In the previous blog in this series, I showed how SharePlex can help migrate data from an on-premise database into a cloud environment, and how it can be used in a cloud environment to provide selective replication or for backup and disaster recovery. In this blog, I'll show why locating the SharePlex components in "the right places" is critical to achieving maximum performance.
Here's a diagram of the SharePlex architecture.
As you can see, SharePlex has three processes that run on the source, two processes that run on the target, two queues that are placed on the source and one on the target; and there's a network connection between the source and target. For a detailed discussion of each of these, please see this great blog by my colleague Mike Shurtz. Let's look at how placement of these processes can impact performance.
Start at the source
Data is transferred between the proceses on the source using two queues. Since the queues are normally contained in memory, we really have to treat all three processes and two queues as a unit. They all need to run on the same server. If your source is "Database as a Service" and don't have access to the underlying source system, these processes and their queues can be run on a separate server; and use Oracle Network Services to access the redo and archive logs; but, this will incur a performance penalty, since capture will perform a network read in order to obtain that data. This leads to tip #1.
#1 - Whenever possible, run the SharePlex processes on the same server as the database.
Now let's consider the queues. SharePlex normally maintains the queues in memory. However, if the database is extremely busy, with more than 100Gb of redo per hour; or if export can't pass the data to import, one or both of the queues can start to spill to disk. It might be tempting to put those files on either SSD, or on some sort of a networked or shared filesystem. However, since the queues are write-intensive and also need to have consistent access speeds; the best selection for the queues is mirrored spinning disks, local to the server. Here's tip #2
#2 - Locate the SharePlex vardir (Variable Directory) on local, mirrored, spinning disks
Consider the network
Once you've located your source processes and queues correctly, you'll need to pay attention to the network and traffic on network. If the network between your source and target is a dedicated high-speed LAN, network latency and bandwidth are probably sufficient. However, if your databases are connected over a wide-area network; or are in separate regions in a cloud environment, both latency and bandwidth may become important.
Consider latency first. If we think of a network as a road, obviously it will take the first byte of data longer to traverse a network from Shanghai to San Francisco than from Los Angeles to San Francisco.
While it's important to understand latency, how long it takes that first byte to get from source to target; bandwidth, or the amount of data that can be moved between source and target over time, is even more critical. Again, using the road example, an 8 lane freeway can move more cars per unit of time than a 2 lane country road. We need to determine how much data we need to move over what period of time to determine how much bandwidth we need to keep the target from falling behind. We can get a good estimate of the amount of data by looking at the size of the redo logs, and the frequency of log switches. This will give us a maximum amount of data SharePlex will need to move. The actual data moved will be something smaller, due to overhead in the redo log and the fact that SharePlex only moves changed data.
For example, let's assume we have 120Mbyte redo logs and the logs are being switched once very two minutes. Since we measure network bandwidth in bits per second, we need to do some simple math. In this case, we're looking at 60Mbytes per minute or 1 Mbyte per second. Since a byte is nominally 8 bits, this means we need to transfer 8Mbits per second. So, in this case, a 1Mbit per second wide-area network would probably not provide sufficient bandwidth, but a 10Mbit per second network wold most likely be OK. Our next tip is:
#3 - "Do the math" to be sure your network has the required latency and bandwidth
Tuning the target
After you've tuned the source, you can turn your attention to the target. Not surprisingly, some of the tips from the source also apply to the target, with some target specifics. Consider that the post process needs to connect to the target database. The fastest way to do this is with a BEQUEATH or non-network connection. If for some reason, such as the target being a pluggable database, you must use Oracle Network Services, it's important that the network connection between the post process and the target database be as fast as possible.
#4 - Keep the post process as "close" to the target as possible
Another complication here might be that the target database is "Database as a Service:" This will mean that the SharePlex processes will need to run somewhere other than the target. The choices here are to run the post process on the source server; or to use an"intermediate" server. If you're replicating across a wide-area network, or between cloud regions, you'll need to do the same calculations as for Tip #3. Considering the efficienty of Oracle Network Services the choice here should almost always be:
#5 - Consider an intermediate server for DBaaS environments
I hope you find these tips helpful. In the next blog, I'll provide some tips more specific to using SharePlex with Oracle databases, including RDS running in the Amazon Web Services environment. To find out more about how SharePlex can help you be successful on your journey to the cloud, visit the Quest website.