Data to AWS Three Ways
A SharePlex Cookbook Part Two
In the first blog, I discussed data and a bit on SharePlex. Now for the recipes!
Recipes
Here are the recipes - three different ways to get data from your Oracle database to either an Amazon File or S3 storage bucket. In subsequent blogs, I’ll go into much more detail about each recipe.
In all three recipes, we use the “target” command from the SharePlex Command Line interface to tell the SharePlex post process where to write the file. There are also additional options, such as how big to allow the file to become before a new file is created, and the file format. There will also probably be some scripting or coding required to deal with file switches and perhaps additional post-processing such as parsing the XML or JSON.
Recipe #1 – Amazon Storage Gateway
Amazon Storage Gateway is perhaps the most flexible recipe. A storage gateway is an “appliance” that runs in a Linux Virtual Machine. The virtual machine can run either in your data center or in an Amazon EC2 instance. There are two types of storage gateways, a file storage gateway and an S3 storage gateway. Which one you choose will depend on your storage target in AWS. If you’re using S3, you’ll want to use the S3 Storage Gateway.
Once installed and configured, the Storage Gateway presents the source as a Linux NFS (Network File System) directory, which can be mounted or connected to a server running either Unix/Linux or Windows. The target is either an AWS file connected to an EC2 instance or an S3 bucket.
In this configuration, the SharePlex server, as well as the Storage Gateway could be located in AWS EC2 instances, in your data center, or even in another cloud vendor’s environment. All that’s required is network connectivity between the servers.
There will be some configuration and maintenance required for the Storage Gateway.
Recipe #2 – Move files to AWS via the AWS Command Line interface
A second way to move files from SharePlex to AWS is via the AWS Command Line. This could also be used for S3 or “plain old files”. Depending on the target, another file or an S3 bucket, several CLI commands can be used to move the data. In this case, SharePlex would write the file to a filesystem on the target server, which must have the AWS CLI installed. Again, the SharePlex server can be located anywhere, as long is it’s accessible through your network.
Recipe #3 – Write files directly to EC2 storage
This is perhaps the simplest way to move data from Oracle to AWS. If your application is already running on an EC2 instance or can read files from an EC2 instance, the only coding or scripting required would be if the XML or JSON data needs to be further processed before being consumed by your application.
To implement this recipe, we simply install and configure SharePlex on an EC2 instance and point the SharePlex source to that EC2 instance. The SharePlex “target” command controls the format of the output file (JSON, XML or SQL), where the file is placed and other parameters that we’ll discuss in a subsequent blog.
Wrapping it all up
In this blog, I’ve discussed three ways to use the SharePlex FILE target to help extract data from your Oracle database, either in your data center or “in the cloud” and land that data in either Amazon EC2 files or Amazon S3.
In subsequent blogs, I’ll go into a bit more depth on each of the three recipes. In the meantime, if you’d like to try a “taste test”, you can find more information and request a free trial here.