Dynamic data movement includes replication of data from a traditional RDBMS like Oracle to databases like MySQL, Azure SQL Database, Kafka and PostgreSQL.
Data is valuable because it contains all of the information that represents your organization. The data in your production databases is constantly changing because of factors like customer transactions, sales inquiries, price fluctuations and purchasing trends. It represents the potential activity with which you achieve your business goals.
There are so many new techniques and technologies that can help your business analysts, marketing specialists, salespeople, data scientists and CXOs interpret and act on data. But you have to find ways to ensure that data can move between systems in real time to derive maximum value from your data.
What is data movement?
Data movement, as we (and IDC) see it, is two sub-markets:
- Database replication for high availability, disaster recovery and scaling.
- Dynamic data movement is the (mostly) heterogeneous movement of data between systems.
Data movement involves a task that’s already familiar to you: increasing the value of data by moving it to the right systems faster.
Traditionally, you use data replication to move it to those systems. That’s the process of simultaneously updating copies of your data in multiple places. One goal of replication is to keep your data moving to the users who rely on it to make decisions and to the customers who need it to perform transactions. Other goals include high availability and disaster recovery, and scaling out beyond what your database is designed for.
In most cases, your data replication is homogeneous with regard to RDBMS vendor. That is, you’re replicating from, say, one Oracle database to another. It’s stable, traditional and familiar.
But as your company adopts open-source and NoSQL databases, you’ll notice the shift from homogeneous replication to dynamic data movement.
What is this new “dynamic data movement?”
The term “dynamic data movement” typically applies to heterogeneous contexts, and uses change data capture (CDC) to move data from databases to other systems. It becomes important as your organization diversifies from enterprise standbys like Oracle to new vendors like MySQL, PostgreSQL and Kafka, for increasingly common use cases like integrating applications, populating data lakes or data warehouses, and running modern analytic applications.
In data warehousing, for example, you’re probably accustomed to traditional ETL (extract, transform, load) products. The problem is that data integration requirements in most companies have evolved beyond ETL. Why?
First, the volume of data you’re processing keeps growing. Second, the sun never sets on operations, so your windows for loading batches have shrunk, pushing you to near-real-time and event-driven data integration. Finally, the trend toward multiple database platforms (mentioned above) leaves you with mismatched sources and targets.
Thus, the market for data movement in general is growing and blurring with ETL. Analyst firm IDC breaks the wider market for data products into three groups:
- Data intelligence — the care and feeding of data quality
- Data access — connectors, data preparation and lifecycle management
- Data integration — increasing the speed of data delivery for making decisions (including dynamic data movement)
Using SharePlex
for data movement
SharePlex by Quest® is built to streamline replication in all the use cases where your business needs it:
- Availability in data replication
- Scalability — for distributed processing, cascading and improved performance
- Reporting and integration — for analytics, database consolidation and low-impact auditing
SharePlex replicates data in both homogeneous and heterogeneous environments, making it ideal for data movement. Typical scenarios include these:
- Integrating an Oracle operational database with open-source databases
- Replicating selected data from an Oracle database to open-source databases for data warehousing or reporting
- Replicating business data from an Oracle database for analysis with cloud tools like Azure Synapse and Azure AI
With SharePlex, you have full data movement from Oracle to targets as varied as SQL Server, Azure SQL Database, Kafka, Azure Event Hubs, MySQL, PostgreSQL and JMS queues.
In all stages of growth, your company faces the prospect of moving production workloads to the cloud, offloading analytics from production databases and implementing a disaster recovery strategy. SharePlex enables you to move beyond traditional, homogenous data replication to dynamic data movement without risking impact to your customers and users.