When working with change data capture (CDC), it’s not always obvious whether everything is properly set up. If you’re capturing change data from a SQL Server database, you need to enable the Server Agent. With Oracle, you need to set up supplemental logging. For MySQL, you need to enable replication. Depending on each source-to-target combination, there are many nuances and points of failure that may hinder proper configuration.
Striim 3.6 introduces the ability to automate the creation of applications that leverage change data capture with various sources and targets via a new easy-to-use CDC wizard.
Traditionally, you would needed a lot of DBA experience to pull this off. Now, you just need the correct permissions. The CDC wizard steps through everything you need to do. It walks through checks along the way, and if there is an issue, it prompts you what to do.
Wizard template selection when creating a new app.
Capturing and moving change data from enterprise databases has never been easier. Fifteen new CDC wizards enable rapid deployment of streaming solutions that move change data from a variety of databases (including Oracle, MS SQL and MySQL), into a variety of targets (including Hadoop, Kafka and Amazon Redshift).
These CDC wizards also facilitate configuration for Heterogeneous Real-Time Database Replication between any of the above-referenced enterprise databases, as well as Amazon Redshift. With change data capture established, Striim is able to apply the relevant operation to the target database, including in-place updates and deletes.
In short, the wizard simplifies end-to-end data flow creation, enables change to be captured from enterprise databases like Oracle and SQL Server, and delivers to targets like Hadoop and the cloud in just a few clicks.
Wizard progress view with change data capture from MySQL to Kafka.
You also have the ability to choose additional streams for additional tables. You can filter out certain values, so there’s fine-tune options on defining what the output streams look like. Using the CDC wizard is extremely easy. Simply:
- Click Add App > Start with Template on the Apps page.
- Choose the application type: Click the button for the desired combination of source and target.
- Specify the source properties: For all sources, you must specify the connection URL, username, and password.
- Test the connection: After you provide the source information, Striim will verify that it can connect to the database, that the user has the necessary privileges, and that it can read the change logs and table metadata.
- Select tables: Specify the tables for which you want to read change data using a wildcard or by selecting from a list of available tables.
- Specify the target properties
After you complete the final step, your application will open in the Flow Designer.
Learn more about Striim’s 3.6 CDC wizards by attending our weekly CTO webinar!
