You have an application that retrieves and stores information in a database. You want faster response times so you utilize a Hazelcast Cache for rapid access to in-memory data. But other applications also make updates which leads to the cache being inconsistent with the database, and the database showing out-of-date information. Hazelcast Striim Hot Cache solves this by synchronizing the cache with the DB using change data capture ensuring the cache and application always have the correct data.
Unedited Transcript:
Today, we would like to introduce the new Hazelcast Striim Hot Cache that uses change data capture to solve the cache consistency problem. You mentioned that you have an application that works by retrieving and storing information in a database to get faster response times. You utilize a Hazelcast in-memory cache for rapid access to data. However, other applications also make database updates which leads to inconsistent data in the cache and the application. No shows out of date or in valid information. Hazelcast Striim Hot Cache solves this by using streaming change data capture to synchronize the cache with the database in real time, ensuring the cache and associated application always have the correct data. In this demo we have a MySQL database – here’s the cache and Striim server. We’re using test programs to work with the MySQL database to create, modify it and dump data and for the Hazelcast hot cache, which can also dump data.
We start by creating a table using the test code and loading it with data. We then use Striim to load the data from the database into the Hazelcast cash. Next, we create a CDC flow from the database and use this at deliver life changes into the cache. We run continuous modifications against the database which are replicated to the cache. After some time we dump the database in the cache to files and run a diff to prove that they are the same. In this demo you’ll see the use of wizards to simplify the CDC setup and delivery to Hazelcast real realtime CDC from the database, the realtime delivery of change to synchronize the cache and a real time custom monitoring of the whole solution.
The first thing we need to do is set up the demo by using our test program to create a table in the MySQL database. You can see the table here and that it is initially empty. Next, we used to test code to insert 200,000 rows of data. You can see the count here to perform the initial load from database to cache. We need a data flow. We use a database reader to extract data from MySQL, which we can figure for our my sequel instance and test table. The target is a Hazelcast writer, which is configured to map the table data into our cache to load the cache we need to deploy it and start the application which streams to for table data from the database into the cache. You can see that the cash now has 200,000 entries for HUD cache. We need to set up change data capture to stream live database changes to the cache first.
We can figure properties to connect to the MySQL database and Striim checks to make sure CDC will work properly. We’ll tell you if the configuration needs changing and what to do. You can then browse and select tables of interest. Next you enter the Hazelcast cluster information and mapping information in the form of a file linking database tables to cache objects. You can see the mapping here. The last step finalizes the configuration of the Hazelcast target. The wizard results in a data flow from MySQL CDC source to the Hazelcast target. When deployed and started, the cache is synced with modifications from database change. Let’s run some modifications in the form of inserts, updates, and deletes against the table. You can see the data flowing here in real time in our application. We stopped the modifications and see the table size, which we compare with a number of entrants in the cache. Both the table and cash have 180551 records. If we dumped this equal data from the database into a file and do the same with the cash data into a different file, we can do a diff between the two files with no results proving that they are the same.
And here is what the dump data looks like in both cases. The full Striim platform can also monitor the end to end data flow using a separate data flow for analytics and fully customizable dashboards to meet your business requirements. You can download Striim today to test out hot cache, and subscribe to our channel for more videos like this one.
