Building pipelines from Salesforce sources
You can read from Salesforce as follows, and write to any target supported by Striim. Typically, you will set up data pipelines that read from Salesforce in two phases—initial load, followed by continuous replication—as explained in this concept article on Pipelines.
For initial load, you can use Salesforce Reader to create a point-in-time copy of the existing source Salesforce dataset at the target, as described in Salesforce initial load.
After initial load has completed, you can start continuous replication by continuously reading the new data created in the same Salesforce dataset after the initial load was started, and then writing this new source data to the target
For continuous real-time replication after initial load, you can use Salesforce CDC Reader (see Salesforce continuous real-time replication using CDC).
For continuous incremental replication after initial load, you can use Salesforce Reader (see Salesforce continuous incremental replication).
You can use automated pipeline wizards to build data pipelines from Salesforce for which Striim manages the lifecycle, or you can manually create applications and manage the lifecycle of the pipeline yourself.
Before building a pipeline, you must complete the steps described in Salesforce initial setup.
Using an automated pipeline wizard: if you want to build real-time data pipelines from Salesforce using CDC and write to a supported target, we recommend that you use an automated pipeline wizard with a Salesforce CDC source. These wizards perform all the following steps automatically:
Create two applications: one for initial load and the other for CDC-based continuous replication.
Create a schema and tables in the target matching those to be synced with the source.
Run the initial load application to perform an initial load of existing data from the source to the target.
When initial load completes, run the CDC application to replicating replicate new data using CDC.
Alternatively, you can use an "Automated mode" wizard with a Salesforce source. That will perform the same steps, but continuous replication will be performed using Salesforce Reader instead of Salesforce CDC Reader.
Not using an automated pipeline wizard: if your use case or policies do not allow using an automated pipeline, create separate applications for initial load and continuous replication:
Before performing initial load, get the current ReplayID as described in Switching from initial load to continuous replication of Snowflake sources.
Create a schema and tables in the target and perform initial load: use a wizard with a Salesforce Reader source.
Switch from initial load to continuous replication: see Switching from initial load to continuous replication of Snowflake sources.
Replicate new data: use a wizard with a Saleforce CDC source (for continuous real-time replication) or a Salesforce Reader source (for continuous incremental replication).
Alternatively, instead of using wizards, you can create applications using Flow Designer, TQL, or Striim's REST API.