Building a pipeline with Zendesk Reader
You can read from Zendesk using the Zendesk Reader and write to any target supported by Striim. Typically, you will set up pipelines in two phases—initial load, followed by continuous incremental replication—as explained in Pipelines.
For initial load, use the Zendesk Reader in Initial Load mode to create a point-in-time copy of selected objects at the target.
After initial load has completed, start continuous replication by reading new or changed records created in the same objects after the initial load began, then writing those changes to the target. For near-real-time continuous replication of new source data, use the Zendesk Reader in Incremental mode.
Using an automated pipeline wizard: if your environment includes a pipeline wizard for Zendesk, you may use it to set up and manage both phases. Wizards typically create separate applications for initial load and incremental polling, prepare target schema, run initial load, then start incremental replication based on your configured watermark and polling interval.
Not using an automated pipeline wizard: if a wizard is not available or you prefer manual control, create separate applications for initial load and continuous replication:
Before performing initial load, identify a stable, monotonically increasing watermark field for each object (for example, a last-modified timestamp) and record the starting point.
Create target schema and perform initial load: copy existing data from Zendesk to the target.
Switch from initial load to continuous replication: provide the last successful watermark value from the initial load as the starting offset.
Replicate new data: configure incremental polling (set the watermark field and polling interval). Configure the target for upsert semantics using appropriate key columns.
Alternatively, you can create applications using Flow Designer, TQL, or Striim’s REST API.
Pre-requisite - The initial setup and configuration you complete in Zendesk and Striim are described in the Initial setup section.
Create a Zendesk Reader application using a wizard
Log in to a Striim instance.
From the Apps menu, click Create an App.
Select Zendesk as your source, and specify your target. Click Get Started.
In Name, enter an application name, then select a Namespace from the drop-down.
Click Save.
Enter the Zendesk Connection URL.
Select an authentication method: Authenticate using OAuth, Authenticate using User Credentials, or Authenticate using API Token, based on your use case.
To use OAuth, ensure token access is enabled in the Zendesk Admin Center (Token access toggle in Settings).
To use user credentials, ensure password access is enabled in the Zendesk Admin Center (Password toggle in Settings).
If authenticating with OAuth, provide the Auth Token.
If authenticating with API Token, provide the Username and API Token.
If authenticating with user credentials, provide the Username and Password of a Zendesk account.
Optionally select Create schemas on the target before moving the data.
Click Next. The wizard validates your authentication information and displays the results.
Click Next. The wizard displays a list of objects available on the Zendesk instance.
Select the objects to ingest and click Next to continue.
Configure target details.
Click Next to review the summary information.
Click Save & Run. The application saves and begins running.
Create a Zendesk Reader application using TQL
The following sample shows a minimal Zendesk Reader application in TQL. Adjust property values to match your environment and authentication method.
CREATE APPLICATION Zendesk_app; CREATE SOURCE zendesk_source USING Global.ZendeskReader ( Mode: 'InitialLoad', MaxConnections: 20, ThreadPoolCount: '0', authflow: false, ZendeskObjects: '%', Password_encrypted: 'true', FetchSize: 1000, ConnectionTimeOut: 60, connectionProfileName: '', PollingInterval: '120s', useConnectionProfile: false, ConnectionRetries: 3, Password: 'example/012345678abcde==', AccessToken: '', ConnectionUrl: 'https://striim1234567890.zendesk.com/', Username: 'example@example.com', MigrateSchema: true ) OUTPUT TO zendesk_output; END APPLICATION Zendesk_app;