Operational considerations for Zendesk Reader

The following links provide additional guidance and best practices for deploying and managing Zendesk integrations in Striim.

Recovery behavior

Initial load: recovery is not supported. If an initial-load run is interrupted or fails, you must rerun the initial load.
Incremental load: supports standard Striim recovery semantics (A1P/E1P) to resume from the most recent checkpoint.

API rate limiting

Zendesk imposes global and per-user/app request limits. When limits are exceeded, the API returns HTTP 429 responses and the reader may halt until limits reset.

Reduce concurrency (Thread pool count) and/or increase Polling interval to lower request frequency.
Limit the scope of sync by narrowing the Objects list instead of using broad wildcards.
Lower Fetch size (maximum 100) to decrease per-call payload and retry pressure.
Stagger multiple applications reading from the same Zendesk account to avoid bursts.
Zendesk applies a lower rate limit to its incremental export endpoints than to its standard endpoints. Incremental load and Automated mode use the incremental export endpoints for tickets and ticket comments, as does Initial load when Include archived tickets is enabled. When reading tickets with many comments, tune Fetch size for incremental so that each poll completes within the polling interval and events reach the target progressively rather than after the entire batch has been read.

Metrics monitored by the Zendesk Reader

Metric	Description
Number of deletes	Number of delete operations
Number of DDLs	Number of data definition language operations
Number of PKUpdates	Number of primary key update operations
Number of updates	Number of update operations
Number of inserts	Number of insert operations

For more information, see Monitoring.

API statistics

The API Statistics value in the MON output for a Zendesk Reader source shows how many calls the reader has made to each Zendesk endpoint, which helps you correlate reader activity with Zendesk rate limits and diagnose slow polls. Call counts are keyed by endpoint template rather than by fully expanded URL, so calls to the per-ticket comments endpoint for all tickets are aggregated under a single entry and the size of the MON output does not grow with the number of tickets read.

For example, an application that reads tickets and ticket_comments in Automated mode produces output similar to the following.

{"/api/v2/incremental/tickets.json":2,"/api/v2/tickets/{id}/comments.json":687}

Limitations

By default, Initial Load for Tickets, Users, and Organizations retrieves only active records. Zendesk automatically archives tickets 120 days after closing, and the standard API used for Initial Load does not return archived records. To include archived tickets and their comments in Initial Load, enable Include archived tickets. To retrieve all records including archived users and organizations, use Automated mode.
Timestamp columns in relational targets: Zendesk Reader emits timestamp values such as created_at and updated_at as strings rather than as org.joda.time.DateTime. When the target is a relational database and the target column uses a timestamp datatype, such as TIMESTAMP, DATETIME, or TIMESTAMP WITH TIME ZONE, the load fails with a datatype conversion error. To work around this, define the target column as a string type, such as VARCHAR, and convert the value to a timestamp in the target after loading.
Custom objects are not supported.
Some secondary objects depend on primary objects (for example, comments- or metrics-related objects may require the corresponding primary ticket object). If a dependent primary is not selected, reads may be empty.
Sherlock scans: Sherlock does not support ticket_comments or other secondary Zendesk objects that have dependencies on primary objects. When a Sherlock scan includes an unsupported object, the scan completes with "Tables scanned: 0" and no rows are processed. To scan supported objects, remove unsupported objects from the scan configuration.