Salesforce Reader
Reads Salesforce sObject data. See also Salesforce Pardot Reader, Salesforce Platform Event Reader, and Salesforce Push Topic Reader.
See API Request Limits and Allocations for information on Salesforce-side limits that may require you to limit how much and how quickly Salesforce Reader ingests data.
Configuring OAuth for Salesforce Reader
Authenticating Striim to Salesforce requires an active Salesforce account and a Striim app connected to Salesforce.
Configuring OAuth for automatic authentication token renewal
From the connected app, get the values of the Consumer Key and Consumer Secret.
In the Salesforce Reader, set the values of the Consumer Key and Consumer Secret.
Generate a security token following the instructions in Salesforce documentation.
In the Salesforce Reader, set the value of the Security token.
Configuring OAuth for manual authentication token renewal
Generate an authentication token using the following command:
curl https://login.salesforce.com/services/oauth2/token -d "grant_type=password"\ -d "client_id=<your consumer key>"\ -d "client_secret=<your consumer secret>"\ -d "username=<your username>"\ -d "password=<your password>"
In the Salesforce Reader, set the value of the authentication token.
Generate a security token following the instructions in Salesforce documentation.
In the Salesforce Reader, set the value of the Security token.
Salesforce Reader properties
property | type | default value | notes |
---|---|---|---|
API End Point | String | The endpoint for your Force.com REST API. | |
Auth Token | com.webaction. security.Password | See Configuring OAuth for Salesforce Reader. When Auto Auth Token Renewal is If autoAuthTokenRenewal is set to false , specify your Salesforce access token (see Set Up Authorization on developer.salesforce.com: the first section, "Setting Up OAuth 2.0," explains how to create a "connected app"; the second section, "Session ID Authorization," explains how to get the token using | |
Auto Auth Token Renewal | String | false | See Configuring OAuth for Salesforce Reader. With the default value of False, when the specified Auth Token expires the application will halt and you will need to modify it to update the auth token before restarting. This setting is recommended only for development and testing, not in a production environment. When this property is False, you must specify Auth Token, Password, and Username. Set to True to renew the auth token automatically. In this case, leave Auth Token blank and set the Consumer Key, Consumer Secret, Password, Security Token, and Username properties. |
Connection Retry Policy | String | retryInterval=30, maxRetries=3 | With the default setting, if a connection attempt is unsuccessful, the adapter will try again in 30 seconds ( |
Consumer Key | String | When Auto Auth Token Renewal is If autoAuthTokenRenewal is set to true, specify the Consumer Key (see Set Up Authorization on developer.salesforce.com). | |
Consumer Secret | com.webaction. security.Password | When Auto Auth Token Renewal is If autoAuthTokenRenewal is set to true, specify the Consumer Secret (see Set Up Authorization on developer.salesforce.com). | |
Custom Objects Only | Boolean | False | By default, both standard and custom objects are included. Set to |
JWT Keystore Path | String | When Auto Auth Token is False or OAuth Authorization Flows is PASSWORD, this property is ignored and not visible in the Flow Designer. | |
JWT Keystore Password | com.webaction. security.Password | When Auto Auth Token is False or OAuth Authorization Flows is PASSWORD, this property is ignored and not visible in the Flow Designer. | |
JWT Certificate Name | String |
| When Auto Auth Token is False or OAuth Authorization Flows is PASSWORD, this property is ignored and not visible in the Flow Designer. |
Migrate Schema | Boolean | False | Do not change this setting. It is reserved for use by applications created using Auto Schema Conversion wizards (see Using Auto Schema Conversion). |
Mode | String | InitialLoad | Use the default setting Set to Set to |
OAuth Authorization Flows | Enum | PASSWORD | When Auto Auth Token Renewal is With the default value of PASSWORD, Salesforce Writer will authorize using OAuth 2.0 username and password (see Salesforce Help> Docs> Identify Your Users and Manage Access > OAuth 2.0 Username-Password Flow for Special Scenarios). In this case, you must specify values for the Consumer Key, Consumer Secret, Password, Security Token, and Username properties. Set to JWT_BEARER to authorize using OAuth 2.0 JWT bearer tokens instead (see Salesforce Help> Docs> Identify Your Users and Manage Access > OAuth 2.0 JWT Bearer Flow for Server-to-Server Integration). In this case, you must specify the Consumer Key, JWT Certificate Name, JWT Keystore Password, JWT Keystore Path, and Username properties. |
Objects | See sObjects. | ||
Password | com.webaction. security.Password | When Auto Auth Token Renewal is False or OAuth Authorization Flows is JWT_BEARER, this property is ignored and not visible in the Flow Designer. When Auto Auth Token Renewal is set to | |
Polling Interval | String | 5 min | This property controls how often the adapter reads from the source. By default, it checks the source for new data every five minutes. If there is new data, the adapter reads it and sends it to the adapter's output stream. If you encounter Salesforce REQUEST_LIMIT_EXCEEDED errors, you may need to increase this value or contact Snowflake to raise your API limits (see Salesforce Developer Limits and Allocations Quick Reference). The maximum value is |
Security Token | String | When Auto Auth Token Renewal is False or OAuth Authorization Flows is JWT_BEARER, this property is ignored and not visible in the Flow Designer. When Auto Auth Token Renewal is set to | |
SObjects | String |
| In the Flow Designer this property is shown as Objects. With the default wildcard value Alternatively, specify one or more objects to read, separating multiple objects with semicolons, for example, The objects must be queryable. The account associated with the specified Auth Token must have View All Data permission for the objects. |
Start Timestamp | String | By default, Salesforce Reader reads only new events. Optionally, specify the time (based on LastModifiedDate) from which to start reading older events in the format | |
Thread Pool Size | Integer | 5 | Set this to match your Salesforce Concurrent API Request Limit (see API Request Limits and Allocations). |
Username | String | When Auto Auth Token Renewal is When Auto Auth Token Renewal is set to true, specify an appropriate username (see Add a Single User on help.salesforce.com). |
The output type is WAEvent.
Salesforce Reader WAEvent example
WAEvent{ data: [ "a025j000004He03AAC", "AA", 100.0, "WA", "USD", null, null, null] metadata: { "LastModifiedDate":1646674721000, "TableName":"Business__c", "IsDeleted":false, "CustomObject":true, "OwnerId":"0055j000004c2D2AAI", "CreatedById":"0055j000004c2D2AAI", "OperationName":"INSERT", "CreatedDate":1646674721000, "attributes": "{type=Business__c, url=\/services\/data\/v51.0\/sobjects\/Business__c\/a025j000004He03AAC}", "LastModifiedById":"0055j000004c2D2AAI", "SystemModstamp":"2022-03-07T17:38:41.000Z"}, } userdata: null before: null dataPresenceBitMap: "HwA=" beforePresenceBitMap: "AAA=" typeUUID: {"uuidstring":"01ed7a69-eb41-3f71-8a71-8cae4cf129d6"} };
Salesforce Reader monitoring
The Salesforce reader monitors the following metrics:
Metric |
---|
Read timestamp (called as Last Event Modified Tiime) |
Number of deletes |
Number of inserts |
Number of updates |
CPU |
CPU rate |
CPU rate per node |
Number of servers |
Number of events seen per monitor snapshot interval |
Source input |
Source rate |
Input |
Input rate |
Rate |
Last event position |
Latest activity |
Read lag |
I/O latency in ms |
Table name |
Verifying Salesforce Reader configuration
Use the following cURL commands (see Using cURL in the REST Examples and curl.haxx.se) to verify your configuration and get necessary information about available resources and sObjects.
Get an access token using the Salesforce login URL.
curl https://login.salesforce.com/services/oauth2/token -d "grant_type=password" \ -d "client_id=<your consumer key>" -d "client_secret=<your consumer secret>" \ -d "username=<your username>" -d "password=<your password>"
Using the access token returned by that command, test the REST API URL for your organization. The instance is typically the first part of the URL you see in your browser when logged into Salesforce, such as "mycompany" in mycompany.salesforce.com. Alternatively, ask your Salesforce technical administrator for access to a connected app. (For more information, see Understanding the Username-Password OAuth Authentication Flow.)
If you do not have a proxy server:
curl https://<your Salesforce instance>.salesforce.com/services/data/ \ -H 'Authorization: Bearer <token>'
If you have a proxy server (change the proxy server URL to match yours):
curl -x http://mycompany.proxy.server.com:8080/ \ https://<your Salesforce instance >.salesforce.com/services/data/ \ -H 'Authorization: Bearer <token>'
List available REST resources and sObjects (see List Available REST Resources and Get a List of Objects).
curl https://<your Salesforce instance>.salesforce.com/services/data/v41.0 \ -H 'Authorization: Bearer <token>' curl https://<your Salesforce instance>.salesforce.com/services/data/v41.0/sobjects \ -H 'Authorization: Bearer <token>'
For additional information, see Salesforce's REST API Developer Guide .
Salesforce Reader example
The following TQL will read from the Business__c
sObject and create an appropriate typed stream:
CREATE SOURCE SFPoller USING SalesforceReader ( sObjects: 'Business__c', authToken: '********', apiEndPoint: '<your organization's endpoint UEL>', mode: 'InitialLoad', autoAuthTokenRenewal: 'false' ) OUTPUT TO DataStream; CREATE TYPE OpStream_Type ( Id String KEY, Name String, POSDataCode__c String, Currency__c String ); CREATE STREAM OpStream OF OpStream_Type; CREATE CQ CQ1 INSERT INTO OpStream SELECT data[0],data[1],data[2],data[3] FROM DataStream;