Skip to main content

Introducing connection profiles

Connection profiles allow you to specify the properties required to connect to an external data source once and use that set of properties in multiple sources and/or targets in multiple applications. Connection profiles also allow you to change connection properties in the web UI, for example to update a rotating token, without undeploying the applications that use it.

In this release, connection profiles support the following authentication types:

  • ADLS Gen2 (for staging area for Databricks and Snowflake only):

    • Access Key

    • Entra ID (OAuth), formerly known as Azure Active Directory

    • SAS

  • Azure Synapse (formerly Azure SQL Data Warehouse):

    • Entra ID

    • Microsoft Entra Service Principal

    • OAuth

    • SQL Password

  • BigQuery (for Database Reader, Incremental Batch Reader, and BigQuery Writer):

    • Service Account Key

  • BigQuery Metastore Catalog (for Iceberg Writer)

  • Databricks:

    • Entra ID (OAuth) for Azure Databricks

    • Personal Access Token

  • Fabric Data Warehouse:

    • Entra ID

    • Microsoft Entra Service Principal

    • OAuth

  • Fabric Lakehouse:

    • Entra ID

    • Microsoft Entra Service Principal

  • Fabric Mirror:

    • Microsoft Entra Password

    • Microsoft Entra Service Principal

  • Generic Iceberg catalog

  • Google Cloud Storage (for Iceberg Writer data lake and external staging only, not for GCS Reader or GCS Writer)

    • Service Account Key

  • Google Dataproc (for Iceberg Writer)

    • Service Account Key

  • Google KMS

    • Service Account Key

  • HubSpot

    • OAuth

    • Private App Token

  • Microsoft Dynamics 365

    • OAuth

  • Nessie Catalog (for Iceberg Writer)

    • Google Open ID

  • Polaris Catalog (for Iceberg Writer)

    • Client Secret

  • S3 (for staging area for Databricks and Snowflake only):

    • Access Key

  • Salesforce (for Salesforce Reader, Salesforce CDC Reader, and Salesforce Writer only, not for other Salesforce adapters):

    • OAuth

  • ServiceNow

    • OAuth

  • Snowflake

    • Key Pair

    • OAuth

    • Password

  • Stripe

    • API Key

    • OAuth

  • Zendesk

    • OAuth

    • Zendesk Creds

Create a connection profile

  1. Select Manage Striim > Connection Profiles > Add Connection Profile.

  2. In Connection Profile Name, enter a name that uniquely identifies the external resource

  3. From the Namespace drop-down, select the namespace where the profile will be stored. All users who will use this profile must have READ and SELECT permissions on the namespace. Known issue (DEV-44065): cannot create new namespace here

  4. From the Endpoint Type drop-down, select the type that matches the external data store (ADLS, BigQuery, Databricks, HubSpot, S3, Salesforce, ServiceNow, Snowflake, Stripe, or Zendesk).

  5. Enter values for the connection properties as described in the inline help, then click Test to validate the connection.

  6. If the test is successful, click Save, otherwise correct the properties as necessary and test again. You can save without a successful test but if the connection profile is not valid applications that use it will halt.

Update a connection profile

  1. Select Manage Striim > Connection Profiles.

  2. Next to the connection profile you want to modify, select ... > Edit.

  3. If a property you want to change is uneditable, undeploy all applications that use the connection profile.

  4. Change the properties as desired, then click Test to validate the connection .

  5. If the test is successful, click Save, otherwise correct the properties as necessary and test again.

Monitoring metrics for connection profiles

Connection profiles generate certain monitoring metrics that can be viewed through the MON command. These include:

  • Recent time of Access Token and Refresh Token retrieval

  • Time taken for Access Token retrieval

  • Total number of Access Tokens retrieved

These metrics help monitor token usage and timing when using connection profiles that manage OAuth-based connections.