Skip to main content

What's new in Striim Platform 5.4.2

Looking for the Striim Cloud documentation? Click here.

The following features are new in Striim Platform 5.4.2.

Release highlights

  • Striim MCP Server for AI Applications and Agents

    The new Striim MCP Server provides a standardized interface for external AI applications and agents to interact with Striim through the Model Context Protocol (MCP).

    MCP-compatible clients, including Claude Code, Claude Desktop, Cursor, and custom agents, can:

    • Create enterprise agentic AI applications with live, real time context

    • Create data pipelines by discovering adapters, generating and validating TQL, and deploying and starting pipelines.

    • Monitor and troubleshoot applications using health information, runtime metrics, checkpoints, lineage, alerts, and pipeline state.

    • Continuously observe live data by querying streams and WActionStores.

    • Securely manage connection profiles, users, roles, permissions, and other platform resources.

    The MCP server is embedded in the Striim server and requires no separate installation or process. It uses Striim’s existing authentication, authorization, namespace permissions, TLS, logging, and auditing infrastructure, ensuring that AI-driven operations remain within established security and governance controls.

    The Striim MCP Server helps developers, operators, and administrators complete complex tasks faster while using the AI tools already built into their workflows. For more information, see Striim MCP Server.

  • Conversational AI and Analytics

    Clara, Striim’s new built-in conversational AI agent, helps you move from questions to informed action through natural-language conversations without leaving Striim Platform. Available from the Ask Clara panel, Clara can:

    • Answer questions about Striim concepts, adapters, configuration, setup, and troubleshooting.

    • Inspect applications, components, configuration, monitoring data, and cluster health, and perform supported application lifecycle operations.

    • Turn a data-movement goal into a proposed Striim application. Clara guides you through configuration and generates a TQL Plan that you can review before approving any changes.

    • Use supported models from OpenAI or Google Gemini for cloud-based AI processing. Customers supply and manage their own AI-provider credentials. Alternatively, Clara can use Ollama on a local or remote server when model processing must remain within the customer’s infrastructure.

    The Clara AI Agent runs locally and supports conversational follow-up questions, helping you investigate issues, operate applications, and build data pipelines more efficiently while retaining control over proposed actions and application changes. For more information, see Clara AI Agent.

  • ClickHouse Integration for Fast Analytics

    The new ClickHouse Writer continuously streams data from databases, data warehouses, message buses, SaaS applications, and other supported sources to ClickHouse for fast analytical querying. Key capabilities include:

    • Support for MergeTree, AggregatingMergeTree, CoalescingMergeTree, ReplacingMergeTree, and other table engines.

    • Merge and Append-Only write modes: Merge applies source inserts, updates, and deletes to keep ClickHouse aligned with the source, while Append Only writes all source operations as new rows.

    • Parallel writes, flexible table and column mappings, and scale-out for higher throughput.

    • Schema evolution to detect and propagate supported source DDL changes.

    • Supports recovery with exactly-once processing or at-least-once processing.

    Fo more information, see ClickHouse.

  • Open-table formats

    Striim can now write to managed Apache Iceberg infrastructure on Amazon S3 Tables and Google Lakehouse for Apache Iceberg. The new S3 Tables Writer and GCP Lakehouse Writer expand Striim’s Apache Iceberg support beyond customer-managed deployments on Google Cloud Storage (GCS) to managed Iceberg infrastructure on AWS and Google Cloud. These new integrations use cloud-managed storage, compute, catalog, and maintenance services, reducing the infrastructure customers must deploy and operate.

    • Amazon S3 Tables: S3 Tables Writer writes Iceberg data to purpose-built Amazon S3 table buckets using AWS EMR for compute. It uses the embedded S3 Tables catalog by default and also supports AWS Glue. Amazon S3 Tables provides storage optimized for Iceberg workloads and managed table-maintenance services such as metadata compaction and unreferenced-file cleanup.

    • Google Lakehouse for Apache Iceberg: GCP Lakehouse Writer stores Iceberg tables in GCS and uses Google Cloud Managed Apache Spark to process appends, merges, compaction, and DDL changes. It uses the GCP Lakehouse Runtime Catalog and supports both the default custom catalog mode and the Apache Iceberg REST Catalog interface. GCS serves as both the data lake and the external staging area.

    Both writers provide the following capabilities:

    • Support for initial loads and continuous replication using CDC or incremental-reading methods.

    • Automatic creation of target namespaces and tables during initial load.

    • Schema evolution that propagates supported source DDL changes to target Iceberg tables.

    • Merge, Append Only, and Optimized Merge write modes.

    • Retry and recovery with at-least-once processing semantics.

    • Server-side encryption at rest using the respective cloud provider’s encryption services, and TLS encryption in transit.

    These integrations make it easier to continuously populate governed and versioned Iceberg tables for analytics, data engineering, AI, and agentic workloads. For more information, see GCP Lakehouse Writer and S3 Tables Writer.

  • Spanner Writer: JSON and ARRAY data types

    Spanner Writer now supports JSON and ARRAY data types for both GoogleSQL and PostgreSQL dialects.

    JSON data type:

    • Write JSON objects and arrays, including nested JSON structures up to Spanner’s limit of 80 levels.

    • Map multiple source columns or tables to one or more JSON columns in a target table.

    • Use existing capabilities such as upsert, key columns, column mapping, and ignorable exception codes.

    • With the GoogleSQL dialect, update or merge individual fields at any nested JSON path without replacing the entire JSON document. This includes adding, updating, or removing elements within JSON arrays.

    JSON support lets you preserve hierarchical and semi-structured data without flattening it into multiple relational columns or tables. Common use cases include application events, customer profiles, product attributes, and other complex records with evolving schemas. For more information, see Support for JSON data types in Spanner targets.

    ARRAY data type:

    • Write ordered collections of values of the same type, such as lists of strings or integers, using either Spanner dialect.

    • Use arrays supplied by sources in native Java formats; Spanner Writer automatically converts them to the corresponding Spanner array type.

    • Use ARRAY columns without enabling additional Spanner Writer properties.

    ARRAY support simplifies the replication of multivalued attributes such as tags, categories, roles, identifiers, and measurements without requiring separate child tables or additional transformations. For more information, see Support for ARRAY data types in Spanner targets.

  • Fast Snapshot Loading

    Fast Snapshot Loading now provides two additional capabilities for maximizing source-read throughput and reducing initial-load and backfill times when using Database Reader with MySQL, Oracle, PostgreSQL, or SQL Server:

    • Labeled multi-query reads: Configure up to 20 labeled SELECT queries to read from one or more source tables. Divide large tables into smaller ranges for concurrent processing, and preview queries before running them.

    • Partition-aware reads: Automatically detect partitioned tables and read their partitions sequentially or concurrently, based on your configuration.

    For more information, see Fast Snapshot Loading.

  • Flow Designer

    The Flow Designer provides improved navigation, layout, component discovery, and debugging capabilities:

    • Use zoom, pan, and scrolling to navigate the canvas. You can also resize property panels and use smart guides to position and align components.

    • Navigate large flows using the new interactive minimap. It displays an overview of the entire canvas and highlights your current viewport. Click or drag within the minimap to move directly to another part of the flow without repeatedly zooming in and out.

    • Find components by browsing a complete list of components in the flow or searching for a component by name.

    • Choose between Classic and Enhanced layout modes. Classic retains the existing layout and remains the default. Enhanced is recommended for large flows and lets you arrange the flow in either a vertical or horizontal orientation.

    • Highlight the upstream data path from the flow’s entry component to the selected component. This makes it easier to understand dependencies and troubleshoot complex or highly parallelized flows.

    • Select source tables visually when reading from MySQL, Oracle, PostgreSQL, or SQL Server using Database Reader, Incremental Batch Reader, or a CDC reader.

  • Striim SDK for Open Processors

    The Striim SDK for Open Processors introduces an annotation-based Java framework that simplifies Open Processor (OP) development. The framework reduces the repetitive setup code required to create an Open Processor, allowing developers to focus on the custom processing logic needed for complex integrations and specialized data transformations.

    The framework supports common development patterns, including:

    • Custom readers and writers.

    • Individual-event and batch processing, with recovery handled automatically where supported.

    • Support for built-in Striim event types, including JsonNodeEvent, AvroEvent, and WAEvent.

    • Access to Striim Vaults.

    • Templates for common Open Processor use cases.

    Open Processors developed with earlier versions of the Striim SDK for Open Processors remain supported. For more information, see Using Striim Open Processors.

  • Google Firestore

    Striim introduces the Firestore Writer, a new target adapter for writing data to Google Cloud Firestore Enterprise edition with MongoDB compatibility. This edition supports existing MongoDB application code, drivers, and tools, making it easier to use data delivered by Striim with MongoDB-compatible applications. For more information, see Firestore Writer.

  • Databricks on Google Cloud

    Databricks Writer now supports Databricks on Google Cloud as a new target, using Google Cloud Storage (GCS) as the staging area. This extends the writer’s existing support for Databricks on AWS with Amazon S3 staging and Databricks on Azure with Azure Data Lake Storage staging, enabling customers to write to Databricks across all three major cloud providers.

    Databricks Connection Profiles and Databricks Writer also support service principal-based machine-to-machine (M2M) authentication for Databricks on AWS, Azure, and Google Cloud. For more information, see Databricks Writer.

Sources and targets

  • The new Spanner Database Reader enables Striim applications to read from Spanner databases using either the GoogleSQL or PostgreSQL dialect. It supports Fast Snapshot Recovery for interrupted initial loads and can automatically quiesce the application after the initial load completes when used with supported targets. For more information, see Spanner Database Reader.

  • The MSJet Reader can capture DDL changes performed within SQL Server user transactions, including transactions containing only DDL or a combination of DDL and DML. It also captures DDL changes made through SQL Server Management Studio (SSMS). Together with existing support for standalone DDL, this provides comprehensive DDL coverage, reduces the risk of missed schema changes, and improves application reliability. For more information, see SQL Server setup to support capturing DDL within transactions or made with SQL Server Management Studio.

  • Striim supports target-managed default columns when writing to BigQuery, Snowflake, Databricks, Azure Synapse, and ClickHouse. When a write operation does not provide a value for a default column, the target data warehouse automatically populates it using a predefined value or expression.

    Examples include timestamps such as CURRENT_TIMESTAMP, generated UUIDs, identity or sequence numbers, constants, and values computed from other columns. You can use default columns for audit tracking, surrogate-key generation, versioning, change tracking, and other target-specific requirements without creating additional mappings or transformations in Striim. For more information, see the notes for the Tables property in the programmer's guides for the individual adapters.

  • BigQuery Writer supports exactly-once processing (E1P) semantics in MERGE mode. This prevents retries or recovery from writing duplicate rows or applying the same change more than once, improving data consistency in BigQuery. For more information, see the notes for the Support Transactions property in BigQuery Writer programmer's reference.

  • MySQL Reader now supports Global Transaction Identifiers (GTIDs). MySQL assigns a unique GTID to each binary-logged transaction committed on its source server, and that identifier remains associated with the transaction throughout the replication topology. When GTID is enabled on the source database, MySQL Reader uses GTID-based positioning to determine where to start or resume reading changes instead of relying on binary-log file names and offsets. This simplifies CDC positioning in MySQL replication environments. For more information, see Understanding Start Position and global transaction identifiers (GTIDs).

Observability (monitoring, alerting, and logging)

  • Non-admin users can now create custom alerts for applications in their own namespace or other namespaces to which they have access. Developers can monitor the applications they work on without requiring administrator privileges, improving operational effectiveness while preserving access controls. For more information, see Allowing non-admin users to manage custom alerts.

  • Two additional log files have been added: striim.applications.log, which records application state transitions such as CREATED → DEPLOYING → DEPLOYED, and striim.system.log, which records information about servers and Forwarding Agents. For more information, see Logs in Striim Platform.

  • The OJet Config property supports a new MONITOR component to support alerting on various conditions. For more information, see OJet properties.

Security and governance

  • Striim now supports Salesforce External Client Apps (ECA) for Salesforce Reader, Salesforce CDC Reader, and Salesforce Writer. For more information, see Salesforce Connected Apps and External Client Apps.

  • Connection Profiles support reader and writer adapters for Oracle, PostgreSQL, SQL Server, MySQL, MariaDB, Lakebase, and ClickHouse. You can reuse centrally managed connection settings across these adapters, simplifying configuration and improving consistency. For more information, see Connection profiles.

  • Azure Data Lake Storage Gen2 Connection Profiles support authentication with a Microsoft Entra service principal. For more information, see Initial setup for ADLS Reader.

  • When reading from MongoDB 4.2 or later, you can now configure the MongoDB Reader to capture changes at the collection, database, or deployment level. This capability supports least-privilege access, allowing you to grant Striim access only to the data it needs. It is especially useful for environments with strict security, privacy, or data-isolation requirements. For more information, see the notes for Change Streams Capture Level in MongoDB Reader properties.

  • Striim Vault supports integration with AWS Secrets Manager, in addition to existing integrations with Azure Key Vault, Google Secrets Manager, Hashicorp Vault, and CyberArk. For more information, see Using vaults.

  • Striim Platform supports SSO integration with PingFederate, in addition to existing integrations with Entra ID (formerly Azure AD) and Okta. For more information, see Single Sign-On with Ping Identity.

  • You can now use the BigQuery Storage Write API in network topologies configured with Google Cloud Private Service Connect endpoints, thus enabling high-throughput writes to BigQuery through private connectivity. For more information, see Using Private Service Connect with BigQuery Writer.

Installation and configuration

  • Striim Platform can now be deployed and operated on Kubernetes. In this release, Google Kubernetes Engine (GKE) is the only supported Kubernetes environment.

    You can use Kubernetes and Helm workflows to configure resources, deploy, scale, monitor, and upgrade Striim while integrating with existing infrastructure-as-code, CI/CD, and cluster-governance practices. This deployment model reduces manual configuration and lets you manage Striim consistently alongside other Kubernetes applications. For the Striim Metadata Repository (MDR), you can use either a self-managed PostgreSQL database or fully managed Cloud SQL for PostgreSQL. For more information, see Running Striim in the Google Cloud Platform.

  • Beginning with this release, the version of Striim installed on the Forwarding Agent and the version installed on the Striim server must have the same first three version components. Some releases may also require the Forwarding Agent to be updated to a specific Striim version; any such requirement will be documented separately. For example, when no additional requirement applies, Striim 5.4.2 installed on the Forwarding Agent will be compatible with Striim Platform versions 5.4.2, 5.4.2.0.B, 5.4.2.2, or 5.4.2.2A.

  • Striim Platform is now compliant with FIPS 140-2, the US government security standard for cryptographic algorithms and implementations. Striim can start and operate on systems with OS-level FIPS mode enabled, supporting deployments in government and other regulated environments.

    To support FIPS compliance, Striim now encrypts all passwords using AES-256 and uses a unique AES key for each Source, Target, Cache, Connection Profile, and Property Variable component.