Release notes

Starting with release 5.4.4, Striim Platform ships with Kafka 4.3.1. This resolves the following High CVEs (see www.cve.org) in Kafka 3.9.1, the version shipped with earlier releases:

Kafka 4.x runs only in KRaft mode and does not use ZooKeeper. If you have scripts or monitoring that start, stop, or check ZooKeeper for Striim's internal Kafka, update them. To start the internal Kafka, see Using Striim's internal Kafka server.

If Striim's internal Kafka runs in ZooKeeper mode, migrate it to KRaft mode before you upgrade to 5.4.4. Kafka does not support upgrading a ZooKeeper-mode cluster directly to Kafka 4.x. See Upgrading the Kafka cluster.

If you are running Forwarding Agents, see Upgrading Forwarding Agents. There is a new, additional step when upgrading to 5.4.2.

Starting with release 5.4.4, the kafkaversion and zk.address properties have been removed from Kafka property sets. When you upgrade, Striim removes them from existing property sets automatically. Remove them from any TQL that you maintain outside Striim or import from an earlier release.

Starting with release 5.4.4, Smart Alert and custom alert notifications sent by email, Slack, or Microsoft Teams identify where an alert originated by including the Striim product name and the cluster or service name. In email notifications, the subject line begins with the cluster or service name. Alerts sent from applications do not include these details. If you use mailbox rules or other automation that matches the full subject line of alert emails, or that parses Slack or Teams alert messages, update it to account for the added details. See Managing Smart Alerts.Managing Smart Alerts

Starting with release 5.4.2, Salesforce CDC Reader changes the metadata and the values that it generates, as follows:

As a result of these changes, the Striim type in an existing application may no longer match the data array that the reader generates. Applications with schema evolution enabled are expected to terminate after you upgrade to 5.4.2.

To resume processing, note the Replay ID of the last successfully processed event, then recreate the application and configure it to start from that Replay ID:

Starting with release 5.4.2, Salesforce CDC Reader does not support undeleted columns. Salesforce can restore a deleted custom field along with its data, but Striim does not replicate that restoration. When CDDL is enabled, dropping a column at the source hard-deletes the corresponding column at the target.

When a custom field is deleted, Salesforce appends a _del suffix to its API name, and columns with this suffix are excluded from the Striim data pipeline. Salesforce advises removing the suffix when you undelete the field. Doing so can return the column to the pipeline, but the values that were in the deleted column are not restored.

Starting with release 5.4.2, you can use the Striim wizard to create an application that combines Salesforce Reader and Salesforce CDC Reader to perform an initial load followed by continuous replication. Salesforce Reader performs the initial load using a SELECT query, and Salesforce CDC Reader performs ongoing replication by reading change events. Because the two readers retrieve data through different mechanisms, the set of columns emitted during the initial load may differ from the set emitted during CDC; use ColumnMap() at the target to align the two schemas.

Starting with release 5.4.2, the default value of ServiceNow Reader's Fetch Size property has been reduced from 10000 to 1000. Applications created in earlier releases retain their existing Fetch Size value.

Starting with release 5.4.2, for Salesforce Reader, Salesforce CDC Reader, Salesforce Writer, and Salesforce Pardot Reader, the Consumer Secret property is no longer used and no longer appears in the Flow Designer when OAuth Authorization Flows is JWT_BEARER.

Starting with release 5.4.0.2, the Oracle Reader property Read Archive Logs Only has been renamed Archive Fence.

Kafka Reader and Kafka Writer versions 0.11 and 2.1.1 are no longer supported. As part of upgrading to Striim 5.4, all Kafka Reader and Kafka Writer adapters in created or quiesced applications will be upgraded to the new versions of the adapters and corresponding connection profiles will be created. If after upgrading you need to import TQL exported from Striim 5.2 or earlier, edit the TQL before importing to use the new properties, and create a connection profile with the appropriate properties. For more information, see Upgrading from Kafka Reader 2.1.

Starting with release 5.4.0.2, the connection profile endpoint type BigQuery Metastore Catalog has been renamed BigLake Metastore Catalog, to reflect the name change by Google. On upgrading, BigQuery Metastore Catalog connection profiles will automatically be converted to BigLake Metastore Catalog connection profiles.

Starting with release 5.2.0, the syntax for defining external sources using the CREATE EXTERNAL SOURCE statement has been updated to simplify configuration and support additional source types. For details and updated examples, see CREATE EXTERNAL SOURCE.CREATE EXTERNAL SOURCE

The event AIData data model for Sentinel was modified in 5.0.6, with the names of two fields updated from 5.0.2. The number of unique sensitive data identifiers is now specified by numIdentifiers (formerly, Identifiers in 5.0.2), and the number of occurrences of all sensitive data identifiers is now specified by numOccurrences (formerly, Occurrences in 5.0.2).

Starting with release 5.4.0.2, Striim Platform requires JDK 17. Due to this change, once you upgrade to Striim 5.4.0.2, it will not be possible to downgrade to a previous release.

Kafka Reader and Kafka Writer versions 0.8, 0.9, and 0.10 are no longer supported. When you upgrade from Striim 4.x to Striim 5.x, any applications using those versions will automatically be updated to use Kafka Reader 2.1 or Kafka Writer 2.1 (which are backward-compatible with the old versions).

As part of upgrading from Striim 4.x to Striim 5.x, a Snowflake Writer with Streaming Upload enabled and Authentication Type set to OAuth or Password will have that property value switched to Key Pair.

Starting with release 4.2.0, TRUNCATE commands are supported by schema evolution (see Handling schema evolution). If you do not want to delete events in the target (for example, because you are writing to a data warehouse in Append Only mode), precede the writer with a CQ with the select statement SELECT * FROM <input stream name> WHERE META(x, OperationName) != 'Truncate'; (replacing <input stream name> with the name of the writer's input stream). Note that there will be no record in the target that the affected events were deleted.Handling schema evolution

OJet requires Oracle Instant Client version 21.6. See Install the Oracle Instant Client in a Striim server or Install the Oracle Instant Client in a Forwarding Agent.

MongoDB Reader no longer supports MongoDB versions prior to 3.6.

MongoDB Reader reads from MongoDB change streams rather than the oplog. Applications created in releases prior to 4.2.0 will continue to read from the oplog after upgrading to 4.2.1. To switch to change streams:

Databricks Writer's Upload Policy's default eventcount value has been increased from 10000 to 100000 and the Hostname property has been removed, since the host name can be retrieved using the connection URL. (See Databricks Writer properties.)

DEV-20673: UI export produces invalid TQL when some property values contain backslash characters

DEV-22056: Equifax - Oracle DBReader - GUI has issues if password contains \ character

DEV-23231: UI Export from BackSlashes not working

DEV-29708: UI creates invalid escaping for fields with \

DEV-32174: Escaping backslash characters not working properly when updated Table properties from UI

DEV-46681: UI improperly persists slashes to all components

DEV-49716: UI - Striim adds 1 more escape character next to the first escape character found in all regex containing fields

DEV-51472: Error while creating a connection profile in Snowflake

DEV-52582: Striim UI doesn't allow multiple HPNonStopSQLMPReader tables to be specified

DEV-55660: StriimConnectException caused by data type conversion error

DEV-55912: System$OTelMonitoring.OtelMonitoringApp got terminated in Azure production

DEV-56376: Recovery Metrics of Mon AppName are seen as N/A

DEV-56850: Address thread leak in MongoDBReader

DEV-57046: Kafka Writer - delay in drop application due to unexpected topic deletion despite auto create topic being disabled

DEV-57127: PostgresReader - data loss on app restart where a transaction performed from multiple sessions has a committed transaction with a lower LSN than a previously processed event from a different session

DEV-57584: GG Trail reader fails with java.lang.ArrayIndexOutOfBoundsException: arraycopy: length -2 is negative

DEV-57612: return value of com.striim.proc.eventprocessor.ApplicationTemplate.getCheckPointProcessor() is null

DEV-57690: performance degraded after upgrade - metric queue full (5000), dropping metric

DEV-57693: unit for the metric average_events_per_collection is not accepted by Striim OTel collector

DEV-57803: application stuck in stopping state repeatedly

DEV-57831: PGReader - exclude tables functionality not excluding tables when also specified in the interested tables list

DEV-58038: Kafka Writer - backpressure

DEV-58068: SalesforceCDCReader - inconsistencies in Striim types between initial run and recovery

DEV-58095: UI is not working properly

DEV-58098: Kafka Writer - event distribution & logging bugs

DEV-58122: export/import upgrade throws NumberFormatException: Invalid version format: 5.4.0.2 - GG - KPS to DBW topology

DEV-58178: Striim console returning empty output while fetching application status

DEV-58265: Kafka Writer - missing events due to missing ACLs on the MSK

DEV-58422: CheckPointTopic didn't get dropped

DEV-58559: BigQueryWriter - failure due to wrong char escape when UI saves the component

DEV-58591: Spanner Writer - PK-UPDATE support for 1..1 table mapping with JSON_OBJECT() or JSON_ARRAY() not working

DEV-58646: Spanner Writer - error on INSERT using JSON_OBJECT when key is present in the event

DEV-58696: DBWriter - Vertica Mon metrics mismatch when IgnorableExceptionCode is DUPLICATE_ROW_EXISTS

DEV-58744: JSON Formatter does not serialize the objects correctly

DEV-58781: Database Reader > Spanner Writer - JSON_OBJECT and JSON_ARRAY fail if USERDATA does not contain the mapped key

DEV-58792: application is hung and in stopping status

DEV-58830: CVEs in Striim 5.4.0.2

DEV-58836: MySQL Reader - index out of bounds error after restart of Striim

DEV-58844: no data written to BigQuery for 6+ days

DEV-58866: quiesce acknowledgement (source disapproved quiesce) silently suppressed after PR #29693

DEV-58956: OJet > KafkaWriter - checkpoint didn't change and holds the archived logs

DEV-59014: Spanner Writer - issue writing JSON array to JSON column

DEV-59266: upgrade fails: problem converting data: Id

DEV-59274: MySQL Reader removed trailing zeros from binary(n)

DEV-59325: woodstox-core-asl-4.2.0.jar causes server startup issue

DEV-59737: replace existing decompression library with Windows ntdll RtlDecompressBufferEx

DEV-59838: Spanner writer - conversion of object of type java.lang.String to Date is not supported for the DueDate datatype

DEV-59948: Db2 for z/OS input objects not getting cleaned up in pipeline drops

DEV-59949: Db2 for z/OS high availability setup issues

DEV-59950: Striim Agent for Db2 for z/OS Agent not starting on certain script actions

DEV-60109: Cause: arraycopy: last destination index 20971521 out of bounds for byte[20971520]

DEV-60266: Spanner Writer does not preserve null for commit timestamp columns

DEV-60678: Db2 for z/OS pipeline failed on startup with "invalid length" errors"

DEV-60900: Db2 for z/OS halts with multiple open Output Targets error when running two apps concurrently on the same Forwarding Agent

DEV-5701: Dashboard queries not dropped with the dashboard or overwritten on import

When you drop a dashboard, its queries are not dropped. If you drop and re-import a dashboard, the queries in the JSON file do not overwrite those already in Striim.

Workaround: drop the namespace or LIST NAMEDQUERIES, then manually drop each one.

DEV-8142: SORTER objects do not appear in the UI

DEV-8933: DatabaseWriter shows no error in UI when MySQL credentials are incorrect

If your DatabaseWriter Username or Password values are incorrect, you will see no error in the UI but no data will be written to MySQL. You will see errors in webaction.server.log regarding DatabaseWriter containing "Failure in Processing query" and "command denied to user."

DEV-11305: DatabaseWriter needs separate checkpoint table for each node when deployed on multiple nodes

DEV-17653: Import of custom Java function fails

IMPORT STATIC may fail. Workaround: use lowercase import static.

DEV-19903: When DatabaseReader Tables property uses wildcard, views are also read

Workaround: use Excluded Tables to exclude the views.