S3 Tables Writer operational considerations
Retry and recovery behavior
S3 Tables Writer offers connection retry configurable at each component's own connection profile (AWS EMR, S3, and S3Tables — see Connection retry support in S3 Tables Writer programmer's reference).
S3 Tables Writer provides recovery capabilities similar to other Striim data warehouse writers, such as the Snowflake and BigQuery writers. During a restart, events from previous runs may be reprocessed and written to the Iceberg tables, maintaining the same order in which they were originally processed. S3 Tables Writer provides at-least-once processing semantics — after a restart, expect some previously written events to be reprocessed rather than silently skipped.
Initial load recovery: supported at table-level granularity. A table that had already completed loading before a restart is not rewritten.
CDC / merge recovery: at-least-once semantics apply; events are reprocessed in original order after a restart.
Optimized merge: when enabled, reduces the data volume S3 Tables Writer must reprocess for partial-image update and delete operations during normal CDC processing (see Optimized merge in S3 Tables Writer programmer's reference).
Best practices
Performance
Use higher event counts (for example 50000 or more) for high-volume CDC; use smaller event counts and shorter intervals when you need lower latency.
Size the EMR cluster for your expected volume and throughput; consider spot instances with an on-demand fallback for cost efficiency.
Use parallel threads for initial snapshot loads; size the thread count to your table count and cluster capacity, and monitor cluster utilization.
Enable Optimized Merge for CDC loads when the source supports partial-image updates.
Group related tables in the same application; avoid configuring an upload policy that produces excessive numbers of small batches.
Security
Prefer IAM roles over explicit access keys; if you do use access keys, rotate them regularly and apply least-privilege permissions.
Rely on S3 Tables' built-in encryption at rest; ensure TLS is used in transit; consider AWS KMS for additional key management where required.
Use VPC endpoints for S3 and S3 Tables where possible; restrict EMR security groups; use private subnets where appropriate.
Operational
Monitor adapter metrics; alert on connection failures and batch errors; watch EMR utilization and cost.
Test recovery scenarios before production use; ensure sufficient checkpoint storage; document your recovery procedures.
Test with representative data volumes; validate DDL propagation; test failover and recovery before going live.
Review upload policies periodically; monitor S3 Tables storage costs and lifecycle policies; keep the EMR cluster's Iceberg libraries current.
Choose the S3 Tables embedded catalog for simplicity and performance; choose AWS Glue Catalog when other AWS services or tools need to query the table metadata through Glue.
Data quality
Use Initial Schema Creation for new tables; validate data types before migration; test DDL propagation in a non-production environment first.
Specify key columns explicitly for sources that lack a usable primary key; validate uniqueness before starting CDC loads; use composite keys where needed.
Use column mapping to handle schema differences between source and target; document your mappings; test with sample data first.
S3 Tables Writer monitoring metrics
S3 Tables Writer exposes two categories of monitoring metrics: table-level Table Write Info, and adapter-level metrics that summarize activity across all tables the application writes to.
Table Write Info
A JSON array of monitoring metrics calculated and displayed at table-level granularity.
Sub-metric | Description | Frequency |
|---|---|---|
Mapped Source Table | The source table the target table is mapped to. | Per batch |
Last batch info | Granular metrics from the last executed batch, including Batch Sequence Number, Batch Event Count, Batch Size (bytes), Max Record Size, Batch Accumulation Time, Micro Batch Count, and Integration Task Time. | Per batch |
Last successful merge time | Time of the last executed task for the table. | Per batch |
Last Applied DDL Time | Time of the last executed DDL for the table. | Per DDL batch |
Last Applied DDL Statement | The last executed DDL statement for the table. | Per DDL batch |
Total Batches Created / Queued / Ignored / Uploaded | Batch counters for the table. | Per batch |
Total event info | Overall event count for the table. | Per batch |
Avg Upload / Compaction / In-Mem Compaction / Merge / Waiting-in-Queue Time (ms) | Average timings across all batches. | Per batch |
Avg Event Count Per Batch, Avg Batch Size (bytes) | Average batch shape. | Per batch |
Avg Stage Resources Management Time (ms) | Average time to clear or create staging resources. | Per batch |
Avg / Min / Max Integration Time (ms) | Time for a processed batch to reach the target table. | Per batch |
Note: Last Applied DDL Statement is restored after a restart only for a single statement. This field can be left empty after a restart when multiple DDL statements were applied in a single batch before the restart. Treat this as a known restore-on-restart limitation for that specific field, not a general failure of DDL tracking.
Adapter level metrics
Metric | Description | Frequency |
|---|---|---|
Write Timestamp | Last time a batch was executed across all tables. | Per batch |
Target Freshness | Time since a batch was executed across all tables. | Per batch |
Discarded Event Count | Total events discarded across all tables (for example, events for excluded tables). | Per event / batch |
Connection Retry Information | Total reconnects and the last known reconnect time. | — |
Queued Batches Size In Bytes | Size of all queued batches. | — |
Limitations
S3 Tables naming constraints. Table bucket names, table names, and column names must comply with AWS S3 Tables naming restrictions: only lowercase letters, numbers, and underscore are supported. Special characters, spaces, and uppercase letters are not supported and can cause DDL operations such as CREATE TABLE to fail.
CREATE TABLE with the AWS Glue catalog requires an AWS-supplied patch. Without the patch, CREATE TABLE against the AWS Glue catalog fails on EMR because EMR's Hadoop configuration ships with pre-configured Glue settings that override the writer's REST catalog configuration, causing Iceberg to use AWS's GlueCatalog implementation instead — which expects a traditional S3 warehouse path rather than the account-id:s3tablescatalog/bucket format S3 Tables uses. AWS has provided a fix consisting of a patched Iceberg RPM and a bootstrap script applied when you create the EMR cluster; this is a third-party AWS artifact that Striim does not bundle or publicly host. Contact Striim Support for guidance if you need CREATE TABLE DDL propagation on the Glue catalog path.
Wildcard table mappings can break against S3 Tables' lowercase-only naming. Many RDBMS sources, including Oracle by default, use uppercase identifiers, so a wildcard table mapping such as Tables:'SCOTT.%' will attempt to create uppercase-named target tables and fail. Use explicit, static, lowercase table mappings instead of wildcards when the source uses uppercase identifiers.
AWS enforces region-level API rate limits on S3 Tables. Striim automatically retries throttled S3 Tables API calls with an internal exponential back-off policy, but under high traffic or with many concurrent S3 Tables Writer applications in the same account and region, retries can still be exhausted, adding latency. This is an AWS platform constraint, not a Striim defect. Monitor for throttling and request an AWS API quota increase proportional to your S3 Tables application count and data volume.
Troubleshooting
Symptom | Likely cause | Resolution |
|---|---|---|
Unable to connect to the AWS EMR cluster. | Incorrect cluster ID or region on the connection profile; insufficient IAM permissions; network connectivity issues. | Verify the cluster ID and region on the AWS EMR connection profile. Confirm the IAM user or role has emr:RunJobFlow, emr:DescribeCluster, and related permissions. Check network and security group configuration. Confirm the cluster is in the WAITING or RUNNING state. |
Authentication failures with S3 Tables. | Invalid or expired credentials; incorrect bucket ARN; insufficient S3 Tables permissions on the EMR instance-profile role. | Confirm the S3TablesBucketARN on the S3Tables connection profile. Verify the EMR instance-profile IAM role has the required s3tables:* permissions — remember that any Access Key / Secret Key entered on the S3Tables connection profile itself has no effect (see Limitations). |
Staging location access errors. | Incorrect bucket or path; insufficient S3 permissions; EMR cannot reach the staging bucket. | Verify ExternalStagingLocation. Confirm s3:GetObject, s3:PutObject, s3:DeleteObject, and s3:ListBucket permissions. Confirm network access from the EMR cluster to the staging bucket, and that regions align. Remember the bucket used always comes from ExternalStagingLocation, not from the S3 connection profile's s3BucketName field. |
Spark job failures on EMR. | Insufficient EMR resources; missing Iceberg libraries; misconfigured Spark settings. | Scale the EMR cluster. Confirm the Iceberg runtime is installed (iceberg-defaults classification with iceberg.enabled=true). Check Spark configuration and EMR logs. |
CREATE TABLE fails on EMR with a Cannot derive default warehouse location error when using the AWS Glue catalog. | EMR's Hadoop configuration overrides the writer's REST catalog setting, forcing Iceberg to use AWS's GlueCatalog implementation, which expects a traditional S3 warehouse path instead of the S3 Tables warehouse format. | Contact Striim Support for guidance on applying the AWS-supplied Iceberg patch and bootstrap script to the EMR cluster (see Limitations). Without the patch, avoid relying on CREATE TABLE DDL propagation with the Glue catalog path. |
DDL operations failing. | Unsupported operation type (see DDL support); catalog permission issues; table already exists (for CREATE TABLE) or does not exist (for ALTER/DROP). | Confirm the operation is one of the supported types listed in DDL support. Check catalog permissions. For CREATE TABLE, confirm the table doesn't already exist; for ALTER or DROP, confirm it does. |
Slow performance during CDC loads. | Upload policy with a very low event count; insufficient cluster resources; too many small batches. | Raise the event count. Scale the EMR cluster. Adjust the interval. Enable OptimizedMerge if the source supports partial-image updates. |
Data type conversion errors. | Unsupported or mismatched source data type. | Review the data type mapping documentation for your source. Use column mapping. Check for special characters or values that don't convert cleanly to the target Iceberg type. |
Throttling or added latency on S3 Tables API calls under heavy load. | AWS region-level API rate limits on S3 Tables, especially with multiple concurrent S3 Tables Writer applications in the same account and region (see Limitations). | Monitor for throttling errors. Request an AWS API quota increase proportional to your S3 Tables application count and data volume. Striim already retries throttled calls automatically with exponential back-off. |
Verify: after applying a resolution above, redeploy or restart the affected application and confirm the symptom no longer appears in the Striim application log or in S3 Tables Writer's monitoring metrics.