Troubleshooting S3 Tables Writer

Unable to connect to the AWS EMR cluster.

Incorrect cluster ID or region on the connection profile; insufficient IAM permissions; network connectivity issues.

Verify the cluster ID and region on the AWS EMR connection profile. Confirm the IAM user or role has emr:RunJobFlow, emr:DescribeCluster, and related permissions. Check network and security group configuration. Confirm the cluster is in the WAITING or RUNNING state.

Authentication failures with S3 Tables.

Invalid or expired credentials; incorrect bucket ARN; insufficient S3 Tables permissions on the EMR instance-profile role.

Confirm the S3TablesBucketARN on the S3Tables connection profile. Verify the EMR instance-profile IAM role has the required s3tables:* permissions — remember that any Access Key / Secret Key entered on the S3Tables connection profile itself has no effect (see Limitations).

Staging location access errors.

Incorrect bucket or path; insufficient S3 permissions; EMR cannot reach the staging bucket.

Verify ExternalStagingLocation. Confirm s3:GetObject, s3:PutObject, s3:DeleteObject, and s3:ListBucket permissions. Confirm network access from the EMR cluster to the staging bucket, and that regions align. Remember the bucket used always comes from ExternalStagingLocation, not from the S3 connection profile's s3BucketName field.

Spark job failures on EMR.

Insufficient EMR resources; missing Iceberg libraries; misconfigured Spark settings.

Scale the EMR cluster. Confirm the Iceberg runtime is installed (iceberg-defaults classification with iceberg.enabled=true). Check Spark configuration and EMR logs.

CREATE TABLE fails on EMR with a Cannot derive default warehouse location error when using the AWS Glue catalog.

EMR's Hadoop configuration overrides the writer's REST catalog setting, forcing Iceberg to use AWS's GlueCatalog implementation, which expects a traditional S3 warehouse path instead of the S3 Tables warehouse format.

Contact Striim Support for guidance on applying the AWS-supplied Iceberg patch and bootstrap script to the EMR cluster (see Limitations). Without the patch, avoid relying on CREATE TABLE DDL propagation with the Glue catalog path.

DDL operations failing.

Unsupported operation type (see DDL support); catalog permission issues; table already exists (for CREATE TABLE) or does not exist (for ALTER/DROP).

Confirm the operation is one of the supported types listed in DDL support. Check catalog permissions. For CREATE TABLE, confirm the table doesn't already exist; for ALTER or DROP, confirm it does.

Slow performance during CDC loads.

Upload policy with a very low event count; insufficient cluster resources; too many small batches.

Raise the event count. Scale the EMR cluster. Adjust the interval. Enable OptimizedMerge if the source supports partial-image updates.

Data type conversion errors.

Unsupported or mismatched source data type.

Review the data type mapping documentation for your source. Use column mapping. Check for special characters or values that don't convert cleanly to the target Iceberg type.

Throttling or added latency on S3 Tables API calls under heavy load.

AWS region-level API rate limits on S3 Tables, especially with multiple concurrent S3 Tables Writer applications in the same account and region (see Limitations).

Monitor for throttling errors. Request an AWS API quota increase proportional to your S3 Tables application count and data volume. Striim already retries throttled calls automatically with exponential back-off.