Troubleshooting Iceberg Writer
See Dataproc > Documentation > Guides > Troubleshoot VM out-of-memory errors and Troubleshoot job delays.
Dealing with backpressure
If Iceberg Writer exhibits backpressure (see Understanding and managing backpressure), you may need to adjust the upload policy or add worker nodes to the Dataproc cluster.
Iceberg Writer load is not progressing
When Iceberg Writer appears to have stalled, open the Striim console and run the show <namespace>.<Iceberg Writer name> status; command to get the Dataproc job ID, then in Google Cloud Console open the Dataproc job details page for that job. If the status is RUNNING but there are no entries in the log, that indicates that there is not enough memory in the Dataproc master node to start the execution of the job. Cancel other Dataproc jobs that are no longer needed, or switch to a Dataproc cluster with a master node with more memory.
If the logs contain multiple entries of WARN YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources, that indicates that the underlying Spark engine is unable to find enough executors to execute the current batch operation. Add additional worker nodes to the cluster or switch to a Dataproc cluster with worker nodes that have more vCPUs and memory.