Skip to main content

BigQuery Writer runtime considerations

BigQuery Writer known limitations

  • If you configure BigQuery Writer to exceed Google's quotas and limits, your application may halt. You can minimize the chances of this happening by configuring BigQuery Writer to use the Storage Write API. Quotas and limits are subject to change by Google without notice; numbers in this document are up to date as of July 2023. For the latest information, see BigQuery > Documentation > Reference > Quotas and limits.

  • When using the Storage Write API, if a single row contains approximately 5MB of data or more, the BigQuery Writer application may terminate. This is due to Storage Write API's AppendRows request size limit of 10MB (metadata may increase the request size to be twice that of the original data).

  • Schema evolution with BigQuery targets does not support MODIFY COLUMN, DROP COLUMN, ADD PRIMARY KEY, or ADD CONSTRAINT.