Skip to content

Not setting gcs chunk_size option overwrites library default #1292

@devoxel

Description

@devoxel

The default is 16MiB, and when you set Writer.ChunkSize to 0, GCS will disable disable chunking, not use the default.

Because the clickhouse-backup library uses the zero value from a yaml config, rather than not setting, it will remove completely the chunking feature by default.

This feels probably unintentional & undesirable, as it will fully disable transient error retries on the GCS side (which appears to be bugged also, see #1291).

For large files, it also increases the chances of hitting these transient errors, because files may take a long time to upload. We consistently observed GOAWAYs on these streams during the duration of the file upload.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions