Skip to content

feat: add ZSTD network compression support and dynamic compression detection - #104

Merged
Protryon merged 5 commits into
masterfrom
zstd-compression
Nov 6, 2026
Merged

Protryon merged 5 commits into
masterfrom
zstd-compression

Conversation

@negezor

@negezor negezor commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Modern ClickHouse instances use ZSTD by default (network_compression_method = 'zstd'), which previously caused the client to crash with a protocol error. Now, it dynamically detects whether the incoming packet is LZ4 or ZSTD and decodes it on the fly. I also fixed some minor double-allocation errors in compress_data.

SET network_compression_method = 'zstd';

Run any heavy SELECT query. It should fetch data successfully instead of throwing ProtocolError / UnexpectedEof.

P.S. Ran into an unpleasant issue in production today, which is why I’m here :)
klickhouse::client: clickhouse client failed: Io(Custom { kind: UnexpectedEof, error: ProtocolError("unexpected compression algorithm identifier: '90', expected 82 (LZ4)") })

P.S.S. I think we should also remove the CompressionMethod requirement for 'None'. Let it automatically check all ClickHouse packet headers so it can determine on its own whether compression is needed or not

@negezor
negezor requested a review from Protryon October 26, 2026 07:33
@negezor

negezor commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

@Protryon The Cargo.toml specifies edition = "2024", which requires Rust version 1.85 or higher. However, the minimum versions set in the CI for both nightly and stable are currently 1.75.

@Protryon
Protryon merged commit 2d81283 into master Nov 6, 2026
2 of 4 checks passed
@Protryon

Protryon commented Nov 6, 2026

Copy link
Copy Markdown
Owner

Thanks, I'll do a CI refresh and re-release today.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants