feat: RFC 8054 COMPRESS DEFLATE wire compression + dep upgrades - #53
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR implements RFC 8054 COMPRESS DEFLATE wire compression for proxy→backend connections and upgrades several key dependencies (foyer 0.21→0.22, nix 0.30→0.31, rand 0.8→0.9, ratatui 0.29→0.30, sysinfo 0.37→0.38).
Changes:
- Adds bidirectional deflate compression support using raw DEFLATE format (no zlib header) with automatic negotiation after authentication
- Implements
DecompressStreamwrapper with efficient poll-based async I/O, separate write/flush buffers, and bandwidth statistics - Adds
compress(auto/require/disable) andcompress_level(0-9) configuration options per backend server - Migrates foyer to use explicit runtime spawner instead of RuntimeOptions, updates rand API calls, and adds ratatui trait bounds
Reviewed changes
Copilot reviewed 22 out of 23 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/compression.rs | Complete rewrite: implements bidirectional deflate compression with poll-based I/O, buffer management, and comprehensive test coverage |
| src/pool/deadpool_connection.rs | Adds compression negotiation after auth, wraps streams in DecompressStream on 206 response |
| src/connection_error.rs | Adds CompressionRequired error variant for when compression is required but not supported |
| src/stream.rs | Adds CompressedPlain and CompressedTls enum variants and helper constructors |
| src/protocol/commands.rs | Adds COMPRESS_DEFLATE command constant |
| src/protocol/mod.rs | Re-exports COMPRESS_DEFLATE command |
| src/protocol/response.rs | Removes #[must_use] from as_u16() method |
| src/config/types.rs | Adds compress and compress_level fields to Server and ServerBuilder |
| src/config/loading.rs | Initializes new compression fields to None in environment parsing |
| src/session/retry.rs | Updates rand 0.9 API: thread_rng() → rng(), gen_range() → random_range() |
| src/cache/hybrid.rs | Migrates foyer 0.22: BlockEngineBuilder → BlockEngineConfig, creates explicit runtime, NoopIoEngineBuilder → NoopIoEngineConfig |
| src/tui/mod.rs | Adds Send + Sync + 'static bound on B::Error for ratatui 0.30 compatibility |
| tests/test_helpers.rs | Adds compress and compress_level fields to test config construction |
| tests/test_properties.proptest-regressions | Proptest regression cases for status code validation |
| src/proxy/mod.rs | Updates test configs with new compression fields |
| src/pool/prewarming.rs | Updates test configs with new compression fields |
| src/pool/provider.rs | Passes compression parameters through to TcpManager |
| scripts/profile*.sh | Adds --features zlib-ng flag for SIMD-optimized profiling builds |
| flake.nix | Adds cmake dependency required for zlib-ng feature |
| Cargo.toml | Upgrades dependencies and adds flate2 with rust_backend, pin-project-lite |
| Cargo.lock | Dependency lock file updates for all upgraded and new dependencies |
mjc
force-pushed
the
feature/wire-compression-rfc8054
branch
from
January 31, 2026 06:17
21ed278 to
6553f4b
Compare
Add bidirectional deflate compression for proxy→backend connections per RFC 8054. After authentication, the proxy sends COMPRESS DEFLATE; on 206 response, the TCP stream is wrapped in a DecompressStream codec that transparently compresses outbound commands and decompresses inbound responses. Key details: - Raw deflate (no zlib header) per RFC 8054 §2.2.2 - FlushCompress::Sync flush for interactive NNTP command/response - Refactored poll_write/poll_flush with DrainState to avoid per-call allocations and correctly handle partial inner writes - Configurable: compress (auto/require/disable) and compress_level (0-9) - Client→proxy traffic remains uncompressed - Pre-allocated buffers reduce heap fragmentation on the hot path
- foyer 0.21→0.22: BlockEngineBuilder→BlockEngineConfig, RuntimeOptions replaced with Spawner wrapping a dedicated tokio runtime, NoopIoEngineBuilder→NoopIoEngineConfig - nix 0.30→0.31: no code changes needed - rand 0.8→0.9: thread_rng()→rng(), gen_range()→random_range() - ratatui 0.29→0.30: Backend::Error is now an associated type, added Send + Sync + 'static bound on run_app generic - sysinfo 0.37→0.38: no code changes needed
mjc
force-pushed
the
feature/wire-compression-rfc8054
branch
from
January 31, 2026 06:29
6553f4b to
cf460c2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
DecompressStreamcodec that transparently compresses outbound commands and decompresses inbound responses. Configurable viacompress(auto/require/disable) andcompress_level(0-9).Compression details
FlushCompress::Syncflush for interactive NNTP command/responsepoll_write/poll_flushwithDrainStateto avoid per-call allocations and handle partial inner writeszlib-ngfeature flag for SIMD-optimized compression (requires cmake)Dependency upgrades
BlockEngineBuilder→BlockEngineConfig,RuntimeOptions→Spawner,NoopIoEngineBuilder→NoopIoEngineConfigthread_rng()→rng(),gen_range()→random_range()Backend::Errorassociated type, addedSend + Sync + 'staticboundTest plan
cargo checkpasses on each commit independentlycargo test -p nntp-proxy --lib— 1240 tests pass on each commitcargo clippy --all-targetsclean on each commitcargo fmt --checkclean on each commit🤖 Generated with Claude Code