Skip to content

feat: RFC 8054 COMPRESS DEFLATE wire compression + dep upgrades - #53

Merged
mjc merged 2 commits into
mainfrom
feature/wire-compression-rfc8054
Jan 31, 2026
Merged

mjc merged 2 commits into
mainfrom
feature/wire-compression-rfc8054

Conversation

@mjc

@mjc mjc commented Jan 31, 2026

Copy link
Copy Markdown
Owner

Summary

  • Implement RFC 8054 COMPRESS DEFLATE for proxy→backend connections: after authentication, the proxy negotiates deflate compression, wrapping the TCP stream in a DecompressStream codec that transparently compresses outbound commands and decompresses inbound responses. Configurable via compress (auto/require/disable) and compress_level (0-9).
  • Upgrade dependencies: foyer 0.22, nix 0.31, rand 0.9, ratatui 0.30, sysinfo 0.38 with corresponding migration code.

Compression details

  • Raw deflate (no zlib header) per RFC 8054 §2.2.2
  • FlushCompress::Sync flush for interactive NNTP command/response
  • poll_write/poll_flush with DrainState to avoid per-call allocations and handle partial inner writes
  • Client→proxy traffic remains uncompressed
  • Optional zlib-ng feature flag for SIMD-optimized compression (requires cmake)

Dependency upgrades

  • foyer 0.21→0.22: BlockEngineBuilderBlockEngineConfig, RuntimeOptionsSpawner, NoopIoEngineBuilderNoopIoEngineConfig
  • rand 0.8→0.9: thread_rng()rng(), gen_range()random_range()
  • ratatui 0.29→0.30: Backend::Error associated type, added Send + Sync + 'static bound
  • nix 0.30→0.31, sysinfo 0.37→0.38: no code changes needed

Test plan

  • cargo check passes on each commit independently
  • cargo test -p nntp-proxy --lib — 1240 tests pass on each commit
  • cargo clippy --all-targets clean on each commit
  • cargo fmt --check clean on each commit

🤖 Generated with Claude Code

Copilot AI review requested due to automatic review settings January 31, 2026 05:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements RFC 8054 COMPRESS DEFLATE wire compression for proxy→backend connections and upgrades several key dependencies (foyer 0.21→0.22, nix 0.30→0.31, rand 0.8→0.9, ratatui 0.29→0.30, sysinfo 0.37→0.38).

Changes:

  • Adds bidirectional deflate compression support using raw DEFLATE format (no zlib header) with automatic negotiation after authentication
  • Implements DecompressStream wrapper with efficient poll-based async I/O, separate write/flush buffers, and bandwidth statistics
  • Adds compress (auto/require/disable) and compress_level (0-9) configuration options per backend server
  • Migrates foyer to use explicit runtime spawner instead of RuntimeOptions, updates rand API calls, and adds ratatui trait bounds

Reviewed changes

Copilot reviewed 22 out of 23 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/compression.rs Complete rewrite: implements bidirectional deflate compression with poll-based I/O, buffer management, and comprehensive test coverage
src/pool/deadpool_connection.rs Adds compression negotiation after auth, wraps streams in DecompressStream on 206 response
src/connection_error.rs Adds CompressionRequired error variant for when compression is required but not supported
src/stream.rs Adds CompressedPlain and CompressedTls enum variants and helper constructors
src/protocol/commands.rs Adds COMPRESS_DEFLATE command constant
src/protocol/mod.rs Re-exports COMPRESS_DEFLATE command
src/protocol/response.rs Removes #[must_use] from as_u16() method
src/config/types.rs Adds compress and compress_level fields to Server and ServerBuilder
src/config/loading.rs Initializes new compression fields to None in environment parsing
src/session/retry.rs Updates rand 0.9 API: thread_rng()rng(), gen_range()random_range()
src/cache/hybrid.rs Migrates foyer 0.22: BlockEngineBuilderBlockEngineConfig, creates explicit runtime, NoopIoEngineBuilderNoopIoEngineConfig
src/tui/mod.rs Adds Send + Sync + 'static bound on B::Error for ratatui 0.30 compatibility
tests/test_helpers.rs Adds compress and compress_level fields to test config construction
tests/test_properties.proptest-regressions Proptest regression cases for status code validation
src/proxy/mod.rs Updates test configs with new compression fields
src/pool/prewarming.rs Updates test configs with new compression fields
src/pool/provider.rs Passes compression parameters through to TcpManager
scripts/profile*.sh Adds --features zlib-ng flag for SIMD-optimized profiling builds
flake.nix Adds cmake dependency required for zlib-ng feature
Cargo.toml Upgrades dependencies and adds flate2 with rust_backend, pin-project-lite
Cargo.lock Dependency lock file updates for all upgraded and new dependencies

Comment thread src/pool/deadpool_connection.rs Outdated
Comment thread src/protocol/response.rs
Comment thread src/config/types.rs
Comment thread src/pool/deadpool_connection.rs
@mjc
mjc force-pushed the feature/wire-compression-rfc8054 branch from 21ed278 to 6553f4b Compare January 31, 2026 06:17
mjc added 2 commits January 30, 2026 23:29
Add bidirectional deflate compression for proxy→backend connections
per RFC 8054. After authentication, the proxy sends COMPRESS DEFLATE;
on 206 response, the TCP stream is wrapped in a DecompressStream codec
that transparently compresses outbound commands and decompresses
inbound responses.

Key details:
- Raw deflate (no zlib header) per RFC 8054 §2.2.2
- FlushCompress::Sync flush for interactive NNTP command/response
- Refactored poll_write/poll_flush with DrainState to avoid per-call
  allocations and correctly handle partial inner writes
- Configurable: compress (auto/require/disable) and compress_level (0-9)
- Client→proxy traffic remains uncompressed
- Pre-allocated buffers reduce heap fragmentation on the hot path
- foyer 0.21→0.22: BlockEngineBuilder→BlockEngineConfig, RuntimeOptions
  replaced with Spawner wrapping a dedicated tokio runtime,
  NoopIoEngineBuilder→NoopIoEngineConfig
- nix 0.30→0.31: no code changes needed
- rand 0.8→0.9: thread_rng()→rng(), gen_range()→random_range()
- ratatui 0.29→0.30: Backend::Error is now an associated type, added
  Send + Sync + 'static bound on run_app generic
- sysinfo 0.37→0.38: no code changes needed
@mjc
mjc force-pushed the feature/wire-compression-rfc8054 branch from 6553f4b to cf460c2 Compare January 31, 2026 06:29
@mjc
mjc requested a review from Copilot January 31, 2026 06:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 26 changed files in this pull request and generated no new comments.

@mjc
mjc merged commit 55c9c1f into main Jan 31, 2026
8 checks passed
@mjc
mjc deleted the feature/wire-compression-rfc8054 branch January 31, 2026 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants