Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,48 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [0.5.2] - 2026-06-11

### Added

- Added per-backend `stat_missing` retry probing, so backends that correctly answer `STAT` with `430` can help retry missing articles faster.
- Runtime CPU pinning support is now available for multi-threaded workloads via a new `CpuPinning` runtime mode, using worker-thread startup hooks so pinned threads are the ones running proxy work.
- Linux CPU pinning now uses [`rustix`](https://docs.rs/rustix)-based affinity handling, so we can remove the direct `nix` pinning dependency path.
- Added TUI session/user-count lifecycle fixes so active sessions stay tied to the session-owned gauge instead of drifting or disappearing.

Comment thread
mjc marked this conversation as resolved.
### Changed

- Wired response write metrics to runtime config and reduced response-metrics logging noise.
- Added queue backpressure routing configuration in `config.full.toml` and documented
`[routing.queue.backpressure]` in operator configuration docs.
- Finalized v0.5.2 release metadata and config coverage, including response-write and
client-writer lock contention interval settings.
- Removed legacy CLI compatibility aliases and aliases that are now obsolete after the cleanup pass: `--no-tui`, `--backend-strategy`, `--cache-capacity`, `--cache-ttl`, `--ttl-secs`, `--cache-articles`, `--store-articles`, and legacy `NNTP_PROXY_CACHE_*` env fallbacks.
- Removed the dead ordered large-transfer pipeline path and associated retry gate/feature code now that it is permanently disabled.
- Replaced visibility and internal API cleanup work from the Rust 1.88/private-first pass, including module privacy tightening and simplification of small conditional branches.
- Replaced `ResponseTransferError::into_anyhow` and `duration_polyfill` with idiomatic Rust 1.88-safe error and duration handling at call sites.
- Adopted private-first encapsulation by tightening crate module visibility where public re-exports already provide the intended external surface.
- Pushed the session-count model through typed metrics snapshots, dashboard conversions, and TUI rendering so count mixups become compile-time errors instead of runtime bugs.

### Fixed

- Backend DNS resolution now uses hickory’s TTL-aware caching behavior, with refreshed lookup handling that avoids unnecessary IPv4 cache clears on IPv6-unreachable failures.
- Tightened `unsafe` boundaries in the pooled-buffer and Windows file-replacement paths, and moved the old `review_claims` coverage into the RFC4643 auth/bypass and buffer test modules.
- Hardened retry-path routing and guard handling around pending counts, capacity-weighted initial article probing, and idle-pool preference.
- Fixed a user-active connection-count regression that could diverge from true active-session totals; the dashboard now consistently reports active sessions from typed user-metric counters.
- Migrated user/session metric collection to typed newtypes (including `ZERO` constructors) and arithmetic helpers so typed counters are incremented at source and cannot be accidentally mixed.
- Fixed metric and stats storage consistency by updating update paths to use typed metrics throughout collection, reducing drift in user gauges and totals.
- Reduced TUI render allocations by writing directly into the buffer, borrowing chart-label names, and using stack-backed formatting where possible.

### Docs

- Documented the `stat_missing` backend option and routing behavior in operator docs.
- Updated `docker-compose.yml`/`Dockerfile` examples and Nix module documentation
to cover `stat_missing` deployment usage.
- Added release metadata/docs updates for v0.5.2 configuration fields such as
response write and client-writer lock contention metric intervals.
- Added documentation for cleanup and migration context from the maintenance PR stack
around private-first cleanup, deprecated options removal, and runtime pinning behavior.

## [0.5.1] - 2026-06-05

Expand Down Expand Up @@ -561,6 +599,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Development documentation

[0.5.1]: https://github.com/mjc/nntp-proxy/compare/v0.5.0...v0.5.1
[0.5.2]: https://github.com/mjc/nntp-proxy/compare/v0.5.1...v0.5.2
[0.5.0]: https://github.com/mjc/nntp-proxy/compare/v0.4.0...v0.5.0
[0.4.0]: https://github.com/mjc/nntp-proxy/compare/v0.3.0...v0.4.0
[0.3.0]: https://github.com/mjc/nntp-proxy/compare/v0.2.3...v0.3.0
Expand Down
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "nntp-proxy"
version = "0.5.1"
version = "0.5.2"
edition = "2024"
rust-version = "1.88"
description = "NNTP proxy server with per-command backend multiplexing, caching, metrics, and TUI dashboard"
Expand Down
2 changes: 2 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ ENV NNTP_PROXY_PORT=8119 \
# ENV NNTP_SERVER_0_HOST=news.example.com
# ENV NNTP_SERVER_0_PORT=119
# ENV NNTP_SERVER_0_NAME="News Server 1"
# ENV NNTP_SERVER_0_STAT_MISSING=1
# ENV NNTP_SERVER_0_USERNAME=""
# ENV NNTP_SERVER_0_PASSWORD=""
# ENV NNTP_SERVER_0_MAX_CONNECTIONS=10
Expand All @@ -68,6 +69,7 @@ ENV NNTP_PROXY_PORT=8119 \
# ENV NNTP_SERVER_1_HOST=news2.example.com
# ENV NNTP_SERVER_1_PORT=119
# ENV NNTP_SERVER_1_NAME="News Server 2"
# ENV NNTP_SERVER_1_STAT_MISSING=1
# ENV NNTP_SERVER_1_USERNAME=""
# ENV NNTP_SERVER_1_PASSWORD=""
# ENV NNTP_SERVER_1_MAX_CONNECTIONS=10
Expand Down
18 changes: 18 additions & 0 deletions config.full.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ validate_yenc = true
# Minimum log level for debug.log file (default: "warn")
# Options: "error", "warn", "info", "debug", "trace"
log_file_level = "warn"
# Optional interval in seconds for response write metrics logging (default: disabled)
# response_write_metrics_secs = 10
# Optional interval in seconds for client writer lock contention metrics (default: disabled)
# client_writer_lock_metrics_secs = 10
# Optional stats persistence path used by the TUI dashboard
# stats_file = "/var/lib/nntp-proxy/stats.json"

Expand All @@ -31,6 +35,20 @@ backend_selection = "least-loaded"
# Enable adaptive availability prechecking for STAT/HEAD commands
adaptive_precheck = false

# Queue backpressure keeps article routing from piling more work onto
# already-saturated backend connection pools.
[routing.queue.backpressure]
enabled = true
# Soft threshold for queued requests per backend connection, as a percentage.
# Above this, routing penalizes the backend but may still select it.
soft_waiters_per_connection_percent = 25
# Hard threshold for queued requests per backend connection, as a percentage.
# Above this, routing filters the backend while alternatives exist in the tier.
hard_waiters_per_connection_percent = 50
# Sleep duration in milliseconds before retrying when every eligible backend
# in the preferred tier is above the hard threshold.
all_busy_sleep_ms = 1

# Memory Configuration
# These settings affect transport buffers and buffer pools, not article storage.
[memory]
Expand Down
4 changes: 4 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ services:
NNTP_SERVER_0_HOST: news.example.com
NNTP_SERVER_0_PORT: 119
NNTP_SERVER_0_NAME: "Primary News Server"
NNTP_SERVER_0_STAT_MISSING: 1 # Probe 430s with STAT on backends that support it
# NNTP_SERVER_0_USERNAME: your_username # Optional: backend authentication
# NNTP_SERVER_0_PASSWORD: your_password # Optional: backend authentication
# NNTP_SERVER_0_MAX_CONNECTIONS: 10 # Optional: max connections (default: 10)
Expand Down Expand Up @@ -87,18 +88,21 @@ services:
NNTP_SERVER_0_HOST: news1.example.com
NNTP_SERVER_0_PORT: 119
NNTP_SERVER_0_NAME: "News Server 1"
NNTP_SERVER_0_STAT_MISSING: 1
NNTP_SERVER_0_USERNAME: ${BACKEND_USER_0:-}
NNTP_SERVER_0_PASSWORD: ${BACKEND_PASS_0:-}

NNTP_SERVER_1_HOST: news2.example.com
NNTP_SERVER_1_PORT: 119
NNTP_SERVER_1_NAME: "News Server 2"
NNTP_SERVER_1_STAT_MISSING: 1
NNTP_SERVER_1_USERNAME: ${BACKEND_USER_1:-}
NNTP_SERVER_1_PASSWORD: ${BACKEND_PASS_1:-}

NNTP_SERVER_2_HOST: news3.example.com
NNTP_SERVER_2_PORT: 119
NNTP_SERVER_2_NAME: "News Server 3"
NNTP_SERVER_2_STAT_MISSING: 1
NNTP_SERVER_2_USERNAME: ${BACKEND_USER_2:-}
NNTP_SERVER_2_PASSWORD: ${BACKEND_PASS_2:-}

Expand Down
28 changes: 21 additions & 7 deletions docs/operator/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,14 @@ Keep the config ordered from general settings to concrete identities:

1. `[proxy]`
2. `[routing]`
3. `[memory]`
4. `[cache]`
5. `[cache.disk]`
6. `[health_check]`
7. `[client_auth]`
8. `[[client_auth.users]]`
9. `[[servers]]`
3. `[routing.queue.backpressure]`
4. `[memory]`
5. `[cache]`
6. `[cache.disk]`
7. `[health_check]`
8. `[client_auth]`
9. `[[client_auth.users]]`
10. `[[servers]]`

See [../../config.full.toml](../../config.full.toml) for a complete example.

Expand All @@ -38,6 +39,8 @@ See [../../config.full.toml](../../config.full.toml) for a complete example.
| `threads` | `1` | Use `0` for CPU-count worker threads |
| `validate_yenc` | `true` | Validate yEnc structure/checksums |
| `log_file_level` | `"warn"` | Filter for the optional local `debug.log` appender |
| `response_write_metrics_secs` | unset | Optional interval for response write metrics logging |
| `client_writer_lock_metrics_secs` | unset | Optional interval for client-writer lock contention metrics logging |
| `stats_file` | unset | If unset, metrics persistence defaults to `stats.json` next to the config file |

### `[routing]`
Expand All @@ -48,6 +51,15 @@ See [../../config.full.toml](../../config.full.toml) for a complete example.
| `backend_selection` | `least-loaded` | Or `weighted-round-robin` |
| `adaptive_precheck` | `false` | Adaptive availability precheck for `STAT`/`HEAD` |

### `[routing.queue.backpressure]`

| Field | Default | Notes |
| --- | --- | --- |
| `enabled` | `true` | Enable queue-pressure-aware backend filtering |
| `soft_waiters_per_connection_percent` | `25` | Penalize backends above this queued-requests-per-connection threshold |
| `hard_waiters_per_connection_percent` | `50` | Filter backends above this threshold while alternatives exist in the tier |
| `all_busy_sleep_ms` | `1` | Sleep before retrying when every eligible backend in the preferred tier is hard-saturated |

### `[memory]`

These settings control transport memory, not stored article bodies.
Expand Down Expand Up @@ -113,6 +125,7 @@ password = "reader-password"
| `name` | required | Friendly name used in logs and the TUI |
| `username` / `password` | unset | Backend auth |
| `max_connections` | `10` | Per-backend pool size |
| `stat_missing` | `0` | Probe missing articles with `STAT` before `ARTICLE`/`BODY`/`HEAD` on this backend. Enable it on backends that correctly return `430` to speed up retrying missing articles. |
| `tier` | `0` | Lower tiers are preferred first |
| `use_tls` | `false` | TLS to the backend |
| `tls_verify_cert` | `true` | Keep enabled in production |
Expand All @@ -137,6 +150,7 @@ Currently supported backend environment fields:
- `USERNAME`
- `PASSWORD`
- `MAX_CONNECTIONS`
- `STAT_MISSING`
- `USE_TLS`
- `TLS_VERIFY_CERT`
- `TLS_CERT_PATH`
Expand Down
3 changes: 3 additions & 0 deletions docs/operator/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,14 @@ docker run -d \
-e NNTP_SERVER_0_HOST=news.example.com \
-e NNTP_SERVER_0_PORT=119 \
-e NNTP_SERVER_0_NAME=Primary \
-e NNTP_SERVER_0_STAT_MISSING=1 \
-e NNTP_SERVER_0_USERNAME="$BACKEND_USER" \
-e NNTP_SERVER_0_PASSWORD="$BACKEND_PASS" \
nntp-proxy
```

Set `NNTP_SERVER_*_STAT_MISSING=1` on backends that correctly return `430` for missing articles if you want the proxy to prefetch those misses with `STAT`.

The repository also includes [../../docker-compose.yml](../../docker-compose.yml).

## Metrics and state files
Expand Down
1 change: 1 addition & 0 deletions nix/module.nix
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,7 @@ in {
host = "news.example.com";
port = 563;
name = "Primary";
stat_missing = 1;
use_tls = true;
tls_verify_cert = true;
max_connections = 20;
Expand Down
Loading