Skip to content

feat(streaming): hedge slow demand-position article fetches on a second connection - #938

Merged
javi11 merged 1 commit into
docs/update-nzb-streaming-bench-resultsfrom
feat/hedged-demand-fetch
Sep 8, 2026
Merged

javi11 merged 1 commit into
docs/update-nzb-streaming-bench-resultsfrom
feat/hedged-demand-fetch

Conversation

@javi11

@javi11 javi11 commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #937 (base branch fix/clamp-range-end-past-eof); only the top commit is this PR.

Why

Instrumenting every article fetch against a real provider (20 connections) showed fetch p50 120–240 ms, p90 300–870 ms, and occasional 2.6–3.7 s fetches, arriving in pairs on one connection (requests queued behind another body). Delivery is in order, so one straggler at the read position stalls the stream while the read-ahead window sits complete behind it; in-flight collapsed to 2–3 for the stall's duration. On a 740 KB-article file, consumer throughput fell from 100–119 MB/s to 26 MB/s on a pass with six such stalls. Steady-state throughput is not the problem (100–150 MB/s), and widening max_prefetch made it worse (80 → 47–50 MB/s vs 30 → 100–119 MB/s; bigger bursts saturate the link and create more tails). The robust lever is cutting tail latency at the demand position.

What

  • A streaming fetch that has run past max(400 ms, 2× rolling median of the reader's last 16 fetch durations), is within demandDepth of the read position, and has received no bytes yet gets a second BodyStreamPriority for the same article. First success wins; the parent context is cancelled on return so the loser hits nntppool's abort-drain.
  • The no-first-byte guard matters: a fetch whose bytes are flowing is slow, not stuck, and is left alone. Without it the simulator's seek-storm profile fetched every 4 MiB article twice (articles_per_read 2.00); with it 1.00, read latency unchanged (p50 1580 → 1583 ms).
  • One hedge per article, at most demandDepth (2) hedges in flight per reader, speculative articles never hedged, hedges never take a speculative-budget slot. ErrArticleNotFound from either side is authoritative; the existing miss re-check path is unchanged. Metrics count each article once.
  • Both attempts publish into the shared articleBuf, length-guarded (hedgeWriter/liveLocked), so a slow original keeps feeding readers until the hedge overtakes it. attemptWriter() clears the hedge marker so retry semantics are unchanged.
  • Known limitation: nntppool dispatches priority requests to any connection with a free slot, so a hedge can land on the straggler's own connection; there is no steering API. In practice the busy connection is less likely to win the pull.

Test plan

  • hedge_test.go: slow demand article hedged and loser cancelled; progressing (bytes flowing) article not hedged; speculative never hedged; at most one hedge per article; 430 keeps miss semantics (SegmentID, MissConfirmed, one STAT); metrics counted once; policy threshold and in-flight cap
  • go vet, golangci-lint run ./internal/usenet/... (0 issues), go test -race -count=1 ./internal/usenet/... ./internal/pool/... ./internal/nzbfilesystem/..., -count=5 -run TestHedge (no flakes), go build ./cmd/altmount
  • Simulator bench (BenchmarkStream): B3 seek-storm/slow-4m articles_per_read 1.00 after the guard; B2 throughput flat (−1.7 % / +3.1 %)
  • Live validation pending. Three identity-verified A/B rounds (control vs this build, cold 256 MiB reads) on the real provider saw no stall events at all that morning, so the hedge never triggered and there is nothing to credit it for yet. Its benefit only shows when provider tails appear; watch for hedging slow demand article fetch / hedged article fetch resolved at Debug level on a bad-provider day, or add a fault-injecting fake to the bench.

…nd connection

Instrumenting every article fetch against a real provider (20 connections)
showed a p50 of 120-240 ms with a tail of 300-870 ms at p90 and occasional
2.6-3.7 s fetches, arriving in pairs on one connection: requests queued behind
another body. Delivery is in order, so one such straggler at the read position
stalls the whole stream while the read-ahead window sits complete behind it;
consumer throughput on a 740 KB-article file fell from 100-119 MB/s to 26 MB/s
on a pass with six stalls, and widening max_prefetch made it worse.

Once a demand-position fetch (read position or the segment after it) has run
past max(400 ms, 2x the rolling median of recent fetches) without receiving a
byte, the reader issues a second BodyStreamPriority for the same article and
takes whichever completes first, cancelling the other so its connection
drains. A fetch whose bytes are already flowing is slow, not stuck, and is
left alone, so large articles on a slow provider are not fetched twice.
Speculative articles are never hedged, an article is hedged at most once, at
most demandDepth hedges are in flight per reader, and a 430 from either
request is authoritative so the miss re-check path is unchanged. Both
attempts publish into the shared article buffer, length-guarded, so a slow
original keeps feeding readers until the hedge overtakes it.
@javi11
javi11 force-pushed the feat/hedged-demand-fetch branch from ecfa01d to 495b494 Compare September 8, 2026 08:11
@javi11
javi11 changed the base branch from fix/clamp-range-end-past-eof to docs/update-nzb-streaming-bench-results September 8, 2026 08:11
javi11 added a commit that referenced this pull request Sep 8, 2026
…ch (#938)

Reran raw+altmount against v0.3.2-92-g495b4949 (PR #938 stacked on this
branch): seq 67.7->71.7 MB/s, p05 34.4->41.4 MB/s, RSS/item 557->537 MiB,
no new capability gaps.
@javi11
javi11 merged commit fc3b42a into docs/update-nzb-streaming-bench-results Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant