feat(streaming): hedge slow demand-position article fetches on a second connection - #938
Merged
javi11 merged 1 commit intoSep 8, 2026
Conversation
…nd connection Instrumenting every article fetch against a real provider (20 connections) showed a p50 of 120-240 ms with a tail of 300-870 ms at p90 and occasional 2.6-3.7 s fetches, arriving in pairs on one connection: requests queued behind another body. Delivery is in order, so one such straggler at the read position stalls the whole stream while the read-ahead window sits complete behind it; consumer throughput on a 740 KB-article file fell from 100-119 MB/s to 26 MB/s on a pass with six stalls, and widening max_prefetch made it worse. Once a demand-position fetch (read position or the segment after it) has run past max(400 ms, 2x the rolling median of recent fetches) without receiving a byte, the reader issues a second BodyStreamPriority for the same article and takes whichever completes first, cancelling the other so its connection drains. A fetch whose bytes are already flowing is slow, not stuck, and is left alone, so large articles on a slow provider are not fetched twice. Speculative articles are never hedged, an article is hedged at most once, at most demandDepth hedges are in flight per reader, and a 430 from either request is authoritative so the miss re-check path is unchanged. Both attempts publish into the shared article buffer, length-guarded, so a slow original keeps feeding readers until the hedge overtakes it.
javi11
force-pushed
the
feat/hedged-demand-fetch
branch
from
September 8, 2026 08:11
ecfa01d to
495b494
Compare
javi11
changed the base branch from
fix/clamp-range-end-past-eof
to
docs/update-nzb-streaming-bench-results
September 8, 2026 08:11
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #937 (base branch
fix/clamp-range-end-past-eof); only the top commit is this PR.Why
Instrumenting every article fetch against a real provider (20 connections) showed fetch p50 120–240 ms, p90 300–870 ms, and occasional 2.6–3.7 s fetches, arriving in pairs on one connection (requests queued behind another body). Delivery is in order, so one straggler at the read position stalls the stream while the read-ahead window sits complete behind it; in-flight collapsed to 2–3 for the stall's duration. On a 740 KB-article file, consumer throughput fell from 100–119 MB/s to 26 MB/s on a pass with six such stalls. Steady-state throughput is not the problem (100–150 MB/s), and widening
max_prefetchmade it worse (80 → 47–50 MB/s vs 30 → 100–119 MB/s; bigger bursts saturate the link and create more tails). The robust lever is cutting tail latency at the demand position.What
max(400 ms, 2× rolling median of the reader's last 16 fetch durations), is withindemandDepthof the read position, and has received no bytes yet gets a secondBodyStreamPriorityfor the same article. First success wins; the parent context is cancelled on return so the loser hits nntppool's abort-drain.articles_per_read2.00); with it 1.00, read latency unchanged (p50 1580 → 1583 ms).demandDepth(2) hedges in flight per reader, speculative articles never hedged, hedges never take a speculative-budget slot.ErrArticleNotFoundfrom either side is authoritative; the existing miss re-check path is unchanged. Metrics count each article once.articleBuf, length-guarded (hedgeWriter/liveLocked), so a slow original keeps feeding readers until the hedge overtakes it.attemptWriter()clears the hedge marker so retry semantics are unchanged.Test plan
hedge_test.go: slow demand article hedged and loser cancelled; progressing (bytes flowing) article not hedged; speculative never hedged; at most one hedge per article; 430 keeps miss semantics (SegmentID, MissConfirmed, one STAT); metrics counted once; policy threshold and in-flight capgo vet,golangci-lint run ./internal/usenet/...(0 issues),go test -race -count=1 ./internal/usenet/... ./internal/pool/... ./internal/nzbfilesystem/...,-count=5 -run TestHedge(no flakes),go build ./cmd/altmountBenchmarkStream): B3 seek-storm/slow-4marticles_per_read1.00 after the guard; B2 throughput flat (−1.7 % / +3.1 %)hedging slow demand article fetch/hedged article fetch resolvedat Debug level on a bad-provider day, or add a fault-injecting fake to the bench.