perf(import): faster, steadier imports — hedged fast-fail STATs, warmed first articles, dead-post fast path

[javi11]

Summary

Import-latency and robustness work driven by the nzb-streaming-benchmarks harness, run head-to-head against AIOStreams on the same corpus in the same session. Nine small commits, each TDD'd:

Fast-fail probe (internal/importer/validation)

• Hedge straggling STATs on an arrival lull, on the priority lane. A debug import showed the real shape: most of the 64 sampled STATs answer in ~150 ms and anywhere from 1 to 22 sit to the 2 s attempt ceiling — STATs pipelined behind slow/cold connections, not slow articles (a priority-lane re-issue answers in 50–90 ms). Once ≥8 answers are in and the sweep goes quiet for clamp(3×median, 250–750 ms), every outstanding id is re-issued with Priority: true. Debug logs record hedge decisions and answers.
• Retry while the sweep converges. A fixed 3×2 s budget refused a healthy release as inconclusive while STATs were still arriving (46→31→15). Retries continue while each attempt shrinks the unanswered set, within a 15 s budget; a non-converging attempt at/after the third still ends it.
• Pass with ≤2 unverified of a full sample. One article whose STAT neither the original nor the hedge could get answered held a healthy import 4.5 s; the probe answers from a sample and the article is handled at stream time like the thousands never sampled. Per-file sweep unchanged.
• Judge a dead post from an 8-STAT first wave and skip its per-file sweep; end a sweep attempt as soon as its answers condemn the release; open the sweep with a 16-STAT chunk. Three dead posts in a row used to cost hundreds of STATs answered as slow 430 spool lookups that stayed pipelined on the connections and poisoned the next import (bugonia 2–5 s, once inconclusive; a damaged-not-dead set swept for 18 s). After: dead posts fail in 2.6–8 s and the 7z imports right after them take 0.6–0.8 s.

Cold open / import critical path

• Publish warmed first articles to the streaming segment store (the 256 MB memory tier is on even without cache_path); cold TTFB 55 → 3–10 ms. Also warm the largest clean-named video (previously skipped) when a store is wired.
• Archive analysis reads warmed articles back (import-scoped cache reads through to the store) and warm-up pre-fetches the last 7z volume's tail: 7z header pass 1/3 → 3/3 cache hits, ~450 ms → ~40 ms.
• Parse's remaining round trips off the critical path: representative middle-segment header fetched during warm-up; PAR2 index reader given the store; store-only fetches detached from the wait the parse blocks on.

Observability (internal/webdav)

• WARN when a streamed body ends on a read error while the client is still connected. A bench playback died silently ("terminated", no health row, no log line); http.ServeContent swallows the reader's error. Client cancels are excluded.

Results (same-session, AIOStreams vs AltMount, 9 shared entries, several passes)

	AIOStreams	AltMount (before → after)
Cold TTFB	6–22 ms	55 → 3–10 ms
Seq MB/s	75–90	70 → 72–92
Playback p05	29–42	36 → 33–56
Full seek	480–540 ms	204–293 ms
CPU s/GiB	6.3–6.7	5.0–5.7
Click→byte	0.61–1.03 s	0.98–1.10 → 0.98–1.10 s (7z entries 2.9 → 0.6–1.0 s; season pack 0.9 → 0.6–0.7 s)
RSS/item	490–560 MiB	640–690 MiB (designed GC headroom + memory tier; not changed)
Capability	1 gap / 0 wrongly served	1 gap / 0 wrongly served

The back-to-back dead posts no longer slow the imports that follow them. Remaining per-entry differences are within the harness's own stated run-to-run noise; in the evening's cleanest pass click→byte was 1.10 s vs 1.03 s.

Test plan

• go test -race ./... green; new tests: fast_fail_hedge_test.go, fast_fail_patience_test.go, fast_fail_deadwave_test.go, segment_cache_fallback_test.go, parser_warm_store_test.go, adapter_get_test.go
• go build ./... && go vet ./...
• Nine full bench passes (raw + AltMount, and raw + AltMount + AIOStreams) plus targeted passes and a debug-logged repro import
• CI golangci-lint (local lint is broken against Go 1.27 export data)