perf(reach): bound concurrent cold-cache scans across distinct node requests #341

Context

Identified by a static comparison on 2026-10-07 against dborup/CoreScope master 6444294d938083f6d935d0445c27bc6a7bbf360e. This is follow-up work for evaluation and implementation, not a claim of an observed production incident. Recheck current master before implementation. No deployment is authorized by this issue alone.

Upstream reference: Kpa-clawbot fork, PR 2127 (reference only; intentionally not linked to avoid upstream notifications)

Benefit and existing coverage

Our cmd/server/node_reach.go coalesces identical cache keys with singleflight, but distinct node/window keys can still compute concurrently without an equivalent global build limit. Limit expensive cold work so other endpoints retain database capacity.

Related #221 evaluates regional /api/nodes pool pressure; that is a different endpoint and is not coverage for Reach. Our Reach leaderboard/cache/privacy work must be preserved.

Required work / acceptance criteria

• Evaluate upstream's nonblocking two-build semaphore and HTTP 429 with Retry-After against our actual connection pool and other workloads.
• Acquire limits only for genuinely cold work after cache recheck, and always release on success, failure or cancellation.
• Keep warm hits responsive and retain same-key coalescing.
• Preserve our visibility filtering, ranking snapshots and response/cache semantics.
• Test many distinct keys, identical keys, cancellation, errors and saturation/recovery.
• Verify unrelated endpoints remain responsive on a representative offline fixture; do not load-test production.

A per-Reach cap is not a complete global database admission-control policy; coordinate any broader work with #221.