B-2 WAN-latency spike: interactive no-go, batch bandwidth-gated

[lai3d]

ROADMAP-AI-V3 gates B-2 (multi-miner pipeline inference) on a WAN-latency measurement spike before committing the 12–16h build. This is that spike: real measurements + a reproducible model + the scope decision. No production code — a runnable calculator (cargo run -p qfc-inference --example pipeline_latency) + two docs.

Real measured anchors

Path	RTT	Bandwidth	How
LAN — AWS us-east-1 intra-VPC (testnet A→B/C/D)	0.25 ms	~5 Gbit/s (assumed)	ping, 5×3 samples
Intercontinental — Singapore ↔ AWS us-east-1 (Virginia)	242 ms median	20 Mbit/s single-stream	TCP-handshake RTT ×8; 20 MB over SSH

The live testnet is single-region (B/C/D are LAN behind a ProxyJump), so it can't directly measure cross-region miner latency — these two anchors bracket it, with published AWS inter-region figures filling the middle.

Findings

1. Interactive autoregressive inference over WAN = NO-GO. Each generated token traverses the full pipeline (K−1 hops), so latency is T·(K−1)·RTT/2. qwen2.5-7b, K=4, 100 tokens — network alone: 0.77s regional, 6.1s continental, 37s intercontinental. Only same-datacenter/same-region passes a 2s bar, which negates the point of geographically distributed miners.

2. Batch prefill = viable but BANDWIDTH-gated — the spike's correction to the roadmap, which framed B-2's risk as RTT only. At the measured 20 Mbit/s, a 7B prefill hop (B8/S512 = 28.7 MB/boundary) is ~12s — transfer-bound, not RTT-bound. Even regional cross-AZ trips the 200ms/hop threshold. Bandwidth is co-equal with RTT.

Decision — ADR-0011

• B-2 drops interactive WAN inference (stays single-miner, as today).
• B-2 targets batch/async single-pass workloads only.
• B3 assignment must model bandwidth + locality (not just RTT): form network-local pipeline groups, carry activation-transfer cost as an explicit term, reject groups over a per-hop budget, prefer fewer stages.
• Activation compression (fp16→int8/fp8) becomes a B-2 requirement, folded into B4 prototype + B5 commitments.
• Hard gate before building B3–B5: re-run the calculator against real cross-region miner-to-miner RTT+BW — needs ≥2 miners in different regions (the single-region testnet can't produce this). Build only if a realistic post-compression locality tier clears the per-hop budget.

Files

• crates/qfc-inference/examples/pipeline_latency.rs — reproducible calculator (4 regression-guard tests)
• docs/spikes/B2-wan-pipeline-latency.md — methodology, measured data, full result tables
• docs/adr/0011-b2-pipeline-scope.md — the decision

🤖 Generated with Claude Code