@@ -210,6 +210,31 @@ states + draft tokens, **≈156 KB/block** — once per block
210210 restricted netns), so real latency was obtained from a real inter-host link
211211 (the reverse-tunnel RTT) rather than synthetic netem.
212212
213+ ** What the ~ 102 ms is (and is not).** It is ** not** a gRPC RTT and ** not** a
214+ floor — it is a raw TCP round-trip through a ** reverse SSH tunnel between two
215+ different-region hosts** . Decomposed by payload over the real path:
216+
217+ | payload | median RTT |
218+ | --- | --- |
219+ | 64 B | 102.6 ms |
220+ | 40 KB | 205.2 ms |
221+ | 80 KB | 206.0 ms |
222+ | 156 KB | 206.9 ms |
223+
224+ The 64 B point (~ 102 ms) is the ** pure inter-region latency + SSH-relay
225+ overhead** ; the flat 205–207 ms from 40 KB up is ** one extra tunnel round-trip**
226+ (TCP windowing / SSH framing), not linear bandwidth. So this is a ** worst-case
227+ far-WAN + SSH artifact** , not a deployment floor. Optimization room is large and
228+ is exactly the architecture's prescription: (1) ** latency** — co-locate the
229+ draft loop on a low-RTT link (same region ~ 5–20 ms, LAN ~ 0.5–2 ms, Thunderbolt
230+ sub-ms; at ≤15 ms the engine is back to 1.8–2.2×, cf. §4.3 / loopback 2.02×);
231+ (2) ** transport** — a direct ** gRPC/QUIC/RDMA** path drops the SSH relay + the
232+ extra round-trip (gRPC would be * lower* , not higher); (3) ** payload** — the
233+ 156 KB/block fp16 aux can be fp8/int8/top-k compressed 2–4×; (4) ** fewer
234+ round-trips** — larger blocks. The invariant is the ratio * per-block RTT :
235+ per-block compute* (break-even ~ 100 ms/block): the strategy is to keep the loop
236+ on a low-RTT link, not to chase a faster WAN.
237+
213238## 5. Decision
214239
2152401 . ** Case 1 is validated** : the session-bound gRPC runtime admits and serves
0 commit comments