test(case2): real two-process socket + real-network cross-host confirmation

[FluffyAIcode]

What

Solidifies Case 2 (ADR 0014) with real two-process socket + real-network measurements and a direct-gRPC transport comparison, replacing the injected-sleep RTT model.

• scripts/research/socket_echo_server.py — raw-TCP echo (remote-host boundary).
• scripts/research/grpc_echo_probe.py — raw-bytes gRPC echo + RTT probe.
• scripts/research/k3_specdecode_gpu_bench.py — --socket-echo-addr (raw) and --grpc-echo-addr (gRPC) per-block real round-trip of the actual ≈156 KB aux payload.

Results (H200 NVL, real models, real network)

transport / path	156 KB RTT	fused tok/s	vs AR
co-located (no network)	—	47.0	1.85–2.20×
loopback gRPC (same host)	1.1 ms	—	—
raw socket over ~102 ms path	207 ms	14.1	0.56×
direct gRPC over ~102 ms path	208 ms	15.95	0.63×

• gRPC vs raw socket: gRPC is modestly better (0.63× vs 0.56×; ~197 vs ~232 ms/block) — it serializes the 156 KB more efficiently — but still a net loss. The ~102 ms geographic RTT dominates, not the transport.
• gRPC's real value is at loopback (1.1 ms for 156 KB) → on a low-RTT link the transport is free and the engine returns to its 1.8–2.2× win. This is "LAN = data plane".

On the ~102 ms (clarification)

It is not a gRPC RTT and not a floor — it's a raw round-trip between two different-region hosts (+ SSH-relay overhead). Decomposed: 64 B → 102.6 ms (pure inter-region latency); 40 KB–156 KB flat at ~206 ms (an extra round-trip, not bandwidth).

Optimization levers (= the architecture's prescription): (1) latency — co-locate the draft loop (same region 5–20 ms, LAN 0.5–2 ms, Thunderbolt sub-ms; ≤15 ms → 1.8–2.2×); (2) transport — gRPC/QUIC (free at low RTT, modest help at high RTT); (3) payload — fp8/int8/top-k cuts the 156 KB 2–4×; (4) fewer round-trips — larger blocks.

Environment notes: tc netem is blocked in the vast container (no NET_ADMIN); the non-SSH vast mapped ports accept SYNs but don't forward data, so real latency came from a real inter-host SSH path (the true direct-gRPC number would be ≤ the tunneled one). Documented in ADR 0014 §4.4.

Files

• scripts/research/{socket_echo_server,grpc_echo_probe}.py (new)
• scripts/research/k3_specdecode_gpu_bench.py (--socket-echo-addr, --grpc-echo-addr, --netem-sweep)
• docs/adr/0014-…md §4.3/§4.4 + README.md
• results/research/k3_crosshost_{realnet,socket_loopback,grpc}_gpu.json

Testing

• ✅ Loopback gRPC on H200 (156 KB → 1.1 ms); co-located fused 1.85–2.20× AR.
• ✅ Real-network (~102 ms) over reverse SSH path: raw socket 0.56× AR, direct gRPC 0.63× AR.
• ✅ RTT decomposed by payload (64 B → 102.6 ms; 156 KB → 206–208 ms) for both raw + gRPC.
• ⚠️ tc netem + non-SSH direct ingress unavailable in the vast container — used a real inter-host path instead.