Skip to content

Commit 508b224

Browse files
docs(adr0014): Appendix A — consolidated test report index, evidence map, and one-line verdict (Case 1 + Case 2)
Co-authored-by: FluffyAIcode <FluffyAIcode@users.noreply.github.com>
1 parent ab6e088 commit 508b224

1 file changed

Lines changed: 44 additions & 0 deletions

File tree

docs/adr/0014-agent-connection-capacity-and-cross-host-topology-tests.md

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -294,3 +294,47 @@ on a low-RTT link, not to chase a faster WAN.
294294
- **Hold live cloud→Mac gRPC sessions for Case 1.** Impossible: the Mac has no
295295
inbound path (the reason the bridge exists). The load test runs co-located on
296296
the Mac, dispatched via the bridge.
297+
298+
## Appendix A — Test report index & evidence
299+
300+
Consolidated record of every run behind this ADR (harnesses, how to reproduce,
301+
the committed evidence JSON, and the headline result).
302+
303+
### A.1 Harnesses & how to reproduce
304+
305+
| Test | Harness / preset | Reproduce |
306+
| --- | --- | --- |
307+
| Case 1 — agent connections (light) | `scripts/research/grpc_agent_capacity_loadtest.py`; preset `agent-capacity-loadtest` | `kakeya_mac.py run --preset agent-capacity-loadtest` |
308+
| Case 1 — agent connections (stress) | same; preset `agent-capacity-stress` (`--context-len`, FD raise) | `kakeya_mac.py run --preset agent-capacity-stress` |
309+
| Case 2 — injected-RTT sweep | `scripts/research/k3_specdecode_gpu_bench.py --rtt-sweep` | H200, real models |
310+
| Case 2 — raw socket (real net) | `socket_echo_server.py` + `k3_specdecode_gpu_bench.py --socket-echo-addr` | echo on host B; reverse-SSH path |
311+
| Case 2 — direct gRPC | `grpc_echo_probe.py` + `k3_specdecode_gpu_bench.py --grpc-echo-addr` | gRPC echo on host B |
312+
313+
### A.2 Consolidated results
314+
315+
**Case 1 (Mac mini M4, gRPC `RuntimeService`, cpu Qwen3-0.6B):**
316+
317+
| run | result | evidence |
318+
| --- | --- | --- |
319+
| light sessions | **256/256 agents, 0 errors**; per-session KV 7.80 MB; node bound ≈2.0 GB; RSS flat ~3.85 GB | `results/research/k3_agent_capacity_mac.json` |
320+
| stress (ctx prefill, FD 100k, cap 2048) | FD not the limit; mem = cap×window (cap 2048→11.5 GB, bound 61 GB>RAM); serialization caps heavy-ctx concurrency at **~8** | `results/research/k3_agent_capacity_stress_mac.json` |
321+
322+
**Case 2 (H200 NVL, Gemma-4-26B + DFlash, fused spec-decode vs AR):**
323+
324+
| transport / RTT | tok/s | vs AR | evidence |
325+
| --- | --- | --- | --- |
326+
| co-located (0 network) | 44–52 | **1.85–2.20×** | (all four JSONs) |
327+
| injected-RTT sweep || 2.20× @0**0.98× @100 ms** → 0.77× @150 ms | `k3_crosshost_rtt_gpu.json` |
328+
| loopback gRPC (156 KB) || 1.1 ms round-trip | `k3_crosshost_grpc_gpu.json` |
329+
| raw socket over ~102 ms path | 14.1 | 0.56× | `k3_crosshost_realnet_gpu.json` |
330+
| direct gRPC over ~102 ms path | 15.95 | 0.63× | `k3_crosshost_grpc_gpu.json` |
331+
| RTT decomposition | 64 B → 102.6 ms; 156 KB → 206–208 ms (both raw + gRPC) || `k3_crosshost_socket_loopback_gpu.json` |
332+
333+
### A.3 One-line verdict
334+
335+
Bounded-memory, admission-controlled multi-agent serving is **validated** (Case 1:
336+
256+ connections, ~2.0 GB node KV ceiling, flat RSS). Cross-host token-level
337+
spec-decode is a **co-located/LAN win (1.8–2.2×)** and a **WAN net loss**
338+
(~0.56–0.63× at ~102 ms RTT, transport-independent) — **WAN = control + tool
339+
plane, LAN = data plane.** The lever is RTT (co-location), not the transport;
340+
gRPC only helps once the link is already low-RTT (loopback 1.1 ms).

0 commit comments

Comments
 (0)