Skip to content

docs: spell out "FD" → open-file-descriptor limit (RLIMIT_NOFILE) #125

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cursor merged 1 commit into from
Jun 14, 2026
Merged

docs: spell out "FD" → open-file-descriptor limit (RLIMIT_NOFILE) #125

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 9 additions & 7 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -247,13 +247,15 @@ channel + session each — against one runtime:
Caveat: v0.3 is **single-tenant** (one shared verifier, RPCs serialized on one
asyncio loop — per-session binding is a v0.4 / PR-A3c item), so create/generate
latency is linear in N and "256" is the max concurrent connections *served*, not
parallel inferences. Pushing further (preset `agent-capacity-stress`, FD raised
to 100k / hard unlimited on the Mac) shows the true ceilings: **FD is not the
limit**; **memory** scales with `capacity × window` (capacity 2048 @ window 256
→ ~11 GB RSS, theoretical node bound ~61 GB > 24 GB RAM, so capacity must be
sized to RAM); and with a per-agent **context** prefill the binding constraint
is **single-tenant serialization** (concurrent heavy-prefill agents serialize
and time out well before any FD/connection limit). Bounded memory is structural:
parallel inferences. Pushing further (preset `agent-capacity-stress`, the
open-file-descriptor limit `RLIMIT_NOFILE` raised to 100k / hard unlimited on
the Mac — each connection uses one descriptor) shows the true ceilings: **the
open-file-descriptor limit is not the constraint**; **memory** scales with
`capacity × window` (capacity 2048 @ window 256 → ~11 GB RSS, theoretical node
bound ~61 GB > 24 GB RAM, so capacity must be sized to RAM); and with a
per-agent **context** prefill the binding constraint is **single-tenant
serialization** (concurrent heavy-prefill agents serialize and time out well
before any file-descriptor / connection limit). Bounded memory is structural:
light-session agent count does **not** grow RSS; the memory lever is the
resident **window**, not the number of agents.

Expand Down
11 changes: 6 additions & 5 deletions docs/adr/0014-agent-connection-capacity-and-cross-host-topology-tests.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,8 @@ behavior; the served **MLX gemma** path is a separate v0.4 gap (§6).

### 3.2b Stress beyond 256 — the real ceilings (preset `agent-capacity-stress`)

Pushing further with the FD limit raised (`RLIMIT_NOFILE` soft 100k, hard
unlimited on the Mac) and a **per-agent context prefill** (window 256,
Pushing further with the open-file-descriptor limit raised (`RLIMIT_NOFILE`
soft 100k, hard unlimited on the Mac) and a **per-agent context prefill** (window 256,
`--context-len 256`, capacity 2048):

| agents | created | create p95 | per-session KV | server RSS |
Expand All @@ -94,7 +94,8 @@ unlimited on the Mac) and a **per-agent context prefill** (window 256,
| 8 | 8/8 | 25.2 s | 29.8 MB | 11 343 MB |
| 16 | 15/16 (1 `RpcCancelled`) | 44.6 s | 29.8 MB | 10 781 MB |

- **FD is not the ceiling** (raised to 100k; Mac hard limit is unlimited).
- **The open-file-descriptor limit is not the ceiling** (raised to 100k; Mac
hard limit is unlimited) — each gRPC channel/session consumes one descriptor.
- **Memory** scales with `capacity × window`: capacity 2048 @ window 256 →
**~11.5 GB RSS**, and the theoretical node bound is **~61 GB > 24 GB RAM** —
so capacity must be **sized to RAM** (it is the memory knob, not agent count).
Expand Down Expand Up @@ -305,7 +306,7 @@ the committed evidence JSON, and the headline result).
| Test | Harness / preset | Reproduce |
| --- | --- | --- |
| Case 1 — agent connections (light) | `scripts/research/grpc_agent_capacity_loadtest.py`; preset `agent-capacity-loadtest` | `kakeya_mac.py run --preset agent-capacity-loadtest` |
| Case 1 — agent connections (stress) | same; preset `agent-capacity-stress` (`--context-len`, FD raise) | `kakeya_mac.py run --preset agent-capacity-stress` |
| Case 1 — agent connections (stress) | same; preset `agent-capacity-stress` (`--context-len`, open-file-descriptor limit raise) | `kakeya_mac.py run --preset agent-capacity-stress` |
| Case 2 — injected-RTT sweep | `scripts/research/k3_specdecode_gpu_bench.py --rtt-sweep` | H200, real models |
| Case 2 — raw socket (real net) | `socket_echo_server.py` + `k3_specdecode_gpu_bench.py --socket-echo-addr` | echo on host B; reverse-SSH path |
| Case 2 — direct gRPC | `grpc_echo_probe.py` + `k3_specdecode_gpu_bench.py --grpc-echo-addr` | gRPC echo on host B |
Expand All @@ -317,7 +318,7 @@ the committed evidence JSON, and the headline result).
| run | result | evidence |
| --- | --- | --- |
| light sessions | **256/256 agents, 0 errors**; per-session KV 7.80 MB; node bound ≈2.0 GB; RSS flat ~3.85 GB | `results/research/k3_agent_capacity_mac.json` |
| stress (ctx prefill, FD 100k, cap 2048) | FD not the limit; mem = cap×window (cap 2048→11.5 GB, bound 61 GB>RAM); serialization caps heavy-ctx concurrency at **~8** | `results/research/k3_agent_capacity_stress_mac.json` |
| stress (ctx prefill, file-descriptor limit 100k, cap 2048) | open-file-descriptor limit not the constraint; mem = cap×window (cap 2048→11.5 GB, bound 61 GB>RAM); serialization caps heavy-ctx concurrency at **~8** | `results/research/k3_agent_capacity_stress_mac.json` |

**Case 2 (H200 NVL, Gemma-4-26B + DFlash, fused spec-decode vs AR):**

Expand Down
Loading