You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
clodex retains a reusable WebSocket per live conversation so later turns send only the new message.
Those connections sit in two fixed-size pools (CLODEX_WS_MAX_NURSERY_CONNECTIONS, CLODEX_WS_MAX_CONNECTIONS). When a pool is full, the least-recently-used idle connection is dropped
and that conversation loses its cache — its next turn resends the full history over a new connection.
Every value we could pick for those caps is wrong for somebody:
It has to be guessed per machine (descriptor limits, memory) and per workload (how many agents the
user actually runs). fix(oauth): keep more cached conversations alive when many subagents run at once #221 raised the nursery cap to 48 explicitly as a safety valve rather than a
measured requirement, because nothing about the observed data identifies a correct number.
The user's actual intent is legible: if they start 200 agents, they want 200 agents. The resource
cost of that concurrency is real and it is theirs to pay. Churning connections underneath the number
they asked for costs more — in cache misses and in new-connection pressure — than holding them.
An arbitrary ceiling turns a resource question the OS can answer authoritatively into a tuning
parameter nobody can set correctly.
Proposal
Let the pools grow organically and let the operating system be the limit, reporting it clearly when it
is reached rather than quietly degrading before it.
Handle EMFILE (and ENFILE) on the connection-open path. There is currently no handling
anywhere in src/ — grep -rn "EMFILE" src/ returns nothing — so exhausting descriptors is an
unhandled failure rather than a degraded mode. This is the blocking prerequisite.
On EMFILE, shed load deliberately instead of failing the request: evict idle pooled
connections (oldest first) and retry the open. That converts descriptor exhaustion into exactly the
behaviour the caps provide today, but triggered by the real limit rather than a guess.
Tell the user, once, in terms they can act on — that the machine's open-file limit was reached,
how many connections were held, and that ulimit -n is the knob. Today the equivalent condition is
invisible unless they go trawling diagnostics.
Then remove the caps as a sizing mechanism. Keep the env vars as an escape hatch for anyone who
wants a hard bound, but let the default be unbounded-until-the-OS-objects. The idle TTLs (5 minutes
nursery, 30 minutes established) remain the real retention policy, and they already are: replaying
a 27.6-hour ledger, head reuse was identical at every cap from 8 to unlimited.
Head reuse over a 27.6-hour real ledger was cap-invariant — 8, 16, 24, 64, unlimited all gave
the same reuse rate. No modelled miss at any cap was caused by an eviction.
The ten real cap evictions in that ledger displaced connections idle 217–284 seconds, already at
the five-minute nursery timeout. The caps were doing nothing useful.
Ordinary pool occupancy was 1–4 nursery for 22 of 24 hours and peaked at 28 established.
Real demand sits far below any cap we would set, which is precisely why a cap is the wrong place to
express the limit.
Notes for whoever picks this up
Neither cap is a hard ceiling today: only idle entries are evictable, so a generation already
exceeds its cap while every head is busy (a 16-conversation fan-out reached 11 against a cap of 8),
and isolated sockets are never registered or counted. So "unbounded" is closer to the current
behaviour than it sounds.
Memory, not just descriptors, scales with retention: each retained head holds its conversation plus
a canonical copy for prefix comparison (fix(oauth): keep prompt caching when many subagents start at the same moment #219 added a second copy while a response is in flight).
Descriptor exhaustion may not be the first limit reached on a memory-constrained box, so consider
whether a heap-pressure signal belongs alongside EMFILE.
Descriptor limits vary enormously — a stock macOS shell commonly carries a 256 soft limit while a
tuned one reports over a million — which is itself the argument for asking the OS instead of
guessing.
The pools are per process. Several clodex processes on one machine multiply held descriptors,
and each has its own connection pacer, so there is no account-wide or machine-wide limit today.
The problem with having a cap at all
clodex retains a reusable WebSocket per live conversation so later turns send only the new message.
Those connections sit in two fixed-size pools (
CLODEX_WS_MAX_NURSERY_CONNECTIONS,CLODEX_WS_MAX_CONNECTIONS). When a pool is full, the least-recently-used idle connection is droppedand that conversation loses its cache — its next turn resends the full history over a new connection.
Every value we could pick for those caps is wrong for somebody:
user actually runs). fix(oauth): keep more cached conversations alive when many subagents run at once #221 raised the nursery cap to 48 explicitly as a safety valve rather than a
measured requirement, because nothing about the observed data identifies a correct number.
one; even now, a user has to read logs to discover that a cap — not their own behaviour — cost them
a cached conversation and a full prompt resend.
cost of that concurrency is real and it is theirs to pay. Churning connections underneath the number
they asked for costs more — in cache misses and in new-connection pressure — than holding them.
An arbitrary ceiling turns a resource question the OS can answer authoritatively into a tuning
parameter nobody can set correctly.
Proposal
Let the pools grow organically and let the operating system be the limit, reporting it clearly when it
is reached rather than quietly degrading before it.
EMFILE(andENFILE) on the connection-open path. There is currently no handlinganywhere in
src/—grep -rn "EMFILE" src/returns nothing — so exhausting descriptors is anunhandled failure rather than a degraded mode. This is the blocking prerequisite.
EMFILE, shed load deliberately instead of failing the request: evict idle pooledconnections (oldest first) and retry the open. That converts descriptor exhaustion into exactly the
behaviour the caps provide today, but triggered by the real limit rather than a guess.
how many connections were held, and that
ulimit -nis the knob. Today the equivalent condition isinvisible unless they go trawling diagnostics.
wants a hard bound, but let the default be unbounded-until-the-OS-objects. The idle TTLs (5 minutes
nursery, 30 minutes established) remain the real retention policy, and they already are: replaying
a 27.6-hour ledger, head reuse was identical at every cap from 8 to unlimited.
Why this is safe to aim for
From the analysis in #219 and #221:
the same reuse rate. No modelled miss at any cap was caused by an eviction.
the five-minute nursery timeout. The caps were doing nothing useful.
Real demand sits far below any cap we would set, which is precisely why a cap is the wrong place to
express the limit.
Notes for whoever picks this up
exceeds its cap while every head is busy (a 16-conversation fan-out reached 11 against a cap of 8),
and isolated sockets are never registered or counted. So "unbounded" is closer to the current
behaviour than it sounds.
a canonical copy for prefix comparison (fix(oauth): keep prompt caching when many subagents start at the same moment #219 added a second copy while a response is in flight).
Descriptor exhaustion may not be the first limit reached on a memory-constrained box, so consider
whether a heap-pressure signal belongs alongside
EMFILE.tuned one reports over a million — which is itself the argument for asking the OS instead of
guessing.
and each has its own connection pacer, so there is no account-wide or machine-wide limit today.