Skip to content

Pacing: a queued request does not re-match a head that freed up while it waited #173

Description

@bman654

Found while reviewing #172 (connection pacing). Not fixed there because it changes head selection
in src/oauth/responses-websocket.ts, which is under the repo's "do not restructure" prime
directive, and it deserves its own tests rather than riding along at the end of another review.

What happens

#172 adds an await between the head scan and connection creation. After that wait it re-reads
the partition and demotes itself to parallel_isolated if a sibling went in flight meanwhile, so
two requests cannot both register a persistent nursery head for one key.

It does not handle the other case: a sibling that completes during the wait. That leaves an idle,
reusable head in the partition — but the queued request was classified against an empty partition and
nothing re-runs continuationMatch after the wait. So it opens a duplicate persistent nursery head
instead of continuing the chain that just became available.

Why it matters

  • The nursery cap is 8 (RESPONSES_WS_MAX_NURSERY_CONNECTIONS). Duplicate heads for one key
    consume it and evict other conversations' reusable heads.
  • An evicted head means the next turn of that conversation resends full context — a cache miss and a
    larger prompt — and opens another connection.
  • More connections means more pacing, which means longer waits, which widens the window in which
    this happens. It partially defeats the feature under its own target workload: heavy parallel
    fan-out on a shared Claude session id.

Fix sketch

Re-run the candidate scan and continuationMatch after admission, not just the in-flight check —
roughly, hoist the existing scan into something callable and run it a second time when
pacingWaitedMs > 0. A delayed request could then reuse a head that freed up while it waited,
which is strictly better than what it does today: it would lower the new-connection rate rather than
just avoiding a duplicate.

Why it needs care

Head selection is exactly the logic the prime directive protects, and getting it wrong risks
continuing a chain whose lineage does not actually match — the failure the exact-prefix validation
exists to prevent. It needs its own discriminating tests, including a negative proving a
non-matching freed head is not continued, plus a mutation showing the re-scan is load-bearing.

Reachability

Requires two same-partition requests where one completes inside the other's queue wait. Reachable
only when pacing actually queues something, i.e. under fan-out past the burst allowance. Not
reachable at all with CLODEX_WS_MAX_NEW_CONNECTIONS_PER_MIN=0.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions