Skip to content

codex-bridge: one wake, one turn — attribute mid-start turn-end signals by turn id - #889

Draft
ainyan03 wants to merge 4 commits into
fujibee:mainfrom
ainyan03:codex-bridge-one-wake-one-turn
Draft

codex-bridge: one wake, one turn — attribute mid-start turn-end signals by turn id#889
ainyan03 wants to merge 4 commits into
fujibee:mainfrom
ainyan03:codex-bridge-one-wake-one-turn

Conversation

@ainyan03

Copy link
Copy Markdown

Fixes #870.

One wake could start two turns: a turn-end signal landing while the turn/start
request was still unanswered re-entered tryStartTurn() with the same wake. This
PR closes the re-entry (the wake is claimed before the request goes out, and
turn-end signals are gated while it is in flight) and attributes mid-start end
signals by turn id so a fast turn still ends promptly. Every claim below is a
local test run at a specific commit of this branch; the one open decision is
whether to keep the wake claim as defense in depth (question at the end).

The series is the verification matrix

You asked for each half reverted individually, with exactly its case going red.
Measured, the halves don't decompose that way (next section), so the PR is
structured as a cumulative series instead — each commit flips exactly the tests
its mechanism owns. Rows 1–4 are git checkout of the Nth commit of this
branch; the last two rows are described below the table.

state duplicate-turn pre-ACK completion stale-idle attribution
1. tests only (bug present) FAIL FAIL FAIL
2. + wake claim & in-flight gate pass FAIL (hang → timeout) pass
3. + turn-id attribution pass pass pass
4. + --quiet (full series) pass pass pass — full test_codex_bridge suite 45/45
full − --quiet only (identical tree to row 3) pass pass pass
full − wake claim only pass pass pass

"full − wake claim only" = on top of row 4, move this.pendingWake = false
back to after the ACK and drop the restore in the catch.

Row 2 is red on pre-ACK completion on purpose: there the gate is a blanket
drop, and a fast turn fully notified before its ACK waits out the idle watchdog
(exit 124 under the test runner). That is the measured reason the gate cannot
ship without the id attribution.

Why the independent-revert matrix doesn't hold

  • Reverting the wake claim alone changes nothing (last row): the in-flight
    gate already drops every turn-end signal that could re-enter tryStartTurn()
    while the request is unanswered, so the claim's timing is unobservable from
    events. It guards the invariant "a wake is spent at most once" at the state
    level; it is not an independently testable half.
  • Reverting the id attribution alone goes red on pre-ACK completion, not on
    stale-idle: removing it also removes the deferral, so the mid-start
    completion is acted on immediately and the bridge shuts down (--max-wakes)
    before the ACK. The stale-idle ordering is instead covered by the step from
    row 1 to row 2.

Which test covers which ordering

  • pre-ACK completiona turn fully notified before its turn/start ACK still ends promptly: the scripted app-server sends turn/started +
    turn/completed (matching turn id) before ACKing turn/start; the test
    asserts no hang and both lifecycle lines. This is the only test that is red
    at row 2.
  • stale-idle attributiona stale idle landing after the new turn was seen starting does not end the running turn: turn/started (new id), then
    the previous turn's thread/status idle, then the ACK, then the real
    completion 2 s later; the test asserts by log-line order that neither the
    second turn nor the bridge's own exit precedes the real completion.

--quiet

Removing only --quiet from the full series leaves everything green — matching
your expectation that nothing may ride on it. It suppresses the symptom (the
sentinel line can never become a prompt), not the cause. The duplicate-turn
test pins that distinction: it asserts the duplicate wake is never spent at
all, not even on an empty re-read that aborts, so --quiet cannot mask a
regression of the actual fix.

Assertions use grep -q in non-last positions, per the existing note in this
test file about [[ ]] under bash 3.2.

The table is from local macOS runs; CI covers both OSes on the PR, so no
manual runs should be needed on your side.

One question: keep the wake claim (early pendingWake = false + restore on
failure) as defense in depth, or drop it for minimality? The series stays green
either way — happy to amend.

… app-server

Three orderings around an in-flight turn/start request, replayed against a
scripted app-server:

- duplicate-turn injection: a wake deferred behind a running turn is
  delivered from onTurnEnded() when turn/completed arrives, and while the
  resulting turn/start request is still unanswered the app-server
  independently emits thread/status idle for that SAME previous turn. The
  second turn-end re-enters tryStartTurn() with the same wake and starts a
  second turn whose whole prompt is inbox.sh literal "No new messages."
  output, the first read having already consumed the rows. Observed live:
  one wakeup, two consecutive started-turn lines, an injected empty turn
  in the TUI.
- pre-ACK completion: the dual ordering. A fast turn is fully notified
  (turn/started, then turn/completed) before the turn/start request is
  ACKed; the end must still be processed promptly instead of waiting out
  the idle watchdog.
- stale-idle attribution: the composition. The new turn is seen starting
  before the ACK, and only THEN does the previous turn idle straggle in;
  the running turn must not be ended early.

All three are red at this commit; the following commits turn them green
one mechanism at a time. The duplicate-turn test also pins that the
duplicate wake is never spent at all (no empty re-read aborting behind a
quiet inbox), so the case stays red until the actual re-entry is closed,
not merely its visible prompt.

Assertions use grep -q in non-last positions per the existing note in
this file: on bash 3.2, which is what macOS CI runs, a false [[ ]] there
reports ok. Negated checks are count comparisons for the same reason.
…/start

A wake deferred behind a running turn is delivered from onTurnEnded() when
turn/completed arrives. While the resulting turn/start request is still in
flight, the app-server independently emits thread/status idle for that SAME
previous turn; both signals funnel into onTurnEnded(). Because the wake
claim (pendingWake) was only cleared after the request resolved, the second
turn-end reset turnActive/threadIdle and re-entered tryStartTurn() with the
same wake -- starting a second turn whose entire prompt was inbox.sh
literal "No new messages." output, the first read having already consumed
the rows. Observed live: one wakeup, two consecutive started-turn lines,
and an injected empty turn in the TUI.

Two mechanisms, both required to close the re-entry (either alone happens
to cover the replayed ordering, so the tests pin the invariant "the
duplicate wake is never spent", which needs both):

- tryStartTurn() claims the wake BEFORE the request goes out and restores
  it on failure, so no concurrent path can spend the same wake twice.
- startInFlight scopes the in-flight window: onTurnEnded() drops every
  turn-end signal that lands inside it, and the idle handler does not flip
  threadIdle under the start it no longer owns.

The duplicate-turn and stale-idle orderings go green here. The pre-ACK
ordering is red AT THIS COMMIT, deliberately: the gate is a blanket drop,
and a fast turn fully notified (started AND completed) before its ACK now
waits out the idle watchdog. That is why this cannot ship without the next
commit, which attributes mid-start end signals by turn id instead of
dropping them wholesale.
Mid-start events are legal in BOTH directions -- the previous turn's tail,
and the NEW turn's own lifecycle, which the app-server may notify before
it ACKs turn/start -- and a stale tail can land AFTER the new turn was
seen starting, so neither a blanket ignore (the previous commit) nor a
phase flag can attribute them. Identity can:

- turn/started (now filtered to our thread, which also stops another
  thread's turns from flipping our state) records the in-flight turn id.
- While the request is unanswered, onTurnCompleted defers the end ONLY
  when its turn id matches that in-flight id; tryStartTurn() then runs one
  onTurnEnded after the ACK, so a fast turn neither waits out the idle
  watchdog nor skips the maxWakes accounting. A different id, an id-less
  completion, or a thread/status idle is unattributable mid-start and is
  dropped -- the idle watchdog closes a genuinely-ended turn (fujibee#41).

This turns the pre-ACK completion ordering green (it hung until the
watchdog under the blanket gate) while keeping the stale-idle ordering
green: a stale idle straggling in after the new turn was seen starting
carries no matching id and is dropped instead of ending the running turn.
All three replayed orderings pass from this commit on.
…r become a prompt

readInboxForPrompt() passes --quiet to inbox.sh (an existing flag), so an
empty inbox reads back as EMPTY and the human-facing "No new messages."
line can never again pass tryStartTurn's emptiness check and become the
entire prompt of a turn.

Defense in depth, not the fix: with the two previous commits in place a
duplicate wake is never spent, so this path is not reachable from the
replayed orderings -- reverting only this commit leaves all three tests
green. It hardens the prompt boundary against any OTHER path that might
reach an empty read, and the duplicate-turn test pins the distinction by
asserting the re-read itself never happens rather than merely that no
sentinel prompt was seen.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

codex-bridge: one wake can start two turns during turn/start

1 participant