Skip to content

fix(native-chat): a host stop decides from the agent's live state, with no wait - #24503

Draft
brennanb2025 wants to merge 3 commits into
mainfrom
brennanb2025/host-stop-decides-from-live-state
Draft

brennanb2025 wants to merge 3 commits into
mainfrom
brennanb2025/host-stop-decides-from-live-state

Conversation

@brennanb2025

@brennanb2025 brennanb2025 commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor
Files Added Deleted Net
Test 8 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​316 $\color{#cf222e}{\Huge{\mathbf{−}}}$​136 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​180
Prod 14 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​122 $\color{#cf222e}{\Huge{\mathbf{−}}}$​69 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​53

ELI5

When Orca stops an agent on its own (you close the chat tab, a worktree is torn down, or a start that never came up is stopped), it first has to decide: was the agent doing anything? If so, it writes down why the work stopped, so the chat can later say "you stopped this" rather than "this failed."

Orca used to answer that by waiting up to a second for the agent's latest messages to be saved, then reading the saved history. Now it asks the agent connection directly what is running at that moment, with no wait.

The problem

Since #24311, every host-side stop (a close of the chat, an eviction, the host stopping a start) writes a lasting Stop event when it ends work. Later, the end of the cut turn reads that event: your close shows as your cancellation (muted "Interrupted after N"); any other stop shows as news ("Failed after N").

To decide "does this stop end work?", the host read the saved history, but the history lags the agent:

  • A send's acceptance is saved straight away, while the turn it opens is saved a moment later, behind the agent's other streamed rows.
  • A turn's end can still be waiting to be saved while the history still shows the turn running.

#24311 bridged that gap with a wait: up to 1 s for the streamed rows to be saved, then a guess when the wait ran long ("an accepted send with no turn yet counts as working").

That wait is the only one of its kind in the Stop path, and both reference designs decide from live state with no wait (see "Differences" below). It also had real gaps:

  • When the wait ran out, a turn the agent opened on its own (not from one of your sends) was not seen as running. Your close of it then read as "Failed after N" instead of your cancellation.
  • When the wait failed outright, the history was read as it stood, with the same miss.
  • Every close and teardown under load paid up to a second before the agent was stopped.

What changes for you

  • Common case: nothing you can see.
  • Under load (the agent streaming faster than Orca saves):
    • Closing a chat whose turn had only just started now reliably reads as your cancellation ("Interrupted after N", muted) rather than "Failed after N".
    • Closing a chat or tearing down a worktree no longer waits up to a second before the agent is stopped.
  • A turn that ended just before your close (its end not yet saved) writes no Stop event, so its own ending stands.

What changed (mechanism)

Each adapter reports what its agent has in flight (liveWork, new optional member on the adapter contract). It reads the provider's own messages as they arrive, before they are saved:

  • Claude: the open turn, or a sent message Claude has not echoed back yet. This is the same check the Stop's wind-down wait already uses (claude-request-end-wait.ts).
  • Codex: the turn it reports running. A turn carrying a command (like /compact) is reported under the command's history turn. Also a send still on its way, or a send Codex answered into a turn it has not opened yet. This is the no-wait half of the read the Stop already does.
  • The adapter router answers for the session's live owner. With none, nothing is in flight.

The host stop decides in its own serialized step, with no wait (workStopEnds):

  • It ends work when the agent reports something in flight, or the history holds a send nobody has answered. A lost reply is not proof the send died: live, unverifiable or exited, never assumed.
  • The event names a turn: the history's running turn when it has one; otherwise the turn the agent reports, whose row has not been saved yet.
  • A person's Stop still holds against a later eviction or host stop while every send in flight is one that Stop stopped. That now includes a send that was accepted but whose turn row has not been saved. Your own close always writes its event.

Removed: the 1 s drain, its timeout and the slow-drain fallback (latestAcceptedSendUnopened as a public read).

SSH and paired hosts: the decision runs on the execution host, where the host and its adapters live, and reads only that host's in-memory state. No wire change, no new persisted field.

Why this approach

  • Asking the agent connection is the freshest view the host has, because the saved history is built from the same messages afterwards. It answers both ways: a turn just opened (row not saved) counts as running, and a turn just ended (end not saved) counts as over.
  • A boolean "something is running" was not enough. The cut turn must be named, or a close of a turn the agent opened by itself binds nothing and reads as "Failed". structured-agent-session-close-verdict.test.ts "records the user's close on a turn whose start landed only as the provider stopped" caught this; it goes red without the name.
  • Considered: keep the wait but shorten it. Rejected: any bound reads wrong at the bound, and the live view needs no bound.
  • Considered: count every accepted send without a turn row as working, with no adapter change. Rejected: a send that never links to a turn row would keep a resting agent "working" forever, and it still misses a turn the agent opened itself.

Differences from the common pattern

  • Matches: a host stop decides from live state, in its own serialized step, with no wait. The decision is whether a turn is running or a send is still unsettled. Here the live state is the adapter's own view of the agent connection; in the common pattern it is the server's thread or session status, set when the send is made and cleared when the turn ends.
  • Matches: the link from a send to a Stop that named no turn follows the common pattern's sequence window. A send handed over before the Stop's row is one it stopped; one made after it is a new instruction.
  • Intended: no "host restarted" stop event at relaunch. Settling from proof that the old agent died already shows "Failed" plus an error row. An extra event would add a second copy of that fact, and loss of contact is never taken as proof of death.
  • Intended: the provider's own success stands after a Stop. One version of the pattern marks the turn interrupted when the interrupt is requested and keeps that over a later completion. Here a turn that finished on its own reads as finished.
  • Intended: when the history already shows a running turn, the event names that turn, and the live view names a turn only when its row has not been saved yet. Turns carrying a command are written by the host before the send, so their history id is always right. The live view's turn id comes from the provider.
    • Remaining edge: if the history still shows an old turn running while the agent has already started a new one (both rows still unsaved), the event names the old turn. The new turn's end then reads as news.
  • Temporary: "the turn ended after the Stop" is judged by the host's clock (when the end was observed, against when the Stop's row was written). Follow-up: compare positions in the history using the position at which each end was observed, not where its row was written. A crash the host sees just before a Stop has its end written after it.
    • fix(native-chat): one ordered journal writer #24127 (one ordered history writer) makes a streamed end's written position equal to its observed one.
    • An end the host writes in a later step (an exit's settle, or a relaunch) still needs the position recorded when the exit was seen. Without it, a crash just before your Stop would read as your cancellation.
  • Temporary: a Codex Stop pressed before Codex opened the turn sends no interrupt and shows the "didn't stop" note, but the send is still the Stop's. If that turn later crashes, or is settled at relaunch, it reads as your cancellation. Follow-up: fix(native-chat): a Stop's note records whether it took, so a Stop that stopped nothing never marks a later turn as yours #24478 records each Stop's answer on its note, and the turn-end rule skips a Stop whose answer is "refused" or "unconfirmed".

Linked Issue

None (maintainer). Follow-up to #24311 (Stop event part B).

Compatibility

  • No wire change: no RPC params, stream frames or persisted fields change. The decision reads in-memory state on the execution host.
  • Older clients and older hosts: unaffected; the Stop event row is unchanged.

Visual Proof

N/A: the chat looks the same. What changes is whether, under load, a turn you close reads "Interrupted" or "Failed". The automated tests below pin that.

Testing

  • I manually tested these changes locally
  • Automated tests added/updated, or explained why not below

New structured-agent-session-host-stop-live-view.test.ts covers each host stop kind (your close, an eviction, the host stopping a start):

  • a running turn: writes, naming it;
  • a turn opened whose row has not landed: writes, naming it;
  • an accepted send whose turn has not opened: writes, naming none;
  • at rest while the turn's end row is still on its way: writes nothing;
  • at rest: writes nothing;
  • a send still unanswered: writes.

It also checks that a person's Stop holds against a later close or eviction, both for the turn it named and for a send it stopped whose turn row has not landed, and does not hold over a send made after it.

Adapter tests:

  • Codex's live view, end to end over the fake Codex: nothing at rest; a send with no turn from the answer until the turn opens, then that turn; nothing after the turn ends, or after the answered turn ends without opening; a command's turn named by its history id.
  • Claude's live view.
  • The router delegates to the live owner.

Existing tests updated:

  • The two drain tests in stop-event-entries and the drain-failure test are replaced by the live-view file.
  • close-verdict's adapter reports the cut turn as in flight.
  • surface-lifetime's drain-failure helper goes back to failing the eviction's own drain, since the stop no longer drains first.

Runs:

  • 52 source test files: 590 passed.
  • Cross-version unit tests (2 files, --no-file-parallelism): 4 passed. The first cold run timed out while materializing the release checkout and passed on rerun.

Ablations (each rule removed alone and the named files rerun; every one goes red):

Rule removed Red
Ignore the live view (history only) 14
Ignore unanswered sends in the history 5
A person's Stop not counting an accepted send whose turn row has not landed 1
Event names only the history's turn 4 (incl. close-verdict)
No precedence for a person's Stop 3
Codex: ignore a turn answered but not opened 1
Codex: no command-turn mapping 1
Claude: ignore an unechoed send 2

Not run:

  • Typecheck: the coordinator runs it.
  • claude-structured-session-integration.test.ts: it may spawn the real CLI.
  • No live app run.

…th no wait

A close, an eviction or the host stopping a start must decide, before it
kills the child, whether it ends work and so must write its Stop event. It
waited up to a second for the event sink to flush the provider's rows into
the journal, and when the wait ran long it guessed from the latest accepted
send. The journal trails the child: an acceptance lands straight in it while
the turn row that follows queues behind the sink, and a turn's end row can
queue while the journal still shows the turn running.

Each adapter now reports what its child has in flight, read off the
provider's own frames as they arrive (`liveWork`): Claude's open turn or
unechoed send, Codex's running turn (by the journal id of a command it
carries), a send on its way, or a send answered into a turn not yet open.
The host stop reads that, plus the sends the journal holds unanswered, in
its own serialized step. The event names the journal's running turn, or the
live one when its row has not landed. A person's Stop still holds against a
later host stop while every send in flight is one it stopped, now counting a
send accepted whose turn row has not landed.

Removes the drain, its one-second bound, and the slow-drain fallback.
@brennanb2025

Copy link
Copy Markdown
Contributor Author

Review status: held, to be superseded after #24127

The problem this PR targets. When Orca stops an agent on its own (you close the chat, a worktree is torn down, a start is stopped), it decides whether the agent was working so the cut turn can read as your cancellation ("Interrupted after N") rather than news ("Failed after N"). Before this PR that decision waited up to 1 s for streamed history rows to be saved, and under load it could miss a turn the agent had just opened, so closing that chat read "Failed".

What this PR does. It drops the wait and asks each agent connection what it has in flight (a new adapter member, liveWork), alongside the saved history.

Why it is held, not shipped. That gives the host two sources for "what is in flight", the saved history and the connection's view, and the review found they disagree in user-visible ways:

  • R1-1 (regression from main): Codex /compact, between Codex's acknowledgement and its turn start. The history already shows the compaction running, but the connection reports nothing. Closing the chat then writes no Stop event, so the compaction reads "Failed after N"; main reads "Interrupted".
  • R2-1: after a Codex process exits on its own, its connection view still reports its last turn as running. A close handled just before the exit is settled writes a Stop event for a turn the crash already ended. That breaks "a stop that stops nothing writes nothing".
  • R3-3: the event names the history's running turn ahead of the connection's. If the history still shows an older turn while the agent is already in a newer one, closing the chat names the old turn, and the new turn reads "Failed".
  • V1-1: if Codex's "turn completed" message is refused because the history writer is full, the connection view has already dropped the turn. No event is written, and a turn that actually completed reads "Failed".

There is also a design reason. A person's own Stop still decides from the saved history alone, so with this PR the two kinds of Stop would answer from different sources.

Plan. #24127 (one ordered history writer) applies each accepted row to the history in the same step it is received, so a stop can read the history directly with no wait. Once it lands, this PR will be superseded by:

  • one synchronous history check, journalWorkInFlight(state, fence): a running turn (host-written command turns included), or an unanswered send from the current agent process. Host stops and the person's Stop both use it, and the event names the turn it returns;
  • liveWork, codexLiveWork and commandJournalTurnId deleted; no wait anywhere in the decision;
  • the refused-message case fixed where it starts: Codex records a turn as running only after the history accepts the message that opened it, so the two can't diverge.

This matches the common pattern by mechanism: one ordered store decides and names the cut turn, with no wait. Stacking on #24127 now was ruled out, because it has conflicts and its base predates #24311.

Interplay with #24478. Whichever lands second must:

Deferred (P3, to fold into the superseding PR):

Verified:

  • Three independent reviewers plus a verifier read the change and the relevant code.
  • R1-1, R2-1 and R3-3 each have a scratch unit test reproducing them on this head (real Codex adapter for R1-1 and R2-1). The PR's own tests pass, and every one of the builder's ablations goes red.
  • fix(native-chat): one ordered journal writer #24127's behaviour was verified by reading its diff at its current head.

Not verified:

The PR stays a draft. No code was changed in this review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant