Skip to content

fix(native-chat): a failed agent start fails only its own message - #26119

Draft
brennanb2025 wants to merge 29 commits into
mainfrom
brennanb2025/start-failure-fails-its-own-message
Draft

brennanb2025 wants to merge 29 commits into
mainfrom
brennanb2025/start-failure-fails-its-own-message

Conversation

@brennanb2025

@brennanb2025 brennanb2025 commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor
Files Added Deleted Net
Test 23 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​1638 $\color{#cf222e}{\Huge{\mathbf{−}}}$​292 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​1346
Prod 30 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​765 $\color{#cf222e}{\Huge{\mathbf{−}}}$​340 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​425

Size: net +415 production lines vs main (+756 added, −341 removed); tests +1.7k. This is the focused rebuild of #24340, which was far larger.

ELI5

When a chat's agent can't start, Orca used to throw away every message you had typed while it was starting, all with the same error. Now only the message that start was for fails. Each message after it gets its own try, so if the problem was a one-off crash, they go through.

What Changed

The problem. In native chat, if you send A, then B and C while the agent is still starting, and that start fails, main rejects A, B and C together with one " couldn't start" row. B and C never get a start of their own. When the failure was a one-off (the CLI crashed during startup, or Orca stopped a start that hung), you lose messages that would have gone through. Cause: the delivery loop's failure path recorded the start failure with rejectsQueued, which rejects every queued message in one write. Its error handler did the same.

What you see now.

Situation Before (main) After
A's start crashes once; B and C were sent during it A, B and C all "Not sent", one error row A "Not sent" with the row. B and C each get their own start and are answered
Not signed in, three sends One row, all three rejected at once Three "Not sent", still one row. Each message tried its own start
The Codex app-server crashes at startup twice, and its stderr differs only by a timestamp One row per attempt One row: the CLI's log text is not part of "the same failure"
/compact fails to start Its row says "Run /compact again" Unchanged. That row never speaks for a later plain message's failure

No new strings or UI elements. Released clients, such as a v1.4.221 desktop paired to a newer remote host and the phone app, still get the durable error row, because the host keeps writing it.

The mechanism.

  • Delivery loop (structured-agent-session-delivery-loop.ts):

    • Each pass fixes the message it is attempting before any await. A failed start rejects only that message, and the loop continues with the next one.
    • Its error handler retries the rejection on that same message, never on the head of the queue.
    • A message whose pass only joined someone else's start is not charged with that start's failure.
    • A child whose start failed is ended before the next message's start, as best effort, after the failure is recorded.
  • The three writers of a failed start each own one message state:

    • the loop: a queued message;
    • the handover: a message being handed to a child that is still starting;
    • the exit: messages handed to a child that never proved its start.

    Each writes the rejection and the start's row together in one journal append (rejectWithStartFailureRow). The row is keyed by the message it speaks for, so a client can tell which message a row is about.

  • One row per run (journal-start-failure-run.ts), decided on the journal's write lane. A message's start that fails with the same structured failure as the latest start-failure row writes no new row, provided no send was accepted since. "Same" is a field comparison of the failure fact (sameAgentSessionFailureFact, now shared by host and renderer). The CLI's own text counts only when it is meant for a person, not when it is log output. A /compact's row is always written and never starts a run.

  • Renderer (structured-agent-session-delivery-notices.ts): a rejected message under its own row, or under the run's row, reads "Your message was not sent." The row says why, once.

Why

  • Why not keep rejecting everything queued: that is the bug. One transient failure discards messages that would have succeeded.
  • Why not hold the queue after a failure: a person's direct sends are not a visible queue they can resume today; the queued-messages capability is not enabled. A hold would strand them with no control to release it.
  • Why one row per run, not one per attempt: with each message trying its own start, a deterministic failure such as being signed out would otherwise add one identical row per message.
  • Why keep the durable row at all: released desktops and phones hide rejected messages. Without the row, they would show nothing for a message sent from another device.

Differences from the common pattern:

  • No automatic retry of a failed message. Temporary: a rejected message is final, and sending again is a new message. An automatic retry is a follow-up once the visible queue ships.
  • After a failed start, later direct sends each try their own start instead of waiting. Intended: direct sends have no visible queue that a person could resume.
  • A phone's own message, sent while the agent starts and then rejected in a run, shows no row of its own. The run's earlier row still states the failure, but the phone draws no "Not sent" on the message itself. Temporary: fix(native-chat): the phone keeps a message the host did not deliver, and a failed /compact is said once #24918 draws "Not sent" in place on the phone.
  • A v1.4.221 desktop compares the CLI's log text, so the second timestamped Codex crash shows full words on that message under the one row. Intended: released code; the count matches main's two rows.
  • A run ends only when a send is accepted, not on a timer. Intended: the journal decides it, with no stored flag to go stale.

Linked Issue

Supersedes #24340 (rebuilt to its core fix).

Visual Proof

Live run on a laptop rig: main 0acf039b5d5 (before) vs this PR at 0c1224262dc (after). Stand-in agents scripted the failures, and an external driver did the clicking.

1. Claude crashes once while starting; A, then B and C sent during the start. Before: all three not sent. After: only A is not sent; B and C are answered by a new start.

Before After
before: A, B, C all not sent after: A not sent, B and C answered

2. Claude not signed in; three sends. Both builds: three "Your message was not sent." and one row. After: each message made its own start, and the row is not repeated.

Before After
before: one row, three not sent after: one row, three not sent

3. Codex start crashes on each send, and its stderr carries a new timestamp each time. Before: three identical rows. After: one row.

Before After
before: three rows after: one row

Testing

  • Host tests: structured-agent-session-start-failure-writer.test.ts (18 cases across every failure path: a refusal before spawn, an exit before and after handover, a dispatch fault while starting, a joined start, a cleanup failure, failures inside the error handler); start-failure-run-lane; command-start-failure-run; signed-out-run (scripted Claude rig, real not-signed-in path); exit settlement unit tests.

  • Each new rule was checked by ablation: turning the rule off fails its test.

  • Typecheck (node, web, cli). The only errors are in files this PR does not touch, all present on main. Lint, changed-code quality, max-lines and reliability gates pass.

  • Live QA (laptop, main vs this PR): the three cases above, plus /compact while signed out (its own row on both builds; the message after it gets its own row). Not covered: the phone (fix(native-chat): the phone keeps a message the host did not deliver, and a failed /compact is said once #24918), and Codex crashing while the chat first opens, which shows "Chat could not be started" with messages left "Sending…" on both builds. That is a separate path, unchanged here.

  • I manually tested these changes locally (macOS laptop rig)

  • Automated tests added/updated

Review

Reviewed in six rounds by independent reviewers. The last round's findings are fixed or labelled above.

Agent skill upstream boundary

  • Not applicable

Notes

  • SSH: every writer runs on the execution host. Losing contact with a client writes nothing.
  • Folder workspaces: delivery is keyed by session; nothing assumes git.
  • Mixed versions: no RPC, capability or field change. Rows and rejections are the same journal records main already writes.

X: @BrennanKB5

Checklist

  • This PR is small and focused
  • I explained what changed and why
  • Before/after screenshots attached
  • Self-reviewed for correctness, security, and performance
  • Cross-platform, SSH/remote impact considered
  • Lint and typecheck pass locally (except errors that already exist on main)

…for; the messages behind it each get their own start
…rejection, so clients that hide a rejected message still see why
…ow its own start wrote, in either write order
…that message; an older host's batch row matches by its failure
…eed announces once when nothing is owed, as on main
… exit's start row; the failed-start check is file-local
… no row, in the tests that counted one per attempt
… on the journal's lane, as its batch is planned
…failure, so a CLI's timestamped stderr keeps one row per run
… starts, a /compact with no item of its own too
…, and that row never speaks for a message's failure
… for, so a row worded for /compact is that command's

The exit wrote its row keyed by the start even when it rejected a handed /compact
and worded the row "Run /compact again". That row then started a run, so a later
message failing alike was read under the /compact's words. Keying the row by the
message its words are for makes it a command's row, which never starts a run and
is never dropped. A /compact's failure now always writes its own row, so the
composer no longer needs to ignore rows loaded before the send.
Main now hands a message to a child still starting (no wait on its start) and holds
the next one until the turn ahead opens. Resolved so a failed start still fails only
its own message: the loop keeps its per-message writer and catch; the exit charges a
start only to a still-queued message it was started for, so the delivery loop's
start-wait tracking (waitingOn / awaits / deliveryAwaits) is gone with the wait.
Tests that drove the removed start wait are rebuilt on refusals, exits before and
after handover, and a dispatch that faults while starting.
… again, and the awaited command reads an empty id as none
…own, and the exit words its row only for a message it was handed

- The delivery loop charges a gone child's failure only to the message that child was
  started for; a message whose pass merely joined that start goes on to its own.
- The exit reads the message its words are for among sends it was handed, so a /compact
  still queued never words, or keys, the exit's row.
- A send a child past its start refuses because it ended writes no start-failure row;
  the exit's own row says why.
- The exit checks for itself whether the message its start was for is still queued.
- Restart continuation is left as on main (its note beside a failed start is main's).
@brennanb2025

Copy link
Copy Markdown
Contributor Author

Status at 0c1224262dc: draft, ready for review by the coordinator. Not merged.

The problem. When a native chat's agent failed to start, Orca rejected every message waiting behind it with the same error. Messages typed while the agent was starting never got their own try. So a one-off crash during startup threw away messages that would have gone through.

What changes for the person.

  • Only the message the failed start was for says "Not sent". Each later message gets its own start: after a one-off crash, they are delivered.
  • If every start fails the same way (not signed in, or a CLI crashing at each start), every message still says "Not sent", but the chat shows one error row for the run, not one per attempt.
  • A failed /compact still gets its own row, and it never stands in for a later message's failure.
  • No new strings or UI. Released desktops and the phone still see the durable error row.

Verified.

  • Live, on a laptop rig, main vs this PR, with screenshots in the body:
    • one crash while starting: B and C delivered (before: all three lost);
    • not signed in: one row, three "Not sent";
    • a Codex start crashing three times with a different timestamp in its stderr each time: one row (before: three).
  • /compact while signed out gets its own row on both builds.
  • Host and renderer tests across every failure path, each new rule proven by ablation (turn it off and its test fails).
  • About 1,540 related test files pass. The only reds are main's own acp restore-failed test, which fix(acp): repair main's typecheck after the legacy journal-path removal #26094 fixes upstream.
  • Typecheck: the only errors are in files this PR does not touch, all already on main. Lint and repo gates pass.
  • Two independent review rounds on the final diff: no P0 or P1 open. Labelled differences are in the body.

Not verified.

  • The phone: a phone's own message rejected inside a run shows no "Not sent" of its own; the run's earlier row still says why. Labelled temporary; fix(native-chat): the phone keeps a message the host did not deliver, and a failed /compact is said once #24918 draws "Not sent" on the phone.
  • Codex crashing while the chat first opens: both builds show "Chat could not be started" and leave messages "Sending…" indefinitely. That path is chat creation, which this PR does not change. It is a real pre-existing bug that needs its own ticket.
  • Platforms: no live run on Windows, Linux or over SSH. Every writer runs on the execution host, and no wire format changed.

Main removed the desktop's saved outbox (#25959): the renderer's delivery notices now come
from in-memory sends and the host's rows. PR A's start-row reading (a row keyed by its
message speaks for it alone; otherwise any loaded row with the same failure, but a
command's) is kept on the new shape, and the start-row test moves to the new API; its
outbox-only and released-host-batch cases are gone. Main's new Claude sign-in prompt
counts start-failure rows through their fact.
@brennanb2025

Copy link
Copy Markdown
Contributor Author

CI note at 6c8d1331025 (merge of main 8fdad2a3af7).

  • static analysis and typecheck is red because of main, not this PR. The localization step reports that en-runtime-required.json no longer covers en.json (two auto.store.slices.worktrees.* keys). Main commit 8fdad2a3af7 (fix(claude): activate account profiles and remove credential replay (Step 4 of 4) #24434) changed that catalog, and nothing on main has fixed it since. This PR touches no localization or store files.
  • Because that step fails first, the test jobs did not run on this head. This PR will re-run once main's catalog is fixed and merged in.
  • The last full CI run, at 0c1224262dc, ran every shard.
    • Its one PR failure, two new journal tests missing from the SQLite runtime list, is fixed in 64e46e9f02e.
    • Its other red, package (windows) timing out in profile-state-writer-stall.electron.test.ts, is in persistence code this PR does not touch.
  • Local runs on this head:
    • the PR's 23 test files pass;
    • a 1,614-file closure of the files this merge touched: 1,606 pass, and both reds are environmental (a cross-version checkout CI prepares, and a temp-dir cleanup that passes alone);
    • the cli typecheck passes, and node and web show only main's existing errors.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant