Repository navigation
feat(native-chat): store each chat's status with its history; startup seeds and settles from it - #24233
feat(native-chat): store each chat's status with its history; startup seeds and settles from it#24233brennanb2025 wants to merge 140 commits into
Conversation
…ailing app startup At startup the chat host re-checks every saved chat's lease and writes the result to agent-sessions.json. If that write failed (the file lock gave up, the file could not be written, or the file was written by a newer Orca and is read-only here), reconcileRestartLeases rejected, the startup IPC call rejected, and the renderer fell into its degraded "Session restore failed. Changes won't be saved until restart" mode. The reconcile is bookkeeping: a lease left unreconciled grants no writer, and every attach, send and read of a chat reconciles its own lease again. So the startup reconcile now reports its failure through a new optional host dependency, onStartupReconcileFailure, and resolves. The runtime routes it to its onError sink under the scope structured-agent-session-startup-reconcile, or logs it when no sink is installed (the desktop installs none).
…eping With native chat on and a chat tab open at quit, the renderer's startup also awaits the chat tab restore (session.tabs.listAll). That restore re-ran the lease reconcile before reading each chat and rethrew its store failure, then recorded each restored tab as visible through a store transaction that throws on a held lock or a read-only store. Either one failed the restore, so startup still fell into "Session restore failed". Reading a chat grants no writer, so the reconcile startup and the restore run is now a reader's: createReaderReconcile never throws, answers whether every lease is settled (recovery is resolved only then; the journal opens either way), and reports each distinct failure once until a reconcile settles. Attach and agent start keep the strict reconcile. The restore's tab republish logs a failed visibility write and still publishes the tab, since a client drops every unpublished chat tab; user-driven publishes still refuse. The host dependency is renamed onLeaseReconcileFailure (scope structured-agent-session-lease-reconcile), since it now also reports for reads.
… startup waiting on the lease check At startup the chat tab list, and the window's first paint behind it, waited for two things: the startup lease check over every saved chat, and opening and replaying the full history of every listed chat. The list itself was then taken from the chats that had opened, so a chat whose history was missing or could not be opened lost its tab after a restart, and one failed open or lease check failed the whole list. Now the tab list comes from the durable records and the tab table, in tab order, filtered to chats this host serves. It opens nothing. Each chat's history is opened after the list is published, by the same restore pass as before, not awaited; a failure there is logged and leaves the list alone. The startup lease check is started and never awaited: neither the list nor paint needs leases. Two things did rely on the startup wait: - a write to a chat whose lease is still the previous run's. Every write (send, Stop, prompt answer, option, goal, rewind, clear, compact, queued drafts, restart-offer resume) enters its chat's queue through the lease check now: it waits for that chat's check, or runs it, when the lease is unreconciled, and enqueues at once otherwise. Attach and agent start already ran the check themselves. - the structured worker release, which the renderer runs right after the startup step. It now runs the check itself before it observes the worker. No schema change, no saved status. Status rows now arrive per chat just after paint instead of all at paint.
…artup' into brennanb2025/chat-tabs-list-without-replay # Conflicts: # src/main/native-chat/agent-session-wire/structured-agent-session-host.ts # src/main/native-chat/agent-session-wire/structured-agent-session-restart-reconcile.ts # src/main/runtime/structured-agent-session-startup-reconcile.test.ts
…read path Round-2 review found two more writes on the startup chat restore that could still fail it and put the app into "Session restore failed": republishing a /clear replacement recorded its tab visibility strictly, and resolving a chat's recovery rethrew its store error. The restore also paid one lock wait per tab and per batch of chats while the lock stayed held. The restore now derives tabs from state it already holds: - publishStructuredAgentSessionTab splits into the strict write and projectStructuredAgentSessionTab, which only updates the runtime's snapshot. The restore and /clear replacements only project: a saved tab index already lists every restored chat, and a /clear moves the tab in the same write that commits it. visibilityWriteMayFail is gone. - Chats a legacy profile restores that the index does not list are recorded in one best-effort transaction (store.showSessionTabs), so a failure leaves the index absent to seed again rather than partial. - The read restore's recovery resolution is caught and reported through onLeaseReconcileFailure, deduplicated with the reconcile's reports. - Once lease bookkeeping fails in a restore pass, the rest of that pass skips it, so a held lock costs one wait for the startup reconcile and one for the restore, however many chats are open. User actions (create, reveal, attach, send, the /clear commit) keep their strict writes.
…op its bookkeeping at the first failure The restore now runs one reader lease check for the pass and lets each chat re-check and resolve recovery only while the pass is still settled. The first refusal or failed write clears it for the rest of the pass, and every chat is still opened for reading. With another process holding the lock, startup waits on it once in prepare and once in the restore, however many chats are open; a legacy profile waits once more for its tab-index seed.
…artup' into brennanb2025/chat-tabs-list-without-replay # Conflicts: # src/main/runtime/orca-runtime-restore-structured-agent-session-tabs-once.ts # src/main/runtime/structured-agent-session-startup-reconcile.test.ts
…listed, one failure per chat The restore that reads each chat's history now runs after the tab list, while the app is already usable, so three things it did before the list existed now show: - A chat whose history failed to open ended one of the restore's four lanes. With four such chats, every chat after them got no status row until opened. Each chat now settles on its own and each failure is logged once. - A chat closed before the restore reached it was opened again and its status row republished. The restore now skips, under the chat's own serialized step, a chat whose tab is no longer listed. - After quit began, the restore kept opening chats. It now stops. Tests: the startup listing tests share the existing rest test rig, extended to several chats, instead of a parallel rig; the status parity test no longer leaves a temp directory behind. Comments that said an unreconciled lease admits no writer, or that a write waits for its own chat's check, now say what happens.
…as answered The restore that reads each chat's history was started, not awaited, from inside the listing call, so its synchronous start (about 110 ms on a 247-chat profile) ran on the listing's own stack and three chats opened before the list answered. It now starts on the next macrotask, after the answer has gone out. Also pins that listing writes nothing to the record store: tabs are projected from the store's own tab index, so a 247-tab list costs no per-tab transaction.
…artup' into brennanb2025/chat-tabs-list-without-replay
…e per-write gate This PR had stopped startup waiting for the lease check (the pass that settles each chat's previous-run owner) and instead made every write to an unchecked chat wait for it. After #23964 that check can no longer fail startup, and a review run measured it at about 30 ms on a 247-chat Mac profile, so the gate bought nothing but a second mechanism. It also reordered two writes that arrived while a chat's lease flipped from unchecked to checked. Startup's structured step, the write path and worker release are back to the base's shape: startup awaits the non-throwing check, writes enter the chat's queue in arrival order, and an agent start still runs its own strict check before it acts. What stays is the listing from records and the tab table, and the chat history restore that starts after the list has answered.
…answer is out The history restore was deferred with setImmediate from inside the tab restore, on the theory that the list answer would be sent by then. It is not: after the tab restore, session.tabs.listAll and the subscribe streams still run the PTY census (daemon, git and SSH calls), and the deferred restore fired during it. Four or five chats opened before the list answered, which is what the deferral was meant to prevent. The tab restore now only records the history restore it owes. Every session.tabs handler that restores tabs (list, listAll, subscribe, subscribeAll) goes through one wrapper that starts the owed restore when that handler's answer has settled: sent, failed or abandoned. The first to settle starts it; the rest find it started. The tab restore itself is no longer callable from a handler without that wrapper. Also: both quit checks in the restore now have a test (one skips the lease bookkeeping, one the open), and a comment no longer says a chat with missing history explains itself; it reads empty.
… timeout Both waited on the provider dispatch with vi.waitFor's default 1 s, and T3's wait failed once under load. Neighbouring structured-session tests wait 10 s for the same kind of assertion.
…t once per tab Every tab snapshot the runtime stores or answers re-applies the /clear replacements, and deriving them walks every chat record. The listing stores one snapshot per restored tab and answers one per worktree, so a 247-tab, 1,301-record profile derived them 289 times per listAll. On main those derivations were split up by the per-tab store writes this PR removed, so here they ran as one long main-process task (about 1.9 s on the Windows measurement machine). Each session.tabs listing request, tab restore included, now derives them at most once, on first use, and every snapshot it stores or answers reuses that value. Anything outside a listing, and any timer or task a listing leaves running after it answers, still derives them fresh. A /clear committed while a listing waits on I/O is still shown: its own commit re-projects the stored snapshot.
…g loop The previous commit kept one derived set of /clear replacements for a whole listing request, reused across the listing's later waits (the PTY census, the authoritative re-collection, subscribeAll's retries). A chat revealed from its history during one of those waits was then answered as its replacement: the stale set still renamed its tab. The request-wide scope is gone. The two loops that carry the cost each derive the set immediately before they run, with nothing awaited in between: the tab restore's projection loop (one snapshot per tab) and the collection's per-worktree answer loop, after its census. Both hand the set to storeMobileSessionSnapshot / toMobileSessionTabsResult through an optional parameter; every other caller derives fresh, as before. A first listing now derives twice (one per loop), later ones once, independent of tab and worktree count.
… a time The history restore that now runs after the tab list ran four chats at once. Its work is synchronous on the main thread (each open folds the chat's journal), so four lanes add no throughput: they only put up to four fold chunks in every event-loop turn, and each step of a user's read or send waits behind a whole turn. On the Mac measurement rig the first send after a restart waited 2.4-7.4 s behind the restore. It now opens one chat at a time. One chat's failure still costs only that chat, and a chat closed, or a host quitting, while it runs is still skipped.
Brings in main at 75040eb. main's record-store test harness moved the host test harness's store from <root>/store to <root>/agent-sessions, so the startup reconcile failure test now opens, edits and reads the relaunched store through agent-session-record-store-test-harness instead of the old path.
…up chat read The lease bookkeeping failure reporter called the host's failure sink directly, so a sink that threw turned a reported, recoverable store failure back into a rejected startup reconcile or read restore. The reporter now catches a sink throw and logs both the original failure and the sink error with console.warn.
…s agent once, after the check
Startup waits for the lease check (which settles each chat's owner from
the previous app run) only on the desktop window's path. A phone, the
CLI or a remote host's peer can send before it has finished, and the
check is not awaited on any RPC write.
This pins what keeps that safe. The send is accepted into the chat at
once, and its delivery's agent start runs the same check itself, so it
waits for the one already in flight. Two sends made while the check is
held start one agent, after the check, and both are delivered.
Without the agent start's own check, both messages are rejected
("couldn't restart", reason: the lease is still being checked) and no
agent starts.
…artup' into brennanb2025/chat-tabs-list-without-replay # Conflicts: # src/main/native-chat/agent-session-wire/structured-agent-session-rest-test-rig.ts
…s-list-without-replay # Conflicts: # src/main/native-chat/agent-session-wire/structured-agent-session-restart-restore.test.ts # src/main/native-chat/agent-session-wire/structured-agent-session-restart-restore.ts # src/main/native-chat/agent-session-wire/structured-agent-session-reveal.ts # src/main/runtime/orca-runtime-restore-structured-agent-session-tabs-once.ts # src/main/runtime/structured-agent-session-startup-tab-restore.test.ts
The runtime module size test failed in CI: orca-runtime-runtime-id.ts had grown to 402 lines. The field for the owed chat-history restore moves to the module whose restoreStructuredAgentSessionTabs owns the tab restore, and an import this PR had reordered goes back to main's order.
…seeds and settles from it Each chat's lifecycle state (whether its next open owes settlement work, and the status summary a session list shows) is stored in journal_session_state, written in the same transaction as the journal row it describes. The open appends one settlement plan built from the same per-entity rules the stored flag reads. Host startup seeds settled listed chats' statuses from stored state and settles every chat that owes work, listed or not, instead of replaying every listed chat after the tab list.
…p step Covers the in-transaction fold and its recovery (T1a-c), a failing state write (T2), stored state equal to a fresh replay after every append (T3), flag and plan agreement (T4, T4b, T15b), seeded rows equal to an open's publish (T5), startup opening only what it must and settling crashed chats with or without a tab (T6, T7, T14), headless startup and first snapshots (T8, T17), older builds (T9), epoch/import/repair writes (T10), the versioned derivation (T11), tab close (T12), status observers (T13), fence independence (T15a), a newer database (T16), and failures in the step (T18).
…tup restoration test
…s-list-without-replay # Conflicts: # src/main/native-chat/agent-session-wire/structured-agent-session-rest-test-rig.ts # src/main/runtime/orca-runtime-restore-structured-agent-session-tabs-once.ts # src/main/runtime/structured-agent-session-startup-tab-restore.test.ts
…out-replay' into brennanb2025/chat-status-stored-with-history Conflicts: - orca-runtime-restore-structured-agent-session-tabs-once.ts: the incoming listing rule (with no saved tab index, the union of tabs opened while the records import was owed and the profile ids) moved into listedStructuredAgentSessionIds, which the startup step and the tab restore share. - structured-agent-session-startup-tab-restore.test.ts: took the incoming rebuild; its startup write counts drop by one because stored state answers every chat, so the background restore opens none.
…t each corpus case owes The corpus's finished roster used "done", which no subagent state is: it read as unverifiable only through the unknown-value fallback. It is "completed" now, and each case's stored flag is pinned, so the flag and the plan agreeing cannot hide both being wrong.
…s-list-without-replay
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
This run reviewed the net delta since the last pullfrog review of the authored code (61e512fa32 → 6a04a45fb9): a single commit, the merge of origin/main (d17351401d) into the branch. The merge's only authored conflict resolutions are two files; everything else in the vs-main delta is main's own churn carried by the merge.
- Stop-row prefix moved to a shared constant.
structured-agent-session-open-settlement.tsandstructured-agent-session-dead-generation-settlement.tsnow build the gone-generationclientMessageIdandsettlementIdfromSTALE_SESSION_ROW_PREFIX('stale-session:') instead of the inline literal. The value is identical, so the settle ids are unchanged and the contract the shared file documents — a stop row must carry theproviderExitedfact or a known prefix, or a client derives a second notice — still holds. - Close-verdict tests reconciled with
main's new cut-turn semantics.structured-agent-session-close-verdict.test.tsadds thesettledTurnAfterQuit()reader (a fresh host, standing in for "what the next launch reads"), threads areaderparameter throughsettledTurn()and returnslabel/notices, and assertsmain's newerworkedForlabel plus the single cut-turnONE_NOTICErow for the quit/evict/crash cases. Test-only.
I verified the merge is faithful (git show --cc shows no other conflict-resolved file), that no production code still uses the literal prefix, and that the affected suites pass locally: close-verdict, open-settlement, dead-generation-settlement, journal-session-status-rules, and journal-session-state (191 tests). No PR-authored production behavior changed in this commit, and the still-open fence-omission thread on journal-session-state.ts is untouched by it.
deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏
…-history Main's #24576 removed journal repair, the rebuild from provider history and the read-only latch: a damaged chat, or one a newer Orca saved, now fails to load and nothing is deleted. This merge takes main's side throughout: - The PR's "load is corrupt, a rebuild is owed" bit is gone, with its status writes on the repair path; journal-repair-marker.ts and agent-session-journal-recovery.ts are deleted as on main. - The status derivation always counts live rosters now, so JOURNAL_SESSION_STATUS_RULES goes to 3 with a corpus case for a roster under a former repair notice (the other 21 cases derive the rules-2 digest). - An open that fails on its rows (damage or a newer build's) drops the chat's stored status in the same step, so startup never selects it again; the catch-up writes no row for such a chat, and startup does not settle a listed chat again after its open dropped the row. - The PR's persisted tab listing already keeps a tab whose chat cannot open, so main's unopened-tab set is not taken; main's test now lists visible tabs. - The rest-test rig counts opens at the journal store's open, since the open no longer asks the adapter for a history file.
…a rollback breaks nothing The status table arrived with schema version 5 so that an older build would open the database read-only and never append history behind a status it does not update. Main's #24576 refuses every chat in a database of a newer version, so a rollback to a build with #24576 but not this change would have opened no chat at all. The table is now created at every writable open with no version bump, as the queued-messages table is, and the database stays at version 4: an older build opens it writable and ignores the table. Each row records the history tip it was derived from (epoch, sequence and timestamp of the newest row, read inside the same statement as the write), and every reader (the startup seed, the catch-up, the per-chat-file restore, settle selection and the open's has-row check) treats a row whose tip is not the chat's tip now as missing. So a row an older build wrote history past, or a same-epoch repair regrew to the same sequence, is derived again; a row whose chat is gone or moved is never selected to settle.
…d one chat's failed write costs only its row A chat whose epoch holds no row (a quit inside an older build's repair, between its delete and its new epoch) made the status write find no tip and throw, which rolled back the whole catch-up batch: up to 15 other listed chats lost their status before the first paint. Such an epoch is now nothing to fold, so it gets no row and its open founds the epoch afresh, and each chat in a batch is written in its own savepoint, so a failed write leaves only that chat without a row.
…-history StructuredAgentSessionStatusBridge: main's failed-start row (#24904) and this PR's batched status projection are both kept. A chat with no summary drops any queued projection for its pane before it shows the failed start or clears the row, so a queued row cannot land after it. Main's failed-start test now waits for the tick its batched row lands in. The status derivation's inputs changed only in comments; the golden digest is unchanged, so the rules stay at 3.
…-history Main's #24991 made the provider handle opaque: the rest-test rig's record link and this PR's test journal identities now use codexProviderHandle. Main's journal-row-schema and journal types changed only the handle type's name; the status derivation is unchanged (golden digest passes), so the rules stay at 3. The rest rig's open counter reads the journal's chat and directory through typed narrowing instead of Reflect.get.
…que provider handle
There was a problem hiding this comment.
ℹ️ No critical issues — one informational note inline about the status table's shape assumption.
Reviewed changes
This run reviewed the authored delta since the last pullfrog review (6a04a45fb9): three branch commits, two origin/main sync merges, and a test-only follow-up.
- No schema bump; the status table is created at every writable open.
JOURNAL_DB_SCHEMA_VERSIONis back to 4 andJOURNAL_DB_RECORDS_VERSIONis gone;ensureJournalSessionStatusTablecreatesjournal_session_stateand its index withIF NOT EXISTSbesideensureQueuedMessagesTable, so a version-4 build opens the database writable and ignores the table and a rollback breaks nothing. - Each row records its history tip. New
(epoch, tip_seq, tip_ts)columns, read inside the status UPSERT'sSELECT;hasJournalSessionStatus, the startup seed, settle selection and the backfill tip check all treat a row not at the chat's current tip as missing and derive it again. - Failed status writes are contained.
writeJournalSessionStatusthrows when the tip select finds nothing; the catch-up folds an empty epoch to nothing and writes each chat in its own savepoint, so one chat's failed write costs only that row. - Merges and tests. The branch adapted its test journal identities and rest-test rig to main's opaque
codexProviderHandle(#24991) and re-applied #24904's failed-start row beside the batched projection; newjournal-session-status-tip.test.ts, and the cross-version writable-downgrade test now pins a main commit and asserts the stale status is re-derived.
I verified the tip predicate against append-past-tip, epoch roll/replace, chat deletion and same-seq rewrite, and traced every writeJournalSessionStatus caller to confirm the changes === 0 throw cannot abort a legitimate append or epoch write. The prior fence-omission thread was retired: main's #24576 removed the repair path that carried the omission.
deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏
| tip_seq INTEGER NOT NULL, | ||
| tip_ts INTEGER NOT NULL | ||
| ); | ||
| CREATE INDEX IF NOT EXISTS journal_session_state_unsettled ON journal_session_state (session_id) |
There was a problem hiding this comment.
CREATE TABLE IF NOT EXISTS skips by name only, so it cannot reconcile shape, and this CREATE INDEX ... WHERE lifecycle ... is prepared against whatever table already exists. On a profile whose journal_session_state predates the tip columns — an interim branch build that already stamped user_version = 4 with the older columns — the table is left as-is and this statement throws no such column: lifecycle, which fails openJournalDatabase and so every chat request. This is dev-profile-only and you note fresh profiles are needed, but the failure mode is a hard host-open error rather than the row-level re-derivation the tip/rules logic otherwise gives; worth a comment here or a one-time shape guard if interim branch profiles should survive.
Technical details
# `journal_session_state` shape is not reconciled on the no-version-bump path
## Affected sites
- `src/main/native-chat/agent-session-journal/journal-session-state.ts:46` — `CREATE TABLE IF NOT EXISTS journal_session_state (... epoch, tip_seq, tip_ts)` leaves an existing old-shape table untouched.
- `src/main/native-chat/agent-session-journal/journal-session-state.ts:60` — `CREATE INDEX IF NOT EXISTS journal_session_state_unsettled ... WHERE lifecycle <> 'idle' ...` is prepared against the existing table and throws `no such column: lifecycle`.
- `src/main/native-chat/agent-session-journal/journal-database.ts:134` — `ensureJournalSessionStatusTable(probe)` runs at every writable open; the throw escapes `openJournalDatabase`.
- Reachable from ancestor commit `f52521dd9b` (in this branch's history), whose `ensureJournalSessionStateTable` created `journal_session_state (session_id, state_version, epoch, seq, owes_work, unverifiable_owner_fences, summary_json, last_activity_at, written_at)` at `user_version = 4` with `IF NOT EXISTS`.
## Required outcome
- Confirm the interim-branch shape is out of scope (say so at `ensureJournalSessionStatusTable`), or make the open reconcile/drop a table whose shape predates the tip columns instead of throwing.
## Suggested approach (optional)
- A `PRAGMA table_info(journal_session_state)` check that drops and recreates when the tip columns are absent, or a guarded index creation. Reproduced in `sqlite3`: create the old shape, then `CREATE INDEX IF NOT EXISTS ... WHERE lifecycle` errors.…provider handle builder itself
…-history - #24660: the row writer's planned write is public writeRows (the new step writer calls it); the transactional write that also stores the chat's status is commitRows, so every path still writes the status row once per transaction. - #25181: the open settlement plan ends a running call as its turn's row ended (runningCallEnd / terminalAgentJournalBody) and revises calls an unproven settle closed once a proof names their owner; the status facts read requiresTerminalSettlement, the same facts as before. Liveness keeps main's lostLiveWorkJournalBody under the per-item roster revision. - #24660: the open plan settles what a gone process left queued through holdUnsentSends (a person's message becomes a held card); its leftovers are exactly the queued sends written before the open. - #25159: reading and settling a chat need no adapter. The persisted tab listing lists every tab with a record, startup settles any chat whose record exists, and adapterSupportsRecord / hostCanSettleRecord are gone, as on main. - Line caps: the runtime's startup step moves into its own layer (orca-runtime-structured-agent-session-startup-step.ts), and the host exposes its restore as one `startup` surface instead of four pass-through methods. - The golden digest is unchanged, so the status rules stay at 3.
…es its agents, as main's host deps require
… chats get their status from the restore after the listing No one runs native chat on a build without the stored-status table yet, so the first launch with it needs no machinery of its own. Startup no longer derives the missing rows of listed chats from their rows alone before the tab listing answers (with batched, unsynced writes), and no longer opens listed chats still in per-chat files before it. A listed chat with no current row (written before the table, by an older build, or by other rules) or still in a per-chat file is left to the restore after the listing, as on main: that open writes its row, settles it and publishes its status. Removed with it: the fold-only status path, the tip read only it used, and the restore's per-call concurrency only the pre-listing pass set. Their tests go too. Two guards are re-homed: the rules-mismatch reader case is in the tip test, and the "no second recovery for a lease that outlived startup" case now runs on a rowless chat through the restore after the listing. The startup-state suite shares one startup and crash helper, merges its seed/read parity tests, and runs the lease-check permutations as one table.
…s reading of each row; the failing-savepoint case tests its rollback The stored-status suite walked the corpus several times for one property. The fold-undo walk already checks, after every real commit, that the stored row equals a fresh derivation; it now also checks at the end of each chat what startup reads: selection against the pinned answer, the partial-index read, the active turn, and a settled summary no fence changes. The separate T3 and T15a walks and the schema-version test (the records-migration test pins version 4) are gone. A new corpus case, a failed start that rejects queued sends, carries the append-status-commit test's failed-COMMIT case into the same walk; the rules digest is re-recorded for it with no rules bump, since no derivation changed. The bookkeeping-savepoint case inserted a status row missing the columns that became NOT NULL, so the INSERT threw before writing and "the savepoint rolled back its write" passed without being tested. It now inserts a complete row, checks the write happened, and runs once on the chat that holds every case.
… into the cases that already cover them The queued-leftover settle check runs inside the corpus-by-evidence agreement test (every rejected leftover is checked, and the no-record plan for the queued case); the "never selects an unverifiable turn" test is dropped, since that agreement test pins it for every evidence. The two host-build failures that open the startup gate run as one table; the slow-build-past-the-ceiling pair is dropped, as the gate-level ceiling table covers each outcome.
…hich nothing can reach now The seed checked each listed chat still had its tab because the catch-up and the per-chat-file restore awaited between the listing and the seed. With both gone, the listing's ids and the seed run in the same task, so no tab can close between them.
…epoch, and the lease-check table names every row and checks the listed chat too Breaking the tip check's epoch condition left every test green: no case moved a chat to a new epoch whose newest row matched the old tip. One does now. The lease-check permutations ran through it.each, whose interpolated titles are shortened past 40 characters, so a test filter could not select the two rows where a command completes the check. They run as a loop now, and each row also checks the listed chat's verdict, which the restore's own lease answers decide.
Conflicts resolved keeping both sides: - status feed/summary: the PR's shared row builder (seed + open) now carries main's Stopping and pinned launch directory; the published map keeps main's first-input key (a seed records none). - journal projection: main's first-input key and Stopping on the PR's stored projection; the journal is rendered only when a Stop is on record. - dead-generation settlement: the PR's plan; main's open-reasoning-row ending moved into journal-open-settlement-plan. - command entry points (attach, options read, send) keep the startup gate with main's new options and comments. - test fakes gain stopMarks; renderer bridge tests await the batched tick. Outside the conflicts: fold undo restores main's renamed latestAcceptedTurnSequence; status-feed's host factory moves to its own module (max-lines); the Codex stopped-send test's interrupt waits for the turn it takes instead of racing a 50 ms poll.
Conflicts resolved keeping both sides: - host-status-feed: main extracted the same factory to the same file; main's file kept. - status summary: main's conversationName from the record joins the PR's shared builder, so a seeded row carries the chat's name as an open's does. - status feed: the PR's subscribers class with main's publishConversationName; the published rows move to structured-agent-session-status-publications (max-lines). - host-tabs: main's deferred tab-hidden notice without the PR-removed unopened set. - runtime deps: commandsReady and onSessionTabHidden both kept. Outside the conflicts: installRuntimeRpc moves unchanged to main-process-runtime-rpc-install.ts; the first merge left main-process-runtime-launch.ts over max-lines.
|
Brought up to date with main at How the conflicts were resolved
Test-fake change ( Line-limit moves (no
Checks on
Locally on the merged tree, the unit tests the skipped |
…ode runtime project
Conflicts resolved keeping both sides: - dead-generation settlement: main's split of the unfinished-work helpers into structured-agent-session-unfinished-work and its watched-exit revisions (exit, withRevisions) kept; settleStaleStructuredAgentSessionState keeps the PR's delegation to planGoneGenerationSettlement. Main's only change to that body was the isInProgressItem rename, which the open settlement's own predicate already covers. - host: the PR's public `startup` field built through main's `reveal.` namespace import. - get-worktree-ps: the PR's startup gate import beside main's isTuiAgent and resolvedTuiAgentArgsBypassPermissions imports.
|
Closing in favour of a much simpler design. After a restart a native chat can only be done, idle, failed or interrupted, so instead of a new per-chat status table written with every history row, native-chat statuses will be saved in the agent status store that CLI agents already persist, and any that were working or waiting when Orca stopped will read Interrupted on the next launch. A new PR follows. |

This PR includes #23978 (lists chat tabs without replaying every chat); the two land together.
Why this PR is big, and how to read it
Where the files go. About two thirds of the added lines are tests. Some files the counter above calls production are test support named as such (
journal-session-state-test-corpus.ts,journal-session-state-test-writes.ts,structured-agent-session-rest-test-rig.ts,-rest-test-observations.ts,-rest-test-journal-opens.ts,-rest-test-startup.ts,journal-host-database-test-support.ts). Real production code is about +2,740 / −600 lines, almost all insrc/main/native-chatandsrc/main/runtime. #23978's own share of that is small, about +210 / −110 lines in 13 files.Why it isn't split further. Each half regresses without the other, which the measurements showed. #23978 alone removes the replay before the tab list, so statuses then fill in seconds after paint and a crash-left turn is only closed out when its chat is opened. This PR alone has nothing to replace: its stored statuses and startup settle are what make removing the replay safe. Together they're faster than main on every machine measured, with every status present at paint. The parts that can land separately already have: the background copy of old chat files was taken out (it becomes its own follow-up), and the crash-cut turn label moved to #25043.
Why the tests are large. A status saved with the history is only safe if it always equals what replaying that history would give. So the corpus tests replay every chat after every successful and every failed write and require the saved status, the in-memory fold and a fresh replay to agree. On top of those: checks that remove each rule in turn and must then fail, and cross-version wire tests (old desktop or phone against a new host and the reverse).
Read it in this order. Each group's main files, with their added lines:
journal-session-state.ts(295) defines the row and its rules version.journal-status-projection.ts(58) derives it from the fold.journal-session-status-writer.ts(149) andjournal-row-writer.tswrite it inside the append.journal-fold-undo.ts(132) andjournal-fold-holder.ts(44) undo the in-memory fold if the commit fails.journal-database.tscreates the status table at every writable open, with no schema version bump.structured-agent-session-startup-state.ts(277) seeds the status store and settles.-startup-lease-phase.ts(93) is the one lease-recovery step.src/main/runtime/structured-agent-session-startup-gate.ts(123) and-startup-step.ts(106) hold chat commands, never the window, until the settle ends (4 s ceiling). A chat with no usable saved row is not handled here: the restore after the tab list opens it, as with fix(native-chat): list chat tabs at startup without opening every chat's history #23978 alone.structured-agent-session-open-settlement.ts(262) andjournal-open-settlement-plan.ts(144) work out and write the verdicts.-dead-generation-settlement.tsshrinks (+22 / −92) because that work moved there.orca worktree ps, mobile and the dashboard.-status-feed.ts,-status-summary.tsand-status-subscribers.tson the host.StructuredAgentSessionStatusBridge.tsxin the renderer, which applies one snapshot's rows as one store update (that removed a 600 ms freeze).src/main/runtime/rpc/methods/session-tabs.ts,session-tabs-inventory.ts,structured-session-tab-restore.ts,orca-runtime-restore-structured-agent-session-tabs-once.ts,structured-agent-session-host-tabs.tsand-restart-restore.ts.docs/reference/agent-status-store.mdis updated to describe where statuses now come from.ELI5
The problem. When Orca starts, it has no saved record of what state each native chat is in: idle, mid-turn, or waiting on an approval. The only way to find out is to open the chat and replay its history, meaning re-read every saved row of it. So on main, before the tab list answers, Orca opens and replays every chat that has a tab, four at a time. Paint, the sidebar's chat statuses and your first send all wait behind that. #23978 (included in this PR) moves the replay to after the tab list. Tabs then appear sooner, but the statuses fill in one chat at a time, seconds after paint, and the first send waits behind that replay. The replay is also where a turn that a crashed Orca left running gets closed out, so it can't simply be skipped.
The fix. Orca now saves each chat's status next to its history, in the same database write. At startup it reads those saved statuses. A chat with nothing left over gets its sidebar status straight away, without being opened. A chat that a crash left mid-turn is closed out at startup, whether or not it still has a tab.
What you will notice. Chat statuses are in the sidebar when the window paints, and in
orca worktree ps, on mobile and on the dashboard. A healthy profile opens no chat at startup. A chat a crash cut off shows its turn as ended, never the stale "working": "Failed after N" when Orca has proof the agent stopped (main's existing label for a crash-cut turn, unchanged here), or "Couldn't confirm" when it can't prove how the turn ended. A chat with no saved status yet (last written before this build, or by an older build after a rollback) gets one the way every chat does with #23978 alone: it is opened in the background right after the tab list, so its status shows shortly after paint, once.This PR and #23978 land together. #23978 deletes the replay before the tab list, and this PR deletes the replay after it.
Words used below.
What Changed
What you see
Chats with no saved status yet. A chat last written before this build, by an older build after a rollback, or by a build with other status rules has no usable row. There is no special pass for it: a listed one is opened by the background restore right after the tab list answers, exactly as with #23978 alone, and that open writes its row, settles anything a crash left and publishes its status. So on the first launch after upgrading, those statuses appear shortly after paint rather than at it; from the next launch on they come from the saved rows. A chat without a tab gets its row when it is next opened. Native chat has no real users yet, so in practice this is a one-time cost for development profiles.
Chats still in an older build's per-chat history file. These are users upgrading from a release before the shared chat database. A listed chat in that state is opened from its file by the same background restore after the tab list, not before it. It is not copied into the database, so this repeats on every launch until the chat is next written to. A follow-up removes the repeat (see Differences).
Chat commands wait for the startup settle. This covers a send, Stop, an approval or question answer, an option change, a queued-message action, a goal change, rewind,
/clear,/compact, and the reads that open a chat (history, subscribe, reveal). Each waits until the startup settle ends, then goes through. None is refused because of it. The wait ends after 4 seconds from launch in any case, which is below the shortest client timeout on a held call: the AI vault gives a chat restore 5 s, while remote desktop and phone give 15 s. If a command goes ahead before the settle reaches its chat, that chat is still settled first, by the command's own open. The only difference is that a crashed turn may then read "Couldn't confirm" instead of the verdict from the evidence startup records. Each launch logs one line saying when the startup step started and ended and what opened the wait.One small behaviour change. Suppose a crash left a message queued in a chat, and you press Stop on that chat after the restart. The message now reads "Orca restarted before this message was sent." instead of "cancelled". Nothing is sent either way. The rejection rule already existed on main, in the delivery loop's first step. It now runs when the chat is settled, which happens before Stop reaches it.
No visible change for chats that were never sent to, SSH (structured chat is not offered there), or folder workspaces (everything here is keyed by chat id, not by git worktree).
How it works
A status row per chat, saved with its history. The new table
journal_session_stateis created at every writable open, beside the queued-messages table, with no chat database version bump. Each row holds:A partial index covers the rows that show unfinished work.
Written in the same transaction as the history row. An append inserts the row, runs the existing queued-draft bookkeeping, applies the row to the fold, writes the status derived from that fold, and commits. Observers hear of the write only after COMMIT.
One set of rules decides what a settle writes and what the row counts.
journal-open-settlement-plan.tshas one rule per kind of leftover work, and both the settle and the stored counts use it. So startup selects a chat exactly when its settle has work to do, and after the settle commits, the row reads settled.Tip check. Every reader (the startup seed, settle selection and the open's "has a row" check) treats a row whose recorded tip is not the chat's tip now as missing, and the chat's next open derives it again, by the same path as a missing row. Each check is two primary-key lookups. A row whose chat is gone or has moved to another epoch is never selected to settle.
Rules version. Each row carries the version of the rules that derived it (
JOURNAL_SESSION_STATUS_RULES). A row derived by other rules, for example by a later build that changes what counts as working, reads as missing and is derived again by the same path as a missing row, at no extra cost. A golden test over the full corpus of chat states fails when the derivation's output changes without a version bump.The startup step, run by the host itself. The wait for chat commands is put in place before any request is served, on desktop and on orcad. The step then starts right away; on Windows desktop it first waits for the shell PATH to be read. It needs neither the terminal daemon nor the hook server's listener.
recoveringstarts in the background, 4 at a time and at most 15 s each. A recovery stops a surviving provider process and records its death, which is the evidence each verdict reads. A recovery whose process check stops answering leaves its lease unverified, never treated as dead, and startup goes on. The tab list waits for the check, never for a recovery. Every startup step asks this one phase about leases and never re-runs the check, and its answers are re-read from the records, never kept. So a check that a command completes later also counts.worktree ps, mobile and the dashboard read. The tab list answers after this.The window applies a burst of statuses as one store update. Statuses that arrive in the same tick are queued and applied through the status store's existing batch update. So every reader of the store runs once per burst instead of once per chat. Without it, a launch that seeds hundreds of statuses at once froze the window while each one was applied separately.
docs/reference/agent-status-store.mdnow says where a structured chat's status is stored and how a chat without a row gets one.Why
Differences from the common pattern
The common pattern: every conversation's status lives in database rows, written in the same transaction as its events. A derived projection carries a version, and a stale one is derived again. Startup recovery runs before commands are served: it selects every conversation whose status shows unfinished work and closes each one out in one write. Old-format data is copied into the database once.
worktree ps, mobile, dashboard) subscribes to that one store (docs/reference/agent-status-store.md). Seeding it from the table is how those readers read the table. A seeded row and the row the chat's open later publishes come from the same builder.recoveringlease.Linked Issue
Part of the chat-startup series: #23964 (merged), then #23978 (list tabs without replaying chats; this PR's base), then this PR. #23978 and this PR land together.
Visual Proof
Screenshots of hidden test windows, from live QA runs on Macs (development builds, app launched in the background). Both rounds ran before this PR dropped its first-launch pass (the derivation of missing rows before the tab list): the crash, relaunch and later-launch shots show paths this head keeps; the first-launch-after-upgrade shot and numbers show the removed pass.
Main at paint. It paints later, because its tab list waits for every listed chat's replay.
This PR at paint, which comes earlier, with every chat's status already shown.
First launch after upgrading, with the removed first-launch pass: a send made right away is answered.
Crash case, before the crash: a chat mid-turn.
Relaunch after the crash, at paint: no stale "Working".
The cut turn is settled, and a send made early in the launch is answered.
A turn a crash cut off reads "Failed after N". That is main's existing label for it, unchanged here and being revisited separately. The stored outcome is the interruption.
Round 2 (head with current main merged)
Setup. Live QA on an M4 MacBook Air, development builds, with a fake Claude agent: this PR at 54da015 against main 5e65871.
Results.
After the merge with current main, expanding a worktree shows a status dot on every chat row.
This PR, relaunch after a crash mid-turn, at first paint.
This PR: the cut turn reads "Failed after 18s", and the early send is answered.
Main, the same crash, at first paint.
Main shows the same "Failed after N" label.
Before the close: the chat is mid-turn.
Relaunch after the close and the crash: the closed chat stays closed.
Session history can't list the rig's fake agent, so reopening the closed chat was not checked.
Testing
At 559dab1:
Every test file this round changed or deleted, and each importer of a changed module, passes when run by explicit path (34 files, 418 tests). The full suite, typecheck and lint gates were last run whole at 54da015; CI and the coordinator's typecheck run them on this head.
Each rule below was checked by breaking the code it guards and watching its test fail: 72 breaks, 69 red. The 3 that stay green are a duplicate republish of an equal row for a chat already open at the seed, recoveries starting at the settle instead of at the lease check (timing only), and the lease phase answering from its first failed check instead of the records, which no verdict at this head depends on (the settle awaits every recovery first). The first-launch pass's breaks went with its code.
The row always matches the history.
journal-fold-undo.test.ts: one walk over every chat state in the corpus. Every append is first tried with a failing COMMIT, then for real; after each, the in-memory fold equals a replay of the disk and the stored row equals a fresh derivation. At the end of each chat, startup's selection reads the pinned answer, the partial-index read agrees, the active turn matches, and a settled summary is the same under any fence. Also: one chat through every state with a tombstone, map order after a tombstone, Stop and Resume marks, an evicted settlement id, and a bookkeeping savepoint that writes a complete row and then fails, whose write must not reach the disk.journal-session-state.test.ts: observers hear of a write only after COMMIT. A failed status write fails the append. A failed COMMIT is undone without re-reading the disk. A fold whose undo failed is re-read before its next use, and the chat stays open. A held submission is never changed in place. Epoch writes carry the row. An open that fails on its rows (damaged, or a newer Orca's) drops the row, and nothing selects the chat again.journal-session-status-tip.test.ts: a row an older build wrote history past, a same-epoch repair that regrows the history to the same sequence, a deleted chat's row, a chat an older build moved to a new epoch, and a row derived by other rules all read as missing, are never selected, and the next open derives them again.journal-append-status-commit.test.ts: a send's ledger answer commits with its row and one status write; a failed start's rows commit with one status write.journal-session-status-rules.test.ts: a golden digest of the derivation over the corpus. It fails when the output changes without a rules-version bump.structured-agent-session-startup-damaged-chat.test.ts: a stored mid-turn row plus a damaged history: one launch logs once and drops the row, and the next launch selects nothing (listed and unlisted).The settle and the row agree.
structured-agent-session-open-settlement.test.ts: for every chat state and every kind of death evidence, a chat is selected when and only when its settle has work, the settle appends exactly that plan (every queued leftover ends rejected), and afterwards the row reads settled. It also covers an item whose key won't parse and a turn a person's Stop found.Startup seed, settle and leases.
structured-agent-session-startup-state.test.ts:/clear, a read) are answered as it ends, and the gate never deadlocks the settle.Gate and listing.
structured-agent-session-startup-restoration.test.ts: commands are held until the settle ends. The gate opens when the host build throws or builds no host. Commands are let go before the AI vault's 5 s restore timeout. The gate closes once per launch. The launch writes exactly one timing line, whatever the order of events. A host no client ever lists is still seeded and settled. No failure fails startup.structured-agent-session-tab-listing-startup.test.ts: the listing opens no chat before it answers, and the history restore starts only after the answer, forlistAllandsubscribeAll. The listing writes nothing to the record store.structured-agent-session-send-compatibility.test.ts: commands that arrive before the host is built build it and are answered.Window.
structured-agent-session-status-bridge-burst.test.tsx: a snapshot of 247 chats reaches the store as one update, with the same rows and tab labels as writing each row on its own. A cleared status still removes its row.Migration and import.
journal-database-records-migration.test.ts: the status table is added with no version bump, and a version-4 build still opens the database writable.journal-per-session-import.test.ts: copying an old file drops the replaced history's row, and the next open writes it.Cross-version (
tests/e2e/cross-version-wire), one file at a time. At 559dab1:agent-session-stop-event-downgrade(an older main build at schema version 4 opens this database writable and appends to it; the status row it left behind reads as missing and this build derives it again),agent-session-newer-chat-refusalandcross-version-session-tabs-retirement-proof, all passed. At 3005424 alsocross-version-agent-session-wire,orchestration-delivery-downgrade,agent-session-resume-marker-downgradeandrelease-checkout, all passed.Measurements
Conditions.
Later launch, host seed
Low-spec Windows, production bundle, later launch, host seed (both arms built with
electron-vite build, renderer loaded from disk; 3 interleaved pairs)What the numbers say.
orca worktree psanswer carried a status for every chat, against 0 on main; that run used the removed first-launch pass. At this head, a headless host seeds every chat that already has its row; a chat without one gets it once a client lists the tabs and the restore after the listing opens it, as with fix(native-chat): list chat tabs at startup without opening every chat's history #23978 alone.Review
Agent skill upstream boundary
docs/reference/agent-skill-sharing-upstream-boundary.mdand copies or mechanically translates no upstream skill-installer source, tests, fixtures, registry entries, path tables, comments, or documentation.Notes
Checklist
N/Awith reasonpnpm lint,pnpm typecheck,pnpm test, andpnpm buildpass (or CI will cover; local preferred)