Repository navigation
feat(native-chat): copy old per-chat files into the chat database in the background - #24462
Closed
brennanb2025 wants to merge 52 commits into
Closed
brennanb2025 wants to merge 52 commits into
brennanb2025 wants to merge 52 commits into
Conversation
…base in the background Step 2 of L'-c, before the stored-state write is wired: the host's background copy job (interval, per-run budget, startup-work gate, disk guard, recorded give-up, listed-first order and walk), the importer's abort checks, copy-side fold, returned load, never-written file deletion and post-publish re-stat, the owed import adopting that load, a supplied load in place of the open's replay, and the teardown phase that stops it first.
…with-history' into brennanb2025/old-chats-copied-in-background Wires the copy's publish to the reshaped status row (written in the publish transaction from the copy's own fold), settles a copied chat by the same unsettled rule the startup settle selects by, and starts the background copy from the startup step after the settle.
…ound copy, and test the copy The version 5 migration creates the status table empty, so a chat already in the host database has no row until it is opened, and startup can't select one a crash cut. The background copy now writes those rows too: the chat folded a batch per task, then only derive and write in one short transaction, settling a chat with work left by the same rule startup selects by. Adds the copy's tests.
…er delete, and settles by startup's rule - A throw inside a run is logged and never reaches an unhandled rejection: one chat's failure (its file's stat, its give-up reads) skips that chat; any other ends the job for this launch. Starting the job can no longer fail startup either. - A settle that fails after a copy published is logged as a settle, not counted as a failed copy; the settle selects by the same rule as startup's (a record this host serves), so the job never settles a chat startup refuses. - Rows a stopped copy staged are deleted a batch per task, abort-checked, not in one statement. - The file's size and mtime are taken before the verify reads it, so a write during the verify is read by it or keeps the file. - The missing-row phase skips chats whose record is gone, and its start and end ignore them. - A missing row that fails for good is given up on, keyed to the chat's epoch and tip and the app version, in the same give-up table as a failed copy (now journal_background_failures). - A chat too big for the free space is skipped for this launch; the job waits only below the 512 MiB floor.
…art each fail logged, never thrown
- Both of the job's settles (a copied chat, a chat given its missing row) go through the startup state's one closed-chat settle, which now takes the copy's load in place of a replay and refuses a chat this host cannot settle. - The missing-row phase skips any chat `canSettle` rejects (record gone, or a provider this host does not serve), and the job's start and end ignore them, so it never rewrites a row startup drops. - The copy's publish still writes the chat's status row from its verify fold; #24233's delete of that row at the import site is replaced by that write, and its test now pins the write.
…; the status-store doc says who writes an old chat's row
6 of 9 tasks
- settleClosedChat keeps #24233's new doc comment, plus this branch's optional load. - The startup step's callback keeps #24233's gate opener and passes the runtime's startup chat work to the copy. The history restore a tab restore owes now lives in that chat-work object, which also answers whether it is owed, so the runtime holds one fact instead of two.
…which the first tab listing waits for; its gate's signals each have a test
…0 ms of each second
…not only aligned ones, holds its share
…d each task by row JSON, not only rows
…e chat it holds, and alone sets the rate - Inside a chat the pace never waits while anyone waits for that chat, and a wait ends when someone starts to; the debt is paid at the next yield between chats, which holds no lock. The task queue says who waits for a chat. Lock waits, disk probes and a restored chat's own import are not charged. - The copy yields between its last batch and the verify, between the verify's two sides, and before the publish, so no chat runs in one task. - No per-run wall budget or chat cap: the 1 s tick only restarts a run the gate or the disk paused, and the pace alone sets the rate.
…huge; only the read-only passes split by size
…eeps; one queue for the copy's locks
… copy of old per-chat files - The copy's failures go to the host's logger, as the rest of the wire layer's now do; the copy's summary line stays on the console, since the logger reports only failures. - The owed history restore keeps living in the startup chat work, with main's logger call. - The restore wiring keeps passing the host's task queue; the lease-reconcile failures go to the logger, as main made them.
…ce when the copy runs it
Inside a chat the copy's tasks only yield to the next macrotask, so anyone who opens, reads or commands that chat, through the host lock or a restored chat's own write queue, waits for the rest of its copy and never for the pace. The pace keeps every task's time as debt and pays it at the yield between chats, which holds no lock. The give-way machinery is gone: the task queue's waiter count and the owed import's yield through the journal write queue are reverted to the base. A restored chat's owed import is charged by its wall time, whoever started it.
…py yield and every pace wait falls between chats
- The missing-row write is #24233's `backfillJournalSessionStatus` now, taken as is; the copy's owed-row read and the status give-up key move to journal-session-status-owed.ts, and the size bound (`charBoundedBatches`, `IMPORT_BATCH_CHARS`) is imported from #24233's file. - Quit reaches that fold as a signal: the host database's import abort is an AbortController. - The startup state keeps the copy's `isSettling` and `loaded`, beside #24233's `deriveMissingStatuses`; reveal keeps the copy's control beside #24233's derive-first restore.
… chat works The background copy of old per-chat files paced itself to a share of the main thread but kept running while a chat streamed or a send was in flight, so it competed with the work the user was watching. It now reads the host's status feed: no chat's copy starts while any chat this host holds open is working (a turn running, or a send journaled that the provider has not answered, for every provider), nor until the chats have been quiet for 5 s, timed from when the work ended. A chat whose copy is under way when one starts working stops at its next batch through a per-chat signal composed with quit: the importer, its verify, a restored chat's owed import and the missing-status fold all take it. Nothing is published, the chat stays owed and is tried first once the chats are quiet; the stop is not a failure, so nothing is recorded and no retry is counted. The staged rows go at the next try's sliced delete. No wait is added inside a chat.
…ts that never go quiet still open whole A chat starting work stops a copy at every one of its task boundaries in turn; once quiet, the shared database holds exactly the file's rows (epoch, seq, ts, JSON) under one epoch, with the status row the chat had. A stopped missing-row fold writes the row a whole fold writes. With a chat streaming for the whole test, the copy starts no chat, and a restored listed chat and a closed unlisted one still show their whole history through the read path a chat not yet copied takes today.
… step it can stop, and counts a turn under a prompt as work A chat starting work now stops the copy before the status derive that precedes the publish, before the settle of a chat it just copied or gave its missing row (startup settles that row), and before the old file is opened when work began between the gate and the chat; the gate is checked again when the chat's signal is taken, after the walk and the disk probe. A running turn waiting on an approval or question reads `attention`, not `working`, so the copy used to resume 5 s into it; the gate now reads the projection's `owesWork`, which holds beneath a pending prompt.
…agent frames, not to turn state The copy's gate now holds only while an open chat has a send handed to its agent and unanswered (queued sends excluded), or any agent frame reached the main thread within the last 5 s. A turn parked on a prompt, or running a long silent tool call, no longer holds it; its next frame stops the copy again. The signal is each provider's frame ingress, not a journal row: streamed text, reasoning and tool output are gathered for 60 ms and then written only as the text grows by an eighth, so a long answer goes many seconds between rows while the main thread parses every delta. Each adapter notes the frame on the chat's host event sink (new optional noteProviderFrame), which forwards it to the bound target and the host's subscribers, where the copy listens. The status feed is back to #24233's; the sink's watermarks moved to their own module. Every R8-1 stop check stays, and nothing waits inside a chat.
… pinned on a thinking delta Claude's thinking and tool-input deltas pass through neither the shared text coalescer (only text deltas are gathered) nor a row write (stream frames write none), so only frame ingress sees them. The adapter's frame delivery now adds one call and keeps its condition as it was.
…delta reach the sink only as a noted frame A sink that records every call shows these frames make no other sink call, so noting activity inside the sink would never see them; the adapter hooks stay.
…hat copy until it is handed over A send to a chat whose agent was not running read as queued (not yet handed over) until the agent started, so the copy could run through the spawn, the history drain and the hand-over. A send not yet handed over now counts as in flight unless the chat's agent is ready, so only a send waiting behind a running turn or prompt is left out. The sink also notes activity when a bind writes rows that frames left before it, since those frames were noted to no one. A Codex approval request is now pinned as noted at receipt.
…round copy either #24233's startup pass leaves a corrupt history rowless, so the next boot opens it and its rebuild runs. The copy now follows the same rule: a copied chat whose fold is corrupt is published with no status row, and the missing-row phase folds with #24233's foldJournalSessionStatus, writes no row for a corrupt fold, and gives the chat up (the existing 'status' give-up, keyed to its epoch and tip plus the app version), so later launches neither re-fold it nor start the job for it until its rows change, which its rebuild does.
… as the job does, now that #24233 dropped its one-chat wrapper
Contributor
Author
Review status: paused, to be superseded (do not merge)Decision. This PR is paused. A follow-up to #24127 will copy each old per-chat file on that chat's first read, with restore's read of a listed tab counting as a first read. That follow-up supersedes this PR.
#23978 + #24233 land safely without this PR.
What this PR proved, carried into the follow-up
Final head Measured on
Low-spec Windows:
M4Air live QA of the copy: all scenarios pass. See the QA comment above for what was copied, the corrupt-chat handling and the quit-mid-copy resume. For the follow-up
Not verified: the real Claude CLI test (it writes the real user profile; skipped by decision), and Linux and SSH hosts. |
The copy's startup gate covered the tab listing, history restores, restoration and the settle step, but not #24233's later startup status pass, the fold of the listed chats with no status row after paint. On the Windows high-spec host seed the copy imported unlisted old files inside that pass, costing it 200-300 ms of main thread. The startup state now reports the pass in flight (isDeriving, from its own calls still running, beside isSettling), and the host's gate holds while it does.
…ot a startup-state flag The startup status pass runs at the front of restoreReadableSessions, before the restorer reports restoring. Reveal now counts that call from entry to its end and holds the copy while one runs, so the derive pass and anything later added to the call are covered without a new signal on the startup state, which is back to #24233's.
…last task took On the low-spec Windows machine the copy's longest task was 438 / 462 ms, one page of a per-chat file: its 512-row commit (204 / 363 ms) and, in the same task, the next page's read and batching (about 195 ms). A fixed page is fine on a fast machine and far too big on a slow one. Each page is now read in a task and written in a task of its own, and a page size per copy follows each task's wall time toward 25 ms: a task over it halves the next page, two tasks in a row under half of it grow it a quarter, within 8 and 512 rows. The leftover delete and both verify sides read the same page; the char ceilings still split a page of huge rows. A fast machine keeps whole 512-row commits; the publish, the verify and every stop check are unchanged.
…ever an earlier one shrank it to The page size is measured within one copy, in memory only, and nothing carries it to the next chat or launch.
… in flight is read once per journal tip A user's own first open, and a restored chat's owed import a read pays, take whole pages as on main. The importer takes its pages as an option, whole by default; the job passes time-sized pages for its own imports, and so does the owed import when the job pays it (with its signal). Whether a chat has a send in flight was re-derived on every gate check by walking all of its submissions, 12-14 ms per check on the low-spec machine. It is now read again only when the chat's journal tip, its fence or its agent's readiness moved (every send and answer is a row, so a new one moves the tip), kept per journal and gone with it. The cold-send rule is unchanged.
Contributor
Author
Live QA on a real Mac: the background copy of old per-chat chat filesBottom line: I found no user-visible problem. Every chat was copied once into the shared chat database with every row intact. The old file was deleted only after its copy. Nothing was duplicated or lost. A quit in the middle of the copy resumed cleanly on the next launch. Two things behaved differently from what I first expected; both are explained below. One corrupt chat is handled by Orca's existing repair before this job reaches it. Also, one or two files go through startup restoration rather than this job. What was tested
1. Before the copy: each chat opens with its full historyExpected: every seeded chat opens with its full history while its old file still exists. Observed: pass. K-1 was kept streaming, which holds the copy back. While it streamed, the long chat, a short repo chat, a folder chat and the open tab each opened with their full history, and each one's old file still existed when it was opened. Long chat L-1 before the copy (left) and after the copy and a relaunch (right). Both long turns ("FIRST DONE") are there, and nothing is repeated:
2. The copy runs with a chat open, and finishesExpected:
Observed: pass.
O-1, the open tab, at launch while its history was still in its old file:
Where startup restoration comes first. One or two files leave through startup restoration before the job starts. These are the visible tab and any tab whose history is corrupt. The file count went 32 → 31 → 30 before the job copied the rest in four runs, and 32 → 31 in the other two. The corrupt chat with no tab (U-1) goes down the job's corrupt path:
The corrupt chat with a tab (X-1) never reaches the job. Startup restoration opens it first:
This is the open-time corruption handling main already has (
3. After the copy: same history, nothing missing or doubledExpected: after a relaunch, every chat opens with the same history as before the copy. Observed: pass.
4. Quit in the middle of the copy, then relaunchExpected: a quit stops the copy partway, and the next launch copies the rest once, with nothing duplicated, lost or left half-copied. Observed: pass.
S-16, which was still in its old file at the quit and was copied after the relaunch, shows its one turn once:
5. The copy waits while a chat is streamingExpected: the copy takes no time while any chat streams, and starts about 5 s after the stream ends. Observed: pass.
Caveat: my file-count watcher stopped early in every run (a rig problem). So these times come from the agent log, the moment the summary line was written, and the per-chat checks. I have no per-file deletion timeline after a stream. Isolation
Not verified
|
…o answer On the Windows high-spec machine the copy imported the unlisted old files from 11.7-11.9 s after launch, before the first paint at 16.2-16.4 s: startup restoration had settled and no listing was yet in flight, so nothing held it. The startup chat work now also holds the copy until a first tab listing has answered. A host no client lists (an orcad no client attaches to, a headless host) opens after a bounded 60 s wait from startup, so it still copies.
Contributor
Author
|
Closing this paused migration branch. Main removed the old per-chat import paths in #26038, and there are no existing native-chat users who need this backfill. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Paused: to be superseded by the copy-on-first-read follow-up to #24127; this PR's measured pieces carry into it.
This PR is step L′-c of: #23978 → #24233 → L′-c (this); the three land together. #23978 lists chat tabs at startup without opening every chat's history. #24233 stores each chat's status beside its history, so startup can show statuses at paint and settle crashed chats without opening them. This PR makes #24233's stored status exist for every chat on an upgraded profile.
ELI5
Older Orca kept each chat's history in its own small file. Newer Orca keeps all chats in one shared database, but it only moved a chat over when you used that chat. So on almost every upgraded profile, nearly every chat is still in its old file, and startup has to read those old files again on every launch before it can show their status. This PR moves the old files into the shared database quietly in the background, a few chats at a time, after the first launch. From the next launch on, every chat's status is there at paint.
What Changed
The problem. #23613 moved chat history into one shared database (
agent-session-journal.db), but copies a chat out of its old per-chat file (agent-session-journal/<workspace>/<session>/journal.db) only on that chat's first use. #24233 stores each chat's status beside its history, and startup seeds statuses from those stored rows. A chat still in its old file has no row, so #24233 cannot seed it: it is re-read from its old file after the tab list answers, on every launch, until someone uses it. On real upgraded profiles that is almost every chat (247 of 247 on the test seeds). With #23978 + #24233 alone, statuses on the Mac appear 4.6–6.6 s after paint, which is worse than main (statuses before paint). On the low-spec Windows machine, re-reading old files costs 13.4 s of CPU per launch inside the listing window.A second gap, from #24233's own migration. #24233's version 5 migration creates the status table empty. A chat already in the shared database then has no status row until it is opened, and startup selects the chats to settle only from rows. So after upgrading, a chat whose tab you closed and that a crash left mid-turn is never settled at startup until someone opens it.
What you see after this change.
orca worktree ps, mobile and the dashboard at paint, as main shows today. No old file is read at startup.The mechanism.
structured-agent-session-per-chat-file-copy.ts), started once per host from the startup step after the settle is kicked off. Desktop, orcad and a headless host all run it.-recovered-v3folder an older build wrote, is never opened or deleted. What is owed is derived from the files on disk and the stored rows; no progress cursor is stored.journal_background_failurestable, keyed to that step's input and the app version: a copy (the file won't open, or the copy didn't read back as the file) to the file's size and modification time and its WAL's; a missing row that could not be written to the chat's epoch and last sequence. The job skips it while all of that holds, and the next launch does not start the job for it; any change makes it owed again for one more try. A transient failure (disk full, busy, I/O) is not recorded: a copy gets at most 3 tries per launch, a missing row one.#24233's settle-on-open fallback is not removed here. L′-c makes it removable once the job has finished: by then every chat has a row, so startup selects every chat with work left. It stays Temporary in #24233 (its D5) until the open-without-row case is handled, because a chat opened before the job reaches it (during the first 10+ seconds, or listed late) still has no row and relies on its open to settle.
No change for: SSH and WSL (structured chats are local-only); folder workspaces (paths come from workspace and session ids); the wire between clients and hosts; a newer build's read-only database (no job, nothing copied).
Going back to an older Orca after this update. What an older build sees depends on what it can read. An Orca from before the shared chat database (v1.4.218 and earlier) reads only the old per-chat files, so once this version has copied your chats, which it does in the background shortly after the first launch, it finds none of your chats' history (what its chat tabs then show has not been checked). An Orca that has the shared database but not #24233's version 5 opens it read-only (see #24233). Your history is not lost: updating again brings it all back. But anything an older build writes to a chat's old file is set aside when you update again: the next launch's job finds that file, keeps it on disk, and never shows or merges it into the chat. Before this change, both effects applied only to chats you had used since updating. Now they apply to every chat.
Why
Why a background copy, not a copy before startup serves anything or on first read. The common pattern moves legacy data into its database before serving, or on the first read of that item. Here the source is up to ~1,300 external SQLite files. Before serving: Orca measured copying 200 chats adding 1.5 s to the first launch, and copying all 1,090 seed chats taking 16.9 s; those numbers came from an earlier single-transaction importer, and the batched importer here is slower in wall time, so they are a lower bound. On first read only, which is what #23613 does today and stays as the path for a chat used before the job reaches it: statuses fill 4.6–6.6 s after paint on the Mac and old files are re-read for 13.4 s of CPU per launch on the low-spec Windows machine. A set-based copy is one long statement per chat; #23613 measured its single-transaction copy blocking the main thread for 650 ms on the largest chat and moved to 512-row batches because of it.
Why this shape. The job takes the common shape for background maintenance sweeps: no client demand, a fixed interval, a bound on its work checked at every step, a yield between steps, a run skipped while one is still going, and failures logged and never thrown. Its bound is a share of the main thread rather than a per-run budget (D1). Its timing, after startup rather than before serving or on first read, is D0 below.
Why every chat, not only listed ones. The startup cost this PR removes comes from listed chats, and copying only those would drop the folder walk and its leftover handling. But #24233's startup settle selects listed and unlisted chats alike from their rows, so an unlisted chat a crash cut needs a row too, and moving every chat into the one database is what later lets #23613's preview and first-use copy path be retired.
Alternatives considered and rejected. A worker thread with its own connection to the shared database: the shared database is one connection by design, and a second writer would make main-thread appends wait on SQLite's lock, which sleeps the main thread. A worker that only reads and hashes the old file (its own SQLite file, no shared lock): its rows would still cross to the main thread to be inserted, and the verify's fold must read the shared database on the main thread, so the main-thread saving would be small; not measured. Iterating every chat record instead of walking the old-file folder: one file-existence check per record (~1,300) on every launch, forever; the walk costs nothing once the folder is gone.
Differences from the common pattern
cd50d5cea0b; 347–378 s atde4bc68c520, the same pace). The job's first deletion comes at 43.5–48.6 s from launch, after the tab strip paints at 38.5–44.2 s; the 12–13 old files that go before it are startup's own opens. A session shorter than that finishes the copy on a later launch: 2 of 3 measured launches reached the rig's 420 s cap with 37 and 3 old files left, and the next launch finished them. Until the copy finishes, those launches still read the remaining old files at startup, as today. On a busy machine the streaming gate stretches it further, since the copy takes no time while a chat streams or a send is in flight.SQLITE_FULLhas been observed.Linked Issue
Part of the native chat startup work in #23978 and #24233.
Visual Proof
Pending. This changes when statuses appear on the second launch after upgrading; the before/after capture comes from the QA lane on a background
$electronrig with an upgraded seed profile. No app was launched for this PR.Testing
tc:nodeclean atf6248fd585c(merged with feat(native-chat): store each chat's status with its history; startup seeds and settles from it #24233 at0786214b740, which includes main through5735ee94e19); the head after it,de4bc68c520, changes one test import.journal-per-chat-file-copy.test.ts,structured-agent-session-per-chat-file-copy.test.ts,structured-agent-session-per-chat-file-copy-host.test.ts(57 tests), plus updates injournal-per-session-import.test.ts,journal-database.test.tsandstructured-agent-session-host-teardown.test.ts.f6248fd585c: 116 files, 1524 tests (this PR's suites, every suite feat(native-chat): store each chat's status with its history; startup seeds and settles from it #24233's last two pushes touched, and every suite of the journal store and its write queue); at5c95a301dfc, 158 files and 1697 tests, including every suite of the files main and this PR both changed (journal database and migrations, status state, import, restore, startup state, reveal, restorer, host teardown, runtime startup, tab restore, launch activation and legacy record import).structured-agent-session-startup-chat-work.test.tspins each runtime signal (restoration not yet settled, resolved or rejected; a listing; an owed and a starting history restore) and that a desktop launch's copy waits for restoration, andstartup-state.test.tspinsisSettling; 7 ablations, one per signal and its wiring, each red.structured-agent-session-per-chat-file-copy-pace.test.tspins that any one-second window (aligned or not, after a long idle) holds at most the share, the burst and the one chat's work that crossed it, that a stall counts, and that quit ends a wait. The job tests pin that a run goes on until the end at the pace's rate, that idle time between runs is not charged, and that every copy and missing row runs inside the chat with that chat's own yield (the missing-row step forwards it). A host test sends to a chat mid-copy with a pace that would wait forever: the send goes through, because the pace never waits inside a chat. The importer test pins a yield between the copy, both verify sides and the publish for a one-batch chat. The pace's tests pin that it never waits inside a chat, pays that chat's debt between chats, and charges nothing while it waits for a chat's lock. 20 ablations, each red. The size bound:journal-per-chat-file-copy.test.tspins that a batch splits so no part exceeds its share of row JSON (a larger row alone), that both verify sides and the missing-row fold go a part per task, and that a copy commit keeps a whole page and splits only past its own larger share; 6 ablations, each red.startup-state.test.tspins that the copy's wait for the settle ends when the settle goes on past a recovery that never answers.structured-agent-session-per-chat-file-copy-production.test.tsrestores a listed chat from its old file and starts the copy through the host (reveal, the control, the job) on its real interval. It pins that the restored chat's owed import is charged to the pace (the next chat waits out its debt) but takes no yield of the copy's, so a reader waiting on it through the chat's write queue never meets a pace wait, and, with a pace that owes a wait at every yield, that every wait falls between chats, none inside one; 7 ablations for this shape, each red, including the owed import given the copy's yield again.structured-agent-session-per-chat-file-copy-activity.test.ts, on a real host and its chat activity, pins that no chat's copy starts while a chat streams frames, running or parked on an approval, or while a send is in flight with no frame at all, a send that starts its chat's agent included from accept through hand-over, while a send queued behind a running agent's turn holds nothing; that a chat parked on an approval, or running a turn that has gone silent, lets every chat be copied once no frame has come for the quiet period, and not 1 ms before; that the quiet period runs from the last frame, which no tick need see; that a frame mid-copy, a new turn or a delta that writes no row, stops that chat before its next batch, with nothing published, the chat still owed, nothing recorded and no settle, and that the same chat is copied first once quiet, its staged rows cleared; that a missing status row stops the same way and is written once quiet; and that a chat whose copy starts after a send went in flight starts stopped.claude-structured-provider-frame-activity.test.tsandcodex-structured-provider-frame-activity.test.tspin, with a sink that records every call made on it, that a text delta between checkpoints (both providers) and a Claude thinking delta reach the host's sink only as the noted frame: without the note the sink gets no call at all, which is why the signal is taken where each provider hands a frame to the host and not inside the sink; a Codex approval request is noted at receipt, before anything it writes. The event sink test pins that rows buffered before a bind are noted when the bind writes them. The production test pins the same through the host's own wiring (reveal, the control, the job on its real interval), including a restored chat's owed import stopped mid-way and still owed. It also pins that a stop leaves nothing half-copied: a frame stops a copy at each of its task boundaries in turn (between copy batches, between verify batches, before the publish, and after it), and once quiet the shared database holds exactly the file's rows (epoch, seq, ts, JSON) under one epoch, with the status row the chat had; a stopped missing-row fold later writes exactly the row a whole fold writes. And with a chat streaming frames for the whole test, the copy starts no chat in 120 ticks, while a restored listed chat and a closed unlisted one both show their whole history. Each ablation is red, including a retry that skips the leftover delete, a stop that publishes its staged rows, a gate that holds on a running turn, and either adapter not noting its frames.tc:nodeclean at27432f846fe(merged with feat(native-chat): store each chat's status with its history; startup seeds and settles from it #24233 at161001fdef6); related suites pass there, 98 files and 993 tests (the runtime's tab-listing and restore suites included, the page sized by its tasks' time included, feat(native-chat): store each chat's status with its history; startup seeds and settles from it #24233's startup status-derive suite included, the copy waiting for feat(native-chat): store each chat's status with its history; startup seeds and settles from it #24233's startup status pass included, the corrupt-history copy and missing-row tests included, feat(native-chat): store each chat's status with its history; startup seeds and settles from it #24233's startup status pass, status derive, fill order, saved-session restoration and tab-listing startup suites included, and this PR's copy, activity, host, pace, production and job suites, reveal, the status feed's five, startup state, status derive, the status bridge burst, the main-agent working agreement, restore without import, the importer, the missing-row backfill, the journal store, the event sink, subscribers, attach reconciliation, conversation stop, append delivery, accept-then-deliver, the Codex adapter suites, and every mocked Claude structured-session suite, 52 files); 29 ablations across the final G3 gate and the corrupt-history rule, each red. The real-Claude-CLI test was not run for the one-line change to the Claude adapter: it drives the real Claude login and writes into the real~/.claude, which this work must not touch; the mocked Claude adapter suites ran instead, and a new one pins that the adapter notes a thinking delta.cd50d5cea0b(quiet, paired runs; table below): main-thread share p50 0.10, max 0.13–0.18; the 70–82 ms longest task is the rig's mid-copy send starting a Claude session, not copy work. Earlier runs, for the record: at661dbfeaf2fthe gate measured a 13–14 s copy at main-thread share p50 0.42–0.59: the pace was not engaged for the listed chats the startup restore had opened (247 of 446 on the file seed), whose copies ran as their own owed import, unpaced. Their profiles show about 7 s at ~90% with no pace frames, then the unlisted chats at 15%. Fixed atc92ee641f82, which also moved every pace wait to between chats (a restored chat is read through its own write queue, not the host's lock); Local, 120 seed chats, load 9–12: the copy took 17–20 s, share p50 0.17, max 0.25–0.26, per-task CPU max 16–26 ms; a chat opened mid-copy waited 19–40 ms. The joint gate's first runs (before the pace) failed: longest block 215 / 56 / 108 ms and share max 0.58 / 0.50 / 0.71. Their main-thread profiles put main-thread share per second at p50 0.20, max 0.28–0.38 (the rig's share counts every thread in the process). Of the five blocks over 50 ms, two (54 and 56 ms) were splittable work, a copy batch run together with the verify in one task, and are now split; the rest were a stall inside a single commit (a WAL checkpoint, with no profiler samples for 203 ms of the 215 ms block) or GC. Local runs, 120 seed chats (load 7–23): the copy at 15% took about 17–19 s against 32–35 s with the earlier pace inside the per-run budget (about 2 s unpaced); share p50 0.18–0.20, max 0.23–0.27 (earlier pace 0.15 / 0.30–0.33); per-task CPU max 12–14 ms; 437 commits and 47 checkpoints (splitting the copy's commits by size, tried and reverted, gave 1053 and 57); a chat opened mid-copy waited 16–19 ms (unpaced: 15–16 ms; the earlier pace: 202–217 ms). Single stalls inside one commit can still exceed 50 ms; see the follow-up above.Measurement gate (results at
cd50d5cea0b). Arms: main at the merge base, #23978, #24233, this PR; at least 3 interleaved rounds, median of paired differences. Every launch on its own dev profile, isolated HOME and CODEX_HOME, launched in the background.cd50d5cea0b): share p50 0.16 / 0.16, max 0.49 / 0.96; longest task 438 / 462 ms, one 512-row page's commit (204 / 363 ms) with the next page's read in the same task. Re-measure on the time-sized page pendingThe low-spec Windows machine is mandatory: its 13.4 s per-launch cost is why this PR exists; its rows above are measured on it.
The joint first-boot gate (all rows from launch ≤ main + about 0.8 s, #24233's area) passes on the Mac (+0.18 s host seed, +0.17 s file seed, medians) and fails on Windows high spec (+1.23 s host, +0.90 s file) and low spec; #24233 is attributing those, see #24233.
Notes
fsandstatfs; nothing is platform-specific.journal_background_failurestable is created at every writable open with no schema version bump. The downgrade cost is stated above.Checklist
pnpm lint,pnpm typecheck,pnpm test, andpnpm buildpass (tc:node clean; CI will cover the rest)