feat(history): file-backed conversation history streamed to providers (Fixes #854)

[acoliver]

TLDR

Conversation history now lives in an on-disk journal instead of in memory. HistoryService is a facade over that journal, and subagents get their own journal files. Long sessions no longer keep a context-sized copy of history in the heap between turns.

OpenAI chat, OpenAI Responses (both the openai provider route and the openai-responses provider) and Anthropic build each request body from journal rows, one row at a time, into a request-scoped spool on disk. The spool is uploaded with Content-Length equal to its sealed size. No whole-history array or whole request object is built on the send, estimate, dump or retry paths for these providers. Vercel and the Gemini plugin still build an array per send; they are declared exceptions (#3869).

Request bytes and token counts match main for the same history.

Part of #854. The remaining #854 work is tracked under #3868.

Dive Deeper

What changed

• History at rest. Membership, pending tickets, the provider normalization snapshot, density folds and the resume resolver are disk-backed. In-memory state between turns is scalars plus state bounded by the number of mutation events.
• Sends. The array send route and its enforcement are deleted. Each provider prepares from requestRows. Stored media references are resolved one row at a time and released at call cleanup.
  • OpenAI chat uses a streaming tool-id mapper and a row-at-a-time body builder. The continuation request after tool calls is a second spool.
  • Responses writes the disk text route for every call. Calls that arrive without rows are staged on scratch disk first. The old progressive fill() body and LazyBoundedJsonBody are deleted (Responses progressive input materializes the whole history when a tool row is present #3858).
  • Anthropic fills a disk-backed message table row by row, applies the finishing passes and cache markers on the table, and emits it into the spool. Image-dimension recovery rebuilds the spool under a smaller image budget instead of keeping a body copy between attempts.
  • Estimates and request dumps for all three providers read the same segment files as the send.
• Durability. A general history replacement (restore, import-driven replacement, compensation, profile adoption) is a replace_begin / rows / replace_commit transaction. Every journal reader, including the CLI scrollback pager, treats an incomplete transaction as absent, so a crash during a replacement resumes with the previous history. Single-record shapes (a suffix cut, a clear, a compression) are one record.
• Shared-history recorders. Journal records carry their origin. Rewinds and inserts are translated into each origin's mirror, and a resumed recorder keeps its origin filter (Shared-history recorder fan-out: unowned rewinds reach every mirror and can over-remove; resumed recorder becomes unfiltered primary #3865).
• Logging. Conversation logging never fails a completed turn. A write is abandoned only when it stops making progress, not after a fixed 5 s (Request-artifact telemetry: fixed 5 s export acknowledgement fails slow-but-healthy writes and leaves abandoned writes running #3866).
• Disposal. Journal disposal is awaited and recorder or scratch cleanup failures are raised (HistoryJournalStore.dispose() is fire-and-forget: recorder dispose errors are swallowed and scratch is removed asynchronously #3867). Scratch roots left by dead processes are swept when a process creates its first root.

Fixes to regressions found in review

• Concurrent o200k counts shared one global regex and miscounted. Each counter now has its own.
• A replace transaction could interleave with another record. It is now guarded. Subscribing a recorder now waits for the previous detach.
• A resume that failed after publishing left no recording. The previous recording is restored.
• A repeated resume re-recorded unchanged rows. It no longer does.
• A resumed transcript replaced rows that arrived while it loaded. It is now prepended.
• A failing BeforeModel hook aborted the request. It is logged and the request goes out unmodified, as on main.
• Body logging blocked requests when the exporter could not carry request artifacts. It is skipped for those exporters.
• /restore left project files changed when the history commit failed. They are rolled back.
• Last-resort tool-response truncation re-projected the history after every replacement. It now decides all replacements in one pass: 722 s to 84 ms at 1,024 rows (Last-resort tool-response truncation is O(candidates x rows) on the journal-backed send path #3857).
• getStatistics, getContextRange and the last-content reads rescanned the journal from byte 0. They now read incrementally: about 1 s and 43 MB per call to 0.04 ms and 4 KB at 8,192 rows.
• Attaching a recorder to a journal with a semantic purge parsed the purge record once per row. It is parsed once.

Defects found while getting CI green

• Mirror durability. A history mutation resolved before the per-origin recorder mirrors had written what they were handed, so restoreTurns could return while the rewind was still in memory. Mutations now settle the fan-out after their publication scope closes (674e522). The new durability test fails 3 of 3 without the fix.
• Per-turn cost that grew with the journal. Every mutation re-folded the whole journal from byte 0 to capture row origins for rollback. CPU per turn in the R3 workload rose from 112 ms to 257 ms over 500 turns. The origins now come from the fold the mutation already runs, which brings it back to about 105 ms, against 95 ms before origin capture existed (a0ba9c5).
• Media lock cycles per row. Resolving stored media on the streamed send path, and admitting deferred history, took one store reservation and release cycle per row. A request now reserves each content id once, so lock work scales with distinct images (7061b52).
• Request staging. A call aborted before it started kept its prepared OpenAI projection (212de1e). The provider normalization stage log kept its 64 KiB append buffer after it switched to reading (9a52af9).
• Scratch cleanup. One CI run failed with EFAULT from Bun's recursive rm of a resolver scratch directory. The cause was not reproduced. Resolver directories, which hold exactly one file, are now removed with unlink and rmdir, so a recurrence reports the real errno (3353ad8). Other scratch owners still use recursive rm.

Memory proof

packages/providers/src/__tests__/sendPeak.*.test.ts sends a 1 MiB and an 8 MiB body (1.4 and 11.6 MiB tool-heavy) on every route. It requires that:

• the bytes equal a pre-change oracle;
• Content-Length equals the bytes sent;
• heap growth and arrayBuffer growth between the two sizes stay under 1 MiB;
• the Theil-Sen trend of retained heap over repeated sends stays under 1 MiB.

Trap variants hold the whole history (array, estimate, fill) or keep it alive across sends (retain), and must break the bound. In the final run every route was inside the bound, with peak growth between -0.92 and +0.50 MB. Every trap broke its bound, by 12.5x to 76.6x.

packages/agents/src/core/__tests__/journal-retained-growth*.test.ts runs a real ChatSession with compression over about 850 requests. It requires settled heap to stay under 1 MiB at six checkpoints, with fitted growth over the late checkpoints under 262,144 B. A retaining trap and a release trap must fail. Locally the normal run takes 67 to 80 s and the traps 81 and 114 s.

Several WeakRef checks require that no released row survives. JavaScriptCore scans the stack conservatively, so a stale slot can keep one dead row alive for a collection or two, and those checks failed now and then on Linux CI with a count of 1. They now count a row only if it survives several collections, each after an event-loop turn (retainedAfterSettling, and a two-consecutive-checks rule in the fan-out pump bound). The bounds stay at 0, and each converted test has a trap that holds a row and must report it.

Speed

Warm send under the context limit, real runtime wiring, local HTTP provider, median of 5, measured at 0d1051f. The numbers are CPU ms, branch against main, because the machine was heavily loaded and CPU time is the fairer comparison:

Provider	1,000 rows	4,000 rows	8,192 rows
OpenAI chat	323 / 180 (1.8x)	1,406 / 922 (1.5x)	3,924 / 3,192 (1.2x)
Responses	475 / 228 (2.1x)	1,763 / 1,042 (1.7x)	4,161 / 3,865 (1.1x)
Anthropic	469 / 222 (2.1x)	1,831 / 1,001 (1.8x)	5,175 / 3,215 (1.6x)

At 1,000 rows the branch still adds about 150 to 250 ms per send, because each send normalizes every surviving row. #3849 tracks making normalization incremental across sends. Prompt estimation at 167k tokens takes about 250 ms (main 154 ms) with identical counts.

Behaviour changes reviewers should know about

• OpenAI chat, Responses and Anthropic uploads send Content-Length again. They briefly went out chunked earlier on this branch.
• An abort during a send surfaces the user's AbortError instead of the provider's "terminated".
• /set retries 0 on Responses providers sent nothing and returned an empty stream, on main as well. It now makes one attempt. A 200 response with no body is an error.
• Model hooks (BeforeModel/AfterModel) run on the disk snapshot path and emit hook_call telemetry from there, with stdout/stderr capped at 4000 characters. MissingSnapshotHookCallbackError is raised for HookSystem subclass overrides, which are not a supported extension point.
• With request logging in strict pre-send mode, a send waits for the request artifact to be acknowledged. Main logged fail-open.
• Density changes are recorded as density_mutation records of at most 64 rows or 64 KiB each.
• SessionRecordingService uses backpressure instead of throwing at the queue limit.
• The per-turn system prompt is read at send time. The race it fixes also exists on main.
• The file-driven scrollback pager (ui.scrollbackPagerEnabled) ships defaulted off. With it off, the transcript uses the fallback ledger, which keeps main's item and byte limits. Unified scrollback to the start of history: one disk-paged surface with the live turn, on by default #3856 tracks the unified scrollback.
• A removed provider-normalization scratch directory is detected at the next batch read instead of the next row, with a clearer error. Rows already read from disk are served normally.
• CI partitions changed: agents run in 3 shards and scripts in 4, balanced by measured duration.

Decisions for the owner

1. PR link. The body says "Part of Cache and load conversation history and highlight what is in context #854", because the unified scrollback (Unified scrollback to the start of history: one disk-paged surface with the live turn, on by default #3856) and the other Journal-backed history follow-ups (#854): remaining work after #3852 #3868 items are deferred. Change it to "Fixes Cache and load conversation history and highlight what is in context #854" if you want Cache and load conversation history and highlight what is in context #854 closed on merge.
2. Strict pre-send logging. On main, request logging is fail-open. Here, strict mode waits for the artifact acknowledgement before sending. Keep strict, or make it fail-open.
3. retries. Main treats retries=N as N total attempts, not N retries, and the branch keeps that, clamping 0 to one attempt. Keep it, or change it to mean "N retries after the first attempt".
4. Codex WebSocket. The Codex WebSocket path still assembles each request as one full frame in memory. The HTTP routes do not.
5. Post-reply logging failures go only to the debug log. There is no user-visible warning channel for them. FileLogExporter is synchronous, so the progress-based liveness check does not cover it.
6. /restore side effects. Each restore makes one shadow commit, and ignored files are not rolled back.
7. Test changes to look at.
   • In LoadBalancingProvider.timeout.test.ts, the "linked listener removal throws" fault now fires only after the first attempt starts, because request staging also removes listeners before any attempt.
   • The p05b4 retry capture count went from 3 to 2, because main's auth policy allows one refreshed retry.
   • Files named *test-helpers* under src still import bun:test. A new guard blocks any other production file from doing so.

Review

One full independent review of this branch found 1 blocker and 2 major defects. Each was reproduced, fixed and covered by a test that failed first. A follow-up check confirmed the hook fix and found the other two fixes incomplete on sibling paths. Those paths are also fixed:

• Unfinished replace transaction on resume (blocker). The resume path left an unfinished replace transaction open in the journal file, so the next replacement made the journal unreadable. Resume now writes the replace_abort first (1761dca). Clone and checkpoint fork, which start a recorder on a copied journal, do the same (605ff99).
• Project-trust withdrawal. Project BeforeModel/AfterModel hooks on the disk route kept running after workspace trust was withdrawn. They now get the project-trust signal, as the command route does (d690f2f).
• Row origins after rollback. Rolling back a failed /clear, pop, batch clear, row transform, validation repair or rollback checkpoint restored rows without their origin. A recorder for another origin attached later then received them. Every rollback now restores each row under its own origin (838ab2e, ce0686c).

Follow-ups (under #3868)

• Done here: OpenAI/Responses requests now upload chunked without Content-Length (on-demand body from #854) #3847, Last-resort tool-response truncation is O(candidates x rows) on the journal-backed send path #3857, Responses progressive input materializes the whole history when a tool row is present #3858, Shared-history recorder fan-out: unowned rewinds reach every mirror and can over-remove; resumed recorder becomes unfiltered primary #3865, Request-artifact telemetry: fixed 5 s export acknowledgement fails slow-but-healthy writes and leaves abandoned writes running #3866 and HistoryJournalStore.dispose() is fire-and-forget: recorder dispose errors are swallowed and scratch is removed asynchronously #3867.
• Still open:
  • Incremental provider normalization across sends (journal-backed history send cost grows with row count) #3849: incremental normalization across sends.
  • Remove array-route leftovers and #854 heap harnesses after the journal-fed send route #3848: dead array helpers and old heap harnesses.
  • Unified scrollback to the start of history: one disk-paged surface with the live turn, on by default #3856: unified scrollback.
  • Gemini plugin and Vercel providers build the whole history as an array on every send #3869: Gemini and Vercel arrays.
  • Record what a history publication means instead of inferring it from row shapes #3870: explicit publication intent.
  • Test coverage follow-ups from the #3852 review (memory proof sizes, deleted and skipped tests, independent expectations) #3871: test-coverage follow-ups.
  • Providers depend on history internals: move request-row and dump-source contracts into runtime/contracts #3872: provider dependency on history internals.
  • Show what is still in context in the transcript, and expand the summary under compressed history #3873: in-context marks in the transcript.

Reviewer Test Plan

• npm run build:types, npm run lint, npm run typecheck, npm run build, npm run test, npm run test:scripts, npm run check:runtime-boundary.
• Gemini plugin: cd plugins/google-gemini && npx tsc --noEmit && bun test.
• Bytes: cd packages/providers && bun test src/__tests__/wireBodyGolden.test.ts. That is 315 cases over 7 routes and 22 histories, recorded before this work.
• Memory: from packages/providers, run each src/__tests__/sendPeak.*.test.ts file on its own. From packages/agents, run each src/core/__tests__/journal-retained-growth*.test.ts file. Trap files pass by observing their trap break the bound.
• Smoke: bun scripts/start.ts --profile-load lunahigh "write me a haiku and nothing else", plus a tool round trip.
• Manual: run a long session with tool calls, then /chat restore to an earlier point. Quit, --continue, and confirm the history and the next request are right.

Testing Matrix

	🍏	🪟	🐧
npm run	✅	❓	✅
npx	❓	❓	❓
Docker	❓	❓	✅
Podman	❓	-	-
Seatbelt	❓	-	-

Linux and Docker are covered by CI (unit shards, E2E with sandbox none and docker, tmux UI). Windows is covered only by the test-infra gate.

Gates at c5d349e. Every CI check passes. Locally: build:types, lint, format (no diff), typecheck, lint:affected-shards, lint:agents-api-surface and check:runtime-boundary pass, and the branch is 0 commits behind main. The haiku and tool round-trip smoke tests passed on the merged tree.

On earlier runs of this branch the Windows test-infra gate failed twice. Both failures were in launcher and runner meta-tests this branch does not touch: a child that sleeps 50 ms ran past its 4 s budget, and an EBUSY came from removing a temp directory. It passed on the next commit.

Linked issues / bugs

Part of #854

Fixes #3847
Fixes #3857
Fixes #3858
Fixes #3865
Fixes #3866
Fixes #3867