refactor(native-chat): read the chat queue by row headers instead of loading every message

[brennanb2025]

	Files	Added	Deleted	Net
Test	10	$\color{#1a7f37}{\Huge{\mathbf{+}}}$​1319	$\color{#cf222e}{\Huge{\mathbf{−}}}$​2	$\color{#1a7f37}{\Huge{\mathbf{+}}}$​1317
Prod	16	$\color{#1a7f37}{\Huge{\mathbf{+}}}$​465	$\color{#cf222e}{\Huge{\mathbf{−}}}$​199	$\color{#1a7f37}{\Huge{\mathbf{+}}}$​266

ELI5

When an agent is busy, messages sent to its chat wait in a queue. Orca keeps each waiting message as its own saved row, but every time it asked a simple question about the queue ("what goes next?", "is anything paused?", "can old receipts be cleaned up?") it loaded every row ever saved for that chat, including the full text of every message and every already-delivered one, and kept all of it in memory. This change makes those questions read only the few small fields they need, from indexes, and stop as soon as they have an answer.

Nothing looks or behaves differently. This is the first step of making the queue a separate, paged read, so that later steps can stop refusing messages because "too much text is waiting".

What Changed

Problem. The host's queue repository had one read, listQueuedMessages, that selected every row of every state (waiting, returned, handed off, and withdrawn receipts) and parsed every message body. The journal cached that whole array per chat. Everything went through it: picking the next message to send (asked on every streamed frame of a running turn), working out whether a Stop, /clear or reopen pauses the queue, the in-transaction recheck before a message is handed to the agent, the open-time repair and cleanup of old receipts, placing kept messages at the head of the queue, and the admission check that sums the size of all waiting text. So memory and per-step work grew with the whole history of the chat's queue, not with what was actually waiting, and the bodies stayed resident on every open chat.

User-facing change. None intended. Same queue order, same pauses and holds, same "Too much text is waiting" limit and wording (it is removed in a later step), same cards on desktop and phone, same /clear behavior.

Mechanism.

• queued-message-headers.ts adds body-free reads over the existing table: an ordered header iterator (waiting / unsettled / all), point lookups by id and by hand-off link, existence checks, a keyset walk over expired handed-off rows for retention, and the admission byte sum computed in SQLite (length(CAST(body_json AS BLOB)), which equals the UTF-8 length of the JSON.stringify(body) the insert stores).
• Three indexes created at open with IF NOT EXISTS, like the table itself (no schema-version bump, so an older build still opens the database): (session_id, position), a partial (session_id, position) over readable waiting/returned rows, and (session_id, state, settled_at, message_id). They only make reads faster (every query carries its full condition), so they are built best-effort after the required schema: if a build fails, for example because one chat's row is already damaged on disk or the disk is full, Orca logs it once and the journal still opens; the next open tries again. Without this, one damaged row would have made every chat on that host refuse to open.
• Every gate and bookkeeping path moved to headers: the drain's pick, pause derivation, reopen detection, the consume recheck, publication's Resume check, owed settlement (now probes only the hand-off links of refused submissions), echo settlement (fingerprints only), retention, and head-of-queue placement. The pause rules in queued-message-pause.ts stay the single source of truth; they now accept an iterable and stop at their first decision.
• Body reads remain only where text is genuinely needed: the one card being handed to the agent, the explicit per-card actions, operation replay receipts (unchanged), /clear's check for queued commands to withdraw, and today's full-list publication to phones and paired computers. That publication now selects only waiting/returned rows in SQL and still recomputes only when the queue changes; the per-chat cache of every row is gone.
• A row whose stored body cannot be read stays invisible to every decision, as before. One definition of a readable row (stored as text, no NUL character, valid JSON, known state; queued-message-readability.ts) is used by every header query and is part of the partial index's condition, so SQLite checks it once when a row is written instead of re-parsing every body on every read. A row the drain can pick is therefore always a row it can load; a corrupt row can't stall the queue.
• Nothing reads the queue on a streamed frame while the agent is working: the next-card pick checks whether the agent is busy before touching the queue, the Resume check returns at once when nothing is paused, and owed settlement only looks at sends that actually took a queued card. Measured: 0 SQL statements per frame while streaming, for any queue size (before this round of review it was 3 statements and about 6 ms per frame with 2,000 small cards).

Why

The root cause of the queue limits is that the host treats its queue as one value that has to be loaded, and later published, whole. The rows are already separate, so the fix is to read them as separate rows instead of adding a second store. This PR does that for every host-side decision. The next PRs give phones and paired computers a separate, paged queue read (so the queue stops riding every chat frame), switch desktop and phone to it, and then delete the aggregate size refusal.

Alternatives considered: keeping the whole-list read and just paging the published list afterwards (still loads every body for every decision), or a second queue table (duplicates the ownership and hand-off rules that already work).

Differences from the common pattern (the common pattern stores queued messages as separate rows and serves them as their own read, invalidated on change):

• INTENDED: waiting drafts stay Orca rows until the existing transactional hand-off, rather than becoming execution entities or being claimed-and-deleted. The hand-off and replay identities already handle delivery; replacing them is unrelated to this defect.
• TEMPORARY: the aggregate "too much text is waiting" refusal is still enforced (now as one SQL sum). It is removed once every reader uses the paged read, in the last PR of this sequence.
• TEMPORARY: phones and paired computers still receive the full waiting list in chat frames. The next PR adds the separate paged read.

Linked Issue

N/A (maintainer). Follows the merged count fix #26553.

Visual Proof

No visual change is intended; these are a smoke test of an earlier head of this PR (7fe88d8, with main's queued /compact merged in; before the last main sync that brought in /clear staying in the same conversation) in a hidden test build on Windows, using a stand-in agent and the queued-messages capability turned on for the test build only.

Three messages queued behind a running turn, in order:

After the turn ends they are sent one at a time, in order, each as its own turn:

Stop with two cards waiting: the cards stay, "Queue paused because you interrupted" with Resume; nothing was sent for about 85 seconds:

After Resume both were sent in order:

A typed card ahead of a /compact card; the typed message ran first, then the compaction:

Testing

• Targeted vitest after the final main sync (which brought in /clear staying in the same conversation, Keep native chat /clear in the same conversation #26579): 59 queue/journal/wire/clear test files, 580 tests pass, including main's /compact and /clear queue suites. The /clear pause now holds exactly the message ids the clear recorded; the header walk keeps that rule and stops once every recorded card is found. New tests: queued-message-headers.test.ts, journal-queued-message-reads.test.ts (header reads never select bodies; next-card pick stops early; retention and owed settlement with thousands of withdrawn rows touch only the rows they act on; SQL byte sum equals the JS measurement for multibyte/emoji/escaped text; FIFO and pauses identical on a mixed queue; a rolled-back consume leaves no stale read), queued-message-read-plans.test.ts (each query uses its intended index), queued-message-readability.test.ts and structured-agent-session-queued-read-regressions.test.ts (a corrupt NUL/BLOB card at the head is never picked and the next card is sent; no queue read while the agent works), journal-queued-message-index-failure.test.ts (a failed index build leaves the journal openable and reads working), queued-command-owed-settlement.test.ts (a refused queued /compact recovered after a restart is spent, not returned).

• Reviewers also compared old and new logic on 300,000 random queue/pause cases and 9,000 bookkeeping cases with zero differences.

• tc:node typecheck clean; orca-ci-checks (lint, code quality, localization, typecheck) clean.

• Platforms: logic is SQLite and shared TypeScript only; no platform, path or process code changed. SSH/paired hosts run the same host code on the same SQLite version.

• I manually tested these changes locally (UI smoke was agent-driven on a hidden Windows test build; see Visual Proof)

• Automated tests added/updated, or explained why not below

Agent skill upstream boundary

• Not applicable, or this change copies or mechanically translates no upstream skill-installer source, tests, fixtures, registry entries, path tables, comments, or documentation.

Notes

Security: no new inputs. Cross-platform: SQLite only. Remote SSH: the execution host owns the queue as before. Mobile: unchanged frames. Backwards compatibility: indexes are additive and created with IF NOT EXISTS; an older build ignores them. Performance: removes whole-table body hydration from the drain and bookkeeping paths.

Author: @BrennanKB5

[brennanb2025]

Review summary

Problem. The host's chat queue answered every question ("what goes next?", "is the queue paused?", "which old receipts can be cleaned up?") by loading every queued row it had ever saved for that chat, full message text included, and keeping that whole list in memory. The work and memory grew with the chat's whole queue history, not with what was actually waiting, and this whole-list read is what forces today's "too much text is waiting" limit.

User-facing change. None intended. Queue order, pauses, holds, Resume, /compact and /clear behave as before, and so does the existing size limit (later PRs in this sequence remove it). Phones and paired computers get the same frames.

Review. Three review rounds plus a readiness pass, by reviewers who did not write the code. Round 1 also had an independent design challenger, which kept the design (read the existing rows by their small header fields; no second store) and found no reason to split the PR. Comparing old and new logic on 300,000 random queue/pause cases and 9,000 bookkeeping cases found zero differences.

Fixed during review:

• Streaming frames ran SQL over the queue (about 6 ms per frame with 2,000 cards). Now the cheap "is the agent busy / is anything paused" checks come first, so streaming does 0 queue SQL.
• Owed-settlement bookkeeping ran on every commit for every refused send. It now probes only sends that actually took a queued card.
• Every header read re-validated message JSON. Now one "readable row" rule (stored as text, no NUL, valid JSON) lives in the partial index's condition, so it is checked once per write.
• A corrupt card at the head of the queue could be picked but not loaded, which stalled the queue. Now every reader uses the same "readable row" rule.
• A tombstone-heavy ordered walk, a test that could never fail, and a shared-iterator hazard (documented).
• A failed index build (for example from one damaged row on disk) would have made every chat on the host refuse to open. Index builds are now best-effort and logged.
• CI: a new real-SQLite test was missing from the Node-runtime test list.

Deferred (nits or later PRs in this sequence):

• Renaming the narrowed full-body list().
• A dispatched row with a corrupt body is never pruned. This already happens on main.
• Orphan indexes in dev databases that ran unshipped heads.
• A withdraw row count is ignored. It only matters for unreachable rows.
• spendRefusedCommandCard throws on a malformed body. This already happens on main.
• Paged client reads need a per-card command flag. That belongs to the next PR.
• /clear (from main) still loads full bodies of every waiting card to find queued commands, which is unbounded once the size limit is gone. The next PR should give it a header-level command flag.

Verified.

• After the final main sync (6f0afa5, which brings in Keep native chat /clear in the same conversation #26579 /clear staying in the same conversation): 59 targeted test files and 580 tests pass, and orca-ci-checks passes 23/23 including typecheck. CI is green (15 pass, the rest skipped by path filters).
• Query-plan tests pin each query to its intended index.
• A hidden Windows test build with a stand-in agent showed queue order, Stop then Resume, and a typed card ahead of a queued /compact (screenshots in the description). That run was on the earlier head 7fe88d8.

Not verified.

• No live run after the final main sync. The /clear pause merge is covered by unit tests only.
• No real Claude/Codex agent, no phone, and no SSH host in the smoke run.
• No measurement on a real user database.