fix(server): bound inbox ACK cost to the partition's non-acked rows

[bestony]

Fixes the quadratic ACK cost described in #1671 (PERF-008).

The problem

ackThroughEntryIdForBoundAgents selected — and, under FOR UPDATE, locked — every
notify=true row in the (inbox_id, chat_id) partition below the ACK cursor, including rows
acked long ago. Those rows were then materialised as JS objects. Cost per ACK was O(chat
history) and O(N²) over a chat's lifetime, and a duplicate ACK that commits nothing paid the
full price.

Measured end-to-end through the real service call (not just the SQL), on a migrated database
with the production index set:

chat history	one duplicate ACK (commits nothing)
50,000	119.6 ms
150,000	341.4 ms
300,000	545.0 ms

Roughly 2 µs per row of history, strictly linear.

Root cause

Not a missing index. idx_inbox_chat_silent (inbox_id, chat_id, notify, status) already
exists and the planner was already using it — the query just never constrained status, so a
four-column index was used as a three-column one. Every row of the partition came back and
id <= cursor was re-checked as a heap filter:

Sort (actual rows=168001)  Sort Method: quicksort  Memory: 21895kB
  -> Index Scan using idx_inbox_chat_silent on inbox_entries (actual rows=168001)
       Index Cond: ((inbox_id = $1) AND (chat_id = $2) AND (notify = $3))
       Filter: (id <= $4)

The locking cost was larger than the scan: with FOR UPDATE, that same statement touched
338,980 buffers and dirtied 2,800 pages, purely to take row locks on rows that were already in
their terminal state.

The change

One clause on the prefix scan, restricting it to the statuses an ACK decision can still be
changed by. No schema change, no new index, no migration.

Why it is an exact equivalence

The prefix rows feed exactly three decisions, and an acked row contributes to none of them:

decision	predicate	acked row
prefix gap	status !== 'acked' && status !== 'delivered' && !isResetDelivered	always false
committable delivered	status === 'delivered'	always false
committable reset-from-pending	status === 'pending' && deliveredAt !== null	always false

This holds because acked is terminal for status transitions. Every UPDATE against
inbox_entries is guarded on 'pending' or 'delivered'; the only statement that touches
acked rows at all is the GC DELETE in pruneStaleSilentEntries, which is restricted to
notify=false and therefore disjoint from this notify=true scan.

The ACK expansion set is unchanged — same inbox, same chat, same cursor. The scan simply stops
reading members of it that cannot affect the outcome.

Why a positive IN rather than <> 'acked'

<> is not sargable. Measured on the same dataset, status <> $4 stays a filter and removes
168,000 rows after reading them — it saves the row locks and nothing else:

spelling	rows scanned	buffers	time
status <> $4	168,020	2,851	9.6 ms
status IN ($4, $5)	20	4	0.03 ms

Why the status set is derived rather than written out

ACK_PREFIX_SCAN_STATUSES is computed from the shared status domain minus the terminal value.
The failure direction is what matters. If a fourth status is ever added, a derived set keeps
scanning it, so an unknown status still rejects the commit as a prefix gap — the same
fail-closed behaviour the original status-agnostic query had. A hardcoded
["pending", "delivered"] would instead skip those rows and let the ACK through, turning a
safe rejection into silent data loss. This is not hypothetical: the column's domain has changed
once already (a legacy failed value, normalised by migration 0066), and
ck_inbox_entries_status was added NOT VALID.

Results

End-to-end service call, median of 40 samples after warm-up, on both PostgreSQL major
versions this repo runs (docker-compose.yml uses postgres:16-alpine, CI uses postgres:17):

chat history	before	after (PG 17.10)	after (PG 16.14)
50,000	119.6 ms	1.46 ms	2.53 ms
150,000	341.4 ms	2.25 ms	2.77 ms
300,000	545.0 ms	2.66 ms	2.61 ms

Flat instead of linear. The residual ~2.5 ms is the transaction's fixed cost — an ACK against a
chat with no history measures 3.0–3.2 ms — so history no longer contributes at all.

Query level, same dataset, verified on 16.14 and 17.10 through the real driver (postgres-js
named prepared statements, repeated executions, default plan_cache_mode):

scenario	before	after
incremental ACK	168,001 rows / 2,940 buffers	1 row / 4 buffers
duplicate ACK	168,000 rows / 2,937 buffers	0 rows / 4 buffers
with FOR UPDATE	338,980 buffers / 2,800 dirtied	44 buffers / 0 dirtied
chat_id IS NULL partition	120,015 rows / 1,582 buffers	15 rows / 4 buffers

LockRows still sits above the Sort, so rows are still locked in ascending id order and the
lock ordering between concurrent ACKs on one partition is unchanged. The rows that stop being
locked are exactly the acked ones, which no production statement mutates.

Operational caveat

On PostgreSQL 16 the status restriction only reaches the index condition under a custom
plan. That is what the default plan_cache_mode = auto produces here, stably: the generic
estimate is hundreds of times more expensive, so the planner keeps rejecting it — verified over
repeated executions and across a plan-pollution sequence (same prepared statement serving a
small partition first, then the large one). But a deployment that sets
plan_cache_mode = force_generic_plan globally would silently return this scan to O(history) on
16. PostgreSQL 17 uses the bound parameters as an index condition either way. Noted in the code
next to the clause.

Scope of the win, stated precisely

Cost now tracks the number of non-acked rows in the partition, not the cursor position and
not the history. id <= cursor is still a filter, so a partition holding a large pending
backlog (an agent offline for a long time — delivered is capped per chat, pending is not)
still costs proportionally to that backlog. Draining such a backlog one ACK at a time remains
O(N²) in the backlog size. That is one to two orders of magnitude better than the previous
behaviour and removes the history dependence #1671 is about, but it is not O(1).

Alternatives considered and rejected

A per-(inbox, chat) high-water cursor (the issue's first suggestion) is not merely
heavier — it is incorrect here. inbox_entries.id comes from a sequence allocated at INSERT
time, while the fan-out INSERT in services/message.ts happens before the only serialisation
point in that transaction (UPDATE chats SET updated_at). Commit order therefore does not
follow id order. Reproduced with two sessions: A allocates id 1 and holds its transaction open,
B allocates id 2 and commits; an observer at that instant sees only id 2. A watermark advanced
to 2 would leave id 1 permanently below it once A commits — the client's ACK for row 1 would
hit the "already acked" short circuit, the row would stay delivered forever, bind-time
recovery would reset it to pending, redeliver, and short-circuit again. An unbounded
redelivery loop. Today's code self-heals in that scenario. Closing the hole would require the
delivery path to walk the watermark backwards — a new invariant spanning delivery, ACK and
recovery, plus a table and a migration.

Extending idx_inbox_chat_silent with a trailing id would make id <= cursor an index
condition. Measured: the planner keeps choosing the existing four-column index even when the
five-column one is available, and that index costs 147 MB against 19 MB on a table that sits on
the message fan-out write path. Not worth it.

Rewriting the three decisions as pure SQL to remove the JS materialisation: once the scan
is bounded to the non-acked window, materialising it is negligible. A larger diff for no
measurable gain.

Not adding a migration is a feature here

inbox_entries already carries five indexes plus a unique constraint on the fan-out write path.
Beyond that, CI pins packages/server/drizzle/LATEST to the latest journal tag specifically so
racing PRs conflict — #2108 landed while this was in progress and took slot 0091. A migration
in this PR would have needed rebasing; there is nothing here to rebase. Drizzle migrations also
run inside a transaction, so CREATE INDEX CONCURRENTLY is unavailable and any new index would
hold a lock on a large table for the duration.

Tests

packages/server/src/__tests__/inbox-ack-prefix-scan.test.ts:

• Cost guard. Runs a real ACK through a logger-instrumented database, captures the SQL the
  service actually emits, and replays each read against EXPLAIN (ANALYZE). Asserts the widest
  scan does not grow when the history grows 5×. It asserts on the service's own statements
  rather than a copy of the query, so inlining the query back into the service or dropping the
  predicate both surface. Fault-injected: removing the status clause turns it red
  (expected 1501 to be less than or equal to 309). The replay happens after the ACK has
  committed, so a correct implementation matches close to zero rows rather than the handful it
  saw live — what is asserted is the presence of the restriction, not the live row count.
• Derived-set guard. Pins that the scanned set is the status domain minus the terminal
  value, and cross-checks the shared enum against the live CHECK constraint so a database-side
  domain change cannot drift away unnoticed.
• Behaviour. ACK-through across a long acked history in the chat_id IS NULL partition (the
  isNull branch of chatPredicate, previously uncovered), and a prefix gap sitting above a
  long acked history still rejected.

Equivalence was additionally checked exhaustively in SQL before implementation: every
(partition × cursor) combination over an adversarial dataset — acked rows interleaved before,
between and after every interesting row, the NULL-chat partition, recovery-reset rows,
cross-inbox interference, gaps on either side of the cursor — derived all three decisions both
ways and diffed them. 60 cases, 0 disagreements; a deliberately broken variant produced 29,
confirming the check has detection power.

pnpm check, pnpm typecheck and the packages/server suite pass.