Money-critical: a pending delivery is stranded on intent completion, and a trust-base rotation freezes the fleet

[MastaP]

Two independent defects on the money path, both found while investigating the mainnet enablement (#764). Neither is caused by that work; both predate it and both become more consequential now that mainnet is live.

1. A certified-but-undelivered token is stranded when the intent completes

modules/payments-v2/machine/TransferMachine.ts:205-209 computes deliveryPending and then calls closeTail(...) on the clean path without consulting it. closeTail (:540-548) calls completeIntent(transferId, payload) unconditionally.

The recipient's certified token bytes exist in exactly one place at that moment: the local delivery journal (TransferMachine.ts:329-337, blobHex, written into the scoped KV at stores.ts:114). Resume only walks server-open intents, so a completed one is never revisited.

Consequences, all silent:

• Any change to the network string, the dataDir, or the browser profile abandons that KV, and the spent token is gone with it.
• Sphere.clear() does the same — it calls storage.clear() with no prefix (core/Sphere.ts:1184).
• There is no error surface and nothing server-side to recover from: the sender's intent is closed, the recipient never received a deposit.

The asymmetry is the problem — the spend is on chain and durable, the delivery obligation is local and disposable.

2. A root-node rotation freezes every deployed wallet, with no runtime recovery

No verification rule in the pinned state-transition-sdk@3.0.1 reads epoch or epochStartRound. UnicityCertificateVerifier checks networkId plus quorum signatures against trustBase.rootNodes only. On a rotation every signature returns "No root node defined" (UnicitySealQuorumSignaturesVerificationRule), quorum fails, and every inclusion proof is rejected — receive drops all incoming tokens.

The trust base is a compile-time literal (assets/trustbase.ts) reached through a single importer. OracleProvider has no refresh member (oracle/oracle-provider.ts:28-41) and initialize() caches once (oracle/UnicityAggregatorProvider.ts:136-150). So recovery requires an npm release reaching every consumer.

Mainnet's rotation budget is exactly one node. It runs 4 root nodes at quorum 3, so one can rotate out invisibly; the second breaks the entire fleet at once.

It also fails in the worst available way: token-engine/certification-outcome.ts matches only TRANSACTION_HASH_MISMATCH, so INVALID_TRUSTBASE falls through to ProofUnconfirmedError — CERTIFICATION_UNCONFIRMED, i.e. keep-open — after the spend has already been submitted (SphereTokenEngine.ts:331 precedes :338). And modules/payments-v2/machine/resume.ts:206 emits transfer:attention only for SPLIT_CHECKPOINT_LOST and CHECKPOINT_TRUSTBASE_MISMATCH, so the intent retries forever with no operator surface, sources reserved, no attempt budget.

An undocumented mitigation does exist and is worth knowing: buildTokenEngine reads getTrustBaseJson() fresh (core/Sphere.ts:3837), so oracle.initialize(newJson) followed by sphere.setOracleApiKey(sameKey) swaps the trust base live. It is shaped like an API-key change, which is why nobody would find it.

Suggested first step

Cheap and separable: give INVALID_TRUSTBASE its own branch in certification-outcome.ts that still keeps the intent open (correct — the spend may be committed) but carries a distinct code, so resume.ts can raise transfer:attention. Roughly ten lines, and it converts a silent permanent freeze into something an operator can see.

Also worth writing down: the rotation budget is one node, and a rotation is a lockstep flag day with wallet-api, which verifies deposits against its own pinned trust base.

---

Evidence gathered 2026-09-01/02 against main at the mainnet enablement merge. File:line references were accurate then; verify before acting.

[MastaP]

Decision (discussed with codex, every claim re-verified against source)

Both halves got an independent codex pass. Each one corrected my framing on a load-bearing
point, so the corrections come first.

---

1. Stranded delivery obligation

Decision: B/D — a backend-owned durable delivery outbox. wallet-api first, SDK second. Plus C in the SDK as defence in depth.

Correction that decides it: option A is not implementable client-side

I had this as "make closeTail skip completeIntent while deliveryPending". That is not
enough, and the design already says why:

inventory/apply … apply completes the intent in-transaction
— docs/PAYMENTS-V2-DESIGN.md Appendix B (verified against wallet-api source)

applyCommitted runs at TransferMachine.ts:208, one line before closeTail at :209. By
the time the explicit complete is skipped the intent is already closed server-side. complete
is only the idempotent backstop it is documented to be (§5.6, "on every shipped path the
evidenced apply already completed the intent").

The one existing mechanism for keeping an intent open past a commit is to skip apply
entirely — what the P5 amendment does for keep-open and unjournaled-committed legs. Reusing it
here would regress the S3 rule the same paragraph pins:

a deferred deposit must not leave spent sources listed active on every device

which is exactly why apply deliberately runs for delivered, deferred and failed-and-journaled
deposits alike. So A costs either that regression or a wallet-api contract change
(apply-without-complete, or a delivery-pending intent state). If we are changing the backend
contract anyway, the right change is the durable obligation, not a longer-lived open intent —
an open intent also keeps the sources pinned by the reservations ledger
(PaymentsFacade.ts:558) and mixes "certification uncertain" (E.2/E.4) with "certified, not yet
transported", which are different obligations with different exits.

Correction 2: the pre-loss surface is not zero

I said poisoned entries are excluded from pendingWork() and invisible. Half right:

• convergence.ts:141 — journal.some((e) => e.undeliverable !== true): poisoned entries do
  drop out of the heartbeat predicate, so they are never retried again. Confirmed.
• convergence.ts:208 — derivePendingTransfers emits a journalOnlyRow for every journal
  group not already seen, poisoned or not. So a poisoned obligation does stay visible in
  pendingTransfers(), alongside the one-shot transfer:attention at
  journal.ts:207.

So before KV loss there is a passive row and one historical event, but no retry and no operator
transition. After KV loss there is nothing at all. That is the real shape.

What the recipient can do: nothing

Once the intent is completed and the sender's journal bytes are gone, there is no supported
recovery path. The recipient never held the certified blob, a completed intent is not listable
(resume.ts:30 → listIntents('open'), compose.ts:267), and wallet-api holds no delivery
obligation to replay. On-chain state proves the sender spent; it is not the recipient's
deliverable token serialization.

Change list

wallet-api (blocking, separate repo + release):

1. A durable delivery-outbox record, idempotent on (transferId, opIndex) — recipient, certified
   token bytes, delivery metadata.
2. An authenticated write-once/idempotent staging endpoint.
3. Intent completion conditional on every certified leg having a durable outbox record — staging
   and completion in one transaction.
4. A backend deposit worker with retained terminal-failure state.
5. Backup/restore must cover that table: it becomes the authoritative outstanding obligation.

sphere-sdk (after that contract exists): stage the obligation server-side right after the
journal insert at TransferMachine.ts:327; gate applyCommitted (:208) on durable
acknowledgement; closeTail (:540) relies on the evidenced completion; keep the local journal
as a latency/offline aid reconciled against the server; update design §5.6 and §6, which today
declare the certification-to-deposit gap client-only.

SDK-side C, worth doing regardless — it improves operability but cannot close the loss
window, so it is not a substitute: persist poison in the pending-transfer view, re-emit
transfer:attention on startup rather than only at the sixth failure, add an explicit
retryDelivery(transferId[, opIndex]) that clears poison and replays, and never silently delete
a poisoned record.

---

2. Trust-base rotation freezes the fleet

Decision: D now (classify, park, surface, and defer rather than reject on receive). Then C + a verified form of B — which needs a protocol 3.0.1 does not currently provide.

Correction that raises the severity: receive does not "drop" — it permanently rejects

modules/payments-v2/receive/Receive.ts:253:

const verdict = await engine.verify(token);
if (!verdict.ok) return { kind: 'ack', ack: rejectAck(entry, 'invalid') };

Every false verdict becomes disposition: 'rejected', reason: 'invalid' on the mailbox entry.
The screen cannot tell a bad token from "this wallet's trust base is stale". So under rotation
the failure is not that incoming money stops arriving — it is that valid incoming deliveries are
permanently rejected at the mailbox, one per drain, with the sender already spent.

That alone makes the cheap patch worth shipping ahead of any rotation design.

The cheap patch is bigger than the ~25 lines I estimated

SphereTokenEngine.verify (token-engine/SphereTokenEngine.ts:575) returns
{ ok: false, reason: String(result.status) }, and result.status is the aggregated
VerificationStatus — FAIL. The granular INVALID_TRUSTBASE
(InclusionProofVerificationRule.js:73) sits in the child results[] trace. So the reason has
to be recovered by walking that trace before anything downstream can branch on it.

Scope: extract the status from the trace; a typed TRUSTBASE_REFRESH_REQUIRED alongside the
existing trust-base checkpoint error in token-engine/errors.ts; map it separately in
token-engine/certification-outcome.ts (which today special-cases only
TRANSACTION_HASH_MISMATCH and folds everything else into ProofUnconfirmedError); surface it at
resume.ts:205 (attention today fires only for the two split-checkpoint codes); park the intent
instead of retrying it every heartbeat, with a refresh explicitly waking convergence; and change
Receive.ts:253 to defer rather than reject when verification failed solely because the
trust base is stale.

Deliberately not an attempt budget: after submission the spend may already be certified, so
exhausting a counter must not abort or complete it. Park + attention + refresh-wake is the safe
terminal state.

Why a remote refresh cannot just fetch JSON

RootTrustBase.fromJSON validates structure, uniqueness and quorum bounds and then accepts the
supplied signatures map without verifying it (RootTrustBase.js:147). TLS or a fixed URL
only moves the root of trust to DNS/CA/server compromise — a worse failure than the outage, since
a compromised endpoint could swap the validator set and make the wallet accept attacker-signed
proofs. A pinned file hash is safe but still needs an npm release to change the hash.

A safe refresh needs one of:

1. a new trust base cryptographically authorised by the currently trusted root set, or
2. a separately embedded, offline-custodied trust-base-update key (a second root of trust, with
   its own rotation and compromise-recovery problem).

(1) is preferable. The signatures, previousEntryHash and changeRecordHash fields suggest
that continuity chain was intended — but 3.0.1 ships no verifier for it, so their presence
must not be mistaken for authentication. refreshTrustBase() (option C) is API ergonomics on top
of that verifier; on its own it establishes no trust.

Note the existing undocumented swap (oracle.initialize(newJson) + setOracleApiKey(sameKey),
Sphere.ts:3975 reads getTrustBaseJson() fresh) does work mechanically, but it is neither
atomic nor named for the job and initialize() accepts arbitrary JSON. It is an operator escape
hatch, not the security model, and should not be documented as one.

Rotation can be detected during the overlap window

With one of four nodes rotated, old clients still reach 3-of-4 on the remaining old signers, so
there is a window before the second rotation breaks everything. Signals available in it: seal
epoch differs from the installed base's; a seal carries signer IDs absent from the installed
base; accepted seals repeatedly reach quorum on exactly the minimum three old signers. Worth a
trustbase:rotation-pending health event while proofs still verify.

Limits worth stating plainly: an unknown signer is not authenticated evidence on its own (a
malicious gateway can append junk signatures), and a seal's epoch is only meaningful while the
seal still reaches the old quorum. If operators rotate straight to a set with fewer than three
old signers without first publishing an old-quorum-authorised transition, an old client cannot
securely learn the new keys at all — no SDK refresh API can repair that. That makes it an
operational requirement, not just an engineering one: publish the next trust base while the old
quorum is still live, wait for fleet uptake, drain open split intents, then activate.

Order

1. Ship D now — classify, park, surface, defer-not-reject.
2. Add authenticated transition verification (state-transition-sdk or here).
3. First-class atomic refreshTrustBase() on top of it.
4. Require rotation publication + an overlap window operationally.
5. Keep the embedded bases as bootstrap anchors, not permanent validator literals.

---

Neither half belongs in #772 (lifecycle/global state) — both are money-path changes and want
their own PRs. #1 is blocked on wallet-api. #2's step 1 is unblocked and I'd take it next.

[MastaP]

Filed the backend half as unicity-sphere/wallet-api#135 — the delivery-outbox contract that item 1 is blocked on. The SDK-side integration listed above waits on it; the SDK-side C hardening (persist poison in the pending view, re-emit attention on startup, an explicit retryDelivery, never silently delete a poisoned record) does not, and can go in the same PR as item 2's patch.