fix(payments): a crash mid-settlement lost the order, silently and forever

[catomean]

Closes finding 4 of #563 — the one with money behind it.

handlePaymentConfirmed flips the intent to paid first — deliberately, because that conditional update is the lock making settlement run exactly once — and only then writes the order, decrements inventory, notifies the seller, grants entitlements and fans out webhooks.

A crash in that gap lost every one of them, permanently and in total silence:

• the row is paid, so every later observer's claimPaidTransition returns false and skips the side-effects as "already settled";
• refreshPaymentStatus short-circuits on terminal statuses;
• the reconcile cron sweeps only CREATED / INVOICE_READY / PENDING_CONFIRMATION, so it never looks at a paid row again.

The buyer's money is gone, their order sits in pending_payment forever, and nothing anywhere reports a problem.

The marker

side_effects_at, stamped when the settlement path completes — on the tip branch and the main one. NULL on a paid intent means the side-effects didn't finish, and the reconcile sweep now reports those.

It rides the existing cron tick rather than adding a second timer, and runs even when there's nothing to reconcile: an incomplete settlement is a paid row, so it never appears among the sweep's own candidates. Skipping the check on a quiet tick would hide exactly the case it exists for.

Why it reports and does not replay

Replaying looks like the obvious fix and is a worse bug. decrement_inventory is a blind inventory_count - 1 with no idempotency key — read from the live database, not assumed — so re-running settlement for one sale decrements twice and quietly destroys stock. Plan grants and entitlements have the same shape.

Trading an invisible loss for a silent corruption is not progress. Safe replay needs per-effect receipts, a separate and larger piece of work. This converts a permanent silent loss into a named, actionable one, which is the part that couldn't wait.

The marker also doesn't claim every async fan-out landed — several effects are deliberately fire-and-forget, so it says the settlement path ran to completion. Stated in the code rather than implied.

Schema

Deliberately not granted to anon/authenticated: payment_intents uses column-level SELECT grants and this is an internal operations marker — same reasoning as nwc_connection_uri. A partial index on (paid_at) WHERE status='paid' AND side_effects_at IS NULL keeps the per-minute check off a seq scan, and is empty whenever things are healthy.

Verification

type-check green, check:schema-columns green, migration versions unique (61), 5 new tests.

Mutation-proven: dropping the side_effects_at filter fails the predicate test; downgrading the report from error to warn fails the loudness test. A detector that goes quiet is indistinguishable from one finding nothing — which is this whole bug.