Skip to content

Cluster: 608 controlled rebuild — 14-PR replay onto a transactional SQLite substrate #648

Description

@tobyhede

Current status — read this first

  1. CLOSED, 14 of 14 merged. PR 14 landed as docs: describe the SQLite substrate and prove the offline snapshot (PR 14/14) #723 (9d6a578b0, 2026-08-11), with fix: drop the dead docsUrl, name no phantom recover command, and keep the probe out of production #724 and fix(core,cli): verify the stash bearer in the transaction that writes the slot (#666) #725 as follow-ups.
  2. The authoritative record is the closure verification comment — 2026-08-12, audited against source on main @ 223c707d9. It supersedes the residual set and several claims in the 2026-08-10 block below.
  3. This issue closed with work outstanding. Residual set: Complete the domain-lock deletion: CompletionLock and DelegationLock survive the single-store cutover #690, lifecycle-command-service.ts measures 59.10% on its dedicated tests — 164 undetected mutants CI has never reported #672, rdpath.ts is unreachable by mutation testing: every mutant survives because its only tests spawn the built binary #698, Cross-process interleave test for the pass/fail --claim-id arm: #706 witnesses routing, not isolation #707, exec_start_id / owner_start_id are declared, CHECK-constrained, and never written — recovery compares PID only #722. (stash --claim-id commits under a rotated bearer: resolve and mutate are separate transactions #666 is closed, via fix(core,cli): verify the stash bearer in the transaction that writes the slot (#666) #725; exec_start_id / owner_start_id are declared, CHECK-constrained, and never written — recovery compares PID only #722 was never added here.)
  4. Every table and checklist below is superseded in place, not rewritten. Earlier dated blocks are retained for provenance and several of them are wrong.
  5. When two blocks disagree, the newest dated one wins — including over the progress checklist, which has gone stale before.

Execution tracker for the 14-PR controlled rebuild that lands the #608 fix (atomic claim check-then-act) on a transactional SQLite substrate.

Plan: docs/superpowers/plans/2026-07-23-608-controlled-rebuild.md, plus one dated per-PR plan (2026-07-23-608-pr01…pr14).

Standing constraints (from the parent plan — restated because they bind every PR):

  • One dependent implementation PR open at a time. Merge before branching the next from freshly fetched origin/main.
  • Salvage tag 608-salvage-2026-07-23 (peeled to 5364e3965) is immutable. Salvage commits are evidence, never merge bases.
  • Every PR: named tests + pnpm run test:mutate:changed (optionally package-scoped or --related-tests) + pnpm run verify, all exit 0. Manual source scopes use package-relative changed-line ranges with --force. Mutation evidence must report non-zero instrumented sources/mutants and disposition every in-scope survivor or NoCoverage result; unscoped campaigns are forbidden.
  • RunbookState.schemaVersion stays 1. No migration, hydration, shim, or dual-read of incompatible persisted JSON.
  • Dated plan files are immutable in CI (check:docs:dated-immutable). Deviations get recorded in a new dated addendum, here, and in the PR description; never edit an existing dated plan.

Progress

Post-PR7 audit corrections — 2026-07-27

Progress is verified at 7 of 14 merged. The authoritative mainline sequence is #637#643#645#649#651#652#655. #640 merged into PR1's feature branch; #643 is the slot-2 merge that reached main.

New immutable corrections:

  • docs/superpowers/plans/2026-07-27-608-pr08-addendum-current-main-adaptations.md — PR8 selective-hunk replay, truthful recovery recording, current site layout, and current mutation gates.
  • docs/superpowers/plans/2026-07-27-608-pr09-pr14-correction-ledger.md — architectural, result-contract, build-order, mutation, and seven-path traceability corrections for PR9–14.

Binding audit decisions:

Implementation PRs remain serial: PR8 → PR9 → PR10 → PR11 → PR12 → PR13 → PR14.

Post-PR8 status verification — 2026-07-30

Supersedes the 2026-07-27 block's progress count and next-PR statement; every other binding decision above stands.

Progress is verified at 8 of 14 merged. PR8 is #661, merged 2026-07-29 at merge commit 43d25a231. The authoritative mainline sequence is #637#643#645#649#651#652#655#661. PR9 is next.

PR8's landing was verified against code on main (b5ba0c92b), not against its PR description. All six claimed deliverables are present:

Deliverable Evidence on main
Strict recovery-reason validation packages/core/src/runbook/execution-recovery-service.ts:148
recoveryRequired hydration + GOTO substep reset packages/core/src/runbook/actor-service.ts:723, compiler.ts:2467-2531
Atomic recovery-attempt commit packages/core/src/runbook/storage/runbook-store.ts:1312-1352
Typed sql.js startup failure packages/core/src/runbook/storage/driver-factory.ts:73
Store close/reopen serialization packages/core/src/runbook/storage/store-registry.ts:218-245
Extended SQLite BUSY/LOCKED codes packages/core/src/runbook/storage/native-sqlite-driver.ts:90-104

PR9 base drift since PR8

Four merges landed after #661. Three of them move surfaces PR9's write-once plan assumes are untouched, so the plan's cherry-pick gate cannot hold as written:

Untouched since PR8, as the plan assumes: runbook-store.ts, session-service.ts, state.ts, packages/core/src/events/types.ts, packages/core/src/runbook/index.ts, and the CLI stash/pop commands.

No PR9 work exists yet. EXECUTION_IN_PROGRESS and RECOVERY_REQUIRED appear in neither zod-schemas.ts nor errors/codes.ts. origin/608-pr9-remediation and origin/608-pr10-coverage are pre-rebuild salvage evidence from the superseded stack (closed PRs #634/#635) and must not be used as merge bases.

Seven-path atomicity status — what "8 of 14" does not mean

The checklist counts merged PRs, not closed paths. Audited against source on b5ba0c92b: the transactional seam is built but not yet wired into any production path. RunbookStore.captureAuthority / captureRunAuthority / saveState / commitOwnedState, SqliteExecutionLeaseService, CoreEffectfulMutationExecutor, and RunbookStoreActorCommitter have no non-test callers — every reference is under packages/core/__tests__/. Production mutations still route RunbookStateManagerRunbookStore.mutateStatewriteStateAtVersion, whose CAS is state_version + exec_token IS NULL only (storage/runbook-store.ts:1233). The claim-generation CAS is reachable only from saveState (:1489). The store's docblock states the consequence at :946: callers "MUST NOT treat a committed result as evidence that their authority was still valid at commit time."

# Path Atomic? Assigned
1 goto No — separate resolve and write, no claim CAS PR11
2 collect No — 3+ transactions; verifyClaimId checks bearer/grants only PR12
3 pass/fail --claim-id No — guardOpenChildren is false on the claim arm (lifecycle-command-service.ts:1640) PR11
4 complete/stop --claim-id No — force→report→release cascade across three transactions PR11
5 abort No — DelegationLock does not exclude claim mutation PR12
6 delegate fresh/retry No — narrowed only; source says so at lifecycle-command-service.ts:1026-1029 PR12 (using PR10)
7 bare / --run pass/fail Partial — open-delegated-children predicate now evaluated in the decisive write's transaction PR7 ✅ + PR11

Only path 7 has a deterministic interleave test (session-service.test.ts:1790). Paths 1–5 have none; the delegate tests at lifecycle-command-service.test.ts:865/:1515 fire inside the injected lock acquire, i.e. before #revalidatePresentedClaim, so they pin the narrowing rather than the residual window.

SessionLock and RunStateLock now have no production instantiation. DelegationLock and CompletionLock remain wired (packages/cli/src/helpers/lifecycle-seam-factory.ts:75,79; completion-service.ts:492,621,738; packages/cli/src/commands/abort.ts:93), so #608's two-lock-domain root cause survives until PR13 — and deleting them before PR11/PR12 land would remove today's narrowing with nothing replacing it. Several comments still read "SessionLock is not held" (lifecycle-command-service.ts:1027, :2259, collection-service.ts:292); the wording is stale, the claim remains true.

PR9 plan corrections beyond the ledger

Recorded in full in docs/superpowers/plans/2026-07-30-608-pr09-addendum-current-main-adaptations.md (seven deltas plus task order); it lands with PR9's branch. Summary follows.

Trial cherry-pick of a823892fe onto b5ba0c92b: 42 paths touched (the plan's count is correct), 10 conflicted, 32 clean. Every conflict resolves against the salvage — current main is newer in each. The salvage is a shape change (TSessionMutationResult<T>) that must be re-applied over main's bodies, never a body replacement. Two are outright regression traps:

Three corrections the plan lacks entirely:

  1. GuardedMutationResult<T> already is SessionMutationResult<T>. storage/mutation-result.ts:223-234 defines both refusal arms field-for-field (only statuskind and kebab→snake differ), and ExecutionEpoch is already branded by fix(core): close two defects in the recovery durability path #664. Derive an Extract<> alias; do not create a parallel union. The salvage's lowercase wire values ('execution_in_progress' as an emitted code) must be discarded per ledger line 58.
  2. Allowlist is 49, not 42. Seven paths break on tsc or are ledger-mandated: packages/claude-code-plugin/__tests__/helpers/test-utils.ts:306, core/__tests__/runbook/guarded-drain-composition.test.ts:152,183, core/__tests__/runbook/storage/fixtures/session-writer-child.ts:70-81, .../fixtures/child-protocol.ts, core/__tests__/testing/session-fixtures.test.ts:39, plus packages/core/src/output/zod-schemas.ts and docs/spec/cli-output.md. The plan's self-review item "Exactly 42 changed paths" must be amended to 49 or the gate blocks correct work.
  3. Build ordering and schema coverage are absent. pnpm run build must run between the core and CLI test steps (CLI resolves core through dist). Registration of the two codes gets zero mutation signalpackages/core/src/output/** is inside core's Stryker exclusion, and OutputEmitter.error takes string, so an unregistered code typechecks and passes verify. The only gate is docs-error-code-drift.repo-asset.test.ts, which requires the docs/spec/cli-output.md fence to land with the registry entry. No intersection with CLI emits at least 10 error codes absent from the closed ErrorCodeSchema enum #662's twelve codes — PR9's two are new.

The plan's §85 whole-file Stryker campaigns are forbidden by current policy (runbook-store.ts alone is ~1450 LOC and has run 17+ minutes without finishing), as is its instruction to delete reports/stryker-incremental.json. Use pnpm run test:mutate:changed.

One retained salvage item is load-bearing: the added driver-contract.test.ts case asserting the trigger-abort message is exactly execution_in_progress across both drivers. mutateSessionGuarded normalizes on exact string equality, and main's only existing assertion is a regex against the store-thrown error, not the SQLite RAISE(ABORT). Land that test before or with the normalizer, or a decorated raise text silently turns the refusal into a crash.

Decided — atomic claim-aware stash is split to #666

Resolved 2026-07-30 under the correction ledger's split clause (lines 60-68). The slice is removed from PR9's scope and tracked in #666, blocked on PR9. #608's closure guarantee has been narrowed in the same action to exclude stash --claim-id and point at #666. PR9's scope is now the typed-refusal slice only: 49 paths, zero behaviour change.

Rationale: PR9's typed-refusal work is a mechanical rewrap across 42 files with no behaviour change, whereas the stash slice is a new authority protocol with new refusal semantics — mixing them makes the diff unreviewable against its own allowlist gate. The stash slice consumes the mutateSessionGuarded primitive PR9 introduces, so it sequences naturally after. And stash --claim-id is an eighth path, not one of #608's seven.

The gap is confirmed real. stash --claim-id resolves the bearer in an unlocked read (stash.ts:30session-service.ts:858), then mutates in a separate transaction (stash.ts:70session-service.ts:1343) whose only check is targetedByClaim (:1352) — "does some claim control this run", never which one. Since mintRunControlClaim (:489) deletes and re-inserts for the same run, a rotated-out bearer still satisfies it. pop --claim-id is already correct (unstashForClaimId verifies the secret in-transaction at :1396); this brings stash to pop's standard. A second consumer is documented in code at lifecycle-command-service.ts:1025-1028, which names #608.

Full defect analysis, scope, and the required rotation-interleave evidence are in #666.

Post-PR9 status verification — 2026-07-30

Supersedes the 2026-07-30 PR8 block's progress count and next-PR statement; every other binding decision and correction remains in force.

Progress is verified at 9 of 14 merged. PR9 is #667, merged 2026-07-30 at merge commit 03c11cfb3. The authoritative mainline sequence is #637#643#645#649#651#652#655#661#667. PR10 is next. The remaining implementation order is PR10 → PR11 → PR12 → PR13 → PR14.

PR9 landed the typed-refusal slice only. Atomic claim-aware stash --claim-id remains split to #666; PR9's merge unblocks that follow-up, but it remains outside #608's seven-path closure and does not replace PR10 in the serial chain.

The delegation-retry review finding from #667 does not expand PR10. PR9's ordering fix prevents a refused child release from superseding the pending outcome first, but delegate retry remains a multi-transaction sequence. Its transactional closure remains assigned to PR12 together with delegate, collect, and abort.

Post-PR13 status verification — 2026-08-08

Supersedes the 2026-07-30 PR9 block's progress count and next-PR statement, and replaces the seven-path table in the 2026-07-30 PR8 block above, which is PR8-era and no longer describes main. Every other binding decision and correction remains in force.

Progress is verified at 13 of 14 merged, audited against source on main @ 0a921937c, not against PR descriptions. The authoritative mainline sequence is #637#643#645#649#651#652#655#661#667#668#669 (+#671) → #673 (+#687) → #674. PR14 is next, and is the last.

The checklist above went four PRs stale between 2026-07-31 and 2026-08-08. Treat the dated blocks, not the checklist, as authoritative when they disagree.

Seven-path atomicity status — all seven now closed

Atomicity bar: one decisive BEGIN IMMEDIATE transaction whose first act is a claim compare-and-swap, refusal rolling back with nothing written.

# Path Atomic? Closed by
1 goto Yes PR11 (#669, f3e94d9ec)
2 collect Yes #687 (a9c07a85a)
3 pass/fail --claim-id Yes #700 — the last one; see below
4 complete/stop --claim-id Yes PR11 (f3e94d9ec) — was three transactions
5 abort Yes PR12 (#673, ed461d7a1)
6 delegate fresh/retry Yes PR12 (#673, 3f9fa37cf)
7 bare / --run pass/fail Yes PR7 (#655, 106b099bc) + PR11

Path 3 was not closed by PR11 as the PR8-era table assigned it, and no document on main recorded that until #700. guardOpenChildren was computed from the resolution shape and excluded claim-shaped resolutions, so the arm took the unguarded branch and no ParentAdvanceGuard reached commitOwnedState. The line was semantically unchanged from the PR8 finding, surviving PR8 through PR13 unremarked.

It was also the arm that matters rather than a residual one: a bare mutation on a delegation-exposed run is refused ACTOR_CONTEXT_REQUIRED and --run cannot carry a bearer, so --claim-id is the only invocation the post-R1 protocol leaves an orchestrator — the guarded shapes were the ones it could not reach. PR11's fence does not cover it transitively, because claimRunbook bumps neither the parent's state_version nor its claim_generation.

PR13 landing verification

All claimed deliverables are present on main:

Deliverable Evidence
paths.ts free of the six removed symbols packages/core/src/paths.ts; zero repo-wide hits outside the changeset and dated plans
No production JSON run/session reader or writer new RunbookStore(...) appears once, storage/store-registry.ts:294
SessionLock / RunStateLock deleted Source files absent; only plugin-local PluginSessionLock survives
runbook_started.statePath removed packages/core/src/events/types.ts:37-43; run identity now RunbookMetadata.runId
WAL check + bounded retry native-sqlite-driver.ts:240-302; 11 attempts, 25→250 ms as claimed
RD-306..309 registered errors/codes.ts:127,135,150,172
New barrel exports All seven present in runbook/index.ts / paths.ts

CompletionLock and DelegationLock survive at exactly six production call sites, matching #690. The PR8-era sites cited above (lifecycle-seam-factory.ts, abort.ts) are gone.

Open items carried into and past PR14

Post-PR13 audit corrections — 2026-08-10

Supersedes the 2026-08-08 block's path-3 attribution, residual set, and release scope. Every other binding decision and correction above remains in force. Audited against source on main @ a7a99c566, with each finding independently re-verified by an adversarial pass instructed to refute it.

Progress is unchanged at 13 of 14 merged. PR14 is still next, and is still the last.

Mainline since the 2026-08-08 baseline

0a921937c is the merge commit of #699 — it is the audit baseline itself, not a PR that landed after it. Five merges followed:

PR Merge Closes What it changed
#701 f4edbeeb6 #680, #681 Machine becomes the single writer of frame entry (removes all 13 external deriveActiveEntry/ensureActiveEntry call sites, incl. two Category-B leaks in packages/cli/src/services/execution.ts); delegate --retry idempotency via resolveRetryIssuance; RD-826/827/828 registered and documented
#705 0b58b0d94 #702 Goto interleave witness; extracts isDelegatedChildClaim; drops the dead guardOpenChildren CLI transition-context field
#706 ff8ce5ad9 #700 The path-3 fix (below)
#708 106421f7d Pins that isDelegatedChildClaim reads linkage, not grants
#709 a7a99c566 #703 Re-lands #704, which merged into an already-merged branch and never reached main

Numbering traps, both of which the 2026-08-08 block fell into: #700 is an issue, closed by PR #706. #704 is a PR that never reached main; the issue it addressed is #703, closed via #709.

Correction — the path-3 row asserted a fix that did not yet exist

The 2026-08-08 seven-path table reads | 3 | pass/fail --claim-id | Yes | **#700** — the last one |. Both halves of that cell are wrong:

The block's surrounding prose is defensible — "no document on main recorded that until #700" is true of the issue. Only the table cell is wrong.

Current source: packages/core/src/runbook/lifecycle-command-service.ts:2569-2570.

Seven-path atomicity — re-verified, with the bar restated

All seven paths meet the bar on a7a99c566. Paths 2 and 5 were re-verified specifically because they aggregate through runAll/commitOwnedRunSet rather than the single-run path: runbook-store.ts:1159 opens one transaction, :1166-1172 classifies every member before :1174-1176 writes any, and classifyCommitRow (:397) checks claim presence/control/status (:406-412) and generation equality (:413-419) ahead of state_version (:434) and execution identity (:441). Session release is folded into the same transaction (:1177-1181).

The bar as previously phrased overstates two things. Restated:

  • "One transaction" applies to the decisive write only. The real shape is one decisive commit behind a three-transaction lease fence — capture (effectful-actor-mutation-runner.ts:354-355), lease.acquireAll (effectful-mutation-executor.ts:336), markEffectStartedAll (:344), then commitOwnedRunSet (:380).
  • "Nothing written" should read "nothing authority-bearing written." Every path commits a recordClaimSeen liveness row beforehand, in its own transaction, deliberately — rationale at collection-service.ts:507-525. It is inert for authority because the schema triggers exclude last_seen_at from the generation bump (see the PR4 deviation note above).

The sql.js driver is not a caveat on the bar. sqljs-driver.ts:251 does issue a plain BEGIN rather than BEGIN IMMEDIATE, but exclusion comes from runLocked (:299-309) plus the load→BEGIN→work→COMMIT→persist→close cycle holding the file lock throughout; the CAS-before-write ordering lives in commitOwnedRunSet, not the driver, so it is identical on both. driver-factory.ts:97-105 additionally refuses sql.js off-WebContainer. Do not record this as "the bar holds only on native" — that would misdirect a future WebContainer audit.

Path 1 should also cite #705 (0b58b0d94) as the source of its in-transaction interleave witness (lifecycle-command-service.test.ts:5235).

Path 3 carries a live caveat: its witness proves routing, not isolation, recorded in-code at packages/core/__tests__/runbook/claim-test-helpers.ts:110-121"would still pass if SQLite offered no isolation whatsoever." That is #707, a coverage gap, not a defect: runGuardedParentAdvance opens no transaction and performs no CAS (session-service.ts:1489-1490 calls it a "cheap pre-check fast-path"); all authority is in assertParentAdvanceAllowed inside commitOwnedState's transaction.

Correction — the residual set

The 2026-08-08 list (#690, #672, #680, #681, #698, #666) is stale in both directions.

Issue 2026-08-08 Actual
#690 residual OPEN — still residual
#672 residual OPEN — still residual, and see CI note below
#680 residual CLOSED 2026-08-08 by #701
#681 residual CLOSED 2026-08-08 by #701
#698 residual OPEN — still residual
#666 residual OPEN — still residual
#707 not listed OPEN — add

#609 is not promoted into this set. The 2026-08-08 block already adjudicated it obsolete and instructed "Close it"; that decision stands and is re-confirmed (the proof block at lifecycle-command-service.ts:2312-2343 has zero hits, session-lock.ts/run-state-lock.ts are deleted, all three cited call sites are structurally impossible). One precondition before closing: #690's body names the surviving DelegationLock → CompletionLock edge against #drainSubstepObservations, which has zero hits repo-wide. Its acceptance item is therefore vacuously satisfied and will prompt nobody. Restate it against the live path — completion-service.ts:1175 → :1236 → :1076 — then close #609.

Explicitly out of the residual set, and why. #684, #685, #675, #676, #677, #659, #662 are all open and all self-identify as deferred from or surfaced by this cluster. They are not blockers on closing #648: none is one of the seven paths, and each is independently tracked. Enumerating the residual set without saying this is what let the previous list go stale.

Release scope — deprioritised by maintainer decision, 2026-08-10

The 2026-08-08 block lists "cutting an actual 2.0.0 release" as PR14 scope. That is withdrawn. Rundown is not near release; treating changesets as a gate generates noise rather than signal.

Recorded for whenever the release track resumes, so it is not rediscovered: .changeset/ holds five changesets (not four — #706 added claim-arm-open-children-guard.md on 08-09), all five packages are in one fixed group so everything lands on 2.0.0, and release PR #636 already contains the generated output. Coverage is 5 changesets against 372 merge PRs since @rundown-org/core@1.0.0 (e2dbb00cc, 2026-01-20), with 14 of 15 breaking-marked commits uncovered. Also note #701's ! marker was erased by the squash-merge (branch commit aedbf2d50 carries it; the merged PR title does not), so conventional-commit breaking markers are not durable in this repo's merge flow. Do not merge #636 until the release track is deliberately resumed.

PR14 scope, re-derived against source

Of PR14's ~22 decomposed tasks: 7 done, 7 invalidated by the correction ledger, 8 remaining. In priority order, what actually remains:

  1. Rewrite docs/internal/architecture.md — the descriptive-doc debt of the entire rebuild, and by far the largest item. Now 1130 lines (the 2026-08-08 block's "980" is stale; it grew via fix(core): make the machine the single writer of frame entry, and make delegate --retry idempotent #701 without gaining storage content), with zero occurrences of SQLite, rundown.db, BEGIN IMMEDIATE, CAS, state_version, exec_token, mutateState, lease, or any domain lock. It is not merely silent but actively false: :20 states | **Persistence** | JSON files | and :30 diagrams [CLI Commands] <---- [Persisted JSON], both falsified by refactor(core): make SQLite the sole state authority #674. No qualifying or historical context surrounds either. docs/reference/runtime.md is already correctly cut over, so this is single-file debt.
  2. Document RD-306..309. Still zero hits under docs/ outside superpowers/. The drift guard is one-way by design (docs-error-code-drift.repo-asset.test.ts:12-16), reads only docs/spec/cli-output.md, and parses only ```json fences — so the prose table in docs/reference/cli.md:1238-1244 is ungoverned in both directions. Adding them to `cli-output.md` is what arms the guard.
  3. Fix docs/spec/language.md:872-874 (was :838-839; shifted +34). The no-migration MUST NOT is scoped to .rundown/runs/, which post-cutover holds only captured outputs — the normative rule binds the wrong directory and does not bind the database at all.
  4. Add a real offline snapshot proof. site/tests/sqlite-substrate.spec.ts:13-15 states outright that it mounts its own files rather than loading public/rundown-snapshot.bin; runbook-runner.spec.ts is unchanged since 2026-01 and asserts one scenario reaching COMPLETE with no sql.js assertion and no pass/fail/goto. This is the ledger's :144 requirement, unmet. Also git mv the probe to site/src/pages/dev/.
  5. WebContainer/sql.js bundling needs no work. The ledger was right: sql.js is a hard dependency of core and site/scripts/prune-sqljs.mjs:148-171 throws if a declared entry point or a retained loader's .wasm is missing.

A new dated addendum under docs/superpowers/plans/ is mandatory before PR14 starts — check:docs:dated-immutable forbids editing the PR14 plan. Note that check runs only in CI, not in the verify fan-out, so a local verify will not catch an edit to it.

Carved to linked issues

Per this issue's own instruction that such items "should be carved to linked issues rather than done inline":

  • exec_start_id / owner_start_id declared and CHECK-constrained (storage/schema.ts:106,200,113-116) but every production write is NULL. Confirmed not redundant: recoverDeadOwner returns {kind:'alive'} at execution-lease.ts:577 and never reaches the token/epoch CAS at :660-666, so epoch and token do not disambiguate the liveness decision. Worse, isProcessAlive fails toward alive on any non-ESRCH error (file-lock.ts:106). Failure mode is a permanent stall, not a safety break.
  • The typed-error documentation URL. Correction to the 2026-08-08 wording: the problem is not a missing route. site/astro.config.mjs:20 sets site: 'https://rundown.cool'; rundown.dev does not resolve at all. rundown-error.ts:88 emits it, and wrapper.ts:178 puts it in the default JSON error envelope, so every typed error ships a dead link. The fix is the URL, not a site route.
  • docs/reference/cli.md:1241 instructs operators to "resolve the interrupted attempt through recovery", but no rundown recover command exists and recovery is automatic, performed by the command that emits the error (effectful-actor-mutation-runner.ts:300-315). Documentation-parity defect.

Note on #672 and CI

#672 persists partly because CI structurally cannot report it. The Mutation Testing producer has never uploaded a baseline since it was sharded on 2026-07-02: 8 scheduled runs, 5 cancelled, 3 failed, zero successes. The cause is scope, not time — scripts/mutation-shard-plan.mjs:261-275 emits whole-file scopes (the failing matrix entry was lifecycle-command-service.ts, 4068 lines) at a flat STRYKER_CONCURRENCY: 4, while the repo's own local policy drops to 1 above 1000 lines. Raising the timeout is a no-op: mutation.yml:96 is a step-level 60-minute timeout beside the job's at :69. Tracked under #670, whose scope needs extending from mutation-pr.yml to this producer.

Recorded deviations

PR 4 — required adaptations to current main

Verified by a full trial replay of 64f840ded a39701d6e 785a3eabb e203ec905 onto 464e94ee1. Three deviations from the plan as written; all three are adaptations to work PR 2/PR 3 already landed, not changes to PR 4's intent.

1. schema.ts conflict in 785a3eabb — resolve in favour of HEAD.

785a3eabb narrows claims_guard_update / claims_bump_gen_update to resolution-affecting columns so a last_seen_at refresh (#519 liveness) is neither refused under ownership nor bumps claim_generation. Current main already does this: 9080c1849 (PR 2) landed BEFORE UPDATE OF key, controlled_run, secret_hash, issued_generation, status, parent_run_id, parent_linkage_version, delegation_json, grants_json, which already excludes last_seen_at and updated_at.

The incoming list is a strict narrowing of main's — it additionally drops key and issued_generation. Taking the incoming side would silently weaken a guard main deliberately holds: rotating a claim's lookup key or issued generation under an active execution owner would stop being refused and stop bumping claim_generation.

Resolution: keep HEAD's column list on both triggers. Keep the commit's two explanatory comments if desired. Keep the commit's tests — they pass against main's schema unchanged, and they are the thing that actually pins the last_seen_at behaviour.

2. Three cherry-picked tests violate PR 3's execution-identity CHECK.

785a3eabb and e203ec905 add tests that simulate an owner with UPDATE runs SET exec_token = 'sha256:live'. PR 3 made the identity columns all-or-nothing (exec_epoch/exec_pid/exec_token together, epoch naming a real execution_attempts row), so those three fail with CHECK constraint failed:

  • mutateState … › refuses immediately while an execution owns the run
  • session persistence and run listing › refuses to delete a run with an active execution owner
  • tombstone preservation (#519 lastSeenAt survives) › records claim activity even while the run has an active execution owner

Fix: use the takeOwnership(txn.tx, state.id) helper already in runbook-store.test.ts (added by PR 3 with a docblock warning against exactly this). Mechanical — three call sites.

3. readRunWithVersion parameter type.

e203ec905's mutateState calls this.readRunWithVersion(tx, runId) inside driver.read(...), but declares the parameter as SqlTransaction. PR 1/3 introduced the narrower SqlReadTransaction that driver.read supplies, so this is a hard tsc error (TS2345: Property 'exec' is missing). Widen the parameter to SqlReadTransaction, matching every other read helper in the file.

Trial-replay evidence (after the three fixes, on 464e94ee1):

  • Named tests: 121 passed, 4 suites.
  • tsc --noEmit on @rundown-org/core: clean.
  • Full core suite: 4525 passed, 2 skipped, 198 suites.
  • Workspace build: clean. Full CLI suite: 3112 passed, 143 suites.
  • Changed paths: exactly the 12 in the plan's ownership block; no CLI lifecycle dispatch.

Not yet run in trial: the scoped Stryker command and full corepack pnpm run verify. Both remain required gates on the real PR.

Metadata

Metadata

Assignees

Labels

DelegationDelegation lifecycle remaining-work roadmap (2026-07-01)P1: highCorrectness bug or blocker, high reachbugSomething isn't workingdocumentationImprovements or additions to documentation

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions