fix(storage): preserve whole recovery generations with resumable moves

[johnny4young]

Summary

Fresh-key recovery preserves the whole database/WAL/SHM/.encrypting/blob generation in one journaled, resumable .unreadable-* directory, and a durable cross-process ownership ledger (.store-ownership.sqlite) keeps recovery and conversion away from live pools. Details below.

Review follow-ups

• dff833f — fix(storage): sync recovery moves and skip empty archives
  • archive creates .unreadable-* only after confirming gancho.sqlite exists (no empty hidden directory when there is no database).
  • The store directory is flushed (F_FULLFSYNC, falling back to fsync) after the journal and archive directory are created, both directories after the moves, and again after the journal is removed.
  • Test: archiving a directory without a database leaves neither an archive directory nor a journal.
• db7ec0c — fix(storage): stamp ownership claims with boot session and start time
  • Each claim now records kern.bootsessionuuid and the claimant's start time (sysctl(KERN_PROC_PID)) in a new owner_identity table. The ledger moves to user_version 2. A version-1 ledger is migrated in place; its claims keep the PID-only rule.
  • A claim is stale when: (a) its boot session differs from the current one; or (b) kill(pid,0) gives ESRCH; or (c) the PID can be signalled and its start time differs from the recorded one. Unknown stamps, unreadable start times and processes that can't be signalled stay fail-closed. Malformed stamps fail the ledger closed.
  • Tests: earlier-boot claim reclaimed even with a "live" PID (and kept for same or unknown boot); reused-PID matrix (live+different start ⇒ reclaimed; same start / unknown start / unverifiable ⇒ kept); v1→v2 migration keeps legacy claims and leaves no orphan identity rows; malformed stamps rejected on write and on read; the current process stamps a valid boot session and its real start time; damaged-ledger test extended to version 3.
  • docs/STORE_RECOVERY.md updated.

Decisions taken

• Boot session plus start time, not a timeout. Both are positive evidence that the claimant is gone, so no live owner can be revoked.
• Start time is compared only when kill succeeds. On iOS other apps' and extensions' PIDs usually return EPERM, so there only the boot-session rule applies. That covers reboots, the common reuse case.
• owner_identity is a separate table rather than ALTER TABLE owner, so the exact-schema check stays strict for both tables.

How it was tested (this round)

• swiftc -parse + swift-format lint --strict (Swift 6.0.3 container). sysctl/kinfo_proc/renameatx_np and GRDB code are not compiled locally; relies on macOS CI (see below).

Known limitations & follow-ups

• Within one boot, a PID reused by a process this one may not signal (another user's, or another sandboxed app's on iOS) still withholds availability until it exits.
• Physical iOS/App Group suspension/reboot qualification remains a release gate, as documented below.

Merge order

Base of the storage stack: merge this first, then #200 (which already merges this branch), then the v0.10.0 release PR #193.

CI: macOS jobs for the new head were queued when this description was written; check the PR checks tab.

---

Original description (state at 681ec09; superseded where it conflicts with the sections above, in particular the PID-reuse paragraph):

Human-review readiness

Scoped implementation and regression coverage are complete at unchanged head 681ec09. Fresh verification confirms both required native jobs and GitGuardian successful, the expected non-blocking beta-canary skip, and no pending discussion comments, submitted reviews or unresolved inline review threads. Review readiness does not authorize merge, release or deployment.

Release qualification remains open: physical iOS/App Group suspension/expiration, foreground resume, restart/reboot and cross-process stress; a quiescent, fully upgraded/restarted app/extension/CLI/MCP cohort; and native execution of the combined repair/MCP tree. Individual CI and a textually clean combination do not qualify this rollout. Mixed-version participants are unsupported. These are documented release/device/integration gates, not incomplete scoped source or a waived required CI check.

Problem

Fresh-key recovery moved database files while old-key blobs and thumbnails stayed in the live content-addressed namespace. Identical recapture could reuse unreadable ciphertext, and maintenance could destroy the archived database's binary recovery material.

Repair

Preserve database/WAL/SHM, interrupted encryption staging and the entire blob/thumbnail directory in one private, journaled recovery generation. Resume interrupted moves before opening any new namespace; ambiguous members fail closed. Durable process-owned generation registrations pin pools and retained GRDB writers. Exclusive recovery conflicts atomically with every generation/blob owner. Plaintext or absent-file initialization retains exclusive ownership through conversion and pool/blob initialization. Authenticate/migrate before legacy blob changes. Replace the earlier lifetime flock with a metadata-only WAL coordinator: transactions and connection-close/checkpoints are fenced against iOS suspension, and no new application-owned kernel lock is retained during pool lifetime or async work.

Regression coverage added

Interrupt and resume each of five generation member moves; verify preserved bytes; reject recovery while an independent shared lease is active; recover binary content under key B, recapture the same bytes successfully, run maintenance, then reopen the preserved generation under key A and read the original payload. Added deterministic WAL idle-writer, scope conflicts, live/exited/unverifiable/reused PID policy, exact old-token release, deferred/SQLite-busy release retry, late-observer suspension rejection, versioned-ledger fail-closed and exclusive conversion tests.

Compatibility and review

The existing fresh-key plus SQLITE_NOTADB trigger is not expanded. Recovery generations are never garbage-collected. Historical per-file unreadable archives remain untouched and may require explicit recovery review. Review this preservation boundary before dependent PR200.

Exact-head validation

Published head: 681ec09
Uploaded/local staged tree: 54d9f6b70ffde30a1d2939672e5c4b90ee77699a (verified identical before publication)

• git diff --check passed locally
• Prior-head macOS compile identified GRDB automaticMemoryManagement as iOS-only. This source scopes it to the existing iOS conditional and expands the metadata Release record's semicolon formatting; ownership semantics and tests are unchanged
• make format, make lint and make test were attempted locally but Swift/Xcode is absent from this Linux executor
• Exact-head native CI 37395952984 passed strict formatter/SwiftLint (0 violations), warning checks, 1514 package tests in 235 suites, 84.2% production-source coverage above the unchanged 80% gate, StoreKit and macOS/iOS builds
• Nineteen isolated SQLite/POSIX/C-API cases passed: the prior 16 preservation/identity cases plus 3 actual installed-SQLite destructor lifetime cases. These are supplementary models, not packaged Swift/SQLCipher/Darwin product execution

Actual-connection generation lifetime

GRDB can retain closed Database/configuration objects in dispatch watchdogs, so a strong prepare callback conservatively outlived the actual connection and blocked valid recovery. Preparation now captures the generation weakly and throws if it is gone. A private random per-connection SQLite scalar function retains one ARC reference through create_function_v2 user data. Its destructor drops only that connection’s reference after the pager closes, including close_v2 zombie completion; sibling writer/readers/snapshots remain pinned. Registration failure also invokes the destructor, so there is no double release. SQLCipher builds use SQLCipher C symbols. No persisted schema or application query uses the function.

Added native tests keep a closed writer/configuration explicitly alive without over-pinning, reject reopening copied configuration after the generation is gone, and prove an independent snapshot blocks recovery beyond its originating pool’s close. The seven previous busy failures and both writer/configuration/snapshot lifetime regressions passed unchanged on this exact published head. Three installed Linux SQLite C-model cases passed for sibling release, delayed zombie destruction, and exactly-once failed registration; they do not qualify the packaged Swift/SQLCipher bridge.

Lifecycle and rollout qualification

Only a positive, bounded PID returning documented ESRCH from signal-zero validity checking is reclaimed. PID reuse/reboot collisions, permissions/sandbox failures, malformed identity and suspended owners stay fail-closed; no timeout or missing heartbeat proves death. Exact UUID release retries only known-finished operations. A reused or unverifiable stale PID can withhold availability; callers report busy rather than silently reset. Older binaries do not participate: ownership safety requires quiescing/upgrading/restarting all app/extension/CLI/MCP participants. Main schema and sealed format remain compatible, but mixed-version ownership is not qualified. The optional raw-key migration retains its separate existing opt-in/device rollout boundary. See store recovery protocol.

Independent source review cleared the revised protocol. Physical iOS/App Group app, keyboard, share/widget suspension, expiration, restart/reboot and cross-process stress remain required. Global suspension seam tests require an isolated iOS invocation. Build success alone does not establish device lifecycle safety.

Combined source review

All five repair trees plus unmerged MCP PR195 at c90e07d combine without textual conflicts in a disposable local tree. Native combined execution remains unverified. MCP optional binary content may become nil after legitimate deletion/GC; the captured row hash is never replaced by a newer body lookup. No stronger availability promise for deleted content or new public-stack dependency is inferred.

Ready for human review only. No merge/release, live Keychain, clipboard, real user library or backup access.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ExmnfUngrDYjAheGvYwMnn

---

Generated by Claude Code

[johnny4young]

Summary: This adds a durable cross-process ownership ledger (.store-ownership.sqlite) with shared and exclusive generation claims. Each actual SQLite connection pins the claim through an sqlite3_create_function_v2 destructor. Unreadable-store recovery now moves the whole generation (db, wal, shm, .encrypting, blobs) into a .unreadable-* directory under a journal that can be resumed after interruption. Migrations now run before blob encryption and the downgrade.

Verdict: the design is sound. I found no correctness bug in the moves, the journal or the claim protocol, and no lock-order deadlock: ownership mutex → lifecycle mutex is used consistently, and nothing re-enters the non-recursive NSLock. Below are risks and small issues.

🟠 Stale claims from a reused or unverifiable PID never clear

StoreProcessOwnership.processState reclaims only on ESRCH. A claim left by a process that was jetsam-killed or crashed, whose PID is later reused (very likely after a reboot, where PIDs restart), returns EPERM or success. Such a claim stays forever, or until that unrelated process exits. In this PR it blocks only exclusive work: recovery, plaintext→encrypted conversion, and first creation of an encrypted store, which also needs exclusivity because needsPlaintextConversion returns true when the file is absent. #200 builds on this and makes it far more visible. Suggestion: store a reuse-proof identity alongside the PID and treat a mismatch as exited:

• the process start time (proc_pidinfo PROC_PIDTBSDINFO pbi_start_tvsec/usec) where it is permitted, and/or
• the boot session (sysctlbyname("kern.bootsessionuuid")). A claim from a previous boot can never be live.

This changes the schema (user_version 2), so I left it to you.

🟡 Nits

• StoreGenerationRecovery.archive creates the .unreadable-* directory before the present.contains("gancho.sqlite") guard, so the no-database path leaves an empty hidden directory behind. Move the createDirectory after the guard.
• The journal is fsynced (synchronize()), but the parent directory and the moves are not. On APFS this is usually fine; consider fcntl(F_FULLFSYNC) on the directory if you want the crash-ordering claim in docs/STORE_RECOVERY.md to hold strictly.
• Opens of an existing encrypted store take a shared claim, so they are not blocked. While another process holds an exclusive claim (first creation or plaintext conversion), however, iOS extensions opening the store get busy immediately and fall back to the ephemeral store. That is expected per the doc; just make sure the keyboard and widgets show that state gracefully.
• Older recovery artifacts (gancho.sqlite.unreadable-* files created by earlier builds) are not migrated or recognized. That is harmless, but worth one line in STORE_RECOVERY.md.

Validation: there is no Swift toolchain in my environment, so this is code reading only. I could not exercise the close_v2 zombie-destructor path. CI is green.

---

Generated by Claude Code