STA-5: Add Store::open_read_only and a shared scan lock

[hexnyl]

Linear: STA-5

StatsAI Desktop reads ~/.statsai/statsai.sqlite while the CLI or daemon may be scanning it. Desktop query connections must never write, and two scanners must never run at once. Both additions are generic, so the CLI can use them on their own.

Read-only store

Store::open_read_only(path) -> Result<ReadStore, ReadOnlyOpenError>

• How it opens: SQLITE_OPEN_READ_ONLY, which is what mode=ro sets; it is not immutable, so WAL commits stay visible. PRAGMA query_only = ON. No checkpoint on close. Defensive mode, a busy timeout, and the same cache and mmap settings as the writer. It shares its open flags with the snapshot reader.

• What it never does: create the file, migrate, backfill, reprice, chmod, or run PRAGMA optimize. Reading the schema version only runs SELECTs, even on a rollback-journal store.

• Typed errors (ReadOnlyOpenError, non-exhaustive):

  • NotFound
  • SchemaTooOld { found, expected }
  • SchemaTooNew { found, expected }
  • Open { reason }, whose message includes the cause.

  The version is read from schema_migrations; CURRENT_SCHEMA_VERSION is 31.

• The schema is checked on every read, not just at open. Each read runs in a read transaction whose first statement re-reads the version, so the check and the read see one snapshot. A store that a newer statsai migrated after the open is refused with SchemaTooNew, inside the anyhow::Error, instead of being misread. A busy or I/O error on that statement stays the read's own error. check_schema() runs the same check without reading anything else.

• Why a wrapper type instead of a plain Store: ReadStore exposes reads only, and each one forwards to the existing Store method. Some Store "reads" also do upkeep: all_sync_rollup_summaries can rebuild stale rollups. ReadStore returns what is stored and leaves that upkeep to the next writer. A write therefore fails to compile instead of failing at runtime.

• Available reads:

  • data_version, with_read_snapshot, schema_version, check_schema
  • session_rollups_in_period, session_stats_in_period
  • all_sync_rollup_summaries, daily_rollups_between, cache_report
  • quota_status, quota_windows, quota_observations, weekly_reset_anchor
  • list_sources, list_accounts, list_source_account_assignments, list_subscriptions, list_sync_states
  • applied_pricing_ruleset_version, event_count, token_total

Scan lock

The lock is scan.lock beside the database (~/.statsai/scan.lock by default; --store is respected). It uses std's File::try_lock: flock on Unix and LockFileEx on Windows. Toolchain 1.97.1 already has it, so no new dependency is needed. The code lives in statsai-store so the daemon crate can use it, and the statsai lib re-exports it.

• API:

  • try_acquire_scan_lock(path) -> Result<Option<ScanLockGuard>>
  • acquire_scan_lock_with_timeout(path, timeout)
  • scan_lock_path(store_path)
  • statsai::default_scan_lock_path()

  Dropping the guard releases the lock. The lock file is never truncated or deleted. A newly created directory gets mode 0700 and the file 0600.

• One lock file per store, however the path is written:

  • scan_lock_path follows a symlink at the store path, even one whose target doesn't exist yet.
  • It resolves the longest existing part of the directory and appends the parts not yet created.
  • So links, /var → /private/var on macOS, and creation order all lead to the same file. Hard links are not detected.

• statsai scan and statsai sync:

  • They take the lock before the store opens and hold it until the command returns. Only sync --status, --verify and --reset-remote --dry-run skip it, because they only read.
  • Bad flags and an unknown --sink are rejected before any wait.
  • If another scanner holds the lock, the command waits up to 5 s, then prints Another statsai scan is running (lock: …); skipping. and exits 0, as STA-5 asks. For a sync the message ends skipping this sync. Nothing was sent., and its --include-*/--exclude-* preference flags are still saved.
  • sync --reset-remote fails instead, having deleted nothing.

• statsai daemon --watch:

  • It takes the lock for each background pass only, never while it waits for file events.
  • While another process holds it, changed paths stay queued and the pass retries after 1 s.
  • After each pass it leaves the lock free for 200 ms, so a waiting scan or sync gets a turn.

• Advisory: if the lock file can't be opened or locked, the CLI warns and runs without it, and the daemon warns once and keeps scanning.

• Breaking change: statsai_daemon::watch_and_serve gains a scan_lock: Option<&Path> parameter. The CLI is its only caller.

README: a note under "Scanning and reporting", and a new subsection, "Reading the store from another process". There is no CHANGELOG file.

Tests

• statsai-store/tests/read_only_store.rs:
  • A missing store returns NotFound and creates nothing.
  • A fresh store opens and reads empty.
  • A current v31 store with data reads the same as the writer. The file is byte-for-byte unchanged afterwards, its permissions are untouched and it is not repriced.
  • An older schema returns SchemaTooOld, including a store that predates schema tracking (version 0). A newer schema returns SchemaTooNew.
  • A non-database file returns Open, and the message carries the cause.
  • The schema version is read without writing, even from a rollback-journal store.
  • A migration after the open is reported by check_schema and refused by every read, including inside with_read_snapshot.
  • Reads succeed during an open write transaction and see the data after commit.
  • data_version changes after a commit, and with_read_snapshot keeps its point in time.
  • A reader thread keeps reading while a writer thread commits 25 times.
• statsai-store/tests/scan_lock.rs:
  • Busy and timeout behaviour.
  • A waiter gets the lock when it's released.
  • 6 threads, each with its own open file, never hold the lock at the same time.
  • Cross-process contention: the test binary re-runs itself as a child that holds the lock.
• Unit tests:
  • The read-only connection refuses INSERT, CREATE, CREATE TEMP and PRAGMA writes.
  • A busy store is the read's own error, not a schema error. A failed open names the path once.
  • Lock path:
    • the lock sits beside the store;
    • it's the same before and after the store's directory exists, through a linked directory;
    • a symlink to the store, existing or dangling, shares the store's lock;
    • permissions.
  • Daemon:
    • a pass is deferred while the lock is held and holds the lock while it scans;
    • a failed pass releases it;
    • an unusable lock doesn't stop scanning.
  • CLI: which commands take the lock, and an unusable lock is reported rather than fatal.
• statsai/tests/scan_lock_cli.rs:
  • While the lock is held, scan and sync skip with exit 0 and don't create a store.
  • Invalid flags and an unknown sink still fail while the lock is held.
  • A skipped sync still records a preference opt-out.
  • A remote reset fails rather than skips.
  • scan waits out a short hold and then runs.

Locally on Linux: cargo fmt --check, clippy --workspace --all-targets -D warnings and cargo test --workspace (1,602 tests) pass. CI passed on Linux and macOS at a1ee0d9.

Review round

The automated review posted 12 findings, and all are addressed in 45cbdd2. Each new behavioural test fails on the commit before its fix.

A verification pass then checked every finding against the fix and hunted for regressions. That led to four more commits:

• 107e501:
  • A busy --reset-remote fails rather than skips.
  • Every read re-checks the schema in its own snapshot. Before, this was an opt-in call.
• 6c5e77a:
  • Dropped a debug_assert that a turned-away sync saving its preferences could trip.
  • The lock docs list every exemption.
• a1ee0d9: one lock path regardless of creation order. This fixes the macOS CLI test.
• f10f02f:
  • Only schema mismatches are typed on reads.
  • The Open message names the path once.
  • The docs no longer promise to catch a migration that is still running.
  • An unknown --sink is rejected before the wait.

Limitations

• WAL files: if no writer has the store open, a reader creates empty -wal/-shm files beside it. In a directory it cannot write, it fails with Open unless those files exist. This is documented on ReadStore.
• No upkeep on read: a reader returns stored rollups as they are and never reprices. The next writer brings both up to date.
• A migration still running: a migration records its version after it changes the schema, so a read in that window isn't refused.
• Skipped syncs exit 0: STA-5 asks for that. The menubar's Upload shows no message for a skip, though "Sync Now" stays in the menu because nothing was sent.

Follow-ups (not in this PR)

• Bring import, conversation collect and source remove --delete-data under the scan lock.
• Show a menubar notice when an Upload is skipped for a running scan.
• Apply each migration and record its version in one transaction, so the per-read check also covers a migration in progress.
• The menubar's fetch_snapshot uses Store::open; consider the read-only path.
• Declare rust-version (File::try_lock needs 1.89+).

🤖 Generated with Claude Code

https://claude.ai/code/session_01G3EZeUTrrXQYg1NwFdgV3K