Read the write-ahead log: Database.AccessMode - #19
Merged
Merged
Conversation
Messages keeps chat.db in WAL mode, and Database(path:) opened it with immutable=1, which makes SQLite ignore chat.db-wal. Every message committed since the last checkpoint was therefore invisible; checkpoints happen every few MB of writes, so the newest messages lagged by hours and then appeared in batches. Database.AccessMode chooses how the file is read: .live (mode=ro, the -wal/-shm companions are read; a read-only grant on the directory is enough), .immutable (the previous behaviour, for a grant that covers chat.db alone) and .automatic (default: live, falling back to immutable when the first read cannot open the companions). init(path:) is unchanged for existing callers, accessMode reports the outcome, and a 1 s busy timeout covers the locks a live connection shares with Messages. Tests build a WAL-mode fixture with one checkpointed row and one still in the log and check the three modes, including the fallback. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0187kUP2mjhwMfJkwRbP5DG7
A sandboxed app that was granted the Messages folder reads it read-only, so SQLite has to work with a shared-memory index it cannot write to. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0187kUP2mjhwMfJkwRbP5DG7
There was a problem hiding this comment.
🟡 Changes recommended
The unresolved query-step error handling can silently return incomplete data.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds WAL-aware database access while preserving immutable fallback behavior.
Changes:
- Introduces live, immutable, and automatic access modes.
- Adds WAL-mode tests and fallback coverage.
- Documents stale-message behavior and configuration.
File summaries
| File | Review |
|---|---|
Tests/iMessageTests/AccessModeTests.swift |
Tests WAL visibility and fallback behavior. |
Sources/iMessage/Database.swift |
Implements access modes and busy timeout. A moderate issue remains: query stepping must throw unless it ends with SQLITE_DONE, preventing silent empty or partial results on SQLITE_BUSY or other errors. |
README.md |
Documents WAL-related message lag. |
Review details
Suppressed comments (1)
Sources/iMessage/Database.swift:153
canReadcollapses every SQLite failure intofalse, so.automaticsilently switches to immutable not only when the WAL companions are inaccessible, but also after transientSQLITE_BUSY, corruption, or an I/O error. In those cases this converts a real failure into a successful connection that can return stale checkpointed data—the exact behavior this mode is intended to avoid. Preserve the SQLite result/extended error code and fall back only for the specific companion-access errors; propagate all other read failures.
if Database.canRead(handle) {
- Files reviewed: 3/3 changed files
- Comments generated: 1
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+105
to
+106
| // A live connection shares locks with Messages; wait briefly instead of failing. | ||
| sqlite3_busy_timeout(handle, 1000) |
The step loop treated every non-row status as a clean end of results, so a step error, or SQLITE_BUSY on a live connection that loses a race with Messages, silently returned partial data. Surface it as Error.queryError instead. Make execute and Bindable internal so the case can be tested directly.
Owner
|
Thanks, @patp. Reading the WAL by default with an immutable fallback is exactly what this needed, and the docs explain the trade-off well. I pushed one follow-up: the step loop now throws on anything other than |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Messages keeps
chat.dbin WAL mode.Database(path:)opens it withimmutable=1, which tells SQLite to ignorechat.db-wal, so every message committed since the last checkpoint is invisible. Checkpoints happen every ~1000 pages (4 MB) of writes: on a typical account the newest messages lag by hours and then appear in a batch. Observed today through iMCP: nothing newer than 12:23 whilechat.db-walheld eight later messages; they all appeared at 17:30, when Messages checkpointed.This adds
Database.AccessMode:.live—mode=roonly; SQLite reads the-wal/-shmcompanions. A read-only grant on the directory is enough (SQLite ≥ 3.22 handles a read-only-shm)..immutable— the previous behaviour, for callers whose grant coverschat.dbalone..automatic(default) — opens live, and falls back to immutable when the first read cannot open the companions.init(path:)keeps working unchanged (mode:defaults to.automatic),accessModereports what was chosen, and a 1 s busy timeout covers the locks a live connection now shares with Messages. The tests build a WAL-mode fixture with one checkpointed row and one still in the log and check the three modes, including the fallback (companions made unreadable). The README gains a FAQ entry.Companion change in iMCP: mattt/iMCP#198
🤖 Generated with Claude Code
https://claude.ai/code/session_0187kUP2mjhwMfJkwRbP5DG7