Skip to content

Messages: grant the Messages folder so recent messages in the write-ahead log are visible - #198

Merged
mattt merged 1 commit into
mattt:mainfrom
patp:messages-folder-access
Sep 17, 2026
Merged

mattt merged 1 commit into
mattt:mainfrom
patp:messages-folder-access

Conversation

@patp

@patp patp commented Sep 1, 2026 •

Copy link
Copy Markdown
Contributor

Problem

chat.db is a WAL-mode SQLite database. Messages.app appends new rows to chat.db-wal and only folds them into the main file at a checkpoint (roughly every 4 MB of changes). iMCP opens the database through Madrid with immutable=1, which ignores the write-ahead log, and its security-scoped bookmark covers the single file chat.db, so the sandbox could not read chat.db-wal / chat.db-shm anyway. The result is that messages_fetch silently misses everything received since the last checkpoint. On a quiet machine that is hours, sometimes days, of messages.

Fix

  • The Messages access panel now asks for the ~/Library/Messages folder instead of the chat.db file (folder-only picker, preset to ~/Library, only the Messages folder is selectable). The bookmark is stored read-only exactly as before.
  • Database access is wrapped in a small DatabaseAccess value that keeps the security scope open for the whole tool call. SQLite opens the -wal / -shm companions lazily on the first read, so stopping the scope right after Database(path:) would break WAL reads.
  • With a folder grant the database is opened in Madrid's .live mode (mode=ro, WAL honoured). An existing single-file grant keeps working in .immutable mode (today's behaviour), with one warning per launch in the log.
  • Tool calls never prompt for the upgrade: a modal alert raised from a tool call parks the whole main-actor server behind a dialog that nobody may be there to answer (observed on a headless Mac). Instead the service reports itself as not fully activated while the grant covers chat.db alone, so switching Messages off and on in the menu offers the folder; declining keeps the old grant and leaves the service on.
  • README: Messages section describes the folder grant and why.

Dependency

Uses Database.AccessMode from Madrid 0.5.0 (mattt/Madrid#19), which main pins since #201. Rebased onto main on 2026-09-03; the branch no longer touches the package pin.

Testing

  • Builds with Xcode 26.6 on macOS 26 (Debug, ad-hoc signed) after the rebase; swift format lint --strict clean.
  • Verified on a Mac where chat.db was last checkpointed at 17:30: with the folder grant, messages_fetch returns messages received at 17:39 and 17:45, which the single-file grant did not see (and still would not, the main file is unchanged). That build has been serving a sync client since 2026-09-01 with no lag behind Messages.app.
  • The Madrid side is covered by tests that create a WAL database with un-checkpointed rows and read it through read-only -wal / -shm companions, the situation under the sandbox.

🤖 Generated with Claude Code

https://claude.ai/code/session_0187kUP2mjhwMfJkwRbP5DG7

@mattt

mattt commented Sep 4, 2026

Copy link
Copy Markdown
Owner

@patp Same as #197: Madrid 0.5.0 is in main now, so this can come off the fork branch whenever you're ready.

The service asked for chat.db alone. Messages keeps that database in WAL
mode: every new message is committed to chat.db-wal first and only reaches
chat.db when SQLite checkpoints the log, every few MB of writes. With a
grant on the single file the log was unreadable, so Madrid opened the
database with immutable=1 and messages_fetch could not see anything newer
than the last checkpoint, often hours behind, then everything at once.

The file picker now asks for ~/Library/Messages (directories only,
constrained to a folder named Messages) and the bookmark covers the
folder, which lets SQLite read chat.db together with its -wal and -shm
companions (Madrid's Database.AccessMode.live). The security scope now
stays open until the fetch is done, since SQLite opens the log lazily.

A bookmark stored by an earlier version (chat.db alone) keeps working in
immutable mode; the first fetch of a launch offers to re-select the folder,
and declining is remembered for that launch.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0187kUP2mjhwMfJkwRbP5DG7
@patp
patp force-pushed the messages-folder-access branch from e8355c3 to 5ce2cbc Compare September 4, 2026 02:05
@patp
patp marked this pull request as ready for review September 4, 2026 02:05
@patp

patp commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Same here: rebased onto main against Madrid 0.5.0, package pin dropped, only App/Services/Messages.swift and the README change. Builds and lints clean on Xcode 26.6. Marked ready for review.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Cancellation handling and warning-state synchronization must be fixed before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Updates Messages access to include SQLite WAL files so recent messages remain visible.

Changes:

  • Requests read-only access to the Messages folder.
  • Keeps security scope active during reads and supports legacy grants.
  • Documents WAL behavior and migration.

Required fixes:

  • Preserve legacy grants when the upgrade folder picker is cancelled.
  • Synchronize the shared one-time warning flag to prevent concurrent races.
File summaries
File Description
README.md Documents folder access and WAL visibility.
App/Services/Messages.swift Adds folder grants, live database access, and legacy fallback handling.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +52 to +53
if !warnedAboutFileGrant {
warnedAboutFileGrant = true
}

let selectedURL = try await showFilePicker()
let selectedURL = try await showFolderPicker()
@mattt

mattt commented Sep 17, 2026

Copy link
Copy Markdown
Owner

Hi @patp. Thanks, and sorry for the wait on this one. I tested it and it works great. I took the same approach for the call history database in #220. Merging now.

@mattt
mattt merged commit 98b5d47 into mattt:main Sep 17, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants