fix(watcher): debounce fsnotify events + atomic index/links updates

[jacky1967]

Summary

Fixes two race conditions in the Obsidian backend's file watcher that surface under rapid concurrent writes (typical of multi-writer scenarios where multiple clients write to the vault through the MCP layer):

1. Lock split between index and backlinks updates — handleEvent performed indexFile() and BuildBacklinks() as two separate lock acquisitions, leaving a brief window where readers could see a page in c.pages whose backlinks did not yet reflect the new content.
2. Atomic temp+rename surfaces a transient missing page — macOS/iCloud sync and atomic write-to-temp+rename (used internally and by external editors) generates Remove followed quickly by Create events on fsnotify (~10 ms gap). During that window the page was temporarily absent from the index.

Background / motivation

This bug surfaced while wiring up the Hermès Agent project to use graphthulhu as a single-writer for an Obsidian vault. Three independent producers (LLM via stdio, plugin via HTTP, cron script via HTTP) share a single graphthulhu subprocess in Streamable HTTP mode. Burst writes (a Scout report followed by a cascade of dispatch writes, or Gmail attachments uploaded in sequence) reliably triggered the second race; the first one was visible as occasional missing backlinks in tests.

The fix itself is generic and does not depend on Hermès — it applies to any setup where multiple clients write to the vault through the MCP layer, or where external editors save with atomic temp+rename.

Solution

Two complementary changes:

1. Atomic helpers (single lock for index + links)

vault/vault.go adds two helpers that acquire c.mu once and run the index update + rebuildLinksLocked() together:

• indexFileWithLinks(relPath, content, info) — replaces indexFile() + BuildBacklinks() in the watcher path.
• removePageWithLinks(lowerName) — replaces removePageFromIndex() + BuildBacklinks().

Existing BuildBacklinks() and removePageFromIndex() are preserved for callers that need them separately (e.g. Reload()).

2. Debounce fsnotify events (per-path coalescer)

New vault/watcher_debounce.go (~120 lines):

• scheduleEventResolution(absPath, relPath) — schedules a time.Timer per path. If a new event arrives for the same path, the existing timer is reset.
• resolveFileState(absPath, relPath) — fires after debounceDelay (default 100 ms) and reads disk state: file present → indexFileWithLinks; file absent → removePageWithLinks. Race-free with respect to atomic rename: regardless of the Remove/Create sequence received, the final state always matches what is on disk at resolution time.

Client struct gains pendingMu, pendingEvents map[string]*pendingEvent, debounceDelay time.Duration (configurable for tests; defaults to defaultDebounceDelay = 100 * time.Millisecond).

handleEvent is simplified: it now delegates Create/Write/Remove/Rename events to scheduleEventResolution (single switch arm).

The 100 ms delay is short enough not to noticeably delay genuine deletes or writes, and long enough to absorb the typical macOS/iCloud atomic-rename window (~10 ms observed on iCloud-synced vaults).

Test plan

5 new tests in vault/watcher_debounce_test.go:

• TestDebounce_AtomicRenameNoMissingPage — page remains accessible during the race window (prior code: race observable; this PR: race eliminated).
• TestDebounce_BurstWritesCoalesce — 10 rapid writes → 1 final reindex (not a cascade of 10).
• TestIndexFileWithLinks_Atomic — 50 reindex calls + concurrent reader → 0 inconsistencies (concurrent reader cannot observe a state where the page exists but backlinks don't reflect it).
• TestRemovePageWithLinks_Atomic — analogous for removal.
• TestScheduleEventResolution_Coalesce — 5 schedules → 1 pending timer → 1 resolve.

vault/vault_test.go TestWatchFile* sleeps adjusted from 100 ms → 250 ms (debounce 100 ms + resolve margin).

Full Go test suite passes (vault, tools, parser, graph, types).

End-to-end smoke against a running MCP HTTP service: 5 burst write_raw_page calls followed by immediate get_page on each → 5/5 accessible, 0 inaccessible. No regression observed in production usage over the past 24h.

Notes

• No public API change; existing callers of BuildBacklinks() and removePageFromIndex() keep working.
• flushPendingEventsForTest() is exposed as a non-public helper for unit tests that need synchronous flushing without time.Sleep.
• The fix targets the Obsidian backend specifically (where fsnotify-driven re-indexing matters); Logseq backends that drive the index through DataScript transactions are unaffected.

[skridlevsky]

Thanks for this, @jacky1967! Solid diagnosis and the architecture is clean.

Rebased onto current main and added a small polish commit on top: translated comments to English, fixed a subtle delete-race in scheduleEventResolution (closure could delete a newer pending event after timer fire), added timer cleanup to Close(), and bumped debounceDelay in two timing-dependent tests for CI flake protection.

Tests green, merging. Really appreciate the work.