Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
f6f3f3c
fix(trash): never sweep a note Obsidian trashed under a recorded name
aliasunder Oct 9, 2026
098823a
fix(review): list the sweep's identity gate in SECURITY.md and word i…
aliasunder Oct 9, 2026
b75f7e9
docs(trash): state why the identity leaves out the device number
aliasunder Oct 9, 2026
d0f012d
fix(trash): create the trash folder under the trash-domain lock
aliasunder Oct 9, 2026
60a6442
style: clarify trash sweep and trash move names, comments, and docs
aliasunder Oct 9, 2026
d21cd45
fix(trash): also match the inode change time before sweeping a file
aliasunder Oct 9, 2026
7bd21af
test(trash): cover the sweep's remaining missing-file paths and pin e…
aliasunder Oct 9, 2026
1ecdad8
fix(trash): bound the change time by the trash moment instead of stor…
aliasunder Oct 9, 2026
643dd3f
fix(trash): compare the whole row in the orphan purge's re-read
aliasunder Oct 9, 2026
0e08d86
docs(trash): note the case-variant sibling a stale trash row can belo…
aliasunder Oct 9, 2026
b5f9fd9
docs(trash): state the change-time limit in the sweep's docstring
aliasunder Oct 9, 2026
f9895e0
style: clarify the trash sweep's change-time names and comments
aliasunder Oct 9, 2026
f9d45f2
test(trash): tighten the change-time limit tests
aliasunder Oct 9, 2026
16e431e
fix(trash): keep a failed folder prune from ending the retention sweep
aliasunder Oct 9, 2026
9f7603e
docs(trash): state that pruneEmptyParents throws on an unsafe path
aliasunder Oct 9, 2026
c75b5ab
docs(trash): state that a re-trash is caught only where renames move …
aliasunder Oct 9, 2026
84e7c9d
docs(contributing): drop the unfinished template sync sentence
aliasunder Oct 9, 2026
a4f3c07
refactor(trash): name the sweep's file check for the verdict it returns
aliasunder Oct 9, 2026
814600a
docs(trash): state that a case-only sibling loses its row on a case-s…
aliasunder Oct 9, 2026
7d60ac0
fix(trash): keep a failed row drop from ending the retention sweep
aliasunder Oct 9, 2026
20c40d7
fix(trash): keep a failed row drop from ending the orphan purge
aliasunder Oct 9, 2026
69bca46
test(trash): prove the unlink ran when a vanished file's row drop fails
aliasunder Oct 9, 2026
958427a
docs(trash): say how the next sweep drops a row an unlink left behind
aliasunder Oct 9, 2026
c9e4b8e
style(trash): name the row-drop helper for its success result and giv…
aliasunder Oct 9, 2026
71a7331
test(trash): require exactly one warning when a folder prune fails
aliasunder Oct 9, 2026
1dd4616
test(trash): require exactly one warning in each sweep and purge warn…
aliasunder Oct 9, 2026
450b848
fix(trash): keep a failed row re-read from ending the sweep or the purge
aliasunder Oct 9, 2026
9287f2d
test(search): assert the whole trash entry survives a vault rebuild
aliasunder Oct 9, 2026
5f47b7b
test(trash): assert whole kept rows, and rebuild a day after recording
aliasunder Oct 9, 2026
89bf9dd
docs(trash): say why the orphan purge treats a row it cannot re-read …
aliasunder Oct 9, 2026
6b3e154
docs(trash): state the window between the identity read and the unlink
aliasunder Oct 9, 2026
9d1adc8
docs(trash): say which gate drops a row whose parent folder is now a …
aliasunder Oct 9, 2026
413b538
docs(trash): say in place why every delete passes the stale-row clear
aliasunder Oct 10, 2026
a6318b2
docs(trash): trim the trash sections of ARCHITECTURE.md and SECURITY.…
aliasunder Oct 10, 2026
25c13c6
docs(security): describe the trash sweep's guards without code-level …
aliasunder Oct 10, 2026
18b9679
docs(trash): restore why the delete log carries trashOption
aliasunder Oct 10, 2026
14a6459
docs(trash): shorten ARCHITECTURE.md's trash handling to the safeguar…
aliasunder Oct 10, 2026
c958ff5
docs(trash): stop ARCHITECTURE.md promising every trash move is recorded
aliasunder Oct 10, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 11 additions & 26 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -1189,39 +1189,24 @@ Docker hardening, and durability seatbelts above.
`withExclusiveMultiFileLock` (all-or-nothing fail-fast) acquires all
locks in one synchronous tick — used by note-mover, which must lock the
source, destination, and every backlink source for the whole
read-plan-write span. The trash move, the orphan purge, and the
retention sweep share one serializing key for the whole `.trash/`
read-plan-write span. The trash move and the cleanup in
`trash-sweeper.ts` share one serializing key for the whole `.trash/`
domain.
- **Properties-block guard** (`frontmatter.ts`): every rewrite refuses a
note whose properties block it cannot keep, and every write's result is
checked before it reaches disk, so no write silently drops a block's
properties.
[Vault read/write](#vault-readwrite) gives the full rules and the
repair path.
- **Trash claim loop** (`moveNoteToTrash` in `vault-filesystem.ts`): a
delete under Obsidian's `system` (default) or `local` trash setting
moves the note into `.trash/`. Each candidate name is claimed with an
exclusive create, so an existing trash copy is never overwritten.
`system`-mapped moves are recorded in the index DB's `trash_entries`
table. Recording is fail-open — a failed row write logs a warning, the
delete still succeeds, and an unrecorded entry is never swept. The
table's primary key is the case-folded path, so a case alias replaces
its stale row instead of leaving one that could purge the wrong sibling
on a case-insensitive mount.
- **Recorded trash bookkeeping** (`trash-sweeper.ts`): two row-driven
operations (neither walks the folder):
- **Orphan purge** — runs once at boot regardless of
`TRASH_RETENTION_DAYS`. Drops rows whose `.trash/` entry no longer
exists on disk (uses lstat, so dangling symlinks are kept), so
manual emptying or `retention=none` never leaves
unbounded stale rows.
- **Retention sweep** — runs at startup and daily. Purges recorded
entries older than `TRASH_RETENTION_DAYS`. Each unlink is
double-guarded: the resolved path and the parent directory's
realpath must both sit inside `.trash/`, so a corrupted row or a
directory symlink cannot reach live notes.
- Both share a serializing lock with the trash move and re-read each
row under it before acting, so neither operates on a stale snapshot.
- **Trash handling** (`moveNoteToTrash` in `vault-filesystem.ts`,
`trash-sweeper.ts`): a delete under Obsidian's `system` (default) or
`local` trash setting moves the note into `.trash/` and never overwrites
an existing trash copy. The server records `system` moves with the
trashed file's identity, and its daily retention sweep deletes only
those recorded files once they pass `TRASH_RETENTION_DAYS`. Before
deleting, it checks that the file still sits inside `.trash/` and is
still the unchanged file it trashed, so a corrupted record, a symlink,
or a note Obsidian later trashed under the same name is left alone.
- **Verify-then-preflight-then-commit move** (`note-mover.ts`): under the
lock, `moveNote` first scans the filesystem for backlinks the search
index missed (closing a lag race); then reads every affected file and
Expand Down
5 changes: 2 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,9 +102,8 @@ development. A test fails if the versions drift.
- Test: `npm test` includes `cli/src/**/*.test.ts`
- Try it: `node cli/dist/bin.js init --help`

**Template sync rule:** `cli/templates/` holds verbatim copies of
The optional env blocks in `cli/src/env.ts` are derived from `deploy/*/.env.example`.
If you change any `.env.example`, run
**Env block sync:** The optional env blocks in `cli/src/env.ts` are derived
from `deploy/*/.env.example`. If you change any `.env.example`, run
`npm run sync:cli-env-blocks` in the same PR — drift tests fail CI otherwise.

**Publishing:** CLI releases are explicit and independent of server releases —
Expand Down
19 changes: 13 additions & 6 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,12 +67,13 @@ mechanism-level detail.
- Memory file names reject `/` and `\` — prevents `../../outside`-style
escapes from the memory directory — and leading dots, which would
create hidden files
- The trash retention sweep deletes only files it previously recorded.
Every removal passes two gates — the recorded path must resolve inside
`.trash/`, and the file's parent directory must realpath-resolve inside
it too. A corrupted or hand-edited index row, or a directory symlink
planted in `.trash/`, is skipped with a warning instead of followed to
a live note
- The trash retention sweep deletes only files it previously recorded,
and only inside `.trash/`. It skips with a warning, instead of
following to a live note:
- a recorded path that points outside `.trash/`, from a corrupted or
hand-edited index row
- a file reached through a folder symlink planted in `.trash/` that
leads elsewhere

### Hidden paths

Expand Down Expand Up @@ -159,6 +160,12 @@ mechanism-level detail.
synced `.obsidian/` settings), the container refuses to start,
preventing the sync engine from interpreting the empty vault as mass
local deletions
- Trash sweep identity gate: the retention sweep deletes a recorded
`.trash/` file only while it still has the identity recorded when the
server trashed it and has not changed since, so a different note that
Obsidian later trashed under a recorded name is kept. It cannot catch a
note restored by hand and trashed again within a minute, or a file
swapped in during the instant between the check and the delete
- Memory shrink guard: refuses writes that would remove >50% of a file's
bytes — defense-in-depth against bugs that would silently erase most of
a memory file
Expand Down
3 changes: 2 additions & 1 deletion cli/src/env.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,8 @@ MAX_PDF_RENDER_PAGES=5
# moved to .trash under that setting are cleaned up — notes you or Obsidian
# put there are never touched. The server tracks its trashed notes even while
# this is none, so switching none to a number cleans up the older backlog on
# the next restart.
# the next restart. Notes trashed by version 0.54.11 or earlier are kept;
# empty .trash by hand to remove them.
TRASH_RETENTION_DAYS=30

# Your IANA timezone — affects daily note resolution and memory timestamps.
Expand Down
3 changes: 2 additions & 1 deletion deploy/local/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,8 @@ MAX_PDF_RENDER_PAGES=5
# moved to .trash under that setting are cleaned up — notes you or Obsidian
# put there are never touched. The server tracks its trashed notes even while
# this is none, so switching none to a number cleans up the older backlog on
# the next restart.
# the next restart. Notes trashed by version 0.54.11 or earlier are kept;
# empty .trash by hand to remove them.
TRASH_RETENTION_DAYS=30

# Your IANA timezone — affects daily note resolution and memory timestamps.
Expand Down
8 changes: 5 additions & 3 deletions deploy/local/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -307,9 +307,11 @@ links):
set `TRASH_RETENTION_DAYS` in `.env` to change the window, or
`TRASH_RETENTION_DAYS=none` to keep them forever. Only notes the server
moved there are cleaned up; anything you or Obsidian put in `.trash` is
never touched. To reclaim space sooner, empty the `.trash` folder
yourself (it's hidden — press Cmd+Shift+. in Finder, or Ctrl+H in most
Linux file managers, to show it).
never touched. Notes the server moved there with version 0.54.11 or
earlier also stay. To reclaim space sooner, or to clear those older
notes, empty the `.trash` folder yourself (it's hidden — press
Cmd+Shift+. in Finder, or Ctrl+H in most Linux file managers, to show
it).
- **"Move to Obsidian trash (.trash folder)"** — a deleted note moves to
the same `.trash` folder and stays there until you empty it, exactly as
in Obsidian.
Expand Down
13 changes: 7 additions & 6 deletions src/vault-mcp/mcp-core/tools/vault-crud-tools.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1135,12 +1135,13 @@ Returns: ${deleteNoteReturns} Notes how many empty folders were pruned when any
// Only "system" moves are swept later, so only they are recorded;
// "local" is Obsidian's keep-forever trash.
recordTrashEntry: trashOption === "system" ? search.recordTrashEntry : undefined,
// The retention sweep deletes the file at each expired
// trash_entries row's path, and a row can outlive its file when
// .trash/ is emptied by hand. A move that does not record clears
// the row at its landed path, so the sweep cannot remove the new
// file, such as a "local" note meant to be kept. A "none" delete
// lands nothing in .trash/ and never calls it.
// Passed for every option, not only "system":
// - Emptying .trash/ by hand leaves rows behind, so a stale row
// can sit at the path this delete lands on.
// - A move the server does not record ("local", or a "system"
// move whose record failed) drops that row, so the retention
// sweep never deletes the note just trashed.
// - A "none" delete lands nothing in .trash/ and never calls it.
clearStaleTrashEntry: search.deleteTrashEntry,
},
reqLogger,
Expand Down
127 changes: 107 additions & 20 deletions src/vault-mcp/search/__tests__/search-index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8270,10 +8270,16 @@ describe("file content vector embeddings", () => {
})

describe("trash entries (retention-sweep bookkeeping)", () => {
/** A stand-in identity — the store keeps whatever string it is given. */
const SAMPLE_FILE_IDENTITY = "4096:12:1791557833301645709"

it("listAllTrashEntries returns every recorded entry", () => {
const trashIndex = createSearchIndex(":memory:")
trashIndex.recordTrashEntry(".trash/a.md")
trashIndex.recordTrashEntry(".trash/sub/b.md")
trashIndex.recordTrashEntry({ trashPath: ".trash/a.md", fileIdentity: SAMPLE_FILE_IDENTITY })
trashIndex.recordTrashEntry({
trashPath: ".trash/sub/b.md",
fileIdentity: SAMPLE_FILE_IDENTITY,
})

const allEntries = trashIndex.listAllTrashEntries()

Expand All @@ -8287,20 +8293,73 @@ describe("trash entries (retention-sweep bookkeeping)", () => {
expect(trashIndex.listAllTrashEntries()).toEqual([])
})

it("round-trips a recorded entry through get and list", () => {
it("round-trips a recorded entry, identity included, through get and both lists", () => {
vi.useFakeTimers()
onTestFinished(() => {
vi.useRealTimers()
})
const trashTime = DateTime.fromISO("2026-01-01T00:00:00Z")
vi.setSystemTime(trashTime.toMillis())
const trashIndex = createSearchIndex(":memory:")
trashIndex.recordTrashEntry(".trash/Notes/gone.md")
trashIndex.recordTrashEntry({
trashPath: ".trash/Notes/gone.md",
fileIdentity: SAMPLE_FILE_IDENTITY,
})
const expectedEntry = {
trashPath: ".trash/Notes/gone.md",
trashedAt: trashTime.toUnixInteger(),
fileIdentity: SAMPLE_FILE_IDENTITY,
}

const entry = trashIndex.getTrashEntry(".trash/Notes/gone.md")
expect(entry?.trashPath).toBe(".trash/Notes/gone.md")
expect(trashIndex.getTrashEntry(".trash/Notes/gone.md")).toEqual(expectedEntry)
expect(trashIndex.listAllTrashEntries()).toEqual([expectedEntry])
// Every recorded entry is "expired" against a cutoff after its stamp.
const listed = trashIndex.listExpiredTrashEntries((entry?.trashedAt ?? 0) + 1)
expect(listed.map((listedEntry) => listedEntry.trashPath)).toEqual([".trash/Notes/gone.md"])
expect(trashIndex.listExpiredTrashEntries(trashTime.toUnixInteger() + 1)).toEqual([
expectedEntry,
])
})

it("adds the file_identity column to a pre-existing trash_entries table, leaving old rows without one", async () => {
const dir = await mkdtemp(join(tmpdir(), "warm-db-"))
onTestFinished(() => rm(dir, { recursive: true }))
vi.useFakeTimers()
onTestFinished(() => {
vi.useRealTimers()
})
const trashTime = DateTime.fromISO("2026-01-01T00:00:00Z")
vi.setSystemTime(trashTime.toMillis())
const dbPath = join(dir, "search.db")
// Simulate a database file created before the file_identity column.
const legacyDb = new Database(dbPath)
legacyDb.exec(`
CREATE TABLE trash_entries (
trash_key TEXT PRIMARY KEY, trash_path TEXT NOT NULL, trashed_at INTEGER NOT NULL
);
INSERT INTO trash_entries VALUES ('.trash/old.md', '.trash/Old.md', 1700000000);
`)
legacyDb.close()

const warmIndex = createSearchIndex(dbPath)
warmIndex.recordTrashEntry({ trashPath: ".trash/new.md", fileIdentity: SAMPLE_FILE_IDENTITY })

expect(warmIndex.getTrashEntry(".trash/Old.md")).toEqual({
trashPath: ".trash/Old.md",
trashedAt: 1700000000,
fileIdentity: null,
})
expect(warmIndex.getTrashEntry(".trash/new.md")).toEqual({
trashPath: ".trash/new.md",
trashedAt: trashTime.toUnixInteger(),
fileIdentity: SAMPLE_FILE_IDENTITY,
})
})

it("treats the cutoff as exclusive — a row stamped exactly at the cutoff is not expired", () => {
const trashIndex = createSearchIndex(":memory:")
trashIndex.recordTrashEntry(".trash/boundary.md")
trashIndex.recordTrashEntry({
trashPath: ".trash/boundary.md",
fileIdentity: SAMPLE_FILE_IDENTITY,
})
const entry = trashIndex.getTrashEntry(".trash/boundary.md")

if (!entry) throw new Error("entry missing after record")
Expand All @@ -8313,7 +8372,7 @@ describe("trash entries (retention-sweep bookkeeping)", () => {
).toEqual([".trash/boundary.md"])
})

it("re-recording the same path restarts the retention clock", () => {
it("re-recording the same path restarts the retention clock and replaces the identity", () => {
vi.useFakeTimers()
onTestFinished(() => {
vi.useRealTimers()
Expand All @@ -8323,24 +8382,32 @@ describe("trash entries (retention-sweep bookkeeping)", () => {
const secondTrashTime = DateTime.fromISO("2026-03-01T00:00:00Z")

vi.setSystemTime(firstTrashTime.toMillis())
trashIndex.recordTrashEntry(".trash/reused.md")
trashIndex.recordTrashEntry({ trashPath: ".trash/reused.md", fileIdentity: "100:5:1" })
const firstEntry = trashIndex.getTrashEntry(".trash/reused.md")

vi.setSystemTime(secondTrashTime.toMillis())
trashIndex.recordTrashEntry(".trash/reused.md")
trashIndex.recordTrashEntry({ trashPath: ".trash/reused.md", fileIdentity: "200:7:2" })
const refreshedEntry = trashIndex.getTrashEntry(".trash/reused.md")

expect(firstEntry?.trashedAt).toBe(firstTrashTime.toUnixInteger())
expect(refreshedEntry?.trashedAt).toBe(secondTrashTime.toUnixInteger())
expect(firstEntry).toEqual({
trashPath: ".trash/reused.md",
trashedAt: firstTrashTime.toUnixInteger(),
fileIdentity: "100:5:1",
})
expect(refreshedEntry).toEqual({
trashPath: ".trash/reused.md",
trashedAt: secondTrashTime.toUnixInteger(),
fileIdentity: "200:7:2",
})
})

it("a case-alias record replaces the stale row instead of adding a second one", () => {
// On a case-insensitive bind mount ".trash/Note.md" and ".trash/note.md"
// are one file — two live rows for it would let a stale expired alias
// purge the fresh copy. The folded primary key collapses them to one.
const trashIndex = createSearchIndex(":memory:")
trashIndex.recordTrashEntry(".trash/Note.md")
trashIndex.recordTrashEntry(".trash/note.md")
trashIndex.recordTrashEntry({ trashPath: ".trash/Note.md", fileIdentity: SAMPLE_FILE_IDENTITY })
trashIndex.recordTrashEntry({ trashPath: ".trash/note.md", fileIdentity: SAMPLE_FILE_IDENTITY })

const farFutureCutoff = DateTime.now().plus({ days: 1 }).toUnixInteger()
const listed = trashIndex.listExpiredTrashEntries(farFutureCutoff)
Expand All @@ -8351,7 +8418,10 @@ describe("trash entries (retention-sweep bookkeeping)", () => {

it("deleteTrashEntry removes the row under any case alias", () => {
const trashIndex = createSearchIndex(":memory:")
trashIndex.recordTrashEntry(".trash/ToDelete.md")
trashIndex.recordTrashEntry({
trashPath: ".trash/ToDelete.md",
fileIdentity: SAMPLE_FILE_IDENTITY,
})

trashIndex.deleteTrashEntry(".trash/todelete.md")

Expand All @@ -8360,15 +8430,32 @@ describe("trash entries (retention-sweep bookkeeping)", () => {
expect(trashIndex.listExpiredTrashEntries(farFutureCutoff)).toEqual([])
})

it("trash entries survive a vault rebuild", async () => {
it("trash entries survive a vault rebuild with their timestamp and identity", async () => {
// Only Date is faked, so the rebuild's own timers and file reads run as usual.
vi.useFakeTimers({ toFake: ["Date"] })
onTestFinished(() => {
vi.useRealTimers()
})
const trashTime = DateTime.fromISO("2026-01-01T00:00:00Z")
vi.setSystemTime(trashTime.toMillis())
const trashIndex = createSearchIndex(":memory:")
trashIndex.recordTrashEntry(".trash/survivor.md")
trashIndex.recordTrashEntry({
trashPath: ".trash/survivor.md",
fileIdentity: SAMPLE_FILE_IDENTITY,
})
Comment thread
aliasunder marked this conversation as resolved.
const emptyVault = await mkdtemp(join(tmpdir(), "trash-rebuild-"))
onTestFinished(() => rm(emptyVault, { recursive: true, force: true }))
// A day later, so a rebuild that re-stamped the row from the clock would
// change trashedAt.
vi.setSystemTime(trashTime.plus({ days: 1 }).toMillis())

await trashIndex.rebuildFromVault({ vaultPath: emptyVault }, logger)

expect(trashIndex.getTrashEntry(".trash/survivor.md")?.trashPath).toBe(".trash/survivor.md")
expect(trashIndex.getTrashEntry(".trash/survivor.md")).toEqual({
trashPath: ".trash/survivor.md",
trashedAt: trashTime.toUnixInteger(),
fileIdentity: SAMPLE_FILE_IDENTITY,
})
})
})

Expand Down
Loading
Loading