Skip to content

An interrupted graph rebuild leaves a full-size candidate file that nothing ever removes #205

Description

@theyashasvipandey

mex graph rebuild builds into graph.db.candidate-<token> and publishes it by
atomic rename. If the process dies partway — Ctrl+C, a CI timeout, a crash — the
candidate stays on disk permanently. No later command removes it, and no command
mentions it.

To reproduce

Kill a running mex graph rebuild, then run each maintenance command in turn:

$ ls .mex/
graph.db                                             6,774,784
graph.db.candidate-ea0c83460fd19a3425f426cd7a32ec…   6,774,784
graph.db.lock                                              112
after candidate present? notes
mex graph status yes reports fresh, says nothing about it
mex graph repair yes
mex graph refresh (successful) yes
mex graph rebuild (full, successful) yes the stale .lock is cleared; only the candidate leaks

The candidate is a byte-for-byte duplicate of the store, so every interruption
costs one full store on disk. It is invisible unless someone lists .mex/.

Cause

src/graph/maintenance.ts calls cleanupOwnedDatabase(paths, candidatePath)
only from within the run that created the candidate — its catch/finally paths.
A killed process never reaches that code, and there is no sweep: the file
contains no readdirSync or globSync of the graph directory at all, so no
later run can discover a previous run's leftovers.

The same applies to the other two owned prefixes in OWNED_DATABASE_PREFIXES
graph.db.rollback- and graph.db.recovery-. Planting one of each confirms
both survive status, repair, refresh and a successful rebuild.

Impact

One full copy of the store leaked silently per interruption. On the 240 MB store
in #140 that is 240 MB each time, and that report describes builds being stopped
by a 30-minute timeout — exactly this path.

Note for anyone triaging this alongside #140

On 0.8.x an interrupted rebuild does not strand a WAL; it strands a
candidate plus a lock. The two failure modes look alike and are not:

stranded graph.db-wal orphan graph.db.candidate-*
what's wrong status misreports a healthy store a full-size copy stays on disk
cost confusion disk
fix today mex graph repair delete it by hand
needs wording cleanup + a warning

The wording half is filed separately as #204.

Suggested fix

At the start of a maintenance run, remove OWNED_DATABASE_PREFIXES files that
no live lock covers, and have graph status warn while one is present.

The lock question is already solved, which keeps this small: the maintenance
lease is per-project and exclusive, and acquireGraphMaintenanceLease already
reclaims a dead lock (readLockOwner / beforeDeadLockReclaim, around
src/graph/maintenance.ts:1537). Sweeping after the lease is held means no
concurrent run can own the files being removed, so the sweep needs no liveness
logic of its own — doing it before acquiring the lease would.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingcode-graphWork related to the AST and Tree-sitter code graphgood first issueGood for newcomers

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions