mex graph rebuild builds into graph.db.candidate-<token> and publishes it by
atomic rename. If the process dies partway — Ctrl+C, a CI timeout, a crash — the
candidate stays on disk permanently. No later command removes it, and no command
mentions it.
To reproduce
Kill a running mex graph rebuild, then run each maintenance command in turn:
$ ls .mex/
graph.db 6,774,784
graph.db.candidate-ea0c83460fd19a3425f426cd7a32ec… 6,774,784
graph.db.lock 112
| after |
candidate present? |
notes |
mex graph status |
yes |
reports fresh, says nothing about it |
mex graph repair |
yes |
|
mex graph refresh (successful) |
yes |
|
mex graph rebuild (full, successful) |
yes |
the stale .lock is cleared; only the candidate leaks |
The candidate is a byte-for-byte duplicate of the store, so every interruption
costs one full store on disk. It is invisible unless someone lists .mex/.
Cause
src/graph/maintenance.ts calls cleanupOwnedDatabase(paths, candidatePath)
only from within the run that created the candidate — its catch/finally paths.
A killed process never reaches that code, and there is no sweep: the file
contains no readdirSync or globSync of the graph directory at all, so no
later run can discover a previous run's leftovers.
The same applies to the other two owned prefixes in OWNED_DATABASE_PREFIXES —
graph.db.rollback- and graph.db.recovery-. Planting one of each confirms
both survive status, repair, refresh and a successful rebuild.
Impact
One full copy of the store leaked silently per interruption. On the 240 MB store
in #140 that is 240 MB each time, and that report describes builds being stopped
by a 30-minute timeout — exactly this path.
Note for anyone triaging this alongside #140
On 0.8.x an interrupted rebuild does not strand a WAL; it strands a
candidate plus a lock. The two failure modes look alike and are not:
|
stranded graph.db-wal |
orphan graph.db.candidate-* |
| what's wrong |
status misreports a healthy store |
a full-size copy stays on disk |
| cost |
confusion |
disk |
| fix today |
mex graph repair |
delete it by hand |
| needs |
wording |
cleanup + a warning |
The wording half is filed separately as #204.
Suggested fix
At the start of a maintenance run, remove OWNED_DATABASE_PREFIXES files that
no live lock covers, and have graph status warn while one is present.
The lock question is already solved, which keeps this small: the maintenance
lease is per-project and exclusive, and acquireGraphMaintenanceLease already
reclaims a dead lock (readLockOwner / beforeDeadLockReclaim, around
src/graph/maintenance.ts:1537). Sweeping after the lease is held means no
concurrent run can own the files being removed, so the sweep needs no liveness
logic of its own — doing it before acquiring the lease would.
mex graph rebuildbuilds intograph.db.candidate-<token>and publishes it byatomic rename. If the process dies partway — Ctrl+C, a CI timeout, a crash — the
candidate stays on disk permanently. No later command removes it, and no command
mentions it.
To reproduce
Kill a running
mex graph rebuild, then run each maintenance command in turn:mex graph statusfresh, says nothing about itmex graph repairmex graph refresh(successful)mex graph rebuild(full, successful).lockis cleared; only the candidate leaksThe candidate is a byte-for-byte duplicate of the store, so every interruption
costs one full store on disk. It is invisible unless someone lists
.mex/.Cause
src/graph/maintenance.tscallscleanupOwnedDatabase(paths, candidatePath)only from within the run that created the candidate — its catch/finally paths.
A killed process never reaches that code, and there is no sweep: the file
contains no
readdirSyncorglobSyncof the graph directory at all, so nolater run can discover a previous run's leftovers.
The same applies to the other two owned prefixes in
OWNED_DATABASE_PREFIXES—graph.db.rollback-andgraph.db.recovery-. Planting one of each confirmsboth survive status, repair, refresh and a successful rebuild.
Impact
One full copy of the store leaked silently per interruption. On the 240 MB store
in #140 that is 240 MB each time, and that report describes builds being stopped
by a 30-minute timeout — exactly this path.
Note for anyone triaging this alongside #140
On 0.8.x an interrupted rebuild does not strand a WAL; it strands a
candidate plus a lock. The two failure modes look alike and are not:
graph.db-walgraph.db.candidate-*mex graph repairThe wording half is filed separately as #204.
Suggested fix
At the start of a maintenance run, remove
OWNED_DATABASE_PREFIXESfiles thatno live lock covers, and have
graph statuswarn while one is present.The lock question is already solved, which keeps this small: the maintenance
lease is per-project and exclusive, and
acquireGraphMaintenanceLeasealreadyreclaims a dead lock (
readLockOwner/beforeDeadLockReclaim, aroundsrc/graph/maintenance.ts:1537). Sweeping after the lease is held means noconcurrent run can own the files being removed, so the sweep needs no liveness
logic of its own — doing it before acquiring the lease would.