Plan rewrites (rhei complete, rhei transition, rhei reset, the dashboard
gate, rhei new) take an fs2 exclusive lock on the plan file itself, then
replace that file by renaming a temp over it. On Unix flock is advisory and
the rename is atomic over the locked file, so the lock spans the whole rewrite.
On Windows the lock is mandatory: the holder is refused its own path read and
its own rename. PR #94 made that work with LockedPlanFile (read through the
handle when the path is refused) and persist_locked (release the lock, then
retry the rename) — but the release opens a window with no lock and no handle
on the plan:
- a writer blocked in
lock_exclusive, or arriving fresh, can acquire the
lock, read the pre-rename content, compute its rewrite, and persist after
us — a silent lost update;
- or, if an outstanding range lock also blocks
MoveFileEx, the single retry
fails as failed to persist temp file under ordinary contention.
The window also has a read side, found in review of PR #97. LockedPlanFile
falls back to reading through its own handle when a path read is refused as
contended, on the reasoning that a replacement would carry no lock and the path
read would have succeeded. That reasoning fails against this window: while a
rewriter is between its release() and its rename, a second process can lock
the file that is about to be orphaned and register it; once the rename lands,
a third process locking the replacement is enough to refuse the second
process's read by path and send it to a handle naming the old content. It reads
stale bytes and cannot tell. Distinguishing the two refusals needs a file-identity
check (a GetFileInformationByHandle / dev+ino comparison), which is a
dependency rhei does not take for this. The sidecar below removes the window and
closes this hole with it — with the plan never locked, the fallback goes away
entirely.
PR #97 closes the half that does not need identity: once a process has released
its own lock, its handle is never read through again and the registry entry is
dropped, so the window a process opens cannot be read through by that same
process. The comment at LockedPlanFile::read_to_string now states the
remaining window rather than claiming it away.
Follow-on from the same cause: in system_transition_execution.rs the
metadata and task writes release their handles before the on_enter callback
and the rollback writes run, so on Windows the tail of a transition runs with
no lock while on Unix both locks span the whole transition. One body, two
transactional guarantees.
Proposed
Lock a sidecar rather than the file being replaced — <plan>.lock beside
the plan, or under the execution root's runtime/locks/ — for every plan
rewrite, on every platform. snapshot_records.rs already does exactly this
for snapshot identities (.lock beside the identity dir) and has none of the
self-refusal. With the plan file never locked, path reads and renames always
work, LockedPlanFile's handle fallback and persist_locked's retry become
unnecessary, and the lock spans the whole transition on Windows as it does on
Unix. Specify the lock's location and lifetime in §FS-rhei-transition-cmd (the
"file locking" contract) and note the compatibility rule for a plan directory
that already carries an older binary's file-level lock.
Until then, the window is documented at file_locks.rs::persist_locked and at
LockedPlanFile::read_to_string (the read side), and in the changelog entry of
#94.
Plan rewrites (
rhei complete,rhei transition,rhei reset, the dashboardgate,
rhei new) take anfs2exclusive lock on the plan file itself, thenreplace that file by renaming a temp over it. On Unix
flockis advisory andthe rename is atomic over the locked file, so the lock spans the whole rewrite.
On Windows the lock is mandatory: the holder is refused its own path read and
its own rename. PR #94 made that work with
LockedPlanFile(read through thehandle when the path is refused) and
persist_locked(release the lock, thenretry the rename) — but the release opens a window with no lock and no handle
on the plan:
lock_exclusive, or arriving fresh, can acquire thelock, read the pre-rename content, compute its rewrite, and persist after
us — a silent lost update;
MoveFileEx, the single retryfails as
failed to persist temp fileunder ordinary contention.The window also has a read side, found in review of PR #97.
LockedPlanFilefalls back to reading through its own handle when a path read is refused as
contended, on the reasoning that a replacement would carry no lock and the path
read would have succeeded. That reasoning fails against this window: while a
rewriter is between its
release()and its rename, a second process can lockthe file that is about to be orphaned and register it; once the rename lands,
a third process locking the replacement is enough to refuse the second
process's read by path and send it to a handle naming the old content. It reads
stale bytes and cannot tell. Distinguishing the two refusals needs a file-identity
check (a
GetFileInformationByHandle/dev+inocomparison), which is adependency rhei does not take for this. The sidecar below removes the window and
closes this hole with it — with the plan never locked, the fallback goes away
entirely.
PR #97 closes the half that does not need identity: once a process has released
its own lock, its handle is never read through again and the registry entry is
dropped, so the window a process opens cannot be read through by that same
process. The comment at
LockedPlanFile::read_to_stringnow states theremaining window rather than claiming it away.
Follow-on from the same cause: in
system_transition_execution.rsthemetadata and task writes release their handles before the
on_entercallbackand the rollback writes run, so on Windows the tail of a transition runs with
no lock while on Unix both locks span the whole transition. One body, two
transactional guarantees.
Proposed
Lock a sidecar rather than the file being replaced —
<plan>.lockbesidethe plan, or under the execution root's
runtime/locks/— for every planrewrite, on every platform.
snapshot_records.rsalready does exactly thisfor snapshot identities (
.lockbeside the identity dir) and has none of theself-refusal. With the plan file never locked, path reads and renames always
work,
LockedPlanFile's handle fallback andpersist_locked's retry becomeunnecessary, and the lock spans the whole transition on Windows as it does on
Unix. Specify the lock's location and lifetime in §FS-rhei-transition-cmd (the
"file locking" contract) and note the compatibility rule for a plan directory
that already carries an older binary's file-level lock.
Until then, the window is documented at
file_locks.rs::persist_lockedand atLockedPlanFile::read_to_string(the read side), and in the changelog entry of#94.