Skip to content

feat(file-lock): add retainOnExit opt-out for process-exit cleanup - #227

Merged
steipete merged 4 commits into
mainfrom
feat/lock-retain-on-exit
Sep 4, 2026
Merged

feat(file-lock): add retainOnExit opt-out for process-exit cleanup#227
steipete merged 4 commits into
mainfrom
feat/lock-retain-on-exit

Conversation

@steipete

@steipete steipete commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Summary

0.8.0 started releasing Root-backed sidecar locks on natural event-loop shutdown. That is the right default, but it broke a legitimate consumer pattern: OpenClaw's dist-artifact ownership deliberately retains its lock file across exit when a build child leaves unjoined work, because PID death alone cannot prove the writer stopped.

This adds retainOnExit?: boolean to acquireFileLock/withFileLock options. When set, the lock's sidecar is exempt from both the beforeExit Root cleanup and the synchronous exit fallback, and remains governed only by the caller's stale policy. Default behavior is unchanged.

Validation

  • New child-process tests prove an unchanged sidecar acquired with retainOnExit: true survives natural exit in both Root-backed and raw modes; existing tests continue to prove default release, changed-sidecar preservation, and single-attempt failure cleanup.
  • Full gate: 7,512 passed / 80 skipped. The two consumer-pnpm-lifecycle failures reproduce identically on unmodified main in this environment (the suite requires a pnpm lifecycle context this shell does not provide); they are unrelated to this change.
  • Docs (docs/sidecar-lock.md) and changelog updated.

Boundaries

Explicit release() still works normally; the option only exempts the process-exit handlers. This is an opt-out for fail-closed ownership records, not a way to skip cleanup silently.

Locks acquired with retainOnExit: true keep their sidecar on natural process exit, for callers whose ownership records are deliberately fail-closed (PID death alone must not release them). Default exit-cleanup behavior is unchanged.
@steipete
steipete requested a review from a team as a code owner September 4, 2026 20:45
@clawsweeper

clawsweeper Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@clawsweeper clawsweeper Bot added P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Sep 4, 2026
@clawsweeper

clawsweeper Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Codex review: blocked before merge. Reviewed September 4, 2026, 5:49 PM ET / 21:49 UTC.

ClawSweeper review

What this changes

This PR adds a retainOnExit file-lock option that preserves selected sidecar ownership records through natural process exit while retaining the default cleanup behavior.

Merge readiness

Blocked before merge - 3 items remain

Keep open for maintainer approval: the final head resolves the prior lifecycle findings and provides direct built-package proof, but it adds a security-sensitive public opt-out whose caller-owned stale-lock policy needs an explicit contract decision.

Priority: P2
Reviewed head: 0b761f6a9e85cb7a64d41351bed8fea58461d0d2
Owner decision: Required. See Decision needed.

Review scores

Measure Result What it means
Overall readiness 🦞 diamond lobster (5/6) A focused public API change with direct runtime proof and comprehensive regressions; only the intended public safety-policy decision remains.
Proof confidence 🦞 diamond lobster (5/6) Sufficient (terminal): The changed lifecycle owner is the sidecar-lock exit cleanup path; the contributor supplied built-dist child-process output showing default sidecar removal and retained-sidecar survival after natural exit, while the added source suite covers Root, raw, legacy-copy, reset, and reentrant cases.
Patch quality 🦞 diamond lobster (5/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Verified Sufficient (terminal): The changed lifecycle owner is the sidecar-lock exit cleanup path; the contributor supplied built-dist child-process output showing default sidecar removal and retained-sidecar survival after natural exit, while the added source suite covers Root, raw, legacy-copy, reset, and reentrant cases.
Evidence reviewed 5 items Retained cleanup is narrowly gated: The exit handler preserves only held locks marked retainOnExit; Root-backed pre-exit cleanup likewise skips only those marked locks, retaining guarded cleanup for the default path.
Legacy handler state fails closed: A retained acquisition rejects with helper-unavailable when an earlier package copy registered cleanup without the retain-aware marker, preventing silent removal by a legacy handler.
Reentrant retention is monotonic: A same-owner reentrant request with retainOnExit: true upgrades the held lock instead of losing the requested exit policy.
Findings None None.
Security None None.

How this fits together

File locks create sidecar ownership records next to protected files to coordinate cross-process access. Exit handlers normally remove those records; this option leaves selected records for a caller-defined stale-lock policy to govern.

flowchart LR
A[Caller acquires file lock] --> B[Sidecar ownership record]
B --> C{Natural process exit}
C -->|Default| D[Guarded cleanup removes sidecar]
C -->|Retain on exit| E[Sidecar remains]
E --> F[Caller stale-lock policy]
D --> G[Later lock acquisition]
F --> G
Loading

Decision needed

Question Recommendation
Should retainOnExit become a supported public file-lock policy for callers that deliberately retain ownership records after natural exit? Approve the retained-lock contract: Accept the documented caller-owned stale policy and merge the opt-in while preserving default cleanup.

Why: The implementation is contained and fail-closed across mixed package copies, but accepting a caller-controlled exemption from shutdown cleanup is a public filesystem-safety boundary decision.

Before merge

  • Resolve merge risk (P1) - This opt-out intentionally leaves a cross-process ownership record after exit, so maintainers must accept caller-owned stale/reclamation policy as part of the public filesystem-safety contract.
  • Complete next step (P2) - Confirm the public retainOnExit contract and its caller-owned stale-lock recovery policy before merge.
  • Resolve maintainer decision - Resolve the maintainer decision shown above before merge.
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Implementation and regression coverage production +41/-4, tests +79 The public lifecycle option is accompanied by child-process, legacy-copy, reset, and reentrancy coverage.

Merge-risk options

Maintainer options:

  1. Approve the fail-closed retention policy (recommended)
    Merge after explicitly accepting that opt-in callers own stale-lock recovery for sidecars retained after exit.
  2. Pause the public API addition
    Do not merge if the package should guarantee cleanup rather than expose a retained-lock lifecycle policy.

Technical review

Best possible solution:

Adopt the documented opt-out only if maintainers endorse retained sidecars as an explicit fail-closed public policy; otherwise keep the v0.8.0 cleanup-only contract.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a bug report: the submitted built-package child-process evidence directly demonstrates the new default and retained exit behaviors.

Is this the best way to solve the issue?

Unclear pending maintainer direction: the implementation is narrow and preserves defaults, but the public retained-lock policy itself requires approval.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 3afa58251b49.

Labels

Label changes:

  • add rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The changed lifecycle owner is the sidecar-lock exit cleanup path; the contributor supplied built-dist child-process output showing default sidecar removal and retained-sidecar survival after natural exit, while the added source suite covers Root, raw, legacy-copy, reset, and reentrant cases.
  • remove rating: 🦐 gold shrimp: Current PR rating is rating: 🦞 diamond lobster, so this older rating label is no longer current.
  • remove status: ⏳ waiting on author: Current PR status label is status: 👀 ready for maintainer look.
  • remove merge-risk: 🚨 compatibility: Current PR review merge-risk labels are merge-risk: 🚨 security-boundary.

Label justifications:

  • P2: This is a bounded public file-lock lifecycle capability with no default-path regression found.
  • merge-risk: 🚨 security-boundary: The new API deliberately changes whether a cross-process ownership record survives process exit.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The changed lifecycle owner is the sidecar-lock exit cleanup path; the contributor supplied built-dist child-process output showing default sidecar removal and retained-sidecar survival after natural exit, while the added source suite covers Root, raw, legacy-copy, reset, and reentrant cases.
  • proof: sufficient: Contributor real behavior proof is sufficient. The changed lifecycle owner is the sidecar-lock exit cleanup path; the contributor supplied built-dist child-process output showing default sidecar removal and retained-sidecar survival after natural exit, while the added source suite covers Root, raw, legacy-copy, reset, and reentrant cases.

Evidence

What I checked:

  • Retained cleanup is narrowly gated: The exit handler preserves only held locks marked retainOnExit; Root-backed pre-exit cleanup likewise skips only those marked locks, retaining guarded cleanup for the default path. (src/sidecar-lock.ts:142, 0b761f6a9e85)
  • Legacy handler state fails closed: A retained acquisition rejects with helper-unavailable when an earlier package copy registered cleanup without the retain-aware marker, preventing silent removal by a legacy handler. (src/sidecar-lock.ts:264, 0b761f6a9e85)
  • Reentrant retention is monotonic: A same-owner reentrant request with retainOnExit: true upgrades the held lock instead of losing the requested exit policy. (src/sidecar-lock-acquire.ts:110, 0b761f6a9e85)
  • Regression coverage and real-package evidence: The process-exit suite covers retained Root and raw sidecars, reset teardown, legacy-handler refusal, and the default-then-retain reentrant sequence. The contributor also recorded a built-dist child-process result showing default cleanup false and retained-sidecar survival true. (test/sidecar-lock-process-exit.test.ts:66, 0b761f6a9e85)
  • Existing lifecycle provenance: The shipped v0.8.0 baseline includes Root-backed natural-exit cleanup; commit a41b18dd37fd98e48d0aeedb5821e6859aac8edc introduced that lifecycle behavior, making this opt-out a new public contract rather than work already on main. (src/sidecar-lock.ts:152, a41b18dd37fd)

Likely related people:

  • steipete: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (3 earlier review cycles)
  • reviewed 2026-09-04T20:49:00.793Z sha f2ad890 :: needs real behavior proof before merge. :: [P2] Keep manager reset independent of retain-on-exit | [P3] List retainOnExit in the published acquire-options shape
  • reviewed 2026-09-04T21:13:11.230Z sha 54f875c :: blocked before merge. :: [P1] Preserve retained locks across preloaded package copies
  • reviewed 2026-09-04T21:34:17.995Z sha 70ea15a :: blocked before merge. :: [P2] Propagate retainOnExit through reentrant acquisition

…he option

Review feedback: manager reset() is explicit teardown, not process exit, so it must still remove retained raw locks; only the exit handlers honor the flag. List retainOnExit in the published acquire-options docs and cover the reset path.
@steipete

steipete commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Addressing the review on head 54f875c (pushed).

P2 — reset() independence: correct; releaseAllLocksSync is shared by the exit handler and FileLockManager.reset(). The exit handler now passes preserveRetained: true; explicit reset() retains its teardown semantics and removes retained raw locks. Covered by a new in-process regression (manager reset still releases a retainOnExit raw lock).

P3 — documented option shape: retainOnExit?: boolean is now listed in the acquire-options block in docs/sidecar-lock.md, and the exit-behavior section notes the exemption.

Real-behavior proof: direct child-process runs against built dist (not the test harness), each acquiring a Root-backed lock then exiting naturally:

{
  "default": { "lockFileSurvivesExit": false },
  "retainOnExit": { "lockFileSurvivesExit": true }
}

The default mode removed the sidecar at exit; the retained mode preserved it. The same proof passes on the child-process suite in CI. Full serial suite: 7,513 passed / 80 skipped; the two consumer-pnpm-lifecycle failures reproduce identically on unmodified main in this shell (missing pnpm lifecycle context) and are unrelated.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Sep 4, 2026
retainOnExit is only honored when this package copy registered the process-exit handlers. When an older copy registered them first, retained acquisition now rejects with helper-unavailable instead of silently losing the guarantee at shutdown.
@steipete

steipete commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Addressing the re-review on head 70ea15a (pushed).

P1 — mixed-version exit handlers: agree, and fixed fail-closed. Exit-handler registration now stamps a retain-aware marker (fsSafe.sidecarLockRetainAwareCleanup) alongside the existing dedup keys; when the cleanup handler is registered but the marker is absent (an older copy owns it), a retainOnExit acquisition rejects with helper-unavailable before touching the filesystem. A retained sidecar can never be silently deleted by a legacy handler. A regression test simulates the legacy-registered state (global dedup keys set, marker absent) and asserts the refusal; normal acquisition is unaffected.

The same reasoning covers the Root-backed beforeExit path: that registration is also keyed and deduplicated globally, and the same marker gates it.

Verification: the exit suite passes 15/15 (retained raw + Root survival, reset independence, legacy refusal, plus the existing default-release and changed-sidecar cases). Full gate: 7,514 passed / 80 skipped; the two consumer-pnpm-lifecycle failures reproduce on unmodified main in this shell and are unrelated. The dist-level child-process proof from the previous comment still holds: default exits remove the sidecar, retained exits keep it.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

A reentrant same-owner acquisition reuses the held entry; a retainOnExit request now upgrades it so the sidecar survives exit. A later default acquisition never downgrades. Covers the default-then-retain sequence in-process and across natural child exit.
@steipete

steipete commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Addressing the re-review on head 0b761f6 (pushed).

P2 — reentrant upgrade: agree. Retention is now monotonic: a same-owner acquisition with retainOnExit: true upgrades the held entry (held.retainOnExit = true) instead of silently reusing the default policy, and a later default acquisition never downgrades it. Two regressions pin this: an in-process check that the held entry's flag upgrades under a doubled refCount, and a child-process natural-exit run for the default-then-retain sequence (sidecar survives, payload intact).

Re-verified against built dist:

{
  "default": { "lockFileSurvivesExit": false },
  "retainOnExit": { "lockFileSurvivesExit": true }
}

Serial suite: 7,516 passed / 80 skipped; the two consumer-pnpm-lifecycle failures reproduce on unmodified main in this shell (missing pnpm lifecycle context) and are unrelated.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. labels Sep 4, 2026
@steipete

steipete commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Maintainer disposition for the remaining review item: retainOnExit is accepted as a supported public contract.

Rationale: the package's documented model is that natural-exit cleanup is best-effort and callers with fail-closed ownership records already exist (OpenClaw's dist-artifact build lock, where PID death cannot prove a detached writer stopped). The option is opt-in, defaults are unchanged, mixed-version processes fail closed rather than silently losing the guarantee, and stale recovery remains the caller's explicit policy. Documented in docs/sidecar-lock.md with the retention and failure modes.

The current-head review (2026-09-04T21:49:07Z, head 0b761f6a) reports no actionable code or security findings; its needs-human marker is not being treated as approval — this comment is the maintainer judgment it asked for.

All exact-head CI, coverage, benchmarks, and CodeQL gates pass. Serial local suite: 7,516 passed / 80 skipped; the two consumer-pnpm-lifecycle failures reproduce on unmodified main in this shell and are unrelated.

@steipete
steipete merged commit 141e2a5 into main Sep 4, 2026
28 checks passed
@steipete
steipete deleted the feat/lock-retain-on-exit branch September 4, 2026 22:07
steipete added a commit to openclaw/openclaw that referenced this pull request Sep 4, 2026
…ct lock on exit

fs-safe 0.8.0 made process-exit sidecar release the default, which broke
the deliberately fail-closed dist-artifact owner record. 0.8.1 adds the
retainOnExit opt-out (openclaw/fs-safe#227); adopt it so only the explicit
release after a joined child removes the record. Also update the whatsapp
session assertion for the conditional parent-directory chmod.
steipete added a commit to openclaw/openclaw that referenced this pull request Sep 5, 2026
* chore(deps): update @openclaw/fs-safe to 0.8.0

fs-safe 0.8.0 stops repairing existing secret-directory permissions and fails closed instead. Wrap OpenClaw secret writes to tighten OpenClaw-owned directory modes before delegating, preserving the documented tighten-then-write behavior. Move the cooldown exclusions to the 0.8.0 family.

* fix(infra): adapt to fs-safe 0.8.0 fail-closed boundaries

- privateFileStore: tighten OpenClaw-owned store roots to 0700 at creation, preserving tighten-then-write behavior for drifted dirs
- doctor transcripts: mask dirMode to permission bits; raw stat.mode includes file-type bits that the strict final directory-mode check rejects
- file-transfer: resolve the symlink canonical target locally when the fs-safe error no longer carries it
- archive test: 0.8.0 preserves the destination when a raced hardlink is rejected (PR197); update assertions

* chore(deps): update @openclaw/fs-safe to 0.8.1 and retain dist-artifact lock on exit

fs-safe 0.8.0 made process-exit sidecar release the default, which broke
the deliberately fail-closed dist-artifact owner record. 0.8.1 adds the
retainOnExit opt-out (openclaw/fs-safe#227); adopt it so only the explicit
release after a joined child removes the record. Also update the whatsapp
session assertion for the conditional parent-directory chmod.

* fix(infra): never chmod through a symlinked secret root

The 0.8.x tightening wrapper resolved rootDir with realpath before its
lstat walk, so a symlinked root had its destination chmodded before
fs-safe could reject the write. Walk the lexical chain instead; any
symlinked component, including the root, is left for fs-safe to reject.
Adds a regression test asserting the destination's mode is untouched.

* chore(deps): narrow the lockfile refresh to the fs-safe bump

The previous lockfile was rebuilt from scratch, drifting unrelated
resolutions. Regenerate from main's lockfile so only the fs-safe family
moves 0.7.2 -> 0.8.1 (temporarily allowing the old entries for the
supply-chain policy check during regeneration).

* fix(infra): pin directory identity through permission tightening

Both 0.8.x permission adapters checked a directory with lstat and then
chmodded it by pathname, leaving a check/use window where a swapped
component redirected the chmod. Open each component with
O_NOFOLLOW|O_DIRECTORY and fchmod the pinned descriptor instead; swapped
or symlinked directories are never mutated and remain fs-safe's to
reject. Adds private-file-store coverage for the tighten and
symlinked-root paths.

* refactor(infra): share descriptor-pinned directory tightening

Both permission adapters now use one module (private-dir-mode): an async
chain walk for secret writes and a sync single-root tighten for store
factories, eliminating the duplicated safety logic. The module documents
the residual same-principal bound: Node has no dirfd-relative open, so
inter-component traversal is by absolute pathname; an actor able to
replace an OpenClaw-owned directory mid-walk already holds write access
to that tree, and fs-safe still validates the final write.

* ci: retrigger checks-ui-e2e on the control-ui runner class

Reruns route to GitHub-hosted runners (run_attempt > 1), where the
dashboard-gallery Playwright test exceeds its 30s locator timeout. The
full shard passes locally (228/228) on this head and the identical UI
code passed CI at 9cbd8c6; a fresh run restores attempt-1 placement.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant