Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/benchmarks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:
- name: Set up pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6.0.10
with:
version: 11.24.0
version: 11.25.0
run_install: false

- name: Set up Node
Expand Down
12 changes: 6 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ jobs:
- name: Set up pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6.0.10
with:
version: 11.24.0
version: 11.25.0
run_install: false

- name: Set up Node
Expand Down Expand Up @@ -95,7 +95,7 @@ jobs:
- name: Set up pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6.0.10
with:
version: 11.24.0
version: 11.25.0
run_install: false

- name: Set up Node
Expand Down Expand Up @@ -140,7 +140,7 @@ jobs:
- name: Test native Root publication verification
env:
FS_SAFE_NATIVE_MODE: require
run: pnpm test test/root-write-mode.test.ts test/root-write-verification.test.ts test/root-write-lifetime.test.ts test/root-write-exact-identity.test.ts
run: pnpm test test/root-write-mode.test.ts test/root-write-verification.test.ts test/root-write-lifetime.test.ts test/root-write-exact-identity.test.ts test/secret-write-publication.test.ts test/native-write-mode-ownership.test.ts test/native-created-cleanup.test.ts test/file-mode-facades.test.ts

- name: Test Root sidecar admission with required native binding
env:
Expand Down Expand Up @@ -169,7 +169,7 @@ jobs:
run: |
docker run --rm -v "$GITHUB_WORKSPACE:/work" -w /work node:24-alpine sh -euxc '
apk add --no-cache cargo rust musl-dev build-base python3
npm install --global pnpm@11.24.0
npm install --global pnpm@11.25.0
pnpm install --frozen-lockfile
cargo test --workspace --locked
pnpm build
Expand All @@ -182,7 +182,7 @@ jobs:
node scripts/sidecar-contention-proof.mjs require
FS_SAFE_NATIVE_MODE=require pnpm test test/root-create-only-preflight.test.ts test/sidecar-lock-root-admission.test.ts test/sidecar-lock-root-ancestry.test.ts test/sidecar-lock-root-budget.test.ts test/sidecar-lock-root-resolver.test.ts test/sidecar-lock-root-unlink.test.ts test/sidecar-lock-unlink-siblings.test.ts test/file-lock-sync-stale.test.ts test/file-lock-sync-release.test.ts
FS_SAFE_PAX_REQUIRE_NATIVE=1 pnpm test test/native-owned-tree.test.ts test/native-write-containment.test.ts test/native-staging-regression.test.ts test/staged-file.test.ts test/staged-file-failures.test.ts test/native-archive-equivalence.test.ts test/native-publish-equivalence.test.ts test/archive-pax.test.ts test/archive-pax-security.test.ts test/archive-pax-compressed.test.ts test/archive-tar-strip.test.ts test/archive-tar-framing.test.ts test/archive-tar-framing-compressed.test.ts test/archive-gzip-integrity.test.ts
FS_SAFE_NATIVE_MODE=require pnpm test test/root-write-mode.test.ts test/root-write-verification.test.ts test/root-write-lifetime.test.ts test/root-write-exact-identity.test.ts
FS_SAFE_NATIVE_MODE=require pnpm test test/root-write-mode.test.ts test/root-write-verification.test.ts test/root-write-lifetime.test.ts test/root-write-exact-identity.test.ts test/secret-write-publication.test.ts test/native-write-mode-ownership.test.ts test/native-created-cleanup.test.ts test/file-mode-facades.test.ts
FS_SAFE_NATIVE_MODE=require pnpm test test/archive-zip-admission.test.ts test/archive-zip-metadata.test.ts test/archive-zip-integrity.test.ts
FS_SAFE_PAX_REQUIRE_NATIVE=1 pnpm test test/archive-filter-paths.test.ts test/archive-filter-compressed.test.ts test/archive-tar-gnu.test.ts test/archive-tar-gnu-meter.test.ts test/archive-tar-ignored.test.ts test/archive-tar-ignored-meter.test.ts test/archive-tar-admission.test.ts test/archive-tar-manifest.test.ts
'
Expand Down Expand Up @@ -240,7 +240,7 @@ jobs:
- name: Set up pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6.0.10
with:
version: 11.24.0
version: 11.25.0
run_install: false

- name: Set up Node
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:
- name: Set up pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6.0.10
with:
version: 11.24.0
version: 11.25.0
run_install: false

- name: Set up Node
Expand Down Expand Up @@ -72,7 +72,7 @@ jobs:
- name: Set up pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6.0.10
with:
version: 11.24.0
version: 11.25.0
run_install: false

- name: Set up Node
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/crabbox-hydrate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ permissions:

env:
NODE_VERSION: "24"
PNPM_VERSION: "11.24.0"
PNPM_VERSION: "11.25.0"

jobs:
hydrate:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/hash-identity-proof.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:
- name: Set up pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6.0.10
with:
version: 11.24.0
version: 11.25.0
run_install: false

- name: Set up Node
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,4 +51,4 @@ jobs:

- name: Deploy
id: deployment
uses: actions/deploy-pages@cd2ce8fcbc39b97be8ca5fce6e763baed58fa128 # v5.0.0
uses: actions/deploy-pages@368f82528645a54fb793d4d04e342629a3f51346 # v5.0.1
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ concurrency:
env:
NODE_VERSION: 24
NPM_VERSION: 12.0.2
PNPM_VERSION: 11.24.0
PNPM_VERSION: 11.25.0

jobs:
validate:
Expand Down
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,14 @@

## 0.7.3 - Unreleased

- Refresh the Node type definitions, align development and CI on pnpm 11.25.0, and update the pinned Pages deployment action to v5.0.1 for polling backoff and jitter. Thanks @dependabot.
- Finalize explicit file modes after content writes across pinned writers, verify all `0o7777` POSIX bits for secret publication, and use exact identities before native Windows mode changes and failed-write cleanup; JavaScript fallback cleanup preserves unverified replacements.
- Complete short synchronous regular-file appends and preserve explicitly requested special mode bits in both append helpers, retaining permission tightening before any data is written.
- Preserve existing secret-directory permissions instead of repairing them; retain lossless directory identities through private locks and native writes, initialize new directories through guarded descriptor authority, honor full directory mode bits, and fail closed for unpinnable parents, including non-root macOS directories created under `umask(0o777)`.
- Keep async Root-backed lock normalization read-only, rejecting deleted or replaced admitted parents without recreating them while preserving explicit in-root sidecars for external target keys.
- Exercise secret-directory admission from isolated npm/pnpm consumer installs, retain real-identity and native-load proof, and honor explicit package-proof output paths.
- Propagate durable queue enqueue parent-sync failures and keep published-file identity checks after synchronization, sharing the guarded writer with migrations while retaining retry state.
- Bound workflow-dispatch test subprocesses and terminate their process groups before fixture cleanup, so stuck shell descendants fail validation instead of hanging the suite.
- Retry Windows Root-backed sidecar exclusive-create denials within the existing eight-retry and caller budgets, using per-call provenance while preserving callback errors and rejecting replayed failure evidence.
- Assign cross-platform sidecar contention proof liveness to its whole-worker watchdog instead of false-failing healthy unfair acquisition; production lock timeout behavior is unchanged.
- Enforce portable FileStore keys consistently across methods: async reads, `exists`, and `remove` now reject parent-segment and backslash aliases with `invalid-path` for existing roots, matching sync and write methods while preserving missing-root error precedence and Root's confined existing-object compatibility.
Expand Down
2 changes: 2 additions & 0 deletions docs/errors.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,8 @@ type FsSafeErrorCode =
| `too-large` | A read or bounded walk exceeded its configured budget. | Caller gave a too-permissive file or traversal limit. |
| `unsupported-platform` | Reserved compatibility code for a platform-specific operation. | No current public helper emits this `FsSafeError` code. Platform-specific APIs currently return a typed unsupported result or use `helper-unavailable`; keep the union member when exhaustively switching across supported package versions. |

Secret writes reject invalid `mode` / `dirMode` values with `invalid-path` before directory creation. Existing secret directories with a mode different from the requested `dirMode` report `insecure-permissions` without chmod; a created directory whose descriptor ownership no longer matches its initializing effective user reports `not-owned`.

Pathname `sha256File()` also reports `path-mismatch` when pre-open, descriptor,
or current-path identity remains unknown after one bounded Windows retry, even
if the file is benign. It never reopens to recover identity. Preview symlinks
Expand Down
13 changes: 11 additions & 2 deletions docs/private-file-store.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,17 @@ const loaded = await store.readJsonIfExists<State>("state.json");

- Writes create parent directories at `0o700` and files at `0o600` unless you
pass stricter `dirMode` / `mode` options.
- Private-mode writes route through the secret-file atomic path, which refuses
symlink parent components and re-asserts mode after rename.
- Async private-mode writes route through the secret-file atomic path, which refuses
symlink parent components and re-asserts mode after rename. Existing directories
must already have the requested mode; writes do not repair their permissions.
New-directory initialization requires guarded descriptor authority and may
fail closed under restrictive platform/umask combinations; see the
[secret-directory policy](secret-file.md#parameters).
- Locked JSON mutations prepare private directories before acquiring their
sidecar and bind the lock to the admitted parent identity. Lock normalization
is read-only: a deleted or replaced admitted parent is rejected, not recreated.
The writer still revalidates directory admission afterward; reads do not create
directories.
- `readText()` and `readJson()` are strict and throw on missing files.
- `readTextIfExists()` and `readJsonIfExists()` return `null` on missing files.
- `write()`, `writeText()`, `writeJson()`, `writeStream()`, and `copyIn()` all
Expand Down
10 changes: 9 additions & 1 deletion docs/regular-file.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,9 +116,17 @@ stalling admission. A confirmed non-regular target is refused before chmod or
append; other open errors propagate unchanged. This safeguard does not change
ordinary regular-file append semantics or require read permission.

The requested mode is applied through the admitted descriptor **before** any
content is appended, so an existing file is tightened first. On successful
completion, explicitly requested POSIX special bits are reapplied after the
content write, which can otherwise clear set-ID bits. An initial chmod failure
leaves the content untouched; a write or final chmod failure can leave appended
content and is not rolled back. Windows does not enforce POSIX mode semantics.

### `appendRegularFileSync(options)`

Synchronous. Same options.
Synchronous. Same options and mode ordering. Writes the complete input through
the already-open descriptor, including when an individual write is short.

### `resolveRegularFileAppendFlags()`

Expand Down
31 changes: 25 additions & 6 deletions docs/secret-file.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ startWebhookVerifier(signingKey);

### `writeSecretFileAtomic(params)`

Async. Creates the parent directory at `dirMode` (default `0o700`) if missing, writes content to a sibling temp file at `mode` (default `0o600`), atomically renames over the destination, and re-asserts the file mode after rename.
Async. Creates the parent directory at `dirMode` (default `0o700`) if missing, writes content to a sibling temp file, finalizes `mode` (default `0o600`) through an owned descriptor after content writes, and atomically renames over the destination. Publication verification checks the final file identity and mode.

Concurrent writes to distinct leaves may share creation of a missing parent.
After a parent-creation race, the helper re-inspects the entry and requires a
Expand All @@ -132,10 +132,21 @@ the requested directory mode before writing either leaf.

Publication verification borrows the writer's still-open descriptor to check
the exact file identity, regular-file and link policy, requested POSIX mode,
and root/parent ancestry before the writer closes it. POSIX mode overrides such
as `0o000` and `0o200` do not require read permission or a readonly reopen, and
verification does not widen the final mode. Windows retains its existing
pathname-identity verification policy without enforcing POSIX mode bits.
and root/parent ancestry before the writer closes it. All `0o7777` mode bits
must match, including explicitly requested special bits; unexpected special
bits are rejected. POSIX mode overrides such as `0o000` and `0o200` do not
require read permission or a readonly reopen, and verification does not widen
the final mode. Windows retains pathname-identity verification without enforcing
POSIX mode bits; its native writer checks the reopened descriptor against the
original lossless file identity before changing the final mode.

Failed JavaScript fallback writes attempt cleanup while retaining the original
descriptor and only after checking parent and file identities. Native cleanup
also compares lossless parent and file identities. Unverifiable paths are left
for caller-managed cleanup, and cleanup failures do not replace the original
write error. These are best-effort identity checks followed by name-based
removal, not atomic conditional unlink. A publication-verification failure
after a completed write does not authorize deleting the published file.

```ts
import { writeSecretFileAtomic } from "@openclaw/fs-safe/secret";
Expand All @@ -159,7 +170,15 @@ type WriteSecretFileParams = {
};
```

The directory mode is asserted on each component along the path: `rootDir`, then any intermediate dirs, then the parent. The helper enforces that every component matches `dirMode` — wider permissions on an existing directory cause the write to fail. Audit and tighten existing secret directories yourself.
The full POSIX directory mode is asserted on each component along the path: `rootDir`, then any intermediate dirs, then the parent. Existing directories, including another creator's `EEXIST` winner, must already match `dirMode` exactly or the write fails with `insecure-permissions`; they are never chmod-repaired. An explicitly requested directory mode such as `0o2750` preserves its setgid bit. Audit and adjust existing secret directories yourself. The admitted directory guards are retained through traversal and the final writer/lock handoff; a fresh pathname lookup cannot silently authorize a replacement. The caller must still trust the selected root and its owners; matching permission bits alone do not establish that trust.

Directory admission and its retained guards use lossless bigint identities, including through private locks and native writes. On Windows, an unknown zero device or inode gets one reinspection that retains known components; a definite mismatch or persistent ambiguity fails with `path-mismatch` rather than authorizing a replacement.

Both mode options must resolve to integers between `0o0000` and `0o7777`; invalid values fail with `invalid-path` before directory creation or file publication. Windows validates the options but does not enforce POSIX permission bits.

After this operation wins directory creation, initialization uses a pinned descriptor bound to the admitted identity and effective user, with ancestor checks before chmod. It does not chmod the caller's pathname. Creation and descriptor admission are separate operations, not an atomic create-and-pin guarantee. A raced directory that has not reached its requested mode yet is rejected rather than repaired; callers may retry after its creator finishes initialization.

Initialization fails closed if the platform cannot safely pin a created directory. In particular, a non-root macOS process cannot pin a new `000` directory produced by `umask(0o777)`; the write fails without repairing that directory or writing a secret. Restrictive masks retaining owner search permission remain usable. Linux x64/arm64 can use the guarded `O_PATH`/procfs descriptor route where available. There is no unguarded pathname-chmod fallback, and a failure may leave a created directory for caller-managed cleanup.

### `createSecretFileAtomic(params)`

Expand Down
9 changes: 9 additions & 0 deletions docs/sidecar-lock.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,15 @@ an existing `Root` capability. `lockPath` must resolve inside that root.
Identity-conditioned removal remains the only release and reclaim deletion
path.

Async Root-backed acquisition normalizes the target's parent without creating
it, checking the retained Root before and after normalization. A deleted or
replaced Root fails before payload execution or held-entry reuse. Missing lock
subdirectories are still created through `Root.create`, never by target-key
normalization. The target is an arbitration key and may be outside the Root
when an explicit in-root `lockPath` is supplied; normalization does not follow a
target-leaf symlink. Non-Root acquisition retains its existing parent-creation
behavior.

An owner can finish releasing while another async acquirer inspects its record.
Create-only Root writes do not open an existing record merely to inherit its
mode. Once a pathname sample and opened descriptor agree, a failed acquisition
Expand Down
2 changes: 2 additions & 0 deletions docs/store.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@ Loading serializes consumers for one ID through a sidecar lock, then creates `pr

Queue and failed directory creation fsyncs every newly-created parent edge from the leaf toward the trusted root. Enqueue and migration writes fsync the temp file and parent; claim, acknowledgement, quarantine, delivered-marker cleanup, and retirement transitions fsync every affected directory and propagate real sync failures. A transition may already be visible when a post-mutation sync fails, so retry the same operation to complete its crash-recovery state. Acknowledgement retries resync the queue directory even when both `.processing` and `.delivered` marker names are already absent, before reporting completion or rejecting a newer pending generation; quarantine retries with only failed evidence resync that destination before repairing the vanished queue source.

`writeJsonDurableQueueEntry()` and migrations share strict parent synchronization inside the atomic writer's retained descriptor and per-path serialization lifetime, followed by published-file identity verification. If sync fails after publication, the write rejects without rolling back the published JSON; retrying writes the entry again and must complete its own sync. This is not a rollback, deduplication, or exactly-once guarantee. The generic `replaceFileAtomic({ syncParentDir: true })` option remains best-effort.

Batch loading skips invalid entry names, malformed, oversized, or unreadable entry content, and caller `read` callback failures. Initially unowned pending entries (hardlinks or unverifiable identities), symlinks, non-files, and absent pending entries are also skipped. Claim, transfer-lock, retirement, and migration write/publication/durability failures reject the batch with the original error, even if earlier entries succeeded. Migration in both loaders strictly syncs the parent directory after successful publication. Visible transitions and earlier processing claims remain for retry; a rejected batch does not acknowledge or roll them back.

Failed destinations are create-only. Quarantine publishes the claimed file by hardlink, so the queue and failed directories must share a filesystem with hardlink support. If `failed/<id>.json` already exists, quarantine rejects while preserving both that earlier evidence and the current claimed entry instead of overwriting either file. The `read` callback continues to receive the logical `.json` path even though bytes are read and migrations are written through the claimed path.
Expand Down
Loading
Loading