Skip to content

native.md

Latest commit

 

History

History
190 lines (163 loc) · 11.8 KB

native.md

File metadata and controls

190 lines (163 loc) · 11.8 KB
Raw
title Native architecture
description The platform-specific native bindings, fd-relative beneath model, platform mechanisms, loader security, and JavaScript fallback contract.

Native architecture

@openclaw/fs-safe uses native bindings that supply mechanisms Node does not expose directly. The Rust layer is deliberately not a second policy engine. TypeScript owns trusted-root selection, path validation, archive filtering, budgets, modes, identity fencing, cleanup decisions, and error normalization. Rust receives already-decided relative operations and performs the smallest platform syscall sequence that can preserve the boundary.

Every operation that has an equivalent safe Node implementation keeps that guarded JavaScript path. Native loading is lazy; installs do not compile Rust, run postinstall code, or fetch binaries at runtime. Seven exact-version optional packages are filtered by OS, CPU, and Linux libc, so an installation receives only its matching prebuilt binding. Native-only formats and creation-time Windows DACL guarantees fail explicitly instead of substituting a weaker implementation.

The beneath model

A trusted directory descriptor is the capability. Native operations accept that descriptor plus a validated relative path and never reconstruct authority from a process working directory. Newly created files use exclusive creation, and TypeScript compares descriptor, pathname, and expected identities before accepting results.

Conceptually, a caller grants authority to an already-open root—not to a path string that can be reinterpreted later:

validated Root handle
  └─ relative components (untrusted)
       └─ open/link/mkdir beneath the handle
            └─ compare descriptor + pathname + expected identity

The TypeScript layer validates and decides. The native layer never decides whether a path, archive entry, mode, owner, or cleanup policy is acceptable.

  • Linux uses openat2(RESOLVE_BENEATH | RESOLVE_NO_MAGICLINKS), fd-relative mkdirat/linkat/renameat/renameat2, FICLONE, and copy_file_range.
  • macOS 15.4 and newer first use openat(O_RESOLVE_BENEATH); older kernels walk components with openat(O_NOFOLLOW) and restart in-root symlinks from the pinned root. Both routes apply an F_GETPATH post-open containment detector, but directory rename races mean the result remains best-effort, not race-atomic. macOS uses renameatx_np(RENAME_EXCL) and permits fclonefileat in an owned, non-shared parent. The clone is normalized inside a private staging directory: flags, ACLs, extended attributes, and broad mode bits are cleared before no-replace publication.
  • Windows uses handle-relative NtCreateFile with OBJ_DONT_REPARSE and FILE_OPEN_REPARSE_POINT, then explicitly rejects reparse points. Rename and hardlink operations stay rooted in already-open handles. Owner/DACL reads use GetSecurityInfo; private directories receive their protected DACL in the CreateDirectoryW call itself.

Archives

Rust streams ZIP and TAR payloads, including gzip, zstd, and bzip2. It first returns a bounded manifest. TypeScript applies the shared path, filter, strip, mode, and byte policies and returns an index-bound extraction plan. Rust then creates only those planned entries beneath a private staging descriptor.

A raw meter sits between decompression and the TAR crate, with matching TypeScript admission before node-tar. It parses 512-byte headers and bounded local PAX x metadata, using supported effective sizes to locate the following member body. GNU long-name/link L/K payloads remain supported. maxMetaEntryBytes bounds each metadata body before allocation; unsupported global/old metadata and sparse forms fail closed rather than being interpreted as ordinary members. See bounded local PAX support.

Every raw pass receives only TypeScript's resolved maxEntries, maxMetaEntryBytes, and maxDecodedBytes. Shared resolution caps metadata and decoded byte fields at JavaScript's safe-integer maximum and entry counts at 2^32 - 1 before backend selection. Large finite limits remain accepted; native conversion mirrors those caps and rejects malformed non-finite or negative direct-call values before casting. Logical member headers count before filtering/stripping; metadata records do not. maxEntryBytes and maxExtractedBytes remain exclusively in TypeScript's accepted-plan builder, after strip/filter policy, and are absent from the raw meter's interface. Bounded reads use the default count/metadata/decoded bounds; public maxBytes bounds only the requested output. TypeScript derives the internal decoded cap by safely adding maxExtractedBytes and maxArchiveBytes, clamped to the safe integer maximum. Every native pass receives that same cap and charges headers, metadata, bodies, padding, EOF blocks, and trailing zeros. It rejects overflow with archive-decoded-size-exceeds-limit; no ratio policy is implied. Extraction and entry reads drain the metered reader through physical EOF after TAR iteration. Trailing framing or decoded-limit failures propagate before directory modes are finalized, staging is published, or selected bytes return. Native reads stop at framing boundaries so a rejected header does not request its body from the decoder; codec buffering can still read ahead internally. Inspection finishes the complete bounded framing pass before parsing. Directory and link bodies, missing two-block EOF, and nonzero trailers reject on both backends, as detailed in raw TAR framing. Raw and padded sizes above JavaScript's safe-integer maximum reject as invalid framing before applying member budgets, including when local PAX overrides the size.

Publication and hashing

Exclusive publication tries a hardlink, then a copy-on-write clone, Linux copy_file_range, and finally the existing asynchronous JavaScript byte loop. All routes preserve wx semantics and the same source/target identity and SHA-256 fencing. Native hashing and Linux whole-file copying run on N-API async workers rather than the JavaScript event loop.

Mode semantics

Mode Native loading Fallback
auto Try once, cache the result Use guarded JavaScript when unavailable
require Try once, cache the result Throw FsSafeError("helper-unavailable")
off Never attempt a binding load Always use guarded JavaScript

Features without a safe JavaScript implementation, including zstd/bzip2 TAR, Windows private-directory creation, and retained-directory staging, fail with helper-unavailable when native support is absent or off. Staging is currently Linux/macOS only and rejects Windows with unsupported-platform.

The staged-file owner also serves POSIX native pinned writes, including streaming. Unpublished files remain at 0600; requested modes are applied through the owned file descriptor only after rename and published-entry identity validation. Post-rename chmod or sync failures retain the publication receipt and final name. Its direct-child exclusive openat hands off the descriptor before any fallible post-open checks; non-following statat compares against that descriptor with exact native identities, and cleanup uses unlinkat in the retained parent. The separate checks and unlink are not atomic conditional deletion. Windows pinned writes and other fallback-capable APIs retain their existing mechanisms. Native writers share root and parent admission, but keep their platform identity checks and leaf ownership. POSIX coordinator disposal uses SuppressedError to retain both an operation failure and a disposal failure, including their receipts; stage preparation and cleanup keep their documented error mappings. Root replacement verification borrows the published descriptor after final mode application, while a private coordinator retains the staged owner until the asynchronous check finishes. The owner never escapes that coordinator; public staging methods and receipts expose no descriptor or verification callback. Verification failures preserve the published name, and disposal still retains both verification and cleanup errors when both fail. The private verification channel carries exact bigint identity from the original owned descriptor (or the content-accepted FUSE descriptor). Root compares it against exact fd and pathname metadata; legacy helper return facts and public read metadata behavior are unchanged. Missing Windows pathname identity still requires a guarded path reopen and comparison with the original retained file; that fallback does not apply to POSIX no-read modes.

JavaScript fallback guarantees and delta

Public policy does not change with the selected mechanism: traversal and link rejection, archive filters/limits/modes, exclusive target creation, source and target identity fencing, publication cleanup receipts, and secret/lock policy remain TypeScript-owned. What changes is the syscall strength or availability:

Capability Native path Guarded JavaScript path
Root-relative opens/mutations Descriptor-relative beneath operations. Pinned writes create parents and publish both replacement and no-replace targets relative to open directory descriptors. Linux reports kernel-atomic; macOS and Windows report best-effort. macOS uses O_RESOLVE_BENEATH when available plus an F_GETPATH detector, while Windows rejects reparse traversal in the object-manager call. Reports best-effort: component-wise alias checks, no-follow opens where Node exposes them, private temp/rename, and post-operation identity verification. A same-privilege peer can replace a writable parent after a guard assertion but before Node resolves the pathname mutation; the mutation may land outside the intended root before the post-check detects it.
ZIP/TAR/gzip Rust streaming decode and fd-relative output creation. JSZip/node-tar into a private stage, then the same guarded merge policy.
Zstd/bzip2 TAR Supported. Unsupported; typed helper-unavailable.
Publication copy Clone, Linux copy_file_range, async native SHA-256. Exclusive wx byte loop and Node SHA-256 with the same content/identity fences.
rename-noreplace Atomic platform no-replace rename. Unsupported; no emulation by check-then-rename.
Windows DACL read Direct GetSecurityInfo; the public facts API exposes ordered basic allow/deny ACE SIDs, masks, and decoded flags without trust policy. Established .NET/icacls inspection fallback for coarse permission checks; raw ACE facts are native-only.
Windows private directory Creation-time protected DACL. Unsupported; no weaker pathname-only substitute.

Use off in CI to keep the fallback contract exercised. Use require when a deployment depends on the stronger mechanism or a native-only feature; do not infer native loading from timing.

Loader security

Importing fs-safe never executes a child process. Linux libc selection uses the Node process report, conventional musl library filenames, and the ELF PT_INTERP field of process.execPath. If all probes are inconclusive, the loader conservatively attempts the glibc package and lets normal module loading fail into auto fallback. The loader requires only the package selected from the detected target; it never probes unrelated packages, downloads code, or runs a postinstall step. A missing or incompatible binary silently selects the JavaScript fallback in auto, throws typed helper-unavailable in require, and is never inspected in off. Tests reject child_process, exec, or spawn usage in the loader.

Related pages