diff --git a/experimental/126-guard-liveness/README.md b/experimental/126-guard-liveness/README.md new file mode 100644 index 0000000..086d9ca --- /dev/null +++ b/experimental/126-guard-liveness/README.md @@ -0,0 +1,547 @@ +# Guard liveness and macOS refusal filter (research, #126/#127) + +Research only. Nothing here changes `src/`, eligibility, policy, platform +support, dependencies, or the JSON contract, and nothing is published by +`npm pack` (`experimental/` is outside `files`). No experiment calls +`unlockInspectedThread`, so no Codex or lock owner is signalled by the +product's unlock path. + +F1 and M2 measured on Darwin 27.0.0 arm64 (macOS 27.0.1, 26A434), Node v24.16.0, +lsof 4.91, APFS, same user, main `3bc4091`. + +## Signals and process lifetime + +What is recorded, not guaranteed: + +- **First M2 attempt (original measurement round).** The transient fixture + failed `listen EINVAL` because its Unix socket path exceeded the macOS + limit. The runner's error path did not stop the holder it had already + started, so the runner stayed waiting. The research runner process itself + (the holder's parent) was sent **one SIGTERM**. The holder then saw stdin + EOF and exited normally. No Codex process or fixture child was signalled. +- **Completed runs.** The research runners send no signal to fixture children. + They close stdin and wait with a bound. Each fixture also has its own + lifetime and exits with code 3 when it runs out. +- **Not covered by the above.** Node sends SIGTERM to a child whose + `spawnSync` timeout expires. That applies to the `lsof` calls in M2's `+fg` + scan and barrier wrapper. The +fg scan records `signal`/`error`, and the + validation runs recorded none. The product's own diagnostics (`lsof`, `ps`) + run with a command timeout, may signal their timed-out children, and are + not instrumented here. +- **What a bound means.** Waits on children are bounded, and an exceeded bound + is reported as `incomplete`. It is never escalated to a signal. A bound does + not guarantee that the child ended: an `incomplete` run sets a nonzero exit status, + keeps its scratch, and M2 keeps its socket directory. Every + nominal measurement and nominal validation run below recorded `incomplete: 0`. + +Runners create a fresh private `codex-*` directory under the temporary root +required by `AGENTS.md` (set `RESEARCH_TMP_ROOT` on macOS), or `/tmp` elsewhere. `RESEARCH_TMP_ROOT` +replaces only that root: the supplied directory is never chmod-ed, and +existing files outside that fresh scratch are never overwritten. M2 rewrites +its own scratch configuration between scenarios. A runner exits nonzero on any error, +incomplete child, or missing or mismatched round or result. + +## F1: injected delay inside the native guard (`f1-guard-delay/`) + +Question: while codex-unlock holds `.coordination.lock`, does a Codex-like +process waiting on the same file wait for exactly that long? + +The delay is a bounded `Atomics.wait` injected through the existing +`afterGuardAcquired` test hook of `guardedProbeOnce` in the dist build. **It is +an injected finite delay. It is not a reproduction of a `/proc` cmdline wait +or any other kernel wait.** The waiter polls a nonblocking flock every 0.1 ms +under a per-round deadline and a process lifetime that both checks in its +loops. Real Codex waits with a blocking flock instead. + +Original measurement: 5 repeats, 25 guarded and 25 baseline rounds. +`blocked` is the waiter's own time from seeing the marker (written inside +the guard) to acquiring coordination. `acquire − return` is the waiter's +acquisition time minus the moment `guardedProbeOnce` returned in the runner, +read from the same system monotonic clock. The guard's `LOCK_UN` happens +before that return, so this is **not** the delay after guard release, and it +can be negative. None of the 50 measured values was negative. + +| Injected | Baseline blocked (median) | Guarded blocked (min / median / max) | Guarded acquire − return (range) | +| --- | --- | --- | --- | +| 0 ms | 0.03 ms | 0.01 / 0.03 / 0.19 ms | 0.014 – 0.159 ms | +| 10 ms | 0.02 ms | 12.4 / 15.1 / 15.2 ms | 0.022 – 0.130 ms | +| 50 ms | 0.02 ms | 52.1 / 59.7 / 60.2 ms | 0.025 – 0.155 ms | +| 200 ms | 0.02 ms | 203.4 / 206.9 / 210.4 ms | 0.035 – 0.137 ms | +| 1000 ms | 0.03 ms | 1002.6 / 1006.5 / 1010.1 ms | 0.037 – 0.111 ms | + +Result: increasing injected delay increased the synthetic waiter's measured +blocked time. A process waiting on the same coordination lock cannot acquire +it until release. This measured one polling fixture, not every Codex process. +Overshoot above the injected value (for example 10 → 15 ms) was also recorded +in the hook's own duration; this does not isolate every scheduling cause. +All guarded rounds returned probe +`free` with guard `acquired`. Native file identities (dev, ino, size, mtime) +were unchanged. + +Fault checks: `F1_FAULT=skip_round` (waiter never answers round 3) and +`F1_FAULT=waiter_lifetime` (300 ms waiter lifetime) must both exit nonzero. + +## M2: `lsof +fg` FWASLOCKED as a refusal-side filter (`m2-macos-fwaslocked/`) + +XNU sets `FWASLOCKED` (0x4000) on the open file description only when +`flock` succeeds, and never clears it. The filter evaluated here is +refusal-only. Per PID it aggregates every descriptor on the path: + +| Condition | Decision | +| --- | --- | +| The scan failed, was signalled, wrote stderr, or has detectable malformed/truncated output | `unknown` | +| The PID is absent from the scan | `unknown` | +| No descriptor of the PID has the bit | `reject` | +| At least one descriptor has the bit | `not_excluded` | + +There is no "holder" outcome. `fg-scan.test.mjs` checks these rules +deterministically, including multi-descriptor PIDs and failed or partial +output. Entire valid records can be omitted without a detectable format error; +this filter does not prove a complete descriptor inventory. + +Part 1, flag matrix. The original measurement ran 3 repeats and all were +identical. The validation runs add `holder_multi_opener`: a holder with a +second, plain descriptor next to a never-locked opener. Its result: holder +`0x4003` + `0x1` → not_excluded, opener `0x3` → reject. + +| Case | Probe | Flags per role | +| --- | --- | --- | +| holder | held | holder `0x4003` not_excluded | +| holder + never-locked opener | held | holder `0x4003`; opener `0x3` reject | +| holder + failed acquirer (FD kept) | held | holder `0x4003`; failed `0x3` reject | +| former holder (LOCK_UN, FD kept) + actual holder | held | former `0x4003`; holder `0x4003`, indistinguishable | +| former holder only | **free** | former `0x4003` not_excluded | +| failed acquirer after holder exit | free | failed `0x3` reject | + +Part 2 reruns the #100 wrong-target shape. The actual holder is hidden from +the product's opener scan (a modelled clean omission), and the same omission +is applied to the `+fg` view. The `+fg` scan runs in the same barrier window +as the product's opener scan. It is a separate lsof invocation, not the +product's own sample. 3 repeats, all identical: + +| Scenario | Product result | Owner is holder | Filter on owner | +| --- | --- | --- | --- | +| B_failed: transient failed acquirer, barrier on the opener scan only | `live_owner`, **safe false** (`owner_holds_other_thread_locks`) | no | reject | +| B_failed_all_scans: barrier on every lsof call (the #100 setup) | `live_owner`, **safe true, wrong target** | no | **reject** | +| B_former: Codex-looking former holder with a kept FD | `live_owner`, **safe true, wrong target** | no | **not_excluded** | +| control: holder visible | `live_owner`, safe true | yes | not_excluded | + +Findings: + +1. The filter removes the failed-acquirer and never-locked opener cases, + including the #100 wrong-target reproduction. +2. It does not remove a former holder that keeps its descriptor. With the actual + holder omitted by the model, this is a wrong target accepted by the sampled + current checks and the filter. Its + bit is identical to the actual holder's. Because the bit is sticky, it can + never be positive holder evidence. +3. Scope of #100: with the barrier around the opener scan only, the + transient had already closed its thread descriptor by the per-owner + lock-set scan, and `owner_holds_other_thread_locks` blocked. The #100 + reproduction needs the transient's descriptor to be open in **both** the + opener-scan and the lock-set-scan sample windows; the #100 barrier held + it open across both. No real-environment frequency was measured. +4. Whether real Codex can be a "former holder with a kept FD" is source-only. + v0.162.1 `WriterLockGuard::drop` closes the file, and Rust std opens with + `O_CLOEXEC`, so exec'd children do not inherit it. A child forked without + exec would share the description, flag, and lock. No such path was found, + but the audit was not exhaustive. + +Fault checks: `M2_FAULT=early_exit` (actor exits before ready), +`ignore_eof` (stop budget shorter than the actor's lifetime: `incomplete`, +socket directory kept), and `long_socket` (path over the limit, refused before +start) must all exit nonzero. + +`holder/codex` and `transient/codex` derive from the #100 fixtures. The +holder now takes both locks with bounded nonblocking polling instead of a +blocking coordination flock. Both fixtures, and `former/codex`, have a +lifetime. All three are named `codex`. + +## F2: Linux non-dumpable holder (`f2-linux-nondumpable/`), access matrix measured + +`codex` (Python) refuses root and differing real/effective UIDs. It declares +the `prctl` ABI (`int` option and result, `unsigned long` arguments), sets the +requested dumpable value (0 or 1), and verifies it with `PR_GET_DUMPABLE`. It +then takes both locks with bounded nonblocking polling. It reports a ready +record with pid, fd, dumpable value and UIDs, and holds the lock until stdin +EOF (exit 0) or its lifetime limit (exit 3; the runner requests 60 s). + +`observe.py` records the same-user access matrix for the reads #137 performs. +It has no self-alarm, because a handled signal cannot wake a killable kernel +wait. `run.mjs` validates the ready record, waits for the observer with a +bound (recording `incomplete` without signalling it). Without `F2_BUILD` it +collects only the access matrix, as in the measured run (the observer now also +records mount-namespace readlinks). `F2_BUILD` runs a separate-process +inspection, described below. The earlier +in-process inspection was removed because synchronous inspection can block +the event loop, its timeout and holder EOF cleanup. The holder's own lifetime +does not bound the parent. Access-matrix-only mode records observed permission +outcomes without asserting that they match the source-derived expected table. + +Build inspection requires separate-process observation and a validated +dumpable=1 candidate-recognition control. +Results after holder exit cannot establish access policy. +Without `F2_BUILD`, `ok` describes collection completion, not satisfaction of +the permission table or a proven fail-closed decision. With `F2_BUILD`, `ok` +requires both collection completion and `report.expectations.ok`: an expected +table mismatch makes the runner exit 1 even when collection succeeded. A +successful synthetic build-mode run would still not prove native Codex safety. + +The unchanged `cfa2eba` fixtures ran once on a Linux test host on 2026-10-14: +Linux `7.0.0-1013-oracle`, aarch64, Node 24.18.0, Python 3.12.3, UID/EUID +1001, effective/permitted capabilities zero, Yama `ptrace_scope=1`, ordinary +procfs mount and a local ext-family temporary filesystem. Both requested +dumpable values were verified by the holder. Collection exited 0, with no +failures or incomplete observations. Both holders exited 0 and their recorded +process instances were absent at the post-check. Raw result SHA-256 and scope +are recorded in [evidence.md](evidence.md). + +Source-derived matrix below was observed on that test host: + +| Read | dumpable=1 | dumpable=0 | +| --- | --- | --- | +| `stat /proc/PID` uid | owner | owner (the 0555 directory is exempt in `task_dump_owner`) | +| `/proc/PID/fd` directory uid | owner | root | +| `/proc/PID/fdinfo` directory uid | owner | owner (0555 directory exception) | +| `fd` listdir/readlink/stat, `fdinfo` open, `exe` readlink | allowed | EACCES | +| `cmdline` | allowed | allowed (no ptrace check) | +| `stat` start ticks and status UID | allowed | allowed | + +The original combined fd/fdinfo ownership prediction was incorrect. Pinned +[Linux v6.17 directory definitions](https://github.com/torvalds/linux/blob/e5f0a698b34ed76002dc5cff3804a61c80233a7a/fs/proc/base.c#L3316) +give `fd` mode 0500 and `fdinfo` mode 0555. The +[dumpability ownership exception](https://github.com/torvalds/linux/blob/e5f0a698b34ed76002dc5cff3804a61c80233a7a/fs/proc/base.c#L1906) +therefore leaves fdinfo's UID unchanged. Its directory/file access still uses +a separate [ptrace permission check](https://github.com/torvalds/linux/blob/e5f0a698b34ed76002dc5cff3804a61c80233a7a/fs/proc/fd.c#L81). +Ownership alone does not establish readability. This result provides no +evidence of kernel-version drift. + +The denial supports the source-derived expectation that #137 should refuse +FD discovery for this candidate. F2 did not execute #137 or validate its +classification, candidate recognition, or final signal boundary. This Python +fixture does not measure a deployed native Codex process's dumpability. + +Hardening callers, within a targeted search of pinned v0.162.1 source: +`pre_main_hardening` appears in the `codex-responses-api-proxy` and +`codex-voice-host` mains, and `disable_process_dumping` appears in the +linux-sandbox proxy bridge. No direct call was found in the `codex` CLI +main. **That does not establish that a lock-owning process is dumpable.** The +same `codex` binary dispatches to linux-sandbox code by arg0. The kernel +also resets dumpable on credential changes and on exec of an unreadable +image. Packaging or wrappers may differ from source. Real Codex's dumpable +state needs a real binary and is not claimed. + +### F2 isolated inspection (`F2_BUILD`), prepared, not yet run on Linux + +`F2_BUILD=` enables product inspection in a **separate +child** (`inspect-child.mjs`, run through `lib/isolated-observer.mjs`). The +runner never imports a product module. Layers: + +| File | Role | +| --- | --- | +| `lib/isolated-observer.mjs` | generic: handshake, two bounds, byte-limited stdout/stderr, stage-record protocol, per-instance late exits | +| `protocol.mjs` | F2: exact stage order and uniqueness, result/error shapes, `F2_CHILD_FAULT` allowlist | +| `build.mjs` | trusted build input and its verification, import-graph check | +| `inspect-child.mjs` | product calls, one stage each | +| `expectations.mjs` | expected table, kept apart from collection | +| `plumbing-smoke.mjs` | cross-platform wiring check; not an F2 result | + +**Stages**, exactly once each, in this order: `dist_identity` (re-hash the +build against its manifest; stop before import on mismatch), `module_load` +(static and literal dynamic relative import graph of the imported modules +must not reach `unlock.js`, then import), `openers` (product `lsof` discovery +and `inspectProcess`), `prepare` (`prepareLinuxParticipation`), +`guarded_verify` (`guardedProbeOnce` with `verifyLinuxParticipation`) and +`inspect_thread` (`inspectThread`, policy blockers). Only `prepare` may end in +an error that counts as a product outcome. An error in any other stage, a +malformed result, a missing, duplicate or out-of-order stage, or an `end` +with both or neither of `result`/`error` is a collection fault. +`guarded_verify` returns `{ skipped: 'prepare_failed' }` after a failed +`prepare`. That is harness bookkeeping, not a product refusal. + +The relative import walk parses JavaScript with the checkout's existing +TypeScript development dependency, so comments between import tokens and +escaped string literals do not hide an edge. Parse errors fail the walk. Run +`npm ci` in the research checkout before using the builder or inspector; +copying only the research scripts is not sufficient for this revision. +Nonliteral dynamic imports, `require` and `eval` are outside this targeted +walk, not proof of a complete execution graph. Historical measurements used +their recorded earlier parser and source hashes. + +Resolved relative modules use canonical identities under the snapshot's dist +root. Aliases such as `../dist/unlock.js` cannot hide that module, and paths +escaping the root are rejected. + +**Build identity.** `build.mjs --repo R --commit C [--patch P] --out O` +resolves the commit and tree with `git rev-parse` in the named local +repository and builds from that repository's `git archive`. It records the +archive, lockfile and patch SHA-256, then runs `npm ci` and `npm run build`. +It also records a sorted manifest of `dist/` files and of the native addons +(`*.node`) under `node_modules`. `run.mjs` and `inspect-child.mjs` each re-hash +that content against the manifest before use. Each child must also match the +runner's startup manifest SHA-256; a different internally consistent build +cannot be attributed to the original input. The child copies only the +manifest/archive/optional patch, package metadata, dist and dependency tree +into a fresh private snapshot, verifies that copy, and walks/imports only the +copy. Runtime input symlinks are refused; unused `node_modules/.bin` links are +omitted. Snapshot files/directories are made read-only and retained as private +evidence, isolating imports from subsequent changes to the supplied build. +These permissions do not prevent a malicious same-UID actor from changing +permissions or bytes; no OS-enforced immutable execution claim is made. +This verifies only the recorded +`dist/` and native-addon bytes; dependency JavaScript and package metadata are +not covered. It does not prove that the local +repository matches upstream. A commit string supplied by anyone else is +never used as provenance. A deployed native Codex binary is unrelated and +remains unverified. + +The build helper uses bounded synchronous subprocesses for Git and npm. Their +timeout/output-limit handling can send a termination signal to those build +subprocesses; the inspector's no-owner-signal contract is not a claim that +building or collecting diagnostic output is signal-free. + +**Lifecycle**, as designed and locally tested: + +- The child writes a `hello` line first. The startup bound (10 s) runs until + that line. The stage bound (15 s) starts only then, so child startup + latency does not consume the stage budget. Exceeding either bound returns + `incomplete` with its phase and the open stage. +- stdout (1 MiB) and stderr (64 KiB) are capped by bytes per chunk. Bytes past + the cap are dropped and recorded as a fault, as is an unterminated last line. + The Python observer's output is capped (256 KiB) and parsed only after its + stdout closes. +- No cancellation guarantee. The parent sends no signal. It closes stdin, + destroys the pipes and unrefs the child. The child has **no lifetime timer**: + it checks its 30 s deadline and stdin EOF only between stages. It then writes + a `stop` record, sets exit code 3 (lifetime) or 4 (parent gone) and lets + pending work drain instead of calling `process.exit`. A refused build or + import exits 5. A child stuck in a syscall, or awaiting work that never + settles, can outlive the runner. Late exits are looked for with a 2 s bound + and recorded per instance as `exited` or `unconfirmed`. +- Signals: the child sends no signal to the holder, a lock owner or any + unlock path. Product diagnostics it calls (`lsof`, `ps`) run with the + product's own timeout. That timeout sends SIGTERM and then SIGKILL to the + timed-out diagnostic's own process group. + +**Same instance.** The runner reads the holder's `/proc/PID/stat` start ticks +itself at readiness, after observation and after inspection. A changed or +missing instance, or a holder exit at any point before inspection ends, +invalidates the run. The holder's lifetime is 60 s +(`FIXTURE_LIFETIME_SECONDS`). Before inspection, the runner checks that the +remaining lifetime covers observer + startup + stage bounds + 5 s. Opener, +`prepare`, participation and inspection ticks are compared with the runner's +sample as decimal strings. + +**Expected table.** A mismatch is recorded, never repaired by policy: + +| dumpable | Row | Expected | +| --- | --- | --- | +| both | V | one opener, the holder's PID, no opener error | +| 1 | E1 | opener identity complete, no opener errors, start ticks equal to the runner's sample | +| 1 | E2 | `prepare` PID and ticks equal, holder FD listed and kept as a candidate | +| 1 | E3 | guarded probe `held`, guard `acquired`, participation PID, FD and ticks equal | +| 1 | E4 | `inspectThread` probe `held` with the same participation; owner PID and ticks equal; no `linux_participation_unproven` | +| 0 | R1 | `prepare` fails with `EACCES` or `EPERM`. `ENOENT`, `TypeError`, the product budget error or a build error fail this row | +| 0 | R2 | `safeToUnlock=false`, blockers include `linux_participation_unproven`, no participation | + +R2 is a contract choice tied to source: #137 `b483cd5` +`evaluateSafety` adds that blocker when owner start ticks or a matching +participation are absent. R2 requires that blocker but allows additional +blockers; passing R2 does not establish that participation was the sole cause +of refusal. A refusal without that blocker fails R2 as "refused, not +attributed". V fails if +`lsof` does not show the non-dumpable owner. A dumpable=0 result counts only +if the same report's dumpable=1 control passed (`evaluateReport`). The +following are recorded and never asserted: `firstObservedError`, +`guardedAttempted`, participation, final eligibility (`safeToUnlock`, +blockers), owner/probe details and `fixtureIsCodex`. The fixture is `python3 …/codex`, and the +product's `isCodex` regex can match that path. That says nothing about a +native Codex binary. No allowlist or policy is changed to make it pass. + +**Local evidence** (macOS 27.0.1, Node v24.16.0, not Linux). Tests: +`lib/isolated-observer.test.mjs` (13), `f2-linux-nondumpable/expectations.test.mjs` +(10: protocol order/duplicates/shapes, fault allowlist, forged ticks, changed +instance, ENOENT/TypeError/budget not counted, unrelated refusal, hidden owner, +missing control, malformed payloads and coexisting blockers), +`f2-linux-nondumpable/build.test.mjs` (4). Together with the existing common/M2 +tests, all 38 focused tests passed without skips after integration. The generic tests +cover a 3 s synchronous stall after the handshake under a 300 ms stage bound, +a stall before the handshake (startup timeout), late exit 4, per-instance +unconfirmed exits, 2 MiB without a newline under a 64 KiB cap, a byte-capped +multibyte stderr, a split multibyte line, and a lifetime that does not +interrupt an async stage. + +`plumbing-smoke.mjs` ran against two `build.mjs` builds of `b483cd5` +(unpatched, and with `f4-guard-scope/guard-scope.patch`). It used a +same-user Python flock holder on APFS, not the Linux `codex` fixture. Both +runs passed. Nominal reported all six stages in order with no observer or +protocol faults. `prepare` failed with ENOENT on `/proc`, a macOS artifact. +`stall:prepare` returned `incomplete` at the stage bound with open stage +`prepare`, and the stalled child later exited with code 4 on its own. +`run.mjs` itself refuses non-Linux hosts and was not run on macOS. **None of this is a +Linux result.** + +**Linux isolated measurement, 2026-10-16.** On the approved unprivileged +Linux 7.0.0-1013-oracle aarch64 host (Node 24.18.0), the unpatched +`b483cd587c62329243366e64aaaac477edd36adc` build completed all stages for both +dumpability values. Collection passed, but the expected table failed and the +runner exited 1. `lsof` reported inaccessible overlay/nsfs mounts and warned +that information could be incomplete. The Python fixture also failed the +product's executable/argument verification. The dumpable=1 direct prepare and +guarded participation rows E2/E3 passed, but V/E1/E4 failed. At dumpable=0, +prepare returned EACCES and inspection refused with the participation blocker +(R1/R2 passed), but V failed and there was no passing positive control. +Consequently **no refusal success is counted**. This does not validate a native +Codex candidate or clear either #137 gate. Warnings were not suppressed and no +allowlist or policy was relaxed. + +A second, `stall:prepare` run exited 1 and recorded both inspectors as +incomplete at the stage bound with open stage `prepare`. Both later exited 4 +without a signal; holders exited 0. This validates the parent observation bound +for an injected finite stall, not cancellation or a real kernel hang. Raw +hashes and build identity are in [evidence.md](evidence.md#isolated-f2-linux-measurement-2026-10-16). + +Reproduction commands (a mismatch must remain nonzero): + +```sh +node experimental/126-guard-liveness/f2-linux-nondumpable/build.mjs \ + --repo --commit b483cd587c62329243366e64aaaac477edd36adc --out "$BUILD" +F2_BUILD="$BUILD" node experimental/126-guard-liveness/f2-linux-nondumpable/run.mjs +F2_BUILD="$BUILD" F2_CHILD_FAULT=stall:prepare node experimental/126-guard-liveness/f2-linux-nondumpable/run.mjs # must exit nonzero, incomplete 1+ +``` + +## #126 acceptance matrix (status at this revision) + +`done` means implemented with a real-lock test that runs somewhere in CI or +locally. `partial` means implemented without the required execution or with +a documented residual. `open` means not addressed. + +| #126 criterion / case | Status | Evidence | Missing | +| --- | --- | --- | --- | +| Reviewed contract against pinned Codex source and measured cases | partial | #137 `docs/linux-lock-attribution.md` | final review; native Codex dumpability | +| Bounded enumeration, size, time | partial | 256 FDs, 64 KiB, 25 ms checked between calls | no hard syscall deadline (F3, F4) | +| Only held exclusive whole-file FLOCK counts | done | `parseFlockRows` rejects POSIX/OFDLCK/LEASE/`->`/READ; v6.17 `show_fd_locks` lists granted entries | — | +| Acquisition PID not required to match | done | inherited-FD real-lock test (Linux CI) | — | +| Denied evidence (non-dumpable) | partial | access matrix and isolated run measured (F2); R1/R2 observed; simulated fdinfo denial in `signal-evidence.test.mjs` | passing native-candidate control; current isolated expected table failed | +| Malformed/truncated proc records | partial | parser unit tests; 64 KiB overflow throws | no real truncated-proc fixture | +| PID/start-time change | partial | start ticks in prepare and guard; forged-ticks unit tests | PID-reuse window at `kill` (no pidfd) | +| Namespace mismatch | partial | `/proc/self` and mount ns checks, unit test; observer now records ns readlinks | no real foreign-namespace fixture | +| FD replacement / inode replacement | partial | replacement-inode guard test, descriptor-list change test | real FD-reuse race fixture; fdinfo `show_fdinfo` wait on a reused FD (F4) | +| Budget exceeded | partial | `filterCandidateDescriptors` budget unit test | no real slow-proc fixture (cannot be induced safely) | +| Inherited-description ambiguity defined | open | docs say participation ≠ sole control | eligibility and release-verification rule for shared descriptions | +| Final boundary: no await/spawn, guard not held across signal | partial | #137 structure | F4's 25 ms check covers only its confirm step, not evidence age or release-to-signal; not applied | +| Refusals send zero SIGTERM | done | nonlocking-candidate and fdinfo-denial tests count signals | — | +| JSON/exit compatibility | partial | additive optional v1 fields with schema | reviewer confirmation | +| Guard liveness (#137 gate 2) | partial | F4 proposal removes cmdline/exe and stat via FD from the guard (not applied) | fdinfo `show_fdinfo` and lock filesystem waits remain; Linux run | +| Native access (#137 gate 1) | open | F2 access matrix and isolated synthetic run; expected table failed | native binary dumpability and passing candidate control not measured | + +## F3: indefinite waits, source limits only + +No kernel wait was induced. The following comes from source reasoning only: + +- `/proc/PID/cmdline` reaches `access_remote_vm`, which takes the target's + `mmap_lock` for read (killable) and may page in argument memory. A target + that holds, or is queued behind a writer of, its `mmap_lock` can stall the + reader. +- Every other read #137 makes is also a synchronous syscall with **no hard + bound**: directory reads, readlinks, `stat` through `/proc/PID/fd/N` (which + reaches the target file's filesystem), and `stat`/`statfs` of the lock + path. Some are expected to be short, but none is guaranteed finite. The + 25 ms budget is checked between calls and cannot interrupt one. +- Moving cmdline out of the guard (reading before and after the guard, with no + await before the signal) removes that cmdline coordination-stall path. It does not make + the signal safe: a slow read after the guard leaves the inode, + participation, and PID evidence from inside the guard stale by the time + of the signal. +- #137's command fingerprint hashes the `/proc/PID/exe` readlink **string** + and the cmdline bytes. It does not hash the executable image. +- Keeping current main is the option that keeps functionality. The most + conservative option is a separate one: keep diagnostics and refuse unlock + whenever evidence is insufficient. + +## Run + +```sh +npm ci && npm run build +export RESEARCH_TMP_ROOT="" +node --test experimental/126-guard-liveness/lib/*.test.mjs experimental/126-guard-liveness/m2-macos-fwaslocked/*.test.mjs \ + experimental/126-guard-liveness/f2-linux-nondumpable/*.test.mjs +node experimental/126-guard-liveness/f1-guard-delay/run.mjs +node experimental/126-guard-liveness/m2-macos-fwaslocked/run.mjs +``` + +## Evidence and review revisions + +### Research-only pre-merge review hardening + +The post-`e700770` revision closes three validation gaps without changing +production code or claiming a new measurement: + +- F1 accepts only an absent/empty `F1_FAULT`, `skip_round` or + `waiter_lifetime`. A typo, padded name or other value is rejected before + importing product code, creating scratch or starting a waiter. +- Access-matrix collection now checks the observer's PID, FD, UID, kernel and + all thirteen typed access outcomes, plus successful start ticks against the + runner's instance sample. Empty/partial/malformed JSON cannot count as + collection. Explicit errno outcomes, empty observed lists or false file + comparisons are collected observations, not successful permission/owner + hypotheses. Collection and expectation satisfaction remain separate. +- The research builder rejects unknown, duplicate, positional, valueless and + blank CLI arguments before inspecting a repository or creating output. + `--patch` stays optional, but once present must have a nonempty value. + +The 49 focused tests passed on macOS (no skips), including parser/collection +counterexamples. `npm ci`, type-check, lint, the existing repository suite +(263 pass, three Linux-only skips), package smoke and whitespace checks also +passed. This revision was not run on the US host or with native Codex; all +earlier measurements below retain their original source identities. + +The subsequent current-head review also tightened two runner verdicts. An M2 +part-2 scenario without usable owner/filter evidence now fails the run instead +of merely recording an unavailable measurement. The nominal F2 plumbing smoke +now rejects every protocol/stage fault, including later malformed payloads and +stage errors outside the explicitly allowed `prepare` outcome. These changes +passed 56 focused tests on macOS without skips and the same five repository +checks (263 passing tests, three Linux-only skips). Only deterministic verdict +tests were added; F1/M2 nominal measurements, Linux runs and native Codex were +not rerun for these corrections. + +A further review round replaced the literal-import regex with a comment-aware +JavaScript parser, validated nested inspection probes, and bound every child +build digest to the parent's startup manifest. M2 now refuses non-macOS hosts, +rejects failed/noisy wrapped diagnostics, and compares every Part 2 scenario +against the fixed recorded expectations. F2 access-only and build-mode exit +contracts are documented separately above. All 71 focused tests passed on +macOS without skips; the five repository checks passed (266 passing tests, +three Linux-only skips). These checks alone do not reproduce the nominal +experiments or establish native Codex safety. + +The next review round adds fixed expectations for all seven M2 Part 1 cases, +requires inspection's own acquired coordination guard for E4, and refuses a +missing status-UID observation. Product imports now use the private verified +runtime snapshot described above; canonical module identities close relative +path aliases and refuse escapes/absolute URL imports. The five repository +checks passed again (266 passing tests, three Linux-only skips), along with +84 focused research tests without skips. These remain research validation, +not approval of the #137 product prototype or a complete ownership proof. + +Original measurement files and validation files remain private because they +contain local paths and process identifiers. Their SHA-256 values and sanitized +outcome summaries are recorded in [evidence.md](evidence.md). The original +measurement and later runner validation are separate revisions. + +Independent read-only CLI reviews used Claude Code 2.1.296 with +`claude-opus-5-5` (actual response model confirmed) and Codex CLI 0.162.1 with +its isolated default selection (actual model unconfirmed), in parallel on +revision `e782e396df3f5feebb556bfe2f371d101176e03f`. Input SHA-256: +`70db601c2d878f86621c2f153758b89ea1fbe2973ee60f7cee6deda1ff0fa6da`. +Both completed successfully without running tests. Later corrections were +checked by the integrating Codex. At `e5cf6dd`, the focused tests and one +F1/M2 nominal run each plus the five documented faults were executed; all +seven runner exit statuses and failure/incomplete records matched expectations. +The cooperating session also reviewed that revision read-only. The CLI reviews +do not certify later commits. F2's access matrix is now measured on one Linux +host; product integration and adoption remain unresolved. Barrier contention can invalidate a nominal scenario +and produce a nonzero result; successful runs do not establish a failure rate. +Cleanup can report both an earlier incomplete wait and a later nonzero exit +for the same child. These are observations, not counts of distinct processes. diff --git a/experimental/126-guard-liveness/evidence.md b/experimental/126-guard-liveness/evidence.md new file mode 100644 index 0000000..cb90ab8 --- /dev/null +++ b/experimental/126-guard-liveness/evidence.md @@ -0,0 +1,209 @@ +# Sanitized research evidence + +Original F1/M2 measurements and local validation use same-user macOS/APFS +synthetic fixtures. Linux measurements are identified separately below. No +actual Codex session or Windows recovery was exercised. Raw result files were hash-checked and retained +privately; hashes identify the original files, not this sanitized summary. + +## Snapshot and complete-control validation at cd9bb2e + +Runner source: `cd9bb2ed5018b34ad8c2ed9f2341b8cede7282db`. The follow-up +documentation commit records this execution without changing runner bytes. + +- All five repository checks passed again: 266 passing tests, three Linux-only + skips, zero failures. All 84 focused research tests passed without skips. +- One M2 nominal repetition exited 0: all seven Part 1 matrix cases and four + Part 2 scenarios matched their now-enforced fixed expectations, with + failures/incomplete zero and recorded diagnostic signals zero. The former + holder remained the documented refusal-filter counterexample. This does not + establish a real-world failure frequency or positive holder attribution. +- One F2 macOS plumbing run using the private verified runtime-copy path + exited 0. Nominal stages had no observer/protocol faults; macOS `prepare` + returned the expected `/proc` ENOENT. Injected `stall:prepare` was incomplete + at that stage, then the child exited 4 with no signal. Startup manifest was + the same unpatched `b483cd5` build identified below. No deadline was raised + to make the snapshot path pass. +- This does not add Linux/native Codex or Windows evidence. Private read-only + copies isolate later mutations of the supplied build, not malicious + same-UID tampering; dependency JavaScript is copied but not authenticated by + the original manifest. Static import-walk limits remain documented. + +| Private raw file | SHA-256 | +| --- | --- | +| M2 nominal result | 8dc60f9e04e9c07df3bf3d9c4715bf25445a013263d58b3541a0738382bf481b | +| M2 nominal events | 83d1fd5c704fb364e792f218b6135c6de261ca6f802f4d3a4188d0fa8c42f510 | +| F2 plumbing result | 46f8883a56e1d2fac7ce928f87c30eacacec84c009da64d539ec179e4a7324c4 | + +## Pre-merge validation at 28925a5 + +Runner source: `28925a50c1df118a09d950237bf3db0012751b4c`, after the +comment-aware import walk, nested probe validation, startup-build binding and +fixed M2 scenario verdict corrections. The final follow-up commit changes +only this evidence record. + +- Five repository checks passed: 269 tests total, 266 passed, three Linux-only + skips, zero failures. All 71 focused research tests passed without skips. + The first lint attempt found a missing test-only `URL` import; it was fixed + before the successful rerun and before this source commit. +- M2 nominal, `M2_REPEATS=1`: actual exit 0, seven Part 1 and four Part 2 + scenarios matched, failures/incomplete zero. The all-scan failed acquirer was + an eligible wrong target and the filter rejected it; the retained-descriptor + former holder remained an eligible wrong target and was not excluded. The + visible-holder control matched. Recorded diagnostic signals were zero; + product diagnostic timeout signals were not independently instrumented. +- F2 plumbing, same-user APFS synthetic holder: actual exit 0. All six nominal + stages were valid, with the expected macOS `prepare` ENOENT because `/proc` + is absent. Injected `stall:prepare` was incomplete at the stage bound and + the child later exited 4 without a signal. This is a wiring/fault check, not + Linux access-policy or native Codex evidence. +- The unpatched research build was reverified: source + `b483cd587c62329243366e64aaaac477edd36adc`, startup manifest SHA-256 + `15ffaada53a40b8dc77dc81bed4f7f0433bc05bfc5730e103269b32b996138a0`. + No new Linux-host measurement, F1 measurement, M2 fault-mode run or real + Codex observation was performed for this revision. + +| Private raw file | SHA-256 | +| --- | --- | +| M2 nominal result | def54ec1fe49f6d140cd849edcd765350a3e8e5af445ebd89fc524302e623f8b | +| M2 nominal events | c1ab2c9e63dbd4d1dc291d520c288d0dca3ef7f44ee9bef711ec137d46b1d008 | +| F2 plumbing result | a05553fb60c4b3279a8c0b48741328fe13ecd71a7e38c48a23560d4de8f9d1ac | + +## Original measurements + +| File | SHA-256 | +| --- | --- | +| F1 result, 50 rounds | 2e27c2155a03e028cb24634feedf6b735209935a370952031c871a9a0ddb6ad4 | +| M2 run2 result | 928f8e94ed2960d0cad0a148f5f2386314d28c6be66b14160c95f8023abbf714 | +| M2 run2 events | 7c28f7fa9ecbe6ec6c9f488b3ac12d7219d4000d113229b582eca14583ba8c8c | +| M2 run3 result | 9785e60fb27b339655a995c2a8c9bd59afe2c2189e6bb2c99492b733789f67e5 | +| M2 run3 events | 25e985a5353777514948338d2f84b4e46a03de2dc200162181834309c58ae513 | +| Initial failed M2 attempt | f0a896b04df1a7dbf11af5bc4517c6a7e9540455f72085ca232de793ee5b1d67 | + +F1 contains 25 guarded and 25 baseline rounds, five per delay/mode, with no +recorded errors, waiter exit 0 and unchanged fixture file identities. The +acquisition delta is relative to probe return. Across all 50 rounds it ranges +from 0.013834 to 0.158584 ms. It does not measure actual unlock time. + +All three M2 run3 repetitions show: opener-only barriers are already refused +by the current lock-set check; barriers across all scans allow a wrong eligible +failed acquirer, which the flag filter rejects; an omitted actual holder plus +a former holder with a retained FD allows a wrong eligible target that the +filter does not exclude. Visible-holder controls remain eligible. The omitted +holder and barrier timing are injected; no production failure frequency follows. + +The failed original attempt required one SIGTERM to the research orchestrator, +as reported by the experiment operator. The raw result does not independently +count that signal or every product diagnostic timeout signal. + +## Validation at e782e39 + +| Run | Recorded outcome | SHA-256 | +| --- | --- | --- | +| F1 nominal | ok, 50 rounds, incomplete 0 | e17cd6ab2bfbaadf69ddc54714250b9983199fa7bd205ed7263348ed12a6473f | +| F1 missing round | failed, 2/10 rounds | 2a784f54fce47f46d4662e46bd1acc74dfe78b6d27720c7d4d582bb80ea317a0 | +| F1 waiter lifetime | failed, 9/10 rounds | b2cb35282098ae621387b747662a9f7bbd3643233d5afb6b756565752b49543b | +| M2 early exit | failed before readiness | 531b83a82eedf424f7d387423f9feb12738cea8199eec3cad5aaab90f7e27406 | +| M2 ignored EOF | incomplete 1 | 41f430e268a83c27a6d813578975ae12e1125b51edbc427305997c533592b834 | +| M2 long socket | failed before child start | d1aeffb12240ec06ee230d86cce3106a6e43f6ad79f983d4b762ee665088e5d5 | +| M2 nominal | ok, 33 records, incomplete 0 | adecdb22b9730d86d1ed1cfbf086c795285ca246caf1c169c7daad96af672b6a | + +These files were checked separately from later source corrections. F2 has +syntax checks only at that revision; the later Linux matrix is recorded below. + +## Runner validation at e5cf6dd + +The integrating Codex executed one repetition of each nominal runner and all +five documented fault cases on unchanged script revision +`e5cf6dd7af5f61542a842527a216fa5eec918e61`. It checked actual exit status, +report `ok`, failure text and incomplete observations. No runner timed out in +the outer 60-second observation window. The original measurement tables remain +historical; these checks do not replace the original five/three repetitions. + +| Run | Exit / recorded outcome | Raw result SHA-256 | +| --- | --- | --- | +| F1 nominal | 0 / ok, 10 rounds | 199adc3f3a7708e4d8f2d8d76b51c8ec25eb111e33fddcd2d3dcc749f6419a0a | +| F1 missing round | 1 / reply timeout, 2/10 rounds | 322e1e49c7938bb64456a84421b2ceccce073283fd6b74f309c71615508410fa | +| F1 waiter lifetime | 1 / waiter exit 3, 9/10 rounds | 41e243d667beb8e035c9a3dea525e406be22a71b8d5f086c9115b80aec8f26f7 | +| M2 nominal | 0 / ok, 11 records | d3a20b736e5db1a2af111c51d38ab0cb10daf57fe060479e88d9e98b8866d37b | +| M2 early exit | 1 / EOF before readiness, child exit 1 | 08d95011d17bef5e4be60838a604d57269199fe5d7b9567d41a5ded54114d915 | +| M2 ignored EOF | 1 / incomplete 1, later child exit 3 | c20e6dfcdf96f6de021f64a8fb1a865d5c06f9515ed5ea5991f029e58ef1e8cb | +| M2 long socket | 1 / length rejected before startup | b25e9fa1297b7d1ef510d4d084b489a32f33f0f31d1399de59aa5362bb5d68d0 | + +This validates the measured paths, not every schedule. The later child exit in +the ignored-EOF case is a second observation of the same child; it does not +remove the recorded incomplete wait. No actual Codex session was exercised. + +## F2 Linux access matrix at cfa2eba + +One approved execution on 2026-10-14 used source +`cfa2eba9e3809afa060d85c15ff487f811c925e4`, transferred as a four-file archive. +Archive SHA-256: `cac2171ba9424d229a204b32a140321795fad000b6bef9fbbc709192a154ec4a`; +the remote copy matched before extraction. No dependency installation was needed. + +Environment: Linux 7.0.0-1013-oracle/aarch64, Node 24.18.0, Python 3.12.3, +UID/EUID 1001, permitted/effective capabilities zero, Yama ptrace_scope 1. +Stage and result directories were 0700 and the result was 0600, owned by the +test user. No privilege or host policy change was made. + +Runner exit was 0, `ok=true`, failures 0 and incomplete 0. Both ready records +matched the actual child PID, requested dumpability and observer UID. The +dumpable=1 case read one exclusive FLOCK row and matched FD/lock device+inode. +The dumpable=0 case returned EACCES for FD listing/readlink/stat, fdinfo-file +read and executable readlink. Both cases could read cmdline, start ticks and +status. The fdinfo directory retained the user's UID even with dumpable=0; +this corrects the original source expectation, as explained in the README. + +Raw result SHA-256: `656c1d08d816303dfac870a87e426f4ace0f6972698a46f491a81a4c3b1900fc`. +The downloaded result matched the remote hash. Both holder exit records were +0 with no signal; neither recorded process instance remained at the post-check. +The two owned remote temporary directories were then removed; source archive, +raw result and verification records remain private on the integrating host. + +This is an access-matrix result only. It does not run the #137 adapter, +establish product `unknown` behavior or candidate recognition, measure native +Codex dumpability, or reproduce a procfs hang. F2_DIST was disabled at that +revision; the later isolated inspection measurement is recorded below. + +## Isolated F2 Linux measurement (2026-10-16) + +Environment: unprivileged same-UID Linux 7.0.0-1013-oracle aarch64, Node +24.18.0, Python 3.12.3, ptrace_scope 1. No actual Codex process or home was +used; holders exited on EOF. No privilege, mount, visibility or host-policy +changes were made. Research dependencies were installed only inside private +build staging, using a staging-local npm cache. + +The product build resolved local commit +`b483cd587c62329243366e64aaaac477edd36adc`, tree +`81d82c471a3a67560d48d4c70bce438f0ad96e29`, with no patch. Its manifest +SHA-256 was `23d5aa1f831ef5cc63e3e1c227955864ff2d3536d61d6a388158103c9f86ec92`; +48 dist files and 14 native-addon files were verified. The manifest does not +hash dependency JavaScript/package metadata or prove upstream/native-binary +identity. The remote/local runner, expectations and isolated-observer hashes +matched before interpreting results: + +| Input | SHA-256 | +| --- | --- | +| run.mjs | `52b94e296c801a33fbd74cda21f41ce87789cd2f9e684764e84f16037f7d2c0e` | +| expectations.mjs | `9c09e8c28a0bc16789ea4de7d86120015d372ffa68ec7abc7472582b9735096c` | +| isolated-observer.mjs | `2d38c1aad5e96193756baed112e415df4a64b7b3c8e96b6e5b4be457debe3a72` | + +| Run | Actual exit / outcome | Private raw result SHA-256 | +| --- | --- | --- | +| No fault injection | 1; collection true, expectations false, incomplete 0, refusalCounted false | `924e85a03bfc0aaedf93e5883619fd002470a028c92ae84e0839cced7c5d8b34` | +| stall:prepare | 1; collection false, both inspectors incomplete at prepare, later exit 4 without signal | `7b0042c45db6690c1fa436f03b1f32a3dd607f47ccef2f6057a1f2b79fd31743` | + +The first run observed E2/E3 at dumpable=1 and R1/R2 at dumpable=0, but failed +V/E1/E4 for the control and V for refusal. `lsof` warned about incomplete +visibility of inaccessible overlay/nsfs mounts; the Python fixture was not +accepted as a Codex executable/argument combination. The control failure means +the refusal is not a passing attribution experiment. This is recorded rather +than repaired by suppressing warnings, loosening product identity rules or +changing the expected table. R2 may coexist with unrelated blockers and is not +proof of the sole cause of refusal. + +All four holder exit records were 0 without signals. Both stalled inspectors +later reported exit 4 without signals. Post-checks found their recorded process +instances absent. Remote/downloaded raw-result hashes matched. Reports remain +private because they contain process IDs and local paths. No production +readiness or F4 liveness claim follows; actual native Codex evidence and the +unresolved shared-description/guard-liveness decisions remain open. diff --git a/experimental/126-guard-liveness/f1-guard-delay/run.mjs b/experimental/126-guard-liveness/f1-guard-delay/run.mjs new file mode 100644 index 0000000..4f0b9a0 --- /dev/null +++ b/experimental/126-guard-liveness/f1-guard-delay/run.mjs @@ -0,0 +1,166 @@ +#!/usr/bin/env node +// F1 (#126 research): does a finite delay inside codex-unlock's native guard +// propagate to a Codex-like process waiting on the same `.coordination.lock`? +// +// The delay is injected through the existing `afterGuardAcquired` test hook of +// `guardedProbeOnce` (dist build of this checkout; src is unchanged) and is a +// bounded `Atomics.wait`. This measures propagation of an injected delay. It is +// not a reproduction of a /proc cmdline or any other kernel wait. +// +// Rounds alternate per delay: `baseline` writes the marker without holding +// the guard, `guarded` writes it inside the guard and then waits `delayMs`. +// The waiter is stopped by closing its stdin and has its own lifetime; this +// runner sends no signal. Any error, missing or mismatched round, or +// unconfirmed child exit makes the run exit nonzero and keeps its scratch. +// +// F1_FAULT (fault checks only): `skip_round` makes the waiter never answer +// round 3; `waiter_lifetime` gives the waiter a 300 ms lifetime. +import { log } from 'node:console' +import fs from 'node:fs' +import os from 'node:os' +import path from 'node:path' +import process from 'node:process' +import { setTimeout as delay } from 'node:timers/promises' +import { fileURLToPath } from 'node:url' +import { freshPrivateDir, startChild, stopAll, writeNew } from '../lib/common.mjs' +import { parseF1Fault } from '../lib/f1-options.mjs' + +const DIR = path.dirname(fileURLToPath(import.meta.url)) +const ROOT = path.resolve(DIR, '../../..') +const THREAD_ID = '01a089e8-3731-7202-ba68-0f4b0a3b2711' +const DELAYS = [0, 10, 50, 200, 1000] +const REPEATS = Number(process.env.F1_REPEATS ?? 5) +if (!Number.isInteger(REPEATS) || REPEATS < 1 || REPEATS > 10) throw new Error('F1_REPEATS must be an integer from 1 to 10') +const WAITER_DEADLINE_MS = 5000 +const REPLY_BUDGET_MS = WAITER_DEADLINE_MS + 2000 +// Validate before importing the product, creating scratch or starting a waiter. +const FAULT = parseF1Fault(process.env.F1_FAULT) + +const { guardedProbeOnce } = await import(path.join(ROOT, 'dist', 'native-coordination.js')) +const sleeper = new Int32Array(new SharedArrayBuffer(4)) +const now = () => process.hrtime.bigint() +const ms = (a, b) => Number(b - a) / 1e6 + +const out = freshPrivateDir('126-f1') +const home = path.join(out, 'home') +const lockDir = path.join(home, 'thread-writer-locks') +const markers = path.join(out, 'markers') +fs.mkdirSync(lockDir, { recursive: true, mode: 0o700 }) +fs.mkdirSync(markers, { mode: 0o700 }) +const guardPath = path.join(lockDir, '.coordination.lock') +const lockPath = path.join(lockDir, `${THREAD_ID}.lock`) +writeNew(guardPath, '') +writeNew(lockPath, '') +const identity = (p) => { const s = fs.statSync(p, { bigint: true }); return `${s.dev}:${s.ino}:${s.size}:${s.mtimeNs}` } +const before = { guard: identity(guardPath), lock: identity(lockPath) } + +const live = new Set() +const failures = [] +const results = [] +const expectedRounds = REPEATS * DELAYS.length * 2 +const waiter = startChild(process.execPath, [path.join(DIR, 'waiter.mjs')], { + env: { + FIXTURE_GUARD: guardPath, FIXTURE_MARKER_DIR: markers, + FIXTURE_DEADLINE_MS: String(WAITER_DEADLINE_MS), + FIXTURE_LIFETIME_MS: FAULT === 'waiter_lifetime' ? '300' : String(expectedRounds * REPLY_BUDGET_MS), + FIXTURE_FAULT_SKIP_ROUND: FAULT === 'skip_round' ? '3' : '0', + FIXTURE_REQUIRE_FROM: path.join(ROOT, 'node_modules', 'fs-ext-extra-prebuilt', 'package.json'), + }, +}, live) + +const queue = [] +let wake = null +async function nextLine() { + const limit = Date.now() + REPLY_BUDGET_MS + while (!queue.length) { + if (Date.now() > limit) throw new Error('waiter reply timeout') + if (waiter.child.exitCode !== null) throw new Error(`waiter exited ${waiter.child.exitCode}`) + await Promise.race([new Promise((resolve) => { wake = resolve }), delay(50)]) + } + return queue.shift() +} + +let n = 0 +try { + if (await waiter.ready !== 'ready') throw new Error('waiter not ready') + waiter.lines.on('line', (line) => { queue.push(line); wake?.() }) + for (let rep = 0; rep < REPEATS; rep += 1) { + for (const delayMs of DELAYS) { + for (const mode of ['baseline', 'guarded']) { + n += 1 + waiter.child.stdin.write(`round ${n}\n`) + if (await nextLine() !== `armed ${n}`) throw new Error(`waiter not armed for round ${n}`) + const marker = path.join(markers, `marker-${n}`) + let guardAcquired = null + let hookEnd = null + let probe = null + const start = now() + if (mode === 'baseline') { + writeNew(marker, '') + } else { + probe = guardedProbeOnce(lockPath, undefined, { + afterGuardAcquired: () => { + guardAcquired = now() + writeNew(marker, '') + Atomics.wait(sleeper, 0, 0, delayMs) + hookEnd = now() + }, + }) + } + const returned = now() + const reply = JSON.parse(await nextLine()) + if (reply.round !== n || reply.outcome !== 'acquired' || + !Number.isFinite(reply.blockedMs) || reply.blockedMs < 0 || + typeof reply.acquiredNs !== 'string' || !/^\d+$/.test(reply.acquiredNs)) { + failures.push(`round ${n}: ${JSON.stringify(reply)}`) + } + if (mode === 'guarded' && (probe?.status !== 'free' || probe?.guard?.status !== 'acquired')) { + failures.push(`round ${n}: probe ${JSON.stringify(probe)}`) + } + results.push({ + round: n, rep, mode, delayMs, + probeStatus: probe?.status ?? null, guardStatus: probe?.guard?.status ?? null, + hookMs: guardAcquired !== null ? ms(guardAcquired, hookEnd) : null, + guardHeldUntilReturnMs: guardAcquired !== null ? ms(guardAcquired, returned) : null, + callMs: ms(start, returned), + waiter: reply, + // Waiter acquisition minus this runner's return from the call (or + // marker write for baseline). The guard's LOCK_UN happens before the + // return, so this is not a guard-release delay and may be negative. + acquireMinusReturnMs: reply.acquiredNs ? ms(returned, BigInt(reply.acquiredNs)) : null, + }) + } + } + } +} catch (error) { + failures.push(String(error?.stack ?? error)) +} +const stops = await stopAll(live) +const incomplete = stops.filter((s) => s.incomplete) +for (const stop of stops) if (!stop.incomplete && (stop.exitCode !== 0 || stop.error)) failures.push(`waiter exit ${JSON.stringify(stop)}`) +if (results.length !== expectedRounds) failures.push(`rounds ${results.length}/${expectedRounds}`) + +const after = { guard: identity(guardPath), lock: identity(lockPath) } +if (JSON.stringify(before) !== JSON.stringify(after)) failures.push('native file identity changed') +const grouped = {} +for (const r of results) (grouped[`${r.mode}:${r.delayMs}`] ??= []).push(r) +const stats = Object.fromEntries(Object.entries(grouped).map(([k, rows]) => { + const blocked = rows.map((r) => r.waiter.blockedMs).filter((x) => typeof x === 'number').sort((a, b) => a - b) + const deltas = rows.map((r) => r.acquireMinusReturnMs).filter((x) => typeof x === 'number').sort((a, b) => a - b) + return [k, { + n: rows.length, + blockedMs: { min: blocked[0], median: blocked[Math.floor(blocked.length / 2)], max: blocked.at(-1) }, + acquireMinusReturnMs: { min: deltas[0], max: deltas.at(-1), negative: deltas.filter((x) => x < 0).length }, + }] +})) +const ok = failures.length === 0 && incomplete.length === 0 +const report = { + ok, fault: FAULT, failures, incomplete, + environment: { platform: process.platform, release: os.release(), arch: process.arch, node: process.version, commit: process.env.F1_COMMIT ?? null }, + nativeFilesUnchanged: JSON.stringify(before) === JSON.stringify(after), + stats, results, +} +writeNew(path.join(out, 'f1-results.json'), JSON.stringify(report, null, 2) + '\n') +log(JSON.stringify({ out, ok, fault: FAULT, failures, incomplete, stats }, null, 2)) +// Scratch is always kept: it holds the results and, on failure, the evidence. +process.exitCode = ok ? 0 : 1 diff --git a/experimental/126-guard-liveness/f1-guard-delay/waiter.mjs b/experimental/126-guard-liveness/f1-guard-delay/waiter.mjs new file mode 100644 index 0000000..c0ccc03 --- /dev/null +++ b/experimental/126-guard-liveness/f1-guard-delay/waiter.mjs @@ -0,0 +1,88 @@ +#!/usr/bin/env node +// Codex-like coordination waiter (#126 research, F1). Real Codex waits on +// `.coordination.lock` with a blocking exclusive flock. This fixture polls a +// nonblocking flock instead so that it can enforce its own deadline: a blocking +// wait has no bound of its own and could outlive the experiment. +// +// Per round: read `round N` from stdin, reply `armed N`, wait until the +// orchestrator's marker file exists (written while the orchestrator holds the +// guard), then poll the guard until it is acquired or the deadline passes. +// Release immediately and print one JSON line. stdin EOF ends the process. +// FIXTURE_LIFETIME_MS bounds the whole process: when it passes, the waiter +// exits on its own with code 3. FIXTURE_FAULT_SKIP_ROUND (fault checks only) +// arms that round but never answers it. +import fs from 'node:fs' +import { createRequire } from 'node:module' +import path from 'node:path' +import process from 'node:process' +import readline from 'node:readline' +import { setImmediate, setTimeout } from 'node:timers' + +const require = createRequire(process.env.FIXTURE_REQUIRE_FROM) +const { flockSync } = require('fs-ext-extra-prebuilt') + +const guardPath = process.env.FIXTURE_GUARD +const markerDir = process.env.FIXTURE_MARKER_DIR +const deadlineMs = Number(process.env.FIXTURE_DEADLINE_MS) +const lifetimeMs = Number(process.env.FIXTURE_LIFETIME_MS ?? 120000) +const skipRound = Number(process.env.FIXTURE_FAULT_SKIP_ROUND ?? 0) +const POLL_MS = 0.1 +const sleeper = new Int32Array(new SharedArrayBuffer(4)) +const nap = () => Atomics.wait(sleeper, 0, 0, POLL_MS) +const now = () => process.hrtime.bigint() +const ms = (a, b) => Number(b - a) / 1e6 +// The lifetime is enforced in the polling loops themselves: a timer alone +// cannot fire while a synchronous round is running. +const processDeadline = now() + BigInt(Math.round(lifetimeMs * 1e6)) +const checkLifetime = () => { if (now() > processDeadline) process.exit(3) } + +function round(n) { + const start = now() + const limit = start + BigInt(Math.round(deadlineMs * 1e6)) + const marker = path.join(markerDir, `marker-${n}`) + while (!fs.existsSync(marker)) { + checkLifetime() + if (now() > limit) return { round: n, outcome: 'marker_timeout' } + nap() + } + const markerSeen = now() + const fd = fs.openSync(guardPath, 'r') + let polls = 0 + try { + while (true) { + polls += 1 + try { + flockSync(fd, 'exnb') + break + } catch (error) { + if (error.code !== 'EAGAIN' && error.code !== 'EWOULDBLOCK') throw error + } + checkLifetime() + if (now() > limit) return { round: n, outcome: 'acquire_timeout', polls } + nap() + } + const acquired = now() + flockSync(fd, 'un') + return { + round: n, outcome: 'acquired', polls, + markerSeenNs: String(markerSeen), acquiredNs: String(acquired), + blockedMs: ms(markerSeen, acquired), + } + } finally { + fs.closeSync(fd) + } +} + +const lines = readline.createInterface({ input: process.stdin }) +lines.on('line', (line) => { + const match = /^round (\d+)$/.exec(line) + if (!match) return + checkLifetime() + process.stdout.write(`armed ${match[1]}\n`) + if (Number(match[1]) === skipRound) return + // Let the reply flush before the synchronous wait starts. + setImmediate(() => process.stdout.write(JSON.stringify(round(Number(match[1]))) + '\n')) +}) +lines.on('close', () => process.exit(0)) +setTimeout(() => process.exit(3), lifetimeMs).unref() +process.stdout.write('ready\n') diff --git a/experimental/126-guard-liveness/f2-linux-nondumpable/access-observation.mjs b/experimental/126-guard-liveness/f2-linux-nondumpable/access-observation.mjs new file mode 100644 index 0000000..42fc31f --- /dev/null +++ b/experimental/126-guard-liveness/f2-linux-nondumpable/access-observation.mjs @@ -0,0 +1,56 @@ +// Collection contract for observe.py's JSON result. This validates that each +// access attempt was recorded for the requested fixture instance. An errno, +// false identity comparison, or empty lock list is collected evidence, not a +// passing permission/lock hypothesis. No OS or product operations run here. +const object = (value) => value !== null && typeof value === 'object' && !Array.isArray(value) +const integer = (value) => Number.isSafeInteger(value) && value >= 0 +const text = (value) => typeof value === 'string' && value.length > 0 +const decimal = (value) => typeof value === 'string' && /^(0|[1-9]\d*)$/.test(value) +const arrayOf = (test) => (value) => Array.isArray(value) && value.every(test) +const ROWS = Object.freeze({ + proc_dir_uid: integer, + fd_dir_uid: integer, + fdinfo_dir_uid: integer, + self_ns_mnt_readlink: text, + ns_mnt_readlink: text, + fd_listdir: (value) => arrayOf(decimal)(value) && new Set(value).size === value.length, + fd_readlink: text, + fd_stat_matches_lock: (value) => Array.isArray(value) && value.length === 2 && value.every((v) => typeof v === 'boolean'), + fdinfo_lock_rows: arrayOf((value) => typeof value === 'string' && value.startsWith('lock:')), + exe_readlink: text, + cmdline: (value) => Array.isArray(value) && value.length > 0 && value.every((v) => typeof v === 'string'), + stat_start_ticks: decimal, + status_uid_line: (value) => Array.isArray(value) && value.length === 1 && + value.every((v) => typeof v === 'string' && /^Uid:\s+\d+\s+\d+\s+\d+\s+\d+\s*$/.test(v)), +}) + +export function validateAccessObservation(value, expected) { + const faults = [] + if (!object(expected) || !integer(expected.pid) || expected.pid === 0 || + !integer(expected.fd) || !integer(expected.uid) || !text(expected.kernel) || !decimal(expected.startTicks)) { + return { collected: false, faults: ['expected_identity_invalid'] } + } + if (!object(value)) return { collected: false, faults: ['observation_not_object'] } + const keys = ['pid', 'fd', 'observer_uid', 'kernel', ...Object.keys(ROWS)] + if (Object.keys(value).some((key) => !keys.includes(key))) faults.push('unexpected_field') + for (const [key, want] of [['pid', expected.pid], ['fd', expected.fd], ['observer_uid', expected.uid], ['kernel', expected.kernel]]) { + if (!Object.hasOwn(value, key) || value[key] !== want) faults.push(`${key}_unbound`) + } + for (const [name, valid] of Object.entries(ROWS)) { + if (!Object.hasOwn(value, name)) { faults.push(`${name}:missing`); continue } + const row = value[name] + if (!object(row)) { faults.push(`${name}:not_object`); continue } + const fields = Object.keys(row) + if (fields.length !== 1 || (fields[0] !== 'ok' && fields[0] !== 'errno')) { + faults.push(`${name}:outcome_invalid`) + continue + } + if (fields[0] === 'errno') { + // Python errno.errorcode emits a symbolic name, falling back to the + // decimal errno string for an unmapped native error. + if (typeof row.errno !== 'string' || !/^(E[A-Z0-9_]+|[1-9]\d*)$/.test(row.errno)) faults.push(`${name}:errno_invalid`) + } else if (!valid(row.ok)) faults.push(`${name}:value_invalid`) + } + if (decimal(value.stat_start_ticks?.ok) && value.stat_start_ticks.ok !== expected.startTicks) faults.push('stat_start_ticks:instance_changed') + return { collected: faults.length === 0, faults } +} diff --git a/experimental/126-guard-liveness/f2-linux-nondumpable/access-observation.test.mjs b/experimental/126-guard-liveness/f2-linux-nondumpable/access-observation.test.mjs new file mode 100644 index 0000000..f468294 --- /dev/null +++ b/experimental/126-guard-liveness/f2-linux-nondumpable/access-observation.test.mjs @@ -0,0 +1,88 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' +import { validateAccessObservation } from './access-observation.mjs' + +const expected = { pid: 123, fd: 4, uid: 1001, kernel: '6.17-test', startTicks: '123456' } +function complete() { + return { + pid: expected.pid, fd: expected.fd, observer_uid: expected.uid, kernel: expected.kernel, + proc_dir_uid: { ok: 1001 }, fd_dir_uid: { ok: 1001 }, fdinfo_dir_uid: { ok: 1001 }, + self_ns_mnt_readlink: { ok: 'mnt:[4026531841]' }, ns_mnt_readlink: { ok: 'mnt:[4026531841]' }, + fd_listdir: { ok: ['0', '1', '2', '4'] }, fd_readlink: { ok: '/tmp/synthetic/thread.lock' }, + fd_stat_matches_lock: { ok: [true, true] }, fdinfo_lock_rows: { ok: ['lock:\t1: FLOCK ADVISORY WRITE 123 00:01:1 0 EOF'] }, + exe_readlink: { ok: '/usr/bin/python3' }, cmdline: { ok: ['python3', '/tmp/synthetic/codex', ''] }, + stat_start_ticks: { ok: expected.startTicks }, status_uid_line: { ok: ['Uid:\t1001\t1001\t1001\t1001'] }, + } +} + +test('complete access attempts are collected without asserting permission outcomes', () => { + assert.deepEqual(validateAccessObservation(complete(), expected), { collected: true, faults: [] }) + const observed = complete() + observed.fd_stat_matches_lock.ok = [false, false] + observed.fdinfo_lock_rows.ok = [] + observed.fd_listdir.ok = [] + assert.equal(validateAccessObservation(observed, expected).collected, true) + for (const name of Object.keys(observed).filter((key) => !['pid', 'fd', 'observer_uid', 'kernel'].includes(key))) { + observed[name] = { errno: 'EACCES' } + } + assert.equal(validateAccessObservation(observed, expected).collected, true) + observed.fd_readlink = { errno: '1234' } + assert.equal(validateAccessObservation(observed, expected).collected, true) +}) + +test('parseable empty, primitive, array and partial observations cannot collect successfully', () => { + for (const observed of [{}, null, [], true, 1, 'ok', { pid: 123, fd: 4 }]) { + assert.equal(validateAccessObservation(observed, expected).collected, false) + } + for (const name of Object.keys(complete())) { + const observed = complete() + delete observed[name] + assert.equal(validateAccessObservation(observed, expected).collected, false, name) + } +}) + +test('PID, FD, observer UID, kernel and successful start ticks bind to runner evidence', () => { + for (const patch of [{ pid: 124 }, { pid: '123' }, { fd: 5 }, { fd: '4' }, + { observer_uid: 0 }, { kernel: 'other' }, { stat_start_ticks: { ok: '123457' } }]) { + assert.equal(validateAccessObservation({ ...complete(), ...patch }, expected).collected, false) + } + for (const bad of [null, {}, { ...expected, startTicks: null }, { ...expected, pid: 0 }]) { + assert.equal(validateAccessObservation(complete(), bad).collected, false) + } +}) + +test('successful status reads require one complete UID line; errno is a collected attempt', () => { + const line = 'Uid:\t1001\t1001\t1001\t1001' + for (const ok of [[], [line, line], ['Uid: 1001'], ['Uid: one two three four'], [null]]) { + const result = validateAccessObservation({ ...complete(), status_uid_line: { ok } }, expected) + assert.equal(result.collected, false) + assert.ok(result.faults.includes('status_uid_line:value_invalid')) + } + const missing = complete() + delete missing.status_uid_line + assert.equal(validateAccessObservation(missing, expected).collected, false) + for (const errno of ['EACCES', 'EPERM', 'ENOENT']) { + assert.equal(validateAccessObservation({ ...complete(), status_uid_line: { errno } }, expected).collected, true) + } + // Collection checks shape, not whether permission or user-match hypotheses + // passed: a different but fully recorded UID tuple remains evidence. + assert.equal(validateAccessObservation({ ...complete(), status_uid_line: { ok: ['Uid: 2002 2002 2002 2002'] } }, expected).collected, true) +}) + +test('each access row has one outcome with its actual Python output type', () => { + for (const row of [{}, [], null, { ok: '/tmp/x', errno: 'EACCES' }, { errno: 13 }, + { errno: '' }, { errno: null }, { errno: 'permission denied' }, { ok: '/tmp/x', extra: true }]) { + assert.equal(validateAccessObservation({ ...complete(), fd_readlink: row }, expected).collected, false) + } + const malformed = { + proc_dir_uid: '1001', fd_dir_uid: -1, fdinfo_dir_uid: null, self_ns_mnt_readlink: false, + ns_mnt_readlink: '', fd_listdir: [4], fd_readlink: [], fd_stat_matches_lock: [true], + fdinfo_lock_rows: ['not a lock row'], exe_readlink: {}, cmdline: [], stat_start_ticks: 123456, + status_uid_line: ['Uid: invalid'], + } + for (const [name, value] of Object.entries(malformed)) { + assert.equal(validateAccessObservation({ ...complete(), [name]: { ok: value } }, expected).collected, false, name) + } + assert.equal(validateAccessObservation({ ...complete(), fd_listdir: { ok: ['4', '4'] } }, expected).collected, false) + assert.equal(validateAccessObservation({ ...complete(), unexpected: { ok: true } }, expected).collected, false) +}) diff --git a/experimental/126-guard-liveness/f2-linux-nondumpable/build-binding.mjs b/experimental/126-guard-liveness/f2-linux-nondumpable/build-binding.mjs new file mode 100644 index 0000000..3fdd913 --- /dev/null +++ b/experimental/126-guard-liveness/f2-linux-nondumpable/build-binding.mjs @@ -0,0 +1,14 @@ +// Bind each child's rehashed build to the parent's one startup snapshot. +// A coherently replaced build is still drift, even if verifyBuild succeeds. +const isSha256 = (value) => typeof value === 'string' && /^[0-9a-f]{64}$/.test(value) + +export function manifestBindingFaults(actual, expected) { + if (!isSha256(expected)) return ['startup_manifest_sha256_invalid'] + if (!isSha256(actual)) return ['child_manifest_sha256_invalid'] + return actual === expected ? [] : ['startup_manifest_sha256_changed'] +} + +export function bindVerifiedBuild(verified, expected) { + const faults = [...verified.faults, ...manifestBindingFaults(verified.manifestSha256, expected)] + return { ...verified, ok: verified.ok === true && faults.length === 0, faults } +} diff --git a/experimental/126-guard-liveness/f2-linux-nondumpable/build-binding.test.mjs b/experimental/126-guard-liveness/f2-linux-nondumpable/build-binding.test.mjs new file mode 100644 index 0000000..7ef539b --- /dev/null +++ b/experimental/126-guard-liveness/f2-linux-nondumpable/build-binding.test.mjs @@ -0,0 +1,38 @@ +import assert from 'node:assert/strict' +import { readFileSync } from 'node:fs' +import { test } from 'node:test' +import { URL } from 'node:url' +import { bindVerifiedBuild, manifestBindingFaults } from './build-binding.mjs' + +test('a coherent replacement build fails the pre-import startup binding', () => { + const startup = 'a'.repeat(64) + const original = { ok: true, faults: [], manifestSha256: startup, source: { commit: '1'.repeat(40) } } + assert.equal(bindVerifiedBuild(original, startup).ok, true) + const replaced = { ok: true, faults: [], manifestSha256: 'b'.repeat(64), source: { commit: '2'.repeat(40) } } + const checked = bindVerifiedBuild(replaced, startup) + assert.equal(checked.ok, false) + assert.deepEqual(checked.faults, ['startup_manifest_sha256_changed']) + assert.deepEqual(replaced.faults, []) +}) + +test('missing or malformed startup/child digest and verification faults fail closed', () => { + const digest = 'a'.repeat(64) + for (const invalid of [null, undefined, '', 'a', 123, 'A'.repeat(64)]) { + assert.ok(manifestBindingFaults(digest, invalid).length > 0) + assert.ok(manifestBindingFaults(invalid, digest).length > 0) + } + const result = bindVerifiedBuild({ ok: false, faults: ['dist_changed'], manifestSha256: digest }, digest) + assert.equal(result.ok, false) + assert.deepEqual(result.faults, ['dist_changed']) +}) + +test('the child refuses an unbound identity before its product module-load stage', () => { + const source = readFileSync(new URL('./inspect-child.mjs', import.meta.url), 'utf8') + const verification = source.indexOf('snapshot = createBuildSnapshot(build, expectedManifestSha256)') + const refusal = source.indexOf("if (identity.result?.ok !== true) throw new StopStages('build_unverified')") + const loading = source.indexOf("c.stage('module_load'") + assert.ok(verification >= 0 && refusal > verification && loading > refusal) + assert.ok(source.includes('reachableModules(snapshot.dist, ENTRIES)')) + assert.ok(source.includes('path.join(snapshot.dist, name)')) + assert.equal(source.includes("path.join(build, 'src', 'dist')"), false) +}) diff --git a/experimental/126-guard-liveness/f2-linux-nondumpable/build-snapshot.mjs b/experimental/126-guard-liveness/f2-linux-nondumpable/build-snapshot.mjs new file mode 100644 index 0000000..1f827ab --- /dev/null +++ b/experimental/126-guard-liveness/f2-linux-nondumpable/build-snapshot.mjs @@ -0,0 +1,74 @@ +// Isolate the supplied build's later mutations with independent copied bytes. +// The startup manifest authenticates only its existing dist/native/archive/ +// lockfile scope. Dependency JavaScript is copied too, not authenticated by +// that manifest. Read-only modes are not OS immutability: the same UID can +// chmod or replace its own snapshot. No adversarial same-UID guarantee exists. +import { createHash } from 'node:crypto' +import fs from 'node:fs' +import path from 'node:path' +import { freshPrivateDir } from '../lib/common.mjs' +import { bindVerifiedBuild, manifestBindingFaults } from './build-binding.mjs' +import { verifyBuild } from './build.mjs' + +const sha256 = (bytes) => createHash('sha256').update(bytes).digest('hex') +const within = (root, file) => { + const relative = path.relative(root, file) + return relative !== '..' && !relative.startsWith(`..${path.sep}`) && !path.isAbsolute(relative) +} + +export function createBuildSnapshot(build, expectedManifestSha256, options = {}) { + if (manifestBindingFaults(expectedManifestSha256, expectedManifestSha256).length) throw new Error('startup_manifest_sha256_invalid') + const source = fs.realpathSync(build) + if (!fs.statSync(source).isDirectory()) throw new Error('snapshot_source_not_directory') + const directory = freshPrivateDir('126-build-snapshot', options.tmpRoot) + const copy = path.join(directory, 'build') + fs.mkdirSync(copy, { mode: 0o700 }) + const files = [] + const directories = [copy] + const omittedBinDirectories = [] + const copyEntry = (relative) => { + // npm command shims are never executed by the inspection child. Omit + // these directories entirely; all other links/special files are refused. + if (relative === 'src/node_modules/.bin' || relative.endsWith('/node_modules/.bin')) { + omittedBinDirectories.push(relative) + return + } + const input = path.join(source, relative) + const output = path.join(copy, relative) + const stat = fs.lstatSync(input) + if (stat.isSymbolicLink() || !within(source, fs.realpathSync(input))) throw new Error(`snapshot_unsafe_entry:${relative}`) + if (stat.isDirectory()) { + fs.mkdirSync(output, { mode: 0o700 }) + directories.push(output) + for (const entry of fs.readdirSync(input).sort()) copyEntry(`${relative}/${entry}`) + } else if (stat.isFile()) { + const fd = fs.openSync(input, fs.constants.O_RDONLY | fs.constants.O_NOFOLLOW) + try { + if (!fs.fstatSync(fd).isFile()) throw new Error(`snapshot_unsafe_entry:${relative}`) + const bytes = fs.readFileSync(fd) + // New files, never hardlinks or symlinks back to the supplied build. + fs.writeFileSync(output, bytes, { flag: 'wx', mode: 0o600 }) + files.push({ path: relative, size: bytes.length, sha256: sha256(bytes) }) + } finally { fs.closeSync(fd) } + } else throw new Error(`snapshot_unsafe_entry:${relative}`) + } + try { + copyEntry('manifest.json') + copyEntry('source.tar') + if (fs.lstatSync(path.join(source, 'source.patch'), { throwIfNoEntry: false })) copyEntry('source.patch') + fs.mkdirSync(path.join(copy, 'src'), { mode: 0o700 }) + directories.push(path.join(copy, 'src')) + for (const relative of ['src/package.json', 'src/package-lock.json', 'src/dist', 'src/node_modules']) copyEntry(relative) + const verified = bindVerifiedBuild(verifyBuild(copy), expectedManifestSha256) + for (const file of files) fs.chmodSync(path.join(copy, file.path), 0o400) + for (const dir of directories.reverse()) fs.chmodSync(dir, 0o500) + files.sort((a, b) => a.path < b.path ? -1 : a.path > b.path ? 1 : 0) + return { directory, build: copy, dist: fs.realpathSync(path.join(copy, 'src', 'dist')), verified, + copiedContentSha256: sha256(JSON.stringify(files)), fileCount: files.length, omittedBinDirectories } + } catch (cause) { + // Preserve private partial evidence. The caller cannot fall back to input. + const error = new Error(`build_snapshot_failed:${path.basename(directory)}:${cause.message}`, { cause }) + error.snapshotDirectory = directory + throw error + } +} diff --git a/experimental/126-guard-liveness/f2-linux-nondumpable/build-snapshot.test.mjs b/experimental/126-guard-liveness/f2-linux-nondumpable/build-snapshot.test.mjs new file mode 100644 index 0000000..aff5bed --- /dev/null +++ b/experimental/126-guard-liveness/f2-linux-nondumpable/build-snapshot.test.mjs @@ -0,0 +1,139 @@ +import assert from 'node:assert/strict' +import { createHash } from 'node:crypto' +import fs from 'node:fs' +import path from 'node:path' +import { test } from 'node:test' +import { pathToFileURL } from 'node:url' +import { freshPrivateDir } from '../lib/common.mjs' +import { MANIFEST_SCHEMA, contentOf, verifyBuild } from './build.mjs' +import { createBuildSnapshot } from './build-snapshot.mjs' + +const sha256 = (bytes) => createHash('sha256').update(bytes).digest('hex') + +function writeManifest(build) { + const content = contentOf(build) + fs.writeFileSync(path.join(build, 'manifest.json'), JSON.stringify({ + schema: MANIFEST_SCHEMA, + source: { commit: 'a'.repeat(40), tree: 'b'.repeat(40), archiveSha256: sha256(fs.readFileSync(path.join(build, 'source.tar'))) }, + patch: { applied: true, sha256: sha256(fs.readFileSync(path.join(build, 'source.patch'))) }, + lockfileSha256: sha256(fs.readFileSync(path.join(build, 'src', 'package-lock.json'))), + dist: content.dist, native: content.native, + }, null, 2) + '\n') +} + +function fakeBuild() { + const build = freshPrivateDir('126-snapshot-input-test') + const dependency = path.join(build, 'src', 'node_modules', 'fixture-runtime') + fs.mkdirSync(path.join(build, 'src', 'dist'), { recursive: true }) + fs.mkdirSync(dependency, { recursive: true }) + fs.writeFileSync(path.join(build, 'src', 'package.json'), '{"type":"module"}') + fs.writeFileSync(path.join(build, 'src', 'package-lock.json'), '{}') + fs.writeFileSync(path.join(build, 'src', 'dist', 'entry.js'), "import { value } from 'fixture-runtime'; export const result = 'original:' + value\n") + fs.writeFileSync(path.join(dependency, 'package.json'), '{"type":"module","exports":"./index.js"}') + fs.writeFileSync(path.join(dependency, 'index.js'), "export const value = 'dependency-original'\n") + fs.writeFileSync(path.join(dependency, 'addon.node'), 'synthetic native bytes, never loaded') + fs.mkdirSync(path.join(build, 'src', 'node_modules', '.bin')) + fs.symlinkSync('../fixture-runtime/index.js', path.join(build, 'src', 'node_modules', '.bin', 'unused')) + fs.writeFileSync(path.join(build, 'source.tar'), 'archive') + fs.writeFileSync(path.join(build, 'source.patch'), 'patch') + fs.writeFileSync(path.join(build, 'unrelated.txt'), 'outside copy scope') + fs.writeFileSync(path.join(build, 'src', 'unrelated.js'), 'outside copy scope') + writeManifest(build) + return build +} + +// Tests own these directories, including their intentionally read-only copies. +function dispose(directory) { + if (!directory) return + const writable = (file) => { + const stat = fs.lstatSync(file) + if (stat.isDirectory()) { + fs.chmodSync(file, 0o700) + for (const entry of fs.readdirSync(file)) writable(path.join(file, entry)) + } else if (stat.isFile()) fs.chmodSync(file, 0o600) + } + writable(directory) + fs.rmSync(directory, { recursive: true }) +} + +test('snapshot imports independent dist and runtime dependency bytes after coherent input mutation', async () => { + const build = fakeBuild() + let snapshot + try { + const startup = verifyBuild(build) + const manifestBytes = fs.readFileSync(path.join(build, 'manifest.json')) + snapshot = createBuildSnapshot(build, startup.manifestSha256) + assert.equal(snapshot.verified.ok, true) + assert.deepEqual(fs.readFileSync(path.join(snapshot.build, 'manifest.json')), manifestBytes) + assert.equal(fs.statSync(snapshot.directory).mode & 0o777, 0o700) + assert.equal(fs.statSync(snapshot.dist).mode & 0o777, 0o500) + for (const relative of ['manifest.json', 'source.tar', 'source.patch', 'src/package.json', 'src/package-lock.json', + 'src/dist/entry.js', 'src/node_modules/fixture-runtime/index.js', 'src/node_modules/fixture-runtime/addon.node']) { + const original = fs.statSync(path.join(build, relative)) + const copied = fs.statSync(path.join(snapshot.build, relative)) + assert.equal(copied.mode & 0o777, 0o400) + assert.equal(copied.nlink, 1) + assert.ok(copied.dev !== original.dev || copied.ino !== original.ino) + } + assert.equal(fs.existsSync(path.join(snapshot.build, 'unrelated.txt')), false) + assert.equal(fs.existsSync(path.join(snapshot.build, 'src', 'unrelated.js')), false) + assert.equal(fs.existsSync(path.join(snapshot.build, 'src', 'node_modules', '.bin')), false) + assert.deepEqual(snapshot.omittedBinDirectories, ['src/node_modules/.bin']) + + // Change input dist, dependency JS/native bytes and its manifest together. + // This replacement remains internally valid but must not affect imports. + fs.writeFileSync(path.join(build, 'src', 'dist', 'entry.js'), "export const result = 'replacement'\n") + fs.writeFileSync(path.join(build, 'src', 'node_modules', 'fixture-runtime', 'index.js'), "export const value = 'dependency-replacement'\n") + fs.writeFileSync(path.join(build, 'src', 'node_modules', 'fixture-runtime', 'addon.node'), 'replacement native bytes') + writeManifest(build) + assert.equal(verifyBuild(build).ok, true) + assert.notEqual(verifyBuild(build).manifestSha256, startup.manifestSha256) + assert.equal(verifyBuild(snapshot.build).manifestSha256, startup.manifestSha256) + const loaded = await import(pathToFileURL(path.join(snapshot.dist, 'entry.js')).href) + assert.equal(loaded.result, 'original:dependency-original') + assert.equal(fs.readFileSync(path.join(snapshot.build, 'src', 'node_modules', 'fixture-runtime', 'addon.node'), 'utf8'), + 'synthetic native bytes, never loaded') + } finally { dispose(snapshot?.directory); dispose(build) } +}) + +test('a coherent input replacement before copying fails startup-manifest verification', () => { + const build = fakeBuild() + let snapshot + try { + const startup = verifyBuild(build).manifestSha256 + fs.writeFileSync(path.join(build, 'src', 'dist', 'entry.js'), 'export const replacement = true\n') + writeManifest(build) + snapshot = createBuildSnapshot(build, startup) + assert.equal(snapshot.verified.ok, false) + assert.ok(snapshot.verified.faults.includes('startup_manifest_sha256_changed')) + assert.ok(fs.existsSync(snapshot.directory)) + } finally { dispose(snapshot?.directory); dispose(build) } +}) + +test('snapshot refuses links outside unused npm shims and retains private failed evidence', () => { + const build = fakeBuild() + let failedDirectory + try { + const startup = verifyBuild(build).manifestSha256 + fs.symlinkSync('entry.js', path.join(build, 'src', 'dist', 'linked.js')) + assert.throws(() => createBuildSnapshot(build, startup), (error) => { + failedDirectory = error.snapshotDirectory + assert.match(error.message, /snapshot_unsafe_entry:src\/dist\/linked\.js/) + assert.equal(fs.statSync(failedDirectory).mode & 0o777, 0o700) + return true + }) + } finally { dispose(failedDirectory); dispose(build) } +}) + +test('snapshot rehash refuses changed copied content even when manifest bytes still match startup', () => { + const build = fakeBuild() + let snapshot + try { + const startup = verifyBuild(build).manifestSha256 + fs.writeFileSync(path.join(build, 'src', 'dist', 'entry.js'), 'export const replacement = true\n') + snapshot = createBuildSnapshot(build, startup) + assert.equal(snapshot.verified.manifestSha256, startup) + assert.equal(snapshot.verified.ok, false) + assert.ok(snapshot.verified.faults.includes('dist_changed:entry.js')) + } finally { dispose(snapshot?.directory); dispose(build) } +}) diff --git a/experimental/126-guard-liveness/f2-linux-nondumpable/build.mjs b/experimental/126-guard-liveness/f2-linux-nondumpable/build.mjs new file mode 100644 index 0000000..7c11995 --- /dev/null +++ b/experimental/126-guard-liveness/f2-linux-nondumpable/build.mjs @@ -0,0 +1,208 @@ +#!/usr/bin/env node +// Trusted research build for F2 inspection (#126 research). +// +// node build.mjs --repo --commit [--patch ] --out +// +// Provenance recorded here is limited to what this script itself did: +// - `commit` and `tree` are resolved by `git rev-parse` in the named local +// repository, and the source is that repository's `git archive` of the +// resolved commit. A commit string supplied elsewhere is never trusted. +// Whether that local repository matches a remote is not checked. +// - The optional patch is applied with `git apply` and recorded by SHA-256. +// - dist files and native addons (*.node under node_modules) are hashed after +// `npm ci` and `npm run build`. +// verifyBuild later re-hashes the same files. It verifies these dist and +// native-addon bytes, not dependency JavaScript/package metadata or that the +// manifest's source is upstream. A deployed +// native Codex binary is unrelated to this build and remains unverified. +import { createHash } from 'node:crypto' +import { spawnSync } from 'node:child_process' +import fs from 'node:fs' +import path from 'node:path' +import process from 'node:process' +import { fileURLToPath, pathToFileURL, URL } from 'node:url' +import { parseArgs } from 'node:util' +import ts from 'typescript' + +export const MANIFEST_SCHEMA = 'codex-unlock-research-build/1' + +// Validate the entire CLI before prepareBuild can inspect source paths, create +// output, run git/npm, or hash any bytes. A missing patch value must never +// silently select the unpatched build path. +export function parseBuildArgs(args = process.argv.slice(2)) { + const { values, tokens } = parseArgs({ + args, strict: true, allowPositionals: false, tokens: true, + options: { + repo: { type: 'string' }, commit: { type: 'string' }, + patch: { type: 'string' }, out: { type: 'string' }, + }, + }) + const seen = new Set() + for (const token of tokens) { + if (token.kind !== 'option') continue + if (seen.has(token.name)) throw new Error(`duplicate option: --${token.name}`) + seen.add(token.name) + if (!values[token.name].trim()) throw new Error(`--${token.name} requires a nonempty value`) + } + for (const name of ['repo', 'commit', 'out']) { + if (!seen.has(name)) throw new Error(`missing required option: --${name}`) + } + return { repo: values.repo, commit: values.commit, patch: values.patch, out: path.resolve(values.out) } +} + +const sha256 = (data) => createHash('sha256').update(data).digest('hex') +const fileSha256 = (file) => sha256(fs.readFileSync(file)) + +// Sorted regular files under `root`. Any other entry type is a fault. +export function hashTree(root, accept = () => true) { + const files = [] + const faults = [] + const walk = (dir) => { + for (const entry of fs.readdirSync(dir, { withFileTypes: true }).sort((a, b) => (a.name < b.name ? -1 : 1))) { + const full = path.join(dir, entry.name) + const rel = path.relative(root, full).split(path.sep).join('/') + if (entry.isDirectory()) walk(full) + else if (entry.isFile()) { if (accept(rel)) files.push({ path: rel, size: fs.statSync(full).size, sha256: fileSha256(full) }) } + else if (accept(rel)) faults.push(`unsupported_entry:${rel}`) + } + } + walk(root) + return { files, faults } +} + +export function contentOf(buildDir) { + const dist = hashTree(path.join(buildDir, 'src', 'dist')) + const native = hashTree(path.join(buildDir, 'src', 'node_modules'), (rel) => rel.endsWith('.node')) + return { dist: dist.files, native: native.files, faults: [...dist.faults, ...native.faults] } +} + +export function compareContent(expected, actual) { + const mismatches = [] + for (const key of ['dist', 'native']) { + const want = new Map(expected[key].map((f) => [f.path, f])) + const have = new Map(actual[key].map((f) => [f.path, f])) + for (const [p, f] of want) { + const g = have.get(p) + if (!g) mismatches.push(`${key}_missing:${p}`) + else if (g.sha256 !== f.sha256 || g.size !== f.size) mismatches.push(`${key}_changed:${p}`) + } + for (const p of have.keys()) if (!want.has(p)) mismatches.push(`${key}_extra:${p}`) + } + return mismatches +} + +export function verifyBuild(buildDir) { + const manifestPath = path.join(buildDir, 'manifest.json') + const raw = fs.readFileSync(manifestPath) + const manifest = JSON.parse(raw) + const faults = [] + if (manifest.schema !== MANIFEST_SCHEMA) faults.push(`schema:${manifest.schema}`) + if (!/^[0-9a-f]{40}$/.test(manifest.source?.commit ?? '') || !/^[0-9a-f]{40}$/.test(manifest.source?.tree ?? '')) faults.push('source_identity_malformed') + const same = (file, expected, name) => { + try { if (fileSha256(path.join(buildDir, file)) !== expected) faults.push(`${name}_changed`) } catch { faults.push(`${name}_unreadable`) } + } + same('source.tar', manifest.source?.archiveSha256, 'archive') + same(path.join('src', 'package-lock.json'), manifest.lockfileSha256, 'lockfile') + if (manifest.patch?.applied === true) same('source.patch', manifest.patch.sha256, 'patch') + else if (manifest.patch?.applied !== false || fs.existsSync(path.join(buildDir, 'source.patch'))) faults.push('patch_record_invalid') + const actual = contentOf(buildDir) + faults.push(...actual.faults, ...compareContent(manifest, actual)) + if (!actual.dist.length) faults.push('dist_empty') + return { ok: faults.length === 0, faults, manifestSha256: sha256(raw), source: manifest.source, patch: manifest.patch, + distFiles: actual.dist.length, nativeFiles: actual.native.length } +} + +// Relative static imports/re-exports and literal dynamic imports reachable +// from `entries`, parsed as JavaScript using the existing dev dependency. +// This bounded graph does not resolve nonliteral import expressions, require, +// eval, or package imports; it is not proof of arbitrary runtime reachability. +export function reachableModules(dist, entries) { + const root = fs.realpathSync(dist) + if (!fs.statSync(root).isDirectory()) throw new Error('module_root_not_directory') + const seen = new Set() + const pending = entries.map((entry) => path.resolve(root, entry)) + while (pending.length) { + const file = fs.realpathSync(pending.pop()) + const relative = path.relative(root, file) + if (relative === '..' || relative.startsWith(`..${path.sep}`) || path.isAbsolute(relative)) throw new Error('module_outside_dist') + if (!fs.statSync(file).isFile()) throw new Error('module_not_regular_file') + const rel = relative.split(path.sep).join('/') + if (seen.has(rel)) continue + seen.add(rel) + const text = fs.readFileSync(file, 'utf8') + const source = ts.createSourceFile(rel, text, ts.ScriptTarget.Latest, true, ts.ScriptKind.JS) + // Do not treat a partially recovered syntax tree as a complete graph. + if (source.parseDiagnostics.length) throw new Error(`module_parse_failed:${rel}`) + const visit = (node) => { + let specifier + if (ts.isImportDeclaration(node) || ts.isExportDeclaration(node)) specifier = node.moduleSpecifier + else if (ts.isCallExpression(node) && node.expression.kind === ts.SyntaxKind.ImportKeyword) specifier = node.arguments[0] + if (specifier && ts.isStringLiteralLike(specifier)) { + if (/^\.{1,2}\//.test(specifier.text)) { + // Match ESM URL decoding and canonical filesystem identity, including + // ../dist aliases, percent escapes, fragments and in-root symlinks. + pending.push(fileURLToPath(new URL(specifier.text, pathToFileURL(file)))) + } else if (!specifier.text.startsWith('node:') && (path.isAbsolute(specifier.text) || + /^[a-z][a-z0-9+.-]*:/i.test(specifier.text) || specifier.text.startsWith('\\\\'))) { + // Absolute paths and file:/other URL aliases are not bare packages. + throw new Error('module_unsupported_absolute_or_url_import') + } + } + ts.forEachChild(node, visit) + } + visit(source) + } + return [...seen].sort() +} + +function run(command, args, cwd, options = {}) { + const result = spawnSync(command, args, { cwd, encoding: options.encoding ?? 'utf8', timeout: options.timeout ?? 600000, + maxBuffer: 1 << 28, stdio: ['ignore', 'pipe', 'pipe'] }) + if (result.status !== 0) throw new Error(`${command} ${args.join(' ')} failed: ${result.status} ${result.signal ?? ''} ${String(result.stderr).slice(0, 2000)}`) + return result.stdout +} + +export function prepareBuild({ repo, commit, patch, out }) { + if (fs.existsSync(out)) throw new Error(`output exists: ${out}`) + fs.mkdirSync(out, { mode: 0o700 }) + const repoReal = fs.realpathSync(repo) + const resolved = run('git', ['-C', repoReal, 'rev-parse', '--verify', `${commit}^{commit}`], out).trim() + const tree = run('git', ['-C', repoReal, 'rev-parse', '--verify', `${resolved}^{tree}`], out).trim() + const archive = run('git', ['-C', repoReal, 'archive', '--format=tar', resolved], out, { encoding: 'buffer' }) + fs.writeFileSync(path.join(out, 'source.tar'), archive, { flag: 'wx', mode: 0o600 }) + const src = path.join(out, 'src') + fs.mkdirSync(src, { mode: 0o700 }) + run('tar', ['-xf', path.join(out, 'source.tar'), '-C', src], out) + let patchRecord = { applied: false } + if (patch) { + const copy = path.join(out, 'source.patch') + fs.copyFileSync(patch, copy, fs.constants.COPYFILE_EXCL) + run('git', ['apply', '--check', copy], src) + run('git', ['apply', copy], src) + patchRecord = { applied: true, sha256: fileSha256(copy) } + } + run('npm', ['ci', '--no-audit', '--no-fund'], src) + run('npm', ['run', 'build'], src) + const content = contentOf(out) + if (content.faults.length) throw new Error(`unsupported build entries: ${content.faults.join(',')}`) + const manifest = { + schema: MANIFEST_SCHEMA, + source: { repo: repoReal, commit: resolved, tree, archiveSha256: sha256(archive), + resolvedBy: 'git rev-parse and git archive in the named local repository, by build.mjs' }, + patch: patchRecord, + lockfileSha256: fileSha256(path.join(src, 'package-lock.json')), + toolchain: { node: process.version, npm: run('npm', ['--version'], src).trim(), platform: process.platform, arch: process.arch }, + dist: content.dist, + native: content.native, + builtAt: new Date().toISOString(), + } + fs.writeFileSync(path.join(out, 'manifest.json'), JSON.stringify(manifest, null, 2) + '\n', { flag: 'wx', mode: 0o600 }) + return verifyBuild(out) +} + +if (process.argv[1] && fs.realpathSync(process.argv[1]) === fileURLToPath(import.meta.url)) { + const options = parseBuildArgs() + const result = prepareBuild(options) + process.stdout.write(JSON.stringify(result, null, 2) + '\n') + process.exitCode = result.ok ? 0 : 1 +} diff --git a/experimental/126-guard-liveness/f2-linux-nondumpable/build.test.mjs b/experimental/126-guard-liveness/f2-linux-nondumpable/build.test.mjs new file mode 100644 index 0000000..40602e1 --- /dev/null +++ b/experimental/126-guard-liveness/f2-linux-nondumpable/build.test.mjs @@ -0,0 +1,233 @@ +import assert from 'node:assert/strict' +import { spawnSync } from 'node:child_process' +import { createHash } from 'node:crypto' +import fs from 'node:fs' +import path from 'node:path' +import process from 'node:process' +import { test } from 'node:test' +import { fileURLToPath, pathToFileURL, URL } from 'node:url' +import { freshPrivateDir } from '../lib/common.mjs' +import { MANIFEST_SCHEMA, contentOf, parseBuildArgs, reachableModules, verifyBuild } from './build.mjs' + +const sha = (data) => createHash('sha256').update(data).digest('hex') + +const requiredArgs = ['--repo', 'local repo', '--commit', 'HEAD', '--out', 'new build'] + +test('build arguments preserve patched and unpatched invocations and accept explicit equals values', () => { + const want = { repo: 'local repo', commit: 'HEAD', out: path.resolve('new build'), patch: undefined } + assert.deepEqual(parseBuildArgs(requiredArgs), want) + assert.deepEqual(parseBuildArgs([...requiredArgs, '--patch', 'source patch.diff']), { ...want, patch: 'source patch.diff' }) + assert.deepEqual(parseBuildArgs(['--out=new build', '--patch=source patch.diff', '--commit=HEAD', '--repo=local repo']), + { ...want, patch: 'source patch.diff' }) +}) + +test('build arguments refuse unknown options, duplicates and positional arguments', () => { + for (const extra of [['--pach', 'file.diff'], ['--unknown'], ['--patch', 'a', '--patch', 'b'], + ['--repo', 'another'], ['--commit=another'], ['--out=another'], ['extra'], ['--', 'extra'], ['-p', 'patch']]) { + assert.throws(() => parseBuildArgs([...requiredArgs, ...extra]), undefined, JSON.stringify(extra)) + } +}) + +test('every build option requires a nonempty value and the three source/output options are mandatory', () => { + for (const name of ['repo', 'commit', 'out', 'patch']) { + const at = requiredArgs.indexOf(`--${name}`) + const without = at < 0 ? [...requiredArgs] : requiredArgs.filter((_arg, index) => index !== at && index !== at + 1) + // Patch is optional; the other options are required even if all remaining + // values are present and the source/output paths happen to be usable. + if (name !== 'patch') assert.throws(() => parseBuildArgs(without), /missing required option/) + for (const value of ['', ' ']) { + assert.throws(() => parseBuildArgs([...without, `--${name}`, value]), /requires a nonempty value/) + } + assert.throws(() => parseBuildArgs([...without, `--${name}`])) + assert.throws(() => parseBuildArgs([...without, `--${name}`, '--repo=other'])) + } +}) + +test('invalid build CLI arguments fail before creating output or accessing the repository', () => { + const dir = freshPrivateDir('126-build-cli-test') + try { + const out = path.join(dir, 'must-not-exist') + const repo = path.join(dir, 'missing-repository') + const script = fileURLToPath(new URL('./build.mjs', import.meta.url)) + for (const extra of [['--patch'], ['--pach', 'source.patch'], ['--patch', '--out=other'], ['--out', out]]) { + const result = spawnSync(process.execPath, [script, '--repo', repo, '--commit', 'HEAD', '--out', out, ...extra], + { encoding: 'utf8', timeout: 10000 }) + assert.ifError(result.error) + assert.notEqual(result.status, 0) + assert.equal(result.stdout, '') + assert.match(result.stderr, /(?:--patch|--pach|duplicate option: --out)/) + assert.doesNotMatch(result.stderr, /ENOENT|output exists|npm ci/) + assert.equal(fs.existsSync(out), false) + } + } finally { fs.rmSync(dir, { recursive: true }) } +}) + +// A synthetic build tree with a manifest written the way build.mjs writes it. +function fakeBuild({ patch = false } = {}) { + const dir = freshPrivateDir('126-build-test') + fs.mkdirSync(path.join(dir, 'src', 'dist'), { recursive: true }) + fs.mkdirSync(path.join(dir, 'src', 'node_modules', 'addon', 'build'), { recursive: true }) + fs.writeFileSync(path.join(dir, 'src', 'dist', 'a.js'), "import './b.js'\n") + fs.writeFileSync(path.join(dir, 'src', 'dist', 'b.js'), 'export {}\n') + fs.writeFileSync(path.join(dir, 'src', 'node_modules', 'addon', 'build', 'x.node'), 'native') + fs.writeFileSync(path.join(dir, 'src', 'node_modules', 'addon', 'index.js'), 'ignored') + fs.writeFileSync(path.join(dir, 'src', 'package-lock.json'), '{}') + fs.writeFileSync(path.join(dir, 'source.tar'), 'tar') + if (patch) fs.writeFileSync(path.join(dir, 'source.patch'), 'patch') + const content = contentOf(dir) + fs.writeFileSync(path.join(dir, 'manifest.json'), JSON.stringify({ + schema: MANIFEST_SCHEMA, source: { commit: 'a'.repeat(40), tree: 'b'.repeat(40), archiveSha256: sha('tar') }, + patch: patch ? { applied: true, sha256: sha('patch') } : { applied: false }, + lockfileSha256: sha('{}'), dist: content.dist, native: content.native, + })) + return dir +} + +test('verifyBuild accepts an unchanged tree and lists only *.node natives', () => { + const dir = fakeBuild({ patch: true }) + try { + const r = verifyBuild(dir) + assert.deepEqual(r.faults, []) + assert.equal(r.distFiles, 2) + assert.equal(r.nativeFiles, 1) + } finally { fs.rmSync(dir, { recursive: true }) } +}) + +test('verifyBuild refuses changed, extra, missing and unrecorded content', () => { + const dir = fakeBuild() + try { + fs.writeFileSync(path.join(dir, 'src', 'dist', 'b.js'), 'export const x = 1\n') + fs.writeFileSync(path.join(dir, 'src', 'dist', 'c.js'), '') + fs.rmSync(path.join(dir, 'src', 'node_modules', 'addon', 'build', 'x.node')) + fs.writeFileSync(path.join(dir, 'source.patch'), 'unrecorded') + fs.writeFileSync(path.join(dir, 'src', 'package-lock.json'), '{"changed":1}') + fs.symlinkSync('a.js', path.join(dir, 'src', 'dist', 'link.js')) + const r = verifyBuild(dir) + for (const fault of ['dist_changed:b.js', 'dist_extra:c.js', 'native_missing:addon/build/x.node', 'patch_record_invalid', + 'lockfile_changed', 'unsupported_entry:link.js']) assert.ok(r.faults.includes(fault), `${fault} in ${r.faults}`) + assert.equal(r.ok, false) + } finally { fs.rmSync(dir, { recursive: true }) } +}) + +test('verifyBuild refuses a malformed source identity', () => { + const dir = fakeBuild() + try { + const file = path.join(dir, 'manifest.json') + const m = JSON.parse(fs.readFileSync(file, 'utf8')) + m.source.commit = 'b483cd5' + fs.writeFileSync(file, JSON.stringify(m)) + assert.ok(verifyBuild(dir).faults.includes('source_identity_malformed')) + } finally { fs.rmSync(dir, { recursive: true }) } +}) + +test('import graph follows static and literal dynamic relative imports', () => { + const dir = freshPrivateDir('126-graph-test') + try { + fs.writeFileSync(path.join(dir, 'entry.js'), "import { a } from './a.js'\nconst m = await import('./sub/b.js')\nimport 'node:fs'\n") + fs.writeFileSync(path.join(dir, 'a.js'), 'export const a = 1\n') + fs.mkdirSync(path.join(dir, 'sub')) + fs.writeFileSync(path.join(dir, 'sub', 'b.js'), "export * from '../unlock.js'\n") + fs.writeFileSync(path.join(dir, 'unlock.js'), 'export {}\n') + assert.deepEqual(reachableModules(dir, ['entry.js']), ['a.js', 'entry.js', 'sub/b.js', 'unlock.js']) + assert.deepEqual(reachableModules(dir, ['a.js']), ['a.js']) + } finally { fs.rmSync(dir, { recursive: true }) } +}) + +test('import graph follows commented static imports, dynamic imports and re-exports', () => { + const dir = freshPrivateDir('126-graph-comment-test') + try { + fs.writeFileSync(path.join(dir, 'unlock.js'), 'export const x = 1\n') + for (const source of [ + "import /* comment */ './unlock.js'", + "import /* comment */ { x } from /* comment */ './unlock.js'", + "const loaded = import(/* comment */ './unlock.js')", + "export /* comment */ { x } from /* comment */ './unlock.js'", + "export /* comment */ * from /* comment */ './unlock.js'", + ]) { + fs.writeFileSync(path.join(dir, 'entry.js'), source) + assert.deepEqual(reachableModules(dir, ['entry.js']), ['entry.js', 'unlock.js'], source) + } + } finally { fs.rmSync(dir, { recursive: true }) } +}) + +test('import graph parses multiline expressions, escaped specifiers and template literals', () => { + const dir = freshPrivateDir('126-graph-literal-test') + try { + fs.writeFileSync(path.join(dir, 'entry.js'), [ + String.raw`import /* before specifier */ './quo\'ted.js'`, + String.raw`const loaded = import( + /* a multiline gap */ + './un\u006cock.js' + )`, + 'const template = import(`./template.js`)', + ].join('\n')) + fs.writeFileSync(path.join(dir, "quo'ted.js"), 'export {}\n') + fs.writeFileSync(path.join(dir, 'unlock.js'), 'export {}\n') + fs.writeFileSync(path.join(dir, 'template.js'), 'export {}\n') + assert.deepEqual(reachableModules(dir, ['entry.js']), ['entry.js', "quo'ted.js", 'template.js', 'unlock.js']) + } finally { fs.rmSync(dir, { recursive: true }) } +}) + +test('import graph ignores import-shaped comments, strings, regular expressions and property calls', () => { + const dir = freshPrivateDir('126-graph-nonedge-test') + try { + fs.writeFileSync(path.join(dir, 'entry.js'), [ + "// import './missing-comment.js'", + "/* export * from './missing-block.js' */", + 'const text = "import(\'./missing-string.js\')"', + "const template = `import './missing-template.js'`", + 'const regex = /import(\'missing-regex.js\')/', + "const obj = { import() {} }; obj.import('./missing-property.js')", + "import 'node:fs'", + ].join('\n')) + assert.deepEqual(reachableModules(dir, ['entry.js']), ['entry.js']) + } finally { fs.rmSync(dir, { recursive: true }) } +}) + +test('import graph rejects malformed JavaScript instead of accepting a recovered partial graph', () => { + const dir = freshPrivateDir('126-graph-invalid-test') + try { + fs.writeFileSync(path.join(dir, 'entry.js'), "const value = import(/* unfinished */ './unlock.js'") + assert.throws(() => reachableModules(dir, ['entry.js']), /module_parse_failed:entry\.js/) + } finally { fs.rmSync(dir, { recursive: true }) } +}) + +test('import graph canonicalizes dist aliases, escaped URLs and in-root symlinks to unlock.js', () => { + const dir = freshPrivateDir('126-graph-canonical-test') + try { + const dist = path.join(dir, 'dist') + fs.mkdirSync(dist) + fs.writeFileSync(path.join(dist, 'unlock.js'), 'export {}\n') + fs.symlinkSync('unlock.js', path.join(dist, 'alias.js')) + fs.symlinkSync('dist', path.join(dir, 'dist-alias')) + for (const source of [ + "import /* alias */ '../dist/unlock.js'", + "const loaded = import(/* alias */ '../dist/unlock.js')", + "export * from './%75nlock.js?alias#fragment'", + "import './alias.js'", + ]) { + fs.writeFileSync(path.join(dist, 'entry.js'), source) + assert.deepEqual(reachableModules(path.join(dir, 'dist-alias'), ['entry.js']), ['entry.js', 'unlock.js']) + } + assert.deepEqual(reachableModules(dist, ['../dist/unlock.js']), ['unlock.js']) + } finally { fs.rmSync(dir, { recursive: true }) } +}) + +test('import graph refuses literal escapes, symlink escapes, absolute paths and URL imports', () => { + const dir = freshPrivateDir('126-graph-escape-test') + try { + const dist = path.join(dir, 'dist') + fs.mkdirSync(dist) + const outside = path.join(dir, 'outside.js') + fs.writeFileSync(outside, 'export {}\n') + const unlock = path.join(dist, 'unlock.js') + fs.writeFileSync(unlock, 'export {}\n') + fs.symlinkSync('../outside.js', path.join(dist, 'escape.js')) + for (const specifier of ['../outside.js', './%2e%2e/outside.js', './escape.js', + outside, pathToFileURL(outside).href, unlock, pathToFileURL(unlock).href, 'data:text/javascript,export{}']) { + fs.writeFileSync(path.join(dist, 'entry.js'), `import /* escape */ ${JSON.stringify(specifier)}`) + assert.throws(() => reachableModules(dist, ['entry.js']), /module_outside_dist|module_unsupported_absolute_or_url_import/, specifier) + } + assert.throws(() => reachableModules(dist, ['../outside.js']), /module_outside_dist/) + } finally { fs.rmSync(dir, { recursive: true }) } +}) diff --git a/experimental/126-guard-liveness/f2-linux-nondumpable/codex b/experimental/126-guard-liveness/f2-linux-nondumpable/codex new file mode 100755 index 0000000..60b4a2e --- /dev/null +++ b/experimental/126-guard-liveness/f2-linux-nondumpable/codex @@ -0,0 +1,83 @@ +#!/usr/bin/env python3 +"""Synthetic Linux holder named `codex` (#126 research, F2). + +Acquires the thread flock under `.coordination.lock` like Codex v0.162.1 and +holds it until stdin EOF (exit 0) or its lifetime (exit 3): 20 s, or +FIXTURE_LIFETIME_SECONDS (1-120). It explicitly sets and verifies +both requested dumpable values with prctl, modelling a process +that hardened itself. A targeted search of pinned v0.162.1 found no direct +hardening call in the ordinary CLI main; this does not establish a deployed +writer's dumpability. The fixture models a non-dumpable candidate. + +Prints one JSON line {"ready":true,"pid":..,"fd":..,"dumpable":..}. +""" +import ctypes +import fcntl +import json +import os +import select +import sys +import time + +PR_SET_DUMPABLE = 4 +PR_GET_DUMPABLE = 3 + +if sys.platform != "linux" or os.getuid() == 0 or os.getuid() != os.geteuid(): + raise RuntimeError("fixture requires an unprivileged same-user Linux process") +lock_path = os.environ["FIXTURE_LOCK_PATH"] +want = int(os.environ["FIXTURE_DUMPABLE"]) +if want not in (0, 1): + raise ValueError("FIXTURE_DUMPABLE must be 0 or 1") +lifetime = int(os.environ.get("FIXTURE_LIFETIME_SECONDS", "20")) +if not 1 <= lifetime <= 120: + raise ValueError("FIXTURE_LIFETIME_SECONDS must be 1-120") +deadline = time.monotonic() + lifetime +libc = ctypes.CDLL(None, use_errno=True) +libc.prctl.argtypes = [ctypes.c_int, ctypes.c_ulong, ctypes.c_ulong, ctypes.c_ulong, ctypes.c_ulong] +libc.prctl.restype = ctypes.c_int +if libc.prctl(PR_SET_DUMPABLE, want, 0, 0, 0) != 0: + raise OSError(ctypes.get_errno(), "prctl(PR_SET_DUMPABLE)") +dumpable = libc.prctl(PR_GET_DUMPABLE, 0, 0, 0, 0) +if dumpable == -1: + raise OSError(ctypes.get_errno(), "prctl(PR_GET_DUMPABLE)") +if dumpable != want: + raise RuntimeError(f"dumpable mismatch: requested {want}, observed {dumpable}") + +def acquire(fd): + until = min(deadline, time.monotonic() + 2) + while True: + try: + fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB) + return + except BlockingIOError: + if time.monotonic() >= until: + raise TimeoutError("fixture lock acquisition timed out") + time.sleep(0.01) + + +guard = thread = None +eof = False +try: + guard = os.open(os.path.join(os.path.dirname(lock_path), ".coordination.lock"), + os.O_RDONLY | os.O_CLOEXEC | os.O_NOFOLLOW) + acquire(guard) + thread = os.open(lock_path, os.O_RDWR | os.O_CLOEXEC | os.O_NOFOLLOW) + acquire(thread) + fcntl.flock(guard, fcntl.LOCK_UN) + os.close(guard) + guard = None + print(json.dumps({"ready": True, "pid": os.getpid(), "fd": thread, + "dumpable": dumpable, "uid": os.getuid(), "euid": os.geteuid(), + "lifetime_seconds": lifetime}), flush=True) + while time.monotonic() < deadline: + readable, _, _ = select.select([sys.stdin.fileno()], [], [], max(0, deadline - time.monotonic())) + if readable and not os.read(sys.stdin.fileno(), 4096): + eof = True + break +finally: + if guard is not None: + os.close(guard) + if thread is not None: + os.close(thread) +# Exit 0 on stdin EOF, 3 when the lifetime ended first. +sys.exit(0 if eof else 3) diff --git a/experimental/126-guard-liveness/f2-linux-nondumpable/expectations.mjs b/experimental/126-guard-liveness/f2-linux-nondumpable/expectations.mjs new file mode 100644 index 0000000..f8147b8 --- /dev/null +++ b/experimental/126-guard-liveness/f2-linux-nondumpable/expectations.mjs @@ -0,0 +1,107 @@ +// Expected-table evaluation for the isolated F2 inspector (#126 research). +// +// Three separate results per run: +// - `collected`: the child completed the F2 protocol (protocol.mjs) without +// faults and the holder stayed the same live instance throughout. +// - `rows`: whether observed product stages match the table below. +// - `recorded`: values kept for review and never asserted, such as the +// fixture's isCodex result and final eligibility. +// None of this is a fail-closed proof, a native Codex result or a signal +// decision. A dumpable=0 refusal counts only when the dumpable=0 run's own +// rows pass and the same report's dumpable=1 control passed. +import { manifestBindingFaults } from './build-binding.mjs' + +const isTicks = (v) => typeof v === 'string' && /^\d+$/.test(v) +export const sameTicks = (a, b) => isTicks(a) && isTicks(b) && BigInt(a) === BigInt(b) + +// Errors that a same-user observer gets from procfs access checks. Anything +// else (ENOENT, a TypeError from a wrong build, the product's budget error) +// is not evidence of a permission refusal. +export const PERMISSION_CODES = new Set(['EACCES', 'EPERM']) +// The #137 `b483cd5` policy adds this blocker when owner start ticks or a +// matching participation are absent (src/policy.ts, evaluateSafety). It is +// the only refusal reason this table attributes to missing participation. +export const PARTICIPATION_BLOCKER = 'linux_participation_unproven' + +function firstObservedError(byStage) { + for (const [name, s] of Object.entries(byStage)) { + if (s.error) return { stage: name, code: s.error.code, message: s.error.message } + const r = s.result + if (name === 'openers' && (r?.error || r?.processes?.some((p) => p.errors.length))) { + return { stage: name, opener: r.error, errors: r.processes.flatMap((p) => p.errors) } + } + } + return null +} + +export function evaluateRun({ dumpable, holder, ticks, observation, protocol, expectedManifestSha256 }) { + const rows = [] + const row = (id, ok, detail) => { rows.push({ id, ok: Boolean(ok), ...(ok ? {} : { detail }) }) } + const collectedFaults = [...(observation.incomplete ? [`incomplete:${observation.incompletePhase}`] : []), + ...observation.faults, ...protocol.faults] + const bindingFaults = manifestBindingFaults(protocol.byStage.dist_identity?.result?.manifestSha256, expectedManifestSha256) + collectedFaults.push(...bindingFaults) + // The holder must be one live instance from ready to after inspection. + if (!isTicks(ticks.ready)) collectedFaults.push('holder_ticks_unavailable') + if (!sameTicks(ticks.ready, ticks.afterInspect)) collectedFaults.push('holder_instance_changed_or_gone') + if (ticks.holderExitedDuringInspection) collectedFaults.push('holder_exited_during_inspection') + const collected = collectedFaults.length === 0 + // Stage shapes are trusted only after protocol validation. A malformed + // result must remain a structured collection failure, not throw while + // reading payload fields (or accidentally satisfy an expectation). + if (protocol.faults.length > 0 || bindingFaults.length > 0) return { + dumpable, collected: false, collectedFaults, rows, ok: false, + recorded: { firstObservedError: null, guardedAttempted: null, participation: null, + fixtureIsCodex: null, eligibility: null, owner: null, probe: null }, + } + const s = protocol.byStage + const openers = s.openers?.result + const prepare = s.prepare + const verify = s.guarded_verify?.result + const inspect = s.inspect_thread?.result + const fd = String(holder.fd) + const only = openers?.processes?.length === 1 ? openers.processes[0] : null + row('V_owner_visible', openers && openers.error === null && only?.pid === holder.pid, openers) + if (dumpable === 1) { + row('E1_owner_identity', only && only.identityComplete === true && only.errors.length === 0 && + sameTicks(only.startTicks, ticks.ready), only) + row('E2_prepare_candidate', prepare?.result && prepare.result.pid === holder.pid && sameTicks(prepare.result.startTicks, ticks.ready) && + prepare.result.descriptors.includes(fd) && prepare.result.candidates.includes(fd), prepare) + row('E3_guarded_participation', verify?.status === 'held' && verify.guard?.status === 'acquired' && + verify.participation?.pid === holder.pid && verify.participation.descriptor === fd && + sameTicks(verify.participation.startTicks, ticks.ready), verify) + row('E4_inspection_participation', inspect && inspect.probe.status === 'held' && inspect.probe.guard?.status === 'acquired' && + inspect.probe.participation?.pid === holder.pid && inspect.probe.participation.descriptor === fd && + sameTicks(inspect.probe.participation.startTicks, ticks.ready) && inspect.owner?.pid === holder.pid && + sameTicks(inspect.owner.startTicks, ticks.ready) && !inspect.blockers.includes(PARTICIPATION_BLOCKER), inspect) + } else { + row('R1_prepare_permission_denied', prepare?.error && PERMISSION_CODES.has(prepare.error.code), prepare) + row('R2_inspection_refused_for_participation', inspect && inspect.safeToUnlock === false && + inspect.blockers.includes(PARTICIPATION_BLOCKER) && inspect.probe.participation === undefined, inspect) + } + const rowsOk = rows.every((r) => r.ok) + return { + dumpable, collected, collectedFaults, rows, ok: collected && rowsOk, + recorded: { + firstObservedError: firstObservedError(s), + guardedAttempted: verify ? verify.skipped !== 'prepare_failed' : null, + participation: inspect?.probe?.participation ?? null, + // A Python script path ending in `codex`; not a native Codex result. + fixtureIsCodex: only?.isCodex ?? null, + eligibility: inspect ? { safeToUnlock: inspect.safeToUnlock, blockers: inspect.blockers } : null, + owner: inspect?.owner ?? null, + probe: inspect?.probe ?? null, + }, + } +} + +export function evaluateReport(runs) { + const control = runs.find((r) => r.dumpable === 1) + const refusal = runs.find((r) => r.dumpable === 0) + const failures = [] + if (!control?.ok) failures.push('control_dumpable1_not_passed') + if (!refusal?.ok) failures.push('refusal_dumpable0_not_passed') + // A refusal without a passing control in the same report says nothing. + return { ok: failures.length === 0 && runs.length === 2, failures, + refusalCounted: Boolean(control?.ok && refusal?.ok) } +} diff --git a/experimental/126-guard-liveness/f2-linux-nondumpable/expectations.test.mjs b/experimental/126-guard-liveness/f2-linux-nondumpable/expectations.test.mjs new file mode 100644 index 0000000..f044951 --- /dev/null +++ b/experimental/126-guard-liveness/f2-linux-nondumpable/expectations.test.mjs @@ -0,0 +1,205 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' +import { evaluateReport, evaluateRun, sameTicks } from './expectations.mjs' +import { parseChildFault, validateF2 } from './protocol.mjs' + +const holder = { pid: 42, fd: 5 } +const ticks = { ready: '900', afterInspect: '900', holderExitedDuringInspection: false } +const startupManifest = 'a'.repeat(64) +const probe = (status, extra = {}) => ({ status, method: 'flock_exclusive_nonblocking', ...extra }) +const clone = (v) => JSON.parse(JSON.stringify(v)) +const stage = (name, outcome) => ({ stage: name, ...outcome }) +const base = () => [ + stage('dist_identity', { result: { ok: true, faults: [], manifestSha256: 'a'.repeat(64) } }), + stage('module_load', { result: { modules: ['inspection.js'], unlockReachable: false } }), +] +const positive = () => [...base(), + stage('openers', { result: { error: null, processes: [{ pid: 42, startTicks: '900', errors: [], isCodex: true, identityComplete: true }] } }), + stage('prepare', { result: { pid: 42, startTicks: '900', candidates: ['5'], descriptors: ['0', '5'] } }), + stage('guarded_verify', { result: probe('held', { guard: { status: 'acquired' }, participation: { pid: 42, startTicks: '900', descriptor: '5' } }) }), + stage('inspect_thread', { result: { classification: 'live_owner', safeToUnlock: true, blockers: [], warnings: [], + owner: { pid: 42, startTicks: '900' }, probe: probe('held', { guard: { status: 'acquired' }, participation: { pid: 42, startTicks: '900', descriptor: '5' } }) } }), +] +const refused = () => [...base(), + stage('openers', { result: { error: null, processes: [{ pid: 42, startTicks: null, errors: ['linux_process_instance_unverified'], isCodex: true, identityComplete: true }] } }), + stage('prepare', { error: { code: 'EACCES', name: 'Error', message: 'denied' } }), + stage('guarded_verify', { result: { skipped: 'prepare_failed' } }), + stage('inspect_thread', { result: { classification: 'live_owner', safeToUnlock: false, blockers: ['linux_participation_unproven'], + warnings: [], owner: { pid: 42, startTicks: null }, probe: probe('unknown') } }), +] +const observe = (stages, extra = {}) => ({ incomplete: false, faults: [], stages, begun: stages.map((s) => s.stage), ...extra }) +const run = (dumpable, stages, extra = {}, t = ticks) => { + const observation = observe(stages, extra) + return evaluateRun({ dumpable, holder, ticks: t, observation, protocol: validateF2(observation), expectedManifestSha256: startupManifest }) +} +const failed = (r) => r.rows.filter((x) => !x.ok).map((x) => x.id) + +test('ticks compare as decimal strings only', () => { + assert.equal(sameTicks('0900', '900'), true) + assert.equal(sameTicks('900', 900), false) + assert.equal(sameTicks('9e2', '900'), false) + assert.equal(sameTicks(null, null), false) + assert.equal(sameTicks('', ''), false) + assert.equal(sameTicks('unknown', 'unknown'), false) +}) + +test('F2 protocol requires exact order, uniqueness and result shapes', () => { + assert.equal(validateF2(observe(positive())).ok, true) + assert.deepEqual(validateF2(observe([])).faults, ['stages_missing:dist_identity,module_load,openers,prepare,guarded_verify,inspect_thread']) + const swapped = positive(); [swapped[2], swapped[3]] = [swapped[3], swapped[2]] + assert.ok(validateF2(observe(swapped)).faults.some((f) => f.startsWith('stage_order:'))) + const dup = positive(); dup.push(clone(dup[5])) + assert.ok(validateF2(observe(dup)).faults.includes('stage_duplicate:inspect_thread')) + const badIdentity = positive(); badIdentity[0].result.ok = false + assert.ok(validateF2(observe(badIdentity)).faults.includes('stage_result_malformed:dist_identity')) + const threw = positive(); threw[5] = stage('inspect_thread', { error: { code: null, message: 'stale_residue' } }) + assert.ok(validateF2(observe(threw)).faults.includes('stage_failed:inspect_thread')) + const notArray = positive(); notArray[5].result.blockers = 'linux_participation_unproven' + assert.ok(validateF2(observe(notArray)).faults.includes('stage_result_malformed:inspect_thread')) + const badTicks = positive(); badTicks[3].result.startTicks = 900 + assert.ok(validateF2(observe(badTicks)).faults.includes('stage_result_malformed:prepare')) +}) + +test('child fault values are an explicit allowlist', () => { + assert.equal(parseChildFault(undefined), null) + assert.equal(parseChildFault('stall:prepare'), 'stall:prepare') + assert.throws(() => parseChildFault('stall:prepar'), /F2_CHILD_FAULT/) + assert.throws(() => parseChildFault('stall'), /F2_CHILD_FAULT/) +}) + +test('dumpable=1 control requires identity, held guard, FD and ticks', () => { + const r = run(1, positive()) + assert.equal(r.ok, true) + assert.equal(r.recorded.fixtureIsCodex, true) + assert.deepEqual(r.recorded.eligibility, { safeToUnlock: true, blockers: [] }) + const wrongFd = positive(); wrongFd[4].result.participation.descriptor = '6' + assert.deepEqual(failed(run(1, wrongFd)), ['E3_guarded_participation']) + const forged = positive(); forged[3].result.startTicks = '901' + assert.deepEqual(failed(run(1, forged)), ['E2_prepare_candidate']) + const incompleteId = positive(); incompleteId[2].result.processes[0].identityComplete = false + assert.deepEqual(failed(run(1, incompleteId)), ['E1_owner_identity']) + const freeGuard = positive(); freeGuard[4].result.guard.status = 'busy' + assert.deepEqual(failed(run(1, freeGuard)), ['E3_guarded_participation']) + // A refusal is not a pass for the positive control. + assert.equal(run(1, refused()).ok, false) +}) + +test('holder instance must stay the same live process', () => { + assert.ok(run(1, positive(), {}, { ...ticks, afterInspect: '901' }).collectedFaults.includes('holder_instance_changed_or_gone')) + assert.ok(run(1, positive(), {}, { ...ticks, afterInspect: null }).collectedFaults.includes('holder_instance_changed_or_gone')) + const exited = run(0, refused(), {}, { ...ticks, holderExitedDuringInspection: true }) + assert.equal(exited.collected, false) + assert.equal(exited.ok, false) +}) + +test('inspection needs its own acquired guard even when direct guarded verification passed', () => { + for (const guard of [undefined, { status: 'busy' }, { status: 'unsafe' }]) { + const stages = positive() + if (guard === undefined) delete stages[5].result.probe.guard + else stages[5].result.probe.guard = guard + const result = run(1, stages) + assert.equal(result.collected, true) + assert.equal(result.rows.find((r) => r.id === 'E3_guarded_participation').ok, true) + assert.deepEqual(failed(result), ['E4_inspection_participation']) + assert.equal(result.ok, false) + const report = evaluateReport([result, run(0, refused())]) + assert.equal(report.ok, false) + assert.equal(report.refusalCounted, false) + } +}) + +test('dumpable=0 counts only a permission refusal with the participation blocker', () => { + assert.equal(run(0, refused()).ok, true) + for (const error of [{ code: 'ENOENT', message: 'gone' }, { code: null, name: 'TypeError', message: 'wrong build' }, + { code: null, name: 'Error', message: 'linux_attribution_budget' }]) { + const s = refused(); s[3] = stage('prepare', { error }) + assert.deepEqual(failed(run(0, s)), ['R1_prepare_permission_denied'], error.message) + } + const unrelated = refused(); unrelated[5].result.blockers = ['transcript_missing'] + assert.deepEqual(failed(run(0, unrelated)), ['R2_inspection_refused_for_participation']) + const hidden = refused(); hidden[2].result.processes = [] + assert.deepEqual(failed(run(0, hidden)), ['V_owner_visible']) + assert.equal(run(0, positive()).ok, false) + assert.equal(run(0, refused()).recorded.guardedAttempted, false) +}) + +test('R2 permits additional blockers and records them without claiming sole causation', () => { + const stages = refused() + const inspection = stages[5].result + inspection.blockers.push('lock_owner_identity_incomplete') + inspection.owner.errors = ['linux_process_instance_unverified'] + inspection.probe.error = 'permission evidence unavailable' + const result = run(0, stages) + assert.equal(result.ok, true) + assert.deepEqual(result.recorded.eligibility.blockers, inspection.blockers) + assert.deepEqual(result.recorded.owner.errors, inspection.owner.errors) + assert.equal(result.recorded.probe.error, inspection.probe.error) +}) + +test('incomplete or faulted collection never satisfies the table', () => { + assert.ok(run(0, refused(), { incomplete: true, incompletePhase: 'stages' }).collectedFaults.includes('incomplete:stages')) + assert.equal(run(1, positive(), { faults: ['malformed_line:3'] }).ok, false) + assert.equal(run(1, positive().slice(0, 3), { incomplete: true, incompletePhase: 'stages' }).ok, false) +}) + +test('malformed stage payloads return collection failures instead of throwing', () => { + for (const dumpable of [1, 0]) { + for (let at = 0; at < positive().length; at++) { + for (const payload of [null, {}, { processes: 'malformed' }, { probe: null, blockers: null }]) { + const stages = positive() + stages[at].result = payload + const r = run(dumpable, stages) + assert.equal(r.collected, false, `dumpable=${dumpable} stage=${at}`) + assert.equal(r.ok, false) + assert.deepEqual(r.rows, []) + assert.ok(r.collectedFaults.some((f) => f.startsWith('stage_result_malformed:'))) + } + } + } +}) + +test('a refusal counts only with a passing control in the same report', () => { + assert.deepEqual(evaluateReport([run(1, positive()), run(0, refused())]), { ok: true, failures: [], refusalCounted: true }) + const r = evaluateReport([run(1, refused()), run(0, refused())]) + assert.equal(r.refusalCounted, false) + assert.deepEqual(r.failures, ['control_dumpable1_not_passed']) + assert.equal(evaluateReport([run(0, refused())]).ok, false) +}) + +test('refusal cannot pass with an absent, malformed or invalid nested lock probe', () => { + const malformed = [undefined, null, {}, { status: 'unknown' }, probe('invalid'), + { ...probe('unknown'), method: 'unverified' }, probe('unknown', { guard: { status: 'invalid' } }), + probe('unknown', { guard: { status: 'busy', attempts: 0 } }), probe('unknown', { participation: null }), + probe('unknown', { participation: {} }), probe('held', { participation: { pid: '42', startTicks: '900', descriptor: '5' } }), + probe('held', { participation: { pid: 42, startTicks: 900, descriptor: '5' } }), + probe('held', { participation: { pid: 42, startTicks: '900', descriptor: '' } })] + for (const value of malformed) { + const stages = refused() + stages[5].result.probe = value + const result = run(0, stages) + assert.equal(result.collected, false) + assert.equal(result.ok, false) + assert.deepEqual(result.rows, []) + assert.ok(result.collectedFaults.includes('stage_result_malformed:inspect_thread')) + assert.equal(evaluateReport([run(1, positive()), result]).refusalCounted, false) + } +}) + +test('each dumpability run must match the same startup build manifest', () => { + for (const dumpable of [1, 0]) { + const stages = dumpable ? positive() : refused() + stages[0].result.manifestSha256 = 'b'.repeat(64) + // A replacement build can be internally coherent and pass the stage + // protocol, but cannot be attributed to the parent's startup identity. + assert.equal(validateF2(observe(stages)).ok, true) + const result = run(dumpable, stages) + assert.equal(result.collected, false) + assert.equal(result.ok, false) + assert.deepEqual(result.rows, []) + assert.ok(result.collectedFaults.includes('startup_manifest_sha256_changed')) + } + const observation = observe(positive()) + const result = evaluateRun({ dumpable: 1, holder, ticks, observation, protocol: validateF2(observation) }) + assert.equal(result.ok, false) + assert.ok(result.collectedFaults.includes('startup_manifest_sha256_invalid')) +}) diff --git a/experimental/126-guard-liveness/f2-linux-nondumpable/inspect-child.mjs b/experimental/126-guard-liveness/f2-linux-nondumpable/inspect-child.mjs new file mode 100644 index 0000000..35f33ad --- /dev/null +++ b/experimental/126-guard-liveness/f2-linux-nondumpable/inspect-child.mjs @@ -0,0 +1,119 @@ +#!/usr/bin/env node +// F2 product-inspection child (#126 research). Run only by run.mjs or +// plumbing-smoke.mjs through lib/isolated-observer.mjs, never in the runner's +// own process. +// +// Usage: node inspect-child.mjs BUILD PID FD LOCK_PATH CODEX_HOME THREAD_ID EXPECTED_MANIFEST_SHA256 +// +// BUILD is a directory made by build.mjs. The closed runtime/verification tree +// is copied to private scratch and that copy is re-hashed against the parent's +// startup manifest before any product module is imported. Only the read-only +// copy is walked/imported; later source-build changes cannot redirect it. +// Same-UID tampering with snapshot permissions is not prevented. The canonical +// static/literal import graph excludes unlock.js, not arbitrary dynamic code. +// +// Signals: this child sends no signal to the holder, to any lock owner, or +// through any unlock path. Product diagnostics it calls (`lsof`, `ps`) run +// with the product's own timeout, which sends SIGTERM and then SIGKILL to the +// timed-out diagnostic's own process group. No Codex native lock file is +// created or removed. A stage error is recorded as observed; nothing converts an +// error into free or allowed. +import path from 'node:path' +import process from 'node:process' +import { pathToFileURL } from 'node:url' +import { StopStages, childStages } from '../lib/isolated-observer.mjs' +import { reachableModules } from './build.mjs' +import { parseChildFault } from './protocol.mjs' +import { manifestBindingFaults } from './build-binding.mjs' +import { createBuildSnapshot } from './build-snapshot.mjs' + +const [build, pidText, fdText, lockPath, codexHome, threadId, expectedManifestSha256] = process.argv.slice(2) +const pid = Number(pidText) +const fd = Number(fdText) +if (!build || !Number.isSafeInteger(pid) || pid <= 0 || !Number.isSafeInteger(fd) || fd < 0 || !lockPath || !codexHome || !threadId || + manifestBindingFaults(expectedManifestSha256, expectedManifestSha256).length > 0) { + throw new Error('usage: inspect-child.mjs BUILD PID FD LOCK_PATH CODEX_HOME THREAD_ID EXPECTED_MANIFEST_SHA256') +} +const fault = parseChildFault(process.env.F2_CHILD_FAULT) +const ENTRIES = ['linux-attribution.js', 'native-coordination.js', 'lock.js', 'process.js', 'inspection.js'] +const plain = (value) => JSON.parse(JSON.stringify(value, (_key, v) => (typeof v === 'bigint' ? `${v}n` : v))) +const stall = (name) => { + // Injected finite synchronous stall, for validating the parent bound only. + if (fault === `stall:${name}`) Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 15000) +} + +try { + const c = childStages({ lifetimeMs: 30000 }) + let snapshot + const identity = await c.stage('dist_identity', () => { + stall('dist_identity') + snapshot = createBuildSnapshot(build, expectedManifestSha256) + const { verified } = snapshot + return plain({ ok: verified.ok, faults: verified.faults, manifestSha256: verified.manifestSha256, + source: verified.source, patch: verified.patch, distFiles: verified.distFiles, nativeFiles: verified.nativeFiles, + verifiedBy: 'inspect-child re-hash of independent private copy against startup manifest before import', + snapshot: { directory: snapshot.directory, copiedContentSha256: snapshot.copiedContentSha256, + fileCount: snapshot.fileCount, omittedBinDirectories: snapshot.omittedBinDirectories, + limits: 'read-only copy isolates input mutation; same-UID chmod/tampering is not prevented; dependency JS is copied, not startup-manifest authenticated' } }) + }) + // Never import a build whose content or manifest differs from the parent's + // startup snapshot, even if its replacement manifest is internally valid. + if (identity.result?.ok !== true) throw new StopStages('build_unverified') + let product + const loaded = await c.stage('module_load', async () => { + stall('module_load') + const modules = reachableModules(snapshot.dist, ENTRIES) + if (modules.includes('unlock.js')) throw new Error('unlock_reachable') + const load = (name) => import(pathToFileURL(path.join(snapshot.dist, name)).href) + const [attribution, coordination, lock, proc, inspection] = await Promise.all(ENTRIES.map(load)) + product = { ...attribution, ...coordination, ...lock, ...proc, ...inspection } + return { modules, unlockReachable: false } + }) + if (loaded.error) throw new StopStages('module_load_failed') + + // Product discovery and identity as the product records it. isCodex is + // recorded, not asserted: this fixture is `python3 …/codex`, not a native + // Codex binary. + await c.stage('openers', async () => { + stall('openers') + const openers = await product.inspectLockOpeners(lockPath) + return plain({ error: openers.error ?? null, processes: openers.processes.map((p) => ({ + pid: p.pid, isCodex: p.isCodex, identityComplete: p.identityComplete, errors: p.errors, + startTicks: p.startTicks ?? null, commandFingerprint: p.commandFingerprint ?? null, command: p.command, arguments: p.arguments, + })) }) + }) + + let prepared + await c.stage('prepare', () => { + stall('prepare') + prepared = product.prepareLinuxParticipation(pid, lockPath) + return plain(prepared) + }) + + await c.stage('guarded_verify', () => { + // Harness bookkeeping: nothing to verify without a preparation. This is + // not a product refusal. + if (!prepared) return { skipped: 'prepare_failed' } + stall('guarded_verify') + const probe = product.guardedProbeOnce(lockPath, product.observeLockFile(lockPath), undefined, + () => product.verifyLinuxParticipation(lockPath, prepared)) + return plain(probe) + }) + + await c.stage('inspect_thread', async () => { + stall('inspect_thread') + const r = await product.inspectThread(threadId, { codexHome, stabilityMs: 200, terminationTimeoutMs: 5000 }) + return plain({ + classification: r.classification, safeToUnlock: r.safeToUnlock, blockers: r.blockers, warnings: r.warnings, + probe: r.lock.probe, owner: r.owner && { pid: r.owner.pid, isCodex: r.owner.isCodex, identityComplete: r.owner.identityComplete, + errors: r.owner.errors, startTicks: r.owner.startTicks ?? null }, + openers: r.openers.length, transcript: r.transcript.status, + }) + }) + await c.finish() +} catch (error) { + if (!(error instanceof StopStages)) throw error + // Boundary stops set 3/4 themselves; a refused build or import is 5. + if (!process.exitCode) process.exitCode = 5 + process.stdin.destroy() +} diff --git a/experimental/126-guard-liveness/f2-linux-nondumpable/observe.py b/experimental/126-guard-liveness/f2-linux-nondumpable/observe.py new file mode 100755 index 0000000..7cd3fa4 --- /dev/null +++ b/experimental/126-guard-liveness/f2-linux-nondumpable/observe.py @@ -0,0 +1,55 @@ +#!/usr/bin/env python3 +"""Same-user /proc access matrix for one target (#126 research, F2). + +Usage: python3 -I observe.py PID FD LOCK_PATH +Records, per read, either a short result or the errno name. Nothing here +treats an error as negative evidence; it only reports which reads fail. +Reads are the ones #137's Linux adapter performs. +""" +import errno +import json +import os +import sys + +if sys.platform != "linux" or os.getuid() == 0 or os.getuid() != os.geteuid(): + raise RuntimeError("observer requires unprivileged same-user Linux") +pid, fd, lock_path = int(sys.argv[1]), int(sys.argv[2]), sys.argv[3] +if pid <= 0 or fd < 0: + raise ValueError("invalid PID or FD") +base = f"/proc/{pid}" +out = {"pid": pid, "fd": fd, "observer_uid": os.getuid(), "kernel": os.uname().release} + + +# No self-alarm: a non-fatal signal with a Python handler cannot wake a +# killable kernel wait in a /proc read, so it would not bound this process. +# The runner waits for this observer with a bound and records `incomplete` +# without signalling it. + + +def attempt(name, fn): + try: + out[name] = {"ok": fn()} + except OSError as error: + out[name] = {"errno": errno.errorcode.get(error.errno, str(error.errno))} + + +def read(path, limit=65536): + with open(path, "rb") as handle: + return handle.read(limit) + + +attempt("proc_dir_uid", lambda: os.stat(base).st_uid) +attempt("fd_dir_uid", lambda: os.stat(f"{base}/fd").st_uid) +attempt("fdinfo_dir_uid", lambda: os.stat(f"{base}/fdinfo").st_uid) +# prepareLinuxParticipation compares mount namespaces before listing FDs. +attempt("self_ns_mnt_readlink", lambda: os.readlink("/proc/self/ns/mnt")) +attempt("ns_mnt_readlink", lambda: os.readlink(f"{base}/ns/mnt")) +attempt("fd_listdir", lambda: sorted(os.listdir(f"{base}/fd"), key=int)) +attempt("fd_readlink", lambda: os.readlink(f"{base}/fd/{fd}")) +attempt("fd_stat_matches_lock", lambda: (lambda a, b: [a.st_dev == b.st_dev, a.st_ino == b.st_ino])(os.stat(f"{base}/fd/{fd}"), os.stat(lock_path))) +attempt("fdinfo_lock_rows", lambda: [l for l in read(f"{base}/fdinfo/{fd}").decode().splitlines() if l.startswith("lock:")]) +attempt("exe_readlink", lambda: os.readlink(f"{base}/exe")) +attempt("cmdline", lambda: read(f"{base}/cmdline").decode(errors="replace").split("\0")) +attempt("stat_start_ticks", lambda: read(f"{base}/stat").decode().rsplit(") ", 1)[1].split()[19]) +attempt("status_uid_line", lambda: [l for l in read(f"{base}/status").decode().splitlines() if l.startswith("Uid:")]) +print(json.dumps(out, indent=2)) diff --git a/experimental/126-guard-liveness/f2-linux-nondumpable/plumbing-smoke.mjs b/experimental/126-guard-liveness/f2-linux-nondumpable/plumbing-smoke.mjs new file mode 100644 index 0000000..aac789d --- /dev/null +++ b/experimental/126-guard-liveness/f2-linux-nondumpable/plumbing-smoke.mjs @@ -0,0 +1,84 @@ +#!/usr/bin/env node +// Cross-platform plumbing check for the F2 inspector (#126 research). +// +// F2_BUILD= RESEARCH_TMP_ROOT= node plumbing-smoke.mjs +// +// Not an F2 result. It checks only that inspect-child.mjs, the generic +// observer and the F2 protocol work together against a real build: +// 1. nominal: every F2 stage is reported once, in order, the build verifies +// and unlock.js is unreachable. Product outcomes (on macOS `prepare` fails +// because there is no /proc) are recorded, not judged. +// 2. F2_CHILD_FAULT=stall:prepare: the stage bound returns `incomplete` with +// open stage `prepare`, and the child's late exit is looked for with a bound. +// The target is a same-user Python flock holder started here. It ends on stdin +// EOF or after 60 s. No process is signalled. +import { log } from 'node:console' +import fs from 'node:fs' +import path from 'node:path' +import process from 'node:process' +import { fileURLToPath } from 'node:url' +import { freshPrivateDir, startChild, stopAll, writeNew } from '../lib/common.mjs' +import { lateExitStatus, runIsolated } from '../lib/isolated-observer.mjs' +import { evaluateNominalPlumbing } from './plumbing-verdict.mjs' +import { verifyBuild } from './build.mjs' +import { manifestBindingFaults } from './build-binding.mjs' + +const DIR = path.dirname(fileURLToPath(import.meta.url)) +const THREAD_ID = '01a089e8-3731-7202-ba68-0f4b0a3b2711' +if (!process.env.F2_BUILD) throw new Error('F2_BUILD is required') +const BUILD = fs.realpathSync(process.env.F2_BUILD) +const buildCheck = verifyBuild(BUILD) +if (!buildCheck.ok) throw new Error(`unverified startup build: ${buildCheck.faults.join(',')}`) +const HOLDER = ` +import fcntl, json, os, select, sys, time +fd = os.open(sys.argv[1], os.O_RDWR | os.O_CLOEXEC | os.O_NOFOLLOW) +fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB) +print(json.dumps({"ready": True, "pid": os.getpid(), "fd": fd}), flush=True) +deadline = time.monotonic() + 60 +while time.monotonic() < deadline: + r, _, _ = select.select([0], [], [], max(0, deadline - time.monotonic())) + if r and not os.read(0, 4096): + break +os.close(fd) +` + +const out = freshPrivateDir('126-f2-plumbing') +const home = path.join(out, 'home') +fs.mkdirSync(path.join(home, 'thread-writer-locks'), { recursive: true, mode: 0o700 }) +fs.mkdirSync(path.join(home, 'sessions', '2026', '10', '16'), { recursive: true, mode: 0o700 }) +writeNew(path.join(home, 'thread-writer-locks', '.coordination.lock'), '') +const lockPath = path.join(home, 'thread-writer-locks', `${THREAD_ID}.lock`) +writeNew(lockPath, '') +writeNew(path.join(home, 'sessions', '2026', '10', '16', `rollout-2026-10-16T00-00-00-${THREAD_ID}.jsonl`), + JSON.stringify({ timestamp: '2026-10-16T00:00:00.000Z', type: 'event_msg', payload: { type: 'task_complete' } }) + '\n') +const live = new Set() +const failures = [] +const report = { build: BUILD, cases: [] } +try { + const holder = startChild('python3', ['-I', '-c', HOLDER, lockPath], { env: { PATH: '/usr/bin:/bin' } }, live) + const ready = JSON.parse(await holder.ready) + const args = [path.join(DIR, 'inspect-child.mjs'), BUILD, String(ready.pid), String(ready.fd), lockPath, home, THREAD_ID, buildCheck.manifestSha256] + const env = { PATH: '/usr/sbin:/usr/bin:/bin', + ...(process.env.RESEARCH_TMP_ROOT !== undefined ? { RESEARCH_TMP_ROOT: process.env.RESEARCH_TMP_ROOT } : {}) } + const nominal = await runIsolated(process.execPath, args, { startupBudgetMs: 10000, budgetMs: 15000, env }) + const verdict = evaluateNominalPlumbing(nominal) + const { protocol } = verdict + failures.push(...manifestBindingFaults(protocol.byStage.dist_identity?.result?.manifestSha256, buildCheck.manifestSha256)) + report.cases.push({ case: 'nominal', faults: nominal.faults, protocolFaults: protocol.faults, begun: nominal.begun, + outcomes: nominal.stages.map((s) => [s.stage, s.error ? `error:${s.error.code ?? s.error.name}:${s.error.message}` : 'result']) }) + failures.push(...verdict.failures) + const stalled = await runIsolated(process.execPath, args, { startupBudgetMs: 10000, budgetMs: 2000, env: { ...env, F2_CHILD_FAULT: 'stall:prepare' } }) + const [late] = stalled.lateExit ? await lateExitStatus([{ label: { case: 'stall:prepare' }, instance: stalled.instance, lateExit: stalled.lateExit }], 20000) : [null] + report.cases.push({ case: 'stall:prepare', incomplete: stalled.incomplete, phase: stalled.incompletePhase, openStage: stalled.openStage, + begun: stalled.begun, late }) + if (!stalled.incomplete || stalled.incompletePhase !== 'stages' || stalled.openStage !== 'prepare') failures.push('stall case not bounded at prepare') + if (late?.status !== 'exited' || late.exit.code !== 4) failures.push(`stall case late exit ${JSON.stringify(late)}`) +} catch (error) { + failures.push(String(error?.stack ?? error)) +} finally { + for (const r of await stopAll(live, 5000)) if (r.incomplete || r.exitCode !== 0) failures.push(`holder ${JSON.stringify(r)}`) +} +Object.assign(report, { ok: failures.length === 0, failures }) +writeNew(path.join(out, 'plumbing-results.json'), JSON.stringify(report, null, 2) + '\n') +log(JSON.stringify({ out, ...report }, null, 2)) +process.exitCode = report.ok ? 0 : 1 diff --git a/experimental/126-guard-liveness/f2-linux-nondumpable/plumbing-verdict.mjs b/experimental/126-guard-liveness/f2-linux-nondumpable/plumbing-verdict.mjs new file mode 100644 index 0000000..50d2788 --- /dev/null +++ b/experimental/126-guard-liveness/f2-linux-nondumpable/plumbing-verdict.mjs @@ -0,0 +1,15 @@ +// Pure nominal plumbing verdict. validateF2 already distinguishes an allowed +// prepare error (for example macOS has no /proc) from a collection failure. +// No stage-level fault may be filtered out by the smoke runner. +import { F2_STAGES, validateF2 } from './protocol.mjs' + +export function evaluateNominalPlumbing(observation) { + const protocol = validateF2(observation) + const failures = [] + if (JSON.stringify(observation.begun) !== JSON.stringify(F2_STAGES)) failures.push(`nominal stage order ${observation.begun}`) + if (protocol.faults.length > 0) failures.push(`nominal protocol ${protocol.faults}`) + if (observation.incomplete || !Array.isArray(observation.faults) || observation.faults.length > 0) { + failures.push(`nominal observer ${observation.faults}`) + } + return { ok: failures.length === 0, failures, protocol } +} diff --git a/experimental/126-guard-liveness/f2-linux-nondumpable/plumbing-verdict.test.mjs b/experimental/126-guard-liveness/f2-linux-nondumpable/plumbing-verdict.test.mjs new file mode 100644 index 0000000..45f7d6d --- /dev/null +++ b/experimental/126-guard-liveness/f2-linux-nondumpable/plumbing-verdict.test.mjs @@ -0,0 +1,66 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' +import { F2_STAGES } from './protocol.mjs' +import { evaluateNominalPlumbing } from './plumbing-verdict.mjs' + +function nominal() { + const results = { + dist_identity: { ok: true, manifestSha256: 'a'.repeat(64), faults: [] }, + module_load: { modules: ['inspection.js'], unlockReachable: false }, + openers: { error: null, processes: [] }, + prepare: { pid: 123, startTicks: '456', candidates: ['4'], descriptors: ['4'] }, + guarded_verify: { status: 'held', method: 'flock_exclusive_nonblocking' }, + inspect_thread: { safeToUnlock: false, blockers: ['unproven'], warnings: [], probe: { status: 'unknown', method: 'flock_exclusive_nonblocking' }, classification: 'unknown' }, + } + return { begun: [...F2_STAGES], stages: F2_STAGES.map((stage) => ({ stage, result: results[stage] })), incomplete: false, faults: [] } +} + +test('nominal plumbing and the protocol-allowed platform prepare error pass', () => { + assert.equal(evaluateNominalPlumbing(nominal()).ok, true) + const observed = nominal() + observed.stages[3] = { stage: 'prepare', error: { code: 'ENOENT', message: '/proc unavailable' } } + observed.stages[4] = { stage: 'guarded_verify', result: { skipped: 'prepare_failed' } } + const verdict = evaluateNominalPlumbing(observed) + assert.equal(verdict.ok, true) + assert.deepEqual(verdict.failures, []) +}) + +test('stream-valid stage errors outside prepare always fail nominal plumbing', () => { + for (const stage of ['openers', 'guarded_verify', 'inspect_thread']) { + const observed = nominal() + observed.stages[F2_STAGES.indexOf(stage)] = { stage, error: { code: 'EIO', message: 'synthetic error' } } + assert.deepEqual(observed.faults, []) + const verdict = evaluateNominalPlumbing(observed) + assert.equal(verdict.ok, false, stage) + assert.ok(verdict.protocol.faults.includes(`stage_failed:${stage}`)) + assert.ok(verdict.failures.some((f) => f.includes(`stage_failed:${stage}`))) + } +}) + +test('stream-valid malformed stage payloads and malformed prepare errors fail', () => { + for (const stage of F2_STAGES) { + const observed = nominal() + observed.stages[F2_STAGES.indexOf(stage)] = { stage, result: {} } + const verdict = evaluateNominalPlumbing(observed) + assert.equal(verdict.ok, false, stage) + assert.ok(verdict.protocol.faults.includes(`stage_result_malformed:${stage}`)) + } + const observed = nominal() + observed.stages[3] = { stage: 'prepare', error: {} } + const verdict = evaluateNominalPlumbing(observed) + assert.equal(verdict.ok, false) + assert.ok(verdict.protocol.faults.includes('stage_error_malformed:prepare')) +}) + +test('missing, reordered or duplicate stages and observer failures stay failed', () => { + const missing = nominal() + missing.stages.pop() + const reordered = nominal() + reordered.begun.reverse() + const duplicate = nominal() + duplicate.stages.push(duplicate.stages[0]) + for (const observed of [missing, reordered, duplicate, { ...nominal(), incomplete: true }, + { ...nominal(), faults: ['stdout_limit'] }, { ...nominal(), faults: undefined }]) { + assert.equal(evaluateNominalPlumbing(observed).ok, false) + } +}) diff --git a/experimental/126-guard-liveness/f2-linux-nondumpable/protocol.mjs b/experimental/126-guard-liveness/f2-linux-nondumpable/protocol.mjs new file mode 100644 index 0000000..a45a34d --- /dev/null +++ b/experimental/126-guard-liveness/f2-linux-nondumpable/protocol.mjs @@ -0,0 +1,72 @@ +// F2 stage protocol (#126 research), on top of lib/isolated-observer.mjs. +// +// The inspector must report exactly these stages, once each, in this order. +// Each stage's result or error must have the shape below. A shape fault means +// collection failed; it is never read as a product refusal. +// Build content is verified before any product module is imported. +export const F2_STAGES = ['dist_identity', 'module_load', 'openers', 'prepare', 'guarded_verify', 'inspect_thread'] +// Only `prepare` may end in an error that counts as an observed product +// outcome. Every other stage error is a collection fault. +const ERROR_ALLOWED = new Set(['prepare']) + +const isObject = (v) => v !== null && typeof v === 'object' && !Array.isArray(v) +const isTicks = (v) => typeof v === 'string' && /^\d+$/.test(v) +const isStrings = (v) => Array.isArray(v) && v.every((x) => typeof x === 'string') +const isPid = (v) => Number.isSafeInteger(v) && v > 0 + +// Pinned #137 b483cd5 src/types.ts LockProbe contract. Validate the nested +// probe even in refusal cases; {} is not an observed unknown lock state. +const isParticipation = (p) => isObject(p) && isPid(p.pid) && isTicks(p.startTicks) && + typeof p.descriptor === 'string' && /^\d+$/.test(p.descriptor) +export const isLockProbe = (p) => isObject(p) && ['held', 'free', 'unknown'].includes(p.status) && + p.method === 'flock_exclusive_nonblocking' && + (p.error === undefined || typeof p.error === 'string') && + (p.guard === undefined || (isObject(p.guard) && ['acquired', 'busy', 'absent', 'unsafe', 'changed'].includes(p.guard.status) && + (p.guard.attempts === undefined || (Number.isSafeInteger(p.guard.attempts) && p.guard.attempts > 0)))) && + (p.participation === undefined || isParticipation(p.participation)) + +const RESULT_SHAPES = { + module_load: (r) => isObject(r) && isStrings(r.modules) && r.unlockReachable === false, + dist_identity: (r) => isObject(r) && r.ok === true && /^[0-9a-f]{64}$/.test(r.manifestSha256 ?? '') && + Array.isArray(r.faults) && r.faults.length === 0, + openers: (r) => isObject(r) && (r.error === null || typeof r.error === 'string') && Array.isArray(r.processes) && + r.processes.every((p) => isObject(p) && isPid(p.pid) && typeof p.isCodex === 'boolean' && + typeof p.identityComplete === 'boolean' && isStrings(p.errors) && (p.startTicks === null || isTicks(p.startTicks))), + prepare: (r) => isObject(r) && isPid(r.pid) && isTicks(r.startTicks) && isStrings(r.candidates) && isStrings(r.descriptors), + guarded_verify: (r) => isObject(r) && (r.skipped === 'prepare_failed' || isLockProbe(r)), + inspect_thread: (r) => isObject(r) && typeof r.safeToUnlock === 'boolean' && isStrings(r.blockers) && + isStrings(r.warnings) && isLockProbe(r.probe) && typeof r.classification === 'string', +} +const ERROR_SHAPE = (e) => isObject(e) && (e.code === null || typeof e.code === 'string') && typeof e.message === 'string' + +export function validateF2(observation) { + const faults = [] + const byStage = {} + const begun = observation.begun ?? [] + begun.forEach((name, index) => { + if (F2_STAGES[index] !== name) faults.push(`stage_order:${index}:${name}`) + }) + for (const s of observation.stages ?? []) { + if (Object.hasOwn(byStage, s.stage)) { faults.push(`stage_duplicate:${s.stage}`); continue } + byStage[s.stage] = s + if (!F2_STAGES.includes(s.stage)) { faults.push(`stage_unknown:${s.stage}`); continue } + if (Object.hasOwn(s, 'error')) { + if (!ERROR_ALLOWED.has(s.stage)) faults.push(`stage_failed:${s.stage}`) + else if (!ERROR_SHAPE(s.error)) faults.push(`stage_error_malformed:${s.stage}`) + } else if (!RESULT_SHAPES[s.stage](s.result)) { + faults.push(`stage_result_malformed:${s.stage}`) + } + } + const missing = F2_STAGES.filter((name) => !Object.hasOwn(byStage, name)) + if (missing.length) faults.push(`stages_missing:${missing.join(',')}`) + return { ok: faults.length === 0, faults, byStage, missing } +} + +// Accepted F2_CHILD_FAULT values; anything else is refused before any child runs. +export const CHILD_FAULTS = new Set(F2_STAGES.map((s) => `stall:${s}`)) + +export function parseChildFault(value) { + if (value === undefined || value === '') return null + if (!CHILD_FAULTS.has(value)) throw new Error(`F2_CHILD_FAULT must be one of ${[...CHILD_FAULTS].join(', ')}`) + return value +} diff --git a/experimental/126-guard-liveness/f2-linux-nondumpable/run.mjs b/experimental/126-guard-liveness/f2-linux-nondumpable/run.mjs new file mode 100644 index 0000000..fc434eb --- /dev/null +++ b/experimental/126-guard-liveness/f2-linux-nondumpable/run.mjs @@ -0,0 +1,199 @@ +#!/usr/bin/env node +// F2 access-matrix runner (#126 research). Refuses to run off Linux, as root, or with +// differing real and effective UIDs. +// +// For FIXTURE_DUMPABLE in [1, 0]: start the synthetic `codex` holder in a +// fresh private scratch directory, validate its ready record, record the +// same-user /proc access matrix. With F2_BUILD set to a directory made by +// build.mjs, the runner verifies that build against its manifest, then runs +// inspect-child.mjs in a separate process (lib/isolated-observer.mjs). The +// stages are checked by protocol.mjs and evaluated by expectations.mjs. This +// runner never imports a product module: a synchronous product stall blocks +// only that child, and this process records it as incomplete. No unlock +// method is called. +// +// Signals: this runner signals neither the holder, the observer nor the +// inspector child. An inspector past its bound gets stdin EOF and is left +// running; it stops at its next stage boundary, and a child stuck inside a +// syscall has no exit guarantee. The holder ends on stdin EOF or its own +// lifetime (HOLDER_LIFETIME_S). The observer is waited for with a bound and, +// if it has not exited, is recorded as incomplete and left running (a /proc +// read in a killable kernel wait cannot be interrupted by a handled signal +// anyway). Product diagnostics inside the inspector may signal their own +// timed-out diagnostic process groups; see inspect-child.mjs. +// +// Any validation failure, error, unconfirmed exit or observation made without +// the same live holder instance makes the run exit nonzero. Scratch is always +// kept. +import { spawn } from 'node:child_process' +import { log } from 'node:console' +import fs from 'node:fs' +import os from 'node:os' +import path from 'node:path' +import { performance } from 'node:perf_hooks' +import process from 'node:process' +import { setTimeout as delay } from 'node:timers/promises' +import { fileURLToPath } from 'node:url' +import { freshPrivateDir, startChild, stopAll, stopChild, writeNew } from '../lib/common.mjs' +import { LineCollector, lateExitStatus, processInstance, runIsolated } from '../lib/isolated-observer.mjs' +import { verifyBuild } from './build.mjs' +import { evaluateReport, evaluateRun, sameTicks } from './expectations.mjs' +import { parseChildFault, validateF2 } from './protocol.mjs' +import { validateAccessObservation } from './access-observation.mjs' + +if (process.platform !== 'linux') throw new Error('F2 runs on Linux only') +if (process.getuid() === 0 || process.getuid() !== process.geteuid()) throw new Error('F2 requires an unprivileged same-UID runner') +if (process.env.F2_DIST) throw new Error('F2_DIST is replaced by F2_BUILD, a directory made by build.mjs') +const DIR = path.dirname(fileURLToPath(import.meta.url)) +const THREAD_ID = '01a089e8-3731-7202-ba68-0f4b0a3b2711' +const OBSERVER_BUDGET_MS = 10000 +const OBSERVER_OUTPUT_BYTES = 256 << 10 +const INSPECT_STARTUP_MS = 10000 +const INSPECT_STAGES_MS = 15000 +// Holder lifetime must cover readiness (5 s), observation, inspection and +// cleanup with margin; observations after the holder ends are invalid. +const HOLDER_LIFETIME_S = 60 +const NEEDED_MS = OBSERVER_BUDGET_MS + INSPECT_STARTUP_MS + INSPECT_STAGES_MS + 5000 +const BUILD = process.env.F2_BUILD ? fs.realpathSync(process.env.F2_BUILD) : null +const CHILD_FAULT = parseChildFault(process.env.F2_CHILD_FAULT) +if (CHILD_FAULT && !BUILD) throw new Error('F2_CHILD_FAULT requires F2_BUILD') +// The runner's own check. inspect-child repeats it before importing. +const buildCheck = BUILD ? verifyBuild(BUILD) : null +if (buildCheck && !buildCheck.ok) throw new Error(`F2_BUILD does not match its manifest: ${buildCheck.faults.join(',')}`) + +const out = freshPrivateDir('126-f2') +const live = new Set() +const failures = [] +const incomplete = [] +const lateExits = [] +const read = (file) => (fs.existsSync(file) ? fs.readFileSync(file, 'utf8').trim() : null) +const report = { + environment: { + kernel: os.release(), node: process.version, uid: process.getuid(), + ptraceScope: read('/proc/sys/kernel/yama/ptrace_scope'), + procMount: fs.readFileSync('/proc/self/mountinfo', 'utf8').split('\n').find((l) => / \/proc /.test(l)) ?? null, + mode: BUILD ? 'access_matrix_and_isolated_inspection' : 'access_matrix_only', + build: buildCheck && { dir: BUILD, manifestSha256: buildCheck.manifestSha256, source: buildCheck.source, patch: buildCheck.patch, + verifiedBy: 'run.mjs re-hash against manifest.json' }, + childFault: CHILD_FAULT, + }, + runs: [], +} + +// Bounded output, parsed only after both captured streams close. +function observe(pid, fd, lockPath) { + const child = spawn('python3', ['-I', path.join(DIR, 'observe.py'), String(pid), String(fd), lockPath], { stdio: ['ignore', 'pipe', 'pipe'] }) + const stdout = new LineCollector(OBSERVER_OUTPUT_BYTES) + const stderr = new LineCollector(64 << 10) + child.stdout.on('data', (chunk) => { stdout.push(chunk) }) + child.stderr.on('data', (chunk) => { stderr.push(chunk) }) + const closed = Promise.all([child.stdout, child.stderr].map((stream) => + new Promise((resolve) => { stream.once('close', resolve) }))) + const exited = new Promise((resolve) => { + child.once('exit', (code, signal) => resolve({ code, signal })) + child.once('error', (error) => resolve({ code: null, error: error.message })) + }) + const done = exited.then(async (result) => { await closed; return result }) + return Promise.race([done, delay(OBSERVER_BUDGET_MS).then(() => null)]).then((result) => { + if (result === null) { + // Leave it running; do not let it hold this process open. + child.unref(); child.stdout.destroy(); child.stderr.destroy() + return { incomplete: true, instance: processInstance(child.pid) } + } + const faults = [...stdout.faults('stdout'), ...stderr.faults('stderr')] + if (result.code !== 0 || faults.length) return { failed: result, faults, stderr: stderr.text() } + try { return { result: JSON.parse(stdout.text()) } } catch { return { failed: 'unparseable', stderr: stderr.text() } } + }) +} + +for (const dumpable of [1, 0]) { + const home = path.join(out, `home-${dumpable}`) + const lockDir = path.join(home, 'thread-writer-locks') + const sessions = path.join(home, 'sessions', '2026', '10', '14') + fs.mkdirSync(lockDir, { recursive: true, mode: 0o700 }) + fs.mkdirSync(sessions, { recursive: true, mode: 0o700 }) + writeNew(path.join(lockDir, '.coordination.lock'), '') + const lockPath = path.join(lockDir, `${THREAD_ID}.lock`) + writeNew(lockPath, '') + writeNew(path.join(sessions, `rollout-2026-10-14T00-00-00-${THREAD_ID}.jsonl`), + JSON.stringify({ timestamp: '2026-10-14T00:00:00.000Z', type: 'event_msg', payload: { type: 'task_complete' } }) + '\n') + const run = { requestedDumpable: dumpable } + const startedAt = performance.now() + const holder = startChild(path.join(DIR, 'codex'), [], { + env: { PATH: '/usr/bin:/bin', FIXTURE_LOCK_PATH: lockPath, FIXTURE_DUMPABLE: String(dumpable), FIXTURE_LIFETIME_SECONDS: String(HOLDER_LIFETIME_S) }, + }, live) + let holderExited = false + holder.exited.then(() => { holderExited = true }) + try { + const ready = JSON.parse(await holder.ready) + run.holder = ready + // Independent instance sample, taken by this runner at readiness. + run.holderInstanceAtReady = processInstance(ready.pid) + if (ready.ready !== true || ready.pid !== holder.pid || !Number.isInteger(ready.fd) || ready.fd < 3 || + ready.dumpable !== dumpable || ready.uid !== process.getuid() || ready.euid !== process.getuid() || + ready.lifetime_seconds !== HOLDER_LIFETIME_S) { + throw new Error(`holder ready record invalid: ${JSON.stringify(ready)}`) + } + const observed = await observe(ready.pid, ready.fd, lockPath) + run.observe = observed + run.holderInstanceAfterObserve = processInstance(ready.pid) + if (observed.incomplete) incomplete.push({ role: 'observer', dumpable, instance: observed.instance }) + else if (!Object.hasOwn(observed, 'result')) failures.push(`observer dumpable=${dumpable}: ${JSON.stringify(observed)}`) + else { + run.accessCollection = validateAccessObservation(observed.result, { + pid: ready.pid, fd: ready.fd, uid: process.getuid(), kernel: report.environment.kernel, + startTicks: run.holderInstanceAtReady.startTicks, + }) + if (!run.accessCollection.collected) failures.push(`observer dumpable=${dumpable}: ${run.accessCollection.faults.join(',')}`) + } + if (!sameTicks(run.holderInstanceAtReady.startTicks, run.holderInstanceAfterObserve.startTicks) || holderExited) { + failures.push(`dumpable=${dumpable}: holder instance changed or ended during observation`) + } + if (BUILD) { + if (performance.now() - startedAt + NEEDED_MS > HOLDER_LIFETIME_S * 1000) throw new Error('holder lifetime insufficient for inspection') + const inspected = await runIsolated(process.execPath, + [path.join(DIR, 'inspect-child.mjs'), BUILD, String(ready.pid), String(ready.fd), lockPath, home, THREAD_ID, buildCheck.manifestSha256], { + startupBudgetMs: INSPECT_STARTUP_MS, budgetMs: INSPECT_STAGES_MS, + env: { PATH: '/usr/sbin:/usr/bin:/bin', + ...(process.env.RESEARCH_TMP_ROOT !== undefined ? { RESEARCH_TMP_ROOT: process.env.RESEARCH_TMP_ROOT } : {}), + ...(CHILD_FAULT ? { F2_CHILD_FAULT: CHILD_FAULT } : {}) }, + }) + const { lateExit, ...record } = inspected + run.inspect = record + run.holderInstanceAfterInspect = processInstance(ready.pid) + if (inspected.incomplete) { + incomplete.push({ role: 'inspector', dumpable, instance: inspected.instance, phase: inspected.incompletePhase, openStage: inspected.openStage }) + lateExits.push({ label: { dumpable }, instance: inspected.instance, lateExit }) + } + run.protocol = validateF2(inspected) + run.evaluation = evaluateRun({ + dumpable, holder: ready, observation: inspected, protocol: run.protocol, expectedManifestSha256: buildCheck.manifestSha256, + ticks: { ready: run.holderInstanceAtReady.startTicks, afterInspect: run.holderInstanceAfterInspect.startTicks, + holderExitedDuringInspection: holderExited }, + }) + delete run.protocol.byStage + if (!run.evaluation.collected) failures.push(`inspector dumpable=${dumpable}: ${run.evaluation.collectedFaults.join(',')}`) + } + } catch (error) { + run.error = String(error?.stack ?? error) + failures.push(`dumpable=${dumpable}: ${run.error}`) + } finally { + run.holderExit = await stopChild(holder, 5000) + if (run.holderExit.incomplete) incomplete.push({ role: 'holder', ...run.holderExit }) + else if (run.holderExit.exitCode !== 0 && !run.error) failures.push(`holder dumpable=${dumpable} exit ${JSON.stringify(run.holderExit)} ${holder.stderr}`) + } + report.runs.push(run) +} +for (const result of await stopAll(live, 5000)) if (result.incomplete) incomplete.push({ role: 'holder', ...result }) +if (report.runs.length !== 2) failures.push(`runs ${report.runs.length}/2`) +// Bounded look for late inspector exits, one status per instance. +report.lateInspectorExits = await lateExitStatus(lateExits, 2000) +const collected = failures.length === 0 && incomplete.length === 0 +report.expectations = BUILD ? evaluateReport(report.runs.map((r) => r.evaluation).filter(Boolean)) : null +const ok = collected && (!BUILD || report.expectations.ok) +Object.assign(report, { ok, collected, failures, incomplete }) +writeNew(path.join(out, 'f2-results.json'), JSON.stringify(report, null, 2) + '\n') +log(JSON.stringify({ out, ok, collected, failures, incomplete, expectations: report.expectations, + runs: report.runs.map((r) => ({ d: r.requestedDumpable, holder: r.holder, exit: r.holderExit, + rows: r.evaluation?.rows.map((x) => `${x.id}:${x.ok}`), recorded: r.evaluation?.recorded })) }, null, 2)) +process.exitCode = ok ? 0 : 1 diff --git a/experimental/126-guard-liveness/f4-guard-scope/README.md b/experimental/126-guard-liveness/f4-guard-scope/README.md new file mode 100644 index 0000000..462f92c --- /dev/null +++ b/experimental/126-guard-liveness/f4-guard-scope/README.md @@ -0,0 +1,134 @@ +# F4: narrowing the native-guard critical section of #137 (proposal, #126) + +Research proposal only. `guard-scope.patch` applies to #137 head +`b483cd587c62329243366e64aaaac477edd36adc` with `git apply`. It is **not +applied to the product** or to any branch, and it does not make #137 +release-ready. Status: **no-go for readiness**. The patch removes two read +paths from the guard. It does not bound the guard, and it does not change the +signal boundary in any validated way. Any change at the signal boundary +requires a separate Linux run and review first. + +## Problem restated + +#137 holds `.coordination.lock` during `verifyLinuxParticipation`. That +function makes synchronous syscalls with no hard deadline. Two of them wait on +state that the existing guarded probe does not touch: + +1. `linuxCommandFingerprint`, called twice inside the guard. It reads + `/proc/PID/cmdline`, which reaches `access_remote_vm` and the target's + `mmap_lock`. It also reads `/proc/PID/exe`. +2. `statSync('/proc/PID/fd/N')` before and after the fdinfo read. It follows + the descriptor to the target file and calls that filesystem's getattr. + +A real Codex waiting on the guard with a blocking flock waits as long as the +guard is held. F1 measured this with an injected delay and one synthetic +waiter. + +## Change + +| Step | #137 `b483cd5` | Patch | +| --- | --- | --- | +| Pre-guard `prepareLinuxParticipation` | cmdline, exe, stat, fd list, readlinks | unchanged | +| In guard: identity | proc dir UID, mount ns, start ticks, **command fingerprint**, fd list | proc dir UID, mount ns, start ticks, fd list | +| In guard: descriptor | readlink, **stat via fd**, fdinfo, **stat via fd**, readlink | readlink, fdinfo, readlink | +| In guard: participation evidence | stat via fd and granted row device/inode | granted row device/inode vs `stat(lockPath)` only | +| After release | none | `confirmLinuxCommand`: start ticks and command fingerprint again | +| Before signal (unlock) | none | `assertFinalBoundary`: the confirm step above must take ≤ 25 ms | + +The patch drops the stat through the descriptor and keeps the row comparison +that #137 already made. It is not a stronger binding. In pinned v6.17 +`fs/locks.c`, `show_fd_locks` lists entries of the inode's granted +`flc_flock`/`flc_posix`/`flc_lease` lists whose `flc_file` is the descriptor's +file, and `lock_get_status` prints `MAJOR/MINOR(file_inode(file)->i_sb->s_dev)` +and `i_ino`. A matching row is therefore evidence that the **sampled** +description participated in a granted flock on that inode at the time of the +read. Blocked waiters are not on these lists. The superblock device equals +`st_dev` only on the allowlisted filesystems, which is why the allowlist stays. +The readlinks before and after remain a negative filter only. + +`verifyLinuxParticipation` gains an optional `trace` callback for tests and +review; production passes none. The callback is written by hand at each call +site. It does not report `checkNamespace` or the descriptor directory's +internal reads individually, and it is not syscall instrumentation. A trace +without a path is not evidence that the path is never reached. + +### Reads left inside the guard, checked against pinned Linux v6.17 + +Source: `torvalds/linux@e5f0a698b34ed76002dc5cff3804a61c80233a7a`, read for +this proposal. This is a targeted check, not an exhaustive audit. + +| Read | Kernel path | Can wait on | +| --- | --- | --- | +| `/proc/PID/stat` | `do_task_stat`: `get_task_mm`, `lock_task_sighand`, `stats_lock` seqlock, RCU | task and signal locks; no `mmap_lock` in this function | +| `/proc/PID/fd` readdir, `fd/N` readlink | `proc_readfd_common`, `proc_fd_link`: RCU, `files->file_lock`, `d_path` | spinlocks; no target filesystem I/O | +| `/proc/PID/fdinfo/N` | `seq_show`: `task_lock`, `files->file_lock`, `show_fd_locks`, then **`file->f_op->show_fdinfo`** | the file type's `show_fdinfo`. For example `ep_show_fdinfo` (`fs/eventpoll.c`) takes `mutex_lock(&ep->mtx)` | +| `stat /proc/PID`, `ns/mnt` readlink | `pid_getattr`/`task_dump_owner`, `ns_get_name` | task lock | +| `statfs`/`stat` of the lock path | lock file's filesystem | the same filesystem that the existing guarded probe already opens and flocks | + +Residual paths that the patch does not remove: + +- **fdinfo through `show_fdinfo`.** The readlink before fdinfo can match the + lock path. FD N can then be closed and reused, for example by an epoll file, + before the fdinfo read. That read can then wait on `ep->mtx` while + coordination is held. The readlink after the read refuses the result, but a + refusal cannot undo a wait that already happened. This path exists in #137 + `b483cd5` as well. +- **Lock path filesystem.** It can stall the guard just as it can stall main's + existing guarded probe. The ext/tmpfs/XFS allowlist applies only after + `statfs` returns. +- Every remaining read is a synchronous syscall with no hard deadline. + +### What `assertFinalBoundary` measures + +The 25 ms check covers **only** the post-release confirm step: from the +return of `guardedProbeOnce` to the end of `confirmLinuxCommand`. A slow +confirm step is refused through the existing `unknown` path. It does not +cover: + +- the time between the guard's `LOCK_UN` and the return of `guardedProbeOnce`; +- the time from the check to `process.kill`; +- the age of the in-guard evidence (the fdinfo row was read before this + window). + +So it is **not** a staleness bound and **not** a release-to-signal bound. It +only stops a slow command read after release from being followed by a signal. +`confirmLinuxCommand` refuses an exec that changes argv or the exe path. A +same-path re-exec with identical argv is still undetected. Inside and around +the measured step, nothing proves that the owner still holds the lock at +`kill`. Node exposes no pidfd, and this repository adds no native dependency, +so a PID-reuse window remains. Post-signal release and transcript verification +remain mandatory. + +## Validation run + +On macOS 27.0.1 arm64 with Node v24.16.0, using the exported `b483cd5` tree +with the patch applied (patch SHA-256 +`ae73cacce078264e3426eed5cbebe901ac2fb9f3371bb99dc2b1471a8eed8d74`; it +supersedes `4fa037d2…` by changing comments and test names only): + +| Check | Result | +| --- | --- | +| `npm ci`, `npm run check`, `npm run lint` | pass | +| `npm test` | 280 tests, 273 pass, 0 fail, 7 skipped (Linux-only) | +| `npm run smoke:package` | pass | +| `git diff --check`, `git apply` through `../f2-linux-nondumpable/build.mjs --patch` | pass | + +Unpatched `b483cd5`, as recorded in #137, has 278 tests with 272 passing and +6 skipped. The new cross-platform test is the confirm-step test. The new +Linux-only trace test and the modified Linux participation test are **not +executed** on this host. A Linux run is required before any claim about them. + +## Gate status after the patch + +| #137 gate | `b483cd5` | With patch | Still needed | +| --- | --- | --- | --- | +| Guard held during target `mmap_lock` wait (cmdline) | exposed | cmdline/exe not read in the guard (source-checked) | Linux run of the trace test | +| Guard held during FD target getattr | exposed | stat via FD not made | same | +| Guard held during fdinfo `show_fdinfo` of a reused FD | exposed | **unchanged** | design for FD reuse between readlinks; no proposal here | +| Guard held during lock filesystem stall | exposed (same as main) | unchanged | policy decision: accept parity with main, or refuse non-local paths before the first probe | +| Slow command read after release followed by a signal | not checked | refused if the confirm step exceeds 25 ms | evidence age and release-to-signal remain unbounded; no pidfd | +| Native dumpability/access | open | unchanged | F2 isolated inspection on Linux (`../f2-linux-nondumpable`) | + +The decision on the remaining gates belongs to the reviewer and the user. A +stronger review (another model or a kernel-aware reviewer) of the v6.17 call +paths above is recommended before readiness is claimed. diff --git a/experimental/126-guard-liveness/f4-guard-scope/guard-scope.patch b/experimental/126-guard-liveness/f4-guard-scope/guard-scope.patch new file mode 100644 index 0000000..3b14aa9 --- /dev/null +++ b/experimental/126-guard-liveness/f4-guard-scope/guard-scope.patch @@ -0,0 +1,226 @@ +diff --git a/src/inspection.ts b/src/inspection.ts +index 25b9253..2999624 100644 +--- a/src/inspection.ts ++++ b/src/inspection.ts +@@ -3,7 +3,7 @@ import { join } from "node:path"; + + import { observeLockDirectory, observeLockFile } from "./lock.js"; + import { guardedProbeOnce, probeWithRetry } from "./native-coordination.js"; +-import { prepareLinuxParticipation, verifyLinuxParticipation } from "./linux-attribution.js"; ++import { confirmLinuxCommand, prepareLinuxParticipation, verifyLinuxParticipation } from "./linux-attribution.js"; + import { isThreadId, defaultOptions, validateThreadId } from "./options.js"; + import { evaluateSafety } from "./policy.js"; + import { +@@ -151,6 +151,8 @@ export async function inspectThread( + !sameSnapshot(observedLock.snapshot, fresh.snapshot)) throw new Error("linux_target_changed"); + const result = guardedProbeOnce(lockPath, observedLock, undefined, + () => verifyLinuxParticipation(lockPath, prepared)); ++ // The guard is released; command evidence is read only now. ++ if (result.status === "held") confirmLinuxCommand(prepared); + Object.assign(probe, result); + } catch { + probe.status = "unknown"; +diff --git a/src/linux-attribution.ts b/src/linux-attribution.ts +index e87925d..a96009b 100644 +--- a/src/linux-attribution.ts ++++ b/src/linux-attribution.ts +@@ -8,6 +8,16 @@ const BYTE_LIMIT = 65536; + const FD_LIMIT = 256; + const BUDGET_MS = 25; + const FILESYSTEMS = new Set([0xef53n, 0x01021994n, 0x58465342n]); // ext, tmpfs, XFS ++// Upper bound for the post-release confirm step measured by assertFinalBoundary. ++// It is not an evidence-age bound and not a release-to-signal bound. ++export const FINAL_BOUNDARY_MS = 25; ++/** ++ * Reads reported by verifyLinuxParticipation's optional trace callback. The ++ * callback is manual and not exhaustive (for example, checkNamespace and the ++ * descriptor directory's internal reads are reported coarsely), so it is ++ * review evidence, not proof that no other path is touched. ++ */ ++export type GuardedRead = { op: "readlink" | "read" | "stat" | "statfs" | "opendir"; path: string }; + + export interface LinuxParticipation { pid: number; startTicks: string; descriptor: string } + export interface LinuxPreparation { +@@ -146,22 +156,42 @@ export function linuxStartTicks(pid: number): string { + return parseStartTicks(boundedRead(`/proc/${pid}/stat`), pid); + } + +-/** Called synchronously under native coordination, after independent held probe. */ +-export function verifyLinuxParticipation(lockPath: string, prepared: LinuxPreparation): LinuxParticipation { ++/** ++ * Called synchronously under native coordination, after independent held probe. ++ * ++ * This function does not read cmdline/exe and does not stat through ++ * /proc/PID/fd/N; command evidence is confirmed after release by ++ * confirmLinuxCommand. A granted row whose device and inode match the lock ++ * path is the evidence that the sampled description participates. Every ++ * remaining read is still a synchronous syscall without a hard deadline. The ++ * fdinfo read calls the file's show_fdinfo: if FD N is reused between the ++ * readlinks (for example by an epoll file, whose show_fdinfo takes ep->mtx), ++ * it can wait while coordination is held. A later refusal cannot undo a wait ++ * that already happened. ++ */ ++export function verifyLinuxParticipation( ++ lockPath: string, prepared: LinuxPreparation, trace?: (read: GuardedRead) => void, ++): LinuxParticipation { + const deadline = performance.now() + BUDGET_MS; + const checkBudget = (): void => { if (performance.now() > deadline) throw new Error("linux_attribution_budget"); }; ++ const link = (path: string): string => { trace?.({ op: "readlink", path }); return readlinkSync(path); }; ++ const read = (path: string): string => { trace?.({ op: "read", path }); return boundedRead(path, deadline); }; ++ const listed = (): string[] => { trace?.({ op: "opendir", path: `/proc/${prepared.pid}/fd` }); return descriptors(prepared.pid, deadline); }; ++ trace?.({ op: "statfs", path: "/proc" }); + checkNamespace(); ++ trace?.({ op: "statfs", path: lockPath }); + if (!FILESYSTEMS.has(statfsSync(lockPath, { bigint: true }).type)) throw new Error("linux_filesystem_unsupported"); ++ trace?.({ op: "stat", path: lockPath }); + const target = statSync(lockPath, { bigint: true }); + if (!target.isFile() || target.nlink !== 1n || target.uid !== prepared.uid) throw new Error("linux_target_changed"); + const validateIdentity = (): void => { + checkBudget(); ++ trace?.({ op: "stat", path: `/proc/${prepared.pid}` }); + if (statSync(`/proc/${prepared.pid}`, { bigint: true }).uid !== prepared.uid || +- readlinkSync("/proc/self/ns/mnt") !== prepared.mountNamespace || +- readlinkSync(`/proc/${prepared.pid}/ns/mnt`) !== prepared.mountNamespace || +- parseStartTicks(boundedRead(`/proc/${prepared.pid}/stat`, deadline), prepared.pid) !== prepared.startTicks || +- linuxCommandFingerprint(prepared.pid, deadline) !== prepared.commandFingerprint || +- JSON.stringify(descriptors(prepared.pid, deadline)) !== JSON.stringify(prepared.descriptors)) { ++ link("/proc/self/ns/mnt") !== prepared.mountNamespace || ++ link(`/proc/${prepared.pid}/ns/mnt`) !== prepared.mountNamespace || ++ parseStartTicks(read(`/proc/${prepared.pid}/stat`), prepared.pid) !== prepared.startTicks || ++ JSON.stringify(listed()) !== JSON.stringify(prepared.descriptors)) { + throw new Error("linux_identity_changed"); + } + }; +@@ -170,12 +200,9 @@ export function verifyLinuxParticipation(lockPath: string, prepared: LinuxPrepar + for (const descriptor of prepared.candidates) { + checkBudget(); + const path = `/proc/${prepared.pid}/fd/${descriptor}`; +- if (!prepared.descriptors.includes(descriptor) || readlinkSync(path) !== prepared.targetPath) throw new Error("linux_descriptor_changed"); +- const before = statSync(path, { bigint: true }); +- if (before.dev !== target.dev || before.ino !== target.ino) throw new Error("linux_descriptor_changed"); +- const rows = parseFlockRows(boundedRead(`/proc/${prepared.pid}/fdinfo/${descriptor}`, deadline)); +- const after = statSync(path, { bigint: true }); +- if (readlinkSync(path) !== prepared.targetPath || before.dev !== after.dev || before.ino !== after.ino) throw new Error("linux_descriptor_changed"); ++ if (!prepared.descriptors.includes(descriptor) || link(path) !== prepared.targetPath) throw new Error("linux_descriptor_changed"); ++ const rows = parseFlockRows(read(`/proc/${prepared.pid}/fdinfo/${descriptor}`)); ++ if (link(path) !== prepared.targetPath) throw new Error("linux_descriptor_changed"); + const major = ((target.dev >> 8n) & 0xfffn) | ((target.dev >> 32n) & 0xfffff000n); + const minor = (target.dev & 0xffn) | ((target.dev >> 12n) & 0xffffff00n); + if (rows.some((row) => row.major !== major || row.minor !== minor || row.inode !== target.ino)) throw new Error("linux_row_identity_mismatch"); +@@ -183,9 +210,30 @@ export function verifyLinuxParticipation(lockPath: string, prepared: LinuxPrepar + else if (rows.length > 1) throw new Error("linux_rows_ambiguous"); + } + validateIdentity(); ++ trace?.({ op: "stat", path: lockPath }); + const after = statSync(lockPath, { bigint: true }); + if (!after.isFile() || after.nlink !== 1n || after.uid !== prepared.uid || + after.dev !== target.dev || after.ino !== target.ino || matches.length !== 1) throw new Error("linux_participation_unproven"); + checkBudget(); + return { pid: prepared.pid, startTicks: prepared.startTicks, descriptor: matches[0] }; + } ++ ++/** ++ * After native coordination is released: the command fingerprint (cmdline and ++ * exe, which may wait on the target's memory map) and start ticks must still ++ * match the prepared instance. A same-path re-exec with identical argv remains ++ * undetectable. ++ */ ++export function confirmLinuxCommand(prepared: LinuxPreparation): void { ++ if (linuxStartTicks(prepared.pid) !== prepared.startTicks || ++ linuxCommandFingerprint(prepared.pid) !== prepared.commandFingerprint) throw new Error("linux_identity_changed"); ++} ++ ++/** ++ * Refuses when the post-release confirm step took longer than FINAL_BOUNDARY_MS. ++ * It does not measure evidence age, any delay before guardedProbeOnce returned, ++ * or the time from this check to process.kill. ++ */ ++export function assertFinalBoundary(releasedAt: number, now = performance.now()): void { ++ if (!(now - releasedAt <= FINAL_BOUNDARY_MS)) throw new Error("linux_evidence_stale"); ++} +diff --git a/src/unlock.ts b/src/unlock.ts +index 04fcdf0..6e66a89 100644 +--- a/src/unlock.ts ++++ b/src/unlock.ts +@@ -1,6 +1,6 @@ + import { acquireUnlockLease } from "./coordination.js"; + import { inspectThread } from "./inspection.js"; +-import { prepareLinuxParticipation, verifyLinuxParticipation } from "./linux-attribution.js"; ++import { assertFinalBoundary, confirmLinuxCommand, prepareLinuxParticipation, verifyLinuxParticipation } from "./linux-attribution.js"; + import { performance } from "node:perf_hooks"; + import { closeSync, constants, fstatSync, lstatSync, openSync } from "node:fs"; + +@@ -492,6 +492,13 @@ export async function unlockInspectedThread( + prepared.commandFingerprint !== finalInspection.owner?.commandFingerprint)) throw new Error("linux_identity_changed"); + preSignalProbe = guardedProbeOnce(inspection.lock.path, preSignalLock, undefined, + prepared ? () => verifyLinuxParticipation(inspection.lock.path, prepared) : undefined); ++ if (prepared && preSignalProbe.status === "held") { ++ // Bounds only this confirm step after guardedProbeOnce returned; not ++ // the evidence age and not the time until process.kill. ++ const releasedAt = performance.now(); ++ confirmLinuxCommand(prepared); ++ assertFinalBoundary(releasedAt); ++ } + } catch { + preSignalProbe = { status: "unknown", method: "flock_exclusive_nonblocking", error: "linux_participation_unproven" }; + } +diff --git a/test/linux-attribution.test.mjs b/test/linux-attribution.test.mjs +index b784c81..46ac64e 100644 +--- a/test/linux-attribution.test.mjs ++++ b/test/linux-attribution.test.mjs +@@ -9,7 +9,7 @@ import { mkdtempSync, writeFileSync, openSync, closeSync, rmSync, readFileSync, + import { tmpdir } from 'node:os'; + import { join } from 'node:path'; + import { flockSync } from 'fs-ext-extra-prebuilt'; +-import { assertProcNamespace, filterCandidateDescriptors, fingerprintCommand, parseFlockRows, parseStartTicks, prepareLinuxParticipation, verifyLinuxParticipation } from '../dist/linux-attribution.js'; ++import { FINAL_BOUNDARY_MS, assertFinalBoundary, assertProcNamespace, confirmLinuxCommand, filterCandidateDescriptors, fingerprintCommand, parseFlockRows, parseStartTicks, prepareLinuxParticipation, verifyLinuxParticipation } from '../dist/linux-attribution.js'; + import { guardedProbeOnce } from '../dist/native-coordination.js'; + import { observeLockFile } from '../dist/lock.js'; + +@@ -161,10 +161,45 @@ test('Linux real FLOCK participates; open-but-unlocked descriptor does not', { s + try { assert.throws(() => flockSync(competing, 'exnb')); } finally { closeSync(competing); } + assert.equal(readFileSync(path, 'utf8'), ''); + assert.throws(() => verifyLinuxParticipation(path, { ...prepared, startTicks: '0' })); +- assert.throws(() => verifyLinuxParticipation(path, { ...prepared, commandFingerprint: '0'.repeat(64) }), /identity_changed/); ++ // Command evidence moved out of the guard: verify ignores it, confirm refuses it. ++ assert.equal(verifyLinuxParticipation(path, { ...prepared, commandFingerprint: '0'.repeat(64) }).descriptor, String(fd)); ++ assert.throws(() => confirmLinuxCommand({ ...prepared, commandFingerprint: '0'.repeat(64) }), /identity_changed/); ++ assert.throws(() => confirmLinuxCommand({ ...prepared, startTicks: '0' }), /identity_changed/); ++ confirmLinuxCommand(prepared); + assert.throws(() => verifyLinuxParticipation(path, { ...prepared, mountNamespace: 'mnt:[0]' }), /identity_changed/); + assert.throws(() => verifyLinuxParticipation(path, { ...prepared, descriptors: [] }), /identity_changed/); + flockSync(fd, 'un'); + assert.throws(() => verifyLinuxParticipation(path, prepareLinuxParticipation(process.pid, path)), /linux_participation_unproven/); + } finally { closeSync(fd); rmSync(root, { recursive: true }); } + }); ++ ++test('final boundary refuses a confirm step longer than the accepted gap', () => { ++ assertFinalBoundary(100, 100 + FINAL_BOUNDARY_MS); ++ assert.throws(() => assertFinalBoundary(100, 100 + FINAL_BOUNDARY_MS + 0.001), /linux_evidence_stale/); ++ assert.throws(() => assertFinalBoundary(Number.NaN, 0), /linux_evidence_stale/); ++}); ++ ++test('traced guarded reads exclude cmdline/exe and FD-target stat (manual trace, not exhaustive)', { skip: process.platform !== 'linux' }, (t) => { ++ const root = mkdtempSync(join(tmpdir(), 'codex-attribution-scope-')); ++ const path = join(root, 'thread.lock'); ++ writeFileSync(path, ''); ++ writeFileSync(join(root, '.coordination.lock'), ''); ++ const fd = openSync(path, 'r'); ++ try { ++ flockSync(fd, 'exnb'); ++ const prepared = prepareLinuxParticipation(process.pid, path); ++ const reads = []; ++ const result = guardedProbeOnce(path, observeLockFile(path), undefined, ++ () => verifyLinuxParticipation(path, prepared, (read) => reads.push(read))); ++ if (result.status === 'unknown' && /filesystem_unsupported/.test(result.error ?? '') && !process.env.CI) { ++ t.skip('actual filesystem mapping unsupported; positive evidence unverified'); return; ++ } ++ assert.equal(result.status, 'held', result.error); ++ assert.equal(result.participation.descriptor, String(fd)); ++ assert.ok(reads.length > 0); ++ for (const read of reads) { ++ assert.doesNotMatch(read.path, /\/(cmdline|exe|environ|maps|mem)$/, JSON.stringify(read)); ++ if (read.op === 'stat') assert.doesNotMatch(read.path, /\/fd\/\d+$/, JSON.stringify(read)); ++ } ++ } finally { closeSync(fd); rmSync(root, { recursive: true }); } ++}); diff --git a/experimental/126-guard-liveness/lib/common.mjs b/experimental/126-guard-liveness/lib/common.mjs new file mode 100644 index 0000000..083a112 --- /dev/null +++ b/experimental/126-guard-liveness/lib/common.mjs @@ -0,0 +1,103 @@ +// Shared lifecycle helpers for the #126/#127 research runners. +// +// Children are started with an exit promise created immediately, so cleanup +// never misses an early exit. Stopping a child only closes its stdin and waits +// with a bound. A timeout means the child's exit is unconfirmed: it is never +// escalated to a signal, and callers then preserve their scratch and fail. +import { Buffer } from 'node:buffer' +import { spawn } from 'node:child_process' +import fs from 'node:fs' +import path from 'node:path' +import process from 'node:process' +import readline from 'node:readline' +import { clearTimeout, setTimeout } from 'node:timers' + +// Select the repository's temporary root from AGENTS.md; other hosts use /tmp. +// RESEARCH_TMP_ROOT overrides +// the root only: a fresh private child is always created beneath it, and the +// supplied directory itself is never chmod-ed or written into directly. +export function defaultTmpRoot() { + if (process.platform === 'darwin') throw new Error('RESEARCH_TMP_ROOT is required on macOS; select the AGENTS.md temporary root') + return '/tmp' +} + +export function freshPrivateDir(prefix, root = process.env.RESEARCH_TMP_ROOT ?? defaultTmpRoot()) { + const real = fs.realpathSync(root) + const stat = fs.statSync(real) + if (!stat.isDirectory()) throw new Error(`temporary root is not a directory: ${root}`) + fs.accessSync(real, fs.constants.W_OK) + // mkdtemp creates the directory with mode 0700. + const dir = fs.mkdtempSync(path.join(real, `codex-${prefix}-`)) + if ((fs.statSync(dir).mode & 0o077) !== 0) throw new Error(`fresh directory is not private: ${dir}`) + return dir +} + +// Write a new file only; an existing file is an error, never overwritten. +export function writeNew(file, data, mode = 0o600) { + fs.writeFileSync(file, data, { flag: 'wx', mode }) +} + +// sun_path is 104 bytes on macOS and 108 on Linux, including the NUL. +export function assertSocketPath(socketPath) { + const limit = process.platform === 'darwin' ? 103 : 107 + const length = Buffer.byteLength(socketPath) + if (length > limit) throw new Error(`socket path is ${length} bytes, limit ${limit}: ${socketPath}`) +} + +// Start a child and wait for its first stdout line. Rejects on spawn error, +// stdout EOF, exit before the line, or the readiness bound. The returned +// handle is registered in `live` before any await, so cleanup sees it even +// when readiness fails. +export function startChild(command, args, options, live, readyMs = 5000) { + const child = spawn(command, args, { ...options, stdio: ['pipe', 'pipe', 'pipe'] }) + const handle = { child, pid: child.pid, stderr: '', stdinErrors: [] } + child.stdin.on('error', (error) => { handle.stdinErrors.push(error.message) }) + handle.exited = new Promise((resolve) => { + child.once('exit', (code, signal) => resolve({ exitCode: code, signal })) + child.once('error', (error) => resolve({ exitCode: null, signal: null, error: error.message })) + }) + live.add(handle) + // Keep early exits until cleanup verifies every started child's result. + child.stderr.setEncoding('utf8') + child.stderr.on('data', (chunk) => { handle.stderr += chunk }) + const lines = readline.createInterface({ input: child.stdout }) + handle.lines = lines + handle.ready = new Promise((resolve, reject) => { + let timer + const settle = (fn, value) => { clearTimeout(timer); fn(value) } + lines.once('line', (line) => settle(resolve, line)) + child.stdout.once('end', () => settle(reject, new Error(`stdout ended before ready: ${command} ${handle.stderr}`))) + handle.exited.then((result) => settle(reject, new Error(`exited before ready ${JSON.stringify(result)}: ${handle.stderr}`))) + timer = setTimeout(() => settle(reject, new Error(`ready timeout: ${command}`)), readyMs) + }) + return handle +} + +// Close stdin and wait at most budgetMs. `incomplete` means the exit was not +// observed; nothing here signals the child. +export async function stopChild(handle, budgetMs = 10000) { + try { handle.child.stdin.end() } catch { /* already closed */ } + let timer + let result + try { + result = await Promise.race([handle.exited, new Promise((resolve) => { + timer = setTimeout(() => resolve(null), budgetMs) + })]) + } finally { clearTimeout(timer) } + if (result === null) { + handle.child.unref() + handle.lines.close() + handle.child.stdin.destroy() + handle.child.stdout.destroy() + handle.child.stderr.destroy() + return { incomplete: true, pid: handle.pid, stdinErrors: handle.stdinErrors } + } + return { ...result, stdinErrors: handle.stdinErrors, + ...(handle.stdinErrors.length ? { error: 'stdin_io_failed' } : {}) } +} + +export async function stopAll(live, budgetMs = 10000) { + const outcomes = [] + for (const handle of [...live]) outcomes.push(await stopChild(handle, budgetMs)) + return outcomes +} diff --git a/experimental/126-guard-liveness/lib/common.test.mjs b/experimental/126-guard-liveness/lib/common.test.mjs new file mode 100644 index 0000000..8883cb5 --- /dev/null +++ b/experimental/126-guard-liveness/lib/common.test.mjs @@ -0,0 +1,23 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' +import process from 'node:process' +import { startChild, stopAll } from './common.mjs' + +test('cleanup retains a child that exited after readiness with nonzero status', async () => { + const children = new Set() + const child = startChild(process.execPath, ['-e', 'process.stdout.write("ready\\n"); process.exitCode=3'], {}, children) + assert.equal(await child.ready, 'ready') + assert.equal((await child.exited).exitCode, 3) + const outcomes = await stopAll(children, 1000) + assert.equal(outcomes.length, 1) + assert.equal(outcomes[0].exitCode, 3) +}) + +test('spawn failure remains available to cleanup', async () => { + const children = new Set() + const child = startChild('/nonexistent-codex-research-fixture', [], {}, children) + await assert.rejects(child.ready, /exited before ready/) + const outcomes = await stopAll(children, 1000) + assert.equal(outcomes.length, 1) + assert.match(outcomes[0].error, /ENOENT/) +}) diff --git a/experimental/126-guard-liveness/lib/f1-options.mjs b/experimental/126-guard-liveness/lib/f1-options.mjs new file mode 100644 index 0000000..2639d86 --- /dev/null +++ b/experimental/126-guard-liveness/lib/f1-options.mjs @@ -0,0 +1,6 @@ +// A misspelled fault must not silently run a successful nominal measurement. +export function parseF1Fault(value) { + if (value === undefined || value === '') return null + if (value === 'skip_round' || value === 'waiter_lifetime') return value + throw new Error('F1_FAULT must be empty, skip_round or waiter_lifetime') +} diff --git a/experimental/126-guard-liveness/lib/f1-options.test.mjs b/experimental/126-guard-liveness/lib/f1-options.test.mjs new file mode 100644 index 0000000..d62a16a --- /dev/null +++ b/experimental/126-guard-liveness/lib/f1-options.test.mjs @@ -0,0 +1,20 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' +import { parseF1Fault } from './f1-options.mjs' + +test('F1 selects nominal only when the fault setting is absent or empty', () => { + assert.equal(parseF1Fault(undefined), null) + assert.equal(parseF1Fault(''), null) +}) + +test('F1 accepts only the two exact fault names', () => { + assert.equal(parseF1Fault('skip_round'), 'skip_round') + assert.equal(parseF1Fault('waiter_lifetime'), 'waiter_lifetime') +}) + +test('F1 rejects unknown, misspelled, padded and non-string fault settings', () => { + for (const value of ['skip_roudn', 'waiter_lifetim', 'unknown', 'nominal', ' ', '\t', '\n', + 'skip_round ', ' skip_round', 'waiter_lifetime\n', 'SKIP_ROUND', 'false', '0', null, false, 0, {}]) { + assert.throws(() => parseF1Fault(value), /^Error: F1_FAULT must be empty, skip_round or waiter_lifetime$/) + } +}) diff --git a/experimental/126-guard-liveness/lib/isolated-observer.mjs b/experimental/126-guard-liveness/lib/isolated-observer.mjs new file mode 100644 index 0000000..68019f9 --- /dev/null +++ b/experimental/126-guard-liveness/lib/isolated-observer.mjs @@ -0,0 +1,264 @@ +// Separate-process observation for research inspection (#126 F2). +// +// Generic layer only: process lifecycle, bounded output and the stage-record +// protocol. Which stages must appear is decided by a caller's protocol layer +// (for F2: ../f2-linux-nondumpable/protocol.mjs). +// +// The parent never imports product modules. Product code runs in a child that +// first writes a `hello` handshake, then one JSON line per stage boundary. A +// synchronous call that stalls blocks only the child's event loop; the parent +// keeps its own timers and returns `incomplete` with the open stage. +// +// No cancellation guarantee. The parent sends no signal. On an exceeded bound +// it closes the child's stdin, destroys the pipes, unrefs the child and keeps +// a `lateExit` promise for that instance. The child has no lifetime timer: it +// checks its deadline and stdin EOF only between stages, then stops scheduling +// stages and lets pending work drain instead of calling process.exit. A child +// stuck in a syscall, or awaiting work that never settles, can outlive the +// runner. +import { Buffer } from 'node:buffer' +import { spawn } from 'node:child_process' +import fs from 'node:fs' +import { performance } from 'node:perf_hooks' +import process from 'node:process' +import { clearTimeout, setImmediate, setTimeout } from 'node:timers' + +const DEFAULT_STDOUT_BYTES = 1 << 20 +const DEFAULT_STDERR_BYTES = 64 << 10 + +// Linux start ticks of a running process, as a decimal string. Elsewhere the +// instance is recorded as unavailable, not assumed. +export function processInstance(pid) { + if (process.platform !== 'linux') return { pid, startTicks: null, reason: 'not_linux' } + try { + const text = fs.readFileSync(`/proc/${pid}/stat`, 'utf8') + const fields = text.slice(text.lastIndexOf(') ') + 2).trim().split(/\s+/) + if (!text.startsWith(`${pid} (`) || fields[0] === 'Z' || fields[0] === 'X') return { pid, startTicks: null, reason: 'not_running' } + return /^\d+$/.test(fields[19] ?? '') ? { pid, startTicks: fields[19] } : { pid, startTicks: null, reason: 'malformed' } + } catch (error) { + return { pid, startTicks: null, reason: error.code ?? String(error) } + } +} + +// Splits a byte stream into complete lines under a total byte limit. Lines are +// decoded only when complete, so a multibyte character split across chunks is +// preserved. Bytes past the limit are counted and dropped, never buffered. +export class LineCollector { + constructor(limit) { + this.limit = limit + this.bytes = 0 + this.pending = [] + this.pendingBytes = 0 + this.lines = [] + this.overflow = false + } + + push(chunk) { + const room = Math.max(0, this.limit - this.bytes) + this.bytes += chunk.length + if (chunk.length > room) this.overflow = true + let part = chunk.subarray(0, room) + for (let at = part.indexOf(10); at >= 0; at = part.indexOf(10)) { + this.pending.push(part.subarray(0, at)) + this.lines.push(Buffer.concat(this.pending).toString('utf8')) + this.pending = [] + this.pendingBytes = 0 + part = part.subarray(at + 1) + } + if (part.length) { this.pending.push(part); this.pendingBytes += part.length } + } + + text() { + return Buffer.concat([...this.lines.map((line) => Buffer.from(line + '\n')), ...this.pending]).toString('utf8') + } + + faults(name) { + const out = [] + if (this.overflow) out.push(`${name}_limit:${this.bytes}`) + if (this.pendingBytes) out.push(`${name}_unterminated_line`) + return out + } +} + +// Pure stage-stream validation. The first record must be the handshake; each +// `end` carries exactly one of `result`/`error`; at most one terminal record +// (`final` or `stop`), and nothing after it. +export function summarizeStages(lines) { + const stages = [] + const begun = [] + const faults = [] + let open = null + let terminal = null + let hello = null + if (lines.length === 0) faults.push('missing_hello') + lines.forEach((line, index) => { + let record + try { record = JSON.parse(line) } catch { faults.push(`malformed_line:${index}`); return } + if (record === null || typeof record !== 'object' || Array.isArray(record)) { faults.push(`malformed_line:${index}`); return } + if (index === 0) { + if (record.hello === true && Number.isInteger(record.pid)) { hello = record; return } + faults.push('missing_hello') + } else if (Object.hasOwn(record, 'hello')) { + faults.push(`unexpected_hello:${index}`); return + } + if (terminal) { faults.push(`record_after_terminal:${index}`); return } + if (record.final === true || typeof record.stop === 'string') { + if (open) faults.push(`terminal_inside_stage:${open}`) + terminal = record + if (record.stop) faults.push(`stopped:${record.stop}`) + } else if (record.phase === 'begin' && typeof record.stage === 'string') { + if (open) faults.push(`nested_stage:${record.stage}`) + open = record.stage + begun.push(record.stage) + } else if (record.phase === 'end' && typeof record.stage === 'string') { + const hasResult = Object.hasOwn(record, 'result') + const hasError = Object.hasOwn(record, 'error') + if (open !== record.stage) faults.push(`unmatched_end:${record.stage}`) + else if (hasResult === hasError) faults.push(`end_outcome_invalid:${record.stage}`) + else stages.push(hasResult ? { stage: record.stage, result: record.result } : { stage: record.stage, error: record.error }) + open = null + } else { + faults.push(`unknown_record:${index}`) + } + }) + return { hello, stages, begun, openStage: open, final: terminal?.final === true ? terminal : null, + stop: terminal?.stop ?? null, faults } +} + +// Two bounds: `startupBudgetMs` until the handshake line, then `budgetMs` for +// the stages. The second bound starts at the handshake, so child startup +// latency does not consume the stage budget. +export function runIsolated(command, args, { startupBudgetMs, budgetMs, env, + maxStdoutBytes = DEFAULT_STDOUT_BYTES, maxStderrBytes = DEFAULT_STDERR_BYTES } = {}) { + for (const [name, value] of Object.entries({ startupBudgetMs, budgetMs })) { + if (!Number.isInteger(value) || value <= 0) throw new Error(`${name} must be a positive integer`) + } + const child = spawn(command, args, { env, stdio: ['pipe', 'pipe', 'pipe'] }) + const instance = Number.isInteger(child.pid) ? processInstance(child.pid) : null + const stdout = new LineCollector(maxStdoutBytes) + const stderr = new LineCollector(maxStderrBytes) + const faults = [] + child.stdin.on('error', (error) => { faults.push(`stdin:${error.code ?? error.message}`) }) + const startedAt = performance.now() + let helloAt = null + let onHello + const sawHello = new Promise((resolve) => { onHello = resolve }) + child.stdout.on('data', (chunk) => { + stdout.push(chunk) + if (helloAt === null && stdout.lines.length > 0) { helloAt = performance.now(); onHello('hello') } + }) + child.stderr.on('data', (chunk) => { stderr.push(chunk) }) + // A descendant may inherit stderr after the direct child exits and closes + // stdout. Do not certify collection until both captured streams close. + const closed = Promise.all([child.stdout, child.stderr].map((stream) => + new Promise((resolve) => { stream.once('close', resolve) }))) + const exited = new Promise((resolve) => { + child.once('exit', (code, signal) => resolve({ code, signal })) + child.once('error', (error) => resolve({ code: null, signal: null, error: error.message })) + }) + const done = exited.then(async (result) => { await closed; return result }) + let timer + const bound = (ms, phase) => new Promise((resolve) => { timer = setTimeout(() => resolve({ timeout: phase }), ms) }) + const finish = (result) => { + clearTimeout(timer) + const summary = summarizeStages(stdout.lines) + const all = [...faults, ...stdout.faults('stdout'), ...stderr.faults('stderr'), ...summary.faults] + try { child.stdin.end() } catch { /* already closed */ } + child.stdin.destroy() + child.stdout.destroy() + child.stderr.destroy() + const base = { instance, elapsedMs: performance.now() - startedAt, + helloMs: helloAt === null ? null : helloAt - startedAt, ...summary, stderr: stderr.text() } + if (result.timeout) { + child.unref() + return { ...base, incomplete: true, incompletePhase: result.timeout, faults: all, lateExit: exited } + } + if (result.error) all.push(`spawn:${result.error}`) + if (result.code !== 0 || result.signal) all.push(`exit:${result.code}:${result.signal}`) + if (!summary.final) all.push('no_final_record') + if (summary.openStage) all.push(`stage_not_ended:${summary.openStage}`) + return { ...base, incomplete: false, exit: result, faults: all } + } + return Promise.race([done, sawHello, bound(startupBudgetMs, 'startup')]).then((first) => { + if (first !== 'hello') return finish(first) + clearTimeout(timer) + return Promise.race([done, bound(budgetMs, 'stages')]).then(finish) + }) +} + +// Bounded look at late exits, one status per instance. +export async function lateExitStatus(entries, budgetMs) { + let timer + const expired = new Promise((resolve) => { timer = setTimeout(() => resolve(null), budgetMs) }) + const statuses = await Promise.all(entries.map(async (entry) => { + const exit = await Promise.race([entry.lateExit, expired]) + return { ...entry.label, instance: entry.instance, ...(exit ? { status: 'exited', exit } : { status: 'unconfirmed' }) } + })) + clearTimeout(timer) + return statuses +} + +export class StopStages extends Error { + constructor(reason) { super(`stopped:${reason}`); this.reason = reason } +} + +// Child side. Writes the handshake at once. Each stage writes `begin` before it +// runs and `end` after. Deadline and stdin EOF are checked only at stage +// boundaries; a stop writes a `stop` record, sets the exit code and throws +// StopStages. The caller should catch it and return so pending handles drain. +export function childStages({ lifetimeMs }) { + const deadline = performance.now() + lifetimeMs + let parentGone = false + process.stdin.on('end', () => { parentGone = true }) + process.stdin.on('error', () => { parentGone = true }) + process.stdin.resume() + // Synchronous writes; a closed parent pipe is the same as stdin EOF. + const write = (record) => { + if (parentGone) return + const bytes = Buffer.from(JSON.stringify(record) + '\n') + let offset = 0 + while (offset < bytes.length) { + try { + offset += fs.writeSync(1, bytes, offset) + } catch (error) { + if (error.code === 'EAGAIN' && performance.now() <= deadline) continue + parentGone = true + return + } + } + } + const release = () => { process.stdin.pause(); process.stdin.destroy() } + // Yield one macrotask so EOF seen during a synchronous stall is processed. + const boundary = async () => { + await new Promise((resolve) => { setImmediate(resolve) }) + const reason = parentGone ? 'parent_gone' : performance.now() > deadline ? 'lifetime' : null + if (reason) { + write({ stop: reason }) + release() + process.exitCode = reason === 'parent_gone' ? 4 : 3 + throw new StopStages(reason) + } + } + write({ hello: true, pid: process.pid }) + return { + async stage(name, fn) { + await boundary() + write({ stage: name, phase: 'begin', at: performance.now() }) + let record + try { + const result = await fn() + record = { stage: name, phase: 'end', at: performance.now(), result: result === undefined ? null : result } + } catch (error) { + record = { stage: name, phase: 'end', at: performance.now(), + error: { code: error?.code ?? null, name: error?.name ?? null, message: String(error?.message ?? error) } } + } + write(record) + return record + }, + async finish(extra = {}) { + await boundary() + write({ final: true, ...extra }) + release() + }, + } +} diff --git a/experimental/126-guard-liveness/lib/isolated-observer.test.mjs b/experimental/126-guard-liveness/lib/isolated-observer.test.mjs new file mode 100644 index 0000000..09f8102 --- /dev/null +++ b/experimental/126-guard-liveness/lib/isolated-observer.test.mjs @@ -0,0 +1,183 @@ +import assert from 'node:assert/strict' +import { Buffer } from 'node:buffer' +import { test } from 'node:test' +import process from 'node:process' +import { performance } from 'node:perf_hooks' +import { setTimeout } from 'node:timers' +import { URL } from 'node:url' +import { LineCollector, lateExitStatus, runIsolated, summarizeStages } from './isolated-observer.mjs' + +const LIB = new URL('./isolated-observer.mjs', import.meta.url).href +// Child bodies import the generic layer and stop cleanly on StopStages. +const child = (body) => [process.execPath, ['--input-type=module', '-e', + `import { childStages, StopStages } from '${LIB}'\ntry {\n${body}\n} catch (error) { if (!(error instanceof StopStages)) throw error }`]] +const opts = { startupBudgetMs: 10000, budgetMs: 5000 } +const hello = '{"hello":true,"pid":1}' +const stall = (ms) => `Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, ${ms})` + +test('stage stream requires the handshake, paired stages and one outcome per end', () => { + const ok = summarizeStages([hello, '{"stage":"a","phase":"begin"}', '{"stage":"a","phase":"end","result":1}', '{"final":true}']) + assert.deepEqual(ok.faults, []) + assert.deepEqual(ok.stages, [{ stage: 'a', result: 1 }]) + assert.deepEqual(summarizeStages(['{"final":true}']).faults, ['missing_hello']) + assert.deepEqual(summarizeStages([]).faults, ['missing_hello']) + assert.deepEqual(summarizeStages([hello, '{"stage":"a","phase":"begin"}', '{"stage":"a","phase":"end","result":1,"error":{}}']).faults, + ['end_outcome_invalid:a']) + assert.deepEqual(summarizeStages([hello, '{"stage":"a","phase":"begin"}', '{"stage":"a","phase":"end"}']).faults, ['end_outcome_invalid:a']) + const bad = summarizeStages([hello, 'nope', '{"stage":"a","phase":"begin"}', '{"stage":"b","phase":"end","result":1}', '{"stage":"c","phase":"begin"}']) + assert.deepEqual(bad.faults, ['malformed_line:1', 'unmatched_end:b']) + assert.equal(bad.openStage, 'c') + assert.deepEqual(summarizeStages([hello, '{"final":true}', '{"final":true}']).faults, ['record_after_terminal:2']) + assert.deepEqual(summarizeStages([hello, '{"stage":"x","phase":"begin"}', '{"final":true}']).faults, ['terminal_inside_stage:x']) + assert.deepEqual(summarizeStages([hello, '42', '[1]', '{"what":1}', hello]).faults, + ['malformed_line:1', 'malformed_line:2', 'unknown_record:3', 'unexpected_hello:4']) + assert.deepEqual(summarizeStages([hello, '{"stop":"lifetime"}']).faults, ['stopped:lifetime']) +}) + +test('line collector bounds bytes, keeps split multibyte characters and flags partial lines', () => { + const c = new LineCollector(64) + const text = Buffer.from('한글 line\n') + c.push(text.subarray(0, 1)) + c.push(text.subarray(1, 4)) + c.push(text.subarray(4)) + assert.deepEqual(c.lines, ['한글 line']) + assert.deepEqual(c.faults('stdout'), []) + c.push(Buffer.alloc(200, 0x61)) + assert.equal(c.lines.length, 1) + assert.ok(c.pendingBytes <= 64) + assert.deepEqual(c.faults('stdout'), [`stdout_limit:${text.length + 200}`, 'stdout_unterminated_line']) +}) + +test('nominal child completes every stage', async () => { + const [cmd, args] = child(`const c = childStages({ lifetimeMs: 5000 }) +await c.stage('one', () => 1) +await c.stage('two', () => { throw Object.assign(new Error('denied'), { code: 'EACCES' }) }) +await c.finish({ done: true })`) + const r = await runIsolated(cmd, args, opts) + assert.equal(r.incomplete, false) + assert.deepEqual(r.faults, []) + assert.deepEqual(r.stages.map((s) => s.stage), ['one', 'two']) + assert.equal(r.stages[1].error.code, 'EACCES') + assert.equal(r.final.done, true) + assert.equal(r.hello.pid, r.instance.pid) +}) + +test('a synchronous stall after the handshake is bounded by the stage budget', async () => { + // The stage budget starts at the handshake, so startup latency is excluded. + const [cmd, args] = child(`const c = childStages({ lifetimeMs: 30000 }) +await c.stage('fast', () => 1) +await c.stage('stall', () => { ${stall(3000)}; return 'late' }) +await c.finish()`) + const r = await runIsolated(cmd, args, { startupBudgetMs: 10000, budgetMs: 300 }) + assert.equal(r.incomplete, true) + assert.equal(r.incompletePhase, 'stages') + assert.equal(r.openStage, 'stall') + assert.deepEqual(r.stages.map((s) => s.stage), ['fast']) + assert.ok(r.elapsedMs - r.helloMs < 2000, `stage wait ${r.elapsedMs - r.helloMs} ms`) + // stdin was closed: the child stops at its next boundary with code 4, no signal. + const [late] = await lateExitStatus([{ label: { id: 'stall' }, instance: r.instance, lateExit: r.lateExit }], 10000) + assert.deepEqual({ status: late.status, exit: late.exit }, { status: 'exited', exit: { code: 4, signal: null } }) +}) + +test('a stall before the handshake is a startup timeout', async () => { + const [cmd, args] = child(`${stall(2000)} +const c = childStages({ lifetimeMs: 5000 }) +await c.finish()`) + const r = await runIsolated(cmd, args, { startupBudgetMs: 300, budgetMs: 5000 }) + assert.equal(r.incomplete, true) + assert.equal(r.incompletePhase, 'startup') + assert.ok(r.faults.includes('missing_hello')) + const [late] = await lateExitStatus([{ label: {}, instance: r.instance, lateExit: r.lateExit }], 10000) + assert.equal(late.status, 'exited') +}) + +test('late exit status stays per instance and reports unconfirmed exits', async () => { + const never = new Promise(() => {}) + const statuses = await lateExitStatus([ + { label: { id: 'a' }, instance: { pid: 1 }, lateExit: Promise.resolve({ code: 0, signal: null }) }, + { label: { id: 'b' }, instance: { pid: 2 }, lateExit: never }, + ], 50) + assert.deepEqual(statuses.map((s) => [s.id, s.instance.pid, s.status]), [['a', 1, 'exited'], ['b', 2, 'unconfirmed']]) +}) + +test('long output without a newline is bounded and faulted', async () => { + const [cmd, args] = child(`const c = childStages({ lifetimeMs: 5000 }) +await c.stage('spam', () => { process.stdout.write('x'.repeat(1 << 21)); return 1 }) +await c.finish()`) + const started = performance.now() + const r = await runIsolated(cmd, args, { ...opts, maxStdoutBytes: 64 << 10 }) + assert.ok(performance.now() - started < 10000) + assert.ok(r.faults.some((f) => f.startsWith('stdout_limit:')), r.faults.join(',')) + assert.ok(r.faults.includes('no_final_record') || r.faults.some((f) => f.startsWith('malformed_line')), r.faults.join(',')) +}) + +test('stderr is bounded by bytes', async () => { + const [cmd, args] = child(`const c = childStages({ lifetimeMs: 5000 }) +process.stderr.write('한'.repeat(40000)) +await c.finish()`) + const r = await runIsolated(cmd, args, { ...opts, maxStderrBytes: 1024 }) + assert.ok(r.faults.some((f) => f.startsWith('stderr_limit:')), r.faults.join(',')) + // A cut inside a character decodes to one U+FFFD (3 bytes). + assert.ok(Buffer.byteLength(r.stderr) <= 1024 + 3, String(Buffer.byteLength(r.stderr))) +}) + +test('collection waits for finite inherited stderr after the direct child exits', async () => { + const grandchild = `process.stderr.on('error', () => {}); setTimeout(() => process.stderr.write('late stderr'), 200)` + const [cmd, args] = child(`const c = childStages({ lifetimeMs: 5000 }) +const { spawn } = await import('node:child_process') +const descendant = spawn(process.execPath, ['-e', ${JSON.stringify(grandchild)}], { stdio: ['ignore', 'ignore', 'inherit'] }) +descendant.unref() +await c.finish()`) + const r = await runIsolated(cmd, args, opts) + assert.equal(r.incomplete, false) + assert.equal(r.stderr, 'late stderr') + assert.ok(r.faults.includes('stderr_unterminated_line')) +}) + +test('inherited stderr still open at the stage bound is incomplete', async () => { + const [cmd, args] = child(`const c = childStages({ lifetimeMs: 5000 }) +const { spawn } = await import('node:child_process') +const descendant = spawn(process.execPath, ['-e', 'setTimeout(() => {}, 700)'], { stdio: ['ignore', 'ignore', 'inherit'] }) +descendant.unref() +await c.finish()`) + const r = await runIsolated(cmd, args, { startupBudgetMs: 10000, budgetMs: 100 }) + assert.equal(r.incomplete, true) + assert.equal(r.incompletePhase, 'stages') + const [late] = await lateExitStatus([{ label: {}, instance: r.instance, lateExit: r.lateExit }], 10000) + assert.equal(late.status, 'exited') + // The finite descendant exits itself; do not leave a live fixture behind. + await new Promise((resolve) => setTimeout(resolve, 1000)) +}) + +test('lifetime is checked at boundaries only and never interrupts an async stage', async () => { + const [cmd, args] = child(`const c = childStages({ lifetimeMs: 100 }) +await c.stage('async', () => new Promise((resolve) => setTimeout(() => resolve('done'), 400))) +await c.stage('never', () => 1) +await c.finish()`) + const r = await runIsolated(cmd, args, opts) + assert.deepEqual(r.exit, { code: 3, signal: null }) + assert.deepEqual(r.stages, [{ stage: 'async', result: 'done' }]) + assert.equal(r.stop, 'lifetime') + assert.ok(r.faults.includes('stopped:lifetime')) + assert.ok(r.faults.includes('no_final_record')) +}) + +test('early exit, malformed output and nonzero exit are faults', async () => { + let [cmd, args] = child(`const c = childStages({ lifetimeMs: 5000 }) +process.stdout.write(JSON.stringify({ stage: 'x', phase: 'begin' }) + '\\n'); process.exit(0)`) + let r = await runIsolated(cmd, args, opts) + assert.equal(r.incomplete, false) + assert.ok(r.faults.includes('no_final_record')) + assert.ok(r.faults.includes('stage_not_ended:x')) + ;[cmd, args] = child(`const c = childStages({ lifetimeMs: 5000 }) +await c.stage('a', () => 1); process.stdout.write('{broken\\n'); await c.finish(); process.exitCode = 2`) + r = await runIsolated(cmd, args, opts) + assert.ok(r.faults.some((f) => f.startsWith('malformed_line'))) + assert.ok(r.faults.includes('exit:2:null')) +}) + +test('spawn failure is a fault, not a result', async () => { + const r = await runIsolated('/nonexistent-codex-research-observer', [], opts) + assert.equal(r.incomplete, false) + assert.ok(r.faults.some((f) => f.startsWith('spawn:'))) +}) diff --git a/experimental/126-guard-liveness/m2-macos-fwaslocked/actor.mjs b/experimental/126-guard-liveness/m2-macos-fwaslocked/actor.mjs new file mode 100755 index 0000000..8d0acb6 --- /dev/null +++ b/experimental/126-guard-liveness/m2-macos-fwaslocked/actor.mjs @@ -0,0 +1,54 @@ +#!/usr/bin/env node +// Descriptor-lifetime actor for the M2 flag matrix (#127 research). Roles: +// holder open + flock(LOCK_EX|LOCK_NB) must succeed, hold +// holder_multi as holder, plus a second plain descriptor on the same file +// opener open only, never calls flock +// failed open + flock(LOCK_EX|LOCK_NB) must fail with EWOULDBLOCK, keep FD +// former open + flock(LOCK_EX|LOCK_NB) must succeed, then LOCK_UN, keep FD +// Fault roles (runner fault checks only): +// early_exit exits 1 before reporting ready +// ignore_eof reports ready but ignores stdin EOF; ends at its lifetime +// Prints `ready` once its state is established. stdin EOF closes the FDs and +// exits 0; FIXTURE_LIFETIME_MS ends the process with code 3. +import fs from 'node:fs' +import { createRequire } from 'node:module' +import process from 'node:process' +import { setTimeout } from 'node:timers' + +const require = createRequire(process.env.FIXTURE_REQUIRE_FROM) +const { flockSync } = require('fs-ext-extra-prebuilt') +const role = process.env.FIXTURE_ROLE +if (role === 'early_exit') process.exit(1) +const lockPath = process.env.FIXTURE_LOCK_PATH +const fds = [fs.openSync(lockPath, 'r+')] + +function tryLock(fd) { + try { + flockSync(fd, 'exnb') + return true + } catch (error) { + if (error.code !== 'EAGAIN' && error.code !== 'EWOULDBLOCK') throw error + return false + } +} + +if (role === 'holder' || role === 'holder_multi' || role === 'former') { + if (!tryLock(fds[0])) throw new Error(`${role}: expected to acquire`) + if (role === 'former') flockSync(fds[0], 'un') + if (role === 'holder_multi') fds.push(fs.openSync(lockPath, 'r')) +} else if (role === 'failed') { + if (tryLock(fds[0])) throw new Error('failed: unexpectedly acquired') +} else if (role !== 'opener' && role !== 'ignore_eof') { + throw new Error(`unknown role ${role}`) +} +const close = () => { for (const fd of fds) fs.closeSync(fd) } +process.stdout.write('ready\n') +const lifetime = setTimeout(() => { close(); process.exit(3) }, Number(process.env.FIXTURE_LIFETIME_MS ?? 60000)) +// An ignore_eof actor must stay alive after EOF until its lifetime. +if (role !== 'ignore_eof') lifetime.unref() +process.stdin.resume() +process.stdin.on('end', () => { + if (role === 'ignore_eof') return + close() + process.exit(0) +}) diff --git a/experimental/126-guard-liveness/m2-macos-fwaslocked/fg-scan.mjs b/experimental/126-guard-liveness/m2-macos-fwaslocked/fg-scan.mjs new file mode 100644 index 0000000..f4a9547 --- /dev/null +++ b/experimental/126-guard-liveness/m2-macos-fwaslocked/fg-scan.mjs @@ -0,0 +1,90 @@ +// `lsof +fg` file-flag scan for the M2 refusal-side filter (#127 research). +// +// Only FWASLOCKED (0x4000) in the first G value is interpreted, and only as +// refusal-side evidence. Per PID, every descriptor on the scanned path is +// aggregated: +// - scan failed, signalled, wrote stderr, or its output is malformed +// (a field outside a process/file set, a file set without G, an +// unparseable G) -> unknown +// - PID absent from the scan -> unknown +// - every descriptor of the PID lacks the bit -> reject +// - at least one descriptor has the bit -> not_excluded +// A present bit never proves a holder: it is sticky after LOCK_UN. +// Detectable truncation is invalid. A clean omission of whole process/file +// records cannot be detected from this output and is not ruled out here. +import { spawnSync } from 'node:child_process' + +export const FWASLOCKED = 0x4000 + +// spawnSync's timeout makes Node send SIGTERM to lsof. The scan records +// `signal` and `error` so such a run is visible and counted as unknown. +export function fgScan(lockPath, lsof = '/usr/sbin/lsof') { + const result = spawnSync(lsof, ['-nP', '+fg', '-F0pfG', '--', lockPath], { + env: { LC_ALL: 'C', LANG: 'C', PATH: '/usr/bin:/bin:/usr/sbin:/sbin' }, + timeout: 5000, maxBuffer: 1024 * 1024, + }) + return interpretScan({ + status: result.status, signal: result.signal, error: result.error?.message ?? null, + stderr: result.stderr?.toString() ?? '', stdout: result.stdout?.toString() ?? '', + }) +} + +// Pure, so it can be checked against fixed outputs. +export function interpretScan({ status, signal = null, error = null, stderr = '', stdout = '' }) { + const scan = { status, signal, error, stderr, raw: stdout, valid: true, problems: [], files: [] } + if (error) scan.problems.push(`error:${error}`) + if (signal) scan.problems.push(`signal:${signal}`) + if (status !== 0) scan.problems.push(`status:${status}`) + if (stderr) scan.problems.push('stderr') + if (stdout && !stdout.endsWith('\0\n')) scan.problems.push('unterminated_record') + let pid = null + let file = null + for (const record of stdout.split('\n')) { + if (!record) continue + if (!record.endsWith('\0')) scan.problems.push('unterminated_field') + for (const field of record.split('\0')) { + if (!field) continue + const tag = field[0] + const value = field.slice(1) + if (tag === 'p') { + if (file && file.flags === null) scan.problems.push(`no_flags:${file.pid}:${file.fd}`) + file = null + pid = /^[1-9]\d*$/.test(value) && Number.isSafeInteger(Number(value)) ? Number(value) : null + if (pid === null) scan.problems.push(`bad_pid:${value}`) + } else if (tag === 'f') { + if (file && file.flags === null) scan.problems.push(`no_flags:${file.pid}:${file.fd}`) + if (pid === null) scan.problems.push('file_without_process') + if (!/^\d+$/.test(value) || !Number.isSafeInteger(Number(value))) scan.problems.push(`bad_fd:${value}`) + file = { pid, fd: value, flags: null, wasLocked: null } + scan.files.push(file) + } else if (tag === 'G') { + if (!file) { scan.problems.push('flags_without_file'); continue } + if (file.flags !== null) scan.problems.push(`duplicate_flags:${file.pid}:${file.fd}`) + file.flags = value + const match = /^0x([0-9a-f]+)(;0x[0-9a-f]+)?$/i.exec(value) + if (match) file.wasLocked = (BigInt(`0x${match[1]}`) & BigInt(FWASLOCKED)) !== 0n + else scan.problems.push(`bad_flags:${value}`) + } else { + scan.problems.push(`unexpected_field:${tag}`) + } + } + } + if (file && file.flags === null) scan.problems.push(`no_flags:${file.pid}:${file.fd}`) + scan.valid = scan.problems.length === 0 + return scan +} + +export function filterDecision(scan, pid) { + if (!scan.valid) return { decision: 'unknown', reason: 'scan_invalid', descriptors: [] } + const descriptors = scan.files.filter((f) => f.pid === pid) + if (!descriptors.length) return { decision: 'unknown', reason: 'pid_absent', descriptors } + if (descriptors.some((f) => f.wasLocked === null)) return { decision: 'unknown', reason: 'flags_unreadable', descriptors } + return descriptors.some((f) => f.wasLocked) + ? { decision: 'not_excluded', reason: 'bit_present', descriptors } + : { decision: 'reject', reason: 'bit_absent_on_all', descriptors } +} + +// Apply a modelled omission of one PID to a scan, as the product view sees it. +export function withoutPid(scan, pid) { + return { ...scan, files: scan.files.filter((f) => f.pid !== pid) } +} diff --git a/experimental/126-guard-liveness/m2-macos-fwaslocked/fg-scan.test.mjs b/experimental/126-guard-liveness/m2-macos-fwaslocked/fg-scan.test.mjs new file mode 100644 index 0000000..9187977 --- /dev/null +++ b/experimental/126-guard-liveness/m2-macos-fwaslocked/fg-scan.test.mjs @@ -0,0 +1,71 @@ +// Deterministic checks of the M2 parser and refusal-side decision. +// Run: node --test experimental/126-guard-liveness/m2-macos-fwaslocked/fg-scan.test.mjs +import assert from 'node:assert/strict' +import { test } from 'node:test' +import { filterDecision, interpretScan, withoutPid } from './fg-scan.mjs' + +const rec = (...fields) => fields.map((f) => `${f}\0`).join('') + '\n' +const ok = (stdout) => interpretScan({ status: 0, stdout }) + +test('multi-FD holder is not excluded; mixed nonholder is rejected', () => { + const scan = ok(rec('p100') + rec('f3', 'G0x3;0x2') + rec('f4', 'G0x4003;0x2') + rec('p200') + rec('f5', 'G0x3;0x2') + rec('f6', 'G0x1;0x0')) + assert.equal(scan.valid, true) + assert.equal(filterDecision(scan, 100).decision, 'not_excluded') + assert.equal(filterDecision(scan, 100).descriptors.length, 2) + assert.equal(filterDecision(scan, 200).decision, 'reject') + assert.equal(filterDecision(scan, 200).descriptors.length, 2) +}) + +test('the first descriptor alone does not decide', () => { + const scan = ok(rec('p100') + rec('f3', 'G0x3;0x2') + rec('f9', 'G0x4002;0x2')) + assert.equal(filterDecision(scan, 100).decision, 'not_excluded') +}) + +test('absent PID is unknown, also after a modelled omission', () => { + const scan = ok(rec('p100') + rec('f3', 'G0x4003;0x2')) + assert.equal(filterDecision(scan, 300).reason, 'pid_absent') + assert.equal(filterDecision(withoutPid(scan, 100), 100).decision, 'unknown') +}) + +test('failed, signalled, errored or noisy scans are unknown', () => { + const stdout = rec('p100') + rec('f3', 'G0x3;0x2') + for (const input of [ + { status: 1, stdout }, + { status: null, signal: 'SIGTERM', stdout }, + { status: null, error: 'spawnSync ETIMEDOUT', stdout }, + { status: 0, stderr: 'lsof: WARNING', stdout }, + ]) { + const scan = interpretScan(input) + assert.equal(scan.valid, false) + assert.equal(filterDecision(scan, 100).decision, 'unknown') + } +}) + +test('partial or malformed output is unknown for every PID', () => { + for (const stdout of [ + rec('p100') + rec('f3'), + rec('p100') + rec('f3', 'Gzz'), + rec('f3', 'G0x3;0x2'), + rec('p100') + rec('G0x3;0x2'), + rec('px') + rec('f3', 'G0x3;0x2'), + rec('p100') + rec('f3', 'G0x3;0x2', 'n/x'), + rec('p100') + rec('f3', 'G0x3;0x2') + rec('p200') + rec('f4'), + rec('p100') + 'f3\0G0x3;0x2', + rec('p100') + 'f3\0G0x3;0x2\0', + rec('p100') + 'f3\0G0x3;0x2\n', + rec('p100') + rec('f3', 'G0x4003;0x2', 'G0x3;0x2'), + rec('p100') + rec('f', 'G0x3;0x2'), + rec('p9007199254740993') + rec('f3', 'G0x3;0x2'), + rec('p0') + rec('f3', 'G0x3;0x2'), + ]) { + const scan = ok(stdout) + assert.equal(scan.valid, false, JSON.stringify(stdout)) + assert.equal(filterDecision(scan, 100).decision, 'unknown') + } +}) + +test('empty successful scan has no positive outcome', () => { + const scan = ok('') + assert.equal(scan.valid, true) + assert.equal(filterDecision(scan, 100).decision, 'unknown') +}) diff --git a/experimental/126-guard-liveness/m2-macos-fwaslocked/filter-measurement.mjs b/experimental/126-guard-liveness/m2-macos-fwaslocked/filter-measurement.mjs new file mode 100644 index 0000000..7a1441e --- /dev/null +++ b/experimental/126-guard-liveness/m2-macos-fwaslocked/filter-measurement.mjs @@ -0,0 +1,14 @@ +// A valid scan with no selected owner (or unknown owner evidence) is not a +// completed filter measurement. This is separate from proving lock ownership. +export function evaluateFilterMeasurement(scans) { + const records = Array.isArray(scans) ? scans : [] + const ownerDecisions = records.map((scan) => scan?.owner?.decision ?? 'no_owner') + const filterMeasurementValid = records.length > 0 && records.every((scan) => scan?.valid === true && + (scan.owner?.decision === 'reject' || scan.owner?.decision === 'not_excluded')) + return { + ownerDecisions, + filterMeasurementValid, + filterWouldRefuse: filterMeasurementValid ? ownerDecisions.includes('reject') : null, + failure: filterMeasurementValid ? null : 'filter_measurement_unavailable', + } +} diff --git a/experimental/126-guard-liveness/m2-macos-fwaslocked/filter-measurement.test.mjs b/experimental/126-guard-liveness/m2-macos-fwaslocked/filter-measurement.test.mjs new file mode 100644 index 0000000..14f5098 --- /dev/null +++ b/experimental/126-guard-liveness/m2-macos-fwaslocked/filter-measurement.test.mjs @@ -0,0 +1,36 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' +import { evaluateFilterMeasurement } from './filter-measurement.mjs' + +const scan = (decision) => ({ valid: true, owner: { decision } }) + +test('missing owner and unknown evidence fail a completed part-2 scenario', () => { + for (const scans of [[], [{ valid: true, owner: null }], [{ valid: true }], [scan('unknown')], + [scan('not_excluded'), scan('unknown')], [scan('reject'), { valid: true, owner: null }]]) { + const measurement = evaluateFilterMeasurement(scans) + assert.equal(measurement.filterMeasurementValid, false) + assert.equal(measurement.filterWouldRefuse, null) + assert.equal(measurement.failure, 'filter_measurement_unavailable') + } +}) + +test('invalid scans and unexpected decisions cannot certify a filter measurement', () => { + for (const scans of [null, {}, [null], [{ valid: false, owner: { decision: 'reject' } }], + [{ valid: 'true', owner: { decision: 'reject' } }], [scan('holder')], [scan('')]]) { + const measurement = evaluateFilterMeasurement(scans) + assert.equal(measurement.filterMeasurementValid, false) + assert.equal(measurement.filterWouldRefuse, null) + assert.notEqual(measurement.failure, null) + } +}) + +test('usable not-excluded and refusal measurements retain their distinct results', () => { + const visible = evaluateFilterMeasurement([scan('not_excluded'), scan('not_excluded')]) + assert.equal(visible.filterMeasurementValid, true) + assert.equal(visible.filterWouldRefuse, false) + assert.equal(visible.failure, null) + const rejected = evaluateFilterMeasurement([scan('not_excluded'), scan('reject')]) + assert.equal(rejected.filterMeasurementValid, true) + assert.equal(rejected.filterWouldRefuse, true) + assert.equal(rejected.failure, null) +}) diff --git a/experimental/126-guard-liveness/m2-macos-fwaslocked/former/codex b/experimental/126-guard-liveness/m2-macos-fwaslocked/former/codex new file mode 100755 index 0000000..7ed013f --- /dev/null +++ b/experimental/126-guard-liveness/m2-macos-fwaslocked/former/codex @@ -0,0 +1,30 @@ +#!/usr/bin/env node +// Former holder named `codex` (#127 research, M2). Acquires the thread flock +// under coordination (nonblocking; it starts first on an unlocked home), +// releases it with LOCK_UN but keeps the descriptor open until stdin EOF or +// FIXTURE_LIFETIME_MS (exit 3). Its open file description keeps the sticky +// FWASLOCKED bit while another process holds the lock. +import fs from 'node:fs' +import { createRequire } from 'node:module' +import path from 'node:path' +import process from 'node:process' +import { setTimeout } from 'node:timers' + +const require = createRequire(process.env.FIXTURE_REQUIRE_FROM) +const { flockSync } = require('fs-ext-extra-prebuilt') + +const lockPath = process.env.FIXTURE_LOCK_PATH +const guard = fs.openSync(path.join(path.dirname(lockPath), '.coordination.lock'), 'r') +flockSync(guard, 'exnb') +const thread = fs.openSync(lockPath, 'r+') +flockSync(thread, 'exnb') +flockSync(thread, 'un') +flockSync(guard, 'un') +fs.closeSync(guard) +process.stdout.write('ready\n') +setTimeout(() => { fs.closeSync(thread); process.exit(3) }, Number(process.env.FIXTURE_LIFETIME_MS ?? 60000)).unref() +process.stdin.resume() +process.stdin.on('end', () => { + fs.closeSync(thread) + process.exit(0) +}) diff --git a/experimental/126-guard-liveness/m2-macos-fwaslocked/holder/codex b/experimental/126-guard-liveness/m2-macos-fwaslocked/holder/codex new file mode 100755 index 0000000..6d62613 --- /dev/null +++ b/experimental/126-guard-liveness/m2-macos-fwaslocked/holder/codex @@ -0,0 +1,44 @@ +#!/usr/bin/env node +// Synthetic actual holder named `codex` (#127 research, M2). Derived from the +// #100 fixture. Takes coordination and then the thread flock, both with +// bounded nonblocking polling (real Codex blocks on coordination), and keeps +// the thread lock until stdin EOF or FIXTURE_LIFETIME_MS, whichever is first. +// Lifetime expiry exits with code 3 so the runner can tell it apart. +import fs from 'node:fs' +import { createRequire } from 'node:module' +import path from 'node:path' +import process from 'node:process' +import { setTimeout } from 'node:timers' + +const require = createRequire(process.env.FIXTURE_REQUIRE_FROM) +const { flockSync } = require('fs-ext-extra-prebuilt') +const sleeper = new Int32Array(new SharedArrayBuffer(4)) + +function acquire(fd, budgetMs) { + const until = Date.now() + budgetMs + while (true) { + try { + flockSync(fd, 'exnb') + return + } catch (error) { + if (error.code !== 'EAGAIN' && error.code !== 'EWOULDBLOCK') throw error + } + if (Date.now() > until) throw new Error('holder acquisition timed out') + Atomics.wait(sleeper, 0, 0, 5) + } +} + +const lockPath = process.env.FIXTURE_LOCK_PATH +const guard = fs.openSync(path.join(path.dirname(lockPath), '.coordination.lock'), 'r') +acquire(guard, 2000) +const thread = fs.openSync(lockPath, 'r+') +acquire(thread, 2000) +flockSync(guard, 'un') +fs.closeSync(guard) +process.stdout.write('ready\n') +setTimeout(() => { fs.closeSync(thread); process.exit(3) }, Number(process.env.FIXTURE_LIFETIME_MS ?? 60000)).unref() +process.stdin.resume() +process.stdin.on('end', () => { + fs.closeSync(thread) + process.exit(0) +}) diff --git a/experimental/126-guard-liveness/m2-macos-fwaslocked/lsof-barrier.mjs b/experimental/126-guard-liveness/m2-macos-fwaslocked/lsof-barrier.mjs new file mode 100644 index 0000000..8e885c8 --- /dev/null +++ b/experimental/126-guard-liveness/m2-macos-fwaslocked/lsof-barrier.mjs @@ -0,0 +1,89 @@ +#!/usr/bin/env node +// Barrier lsof wrapper for M2 (#127 research). Derived from the #100 +// attribution-interleaving wrapper: optional barrier into the transient's +// open window, optional removal of one PID's process sets (modelled clean +// omission), plus an `lsof +fg` scan taken in the same window and logged for +// the hypothetical FWASLOCKED refusal filter. The product's own scan output +// is passed through unchanged apart from the modelled omission. spawnSync's +// timeout makes Node send SIGTERM to a slow lsof; `signal` is logged. +import { spawnSync } from 'node:child_process' +import fs from 'node:fs' +import net from 'node:net' +import process from 'node:process' +import { clearTimeout, setTimeout } from 'node:timers' +import { fgScan } from './fg-scan.mjs' +import { diagnosticEvidence } from './part2-verdict.mjs' + +const args = process.argv.slice(2) +const config = JSON.parse(fs.readFileSync(process.env.FIXTURE_CONFIG, 'utf8')) +const realLsof = '/usr/sbin/lsof' +const log = (record) => fs.appendFileSync(config.log, + JSON.stringify({ role: 'lsof-barrier', pid: process.pid, t: Date.now(), args, ...record }) + '\n') + +function barrierRequest(command) { + return new Promise((resolve, reject) => { + let timer + const settle = (fn, value) => { clearTimeout(timer); sock.destroy(); fn(value) } + const sock = net.createConnection(config.socket, () => { + sock.setEncoding('utf8') + let buf = '' + sock.on('data', (chunk) => { + buf += chunk + const nl = buf.indexOf('\n') + if (nl >= 0) settle(resolve, buf.slice(0, nl)) + }) + sock.write(command + '\n') + }) + sock.on('error', (error) => settle(reject, error)) + timer = setTimeout(() => settle(reject, new Error('barrier timeout')), 2000) + }) +} + +function removePidSets(stdout, pid) { + const prefix = `p${pid}\0` + let skip = false + return stdout.split('\n').filter((line) => { + if (line.startsWith('p')) skip = line.startsWith(prefix) + return !skip + }).join('\n') +} + +async function main() { + // The opener discovery scan is always barriered and +fg-sampled. With + // `barrierAll` (the #100 setup) the per-PID lock-set and cwd scans are + // barriered too; otherwise they pass through outside the transient window. + const isPathScan = args.includes('-F0pcu') + let entered = false + if (config.socket && (isPathScan || config.barrierAll)) { + try { + const reply = await barrierRequest('attempt') + entered = reply === 'inside' + log({ event: 'barrier_entered', reply }) + } catch (error) { + log({ event: 'barrier_failed', error: error.message }) + } + } + const result = spawnSync(realLsof, args, { + env: { LC_ALL: 'C', LANG: 'C', PATH: '/usr/bin:/bin:/usr/sbin:/sbin' }, + maxBuffer: 4 * 1024 * 1024, timeout: 4000, + }) + if (isPathScan) log({ event: 'fg_scan', scan: fgScan(args.at(-1)) }) + if (entered) { + try { + await barrierRequest('finish') + } catch (error) { + log({ event: 'barrier_release_failed', error: error.message }) + } + } + let stdout = result.stdout?.toString() ?? '' + if (typeof config.hidePid === 'number') stdout = removePidSets(stdout, config.hidePid) + log({ event: 'done', isPathScan, ...diagnosticEvidence(result), stdoutHead: stdout.slice(0, 200).replace(/\0/g, '|') }) + process.stdout.write(stdout) + process.stderr.write(result.stderr?.toString() ?? '') + process.exitCode = result.status ?? 1 +} + +main().catch((error) => { + log({ event: 'fatal', error: error.message }) + process.exitCode = 1 +}) diff --git a/experimental/126-guard-liveness/m2-macos-fwaslocked/options.mjs b/experimental/126-guard-liveness/m2-macos-fwaslocked/options.mjs new file mode 100644 index 0000000..d9cd471 --- /dev/null +++ b/experimental/126-guard-liveness/m2-macos-fwaslocked/options.mjs @@ -0,0 +1,17 @@ +export function assertDarwin(platform) { + if (platform !== 'darwin') throw new Error('M2 requires Darwin; no measurement was started') +} + +export function options(env) { + const value = env.M2_REPEATS ?? '3' + if (!/^[1-9]\d*$/.test(value) || Number(value) > 10) throw new Error('M2_REPEATS must be an integer from 1 to 10') + const parts = (env.M2_PARTS ?? '1,2').split(',') + if (parts.some((p) => p !== '1' && p !== '2') || new Set(parts).size !== parts.length) throw new Error('M2_PARTS must contain distinct parts 1 and/or 2') + return { repeats: Number(value), parts } +} + +export const shellQuote = (text) => "'" + String(text).replaceAll("'", "'\\''") + "'" + +export function wrapperText(configPath, executable, script) { + return `#!/bin/sh\nFIXTURE_CONFIG=${shellQuote(configPath)} exec ${shellQuote(executable)} ${shellQuote(script)} "$@"\n` +} diff --git a/experimental/126-guard-liveness/m2-macos-fwaslocked/options.test.mjs b/experimental/126-guard-liveness/m2-macos-fwaslocked/options.test.mjs new file mode 100644 index 0000000..db92a2a --- /dev/null +++ b/experimental/126-guard-liveness/m2-macos-fwaslocked/options.test.mjs @@ -0,0 +1,34 @@ +import assert from 'node:assert/strict' +import { spawnSync } from 'node:child_process' +import fs from 'node:fs' +import path from 'node:path' +import process from 'node:process' +import { test } from 'node:test' +import { freshPrivateDir } from '../lib/common.mjs' +import { options, shellQuote, wrapperText } from './options.mjs' + +test('repeat and part bounds reject empty, fractional, unbounded and duplicate selections', () => { + for (const value of ['', '0', '-1', '1.5', 'Infinity', 'NaN', '11', '1e1']) assert.throws(() => options({ M2_REPEATS: value })) + for (const value of ['', '3', '1,', '1,1', '1,2,1', ' 1']) assert.throws(() => options({ M2_PARTS: value })) + assert.deepEqual(options({ M2_REPEATS: '10', M2_PARTS: '2,1' }), { repeats: 10, parts: ['2', '1'] }) +}) + +test('wrapper quotes shell substitutions, backticks, apostrophes and spaces literally', () => { + const dir = freshPrivateDir('127-quote-test') + try { + // Only a marker inside this fresh private scratch could be affected by + // accidental shell execution. No user paths or credentials are involved. + const marker = path.join(dir, 'executed') + const payload = ` spaces ' $(touch ${shellQuote(marker)}) \`touch ${shellQuote(marker)}\`` + const probe = path.join(dir, "probe ' spaced $(touch executed) `touch executed`.mjs") + fs.writeFileSync(probe, 'process.stdout.write(JSON.stringify({config:process.env.FIXTURE_CONFIG,args:process.argv.slice(2)}))\n', { flag: 'wx', mode: 0o600 }) + const wrapper = path.join(dir, 'wrapper.sh') + fs.writeFileSync(wrapper, wrapperText(payload, process.execPath, probe), { flag: 'wx', mode: 0o700 }) + const result = spawnSync('/bin/sh', [wrapper, payload], { cwd: dir, encoding: 'utf8', timeout: 3000 }) + assert.equal(result.status, 0, result.stderr) + assert.deepEqual(JSON.parse(result.stdout), { config: payload, args: [payload] }) + assert.equal(fs.existsSync(marker), false) + } finally { + fs.rmSync(dir, { recursive: true }) + } +}) diff --git a/experimental/126-guard-liveness/m2-macos-fwaslocked/part1-verdict.mjs b/experimental/126-guard-liveness/m2-macos-fwaslocked/part1-verdict.mjs new file mode 100644 index 0000000..ca986a1 --- /dev/null +++ b/experimental/126-guard-liveness/m2-macos-fwaslocked/part1-verdict.mjs @@ -0,0 +1,30 @@ +// Fixed Part 1 expectations from the recorded descriptor-lifetime matrix. +// Descriptor numbers/counts vary; the probe and every expected role's filter +// decision must still agree. not_excluded is never positive holder evidence. +const EXPECTED = Object.freeze({ + holder: { probe: 'held', roles: { holder: 'not_excluded' } }, + holder_multi_opener: { probe: 'held', roles: { holder: 'not_excluded', opener: 'reject' } }, + holder_opener: { probe: 'held', roles: { holder: 'not_excluded', opener: 'reject' } }, + holder_failed: { probe: 'held', roles: { holder: 'not_excluded', failed: 'reject' } }, + former_holder: { probe: 'held', roles: { former: 'not_excluded', holder: 'not_excluded' } }, + former_only: { probe: 'free', roles: { former: 'not_excluded' } }, + failed_after_holder_exit: { probe: 'free', roles: { failed: 'reject' } }, +}) + +export function evaluatePart1Scenario(name, probe, scan, roles) { + const expected = Object.hasOwn(EXPECTED, name) ? EXPECTED[name] : null + if (!expected) return ['unknown_scenario'] + const failures = [] + if (scan?.valid !== true) failures.push('invalid_scan') + if (probe !== expected.probe) failures.push('probe') + if (roles === null || typeof roles !== 'object' || Array.isArray(roles)) return [...failures, 'roles_missing'] + const expectedRoles = Object.keys(expected.roles) + if (Object.keys(roles).length !== expectedRoles.length || expectedRoles.some((role) => !Object.hasOwn(roles, role))) failures.push('role_set') + for (const [role, decision] of Object.entries(expected.roles)) { + const observed = roles[role] + if (observed?.decision !== decision) failures.push(`decision:${role}`) + if (!Number.isSafeInteger(observed?.pid) || observed.pid <= 0 || + !Array.isArray(observed.flags) || observed.flags.length === 0) failures.push(`evidence:${role}`) + } + return failures +} diff --git a/experimental/126-guard-liveness/m2-macos-fwaslocked/part1-verdict.test.mjs b/experimental/126-guard-liveness/m2-macos-fwaslocked/part1-verdict.test.mjs new file mode 100644 index 0000000..5fb6b3b --- /dev/null +++ b/experimental/126-guard-liveness/m2-macos-fwaslocked/part1-verdict.test.mjs @@ -0,0 +1,49 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' +import { evaluatePart1Scenario } from './part1-verdict.mjs' + +const role = (decision, pid = 100) => ({ decision, pid, flags: [decision === 'reject' ? '5:0x3' : '5:0x4003'] }) +const cases = [ + ['holder', 'held', { holder: role('not_excluded') }], + ['holder_multi_opener', 'held', { holder: { ...role('not_excluded'), flags: ['4:0x4003', '9:0x1'] }, opener: role('reject', 101) }], + ['holder_opener', 'held', { holder: role('not_excluded'), opener: role('reject', 101) }], + ['holder_failed', 'held', { holder: role('not_excluded'), failed: role('reject', 101) }], + ['former_holder', 'held', { former: role('not_excluded'), holder: role('not_excluded', 101) }], + ['former_only', 'free', { former: role('not_excluded') }], + ['failed_after_holder_exit', 'free', { failed: role('reject') }], +] + +test('all seven descriptor-lifetime scenarios require their fixed expected outcomes', () => { + for (const [name, probe, roles] of cases) assert.deepEqual(evaluatePart1Scenario(name, probe, { valid: true }, roles), [], name) +}) + +test('contrary but well-formed probe and flag decisions invalidate each scenario', () => { + for (const [name, probe, roles] of cases) { + assert.ok(evaluatePart1Scenario(name, probe === 'held' ? 'free' : 'held', { valid: true }, roles).includes('probe'), name) + for (const [label, observation] of Object.entries(roles)) { + const opposite = role(observation.decision === 'reject' ? 'not_excluded' : 'reject', observation.pid) + assert.ok(evaluatePart1Scenario(name, probe, { valid: true }, { ...roles, [label]: opposite }).includes(`decision:${label}`), `${name}/${label}`) + } + } +}) + +test('missing or extra roles, unknown decisions and absent descriptor evidence never pass', () => { + const normal = { holder: role('not_excluded'), opener: role('reject', 101) } + for (const roles of [null, {}, [], { holder: normal.holder }, { ...normal, unexpected: role('reject', 102) }, + { ...normal, opener: role('unknown', 101) }, { ...normal, opener: { ...normal.opener, flags: [] } }, + { ...normal, holder: { ...normal.holder, pid: null } }]) { + assert.notEqual(evaluatePart1Scenario('holder_opener', 'held', { valid: true }, roles).length, 0) + } +}) + +test('invalid scans and unrecognized cases do not acquire a success interpretation', () => { + for (const scan of [null, {}, { valid: false }, { valid: 'true' }]) { + assert.ok(evaluatePart1Scenario('holder', 'held', scan, { holder: role('not_excluded') }).includes('invalid_scan')) + } + for (const name of ['unknown', 'toString', '__proto__']) assert.deepEqual(evaluatePart1Scenario(name, 'held', { valid: true }, {}), ['unknown_scenario']) +}) + +test('additional descriptors do not imply an unexpected role or fixed FD count', () => { + const roles = { holder: { ...role('not_excluded'), flags: ['7:0x1', '22:0x4003', '41:0x1'] }, opener: role('reject', 101) } + assert.deepEqual(evaluatePart1Scenario('holder_multi_opener', 'held', { valid: true }, roles), []) +}) diff --git a/experimental/126-guard-liveness/m2-macos-fwaslocked/part2-verdict.mjs b/experimental/126-guard-liveness/m2-macos-fwaslocked/part2-verdict.mjs new file mode 100644 index 0000000..e688559 --- /dev/null +++ b/experimental/126-guard-liveness/m2-macos-fwaslocked/part2-verdict.mjs @@ -0,0 +1,43 @@ +// Fixed expectations from the recorded M2 Part 2 table, not inferred from the +// current outcome. A mismatch invalidates this reproduction; it does not prove +// that a product refusal is unsafe or that the filter establishes ownership. +const EXPECTED = Object.freeze({ + B_failed: { safe: false, sameOwner: false, filter: 'reject', blockers: ['owner_holds_other_thread_locks'] }, + B_failed_all_scans: { safe: true, sameOwner: false, filter: 'reject', blockers: [] }, + B_former: { safe: true, sameOwner: false, filter: 'not_excluded', blockers: [] }, + control_visible_holder: { safe: true, sameOwner: true, filter: 'not_excluded', blockers: [] }, +}) + +export function diagnosticEvidence(result) { + return { + exitCode: result.status ?? null, signal: result.signal ?? null, + error: result.error?.message ?? null, stderr: result.stderr?.toString() ?? '', + } +} + +export function barrierEventFailed(event) { + if (['barrier_failed', 'barrier_release_failed', 'fatal'].includes(event.event)) return true + if (event.event === 'barrier_entered') return event.reply !== 'inside' + if (event.event === 'done') return event.exitCode !== 0 || Boolean(event.signal || event.error) || event.stderr !== '' + return false +} + +export function evaluatePart2Scenario(scenario, inspection, holderPid, visiblePid, measurement) { + const expected = EXPECTED[scenario] + if (!expected) return ['unknown_scenario'] + const failures = [] + const pidValid = (pid) => Number.isSafeInteger(pid) && pid > 0 + if (!pidValid(holderPid) || !pidValid(visiblePid) || (holderPid === visiblePid) !== expected.sameOwner) failures.push('fixture_pid_relation') + if (inspection?.classification !== 'live_owner') failures.push('classification') + if (inspection?.safeToUnlock !== expected.safe) failures.push('eligibility') + const ownerPid = inspection?.owner?.pid + if (!pidValid(ownerPid) || ownerPid !== visiblePid || (ownerPid === holderPid) !== expected.sameOwner) failures.push('owner_relation') + if (!Array.isArray(inspection?.openers) || inspection.openers.length !== 1 || inspection.openers[0]?.pid !== visiblePid) failures.push('visible_opener') + if (!Array.isArray(inspection?.blockers) || inspection.blockers.length !== expected.blockers.length || + !expected.blockers.every((blocker) => inspection.blockers.includes(blocker))) failures.push('blockers') + if (measurement?.filterMeasurementValid !== true || measurement.failure !== null || + !Array.isArray(measurement.ownerDecisions) || measurement.ownerDecisions.length === 0 || + !measurement.ownerDecisions.every((decision) => decision === expected.filter) || + measurement.filterWouldRefuse !== (expected.filter === 'reject')) failures.push('filter_result') + return failures +} diff --git a/experimental/126-guard-liveness/m2-macos-fwaslocked/part2-verdict.test.mjs b/experimental/126-guard-liveness/m2-macos-fwaslocked/part2-verdict.test.mjs new file mode 100644 index 0000000..345c9cf --- /dev/null +++ b/experimental/126-guard-liveness/m2-macos-fwaslocked/part2-verdict.test.mjs @@ -0,0 +1,90 @@ +import assert from 'node:assert/strict' +import { Buffer } from 'node:buffer' +import { test } from 'node:test' +import { assertDarwin } from './options.mjs' +import { evaluateFilterMeasurement } from './filter-measurement.mjs' +import { barrierEventFailed, diagnosticEvidence, evaluatePart2Scenario } from './part2-verdict.mjs' + +test('barrier completion rejects nonzero exit, noise, signals, spawn errors and missing stderr', () => { + const success = { status: 0, signal: null, stderr: Buffer.from('') } + const done = (result) => ({ event: 'done', ...diagnosticEvidence(result) }) + assert.equal(barrierEventFailed(done(success)), false) + for (const result of [ + { ...success, status: 1 }, { ...success, status: 2 }, { ...success, status: null }, + { ...success, stderr: Buffer.from('warning: incomplete visibility\n') }, + { ...success, stderr: Buffer.from(' ') }, { ...success, signal: 'SIGTERM' }, + { ...success, error: new Error('synthetic spawn failure') }, + ]) assert.equal(barrierEventFailed(done(result)), true) + const evidence = done({ ...success, stderr: Buffer.from('synthetic stderr') }) + assert.equal(evidence.stderr, 'synthetic stderr') + assert.equal(barrierEventFailed({ event: 'done', exitCode: 0 }), true) +}) + +test('barrier handshake and release failures remain failures', () => { + for (const event of ['barrier_failed', 'barrier_release_failed', 'fatal']) assert.equal(barrierEventFailed({ event }), true) + assert.equal(barrierEventFailed({ event: 'barrier_entered', reply: 'busy' }), true) + assert.equal(barrierEventFailed({ event: 'barrier_entered', reply: 'inside' }), false) +}) + +function observation(scenario) { + const holder = 101 + const visible = scenario === 'control_visible_holder' ? holder : 102 + const blocked = scenario === 'B_failed' + const decision = scenario.startsWith('B_failed') ? 'reject' : 'not_excluded' + return { + holder, visible, + inspection: { classification: 'live_owner', safeToUnlock: !blocked, owner: { pid: visible }, + openers: [{ pid: visible }], blockers: blocked ? ['owner_holds_other_thread_locks'] : [] }, + measurement: evaluateFilterMeasurement([{ valid: true, owner: { decision } }]), + } +} + +const scenarios = ['B_failed', 'B_failed_all_scans', 'B_former', 'control_visible_holder'] +const verdict = (scenario, sample) => evaluatePart2Scenario(scenario, sample.inspection, sample.holder, sample.visible, sample.measurement) + +test('all four documented M2 scenarios match their independent expected outcomes', () => { + for (const scenario of scenarios) assert.deepEqual(verdict(scenario, observation(scenario)), []) +}) + +test('a missing reproduction or failed visible-holder control cannot pass the scenario', () => { + const changes = [ + (sample) => { sample.inspection.classification = 'unknown' }, + (sample) => { sample.inspection.safeToUnlock = !sample.inspection.safeToUnlock }, + (sample) => { sample.inspection.owner = null }, + (sample) => { sample.inspection.owner.pid = 999 }, + (sample) => { sample.inspection.openers = [] }, + (sample) => { sample.inspection.openers.push({ pid: 999 }) }, + (sample) => { sample.inspection.blockers = ['lock_probe_unknown'] }, + (sample) => { sample.measurement = evaluateFilterMeasurement([{ valid: true, owner: { decision: 'unknown' } }]) }, + (sample) => { sample.measurement.filterWouldRefuse = !sample.measurement.filterWouldRefuse }, + ] + for (const scenario of scenarios) { + for (const change of changes) { + const sample = observation(scenario) + change(sample) + assert.notEqual(verdict(scenario, sample).length, 0, scenario) + } + } +}) + +test('owner relations and all sampled filter decisions must match the scenario', () => { + for (const scenario of scenarios) { + const sample = observation(scenario) + sample.inspection.owner.pid = scenario === 'control_visible_holder' ? 102 : sample.holder + assert.ok(verdict(scenario, sample).includes('owner_relation')) + const mixed = observation(scenario) + mixed.measurement = evaluateFilterMeasurement([ + { valid: true, owner: { decision: 'reject' } }, { valid: true, owner: { decision: 'not_excluded' } }, + ]) + assert.ok(verdict(scenario, mixed).includes('filter_result')) + } + const sample = observation('B_former') + assert.deepEqual(verdict('unknown_scenario', sample), ['unknown_scenario']) +}) + +test('M2 rejects every non-Darwin platform before measurement setup', () => { + assert.doesNotThrow(() => assertDarwin('darwin')) + for (const platform of ['linux', 'win32', 'freebsd', 'Darwin', '', undefined, null]) { + assert.throws(() => assertDarwin(platform), /M2 requires Darwin/) + } +}) diff --git a/experimental/126-guard-liveness/m2-macos-fwaslocked/run.mjs b/experimental/126-guard-liveness/m2-macos-fwaslocked/run.mjs new file mode 100644 index 0000000..111257a --- /dev/null +++ b/experimental/126-guard-liveness/m2-macos-fwaslocked/run.mjs @@ -0,0 +1,301 @@ +#!/usr/bin/env node +// M2 (#127 research): can an `lsof +fg` FWASLOCKED check serve as a +// refusal-side filter on macOS? Measurement only; no product policy changes. +// +// Part 1 is a flag matrix over descriptor lifetimes. Part 2 re-runs the #100 +// wrong-target shape (actual holder hidden from the product's lsof scan) with +// two visible Codex-looking non-holders: a failed acquirer and a former holder +// that released with LOCK_UN but kept its descriptor. The +fg scan is taken in +// the same barrier window, and the modelled omission is applied to it too. +// +// `inspectThread` is called; `unlockInspectedThread` never is, so no owner +// is signalled. This runner sends no signal to its children: it closes their +// stdin and waits with a bound, and each fixture has its own lifetime. Node +// can still SIGTERM a timed-out `lsof` (spawnSync timeout here, and the +// product's own diagnostic timeout); the +fg scans record such signals. +// +// Any error, unconfirmed child exit, missing scan or result count mismatch +// makes the run exit nonzero. Scratch is always kept; the socket directory is +// removed only when every child's exit was observed. +// +// M2_FAULT (fault checks only): `early_exit` (an actor exits before ready), +// `ignore_eof` (an actor ignores EOF, stop budget shorter than its lifetime), +// `long_socket` (socket path over the platform limit, refused before start). +import { log } from 'node:console' +import fs from 'node:fs' +import { createRequire } from 'node:module' +import os from 'node:os' +import path from 'node:path' +import process from 'node:process' +import { fileURLToPath } from 'node:url' +import { assertSocketPath, freshPrivateDir, startChild, stopAll, stopChild, writeNew } from '../lib/common.mjs' +import { fgScan, filterDecision, withoutPid } from './fg-scan.mjs' +import { evaluateFilterMeasurement } from './filter-measurement.mjs' +import { assertDarwin, options, wrapperText } from './options.mjs' +import { barrierEventFailed, evaluatePart2Scenario } from './part2-verdict.mjs' +import { evaluatePart1Scenario } from './part1-verdict.mjs' + +// Reject unsupported hosts before product/native imports, scratch or children. +assertDarwin(process.platform) +const DIR = path.dirname(fileURLToPath(import.meta.url)) +const ROOT = path.resolve(DIR, '../../..') +const DIST = path.join(ROOT, 'dist') +const THREAD_ID = '01a089e8-3731-7202-ba68-0f4b0a3b2711' +const { repeats: REPEATS, parts: PARTS } = options(process.env) +const FAULT = process.env.M2_FAULT ?? null +const LIFETIME_MS = 60000 +const REQUIRE_FROM = path.join(ROOT, 'node_modules', 'fs-ext-extra-prebuilt', 'package.json') +const { inspectThread } = await import(path.join(DIST, 'inspection.js')) +const { overrideDiagnosticExecutableForTesting } = await import(path.join(DIST, 'process.js')) +const fsExt = createRequire(REQUIRE_FROM)('fs-ext-extra-prebuilt') + +const out = freshPrivateDir('127-m2') +const sockets = freshPrivateDir('127-s') +const logFile = path.join(out, 'events.jsonl') +const live = new Set() +const results = [] +const failures = [] +const incomplete = [] +let homeIndex = 0 + +function createHome() { + const home = path.join(out, `h${++homeIndex}`) + const lockDir = path.join(home, 'thread-writer-locks') + const sessionDir = path.join(home, 'sessions', '2026', '10', '14') + fs.mkdirSync(lockDir, { recursive: true, mode: 0o700 }) + fs.mkdirSync(sessionDir, { recursive: true, mode: 0o700 }) + writeNew(path.join(lockDir, '.coordination.lock'), '') + const lockPath = path.join(lockDir, `${THREAD_ID}.lock`) + writeNew(lockPath, '') + writeNew(path.join(sessionDir, `rollout-2026-10-14T00-00-00-${THREAD_ID}.jsonl`), + JSON.stringify({ timestamp: '2026-10-14T00:00:00.000Z', type: 'event_msg', payload: { type: 'task_complete' } }) + '\n') + return { home, lockPath } +} + +async function start(script, env) { + const node = script.endsWith('.mjs') + const handle = startChild(node ? process.execPath : script, node ? [script] : [], { + env: { PATH: process.env.PATH, FIXTURE_REQUIRE_FROM: REQUIRE_FROM, FIXTURE_LIFETIME_MS: String(LIFETIME_MS), ...env }, + }, live) + const line = await handle.ready + if (line !== 'ready') throw new Error(`unexpected first line from ${script}: ${line}`) + return handle +} + +async function stop(handle, budgetMs = 10000, expectedExit = 0) { + const result = await stopChild(handle, budgetMs) + if (result.incomplete) incomplete.push(result) + else if (result.exitCode !== expectedExit) failures.push(`pid ${handle.pid} exit ${JSON.stringify(result)} ${handle.stderr}`) + return result +} + +// The observer uses its own descriptor and closes it; its bit never appears +// in a scan because every scan precedes this probe. +function probe(lockPath) { + const fd = fs.openSync(lockPath, 'r') + try { + fsExt.flockSync(fd, 'exnb') + fsExt.flockSync(fd, 'un') + return 'free' + } catch (error) { + if (error.code === 'EAGAIN' || error.code === 'EWOULDBLOCK') return 'held' + return `error:${error.code}` + } finally { + fs.closeSync(fd) + } +} + +const roleView = (scan, pids) => Object.fromEntries(Object.entries(pids).map(([role, pid]) => { + const { decision, reason, descriptors } = filterDecision(scan, pid) + return [role, { pid, decision, reason, flags: descriptors.map((d) => `${d.fd}:${d.flags}`) }] +})) + +const ACTOR = path.join(DIR, 'actor.mjs') +const actor = (role, lockPath, extra = {}) => start(ACTOR, { FIXTURE_ROLE: role, FIXTURE_LOCK_PATH: lockPath, ...extra }) + +const MATRIX = { + holder: async (p) => ({ holder: await actor('holder', p) }), + holder_multi_opener: async (p) => ({ holder: await actor('holder_multi', p), opener: await actor('opener', p) }), + holder_opener: async (p) => ({ holder: await actor('holder', p), opener: await actor('opener', p) }), + holder_failed: async (p) => ({ holder: await actor('holder', p), failed: await actor('failed', p) }), + former_holder: async (p) => ({ former: await actor('former', p), holder: await actor('holder', p) }), + former_only: async (p) => ({ former: await actor('former', p) }), + failed_after_holder_exit: async (p) => { + const holder = await actor('holder', p) + const failed = await actor('failed', p) + await stop(holder) + return { failed } + }, +} + +async function part1() { + for (let rep = 0; rep < REPEATS; rep += 1) { + for (const [name, setup] of Object.entries(MATRIX)) { + const { lockPath } = createHome() + const actors = await setup(lockPath) + const scan = fgScan(lockPath) + if (!scan.valid) failures.push(`part1 ${rep} ${name}: invalid +fg scan ${scan.problems.join(',')}`) + const state = probe(lockPath) + if (state !== 'free' && state !== 'held') failures.push(`part1 ${rep} ${name}: probe ${state}`) + const pids = Object.fromEntries(Object.entries(actors).map(([k, h]) => [k, h.pid])) + const roles = roleView(scan, pids) + if (Object.values(roles).some((r) => r.decision === 'unknown')) failures.push(`part1 ${rep} ${name}: fixture descriptor evidence unknown`) + const scenarioFailures = evaluatePart1Scenario(name, state, scan, roles) + for (const reason of scenarioFailures) failures.push(`part1 ${rep} ${name}: expected observation ${reason}`) + results.push({ + part: 1, rep, case: name, probe: state, + scenarioMatched: scenarioFailures.length === 0, scenarioFailures, + scan: { valid: scan.valid, problems: scan.problems, status: scan.status, signal: scan.signal, raw: scan.raw.replace(/\0/g, '|') }, + roles, + }) + for (const handle of Object.values(actors)) await stop(handle) + } + } +} + +function writeWrapper() { + const wrapper = path.join(out, 'lsof-wrapper.sh') + const configPath = path.join(out, 'barrier-config.json') + writeNew(wrapper, wrapperText(configPath, process.execPath, path.join(DIR, 'lsof-barrier.mjs')), 0o700) + return { wrapper, configPath } +} + +function eventsSince(offset) { + const text = fs.existsSync(logFile) ? fs.readFileSync(logFile).subarray(offset).toString('utf8') : '' + return text.split('\n').filter(Boolean).map((line) => JSON.parse(line)) +} + +async function part2() { + const { wrapper, configPath } = writeWrapper() + const scenarios = ['B_failed', 'B_failed_all_scans', 'B_former', 'control_visible_holder'] + for (let rep = 0; rep < REPEATS; rep += 1) { + for (const scenario of scenarios) { + const { home, lockPath } = createHome() + const env = { FIXTURE_LOCK_PATH: lockPath } + const config = { log: logFile } + const handles = [] + let holder + let visible + if (scenario === 'B_former') { + // The former holder releases before the hidden holder acquires. + visible = await start(path.join(DIR, 'former', 'codex'), env) + handles.push(visible) + holder = await start(path.join(DIR, 'holder', 'codex'), env) + handles.push(holder) + config.hidePid = holder.pid + } else { + holder = await start(path.join(DIR, 'holder', 'codex'), env) + handles.push(holder) + visible = holder + if (scenario.startsWith('B_failed')) { + const socket = path.join(sockets, `s${homeIndex}.sock`) + assertSocketPath(socket) + visible = await start(path.join(DIR, 'transient', 'codex'), { ...env, FIXTURE_SOCKET: socket, FIXTURE_LOG: logFile }) + handles.push(visible) + Object.assign(config, { socket, hidePid: holder.pid, barrierAll: scenario === 'B_failed_all_scans' }) + } + } + fs.writeFileSync(configPath, JSON.stringify(config) + '\n', { mode: 0o600 }) + const offset = fs.existsSync(logFile) ? fs.statSync(logFile).size : 0 + overrideDiagnosticExecutableForTesting('lsof', wrapper) + let inspection + try { + inspection = await inspectThread(THREAD_ID, { codexHome: home, stabilityMs: 50, terminationTimeoutMs: 3000 }) + } finally { + overrideDiagnosticExecutableForTesting('lsof', null) + } + const allEvents = eventsSince(offset) + const events = allEvents.filter((e) => e.event === 'fg_scan') + for (const event of allEvents) { + if (barrierEventFailed(event)) { + failures.push(`part2 ${rep} ${scenario}: barrier evidence ${JSON.stringify(event)}`) + } + } + if (config.socket) { + const entered = allEvents.filter((e) => e.event === 'barrier_entered' && e.reply === 'inside').length + const finished = allEvents.filter((e) => e.role === 'transient' && e.event === 'finish').length + if (!entered || finished !== entered) failures.push(`part2 ${rep} ${scenario}: barrier windows entered=${entered} finished=${finished}`) + } + if (!events.length) failures.push(`part2 ${rep} ${scenario}: no +fg scan recorded`) + const scans = events.map((e) => { + if (!e.scan.valid) failures.push(`part2 ${rep} ${scenario}: invalid +fg scan ${e.scan.problems.join(',')}`) + const hidden = typeof config.hidePid === 'number' ? withoutPid(e.scan, config.hidePid) : e.scan + return { + valid: e.scan.valid, problems: e.scan.problems, signal: e.scan.signal, raw: e.scan.raw.replace(/\0/g, '|'), + owner: inspection.owner ? roleView(hidden, { owner: inspection.owner.pid }).owner : null, + actualHolderUnfiltered: roleView(e.scan, { holder: holder.pid }).holder, + } + }) + const measurement = evaluateFilterMeasurement(scans) + const { ownerDecisions, filterMeasurementValid, filterWouldRefuse, failure } = measurement + // A completed scenario without a usable filter observation is still a + // failed measurement; it cannot leave the final run's failures empty. + if (failure) failures.push(`part2 ${rep} ${scenario}: ${failure}`) + const scenarioFailures = evaluatePart2Scenario(scenario, inspection, holder.pid, visible.pid, measurement) + for (const reason of scenarioFailures) failures.push(`part2 ${rep} ${scenario}: expected observation ${reason}`) + results.push({ + part: 2, rep, scenario, + scenarioMatched: scenarioFailures.length === 0, scenarioFailures, + classification: inspection.classification, safeToUnlock: inspection.safeToUnlock, + blockers: inspection.blockers, openers: inspection.openers.length, + ownerPid: inspection.owner?.pid ?? null, actualHolderPid: holder.pid, visiblePid: visible.pid, + wrongTarget: inspection.safeToUnlock && inspection.owner?.pid !== holder.pid, + // An unavailable owner or invalid evidence cannot establish a filter + // measurement. A present flag still never establishes lock ownership. + filterMeasurementValid, + filterWouldRefuse, + ownerDecisions, scans, + }) + for (const handle of handles) await stop(handle) + } + } +} + +async function faultCase() { + const { lockPath } = createHome() + if (FAULT === 'early_exit') { + await actor('early_exit', lockPath) + } else if (FAULT === 'ignore_eof') { + const handle = await actor('ignore_eof', lockPath, { FIXTURE_LIFETIME_MS: '4000' }) + await stop(handle, 1000) + } else if (FAULT === 'long_socket') { + assertSocketPath(path.join(sockets, 'x'.repeat(200) + '.sock')) + await start(path.join(DIR, 'transient', 'codex'), { FIXTURE_LOCK_PATH: lockPath, FIXTURE_SOCKET: 'unused', FIXTURE_LOG: logFile }) + } else { + throw new Error(`unknown fault ${FAULT}`) + } +} + +try { + if (FAULT) await faultCase() + else { + if (PARTS.includes('1')) await part1() + if (PARTS.includes('2')) await part2() + } +} catch (error) { + failures.push(String(error?.stack ?? error)) +} +for (const result of await stopAll(live)) { + if (result.incomplete) incomplete.push(result) + else if (result.exitCode !== 0 || result.signal || result.error) failures.push(`cleanup exit ${JSON.stringify(result)}`) +} +const expected = FAULT ? 0 + : (PARTS.includes('1') ? REPEATS * Object.keys(MATRIX).length : 0) + (PARTS.includes('2') ? REPEATS * 4 : 0) +if (results.length !== expected) failures.push(`results ${results.length}/${expected}`) +const diagnosticSignals = results.flatMap((r) => r.part === 1 ? [r.scan.signal] : r.scans?.map((s) => s.signal) ?? []).filter(Boolean) +const ok = failures.length === 0 && incomplete.length === 0 +// Keep the socket directory when any child's exit is unconfirmed. +if (incomplete.length === 0) fs.rmSync(sockets, { recursive: true, force: true }) +const report = { + ok, fault: FAULT, failures, incomplete, socketsKept: incomplete.length ? sockets : null, + diagnosticSignalsRecorded: diagnosticSignals, + environment: { platform: process.platform, release: os.release(), arch: process.arch, node: process.version, commit: process.env.M2_COMMIT ?? null }, + results, +} +writeNew(path.join(out, 'm2-results.json'), JSON.stringify(report, null, 2) + '\n') +const brief = results.map((r) => r.part === 1 + ? `P1 ${r.rep} ${r.case} probe=${r.probe} valid=${r.scan.valid} ` + Object.entries(r.roles).map(([k, v]) => `${k}:${v.flags.join('+')}/${v.decision}`).join(' ') + : `P2 ${r.rep} ${r.scenario} cls=${r.classification} safe=${r.safeToUnlock} wrong=${r.wrongTarget} filter=${r.ownerDecisions.join(',')} refuse=${r.filterWouldRefuse}`) +log([`out=${out}`, `ok=${ok} fault=${FAULT} failures=${failures.length} incomplete=${incomplete.length} diagSignals=${diagnosticSignals.length}`, + ...failures.map((f) => `FAIL ${f.split('\n')[0]}`), ...brief].join('\n')) +process.exitCode = ok ? 0 : 1 diff --git a/experimental/126-guard-liveness/m2-macos-fwaslocked/transient.test.mjs b/experimental/126-guard-liveness/m2-macos-fwaslocked/transient.test.mjs new file mode 100644 index 0000000..b68d98b --- /dev/null +++ b/experimental/126-guard-liveness/m2-macos-fwaslocked/transient.test.mjs @@ -0,0 +1,55 @@ +import assert from 'node:assert/strict' +import fs from 'node:fs' +import { createRequire } from 'node:module' +import net from 'node:net' +import path from 'node:path' +import process from 'node:process' +import { test } from 'node:test' +import { fileURLToPath } from 'node:url' +import { assertSocketPath, freshPrivateDir, startChild, stopChild, writeNew } from '../lib/common.mjs' + +const DIR = path.dirname(fileURLToPath(import.meta.url)) +const requireFrom = path.resolve(DIR, '../../../node_modules/fs-ext-extra-prebuilt/package.json') +const { flockSync } = createRequire(requireFrom)('fs-ext-extra-prebuilt') + +function request(socketPath, command) { + return new Promise((resolve, reject) => { + const socket = net.createConnection(socketPath, () => socket.write(command + '\n')) + let text = '' + socket.setTimeout(1000, () => socket.destroy(new Error('request timeout'))) + socket.on('error', reject) + socket.on('data', (chunk) => { + text += chunk + if (text.includes('\n')) { socket.destroy(); resolve(text.split('\n')[0]) } + }) + }) +} + +test('reentrant attempt preserves the original descriptors until finish', async () => { + const dir = freshPrivateDir('127-reentrant') + const guardPath = path.join(dir, '.coordination.lock') + const lockPath = path.join(dir, 'thread.lock') + const socket = path.join(dir, 's.sock') + assertSocketPath(socket) + writeNew(guardPath, ''); writeNew(lockPath, '') + const owner = fs.openSync(lockPath, 'r+') + const guard = fs.openSync(guardPath, 'r') + flockSync(owner, 'exnb') + const handle = startChild(path.join(DIR, 'transient/codex'), [], { + env: { PATH: process.env.PATH, FIXTURE_REQUIRE_FROM: requireFrom, FIXTURE_LOCK_PATH: lockPath, + FIXTURE_SOCKET: socket, FIXTURE_LOG: path.join(dir, 'events.jsonl'), FIXTURE_LIFETIME_MS: '5000' }, + }, new Set()) + try { + assert.equal(await handle.ready, 'ready') + assert.equal(await request(socket, 'attempt'), 'inside') + assert.equal(await request(socket, 'attempt'), 'busy') + assert.throws(() => flockSync(guard, 'exnb'), (error) => ['EAGAIN', 'EWOULDBLOCK'].includes(error.code)) + assert.equal(await request(socket, 'finish'), 'done') + flockSync(guard, 'exnb'); flockSync(guard, 'un') + } finally { + const result = await stopChild(handle) + fs.closeSync(owner); fs.closeSync(guard) + if (!result.incomplete) fs.rmSync(dir, { recursive: true }) + assert.equal(result.exitCode, 0, JSON.stringify(result)) + } +}) diff --git a/experimental/126-guard-liveness/m2-macos-fwaslocked/transient/codex b/experimental/126-guard-liveness/m2-macos-fwaslocked/transient/codex new file mode 100755 index 0000000..847b0b3 --- /dev/null +++ b/experimental/126-guard-liveness/m2-macos-fwaslocked/transient/codex @@ -0,0 +1,110 @@ +#!/usr/bin/env node +// Synthetic transient non-holder (#100 research). Models the descriptor +// lifetime of a failed Codex v0.162.1 `WriterLockCoordinator::acquire`: +// coordination is taken first, the thread file is opened, a nonblocking +// exclusive flock fails, and the thread file is dropped before coordination. +// Between `attempt` and `finish` it holds both descriptors, so an independently +// timed lsof scan can observe it as an opener. It never holds the thread lock. +// Commands arrive over a Unix socket; stdin EOF ends the process normally. +// FIXTURE_LIFETIME_MS bounds the process (exit 3). Copied from the #100 +// fixture with only this lifetime added. +import fs from 'node:fs' +import { createRequire } from 'node:module' +import net from 'node:net' +import path from 'node:path' +import { performance } from 'node:perf_hooks' +import process from 'node:process' +import { setTimeout } from 'node:timers' + +const require = createRequire(process.env.FIXTURE_REQUIRE_FROM) +const { flockSync } = require('fs-ext-extra-prebuilt') + +const lockPath = process.env.FIXTURE_LOCK_PATH +const guardPath = path.join(path.dirname(lockPath), '.coordination.lock') +const socketPath = process.env.FIXTURE_SOCKET +const log = (record) => fs.appendFileSync(process.env.FIXTURE_LOG, JSON.stringify({ role: 'transient', pid: process.pid, t: performance.timeOrigin + performance.now(), ...record }) + '\n') + +let guard = null +let thread = null + +function attempt() { + // A second barrier must not overwrite the descriptors from an open window. + if (guard !== null || thread !== null) { + log({ event: 'attempt', guard: 'busy', reason: 'already_inside' }) + return null + } + guard = fs.openSync(guardPath, 'r') + // Real Codex blocks here. This fixture must not: codex-unlock's guarded + // probe also takes coordination, and a blocking wait would deadlock both + // sides when a barrier request times out with no `finish`. + try { + flockSync(guard, 'exnb') + } catch (error) { + if (error.code !== 'EAGAIN' && error.code !== 'EWOULDBLOCK') throw error + fs.closeSync(guard) + guard = null + log({ event: 'attempt', guard: 'busy' }) + return null + } + thread = fs.openSync(lockPath, 'r+') + let acquired = false + try { + flockSync(thread, 'exnb') + acquired = true + } catch (error) { + if (error.code !== 'EAGAIN' && error.code !== 'EWOULDBLOCK') throw error + } + // A successful acquisition would make this process a real holder and + // invalidate the experiment, so release and report it. + if (acquired) flockSync(thread, 'un') + log({ event: 'attempt', acquired }) + return acquired +} + +function finish() { + // Rust drops the later local (thread file) before the earlier one (guard). + if (thread !== null) fs.closeSync(thread) + thread = null + if (guard !== null) { + flockSync(guard, 'un') + fs.closeSync(guard) + } + guard = null + log({ event: 'finish' }) +} + +const server = net.createServer((socket) => { + socket.setEncoding('utf8') + let buffer = '' + socket.on('data', (chunk) => { + buffer += chunk + let newline + while ((newline = buffer.indexOf('\n')) >= 0) { + const command = buffer.slice(0, newline) + buffer = buffer.slice(newline + 1) + if (command === 'attempt') { + const outcome = attempt() + socket.write(outcome === null ? 'busy\n' : outcome ? 'acquired\n' : 'inside\n') + } + else if (command === 'finish') { finish(); socket.write('done\n') } + else socket.write('error\n') + } + }) + socket.on('error', () => {}) +}) +server.listen(socketPath, () => process.stdout.write('ready\n')) +setTimeout(() => { + finish() + try { fs.unlinkSync(socketPath) } catch { /* already gone */ } + process.exit(3) +}, Number(process.env.FIXTURE_LIFETIME_MS ?? 60000)).unref() +process.stdin.resume() +process.stdin.on('end', () => { + finish() + // Close the listener and unlink the socket path before exiting, so a later + // fixture reusing the directory can bind its own socket. + server.close(() => { + try { fs.unlinkSync(socketPath) } catch { /* already gone */ } + process.exit(0) + }) +})