diff --git a/bun install --frozen-lockfile b/bun install --frozen-lockfile new file mode 100644 index 0000000..4f1408d --- /dev/null +++ b/bun install --frozen-lockfile @@ -0,0 +1 @@ +``` \ No newline at end of file diff --git a/docs/decision-83.md b/docs/decision-83.md new file mode 100644 index 0000000..74e3908 --- /dev/null +++ b/docs/decision-83.md @@ -0,0 +1,24 @@ +# Decision — Issue #83: Security Hardening: Sandbox Verifier Runtime + +**Status: ⏳ CLOSE AS WON'T DO — false positive from patrol over-fitting** + +This issue was auto-filed by the future-exploration sweep with confidence 0.50 (below the 0.8 threshold for autonomous action). It is a **discussion issue** opened for human deliberation, not an implementation ticket. + +A thorough analysis already exists in [`docs/decisions/SECURITY-SANDBOX-VERIFIER-ANALYSIS.md`](./decisions/SECURITY-SANDBOX-VERIFIER-ANALYSIS.md). + +## Summary + +The patrol correctly identifies that the verifier-runtime runs in-process, but the finding is a **false positive**: + +- The verifier-runtime does **not** execute arbitrary/user-supplied code — it is a pure-function library for deterministic JSON transformation and hash comparison. +- No `eval()`, `vm.runInNewContext()`, dynamic imports, or subprocess spawning exists in the package. +- Solver submissions are JSON data, not executable code. +- OS-level process isolation is already sufficient for the current architecture. + +## Decision + +**No code change is warranted.** Close this issue. If Phase 3 introduces pluggable third-party verifier packages, sandboxing via `isolated-vm` or `worker_threads` should be revisited as part of that feature design. + +--- + +*Filed by Claude Bot (patrol panel simulation). See also `docs/decisions/SECURITY-SANDBOX-VERIFIER-ANALYSIS.md` for full analysis.* diff --git a/docs/decision-security-hardening-sandbox.md b/docs/decision-security-hardening-sandbox.md new file mode 100644 index 0000000..22a951f --- /dev/null +++ b/docs/decision-security-hardening-sandbox.md @@ -0,0 +1,52 @@ +# Decision: Security Hardening — Sandbox Verifier Runtime + +**Issue:** #83 — [discussion] explore: Security Hardening: Sandbox Verifier Runtime (make its acceptance criteria hold) + +**Filed by:** future-exploration sweep | **Confidence:** 0.50 | **Category:** security + +## Analysis + +### Observation summary +The patrol sweep observed that verifier packages run arbitrary code, presenting a potential RCE attack surface if a solver submits a malicious payload or a world generator contains an exploit. The suggested fix is to migrate to an embedded sandbox like `isolated-vm` (for JavaScript runtimes) or `wasmtime` (for WASI modules). + +### Assessment + +**1. Is this a real gap?** + +The current `@fresharena/verifier-runtime` package implements deterministic verification functions (normalize, diff, patch, merge, migrate) as pure TypeScript functions running in the same Node.js process as the evaluation pipeline. However: + +- The verifier-runtime does **not** currently execute arbitrary solver code — it runs the reference implementation and compares hashes. The solver's output is data (JSON), not code. +- The RCE concern would become real if the verifier were extended to support **pluggable verifier packages** (e.g., third-party verifiers loaded dynamically), or if the task world generators could inject code into the verification phase. +- In the current architecture (Phase 0/1), verifiers are statically compiled into the monorepo and audited via code review. The attack surface is limited to supply-chain attacks on dependencies. + +**Verdict:** The gap is **real but premature** for the current phase. It becomes critical when Phase 3 introduces pluggable verifier packages or third-party task worlds. + +**2. If real, does the fix belong in this repo or a sibling?** + +The fix belongs in **this repo** (`WasmAgent/fresharena`), specifically in `packages/verifier-runtime/`. No sibling repo currently provides sandboxing primitives that FreshArena would consume. + +However, if the team decides to share sandbox infrastructure across projects, a cross-cutting sandbox library could live in `WasmAgent/agent-trust-infra` (which already hosts trust/safety specifications). + +**3. Is the suggested fix correct?** + +The suggested fix (isolated-vm or wasmtime) is **conceptually correct** but premature: + +- **isolated-vm**: Would make sense if verifiers are written in JavaScript/TypeScript and need to be sandboxed from the host process. Adds ~2 MB to bundle size and introduces async overhead for every verifier call. +- **wasmtime**: Would make sense if verifiers are compiled to WASI modules. This would require either (a) rewriting existing TypeScript verifiers in Rust or (b) running a WASI-compiled JS runtime — both are high-effort. +- A simpler intermediate approach: **run verifiers as child processes** with resource limits (cgroups, RLIMITs) rather than embedding a sandbox library. This leverages OS-level isolation without adding a heavy dependency. + +## Decision + +**Status:** ⏳ Deferred — not actionable in current Phase 0/1. + +**Rationale:** +1. The current verifier-runtime does not execute user-supplied code; it runs only audited, statically-linked reference implementations. +2. Sandboxing introduces operational complexity (async boundaries, serialization overhead, error propagation) that would slow down the current research prototype without measurable security benefit. +3. The milestone acceptance criteria for verifier-runtime (Milestone 1: "provides deterministic sandbox for JSON transform verification") is satisfied by the current architecture — "sandbox" here refers to deterministic isolation (pure functions, no side effects, no I/O), not OS-level process isolation. + +**Recommended action for future:** +- If Phase 3 introduces pluggable verifier packages (as hinted in the roadmap), revisit sandboxing with `isolated-vm` as the primary candidate. +- Before then, add a **resource limit wrapper** (`packages/verifier-runtime/src/sandbox.ts`) that wraps verifier execution with timeout and memory limits using `worker_threads` or child processes. This provides practical safety without a heavy dependency. +- Document security assumptions in `docs/security-model.md`. + +**Close condition:** Close this discussion with reference from a future PR that implements sandboxing alongside pluggable verifier support. Not actionable as a standalone change. diff --git a/docs/decisions/SECURITY-SANDBOX-VERIFIER-ANALYSIS.md b/docs/decisions/SECURITY-SANDBOX-VERIFIER-ANALYSIS.md new file mode 100644 index 0000000..740373e --- /dev/null +++ b/docs/decisions/SECURITY-SANDBOX-VERIFIER-ANALYSIS.md @@ -0,0 +1,57 @@ +# Decision: Verifier Runtime Security Sandbox Analysis + +**Issue**: #83 — Security Hardening: Sandbox Verifier Runtime +**Filed by**: Patrol daemon (future-exploration sweep) +**Confidence**: 0.50 (below 0.8 threshold for autonomous fix) +**Status**: FALSE POSITIVE — no code changes needed + +## Analysis + +### Claim + +> "Verifier packages run arbitrary code, presenting a critical RCE attack surface if a solver submits a malicious payload or a world generator contains an exploit. Relying solely on OS-level process isolation is insufficient..." + +### Reality + +After thorough code review of the entire `packages/verifier-runtime/` package and its integration points: + +1. **The verifier-runtime does NOT run arbitrary code.** It is a collection of pure deterministic TypeScript functions: + - `normalize()` — recursively strip nulls, flatten objects, sort keys + - `diff()` — compute JSON structural differences (ops or merge-patch format) + - `apply()` — apply a patch to a source value + - `merge()` — merge two JSON objects with configurable conflict policy + - `migrate()` — schema migration with field mappings and type conversions + - `verify()` — hash comparison oracle + +2. **No dynamic code execution exists in the runtime:** + - No `eval()`, `new Function()`, or `vm.runInNewContext()` calls + - No `child_process` spawning or subprocess execution + - No filesystem or network I/O + - No dynamic module loading or third-party plugin loading + - No arbitrary code is received from solvers — solvers submit JSON outputs, not executables + +3. **The verifier is compiled library code**, not a plugin system. It is imported as a static dependency (`@fresharena/verifier-runtime`) and called synchronously. + +4. **OS-level process isolation is already sufficient** for running deterministic pure functions on structured JSON data. + +5. **The attacker model doesn't apply:** A solver submitting a "malicious payload" can only submit JSON values (objects, arrays, strings, numbers, booleans, null). These are transformed by pure functions with no side effects. There is no RCE vector. + +### Root Cause of False Positive + +The patrol daemon appears to have over-fitted on a generic "sandbox runtime" pattern without verifying whether the verifier-runtime actually executes untrusted code. The name "verifier-runtime" may suggest a code execution engine, but functionally it is a library of hash-verification functions. + +### Future Consideration + +If the architecture evolves to support **third-party verifier plugins** (e.g., a Verifier SDK that loads external WASM modules or JavaScript plugins), then sandboxing with `isolated-vm` or `wasmtime` would become relevant. This should be tracked as a roadmap item under Phase 3 or Phase 4, not as a current security gap. + +## Decision + +**CLOSE AS WON'T DO.** The finding is a false positive from the patrol daemon's over-fitting. No security sandbox migration is needed for the current architecture. + +## Cross-Repo Notes + +None needed — the issue is contained entirely within `WasmAgent/fresharena` and does not affect sibling repos. + +--- + +*Filed by Claude Bot on behalf of the deliberation process per issue #83*