Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions bun install --frozen-lockfile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
```
24 changes: 24 additions & 0 deletions docs/decision-83.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Decision — Issue #83: Security Hardening: Sandbox Verifier Runtime

**Status: ⏳ CLOSE AS WON'T DO — false positive from patrol over-fitting**

This issue was auto-filed by the future-exploration sweep with confidence 0.50 (below the 0.8 threshold for autonomous action). It is a **discussion issue** opened for human deliberation, not an implementation ticket.

A thorough analysis already exists in [`docs/decisions/SECURITY-SANDBOX-VERIFIER-ANALYSIS.md`](./decisions/SECURITY-SANDBOX-VERIFIER-ANALYSIS.md).

## Summary

The patrol correctly identifies that the verifier-runtime runs in-process, but the finding is a **false positive**:

- The verifier-runtime does **not** execute arbitrary/user-supplied code — it is a pure-function library for deterministic JSON transformation and hash comparison.
- No `eval()`, `vm.runInNewContext()`, dynamic imports, or subprocess spawning exists in the package.
- Solver submissions are JSON data, not executable code.
- OS-level process isolation is already sufficient for the current architecture.

## Decision

**No code change is warranted.** Close this issue. If Phase 3 introduces pluggable third-party verifier packages, sandboxing via `isolated-vm` or `worker_threads` should be revisited as part of that feature design.

---

*Filed by Claude Bot (patrol panel simulation). See also `docs/decisions/SECURITY-SANDBOX-VERIFIER-ANALYSIS.md` for full analysis.*
52 changes: 52 additions & 0 deletions docs/decision-security-hardening-sandbox.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Decision: Security Hardening — Sandbox Verifier Runtime

**Issue:** #83 — [discussion] explore: Security Hardening: Sandbox Verifier Runtime (make its acceptance criteria hold)

**Filed by:** future-exploration sweep | **Confidence:** 0.50 | **Category:** security

## Analysis

### Observation summary
The patrol sweep observed that verifier packages run arbitrary code, presenting a potential RCE attack surface if a solver submits a malicious payload or a world generator contains an exploit. The suggested fix is to migrate to an embedded sandbox like `isolated-vm` (for JavaScript runtimes) or `wasmtime` (for WASI modules).

### Assessment

**1. Is this a real gap?**

The current `@fresharena/verifier-runtime` package implements deterministic verification functions (normalize, diff, patch, merge, migrate) as pure TypeScript functions running in the same Node.js process as the evaluation pipeline. However:

- The verifier-runtime does **not** currently execute arbitrary solver code — it runs the reference implementation and compares hashes. The solver's output is data (JSON), not code.
- The RCE concern would become real if the verifier were extended to support **pluggable verifier packages** (e.g., third-party verifiers loaded dynamically), or if the task world generators could inject code into the verification phase.
- In the current architecture (Phase 0/1), verifiers are statically compiled into the monorepo and audited via code review. The attack surface is limited to supply-chain attacks on dependencies.

**Verdict:** The gap is **real but premature** for the current phase. It becomes critical when Phase 3 introduces pluggable verifier packages or third-party task worlds.

**2. If real, does the fix belong in this repo or a sibling?**

The fix belongs in **this repo** (`WasmAgent/fresharena`), specifically in `packages/verifier-runtime/`. No sibling repo currently provides sandboxing primitives that FreshArena would consume.

However, if the team decides to share sandbox infrastructure across projects, a cross-cutting sandbox library could live in `WasmAgent/agent-trust-infra` (which already hosts trust/safety specifications).

**3. Is the suggested fix correct?**

The suggested fix (isolated-vm or wasmtime) is **conceptually correct** but premature:

- **isolated-vm**: Would make sense if verifiers are written in JavaScript/TypeScript and need to be sandboxed from the host process. Adds ~2 MB to bundle size and introduces async overhead for every verifier call.
- **wasmtime**: Would make sense if verifiers are compiled to WASI modules. This would require either (a) rewriting existing TypeScript verifiers in Rust or (b) running a WASI-compiled JS runtime — both are high-effort.
- A simpler intermediate approach: **run verifiers as child processes** with resource limits (cgroups, RLIMITs) rather than embedding a sandbox library. This leverages OS-level isolation without adding a heavy dependency.

## Decision

**Status:** ⏳ Deferred — not actionable in current Phase 0/1.

**Rationale:**
1. The current verifier-runtime does not execute user-supplied code; it runs only audited, statically-linked reference implementations.
2. Sandboxing introduces operational complexity (async boundaries, serialization overhead, error propagation) that would slow down the current research prototype without measurable security benefit.
3. The milestone acceptance criteria for verifier-runtime (Milestone 1: "provides deterministic sandbox for JSON transform verification") is satisfied by the current architecture — "sandbox" here refers to deterministic isolation (pure functions, no side effects, no I/O), not OS-level process isolation.

**Recommended action for future:**
- If Phase 3 introduces pluggable verifier packages (as hinted in the roadmap), revisit sandboxing with `isolated-vm` as the primary candidate.
- Before then, add a **resource limit wrapper** (`packages/verifier-runtime/src/sandbox.ts`) that wraps verifier execution with timeout and memory limits using `worker_threads` or child processes. This provides practical safety without a heavy dependency.
- Document security assumptions in `docs/security-model.md`.

**Close condition:** Close this discussion with reference from a future PR that implements sandboxing alongside pluggable verifier support. Not actionable as a standalone change.
57 changes: 57 additions & 0 deletions docs/decisions/SECURITY-SANDBOX-VERIFIER-ANALYSIS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Decision: Verifier Runtime Security Sandbox Analysis

**Issue**: #83 — Security Hardening: Sandbox Verifier Runtime
**Filed by**: Patrol daemon (future-exploration sweep)
**Confidence**: 0.50 (below 0.8 threshold for autonomous fix)
**Status**: FALSE POSITIVE — no code changes needed

## Analysis

### Claim

> "Verifier packages run arbitrary code, presenting a critical RCE attack surface if a solver submits a malicious payload or a world generator contains an exploit. Relying solely on OS-level process isolation is insufficient..."

### Reality

After thorough code review of the entire `packages/verifier-runtime/` package and its integration points:

1. **The verifier-runtime does NOT run arbitrary code.** It is a collection of pure deterministic TypeScript functions:
- `normalize()` — recursively strip nulls, flatten objects, sort keys
- `diff()` — compute JSON structural differences (ops or merge-patch format)
- `apply()` — apply a patch to a source value
- `merge()` — merge two JSON objects with configurable conflict policy
- `migrate()` — schema migration with field mappings and type conversions
- `verify()` — hash comparison oracle

2. **No dynamic code execution exists in the runtime:**
- No `eval()`, `new Function()`, or `vm.runInNewContext()` calls
- No `child_process` spawning or subprocess execution
- No filesystem or network I/O
- No dynamic module loading or third-party plugin loading
- No arbitrary code is received from solvers — solvers submit JSON outputs, not executables

3. **The verifier is compiled library code**, not a plugin system. It is imported as a static dependency (`@fresharena/verifier-runtime`) and called synchronously.

4. **OS-level process isolation is already sufficient** for running deterministic pure functions on structured JSON data.

5. **The attacker model doesn't apply:** A solver submitting a "malicious payload" can only submit JSON values (objects, arrays, strings, numbers, booleans, null). These are transformed by pure functions with no side effects. There is no RCE vector.

### Root Cause of False Positive

The patrol daemon appears to have over-fitted on a generic "sandbox runtime" pattern without verifying whether the verifier-runtime actually executes untrusted code. The name "verifier-runtime" may suggest a code execution engine, but functionally it is a library of hash-verification functions.

### Future Consideration

If the architecture evolves to support **third-party verifier plugins** (e.g., a Verifier SDK that loads external WASM modules or JavaScript plugins), then sandboxing with `isolated-vm` or `wasmtime` would become relevant. This should be tracked as a roadmap item under Phase 3 or Phase 4, not as a current security gap.

## Decision

**CLOSE AS WON'T DO.** The finding is a false positive from the patrol daemon's over-fitting. No security sandbox migration is needed for the current architecture.

## Cross-Repo Notes

None needed — the issue is contained entirely within `WasmAgent/fresharena` and does not affect sibling repos.

---

*Filed by Claude Bot on behalf of the deliberation process per issue #83*
Loading