diff --git a/.env.example b/.env.example index d551470..ffa356f 100644 --- a/.env.example +++ b/.env.example @@ -22,7 +22,11 @@ FOREMAN_WORKSPACE_BRIDGE_URL= FOREMAN_WORKSPACE_ALLOWED_SCOPE= # JSON argv list; commands are run by Foreman in a disposable validation # workspace. A workspace workflow requires a source repo, bridge URL, allowed -# scope, and at least one configured command together. +# scope, and at least one configured command together. Commands run without +# network access unless they set "network": true (true or false only), for +# example [{"name":"Install","command":"pnpm","args":["install","--frozen-lockfile"],"network":true}]. +# Without the field, a pnpm/npm/yarn install (install, i, ci, add, or bare yarn) gets the +# network and every other command does not. FOREMAN_VALIDATION_COMMANDS=[] FOREMAN_VALIDATION_TIMEOUT_MS=120000 FOREMAN_VALIDATION_MAX_OUTPUT_BYTES=1048576 diff --git a/README.md b/README.md index f8dbdcc..be20d9c 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ Human ↔ Planner → Orchestrator → Worker → checks → Reviewer 3. **Worker.** The Worker edits an isolated workspace seeded from the pinned Git base. Foreman fetches the complete snapshot, byte-compares it against the pinned commit, enforces allowed scope, and computes the exact diff. 4. **Checks.** Foreman runs every configured validation command in a disposable workspace. An empty result (no file changes) fails the result gate even if commands pass. 5. **Reviewer.** The Reviewer reads the verified diff and check results in read-only mode. Its recommendation is advisory; it cannot modify the result. A `request_changes` or `reject` verdict triggers a bounded Orchestrator → Worker → Reviewer correction while budget remains. -6. **Your decision.** The "Ready for your review" panel shows a checklist, the checks pipeline (local Foreman checks and remote GitHub CI), and an Approve / Reject pair. Approval records an immutable decision bound to the evidence. It does not change Git. +6. **Your decision.** The "Ready for your review" panel shows a checklist, the checks pipeline (local Foreman checks and remote GitHub CI), and an Approve / Reject pair. Approval records an immutable decision bound to the evidence digest you reviewed. It does not change Git. 7. **Promote → GitHub.** Approved results can be promoted to a local commit, pushed, opened as a PR, and merged, each as a separate confirmed action. ## Quick start @@ -70,13 +70,17 @@ The checks pipeline in the review panel shows local Foreman results alongside re The Worker's files are untrusted, and validation runs them (`pnpm install` lifecycle scripts, test files). Foreman therefore runs every validation command inside a [bubblewrap](https://github.com/containers/bubblewrap) (`bwrap`) sandbox by default. On Linux, install it with `apt install bubblewrap` or `dnf install bubblewrap`. If `bwrap` is missing or cannot create a sandbox, validation fails with an error and does not run on the host. - **Hidden from the command:** your home directory (`/home`, `/root`, `$HOME`), Foreman's data directory, the source repository checkout, `/tmp`, `/var/tmp`, `/run` (Docker and D-Bus sockets, the user runtime directory), `/mnt`, `/media` and `/srv` are replaced by empty temporary directories. The rest of the host is mounted read-only. -- **Provided:** the workspace, read-write at `/tmp/workspace`; a fresh `HOME`; and an environment of only `PATH`, `LANG`, `LC_ALL`, `HOME`, `TMPDIR` and the cache variables below. The command has its own PID, IPC and UTS namespaces and no capabilities. A timeout or output overflow kills its whole process tree. +- **Provided:** the workspace, read-write at `/tmp/workspace`; a fresh `HOME`; and an environment of only `PATH`, `LANG`, `LC_ALL`, `HOME`, `TMPDIR` and the cache variables below. The command has its own PID, IPC, UTS and network namespaces and no capabilities. A timeout or output overflow kills its whole process tree. - **Toolchains under a hidden directory** (nvm, pnpm in `~/.local`) are mounted back read-only: `PATH` entries, the install prefix of the command and of `node` (when it is at least two levels below the home directory), and the paths in `FOREMAN_VALIDATION_SANDBOX_RO_PATHS`. Version managers that keep state beside their shims (Volta `~/.volta`, asdf `~/.asdf`, rustup `~/.rustup`, Homebrew on Linux) need their directory listed there. - **Package-manager cache:** `/validation-cache` (owner-only) is mounted read-write at `/tmp/foreman-cache`, with `XDG_CACHE_HOME`, `XDG_DATA_HOME`, `npm_config_cache`, `npm_config_store_dir` and `pnpm_config_store_dir` (pnpm 10 and 11), `YARN_CACHE_FOLDER` and `COREPACK_HOME` pointing into it, so installs do not download everything each time. It is shared by every validation and project. pnpm and npm verify package integrity against the lockfile, but a command that ran hostile code could still leave bad data behind, for example a swapped pnpm binary under `XDG_DATA_HOME`. Delete the directory if you suspect that. -- **Accepted risk, network access:** the sandbox keeps the host network because installs need it. A validation command can reach the internet, any service listening on localhost (including Foreman's own API) and the host's abstract-namespace Unix sockets, though it does not receive your proxy or credential variables. Do not treat the sandbox as a network boundary. +- **Network policy: none, except commands that explicitly need it.** Every validation command runs in an empty network namespace (`bwrap --unshare-net`). It has no interface except a working loopback, so tests that bind and connect to `127.0.0.1` inside the sandbox still work, but it cannot reach Foreman's own API, the per-project bridge or any other service on the host's loopback, the host's abstract-namespace Unix sockets, the cloud instance metadata endpoint, the local network or the internet, and DNS lookups fail. A hostile test therefore cannot drive Foreman's API for another run or fetch instance credentials. If bubblewrap cannot create the network namespace on your host (some containers refuse), the sandbox probe fails and validation reports the sandbox as unavailable; it never falls back to the host network. +- **The `network` flag.** Every validation command takes an optional boolean `network`: in `FOREMAN_VALIDATION_COMMANDS` (`{"name","command","args","cwd?","network?"}`), in a saved workspace setup, in the task-start command overrides, and as the per-command **Network** checkbox in the open-repository dialog and the task's advanced options. Only `true` keeps the host network for that command; only the JSON values `true` and `false` are accepted, anything else is rejected. Each check records what it got as `network` (with `sandbox: none` every check is recorded as `network: true`, since nothing isolates it), and the checks pipeline shows a **Network** badge on checks that ran with network access. +- **Which commands get the network by default.** The open-repository dialog starts each command from the repository inspector's suggestion: the dependency install (`pnpm install --frozen-lockfile`, `npm ci`) has `network: true`, and so do `cargo test` and `go test ./...` because they download dependencies on first run. Every other suggestion (`pnpm run test`, `pnpm run typecheck`, `python -m pytest`, ...) is offline; the install step puts `node_modules` in the shared validation workspace, so the checks after it need no network. A command that fetches tooling implicitly (for example pnpm downloading the version pinned in `packageManager`) fails offline and needs the checkbox. +- **Older setups and configs.** A command with no `network` field, in a saved workspace setup, in `FOREMAN_VALIDATION_COMMANDS` or in a task-start override, is treated as `network: true` if it is a recognised package-manager install and `network: false` otherwise. Recognised means the executable is exactly `pnpm`, `npm` or `yarn` with first argument `install`, `i`, `ci` or `add`, or bare `yarn` with no arguments. An explicit `network` always wins, so `{"command":"pnpm","args":["install","--offline"],"network":false}` stays offline. Runs stored before the flag existed are not rewritten (their approval is bound to a digest of the stored command list); a stored command without the field gets the same default when it runs, and the check records the effective value. +- **Residual risk, network-enabled commands.** A command with `network: true` shares the host network namespace, including the host's loopback services and the metadata endpoint. Package lifecycle scripts (`preinstall`, `postinstall`, `prepare`) run during a network-enabled install, so a hostile dependency or Worker-edited manifest gets the network for the length of that install. Grant `network` only to the install step, and consider `--ignore-scripts` if your dependencies allow it. The offline default protects every other command, including the tests. - Run Foreman as an unprivileged user. The sandbox hides paths but cannot stop a root process from reading root-only files that remain visible, such as `/etc/shadow`. -`FOREMAN_VALIDATION_SANDBOX=none` is an explicit, unsafe opt-out for hosts without bubblewrap (for example macOS). Worker-authored code then runs directly on your machine with your credentials. Foreman prints a warning at startup, marks each check with the mode it ran in, and reports `validationSandbox: {mode, available}` from `GET /api/status`. Foreman's own tests run validation through the sandbox, so `pnpm test` needs `bwrap` as well. +`FOREMAN_VALIDATION_SANDBOX=none` is an explicit, unsafe opt-out for hosts without bubblewrap (for example macOS). Worker-authored code then runs directly on your machine with your credentials and the host network, and the `network` flag has no effect. Foreman prints a warning at startup, marks each check with the mode it ran in, and reports `validationSandbox: {mode, available}` from `GET /api/status`. Foreman's own tests run validation through the sandbox, so `pnpm test` needs `bwrap` as well. ## Reviewing and approving results @@ -87,7 +91,7 @@ When a run reaches `awaiting_approval`, the **Ready for your review** panel appe - **Approve result** / **Reject** buttons. - A stepper: Review → Approve → Promote → Push → PR → Merge. -**Approve result** records an immutable decision bound to the evidence digest. It does not change Git. +**Approve result** records an immutable decision bound to the evidence digest you reviewed: the panel loads the digest for the evidence it displays (`GET /api/runs/:id/decision`) and sends it with Approve or Reject. If the evidence changed after that (a validation retry, a correction cycle, another tab), Foreman refuses the decision with a message asking you to review the current result. It does not change Git. **Promote approved result** creates a local commit rooted at the pinned base and a result branch. Your checkout is unchanged. @@ -140,7 +144,7 @@ The standard local flow requires no environment variables. The following setting | `FOREMAN_WORKSPACE_SOURCE_REPO` | — | Local Git repository for snapshot verification. Required with the manual bridge workflow. | | `FOREMAN_WORKSPACE_BRIDGE_URL` | — | Loopback URL for an external workspace bridge. | | `FOREMAN_WORKSPACE_ALLOWED_SCOPE` | — | Comma-separated exact paths or directory prefixes ending in `/` allowed in Worker results. | -| `FOREMAN_VALIDATION_COMMANDS` | — | JSON array of `{"name","command","args","cwd?"}` entries run in the disposable validation workspace. | +| `FOREMAN_VALIDATION_COMMANDS` | — | JSON array of `{"name","command","args","cwd?","network?"}` entries run in the disposable validation workspace. `network` is `true` or `false`; validation runs offline unless it is `true`, and a package-manager install without the field defaults to `true`. See [Validation sandbox](#validation-sandbox). | | `FOREMAN_VALIDATION_TIMEOUT_MS` | `120000` | Per-command time limit (max 600,000 ms). | | `FOREMAN_VALIDATION_MAX_OUTPUT_BYTES` | `1048576` | Per-command output capture bound (max 16 MiB). | | `FOREMAN_VALIDATION_SANDBOX` | `bwrap` | `bwrap` runs every validation command in a bubblewrap sandbox and fails if it is unavailable. `none` runs them directly on the host with your credentials, and is unsafe. See [Validation sandbox](#validation-sandbox). | diff --git a/config.schema.json b/config.schema.json index 3610621..b7f72c3 100644 --- a/config.schema.json +++ b/config.schema.json @@ -17,7 +17,7 @@ "FOREMAN_WORKSPACE_SOURCE_REPO": { "type": "string", "description": "Local Git source repository used for pinned Worker snapshot verification" }, "FOREMAN_WORKSPACE_BRIDGE_URL": { "type": "string", "format": "uri", "description": "Loopback URL for the external workspace bridge" }, "FOREMAN_WORKSPACE_ALLOWED_SCOPE": { "type": "string", "description": "Comma-separated exact paths or directory prefixes ending in / allowed for Worker changes" }, - "FOREMAN_VALIDATION_COMMANDS": { "type": "string", "description": "JSON array of validation command objects with explicit command and args" }, + "FOREMAN_VALIDATION_COMMANDS": { "type": "string", "description": "JSON array of validation command objects {name, command, args, cwd?, network?} with explicit command and args. network is true or false (no other value is accepted): validation commands run without network access unless it is true. When omitted, a pnpm, npm or yarn install (first argument install, i, ci or add, or bare yarn) defaults to true and every other command to false" }, "FOREMAN_VALIDATION_TIMEOUT_MS": { "type": "integer", "minimum": 1, "maximum": 600000, "default": 120000 }, "FOREMAN_VALIDATION_MAX_OUTPUT_BYTES": { "type": "integer", "minimum": 1, "maximum": 16777216, "default": 1048576 }, "FOREMAN_VALIDATION_SANDBOX": { "type": "string", "enum": ["bwrap", "none"], "default": "bwrap", "description": "bwrap runs every validation command inside a bubblewrap sandbox and fails closed when it is unavailable; none runs them unsandboxed on the host (unsafe opt-out)" }, diff --git a/docs/architecture.md b/docs/architecture.md index 06b5682..c29b38f 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -22,4 +22,4 @@ When the Orchestrator returns an empty or soft-denied response (e.g. AGY invokin ## Approval and Git promotion -Human approval records an immutable decision bound to the evidence digest and pinned input base. Its `evidenceCommit` names the pinned base; it is not the result commit. Promotion is a separate explicit operation. Before creating a result commit, the controller rechecks the stored snapshot bytes and modes, allowed scope, pinned base, validation record, and Reviewer binding. It builds the tree in an isolated worktree, commits with the pinned base as parent, then compares the complete result tree to the approved snapshot. Only that verified result commit and tree are reported as applied. A failed or interrupted attempt remains distinct from the human decision and can be retried idempotently; conflicting branch state is rejected. +Human approval records an immutable decision bound to the evidence digest and pinned input base. The digest is the one the operator reviewed: `GET /api/runs/:id/decision` returns it with the Worker response, validation, and Reviewer recommendation IDs it covers (only at a decision checkpoint), and the approve/reject request must carry it as `evidenceDigest`. `approveRun` recomputes the digest inside the same store mutation and refuses a missing or malformed value (400) or a stale one (409), so an approval or rejection never applies to evidence the operator did not see. Its `evidenceCommit` names the pinned base; it is not the result commit. Promotion is a separate explicit operation. Before creating a result commit, the controller rechecks the stored snapshot bytes and modes, allowed scope, pinned base, validation record, and Reviewer binding. It builds the tree in an isolated worktree, commits with the pinned base as parent, then compares the complete result tree to the approved snapshot. Only that verified result commit and tree are reported as applied. A failed or interrupted attempt remains distinct from the human decision and can be retried idempotently; conflicting branch state is rejected. diff --git a/docs/workspace-bridge.md b/docs/workspace-bridge.md index 65b0201..07d3bbd 100644 --- a/docs/workspace-bridge.md +++ b/docs/workspace-bridge.md @@ -108,7 +108,7 @@ The controller materializes the verified diff in a disposable workspace from the 2. Create the project, task, and run, select the Worker/Reviewer harnesses and models, then pin and seed the run with `POST /api/runs/{runId}/workspace` and `{ "baseCommit": "" }`. 3. Submit the Worker with `POST /api/runs/{runId}/assignments` and `{ "roleId": "worker", "prompt": "..." }`. After its assignment succeeds, call `POST /api/runs/{runId}/worker-evidence` with `{ "assignmentId": "" }`. Foreman retrieves the Worker response and bridge snapshot, verifies the exact pinned base/scope, materializes the verified output, and runs the configured checks. 4. After controller validation passes, submit the Reviewer through `POST /api/runs/{runId}/reviewer`. The bridge opens a fresh review context with no `workspace_id` or checkout; Foreman sends only the exact verified diff and controller-observed validation evidence. Claude has an empty tool allowlist. Codex uses `--sandbox read-only`, which may retain read-only shell tools; this Codex boundary was fixture-tested only. The recorded live Claude response is in [actual-reviewer-smoke.json](../investigations/local-cli-uhp/evidence/actual-reviewer-smoke.json); the separate checked-in integration fixture labels its Reviewer `simulated_fixture` and is not live model evidence. Inspect rationale, requested and actual model, response/session IDs, measured usage, provenance, and prior recommendation history in the UI. -5. Make the human decision in the UI or with `POST /api/runs/{runId}/approve` and `{ "approved": true|false, "rationale": "..." }`. Foreman rejects approval unless snapshot/scope verification, configured validation, and a Reviewer recommendation are present. The decision records `evidenceCommit` as the pinned input base; it does not create a result commit. +5. Make the human decision in the UI, or over HTTP in two steps. First review the evidence and fetch its digest with `GET /api/runs/{runId}/decision`; it returns `{ "runId", "evidenceDigest", "pinnedBaseCommit", "workerResponseId", "validationId", "recommendationId" }` for a run at a decision checkpoint (controller inactive, verified evidence, passing validation, a Reviewer recommendation, no decision yet), and answers 409 when the run is not at one. Then send `POST /api/runs/{runId}/approve` with `{ "approved": true|false, "evidenceDigest": "", "rationale": "..." }`; `evidenceDigest` is required for both approval and rejection, `rationale` is optional, and a missing or malformed digest is a 400. If the evidence changed after that digest was issued, Foreman answers 409 and records nothing; re-fetch the digest, review the current result, and decide again. The UI fetches the digest when it renders the decision panel, not when you click, so the decision is bound to the evidence you were looking at. Foreman also rejects approval unless snapshot/scope verification, configured validation, and a Reviewer recommendation are present. The decision records `evidenceCommit` as the pinned input base; it does not create a result commit. 6. After an approved decision, explicitly request Git promotion in the UI or with `POST /api/runs/{runId}/promotion` and optional `{ "destinationBranch": "..." }`. Inspect separate promotion status and result commit/tree. A retry reuses the operation identity and refuses changed evidence, base drift, missing Git objects, and a conflicting destination branch. A human rejection is terminal for that run. To retry failed validation, use `POST /api/runs/{runId}/validation/retry`. To authorize a Reviewer retry after a non-recommending result, use `POST /api/runs/{runId}/reviewer/retry`, then submit a new Reviewer request. Previous recommendation records remain inspectable. A new Worker attempt requires a new run and workspace seeded from its pinned base. The previously approved live smoke run has not been promoted. Its legacy decision lacks an evidence digest and cannot pass the current promotion binding check; a new bound decision or explicit migration would be required, and neither is performed here. The checked-in Claude Worker smoke response can be loaded through Foreman's recorded evidence import path. Foreman verifies its exact response/snapshot bytes against the full pinned Git base and recomputes scope and diff before storing it as `recorded_live_import`; importing the evidence does not invoke the Worker. The integration test separately replays the sanitized evidence as `recorded_replay` and uses its own deterministic simulated Reviewer. Neither fixture is a live model run, and neither the smoke record nor fixture is approved automatically. diff --git a/src/config.ts b/src/config.ts index cba0aa4..bccf843 100644 --- a/src/config.ts +++ b/src/config.ts @@ -1,7 +1,7 @@ import { existsSync } from 'node:fs'; import { isAbsolute, resolve } from 'node:path'; import { loadEnvFile } from 'node:process'; -import { parseSandboxMode, type ValidationSandboxMode } from './validation-sandbox.js'; +import { defaultNetworkAccess, parseSandboxMode, type ValidationSandboxMode } from './validation-sandbox.js'; export interface ForemanConfig { host: string; @@ -17,7 +17,7 @@ export interface ForemanConfig { workspaceSourceRepo?: string; workspaceBridgeUrl?: string; workspaceAllowedScope: string[]; - validationCommands: Array<{name:string;command:string;args:string[];cwd?:string}>; + validationCommands: Array<{name:string;command:string;args:string[];cwd?:string;network:boolean}>; validationTimeoutMs: number; validationMaxOutputBytes: number; validationSandbox: { mode: ValidationSandboxMode; roPaths: string[]; dataDir: string; cacheDir: string }; @@ -52,7 +52,7 @@ export function loadConfig(): ForemanConfig { if (Boolean(uhpHarnessId) !== Boolean(uhpModel)) throw new Error('UHP_HARNESS_ID and UHP_MODEL must be configured together'); let validationCommands: ForemanConfig['validationCommands'] = []; const rawCommands=process.env.FOREMAN_VALIDATION_COMMANDS?.trim(); - if(rawCommands){try{const value=JSON.parse(rawCommands);if(!Array.isArray(value))throw new Error();validationCommands=value.map((item:unknown)=>{if(!item||typeof item!=='object')throw new Error();const x=item as Record;if(typeof x.name!=='string'||!x.name.trim()||typeof x.command!=='string'||!x.command||!Array.isArray(x.args)||x.args.some(arg=>typeof arg!=='string')||(x.cwd!==undefined&&typeof x.cwd!=='string'))throw new Error();return {name:x.name,command:x.command,args:x.args as string[],...(typeof x.cwd==='string'?{cwd:x.cwd}:{})};});}catch{throw new Error('FOREMAN_VALIDATION_COMMANDS must be a JSON array of {name,command,args,cwd?}');}} + if(rawCommands){try{const value=JSON.parse(rawCommands);if(!Array.isArray(value))throw new Error();validationCommands=value.map((item:unknown)=>{if(!item||typeof item!=='object')throw new Error();const x=item as Record;if(typeof x.name!=='string'||!x.name.trim()||typeof x.command!=='string'||!x.command||!Array.isArray(x.args)||x.args.some(arg=>typeof arg!=='string')||(x.cwd!==undefined&&typeof x.cwd!=='string')||(x.network!==undefined&&typeof x.network!=='boolean'))throw new Error();return {name:x.name,command:x.command,args:x.args as string[],...(typeof x.cwd==='string'?{cwd:x.cwd}:{}),network:(x.network as boolean|undefined)??defaultNetworkAccess(x.command,x.args as string[])};});}catch{throw new Error('FOREMAN_VALIDATION_COMMANDS must be a JSON array of {name,command,args,cwd?,network?} where network is true or false');}} const workspaceAllowedScope=(process.env.FOREMAN_WORKSPACE_ALLOWED_SCOPE??'').split(',').map(x=>x.trim()).filter(Boolean); const sandboxMode = parseSandboxMode(process.env.FOREMAN_VALIDATION_SANDBOX); if (!sandboxMode) throw new Error('FOREMAN_VALIDATION_SANDBOX must be "bwrap" or "none"'); diff --git a/src/controller.ts b/src/controller.ts index ab9897b..a3d65a0 100644 --- a/src/controller.ts +++ b/src/controller.ts @@ -16,7 +16,7 @@ import type { WorkerEvidence, ValidationCheck } from './domain.js'; import { buildRepoDigest, extractKeywords } from './repo-digest.js'; import { promoteSnapshotToGit } from './git-promotion.js'; import { parseCiScripts, ciChecksNotConfigured as computeCiChecksNotConfigured } from './repository-inspector.js'; -import { validationSandboxStatus, type ValidationSandboxConfig } from './validation-sandbox.js'; +import { defaultNetworkAccess, validationSandboxStatus, type ValidationSandboxConfig } from './validation-sandbox.js'; const execFileAsync=promisify(execFile); export interface UhpAdapter { @@ -871,7 +871,16 @@ export class Controller { async recordReviewerRecommendation(runId:string,reviewerAssignmentId:string,provenance:'uhp_response'|'simulated_fixture'='uhp_response',authorization?:symbol):Promise{if(this.findRun(await this.store.load(),runId).controller?.active&&authorization!==AUTOMATIC_RUN)throw Object.assign(new Error('Reviewer recommendation recording is controlled by the active run'),{statusCode:409});try{return await this.store.mutate(s=>{const run=this.findRun(s,runId),reviewer=must(run.assignments.find(a=>a.id===reviewerAssignmentId),'Reviewer assignment');if(!run.workerEvidence?.scopeVerified||run.validation?.status!=='passed')throw Object.assign(new Error('Verified Worker evidence and passing validation are required'),{statusCode:409});if(provenance==='simulated_fixture'&&!['recorded_replay','recorded_live_import'].includes(run.workerEvidence.provenance))throw Object.assign(new Error('Simulated Reviewer provenance is reserved for deterministic recorded fixtures'),{statusCode:409});if(reviewer.roleId!=='reviewer'||reviewer.status!=='succeeded'||!reviewer.responseId||!reviewer.sessionId||!reviewerAssignmentModelEvidence(reviewer)||reviewer.reviewerExecution?.mode!=='read_only'||reviewer.reviewerExecution.mutationAttempted||!this.reviewerAssignmentMatchesEvidence(run,reviewer))throw Object.assign(new Error('Reviewer response lacks complete model-reporting, session, or read-only execution evidence'),{statusCode:409});if(reviewer.responseId===run.workerEvidence.responseId||reviewer.sessionId===run.assignments.find(a=>a.id===run.workerEvidence?.workerAssignmentId)?.sessionId)throw Object.assign(new Error('Reviewer response and session IDs must be distinct from the Worker'),{statusCode:409});if(run.reviewerRecommendation)throw Object.assign(new Error('Reviewer recommendation already recorded'),{statusCode:409});const raw=typeof reviewer.result==='string'?reviewer.result:JSON.stringify(reviewer.result??'');let parsed:unknown;try{parsed=JSON.parse(raw);}catch{}const value=parsed&&typeof parsed==='object'?parsed as Record:{};const verdict=['recommend','request_changes','reject'].includes(String(value.verdict))?value.verdict as 'recommend'|'request_changes'|'reject':undefined;const rationale=(typeof value.rationale==='string'?value.rationale:'').trim();if(!verdict||!rationale)throw Object.assign(new Error('Reviewer response must contain a supported verdict and non-empty rationale'),{statusCode:422});const recommendation={id:id('recommendation'),status:'proposed' as const,provenance,reviewerAssignmentId:reviewer.id,harnessId:reviewer.selectedHarnessId??reviewer.actualConfig?.harnessId??reviewer.requestedConfig.harnessId,model:reviewer.requestedConfig.model,...(reviewer.actualModelStatus==='observed'&&reviewer.actualConfig?.model?{actualModel:reviewer.actualConfig.model}:{}),responseId:reviewer.responseId,sessionId:reviewer.sessionId,...(reviewer.usage?.measured?{usage:structuredClone(reviewer.usage)}:{}),reviewMode:'read_only' as const,mutationAttempted:false as const,verdict,rationale,createdAt:now()};run.reviewerRecommendation=recommendation;run.reviewerRecommendationHistory??=[];run.reviewerRecommendationHistory.push(structuredClone(recommendation));run.status='awaiting_approval';s.events.push(event('reviewer.recommendation_recorded','run',runId,{recommendationId:recommendation.id,reviewerAssignmentId:reviewer.id,responseId:reviewer.responseId,sessionId:reviewer.sessionId,harnessId:recommendation.harnessId,model:recommendation.model,actualModel:recommendation.actualModel,usage:recommendation.usage,verdict,provenance,reviewMode:recommendation.reviewMode,mutationAttempted:false}));return recommendation;});}catch(error){await this.store.mutate(s=>{const run=this.findRun(s,runId),reviewer=run.assignments.find(a=>a.id===reviewerAssignmentId);if(reviewer?.roleId==='reviewer'&&reviewer.status==='succeeded'&&!run.reviewerRecommendation){reviewer.status='failed';reviewer.error=error instanceof Error?error.message:String(error);s.events.push(event('reviewer.execution_rejected','assignment',reviewer.id,{responseId:reviewer.responseId,error:reviewer.error,source:'recommendation-validation'}));}}).catch(()=>undefined);throw error;}} async addReview(runId:string,input:{reviewerAssignmentId:string;implementationAssignmentIds:string[];verdict:'clear'|'changes_requested'|'rejected';scope:string[];summary:string}):Promise{return this.store.mutate(s=>{const run=this.findRun(s,runId),reviewer=must(run.assignments.find(a=>a.id===input.reviewerAssignmentId),'Reviewer assignment');if(reviewer.roleId!=='reviewer'||reviewer.status!=='succeeded')throw new Error('Reviewer proposal requires a completed Reviewer assignment');if(!input.implementationAssignmentIds.length||input.implementationAssignmentIds.includes(reviewer.id))throw new Error('Reviewer cannot approve its own implementation');for(const assignmentId of input.implementationAssignmentIds){const impl=must(run.assignments.find(a=>a.id===assignmentId),'Implementation assignment');if(impl.roleId!=='worker'||impl.status!=='succeeded')throw new Error('Review scope must contain completed Worker assignments');}const implemented=this.currentSuccessfulWorkers(run).map(a=>a.id).sort();if(JSON.stringify(implemented)!==JSON.stringify([...input.implementationAssignmentIds].sort()))throw new Error('Reviewer proposal must cover every completed Worker assignment');if(!['clear','changes_requested','rejected'].includes(input.verdict)||!input.scope.length)throw new Error('Review verdict and scope are required');const review={id:id('review'),status:'proposed' as const,reviewerAssignmentId:reviewer.id,implementationAssignmentIds:[...input.implementationAssignmentIds],verdict:input.verdict,scope:input.scope.map(x=>x.trim()).filter(Boolean),summary:input.summary.trim(),createdAt:now()};run.reviews.push(review);run.status='review';s.events.push(event('review.proposed','review',review.id,{runId,verdict:review.verdict,implementationAssignmentIds:review.implementationAssignmentIds,authority:'unverified-agent-proposal'}));return review;});} async recordValidation(runId:string,input:{passed:boolean;checks:Array<{name:string;passed:boolean;details?:string}>;gitEvidence?:{commit?:string;tree?:string;changedPaths?:string[]}}):Promise{return this.store.mutate(s=>{const run=this.findRun(s,runId);if(run.workerEvidence||run.validation?.observations)throw Object.assign(new Error('Client-reported validation cannot overwrite controller-observed evidence'),{statusCode:409});if(!run.reviews.some(r=>r.status==='verified'&&r.verdict==='clear'))throw Object.assign(new Error('A controller-verified clear reviewer verdict is required before validation'),{statusCode:409});if(!Array.isArray(input.checks)||!input.checks.length||input.checks.some(c=>!c.name||typeof c.passed!=='boolean'))throw new Error('Validation requires named check results');const checks=structuredClone(input.checks),reportedPassed=input.passed&&checks.every(c=>c.passed);const record={id:id('valid'),status:'unverified' as const,passed:false,reportedPassed,checks,createdAt:now(),...(input.gitEvidence?{gitEvidence:{status:'unverified' as const,...input.gitEvidence,submittedAt:now()}}:{})};run.validation=record;run.status='validation';s.events.push(event('validation.reported','validation',record.id,{runId,reportedPassed,authority:'unverified-client-report',gitEvidenceStatus:record.gitEvidence?.status??'missing'}));return record;});} - async approveRun(runId:string,input:{approved:boolean;evidenceCommit?:string;rationale?:string}):Promise{ + /** The evidence a human is about to decide on. approveRun refuses any decision that does not present this exact digest, so the decision binds to what the operator reviewed, not to whatever exists at click time. */ + async decisionEvidence(runId:string):Promise<{runId:string;evidenceDigest:string;pinnedBaseCommit:string;workerResponseId:string;validationId:string;recommendationId:string}>{ + const run=this.findRun(await this.store.load(),runId); + if(run.controller?.active)throw Object.assign(new Error('Human acceptance is available after the controller reaches a decision checkpoint'),{statusCode:409}); + if(run.approval)throw Object.assign(new Error('A human decision is final for this run'),{statusCode:409}); + if(!run.workerEvidence?.scopeVerified||run.workerEvidence.acceptance!=='not_decided'||!run.validation||run.validation.status!=='passed'||!run.validation.observations?.length||!run.reviewerRecommendation)throw Object.assign(new Error('Human decision requires verified Worker scope, passing controller-observed validation, and a recorded independent Reviewer recommendation'),{statusCode:409}); + return {runId:run.id,evidenceDigest:this.promotionEvidenceDigest(run),pinnedBaseCommit:run.workerEvidence.pinnedBaseCommit,workerResponseId:run.workerEvidence.responseId,validationId:run.validation.id,recommendationId:run.reviewerRecommendation.id}; + } + async approveRun(runId:string,input:{approved:boolean;evidenceDigest:string;evidenceCommit?:string;rationale?:string}):Promise{ + if(typeof input?.evidenceDigest!=='string'||!/^[0-9a-f]{64}$/i.test(input.evidenceDigest))throw Object.assign(new Error('The evidence digest you reviewed is required to record a decision (64 hexadecimal characters from the decision evidence)'),{statusCode:400}); const approval=await this.store.mutate(s=>{ const context=this.findRunContext(s,runId),run=context.run; if(run.controller?.active)throw Object.assign(new Error('Human acceptance is available after the controller reaches a decision checkpoint'),{statusCode:409}); @@ -887,6 +896,7 @@ export class Controller { if(!run.pinnedBaseCommit||run.workerEvidence.pinnedBaseCommit!==run.pinnedBaseCommit||completedWorkers.length!==1||completedWorkers[0]?.id!==run.workerEvidence.workerAssignmentId||completedWorkers[0]?.responseId!==run.workerEvidence.responseId)throw Object.assign(new Error('Human decision requires evidence bound to the run pinned base and its sole completed Worker response'),{statusCode:409}); if(input.evidenceCommit&&input.evidenceCommit!==run.workerEvidence.pinnedBaseCommit)throw Object.assign(new Error('Decision evidence commit must match the pinned Worker base'),{statusCode:409}); const evidenceDigest=this.promotionEvidenceDigest(run); + if(input.evidenceDigest.toLowerCase()!==evidenceDigest)throw Object.assign(new Error('The evidence changed since you reviewed it; refresh and review the current result before deciding.'),{statusCode:409}); const accepted={id:id('approval'),approved:input.approved,decision:input.approved?'approved' as const:'rejected' as const,evidenceCommit:run.workerEvidence.pinnedBaseCommit,evidenceDigest,rationale:input.rationale?.trim(),createdAt:now()};run.approval=accepted;run.workerEvidence.acceptance=input.approved?'accepted':'rejected';run.status=input.approved?'completed':'failed'; if(input.approved)run.promotion={status:'not_started',evidenceDigest,updatedAt:now()}; s.events.push(event(input.approved?'run.human_approved':'run.human_rejected','run',run.id,{approvalId:accepted.id,recommendationId:run.reviewerRecommendation.id,decision:accepted.decision,evidenceDigest})); @@ -1200,7 +1210,7 @@ function parseProjectPlannerReply(raw:string):{reply?:string;tasks?:unknown[];pr if(!Array.isArray(parsed.tasks))return {reply:typeof parsed.reply==='string'?parsed.reply.slice(0,8000):prefix.slice(0,8000)||'Planner proposed tasks, but the reply field was missing.',proposalError:'Planner task proposal must contain a tasks array; no tasks were created.'}; return {reply:typeof parsed.reply==='string'?parsed.reply.slice(0,8000):prefix.slice(0,8000)||'Proposed tasks are ready for review.',tasks:parsed.tasks}; } -function normalizeValidationCommands(input:unknown):Array<{name:string;command:string;args:string[];cwd?:string}>{if(!Array.isArray(input)||!input.length||input.length>20)throw Object.assign(new Error('At least one and at most 20 validation commands are required'),{statusCode:422});const names=new Set();return input.map((raw,i)=>{if(!raw||typeof raw!=='object')throw Object.assign(new Error(`Validation command ${i+1} is invalid`),{statusCode:422});const x=raw as Record;if(typeof x.name!=='string'||!x.name.trim()||x.name.length>120||names.has(x.name.trim())||typeof x.command!=='string'||!x.command.trim()||x.command.length>1024||x.command.includes('\0')||!Array.isArray(x.args)||x.args.some(a=>typeof a!=='string'||a.length>4096||a.includes('\0'))||x.args.length>128)throw Object.assign(new Error(`Validation command ${i+1} has invalid name, command, or arguments`),{statusCode:422});let cwd:string|undefined;if(x.cwd!==undefined){if(typeof x.cwd!=='string'||!x.cwd||x.cwd.startsWith('/')||x.cwd.includes('\\')||x.cwd.includes('\0')||x.cwd.split('/').some(part=>!part||part==='.'||part==='..'))throw Object.assign(new Error(`Validation command ${i+1} cwd must be a safe relative directory`),{statusCode:422});cwd=x.cwd;}names.add(x.name.trim());return {name:x.name.trim(),command:x.command.trim(),args:[...(x.args as string[])],...(cwd?{cwd}:{})};});} +export function normalizeValidationCommands(input:unknown):Array<{name:string;command:string;args:string[];cwd?:string;network:boolean}>{if(!Array.isArray(input)||!input.length||input.length>20)throw Object.assign(new Error('At least one and at most 20 validation commands are required'),{statusCode:422});const names=new Set();return input.map((raw,i)=>{if(!raw||typeof raw!=='object')throw Object.assign(new Error(`Validation command ${i+1} is invalid`),{statusCode:422});const x=raw as Record;if(typeof x.name!=='string'||!x.name.trim()||x.name.length>120||names.has(x.name.trim())||typeof x.command!=='string'||!x.command.trim()||x.command.length>1024||x.command.includes('\0')||!Array.isArray(x.args)||x.args.some(a=>typeof a!=='string'||a.length>4096||a.includes('\0'))||x.args.length>128)throw Object.assign(new Error(`Validation command ${i+1} has invalid name, command, or arguments`),{statusCode:422});let cwd:string|undefined;if(x.cwd!==undefined){if(typeof x.cwd!=='string'||!x.cwd||x.cwd.startsWith('/')||x.cwd.includes('\\')||x.cwd.includes('\0')||x.cwd.split('/').some(part=>!part||part==='.'||part==='..'))throw Object.assign(new Error(`Validation command ${i+1} cwd must be a safe relative directory`),{statusCode:422});cwd=x.cwd;}if(x.network!==undefined&&typeof x.network!=='boolean')throw Object.assign(new Error(`Validation command ${i+1} network must be true or false`),{statusCode:422});const command=x.command.trim(),args=[...(x.args as string[])];names.add(x.name.trim());return {name:x.name.trim(),command,args,...(cwd?{cwd}:{}),network:(x.network as boolean|undefined)??defaultNetworkAccess(command,args)};});} function pathsOverlap(a:string,b:string):boolean{const left=a.replace(/\\/g,'/'),right=b.replace(/\\/g,'/'),ld=left.endsWith('/'),rd=right.endsWith('/'),lb=left.replace(/\*.*$/,'').replace(/\/$/,''),rb=right.replace(/\*.*$/,'').replace(/\/$/,'');if(lb===rb)return true;if(left.includes('*'))return right===lb||right.startsWith(`${lb}/`)||lb.startsWith(`${rb}/`);if(right.includes('*'))return left===rb||left.startsWith(`${rb}/`)||rb.startsWith(`${lb}/`);if(ld)return right===lb||right.startsWith(`${lb}/`);if(rd)return left===rb||left.startsWith(`${rb}/`);return left.startsWith(`${right}/`)||right.startsWith(`${left}/`);} function scopeContainedBy(path:string,allowed:string[]):boolean{return allowed.some(root=>{const dir=root.endsWith('/'),wild=root.endsWith('*'),a=root.replace(/\*$/,'').replace(/\/$/,''),p=path.replace(/\/$/,'');return p===a||wild&&p.startsWith(a)||dir&&p.startsWith(`${a}/`);});} function expandPlannerPaths(paths:string[],allowedScope:string[]):string[]{const expanded:string[]=[];for(const path of paths){if(!path.includes('*')){expanded.push(path);continue;}const segments=path.split('/'),wildcardSegments=segments.map((segment,index)=>segment.includes('*')?index:-1).filter(index=>index>=0);if(segments.some(segment=>segment.includes('*')&&segment!=='*'&&segment!=='**')||wildcardSegments.some(index=>segments[index]==='*'&&index!==segments.length-1)||wildcardSegments.some(index=>segments[index]==='**'&&index!==0&&index!==segments.length-1))throw Object.assign(new Error(`Planner path suggestion cannot be safely narrowed: ${path}`),{statusCode:422});let concrete=path.replace(/\*+$/,'');if(concrete.endsWith('/'))concrete=concrete.slice(0,-1);if(concrete.startsWith('**/'))concrete=concrete.slice(3);if(concrete.includes('*'))throw Object.assign(new Error(`Planner path suggestion cannot be safely narrowed: ${path}`),{statusCode:422});if(!concrete)throw Object.assign(new Error(`Planner path suggestion cannot be safely narrowed: ${path}`),{statusCode:422});const normalized=`${concrete}/`,matchingRoots=path.startsWith('**/')?allowedScope.filter(root=>{const value=root.replace(/\/$/,'');return value===concrete||value.endsWith(`/${concrete}`);}):[],candidates=matchingRoots.length?matchingRoots:[normalized];for(const candidate of candidates){if(scopeContainedBy(candidate,allowedScope)){expanded.push(candidate);continue;}for(const root of allowedScope)if(scopeContainedBy(root,[candidate]))expanded.push(root);}}return [...new Set(expanded)];} diff --git a/src/domain.ts b/src/domain.ts index ca21ffb..078253f 100644 --- a/src/domain.ts +++ b/src/domain.ts @@ -11,7 +11,7 @@ export interface Role { id: string; name: string; kind: RoleKind; enabled: boole export interface ProjectPlannerMessage { id:string; role:'user'|'planner'; text:string; createdAt:string; assignmentId?:string; createdTaskIds?:string[] } export interface ProjectPlannerSession extends RoleSession { projectId:string } export interface TaskSpec { title:string; goal:string; suggestedAllowedPaths:string[]; validationCriteria:string[]; dependsOn?:string[] } -export interface ApprovedTaskPlan { scope:string[]; roleConfigs:Record; validationCommands:Array<{name:string;command:string;args:string[];cwd?:string}>; budgets:RunBudget; baseCommit?:string } +export interface ApprovedTaskPlan { scope:string[]; roleConfigs:Record; validationCommands:Array<{name:string;command:string;args:string[];cwd?:string;network?:boolean}>; budgets:RunBudget; baseCommit?:string } export interface TaskPlanApproval { status:'approved'; approvedAt:string; specDigest:string; plan:ApprovedTaskPlan; runId:string } export interface Project { id: string; name: string; status: ProjectStatus; defaultRoleConfigs: Record; roleConfigOverridesMigrated?: true; createdAt: string; tasks: Task[]; plannerSession?:ProjectPlannerSession; plannerSessionHistory?:ProjectPlannerSession[]; plannerMessages?:ProjectPlannerMessage[]; plannerAssignments?:Assignment[]; usage?:Usage; usageByHarnessModel?:Record } export interface Task { id: string; title: string; goal?:string; suggestedAllowedPaths?:string[]; validationCriteria?:string[]; dependsOn?:string[]; planApproval?:TaskPlanApproval; status: TaskStatus; createdAt: string; runs: Run[] } @@ -19,13 +19,13 @@ export interface RoleSession { localId: string; roleId: 'planner'|'orchestrator' export interface ReviewRecord { id: string; status:'proposed'|'verified'; reviewerAssignmentId: string; implementationAssignmentIds: string[]; verdict: 'clear'|'changes_requested'|'rejected'; scope: string[]; summary: string; createdAt: string } export interface WorkerEvidence { provenance:'bridge_snapshot'|'recorded_replay'|'recorded_live_import'; workerAssignmentId:string; responseId:string; requestedModel?:string; actualModel?:string; actualModelStatus?:'observed'|'unavailable'; cliInvocation?:{executable:string;hostExecutable?:string;args:string[]}; usage?:Usage; workspaceId?:string; pinnedBaseCommit:string; completeSnapshot:{reportedComplete:true;reportedErrors:0;entryCount:number}; scopeVerified:true; allowedScope:string[]; entries:import('./workspace-snapshot.js').SnapshotEntry[]; changes:import('./workspace-snapshot.js').SnapshotChange[]; reviewDiff:string; acceptance:'not_decided'|'accepted'|'rejected' } export interface GitEvidence { status: 'unverified'|'verified'; commit?: string; tree?: string; changedPaths?: string[]; submittedAt: string } -export interface ValidationCheck { name:string; command:string; args:string[]; exitCode:number|null; signal?:string; timedOut:boolean; output:string; outputTruncated:boolean; startedAt:string; finishedAt:string; passed:boolean; sandbox?:'bwrap'|'none' } +export interface ValidationCheck { name:string; command:string; args:string[]; exitCode:number|null; signal?:string; timedOut:boolean; output:string; outputTruncated:boolean; startedAt:string; finishedAt:string; passed:boolean; sandbox?:'bwrap'|'none'; network?:boolean } export interface ValidationRecord { id: string; status:'unverified'|'passed'|'failed'; passed: boolean; reportedPassed:boolean; checks: Array<{name:string;passed:boolean;details?:string}>; observations?:ValidationCheck[]; resultGate?:{passed:boolean;reason:string}; policy?:{requireAllChecksPass:true;configuredCheckCount:number;commandDigest?:string}; gitEvidence?: GitEvidence; createdAt: string } export interface ReviewerRecommendation { id:string; status:'proposed'; provenance:'uhp_response'|'simulated_fixture'; reviewerAssignmentId:string; harnessId:string; model:string; actualModel?:string; responseId:string; sessionId?:string; usage?:Usage; reviewMode?:'read_only'; mutationAttempted?:false; verdict:'recommend'|'request_changes'|'reject'; rationale:string; createdAt:string } export interface ApprovalRecord { id: string; approved: boolean; decision:'approved'|'rejected'; evidenceCommit?:string; evidenceDigest?:string; rationale?:string; createdAt: string } export interface PromotionRecord { status:'not_started'|'promoting'|'applied'|'failed'|'abandoned'; operationId?:string; evidenceDigest?:string; destinationBranch?:string; resultCommit?:string; resultTree?:string; error?:string; updatedAt:string } export interface PrDraft { title: string; body: string; source: 'planner'|'template'|'edited'; generatedAt: string; assignmentId?: string; fallbackReason?: string } -export interface Run { id: string; status: RunStatus; createdAt: string; allowedScope?:string[]; baseFilePaths?:string[]; validationCommands?:Array<{name:string;command:string;args:string[];cwd?:string}>; projectPlannerContext?:string; pinnedBaseCommit?:string; workspaceId?:string; controller?:RunControllerState; reviewerRetryAuthorized?:boolean; reviewerRecommendationHistory?:ReviewerRecommendation[]; plannerSessionId?: string; orchestratorSessionId?: string; sessions: {planner:RoleSession;orchestrator:RoleSession}; sessionHistory:RoleSession[]; roleConfigs: Record; guidance: Guidance[]; assignments: Assignment[]; workerProposal?:WorkerProposal; workerProposalHistory?:WorkerProposal[]; rejectedWorkerAttempts?:RejectedWorkerAttempt[]; orchestratorInbox?:OrchestratorResultInbox; orchestratorInboxHistory?:OrchestratorResultInbox[]; reviews:ReviewRecord[]; workerEvidence?:WorkerEvidence; workerEvidenceHistory?:WorkerEvidence[]; validation?:ValidationRecord; validationHistory?:ValidationRecord[]; reviewerRecommendation?:ReviewerRecommendation; approval?:ApprovalRecord; promotion?:PromotionRecord; prDraft?:PrDraft; usage?:Usage; usageByRole?:Record; usageByHarnessModel?:Record } +export interface Run { id: string; status: RunStatus; createdAt: string; allowedScope?:string[]; baseFilePaths?:string[]; validationCommands?:Array<{name:string;command:string;args:string[];cwd?:string;network?:boolean}>; projectPlannerContext?:string; pinnedBaseCommit?:string; workspaceId?:string; controller?:RunControllerState; reviewerRetryAuthorized?:boolean; reviewerRecommendationHistory?:ReviewerRecommendation[]; plannerSessionId?: string; orchestratorSessionId?: string; sessions: {planner:RoleSession;orchestrator:RoleSession}; sessionHistory:RoleSession[]; roleConfigs: Record; guidance: Guidance[]; assignments: Assignment[]; workerProposal?:WorkerProposal; workerProposalHistory?:WorkerProposal[]; rejectedWorkerAttempts?:RejectedWorkerAttempt[]; orchestratorInbox?:OrchestratorResultInbox; orchestratorInboxHistory?:OrchestratorResultInbox[]; reviews:ReviewRecord[]; workerEvidence?:WorkerEvidence; workerEvidenceHistory?:WorkerEvidence[]; validation?:ValidationRecord; validationHistory?:ValidationRecord[]; reviewerRecommendation?:ReviewerRecommendation; approval?:ApprovalRecord; promotion?:PromotionRecord; prDraft?:PrDraft; usage?:Usage; usageByRole?:Record; usageByHarnessModel?:Record } export interface RunBudget { roleTurns: Record; workerAttempts: number } export type RunBudgetOverrides = { roleTurns?: Partial>; workerAttempts?: number }; export interface RunControllerState { startedAt: string; active: boolean; phase:'orchestrating'|'dispatching'|'verifying'|'validating'|'reviewing'|'stopped'|'awaiting_approval'; budgets:RunBudget; stoppedReason?:string } diff --git a/src/repository-inspector.ts b/src/repository-inspector.ts index 91ddd46..0c67e60 100644 --- a/src/repository-inspector.ts +++ b/src/repository-inspector.ts @@ -222,7 +222,7 @@ export async function inspectRepository(selectedPath: string): Promise<{ const pkgScripts = pkg.scripts ?? {}; const lockfile = runner === 'pnpm' ? 'pnpm-lock.yaml' : 'package-lock.json'; const installCmd: ValidationSuggestion | undefined = allFiles.includes(lockfile) - ? { name: 'Install dependencies', command: runner, args: runner === 'pnpm' ? ['install', '--frozen-lockfile'] : ['ci'], source: 'ci' } + ? { name: 'Install dependencies', command: runner, args: runner === 'pnpm' ? ['install', '--frozen-lockfile'] : ['ci'], network: true, source: 'ci' } : undefined; // Try CI workflow first — pass known script names so bare `pnpm X` is only @@ -260,9 +260,10 @@ export async function inspectRepository(selectedPath: string): Promise<{ } } catch { /* malformed package metadata has no automatic validation suggestion */ } } else if (allFiles.includes('Cargo.toml')) { - suggestedValidationCommands.push({ name: 'Tests', command: 'cargo', args: ['test'], source: 'package-script' }); + // cargo and go fetch dependencies implicitly on first run, so they get the network like the install step; every other suggestion runs offline. + suggestedValidationCommands.push({ name: 'Tests', command: 'cargo', args: ['test'], network: true, source: 'package-script' }); } else if (allFiles.includes('go.mod')) { - suggestedValidationCommands.push({ name: 'Tests', command: 'go', args: ['test', './...'], source: 'package-script' }); + suggestedValidationCommands.push({ name: 'Tests', command: 'go', args: ['test', './...'], network: true, source: 'package-script' }); } else if (allFiles.includes('pyproject.toml')) { suggestedValidationCommands.push({ name: 'Tests', command: 'python', args: ['-m', 'pytest'], source: 'package-script' }); } diff --git a/src/server.ts b/src/server.ts index 76a39bb..fb1b4e7 100644 --- a/src/server.ts +++ b/src/server.ts @@ -219,6 +219,7 @@ const server=createServer(async(req,res)=>{ m=path.match(/^\/api\/runs\/([^/]+)\/reviewer-correction\/resume$/);if(req.method==='POST'&&m){json(res,202,await activeController.resumeReviewerCorrection(decodeURIComponent(m[1]!)));return;} m=path.match(/^\/api\/runs\/([^/]+)\/reviewer-correction\/continue$/);if(req.method==='POST'&&m){json(res,202,await activeController.continueReviewerCorrection(decodeURIComponent(m[1]!)));return;} m=path.match(/^\/api\/runs\/([^/]+)\/initial-proposal\/resume$/);if(req.method==='POST'&&m){json(res,202,await activeController.resumeInitialProposal(decodeURIComponent(m[1]!)));return;} + m=path.match(/^\/api\/runs\/([^/]+)\/decision$/);if(req.method==='GET'&&m){json(res,200,await activeController.decisionEvidence(decodeURIComponent(m[1]!)));return;} m=path.match(/^\/api\/runs\/([^/]+)\/approve$/);if(req.method==='POST'&&m){json(res,200,await activeController.approveRun(decodeURIComponent(m[1]!),await body(req)));return;} m=path.match(/^\/api\/runs\/([^/]+)\/abandon-result$/);if(req.method==='POST'&&m){json(res,200,await activeController.abandonApprovedResult(decodeURIComponent(m[1]!),await body(req)));return;} m=path.match(/^\/api\/runs\/([^/]+)\/promotion$/);if(req.method==='POST'&&m){json(res,200,await activeController.promoteRun(decodeURIComponent(m[1]!),await body(req)));return;} diff --git a/src/validation-sandbox.ts b/src/validation-sandbox.ts index acba2ad..128f492 100644 --- a/src/validation-sandbox.ts +++ b/src/validation-sandbox.ts @@ -35,6 +35,8 @@ export interface SandboxPlanInput { cwd?: string; command: string; args: readonly string[]; + /** Keep the host network namespace. Only an explicit `true` does; anything else runs in an empty namespace with just a loopback. */ + network?: boolean; /** Host environment values that are carried into the sandbox. */ env: { path: string; lang: string; lcAll: string }; /** Operator home directory. */ @@ -54,6 +56,18 @@ const isDirectory = (path: string): boolean => { try { return statSync(path).isD const isFile = (path: string): boolean => { try { return statSync(path).isFile(); } catch { return false; } }; const within = (path: string, root: string): boolean => path === root || path.startsWith(root === '/' ? '/' : `${root}/`); +const INSTALL_SUBCOMMANDS = new Set(['install', 'i', 'ci', 'add']); + +/** + * Network default for a validation command that carries no `network` flag (saved workspace setups, env configs, API clients written before the flag existed): + * true for a package-manager install (`pnpm`, `npm` or `yarn` with first argument install, i, ci or add, and bare `yarn`), false for everything else. + */ +export function defaultNetworkAccess(command: string, args: readonly string[]): boolean { + const name = command.trim(); + if (name === 'yarn' && args.length === 0) return true; + return (name === 'pnpm' || name === 'npm' || name === 'yarn') && INSTALL_SUBCOMMANDS.has(args[0] ?? ''); +} + /** Parse FOREMAN_VALIDATION_SANDBOX; empty means the default, anything unrecognised is undefined. */ export function parseSandboxMode(raw: string | undefined): ValidationSandboxMode | undefined { const value = raw?.trim().toLowerCase(); @@ -152,6 +166,8 @@ export function buildSandboxArgs(input: SandboxPlanInput): string[] { }; return [ '--die-with-parent', '--new-session', '--unshare-pid', '--unshare-ipc', '--unshare-uts', '--cap-drop', 'ALL', + // A private network namespace hides the host's loopback services (Foreman's own API, the bridge), cloud metadata and abstract Unix sockets; bwrap brings up its own lo. + ...(input.network === true ? [] : ['--unshare-net']), '--ro-bind', '/', '/', ...hidden.flatMap(path => ['--tmpfs', path]), '--proc', '/proc', '--dev', '/dev', @@ -173,7 +189,7 @@ export async function ensureSandboxCache(cacheDir: string): Promise { const usableBwrap = new Set(); -/** Run a trivial command through the same isolation flags to prove bwrap can really create the sandbox here. */ +/** Run a trivial command through the same isolation flags (including the private network namespace) to prove bwrap can really create the sandbox here. */ export function probeBwrap(bwrapPath = 'bwrap'): Promise<{ ok: true } | { ok: false; detail: string }> { if (usableBwrap.has(bwrapPath)) return Promise.resolve({ ok: true }); return new Promise(resolvePromise => { @@ -184,7 +200,7 @@ export function probeBwrap(bwrapPath = 'bwrap'): Promise<{ ok: true } | { ok: fa if (result.ok) usableBwrap.add(bwrapPath); resolvePromise(result); }; - const child = spawn(bwrapPath, ['--die-with-parent', '--new-session', '--unshare-pid', '--unshare-ipc', '--unshare-uts', '--cap-drop', 'ALL', '--ro-bind', '/', '/', '--tmpfs', '/tmp', '--proc', '/proc', '--dev', '/dev', '--', '/bin/sh', '-c', ':'], { stdio: ['ignore', 'ignore', 'pipe'], env: { PATH: process.env.PATH ?? '' } }); + const child = spawn(bwrapPath, ['--die-with-parent', '--new-session', '--unshare-pid', '--unshare-ipc', '--unshare-uts', '--unshare-net', '--cap-drop', 'ALL', '--ro-bind', '/', '/', '--tmpfs', '/tmp', '--proc', '/proc', '--dev', '/dev', '--', '/bin/sh', '-c', ':'], { stdio: ['ignore', 'ignore', 'pipe'], env: { PATH: process.env.PATH ?? '' } }); const timer = setTimeout(() => { child.kill('SIGKILL'); finish({ ok: false, detail: 'timed out' }); }, 10_000); child.stderr.on('data', chunk => { if (stderr.length < 2000) stderr += chunk; }); child.once('error', error => finish({ ok: false, detail: error.message })); diff --git a/src/verified-workspace.ts b/src/verified-workspace.ts index 9ba6b0c..6410367 100644 --- a/src/verified-workspace.ts +++ b/src/verified-workspace.ts @@ -4,7 +4,7 @@ import { homedir, tmpdir } from 'node:os'; import { dirname, isAbsolute, join, resolve, sep } from 'node:path'; import { spawn } from 'node:child_process'; import { verifyGitSnapshotScope } from './git-workspace.js'; -import { assertBwrapUsable, buildSandboxArgs, ensureSandboxCache, type ValidationSandboxConfig } from './validation-sandbox.js'; +import { assertBwrapUsable, buildSandboxArgs, defaultNetworkAccess, ensureSandboxCache, type ValidationSandboxConfig } from './validation-sandbox.js'; import type { SnapshotChange, SnapshotEntry } from './workspace-snapshot.js'; export interface BridgeSnapshotEntry { path: string; kind: 'file'|'symlink'; mode: '100644'|'100755'|'120000'; size: number; sha256: string; contentBase64?: string; target?: string } @@ -14,8 +14,8 @@ export interface VerifiedWorkerWorkspace { provenance: 'bridge_snapshot'|'recorded_replay'; pinnedBaseCommit: string; completeSnapshot: { reportedComplete: true; reportedErrors: 0; entryCount: number }; scopeVerified: true; allowedScope: string[]; entries: SnapshotEntry[]; changes: ChangeEvidence[]; reviewDiff: string; } -export interface ValidationCommand { name: string; command: string; args: string[]; cwd?: string } -export interface ValidationObservation { name: string; command: string; args: string[]; exitCode: number|null; signal?: string; timedOut: boolean; output: string; outputTruncated: boolean; startedAt: string; finishedAt: string; sandbox: 'bwrap'|'none' } +export interface ValidationCommand { name: string; command: string; args: string[]; cwd?: string; /** Keep the host network for this command. Unset means a recognised package-manager install gets it and everything else runs offline (`defaultNetworkAccess`); an explicit boolean always wins. */ network?: boolean } +export interface ValidationObservation { name: string; command: string; args: string[]; exitCode: number|null; signal?: string; timedOut: boolean; output: string; outputTruncated: boolean; startedAt: string; finishedAt: string; sandbox: 'bwrap'|'none'; /** True when the command could reach the network: it asked for it, or nothing isolated it (sandbox none). */ network: boolean } export interface ControllerValidationEvidence { passed: boolean; checks: ValidationObservation[] } const SHA=/^(?:[a-f0-9]{40}|[a-f0-9]{64})$/i; @@ -161,12 +161,13 @@ export async function validateWorkerOutput(input:{repoPath:string;evidence:Verif async function runOne(root:string,repoPath:string,command:ValidationCommand,timeoutMs:number,maxBytes:number,sandbox:ValidationSandboxConfig):Promise{ if(!Number.isSafeInteger(timeoutMs)||timeoutMs<1||!Number.isSafeInteger(maxBytes)||maxBytes<1)throw new Error('Invalid validation bounds'); + if(command.network!==undefined&&typeof command.network!=='boolean')throw new Error('Validation command network must be a boolean'); const cwd=command.cwd?resolve(root,command.cwd):root;if(cwd!==root&&!cwd.startsWith(root+sep))throw new Error('Validation cwd escapes disposable workspace'); const relCwd=cwd.slice(root.length+1).split(sep);let traversed=root;for(const part of relCwd){traversed=join(traversed,part);try{if((await lstat(traversed)).isSymbolicLink())throw new Error('Validation cwd follows a symlink');}catch(error){if(error instanceof Error&&error.message==='Validation cwd follows a symlink')throw error;}} - const mode=sandbox.mode??'bwrap',hostEnv={PATH:process.env.PATH??'',LANG:process.env.LANG??'C.UTF-8',LC_ALL:process.env.LC_ALL??'C.UTF-8'}; + const mode=sandbox.mode??'bwrap',network=mode==='none'||(command.network??defaultNetworkAccess(command.command,command.args)),hostEnv={PATH:process.env.PATH??'',LANG:process.env.LANG??'C.UTF-8',LC_ALL:process.env.LC_ALL??'C.UTF-8'}; // Sandboxed: bwrap is the direct child, so killing it tears down the whole PID namespace (--die-with-parent + --unshare-pid). const launch=mode==='bwrap' - ?{file:sandbox.bwrapPath??'bwrap',args:buildSandboxArgs({workspacePath:root,cwd:relCwd.join(sep),command:command.command,args:command.args,env:{path:hostEnv.PATH,lang:hostEnv.LANG,lcAll:hostEnv.LC_ALL},home:homedir(),tmpDir:tmpdir(),repoPath,dataDir:sandbox.dataDir,cacheDir:sandbox.cacheDir,roPaths:sandbox.roPaths}),cwd:root,env:{PATH:hostEnv.PATH}} + ?{file:sandbox.bwrapPath??'bwrap',args:buildSandboxArgs({workspacePath:root,cwd:relCwd.join(sep),command:command.command,args:command.args,network,env:{path:hostEnv.PATH,lang:hostEnv.LANG,lcAll:hostEnv.LC_ALL},home:homedir(),tmpDir:tmpdir(),repoPath,dataDir:sandbox.dataDir,cacheDir:sandbox.cacheDir,roPaths:sandbox.roPaths}),cwd:root,env:{PATH:hostEnv.PATH}} :{file:command.command,args:command.args,cwd,env:hostEnv}; const startedAt=new Date().toISOString();return new Promise(resolvePromise=>{ const child=spawn(launch.file,launch.args,{cwd:launch.cwd,stdio:['ignore','pipe','pipe'],windowsHide:true,detached:process.platform!=='win32',env:launch.env}); @@ -174,8 +175,8 @@ async function runOne(root:string,repoPath:string,command:ValidationCommand,time const collect=(chunk:Buffer)=>{if(size=maxBytes&&chunk.length>0){truncated=true;killTree(child);}}; child.stdout.on('data',collect);child.stderr.on('data',collect); const timer=setTimeout(()=>{timedOut=true;killTree(child);},timeoutMs); - child.once('error',()=>{clearTimeout(timer);exitCode=127;resolvePromise({name:command.name,command:command.command,args:[...command.args],exitCode,signal,timedOut,output:Buffer.concat(chunks).toString('utf8'),outputTruncated:truncated,startedAt,finishedAt:new Date().toISOString(),sandbox:mode});}); - child.once('close',(code,term)=>{clearTimeout(timer);exitCode=code;signal=term??undefined;resolvePromise({name:command.name,command:command.command,args:[...command.args],exitCode,signal,timedOut,output:Buffer.concat(chunks).toString('utf8'),outputTruncated:truncated,startedAt,finishedAt:new Date().toISOString(),sandbox:mode});}); + child.once('error',()=>{clearTimeout(timer);exitCode=127;resolvePromise({name:command.name,command:command.command,args:[...command.args],exitCode,signal,timedOut,output:Buffer.concat(chunks).toString('utf8'),outputTruncated:truncated,startedAt,finishedAt:new Date().toISOString(),sandbox:mode,network});}); + child.once('close',(code,term)=>{clearTimeout(timer);exitCode=code;signal=term??undefined;resolvePromise({name:command.name,command:command.command,args:[...command.args],exitCode,signal,timedOut,output:Buffer.concat(chunks).toString('utf8'),outputTruncated:truncated,startedAt,finishedAt:new Date().toISOString(),sandbox:mode,network});}); }); } diff --git a/src/workspace-setup.test.ts b/src/workspace-setup.test.ts index e45765b..4847612 100644 --- a/src/workspace-setup.test.ts +++ b/src/workspace-setup.test.ts @@ -1,4 +1,4 @@ -import { mkdtemp, mkdir, rm, writeFile } from 'node:fs/promises'; +import { mkdtemp, mkdir, readFile, rm, writeFile } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { afterEach, describe, expect, it } from 'vitest'; @@ -61,4 +61,61 @@ describe('workspace setup', () => { await expect(validateWorkspaceSetup({ repoPath: '.', allowedScope: ['src/'], validationCommands: [{ name: 'Build', command: 'npm', args: [] }] })).rejects.toThrow('absolute repository folder'); await expect(validateWorkspaceSetup({ repoPath: '/tmp', allowedScope: ['src/'], validationCommands: [] })).rejects.toThrow('validation commands'); }); + + describe('network flag', () => { + const networkOf = (setup: { validationCommands: Array<{ name: string; network?: boolean }> }) => Object.fromEntries(setup.validationCommands.map(check => [check.name, check.network])); + + it('gives a command with no flag network access only when it is a recognised package-manager install', async () => { + const repoPath = await gitRepo(); + const commands = [ + { name: 'pnpm install', command: 'pnpm', args: ['install', '--frozen-lockfile'] }, + { name: 'npm ci', command: 'npm', args: ['ci'] }, + { name: 'npm i', command: 'npm', args: ['i', '--no-audit'] }, + { name: 'pnpm add', command: 'pnpm', args: ['add', 'left-pad'] }, + { name: 'bare yarn', command: 'yarn', args: [] }, + { name: 'yarn install', command: 'yarn', args: ['install', '--immutable'] }, + { name: 'trimmed', command: ' npm ', args: ['ci'] }, + { name: 'pnpm test', command: 'pnpm', args: ['test'] }, + { name: 'npm run install', command: 'npm', args: ['run', 'install'] }, + { name: 'pnpm filter install', command: 'pnpm', args: ['--filter', 'app', 'install'] }, + { name: 'yarn flags only', command: 'yarn', args: ['--immutable'] }, + { name: 'cargo', command: 'cargo', args: ['test'] }, + { name: 'other install', command: 'bun', args: ['install'] }, + { name: 'shim path', command: '/usr/local/bin/pnpm', args: ['install'] }, + ]; + const setup = await validateWorkspaceSetup({ repoPath, allowedScope: ['src/'], validationCommands: commands }); + expect(networkOf(setup)).toEqual({ + 'pnpm install': true, 'npm ci': true, 'npm i': true, 'pnpm add': true, 'bare yarn': true, 'yarn install': true, trimmed: true, + 'pnpm test': false, 'npm run install': false, 'pnpm filter install': false, 'yarn flags only': false, cargo: false, 'other install': false, 'shim path': false, + }); + }); + + it('keeps an explicit flag on every command, including an install that opts out and a test that opts in', async () => { + const repoPath = await gitRepo(); + const setup = await validateWorkspaceSetup({ repoPath, allowedScope: ['src/'], validationCommands: [ + { name: 'Install offline', command: 'pnpm', args: ['install', '--offline'], network: false }, + { name: 'Cargo', command: 'cargo', args: ['test'], network: true }, + { name: 'Tests', command: 'pnpm', args: ['test'], network: false }, + ] }); + expect(networkOf(setup)).toEqual({ 'Install offline': false, Cargo: true, Tests: false }); + }); + + it('persists the resolved flag and defaults a saved setup written before the flag existed', async () => { + const repoPath = await gitRepo(), dataDir = join(await temp(), 'data'); + const saved = await saveWorkspaceSetup(dataDir, 'project-net', { repoPath, allowedScope: ['src/'], validationCommands: [{ name: 'Install', command: 'pnpm', args: ['install'] }, { name: 'Tests', command: 'pnpm', args: ['test'], network: true }] }); + expect(saved.validationCommands.map(check => check.network)).toEqual([true, true]); + expect(JSON.parse(await readFile(join(dataDir, 'workspaces', 'project-net.json'), 'utf8')).validationCommands.map((check: { network: boolean }) => check.network)).toEqual([true, true]); + // An older file: no network field anywhere. + const legacy = { repoPath, head: saved.head, dirty: false, allowedScope: ['src/'], validationCommands: [{ name: 'Install', command: 'npm', args: ['ci'] }, { name: 'Tests', command: 'npm', args: ['test'] }] }; + await writeFile(join(dataDir, 'workspaces', 'project-legacy.json'), JSON.stringify(legacy)); + expect(networkOf((await loadWorkspaceSetup(dataDir, 'project-legacy'))!)).toEqual({ Install: true, Tests: false }); + }); + + it('accepts only true or false', async () => { + const base = { repoPath: '/path/that/does/not/exist', allowedScope: ['src/'] }; + for (const network of ['true', 'false', 1, 0, null, {}, []]) { + await expect(validateWorkspaceSetup({ ...base, validationCommands: [{ name: 'Install', command: 'pnpm', args: ['install'], network }] }), JSON.stringify(network)).rejects.toThrow('network must be true or false'); + } + }); + }); }); diff --git a/src/workspace-setup.ts b/src/workspace-setup.ts index aa6b7c0..21e17c0 100644 --- a/src/workspace-setup.ts +++ b/src/workspace-setup.ts @@ -1,12 +1,15 @@ import { spawn } from 'node:child_process'; import { mkdir, readFile, rename, rm, writeFile } from 'node:fs/promises'; import { isAbsolute, resolve, sep } from 'node:path'; +import { defaultNetworkAccess } from './validation-sandbox.js'; export interface WorkspaceValidationCommand { name: string; command: string; args: string[]; cwd?: string; + /** Keep the host network for this command. Validation runs offline unless this is true; saved setups without the flag default by `defaultNetworkAccess`. */ + network?: boolean; } export interface WorkspaceSetupConfig { @@ -129,13 +132,15 @@ function validateCommands(value: unknown): WorkspaceValidationCommand[] { if (typeof command.name !== 'string' || !command.name.trim() || command.name.length > 120 || names.has(command.name.trim())) throw new Error('Validation command names must be unique and non-empty'); if (typeof command.command !== 'string' || !command.command.trim() || command.command.length > 1024 || command.command.includes('\0')) throw new Error('Validation commands require an executable name'); if (!Array.isArray(command.args) || command.args.length > MAX_ARGS || command.args.some(arg => typeof arg !== 'string' || arg.length > MAX_ARG_LENGTH || arg.includes('\0'))) throw new Error(`Validation argv must contain at most ${MAX_ARGS} bounded string arguments`); + if (command.network !== undefined && typeof command.network !== 'boolean') throw new Error('Validation command network must be true or false'); let cwd: string | undefined; if (command.cwd !== undefined) { if (typeof command.cwd !== 'string' || !command.cwd || command.cwd.startsWith('/') || command.cwd.includes('\\') || command.cwd.includes('\0') || command.cwd.split('/').some(part => !part || part === '.' || part === '..')) throw new Error('Validation command cwd must be a safe relative directory'); cwd = command.cwd; } names.add(command.name.trim()); - return { name: command.name.trim(), command: command.command.trim(), args: [...command.args] as string[], ...(cwd ? { cwd } : {}) }; + const executable = command.command.trim(), args = [...command.args] as string[]; + return { name: command.name.trim(), command: executable, args, ...(cwd ? { cwd } : {}), network: (command.network as boolean | undefined) ?? defaultNetworkAccess(executable, args) }; }); } diff --git a/tests/controller.test.ts b/tests/controller.test.ts index 954bf7e..c805036 100644 --- a/tests/controller.test.ts +++ b/tests/controller.test.ts @@ -12,6 +12,7 @@ import { JsonStore } from '../src/store.js'; import { deleteWorkspaceSetup, findSavedProjectForRepository } from '../src/workspace-setup.js'; import { UhpClient } from '../src/uhp.js'; import { startUhpFixture, type UhpFixture } from './fixtures/uhp-server.js'; +import { decisionDigest } from './decision-helper.js'; const workspaceMocks=vi.hoisted(()=>({verify:vi.fn(),validate:vi.fn(),useVerify:false,useValidate:false})); vi.mock('../src/verified-workspace.js',async importOriginal=>{const original=await importOriginal();return {...original,verifyWorkerSnapshot:(input:any)=>workspaceMocks.useVerify?workspaceMocks.verify(input):original.verifyWorkerSnapshot(input),validateWorkerOutput:(input:any)=>workspaceMocks.useValidate?workspaceMocks.validate(input):original.validateWorkerOutput(input)};}); @@ -375,7 +376,7 @@ describe('controller persistence and state transitions',()=>{ await addInbox('tampered diff');await expect(controller.requestReviewer(run.id)).rejects.toThrow('Orchestrator receipt is not bound');expect(calls).toHaveLength(0); await addInbox('exact verified diff');const recommendation:any=await controller.requestReviewer(run.id);expect(recommendation).toMatchObject({verdict:'recommend',responseId:'resp_reviewer_handoff'});expect(submittedEvidence).toMatchObject({orchestratorInboxId:'inbox_handoff',baseCommit:base,allowedScope:['README.md'],workerResponseId:'resp_worker_handoff',reviewDiff:'exact verified diff',controllerValidation:receiptValidation}); const state=await store.load(),savedRun=state.projects[0]!.tasks[0]!.runs[0]!,reviewer=savedRun.assignments.find(a=>a.roleId==='reviewer')!,handoff=state.events.find(e=>e.type==='reviewer.evidence_handoff');expect(reviewer).toMatchObject({reviewerInboxId:'inbox_handoff',reviewerEvidenceDigest:savedRun.orchestratorInbox!.evidenceDigest});expect(handoff?.data).toMatchObject({assignmentId:reviewer.id,inboxId:'inbox_handoff',evidenceDigest:savedRun.orchestratorInbox!.evidenceDigest}); - await expect(controller.approveRun(run.id,{approved:true,evidenceCommit:base})).resolves.toMatchObject({approved:true}); + await expect(controller.approveRun(run.id,{approved:true,evidenceCommit:base,evidenceDigest:await decisionDigest(controller,run.id)})).resolves.toMatchObject({approved:true}); }); it('accepts an Antigravity Reviewer unavailable-model report only with exact CLI and read-only evidence',async()=>{ const {controller,store}=await setup({discover:async()=>({version:'fixture',capabilities:{readOnlyReviewer:true},harnesses:[{id:'fixture'},{id:'antigravity-cli',models:[{id:'gemini-3.8-flash-medium',available:true}]}]}),submit:async input=>({externalId:'resp_reviewer',responseId:'resp_reviewer',sessionId:'session_reviewer',status:'completed',requestedModel:'gemini-3.8-flash-medium',actualModelStatus:'unavailable',selectedHarnessId:'antigravity-cli',reportedHarnessId:'antigravity-cli',cliInvocation:{executable:'/opt/agy',hostExecutable:'/usr/bin/agy',args:['-p','review','--output-format','stream-json','--model','gemini-3.8-flash-medium']},reviewerExecution:{mode:'read_only',mutationAttempted:false,validation:(input.config.reviewEvidence as any).controllerValidation},outputText:JSON.stringify({verdict:'recommend',rationale:'The supplied diff and validation evidence are acceptable.'}),usage:{inputTokens:8,outputTokens:4,thinkingTokens:2,cachedInputTokens:3}})}); @@ -413,7 +414,7 @@ describe('controller persistence and state transitions',()=>{ controller.configureVerifiedWorkspace({repoPath:repo,allowedScope:['README.md'],commands:[{name:'fixture validation',command:'/bin/true',args:[]}]});const project:any=await controller.createProject('Retry validation'),task:any=await controller.createTask(project.id,'Recover saved evidence'),run:any=await controller.createRun(task.id); await store.mutate(s=>{const r=s.projects[0]!.tasks[0]!.runs[0]!;r.pinnedBaseCommit=base;r.controller={startedAt:stamp,active:false,phase:'stopped',budgets:{roleTurns:{planner:1,orchestrator:1,worker:1,reviewer:1},workerAttempts:1},stoppedReason:'Controller validation evidence exceeds the 16,000 byte inbox limit'};r.assignments.push({id:'retry_worker',roleId:'worker',status:'succeeded',requestedConfig:{harnessId:'fixture',model:'model-fixture'},responseId:'retry_worker_response',prompt:'fixture',submissionId:'retry_worker_submission',idempotencyKey:'retry_worker_key',createdAt:stamp} as any);r.workerEvidence={provenance:'recorded_replay',workerAssignmentId:'retry_worker',responseId:'retry_worker_response',pinnedBaseCommit:base,completeSnapshot:{reportedComplete:true,reportedErrors:0,entryCount:1},scopeVerified:true,allowedScope:['README.md'],entries:[{path:'README.md',kind:'file',executable:false,contentBase64}],changes:[],reviewDiff:'No workspace changes.\n',acceptance:'not_decided'} as any;s.events.push({id:'evt_previous_stop',type:'controller.stopped',entityType:'run',entityId:r.id,at:stamp,data:{reason:'Controller validation evidence exceeds the 16,000 byte inbox limit'}});}); const record:any=await controller.retryValidation(run.id),state=await store.load(),saved=state.projects[0]!.tasks[0]!.runs[0]!; - expect(record).toMatchObject({status:'failed',passed:false,reportedPassed:true,resultGate:{passed:false}});expect(saved.validation?.observations).toEqual(record.observations);expect(record.observations.map((o:any)=>o.sandbox)).toEqual(['bwrap']);expect(saved.orchestratorInbox?.validation.observations).toEqual(record.observations.map(({sandbox:_sandbox,...o}:any)=>o));expect(saved.controller).toMatchObject({active:false,phase:'stopped'});expect(saved.controller?.stoppedReason).toBeUndefined();expect(saved.assignments.filter(a=>a.roleId==='reviewer')).toHaveLength(0);expect(state.events.some(e=>e.id==='evt_previous_stop')).toBe(true);expect(state.events.some(e=>e.type==='validation.retry_observed'&&e.data.passed===false)).toBe(true); + expect(record).toMatchObject({status:'failed',passed:false,reportedPassed:true,resultGate:{passed:false}});expect(saved.validation?.observations).toEqual(record.observations);expect(record.observations.map((o:any)=>o.sandbox)).toEqual(['bwrap']);expect(saved.orchestratorInbox?.validation.observations).toEqual(record.observations.map(({sandbox:_sandbox,network:_network,...o}:any)=>o));expect(saved.controller).toMatchObject({active:false,phase:'stopped'});expect(saved.controller?.stoppedReason).toBeUndefined();expect(saved.assignments.filter(a=>a.roleId==='reviewer')).toHaveLength(0);expect(state.events.some(e=>e.id==='evt_previous_stop')).toBe(true);expect(state.events.some(e=>e.type==='validation.retry_observed'&&e.data.passed===false)).toBe(true); }); it('passes the configured validation sandbox through to retried validation and reports it in the service status',async()=>{ const retry=async(sandbox?:any)=>{ @@ -443,7 +444,8 @@ describe('controller persistence and state transitions',()=>{ }); it('exposes only discovered combinations and leaves an explicit choice unset until selected',async()=>{const fixture=await startUhpFixture();fixtures.push(fixture);const client=new UhpClient({baseUrl:fixture.baseUrl});const dir=await mkdtemp(join(tmpdir(),'foreman-'));dirs.push(dir);const store=new JsonStore(join(dir,'state.json'));const controller=new Controller(store,client,false,true);await controller.refreshDiscovery();let state=await store.load();const role=state.roles.find(r=>r.id==='worker')!;expect(role.availableConfigs).toEqual([{harnessId:'chrn_fixture',model:'model-fixture'},{harnessId:'chrn_other',model:'model-other'}]);expect(role.config).toEqual({harnessId:'',model:''});await expect(controller.selectRoleConfig('worker',{harnessId:'chrn_fixture',model:'model-offline'})).rejects.toThrow('Unsupported');await controller.selectRoleConfig('worker',{harnessId:'chrn_other',model:'model-other'});state=await store.load();expect(state.roles.find(r=>r.id==='worker')!.config).toEqual({harnessId:'chrn_other',model:'model-other'});}); it('validates role configuration before creating submission intent',async()=>{const {controller,store}=await setup();const p:any=await controller.createProject('Example');const t:any=await controller.createTask(p.id,'Task');const r:any=await controller.createRun(t.id);await expect(controller.assign(r.id,'worker','do work',{harnessId:'invalid',model:'bad'})).rejects.toThrow('Unsupported');expect((await store.load()).projects[0]!.tasks[0]!.runs[0]!.assignments).toHaveLength(0);}); - it('persists a live Worker response ID, accepts ordered Planner guidance, cancels at UHP, and gets Orchestrator checkpoint acknowledgments',async()=>{const fixture=await startUhpFixture({holdUntilCancel:true,holdRole:'worker',uniqueResponseIds:true});fixtures.push(fixture);const client=new UhpClient({baseUrl:fixture.baseUrl,timeoutMs:45000});let controller:Controller;const adapter:UhpAdapter={discover:client.discover.bind(client),retrieve:client.retrieve.bind(client),cancel:client.cancel.bind(client),submit:async input=>{if(input.roleId==='orchestrator'){const ids=[...input.prompt.matchAll(/(guide_[\w-]+) \|/g)].map(m=>m[1]!);return {externalId:'resp_orchestrator',responseId:'resp_orchestrator',sessionId:'session_orchestrator',status:'completed',actualModel:'model-fixture',selectedHarnessId:'chrn_fixture',outputText:JSON.stringify({acknowledgments:ids.map((guidanceId,index)=>({guidanceId,action:(['applied','replan','waiting'] as const)[index]??'waiting',reason:'Fixture checkpoint acknowledgment'}))})};}return client.submit(input);}};const dir=await mkdtemp(join(tmpdir(),'foreman-'));dirs.push(dir);const store=new JsonStore(join(dir,'state.json'));controller=new Controller(store,adapter,false,true);await controller.refreshDiscovery();const selected={harnessId:'chrn_fixture',model:'model-fixture'};for(const role of ['planner','orchestrator','worker'])await controller.selectRoleConfig(role,selected);const project:any=await controller.createProject('Fixture project'),task:any=await controller.createTask(project.id,'Run held worker'),run:any=await controller.createRun(task.id);const workerPromise=controller.assign(run.id,'worker','hold while operator gives input');let active:any;for(let i=0;i<100;i++){const state=await store.load();active=state.projects[0]!.tasks[0]!.runs[0]!.assignments.find(a=>a.roleId==='worker');if(active?.responseId)break;await new Promise(resolve=>setTimeout(resolve,10));}expect(active?.responseId).toMatch(/^resp_/);expect(active?.status).toBe('running');const recoveredController=new Controller(new JsonStore(store.filePath),adapter,false,true);await recoveredController.recover();expect(fixture.executionCount).toBe(1);let persisted=await new JsonStore(store.filePath).load();expect(persisted.projects[0]!.tasks[0]!.runs[0]!.assignments.find(a=>a.roleId==='worker')!.responseId).toBe(active.responseId);const first:any=await controller.addGuidance(run.id,'First ordered direction');const second:any=await controller.addGuidance(run.id,'Second ordered direction');expect([first.sequence,second.sequence]).toEqual([1,2]);expect([first.status,second.status]).toEqual(['queued','queued']);const plannerTurns=(await store.load()).projects[0]!.tasks[0]!.runs[0]!.assignments.filter(a=>a.roleId==='planner');expect(plannerTurns).toHaveLength(2);expect(plannerTurns[0]!.prompt).toContain('Source checkout: none');expect(plannerTurns[0]!.prompt).toContain('Do not infer that a file exists or is absent');expect(plannerTurns[0]!.prompt).toContain('Pinned base commit: not yet pinned');const plannerRequests=fixture.requests.filter(r=>r.method==='POST'&&r.path==='/v1/responses'&&JSON.parse(r.body).metadata.foreman_role_id==='planner');expect(JSON.parse(plannerRequests[1]!.body).previous_response_id).toBe(plannerTurns[0]!.responseId);await controller.cancelAssignment(active.id);await expect(workerPromise).resolves.toMatchObject({status:'cancelled'});const acknowledged:any[]=await controller.deliverGuidanceCheckpoint(run.id);expect(acknowledged.map(x=>x.acknowledgment)).toEqual(['applied','replan']);expect(acknowledged[0].checkpointHandoffAssignmentId).toBeTruthy();let state=await controller.state();expect(state.projects[0]!.tasks[0]!.runs[0]!.assignments.find(a=>a.roleId==='worker')!.externalId).toBe(active.responseId);expect(state.projects[0]!.tasks[0]!.runs[0]!.sessions.planner.uhpSessionId).toMatch(/^hsess_/);expect(state.projects[0]!.tasks[0]!.runs[0]!.sessions.orchestrator.uhpSessionId).toBe('session_orchestrator');expect(state.projects[0]!.tasks[0]!.runs[0]!.assignments.find(a=>a.roleId==='planner')!.usage?.inputTokens).toBeGreaterThan(0);const originalSession=state.projects[0]!.tasks[0]!.runs[0]!.sessions.planner.uhpSessionId;await controller.selectRoleConfig('planner',{harnessId:'chrn_other',model:'model-other'},undefined,run.id);state=await controller.state();expect(state.projects[0]!.tasks[0]!.runs[0]!.sessionHistory.at(-1)).toMatchObject({uhpSessionId:originalSession,status:'rotated'});expect(state.projects[0]!.tasks[0]!.runs[0]!.sessions.planner.uhpSessionId).toBeUndefined();}); + // The Worker wait below polls the on-disk file through a fresh JsonStore, not store.load(): the store publishes a mutation in memory before its atomic write lands, and recover() below reads the disk. Polling memory let recover() see the stale 'submitting' assignment, replay the held POST, and block until the test timed out. + it('persists a live Worker response ID, accepts ordered Planner guidance, cancels at UHP, and gets Orchestrator checkpoint acknowledgments',async()=>{const fixture=await startUhpFixture({holdUntilCancel:true,holdRole:'worker',uniqueResponseIds:true});fixtures.push(fixture);const client=new UhpClient({baseUrl:fixture.baseUrl,timeoutMs:45000});let controller:Controller;const adapter:UhpAdapter={discover:client.discover.bind(client),retrieve:client.retrieve.bind(client),cancel:client.cancel.bind(client),submit:async input=>{if(input.roleId==='orchestrator'){const ids=[...input.prompt.matchAll(/(guide_[\w-]+) \|/g)].map(m=>m[1]!);return {externalId:'resp_orchestrator',responseId:'resp_orchestrator',sessionId:'session_orchestrator',status:'completed',actualModel:'model-fixture',selectedHarnessId:'chrn_fixture',outputText:JSON.stringify({acknowledgments:ids.map((guidanceId,index)=>({guidanceId,action:(['applied','replan','waiting'] as const)[index]??'waiting',reason:'Fixture checkpoint acknowledgment'}))})};}return client.submit(input);}};const dir=await mkdtemp(join(tmpdir(),'foreman-'));dirs.push(dir);const store=new JsonStore(join(dir,'state.json'));controller=new Controller(store,adapter,false,true);await controller.refreshDiscovery();const selected={harnessId:'chrn_fixture',model:'model-fixture'};for(const role of ['planner','orchestrator','worker'])await controller.selectRoleConfig(role,selected);const project:any=await controller.createProject('Fixture project'),task:any=await controller.createTask(project.id,'Run held worker'),run:any=await controller.createRun(task.id);const workerPromise=controller.assign(run.id,'worker','hold while operator gives input');let active:any;for(let i=0;i<100;i++){const state=await new JsonStore(store.filePath).load();active=state.projects[0]!.tasks[0]!.runs[0]!.assignments.find(a=>a.roleId==='worker');if(active?.responseId)break;await new Promise(resolve=>setTimeout(resolve,10));}expect(active?.responseId).toMatch(/^resp_/);expect(active?.status).toBe('running');const recoveredController=new Controller(new JsonStore(store.filePath),adapter,false,true);await recoveredController.recover();expect(fixture.executionCount).toBe(1);let persisted=await new JsonStore(store.filePath).load();expect(persisted.projects[0]!.tasks[0]!.runs[0]!.assignments.find(a=>a.roleId==='worker')!.responseId).toBe(active.responseId);const first:any=await controller.addGuidance(run.id,'First ordered direction');const second:any=await controller.addGuidance(run.id,'Second ordered direction');expect([first.sequence,second.sequence]).toEqual([1,2]);expect([first.status,second.status]).toEqual(['queued','queued']);const plannerTurns=(await store.load()).projects[0]!.tasks[0]!.runs[0]!.assignments.filter(a=>a.roleId==='planner');expect(plannerTurns).toHaveLength(2);expect(plannerTurns[0]!.prompt).toContain('Source checkout: none');expect(plannerTurns[0]!.prompt).toContain('Do not infer that a file exists or is absent');expect(plannerTurns[0]!.prompt).toContain('Pinned base commit: not yet pinned');const plannerRequests=fixture.requests.filter(r=>r.method==='POST'&&r.path==='/v1/responses'&&JSON.parse(r.body).metadata.foreman_role_id==='planner');expect(JSON.parse(plannerRequests[1]!.body).previous_response_id).toBe(plannerTurns[0]!.responseId);await controller.cancelAssignment(active.id);await expect(workerPromise).resolves.toMatchObject({status:'cancelled'});const acknowledged:any[]=await controller.deliverGuidanceCheckpoint(run.id);expect(acknowledged.map(x=>x.acknowledgment)).toEqual(['applied','replan']);expect(acknowledged[0].checkpointHandoffAssignmentId).toBeTruthy();let state=await controller.state();expect(state.projects[0]!.tasks[0]!.runs[0]!.assignments.find(a=>a.roleId==='worker')!.externalId).toBe(active.responseId);expect(state.projects[0]!.tasks[0]!.runs[0]!.sessions.planner.uhpSessionId).toMatch(/^hsess_/);expect(state.projects[0]!.tasks[0]!.runs[0]!.sessions.orchestrator.uhpSessionId).toBe('session_orchestrator');expect(state.projects[0]!.tasks[0]!.runs[0]!.assignments.find(a=>a.roleId==='planner')!.usage?.inputTokens).toBeGreaterThan(0);const originalSession=state.projects[0]!.tasks[0]!.runs[0]!.sessions.planner.uhpSessionId;await controller.selectRoleConfig('planner',{harnessId:'chrn_other',model:'model-other'},undefined,run.id);state=await controller.state();expect(state.projects[0]!.tasks[0]!.runs[0]!.sessionHistory.at(-1)).toMatchObject({uhpSessionId:originalSession,status:'rotated'});expect(state.projects[0]!.tasks[0]!.runs[0]!.sessions.planner.uhpSessionId).toBeUndefined();}); it('persists the idempotency intent before submit and saves returned identifiers',async()=>{let path='';let observed:any;const {controller,store}=await setup({submit:async()=>{observed=JSON.parse(await readFile(path,'utf8'));return {externalId:'ext',sessionId:'sess',responseId:'resp',status:'completed',outputText:'done'};}});path=store.filePath;const p:any=await controller.createProject('Example');const t:any=await controller.createTask(p.id,'Task');const r:any=await controller.createRun(t.id);const a:any=await controller.assign(r.id,'worker','work');const saved=observed.projects[0].tasks[0].runs[0].assignments[0];expect(saved.status).toBe('submitting');expect(saved.idempotencyKey).toBe(a.idempotencyKey);expect(a.externalId).toBe('ext');expect(a.sessionId).toBe('sess');expect(a.responseId).toBe('resp');expect(a.status).toBe('succeeded');const reloaded=await new JsonStore(path).load();expect(reloaded.projects[0]!.tasks[0]!.runs[0]!.assignments[0]!.externalId).toBe('ext');}); it('recovers an uncertain submission with its original idempotency key',async()=>{const {controller,store}=await setup({submit:async()=>{throw new Error('offline');}});const p:any=await controller.createProject('Example');const t:any=await controller.createTask(p.id,'Task');const r:any=await controller.createRun(t.id);await expect(controller.assign(r.id,'worker','work')).rejects.toThrow('offline');const s=await store.load();const key=s.projects[0]!.tasks[0]!.runs[0]!.assignments[0]!.idempotencyKey;let recoveryKey='';const recovered=new Controller(store,{submit:async input=>{recoveryKey=input.idempotencyKey;return {externalId:'ext',status:'completed'};},cancel:async()=>({status:'cancelled'})});await recovered.recover();expect(recoveryKey).toBe(key);expect((await store.load()).projects[0]!.tasks[0]!.runs[0]!.assignments[0]!.status).toBe('succeeded');}); it('marks deterministic UHP 4xx rejections failed and does not resubmit an old Reviewer 400 on recovery',async()=>{const {controller,store}=await setup({submit:async()=>{throw Object.assign(new Error('UHP request failed (400): review_evidence_invalid'),{statusCode:400});}});const project:any=await controller.createProject('Deterministic reject'),task:any=await controller.createTask(project.id,'No automatic replay'),run:any=await controller.createRun(task.id);await expect(controller.assign(run.id,'worker','work')).rejects.toThrow('review_evidence_invalid');let saved=(await store.load()).projects[0]!.tasks[0]!.runs[0]!,first=saved.assignments[0]!;expect(first.status).toBe('failed');expect((await store.load()).events.some(e=>e.type==='assignment.submit_rejected'&&e.entityId===first.id)).toBe(true);expect((await store.load()).events.some(e=>e.type==='assignment.submit_uncertain'&&e.entityId===first.id)).toBe(false);await store.mutate(s=>{const a=s.projects[0]!.tasks[0]!.runs[0]!.assignments[0]!;a.roleId='reviewer';a.status='submitting';a.error='UHP request failed (400): review_evidence_invalid';});let calls=0;const recovered=new Controller(store,{submit:async()=>{calls++;throw new Error('recovery must not resubmit');},cancel:async()=>({status:'cancelled'})});await recovered.recover();saved=(await store.load()).projects[0]!.tasks[0]!.runs[0]!;expect(saved.assignments[0]).toMatchObject({roleId:'reviewer',status:'failed'});expect(saved.assignments[0]!.error).toContain('explicit retry is available');expect(calls).toBe(0);expect((await store.load()).events.some(e=>e.type==='assignment.submit_rejected'&&e.entityId===first.id&&e.data.recovered===true)).toBe(true);}); @@ -476,7 +478,7 @@ describe('controller persistence and state transitions',()=>{ it('replays an intent-only crash with UHP idempotency and blocks retry when the server lacks it',async()=>{const fixture=await startUhpFixture({uniqueResponseIds:true});fixtures.push(fixture);const client=new UhpClient({baseUrl:fixture.baseUrl});const dir=await mkdtemp(join(tmpdir(),'foreman-'));dirs.push(dir);const store=new JsonStore(join(dir,'state.json'));const interrupted=new Controller(store,{discover:client.discover.bind(client),retrieve:client.retrieve.bind(client),cancel:client.cancel.bind(client),submit:async()=>{throw new Error('simulated crash before network send');}},false,true);await interrupted.refreshDiscovery();await interrupted.selectRoleConfig('worker',{harnessId:'chrn_fixture',model:'model-fixture'});const p:any=await interrupted.createProject('Crash recovery'),t:any=await interrupted.createTask(p.id,'intent only'),r:any=await interrupted.createRun(t.id);await expect(interrupted.assign(r.id,'worker','bounded work')).rejects.toThrow('simulated crash');const intent=(await store.load()).projects[0]!.tasks[0]!.runs[0]!.assignments[0]!;expect(intent.status).toBe('submitting');const recovered=new Controller(new JsonStore(store.filePath),client,false,true);await recovered.refreshDiscovery();await recovered.recover();expect(fixture.executionCount).toBe(1);const post=fixture.requests.find(q=>q.method==='POST'&&q.path==='/v1/responses');expect(post?.headers['idempotency-key']).toBe(intent.idempotencyKey);const noRetry=await startUhpFixture({capabilityIdempotency:false});fixtures.push(noRetry);const noRetryClient=new UhpClient({baseUrl:noRetry.baseUrl});const secondDir=await mkdtemp(join(tmpdir(),'foreman-'));dirs.push(secondDir);const secondStore=new JsonStore(join(secondDir,'state.json'));const noRetryController=new Controller(secondStore,noRetryClient,false,true);await noRetryController.refreshDiscovery();await noRetryController.selectRoleConfig('worker',{harnessId:'chrn_fixture',model:'model-fixture'});const p2:any=await noRetryController.createProject('No idempotency'),t2:any=await noRetryController.createTask(p2.id,'held intent'),r2:any=await noRetryController.createRun(t2.id);await expect(noRetryController.assign(r2.id,'worker','do not duplicate')).rejects.toThrow('idempotency');await noRetryController.recover();expect(noRetry.executionCount).toBe(0);}); it('recovers uncertain-after-send by replaying the same idempotent UHP request',async()=>{const fixture=await startUhpFixture({uniqueResponseIds:true});fixtures.push(fixture);const client=new UhpClient({baseUrl:fixture.baseUrl});const dir=await mkdtemp(join(tmpdir(),'foreman-'));dirs.push(dir);const store=new JsonStore(join(dir,'state.json'));const crashed=new Controller(store,{discover:client.discover.bind(client),retrieve:client.retrieve.bind(client),cancel:client.cancel.bind(client),submit:async input=>{const {onEvent:_callback,...noCallback}=input;await client.submit(noCallback);throw new Error('simulated crash after UHP accepted request');}},false,true);await crashed.refreshDiscovery();await crashed.selectRoleConfig('worker',{harnessId:'chrn_fixture',model:'model-fixture'});const p:any=await crashed.createProject('After-send recovery'),t:any=await crashed.createTask(p.id,'uncertain response'),r:any=await crashed.createRun(t.id);await expect(crashed.assign(r.id,'worker','bounded work')).rejects.toThrow('after UHP accepted');const intent=(await store.load()).projects[0]!.tasks[0]!.runs[0]!.assignments[0]!;expect(intent.status).toBe('submitting');expect(intent.responseId).toBeUndefined();expect(fixture.executionCount).toBe(1);const restarted=new Controller(new JsonStore(store.filePath),client,false,true);await restarted.refreshDiscovery();await restarted.recover();expect(fixture.executionCount).toBe(1);expect((await new JsonStore(store.filePath).load()).projects[0]!.tasks[0]!.runs[0]!.assignments[0]!.status).toBe('succeeded');}); it('keeps cancellation intent until a response ID arrives, then retries cancellation after restart',async()=>{let submits=0;const cancelKeys:string[]=[];const adapter:UhpAdapter={submit:async input=>{submits++;if(submits===1)throw new Error('uncertain initial send');await input.onEvent?.({type:'response.created',responseId:'response-before-cancel',sessionId:'session-1',event:{type:'response.created'}});return {externalId:'response-before-cancel',responseId:'response-before-cancel',sessionId:'session-1',status:'completed'};},cancel:async input=>{cancelKeys.push(input.idempotencyKey);return {status:'cancelled'};}};const setupState=await setup(adapter);const store=setupState.store;const p:any=await setupState.controller.createProject('Cancel recovery'),t:any=await setupState.controller.createTask(p.id,'Pending ID'),r:any=await setupState.controller.createRun(t.id);await expect(setupState.controller.assign(r.id,'worker','work')).rejects.toThrow('uncertain initial send');const intent=(await store.load()).projects[0]!.tasks[0]!.runs[0]!.assignments[0]!;const pending:any=await setupState.controller.cancelAssignment(intent.id);expect(pending.status).toBe('cancel_requested');expect(cancelKeys).toHaveLength(0);const restartedStore=new JsonStore(store.filePath),restarted=new Controller(restartedStore,adapter);await restarted.recover();const state=await restartedStore.load(),recovered=state.projects[0]!.tasks[0]!.runs[0]!.assignments[0]!;expect(submits).toBe(2);expect(cancelKeys).toHaveLength(1);expect(state.events.filter(e=>e.type==='assignment.cancel_error')).toEqual([]);expect(recovered.status).toBe('cancelled');expect(recovered.externalId).toBe('response-before-cancel');expect(recovered.cancelIdempotencyKey).toBe(cancelKeys[0]);}); - it('keeps review and validation reports unverified and blocks completion without independent evidence',async()=>{const {controller,store}=await setup();const p:any=await controller.createProject('Review gate'),t:any=await controller.createTask(p.id,'Implementation'),r:any=await controller.createRun(t.id);const worker:any=await controller.assign(r.id,'worker','implement');const reviewer:any={id:'reviewer_fixture',roleId:'reviewer',status:'succeeded'};await store.mutate(s=>{s.projects[0]!.tasks[0]!.runs[0]!.assignments.push(reviewer);});const proposal:any=await controller.addReview(r.id,{reviewerAssignmentId:reviewer.id,implementationAssignmentIds:[worker.id],verdict:'clear',scope:['all worker changes'],summary:'Looks good'});expect(proposal.status).toBe('proposed');await expect(controller.recordValidation(r.id,{passed:true,checks:[{name:'tests',passed:true}],gitEvidence:{commit:'abc',tree:'tree'}})).rejects.toThrow('controller-verified');await store.mutate(s=>{s.projects[0]!.tasks[0]!.runs[0]!.reviews[0]!.status='verified';});const validation:any=await controller.recordValidation(r.id,{passed:true,checks:[{name:'tests',passed:true}],gitEvidence:{commit:'abc',tree:'tree'}});expect(validation).toMatchObject({status:'unverified',passed:false,reportedPassed:true,gitEvidence:{status:'unverified',commit:'abc'}});await expect(controller.approveRun(r.id,{approved:true,evidenceCommit:'abc'})).rejects.toThrow('Human decision requires verified Worker scope');}); + it('keeps review and validation reports unverified and blocks completion without independent evidence',async()=>{const {controller,store}=await setup();const p:any=await controller.createProject('Review gate'),t:any=await controller.createTask(p.id,'Implementation'),r:any=await controller.createRun(t.id);const worker:any=await controller.assign(r.id,'worker','implement');const reviewer:any={id:'reviewer_fixture',roleId:'reviewer',status:'succeeded'};await store.mutate(s=>{s.projects[0]!.tasks[0]!.runs[0]!.assignments.push(reviewer);});const proposal:any=await controller.addReview(r.id,{reviewerAssignmentId:reviewer.id,implementationAssignmentIds:[worker.id],verdict:'clear',scope:['all worker changes'],summary:'Looks good'});expect(proposal.status).toBe('proposed');await expect(controller.recordValidation(r.id,{passed:true,checks:[{name:'tests',passed:true}],gitEvidence:{commit:'abc',tree:'tree'}})).rejects.toThrow('controller-verified');await store.mutate(s=>{s.projects[0]!.tasks[0]!.runs[0]!.reviews[0]!.status='verified';});const validation:any=await controller.recordValidation(r.id,{passed:true,checks:[{name:'tests',passed:true}],gitEvidence:{commit:'abc',tree:'tree'}});expect(validation).toMatchObject({status:'unverified',passed:false,reportedPassed:true,gitEvidence:{status:'unverified',commit:'abc'}});await expect(controller.decisionEvidence(r.id)).rejects.toMatchObject({statusCode:409});await expect(controller.approveRun(r.id,{approved:true,evidenceCommit:'abc',evidenceDigest:'0'.repeat(64)})).rejects.toThrow('Human decision requires verified Worker scope');}); it('allows human decisions for CLI Reviewer recommendations with bound unavailable actual-model evidence',async()=>{ for(const [harnessId,model,approved] of [['codex-cli','gpt-6-sol',true],['antigravity-cli','gemini-3.8-flash-medium',false]] as const){ const {controller,store}=await setup(); @@ -487,11 +489,11 @@ describe('controller persistence and state transitions',()=>{ await store.mutate(s=>{const stored=s.projects[0]!.tasks[0]!.runs[0]!;stored.pinnedBaseCommit=base;stored.status='review';stored.assignments.push({id:'worker_decision',roleId:'worker',status:'succeeded',requestedConfig:{harnessId:'fixture',model:'worker-model'},actualConfig:{harnessId:'fixture',model:'worker-model'},requestedModel:'worker-model',actualModelStatus:'observed',responseId:'resp_worker_decision',sessionId:'session_worker_decision'} as any);stored.workerEvidence={provenance:'recorded_replay',workerAssignmentId:'worker_decision',responseId:'resp_worker_decision',pinnedBaseCommit:base,completeSnapshot:{reportedComplete:true,reportedErrors:0,entryCount:1},scopeVerified:true,allowedScope:['README.md'],entries:[],changes:[{kind:'modify',path:'README.md'}],reviewDiff:'fixture diff',acceptance:'not_decided'};stored.validation={id:'validation_decision',status:'passed',passed:true,reportedPassed:true,checks:[{name:'fixture check',passed:true}],observations:[observation],policy,gitEvidence:{status:'verified',commit:base,changedPaths:['README.md'],submittedAt:stamp},createdAt:stamp};stored.assignments.push({id:'reviewer_decision',roleId:'reviewer',status:'succeeded',requestedConfig:{harnessId,model},requestedModel:model,selectedHarnessId:harnessId,reportedHarnessId:harnessId,actualModelStatus:'unavailable',cliInvocation:{executable:`/opt/${harnessId}`,args:['-p','review','--model',model]},responseId:`resp_${harnessId}`,sessionId:`session_${harnessId}`,result:JSON.stringify({verdict:'recommend',rationale:'Controller evidence supports this change.'}),reviewerExecution:{mode:'read_only',mutationAttempted:false,validation:reviewValidation}} as any);}); const recommendation:any=await controller.recordReviewerRecommendation(run.id,'reviewer_decision'); expect(recommendation).toMatchObject({harnessId,model,verdict:'recommend',responseId:`resp_${harnessId}`,sessionId:`session_${harnessId}`});expect(recommendation.actualModel).toBeUndefined(); - await expect(controller.approveRun(run.id,{approved,evidenceCommit:base})).resolves.toMatchObject({approved}); + await expect(controller.approveRun(run.id,{approved,evidenceCommit:base,evidenceDigest:await decisionDigest(controller,run.id)})).resolves.toMatchObject({approved}); } }); - it('returns after durable approval while accepted-outcome retention remains pending',async()=>{const {controller,store,uhp}=await setup();const p:any=await controller.createProject('Async retention'),t:any=await controller.createTask(p.id,'Approved result'),r:any=await controller.createRun(t.id),worker:any=await controller.assign(r.id,'worker','implement');await store.mutate(s=>{const run=s.projects[0]!.tasks[0]!.runs[0]!;run.reviews.push({id:'review_verified',status:'verified',reviewerAssignmentId:'reviewer-proof',implementationAssignmentIds:[worker.id],verdict:'clear',scope:['all changes'],summary:'independently reviewed',createdAt:new Date().toISOString()});run.validation={id:'validation_verified',status:'passed',passed:true,reportedPassed:true,checks:[{name:'controller tests',passed:true}],gitEvidence:{status:'verified',commit:'abc123',submittedAt:new Date().toISOString()},createdAt:new Date().toISOString()};run.pinnedBaseCommit='a'.repeat(40);const storedWorker=run.assignments.find(a=>a.id===worker.id)!;storedWorker.responseId='resp_worker';run.workerEvidence={provenance:'recorded_replay',workerAssignmentId:worker.id,responseId:storedWorker.responseId,pinnedBaseCommit:run.pinnedBaseCommit,completeSnapshot:{reportedComplete:true,reportedErrors:0,entryCount:0},scopeVerified:true,allowedScope:['README.md'],entries:[],changes:[{kind:'modify',path:'README.md'}],reviewDiff:'fixture diff',acceptance:'not_decided'};run.validation.observations=[{name:'fixture check',command:'true',args:[],exitCode:0,timedOut:false,output:'passed',outputTruncated:false,startedAt:new Date().toISOString(),finishedAt:new Date().toISOString(),passed:true}];const validationEvidence={passed:true,policy:run.validation.policy,observations:run.validation.observations};run.assignments.push({id:'reviewer-proof',roleId:'reviewer',status:'succeeded',requestedConfig:{harnessId:'fixture',model:'simulated'},requestedModel:'simulated',selectedHarnessId:'fixture',reportedHarnessId:'fixture',actualModelStatus:'observed',actualConfig:{harnessId:'fixture',model:'simulated'},responseId:'resp_reviewer',sessionId:'session_reviewer',reviewerExecution:{mode:'read_only',mutationAttempted:false,validation:validationEvidence}} as any);run.reviewerRecommendation={id:'recommendation_fixture',status:'proposed',provenance:'simulated_fixture',reviewerAssignmentId:'reviewer-proof',harnessId:'fixture',model:'simulated',actualModel:'simulated',responseId:'resp_reviewer',sessionId:'session_reviewer',reviewMode:'read_only',mutationAttempted:false,verdict:'recommend',rationale:'Simulated test recommendation',createdAt:new Date().toISOString()};});let retainCalls=0;const hindsight={recall:async()=>({status:'ready' as const,bankId:'bank',memories:[]}),retainOutcome:()=>{retainCalls++;return new Promise(()=>{});}};const approvedController=new Controller(store,uhp,false,false,undefined,hindsight);const approval=await Promise.race([approvedController.approveRun(r.id,{approved:true,evidenceCommit:'a'.repeat(40)}),new Promise((_,reject)=>setTimeout(()=>reject(new Error('approval waited on retention')),100))]);expect(approval).toMatchObject({approved:true,evidenceCommit:'a'.repeat(40)});const snapshot=await store.load();const intent=snapshot.events.find(e=>e.type==='memory.retention_intent');expect(intent?.data).toMatchObject({retentionId:worker.submissionId,idempotencyReference:`foreman-submission-${worker.submissionId}`,status:'pending'});await new Promise(resolve=>setTimeout(resolve,0));expect(retainCalls).toBe(1);await expect(approvedController.approveRun(r.id,{approved:true,evidenceCommit:'a'.repeat(40)})).rejects.toThrow('human decision is final');expect(retainCalls).toBe(1);}); - it('retries degraded retained outcomes after restart using the same idempotency reference',async()=>{const {controller,store,uhp}=await setup();const p:any=await controller.createProject('Retention retry'),t:any=await controller.createTask(p.id,'Approved outcome'),r:any=await controller.createRun(t.id),worker:any=await controller.assign(r.id,'worker','implement');await store.mutate(s=>{const run=s.projects[0]!.tasks[0]!.runs[0]!;run.reviews.push({id:'review_verified',status:'verified',reviewerAssignmentId:'reviewer-proof',implementationAssignmentIds:[worker.id],verdict:'clear',scope:['all changes'],summary:'independently reviewed',createdAt:new Date().toISOString()});run.validation={id:'validation_verified',status:'passed',passed:true,reportedPassed:true,checks:[{name:'controller tests',passed:true}],gitEvidence:{status:'verified',commit:'abc123',submittedAt:new Date().toISOString()},createdAt:new Date().toISOString()};run.pinnedBaseCommit='a'.repeat(40);const storedWorker=run.assignments.find(a=>a.id===worker.id)!;storedWorker.responseId='resp_worker';run.workerEvidence={provenance:'recorded_replay',workerAssignmentId:worker.id,responseId:storedWorker.responseId,pinnedBaseCommit:run.pinnedBaseCommit,completeSnapshot:{reportedComplete:true,reportedErrors:0,entryCount:0},scopeVerified:true,allowedScope:['README.md'],entries:[],changes:[{kind:'modify',path:'README.md'}],reviewDiff:'fixture diff',acceptance:'not_decided'};run.validation.observations=[{name:'fixture check',command:'true',args:[],exitCode:0,timedOut:false,output:'passed',outputTruncated:false,startedAt:new Date().toISOString(),finishedAt:new Date().toISOString(),passed:true}];const validationEvidence={passed:true,policy:run.validation.policy,observations:run.validation.observations};run.assignments.push({id:'reviewer-proof',roleId:'reviewer',status:'succeeded',requestedConfig:{harnessId:'fixture',model:'simulated'},requestedModel:'simulated',selectedHarnessId:'fixture',reportedHarnessId:'fixture',actualModelStatus:'observed',actualConfig:{harnessId:'fixture',model:'simulated'},responseId:'resp_reviewer',sessionId:'session_reviewer',reviewerExecution:{mode:'read_only',mutationAttempted:false,validation:validationEvidence}} as any);run.reviewerRecommendation={id:'recommendation_fixture',status:'proposed',provenance:'simulated_fixture',reviewerAssignmentId:'reviewer-proof',harnessId:'fixture',model:'simulated',actualModel:'simulated',responseId:'resp_reviewer',sessionId:'session_reviewer',reviewMode:'read_only',mutationAttempted:false,verdict:'recommend',rationale:'Simulated test recommendation',createdAt:new Date().toISOString()};});const unavailable={recall:async()=>({status:'ready' as const,bankId:'bank',memories:[]}),retainOutcome:async()=>({status:'degraded' as const,bankId:'bank',error:'temporary outage'})};const approving=new Controller(store,uhp,false,false,undefined,unavailable);await approving.approveRun(r.id,{approved:true,evidenceCommit:'a'.repeat(40)});let state=await store.load();for(let i=0;i<100&&!state.events.some(e=>e.type==='memory.retention_attempt_failed');i++){await new Promise(resolve=>setTimeout(resolve,2));state=await store.load();}expect(state.events.some(e=>e.type==='memory.retention_attempt_failed')).toBe(true);expect(state.events.some(e=>e.type==='memory.retention_completed')).toBe(false);const restartedStore=new JsonStore(store.filePath);let retries=0;const recovered=new Controller(restartedStore,uhp,false,false,undefined,{recall:unavailable.recall,retainOutcome:async outcome=>{retries++;expect(outcome.submissionId).toBe(worker.submissionId);return {status:'accepted',bankId:'bank',operationId:'op-retry'};}});await recovered.resumePendingRetentions();expect(retries).toBe(1);state=await restartedStore.load();for(let i=0;i<100&&!state.events.some(e=>e.type==='memory.retention_completed');i++){await new Promise(resolve=>setTimeout(resolve,2));state=await restartedStore.load();}expect(state.events.find(e=>e.type==='memory.retention_intent')?.data.idempotencyReference).toBe(`foreman-submission-${worker.submissionId}`);expect(state.events.find(e=>e.type==='memory.retention_completed')?.data).toMatchObject({retentionId:worker.submissionId,status:'accepted',operationId:'op-retry'});}); + it('returns after durable approval while accepted-outcome retention remains pending',async()=>{const {controller,store,uhp}=await setup();const p:any=await controller.createProject('Async retention'),t:any=await controller.createTask(p.id,'Approved result'),r:any=await controller.createRun(t.id),worker:any=await controller.assign(r.id,'worker','implement');await store.mutate(s=>{const run=s.projects[0]!.tasks[0]!.runs[0]!;run.reviews.push({id:'review_verified',status:'verified',reviewerAssignmentId:'reviewer-proof',implementationAssignmentIds:[worker.id],verdict:'clear',scope:['all changes'],summary:'independently reviewed',createdAt:new Date().toISOString()});run.validation={id:'validation_verified',status:'passed',passed:true,reportedPassed:true,checks:[{name:'controller tests',passed:true}],gitEvidence:{status:'verified',commit:'abc123',submittedAt:new Date().toISOString()},createdAt:new Date().toISOString()};run.pinnedBaseCommit='a'.repeat(40);const storedWorker=run.assignments.find(a=>a.id===worker.id)!;storedWorker.responseId='resp_worker';run.workerEvidence={provenance:'recorded_replay',workerAssignmentId:worker.id,responseId:storedWorker.responseId,pinnedBaseCommit:run.pinnedBaseCommit,completeSnapshot:{reportedComplete:true,reportedErrors:0,entryCount:0},scopeVerified:true,allowedScope:['README.md'],entries:[],changes:[{kind:'modify',path:'README.md'}],reviewDiff:'fixture diff',acceptance:'not_decided'};run.validation.observations=[{name:'fixture check',command:'true',args:[],exitCode:0,timedOut:false,output:'passed',outputTruncated:false,startedAt:new Date().toISOString(),finishedAt:new Date().toISOString(),passed:true}];const validationEvidence={passed:true,policy:run.validation.policy,observations:run.validation.observations};run.assignments.push({id:'reviewer-proof',roleId:'reviewer',status:'succeeded',requestedConfig:{harnessId:'fixture',model:'simulated'},requestedModel:'simulated',selectedHarnessId:'fixture',reportedHarnessId:'fixture',actualModelStatus:'observed',actualConfig:{harnessId:'fixture',model:'simulated'},responseId:'resp_reviewer',sessionId:'session_reviewer',reviewerExecution:{mode:'read_only',mutationAttempted:false,validation:validationEvidence}} as any);run.reviewerRecommendation={id:'recommendation_fixture',status:'proposed',provenance:'simulated_fixture',reviewerAssignmentId:'reviewer-proof',harnessId:'fixture',model:'simulated',actualModel:'simulated',responseId:'resp_reviewer',sessionId:'session_reviewer',reviewMode:'read_only',mutationAttempted:false,verdict:'recommend',rationale:'Simulated test recommendation',createdAt:new Date().toISOString()};});let retainCalls=0;const hindsight={recall:async()=>({status:'ready' as const,bankId:'bank',memories:[]}),retainOutcome:()=>{retainCalls++;return new Promise(()=>{});}};const approvedController=new Controller(store,uhp,false,false,undefined,hindsight);const reviewedDigest=await decisionDigest(approvedController,r.id);const approval=await Promise.race([approvedController.approveRun(r.id,{approved:true,evidenceCommit:'a'.repeat(40),evidenceDigest:reviewedDigest}),new Promise((_,reject)=>setTimeout(()=>reject(new Error('approval waited on retention')),100))]);expect(approval).toMatchObject({approved:true,evidenceCommit:'a'.repeat(40)});const snapshot=await store.load();const intent=snapshot.events.find(e=>e.type==='memory.retention_intent');expect(intent?.data).toMatchObject({retentionId:worker.submissionId,idempotencyReference:`foreman-submission-${worker.submissionId}`,status:'pending'});await new Promise(resolve=>setTimeout(resolve,0));expect(retainCalls).toBe(1);await expect(approvedController.approveRun(r.id,{approved:true,evidenceCommit:'a'.repeat(40),evidenceDigest:reviewedDigest})).rejects.toThrow('human decision is final');expect(retainCalls).toBe(1);}); + it('retries degraded retained outcomes after restart using the same idempotency reference',async()=>{const {controller,store,uhp}=await setup();const p:any=await controller.createProject('Retention retry'),t:any=await controller.createTask(p.id,'Approved outcome'),r:any=await controller.createRun(t.id),worker:any=await controller.assign(r.id,'worker','implement');await store.mutate(s=>{const run=s.projects[0]!.tasks[0]!.runs[0]!;run.reviews.push({id:'review_verified',status:'verified',reviewerAssignmentId:'reviewer-proof',implementationAssignmentIds:[worker.id],verdict:'clear',scope:['all changes'],summary:'independently reviewed',createdAt:new Date().toISOString()});run.validation={id:'validation_verified',status:'passed',passed:true,reportedPassed:true,checks:[{name:'controller tests',passed:true}],gitEvidence:{status:'verified',commit:'abc123',submittedAt:new Date().toISOString()},createdAt:new Date().toISOString()};run.pinnedBaseCommit='a'.repeat(40);const storedWorker=run.assignments.find(a=>a.id===worker.id)!;storedWorker.responseId='resp_worker';run.workerEvidence={provenance:'recorded_replay',workerAssignmentId:worker.id,responseId:storedWorker.responseId,pinnedBaseCommit:run.pinnedBaseCommit,completeSnapshot:{reportedComplete:true,reportedErrors:0,entryCount:0},scopeVerified:true,allowedScope:['README.md'],entries:[],changes:[{kind:'modify',path:'README.md'}],reviewDiff:'fixture diff',acceptance:'not_decided'};run.validation.observations=[{name:'fixture check',command:'true',args:[],exitCode:0,timedOut:false,output:'passed',outputTruncated:false,startedAt:new Date().toISOString(),finishedAt:new Date().toISOString(),passed:true}];const validationEvidence={passed:true,policy:run.validation.policy,observations:run.validation.observations};run.assignments.push({id:'reviewer-proof',roleId:'reviewer',status:'succeeded',requestedConfig:{harnessId:'fixture',model:'simulated'},requestedModel:'simulated',selectedHarnessId:'fixture',reportedHarnessId:'fixture',actualModelStatus:'observed',actualConfig:{harnessId:'fixture',model:'simulated'},responseId:'resp_reviewer',sessionId:'session_reviewer',reviewerExecution:{mode:'read_only',mutationAttempted:false,validation:validationEvidence}} as any);run.reviewerRecommendation={id:'recommendation_fixture',status:'proposed',provenance:'simulated_fixture',reviewerAssignmentId:'reviewer-proof',harnessId:'fixture',model:'simulated',actualModel:'simulated',responseId:'resp_reviewer',sessionId:'session_reviewer',reviewMode:'read_only',mutationAttempted:false,verdict:'recommend',rationale:'Simulated test recommendation',createdAt:new Date().toISOString()};});const unavailable={recall:async()=>({status:'ready' as const,bankId:'bank',memories:[]}),retainOutcome:async()=>({status:'degraded' as const,bankId:'bank',error:'temporary outage'})};const approving=new Controller(store,uhp,false,false,undefined,unavailable);await approving.approveRun(r.id,{approved:true,evidenceCommit:'a'.repeat(40),evidenceDigest:await decisionDigest(approving,r.id)});let state=await store.load();for(let i=0;i<100&&!state.events.some(e=>e.type==='memory.retention_attempt_failed');i++){await new Promise(resolve=>setTimeout(resolve,2));state=await store.load();}expect(state.events.some(e=>e.type==='memory.retention_attempt_failed')).toBe(true);expect(state.events.some(e=>e.type==='memory.retention_completed')).toBe(false);const restartedStore=new JsonStore(store.filePath);let retries=0;const recovered=new Controller(restartedStore,uhp,false,false,undefined,{recall:unavailable.recall,retainOutcome:async outcome=>{retries++;expect(outcome.submissionId).toBe(worker.submissionId);return {status:'accepted',bankId:'bank',operationId:'op-retry'};}});await recovered.resumePendingRetentions();expect(retries).toBe(1);state=await restartedStore.load();for(let i=0;i<100&&!state.events.some(e=>e.type==='memory.retention_completed');i++){await new Promise(resolve=>setTimeout(resolve,2));state=await restartedStore.load();}expect(state.events.find(e=>e.type==='memory.retention_intent')?.data.idempotencyReference).toBe(`foreman-submission-${worker.submissionId}`);expect(state.events.find(e=>e.type==='memory.retention_completed')?.data).toMatchObject({retentionId:worker.submissionId,status:'accepted',operationId:'op-retry'});}); it('records terminal completion when cancellation races with a completed UHP response',async()=>{let release!:()=>void,started!:()=>void;const gate=new Promise(resolve=>{release=resolve;}),created=new Promise(resolve=>{started=resolve;});const {controller,store}=await setup({submit:async input=>{await input.onEvent?.({type:'response.created',responseId:'race-response',event:{type:'response.created'}});started();await gate;return {externalId:'race-response',responseId:'race-response',status:'completed',result:'completed before cancel'};},cancel:async()=>({status:'completed'})});const p:any=await controller.createProject('Cancel race'),t:any=await controller.createTask(p.id,'Completed during cancel'),r:any=await controller.createRun(t.id);const assignmentPromise=controller.assign(r.id,'worker','finish task');await created;const assignment=(await store.load()).projects[0]!.tasks[0]!.runs[0]!.assignments[0]!;const canceled:any=await controller.cancelAssignment(assignment.id);expect(canceled.status).toBe('succeeded');expect((await store.load()).events.some(e=>e.type==='assignment.cancel_raced_terminal'&&e.data.source==='cancel-response')).toBe(true);release();await expect(assignmentPromise).resolves.toMatchObject({status:'succeeded',result:'completed before cancel'});}); }); diff --git a/tests/decision-digest.test.ts b/tests/decision-digest.test.ts new file mode 100644 index 0000000..4012b33 --- /dev/null +++ b/tests/decision-digest.test.ts @@ -0,0 +1,374 @@ +import { describe, expect, it, vi } from 'vitest'; +import { + DecisionDigestStore, EVIDENCE_CHANGED_MESSAGE, decisionKey, deriveDigestView, postDecision, + type DecisionSnapshot, type EvidenceIdentity, +} from '../ui/decision-digest.js'; + +// ── Test helpers: a mocked fetch and a settle tick (no DOM needed) ───────── + +const hex = (char: string) => char.repeat(64); +const identity = (over: Partial = {}): EvidenceIdentity => ({ + runId: 'run-1', workerResponseId: 'resp-1', validationId: 'val-1', recommendationId: 'rec-1', controllerActive: false, ...over, +}); +/** Body of GET /api/runs/:id/decision for the evidence in `identity`. */ +const checkpoint = (over: Record = {}) => ({ + runId: 'run-1', evidenceDigest: hex('a'), pinnedBaseCommit: 'c'.repeat(40), + workerResponseId: 'resp-1', validationId: 'val-1', recommendationId: 'rec-1', ...over, +}); +const reply = (status: number, body: unknown, statusText = ''): Response => + ({ ok: status >= 200 && status < 300, status, statusText, json: async () => body }) as unknown as Response; +const flush = () => new Promise(resolve => setTimeout(resolve, 0)); +function deferred() { + let resolve!: (value: T) => void; + const promise = new Promise(res => { resolve = res; }); + return { promise, resolve }; +} + +interface Call { method: string; url: string; body?: Record } +/** A fetch mock that records every call; `handler` answers it. */ +function mockFetch(handler: (call: Call) => Promise | Response) { + const calls: Call[] = []; + const fetchImpl = vi.fn(async (url: string, init?: RequestInit) => { + const call: Call = { method: init?.method ?? 'GET', url, body: typeof init?.body === 'string' ? JSON.parse(init.body) as Record : undefined }; + calls.push(call); + return handler(call); + }); + return { fetchImpl, calls, gets: () => calls.filter(c => c.method === 'GET'), posts: () => calls.filter(c => c.method === 'POST') }; +} +const snap = (store: DecisionDigestStore): DecisionSnapshot => store.getSnapshot(); + +describe('decision digest: fetch on render, send on click', () => { + it('fetches GET /api/runs/:id/decision when the panel renders and a click sends that digest with no extra GET', async () => { + const api = mockFetch(call => call.method === 'GET' ? reply(200, checkpoint()) : reply(200, { id: 'approval-1' })); + const store = new DecisionDigestStore(api.fetchImpl); + + store.sync(identity()); + expect(api.calls).toEqual([{ method: 'GET', url: '/api/runs/run-1/decision', body: undefined }]); + expect(snap(store).status).toBe('loading'); + await flush(); + expect(snap(store)).toMatchObject({ status: 'ready', digest: { evidenceDigest: hex('a'), pinnedBaseCommit: 'c'.repeat(40) } }); + + await store.decide(digest => postDecision('run-1', { approved: true, evidenceDigest: digest }, api.fetchImpl)); + expect(api.gets()).toHaveLength(1); + expect(api.posts()).toEqual([{ method: 'POST', url: '/api/runs/run-1/approve', body: { approved: true, evidenceDigest: hex('a') } }]); + }); + + it('sends the same digest for Reject, with the optional rationale', async () => { + const api = mockFetch(call => call.method === 'GET' ? reply(200, checkpoint()) : reply(200, {})); + const store = new DecisionDigestStore(api.fetchImpl); + store.sync(identity()); + await flush(); + + await store.decide(digest => postDecision('run-1', { approved: false, evidenceDigest: digest, rationale: 'Out of scope' }, api.fetchImpl)); + expect(api.gets()).toHaveLength(1); + expect(api.posts()[0]?.body).toEqual({ approved: false, evidenceDigest: hex('a'), rationale: 'Out of scope' }); + }); + + it('does nothing on a click before the digest is ready, or while it failed', async () => { + const pendingGet = deferred(); + const api = mockFetch(() => pendingGet.promise); + const store = new DecisionDigestStore(api.fetchImpl); + const submit = vi.fn(async () => ({})); + + store.sync(identity()); + await store.decide(submit); + expect(submit).not.toHaveBeenCalled(); + + pendingGet.resolve(reply(500, { error: 'boom' })); + await flush(); + expect(snap(store).status).toBe('error'); + await store.decide(submit); + expect(submit).not.toHaveBeenCalled(); + expect(api.gets()).toHaveLength(1); + }); + + it('ignores a second click while a decision is in flight', async () => { + const api = mockFetch(() => reply(200, checkpoint())); + const store = new DecisionDigestStore(api.fetchImpl); + store.sync(identity()); + await flush(); + const gate = deferred(); + const submit = vi.fn(async () => { await gate.promise; return {}; }); + + const first = store.decide(submit); + await store.decide(submit); + gate.resolve(); + await first; + expect(submit).toHaveBeenCalledTimes(1); + }); +}); + +describe('decision digest: keyed on the evidence identity', () => { + it('does not refetch for the same evidence (unrelated re-renders)', async () => { + const api = mockFetch(() => reply(200, checkpoint())); + const store = new DecisionDigestStore(api.fetchImpl); + store.sync(identity()); + await flush(); + store.sync(identity()); + store.sync({ ...identity() }); + await flush(); + expect(api.gets()).toHaveLength(1); + expect(snap(store).status).toBe('ready'); + }); + + it.each([ + ['Worker response', { workerResponseId: 'resp-2' }], + ['validation', { validationId: 'val-2' }], + ['Reviewer recommendation', { recommendationId: 'rec-2' }], + ])('refetches when the %s changes', async (_name, change) => { + // Each GET is answered for the evidence it was requested for. + let requests = 0; + const api = mockFetch(() => reply(200, ++requests === 1 ? checkpoint({ evidenceDigest: hex('1') }) : checkpoint({ evidenceDigest: hex('2'), ...change }))); + const store = new DecisionDigestStore(api.fetchImpl); + store.sync(identity()); + await flush(); + expect(snap(store).digest?.evidenceDigest).toBe(hex('1')); + + store.sync(identity(change)); + expect(snap(store).status).toBe('loading'); + await flush(); + expect(api.gets()).toHaveLength(2); + expect(snap(store)).toMatchObject({ status: 'ready', digest: { evidenceDigest: hex('2') } }); + }); + + it('refetches for a different run', async () => { + const api = mockFetch(call => reply(200, checkpoint({ runId: call.url.includes('run-2') ? 'run-2' : 'run-1' }))); + const store = new DecisionDigestStore(api.fetchImpl); + store.sync(identity()); + await flush(); + store.sync(identity({ runId: 'run-2' })); + await flush(); + expect(api.gets().map(c => c.url)).toEqual(['/api/runs/run-1/decision', '/api/runs/run-2/decision']); + expect(snap(store).digest?.runId).toBe('run-2'); + }); + + it('does not fetch while the controller is active, and fetches again once it stops', async () => { + const api = mockFetch(() => reply(200, checkpoint())); + const store = new DecisionDigestStore(api.fetchImpl); + store.sync(identity({ controllerActive: true })); + await flush(); + expect(api.calls).toHaveLength(0); + expect(snap(store).status).toBe('idle'); + + store.sync(identity()); + await flush(); + expect(api.gets()).toHaveLength(1); + expect(snap(store).status).toBe('ready'); + + // A new controller cycle with the very same evidence ids still refetches afterwards. + store.sync(identity({ controllerActive: true })); + expect(snap(store).status).toBe('idle'); + store.sync(identity()); + await flush(); + expect(api.gets()).toHaveLength(2); + }); + + it('derives the key from the three evidence ids plus the controller being inactive', () => { + expect(decisionKey(undefined)).toBeUndefined(); + expect(decisionKey(identity({ controllerActive: true }))).toBeUndefined(); + expect(decisionKey(identity())).toBe(decisionKey(identity())); + expect(decisionKey(identity())).not.toBe(decisionKey(identity({ validationId: 'val-2' }))); + expect(decisionKey(identity({ recommendationId: undefined }))).not.toBe(decisionKey(identity())); + }); + + it('never shows the previous digest for new evidence, even before the refetch starts', () => { + const ready: DecisionSnapshot = { status: 'ready', key: decisionKey(identity()), digest: checkpoint() }; + expect(deriveDigestView(ready, decisionKey(identity()))).toMatchObject({ status: 'ready', digest: { evidenceDigest: hex('a') } }); + const next = deriveDigestView(ready, decisionKey(identity({ validationId: 'val-2' }))); + expect(next.status).toBe('loading'); + expect(next.digest).toBeUndefined(); + expect(deriveDigestView(ready, undefined).status).toBe('idle'); + }); +}); + +describe('decision digest: out-of-order responses', () => { + it('ignores a stale response that arrives after the newer one', async () => { + const first = deferred(), second = deferred(); + let requests = 0; + const api = mockFetch(() => (++requests === 1 ? first : second).promise); + const store = new DecisionDigestStore(api.fetchImpl); + + store.sync(identity()); + store.sync(identity({ validationId: 'val-2' })); + second.resolve(reply(200, checkpoint({ validationId: 'val-2', evidenceDigest: hex('b') }))); + await flush(); + expect(snap(store)).toMatchObject({ status: 'ready', digest: { evidenceDigest: hex('b') } }); + + first.resolve(reply(200, checkpoint({ evidenceDigest: hex('a') }))); + await flush(); + expect(snap(store)).toMatchObject({ status: 'ready', digest: { evidenceDigest: hex('b') } }); + }); + + it('ignores a stale response that arrives before the newer one, and a stale failure too', async () => { + const first = deferred(), second = deferred(); + let requests = 0; + const api = mockFetch(() => (++requests === 1 ? first : second).promise); + const store = new DecisionDigestStore(api.fetchImpl); + + store.sync(identity()); + store.sync(identity({ validationId: 'val-2' })); + first.resolve(reply(500, { error: 'stale failure' })); + await flush(); + expect(snap(store).status).toBe('loading'); + + second.resolve(reply(200, checkpoint({ validationId: 'val-2', evidenceDigest: hex('b') }))); + await flush(); + expect(snap(store)).toMatchObject({ status: 'ready', digest: { evidenceDigest: hex('b') } }); + }); + + it('drops an in-flight response once the run stops being decidable', async () => { + const pendingGet = deferred(); + const api = mockFetch(() => pendingGet.promise); + const store = new DecisionDigestStore(api.fetchImpl); + store.sync(identity()); + store.sync(undefined); + pendingGet.resolve(reply(200, checkpoint())); + await flush(); + expect(snap(store).status).toBe('idle'); + expect(snap(store).digest).toBeUndefined(); + }); + + it('notifies subscribers on every state change', async () => { + const api = mockFetch(() => reply(200, checkpoint())); + const store = new DecisionDigestStore(api.fetchImpl); + const listener = vi.fn(); + const unsubscribe = store.subscribe(listener); + store.sync(identity()); + await flush(); + expect(listener).toHaveBeenCalledTimes(2); // loading, ready + unsubscribe(); + store.sync(identity({ validationId: 'val-2' })); + expect(listener).toHaveBeenCalledTimes(2); + }); +}); + +describe('decision digest: failures keep the buttons disabled and can be retried', () => { + it('reports HTTP errors with the server message and recovers on retry', async () => { + let fail = true; + const api = mockFetch(() => fail ? reply(409, { error: 'Run is not at a decision checkpoint' }) : reply(200, checkpoint())); + const store = new DecisionDigestStore(api.fetchImpl); + store.sync(identity()); + await flush(); + expect(snap(store)).toMatchObject({ status: 'error', error: 'Run is not at a decision checkpoint' }); + expect(snap(store).digest).toBeUndefined(); + + fail = false; + store.retry(); + expect(snap(store).status).toBe('loading'); + await flush(); + expect(api.gets()).toHaveLength(2); + expect(snap(store).status).toBe('ready'); + }); + + it('reports network failures and falls back to the HTTP status without a JSON error', async () => { + let requests = 0; + const api = mockFetch(() => { if (++requests === 1) throw new Error('Failed to fetch'); return reply(404, undefined, 'Not Found'); }); + const store = new DecisionDigestStore(api.fetchImpl); + store.sync(identity()); + await flush(); + expect(snap(store)).toMatchObject({ status: 'error', error: 'Failed to fetch' }); + store.retry(); + await flush(); + expect(snap(store)).toMatchObject({ status: 'error', error: '404 Not Found' }); + }); + + it.each([ + ['a malformed digest', checkpoint({ evidenceDigest: 'abc' })], + ['an incomplete checkpoint', { runId: 'run-1', evidenceDigest: hex('a') }], + ])('rejects %s', async (_name, body) => { + const api = mockFetch(() => reply(200, body)); + const store = new DecisionDigestStore(api.fetchImpl); + store.sync(identity()); + await flush(); + expect(snap(store).status).toBe('error'); + expect(snap(store).digest).toBeUndefined(); + }); + + it('refuses a digest issued for evidence other than what the panel shows', async () => { + // The server already moved to a newer validation; the view has not refreshed yet. + const api = mockFetch(() => reply(200, checkpoint({ validationId: 'val-newer' }))); + const store = new DecisionDigestStore(api.fetchImpl); + store.sync(identity()); + await flush(); + expect(snap(store).status).toBe('error'); + expect(snap(store).error).toMatch(/no longer matches what is on screen/); + await store.decide(async () => ({})); + expect(api.posts()).toHaveLength(0); + }); +}); + +describe('decision digest: a 409 (evidence changed)', () => { + const serverMessage = 'The evidence changed since you reviewed it; refresh and review the current result before deciding.'; + + it('surfaces the message, refreshes the state, refetches the digest, and lets the operator decide again', async () => { + let current = hex('a'); + const log: string[] = []; + const api = mockFetch(call => { + log.push(call.method); + if (call.method === 'GET') return reply(200, checkpoint({ evidenceDigest: current })); + if (call.body?.evidenceDigest === hex('a')) { current = hex('b'); return reply(409, { error: serverMessage }); } + return reply(200, { id: 'approval-1' }); + }); + const store = new DecisionDigestStore(api.fetchImpl); + store.sync(identity()); + await flush(); + + // Same shape as the App's decision handler: POST, then the normal state refresh. + const decideLikeApp = (approved: boolean) => store.decide(async digest => { + const outcome = await postDecision('run-1', { approved, evidenceDigest: digest }, api.fetchImpl); + log.push('refresh'); + return outcome; + }); + + await decideLikeApp(true); + expect(snap(store).conflict).toBe(serverMessage); + expect(snap(store).status).toBe('loading'); // buttons stay disabled until the new digest arrives + expect(snap(store).digest).toBeUndefined(); + await flush(); + expect(log).toEqual(['GET', 'POST', 'refresh', 'GET']); + expect(snap(store)).toMatchObject({ status: 'ready', conflict: serverMessage, digest: { evidenceDigest: hex('b') } }); + + // Deciding again uses the new digest and clears the message. + await decideLikeApp(true); + expect(api.posts().map(c => c.body?.evidenceDigest)).toEqual([hex('a'), hex('b')]); + expect(snap(store).conflict).toBeUndefined(); + expect(api.gets()).toHaveLength(2); + }); + + it('keeps the message while the view refreshes to new evidence', async () => { + const api = mockFetch(call => call.method === 'GET' ? reply(200, checkpoint()) : reply(409, { error: serverMessage })); + const store = new DecisionDigestStore(api.fetchImpl); + store.sync(identity()); + await flush(); + await store.decide(digest => postDecision('run-1', { approved: false, evidenceDigest: digest }, api.fetchImpl)); + await flush(); + api.fetchImpl.mockImplementation(async () => reply(200, checkpoint({ validationId: 'val-2' }))); + store.sync(identity({ validationId: 'val-2' })); + await flush(); + expect(snap(store)).toMatchObject({ status: 'ready', conflict: serverMessage }); + // A different run starts clean. + api.fetchImpl.mockImplementation(async () => reply(200, checkpoint({ runId: 'run-2' }))); + store.sync(identity({ runId: 'run-2' })); + await flush(); + expect(snap(store).conflict).toBeUndefined(); + }); + + it('postDecision resolves the conflict message, with a standard fallback', async () => { + const withBody = mockFetch(() => reply(409, { error: serverMessage })); + expect(await postDecision('run-1', { approved: true, evidenceDigest: hex('a') }, withBody.fetchImpl)).toEqual({ conflict: serverMessage }); + const withoutBody = mockFetch(() => ({ ok: false, status: 409, statusText: 'Conflict', json: async () => { throw new Error('no body'); } }) as unknown as Response); + expect(await postDecision('run-1', { approved: true, evidenceDigest: hex('a') }, withoutBody.fetchImpl)).toEqual({ conflict: EVIDENCE_CHANGED_MESSAGE }); + }); + + it('other errors behave as today: the message is thrown, with no conflict and no refetch', async () => { + const api = mockFetch(call => call.method === 'GET' ? reply(200, checkpoint()) : reply(400, { error: 'evidenceDigest must be a 64-character hex string' })); + const store = new DecisionDigestStore(api.fetchImpl); + store.sync(identity()); + await flush(); + await expect(store.decide(digest => postDecision('run-1', { approved: true, evidenceDigest: digest }, api.fetchImpl))) + .rejects.toThrow('evidenceDigest must be a 64-character hex string'); + await flush(); + expect(snap(store)).toMatchObject({ status: 'ready', conflict: undefined }); + expect(api.gets()).toHaveLength(1); + }); +}); diff --git a/tests/decision-helper.ts b/tests/decision-helper.ts new file mode 100644 index 0000000..b34c84b --- /dev/null +++ b/tests/decision-helper.ts @@ -0,0 +1,6 @@ +import type { Controller } from '../src/controller.js'; + +/** The digest an operator would have fetched from GET /api/runs/:id/decision when the decision panel rendered. */ +export async function decisionDigest(controller: Controller, runId: string): Promise { + return (await controller.decisionEvidence(runId)).evidenceDigest; +} diff --git a/tests/project-planner.test.ts b/tests/project-planner.test.ts index fb4d655..0ba1beb 100644 --- a/tests/project-planner.test.ts +++ b/tests/project-planner.test.ts @@ -137,7 +137,7 @@ describe('project Planner workflow',()=>{ (controller as any).verifyWorkerOutput=async(runId:string,workerId:string)=>{let result:any;await store.mutate(state=>{const run=state.projects[0]!.tasks.flatMap(task=>task.runs).find(item=>item.id===runId)!,worker=run.assignments.find(item=>item.id===workerId)!,commands=run.validationCommands!,stamp=new Date().toISOString(),digest=createHash('sha256').update(JSON.stringify(commands.map(command=>({args:command.args,command:command.command,name:command.name})))).digest('hex');run.workerEvidence={provenance:'bridge_snapshot',workerAssignmentId:worker.id,responseId:worker.responseId!,workspaceId:run.workspaceId,pinnedBaseCommit:run.pinnedBaseCommit!,completeSnapshot:{reportedComplete:true,reportedErrors:0,entryCount:1},scopeVerified:true,allowedScope:run.allowedScope!,entries:[],changes:[{path:'README.md',kind:'modify'}],reviewDiff:'diff --git a/README.md b/README.md\n+Fixture note',acceptance:'not_decided'};run.validation={id:'fixture-validation',status:'passed',passed:true,reportedPassed:true,checks:commands.map(command=>({name:command.name,passed:true})),observations:commands.map(command=>({name:command.name,command:command.command,args:command.args,exitCode:0,timedOut:false,output:'passed',outputTruncated:false,startedAt:stamp,finishedAt:stamp,passed:true})),policy:{requireAllChecksPass:true,configuredCheckCount:commands.length,commandDigest:digest},gitEvidence:{status:'verified',commit:run.pinnedBaseCommit!,changedPaths:['README.md'],submittedAt:new Date().toISOString()},createdAt:stamp};run.sessions.orchestrator.uhpSessionId=run.assignments.filter(item=>item.roleId==='orchestrator').at(-1)?.sessionId;run.orchestratorInbox=(controller as any).buildOrchestratorInbox(runId,run);result={workerEvidence:run.workerEvidence,validation:run.validation};});return result;}; const started=await controller.startTaskWork(readmeTask!.id,{scope:['README.md'],validationCommands:[{name:'task README check',command:'true',args:[]}],budgets:{roleTurns:{planner:1,orchestrator:1,worker:1,reviewer:1},workerAttempts:1}});expect(started.id).toMatch(/^run_/); let completed:any;for(let i=0;i<500;i++){const latest=(await store.load()).projects[0]!.tasks.find(task=>task.id===readmeTask!.id)!.runs[0]!;if(latest.controller&&!latest.controller.active){completed=latest;expect(latest.workerEvidence).toMatchObject({workerAssignmentId:expect.any(String),pinnedBaseCommit:workspace.head,allowedScope:['README.md']});expect(latest.validation).toMatchObject({status:'passed',policy:{configuredCheckCount:1},observations:[{name:'task README check',command:'true'}]});expect(latest.orchestratorInbox?.runId).toBe(latest.id);break;}await new Promise(resolve=>setTimeout(resolve,5));if(i===499)throw new Error('The explicitly started task did not finish its simulated run');} - const saved=(await store.load()).projects[0]!,selected=saved.tasks.find(task=>task.id===readmeTask!.id)!,other=saved.tasks.find(task=>task.id!==readmeTask!.id)!;expect(selected.runs).toHaveLength(1);expect(other.runs).toHaveLength(0);expect(selected.runs[0]!.workspaceId).toBe('task-isolated-workspace');expect(selected.runs[0]!.validationCommands).toEqual([{name:'task README check',command:'true',args:[]}]);const reviewer=completed.assignments.find((assignment:any)=>assignment.roleId==='reviewer');expect(reviewer).toMatchObject({reviewerInboxId:completed.orchestratorInbox.id,reviewerEvidenceDigest:completed.orchestratorInbox.evidenceDigest});expect(submissions.map(call=>call.roleId)).toEqual(['planner','orchestrator','worker','reviewer']); + const saved=(await store.load()).projects[0]!,selected=saved.tasks.find(task=>task.id===readmeTask!.id)!,other=saved.tasks.find(task=>task.id!==readmeTask!.id)!;expect(selected.runs).toHaveLength(1);expect(other.runs).toHaveLength(0);expect(selected.runs[0]!.workspaceId).toBe('task-isolated-workspace');expect(selected.runs[0]!.validationCommands).toEqual([{name:'task README check',command:'true',args:[],network:false}]);const reviewer=completed.assignments.find((assignment:any)=>assignment.roleId==='reviewer');expect(reviewer).toMatchObject({reviewerInboxId:completed.orchestratorInbox.id,reviewerEvidenceDigest:completed.orchestratorInbox.evidenceDigest});expect(submissions.map(call=>call.roleId)).toEqual(['planner','orchestrator','worker','reviewer']); }); it('blocks overlapping work until promotion and requires the current Git base',async()=>{ diff --git a/tests/recorded-integration.test.ts b/tests/recorded-integration.test.ts index aa8162b..fe9b654 100644 --- a/tests/recorded-integration.test.ts +++ b/tests/recorded-integration.test.ts @@ -1,13 +1,15 @@ import { execFileSync } from 'node:child_process'; import { createHash } from 'node:crypto'; -import { mkdtemp, readFile, rm } from 'node:fs/promises'; -import { tmpdir } from 'node:os'; +import { chmod, mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; +import { homedir, tmpdir } from 'node:os'; import { join, resolve } from 'node:path'; -import { afterEach, describe, expect, it } from 'vitest'; +import { afterEach, describe, expect, it, vi } from 'vitest'; import { Controller, type UhpAdapter } from '../src/controller.js'; import { JsonStore } from '../src/store.js'; import { promoteSnapshotToGit } from '../src/git-promotion.js'; import { formatReviewDiff } from '../src/verified-workspace.js'; +import { decisionDigest } from './decision-helper.js'; +import { defaultNetworkAccess } from '../src/validation-sandbox.js'; const dirs:string[]=[]; const bundle=resolve('tests/fixtures/recorded-worker-base.bundle'); @@ -30,14 +32,14 @@ async function setup(validationPass=true,reviewVerdict:'recommend'|'reject'='rec if(assignWorker){await store.mutate(s=>{s.projects[0]!.tasks[0]!.runs[0]!.workspaceId='recorded-workspace-fixture';});await controller.addGuidance(run.id,'Replay the recorded bounded README task.');const orchestration:any=await controller.orchestrate(run.id,'Prepare one bounded Worker assignment for the recorded README.md task.');worker=await controller.dispatchWorkerProposal(run.id,orchestration.proposal.id);} return {dir,repoPath,recorded,store,controller,runId:run.id,workerId:worker?.id,uhpCalls}; } -afterEach(async()=>{await Promise.all(dirs.splice(0).map(dir=>rm(dir,{recursive:true,force:true})));}); +afterEach(async()=>{vi.unstubAllEnvs();await Promise.all(dirs.splice(0).map(dir=>rm(dir,{recursive:true,force:true})));}); describe('recorded Worker integration fixture (simulated Reviewer; no live model call)',()=>{ async function approveRecordedRun(fixture: Awaited>) { await fixture.controller.replayRecordedWorkerOutput(fixture.runId,fixture.workerId,fixture.recorded); const recommendation:any=await fixture.controller.requestReviewer(fixture.runId,'simulated_fixture'); expect(recommendation).toMatchObject({provenance:'simulated_fixture',verdict:'recommend',reviewMode:'read_only'}); - return fixture.controller.approveRun(fixture.runId,{approved:true,rationale:'Accepted the verified recorded result after inspecting its simulated review.'}); + return fixture.controller.approveRun(fixture.runId,{approved:true,evidenceDigest:await decisionDigest(fixture.controller,fixture.runId),rationale:'Accepted the verified recorded result after inspecting its simulated review.'}); } it('keeps the human decision immutable and distinct from an explicitly applied Git result',async()=>{ @@ -45,7 +47,7 @@ describe('recorded Worker integration fixture (simulated Reviewer; no live model const approval:any=await approveRecordedRun(fixture); expect(approval).toMatchObject({approved:true,decision:'approved',evidenceCommit:fixture.recorded.baseCommit}); expect((await fixture.store.load()).projects[0]!.tasks[0]!.runs[0]!.promotion).toMatchObject({status:'not_started'}); - await expect(fixture.controller.approveRun(fixture.runId,{approved:false,rationale:'Attempt to replace approval'})).rejects.toThrow('A human decision is final for this run'); + await expect(fixture.controller.approveRun(fixture.runId,{approved:false,evidenceDigest:approval.evidenceDigest,rationale:'Attempt to replace approval'})).rejects.toThrow('A human decision is final for this run'); const promoted:any=await fixture.controller.promoteRun(fixture.runId,{destinationBranch:'foreman/results/controller-success'}); expect(promoted.approval).toEqual(approval); @@ -57,6 +59,37 @@ describe('recorded Worker integration fixture (simulated Reviewer; no live model expect(after.promotion).toEqual(promoted.promotion); }); + it('promotes a legacy run whose stored install command has no network flag: the default applies when it runs and the command digest is untouched',async()=>{ + // A fake pnpm under a re-exposed toolchain prefix, so no real install runs. + const home=await mkdtemp(join(homedir(),'.foreman-legacy-install-'));dirs.push(home); + const bin=join(home,'versions','v1','bin');await mkdir(bin,{recursive:true}); + await writeFile(join(bin,'pnpm'),'#!/bin/sh\necho "fake install $*"\n');await chmod(join(bin,'pnpm'),0o755); + vi.stubEnv('PATH',`${bin}:${process.env.PATH??''}`); + const fixture=await setup(); + // Legacy shape: a run stored before the network flag existed, with an install and a check, neither carrying a network field. + const legacy=[{name:'install',command:'pnpm',args:['install','--frozen-lockfile']},{name:'recorded README assertion',command:process.execPath,args:['-e',"const fs=require('fs');if(!fs.readFileSync('README.md','utf8').includes('Bridge smoke'))process.exit(2);console.log('recorded README assertion passed')"]}]; + await fixture.store.mutate(s=>{s.projects[0]!.tasks[0]!.runs[0]!.validationCommands=structuredClone(legacy);}); + await approveRecordedRun(fixture); + const stored=async():Promise=>(await fixture.store.load()).projects[0]!.tasks[0]!.runs[0]!; + const validated=await stored(); + // The install ran with the network by default, the check without, and the observation says so. + expect(validated.validation.observations.map((o:any)=>[o.name,o.exitCode,o.network])).toEqual([['install',0,true],['recorded README assertion',0,false]]); + expect(validated.validation.observations[0].output).toBe('fake install install --frozen-lockfile\n'); + // The stored commands were not rewritten, so the digest the approval is bound to is the digest of the legacy list. + const sortKeys=(value:unknown):unknown=>Array.isArray(value)?value.map(sortKeys):value&&typeof value==='object'?Object.fromEntries(Object.entries(value).filter(([,v])=>v!==undefined).sort(([a],[b])=>a.localeCompare(b)).map(([k,v])=>[k,sortKeys(v)])):value; + const digest=(commands:unknown)=>createHash('sha256').update(JSON.stringify(sortKeys(commands))).digest('hex'); + expect(validated.validationCommands).toEqual(legacy); + expect(validated.validationCommands.some((command:any)=>'network' in command)).toBe(false); + expect(validated.validation.policy.commandDigest).toBe(digest(legacy)); + // Filling the flag into the stored list would have changed that digest and broken promotion. + expect(digest(legacy.map(command=>({...command,network:defaultNetworkAccess(command.command,command.args)})))).not.toBe(validated.validation.policy.commandDigest); + const promoted:any=await fixture.controller.promoteRun(fixture.runId,{destinationBranch:'foreman/results/legacy-install'}); + expect(promoted.promotion).toMatchObject({status:'applied',destinationBranch:'foreman/results/legacy-install',resultCommit:expect.any(String)}); + const after=await stored(); + expect(after.validationCommands).toEqual(legacy); + expect(after.validation.policy.commandDigest).toBe(digest(legacy)); + }); + it('recovers persisted promotion intent after Git created the result branch but before state recorded applied',async()=>{ const fixture=await setup(); await approveRecordedRun(fixture); @@ -161,9 +194,10 @@ describe('recorded Worker integration fixture (simulated Reviewer; no live model const replayed=await afterReviewRestart.replayRecordedWorkerOutput(fixture.runId,fixture.workerId,fixture.recorded); expect(replayed).toMatchObject({workerEvidence:{provenance:'recorded_replay',acceptance:'not_decided'},validation:{status:'passed'}}); await expect(afterReviewRestart.requestReviewer(fixture.runId)).rejects.toThrow('already exists'); - const rejected=await afterReviewRestart.approveRun(fixture.runId,{approved:false,rationale:'Human rejected after inspecting simulated recommendation'}); + const reviewedDigest=await decisionDigest(afterReviewRestart,fixture.runId); + const rejected=await afterReviewRestart.approveRun(fixture.runId,{approved:false,evidenceDigest:reviewedDigest,rationale:'Human rejected after inspecting simulated recommendation'}); expect(rejected).toMatchObject({decision:'rejected',approved:false}); - await expect(afterReviewRestart.approveRun(fixture.runId,{approved:true,rationale:'Attempt to replace the recorded human rejection'})).rejects.toThrow('A human decision is final for this run'); + await expect(afterReviewRestart.approveRun(fixture.runId,{approved:true,evidenceDigest:reviewedDigest,rationale:'Attempt to replace the recorded human rejection'})).rejects.toThrow('A human decision is final for this run'); await expect(afterReviewRestart.assign(fixture.runId,'planner','new assignment after human rejection')).rejects.toThrow('No assignments are allowed after the human decision'); }); @@ -193,3 +227,96 @@ describe('recorded Worker integration fixture (simulated Reviewer; no live model expect(outscope.uhpCalls.filter(x=>x.role==='reviewer')).toHaveLength(0); }); }); + +describe('human decision is bound to the evidence digest the operator reviewed',()=>{ + const staleMessage='The evidence changed since you reviewed it; refresh and review the current result before deciding.'; + async function atCheckpoint(verdict:'recommend'|'reject'='recommend') { + const fixture=await setup(true,verdict); + await fixture.controller.replayRecordedWorkerOutput(fixture.runId,fixture.workerId,fixture.recorded); + await fixture.controller.requestReviewer(fixture.runId,'simulated_fixture'); + return fixture; + } + const savedRun=async(fixture:Awaited>):Promise=>(await fixture.store.load()).projects[0]!.tasks[0]!.runs[0]!; + const decisionEvents=async(fixture:Awaited>)=>(await fixture.store.load()).events.filter(e=>e.type==='run.human_approved'||e.type==='run.human_rejected'); + + it('serves the evidence identity and the exact digest that approval records',async()=>{ + const fixture=await atCheckpoint(); + const evidence=await fixture.controller.decisionEvidence(fixture.runId); + const run=await savedRun(fixture); + expect(evidence).toEqual({runId:fixture.runId,evidenceDigest:expect.stringMatching(/^[0-9a-f]{64}$/),pinnedBaseCommit:fixture.recorded.baseCommit,workerResponseId:fixture.recorded.responseId,validationId:run.validation.id,recommendationId:run.reviewerRecommendation.id}); + expect(await fixture.controller.decisionEvidence(fixture.runId)).toEqual(evidence); + const approval:any=await fixture.controller.approveRun(fixture.runId,{approved:true,evidenceDigest:evidence.evidenceDigest}); + expect(approval).toMatchObject({approved:true,decision:'approved',evidenceDigest:evidence.evidenceDigest,evidenceCommit:fixture.recorded.baseCommit}); + expect((await savedRun(fixture)).promotion).toMatchObject({status:'not_started',evidenceDigest:evidence.evidenceDigest}); + }); + + it('accepts an uppercase copy of the digest and records the canonical lowercase value',async()=>{ + const fixture=await atCheckpoint(); + const {evidenceDigest}=await fixture.controller.decisionEvidence(fixture.runId); + const approval:any=await fixture.controller.approveRun(fixture.runId,{approved:true,evidenceDigest:evidenceDigest.toUpperCase()}); + expect(approval.evidenceDigest).toBe(evidenceDigest); + }); + + it('rejects a missing or malformed digest with 400 for both approval and rejection and records nothing',async()=>{ + const fixture=await atCheckpoint(); + const {evidenceDigest}=await fixture.controller.decisionEvidence(fixture.runId); + const before=await savedRun(fixture); + // Input validation comes before any run lookup, so even an unknown run reports the missing digest. + await expect(fixture.controller.approveRun('does-not-exist',{approved:true} as any)).rejects.toMatchObject({statusCode:400}); + for(const approved of [true,false]){ + const missing:any={approved}; + await expect(fixture.controller.approveRun(fixture.runId,missing)).rejects.toMatchObject({statusCode:400}); + for(const bad of ['',evidenceDigest.slice(1),`${evidenceDigest}0`,'z'.repeat(64),` ${evidenceDigest}`,42,null,{digest:evidenceDigest}]){ + await expect(fixture.controller.approveRun(fixture.runId,{approved,evidenceDigest:bad as any})).rejects.toMatchObject({statusCode:400}); + } + } + const after=await savedRun(fixture); + expect(after.approval).toBeUndefined();expect(after.status).toBe(before.status);expect(after.workerEvidence.acceptance).toBe('not_decided');expect(after.promotion).toBeUndefined(); + expect(await decisionEvents(fixture)).toEqual([]); + }); + + it('refuses a digest from before a Reviewer retry with 409, records no decision, and accepts the refreshed digest',async()=>{ + const fixture=await atCheckpoint('reject'); + const reviewed=await fixture.controller.decisionEvidence(fixture.runId); + // A real evidence change: the operator's tab still shows the first recommendation while the Reviewer is retried. + await fixture.controller.retryReviewer(fixture.runId); + await expect(fixture.controller.decisionEvidence(fixture.runId)).rejects.toMatchObject({statusCode:409}); + await fixture.controller.requestReviewer(fixture.runId,'simulated_fixture'); + const current=await fixture.controller.decisionEvidence(fixture.runId); + expect(current.recommendationId).not.toBe(reviewed.recommendationId); + expect(current.evidenceDigest).not.toBe(reviewed.evidenceDigest); + for(const approved of [true,false])await expect(fixture.controller.approveRun(fixture.runId,{approved,evidenceDigest:reviewed.evidenceDigest})).rejects.toMatchObject({statusCode:409,message:staleMessage}); + const run=await savedRun(fixture); + expect(run.approval).toBeUndefined();expect(run.workerEvidence.acceptance).toBe('not_decided');expect(run.status).toBe('awaiting_approval');expect(run.promotion).toBeUndefined(); + expect(await decisionEvents(fixture)).toEqual([]); + await expect(fixture.controller.approveRun(fixture.runId,{approved:true,evidenceDigest:current.evidenceDigest})).resolves.toMatchObject({approved:true,evidenceDigest:current.evidenceDigest}); + }); + + it('requires the digest for rejection as well and binds the rejection to it',async()=>{ + const fixture=await atCheckpoint('reject'); + const {evidenceDigest}=await fixture.controller.decisionEvidence(fixture.runId); + await expect(fixture.controller.approveRun(fixture.runId,{approved:false,rationale:'no digest'} as any)).rejects.toMatchObject({statusCode:400}); + expect((await savedRun(fixture)).approval).toBeUndefined(); + const rejected:any=await fixture.controller.approveRun(fixture.runId,{approved:false,evidenceDigest,rationale:'Not what I reviewed'}); + expect(rejected).toMatchObject({approved:false,decision:'rejected',evidenceDigest}); + expect((await decisionEvents(fixture)).map(e=>e.type)).toEqual(['run.human_rejected']); + }); + + it('does not serve a digest unless the run is at a decision checkpoint',async()=>{ + const noWorker=await setup(true,'recommend',false); + await expect(noWorker.controller.decisionEvidence('does-not-exist')).rejects.toMatchObject({statusCode:404}); + await expect(noWorker.controller.decisionEvidence(noWorker.runId)).rejects.toMatchObject({statusCode:409,message:expect.stringContaining('Human decision requires')}); + const noReviewer=await setup(); + await noReviewer.controller.replayRecordedWorkerOutput(noReviewer.runId,noReviewer.workerId,noReviewer.recorded); + await expect(noReviewer.controller.decisionEvidence(noReviewer.runId)).rejects.toMatchObject({statusCode:409}); + await expect(noReviewer.controller.approveRun(noReviewer.runId,{approved:true,evidenceDigest:'0'.repeat(64)})).rejects.toMatchObject({statusCode:409}); + + const active=await atCheckpoint(); + await active.store.mutate(s=>{s.projects[0]!.tasks[0]!.runs[0]!.controller={startedAt:new Date().toISOString(),active:true,phase:'reviewing',budgets:{roleTurns:{planner:1,orchestrator:1,worker:1,reviewer:1},workerAttempts:1}};}); + await expect(active.controller.decisionEvidence(active.runId)).rejects.toMatchObject({statusCode:409,message:expect.stringContaining('decision checkpoint')}); + + const decided=await atCheckpoint(); + await decided.controller.approveRun(decided.runId,{approved:true,evidenceDigest:await decisionDigest(decided.controller,decided.runId)}); + await expect(decided.controller.decisionEvidence(decided.runId)).rejects.toMatchObject({statusCode:409,message:'A human decision is final for this run'}); + }); +}); diff --git a/tests/repository-inspector.test.ts b/tests/repository-inspector.test.ts index c872ae9..c437f3e 100644 --- a/tests/repository-inspector.test.ts +++ b/tests/repository-inspector.test.ts @@ -282,6 +282,59 @@ jobs: }); }); + describe('network flag on suggestions', () => { + /** A committed repository with exactly the given files (no default lockfile). */ + async function repoWithFiles(files: Record): Promise { + const root = await mkdtemp(join(tmpdir(), 'foreman-inspector-net-')); + dirs.push(root); + git(root, 'init', '-q'); + git(root, 'config', 'user.name', 'Test'); + git(root, 'config', 'user.email', 'test@example.invalid'); + for (const [name, content] of Object.entries(files)) { + await mkdir(dirname(join(root, name)), { recursive: true }); + await writeFile(join(root, name), content); + } + git(root, 'add', '-A'); + git(root, 'commit', '-qm', 'init'); + return root; + } + const networkByName = (suggestions: Array<{ name: string; network?: boolean }>) => Object.fromEntries(suggestions.map(c => [c.name, c.network === true])); + const CI = 'on: [push]\njobs:\n test:\n runs-on: ubuntu-latest\n steps:\n - run: pnpm run lint\n - run: pnpm test\n'; + + it('marks only the pnpm install network-enabled when the suggestions come from CI', async () => { + const repo = await repoWithPackage({ scripts: { lint: 'eslint', test: 'vitest' }, workflowFiles: { 'ci.yml': CI } }); + const { suggestedValidationCommands } = await inspectRepository(repo); + expect(suggestedValidationCommands.map(c => c.args.join(' '))).toEqual(['install --frozen-lockfile', 'run lint', 'run test']); + expect(networkByName(suggestedValidationCommands)).toEqual({ 'Install dependencies': true, Lint: false, Tests: false }); + expect(suggestedValidationCommands.find(c => c.name === 'Install dependencies')?.network).toBe(true); + for (const check of suggestedValidationCommands.filter(c => c.name !== 'Install dependencies')) expect(check.network === undefined || check.network === false).toBe(true); + }); + + it('does the same for the package-script fallback and for npm ci', async () => { + const pnpm = await inspectRepository(await repoWithPackage({ scripts: { test: 'vitest', typecheck: 'tsc', build: 'esbuild' } })); + expect(networkByName(pnpm.suggestedValidationCommands)).toEqual({ 'Install dependencies': true, Typecheck: false, Tests: false, Build: false }); + const npm = await inspectRepository(await repoWithFiles({ 'package.json': JSON.stringify({ scripts: { test: 'vitest', build: 'tsc' } }), 'package-lock.json': '{}' })); + expect(npm.suggestedValidationCommands.map(c => [c.command, c.args.join(' ')])).toEqual([['npm', 'ci'], ['npm', 'run test'], ['npm', 'run build']]); + expect(networkByName(npm.suggestedValidationCommands)).toEqual({ 'Install dependencies': true, Tests: false, Build: false }); + }); + + it('keeps a package repository without a lockfile fully offline', async () => { + const { suggestedValidationCommands } = await inspectRepository(await repoWithFiles({ 'package.json': JSON.stringify({ scripts: { test: 'vitest' } }) })); + expect(suggestedValidationCommands.map(c => c.name)).toEqual(['Tests']); + expect(networkByName(suggestedValidationCommands)).toEqual({ Tests: false }); + }); + + it('lets cargo and go fetch dependencies but keeps python offline', async () => { + const cargo = await inspectRepository(await repoWithFiles({ 'Cargo.toml': '[package]\nname = "x"\n' })); + const go = await inspectRepository(await repoWithFiles({ 'go.mod': 'module x\n' })); + const python = await inspectRepository(await repoWithFiles({ 'pyproject.toml': '[project]\nname = "x"\n' })); + expect(cargo.suggestedValidationCommands).toMatchObject([{ command: 'cargo', args: ['test'], network: true }]); + expect(go.suggestedValidationCommands).toMatchObject([{ command: 'go', args: ['test', './...'], network: true }]); + expect(python.suggestedValidationCommands).toHaveLength(1); + expect(python.suggestedValidationCommands[0]?.network === true).toBe(false); + }); + }); + describe('ciChecksNotConfigured', () => { it('returns CI scripts not covered by any configured command', () => { const ciScripts = ['format:check', 'lint', 'typecheck', 'test', 'build']; diff --git a/tests/ui.test.tsx b/tests/ui.test.tsx index a2e8b88..72bdf75 100644 --- a/tests/ui.test.tsx +++ b/tests/ui.test.tsx @@ -1,11 +1,13 @@ -import { createElement } from 'react'; +import { createElement, type ReactElement, type ReactNode } from 'react'; import { renderToStaticMarkup } from 'react-dom/server'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import { App, debounce, type State, type TaskStartPreview } from '../ui/main.js'; -import { DecisionPanel, type DecisionPanelProps } from '../ui/decision-panel.js'; +import { DecisionPanel, DecisionDigestNotice, type DecisionPanelProps } from '../ui/decision-panel.js'; +import type { DecisionDigestView } from '../ui/decision-digest.js'; import { ChecksPipeline, type StationObservation, type GithubCheckEntry } from '../ui/checks-pipeline.js'; import { parseChecksSummary } from '../ui/check-output.js'; import { PrDraftPanel, type PrDraftData } from '../ui/pr-draft.js'; +import { NetworkToggle, RepoChecksEditor, checksFromSuggestions, validationCommandsPayload, type RepoCheck } from '../ui/repo-checks.js'; import { Badge, toneForStatus } from '../ui/badge.js'; describe('debounce helper',()=>{ @@ -406,11 +408,18 @@ describe('usage dock refresh stability',()=>{ // ── Shared fixture helpers ───────────────────────────────────────────────── +/** Evidence-digest state as the panel receives it from useDecisionDigest. */ +const digestView=(over:Partial={}):DecisionDigestView=>({ + status:'ready', + digest:{runId:'run-1',evidenceDigest:'a'.repeat(64),pinnedBaseCommit:'c'.repeat(40),workerResponseId:'resp-1',validationId:'val-1',recommendationId:'rec-1'}, + retry:()=>{},decide:async()=>{},...over, +}); + const baseDecisionProps:DecisionPanelProps={ filesChanged:3,allowedScope:['src/'],validationPassedCount:2,validationTotalCount:2, validationPassed:true,observations:[],reviewerVerdict:'recommend', reviewerRationaleSnippet:'Looks good.',approvable:true,reviewerRecommends:true, - pending:false,onApprove:()=>{},onReject:()=>{}, + pending:false,digest:digestView(),onApprove:()=>{},onReject:()=>{}, approved:false,promoted:false,branchPushed:false,prOpen:false,prMerged:false, }; @@ -480,6 +489,117 @@ describe('decision panel',()=>{ }); }); +// ── Decision panel: evidence digest states ───────────────────────────────── + +/** Opening tags of the Approve/Reject/Retry buttons, keyed by aria-label. */ +const decisionButtons=(html:string)=>{ + const tag=(label:string)=>html.match(new RegExp(`]*aria-label="${label}"[^>]*>`))?.[0]; + return {approve:tag('Approve result'),reject:tag('Reject result'),retry:tag('Retry loading the evidence digest')}; +}; + +describe('decision panel evidence digest',()=>{ + it('enables Approve and Reject once the digest is ready',()=>{ + const html=renderToStaticMarkup(createElement(DecisionPanel,baseDecisionProps)); + const {approve,reject,retry}=decisionButtons(html); + expect(approve).toBeDefined(); + expect(approve).not.toContain('disabled'); + expect(reject).not.toContain('disabled'); + expect(retry).toBeUndefined(); + expect(html).toContain('Approving records your decision only'); + expect(html).not.toContain('Loading the evidence digest'); + expect(html).not.toContain('class="workflow-stop decision-conflict"'); + }); + + it('disables both buttons with a reason while the digest is loading',()=>{ + const html=renderToStaticMarkup(createElement(DecisionPanel,{...baseDecisionProps,digest:digestView({status:'loading',digest:undefined})})); + const {approve,reject,retry}=decisionButtons(html); + expect(approve).toContain('disabled=""'); + expect(reject).toContain('disabled=""'); + expect(retry).toBeUndefined(); + expect(html).toContain('class="decision-approve-note" aria-live="polite">Loading the evidence digest…

'); + expect(html).not.toContain('Approving records your decision only'); + expect(html).toContain('data-digest="loading"'); + }); + + it('disables both buttons and offers a retry when the digest failed',()=>{ + const html=renderToStaticMarkup(createElement(DecisionPanel,{...baseDecisionProps,digest:digestView({status:'error',digest:undefined,error:'Run is not at a decision checkpoint.'})})); + const {approve,reject,retry}=decisionButtons(html); + expect(approve).toContain('disabled=""'); + expect(reject).toContain('disabled=""'); + expect(retry).toBeDefined(); + expect(retry).not.toContain('disabled'); + expect(html).toContain('Evidence digest unavailable: Run is not at a decision checkpoint.

'); + expect(html).toContain('>Retry<'); + expect(html).toContain('data-digest="error"'); + }); + + it('shows the server message prominently after a 409 while the new digest loads, and re-enables once it arrives',()=>{ + const message='The evidence changed since you reviewed it; refresh and review the current result before deciding.'; + const reloading=renderToStaticMarkup(createElement(DecisionPanel,{...baseDecisionProps,digest:digestView({status:'loading',digest:undefined,conflict:message})})); + expect(reloading).toContain(''); + expect(decisionButtons(reloading).approve).toContain('disabled=""'); + expect(decisionButtons(reloading).reject).toContain('disabled=""'); + + const ready=renderToStaticMarkup(createElement(DecisionPanel,{...baseDecisionProps,digest:digestView({conflict:message})})); + expect(ready).toContain(message); + expect(decisionButtons(ready).approve).not.toContain('disabled'); + expect(decisionButtons(ready).reject).not.toContain('disabled'); + }); + + it('keeps the buttons disabled while the controller runs, with a reason',()=>{ + const html=renderToStaticMarkup(createElement(DecisionPanel,{...baseDecisionProps,controllerActive:true,digest:digestView({status:'idle',digest:undefined})})); + expect(decisionButtons(html).approve).toContain('disabled=""'); + expect(decisionButtons(html).reject).toContain('disabled=""'); + expect(html).toContain('Decisions are paused while the controller is running.'); + }); + + it('does not add digest messaging to a result that is not approvable',()=>{ + const html=renderToStaticMarkup(createElement(DecisionPanel,{...baseDecisionProps,approvable:false,digest:digestView({status:'loading',digest:undefined})})); + expect(html).not.toContain('Loading the evidence digest'); + expect(html).toContain('data-digest="idle"'); + expect(decisionButtons(html).approve).toContain('disabled=""'); + }); + + it('shows no decision controls or digest messaging once approved',()=>{ + const html=renderToStaticMarkup(createElement(DecisionPanel,{...baseDecisionProps,approved:true,digest:digestView({status:'idle',digest:undefined,conflict:'stale'})})); + expect(html).not.toContain('>Approve result<'); + expect(html).not.toContain('decision-conflict'); + }); + + it('sends the digest fetched for the evidence on screen when Approve or Reject is clicked',async()=>{ + const view=digestView(); + // Stands in for the store: hands the ready digest to the submit callback, like DecisionDigestStore.decide. + const decide=vi.fn(async(submit:(digest:string)=>unknown)=>{await submit(view.digest!.evidenceDigest);}); + const onApprove=vi.fn(),onReject=vi.fn(); + const findElement=(node:unknown,match:(props:Record)=>boolean):{props:Record}|undefined=>{ + if(Array.isArray(node)){for(const child of node){const hit=findElement(child,match);if(hit)return hit;}return undefined;} + if(typeof node!=='object'||node===null||!('props' in node))return undefined; + const props=(node as {props:Record}).props; + return match(props)?node as {props:Record}:findElement(props.children,match); + }; + const tree=DecisionPanel({...baseDecisionProps,onApprove,onReject,digest:{...view,decide}}); + + findElement(tree,p=>p['aria-label']==='Approve result')!.props.onClick(); + expect(onApprove).toHaveBeenCalledExactlyOnceWith('a'.repeat(64)); + expect(onReject).not.toHaveBeenCalled(); + findElement(tree,p=>p['aria-label']==='Reject result')!.props.onClick(); + expect(onReject).toHaveBeenCalledExactlyOnceWith('a'.repeat(64)); + expect(decide).toHaveBeenCalledTimes(2); + }); + + it('renders the same digest states beside the Human decision buttons',()=>{ + expect(renderToStaticMarkup(createElement(DecisionDigestNotice,{digest:digestView()}))).toBe(''); + const loading=renderToStaticMarkup(createElement(DecisionDigestNotice,{digest:digestView({status:'loading',digest:undefined})})); + expect(loading).toContain('Loading the evidence digest…'); + const failed=renderToStaticMarkup(createElement(DecisionDigestNotice,{digest:digestView({status:'error',digest:undefined,error:'boom'})})); + expect(failed).toContain('Evidence digest unavailable: boom.'); + expect(failed).toContain('>Retry<'); + const conflict=renderToStaticMarkup(createElement(DecisionDigestNotice,{digest:digestView({conflict:'The evidence changed'})})); + expect(conflict).toContain('role="alert"'); + expect(conflict).toContain('The evidence changed'); + }); +}); + // ── Decision panel stepper ───────────────────────────────────────────────── describe('decision panel stepper',()=>{ @@ -578,6 +698,84 @@ describe('checks pipeline',()=>{ expect(html).toContain('tone-running'); expect(html).toContain('Validating'); }); + + it('badges only the checks that ran with network access',()=>{ + const obs=[{...sampleObservation('install',true),network:true},{...sampleObservation('tests',true),network:false},sampleObservation('legacy',true)]; + const html=renderToStaticMarkup(createElement(ChecksPipeline,{observations:obs})); + const station=(name:string)=>html.split('part.includes(`class="station-name">${name}<`))??''; + expect(html.match(/status-pill tone-info/g)).toHaveLength(1); + expect(station('install')).toContain('class="status-pill tone-info" title="This check ran with network access" aria-label="Ran with network access">Network<'); + expect(station('install')).toContain('aria-label="install: Passed — ran with network access"'); + expect(station('tests')).toContain('aria-label="tests: Passed"'); + expect(station('tests')).not.toContain('Network'); + expect(station('legacy')).not.toContain('Network'); + }); +}); + +// ── Open-repository dialog: validation command list ────────────────────────── + +type Node=ReactNode; +/** Walk an element tree without rendering it, collecting elements of one component type. */ +const findElements=(node:Node,type:unknown):ReactElement[]=>{ + if(!node||typeof node!=='object')return []; + if(Array.isArray(node))return node.flatMap(child=>findElements(child,type)); + const element=node as ReactElement,own=element.type===type?[element]:[]; + return [...own,...findElements(element.props?.children,type)]; +}; + +describe('open-repository validation commands',()=>{ + const suggestions=[{name:'Install dependencies',command:'pnpm',args:['install','--frozen-lockfile'],network:true,source:'ci' as const},{name:'Tests',command:'pnpm',args:['run','test'],source:'ci' as const}]; + + it('defaults each Network toggle from the suggestion and explains why it is off by default',()=>{ + const checks=checksFromSuggestions(suggestions); + expect(checks.map(check=>check.network)).toEqual([true,false]); + const html=renderToStaticMarkup(createElement(RepoChecksEditor,{checks,onChange:()=>{}})); + const rows=html.split('class="repo-check"').slice(1); + expect(rows).toHaveLength(2); + expect(rows[0]).toContain(''); + expect(rows[1]).toContain(''); + expect(html).toContain('Network is off by default so a check cannot reach services on this computer or cloud credentials.'); + expect(html.match(/class="repo-check-network"/g)).toHaveLength(2); + }); + + it('round-trips the checkbox into the submitted validation commands',()=>{ + let checks=checksFromSuggestions(suggestions); + const onChange=vi.fn((next:RepoCheck[])=>{checks=next;}); + const toggles=()=>findElements(RepoChecksEditor({checks,onChange}),NetworkToggle); + expect(toggles().map(toggle=>toggle.props.checked)).toEqual([true,false]); + expect(validationCommandsPayload(checks)).toEqual([ + {name:'Install dependencies',command:'pnpm',args:['install','--frozen-lockfile'],network:true}, + {name:'Tests',command:'pnpm',args:['run','test'],network:false}, + ]); + // Turn Tests on and Install off, as the checkboxes would. + toggles()[1]!.props.onChange(true); + expect(checks.map(check=>check.network)).toEqual([true,true]); + toggles()[0]!.props.onChange(false); + expect(onChange).toHaveBeenCalledTimes(2); + expect(toggles().map(toggle=>toggle.props.checked)).toEqual([false,true]); + expect(validationCommandsPayload(checks)).toEqual([ + {name:'Install dependencies',command:'pnpm',args:['install','--frozen-lockfile'],network:false}, + {name:'Tests',command:'pnpm',args:['run','test'],network:true}, + ]); + // What the server stores is what was ticked. + expect(JSON.parse(JSON.stringify(validationCommandsPayload(checks))).map((command:{network:boolean})=>command.network)).toEqual([false,true]); + }); + + it('submits an explicit false for a new or edited command and keeps the flag when other fields change',()=>{ + let checks:RepoCheck[]=[]; + const editor=()=>RepoChecksEditor({checks,onChange:next=>{checks=next;}}); + const addCheck=findElements(editor(),'button').find(button=>button.props.children==='Add check')!; + addCheck.props.onClick(); + expect(checks).toEqual([{name:'',command:'',args:'',network:false}]); + // Each event fires on a fresh render, as React re-renders between events. + const type=(label:string,value:string)=>findElements(editor(),'input').find(input=>input.props['aria-label']===label)!.props.onChange({target:{value}}); + type('Check name','Fetch');type('Executable','cargo');type('Arguments','fetch --locked'); + findElements(editor(),NetworkToggle)[0]!.props.onChange(true); + type('Arguments','fetch'); + expect(validationCommandsPayload(checks)).toEqual([{name:'Fetch',command:'cargo',args:['fetch'],network:true}]); + // Commands left blank are not submitted. + expect(validationCommandsPayload([...checks,{name:'',command:' ',args:'',network:true}])).toHaveLength(1); + }); }); // ── parseChecksSummary ───────────────────────────────────────────────────── diff --git a/tests/validation-sandbox.test.ts b/tests/validation-sandbox.test.ts index 2268ebc..81a5a06 100644 --- a/tests/validation-sandbox.test.ts +++ b/tests/validation-sandbox.test.ts @@ -2,12 +2,13 @@ import { createHash } from 'node:crypto'; import { execFileSync, spawnSync } from 'node:child_process'; import { chmod, mkdir, mkdtemp, readFile, rm, stat, symlink, writeFile } from 'node:fs/promises'; import { accessSync, constants, existsSync, readFileSync, readdirSync, realpathSync } from 'node:fs'; -import { createServer } from 'node:http'; +import { createServer as createTcpServer, type AddressInfo } from 'node:net'; import { homedir, tmpdir } from 'node:os'; import { dirname, join, relative } from 'node:path'; import { afterEach, describe, expect, it, vi } from 'vitest'; import { loadConfig } from '../src/config.js'; -import { SANDBOX_UNAVAILABLE_MESSAGE, buildSandboxArgs, parseSandboxMode, probeBwrap, validationSandboxStatus, type SandboxPlanInput } from '../src/validation-sandbox.js'; +import { SANDBOX_UNAVAILABLE_MESSAGE, buildSandboxArgs, defaultNetworkAccess, parseSandboxMode, probeBwrap, validationSandboxStatus, type SandboxPlanInput } from '../src/validation-sandbox.js'; +import { normalizeValidationCommands } from '../src/controller.js'; import { validateWorkerOutput, verifyWorkerSnapshot, type ValidationCommand } from '../src/verified-workspace.js'; vi.setConfig({ testTimeout: 30_000 }); @@ -153,16 +154,6 @@ describe('validation sandbox (bubblewrap)', () => { expect(second.output.trim()).toBe('cached'); }); - it('can still reach the network (installs need it): a loopback server answers', async () => { - const server = createServer((_req, res) => res.end('pong')); - await new Promise(resolve => server.listen(0, '127.0.0.1', resolve)); - try { - const { port } = server.address() as import('node:net').AddressInfo; - const check = await run(await fixture(), node("fetch('http://127.0.0.1:'+process.argv[1]).then(r=>r.text()).then(t=>console.log(t),e=>{console.log('ERR '+e.cause?.code);process.exit(1)})", String(port))); - expect(check).toMatchObject({ exitCode: 0, output: 'pong\n' }); - } finally { await new Promise(resolve => server.close(resolve)); } - }); - it('re-exposes a toolchain that lives under the hidden home directory', async () => { const home = await scratch(homedir(), '.foreman-sandbox-toolchain-'); const prefix = join(home, 'versions', 'v1'), bin = join(prefix, 'bin'); @@ -231,6 +222,151 @@ describe('validation sandbox (bubblewrap)', () => { }); }); +/** A TCP (or, with `abstractName`, abstract-namespace Unix) listener on the host that counts connections. */ +async function hostListener(abstractName?: string) { + let connections = 0; + const server = createTcpServer(socket => { connections++; socket.end('pong'); }); + await new Promise(resolve => abstractName ? server.listen(`\0${abstractName}`, resolve) : server.listen(0, '127.0.0.1', resolve)); + return { port: abstractName ? 0 : (server.address() as AddressInfo).port, connections: () => connections, close: () => new Promise(resolve => server.close(() => resolve())) }; +} +const CONNECT_TCP = "const c=require('net').connect({host:process.argv[1],port:Number(process.argv[2])});let data='';c.on('data',d=>data+=d);c.on('end',()=>console.log(JSON.stringify({data})));c.on('error',e=>console.log(JSON.stringify({error:e.code})))"; +const CONNECT_ABSTRACT = "const c=require('net').connect({path:'\\0'+process.argv[1]});let data='';c.on('data',d=>data+=d);c.on('end',()=>console.log(JSON.stringify({data})));c.on('error',e=>console.log(JSON.stringify({error:e.code})))"; +/** Bind and connect to loopback from inside: a raw TCP echo, then an HTTP request, as a test suite would. */ +const LOOPBACK = "const net=require('net'),http=require('http'),os=require('os');const tcp=net.createServer(s=>s.end('inside')).listen(0,'127.0.0.1',()=>{const c=net.connect({host:'127.0.0.1',port:tcp.address().port});let data='';c.on('data',d=>data+=d);c.on('end',()=>{tcp.close();const web=http.createServer((q,r)=>r.end('web')).listen(0,'127.0.0.1',()=>{fetch('http://127.0.0.1:'+web.address().port).then(r=>r.text()).then(body=>{web.close();console.log(JSON.stringify({data,body,interfaces:Object.keys(os.networkInterfaces())}))},e=>{console.log(JSON.stringify({error:String(e.cause&&e.cause.code)}));process.exit(1)})})});c.on('error',e=>{console.log(JSON.stringify({error:e.code}));process.exit(1)})})"; + +describe('validation network policy (live bubblewrap)', () => { + it('cannot connect to a TCP listener on the host loopback unless the command asks for the network', async () => { + const listener = await hostListener(); + try { + const f = await fixture(); + for (const network of [undefined, false]) { + const check = await run(f, { ...node(CONNECT_TCP, '127.0.0.1', String(listener.port)), ...(network === undefined ? {} : { network }) }); + expect(check, `network ${network}`).toMatchObject({ exitCode: 0, sandbox: 'bwrap', network: false }); + expect(json(check.output)).toEqual({ error: 'ECONNREFUSED' }); + } + expect(listener.connections()).toBe(0); + + const allowed = await run(f, { ...node(CONNECT_TCP, '127.0.0.1', String(listener.port)), network: true }); + expect(allowed).toMatchObject({ exitCode: 0, sandbox: 'bwrap', network: true }); + expect(json(allowed.output)).toEqual({ data: 'pong' }); + expect(listener.connections()).toBe(1); + } finally { await listener.close(); } + }); + + it('cannot reach the host abstract Unix sockets without the network, and can with it', async () => { + const name = `foreman-net-test-${process.pid}-${Date.now()}`, listener = await hostListener(name); + try { + const f = await fixture(); + const denied = json((await run(f, node(CONNECT_ABSTRACT, name))).output); + expect(['ECONNREFUSED', 'ENOENT']).toContain(denied.error); + expect(listener.connections()).toBe(0); + expect(json((await run(f, { ...node(CONNECT_ABSTRACT, name), network: true })).output)).toEqual({ data: 'pong' }); + expect(listener.connections()).toBe(1); + } finally { await listener.close(); } + }); + + it('has no route out (cloud metadata address, public addresses) and DNS lookups fail', async () => { + const f = await fixture(); + const connect = "const c=require('net').connect({host:process.argv[1],port:Number(process.argv[2]),timeout:5000});c.on('connect',()=>{console.log(JSON.stringify({connected:true}));c.destroy()});c.on('timeout',()=>{console.log(JSON.stringify({error:'TIMEOUT'}));c.destroy()});c.on('error',e=>console.log(JSON.stringify({error:e.code})))"; + for (const [host, port] of [['169.254.169.254', '80'], ['8.8.8.8', '53'], ['1.1.1.1', '443']]) { + expect(json((await run(f, node(connect, host!, port!))).output), `${host}:${port}`).toEqual({ error: 'ENETUNREACH' }); + } + const lookup = json((await run(f, node("require('dns').lookup('example.com',(e,a)=>console.log(JSON.stringify({error:e&&e.code,address:a})))"))).output); + expect(['ENOTFOUND', 'EAI_AGAIN']).toContain(lookup.error); + expect(lookup.address).toBeUndefined(); + }); + + it('keeps loopback up inside the private namespace: bind, connect and HTTP to 127.0.0.1 work without network', async () => { + const check = await run(await fixture(), node(LOOPBACK)); + expect(check).toMatchObject({ exitCode: 0, network: false }); + expect(json(check.output)).toEqual({ data: 'inside', body: 'web', interfaces: ['lo'] }); + // The namespace is the same when the command is explicitly offline, and loopback also works with network on. + const explicit = await run(await fixture(), { ...node(LOOPBACK), network: false }); + expect(json(explicit.output)).toMatchObject({ data: 'inside', body: 'web', interfaces: ['lo'] }); + const online = await run(await fixture(), { ...node(LOOPBACK), network: true }); + expect(json(online.output)).toMatchObject({ data: 'inside', body: 'web' }); + }); + + it('decides per command and records the decision on every observation', async () => { + const listener = await hostListener(); + try { + const f = await fixture(), probe = (name: string, network?: boolean): ValidationCommand => ({ ...node(CONNECT_TCP, '127.0.0.1', String(listener.port)), name, ...(network === undefined ? {} : { network }) }); + const result = await validateWorkerOutput({ repoPath: f.repo, evidence: f.evidence, commands: [probe('install', true), probe('tests', false), probe('lint')], timeoutMs: 20_000 }); + expect(result.checks.map(check => [check.name, check.network, json(check.output).data ?? json(check.output).error])).toEqual([['install', true, 'pong'], ['tests', false, 'ECONNREFUSED'], ['lint', false, 'ECONNREFUSED']]); + expect(listener.connections()).toBe(1); + } finally { await listener.close(); } + }); + + it('applies the install default at execution time: a legacy command list with no network fields gives only the install the network', async () => { + const listener = await hostListener(); + try { + // Fake pnpm/npm/yarn that connect to the host listener (the port is read from a file inside the re-exposed toolchain prefix), so no real install runs. + const home = await scratch(homedir(), '.foreman-sandbox-legacy-'), prefix = join(home, 'versions', 'v1'), bin = join(prefix, 'bin'), lib = join(prefix, 'lib'); + await mkdir(bin, { recursive: true }); await mkdir(lib); + await writeFile(join(lib, 'port'), String(listener.port)); + await writeFile(join(lib, 'probe.js'), "const c=require('net').connect({host:'127.0.0.1',port:Number(require('fs').readFileSync(__dirname+'/port','utf8'))});let d='';c.on('data',x=>d+=x);c.on('end',()=>console.log(JSON.stringify({data:d})));c.on('error',e=>console.log(JSON.stringify({error:e.code})))"); + for (const tool of ['pnpm', 'npm', 'yarn']) { await writeFile(join(bin, tool), `#!/bin/sh\nexec '${process.execPath}' '${join(lib, 'probe.js')}' "$@"\n`); await chmod(join(bin, tool), 0o755); } + vi.stubEnv('PATH', `${bin}:${process.env.PATH ?? ''}`); + + const legacy: ValidationCommand[] = [ + { name: 'pnpm install', command: 'pnpm', args: ['install', '--frozen-lockfile'] }, + { name: 'npm ci', command: 'npm', args: ['ci'] }, + { name: 'bare yarn', command: 'yarn', args: [] }, + { name: 'yarn add', command: 'yarn', args: ['add', 'left-pad'] }, + { name: 'pnpm test', command: 'pnpm', args: ['run', 'test'] }, + { name: 'npm test', command: 'npm', args: ['test'] }, + { name: 'yarn flags', command: 'yarn', args: ['--immutable'] }, + { name: 'explicit off', command: 'pnpm', args: ['install'], network: false }, + { name: 'explicit on', command: 'pnpm', args: ['run', 'test'], network: true }, + ]; + const before = structuredClone(legacy), f = await fixture(); + const result = await validateWorkerOutput({ repoPath: f.repo, evidence: f.evidence, commands: legacy, timeoutMs: 20_000 }); + expect(result.checks.map(check => [check.name, check.network, json(check.output).data ?? json(check.output).error])).toEqual([ + ['pnpm install', true, 'pong'], ['npm ci', true, 'pong'], ['bare yarn', true, 'pong'], ['yarn add', true, 'pong'], + ['pnpm test', false, 'ECONNREFUSED'], ['npm test', false, 'ECONNREFUSED'], ['yarn flags', false, 'ECONNREFUSED'], + ['explicit off', false, 'ECONNREFUSED'], ['explicit on', true, 'pong'], + ]); + expect(listener.connections()).toBe(5); + // The stored command list is not rewritten: the default is decided when the command runs. + expect(legacy).toEqual(before); + } finally { await listener.close(); } + }); + + it('rejects a network flag that is not a boolean instead of guessing', async () => { + const f = await fixture(), marker = join(f.root, 'ran'); + for (const network of ['true', 'false', 1, 0, null]) { + await expect(run(f, { ...node("require('fs').writeFileSync(process.argv[1],'ran')", marker), network: network as unknown as boolean }), JSON.stringify(network)).rejects.toThrow('network must be a boolean'); + } + expect(existsSync(marker)).toBe(false); + }); + + it('mode none leaves the host network alone and says so on the observation', async () => { + const listener = await hostListener(); + try { + const f = await fixture(); + for (const network of [undefined, false]) { + const check = await run(f, { ...node(CONNECT_TCP, '127.0.0.1', String(listener.port)), ...(network === undefined ? {} : { network }) }, { sandbox: { mode: 'none' } }); + expect(check).toMatchObject({ sandbox: 'none', network: true }); + expect(json(check.output)).toEqual({ data: 'pong' }); + } + expect(listener.connections()).toBe(2); + } finally { await listener.close(); } + }); + + it('probes bwrap with the private network namespace, so a host that cannot create one fails closed instead of running networked', async () => { + const dir = await scratch(tmpdir(), 'foreman-fake-bwrap-'), log = join(dir, 'argv'), recording = join(dir, 'bwrap-ok'), failing = join(dir, 'bwrap-no-netns'); + await writeFile(recording, `#!/bin/sh\nprintf '%s\\n' "$@" > '${log}'\n`); await chmod(recording, 0o755); + expect(await probeBwrap(recording)).toEqual({ ok: true }); + expect((await readFile(log, 'utf8')).split('\n')).toContain('--unshare-net'); + await writeFile(failing, '#!/bin/sh\nfor arg in "$@"; do [ "$arg" = --unshare-net ] && { echo "bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted" >&2; exit 1; }; done\nexit 0\n'); await chmod(failing, 0o755); + const f = await fixture(), marker = join(f.root, 'ran'); + const command = node("require('fs').writeFileSync(process.argv[1],'ran')", marker); + await expect(run(f, command, { sandbox: { bwrapPath: failing } })).rejects.toThrow(/Validation sandbox unavailable.*RTM_NEWADDR/); + await expect(run(f, { ...command, network: true }, { sandbox: { bwrapPath: failing } })).rejects.toThrow(SANDBOX_UNAVAILABLE_MESSAGE); + expect(existsSync(marker)).toBe(false); + }); +}); + describe('sandbox mount plan (pure)', () => { async function plan(overrides: Partial = {}) { const base = await scratch(tmpdir(), 'foreman-plan-'); @@ -351,6 +487,36 @@ describe('sandbox mount plan (pure)', () => { expect(Object.keys(env).sort()).toEqual(['COREPACK_HOME', 'HOME', 'LANG', 'LC_ALL', 'PATH', 'TMPDIR', 'XDG_CACHE_HOME', 'XDG_DATA_HOME', 'YARN_CACHE_FOLDER', 'npm_config_cache', 'npm_config_store_dir', 'pnpm_config_store_dir']); expect(withCache.indexOf('--clearenv')).toBeLessThan(withCache.indexOf('--setenv')); }); + + it('unshares the network namespace unless the command explicitly asks for the network', async () => { + const { input } = await plan(); + for (const network of [undefined, false]) { + const args = buildSandboxArgs({ ...input, network }); + expect(args, `network ${network}`).toContain('--unshare-net'); + expect(args.indexOf('--unshare-net')).toBeLessThan(args.indexOf('--ro-bind')); + } + expect(buildSandboxArgs({ ...input, network: true })).not.toContain('--unshare-net'); + // Only a real `true` opens the network: a truthy string or number does not. + for (const network of ['true', 1, {}]) expect(buildSandboxArgs({ ...input, network: network as unknown as boolean })).toContain('--unshare-net'); + // The flag is the only difference between the two plans, and it is a bwrap option, not part of the sandboxed command's argv. + const offline = buildSandboxArgs(input), online = buildSandboxArgs({ ...input, network: true }); + expect(offline.filter(arg => arg !== '--unshare-net')).toEqual(online); + expect(offline.indexOf('--unshare-net')).toBeLessThan(offline.indexOf('--')); + expect(buildSandboxArgs({ ...input, network: true, args: ['--unshare-net'] }).slice(-2)).toEqual(['true', '--unshare-net']); + }); + + it('recognises package-manager installs for the compatibility default and nothing else', () => { + const table: Array<[string, string[], boolean]> = [ + ['pnpm', ['install', '--frozen-lockfile'], true], ['pnpm', ['i'], true], ['pnpm', ['ci'], true], ['pnpm', ['add', 'left-pad'], true], + ['npm', ['ci'], true], ['npm', ['install'], true], ['npm', ['i', '-D', 'x'], true], ['npm', ['add', 'x'], true], + ['yarn', [], true], ['yarn', ['install', '--immutable'], true], ['yarn', ['add', 'x'], true], [' pnpm ', ['install'], true], + ['pnpm', ['test'], false], ['pnpm', ['run', 'install'], false], ['pnpm', ['--filter', 'app', 'install'], false], ['pnpm', [], false], ['npm', [], false], + ['npm', ['run', 'build'], false], ['npm', ['test'], false], ['yarn', ['test'], false], ['yarn', ['--immutable'], false], + ['npx', ['install'], false], ['bun', ['install'], false], ['cargo', ['test'], false], ['go', ['test', './...'], false], ['python', ['-m', 'pytest'], false], + ['/usr/local/bin/pnpm', ['install'], false], ['sh', ['-c', 'pnpm install'], false], ['true', [], false], + ]; + for (const [command, args, expected] of table) expect(defaultNetworkAccess(command, args), `${command} ${args.join(' ')}`).toBe(expected); + }); }); describe('sandbox configuration', () => { @@ -379,3 +545,50 @@ describe('sandbox configuration', () => { } finally { write.mockRestore(); } }); }); + +describe('validation command network flag', () => { + const list = (...commands: unknown[]) => JSON.stringify(commands); + + it('FOREMAN_VALIDATION_COMMANDS keeps an explicit flag and defaults an unflagged command by the install rule', () => { + vi.stubEnv('FOREMAN_VALIDATION_COMMANDS', list( + { name: 'Install', command: 'pnpm', args: ['install', '--frozen-lockfile'] }, + { name: 'Install offline', command: 'npm', args: ['ci', '--offline'], network: false }, + { name: 'Tests', command: 'pnpm', args: ['test'] }, + { name: 'Cargo', command: 'cargo', args: ['test'], network: true }, + { name: 'Sub', command: 'yarn', args: [], cwd: 'app' }, + )); + expect(loadConfig().validationCommands).toEqual([ + { name: 'Install', command: 'pnpm', args: ['install', '--frozen-lockfile'], network: true }, + { name: 'Install offline', command: 'npm', args: ['ci', '--offline'], network: false }, + { name: 'Tests', command: 'pnpm', args: ['test'], network: false }, + { name: 'Cargo', command: 'cargo', args: ['test'], network: true }, + { name: 'Sub', command: 'yarn', args: [], cwd: 'app', network: true }, + ]); + }); + + it('FOREMAN_VALIDATION_COMMANDS accepts only true or false', () => { + for (const network of ['true', 'false', 1, 0, null, {}, []]) { + vi.stubEnv('FOREMAN_VALIDATION_COMMANDS', list({ name: 'Install', command: 'pnpm', args: ['install'], network })); + expect(() => loadConfig(), JSON.stringify(network)).toThrow('FOREMAN_VALIDATION_COMMANDS must be a JSON array of {name,command,args,cwd?,network?} where network is true or false'); + } + }); + + it('task-start validation commands: explicit flag, install default, strict booleans', () => { + expect(normalizeValidationCommands([ + { name: ' Install ', command: ' pnpm ', args: ['install'] }, + { name: 'Tests', command: 'pnpm', args: ['test'] }, + { name: 'Cargo', command: 'cargo', args: ['test'], network: true, cwd: 'crate' }, + { name: 'Offline install', command: 'npm', args: ['ci'], network: false }, + ])).toEqual([ + { name: 'Install', command: 'pnpm', args: ['install'], network: true }, + { name: 'Tests', command: 'pnpm', args: ['test'], network: false }, + { name: 'Cargo', command: 'cargo', args: ['test'], cwd: 'crate', network: true }, + { name: 'Offline install', command: 'npm', args: ['ci'], network: false }, + ]); + for (const network of ['true', 'false', 1, 0, null, {}]) { + const attempt = () => normalizeValidationCommands([{ name: 'Install', command: 'pnpm', args: ['install'], network }]); + expect(attempt, JSON.stringify(network)).toThrow('network must be true or false'); + expect(attempt, JSON.stringify(network)).toThrow(expect.objectContaining({ statusCode: 422 })); + } + }); +}); diff --git a/ui/checks-pipeline.tsx b/ui/checks-pipeline.tsx index f995e50..7a11fff 100644 --- a/ui/checks-pipeline.tsx +++ b/ui/checks-pipeline.tsx @@ -20,6 +20,8 @@ export interface StationObservation { passed?: boolean; startedAt?: string; finishedAt?: string; + /** True when the check ran with network access (an install, or no sandbox). */ + network?: boolean; } export interface CiJobFailure { @@ -123,12 +125,13 @@ export function ChecksPipeline({ observations, running, ciChecks, ciChecksNotCon )} {localStations.map(({ obs, status, summary }) => { const dur = durationLabel(obs.startedAt, obs.finishedAt); - const ariaLabel = `${obs.name}: ${statusLabel(status, obs.timedOut)}${summary ? ` — ${summary}` : ''}`; + const ariaLabel = `${obs.name}: ${statusLabel(status, obs.timedOut)}${summary ? ` — ${summary}` : ''}${obs.network === true ? ' — ran with network access' : ''}`; return (
{obs.name} {dur && {dur}} + {obs.network === true && Network} {statusLabel(status, obs.timedOut)} diff --git a/ui/decision-digest.ts b/ui/decision-digest.ts new file mode 100644 index 0000000..4c5903d --- /dev/null +++ b/ui/decision-digest.ts @@ -0,0 +1,262 @@ +/** + * Evidence digest binding for the human Approve/Reject decision. + * + * The operator's decision must be bound to the evidence they actually looked + * at. When the decision panel shows a decidable run it fetches + * `GET /api/runs/:id/decision` (keyed on the evidence identity: Worker + * response, validation, Reviewer recommendation, controller inactive) and every + * Approve/Reject click sends that digest back. The digest is never fetched at + * click time. + * + * Everything except `useDecisionDigest` is framework-agnostic so it can be unit + * tested with a mocked `fetch` and no DOM. + */ +import { useEffect, useMemo, useState, useSyncExternalStore } from 'react'; + +/** What the panel is showing: changes exactly when the reviewed evidence changes. */ +export interface EvidenceIdentity { + runId: string; + /** `run.workerEvidence.responseId` */ + workerResponseId?: string; + /** `run.validation.id` */ + validationId?: string; + /** `run.reviewerRecommendation.id` */ + recommendationId?: string; + /** A run whose controller is active is not at a decision checkpoint. */ + controllerActive: boolean; +} + +/** Body of `GET /api/runs/:id/decision`. */ +export interface DecisionDigest { + runId: string; + evidenceDigest: string; + pinnedBaseCommit: string; + workerResponseId: string; + validationId: string; + recommendationId: string; +} + +export type DigestStatus = 'idle' | 'loading' | 'ready' | 'error'; + +export interface DecisionSnapshot { + status: DigestStatus; + /** Evidence key the status/digest/error belongs to; undefined while idle. */ + key?: string; + digest?: DecisionDigest; + error?: string; + /** Server message from the last 409; kept until the operator decides again. */ + conflict?: string; +} + +/** Result of submitting a decision: `conflict` is set when the server answered 409. */ +export interface DecisionOutcome { conflict?: string } + +export const EVIDENCE_CHANGED_MESSAGE = 'The evidence changed since you reviewed it; refresh and review the current result before deciding.'; + +type FetchLike = (input: string, init?: RequestInit) => Promise; + +const DIGEST_PATTERN = /^[0-9a-f]{64}$/i; + +/** Key for the fetch, or undefined when the run is not at a decision checkpoint. */ +export function decisionKey(identity: EvidenceIdentity | undefined): string | undefined { + if (!identity || identity.controllerActive) return undefined; + return JSON.stringify([identity.runId, identity.workerResponseId ?? null, identity.validationId ?? null, identity.recommendationId ?? null, 'controller-inactive']); +} + +async function errorDetail(response: Response): Promise { + let detail = `${response.status} ${response.statusText}`.trim(); + try { + const body = await response.json() as { error?: string; message?: string }; + detail = body.error ?? body.message ?? detail; + } catch { /* Keep the HTTP status when the response has no JSON error. */ } + return detail; +} + +function parseDigest(value: unknown): DecisionDigest { + const record = (typeof value === 'object' && value !== null ? value : {}) as Record; + const text = (field: string): string => { + const item = record[field]; + if (typeof item !== 'string') throw new Error(`The server returned an incomplete decision checkpoint (missing ${field})`); + return item; + }; + const digest: DecisionDigest = { + runId: text('runId'), evidenceDigest: text('evidenceDigest'), pinnedBaseCommit: text('pinnedBaseCommit'), + workerResponseId: text('workerResponseId'), validationId: text('validationId'), recommendationId: text('recommendationId'), + }; + if (!DIGEST_PATTERN.test(digest.evidenceDigest)) throw new Error('The server returned a malformed evidence digest'); + return digest; +} + +/** + * A digest issued for evidence other than what the panel is showing (the server + * moved on before the view refreshed) must not enable the buttons. + */ +function assertMatchesIdentity(digest: DecisionDigest, identity: EvidenceIdentity): void { + const mismatch = digest.runId !== identity.runId + || (identity.workerResponseId !== undefined && digest.workerResponseId !== identity.workerResponseId) + || (identity.validationId !== undefined && digest.validationId !== identity.validationId) + || (identity.recommendationId !== undefined && digest.recommendationId !== identity.recommendationId); + if (mismatch) throw new Error('The result on the server no longer matches what is on screen; wait for the view to refresh, then retry'); +} + +/** Holds the digest for the evidence currently shown. Compatible with `useSyncExternalStore`. */ +export class DecisionDigestStore { + private snapshot: DecisionSnapshot = { status: 'idle' }; + private identity: EvidenceIdentity | undefined; + private readonly listeners = new Set<() => void>(); + /** Bumped on every load and on dispose: only the newest request may write. */ + private generation = 0; + private deciding = false; + + constructor(private readonly fetchImpl?: FetchLike) {} + + subscribe = (listener: () => void): (() => void) => { + this.listeners.add(listener); + return () => { this.listeners.delete(listener); }; + }; + + getSnapshot = (): DecisionSnapshot => this.snapshot; + + /** + * Point the store at the evidence the panel is showing. Fetches when the + * evidence key changes; the same key (an unrelated re-render) is a no-op. + */ + sync(identity: EvidenceIdentity | undefined): void { + const key = decisionKey(identity); + const conflict = identity && this.identity?.runId === identity.runId ? this.snapshot.conflict : undefined; + this.identity = identity; + if (key !== undefined && key === this.snapshot.key) return; + if (key === undefined) { + this.generation += 1; + if (this.snapshot.status !== 'idle' || this.snapshot.conflict !== conflict) this.set({ status: 'idle', conflict }); + return; + } + this.load(identity as EvidenceIdentity, key, conflict); + } + + /** Refetch the digest for the current evidence (the Retry button). */ + retry(): void { + const key = decisionKey(this.identity); + if (key !== undefined && this.identity) this.load(this.identity, key, this.snapshot.conflict); + } + + /** The server rejected the digest as stale: show its message and fetch a fresh digest. */ + reportConflict(message: string): void { + const key = decisionKey(this.identity); + if (key !== undefined && this.identity) this.load(this.identity, key, message); + else this.set({ status: 'idle', conflict: message }); + } + + /** + * Submit a decision with the digest the operator was looking at. Never + * fetches at click time, and does nothing unless a digest is ready. + */ + async decide(submit: (evidenceDigest: string) => Promise | void): Promise { + const { status, digest } = this.snapshot; + if (this.deciding || status !== 'ready' || !digest) return; + this.deciding = true; + if (this.snapshot.conflict) this.set({ ...this.snapshot, conflict: undefined }); + try { + const outcome = await submit(digest.evidenceDigest); + if (outcome?.conflict) this.reportConflict(outcome.conflict); + } finally { + this.deciding = false; + } + } + + /** Drop in-flight work (unmount). A later sync fetches again. */ + dispose(): void { + this.generation += 1; + this.identity = undefined; + this.snapshot = { status: 'idle' }; + } + + private set(next: DecisionSnapshot): void { + this.snapshot = next; + for (const listener of [...this.listeners]) listener(); + } + + private load(identity: EvidenceIdentity, key: string, conflict: string | undefined): void { + const generation = ++this.generation; + this.set({ status: 'loading', key, conflict }); + const finish = (next: Omit) => { + // A newer key, retry or dispose superseded this request: drop it. + if (generation !== this.generation) return; + this.set({ ...next, key, conflict: this.snapshot.conflict }); + }; + const doFetch: FetchLike = this.fetchImpl ?? ((input, init) => fetch(input, init)); + void (async () => { + try { + const response = await doFetch(`/api/runs/${encodeURIComponent(identity.runId)}/decision`, { headers: { accept: 'application/json' } }); + if (!response.ok) throw new Error(await errorDetail(response)); + const digest = parseDigest(await response.json()); + assertMatchesIdentity(digest, identity); + finish({ status: 'ready', digest }); + } catch (e) { + finish({ status: 'error', error: e instanceof Error ? e.message : 'Could not load the evidence digest' }); + } + })(); + } +} + +/** What the panel renders: a snapshot re-keyed to the evidence being shown right now. */ +export interface DigestViewState { + status: DigestStatus; + digest?: DecisionDigest; + error?: string; + conflict?: string; +} + +export interface DecisionDigestView extends DigestViewState { + retry: () => void; + decide: (submit: (evidenceDigest: string) => Promise | void) => Promise; +} + +/** + * A snapshot for a different key is never shown: the first render after the + * evidence changes reads as loading, not as the old digest on new evidence. + */ +export function deriveDigestView(snapshot: DecisionSnapshot, key: string | undefined): DigestViewState { + if (key === undefined) return { status: 'idle', conflict: snapshot.conflict }; + if (snapshot.key !== key) return { status: 'loading', conflict: snapshot.conflict }; + return { status: snapshot.status, digest: snapshot.digest, error: snapshot.error, conflict: snapshot.conflict }; +} + +/** Fetch on render, keyed on the evidence identity. Pass undefined when the run is not decidable. */ +export function useDecisionDigest(identity: EvidenceIdentity | undefined): DecisionDigestView { + const [store] = useState(() => new DecisionDigestStore()); + const snapshot = useSyncExternalStore(store.subscribe, store.getSnapshot, store.getSnapshot); + const key = decisionKey(identity); + useEffect(() => { store.sync(identity); }, [store, key]); + useEffect(() => () => store.dispose(), [store]); + return useMemo(() => ({ + ...deriveDigestView(snapshot, key), + retry: () => store.retry(), + decide: (submit) => store.decide(submit), + }), [store, snapshot, key]); +} + +/** + * `POST /api/runs/:id/approve` with the reviewed digest. Resolves `{}` on + * success, `{ conflict }` on a 409 (evidence changed), and throws the server's + * message for any other failure. + */ +export async function postDecision( + runId: string, + decision: { approved: boolean; evidenceDigest: string; rationale?: string }, + fetchImpl?: FetchLike, +): Promise { + const doFetch: FetchLike = fetchImpl ?? ((input, init) => fetch(input, init)); + const response = await doFetch(`/api/runs/${encodeURIComponent(runId)}/approve`, { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify(decision), + }); + if (response.status === 409) { + let message = EVIDENCE_CHANGED_MESSAGE; + try { const body = await response.json() as { error?: string }; if (body.error) message = body.error; } catch { /* Fall back to the standard message. */ } + return { conflict: message }; + } + if (!response.ok) throw new Error(await errorDetail(response)); + return {}; +} diff --git a/ui/decision-panel.tsx b/ui/decision-panel.tsx index 586bf3e..911b56e 100644 --- a/ui/decision-panel.tsx +++ b/ui/decision-panel.tsx @@ -8,6 +8,7 @@ import React from 'react'; import { ChecksPipeline, type StationObservation, type GithubCheckEntry } from './checks-pipeline.js'; import { Badge, type Tone } from './badge.js'; +import type { DecisionDigestView, DecisionOutcome } from './decision-digest.js'; export interface DecisionPanelProps { /** Number of files changed (from workerEvidence.changes.length). */ @@ -32,10 +33,20 @@ export interface DecisionPanelProps { reviewerRecommends: boolean; /** True while an API request is in flight. */ pending: boolean; - /** Fire when the user clicks Approve. */ - onApprove: () => void; - /** Fire when the user clicks Reject. */ - onReject: () => void; + /** + * Evidence digest for the evidence on screen (`useDecisionDigest`). Approve + * and Reject stay disabled until it is ready. + */ + digest: DecisionDigestView; + /** True while the controller is running; decisions are paused until it stops. */ + controllerActive?: boolean; + /** + * Fire when the user clicks Approve, with the digest fetched for the evidence + * on screen. Resolve `{ conflict }` when the server answered 409. + */ + onApprove: (evidenceDigest: string) => Promise | void; + /** Fire when the user clicks Reject; same contract as `onApprove`. */ + onReject: (evidenceDigest: string) => Promise | void; // ── Stepper state ────────────────────────────────────────────────────────── /** Whether the run has a human approval recorded. */ @@ -110,15 +121,57 @@ function verdictBadge(verdict: DecisionPanelProps['reviewerVerdict']): string { return 'Pending'; } +/** Loading / failure text for the evidence digest, or undefined when there is nothing to say. */ +function digestStatusText(digest: DecisionDigestView): string | undefined { + if (digest.status === 'loading') return 'Loading the evidence digest…'; + if (digest.status === 'error') return `Evidence digest unavailable: ${(digest.error ?? 'unknown error').replace(/[.\s]+$/, '')}.`; + return undefined; +} + +/** The server refused the digest: the evidence changed since it was reviewed. */ +function DecisionConflict({ message }: { message: string }): React.ReactElement { + return ( +
+ Your decision was not recorded + {message} +
+ ); +} + +/** Digest status for other places that offer Approve/Reject (the Human decision evidence block). */ +export function DecisionDigestNotice({ digest }: { digest: DecisionDigestView }): React.ReactElement | null { + const status = digestStatusText(digest); + if (!status && !digest.conflict) return null; + return ( + <> + {digest.conflict && } + {status && ( + + {status}{' '} + {digest.status === 'error' && } + + )} + + ); +} + export function DecisionPanel(props: DecisionPanelProps): React.ReactElement { const { filesChanged, allowedScope, validationPassedCount, validationTotalCount, validationPassed, observations, reviewerVerdict, reviewerRationaleSnippet, approvable, reviewerRecommends, pending, onApprove, onReject, approved, promotedBranch, branchPushed, prOpen, prUrl, prMerged, - ciChecks, ciChecksNotConfigured, + ciChecks, ciChecksNotConfigured, controllerActive, digest, } = props; + // Both decisions go out with the digest fetched for the evidence on screen, and never before it is ready. + const digestReady = digest.status === 'ready'; + const decide = (approve: boolean) => { void digest.decide((evidenceDigest) => (approve ? onApprove : onReject)(evidenceDigest)); }; + // The reason shown next to the buttons; only for a result that could otherwise be decided. + const digestNote = !approvable ? undefined + : digestStatusText(digest) + ?? (digest.status === 'idle' && controllerActive ? 'Decisions are paused while the controller is running.' : undefined); + const steps = buildStepper(props); const currentStep = steps.find(s => s.status === 'current'); @@ -127,7 +180,7 @@ export function DecisionPanel(props: DecisionPanelProps): React.ReactElement { const reviewTone = verdictTone(reviewerVerdict); return ( -
+

Ready for your review

@@ -215,14 +268,16 @@ export function DecisionPanel(props: DecisionPanelProps): React.ReactElement { /> )} + {!approved && digest.conflict && } + {/* Approve / Reject controls — only when awaiting approval */} {!approved && (
-

- {reviewerRecommends + {approvable && digest.status === 'error' && ( + + )} +

+ {digestNote ?? (reviewerRecommends ? 'Approving records your decision only. Nothing is committed to Git until you promote it.' - : 'The Reviewer has not recommended approving. Approving accepts the current result as-is.'} + : 'The Reviewer has not recommended approving. Approving accepts the current result as-is.')}

)} diff --git a/ui/main.tsx b/ui/main.tsx index 66b9392..3e9f79b 100644 --- a/ui/main.tsx +++ b/ui/main.tsx @@ -1,7 +1,9 @@ import React, { FormEvent, useCallback, useEffect, useMemo, useRef, useState } from 'react'; import { summarizeCheckOutput } from './check-output.js'; -import { DecisionPanel } from './decision-panel.js'; +import { DecisionPanel, DecisionDigestNotice } from './decision-panel.js'; +import { postDecision, useDecisionDigest, type DecisionOutcome } from './decision-digest.js'; import { ChecksPipeline, type CiJobFailure } from './checks-pipeline.js'; +import { NetworkToggle, RepoChecksEditor, checksFromSuggestions, validationCommandsPayload, type RepoCheck, type RepoCheckSuggestion } from './repo-checks.js'; import { PrDraftPanel, type PrDraftData } from './pr-draft.js'; import { Badge, toneForStatus, type Tone } from './badge.js'; import { createRoot } from 'react-dom/client'; @@ -46,10 +48,10 @@ export type Task = { id: string; title: string; goal?:string; suggestedAllowedPa export type Project = { id: string; name: string; repoPath?:string; status?: string; plannerSession?:Session; plannerMessages?:PlannerMessage[]; plannerAssignments?:Assignment[]; plannerTurnCount?:number; defaultRoleConfigs?: Record; usage?: UsageMetrics; usageByHarnessModel?: Record; tasks?: Task[] }; type RepoEntry={name:string;path:string;isGitRepo:boolean}; type RepoBrowse={currentPath:string;parentPath?:string;entries:RepoEntry[];roots:{name:string;path:string}[];truncated:boolean}; -type WorkspaceSetup={repoPath:string;head:string;dirty:boolean;allowedScope:string[];validationCommands:{name:string;command:string;args:string[]}[];bridgeStatus?:string}; -export type TaskStartPreview={taskId?:string;task?:Task;roleConfigs?:Record;scope?:string[];validationCriteria?:string[];validationCommands?:{name:string;command:string;args:string[]}[];budgets?:RunBudgets;baseCommit?:string;currentHead?:string;requiredBaseCommits?:string[];requiresExplicitBase:boolean;reasons?:string[];overlappingTaskIds?:string[];dependencyTaskIds?:string[];dependencies?:Task[];canStart?:boolean;ciChecksNotConfigured?:string[]}; +type WorkspaceSetup={repoPath:string;head:string;dirty:boolean;allowedScope:string[];validationCommands:{name:string;command:string;args:string[];network?:boolean}[];bridgeStatus?:string}; +export type TaskStartPreview={taskId?:string;task?:Task;roleConfigs?:Record;scope?:string[];validationCriteria?:string[];validationCommands?:{name:string;command:string;args:string[];network?:boolean}[];budgets?:RunBudgets;baseCommit?:string;currentHead?:string;requiredBaseCommits?:string[];requiresExplicitBase:boolean;reasons?:string[];overlappingTaskIds?:string[];dependencyTaskIds?:string[];dependencies?:Task[];canStart?:boolean;ciChecksNotConfigured?:string[]}; type PrDraft = PrDraftData; -type RepoInspect={repoPath:string;head:string;dirty:boolean;suggestedAllowedScope:string[];suggestedValidationCommands:{name:string;command:string;args:string[];source?:'ci'|'package-script'}[];trackedFiles:string[];ciScripts?:string[]}; +type RepoInspect={repoPath:string;head:string;dirty:boolean;suggestedAllowedScope:string[];suggestedValidationCommands:RepoCheckSuggestion[];trackedFiles:string[];ciScripts?:string[]}; type Role = { id: string; name: string; kind?: string; enabled: boolean; configSchema?: unknown; config?: RoleConfig; availableConfigs?: RoleConfig[]; usage?: UsageMetrics; usageByHarnessModel?: Record }; type EventItem = { id: string; type: string; entityType?: string; entityId?: string; at: string; data?: Record; response?: Record; activity?: { summary?: string; [key:string]: unknown } }; export type State = { projects: Project[]; roles?: Role[]; events?: EventItem[] }; @@ -154,7 +156,7 @@ export function App({initialState,initialWorkspaceSetup,initialTaskStartPreview, const [browsePath,setBrowsePath]=useState(''); const [repoPath,setRepoPath]=useState(''); const [scopePaths,setScopePaths]=useState(['']); - const [checks,setChecks]=useState<{name:string;command:string;args:string;source?:'ci'|'package-script'}[]>([{name:'Tests',command:'npm',args:'test'}]); + const [checks,setChecks]=useState([{name:'Tests',command:'npm',args:'test',network:false}]); const [workspaceSetup,setWorkspaceSetup]=useState(initialWorkspaceSetup); const [repoInspect,setRepoInspect]=useState(); const [addingTask,setAddingTask]=useState(false); @@ -174,7 +176,7 @@ export function App({initialState,initialWorkspaceSetup,initialTaskStartPreview, const [taskCriteria,setTaskCriteria]=useState(''); const [taskDeps,setTaskDeps]=useState(''); const [workScope,setWorkScope]=useState(''); - const [workChecks,setWorkChecks]=useState<{name:string;command:string;args:string}[]>([]); + const [workChecks,setWorkChecks]=useState<{name:string;command:string;args:string;network?:boolean}[]>([]); const [taskAdvancedEdited,setTaskAdvancedEdited]=useState(false); const [roleChoices,setRoleChoices]=useState>(()=>Object.fromEntries((initialState?.roles??[]).filter(r=>r.enabled).map(r=>[r.id,r.config??{}]))); const [taskStartPreview,setTaskStartPreview]=useState(initialTaskStartPreview); @@ -323,14 +325,16 @@ export function App({initialState,initialWorkspaceSetup,initialTaskStartPreview, catch (e) { setError(e instanceof Error ? e.message : 'Could not verify Worker result'); } finally { setPending(false); } }; - const decideRun = async (approved: boolean) => { + // `evidenceDigest` is the one fetched when the decision panel rendered this evidence (never fetched at click time). + // A 409 (evidence changed) resolves `{ conflict }` after the state refresh so the panel can show it and refetch the digest. + const decideRun = async (approved: boolean, evidenceDigest: string): Promise => { if (!run) return; if (approved && run.reviewerRecommendation?.verdict && run.reviewerRecommendation.verdict !== 'recommend' && !window.confirm('The Reviewer requested changes. Approving accepts the current result as-is and finalizes this run; it will not send work back to the Orchestrator. Cancel to use a correction action if one is available. Approve this result anyway?')) return; const rejectionRationale = approved ? undefined : window.prompt('Reason for rejecting this result (optional)'); if (!approved && rejectionRationale === null) return; const rationale = rejectionRationale || undefined; setPending(true); setError(''); - try { await api(`/api/runs/${run.id}/approve`, { method: 'POST', body: JSON.stringify({ approved, rationale }) }); await refresh(); } + try { const outcome = await postDecision(run.id, { approved, evidenceDigest, rationale }); await refresh(); return outcome; } catch (e) { setError(e instanceof Error ? e.message : 'Could not record human decision'); } finally { setPending(false); } }; @@ -407,15 +411,15 @@ export function App({initialState,initialWorkspaceSetup,initialTaskStartPreview, }; const loadBrowse = async(path?:string)=>{setPending(true);setError('');try{const query=path?`?path=${encodeURIComponent(path)}`:'';const next=await api(`/api/repositories/browse${query}`);setBrowse(next);setBrowsePath(next.currentPath);}catch(e){setError(e instanceof Error?e.message:'Could not browse local folders');}finally{setPending(false);}}; const openRepoDialog=()=>{setRepoDialog(true);setRepoAdvancedOpen(false);setRepoPath('');setRepoInspect(undefined);setScopePaths(['']);setChecks([]);void loadBrowse();}; - const selectRepo=async(path:string)=>{setRepoPath(path);setRepoInspect(undefined);setPending(true);setError('');try{const info=await api(`/api/repositories/inspect?path=${encodeURIComponent(path)}`);setRepoInspect(info);setRepoAdvancedOpen(info.suggestedValidationCommands.length===0);setScopePaths(info.suggestedAllowedScope.length?info.suggestedAllowedScope:['']);setChecks(info.suggestedValidationCommands.map(c=>({name:c.name,command:c.command,args:c.args.join(' '),source:c.source})));}catch(e){setError(e instanceof Error?e.message:'Could not inspect repository');}finally{setPending(false);}}; - const openRepository=async()=>{if(!repoPath)return;setPending(true);setError('');try{const opened=await api('/api/projects/open',{method:'POST',body:JSON.stringify({repoPath,allowedScope:scopePaths.map(x=>x.trim()).filter(Boolean),validationCommands:checks.filter(c=>c.command.trim()).map(c=>({name:c.name||c.command,command:c.command.trim(),args:c.args.trim()?c.args.trim().split(/\s+/):[]}))})});await refresh();setSelectedProject(opened.id);setSelectedTask('');setSelectedRun('');setRepoDialog(false);setView('overview');}catch(e){setError(e instanceof Error?e.message:'Could not open repository');}finally{setPending(false);}}; + const selectRepo=async(path:string)=>{setRepoPath(path);setRepoInspect(undefined);setPending(true);setError('');try{const info=await api(`/api/repositories/inspect?path=${encodeURIComponent(path)}`);setRepoInspect(info);setRepoAdvancedOpen(info.suggestedValidationCommands.length===0);setScopePaths(info.suggestedAllowedScope.length?info.suggestedAllowedScope:['']);setChecks(checksFromSuggestions(info.suggestedValidationCommands));}catch(e){setError(e instanceof Error?e.message:'Could not inspect repository');}finally{setPending(false);}}; + const openRepository=async()=>{if(!repoPath)return;setPending(true);setError('');try{const opened=await api('/api/projects/open',{method:'POST',body:JSON.stringify({repoPath,allowedScope:scopePaths.map(x=>x.trim()).filter(Boolean),validationCommands:validationCommandsPayload(checks)})});await refresh();setSelectedProject(opened.id);setSelectedTask('');setSelectedRun('');setRepoDialog(false);setView('overview');}catch(e){setError(e instanceof Error?e.message:'Could not open repository');}finally{setPending(false);}}; const addProject = openRepoDialog; const addTask = () => { setTaskTitle(''); setAddingTask(true); }; const submitTask = async (event:FormEvent) => {event.preventDefault();if(!project||!taskTitle.trim())return;setPending(true);setError('');try{const created=await api(`/api/projects/${project.id}/tasks`,{method:'POST',body:JSON.stringify({title:taskTitle.trim(),goal:taskTitle.trim()})});setTaskTitle('');setAddingTask(false);await refresh();setSelectedTask(created.id);setSelectedRun('');setView('overview');}catch(e){setError(e instanceof Error?e.message:'Could not create task');}finally{setPending(false);}}; const lines = (value:string) => value.split('\n').map(item=>item.trim()).filter(Boolean); const sendProjectPlanner = async (event:FormEvent) => { event.preventDefault(); if(!project||!plannerDraft.trim()||plannerSending)return; const text=plannerDraft.trim();plannerFollowTail.current=true;setPlannerPendingText(text);setPlannerPendingProjectId(project.id);setPlannerSending(true);setError('');setPlannerTurnError('');setProposalError(''); try { const result=await api(`/api/projects/${project.id}/planner/messages`,{method:'POST',body:JSON.stringify({text})});setProposalError(result.proposalError??'');setPlannerDraft('');await refresh();requestAnimationFrame(()=>{if(plannerFollowTail.current&&plannerCurrentProjectRef.current===project.id&&plannerChatRef.current)plannerChatRef.current.scrollTop=plannerChatRef.current.scrollHeight;}); } catch(e) {setPlannerTurnError(e instanceof Error?e.message:'Could not send message to project Planner');} finally {setPlannerPendingText('');setPlannerPendingProjectId('');setPlannerSending(false);} }; const saveTaskSpec = async (event:FormEvent) => { event.preventDefault();if(!task)return;setPending(true);setError('');try {await api(`/api/tasks/${task.id}`,{method:'PATCH',body:JSON.stringify({title:taskTitle.trim(),goal:taskGoal,suggestedAllowedPaths:lines(taskPaths),validationCriteria:lines(taskCriteria),dependsOn:taskDeps.split(',').map(x=>x.trim()).filter(Boolean)})});setTaskEdit(false);await refresh();}catch(e){setError(e instanceof Error?e.message:'Could not save task specification');}finally{setPending(false);}}; - const startTask = async () => {if(!task)return;setPending(true);setError('');try{const options=taskAdvancedEdited?{roleConfigs:roleChoices,scope:lines(workScope),validationCommands:workChecks.filter(c=>c.command.trim()&&c.name.trim()).map(c=>({name:c.name,command:c.command.trim(),args:c.args.trim()?c.args.trim().split(/\s+/):[]})),budgets,baseCommit:taskBaseCommit.trim()||undefined}:{};const newRun=await api(`/api/tasks/${task.id}/start`,{method:'POST',body:JSON.stringify(options)});await refresh();setSelectedRun(newRun.id);setView('run');}catch(e){await refresh();setError(e instanceof Error?e.message:'Could not start task work');}finally{setPending(false);}}; + const startTask = async () => {if(!task)return;setPending(true);setError('');try{const options=taskAdvancedEdited?{roleConfigs:roleChoices,scope:lines(workScope),validationCommands:workChecks.filter(c=>c.command.trim()&&c.name.trim()).map(c=>({name:c.name,command:c.command.trim(),args:c.args.trim()?c.args.trim().split(/\s+/):[],...(c.network===undefined?{}:{network:c.network})})),budgets,baseCommit:taskBaseCommit.trim()||undefined}:{};const newRun=await api(`/api/tasks/${task.id}/start`,{method:'POST',body:JSON.stringify(options)});await refresh();setSelectedRun(newRun.id);setView('run');}catch(e){await refresh();setError(e instanceof Error?e.message:'Could not start task work');}finally{setPending(false);}}; const steerTaskRun = async (event:FormEvent) => {event.preventDefault();if(!task||!message.trim())return;setPending(true);setError('');try{await api(`/api/tasks/${task.id}/steer`,{method:'POST',body:JSON.stringify({text:message.trim()})});setMessage('');await refresh();}catch(e){setError(e instanceof Error?e.message:'Could not steer task at a safe checkpoint');}finally{setPending(false);}}; const configKey = (config?: RoleConfig) => config?.harnessId&&config.model ? JSON.stringify([config.harnessId,config.model]) : ''; const decodeConfig = (value: string): RoleConfig|undefined => { if(!value) return undefined; try { const [harnessId,model]=JSON.parse(value) as [string,string]; return {harnessId,model}; } catch { return undefined; } }; @@ -770,6 +774,12 @@ export function App({initialState,initialWorkspaceSetup,initialTaskStartPreview, trackViewport();window.addEventListener('resize',trackViewport);return()=>window.removeEventListener('resize',trackViewport); },[]); + // Evidence digest for the human decision: fetched when a decidable result renders (keyed on the evidence shown), never at click time. + const decisionEvidence = run && !run.approval && run.reviewerRecommendation && run.workerEvidence?.scopeVerified === true && run.workerEvidence.completeSnapshot?.reportedComplete === true && run.workerEvidence.completeSnapshot.reportedErrors === 0 && run.validation?.status === 'passed' && run.validation.observations?.length + ? { runId: run.id, workerResponseId: run.workerEvidence.responseId, validationId: run.validation.id, recommendationId: run.reviewerRecommendation.id, controllerActive: Boolean(run.controller?.active) } + : undefined; + const decisionDigest = useDecisionDigest(decisionEvidence); + // Decision panel props — derived here to keep JSX concise. const decisionPanelProps = (() => { if (!run || run.workerEvidence?.scopeVerified !== true) return null; @@ -785,7 +795,7 @@ export function App({initialState,initialWorkspaceSetup,initialTaskStartPreview, const jobFailures = ciFailures?.runId === run.id ? ciFailures.failures.filter(f => f.jobName === c.name || c.name.toLowerCase().includes(f.jobName.toLowerCase())) : []; return { ...c, failures: jobFailures.length ? jobFailures : undefined }; }); - return { filesChanged: run.workerEvidence.changes?.length ?? 0, allowedScope: run.workerEvidence.allowedScope ?? [], validationPassedCount: obs.filter(o => o.passed).length, validationTotalCount: obs.length, validationPassed: run.validation?.status === 'passed', observations: obs, reviewerVerdict: run.reviewerRecommendation?.verdict, reviewerRationaleSnippet: run.reviewerRecommendation?.rationale?.match(/[^.!?]*[.!?]/)?.[0]?.trim(), approvable, reviewerRecommends: !run.reviewerRecommendation?.verdict || run.reviewerRecommendation.verdict === 'recommend', pending, onApprove: () => void decideRun(true), onReject: () => void decideRun(false), approved, promoted, promotedBranch: run.promotion?.destinationBranch, branchPushed, prOpen, prUrl: github?.pullRequest?.url, prMerged, ciChecks: ciChecksWithFailures, ciChecksNotConfigured: taskStartPreview?.ciChecksNotConfigured }; + return { filesChanged: run.workerEvidence.changes?.length ?? 0, allowedScope: run.workerEvidence.allowedScope ?? [], validationPassedCount: obs.filter(o => o.passed).length, validationTotalCount: obs.length, validationPassed: run.validation?.status === 'passed', observations: obs, reviewerVerdict: run.reviewerRecommendation?.verdict, reviewerRationaleSnippet: run.reviewerRecommendation?.rationale?.match(/[^.!?]*[.!?]/)?.[0]?.trim(), approvable, reviewerRecommends: !run.reviewerRecommendation?.verdict || run.reviewerRecommendation.verdict === 'recommend', pending, digest: decisionDigest, controllerActive: Boolean(run.controller?.active), onApprove: (evidenceDigest: string) => decideRun(true, evidenceDigest), onReject: (evidenceDigest: string) => decideRun(false, evidenceDigest), approved, promoted, promotedBranch: run.promotion?.destinationBranch, branchPushed, prOpen, prUrl: github?.pullRequest?.url, prMerged, ciChecks: ciChecksWithFailures, ciChecksNotConfigured: taskStartPreview?.ciChecksNotConfigured }; })(); return
@@ -866,7 +876,7 @@ export function App({initialState,initialWorkspaceSetup,initialTaskStartPreview, {taskPreviewBlockersRemain.length?
Resolve these blockers before starting
    {taskPreviewBlockersRemain.map((reason,index)=>
  • {reason}
  • )}
:null} {requiresExplicitTaskBase&&
Related work must be integrated first{baseTaskIds.length?`Related tasks: ${baseTaskIds.join(', ')}. `:''}Integrate any required promoted commits into the repository. Foreman then pins the updated current HEAD automatically.
}
Advanced options Optional overrides for scope, roles, checks, run limits, and Git base