diff --git a/.gitignore b/.gitignore index f8c9e85..9d7148c 100644 --- a/.gitignore +++ b/.gitignore @@ -11,4 +11,5 @@ research/ /.review-board-*/ /.security-gate-*/ +/.compound-engineering-*/ /.spec-driven-development-research-*/ diff --git a/README.md b/README.md index 47f3029..7b6b468 100644 --- a/README.md +++ b/README.md @@ -99,6 +99,15 @@ Use workflow paths relative to the package root, such as `workflows/review-board These workflows are provided by this registry package after installation. See [`workflows/README.md`](./workflows/README.md) for the current workflow index, details, and settings filter examples. +| Workflow | Use | +| --- | --- | +| `codebase-migration` | Research and execute bounded migration passes with Ralph handoffs. | +| `compound-engineering` | Safe Compound Engineering-inspired brainstorm/plan/approval/review/learning loop; defaults to handoff-only but explicit `goal`/`ralph` runners execute after approval, with file-backed review evidence and optional post-validation learning. | +| `descent` | Bounded iterative code optimization with validation loops. | +| `review-board` | Read-only specialist review board and synthesized report. | +| `security-gate` | Read-only security risk gate for a PR, branch, diff, path, or repo. | +| `spec-driven-development` | Brainstorm/direct intake through approved spec and Ralph handoff metadata. | + ## Customize these workflow recipes These workflows are deliberately readable TypeScript recipes, not black boxes. Copy one into your project or your own workflow package and adapt the inputs, prompts, stages, parallel specialists, validation policy, and output format. diff --git a/workflows/README.md b/workflows/README.md index 9aff8c9..469d3e4 100644 --- a/workflows/README.md +++ b/workflows/README.md @@ -7,6 +7,7 @@ Each workflow has its own subfolder with an `index.ts` entrypoint and local docu | Workflow | Source | Details | | --- | --- | --- | | `codebase-migration` | [`codebase-migration/index.ts`](./codebase-migration/index.ts) | [`codebase-migration/README.md`](./codebase-migration/README.md) | +| `compound-engineering` | [`compound-engineering/index.ts`](./compound-engineering/index.ts) | [`compound-engineering/README.md`](./compound-engineering/README.md) — safe default handoff-only; explicit `goal`/`ralph` run after approval; file-backed review gates and optional post-validation learning. | | `descent` | [`descent/index.ts`](./descent/index.ts) | [`descent/README.md`](./descent/README.md) | | `review-board` | [`review-board/index.ts`](./review-board/index.ts) | [`review-board/README.md`](./review-board/README.md) | | `security-gate` | [`security-gate/index.ts`](./security-gate/index.ts) | [`security-gate/README.md`](./security-gate/README.md) | @@ -21,6 +22,7 @@ From an Atomic chat session: ```text /workflow list /workflow inputs codebase-migration +/workflow inputs compound-engineering /workflow inputs descent /workflow inputs review-board /workflow inputs security-gate @@ -31,15 +33,16 @@ From an Atomic chat session: Reporting workflows write their final Markdown report to disk and return compact metadata instead of returning the full report inline. -Reporting workflows save final reports under project-root output folders: +Reporting workflows save final reports under project-root output folders. Compound Engineering visible Markdown artifacts are collision-safe and use `-2`, `-3`, ... suffixes when a same-day slug already exists: ```text +./compound-engineering/YYYY-MM-DD-.md ./migrations/YYYY-MM-DD-(-N).md ./review-board/YYYY-MM-DD-(-N).md ./security-gate/YYYY-MM-DD-(-N).md ``` -Intermediate workflow outputs are preserved under hidden run-specific artifact directories such as `./.review-board-/` and `./.security-gate-/`. Each artifact directory includes markdown stage outputs created by that run and a `manifest.json` recording the run id, timestamps, user input, final report path, and actual artifact paths. Some workflows may intentionally create a smaller artifact set when they short-circuit. +Intermediate workflow outputs are preserved under hidden run-specific artifact directories such as `./.compound-engineering-/`, `./.review-board-/`, and `./.security-gate-/`. Each artifact directory includes markdown stage outputs created by that run and a `manifest.json` recording the run id, timestamps, user input, final report path, and actual artifact paths. Some workflows may intentionally create a smaller artifact set when they short-circuit. The return object includes `summary`, `report_path`, `filename_summary`, `artifact_dir`, `manifest_path`, and `stages`. diff --git a/workflows/compound-engineering/README.md b/workflows/compound-engineering/README.md new file mode 100644 index 0000000..78d2b03 --- /dev/null +++ b/workflows/compound-engineering/README.md @@ -0,0 +1,77 @@ +# compound-engineering + +`compound-engineering` is a safe, artifact-backed Atomic workflow inspired by EveryInc's MIT-licensed [Compound Engineering Plugin](https://github.com/EveryInc/compound-engineering-plugin). It adapts the process vocabulary of brainstorm → plan → work → review → compound learning into an Atomic workflow graph. + +Iteration 3 is still safe by default, but explicit implementation runners are active after approval: it can classify intake, scout repo memory, write brainstorm/plan/spec/review/learning artifacts, ask for human approval, and then either stop at handoff metadata or launch the selected child workflow. File-only review artifacts are read from disk before evidence parsing, Ralph receives the effective Git worktree root, and validated implementation can capture learning according to `learning_mode`. + +## Safe posture + +- no implementation before approval: the workflow always requires a human approval gate before emitting implementation handoff metadata. +- `runner=auto` resolves to `handoff-only` in iteration 3, so the default remains non-mutating. +- `runner=handoff-only` never launches a child workflow and omits command metadata unless there is a concrete command string. +- explicit `runner=goal` or `runner=ralph` runs after the human approval gate via Atomic built-ins. +- `goal` receives `objective`, `max_turns`, and `base_branch`. +- `ralph` receives `prompt`, `max_loops`, `base_branch`, strict boolean `create_pr`, and the effective Git worktree root derived with `git rev-parse --show-toplevel` when `git_worktree_dir` is non-empty. +- `create_pr` defaults to `false`; only strict boolean `true` can authorize Ralph PR creation. +- Review and scout stages are read-only by prompt contract; implementation completion is gated by the child workflow's structured `compound_engineering_evidence`, not by parent supplemental review or writer self-attestation. +- File-only review stages save Markdown artifacts; compact inline strings such as `Output saved to: ...` are not treated as review evidence. +- Child implementation evidence must be emitted in child outputs and/or a review artifact as a `compound_engineering_evidence` JSON block with all six sufficiency criteria, severity counts, and validation command summaries/exit codes. Missing, malformed, prose-only, blocked, conflicted, failed-validation, or P0/P1 evidence fails closed. +- After a validated implementation, `learning_mode=lightweight|full` writes one `docs/solutions/...` artifact, `off` writes nothing, and `ask` presents the existing lightweight/full/off selection. + +## Inputs + +| Input | Default | Description | +| --- | --- | --- | +| `prompt` | required | Idea, spec/plan path, work request, review target, or learning-capture request. | +| `mode` | `auto` | `auto`, `brainstorm`, `plan`, `work`, `review`, or `compound-only`. | +| `runner` | `auto` | `auto`, `goal`, `ralph`, or `handoff-only`. Auto is handoff-only in iteration 3; explicit `goal`/`ralph` run after approval. | +| `max_loops` | `5` | Bound for approval revision loops and child runner loops (`goal` maps it to `max_turns`). | +| `base_branch` | `origin/main` | Base branch copied into implementation handoff metadata. | +| `git_worktree_dir` | `""` | Optional reusable worktree directory bound with `.worktreeFromInputs`; Ralph receives `""` when empty or the Git top-level root derived from `ctx.cwd` when non-empty. | +| `create_pr` | `false` | Strict `true` only; safe default is no PR creation. | +| `learning_mode` | `ask` | `ask`, `off`, `lightweight`, or `full`; honored for learning-only and post-validation implementation capture. | +| `memory_scope` | `repo` | `repo` scouts CE-style memory anchors; `none` skips repo memory. | + +## Outputs + +Required outputs: `status`, `mode`, `runner`, `approved`, `artifact_dir`, `manifest_path`, and `message`. + +Optional outputs: `brainstorm_path`, `plan_path`, `spec_path`, `approved_spec_path`, `implementation`, `review_report_path`, and `learning_doc_path`. + +Statuses are honest fixed strings: `complete`, `approved`, `handoff_ready`, `review_only`, `blocked`, `needs_human`, `rejected`, or `stopped`. + +## Artifacts + +The workflow writes a hidden run directory such as: + +```text +./.compound-engineering-/ + 00-intake.md + 01-memory-context.md + manifest.json +``` + +It may also write visible durable docs: + +```text +docs/brainstorms/YYYY-MM-DD-.md +docs/plans/YYYY-MM-DD-.md +specs/YYYY-MM-DD-.md +docs/solutions/YYYY-MM-DD-.md +compound-engineering/YYYY-MM-DD-.md +``` + +Visible Markdown writes use exclusive create and preserve existing files with `-2`, `-3`, ... suffixes. The manifest records sanitized inputs, timestamps, final report path, and artifact paths. + +## Examples + +```text +/workflow compound-engineering prompt="Improve onboarding activation" mode=auto +/workflow compound-engineering prompt="specs/2026-06-05-rate-limit.md" mode=work runner=ralph create_pr=false +/workflow compound-engineering prompt="main..feature/auth" mode=review runner=handoff-only +/workflow compound-engineering prompt="Capture lessons from the last fix" mode=compound-only learning_mode=lightweight +``` + +## Attribution + +This is not the upstream plugin. It is an Atomic workflow inspired by/adapting open-source Compound Engineering process language and severity vocabulary from EveryInc's Compound Engineering Plugin. The upstream project is MIT licensed; retain this notice when copying adapted workflow prompts or documentation. diff --git a/workflows/compound-engineering/helpers.test.ts b/workflows/compound-engineering/helpers.test.ts new file mode 100644 index 0000000..1af32dd --- /dev/null +++ b/workflows/compound-engineering/helpers.test.ts @@ -0,0 +1,2147 @@ +import { mkdir, mkdtemp, readFile, realpath, rm, writeFile } from "node:fs/promises"; +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import { tmpdir } from "node:os"; +import { describe, expect, mock, test } from "bun:test"; +import { + buildChildHandoff, + datedMarkdownPath, + displayPath, + loadSavedStageReport, + missingReviewCriteria, + normalizeCreatePr, + parseApprovalDecision, + reduceReviewEvidence, + resolveEffectiveWorktreeRoot, + resolveMode, + resolveRunner, + slugifyTopic, + writeMarkdown, +} from "./helpers.ts"; + +function readFixture(relativePath: string): string { + return readFileSync(new URL(relativePath, import.meta.url), "utf8"); +} + +const workflowSource = () => readFixture("./index.ts"); +const workflowReadme = () => readFixture("./README.md"); +const registryReadme = () => readFixture("../README.md"); +const rootReadme = () => readFixture("../../README.md"); +const rootGitignore = () => readFixture("../../.gitignore"); + +async function gitInit(cwd: string): Promise { + const proc = Bun.spawn(["git", "init"], { cwd, stdout: "ignore", stderr: "ignore" }); + const exitCode = await proc.exited; + if (exitCode !== 0) throw new Error(`git init failed in ${cwd}`); +} + +function structuredEvidence(overrides: Record = {}): Record { + return { + independent: { satisfied: true, evidence: "Child internal reviewer ran independently after implementation.", source: "child internal review" }, + acceptance_mapped: { satisfied: true, evidence: "Child mapped each approved acceptance criterion to implemented changes." }, + diff_aware: { satisfied: true, evidence: "Child inspected git diff and changed files before approval." }, + validation_backed: { + satisfied: true, + evidence: "Validation completed successfully.", + commands: [{ command: "bun test", exit_code: 0, summary: "bun test passed" }], + }, + risk_aware: { satisfied: true, evidence: "Child documented residual risks after review." }, + fresh: { satisfied: true, evidence: "Evidence was collected after the final diff/latest change." }, + severity_counts: { p0: 0, p1: 0, p2: 0, p3: 0 }, + ...overrides, + }; +} + +function structuredEvidenceReport(evidence: Record = structuredEvidence()): string { + return [ + "Independent separate reviewer mapped acceptance criteria against the approved spec.", + "Inspected git diff and changed files.", + "Validation commands run: bun test passed.", + "Residual risk review completed.", + "Fresh after final diff.", + "P0: none", + "P1: no findings", + "", + "```json", + JSON.stringify({ compound_engineering_evidence: evidence }, null, 2), + "```", + ].join("\n"); +} + +function proseOnlySufficientReviewReport(): string { + return [ + "Independent separate reviewer mapped acceptance criteria against the approved spec.", + "Inspected git diff and changed files.", + "Validation commands run: bun test passed.", + "Residual risk review completed.", + "Fresh after final diff.", + "P0: none", + "P1: no findings", + ].join("\n"); +} + +function sufficientReviewReport(): string { + return structuredEvidenceReport(); +} + +type MockSchema = Record; +type MockTypeOptions = Record; + +function mockSchema(type: string, options: MockTypeOptions = {}): MockSchema { + return { type, ...options }; +} + +const Type = { + String: (options?: MockTypeOptions): MockSchema => mockSchema("string", options), + Number: (options?: MockTypeOptions): MockSchema => mockSchema("number", options), + Boolean: (options?: MockTypeOptions): MockSchema => mockSchema("boolean", options), + Literal: (value: unknown): MockSchema => ({ const: value }), + Union: (variants: MockSchema[], options: MockTypeOptions = {}): MockSchema => ({ anyOf: variants, ...options }), + Object: (properties: Record, options: MockTypeOptions = {}): MockSchema => ({ type: "object", properties, ...options }), + Optional: (schema: MockSchema): MockSchema => ({ ...schema, optional: true }), +}; + +mock.module("@bastani/workflows/builtin", () => ({ + deepResearchCodebase: Object.freeze({ name: "deep-research-codebase" }), + goal: Object.freeze({ name: "goal" }), + ralph: Object.freeze({ name: "ralph" }), +})); + +mock.module("@bastani/workflows", () => ({ + Type, + defineWorkflow(name: string) { + const state: { description?: string; inputs: Record; outputs: Record; run?: unknown; worktree?: unknown } = { + inputs: {}, + outputs: {}, + }; + const builder = { + description(value: string) { + state.description = value; + return builder; + }, + input(key: string, schema: MockSchema) { + state.inputs[key] = schema; + return builder; + }, + output(key: string, schema: MockSchema) { + state.outputs[key] = schema; + return builder; + }, + worktreeFromInputs(value: unknown) { + state.worktree = value; + return builder; + }, + run(fn: unknown) { + state.run = fn; + return builder; + }, + compile() { + return Object.freeze({ name, ...state }); + }, + }; + return builder; + }, +})); + +const workflowModulePromise = import("./index.ts"); +const workflowPromise = workflowModulePromise.then((module) => module.default as { + name: string; + inputs: Record; + outputs: Record; + worktree?: unknown; + run: (ctx: unknown) => Promise>; +}); + +describe("compound-engineering mode routing", () => { + test("explicit modes are preserved", () => { + expect(resolveMode("anything", "review")).toBe("review"); + expect(resolveMode("anything", "compound-only")).toBe("compound-only"); + }); + + test("auto routes path, review, learning, vague, and concrete prompts", () => { + expect(resolveMode("specs/2026-06-05-rate-limit.md", "auto")).toBe("work"); + expect(resolveMode("docs/plans/2026-06-05-rate-limit.md", "auto")).toBe("work"); + expect(resolveMode("docs/brainstorms/onboarding.md", "auto")).toBe("plan"); + expect(resolveMode("main..feature/auth", "auto")).toBe("review"); + expect(resolveMode("Capture lessons learned from the rate limit fix", "auto")).toBe("compound-only"); + expect(resolveMode("Improve onboarding activation", "auto")).toBe("brainstorm"); + expect(resolveMode("Implement the TypeScript CLI config parser and add tests", "auto")).toBe("work"); + }); +}); + +describe("compound-engineering runner and approval helpers", () => { + test("auto runner resolves to safe handoff-only for iteration 3 and explicit runners are preserved", () => { + expect(resolveRunner("auto")).toBe("handoff-only"); + expect(resolveRunner("goal")).toBe("goal"); + expect(resolveRunner("ralph")).toBe("ralph"); + expect(resolveRunner("handoff-only")).toBe("handoff-only"); + }); + + test("approval parser recognizes approve, reject, revise, and stop intents", () => { + expect(parseApprovalDecision("Approved, proceed")).toBe("approved"); + expect(parseApprovalDecision("LGTM ship it")).toBe("approved"); + expect(parseApprovalDecision("reject this scope")).toBe("rejected"); + expect(parseApprovalDecision("please revise validation")).toBe("revise"); + expect(parseApprovalDecision("cancel for now")).toBe("stopped"); + expect(parseApprovalDecision("not sure yet")).toBe("revise"); + }); + + test("create_pr is strict true only and child handoff inputs default safely", () => { + expect(normalizeCreatePr(true)).toBe(true); + expect(normalizeCreatePr("true")).toBe(false); + expect(normalizeCreatePr(1)).toBe(false); + + const ralphHandoff = buildChildHandoff({ + runner: "ralph", + approvedPath: "specs/approved.md", + prompt: "Implement the approved spec", + maxLoops: 5, + baseBranch: "origin/main", + gitWorktreeDir: "", + createPr: false, + }); + + expect(ralphHandoff.workflow).toBe("ralph"); + expect(ralphHandoff.inputs.create_pr).toBe(false); + expect(ralphHandoff.inputs.max_loops).toBe(5); + expect(ralphHandoff.inputs.base_branch).toBe("origin/main"); + expect(ralphHandoff.command).toContain("/workflow ralph"); + expect(ralphHandoff.safe_note).toContain("explicit goal/ralph"); + + const goalHandoff = buildChildHandoff({ + runner: "goal", + approvedPath: "specs/approved.md", + prompt: "Implement the approved spec", + maxLoops: 5, + baseBranch: "origin/main", + gitWorktreeDir: "ignored-for-goal", + createPr: true, + }); + expect(goalHandoff.inputs).toMatchObject({ + max_turns: 5, + base_branch: "origin/main", + }); + expect(goalHandoff.inputs.objective).toContain("Implement the approved Compound Engineering plan/spec at specs/approved.md. Original request: Implement the approved spec"); + expect(goalHandoff.inputs.objective).toContain("compound_engineering_evidence"); + expect(goalHandoff.command).toContain("/workflow goal"); + expect(goalHandoff.command).toContain("max_turns=5"); + + const handoffOnly = buildChildHandoff({ + runner: "handoff-only", + approvedPath: "specs/approved.md", + prompt: "Implement the approved spec", + maxLoops: 5, + baseBranch: "origin/main", + gitWorktreeDir: "", + createPr: true, + }); + expect(handoffOnly.workflow).toBe("handoff-only"); + expect(handoffOnly.inputs.create_pr).toBe(false); + expect(handoffOnly.command).toBeUndefined(); + }); +}); + +describe("compound-engineering child completion and review artifact gates", () => { + test("child completion gate vetoes non-approval statuses and missing approval", async () => { + const { gateChildRunCompletion } = await workflowModulePromise; + + expect(gateChildRunCompletion({ approved: false }, "ralph")).toMatchObject({ state: "non_approved", parent_status: "needs_human" }); + for (const status of ["needs_human", "rejected", "stopped", "active"] as const) { + expect(gateChildRunCompletion({ approved: true, status }, "goal")).toMatchObject({ state: "non_approved", parent_status: "needs_human", status }); + } + for (const status of ["blocked", "failed", "failure", "error", "errored"] as const) { + expect(gateChildRunCompletion({ approved: true, status }, "goal")).toMatchObject({ state: "blocked", parent_status: "blocked", status }); + } + expect(gateChildRunCompletion({ status: "complete" }, "goal")).toMatchObject({ state: "missing_approval", parent_status: "needs_human" }); + expect(gateChildRunCompletion({ approved: true, status: "mystery" }, "goal")).toMatchObject({ state: "non_approved", parent_status: "needs_human" }); + expect(gateChildRunCompletion({ approved: true, status: "complete" }, "goal")).toMatchObject({ state: "approved" }); + }); + + test("supplemental review cannot override child approved false", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-child-veto-")); + const workflow = await workflowPromise; + let supplementalTaskRan = false; + try { + const sufficientReport = sufficientReviewReport(); + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "ralph", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (name: string, options: { output?: string }) => { + if (name === "gate-review-evidence") supplementalTaskRan = true; + if (options.output) await writeMarkdown(options.output, sufficientReport); + return { text: "draft" }; + }, + workflow: async () => ({ outputs: { approved: false, status: "needs_human", review_report: sufficientReport } }), + }); + + expect(supplementalTaskRan).toBe(false); + expect(result.status).toBe("needs_human"); + expect(result.approved).toBe(false); + expect(result.message).toContain("child completion gate"); + expect((result.implementation as Record).gate_child_run_completion).toMatchObject({ state: "non_approved" }); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("child review_report_path is loaded before inline compact review_report", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-child-review-path-")); + const workflow = await workflowPromise; + let supplementalTaskRan = false; + try { + const reviewPath = join(dir, "ralph-review.json"); + await writeFile(reviewPath, JSON.stringify({ + reviewer: "ralph", + compound_engineering_evidence: structuredEvidence(), + }), "utf8"); + + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "ralph", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (name: string, options: { output?: string }) => { + if (name === "gate-review-evidence") supplementalTaskRan = true; + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: "draft" }; + }, + workflow: async () => ({ outputs: { approved: true, review_report_path: reviewPath, review_report: "Output saved to: /tmp/compact.md" } }), + }); + + expect(supplementalTaskRan).toBe(false); + expect(result.status).toBe("complete"); + expect(result.review_report_path).toBe(displayPath(reviewPath)); + expect(((result.implementation as Record).evidence as Record).child_review_artifact).toMatchObject({ source: "review_report_path", loaded: true }); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("compact child review pointer with missing path fails closed", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-child-pointer-")); + const workflow = await workflowPromise; + let supplementalTaskRan = false; + try { + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "ralph", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (name: string, options: { output?: string }) => { + if (name === "gate-review-evidence") supplementalTaskRan = true; + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: "draft" }; + }, + workflow: async () => ({ outputs: { approved: true, review_report_path: join(dir, "missing-review.json"), review_report: "Output saved to: /tmp/compact.md" } }), + }); + + expect(supplementalTaskRan).toBe(false); + expect(result.status).toBe("blocked"); + expect(result.approved).toBe(false); + expect(result.message).toContain("review evidence gate returned blocked"); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("Ralph latest review artifact pointer is not inline evidence when missing", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-ralph-pointer-missing-")); + const workflow = await workflowPromise; + let supplementalTaskRan = false; + try { + const missingReviewPath = join(dir, "missing-ralph-review.json"); + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "ralph", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (name: string, options: { output?: string }) => { + if (name === "gate-review-evidence") supplementalTaskRan = true; + if (options.output) await writeMarkdown(options.output, sufficientReviewReport()); + return { text: "draft" }; + }, + workflow: async () => ({ outputs: { approved: true, review_report: `Latest review round artifact: ${missingReviewPath}` } }), + }); + + expect(supplementalTaskRan).toBe(false); + expect(result.status).toBe("blocked"); + expect(result.approved).toBe(false); + expect(result.message).toContain("review evidence gate returned blocked"); + expect(((result.implementation as Record).evidence as Record).child_review_artifact).toMatchObject({ compact_pointer: true, loaded: false, fail_closed: true }); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("Ralph latest review artifact pointer loads referenced artifact", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-ralph-pointer-loaded-")); + const workflow = await workflowPromise; + let supplementalTaskRan = false; + try { + const reviewPath = join(dir, "ralph-review.md"); + await writeFile(reviewPath, sufficientReviewReport(), "utf8"); + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "ralph", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (name: string, options: { output?: string }) => { + if (name === "gate-review-evidence") supplementalTaskRan = true; + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: "draft" }; + }, + workflow: async () => ({ outputs: { approved: true, review_report: `Latest review round artifact: ${reviewPath}` } }), + }); + + expect(supplementalTaskRan).toBe(false); + expect(result.status).toBe("complete"); + expect(result.review_report_path).toBe(displayPath(reviewPath)); + expect(((result.implementation as Record).evidence as Record).child_review_artifact).toMatchObject({ source: "inline_review_report_pointer", compact_pointer: true, loaded: true }); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("Latest review round artifact pointer with spaces loads referenced artifact", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound engineering latest pointer ")); + const workflow = await workflowPromise; + let supplementalTaskRan = false; + try { + const reviewDir = join(dir, "review folder with spaces"); + await mkdir(reviewDir, { recursive: true }); + const reviewPath = join(reviewDir, "ralph review artifact.md"); + await writeFile(reviewPath, sufficientReviewReport(), "utf8"); + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "ralph", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (name: string, options: { output?: string }) => { + if (name === "gate-review-evidence") supplementalTaskRan = true; + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: "draft" }; + }, + workflow: async () => ({ outputs: { approved: true, review_report: `Latest review round artifact: ${reviewPath}` } }), + }); + + expect(supplementalTaskRan).toBe(false); + expect(result.status).toBe("complete"); + expect(result.review_report_path).toBe(displayPath(reviewPath)); + expect(((result.implementation as Record).evidence as Record).child_review_artifact).toMatchObject({ + source: "inline_review_report_pointer", + path: reviewPath, + compact_pointer: true, + pointer_kind: "latest_review_round_artifact", + loaded: true, + }); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("Output saved to pointer with spaces and Atomic suffix loads referenced artifact", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound engineering output pointer ")); + const workflow = await workflowPromise; + let supplementalTaskRan = false; + try { + const reviewDir = join(dir, "review folder with spaces"); + await mkdir(reviewDir, { recursive: true }); + const reviewPath = join(reviewDir, "saved review artifact.md"); + await writeFile(reviewPath, sufficientReviewReport(), "utf8"); + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "ralph", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (name: string, options: { output?: string }) => { + if (name === "gate-review-evidence") supplementalTaskRan = true; + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: "draft" }; + }, + workflow: async () => ({ outputs: { approved: true, review_report: `Output saved to: ${reviewPath} (48.2 KB, 2847 lines). Read this file if needed.` } }), + }); + + expect(supplementalTaskRan).toBe(false); + expect(result.status).toBe("complete"); + expect(result.review_report_path).toBe(displayPath(reviewPath)); + expect(((result.implementation as Record).evidence as Record).child_review_artifact).toMatchObject({ + source: "inline_review_report_pointer", + path: reviewPath, + compact_pointer: true, + pointer_kind: "output_saved_to", + loaded: true, + }); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("missing compact pointer path with spaces fails closed and traces full path", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound engineering missing pointer ")); + const workflow = await workflowPromise; + let supplementalTaskRan = false; + try { + const missingReviewPath = join(dir, "missing review artifact with spaces.md"); + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "ralph", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (name: string, options: { output?: string }) => { + if (name === "gate-review-evidence") supplementalTaskRan = true; + if (options.output) await writeMarkdown(options.output, sufficientReviewReport()); + return { text: "draft" }; + }, + workflow: async () => ({ outputs: { approved: true, review_report: `Saved output to: ${missingReviewPath}` } }), + }); + + expect(supplementalTaskRan).toBe(false); + expect(result.status).toBe("blocked"); + expect(result.approved).toBe(false); + expect(result.message).toContain("review evidence gate returned blocked"); + expect(((result.implementation as Record).evidence as Record).child_review_artifact).toMatchObject({ + source: "inline_review_report_pointer", + path: missingReviewPath, + compact_pointer: true, + pointer_kind: "output_saved_to", + loaded: false, + fail_closed: true, + }); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("missing child review report fails closed without parsing generic result self-attestation", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-missing-child-review-")); + const workflow = await workflowPromise; + let supplementalTaskRan = false; + try { + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "ralph", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (name: string, options: { output?: string }) => { + if (name === "gate-review-evidence") supplementalTaskRan = true; + if (options.output) await writeMarkdown(options.output, sufficientReviewReport()); + return { text: "draft" }; + }, + workflow: async () => ({ + outputs: { + result: sufficientReviewReport(), + approved: true, + }, + }), + }); + + expect(supplementalTaskRan).toBe(false); + expect(result.status).toBe("blocked"); + expect(result.approved).toBe(false); + expect(((result.implementation as Record).evidence as Record).child_review_artifact).toMatchObject({ + source: "missing_child_review_report", + loaded: false, + fail_closed: true, + }); + expect(((result.implementation as Record).evidence as Record).child).toMatchObject({ blocked: true }); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("implementation receipt and final report omit raw child plan ledger and receipts", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-compact-child-output-")); + const workflow = await workflowPromise; + try { + const rawPlanSentinel = "RAW-PLAN-SENTINEL"; + const rawLedgerSentinel = "RAW-LEDGER-SENTINEL"; + const rawReceiptSentinel = "RAW-RECEIPT-SENTINEL"; + const ledgerPath = join(dir, "goal-ledger.json"); + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "ralph", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (_name: string, options: { output?: string }) => { + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: "draft" }; + }, + workflow: async () => ({ + outputs: { + result: "implementation done", + approved: true, + review_report: sufficientReviewReport(), + review_report_path: undefined, + ledger_path: ledgerPath, + plan: `${rawPlanSentinel} ${"x".repeat(4096)}`, + ledger: { details: rawLedgerSentinel }, + receipts: [{ details: rawReceiptSentinel }], + }, + }), + }); + + expect(result.status).toBe("complete"); + const implementation = result.implementation as Record; + const outputs = implementation.outputs as Record; + expect(outputs.ledger_path).toBe(ledgerPath); + expect(() => JSON.stringify(result)).not.toThrow(); + expect(outputs).not.toHaveProperty("raw"); + expect(outputs).not.toHaveProperty("plan"); + expect(outputs).not.toHaveProperty("ledger"); + expect(outputs).not.toHaveProperty("receipts"); + expect(outputs.omitted_child_output_keys).toEqual(expect.arrayContaining(["ledger", "plan", "receipts"])); + expect(JSON.stringify(result)).not.toContain(rawPlanSentinel); + expect(JSON.stringify(result)).not.toContain(rawLedgerSentinel); + expect(JSON.stringify(result)).not.toContain(rawReceiptSentinel); + + const manifest = JSON.parse(await readFile(result.manifest_path as string, "utf8")); + const finalReport = await readFile(manifest.finalReportPath, "utf8"); + expect(finalReport).not.toContain(rawPlanSentinel); + expect(finalReport).not.toContain(rawLedgerSentinel); + expect(finalReport).not.toContain(rawReceiptSentinel); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("implementation receipt compacting sanitizes non-JSON primitives and cycles", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-json-safe-child-output-")); + const workflow = await workflowPromise; + try { + const cyclic: Record = { kept_path: "/tmp/validation.log", nested: { command: "bun test" } }; + cyclic.self = cyclic; + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "ralph", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (_name: string, options: { output?: string }) => { + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: "draft" }; + }, + workflow: async () => ({ + outputs: { + result: 123n, + approved: true, + artifact_dir: "/tmp/child-artifacts", + review_report: sufficientReviewReport(), + validation_output: cyclic, + changed_files: ["src/index.ts", undefined, Symbol("skip"), () => "skip", Number.POSITIVE_INFINITY, Number.NaN, 7n], + }, + }), + }); + + expect(result.status).toBe("complete"); + expect(() => JSON.stringify(result)).not.toThrow(); + const outputs = ((result.implementation as Record).outputs as Record); + expect(outputs.result).toBe("123"); + expect(outputs.artifact_dir).toBe("/tmp/child-artifacts"); + expect(outputs.validation_output).toMatchObject({ kept_path: "/tmp/validation.log", nested: { command: "bun test" }, self: "[Circular]" }); + expect(outputs.changed_files).toEqual(["src/index.ts", null, null, null, null, null, "7"]); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("implementation receipt caps wide selected child output objects", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-wide-child-output-")); + const workflow = await workflowPromise; + try { + const omittedValueSentinel = "OMITTED-WIDE-VALIDATION-VALUE"; + const validationOutput = Object.fromEntries(Array.from({ length: 60 }, (_value, index) => [ + `key_${String(index).padStart(2, "0")}`, + index === 55 ? omittedValueSentinel : `value_${index}`, + ])); + + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "ralph", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (_name: string, options: { output?: string }) => { + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: "draft" }; + }, + workflow: async () => ({ + outputs: { + result: "implementation done", + approved: true, + review_report: sufficientReviewReport(), + validation_output: validationOutput, + }, + }), + }); + + expect(result.status).toBe("complete"); + expect(() => JSON.stringify(result)).not.toThrow(); + const outputs = ((result.implementation as Record).outputs as Record); + const compactValidationOutput = outputs.validation_output as Record; + expect(compactValidationOutput.key_00).toBe("value_0"); + expect(compactValidationOutput.key_49).toBe("value_49"); + expect(compactValidationOutput).not.toHaveProperty("key_55"); + expect(compactValidationOutput.original_keys).toBe(60); + expect(compactValidationOutput.omitted_key_count).toBe(10); + expect(compactValidationOutput.omitted_keys).toEqual(expect.arrayContaining(["key_55"])); + expect(JSON.stringify(result)).not.toContain(omittedValueSentinel); + + const manifest = JSON.parse(await readFile(result.manifest_path as string, "utf8")); + const finalReport = await readFile(manifest.finalReportPath, "utf8"); + expect(finalReport).not.toContain(omittedValueSentinel); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); +}); + +describe("compound-engineering review evidence parser and reducer", () => { + test("severity parser ignores empty headings and counts real finding lines", async () => { + const { parseEvidenceFromText } = await workflowModulePromise; + const evidence = parseEvidenceFromText([ + "## P0", + "P0: none", + "P1: no findings", + "P0: none found", + "P1: none identified", + "P0: none reported", + "P1: no issues", + "P1: no blockers", + "- none", + "### P2", + "- P1: data loss bug", + "[P0] deletes user data", + "p2 - issue in fallback", + "P3: small docs nit", + ].join("\n")); + + expect(evidence.severity_counts).toEqual({ p0: 1, p1: 1, p2: 1, p3: 1 }); + }); + + test("validation availability alone does not satisfy validation-backed evidence", async () => { + const { parseEvidenceFromText } = await workflowModulePromise; + const evidence = parseEvidenceFromText([ + "Independent separate reviewer mapped acceptance criteria against the approved spec.", + "Inspected git diff and changed files.", + "Validation commands available: bun test.", + "Residual risk review completed.", + "Fresh after final diff.", + "P0: none", + "P1: no findings", + ].join("\n")); + + expect(evidence.validation_failed).toBe(false); + expect(evidence.validation_backed).toBeUndefined(); + expect(reduceReviewEvidence(evidence).missing).toContain("validation_backed"); + expect(reduceReviewEvidence(evidence).decision).not.toBe("sufficient"); + }); + + test("bare validation execution wording does not satisfy validation-backed evidence", async () => { + const { parseEvidenceFromText } = await workflowModulePromise; + const evidence = parseEvidenceFromText([ + "Independent separate reviewer mapped acceptance criteria against the approved spec.", + "Inspected git diff and changed files.", + "Validation commands run: bun test.", + "Residual risk review completed.", + "Fresh after final diff.", + "P0: none", + "P1: no findings", + ].join("\n")); + const reduction = reduceReviewEvidence(evidence); + + expect(evidence.validation_failed).toBe(false); + expect(evidence.validation_backed).toBeUndefined(); + expect(reduction.missing).toContain("validation_backed"); + expect(reduction.decision).not.toBe("sufficient"); + }); + + test("negated validation success wording fails closed before success matching", async () => { + const { parseEvidenceFromText } = await workflowModulePromise; + const negatedSuccessLines = [ + "Validation commands run: bun test not successful.", + "Validation commands run: bun test not passed.", + "Validation commands run: bun test. Not successful.", + "Validation commands run: not all tests passed.", + "Validation commands run: bun test. Not all tests passed.", + ]; + + for (const line of negatedSuccessLines) { + const evidence = parseEvidenceFromText([ + "Independent separate reviewer mapped acceptance criteria against the approved spec.", + "Inspected git diff and changed files.", + line, + "Residual risk review completed.", + "Fresh after final diff.", + "P0: none", + "P1: no findings", + ].join("\n")); + const reduction = reduceReviewEvidence(evidence); + + expect(evidence.validation_failed).toBe(true); + expect(evidence.validation_backed).toBe(false); + expect(reduction.decision).not.toBe("sufficient"); + } + }); + + test("validation no-run wording fails closed instead of satisfying validation evidence", async () => { + const { parseEvidenceFromText } = await workflowModulePromise; + const noRunLines = [ + "Validation commands run: none.", + "Validation commands run: no commands.", + "Validation commands run: zero.", + "Tests run: none.", + "No validation commands were run.", + "No validation was run.", + "No validation ran.", + "No tests were run.", + "Validation skipped.", + ]; + + for (const line of noRunLines) { + const evidence = parseEvidenceFromText([ + "Independent separate reviewer mapped acceptance criteria against the approved spec.", + "Inspected git diff and changed files.", + line, + "Residual risk review completed.", + "Fresh after final diff.", + "P0: none", + "P1: no findings", + ].join("\n")); + const reduction = reduceReviewEvidence(evidence); + + expect(evidence.validation_failed).toBe(false); + expect(evidence.validation_backed).toBe(false); + expect(reduction.missing).toContain("validation_backed"); + expect(reduction.decision).not.toBe("sufficient"); + } + }); + + test("failed validation outcomes set validation_failed and cannot satisfy validation evidence", async () => { + const { parseEvidenceFromText } = await workflowModulePromise; + const failureLines = [ + "Validation commands run: bun test did not pass.", + "Validation commands run: bun test reported 2 failures.", + "Validation commands run: bun test reported 3 errors.", + "bun test failed.", + "Validation command exited with exit code 1.", + "Validation command exited with code 1.", + "Validation command exited with status 2.", + "Validation command returned code 3.", + "Validation command returned status 4.", + "Validation command status: 2.", + "Validation command code: 1.", + "Validation command exit code: 1.", + "Validation command exit status: 5.", + "Validation command failed with exit status 5.", + "Tests completed with nonzero status.", + "Validation command returned non-zero exit.", + "Test command errored before completion.", + "Validation is failing in CI.", + ]; + + for (const line of failureLines) { + const evidence = parseEvidenceFromText([ + "Independent separate reviewer mapped acceptance criteria against the approved spec.", + "Inspected git diff and changed files.", + line, + "Residual risk review completed.", + "Fresh after final diff.", + "P0: none", + "P1: no findings", + ].join("\n")); + + expect(evidence.validation_failed).toBe(true); + expect(evidence.validation_backed).toBe(false); + expect(reduceReviewEvidence(evidence).decision).not.toBe("sufficient"); + } + }); + + test("adjacent validation exit code and status segments fail closed", async () => { + const { parseEvidenceFromText } = await workflowModulePromise; + const adjacentFailures = [ + "Validation commands run: bun test. Exit code: 1.", + "Validation commands run: bun test; status: 2.", + "Validation commands run: bun test. Returned code: 3.", + "Validation commands run: bun test; nonzero status.", + ]; + + for (const line of adjacentFailures) { + const evidence = parseEvidenceFromText([ + "Independent separate reviewer mapped acceptance criteria against the approved spec.", + "Inspected git diff and changed files.", + line, + "Residual risk review completed.", + "Fresh after final diff.", + "P0: none", + "P1: no findings", + ].join("\n")); + + expect(evidence.validation_failed).toBe(true); + expect(evidence.validation_backed).toBe(false); + expect(reduceReviewEvidence(evidence).decision).not.toBe("sufficient"); + } + }); + + test("contextual validation errors fail but expected negative-path errors can pass", async () => { + const { parseEvidenceFromText } = await workflowModulePromise; + const contextualErrors = [ + "Validation commands run: bun test reported 3 errors.", + "Validation commands run: bun test had 3 errors.", + "Validation commands run: bun test error count: 3.", + "Validation commands run: bun test errors: 3.", + "Validation commands run: bun test failed with errors.", + "Validation commands run: bun test errored.", + ]; + + for (const line of contextualErrors) { + const evidence = parseEvidenceFromText([ + "Independent separate reviewer mapped acceptance criteria against the approved spec.", + "Inspected git diff and changed files.", + line, + "Residual risk review completed.", + "Fresh after final diff.", + "P0: none", + "P1: no findings", + ].join("\n")); + + expect(evidence.validation_failed).toBe(true); + expect(evidence.validation_backed).toBe(false); + expect(reduceReviewEvidence(evidence).decision).not.toBe("sufficient"); + } + + const expectedErrorEvidence = parseEvidenceFromText([ + "Independent separate reviewer mapped acceptance criteria against the approved spec.", + "Inspected git diff and changed files.", + "Validation commands run: errors were expected in negative-path tests and the command passed.", + "Residual risk review completed.", + "Fresh after final diff.", + "P0: none", + "P1: no findings", + ].join("\n")); + + expect(expectedErrorEvidence.validation_failed).toBe(false); + expect(expectedErrorEvidence.validation_backed).toBe(true); + expect(reduceReviewEvidence(expectedErrorEvidence).decision).toBe("sufficient"); + }); + + test("explicit pass and status-zero wording satisfies validation-backed evidence", async () => { + const { parseEvidenceFromText } = await workflowModulePromise; + const successLines = [ + "Validation commands run: bun test passed.", + "Validation commands run: bun test succeeded.", + "Validation commands run: bun test status: 0.", + "Validation commands run: bun test. Exit code: 0.", + "bun test returned status: 0.", + ]; + + for (const line of successLines) { + const evidence = parseEvidenceFromText([ + "Independent separate reviewer mapped acceptance criteria against the approved spec.", + "Inspected git diff and changed files.", + line, + "Residual risk review completed.", + "Fresh after final diff.", + "P0: none", + "P1: no findings", + ].join("\n")); + + expect(evidence.validation_failed).toBe(false); + expect(evidence.validation_backed).toBe(true); + expect(reduceReviewEvidence(evidence).decision).toBe("sufficient"); + } + }); + + test("negated validation failure wording does not set validation_failed", async () => { + const { parseEvidenceFromText } = await workflowModulePromise; + const noFailureLines = [ + "Validation commands run: bun test passed; no validation commands failed.", + "Validation commands run: bun test passed; No tests failed.", + "Validation commands run: bun test passed; zero validation failures.", + "Validation commands run: bun test passed; 0 failures.", + "Validation commands run: bun test passed; Validation failures: none.", + "Validation commands run: bun test passed; Validation errors: none.", + "Validation commands run: error handling tests passed.", + "Validation commands run: bun test passed without validation failures.", + ]; + + for (const line of noFailureLines) { + const evidence = parseEvidenceFromText([ + "Independent separate reviewer mapped acceptance criteria against the approved spec.", + "Inspected git diff and changed files.", + line, + "Residual risk review completed.", + "Fresh after final diff.", + "P0: none", + "P1: no findings", + ].join("\n")); + + expect(evidence.validation_failed).toBe(false); + expect(evidence.validation_backed).toBe(true); + expect(reduceReviewEvidence(evidence).decision).toBe("sufficient"); + } + + expect(parseEvidenceFromText("No tests failed.").validation_failed).toBe(false); + expect(parseEvidenceFromText("zero validation failures.").validation_failed).toBe(false); + expect(parseEvidenceFromText("0 failures.").validation_failed).toBe(false); + }); + + test("Ralph JSON review artifacts expose findings and raw text as evidence", async () => { + const { parseEvidenceFromText, reviewArtifactToEvidenceText } = await workflowModulePromise; + const evidenceText = reviewArtifactToEvidenceText(JSON.stringify({ + reviewer: "reviewer-a", + decision: { + findings: [ + { title: "[P2] fallback bug", body: "The fallback misses an edge case.", priority: 2 }, + ], + overall_correctness: "patch is correct", + overall_explanation: "Reviewed after final diff.", + }, + raw_text: "Independent separate reviewer mapped acceptance criteria against the approved spec. Inspected git diff and changed files. Validation commands run: bun test passed. Residual risk review completed. Fresh after final diff.", + validation_notes: "No validation commands failed.", + severity_counts: { p0: 0, p1: 0, p2: 1, p3: 0 }, + })); + const evidence = parseEvidenceFromText(evidenceText); + + expect(evidenceText).toContain("overall_correctness: patch is correct"); + expect(evidenceText).toContain("finding_title: [P2] fallback bug"); + expect(evidenceText).toContain("validation_notes: No validation commands failed."); + expect(evidence.severity_counts).toMatchObject({ p0: 0, p1: 0, p2: 3, p3: 0 }); + expect(evidence.validation_failed).toBe(false); + }); + + test("acceptance evidence requires explicit mapping, checking, tracing, verification, or coverage", async () => { + const { parseEvidenceFromText } = await workflowModulePromise; + const bareMentions = parseEvidenceFromText([ + "Independent separate reviewer completed the review.", + "The latest approved spec and criteria were mentioned.", + "Inspected git diff and changed files.", + "Validation commands run: bun test passed.", + "Residual risk review completed.", + "Fresh after final diff.", + ].join("\n")); + const positiveControls = [ + "Acceptance criteria were checked and traced against the approved spec.", + "Reviewer verified approved plan coverage for each acceptance criterion.", + ]; + const negativeControls = [ + "The approved spec was not checked against acceptance criteria.", + "Unchecked spec; acceptance criteria were not traced.", + "Spec not verified against acceptance criteria.", + ]; + + expect(bareMentions.acceptance_mapped).toBeUndefined(); + expect(reduceReviewEvidence(bareMentions).missing).toContain("acceptance_mapped"); + for (const line of positiveControls) { + expect(parseEvidenceFromText(line).acceptance_mapped).toBe(true); + } + for (const line of negativeControls) { + expect(parseEvidenceFromText(line).acceptance_mapped).toBe(false); + } + }); + + test("freshness requires review after the final or current diff/latest change", async () => { + const { parseEvidenceFromText } = await workflowModulePromise; + + expect(parseEvidenceFromText("Latest change is mentioned in the review summary.").fresh).toBeUndefined(); + expect(parseEvidenceFromText("Reviewed after latest change.").fresh).toBe(true); + expect(parseEvidenceFromText("Fresh after final diff.").fresh).toBe(true); + expect(parseEvidenceFromText("Review evidence after current diff was checked.").fresh).toBe(true); + expect(parseEvidenceFromText("Review predates latest change.").fresh).toBe(false); + expect(parseEvidenceFromText("Reviewed before final diff.").fresh).toBe(false); + }); + + test("diff unavailable wording prevents diff-aware sufficiency", async () => { + const { parseEvidenceFromText } = await workflowModulePromise; + const unavailableLines = [ + "Unable to inspect git diff for this run.", + "Could not inspect changed files.", + "Cannot review current diff.", + "Diff unavailable.", + "Diff was unavailable.", + "Diff could not be inspected.", + "No git diff was available.", + "Diff was not available.", + "Changed files were unavailable.", + ]; + + for (const line of unavailableLines) { + const evidence = parseEvidenceFromText([ + "Independent separate reviewer mapped acceptance criteria against the approved spec.", + line, + "Validation commands run: bun test passed.", + "Residual risk review completed.", + "Fresh after final diff.", + "P0: none", + "P1: no findings", + ].join("\n")); + + expect(evidence.diff_aware).toBe(false); + expect(reduceReviewEvidence(evidence).missing).toContain("diff_aware"); + expect(reduceReviewEvidence(evidence).decision).not.toBe("sufficient"); + } + }); + + test("negated and skipped review phrases do not satisfy evidence criteria", async () => { + const { parseEvidenceFromText } = await workflowModulePromise; + const evidence = parseEvidenceFromText([ + "Review is not independent.", + "No acceptance mapping was provided.", + "Diff not inspected.", + "Validation commands skipped; no tests run.", + "No residual risk assessment.", + "Not fresh.", + "P0: none", + "P1: no findings", + ].join("\n")); + + expect(evidence).toMatchObject({ + independent: false, + acceptance_mapped: false, + diff_aware: false, + validation_backed: false, + risk_aware: false, + fresh: false, + severity_counts: { p0: 0, p1: 0, p2: 0, p3: 0 }, + }); + expect(reduceReviewEvidence(evidence).decision).not.toBe("sufficient"); + }); + + test("negated criterion variants stay missing even with other affirmative evidence", async () => { + const { parseEvidenceFromText } = await workflowModulePromise; + const positiveLines = { + independent: "Independent separate reviewer completed the review.", + acceptance_mapped: "Acceptance criteria were mapped to the approved spec.", + diff_aware: "Inspected git diff and changed files.", + validation_backed: "Validation commands run: bun test passed.", + risk_aware: "Residual risk review completed.", + fresh: "Fresh after final diff.", + } as const; + const cases = [ + { criterion: "independent", negatedLine: "This review was not independent." }, + { criterion: "acceptance_mapped", negatedLine: "Acceptance criteria were not mapped to the approved spec." }, + { criterion: "diff_aware", negatedLine: "No git diff or changed files were inspected." }, + { criterion: "validation_backed", negatedLine: "Validation was skipped; no tests run." }, + { criterion: "risk_aware", negatedLine: "Risk is unknown." }, + { criterion: "fresh", negatedLine: "Evidence is stale." }, + ] as const; + + for (const { criterion, negatedLine } of cases) { + const report = Object.entries(positiveLines) + .map(([key, line]) => key === criterion ? negatedLine : line) + .concat(["P0: none", "P1: no findings"]) + .join("\n"); + const evidence = parseEvidenceFromText(report); + const reduction = reduceReviewEvidence(evidence); + + expect(evidence[criterion]).toBe(false); + expect(reduction.missing).toContain(criterion); + expect(reduction.decision).not.toBe("sufficient"); + } + }); + + test("common negated acceptance and validation variants do not satisfy sufficiency", async () => { + const { parseEvidenceFromText } = await workflowModulePromise; + const evidence = parseEvidenceFromText([ + "Independent separate reviewer completed the review.", + "No acceptance criteria were mapped.", + "Inspected git diff and changed files.", + "No tests were run.", + "Residual risk review completed.", + "Fresh after final diff.", + "P0: none", + "P1: no findings", + ].join("\n")); + const reduction = reduceReviewEvidence(evidence); + + expect(evidence.acceptance_mapped).toBe(false); + expect(evidence.validation_backed).toBe(false); + expect(reduction.missing).toEqual(expect.arrayContaining(["acceptance_mapped", "validation_backed"])); + expect(reduction.decision).not.toBe("sufficient"); + }); + + test("common negated independent, acceptance, diff, and test variants fail closed", async () => { + const { parseEvidenceFromText } = await workflowModulePromise; + const evidence = parseEvidenceFromText([ + "No separate reviewer completed this.", + "Acceptance mapping was not provided.", + "Diff review skipped.", + "Tests skipped.", + "Residual risk review completed.", + "Fresh after final diff.", + "P0: none", + "P1: no findings", + ].join("\n")); + const reduction = reduceReviewEvidence(evidence); + + expect(evidence.independent).toBe(false); + expect(evidence.acceptance_mapped).toBe(false); + expect(evidence.diff_aware).toBe(false); + expect(evidence.validation_backed).toBe(false); + expect(reduction.missing).toEqual(expect.arrayContaining(["independent", "acceptance_mapped", "diff_aware", "validation_backed"])); + expect(reduction.decision).not.toBe("sufficient"); + }); + + test("affirmative non-negated review evidence still satisfies all criteria", async () => { + const { parseEvidenceFromText } = await workflowModulePromise; + const evidence = parseEvidenceFromText([ + "Independent separate reviewer mapped acceptance criteria against the approved spec.", + "Inspected git diff and changed files.", + "Validation commands run: bun test passed.", + "Residual risk review completed.", + "Fresh after final diff.", + "P0: none", + "P1: no findings", + ].join("\n")); + + expect(evidence).toMatchObject({ + independent: true, + acceptance_mapped: true, + diff_aware: true, + validation_backed: true, + risk_aware: true, + fresh: true, + severity_counts: { p0: 0, p1: 0, p2: 0, p3: 0 }, + }); + expect(reduceReviewEvidence(evidence).decision).toBe("sufficient"); + }); + + test("sufficient evidence passes only when every criterion is true", () => { + const reduction = reduceReviewEvidence({ + independent: true, + acceptance_mapped: true, + diff_aware: true, + validation_backed: true, + risk_aware: true, + fresh: true, + }); + + expect(reduction.decision).toBe("sufficient"); + expect(reduction.missing).toEqual([]); + }); + + test("one missing non-fresh criterion asks for targeted review", () => { + const reduction = reduceReviewEvidence({ + independent: true, + acceptance_mapped: true, + diff_aware: false, + validation_backed: true, + risk_aware: true, + fresh: true, + }); + + expect(reduction.decision).toBe("targeted_review"); + expect(reduction.missing).toEqual(["diff_aware"]); + }); + + test("stale or multiple missing criteria require full review", () => { + expect(reduceReviewEvidence({ + independent: true, + acceptance_mapped: true, + diff_aware: true, + validation_backed: true, + risk_aware: true, + fresh: false, + }).decision).toBe("full_review"); + + expect(reduceReviewEvidence({ + independent: false, + acceptance_mapped: true, + diff_aware: false, + validation_backed: true, + risk_aware: true, + fresh: true, + }).decision).toBe("full_review"); + }); + + test("blocking severities, conflicts, validation failures, and blockers fail closed", () => { + expect(reduceReviewEvidence({ severity_counts: { p1: 1 } }).decision).toBe("fixes_needed"); + expect(reduceReviewEvidence({ conflicted: true }).decision).toBe("needs_human"); + expect(reduceReviewEvidence({ validation_failed: true }).decision).toBe("needs_human"); + expect(reduceReviewEvidence({ blocked: true }).decision).toBe("blocked"); + expect(missingReviewCriteria({})).toEqual([ + "independent", + "acceptance_mapped", + "diff_aware", + "validation_backed", + "risk_aware", + "fresh", + ]); + }); +}); + +describe("compound-engineering artifact path helpers and discoverability", () => { + test("markdown writes use exclusive collision-safe suffixes", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-")); + try { + const desired = join(dir, "report.md"); + const first = await writeMarkdown(desired, "first"); + const second = await writeMarkdown(desired, "second"); + const third = await writeMarkdown(desired, "third"); + + expect(first).toBe(desired); + expect(second).toBe(join(dir, "report-2.md")); + expect(third).toBe(join(dir, "report-3.md")); + expect(await readFile(first, "utf8")).toBe("first\n"); + expect(await readFile(second, "utf8")).toBe("second\n"); + expect(await readFile(third, "utf8")).toBe("third\n"); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("saved stage report reads saved Markdown before inline file-only references", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-report-")); + const suffixedPointer = "Output saved to: /tmp/P1-compact-reference.md (48.2 KB, 2847 lines). Read this file if needed."; + try { + const reportPath = join(dir, "review.md"); + await writeFile(reportPath, "saved sufficient review evidence\n", "utf8"); + const loaded = await loadSavedStageReport(reportPath, { text: suffixedPointer }); + + expect(loaded).toEqual({ path: reportPath, body: "saved sufficient review evidence\n", source: "saved-file" }); + await expect(loadSavedStageReport(join(dir, "missing.md"), { text: "Output saved to: /tmp/P1-compact-reference.md" })).rejects.toThrow("compact output reference"); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("missing saved stage report rejects suffixed Atomic file-only pointers", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-missing-report-")); + try { + const missingPath = join(dir, "missing.md"); + for (const pointer of [ + "Output saved to: /tmp/review.md (48.2 KB, 2847 lines). Read this file if needed.", + "Saved output to: /tmp/review.md (48.2 KB, 2847 lines). Read this file if needed.", + "Saved to: /tmp/review.md (48.2 KB, 2847 lines). Read this file if needed.", + ]) { + await expect(loadSavedStageReport(missingPath, { text: pointer })).rejects.toThrow("file-only pointer"); + } + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("substantive inline stage report remains an accepted fallback", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-inline-report-")); + try { + const missingPath = join(dir, "missing.md"); + const loaded = await loadSavedStageReport(missingPath, { text: "Independent review body with substantive evidence." }); + + expect(loaded).toEqual({ path: missingPath, body: "Independent review body with substantive evidence.", source: "inline-fallback" }); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("effective worktree root resolves Git top-level from a subdirectory for Ralph", async () => { + const tempDir = await realpath(await mkdtemp(join(tmpdir(), "compound-engineering-git-"))); + try { + await gitInit(tempDir); + const subdir = join(tempDir, "nested", "cwd"); + await mkdir(subdir, { recursive: true }); + + expect(await resolveEffectiveWorktreeRoot("../requested-wt", subdir)).toBe(tempDir); + expect(await resolveEffectiveWorktreeRoot("", subdir)).toBe(""); + } finally { + await rm(tempDir, { recursive: true, force: true }); + } + }); + + test("slugs are date-prefixed, compact, and traversal-safe", () => { + const now = new Date("2026-06-05T12:00:00.000Z"); + expect(slugifyTopic("../../Add OAuth Login!!! and docs", "fallback")).toBe("add-oauth-login-and-docs"); + expect(datedMarkdownPath("docs/plans", "../../Add OAuth Login!!! and docs", "plan", now)).toBe("docs/plans/2026-06-05-add-oauth-login-and-docs.md"); + }); + + test("workflow is discoverable and documents safe iteration-3 posture", () => { + const source = workflowSource(); + expect(source).toContain('defineWorkflow("compound-engineering")'); + expect(source).toContain('const WORKFLOW_NAME = "compound-engineering"'); + expect(source).toContain('.worktreeFromInputs({'); + expect(source).toContain('ctx.workflow(runner === "goal" ? goal : ralph'); + expect(source).toContain('loadSavedStageReport(reviewPath, review)'); + expect(source).not.toContain('ctx.task("gate-review-evidence"'); + expect(source).toContain('resolveEffectiveWorktreeRoot(gitWorktreeDir, cwd)'); + expect(source).toContain('captureLearningArtifact({'); + expect(source).toContain('.compile()'); + expect(source).toContain('child_workflow_launched: false'); + expect(source).toContain('child_workflow_launched: true'); + expect(source).toContain('create_pr'); + + expect(workflowReadme()).toContain("EveryInc"); + expect(workflowReadme()).toContain("no implementation before approval"); + expect(workflowReadme()).toContain("explicit `runner=goal` or `runner=ralph`"); + expect(registryReadme()).toContain("compound-engineering"); + expect(rootReadme()).toContain("compound-engineering"); + expect(rootGitignore()).toContain("/.compound-engineering-*/"); + }); + + test("explicit goal and ralph run child workflows only after approval with runner-specific inputs", async () => { + for (const runner of ["goal", "ralph"] as const) { + const dir = await realpath(await mkdtemp(join(tmpdir(), `compound-engineering-${runner}-`))); + if (runner === "ralph") await gitInit(dir); + const workflow = await workflowPromise; + const workflowCalls: Array<{ workflow: { name: string }; options: { inputs: Record } }> = []; + const cwd = runner === "ralph" ? join(dir, "nested", "cwd") : dir; + try { + if (runner === "ralph") await mkdir(cwd, { recursive: true }); + const result = await workflow.run({ + cwd, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner, + max_loops: 3, + base_branch: "origin/main", + git_worktree_dir: runner === "ralph" ? "../requested-wt" : "", + create_pr: "true", + learning_mode: "off", + memory_scope: "none", + }, + ui: { + select: async () => "Approve", + input: async () => "", + }, + task: async (_name: string, options: { output?: string }) => { + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: sufficientReviewReport() }; + }, + workflow: async (childWorkflow: { name: string }, options: { inputs: Record }) => { + workflowCalls.push({ workflow: childWorkflow, options }); + return { + outputs: { + result: "implementation done", + review_report: sufficientReviewReport(), + approved: true, + }, + }; + }, + }); + + expect(result.status).toBe("complete"); + expect(result.implementation).toBeDefined(); + expect(JSON.stringify(result)).not.toContain(":undefined"); + expect(workflowCalls).toHaveLength(1); + expect(workflowCalls[0].workflow.name).toBe(runner); + if (runner === "goal") { + expect(workflowCalls[0].options.inputs).toMatchObject({ + objective: expect.stringContaining("Implement the approved Compound Engineering plan/spec"), + max_turns: 3, + base_branch: "origin/main", + }); + expect(workflowCalls[0].options.inputs).not.toHaveProperty("create_pr"); + } else { + expect(workflowCalls[0].options.inputs).toMatchObject({ + prompt: expect.stringContaining("Implement the approved Compound Engineering plan/spec"), + max_loops: 3, + base_branch: "origin/main", + git_worktree_dir: dir, + create_pr: false, + }); + } + } finally { + await rm(dir, { recursive: true, force: true }); + } + } + }); + + test("review-only evidence uses saved file-only body instead of compact reference", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-review-only-")); + const workflow = await workflowPromise; + try { + const sufficientReport = sufficientReviewReport(); + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "main..feature/auth", + mode: "review", + runner: "handoff-only", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (_name: string, options: { output?: string }) => { + if (options.output) await writeMarkdown(options.output, sufficientReport); + return { text: "Output saved to: /tmp/P1-compact-reference.md" }; + }, + workflow: async () => ({}), + }); + + expect(result.status).toBe("review_only"); + expect(result.approved).toBe(true); + expect(result.message).toContain("evidence gate decision: sufficient"); + expect(result.review_report_path).toContain("compound-engineering/"); + expect(JSON.stringify(result)).not.toContain(":undefined"); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("review-only missing saved compact report fails closed", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-review-only-missing-")); + const workflow = await workflowPromise; + try { + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "main..feature/auth", + mode: "review", + runner: "handoff-only", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async () => ({ text: "Output saved to: /tmp/P1-compact-reference.md" }), + workflow: async () => ({}), + }); + + expect(result.status).toBe("blocked"); + expect(result.approved).toBe(false); + expect(result.message).toContain("Review-only evidence gate returned blocked"); + expect(result.message).not.toContain("Review-only report produced"); + expect(result).not.toHaveProperty("review_report_path"); + expect(JSON.stringify(result)).not.toContain(":undefined"); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("review-only missing saved suffixed pointer report fails closed", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-review-only-missing-suffixed-")); + const workflow = await workflowPromise; + try { + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "main..feature/auth", + mode: "review", + runner: "handoff-only", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async () => ({ text: "Output saved to: /tmp/review.md (48.2 KB, 2847 lines). Read this file if needed." }), + workflow: async () => ({}), + }); + + expect(result.status).toBe("blocked"); + expect(result.approved).toBe(false); + expect(result.message).toContain("file-only pointer"); + expect(result.message).not.toContain("Review-only report produced"); + expect(result).not.toHaveProperty("review_report_path"); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("structured evidence with all criteria true returns complete", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-structured-complete-")); + const workflow = await workflowPromise; + let supplementalTaskRan = false; + try { + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "goal", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (name: string, options: { output?: string }) => { + if (name === "gate-review-evidence") supplementalTaskRan = true; + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: "draft" }; + }, + workflow: async () => ({ outputs: { result: "implementation done", compound_engineering_evidence: structuredEvidence(), approved: true } }), + }); + + expect(supplementalTaskRan).toBe(false); + expect(result.status).toBe("complete"); + expect(result.approved).toBe(true); + expect(((result.implementation as Record).gate_review_evidence as Record).decision).toBe("sufficient"); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("structured evidence missing one criterion returns needs_human without supplemental review", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-structured-missing-")); + const workflow = await workflowPromise; + let supplementalTaskRan = false; + try { + const incompleteEvidence = structuredEvidence({ risk_aware: { satisfied: false, evidence: "Child did not complete residual risk review." } }); + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "goal", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (name: string, options: { output?: string }) => { + if (name === "gate-review-evidence") supplementalTaskRan = true; + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: "draft" }; + }, + workflow: async () => ({ outputs: { result: "implementation done with incomplete review evidence", compound_engineering_evidence: incompleteEvidence, approved: true } }), + }); + + expect(supplementalTaskRan).toBe(false); + expect(result.status).toBe("needs_human"); + expect(result.approved).toBe(false); + expect(result.message).toContain("Missing structured evidence criteria: risk_aware"); + expect(((result.implementation as Record).gate_review_evidence as Record).missing).toContain("risk_aware"); + expect(JSON.stringify(result)).not.toContain(":undefined"); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("structured evidence with validation_failed true does not complete", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-validation-failed-")); + const workflow = await workflowPromise; + try { + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "goal", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (_name: string, options: { output?: string }) => { + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: "draft" }; + }, + workflow: async () => ({ outputs: { result: "implementation done", compound_engineering_evidence: structuredEvidence({ validation_failed: true }), approved: true } }), + }); + + expect(result.status).toBe("needs_human"); + expect(result.approved).toBe(false); + expect(result.message.toLowerCase()).toContain("validation failed"); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("structured evidence with p0 or p1 findings does not complete", async () => { + for (const severity of ["p0", "p1"] as const) { + const dir = await mkdtemp(join(tmpdir(), `compound-engineering-${severity}-finding-`)); + const workflow = await workflowPromise; + try { + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "goal", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (_name: string, options: { output?: string }) => { + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: "draft" }; + }, + workflow: async () => ({ outputs: { result: "implementation done", compound_engineering_evidence: structuredEvidence({ severity_counts: { p0: severity === "p0" ? 1 : 0, p1: severity === "p1" ? 1 : 0, p2: 0, p3: 0 } }), approved: true } }), + }); + + expect(result.status).toBe("needs_human"); + expect(result.approved).toBe(false); + expect(((result.implementation as Record).gate_review_evidence as Record).decision).toBe("fixes_needed"); + } finally { + await rm(dir, { recursive: true, force: true }); + } + } + }); + + test("structured evidence with blocked or conflicted true does not complete", async () => { + for (const flag of ["blocked", "conflicted"] as const) { + const dir = await mkdtemp(join(tmpdir(), `compound-engineering-${flag}-flag-`)); + const workflow = await workflowPromise; + try { + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "goal", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (_name: string, options: { output?: string }) => { + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: "draft" }; + }, + workflow: async () => ({ outputs: { result: "implementation done", compound_engineering_evidence: structuredEvidence({ [flag]: true }), approved: true } }), + }); + + expect(result.status).toBe(flag === "blocked" ? "blocked" : "needs_human"); + expect(result.approved).toBe(false); + expect(JSON.stringify(result.implementation)).toContain(flag); + } finally { + await rm(dir, { recursive: true, force: true }); + } + } + }); + + test("structured validation commands with failing exit codes do not complete", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-command-failed-")); + const workflow = await workflowPromise; + try { + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "goal", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (_name: string, options: { output?: string }) => { + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: "draft" }; + }, + workflow: async () => ({ + outputs: { + result: "implementation done", + compound_engineering_evidence: structuredEvidence({ + validation_backed: { satisfied: true, evidence: "Validation was attempted.", commands: [{ command: "bun test", exit_code: 1, summary: "bun test failed" }] }, + }), + approved: true, + }, + }), + }); + + expect(result.status).toBe("needs_human"); + expect(result.approved).toBe(false); + expect(result.message).toContain("validation_backed.commands"); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("prose-only child review reports do not produce complete", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-prose-only-child-")); + const workflow = await workflowPromise; + let supplementalTaskRan = false; + try { + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "goal", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (name: string, options: { output?: string }) => { + if (name === "gate-review-evidence") supplementalTaskRan = true; + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: "draft" }; + }, + workflow: async () => ({ outputs: { result: "implementation done", review_report: proseOnlySufficientReviewReport(), approved: true } }), + }); + + expect(supplementalTaskRan).toBe(false); + expect(result.status).toBe("needs_human"); + expect(result.approved).toBe(false); + expect(result.message).toContain("missing named compound_engineering_evidence block"); + expect(result.message).not.toContain("review evidence gate is sufficient"); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("parent no longer invokes gate-review-evidence", async () => { + const source = workflowSource(); + expect(source).not.toContain('ctx.task("gate-review-evidence"'); + expect(source).not.toContain("mergeEvidence("); + }); + + test("post-validation learning writes docs solutions path for lightweight mode", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-learning-")); + const workflow = await workflowPromise; + const taskNames: string[] = []; + try { + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "goal", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "lightweight", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (name: string, options: { output?: string }) => { + taskNames.push(name); + if (options.output) await writeMarkdown(options.output, name === "capture-learning" ? "learning doc" : "task output"); + return { text: "draft" }; + }, + workflow: async () => ({ + outputs: { + result: "implementation done", + approved: true, + review_report: sufficientReviewReport(), + }, + }), + }); + + expect(result.status).toBe("complete"); + expect(taskNames).toContain("capture-learning"); + expect(result.learning_doc_path).toContain("docs/solutions/"); + expect(await readFile(result.learning_doc_path as string, "utf8")).toBe("learning doc\n"); + const manifest = JSON.parse(await readFile(result.manifest_path as string, "utf8")); + expect(manifest.input.selected_learning_mode).toBe("lightweight"); + expect(manifest.artifacts["learning-doc"]).toBe(result.learning_doc_path); + expect(JSON.stringify(result)).not.toContain(":undefined"); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("post-validation learning off does not write docs solutions path", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-learning-off-")); + const workflow = await workflowPromise; + const taskNames: string[] = []; + try { + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "goal", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: false, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (name: string, options: { output?: string }) => { + taskNames.push(name); + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: "draft" }; + }, + workflow: async () => ({ + outputs: { + result: "implementation done", + approved: true, + review_report: sufficientReviewReport(), + }, + }), + }); + + expect(result.status).toBe("complete"); + expect(taskNames).not.toContain("capture-learning"); + expect(result).not.toHaveProperty("learning_doc_path"); + expect(JSON.stringify(result)).not.toContain(":undefined"); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("handoff-only does not launch child workflow and omits undefined command", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-handoff-")); + const workflow = await workflowPromise; + let workflowCalls = 0; + try { + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt: "Implement approved thing", + mode: "work", + runner: "handoff-only", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: true, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (_name: string, options: { output?: string }) => { + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: "draft" }; + }, + workflow: async () => { + workflowCalls += 1; + return {}; + }, + }); + + expect(result.status).toBe("handoff_ready"); + expect(workflowCalls).toBe(0); + expect((result.implementation as Record).child_workflow_launched).toBe(false); + expect(result.implementation).not.toHaveProperty("command"); + expect(JSON.stringify(result)).not.toContain(":undefined"); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("workflow records suffixed artifact paths when visible outputs collide", async () => { + const dir = await mkdtemp(join(tmpdir(), "compound-engineering-collisions-")); + const workflow = await workflowPromise; + const prompt = "Implement approved thing"; + const now = new Date(); + const desiredPlan = join(dir, datedMarkdownPath("docs/plans", prompt, "plan", now)); + const desiredSpec = join(dir, datedMarkdownPath("specs", prompt, "spec", now)); + const desiredFinalReport = join(dir, datedMarkdownPath("compound-engineering", prompt, "final-report", now)); + try { + await mkdir(join(dir, "docs/plans"), { recursive: true }); + await mkdir(join(dir, "specs"), { recursive: true }); + await mkdir(join(dir, "compound-engineering"), { recursive: true }); + await writeFile(desiredPlan, "existing plan\n", "utf8"); + await writeFile(desiredSpec, "existing spec\n", "utf8"); + await writeFile(desiredFinalReport, "existing final report\n", "utf8"); + + const result = await workflow.run({ + cwd: dir, + inputs: { + prompt, + mode: "work", + runner: "handoff-only", + max_loops: 2, + base_branch: "origin/main", + git_worktree_dir: "", + create_pr: true, + learning_mode: "off", + memory_scope: "none", + }, + ui: { select: async () => "Approve", input: async () => "" }, + task: async (_name: string, options: { output?: string }) => { + if (options.output) await writeMarkdown(options.output, "task output"); + return { text: "draft" }; + }, + workflow: async () => ({}), + }); + + const actualPlan = desiredPlan.replace(/\.md$/, "-2.md"); + const actualSpec = desiredSpec.replace(/\.md$/, "-2.md"); + const actualFinalReport = desiredFinalReport.replace(/\.md$/, "-2.md"); + expect(result.plan_path).toBe(displayPath(actualPlan)); + expect(result.spec_path).toBe(displayPath(actualSpec)); + expect(result.approved_spec_path).toBe(displayPath(actualSpec)); + + const manifest = JSON.parse(await readFile(result.manifest_path as string, "utf8")); + expect(manifest.finalReportPath).toBe(displayPath(actualFinalReport)); + expect(manifest.artifacts.plan).toBe(displayPath(actualPlan)); + expect(manifest.artifacts.spec).toBe(displayPath(actualSpec)); + expect(manifest.artifacts["final-report"]).toBe(displayPath(actualFinalReport)); + + const finalReport = await readFile(actualFinalReport, "utf8"); + expect(finalReport).toContain(`- final-report: ${displayPath(actualFinalReport)}`); + expect(finalReport).toContain(`- plan: ${displayPath(actualPlan)}`); + expect(finalReport).toContain(`- spec: ${displayPath(actualSpec)}`); + expect(finalReport).not.toContain(`- plan: ${displayPath(desiredPlan)}\n`); + expect(finalReport).not.toContain(`- spec: ${displayPath(desiredSpec)}\n`); + } finally { + await rm(dir, { recursive: true, force: true }); + } + }); + + test("compiled workflow declares required inputs, defaults, and outputs", async () => { + const workflow = await workflowPromise; + expect(workflow.name).toBe("compound-engineering"); + expect(Object.keys(workflow.inputs)).toEqual([ + "prompt", + "mode", + "runner", + "max_loops", + "base_branch", + "git_worktree_dir", + "create_pr", + "learning_mode", + "memory_scope", + ]); + expect(workflow.inputs.mode.default).toBe("auto"); + expect(workflow.inputs.runner.default).toBe("auto"); + expect(workflow.inputs.max_loops.default).toBe(5); + expect(workflow.inputs.base_branch.default).toBe("origin/main"); + expect(workflow.inputs.git_worktree_dir.default).toBe(""); + expect(workflow.inputs.create_pr.default).toBe(false); + expect(workflow.inputs.learning_mode.default).toBe("ask"); + expect(workflow.inputs.memory_scope.default).toBe("repo"); + expect(workflow.worktree).toEqual({ gitWorktreeDir: "git_worktree_dir", baseBranch: "base_branch" }); + expect(Object.keys(workflow.outputs)).toEqual([ + "status", + "mode", + "runner", + "approved", + "artifact_dir", + "manifest_path", + "message", + "brainstorm_path", + "plan_path", + "spec_path", + "approved_spec_path", + "implementation", + "review_report_path", + "learning_doc_path", + ]); + expect(typeof workflow.run).toBe("function"); + }); +}); diff --git a/workflows/compound-engineering/helpers.ts b/workflows/compound-engineering/helpers.ts new file mode 100644 index 0000000..b0cd2af --- /dev/null +++ b/workflows/compound-engineering/helpers.ts @@ -0,0 +1,514 @@ +import { execFile } from "node:child_process"; +import { access, mkdir, readFile, writeFile } from "node:fs/promises"; +import { dirname, extname, isAbsolute, join, resolve } from "node:path"; +import { promisify } from "node:util"; + +const execFileAsync = promisify(execFile); + +export const COMPOUND_MODES = ["auto", "brainstorm", "plan", "work", "review", "compound-only"] as const; +export type CompoundMode = (typeof COMPOUND_MODES)[number]; +export type ResolvedCompoundMode = Exclude; + +export const IMPLEMENTATION_RUNNERS = ["auto", "goal", "ralph", "handoff-only"] as const; +export type ImplementationRunner = (typeof IMPLEMENTATION_RUNNERS)[number]; +export type ResolvedImplementationRunner = Exclude; + +export const LEARNING_MODES = ["ask", "off", "lightweight", "full"] as const; +export type LearningMode = (typeof LEARNING_MODES)[number]; + +export const MEMORY_SCOPES = ["repo", "none"] as const; +export type MemoryScope = (typeof MEMORY_SCOPES)[number]; + +export const WORKFLOW_STATUSES = [ + "complete", + "approved", + "handoff_ready", + "review_only", + "blocked", + "needs_human", + "rejected", + "stopped", +] as const; +export type WorkflowStatus = (typeof WORKFLOW_STATUSES)[number]; + +export const REVIEW_CRITERIA = [ + "independent", + "acceptance_mapped", + "diff_aware", + "validation_backed", + "risk_aware", + "fresh", +] as const; +export type ReviewCriterion = (typeof REVIEW_CRITERIA)[number]; + +export type ReviewGateDecision = "sufficient" | "targeted_review" | "full_review" | "fixes_needed" | "needs_human" | "blocked"; + +export type SeverityCounts = { + p0?: number; + p1?: number; + p2?: number; + p3?: number; +}; + +export type ReviewEvidence = { + independent?: boolean; + acceptance_mapped?: boolean; + diff_aware?: boolean; + validation_backed?: boolean; + risk_aware?: boolean; + fresh?: boolean; + severity_counts?: SeverityCounts; + conflicted?: boolean; + validation_failed?: boolean; + blocked?: boolean; +}; + +export type ReviewEvidenceReduction = { + decision: ReviewGateDecision; + missing: ReviewCriterion[]; + severity_counts: Required; + reason: string; +}; + +export type ChildHandoff = { + workflow: "goal" | "ralph" | "handoff-only"; + inputs: Record; + command?: string; + safe_note: string; +}; + +export type SavedStageReport = { + path: string; + body: string; + source: "saved-file" | "inline-fallback"; +}; + +export function text(value: unknown, fallback = ""): string { + const result = String(value ?? fallback).trim(); + return result.length > 0 ? result : fallback; +} + +function unknownToText(value: unknown): string { + if (typeof value === "string") return value; + if (value === undefined || value === null) return ""; + if (typeof value === "object" && "text" in value) { + const textValue = (value as { text?: unknown }).text; + if (typeof textValue === "string") return textValue; + } + try { + return JSON.stringify(value); + } catch { + return String(value); + } +} + +function isFileOnlySavedOutputReference(value: string): boolean { + const firstSubstantiveLine = value + .split(/\r?\n/) + .map((line) => line.trim()) + .find((line) => line.length > 0) ?? ""; + + return /^(?:output saved to|saved output to|saved to):\s*\S+/i.test(firstSubstantiveLine); +} + +export async function loadSavedStageReport(path: string, stageResult: unknown): Promise { + try { + const body = await readFile(path, "utf8"); + if (body.trim().length === 0) throw new Error(`Saved stage report is empty: ${path}`); + return { path, body, source: "saved-file" }; + } catch (error) { + if (!isFileNotFoundError(error)) { + const message = error instanceof Error ? error.message : String(error); + throw new Error(`Saved stage report is unreadable: ${path}: ${message}`); + } + // Controlled fallback below keeps tests/mocks usable while still rejecting + // compact file-only references as evidence. + } + + const inlineBody = unknownToText(stageResult); + if (inlineBody.trim().length > 0 && !isFileOnlySavedOutputReference(inlineBody)) { + return { path, body: inlineBody, source: "inline-fallback" }; + } + + throw new Error(`Saved stage report is missing, empty, or only a compact output reference/file-only pointer: ${path}`); +} + +async function git(args: string[], cwd: string): Promise { + const { stdout } = await execFileAsync("git", args, { cwd }); + return text(stdout); +} + +async function resolveGitTopLevel(cwd: string): Promise { + try { + const gitRoot = await git(["rev-parse", "--show-toplevel"], cwd); + if (gitRoot.length === 0) throw new Error("git returned an empty top-level path"); + return isAbsolute(gitRoot) ? gitRoot : resolve(cwd, gitRoot); + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + throw new Error( + `compound-engineering could not derive the effective worktree root from ctx.cwd (${cwd}). ` + + `When git_worktree_dir is non-empty, ctx.cwd must be inside the parent-bound Git worktree. git rev-parse --show-toplevel failed: ${message}`, + ); + } +} + +export async function resolveEffectiveWorktreeRoot(requestedGitWorktreeDir: string, effectiveWorkflowCwd: string): Promise { + if (text(requestedGitWorktreeDir).length === 0) return ""; + return resolveGitTopLevel(effectiveWorkflowCwd); +} + +export function positiveInteger(value: unknown, fallback: number): number { + return typeof value === "number" && Number.isFinite(value) && value > 0 + ? Math.floor(value) + : fallback; +} + +export function today(now = new Date()): string { + return now.toISOString().slice(0, 10); +} + +export function isCompoundMode(value: unknown): value is CompoundMode { + return COMPOUND_MODES.includes(value as CompoundMode); +} + +export function isImplementationRunner(value: unknown): value is ImplementationRunner { + return IMPLEMENTATION_RUNNERS.includes(value as ImplementationRunner); +} + +export function isLearningMode(value: unknown): value is LearningMode { + return LEARNING_MODES.includes(value as LearningMode); +} + +export function isMemoryScope(value: unknown): value is MemoryScope { + return MEMORY_SCOPES.includes(value as MemoryScope); +} + +function promptHasReviewKeyword(value: string): boolean { + return /\b(review|audit|gate|security check|code review)\b/i.test(value); +} + +function promptLooksLikePullRequestReference(value: string): boolean { + return /^#?\d+$/.test(value) + || /^https?:\/\/\S+\/pull\/\d+\b/i.test(value) + || /^pr[:#\s]+\d+$/i.test(value); +} + +function promptLooksLikeBranchRange(value: string): boolean { + return /^[\w./-]+\.\.[\w./-]+$/.test(value); +} + +export function promptLooksLikeReviewTarget(prompt: string): boolean { + const value = prompt.trim(); + return promptHasReviewKeyword(value) + || promptLooksLikePullRequestReference(value) + || promptLooksLikeBranchRange(value); +} + +export function promptLooksLikeLearningCapture(prompt: string): boolean { + return /\b(learned|learning|lessons?|solution note|capture knowledge|update concepts|docs\/solutions|postmortem)\b/i.test(prompt); +} + +export function promptLooksLikePlanOrSpecPath(prompt: string): "spec" | "plan" | "brainstorm" | undefined { + const value = prompt.trim().replace(/^['"]|['"]$/g, ""); + if (/\s/.test(value) || value.length === 0) return undefined; + if (/^(?:\.\/)?specs\/[^\0]+\.md$/i.test(value)) return "spec"; + if (/^(?:\.\/)?docs\/plans\/[^\0]+\.md$/i.test(value)) return "plan"; + if (/^(?:\.\/)?docs\/brainstorms\/[^\0]+\.md$/i.test(value)) return "brainstorm"; + return undefined; +} + +export function promptLooksVagueOrProductShaped(prompt: string): boolean { + const words = prompt.split(/\s+/).filter(Boolean); + const productSignal = /\b(improve|better|onboarding|activation|experience|flow|idea|explore|maybe|help users|make it easier|product|strategy|roadmap)\b/i.test(prompt); + const concreteSignal = /\b(api|endpoint|database|schema|migration|test|bug|error|auth|cli|config|typescript|react|sql|cache|worker|specs?\/)\b/i.test(prompt); + return words.length < 8 || (productSignal && !concreteSignal); +} + +export function resolveMode(promptInput: string, requestedMode: CompoundMode = "auto"): ResolvedCompoundMode { + if (requestedMode !== "auto") return requestedMode; + + const prompt = text(promptInput); + const pathKind = promptLooksLikePlanOrSpecPath(prompt); + if (pathKind === "spec" || pathKind === "plan") return "work"; + if (pathKind === "brainstorm") return "plan"; + if (promptLooksLikeReviewTarget(prompt)) return "review"; + if (promptLooksLikeLearningCapture(prompt)) return "compound-only"; + if (promptLooksVagueOrProductShaped(prompt)) return "brainstorm"; + return "work"; +} + +export function resolveRunner(requestedRunner: ImplementationRunner = "auto"): ResolvedImplementationRunner { + if (requestedRunner !== "auto") return requestedRunner; + + // Iteration 3 keeps automatic runner selection safe-by-default: automatic + // selection never launches a code-changing child workflow. Only explicit + // goal/ralph requests may run implementation after human approval. + return "handoff-only"; +} + +export type ApprovalDecision = "approved" | "rejected" | "revise" | "stopped"; + +export function parseApprovalDecision(input: unknown): ApprovalDecision { + const value = text(input).toLowerCase(); + if (/\b(stop|cancel|abort|quit)\b/.test(value)) return "stopped"; + if (/\b(reject|rejected|no|decline|do not proceed)\b/.test(value)) return "rejected"; + if (/\b(revise|revision|change|adjust|edit|needs work)\b/.test(value)) return "revise"; + if (/\b(approve|approved|yes|proceed|ship|looks good|lgtm)\b/.test(value)) return "approved"; + return "revise"; +} + +function countSeverity(value: unknown): number { + return typeof value === "number" && Number.isFinite(value) && value > 0 ? Math.floor(value) : 0; +} + +export function normalizeSeverityCounts(counts: SeverityCounts = {}): Required { + return { + p0: countSeverity(counts.p0), + p1: countSeverity(counts.p1), + p2: countSeverity(counts.p2), + p3: countSeverity(counts.p3), + }; +} + +export function missingReviewCriteria(evidence: ReviewEvidence): ReviewCriterion[] { + return REVIEW_CRITERIA.filter((criterion) => evidence[criterion] !== true); +} + +export function reduceReviewEvidence(evidence: ReviewEvidence): ReviewEvidenceReduction { + const severity_counts = normalizeSeverityCounts(evidence.severity_counts); + const missing = missingReviewCriteria(evidence); + + if (evidence.blocked === true) { + return { decision: "blocked", missing, severity_counts, reason: "Review or validation is blocked by missing dependencies or unavailable evidence." }; + } + if (evidence.conflicted === true) { + return { decision: "needs_human", missing, severity_counts, reason: "Review evidence is conflicted and needs a human decision." }; + } + if (severity_counts.p0 > 0 || severity_counts.p1 > 0) { + return { decision: "fixes_needed", missing, severity_counts, reason: "Blocking P0/P1 findings remain." }; + } + if (evidence.validation_failed === true) { + return { decision: "needs_human", missing, severity_counts, reason: "Validation failed after implementation or fixes." }; + } + if (missing.length === 0) { + return { decision: "sufficient", missing, severity_counts, reason: "Review evidence satisfies all sufficiency criteria." }; + } + if (missing.includes("fresh") || missing.length > 1) { + return { decision: "full_review", missing, severity_counts, reason: "Evidence is stale or missing multiple required review dimensions." }; + } + return { decision: "targeted_review", missing, severity_counts, reason: `Evidence is missing ${missing[0]}.` }; +} + +export function slugifyTopic(value: unknown, fallback = "compound-engineering"): string { + const slug = text(value, fallback) + .normalize("NFKD") + .replace(/[\u0300-\u036f]/g, "") + .toLowerCase() + .replace(/[^a-z0-9]+/g, "-") + .replace(/^-+|-+$/g, "") + .split("-") + .filter(Boolean) + .slice(0, 8) + .join("-"); + return slug.length > 0 ? slug : fallback; +} + +export function datedMarkdownPath(directory: string, topic: unknown, fallback: string, now = new Date()): string { + return join(directory, `${today(now)}-${slugifyTopic(topic, fallback)}.md`); +} + +export function normalizeCreatePr(value: unknown): boolean { + return value === true; +} + +function quoteWorkflowValue(value: unknown): string { + return `"${String(value ?? "").replace(/\\/g, "\\\\").replace(/"/g, "\\\"")}"`; +} + +function buildChildWorkflowCommand(options: { + runner: Exclude; + prompt: string; + maxLoops: number; + baseBranch: string; + gitWorktreeDir: string; + createPr: boolean; +}): string { + if (options.runner === "goal") { + return [ + "/workflow", + "goal", + `objective=${quoteWorkflowValue(options.prompt)}`, + `max_turns=${options.maxLoops}`, + `base_branch=${quoteWorkflowValue(options.baseBranch)}`, + ].join(" "); + } + + return [ + "/workflow", + "ralph", + `prompt=${quoteWorkflowValue(options.prompt)}`, + `max_loops=${options.maxLoops}`, + `base_branch=${quoteWorkflowValue(options.baseBranch)}`, + `git_worktree_dir=${quoteWorkflowValue(options.gitWorktreeDir)}`, + `create_pr=${options.createPr ? "true" : "false"}`, + ].join(" "); +} + +export function buildChildHandoff(options: { + runner: ResolvedImplementationRunner; + approvedPath: string; + prompt: string; + maxLoops: number; + baseBranch: string; + gitWorktreeDir: string; + createPr: boolean; +}): ChildHandoff { + const safe_note = "Iteration 3 launches explicit goal/ralph runners only after human approval; auto and handoff-only remain non-mutating."; + if (options.runner === "handoff-only") { + return { + workflow: "handoff-only", + inputs: { + approved_path: options.approvedPath, + prompt: options.prompt, + max_loops: options.maxLoops, + base_branch: options.baseBranch, + git_worktree_dir: options.gitWorktreeDir, + create_pr: false, + }, + safe_note, + }; + } + + const childPrompt = [ + `Implement the approved Compound Engineering plan/spec at ${options.approvedPath}. Original request: ${options.prompt}`, + "", + "After implementation and your own internal review, emit a structured evidence block named `compound_engineering_evidence` in child outputs and/or the `review_report` artifact.", + "The block must reflect the child workflow's own implementation and internal review results; the parent will not fabricate missing evidence or run a duplicative supplemental review.", + "Use exactly this compact JSON-serializable shape:", + "{", + " independent: { satisfied: boolean, evidence?: string, source?: string },", + " acceptance_mapped: { satisfied: boolean, evidence?: string },", + " diff_aware: { satisfied: boolean, evidence?: string },", + " validation_backed: { satisfied: boolean, evidence?: string, commands?: [{ command?: string, exit_code?: number, summary?: string }] },", + " risk_aware: { satisfied: boolean, evidence?: string },", + " fresh: { satisfied: boolean, evidence?: string },", + " severity_counts: { p0: number, p1: number, p2: number, p3: number },", + " blocked?: boolean,", + " conflicted?: boolean,", + " validation_failed?: boolean", + "}", + "Every satisfied criterion must include explicit evidence text or, for validation, passing command summaries/zero exit codes. Set blocked/conflicted/validation_failed truthfully when applicable.", + ].join("\n"); + const createPr = normalizeCreatePr(options.createPr); + const inputs: Record = options.runner === "goal" + ? { + objective: childPrompt, + max_turns: options.maxLoops, + base_branch: options.baseBranch, + } + : { + prompt: childPrompt, + max_loops: options.maxLoops, + base_branch: options.baseBranch, + git_worktree_dir: options.gitWorktreeDir, + create_pr: createPr, + }; + + return { + workflow: options.runner, + inputs, + command: buildChildWorkflowCommand({ + runner: options.runner, + prompt: childPrompt, + maxLoops: options.maxLoops, + baseBranch: options.baseBranch, + gitWorktreeDir: options.gitWorktreeDir, + createPr, + }), + safe_note, + }; +} + +function sanitizeRunId(value: string): string { + const sanitized = value + .trim() + .replace(/[^A-Za-z0-9._-]+/g, "-") + .replace(/^-+|-+$/g, ""); + return sanitized.length > 0 ? sanitized : "run"; +} + +export function timestampRunId(now = new Date()): string { + return sanitizeRunId(now.toISOString().replace(/[:.]/g, "-")); +} + +export function displayPath(path: string): string { + return path.replace(/\\/g, "/"); +} + +export async function createArtifactRun(workflowName: string, startedAt: Date, cwd = process.cwd()): Promise<{ runId: string; artifactDir: string }> { + const runId = timestampRunId(startedAt); + const artifactDir = join(cwd, `.${workflowName}-${runId}`); + await mkdir(artifactDir, { recursive: true }); + return { runId, artifactDir }; +} + +function isFileExistsError(error: unknown): boolean { + return typeof error === "object" && error !== null && "code" in error && error.code === "EEXIST"; +} + +function isFileNotFoundError(error: unknown): boolean { + return typeof error === "object" && error !== null && "code" in error && error.code === "ENOENT"; +} + +function suffixedPath(path: string, suffix: number): string { + const extension = extname(path); + if (extension.length === 0) return `${path}-${suffix}`; + return `${path.slice(0, -extension.length)}-${suffix}${extension}`; +} + +async function writeNewFile(path: string, content: string): Promise { + try { + await writeFile(path, content, { encoding: "utf8", flag: "wx" }); + return true; + } catch (error) { + if (isFileExistsError(error)) return false; + throw error; + } +} + +async function pathExists(path: string): Promise { + try { + await access(path); + return true; + } catch (error) { + if (isFileNotFoundError(error)) return false; + throw error; + } +} + +export async function nextAvailablePath(path: string): Promise { + await mkdir(dirname(path), { recursive: true }); + if (!await pathExists(path)) return path; + + for (let suffix = 2; ; suffix += 1) { + const candidate = suffixedPath(path, suffix); + if (!await pathExists(candidate)) return candidate; + } +} + +export async function writeMarkdown(path: string, content: string): Promise { + await mkdir(dirname(path), { recursive: true }); + const body = `${content.trimEnd()}\n`; + if (await writeNewFile(path, body)) return path; + + for (let suffix = 2; ; suffix += 1) { + const candidate = suffixedPath(path, suffix); + if (await writeNewFile(candidate, body)) return candidate; + } +} + +export async function writeJson(path: string, value: unknown): Promise { + await mkdir(dirname(path), { recursive: true }); + await writeFile(path, `${JSON.stringify(value, null, 2)}\n`, "utf8"); + return path; +} diff --git a/workflows/compound-engineering/index.ts b/workflows/compound-engineering/index.ts new file mode 100644 index 0000000..327de9f --- /dev/null +++ b/workflows/compound-engineering/index.ts @@ -0,0 +1,1517 @@ +import { readFile } from "node:fs/promises"; +import { join } from "node:path"; +import { defineWorkflow, Type } from "@bastani/workflows"; +import { goal, ralph } from "@bastani/workflows/builtin"; +import { + buildChildHandoff, + createArtifactRun, + datedMarkdownPath, + displayPath, + isCompoundMode, + isImplementationRunner, + isLearningMode, + isMemoryScope, + loadSavedStageReport, + nextAvailablePath, + normalizeCreatePr, + normalizeSeverityCounts, + parseApprovalDecision, + positiveInteger, + REVIEW_CRITERIA, + reduceReviewEvidence, + resolveEffectiveWorktreeRoot, + resolveMode, + resolveRunner, + text, + writeJson, + writeMarkdown, + WORKFLOW_STATUSES, + type CompoundMode, + type ImplementationRunner, + type LearningMode, + type MemoryScope, + type ResolvedCompoundMode, + type ResolvedImplementationRunner, + type ReviewCriterion, + type ReviewEvidence, + type SeverityCounts, + type WorkflowStatus, +} from "./helpers.js"; + +const WORKFLOW_NAME = "compound-engineering"; +const DEFAULT_MAX_LOOPS = 5; +const DEFAULT_BASE_BRANCH = "origin/main"; +const FILE_ONLY_OUTPUT = "file-only" as const; +const EVERY_INC_CREDIT = "Inspired by EveryInc's MIT-licensed Compound Engineering Plugin process vocabulary: brainstorm → plan → work → review → compound learning."; + +const CHILD_OUTPUT_KEYS = [ + "result", + "status", + "message", + "summary", + "approved", + "artifact_dir", + "manifest_path", + "plan_path", + "implementation_notes_path", + "ledger_path", + "pr_report", + "iterations_completed", + "review_report", + "review_report_path", + "validation_output", + "changed_files", + "compound_engineering_evidence", +] as const; +const CHILD_OUTPUT_KEY_SET = new Set(CHILD_OUTPUT_KEYS); + +const resolvedModeSchema = Type.Union([ + Type.Literal("brainstorm"), + Type.Literal("plan"), + Type.Literal("work"), + Type.Literal("review"), + Type.Literal("compound-only"), +]); + +const resolvedRunnerSchema = Type.Union([ + Type.Literal("goal"), + Type.Literal("ralph"), + Type.Literal("handoff-only"), +]); + +const statusSchema = Type.Union(WORKFLOW_STATUSES.map((status) => Type.Literal(status))); + +function fileOnlyOutput(output: string): { output: string; outputMode: typeof FILE_ONLY_OUTPUT } { + return { output, outputMode: FILE_ONLY_OUTPUT }; +} + +function artifactMarkdown(title: string, body: string): string { + return [`# ${title}`, "", body.trim(), "", `Attribution: ${EVERY_INC_CREDIT}`].join("\n"); +} + +function manifestArtifacts(paths: ReadonlyMap, manifestPath: string): Record { + const artifacts: Record = {}; + for (const [name, path] of paths) { + artifacts[name] = displayPath(path); + } + artifacts.manifest = displayPath(manifestPath); + return artifacts; +} + +type SufficiencyCriterion = ReviewCriterion; +type SeverityLevel = keyof Required; + +const SEVERITY_LEVELS: readonly SeverityLevel[] = ["p0", "p1", "p2", "p3"]; + +type CriterionPolicy = { + positive: readonly RegExp[]; + negative: readonly RegExp[]; +}; + +const VALIDATION_NO_RUN_PATTERN = /\b(?:validation\s+commands?|tests?|test\s+commands?|commands?)\s+(?:run|ran)\s*:?\s*(?:none|no\s+commands?|zero|0|n\/a)\b|\bno\s+validation\s+(?:(?:was|were)\s+)?(?:run|ran|performed)\b|\bno\s+validation\s+commands?\s+(?:(?:was|were)\s+)?run\b|\bno\s+tests?\s+(?:(?:was|were)\s+)?run\b|\bno\s+commands?\s+(?:(?:was|were)\s+)?run\b|\bvalidation\s+(?:commands?\s+)?skipped\b|\bvalidation\b.{0,50}\bskipped\b|\bvalidation\s+not\s+(?:run|performed|backed|available)\b|\bvalidation\b.{0,50}\b(?:was|were)\s+not\s+(?:run|performed|available)\b|\btests?\b.{0,40}\bskipped\b|\btests?\s+(?:were\s+)?not\s+run\b/; +const CONTEXTUAL_VALIDATION_ERROR_PATTERN = /\b(?:reported\s+)?[1-9]\d*\s+errors\b|\berror\s+count\s*:?\s*[1-9]\d*\b|\berrors\s*:?\s*[1-9]\d*\b|\bfailed\s+with\s+errors\b/; +const VALIDATION_NEGATED_SUCCESS_PATTERN = /\b(?:not|never)\s+(?:(?:all|any|every)\s+(?:[\w-]+\s+){0,6})?(?:pass(?:ed|es|ing)?|successful|success|succeed(?:ed|s|ing)?)\b|\bunsuccessful\b/; +const VALIDATION_SUCCESS_PATTERN = /\b(?:passed|passes|passing|succeeded|successful|success|status\s*:?\s*0|code\s*:?\s*0|exit\s+(?:code|status)\s*:?\s*0|exited\s+with\s+(?:exit\s+)?(?:code|status)\s*:?\s*0|returned\s+(?:exit\s+)?(?:code|status)\s*:?\s*0)\b/; + +const REVIEW_EVIDENCE_POLICIES = { + independent: { + positive: [/\bindependent(?:ly)?\b/, /\bseparate reviewer\b/, /\bfresh[-\s]context reviewer\b/], + negative: [ + /\bnot\s+independent\b/, + /\bno\s+(?:independent|separate|fresh[-\s]context)\s+review(?:er)?\b/, + /\bwithout\s+(?:an?\s+)?(?:independent|separate|fresh[-\s]context)\s+review(?:er)?\b/, + /\bself[-\s]review\b/, + ], + }, + acceptance_mapped: { + positive: [ + /\b(?:map(?:ped)?|mapping|check(?:ed)?|verify|verified|trace(?:d)?|cover(?:ed)?|coverage)\b.{0,80}\b(?:acceptance(?:\s+criteria)?|approved\s+(?:plan|spec))\b/, + /\b(?:acceptance(?:\s+criteria)?|approved\s+(?:plan|spec))\b.{0,80}\b(?:map(?:ped)?|mapping|check(?:ed)?|verify|verified|trace(?:d)?|cover(?:ed)?|coverage)\b/, + ], + negative: [ + /\bno\s+acceptance(?:\s+criteria)?\s+(?:mapping|mapped|traceability|coverage)\b/, + /\bno\s+acceptance(?:\s+criteria)?\s+(?:(?:was|were)\s+)?(?:mapped|checked|verified|traced|covered|inspected)\b/, + /\bnot\s+acceptance[-\s]mapped\b/, + /\bacceptance\s+(?:mapping\s+)?(?:not\s+)?(?:missing|skipped|absent)\b/, + /\b(?:acceptance(?:\s+criteria|\s+mapping)?|approved\s+(?:plan|spec)|spec)\b.{0,80}\b(?:not\s+(?:mapped|provided|performed|done|checked|verified|traced|covered|inspected)|(?:was|were)\s+not\s+(?:mapped|provided|performed|done|checked|verified|traced|covered|inspected)|unmapped|unchecked|unverified|untraced|uncovered|skipped|missing|absent|unavailable)\b/, + /\b(?:did not|does not|not)\s+(?:map|mapped|check|checked|verify|verified|trace|traced|cover|covered|inspect|inspected)\b.{0,80}\b(?:acceptance|approved\s+(?:plan|spec)|spec)\b/, + /\b(?:unchecked|unmapped|unverified|untraced|uncovered)\s+(?:acceptance(?:\s+criteria)?|approved\s+)?(?:spec|plan|criteria)\b/, + ], + }, + diff_aware: { + positive: [/\bdiff\b/, /\bchanged files?\b/, /\bgit status\b/, /\bgit diff\b/], + negative: [ + /\bdiff\s+(?:was\s+|were\s+)?not\s+(?:inspected|reviewed|checked)\b/, + /\bnot\s+(?:inspect(?:ed)?|review(?:ed)?|check(?:ed)?)\b.{0,40}\bdiff\b/, + /\b(?:did not|does not)\s+(?:inspect|review|check)\b.{0,40}\bdiff\b/, + /\bdiff(?:\s+review)?\b.{0,40}\b(?:skipped|missing|absent|not\s+(?:inspected|reviewed|checked|performed)|(?:was|were)\s+not\s+(?:inspected|reviewed|checked|performed))\b/, + /\b(?:unable|could not|cannot)\s+(?:to\s+)?(?:inspect|review|check)\b.{0,60}\b(?:current\s+)?(?:git\s+)?diff\b/, + /\bcould not\s+inspect\s+changed files?\b/, + /\bdiff\s+(?:was\s+)?unavailable\b/, + /\bdiff\s+(?:could\s+not|cannot)\s+be\s+(?:inspected|reviewed|checked)\b/, + /\bdiff\s+was\s+not\s+available\b/, + /\bno\s+(?:git\s+)?diff\s+(?:was\s+)?available\b/, + /\bchanged files?\s+(?:was\s+|were\s+)?(?:unavailable|not\s+(?:inspected|reviewed|checked|available))\b/, + /\bno\s+(?:git\s+)?diff\b.{0,80}\b(?:inspected|reviewed|checked)\b/, + /\bno\s+changed files?\b.{0,80}\b(?:inspected|reviewed|checked)\b/, + ], + }, + validation_backed: { + positive: [ + /\b(?:validation|tests?|commands?|bun\s+test|cargo\s+test|npm\s+test|pnpm\s+test|yarn\s+test|test\s+command)\b.{0,80}\b(?:passed|passes|passing|succeeded|successful|success|status\s*:?\s*0|code\s*:?\s*0|exit\s+(?:code|status)\s*:?\s*0|exited\s+with\s+(?:exit\s+)?(?:code|status)\s*:?\s*0|returned\s+(?:exit\s+)?(?:code|status)\s*:?\s*0)\b/, + /\b(?:passed|passes|passing|succeeded|successful|success|status\s*:?\s*0|code\s*:?\s*0|exit\s+(?:code|status)\s*:?\s*0|exited\s+with\s+(?:exit\s+)?(?:code|status)\s*:?\s*0|returned\s+(?:exit\s+)?(?:code|status)\s*:?\s*0)\b.{0,80}\b(?:validation|tests?|commands?|bun\s+test|cargo\s+test|npm\s+test|pnpm\s+test|yarn\s+test|test\s+command)\b/, + ], + negative: [ + VALIDATION_NO_RUN_PATTERN, + /\b(?:validation|tests?|commands?|bun\s+test|cargo\s+test)\b.{0,80}\b(?:failed|failures?|failing|errored|did\s+not\s+pass|does\s+not\s+pass|not\s+passing|non[-\s]?zero|(?:exit\s+)?(?:code|status)\s*:?\s*[1-9]\d*)\b/, + /\b(?:failed|failures?|failing|errored|did\s+not\s+pass|does\s+not\s+pass|not\s+passing|non[-\s]?zero|(?:exit\s+)?(?:code|status)\s*:?\s*[1-9]\d*)\b.{0,80}\b(?:validation|tests?|commands?|bun\s+test|cargo\s+test)\b/, + /\b(?:validation|tests?|commands?|bun\s+test|cargo\s+test)\b.{0,80}\b(?:reported\s+)?[1-9]\d*\s+errors\b/, + /\b(?:reported\s+)?[1-9]\d*\s+errors\b.{0,80}\b(?:validation|tests?|commands?|bun\s+test|cargo\s+test)\b/, + /\b(?:validation|tests?|commands?|bun\s+test|cargo\s+test)\b.{0,80}\b(?:error\s+count\s*:?\s*[1-9]\d*|errors\s*:?\s*[1-9]\d*|failed\s+with\s+errors)\b/, + /\b(?:error\s+count\s*:?\s*[1-9]\d*|errors\s*:?\s*[1-9]\d*|failed\s+with\s+errors)\b.{0,80}\b(?:validation|tests?|commands?|bun\s+test|cargo\s+test)\b/, + ], + }, + risk_aware: { + positive: [/\brisk\b/, /\bresidual\b/, /\bsecurity\b/], + negative: [ + /\bno\s+residual\s+risk\s+(?:assessment|review|analysis)\b/, + /\bno\s+risk\s+(?:assessment|review|analysis)\b/, + /\brisk\s+(?:assessment|review|analysis)\s+(?:not\s+)?(?:missing|skipped|absent)\b/, + /\brisk\b.{0,50}\bunknown\b/, + /\brisk\b.{0,50}\b(?:not\s+(?:assessed|reviewed|evaluated)|(?:was|were)\s+not\s+(?:assessed|reviewed|evaluated|performed))\b/, + /\bnot\s+(?:risk|residual\s+risk)[-\s]aware\b/, + ], + }, + fresh: { + positive: [ + /\b(?:review(?:ed)?|fresh|evidence)\b.{0,60}\bafter\s+(?:the\s+)?(?:final\s+diff|latest\s+change|current\s+diff)\b/, + /\bafter\s+(?:the\s+)?(?:final\s+diff|latest\s+change|current\s+diff)\b.{0,60}\b(?:review(?:ed)?|fresh|evidence)\b/, + ], + negative: [ + /\bnot\s+fresh\b/, + /\bstale\b/, + /\b(?:review(?:ed)?|evidence)\b.{0,60}\b(?:predates|before)\b.{0,40}\b(?:final\s+diff|latest\s+change|current\s+diff)\b/, + /\b(?:predates|before)\s+(?:the\s+)?(?:final\s+diff|latest\s+change|current\s+diff)\b/, + /\bnot\s+(?:after\s+)?(?:the\s+)?latest\s+change\b/, + /\bcurrent diff\b.{0,60}\bnot\s+(?:inspected|reviewed|checked)\b/, + ], + }, +} as const satisfies Record; + +function reviewSegments(value: string): string[] { + return value + .toLowerCase() + .split(/[\r\n.;]+/) + .map((segment) => segment.trim()) + .filter((segment) => segment.length > 0); +} + +function matchesAnyPattern(value: string, patterns: readonly RegExp[]): boolean { + return patterns.some((pattern) => pattern.test(value)); +} + +function criterionEvidenceState(segments: readonly string[], policy: CriterionPolicy): boolean | undefined { + if (segments.some((segment) => matchesAnyPattern(segment, policy.negative))) return false; + if (segments.some((segment) => matchesAnyPattern(segment, policy.positive))) return true; + return undefined; +} + +const VALIDATION_SUBJECT_PATTERN = /\b(?:validation|tests?|commands?|bun\s+test|cargo\s+test|npm\s+test|pnpm\s+test|yarn\s+test|test\s+command)\b/; +const VALIDATION_RUN_CONTEXT_PATTERN = /\b(?:run|ran|completed|bun\s+test|cargo\s+test|npm\s+test|pnpm\s+test|yarn\s+test|test\s+command)\b/; +const NO_VALIDATION_FAILURE_PATTERN = /\b(?:no|zero|0)\s+(?:validation\s+)?(?:commands?\s+)?(?:failed|failures?|errors?)\b|\b(?:validation\s+)?(?:failures?|errors?)\s*:\s*(?:none|no|zero|0)\b|\b(?:no|zero|0)\s+tests?\s+(?:failed|failing)\b|\bwithout\s+(?:validation\s+|test\s+)?(?:failures?|errors?)\b/g; +const VALIDATION_FAILURE_PATTERN = /\b(?:failed|failures?|failing|errored|did\s+not\s+pass|does\s+not\s+pass|not\s+passing|non[-\s]?zero|(?:exit\s+)?(?:code|status)\s*:?\s*[1-9]\d*|exited\s+with\s+(?:exit\s+)?(?:code|status)\s*:?\s*[1-9]\d*|returned\s+(?:exit\s+)?(?:code|status)\s*:?\s*[1-9]\d*)\b|\b(?:reported\s+)?[1-9]\d*\s+errors\b|\berror\s+count\s*:?\s*[1-9]\d*\b|\berrors\s*:?\s*[1-9]\d*\b|\bfailed\s+with\s+errors\b/; + +function removeNoValidationFailureAssertions(segment: string): string { + return segment.replace(NO_VALIDATION_FAILURE_PATTERN, " ").trim(); +} + +function hasValidationRunContext(segment: string): boolean { + return VALIDATION_SUBJECT_PATTERN.test(segment) && VALIDATION_RUN_CONTEXT_PATTERN.test(segment) && !VALIDATION_NO_RUN_PATTERN.test(segment); +} + +function validationConditionHasContext(segment: string, previousSegment: string): boolean { + return VALIDATION_SUBJECT_PATTERN.test(segment) || (previousSegment.length > 0 && hasValidationRunContext(previousSegment)); +} + +function hasValidationFailure(segments: readonly string[]): boolean { + for (let index = 0; index < segments.length; index += 1) { + const scrubbed = removeNoValidationFailureAssertions(segments[index]); + if (scrubbed.length === 0) continue; + + const hasFailure = VALIDATION_FAILURE_PATTERN.test(scrubbed) || CONTEXTUAL_VALIDATION_ERROR_PATTERN.test(scrubbed); + const hasNegatedSuccess = VALIDATION_NEGATED_SUCCESS_PATTERN.test(scrubbed); + if (!hasFailure && !hasNegatedSuccess) continue; + + const previous = index > 0 ? removeNoValidationFailureAssertions(segments[index - 1]) : ""; + if (validationConditionHasContext(scrubbed, previous)) return true; + } + return false; +} + +function validationEvidenceState(segments: readonly string[]): boolean | undefined { + const scrubbedSegments = segments + .map(removeNoValidationFailureAssertions) + .filter((segment) => segment.length > 0); + const validationPolicy = REVIEW_EVIDENCE_POLICIES.validation_backed; + + for (let index = 0; index < scrubbedSegments.length; index += 1) { + const segment = scrubbedSegments[index]; + if (!VALIDATION_NEGATED_SUCCESS_PATTERN.test(segment)) continue; + + const previous = index > 0 ? scrubbedSegments[index - 1] : ""; + if (validationConditionHasContext(segment, previous)) return false; + } + + if (scrubbedSegments.some((segment) => matchesAnyPattern(segment, validationPolicy.negative))) return false; + if (scrubbedSegments.some((segment) => matchesAnyPattern(segment, validationPolicy.positive))) return true; + + for (let index = 1; index < scrubbedSegments.length; index += 1) { + if (VALIDATION_SUCCESS_PATTERN.test(scrubbedSegments[index]) && hasValidationRunContext(scrubbedSegments[index - 1])) return true; + } + return undefined; +} + +function emptySeverityCounts(): Required { + return { p0: 0, p1: 0, p2: 0, p3: 0 }; +} + +function severityFindingLines(value: string, severity: SeverityLevel): number { + const findingPattern = new RegExp(`(?:^|\\s)(?:-\\s*)?(?:\\[${severity}\\]|${severity}\\s*(?::\\s*\\S|[-–—]\\s*\\S))`, "i"); + return value + .split(/\r?\n/) + .map((line) => line.trim()) + .filter((line) => { + if (line.length === 0) return false; + if (/^#{1,6}\s*(?:p[0-3])\b/i.test(line)) return false; + if (/^(?:-\s*)?(?:p[0-3]\s*:\s*)?(?:none(?:\s+(?:found|identified|reported))?|no\s+(?:findings?|issues?|blockers?)|n\/a|nothing)\.?$/i.test(line)) return false; + return findingPattern.test(line); + }) + .length; +} + +export function parseEvidenceFromText(value: string): ReviewEvidence { + const textValue = value.toLowerCase(); + const segments = reviewSegments(value); + const validationFailed = hasValidationFailure(segments); + const severity_counts = emptySeverityCounts(); + for (const severity of SEVERITY_LEVELS) { + severity_counts[severity] = severityFindingLines(value, severity); + } + + return { + independent: criterionEvidenceState(segments, REVIEW_EVIDENCE_POLICIES.independent), + acceptance_mapped: criterionEvidenceState(segments, REVIEW_EVIDENCE_POLICIES.acceptance_mapped), + diff_aware: criterionEvidenceState(segments, REVIEW_EVIDENCE_POLICIES.diff_aware), + validation_backed: validationFailed ? false : validationEvidenceState(segments), + risk_aware: criterionEvidenceState(segments, REVIEW_EVIDENCE_POLICIES.risk_aware), + fresh: criterionEvidenceState(segments, REVIEW_EVIDENCE_POLICIES.fresh), + severity_counts, + conflicted: /conflict(ed|ing)? evidence/.test(textValue), + validation_failed: validationFailed, + blocked: /blocked|unable to review|missing dependency/.test(textValue), + }; +} + +function childOutputs(result: unknown): Record { + if (typeof result !== "object" || result === null) return {}; + const record = result as Record; + const nested = record.outputs; + return typeof nested === "object" && nested !== null ? nested as Record : record; +} + +function omitUndefined(value: unknown, seen = new WeakSet()): unknown { + if (value === undefined || typeof value === "function" || typeof value === "symbol") return undefined; + if (typeof value === "bigint") return value.toString(); + if (typeof value === "number") return Number.isFinite(value) ? value : null; + if (Array.isArray(value)) { + if (seen.has(value)) return "[Circular]"; + seen.add(value); + return value.map((item) => omitUndefined(item, seen) ?? null); + } + if (typeof value !== "object" || value === null) return value; + if (seen.has(value)) return "[Circular]"; + seen.add(value); + + const output: Record = {}; + for (const [key, childValue] of Object.entries(value as Record)) { + const sanitized = omitUndefined(childValue, seen); + if (sanitized !== undefined) output[key] = sanitized; + } + return output; +} + +function outputRecord(value: Record): Record { + return omitUndefined(value) as Record; +} + +function textFromUnknown(value: unknown): string { + if (typeof value === "string") return value; + if (value === undefined || value === null) return ""; + try { + return JSON.stringify(value); + } catch { + return String(value); + } +} + +function normalizedChildOutput(result: unknown): Record { + const outputs = childOutputs(result); + const selected: Record = {}; + for (const key of CHILD_OUTPUT_KEYS) { + if (outputs[key] !== undefined) selected[key] = outputs[key]; + } + + const omittedChildOutputKeys = Object.keys(outputs) + .filter((key) => !CHILD_OUTPUT_KEY_SET.has(key)) + .sort(); + if (omittedChildOutputKeys.length > 0) selected.omitted_child_output_keys = omittedChildOutputKeys; + + return outputRecord(selected); +} + +const MAX_RECEIPT_STRING_CHARS = 2000; +const MAX_RECEIPT_COLLECTION_ITEMS = 50; +const MAX_RECEIPT_OBJECT_KEYS = 50; + +function compactTextSummary(value: string): Record { + return { + summary: value.slice(0, MAX_RECEIPT_STRING_CHARS), + original_chars: value.length, + omitted_chars: Math.max(0, value.length - MAX_RECEIPT_STRING_CHARS), + }; +} + +function compactReceiptValue(value: unknown, depth = 0, seen = new WeakSet()): unknown { + if (value === undefined || typeof value === "function" || typeof value === "symbol") return undefined; + if (typeof value === "bigint") return value.toString(); + if (typeof value === "number") return Number.isFinite(value) ? value : null; + if (typeof value === "string") { + return value.length > MAX_RECEIPT_STRING_CHARS ? compactTextSummary(value) : value; + } + if (Array.isArray(value)) { + if (seen.has(value)) return "[Circular]"; + seen.add(value); + const items = value.slice(0, MAX_RECEIPT_COLLECTION_ITEMS).map((item) => compactReceiptValue(item, depth + 1, seen) ?? null); + if (value.length <= MAX_RECEIPT_COLLECTION_ITEMS) return items; + + return { items, original_items: value.length, omitted_items: value.length - MAX_RECEIPT_COLLECTION_ITEMS }; + } + if (typeof value !== "object" || value === null) return value; + if (seen.has(value)) return "[Circular]"; + seen.add(value); + + if (depth >= 2) { + const textValue = textFromUnknown(value); + return textValue.length > MAX_RECEIPT_STRING_CHARS ? compactTextSummary(textValue) : textValue; + } + + const compacted: Record = {}; + const entries = Object.entries(value as Record); + const keptEntries = entries.slice(0, MAX_RECEIPT_OBJECT_KEYS); + for (const [key, childValue] of keptEntries) { + const compactedValue = compactReceiptValue(childValue, depth + 1, seen); + if (compactedValue !== undefined) compacted[key] = compactedValue; + } + + const omittedEntries = entries.slice(MAX_RECEIPT_OBJECT_KEYS); + if (omittedEntries.length > 0) { + const omittedKeys = omittedEntries.map(([key]) => key); + compacted.original_keys = entries.length; + compacted.omitted_key_count = omittedEntries.length; + compacted.omitted_keys = omittedKeys.slice(0, MAX_RECEIPT_OBJECT_KEYS); + } + + return compacted; +} + +function compactChildOutputForReceipt(childOutput: Record): Record { + const compacted: Record = {}; + for (const [key, value] of Object.entries(childOutput)) { + compacted[key] = compactReceiptValue(value); + } + return outputRecord(compacted); +} + +type ChildRunGate = + | { state: "approved"; parent_status?: undefined; reason: string; approved: true; status?: string } + | { state: "non_approved"; parent_status: "needs_human"; reason: string; approved?: boolean; status?: string } + | { state: "blocked"; parent_status: "blocked"; reason: string; approved?: boolean; status: string } + | { state: "missing_approval"; parent_status: "needs_human"; reason: string; approved?: boolean; status?: string }; + +type ChildReviewArtifact = { + body: string; + evidenceText: string; + trace: Record; + reportPath?: string; +}; + +function optionalBoolean(value: unknown): boolean | undefined { + return typeof value === "boolean" ? value : undefined; +} + +function optionalNonEmptyString(value: unknown): string | undefined { + const stringValue = typeof value === "string" ? value.trim() : ""; + return stringValue.length > 0 ? stringValue : undefined; +} + +function errorMessage(error: unknown): string { + return error instanceof Error ? error.message : String(error); +} + +const CHILD_FAILURE_STATUSES = new Set(["blocked", "failed", "failure", "error", "errored"]); +const CHILD_NEEDS_HUMAN_STATUSES = new Set(["needs_human", "rejected", "stopped", "active", "pending", "running"]); +const CHILD_SUCCESS_STATUSES = new Set(["complete", "completed", "success", "succeeded", "passed"]); + +function childStatusIs(status: string | undefined, statuses: ReadonlySet): boolean { + return status !== undefined && statuses.has(status); +} + +export function gateChildRunCompletion(childOutput: Record, runner: Exclude): ChildRunGate { + const approved = optionalBoolean(childOutput.approved); + const status = optionalNonEmptyString(childOutput.status); + const normalizedStatus = status?.toLowerCase(); + + if (childStatusIs(normalizedStatus, CHILD_FAILURE_STATUSES)) { + return { state: "blocked", parent_status: "blocked", reason: `Child ${runner} returned status=${status}.`, approved, status }; + } + if (approved === false) { + return { state: "non_approved", parent_status: "needs_human", reason: `Child ${runner} returned approved=false.`, approved, status }; + } + if (childStatusIs(normalizedStatus, CHILD_NEEDS_HUMAN_STATUSES)) { + return { state: "non_approved", parent_status: "needs_human", reason: `Child ${runner} returned status=${status}.`, approved, status }; + } + if (approved !== true) { + return { state: "missing_approval", parent_status: "needs_human", reason: `Child ${runner} did not return approved=true.`, approved, status }; + } + if (normalizedStatus !== undefined && !childStatusIs(normalizedStatus, CHILD_SUCCESS_STATUSES)) { + return { state: "non_approved", parent_status: "needs_human", reason: `Child ${runner} returned unknown status=${status}.`, approved, status }; + } + + return { state: "approved", reason: `Child ${runner} returned approved=true.`, approved, status }; +} + +type CompactReviewPointer = { + kind: "output_saved_to" | "latest_review_round_artifact"; + path?: string; +}; + +const ATOMIC_FILE_ONLY_SUFFIX_PATTERN = /\s+\(\d+(?:\.\d+)?\s*(?:B|KB|MB|GB),\s*\d+\s+lines?\)\.?(?:\s+Read this file if needed\.?)?$/i; + +function unquoteMatchingPath(value: string): string { + const quote = value[0]; + if ((quote === '"' || quote === "'") && value[value.length - 1] === quote) return value.slice(1, -1); + return value; +} + +function compactReviewPointerKind(label: string): CompactReviewPointer["kind"] { + return /^latest review round artifact$/i.test(label) ? "latest_review_round_artifact" : "output_saved_to"; +} + +function parseCompactReviewPointer(value: string): CompactReviewPointer | undefined { + const firstSubstantiveLine = value + .split(/\r?\n/) + .map((line) => line.trim()) + .find((line) => line.length > 0) ?? ""; + + const match = firstSubstantiveLine.match(/^(output saved to|saved output to|saved to|latest review round artifact):\s*(.*)$/i); + if (!match) return undefined; + + const kind = compactReviewPointerKind(match[1]); + const target = match[2].trim(); + if (target.length === 0) return { kind }; + + const path = unquoteMatchingPath(target.replace(ATOMIC_FILE_ONLY_SUFFIX_PATTERN, "").trim()); + return { kind, path }; +} + +function appendMultilineReviewArtifactField(lines: string[], label: string, value: string): void { + lines.push(`${label}:`); + lines.push(...value + .replace(/\.\s+(?=P[0-3]\s*:)/gi, ".\n") + .split(/\r?\n/) + .map((line) => line.trim()) + .filter((line) => line.length > 0)); +} + +function appendReviewArtifactField(lines: string[], label: string, value: unknown): void { + if (typeof value === "number" || typeof value === "boolean") { + lines.push(`${label}: ${String(value)}`); + return; + } + if (typeof value !== "string") return; + + const trimmed = value.trim(); + if (trimmed.length === 0) return; + + if (label === "raw_text" || label === "overall_explanation") { + appendMultilineReviewArtifactField(lines, label, trimmed); + } else { + lines.push(`${label}: ${trimmed}`); + } + + if (label === "overall_correctness" && /\b(?:incorrect|wrong|not\s+correct|failed|failure)\b/i.test(trimmed)) { + lines.push(`P1: overall_correctness ${trimmed}`); + } +} + +function prioritySeverity(value: unknown): SeverityLevel | undefined { + if (typeof value === "number" && Number.isInteger(value) && value >= 0 && value <= 3) return `p${value}` as SeverityLevel; + const match = typeof value === "string" ? value.match(/\bp([0-3])\b|^\s*([0-3])\s*$/i) : undefined; + const digit = match?.[1] ?? match?.[2]; + return digit ? `p${digit}` as SeverityLevel : undefined; +} + +function appendReviewFinding(lines: string[], finding: Record): void { + const title = optionalNonEmptyString(finding.title); + const body = optionalNonEmptyString(finding.body) ?? optionalNonEmptyString(finding.description); + const priority = prioritySeverity(finding.priority ?? finding.severity); + + if (priority !== undefined) lines.push(`${priority.toUpperCase()}: ${title ?? body ?? "structured finding"}`); + if (title !== undefined) lines.push(`finding_title: ${title}`); + if (body !== undefined) lines.push(`finding_body: ${body}`); + appendReviewArtifactField(lines, "priority", finding.priority ?? finding.severity); + + const codeLocation = finding.code_location; + if (typeof codeLocation === "object" && codeLocation !== null) { + const path = optionalNonEmptyString((codeLocation as Record).absolute_file_path); + if (path !== undefined) lines.push(`file: ${path}`); + } +} + +function appendSeverityCounts(lines: string[], value: Record): void { + for (const severity of SEVERITY_LEVELS) { + const count = value[severity] ?? value[severity.toUpperCase()]; + if (typeof count === "number" && Number.isFinite(count) && count > 0) { + lines.push(`${severity.toUpperCase()}: severity count reported ${Math.floor(count)}`); + } else if (typeof count === "number" && count === 0) { + lines.push(`${severity.toUpperCase()}: none`); + } + } +} + +function appendReviewArtifactJson(lines: string[], value: unknown, key = "root"): void { + if (Array.isArray(value)) { + for (const item of value) appendReviewArtifactJson(lines, item, key); + return; + } + if (typeof value !== "object" || value === null) return; + + const record = value as Record; + if (key === "findings") { + appendReviewFinding(lines, record); + } + + for (const [childKey, childValue] of Object.entries(record)) { + if (childKey === "findings" && Array.isArray(childValue)) { + for (const finding of childValue) { + if (typeof finding === "object" && finding !== null) appendReviewFinding(lines, finding as Record); + } + continue; + } + if (childKey === "severity_counts" && typeof childValue === "object" && childValue !== null) { + appendSeverityCounts(lines, childValue as Record); + continue; + } + if (/^(?:reviewer|overall_correctness|overall_explanation|raw_text|validation(?:_notes|_output|_summary|_results)?|commands_run|notes?)$/i.test(childKey)) { + appendReviewArtifactField(lines, childKey, childValue); + continue; + } + appendReviewArtifactJson(lines, childValue, childKey); + } +} + +export function reviewArtifactToEvidenceText(body: string): string { + try { + const parsed = JSON.parse(body) as unknown; + const lines: string[] = []; + appendReviewArtifactJson(lines, parsed); + return lines.length > 0 ? lines.join("\n") : body; + } catch { + return body; + } +} + +async function loadReviewArtifactPath(path: string): Promise { + const body = await readFile(path, "utf8"); + if (body.trim().length === 0) throw new Error(`Child review artifact is empty: ${path}`); + return body; +} + +async function loadReviewEvidenceArtifact(path: string, trace: Record): Promise { + const body = await loadReviewArtifactPath(path); + return { + body, + evidenceText: reviewArtifactToEvidenceText(body), + reportPath: path, + trace, + }; +} + +export async function loadChildReviewArtifact(childOutput: Record): Promise { + const reviewReportPath = optionalNonEmptyString(childOutput.review_report_path); + const inlineReviewReport = optionalNonEmptyString(childOutput.review_report); + const inlineCompactPointer = inlineReviewReport !== undefined ? parseCompactReviewPointer(inlineReviewReport) : undefined; + + if (reviewReportPath !== undefined) { + try { + return await loadReviewEvidenceArtifact(reviewReportPath, { source: "review_report_path", path: reviewReportPath, loaded: true }); + } catch (error) { + const reviewReportPathError = errorMessage(error); + return { + body: "", + evidenceText: `blocked unable to review missing dependency: child review_report_path unreadable: ${reviewReportPath}: ${reviewReportPathError}`, + trace: { source: "review_report_path", path: reviewReportPath, loaded: false, error: reviewReportPathError, fail_closed: true }, + }; + } + } + + if (inlineReviewReport !== undefined) { + if (inlineCompactPointer !== undefined) { + if (inlineCompactPointer.path !== undefined) { + try { + return await loadReviewEvidenceArtifact(inlineCompactPointer.path, { + source: "inline_review_report_pointer", + path: inlineCompactPointer.path, + loaded: true, + compact_pointer: true, + pointer_kind: inlineCompactPointer.kind, + }); + } catch (error) { + const pointerErrorMessage = errorMessage(error); + return { + body: "", + evidenceText: `blocked unable to review missing dependency: compact child review_report pointer unreadable: ${inlineCompactPointer.path}: ${pointerErrorMessage}`, + trace: { source: "inline_review_report_pointer", path: inlineCompactPointer.path, loaded: false, compact_pointer: true, pointer_kind: inlineCompactPointer.kind, error: pointerErrorMessage, fail_closed: true }, + }; + } + } + return { + body: "", + evidenceText: "blocked unable to review missing dependency: child review_report is only a compact file-only pointer without a readable path", + trace: { source: "inline_review_report_pointer", loaded: false, compact_pointer: true, pointer_kind: inlineCompactPointer.kind, fail_closed: true }, + }; + } + return { + body: inlineReviewReport, + evidenceText: reviewArtifactToEvidenceText(inlineReviewReport), + trace: { source: "inline_review_report", loaded: true }, + }; + } + + return { + body: "", + evidenceText: "blocked unable to review missing dependency: child output did not include review_report_path or substantive review_report", + trace: { source: "missing_child_review_report", loaded: false, fail_closed: true }, + }; +} + +const COMPOUND_ENGINEERING_EVIDENCE_KEY = "compound_engineering_evidence"; + +type StructuredEvidenceCommand = { + command?: string; + exit_code?: number; + summary?: string; +}; + +type StructuredEvidenceCriterion = { + satisfied: boolean; + evidence?: string; + source?: string; + commands?: StructuredEvidenceCommand[]; +}; + +type CompoundEngineeringEvidence = { + independent: StructuredEvidenceCriterion; + acceptance_mapped: StructuredEvidenceCriterion; + diff_aware: StructuredEvidenceCriterion; + validation_backed: StructuredEvidenceCriterion; + risk_aware: StructuredEvidenceCriterion; + fresh: StructuredEvidenceCriterion; + severity_counts: Required; + blocked?: boolean; + conflicted?: boolean; + validation_failed?: boolean; +}; + +type StructuredEvidenceGate = { + childEvidence: ReviewEvidence; + trace: Record; + missing: ReviewCriterion[]; + errors: string[]; + childReviewArtifact?: ChildReviewArtifact; +}; + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function compactErrorList(values: readonly string[]): string[] { + return [...new Set(values)].slice(0, 20); +} + +function hasSubstantiveEvidence(value: unknown): boolean { + return typeof value === "string" && value.trim().length > 0; +} + +const PASSING_COMMAND_SUMMARY_PATTERN = /\b(?:pass(?:ed|es|ing)?|succeed(?:ed|s|ing)?|success(?:ful|fully)?|exit\s+(?:code|status)\s*:?\s*0|(?:exit_)?code\s*:?\s*0|status\s*:?\s*0|0\s+(?:failures?|errors?)|zero\s+(?:failures?|errors?)|no\s+(?:failures?|errors?)|all\s+tests?\s+passed)\b/i; +const FAILING_COMMAND_SUMMARY_PATTERN = /\b(?:fail(?:ed|s|ing|ures?)|errored|non[-\s]?zero|exit\s+(?:code|status)\s*:?\s*[1-9]\d*|(?:exit_)?code\s*:?\s*[1-9]\d*|status\s*:?\s*[1-9]\d*)\b/i; + +function commandSummaryPasses(summary: unknown): boolean { + if (typeof summary !== "string") return false; + const value = summary.trim(); + return value.length > 0 && PASSING_COMMAND_SUMMARY_PATTERN.test(value) && !FAILING_COMMAND_SUMMARY_PATTERN.test(value); +} + +function validationCommandsPass(commands: unknown): boolean { + if (commands === undefined) return true; + if (!Array.isArray(commands)) return false; + for (const command of commands) { + if (!isRecord(command)) return false; + const exitCode = command.exit_code; + if (typeof exitCode === "number" && Number.isFinite(exitCode)) { + if (Math.trunc(exitCode) !== 0) return false; + continue; + } + if (!commandSummaryPasses(command.summary)) return false; + } + return true; +} + +function structuredEvidenceHasSupport(criterion: ReviewCriterion, block: Record): boolean { + if (hasSubstantiveEvidence(block.evidence)) return true; + return criterion === "validation_backed" && Array.isArray(block.commands) && block.commands.length > 0 && validationCommandsPass(block.commands); +} + +function readSeverityCounts(value: unknown, errors: string[]): Required | undefined { + if (!isRecord(value)) { + errors.push("severity_counts must be an object with p0, p1, p2, and p3 numbers"); + return undefined; + } + + const counts: Partial> = {}; + for (const severity of SEVERITY_LEVELS) { + const count = value[severity]; + if (typeof count !== "number" || !Number.isFinite(count) || count < 0) { + errors.push(`severity_counts.${severity} must be a non-negative number`); + return undefined; + } + counts[severity] = Math.floor(count); + } + + return counts as Required; +} + +function structuredEvidenceToReviewEvidence(value: unknown): { evidence: ReviewEvidence; missing: ReviewCriterion[]; errors: string[] } { + const errors: string[] = []; + const evidence: ReviewEvidence = {}; + const missing: ReviewCriterion[] = []; + + if (!isRecord(value)) { + return { + evidence, + missing: [...REVIEW_CRITERIA], + errors: [`${COMPOUND_ENGINEERING_EVIDENCE_KEY} must be an object`], + }; + } + + for (const criterion of REVIEW_CRITERIA) { + const block = value[criterion]; + if (!isRecord(block)) { + missing.push(criterion); + errors.push(`${criterion} must be an object with satisfied boolean and evidence`); + continue; + } + + if (typeof block.satisfied !== "boolean") { + missing.push(criterion); + errors.push(`${criterion}.satisfied must be boolean`); + continue; + } + + if (block.satisfied !== true) { + evidence[criterion] = false; + missing.push(criterion); + continue; + } + + if (!structuredEvidenceHasSupport(criterion, block)) { + missing.push(criterion); + errors.push(`${criterion} is satisfied but lacks explicit evidence${criterion === "validation_backed" ? " or passing command summaries" : ""}`); + continue; + } + + evidence[criterion] = true; + } + + const severity_counts = readSeverityCounts(value.severity_counts, errors); + if (severity_counts !== undefined) evidence.severity_counts = severity_counts; + + if (typeof value.blocked === "boolean") evidence.blocked = value.blocked; + if (typeof value.conflicted === "boolean") evidence.conflicted = value.conflicted; + if (typeof value.validation_failed === "boolean") evidence.validation_failed = value.validation_failed; + if (isRecord(value.validation_backed) && validationCommandsPass(value.validation_backed.commands) === false) { + evidence.validation_backed = false; + evidence.validation_failed = true; + if (!missing.includes("validation_backed")) missing.push("validation_backed"); + errors.push("validation_backed.commands must all have zero exit codes or explicit passing summaries"); + } + + return { evidence, missing, errors: compactErrorList(errors) }; +} + +function findNamedEvidence(value: unknown, seen = new WeakSet()): unknown | undefined { + if (!isRecord(value) && !Array.isArray(value)) return undefined; + if (seen.has(value)) return undefined; + seen.add(value); + + if (isRecord(value) && Object.prototype.hasOwnProperty.call(value, COMPOUND_ENGINEERING_EVIDENCE_KEY)) { + return value[COMPOUND_ENGINEERING_EVIDENCE_KEY]; + } + + const values = Array.isArray(value) ? value : Object.values(value); + for (const childValue of values) { + const found = findNamedEvidence(childValue, seen); + if (found !== undefined) return found; + } + return undefined; +} + +function parseJsonCandidate(value: string): unknown | undefined { + try { + return JSON.parse(value) as unknown; + } catch { + return undefined; + } +} + +function extractJsonEvidenceFromBody(body: string): unknown | undefined { + const parsedBody = parseJsonCandidate(body); + const direct = parsedBody !== undefined ? findNamedEvidence(parsedBody) : undefined; + if (direct !== undefined) return direct; + + const fencePattern = /```(?:json)?\s*([\s\S]*?)```/gi; + for (const match of body.matchAll(fencePattern)) { + const parsedFence = parseJsonCandidate(match[1].trim()); + const fromFence = parsedFence !== undefined ? findNamedEvidence(parsedFence) : undefined; + if (fromFence !== undefined) return fromFence; + } + + return undefined; +} + +function structuredEvidenceNeedsHumanReduction(gate: StructuredEvidenceGate, fallback: ReviewEvidenceReduction): ReviewEvidenceReduction { + if (fallback.decision === "sufficient" && gate.errors.length > 0) { + return { + decision: "needs_human", + missing: gate.missing, + severity_counts: normalizeSeverityCounts(gate.childEvidence.severity_counts), + reason: `Structured child evidence is incomplete or malformed: ${gate.errors.join("; ")}.`, + }; + } + return fallback; +} + +async function loadStructuredChildEvidence(childOutput: Record): Promise { + const directEvidence = childOutput[COMPOUND_ENGINEERING_EVIDENCE_KEY]; + if (directEvidence !== undefined) { + const converted = structuredEvidenceToReviewEvidence(directEvidence); + return { + childEvidence: converted.evidence, + trace: { + source: "child_output", + contract: COMPOUND_ENGINEERING_EVIDENCE_KEY, + loaded: true, + missing: converted.missing, + errors: converted.errors, + }, + missing: converted.missing, + errors: converted.errors, + }; + } + + const childReviewArtifact = await loadChildReviewArtifact(childOutput); + if (childReviewArtifact.trace.fail_closed === true) { + return { + childEvidence: { blocked: true, severity_counts: emptySeverityCounts() }, + trace: { + ...childReviewArtifact.trace, + contract: COMPOUND_ENGINEERING_EVIDENCE_KEY, + loaded: false, + errors: ["child review artifact was missing or unreadable"], + }, + missing: [...REVIEW_CRITERIA], + errors: ["child review artifact was missing or unreadable"], + childReviewArtifact, + }; + } + + const artifactEvidence = extractJsonEvidenceFromBody(childReviewArtifact.body); + if (artifactEvidence === undefined) { + const missingError = `missing named ${COMPOUND_ENGINEERING_EVIDENCE_KEY} block in child review artifact`; + return { + childEvidence: { severity_counts: emptySeverityCounts() }, + trace: { + ...childReviewArtifact.trace, + contract: COMPOUND_ENGINEERING_EVIDENCE_KEY, + loaded: false, + errors: [missingError], + }, + missing: [...REVIEW_CRITERIA], + errors: [missingError], + childReviewArtifact, + }; + } + + const converted = structuredEvidenceToReviewEvidence(artifactEvidence); + return { + childEvidence: converted.evidence, + trace: { + ...childReviewArtifact.trace, + contract: COMPOUND_ENGINEERING_EVIDENCE_KEY, + loaded: true, + missing: converted.missing, + errors: converted.errors, + }, + missing: converted.missing, + errors: converted.errors, + childReviewArtifact, + }; +} + +function runnerSafetyNote(requestedRunner: ImplementationRunner, runner: ResolvedImplementationRunner): string { + if (requestedRunner === "auto") return "auto resolved to handoff-only for iteration 3"; + if (runner === "handoff-only") return "handoff-only is non-mutating"; + return "explicit runner may launch only after approval"; +} + +async function writeFinalManifest(options: { + manifestPath: string; + runId: string; + startedAt: Date; + input: Record; + finalReportPath: string; + artifacts: ReadonlyMap; +}): Promise { + await writeJson(options.manifestPath, { + runId: options.runId, + startedAt: options.startedAt.toISOString(), + completedAt: new Date().toISOString(), + input: options.input, + finalReportPath: displayPath(options.finalReportPath), + artifacts: manifestArtifacts(options.artifacts, options.manifestPath), + }); +} + +async function writeCompactFinalReport(options: { + path: string; + status: WorkflowStatus; + mode: ResolvedCompoundMode; + runner: ResolvedImplementationRunner; + prompt: string; + message: string; + paths: ReadonlyMap; + implementation?: Record; +}): Promise { + const implementationBlock = options.implementation + ? `\n\n## Implementation handoff\n\n\`\`\`json\n${JSON.stringify(options.implementation, null, 2)}\n\`\`\`` + : ""; + const pathLines = [...options.paths] + .map(([name, path]) => `- ${name}: ${displayPath(path)}`) + .join("\n"); + + return writeMarkdown(options.path, artifactMarkdown("Compound Engineering final report", [ + `Status: ${options.status}`, + `Mode: ${options.mode}`, + `Runner: ${options.runner}`, + "", + "## Request", + options.prompt, + "", + "## Summary", + options.message, + "", + "## Artifacts", + pathLines || "- No additional artifacts.", + implementationBlock, + ].join("\n"))); +} + +const compoundEngineeringWorkflow = defineWorkflow("compound-engineering") + .description("Safe Compound Engineering loop: classify intake, scout memory, brainstorm/plan, require approval, run explicit implementation runners, gate review evidence, and optionally capture learning.") + .input("prompt", Type.String({ description: "Idea, spec/plan path, work request, review target, or learning-capture request." })) + .input("mode", Type.Union([ + Type.Literal("auto"), + Type.Literal("brainstorm"), + Type.Literal("plan"), + Type.Literal("work"), + Type.Literal("review"), + Type.Literal("compound-only"), + ], { default: "auto", description: "auto, brainstorm, plan, work, review, or compound-only." })) + .input("runner", Type.Union([ + Type.Literal("auto"), + Type.Literal("goal"), + Type.Literal("ralph"), + Type.Literal("handoff-only"), + ], { default: "auto", description: "auto, goal, ralph, or handoff-only. Iteration 3 auto resolves to handoff-only; explicit goal/ralph run after approval." })) + .input("max_loops", Type.Number({ default: DEFAULT_MAX_LOOPS, description: "Maximum implementation/fix/review loops to include in handoff metadata." })) + .input("base_branch", Type.String({ default: DEFAULT_BASE_BRANCH, description: "Base branch for implementation/review handoff." })) + .input("git_worktree_dir", Type.String({ default: "", description: "Optional reusable worktree directory for implementation handoff." })) + .input("create_pr", Type.Boolean({ default: false, description: "Strict true authorizes PR creation in handoff inputs; default false." })) + .input("learning_mode", Type.Union([ + Type.Literal("ask"), + Type.Literal("off"), + Type.Literal("lightweight"), + Type.Literal("full"), + ], { default: "ask", description: "ask, off, lightweight, or full." })) + .input("memory_scope", Type.Union([ + Type.Literal("repo"), + Type.Literal("none"), + ], { default: "repo", description: "repo or none." })) + .worktreeFromInputs({ + gitWorktreeDir: "git_worktree_dir", + baseBranch: "base_branch", + }) + .output("status", statusSchema) + .output("mode", resolvedModeSchema) + .output("runner", resolvedRunnerSchema) + .output("approved", Type.Boolean({ description: "Whether the workflow reached an approved or complete safe exit." })) + .output("artifact_dir", Type.String({ description: "Hidden per-run artifact directory." })) + .output("manifest_path", Type.String({ description: "Path to manifest JSON." })) + .output("message", Type.String({ description: "Compact user-facing summary." })) + .output("brainstorm_path", Type.Optional(Type.String({ description: "Saved brainstorm brief path." }))) + .output("plan_path", Type.Optional(Type.String({ description: "Saved plan path." }))) + .output("spec_path", Type.Optional(Type.String({ description: "Saved spec path." }))) + .output("approved_spec_path", Type.Optional(Type.String({ description: "Approved plan/spec path." }))) + .output("implementation", Type.Optional(Type.Object({}, { additionalProperties: true, description: "Guarded handoff metadata or child-run summary." }))) + .output("review_report_path", Type.Optional(Type.String({ description: "Saved normalized review/evidence report path." }))) + .output("learning_doc_path", Type.Optional(Type.String({ description: "Saved docs/solutions learning artifact path." }))) + .run(async (ctx) => { + const startedAt = new Date(); + const cwd = text((ctx as { cwd?: unknown }).cwd, process.cwd()); + const prompt = text(ctx.inputs.prompt); + const requestedMode: CompoundMode = isCompoundMode(ctx.inputs.mode) ? ctx.inputs.mode : "auto"; + const requestedRunner: ImplementationRunner = isImplementationRunner(ctx.inputs.runner) ? ctx.inputs.runner : "auto"; + const learningMode: LearningMode = isLearningMode(ctx.inputs.learning_mode) ? ctx.inputs.learning_mode : "ask"; + const memoryScope: MemoryScope = isMemoryScope(ctx.inputs.memory_scope) ? ctx.inputs.memory_scope : "repo"; + const mode = resolveMode(prompt, requestedMode); + const runner = resolveRunner(requestedRunner); + const maxLoops = positiveInteger(ctx.inputs.max_loops, DEFAULT_MAX_LOOPS); + const baseBranch = text(ctx.inputs.base_branch, DEFAULT_BASE_BRANCH); + const gitWorktreeDir = text(ctx.inputs.git_worktree_dir); + const createPr = normalizeCreatePr(ctx.inputs.create_pr); + const { runId, artifactDir } = await createArtifactRun(WORKFLOW_NAME, startedAt, cwd); + const artifacts = new Map(); + const addArtifact = (name: string, path: string): string => { + artifacts.set(name, path); + return path; + }; + const prepareArtifactPath = async (name: string, desiredPath: string): Promise => addArtifact(name, await nextAvailablePath(desiredPath)); + const writeArtifactMarkdown = async (name: string, desiredPath: string, content: string): Promise => addArtifact(name, await writeMarkdown(desiredPath, content)); + const manifestPath = join(artifactDir, "manifest.json"); + let finalReportPath = join(cwd, datedMarkdownPath(WORKFLOW_NAME, prompt, "final-report", startedAt)); + const writeFinalReport = async (options: Omit[0], "path" | "paths">): Promise => { + finalReportPath = await prepareArtifactPath("final-report", finalReportPath); + finalReportPath = await writeCompactFinalReport({ ...options, path: finalReportPath, paths: artifacts }); + return addArtifact("final-report", finalReportPath); + }; + + const baseInput = { + prompt, + requested_mode: requestedMode, + mode, + requested_runner: requestedRunner, + runner, + max_loops: maxLoops, + base_branch: baseBranch, + git_worktree_dir: gitWorktreeDir, + create_pr: createPr, + learning_mode: learningMode, + memory_scope: memoryScope, + }; + + const captureLearningArtifact = async (options: { + reads: string[]; + summary: string; + uiPrompt: string; + }): Promise<{ selectedLearningMode: LearningMode; learningDocPath?: string }> => { + let selectedLearningMode = learningMode; + if (learningMode === "ask") { + selectedLearningMode = await ctx.ui.select( + options.uiPrompt, + ["lightweight", "full", "off"] as const, + ) as LearningMode; + } + + if (selectedLearningMode === "off") return { selectedLearningMode }; + + const learningDocPath = await prepareArtifactPath("learning-doc", join(cwd, datedMarkdownPath("docs/solutions", prompt, "learning", startedAt))); + await ctx.task("capture-learning", { + reads: options.reads, + prompt: `Capture a ${selectedLearningMode} solved-problem learning note.\n\nRequest:\n${prompt}\n\n${options.summary}\n\nUse available artifacts as background. Include problem, context, solution/lesson, validation/evidence if known, and when to reuse the lesson. Do not include secrets, raw environment dumps, credentials, or unnecessary transcripts. Do not edit CONCEPTS.md or other discoverability anchors in iteration 3; produce only this docs/solutions Markdown artifact.\n\n${EVERY_INC_CREDIT}`, + ...fileOnlyOutput(learningDocPath), + }); + + return { selectedLearningMode, learningDocPath }; + }; + + if (prompt.length === 0) { + const message = "Blocked: prompt is required."; + await writeFinalReport({ status: "blocked", mode, runner, prompt, message }); + await writeFinalManifest({ manifestPath, runId, startedAt, input: baseInput, finalReportPath, artifacts }); + return outputRecord({ status: "blocked", mode, runner, approved: false, artifact_dir: displayPath(artifactDir), manifest_path: displayPath(manifestPath), message }); + } + + await writeArtifactMarkdown("intake", join(artifactDir, "00-intake.md"), artifactMarkdown("Compound Engineering intake", [ + `Resolved mode: ${mode}`, + `Resolved runner: ${runner}`, + `Safe runner note: ${runnerSafetyNote(requestedRunner, runner)}`, + `Create PR: ${createPr}`, + "", + "## Prompt", + prompt, + ].join("\n"))); + + let contextPath = await prepareArtifactPath("memory-context", join(artifactDir, "01-memory-context.md")); + if (memoryScope === "repo") { + await ctx.task("scout-memory-and-context", { + prompt: `Scout only lightweight repo memory for a Compound Engineering run.\n\nPrompt:\n${prompt}\n\nMode: ${mode}\n\nLook for high-signal anchors if present: STRATEGY.md, CONCEPTS.md, docs/brainstorms/, docs/plans/, docs/solutions/, research/docs/, specs/, README files, and similar workflow artifacts. Do not modify files. Summarize useful context, unresolved ambiguities, and any referenced file/path that appears missing.\n\n${EVERY_INC_CREDIT}`, + ...fileOnlyOutput(contextPath), + }); + } else { + contextPath = await writeArtifactMarkdown("memory-context", contextPath, artifactMarkdown("Memory context", "memory_scope=none; repo memory scout skipped.")); + } + + if (mode === "review") { + const reviewPath = await nextAvailablePath(join(cwd, datedMarkdownPath(WORKFLOW_NAME, `${prompt} review`, "review", startedAt))); + const review = await ctx.task("review-only-evidence-report", { + reads: [contextPath], + prompt: `Run a read-only Compound Engineering review for this target.\n\nTarget/request:\n${prompt}\n\nRead memory context at ${displayPath(contextPath)}. Infer whether the target is a branch range, PR, path, or current repo. Use safe local read-only evidence only. Do not modify files, install dependencies, create commits, post comments, or create a PR.\n\nWrite a compact report with: scope, evidence inspected, findings by P0/P1/P2/P3, validation commands available/run/skipped, review sufficiency dimensions (independent, acceptance-mapped, diff-aware, validation-backed, risk-aware, fresh), and residual risks.`, + ...fileOnlyOutput(reviewPath), + }); + let reviewBody = "blocked unable to review missing dependency"; + let reviewReportPath: string | undefined; + let reportLoadError: string | undefined; + try { + const loadedReview = await loadSavedStageReport(reviewPath, review); + reviewBody = loadedReview.body; + if (loadedReview.source === "saved-file") reviewReportPath = addArtifact("review-report", reviewPath); + } catch (error) { + reportLoadError = error instanceof Error ? error.message : String(error); + reviewBody = `blocked unable to review missing dependency: ${reportLoadError}`; + } + const reduction = reduceReviewEvidence(parseEvidenceFromText(reviewBody)); + const approved = reduction.decision === "sufficient"; + let status: WorkflowStatus = "needs_human"; + if (approved) { + status = "review_only"; + } else if (reduction.decision === "blocked") { + status = "blocked"; + } + + const loadErrorMessage = reportLoadError ? ` (${reportLoadError})` : ""; + const message = approved + ? `Review-only report produced; evidence gate decision: ${reduction.decision}.` + : `Review-only evidence gate returned ${reduction.decision}: ${reduction.reason}${loadErrorMessage}`; + await writeFinalReport({ status, mode, runner, prompt, message }); + await writeFinalManifest({ manifestPath, runId, startedAt, input: baseInput, finalReportPath, artifacts }); + return outputRecord({ + status, + mode, + runner, + approved, + artifact_dir: displayPath(artifactDir), + manifest_path: displayPath(manifestPath), + message, + review_report_path: reviewReportPath ? displayPath(reviewReportPath) : undefined, + }); + } + + if (mode === "compound-only") { + const { selectedLearningMode, learningDocPath } = await captureLearningArtifact({ + reads: [contextPath], + uiPrompt: "Capture a Compound Engineering learning artifact for this request?", + summary: `Repo memory context: ${displayPath(contextPath)}. This is a learning-only Compound Engineering run.`, + }); + const message = learningDocPath + ? `Learning artifact written to ${displayPath(learningDocPath)}.` + : "Learning capture skipped by mode selection."; + const status: WorkflowStatus = learningDocPath ? "complete" : "needs_human"; + await writeFinalReport({ status, mode, runner, prompt, message }); + await writeFinalManifest({ manifestPath, runId, startedAt, input: { ...baseInput, selected_learning_mode: selectedLearningMode }, finalReportPath, artifacts }); + return outputRecord({ + status, + mode, + runner, + approved: learningDocPath !== undefined, + artifact_dir: displayPath(artifactDir), + manifest_path: displayPath(manifestPath), + message, + learning_doc_path: learningDocPath ? displayPath(learningDocPath) : undefined, + }); + } + + let brainstormPath: string | undefined; + if (mode === "brainstorm") { + brainstormPath = await prepareArtifactPath("brainstorm", join(cwd, datedMarkdownPath("docs/brainstorms", prompt, "brainstorm", startedAt))); + await ctx.task("brainstorm-requirements", { + reads: [contextPath], + prompt: `Create a concise Compound Engineering brainstorm brief for this request.\n\nPrompt:\n${prompt}\n\nRead memory context at ${displayPath(contextPath)}. Ask no questions in this stage; record unresolved questions explicitly instead. Include user/problem, desired outcome, options, recommendation, assumptions, non-goals, and what would make the work concrete enough to plan.\n\n${EVERY_INC_CREDIT}`, + ...fileOnlyOutput(brainstormPath), + }); + } + + let planPath = join(cwd, datedMarkdownPath("docs/plans", prompt, "plan", startedAt)); + let specPath = await prepareArtifactPath("spec", join(cwd, datedMarkdownPath("specs", prompt, "spec", startedAt))); + const plan = await ctx.task("draft-plan-or-spec", { + reads: [contextPath, ...(brainstormPath ? [brainstormPath] : [])], + prompt: `Draft a plan/spec for the Compound Engineering approval gate.\n\nMode: ${mode}\nOriginal request:\n${prompt}\n\nRead available artifacts:\n- Memory context: ${displayPath(contextPath)}${brainstormPath ? `\n- Brainstorm brief: ${displayPath(brainstormPath)}` : ""}\n\nWrite a compact Markdown plan/spec suitable for human approval. Include status Draft, scope, acceptance criteria, implementation approach, validation plan, review/evidence requirements, risks, non-goals, and explicit note that implementation must not start until approval. If the prompt references an existing spec/plan path, treat it as source material and still produce this approval artifact.\n\n${EVERY_INC_CREDIT}`, + ...fileOnlyOutput(specPath), + }); + planPath = await writeArtifactMarkdown("plan", planPath, artifactMarkdown("Compound Engineering plan", `This companion plan points to the generated approval spec.\n\nSpec path: ${displayPath(specPath)}\n\nDraft summary:\n${plan.text}`)); + + let approvalDecision = parseApprovalDecision(await ctx.ui.select( + `Approve this Compound Engineering plan/spec before any implementation handoff?\n\nSpec: ${displayPath(specPath)}\nPlan: ${displayPath(planPath)}\n\nChoose Approve only if the scope and validation plan are acceptable.`, + ["Approve", "Revise", "Reject", "Stop"] as const, + )); + let revisions = 0; + + while (approvalDecision === "revise" && revisions < maxLoops) { + revisions += 1; + const revisionRequest = await ctx.ui.input(`What should change before approval? Revision ${revisions}/${maxLoops}.`); + const previousSpecPath = specPath; + specPath = await prepareArtifactPath("spec", specPath); + await ctx.task(`revise-plan-or-spec-${revisions}`, { + reads: [previousSpecPath, planPath, contextPath], + prompt: `Revise the Compound Engineering approval spec in place conceptually and write the new full Markdown artifact.\n\nExisting spec: ${displayPath(previousSpecPath)}\nExisting plan: ${displayPath(planPath)}\nRevision request:\n${text(revisionRequest, "No specific revision text provided.")}\n\nPreserve the no-implementation-before-approval guardrail.`, + ...fileOnlyOutput(specPath), + }); + approvalDecision = parseApprovalDecision(await ctx.ui.select( + `Review the revised Compound Engineering spec.\n\nSpec: ${displayPath(specPath)}\n\nApprove before implementation handoff?`, + ["Approve", "Revise", "Reject", "Stop"] as const, + )); + } + + if (approvalDecision === "revise") approvalDecision = "stopped"; + + if (approvalDecision !== "approved") { + const status: WorkflowStatus = approvalDecision === "rejected" ? "rejected" : "stopped"; + const message = status === "rejected" + ? "Plan/spec rejected; no implementation handoff emitted." + : "Approval loop stopped or exhausted; no implementation handoff emitted."; + await writeFinalReport({ status, mode, runner, prompt, message }); + await writeFinalManifest({ manifestPath, runId, startedAt, input: { ...baseInput, approval_decision: approvalDecision, revisions }, finalReportPath, artifacts }); + return outputRecord({ + status, + mode, + runner, + approved: false, + artifact_dir: displayPath(artifactDir), + manifest_path: displayPath(manifestPath), + message, + brainstorm_path: brainstormPath ? displayPath(brainstormPath) : undefined, + plan_path: displayPath(planPath), + spec_path: displayPath(specPath), + }); + } + + const approvedSpecPath = specPath; + let effectiveGitWorktreeDir = gitWorktreeDir; + if (runner === "ralph") { + try { + effectiveGitWorktreeDir = await resolveEffectiveWorktreeRoot(gitWorktreeDir, cwd); + } catch (error) { + const message = `Plan/spec approved, but Ralph worktree root resolution failed: ${error instanceof Error ? error.message : String(error)}`; + const implementation = outputRecord({ + kind: "worktree_root_resolution_failed", + requested_runner: requestedRunner, + resolved_runner: runner, + workflow: runner, + requested_git_worktree_dir: gitWorktreeDir, + safe_note: "Ralph was not launched because a non-empty git_worktree_dir must resolve to a Git worktree root from ctx.cwd.", + child_workflow_launched: false, + }); + await writeFinalReport({ status: "blocked", mode, runner, prompt, message, implementation }); + await writeFinalManifest({ manifestPath, runId, startedAt, input: { ...baseInput, approval_decision: approvalDecision, revisions }, finalReportPath, artifacts }); + return outputRecord({ + status: "blocked", + mode, + runner, + approved: false, + artifact_dir: displayPath(artifactDir), + manifest_path: displayPath(manifestPath), + message, + brainstorm_path: brainstormPath ? displayPath(brainstormPath) : undefined, + plan_path: displayPath(planPath), + spec_path: displayPath(specPath), + approved_spec_path: displayPath(approvedSpecPath), + implementation, + }); + } + } + const handoff = buildChildHandoff({ + runner, + approvedPath: displayPath(approvedSpecPath), + prompt, + maxLoops, + baseBranch, + gitWorktreeDir: effectiveGitWorktreeDir, + createPr, + }); + + if (runner === "handoff-only") { + const implementation = outputRecord({ + kind: "guarded_handoff", + requested_runner: requestedRunner, + resolved_runner: runner, + workflow: handoff.workflow, + inputs: handoff.inputs, + safe_note: handoff.safe_note, + child_workflow_launched: false, + }); + const status: WorkflowStatus = mode === "work" || requestedRunner !== "handoff-only" ? "handoff_ready" : "approved"; + const message = "Plan/spec approved. Iteration 3 stopped safely with handoff-only metadata and did not implement code."; + + await writeFinalReport({ status, mode, runner, prompt, message, implementation }); + await writeFinalManifest({ manifestPath, runId, startedAt, input: { ...baseInput, approval_decision: approvalDecision, revisions }, finalReportPath, artifacts }); + + return outputRecord({ + status, + mode, + runner, + approved: true, + artifact_dir: displayPath(artifactDir), + manifest_path: displayPath(manifestPath), + message, + brainstorm_path: brainstormPath ? displayPath(brainstormPath) : undefined, + plan_path: displayPath(planPath), + spec_path: displayPath(specPath), + approved_spec_path: displayPath(approvedSpecPath), + implementation, + }); + } + + const childInputs = handoff.inputs; + const childResult = await ctx.workflow(runner === "goal" ? goal : ralph, { + stageName: `${runner} implementation`, + inputs: childInputs, + }); + const childOutput = normalizedChildOutput(childResult); + const receiptChildOutput = compactChildOutputForReceipt(childOutput); + const childGate = gateChildRunCompletion(childOutput, runner); + let reviewReportPath: string | undefined; + + if (childGate.state !== "approved") { + const status = childGate.parent_status; + const implementation = outputRecord({ + kind: "child_workflow_receipt", + requested_runner: requestedRunner, + resolved_runner: runner, + workflow: runner, + inputs: childInputs, + safe_note: handoff.safe_note, + child_workflow_launched: true, + outputs: receiptChildOutput, + gate_child_run_completion: childGate, + }); + const message = status === "blocked" + ? `Plan/spec approved and ${runner} child workflow ran, but child completion gate returned blocked: ${childGate.reason}` + : `Plan/spec approved and ${runner} child workflow ran, but child completion gate requires human review: ${childGate.reason}`; + + await writeFinalReport({ status, mode, runner, prompt, message, implementation }); + await writeFinalManifest({ manifestPath, runId, startedAt, input: { ...baseInput, approval_decision: approvalDecision, revisions }, finalReportPath, artifacts }); + + return outputRecord({ + status, + mode, + runner, + approved: false, + artifact_dir: displayPath(artifactDir), + manifest_path: displayPath(manifestPath), + message, + brainstorm_path: brainstormPath ? displayPath(brainstormPath) : undefined, + plan_path: displayPath(planPath), + spec_path: displayPath(specPath), + approved_spec_path: displayPath(approvedSpecPath), + implementation, + }); + } + + const structuredGate = await loadStructuredChildEvidence(childOutput); + const childReviewArtifact = structuredGate.childReviewArtifact; + if (childReviewArtifact?.reportPath !== undefined) reviewReportPath = addArtifact("child-review-report", childReviewArtifact.reportPath); + const childEvidence = structuredGate.childEvidence; + const initialReduction = reduceReviewEvidence(childEvidence); + const reduction = structuredEvidenceNeedsHumanReduction(structuredGate, initialReduction); + + let status: WorkflowStatus = "needs_human"; + if (reduction.decision === "sufficient") { + status = "complete"; + } else if (reduction.decision === "blocked") { + status = "blocked"; + } + const implementation = outputRecord({ + kind: "child_workflow_receipt", + requested_runner: requestedRunner, + resolved_runner: runner, + workflow: runner, + inputs: childInputs, + safe_note: handoff.safe_note, + child_workflow_launched: true, + outputs: receiptChildOutput, + evidence: { + child: childEvidence, + structured_child: structuredGate.trace, + child_review_artifact: childReviewArtifact?.trace, + }, + gate_child_run_completion: childGate, + gate_review_evidence: reduction, + }); + let selectedLearningMode: LearningMode | undefined; + let learningDocPath: string | undefined; + if (status === "complete") { + const learning = await captureLearningArtifact({ + reads: [contextPath, approvedSpecPath, ...(reviewReportPath ? [reviewReportPath] : [])], + uiPrompt: "Review evidence is sufficient. Capture a Compound Engineering learning artifact for this validated implementation?", + summary: `Validated implementation summary:\n- Runner: ${runner}\n- Approved spec: ${displayPath(approvedSpecPath)}\n- Review decision: ${reduction.decision}\n- Review reason: ${reduction.reason}\n- Child review report: ${reviewReportPath ? displayPath(reviewReportPath) : "structured child output"}`, + }); + selectedLearningMode = learning.selectedLearningMode; + learningDocPath = learning.learningDocPath; + } + + let learningMessage = ""; + if (status === "complete") { + learningMessage = learningDocPath + ? ` Learning artifact written to ${displayPath(learningDocPath)}.` + : ` Learning capture skipped (${selectedLearningMode ?? "not requested"}).`; + } + const structuredGapMessage = structuredGate.errors.length > 0 || structuredGate.missing.length > 0 + ? ` Missing structured evidence criteria: ${structuredGate.missing.length > 0 ? structuredGate.missing.join(", ") : "none"}. ${structuredGate.errors.length > 0 ? `Structured evidence errors: ${structuredGate.errors.join("; ")}.` : ""}` + : ""; + const message = status === "complete" + ? `Plan/spec approved, ${runner} child workflow ran, and structured review evidence gate is sufficient.${learningMessage}` + : `Plan/spec approved and ${runner} child workflow ran, but structured review evidence gate returned ${reduction.decision}: ${reduction.reason}${structuredGapMessage}`; + + await writeFinalReport({ status, mode, runner, prompt, message, implementation }); + await writeFinalManifest({ manifestPath, runId, startedAt, input: { ...baseInput, approval_decision: approvalDecision, revisions, selected_learning_mode: selectedLearningMode }, finalReportPath, artifacts }); + + return outputRecord({ + status, + mode, + runner, + approved: status === "complete", + artifact_dir: displayPath(artifactDir), + manifest_path: displayPath(manifestPath), + message, + brainstorm_path: brainstormPath ? displayPath(brainstormPath) : undefined, + plan_path: displayPath(planPath), + spec_path: displayPath(specPath), + approved_spec_path: displayPath(approvedSpecPath), + implementation, + review_report_path: reviewReportPath ? displayPath(reviewReportPath) : undefined, + learning_doc_path: learningDocPath ? displayPath(learningDocPath) : undefined, + }); + }) + .compile(); + +export { compoundEngineeringWorkflow }; +export default compoundEngineeringWorkflow;