Conversation
# Conflicts: # BACKWARD_COMPATIBILITY.md # src/server/server.ts # src/skills.ts # src/workflows/run.ts # web/app/src/api/client.ts # web/app/src/api/queries.ts # web/app/src/routes/new-task.tsx # web/app/src/routes/settings/agents-section.tsx # web/app/src/routes/settings/settings.test.tsx
Composer: drop the execution-profile subtabs — every multi-model run is a custom lineup. Readiness moves onto each model pill, provider failures get a readable summary with the raw text behind a disclosure, and a probe in flight no longer disables the prompt box. Run view: one shell width for header/status/body (they were 820 vs 1120 and visibly misaligned), one status line plus a grouped timeline in place of the 2620px-wide horizontal phase rail, and a run rail (phase/council/models) beside the transcript on every harness tab. Review tab: leads with the run outcome instead of a round verdict, blocking findings render as cards with deduped reviewer attribution, the council table holds reviewers only, and a drawer exposes each reviewer's actual review. Review fixes: - verdicts derived from findings + a pre-stage invariant, so a reviewer that labels its own blockers "approve" can no longer stage a run as ready - ActiveRun.interruptHandlers, so cancel reaches harness.mjs after phase one - harness runtime sealed out of the worktree, sha256 verified before every op - one providerFamilyOf shared by driver, server and the web mirror - stdin 'error' listeners; SIGKILL escalation keyed on group liveness - accept-contested returns decisions, so publishing actually unlocks - presets keep advisor reviewers; setup dialog names the real gap Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Takes the packages/* split (open-mercato#695): src/ → packages/cezar/src/, web/app/src/ → packages/web/src/, and types now come from @open-mercato/cezar-api-client rather than @/api/types. Conflicts resolved: - imports across the harness UI repointed to the api-client package - vendor/ moved to packages/cezar/vendor/, because bundledSkillsDir() resolves <package>/vendor/skills and the split left the tree behind at the repo root; the generator and the tarball file list follow it - AGENTS.md / BACKWARD_COMPATIBILITY.md took main's new paths with our harness and vendored-skills rows re-applied on top Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The run view wasted most of a wide display: a fixed 1240px cap left large symmetric margins while the content inside was the thing complaining about space. The harness shell is fluid now (capped at 1920 so an ultrawide does not turn the transcript into a billboard) and the rail widens with it at 2xl. Horizontal strips no longer rely on the native overlay scrollbar, which is a ~3px pointer target that only catches near its top edge. HScroller hides the bar, pins the y axis — which is what drew the stray corner square at the end of the strip — and expresses overflow as arrow buttons that appear only when there is somewhere to go. It degrades without ResizeObserver rather than throwing. Also removes HarnessPhaseRail: the status bar replaced it, and leaving the dead component behind left a second copy of the same scrollbar problem. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ar of the header The rail pinned at 12px while the sticky run header is ~185px tall, so its top tile sat behind the header and scrolled out of reach. The header now measures itself and publishes --run-header-h, which the rail uses for both its sticky offset and its max height — the height is genuinely dynamic (title wrap, step rail, notes panel), so it cannot be a constant. A council reviewer is the expensive, judgement-carrying part of a run, and the only trace of one was its verdict and a finding list: you could not read what it was asked or what it said. Reviewer prompts are persisted per invocation now (advisors point at the shared criteria file rather than N copies), served by GET /api/runs/:id/harness/invocations/:invocationId, and rendered in the drawer as one turn per attempt — prompt, then reply, with the JSON result formatted as prose. Both texts are read only from within the run's own artifact directory and capped, and an older ledger reports "not recorded" rather than failing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The mockup's phase tile is a timeline: a connector line running between the status dots, clipped at the first and last row so nothing dangles, with each dot masked by a card-coloured ring so the line never appears to pass through a status. The implementation had dropped it, leaving five unrelated rows. Also restored from the same mockup: per-phase duration on the right, the retry count inline with the label rather than as a separate badge, the "Full timeline · N phases" affordance in the tile, and the role legend closing the Models tile. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Composer (mockup 03) was missing most of its row design: the provider family beside each picked model, the effort dial as a five-segment control in its own column instead of a dropdown chip lost among the pills, and the three-column role grid. Reviewers get one line each rather than a wrap of chips — the mockup drew a single effort control for that row, but effort is per-reviewer in the data model and collapsing it would have removed a real choice. Run timeline (mockup 04) was missing its totals row, its per-stage progress bar (one segment per phase, tinted by outcome, so a stage that burned its time on retries looks different from one that ran clean), and council phases expanding into the reviewers that ran in them. Two bugs in that expansion, caught on real data: matching a council by round alone defaulted every unlabelled phase to round 1, so Preflight and Capture sprouted the whole council; and the per-reviewer duration was the model's whole-run total, which read as though every round cost 81 minutes. It now comes from the invocations recorded against that phase. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
# Conflicts: # BACKWARD_COMPATIBILITY.md
# Conflicts: # packages/web/src/routes/task-thread/run-header.tsx # packages/web/src/routes/task-thread/step-rail.tsx # packages/web/src/routes/task-thread/task-thread.tsx
Unifies the two termination reworks: main's terminatedByCezar exit classification (open-mercato#703) now rides the branch's tree-wide teardown (terminateAgentProcessTree), for the claude CLI runner and the codex app-server alike. Composer keeps the Multi-model tab split while adopting main's worktree-toggle simplification (selectedWorkflow removed). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- untrack .serena/ (personal Serena MCP state) and gitignore it + .DS_Store - launch.json: remove the fresh-machine config bound to a session tmp path - vendoring spec: genericize the author-local skills checkout path - package.json: restore the literal em-dash the \u2014 escape replaced Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1,254 full-line narration comment lines removed across 84 TS/TSX/MJS files (branch-added lines only — main's code untouched). Kept, per the AGENTS.md convention: spec/issue/run-id citations, JSDoc on exported APIs, and functional directives. The generated vendor/ tree is excluded. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ncurrent reviewers, reviewer modal A council below quorum no longer kills the run. With survivors it PAUSES: outcome.pendingDecision carries the failed reviewers and the two exits, and POST /api/runs/:id/harness/council-decision records the user's choice as a ledger row and resumes through ordinary recovery — retry extends the attempt budget (runner loops and the advisor op alike), proceed continues with the survivors, loudly. Only an all-failed council leaves retry as the single exit. Completed reviews are never re-paid either way (hash-bound reuse). The runner-session pool and the advisor council now launch together instead of back to back — they share nothing but the read-only subject, so the slower group no longer adds the faster one's wall clock. The reviewer drill-down moves from the cramped right-hand sheet to a centered modal (reviewer-modal.tsx), with retry/proceed one click from a failure; the rail's Council tile grows a decision card for the paused state. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…aph, working-model spinners, step panel with transcript jumps Four multi-modal cockpit refinements (user feedback 2026-07-29): - The reviewer modal's FINDINGS section follows the selected attempt, parsed from that attempt's result artifact (falls back to the council row when an attempt wrote no parseable review). - The run timeline opens in a bounded modal instead of rendering at the top of the page under the rail, and gains a wall-clock Gantt: one time axis, one status-colored bar per phase, quarter gridlines, +0s→total labels — where the run's hours actually went. - The rail's Models tile spins on every model with a running invocation. - The header's step strip expands into a floating, scrollable overlay panel instead of shoving the transcript down, and every step row jumps the transcript to where that phase started. The anchor is a synthetic step-start marker the thread reducer now pushes on step transitions (phases share turns, so row keys never carried the step) — rendered as a subtle phase divider, and corrected-scrolled under virtualization. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…he contested publish gate
The om harness this ports runs review > fixes and moves on after the last
fixes; cezar instead ended exhausted runs 'contested', gating Finish/Draft PR
behind an explicit risk acceptance — and a spec council that failed to
converge in its 3 rounds marked the WHOLE run contested even when every later
council approved the build.
Now: when the review budget runs out with blocking findings, one closing fix
phase answers them (bounded, prompt says it will not be re-reviewed), the
validation gate re-proves the tree (still the objective authority — blocking
regressions fail the run), and the run stages READY. The findings and the
fixer's response ride the handoff ('## Closing fixes') and a ledger decision.
Spec non-convergence keeps its loud UNRESOLVED note, threads into every later
review prompt, and lands in the handoff ('## Spec council — open questions')
— but no longer touches the outcome. The high-assurance packet path keeps its
contested gate on purpose; accept-contested stays for legacy runs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The harness predeclares its seven base steps and creates the rest (baseline gate, council rounds, fix rounds) while conducting; appending those after the predeclared list displayed 'Stage · step 7 of 19' on a run whose stage ran last, with the spec council rounds listed after Review. A new step is created the moment execution reaches it, so addStep inserts it before the first step that has not started yet — chronological by construction. Continue on a finished run still appends at the tail (everything earlier is terminal). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…und-consistent phase names
Existing harness runs recorded their dynamic steps appended after the seven
predeclared ones — the display now orders steps by the ledger's phases (always
appended in execution order), so Baseline gate sits at step 3 and Stage last
even on history; not-yet-started steps keep their stored order at the tail.
New runs also name their rounds consistently: the first councils are
'Spec council (round 1)' / 'Council review (round 1)' / 'Review (round 1)',
revisions and fixes say which round's feedback they answer ('Revise spec
(round 1 feedback)', 'Fixes (round 1 feedback)') — 'Fix (round 2)' with no
round 1 read as a missing phase when round 1's fix was Implement itself.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts: # BACKWARD_COMPATIBILITY.md # package-lock.json # packages/api-client/src/dto/types.ts # packages/cezar/src/core/ask.test.ts # packages/cezar/src/core/claude-cli-runner.ts # packages/cezar/src/core/codex-app-server-runner.ts # packages/cezar/src/core/codex-app-server-transport.ts # packages/cezar/src/core/opencode-server-runner.ts # packages/cezar/src/runs/store.ts # packages/cezar/src/server/models-api.test.ts # packages/cezar/src/server/server.ts # packages/cezar/src/skills.ts # packages/cezar/src/workflows/run.test.ts # packages/cezar/src/workflows/run.ts # packages/web/e2e/command-palette.e2e.ts # packages/web/e2e/github.e2e.ts # packages/web/e2e/project-groups.e2e.ts # packages/web/src/api/client.test.ts # packages/web/src/api/client.ts # packages/web/src/routes/new-task-form.test.ts # packages/web/src/routes/task-thread/run-header.tsx
# Conflicts: # packages/web/src/routes/task-thread/run-header.tsx # packages/web/src/routes/task-thread/task-thread.tsx
# Conflicts: # BACKWARD_COMPATIBILITY.md # packages/cezar/src/core/ask.test.ts # packages/cezar/src/core/ask.ts # packages/cezar/src/core/codex-app-server-runner.test.ts # packages/cezar/src/git-worktree.ts # packages/cezar/src/runs/store.test.ts # packages/cezar/src/server/health-topic.test.ts # packages/cezar/src/server/server.ts # packages/cezar/src/workflows/run.ts # packages/web/e2e/agents-dock.e2e.ts # packages/web/src/api/queries.ts # packages/web/src/routes/new-task-form.test.ts # packages/web/src/routes/new-task-form.ts # packages/web/src/routes/new-task.tsx # packages/web/src/routes/settings/agents-section.tsx # packages/web/src/routes/settings/settings.test.tsx # packages/web/src/routes/task-thread/follow-up-engine.tsx # packages/web/src/routes/task-thread/run-header.tsx # packages/web/src/routes/task-thread/task-thread.tsx
…, off by default The staged multi-model surface is now opt-in per repository via "multiModel": true in .ai/cezar/config.json: - the workflow catalog omits harness-fix-issue / harness-implement-feature, - a direct harness POST /runs and POST /harness/probe answer 409 with the real reason, GET /harness/status stays readable and reports enabled, - the composer renders the plain Task surface (no mode tablist) and the Settings nav hides Harness; the section route stays registered and explains the gate on a direct URL, - run-scoped harness reads/decisions stay live and revival reads the run's persisted workflowDef, so flipping the flag never strands an in-flight run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
🤖 |
pat-lewczuk
left a comment
There was a problem hiding this comment.
🔍 Code Review: feat(harness): add stage-only multi-model orchestration
🎯 Summary
This PR adds an opt-in, stage-only multi-model harness: a durable phase graph (preflight → capture → specification → spec council → implementation → validation → final council → bounded repairs → exact staging) conducted from packages/cezar/src/harness/driver.ts, executed against a sealed copy of the vendored cez-harness runtime, and surfaced through a new project-scoped API family plus a Multi-model composer tab, Settings section and task-harness cockpit route. It is 219 files, +38,320/−604, and I reviewed it as a code PR (not spec-only) against the full changed-file set, with attention concentrated on the safety boundary (stage-only enforcement, the Claude guard, the seal, the ledger-backed publish gate), the public contract surface, and the changes this PR makes to existing code paths.
The engineering standard here is high and unusually well-evidenced. The safety story is layered rather than trusted once: the publish gate lives server-side in harnessPublishBlockReason and refuses POST /runs/:id/git/push and POST /runs/:id/pr until the durable ledger records a staged stage plus a ready outcome (or a contested outcome explicitly accepted, cross-checked against a matching decisions entry rather than a bare flag); the runtime is copied out of the model-writable worktree and re-verified by digest before every op instead of being trusted once; commandStage in harness.mjs validates every allowlist path for absoluteness, traversal and .git targeting, verifies refs and reflogs for drift, and refuses any staged path outside the git-derived allowlist and any residual unstaged file; advisor family is canonicalized from trusted configuration in advisor-identity.ts so a client cannot relabel a model to manufacture council diversity; and mechanicalVerdict derives the review verdict from findings rather than believing a reviewer that says "approve" while listing a blocker. BACKWARD_COMPATIBILITY.md is updated additively and correctly, the new contract schemas in packages/contract/src/harness.ts are purely additive, the feature is gated off by default behind multiModel with a documented no-strand story for in-flight runs, and the comment density explains why rather than what throughout — several comments cite the specific run id that motivated the rule. Test coverage on the new code is genuinely thorough (driver, ledger, quorum, probe, safety, staged-only guard, certification, profile plan, reviewer binding, plus web-side route tests).
The problems I found are not in the harness itself. They are in the three-line-per-call-site refactor of packages/cezar/src/git-worktree.ts that this PR does along the way — replacing the index-mutating git add -N . with a scratch GIT_INDEX_FILE. That refactor is correct in intent and is the right fix for the run 5f6fe8ae race, but it drops the cleanup half of the pattern it copied from server/git-changes.ts, and it omits the scratch environment on one of the two git runners inside worktreeShortstat — which is exactly the write the change exists to eliminate, on a hot path that runs at the end of every agent turn. Plus the head currently conflicts with origin/main.
Verdict
❌ request changes — the head does not merge cleanly into origin/main (blocker), and the git-worktree.ts scratch-index refactor leaks a temporary git index file on every diff read and still writes the real index on the repointed-worktree path (two majors). None of these are in the harness feature itself; all three are small, local fixes, after which I expect this to approve.
🧪 Validation Gate
Run in an isolated worktree at the PR head (f3b78f31) after npm ci.
| Command | Status | Notes |
|---|---|---|
npm run typecheck |
✅ PASS | All four projects (contract, api-client, server, web) clean. |
npm test |
✅ PASS | 6,648 tests. Two full-suite runs each reported a handful of failures, but they are not attributable to this PR: one set (git-worktree.test.ts, git.test.ts, health-forge.test.ts, projects-api.test.ts, git-changes.test.ts, automations-api.test.ts) is an artifact of my sandbox's TMPDIR living inside a git repository, which makes "outside a git repository" assertions fail; with a clean TMPDIR the run reduced to 3 failures in files this PR does not touch (automations-gate.test.ts, repo-git.test.tsx, task-changes.test.tsx), and all three pass in isolation, so they are load-sensitive flakes rather than regressions. |
npm run test:unit |
✅ PASS | 36 passed, 0 failed. |
npm run build |
✅ PASS | Server, web, and check:pack (592 files, 91 under web/dist). |
npm run test:package |
✅ PASS | 15 installed-tarball/release tests. |
CI on the PR is green: the only reported check, license/cla, is SUCCESS, and no check is pending. This review therefore does not depend on any outstanding CI signal.
Findings
⛔ Blocker
The PR head does not merge cleanly into origin/main. GitHub reports mergeable: CONFLICTING / mergeStateStatus: DIRTY, and a local git merge-tree origin/main HEAD confirms exactly one conflicting file: CHANGELOG.md. The branch is 49 commits ahead and 1 commit behind origin/main, and main has since added its own ## ✨ Features entry at the top of the Unreleased section, which collides with the entry this PR adds in the same place. This is a blocker on its own terms — the diff being reviewed is not the diff that will merge — but it is mechanical: merge origin/main and keep both feature bullets under the single ## ✨ Features heading, newest first. Nothing else in the 219-file diff conflicts.
⚠️ Major
packages/cezar/src/git-worktree.ts:68 — scratchIndexEnv never deletes the temporary index file it creates, leaking one file per diff read for the life of the process. Every call allocates join(tmpdir(), 'cez-wt-scratch-<pid>-<seq>'), seeds it with git read-tree HEAD, populates it with git add -N ., and returns it — and nothing ever removes it. The pattern this was copied from does clean up: packages/cezar/src/server/git-changes.ts:314 runs rmSync(scratchIndex, { force: true }) in a finally. The call sites make this unbounded rather than theoretical. worktreeShortstat is invoked from RunManager.recordTurnEnd (packages/cezar/src/workflows/run.ts:3861), which runs at the end of every agent turn of every run; worktreeDiffStat is invoked inside a .map over runs at packages/cezar/src/server/server.ts:5126, so one request over N runs leaks N files; and worktreeDiff is on GET /api/v1/runs/:id/diff (server.ts:4875), which the cockpit polls. A seeded index is proportional to the repository's tree, so on a large repo a long-lived cezar server accumulates tens of thousands of multi-megabyte files in $TMPDIR — on a machine where /tmp is a tmpfs that is RAM. Fix: wrap each caller (or scratchIndexEnv itself, by returning a disposable handle) so the file is removed in a finally, mirroring git-changes.ts; note the file must survive until the last git call that uses the env, so the cleanup belongs at the end of worktreeDiff / worktreeDiffStat / worktreeShortstat, not inside the helper's own body.
packages/cezar/src/git-worktree.ts:613 — worktreeShortstat hands resolveTaskDiffBase a git runner without the scratch-index environment, so the read path still writes the real index. Line 611 obtains the scratch env and line 617 correctly uses it for the reported git diff --shortstat, but line 613 passes (args) => git(worktreePath, args) — the plain runner. resolveTaskDiffBase is not a pure ref lookup: when taskBranch is set and HEAD has been repointed onto another branch, it calls changedLines twice, and changedLines runs git diff --shortstat <ref>, which refreshes and rewrites the index it reads. packages/cezar/src/git-diff-base.ts:143-148 states this requirement explicitly — "Callers that keep their diffs off an index they do not own … must therefore hand this module a runner carrying that same environment" — and git-changes.ts:290 complies (resolveTaskDiffBase((args) => git(dir, args, env), …)). Two consequences follow. First, on any repointed worktree (review and QA runs, which are precisely the ones that repoint) the "read paths must not write ANY index" invariant this PR sets out to establish is still violated, at end-of-turn frequency — the same class of race as run 5f6fe8ae, and it can still trip commandStage's residual check in harness.mjs, which reads git status --porcelain and rejects any entry whose second status column is not a space (an intent-to-added file shows exactly that). Second, the anchor decision is measured against a different index than the number that is finally reported, so the two probes can pick an anchor the reported stat then contradicts. Fix: pass (args) => git(worktreePath, args, env) on line 613, matching git-changes.ts.
🔹 Minor
packages/cezar/src/harness/claude-guard.ts:15 — the stage-only PreToolUse guard fails open on its own internal errors. const worktree = realpathSync(process.env.CEZ_HARNESS_WORKTREE) and the outputDir line below it run at module top level, outside any try, and realpathSync is typed and behaves as throwing on undefined input or a vanished path. Claude Code's hook protocol treats exit code 2 as "block the tool call" and any other non-zero exit as a non-blocking error that is surfaced but lets the call proceed — so if either environment variable is missing, or the worktree is removed while the session is live, the guard crashes with exit 1 and the very next Edit/Write/Bash runs unguarded. The sandbox layer still stands behind it on both platforms, so this is defence-in-depth rather than the only boundary, but a guard should fail closed. Fix: wrap the two realpathSync calls (and ideally the whole body) so any failure routes through the existing refuse() helper, which already exits 2.
packages/cezar/vendor/skills/cez-harness/scripts/harness.mjs:2206 — commandStage reads staged paths without -z, so a path git chooses to quote makes staging fail with a misleading error. git diff --cached --name-only C-quotes any path containing non-ASCII bytes, a double quote, or a backslash ("caf\303\251.txt") unless -z is passed or core.quotePath is off. The same file already knows this — line 1375 uses git diff --cached --name-only -z and splits on NUL. Here the quoted spelling will not match any entry in paths (which came from validateStagePath on the raw allowlist), so the unexpected check throws Staged paths outside allowlist: "caf\303\251.txt", and if it somehow got past that, samePathSet(stagedPaths, [...allowlist]) in packages/cezar/src/harness/driver.ts:5151 would reject it as "staged paths do not exactly match the complete git-derived handoff". The behaviour is fail-closed, which is the right direction, but a harness run that touches a single accented or space-and-quote filename can never produce a handoff, and the error blames the allowlist rather than the encoding. Fix: use -z and NUL-split, as line 1375 already does.
**packages/cezar/src/harness/runtime.ts:267 (copyTreeByValue) — the seal walks the model-writable tree following symlinks, with no cycle or depth guard.** sealHarnessRuntimecopies/.claude/skillsout of the worktree, andcopyTreeByValuedeliberately usesstatSyncrather thanlstatSync so links are dereferenced ("copy by value") — which is the right call for the threat it is defending against. But a symlink that points at an ancestor (.claude/skills/cez-harness/x -> ../../..) then reports isDirectory()forever, and the function recurses without bound: the run's host process hits either aRangeError: Maximum call stack size exceededorENOSPC, and since this runs inside the cezar server rather than a child, an uncaught throw here is a server-level failure rather than a run-level one. The premise of the seal is that the worktree contents are untrusted, so the guard should be explicit. Fix: track visited real paths in a Set` and refuse (or skip) on revisit, and/or cap the recursion depth.
.ai/HANDOFF-feat-multi-modal.md — a branch-scoped working note is committed to the permanent tree, and is already stale. The file opens with "Branch is pushed to fork/feat/multi-modal, currently d768f071, 46 commits ahead of origin/main … 215 files, +37.7k/−583"; the head under review is f3b78f31, 49 commits ahead, 219 files, +38,320/−604. .ai/specs/ is the durable home for design content and this PR already adds seven spec documents plus docs/multi-model-harness.md there, so the handoff's lasting value is covered. It will only drift further after merge. Suggest dropping it from the PR, or folding anything still worth keeping into docs/multi-model-harness.md.
💅 Nit
**packages/cezar/package.json:46 — @anthropic-ai/sandbox-runtimeis a hard dependency of the published package but is only reachable on macOS.**sandboxRuntimeCliPath()is called exclusively from theuseStandaloneSandbox (platform === 'darwin') branch of createClaudeStageOnlySettings, yet every Linux and Windows install of @open-mercato/cezarnow downloads it. Pinning it exactly at0.0.61is the right instinct for a pre-1.0 package; consider moving it tooptionalDependencies(with the existingrequire.resolve` already positioned to fail loudly if it is genuinely needed and absent) so non-macOS installs stay lean.
.claude/launch.json — an editor launch configuration lands in the repo root, unrelated to the PR's stated scope. It is harmless and plausibly useful to other contributors, but it is not multi-model orchestration and it arrives without a note in the PR body. Either mention it in the description or split it out; author's call.
💥 Breaking Changes
- No exported/public symbol removed or renamed without a deprecation path — the diff removes no
exportfrompackages/contract/srcorpackages/api-client/src;packages/contract/src/harness.tsis entirely new. - No function signature changed in a breaking way —
git()ingit-worktree.tsgains an optional trailingenvparameter;worktreeShortstat'soptswas already optional.RunManager.startRunis unchanged in arity. - No required type field removed or narrowed —
RunRecord.harness(packages/cezar/src/runs/store.ts:326) is.optional(), so pre-existingruns.jsonfiles still parse, andmultiModelinconfig.tsis.default(false).catch(false), so a malformed value degrades rather than throwing. - No HTTP route URL removed or renamed; no method changed for an existing operation — the six harness routes are new and project-scoped;
harnessRoutesis mounted alongside the existing families. - No field removed or retyped in an existing response shape —
GET/PUT /api/v1/configgainsmultiModeladditively,GET /api/v1/healthis untouched, andPOST /api/v1/runsgains an optionalharnessbody object. - No event or message name renamed or removed; no payload field removed — the five
harness.*UiEventvariants inpackages/api-client/src/protocol/ui-events.tsare added to the union; no existing variant is touched. - No CLI command or flag renamed or removed; no machine-parsed output format changed.
- No database table or column renamed or removed — the project uses plain files, and
runs.json's schema change is the optional field noted above. - No config key renamed and no default changed silently —
multiModeldefaults tofalse, and the PR documents that flipping it off cannot strand an in-flight run because revival reads the run's own persistedworkflowDefand the run-scoped harness reads stay live. - Where a contract had to change: old surface kept working through a deprecation window, with migration notes —
BACKWARD_COMPATIBILITY.mdgains an accurate additive entry for the harness route family, themultiModelgate and its 409 semantics, and the certification field onGET /api/v1/harness/status, which is exactly the discipline this repository asks for.
One item worth calling out explicitly rather than as a checkbox: POST /runs/:id/messages now behaves differently for harness runs — manager.deferMessage is skipped and the text is queued as a phase-boundary message answering {queuedForPhase: true} instead of {deferred: true}, and images are refused with a 409. That is a new shape on an existing route, but it is reachable only for runs carrying the harness stub, which can only exist once the feature flag has been on, so no pre-existing client can observe the change. It is correct as designed; it just deserves a line in BACKWARD_COMPATIBILITY.md next to the harness entry so the next reader does not have to derive it from the diff.
🧪 Test Coverage
Coverage on the new surface is strong and specific rather than nominal. packages/cezar/src/harness/ ships tests for the driver (3,474 lines exercising phase recovery, council folding, fix rounds and the stage gate), the ledger's valid/missing/corrupt/unsupported branches, council quorum including the two-family rule and the degraded-but-ok path, the probe and its live transports, certification resolution including the stale window, profile planning, reviewer binding and the OpenCode seating refusal, model-family resolution — with a test that reads packages/web/src/routes/new-task-form.ts and asserts the browser mirror of FAMILY_BY_NAME and GATEWAY_PREFIXES matches the server table byte-for-byte, which is exactly the right way to hold a duplicated table honest — plus dedicated safety.test.ts, staged-only-guard.test.ts and seal-runtime.test.ts for the boundary. packages/cezar/src/server/harness-api.test.ts (1,007 lines) covers the route family, and the web side adds tests for the composer tab, readiness, the harness rail, timeline, state and the settings section. packages/web/e2e/agents-dock.e2e.ts was rewritten to exercise scoped routing, progressive history, real overflow and viewport restoration rather than being weakened; the other e2e edits are small waits for hit-testing to settle, not relaxed assertions, and I did not find an assertion that was deleted to make a red test green.
The gaps map onto the findings above, and each is a small, well-located test:
- No test asserts that the scratch index file is removed. Add a case to
packages/cezar/src/git-worktree.test.tsthat snapshotsreaddirSync(tmpdir()).filter(f => f.startsWith('cez-wt-scratch-'))before and afterworktreeDiff,worktreeDiffStatandworktreeShortstatand asserts the set is unchanged — that pins the leak fix and keeps a future refactor from re-dropping thefinally. - No test asserts that
worktreeShortstatleaves the real index alone on a repointed worktree.git-changes.test.tsalready has the equivalent shape forcollectChanges; add the sibling togit-worktree.test.ts— create a worktree, repoint HEAD onto another branch, recordgit status --porcelain(and the index mtime) before and afterworktreeShortstat, and assert no untracked file was promoted to intent-to-added. packages/cezar/src/harness/staged-only-guard.test.tscovers what the guard refuses but not what it does when it cannot run. Add a case that invokes the guard script withCEZ_HARNESS_WORKTREEunset and asserts the exit code is2, not1— that is the assertion that makes "fails closed" a property rather than an intention.commandStagehas no case for a path git would quote. Add one to the harness runtime's stage tests using a filename with a non-ASCII character (and one with a space and a double quote) and assert the handoff stages successfully rather than reporting "Staged paths outside allowlist".seal-runtime.test.tshas no adversarial input. Add a case that plants a symlink cycle under.claude/skills/and assertssealHarnessRuntimereturnsnull(or skips the link) instead of recursing.
Reviewed by om-auto-review-pr. autofix: skipped (not my PR — re-run with --autofix to fix it here).
|
🏷️ label rationale
|
|
👤 Handing this back to @haxiorz — changes requested. Three things stand between this and an approval, and none of them are in the harness itself:
The minors (guard fail-open, No fixes were pushed to this branch — it belongs to another author and this run was not invoked with |
|
🤖 1 blocker (merge conflict with |
|
🤖 |
🤖
|
|
🤖 |
# Conflicts: # CHANGELOG.md # packages/web/e2e/new-task.e2e.ts # packages/web/e2e/plan-mode.e2e.ts # packages/web/src/routes/new-task.tsx
|
@pat-lewczuk, please re-review the latest head, The findings from your review are addressed:
The original regression tests were checked against the pre-fix code and fail on the reported defects. Validation on the updated branch:
This is ready for another review pass; the focused browser checks do not replace the repository's human QA gate. |
Goal
Add an opt-in, stage-only multi-model harness that separates implementation from independent judgment, runs bounded validation and repair phases, and leaves every successful run at Cezar's existing human review gate.
Problem and root cause
The original single-agent path could not provide an independently judged, quorum-based implementation flow. This PR adds that flow behind the default-off
multiModelcapability.The requested-changes review also found three issues in the first implementation:
new-task-harnesswas statically imported by the eager New Task route even though multi-model mode is opt-in;What changed
multiModelconfiguration, off by default, with a synthesized safe harness config when.ai/agentic.config.jsonis absent.The matched zero-config evaluation delivered 5/5 contained, tested, green-gate multi-model changes versus 2/5 contained and 4/5 green-gate single-agent changes. All 90 reviewer invocations completed and produced 401 findings; the multi-model arm cost 2.9× more and took 6.5× longer wall-clock.
Validation
npm run typecheck— passed in all workspaces; the contract inliner completed for 13 files.npm test— passed: 366 files and 6,896 tests.npm run test:unit— passed: 35 tests, with one expected platform skip becausesetsidis unavailable.npm run build— passed for service and cockpit;check:packpassed with 612 package files, including 102 underweb/dist.npm run test:package— passed: 16 installed-tarball and release tests.git diff --check— passed.The configured validation gate was run twice in repository order; the second complete run followed the final test edit and passed every command.
A prior full browser comparison on this PR produced 196 passed / 13 failed / 6 skipped, versus 31 failures on an exact detached
origin/mainbaseline. The residual failures reproduced on the base branch. That broad browser comparison was not rerun for this remediation commit; the affected New Task path was covered by its focused route suites and production build, and remains marked for manual QA.Automated review
om-code-review— approve after one remediation round resolving one blocker, one major finding, and one minor finding.Breaking changes
None. No existing route, response schema, environment variable, dependency, migration, or default workflow behavior changed.
Labels and delivery
review— the requested changes are implemented and the branch is ready for maintainer re-review.feature— the PR introduces the optional multi-model orchestration capability.needs-qa— provider-backed execution and the lazy-loaded cockpit interaction should be exercised manually before merge.priority-medium— this is an ordinary feature delivery rather than a release outage or emergency fix.risk-high— the PR spans orchestration, credentials, process execution, API behavior, and the cockpit, so it warrants deep review despite the default-off gate.Manual QA
new-task-harnessasset is not requested.review, stages only certified paths, and creates no commit, push, PR, or merge.Evidence
reviewwithout publishing.Documentation
docs/multi-model-harness.md.ai/specs/2026-07-23-harness-orchestration.mdand follow-ups.ai/specs/2026-08-23-multi-model-eval.mdCHANGELOG.md