feat(gate-findings): frame findings as hypotheses at evaluator-fed fix stages by aaddrick · Pull Request #185 · aaddrick/ticketmill

aaddrick · 2026-07-28T19:47:21Z

Closes #167

Summary

Applies the verify-then-adopt-or-rebut framing already used at the two contrarian revision stages to the three evaluator-fed fix stages (quality-fix, test-quality-fix, pr-fix), while leaving the two oracle-fed stages (test-fix, browser-fix) untouched since they already carry a correct anti-rebuttal guard that this framing would invert.

Implementation

Added rebutted: [{finding_id, evidence}] to FIX_SCHEMA (not in required), mirroring how issues stays out of REVIEW_SCHEMA.required.
New shared FINDING_HYPOTHESIS_ASK constant beside HANDOFF_ASK/COMMIT_SHA_ASK, rendered only when structured findings are present, and wired into exactly the three evaluator-fed fix prompts (quality-fix, test-quality-fix, pr-fix).
Pure helpers normalizeRebuttals, contestedBlock, and retypeGateDisposition, all fail-toward-existing-behavior on malformed input.
Deterministic rebuttalOnly predicate (rebutted findings present, fixes_applied and files_changed both empty) evaluated uniformly at all three gates, after existing handoff/commit-SHA bookkeeping so those checks still fire.
Quality loop and test loop: rebuttal-only rounds exit their loop via a dedicated flag (kept separate from degraded) rather than falling through to the iteration cap path, and increment ctx.metrics.rebuttal_only_rounds.
pr-review merge gate: continues once on a rebuttal-only round, then halts to needs_human with a carried-unresolved disposition and a Verification Gaps line on a second consecutive rebuttal-only round.
Kept the lockstep .claude/workflows/ copy in sync via node scripts/lint-engine.js --fix.
Documented the framing in docs/architecture/gate-hygiene.md.

Key decisions

Framing renders conditionally on findings actually being present, so the prose-fallback path (no finding ids) never sees rebuttal instructions it can't act on.
Only findings rendered with a bracketed id are rebuttable; unlabeled findings must be fixed or explained in the summary.
rebuttalOnly is evaluated after collectNotes/collectPostedCommit at each site so handoff notes and the fabricated-SHA check still run first.
tallyTouches is a verified no-op on an empty rebuttal array, so metrics accounting stays correct when nothing was rebutted.
Oracle-fed stages (test-fix, browser-fix) were explicitly left untouched to preserve their existing "fix the real defect, do NOT delete/weaken assertions" guard.
Stripping reviewer attribution from fix prompts was explicitly dropped from scope.

Testing

45 new unit tests added to tests/gate-findings.test.js covering the new primitives.
Full suite passes: node --test.

Token usage (approximate, this issue only): 516192 output tokens

…ages (#167) Task 1 of #167: behaviour-neutral primitives that let a fixer treat a reviewer's finding as a hypothesis to verify rather than a command to obey. - FIX_SCHEMA.rebutted ({finding_id, evidence}), not in required — mirrors REVIEW_SCHEMA.issues staying out of its own required list. - FINDING_HYPOTHESIS_ASK beside HANDOFF_ASK/COMMIT_SHA_ASK: verify before acting, record disproof in `rebutted` if wrong, rebut only what you didn't fix, only bracketed-id findings are rebuttable, and a gate-agnostic consequence clause (no immediate-exit claim, since pr-review continues into another review round). - normalizeRebuttals(raw, findings): pure, fails toward today's behavior on every drop (non-array, blank finding_id/evidence, unrendered id); carries the matched finding's summary through. - contestedBlock(ctx): pure, renders rebutted-but-not-adjudicated findings back to the next reviewer with an inverted (vs settledBlock) closing contract, and states the iteration-2+ do-not-re-flag instruction doesn't apply to a contested entry. Never calls settleDecision(). Rendered immediately after settledBlock() at the quality/spec/code review prompts; emits '' until a later task populates ctx.contested. - retypeGateDisposition(ctx, gate, from, to): pure, moves one disposition count between buckets without touching count/severity; from === to is a documented count-preserving no-op, reachable at MAX_QUALITY_ITERATIONS. - ctx.metrics.rebuttal_only_rounds, plus tests/harness.js freshMetrics() parity. None of these are wired into the fix stages' control flow yet (task 2/3); FINDING_HYPOTHESIS_ASK and retypeGateDisposition are unused in production until then. 45 new unit tests in tests/gate-findings.test.js. node scripts/lint-engine.js --fix kept .claude/workflows/ticketmill.js in lockstep. node --test: 763/763 passing.

…st loops (#167) Renders FINDING_HYPOTHESIS_ASK in the quality-fix and test-quality-fix prompts only when the reviewer/validator named structured findings, and wires normalizeRebuttals()/retypeGateDisposition()/contestedBlock() (from task 1) into both evaluator-fed fix loops so a fix that rebuts every finding and applies none can never approve its own gate: - runQualityLoop: a third loop-exit flag `rebutted`, kept separate from `degraded` so the rolling degrade window is untouched, retypes the disposition already booked for that iteration to 'carried-unresolved', records the rebuttal as contested (not settled), and rolls up to exactly one VERIFY_SKIPS line per issue (ctx.quality_rebuttals / quality_rebuttal_skip_index), mirroring the existing cap roll-up. - runTestLoop: same predicate at test-quality-fix, exits through the loop's own { ok: true } path (never ok:false) so a rebuttal-only round can never route through the 'test-loop' merge-block stage key. No gate key to retype here (test-quality books none today). The two ORACLE-fed fix stages (test-fix, browser-fix) are untouched and keep their anti-rebuttal guards — a failing test is ground truth, not a hypothesis to verify. Extends tests/quality-loop.test.js and tests/test-loop.test.js with the rebuttal-only exit, partial-rebuttal/omitted-field/blank-evidence non-triggers, the rolled-up VERIFY_SKIPS line, and scope-pinning assertions proving FINDING_HYPOTHESIS_ASK renders only at the three evaluator-fed sites and never at test-fix.

…ntinue Renders FINDING_HYPOTHESIS_ASK in the pr-fix prompt when either reviewer returned structured findings, and evaluates the rebuttalOnly predicate over the union of both reviewers' rendered finding sets. Unlike the quality and test loops (which exit their gate on a rebuttal-only round), pr-review CONTINUES: it retypes the iteration's disposition to carried-unresolved, carries the dispute into ctx.contested so the next iteration's reviewers see it, records one un-rolled Verification Gaps line, and moves on to the next review iteration instead of ending the gate. A per-call counter permits exactly one such round per issue; a second sets haltReason and falls through to the existing needs_human path, leaving the PR open. The `continue` precedes runQualityLoop so a rebuttal-only round (empty files_changed) never burns a quality gate against an untouched tree. Issue #167

…ix stages (#167) Add a "Rebuttal: a finding is a hypothesis, not a command" section to gate-hygiene.md covering what a rebuttal is and its evidence bar, why the framing is scoped to quality-fix/test-quality-fix/pr-fix and not test-fix/browser-fix, FIX_SCHEMA.rebutted being schema-wide but read at only three sites, normalizeRebuttals' fail-toward-today drops, the three per-gate exits and why only pr-review can block a merge, contestedBlock vs settledBlock's inverted contract, and rebuttal_only_rounds (a continuation, not an exit, on its first pr-review increment). Extend the gate-hygiene.md row in index.md's file map, and correct three now-stale passages in engine-internals.md that #167 falsifies: the carried-unresolved disposition enumeration (missing the rebuttal-only route), the "only findings_empty_exits distinguishes them" claim at pr-review, and findingsBlock's description of a rendered finding's job list.

aaddrick · 2026-07-28T19:50:42Z

Spec Review (Iteration 1)

Verdict: approved

PR #185 implements issue #167 in full. Verified against .worktrees/issue-167 at commit 6e35ded (branch issue-167-a-finding-is-a-hypothesis-at-the-evaluator-fed-fix), node --test green (781/781), node scripts/lint-engine.js clean, and the lockstep .claude/workflows/ticketmill.js copy byte-identical to workflows/ticketmill.js.

Acceptance criteria, checked against the diff:

The three evaluator-fed stages (quality-fix, test-quality-fix, pr-fix) render FINDING_HYPOTHESIS_ASK, conditioned on structured findings being present; the two oracle-fed stages (test-fix, browser-fix) are untouched and keep their anti-rebuttal guards verbatim — confirmed both in the diff and via dedicated scope-pinning tests (tests/quality-loop.test.js, tests/test-loop.test.js, tests/browser-check.test.js, tests/pr-review-gate.test.js) that assert the string is present/absent at each of the five fix prompts.
rebutted entries require cited evidence — normalizeRebuttals drops any entry with blank finding_id or blank evidence, and drops ids not present in the finding set actually rendered to that fixer (fabrication-proof).
The JS rule (normalizeRebuttals, contestedBlock, retypeGateDisposition) sits above the TICKETMILL-TEST-HARNESS-SPLIT marker and is unit-tested directly through tests/gate-findings.test.js (45 tests).
A rebuttal-only round (rebuts something, fixes/changes nothing) never resolves as approved: quality and test-quality exit their loop immediately with a VERIFY_SKIPS line and a retyped carried-unresolved disposition (quality) or contested-ledger entry (test-quality, which has no gate key to retype); pr-review — the one gate that can actually block a merge — never sets approved = true on a rebuttal-only round, continues once, and halts to needs_human with its own Verification Gaps line on a second consecutive rebuttal-only round. This per-gate shape (continue-once at pr-review vs. immediate-exit at the other two) matches what was explicitly reasoned through and adjudicated across the issue's own evaluate/contrarian rounds, and is documented in docs/architecture/gate-hygiene.md.
A partial rebuttal (some findings rebutted, real fixes applied) proceeds normally — explicitly tested (tests/quality-loop.test.js:643).
Omitting rebutted entirely is byte-identical to today — explicitly tested (tests/quality-loop.test.js:687, tests/test-loop.test.js:255).
node --test stays green; tests/quality-loop.test.js, tests/pr-review-gate.test.js, and tests/test-loop.test.js were all extended as required.
Documented in docs/architecture/gate-hygiene.md (new "Rebuttal" section) with its docs/architecture/index.md file-map row extended; stale passages in engine-internals.md (the carried-unresolved disposition enumeration, the pr-review distinguishing-signal claim, and findingsBlock's job-list description) were corrected to match, all falling outside the hash-frozen pipeline.md/metrics.md/failure-semantics.md files. CHANGELOG.md is not touched by this PR, but that is correct: the batch's release stage generates the changelog entry automatically from shipped-issue titles and conventional-commit types at merge time, not per-issue.
Explicitly-out-of-scope item (stripping reviewer attribution from fix prompts) was correctly left alone.

No scope creep found. No changes requested.

aaddrick · 2026-07-28T19:53:41Z

Code Review (Iteration 1)

Verdict: changes_requested (1 major, 4 minor)

Baseline is clean: node --check passes on both copies, node scripts/lint-engine.js reports the two lockstep pairs in sync, and node --test is 781/781 green. No sandbox violations (Date.now(), Math.random(), argless new Date(), filesystem/Node APIs, TS syntax) in the diff, no agentType in any agent() call, and no profile-shape change. The scoping decision (three evaluator-fed stages, not the two oracle-fed ones) is pinned by real prompt assertions in tests/browser-check.test.js:147 and tests/test-loop.test.js:281, both of which check the existing anti-rebuttal guards verbatim. Every rebuttal-only exit pushes a VERIFY_SKIPS line, so no new path skips a check silently.

Major

1. The engine grew 17,355 bytes and crossed the 92% Workflow-cap warning line that the immediately preceding commit added to prevent a repeat incident.
workflows/ticketmill.js goes 465,677 -> 483,032 bytes. On main the lint is silent; on this branch it prints WARNING — within 41,256 bytes of the 524,288-byte Workflow script cap. That threshold and that warning were both added by b591d71 ("move standing prose to docs so the engine can launch again"), which existed because v0.2.0 shipped 32,380 bytes past the cap with node --check passing, all tests passing, and CI green — a released engine that could not be run. That commit recovered 58,611 bytes of headroom by moving 36 blocks of standing commentary into docs/architecture/engine-internals.md and leaving a one-sentence pointer at each site. This PR spends 30% of that recovered headroom in a single issue, and 10,232 of the 17,985 added bytes (57%) are pure comment lines — most of it prose this same commit also wrote, in places near-verbatim, into docs/architecture/gate-hygiene.md.

The engine still launches, so this is not blocking, but it regresses the guard from one commit ago and the fix is mechanical with zero behaviour change.

Minor

2. A second rebuttal-only pr-fix round is the only terminal exit from reviewAndMerge's loop that leaves re-litigated booked. workflows/ticketmill.js:5098-5100 sets haltReason and breaks without calling retypeGateDisposition. bothNothingToFix and capReached both reach needs_human with carried-unresolved booked by construction (:5005); this one books re-litigated for an iteration that never got re-litigated, so the Gate Yield rollup at :5696-5697 over-counts relitigated and under-counts carried by one on exactly the runs a human is being asked to look at. docs/architecture/gate-hygiene.md also says of pr-fix that "retypeGateDisposition still fires here too", which is only true of the first round.

3. docs/architecture/gate-hygiene.md:540-544 overstates what normalizeRebuttals enforces. It claims the helper "enforces the same bar mechanically" as the prompt's concrete-evidence ask, and that "bare disagreement can never survive normalization." normalizeRebuttals (:2185) only rejects blank strings — evidence: 'I disagree' passes every check and becomes a rebuttal the engine acts on. The mechanical bar is non-blankness; the evidence-quality bar is prose-only.

4. contestedBlock's override clause points at an instruction that exists at only one of its three render sites. workflows/ticketmill.js:1916-1918 reads "The iteration-2+ instruction elsewhere in this prompt not to re-flag issues already addressed or accepted does NOT apply..." That instruction is in the code-review prompt (:4973) only. The quality-review prompt (:3051) has no such line, and the spec-review prompt (:4946) carries a different one ("stay consistent with your own prior spec reviews — do not reverse a prior scope approval without new information"), which the override does not name and therefore does not neutralise.

5. Nothing ever closes a ctx.contested entry. Entries are only ever pushed (:3114-3117, :3655-3658, :5102-5105); there is no removal path anywhere in the engine. contestedBlock tells the reviewer to "re-raise it as a finding this iteration — do not let it sit contested indefinitely with neither outcome" and the code comment at :1900-1901 plus docs/architecture/gate-hygiene.md:682-683 both assert that "only a reviewer (or eventually a human) closes a contested entry". No reviewer verdict closes anything: once pushed, an entry is re-rendered to every later quality review and every later pr-review iteration for the rest of the issue, still carrying the explicit "the do-not-re-flag guard does NOT apply here" override, even after a reviewer has adjudicated it. Since the quality reviewer's diff is the whole branch (git diff TARGET...HEAD), a contested entry from task 1 stays actionable, and re-raisable, through task N.

Observed, deliberately not filed

A partial rebuttal (rebut some findings, fix others) discards normRebuttals entirely at all three sites — no ctx.contested entry, no VERIFY_SKIPS line, no counter — even though FINDING_HYPOTHESIS_ASK explicitly invites that shape ("rebut ONLY the findings you did not fix"). I am not filing it: "Partial rebuttal plus real fixes proceeds untouched, which is the common and correct case" was adjudicated upstream on this issue, and tests/quality-loop.test.js:643 pins the behaviour deliberately. Flagging it here would be re-litigating a settled decision. Noting it only so the next reader knows it was seen and left alone.

…ally gap - Move the #167 rebuttal-framing comment blocks (contestedBlock, FINDING_HYPOTHESIS_ASK, retypeGateDisposition, normalizeRebuttals, and the three in-loop rebuttalOnly explainers) out of workflows/ticketmill.js into one-sentence pointers at docs/architecture/gate-hygiene.md, where the same rationale already lives — clears lint-engine's 92% size warning. - reviewAndMerge()'s second (halting) rebuttal-only pr-fix round now calls retypeGateDisposition before breaking, so gate_findings['pr-review'] never leaves a leftover 're-litigated' count on the run a human is sent to inspect. - Reword gate-hygiene.md's normalizeRebuttals claim: it enforces only non-blankness, not the prompt's concrete-evidence bar. - Make contestedBlock's do-not-re-flag override site-agnostic so it also covers spec review's "stay consistent with prior reviews" instruction. - Correct the contestedBlock comment and gate-hygiene.md to state plainly that nothing closes a ctx.contested entry today — a known gap, not the "only a reviewer closes it" claim that didn't match the code. Fixes code review findings code-i1-1 through code-i1-5 on PR #185.

aaddrick · 2026-07-28T20:03:06Z

PR Review Fix (iteration 1)

Commit: eaa7458

[code-i1-1] moved the standing A finding is a hypothesis at the evaluator-fed fix stages #167 rebuttal-framing comment blocks (contestedBlock, FINDING_HYPOTHESIS_ASK, retypeGateDisposition, normalizeRebuttals, three in-loop explainers) out of workflows/ticketmill.js into one-sentence pointers at gate-hygiene.md; lint-engine's 92% warning is gone (478,435/524,288 bytes).
[code-i1-2] the second (halting) rebuttal-only pr-fix round now calls retypeGateDisposition before breaking, so gate_findings['pr-review'] never leaves a leftover 're-litigated' count; extended the test at tests/pr-review-gate.test.js:537.
[code-i1-3] reworded gate-hygiene.md: normalizeRebuttals enforces only non-blankness, not the prompt's concrete-evidence bar.
[code-i1-4] made contestedBlock's do-not-re-flag override site-agnostic so it also names spec review's "stay consistent with prior reviews" instruction.
[code-i1-5] corrected the contestedBlock comment and gate-hygiene.md: nothing closes a ctx.contested entry today — documented as a known gap, not the "only a reviewer closes it" claim that didn't match the code.

node --test: 781/781 passing. node scripts/lint-engine.js: clean, no size warning.

aaddrick · 2026-07-28T20:06:56Z

Spec Review (Iteration 2)

Verdict: approved

Re-verified PR #185 against issue #167 at HEAD eaa7458 (branch issue-167-a-finding-is-a-hypothesis-at-the-evaluator-fed-fix) after the code-review fix round. node --test 781/781 green, node scripts/lint-engine.js clean (478,435/524,288 bytes, well clear of the 92% warning line), lockstep .claude/workflows/ticketmill.js byte-identical to workflows/ticketmill.js.

My iteration-1 approval already confirmed the PR meets every acceptance criterion in #167. This pass checks whether the intervening "PR Review Fix (iteration 1)" commit (eaa7458) preserved that, since a code-review fix round is exactly where scope creep or spec regressions get introduced silently.

Diff-audited eaa7458:

code-i1-1 (prose relocation): the standing A finding is a hypothesis at the evaluator-fed fix stages #167 rationale comment blocks moved out of workflows/ticketmill.js into one-sentence pointers at docs/architecture/gate-hygiene.md. Confirmed contestedBlock, FINDING_HYPOTHESIS_ASK, retypeGateDisposition, and normalizeRebuttals all still sit above the TICKETMILL-TEST-HARNESS-SPLIT marker (real marker at :6688), so the "JS rule is unit-tested above the split marker" acceptance criterion still holds. This is a comment-only diff — verified no logic lines changed alongside the trims.
code-i1-2 (the one functional change): reviewAndMerge()'s second, halting rebuttal-only pr-fix round now calls retypeGateDisposition(ctx, 'pr-review', prReviewDisposition, 'carried-unresolved') before its break, closing the leftover re-litigated tally gap. Matches the commit's own description; pinned by a new assertion in tests/pr-review-gate.test.js:576+ that distinguishes the continuing (iteration 1) round's retype from the halting round's. Does not change the acceptance criterion "rebuts every finding, applies none -> terminates as carried-unresolved, never approved" — if anything it makes that criterion's bookkeeping more accurate.
code-i1-3/4/5: doc-only corrections (normalizeRebuttals's non-blankness-only bar, the do-not-re-flag override's site-agnostic wording, and the "nothing closes a ctx.contested entry" gap stated plainly). No engine behavior change; these bring gate-hygiene.md back into agreement with what the code actually does, which is what issue A finding is a hypothesis at the evaluator-fed fix stages #167's documentation-target section asked for.

Grepped the full + side of eaa7458's workflows/ticketmill.js diff for anything outside the five named findings: none found. All acceptance criteria from iteration 1 still hold; the fix round introduced no scope creep and no spec regression.

aaddrick · 2026-07-28T20:09:28Z

Code Review (Iteration 2)

Verdict: changes_requested (1 minor)

Re-reviewed at eaa7458 in .worktrees/issue-167 against base Batch_2026-07-28_134922 (b591d71). Baseline is clean: node --check passes on both copies, node scripts/lint-engine.js reports clean (workflows/ticketmill.js, 2 lockstep pairs in sync, 478,435/524,288 bytes), the lockstep .claude/workflows/ticketmill.js is byte-identical (cmp), and node --test is 781/781 green. No sandbox violations in the diff (Date.now(), Math.random(), argless new Date(), require/process/fs, TS syntax), no agentType in any agent() call, no profile-shape change.

Iteration-1 findings, re-verified as fixed

code-i1-1 (size) — fixed. The engine is 478,435 bytes, back under the 92% warning line the preceding commit added; the trimmed comment blocks now carry one-sentence pointers, and all seven docs/architecture/gate-hygiene.md#... anchors in the engine resolve to real headings in that file (checked each against the heading list).
code-i1-2 (leftover re-litigated) — fixed at workflows/ticketmill.js:5029: the halting second rebuttal-only round now calls retypeGateDisposition(ctx, 'pr-review', prReviewDisposition, 'carried-unresolved') before break. prReviewDisposition at that point is always 're-litigated' by construction (bothNothingToFix/capReached both break above the fix stage), so the retype always finds its from bucket. Pinned by tests/pr-review-gate.test.js:580.
code-i1-3 (normalizeRebuttals claim) — fixed: gate-hygiene.md:540-548 now says the mechanical bar is non-blankness only and names evidence: 'I disagree' as passing it.
code-i1-4 (override clause) — fixed at :1908-1910: the clause is now site-agnostic and names both the code-review and spec-review instructions.
code-i1-5 (nothing closes a contested entry) — fixed: both the contestedBlock comment (:1892-1896) and gate-hygiene.md:700-713 now state plainly that no code path closes a ctx.contested entry, flagged as a known gap for a follow-up rather than as a design claim.

No regressions from the fix commit: the trim is comments-only apart from the one retypeGateDisposition line, and the postGateState payload carries only gate_budgets/settled (:4214-4230), so retyping a disposition after that boundary post can never leave the durable gate-state comment disagreeing with the ledger.

Minor

1. docs/architecture/gate-hygiene.md:714-729 says rebuttal_only_rounds increments on the halting second pr-fix round; the code deliberately does not. The section header is "at pr-review, the first increment is a continuation, not an exit", the body says the counter "increments at all three rebuttal-only exits above", and it then calls a second rebuttal-only round "an exit" that "bounds an issue to at most two rebuttal-only pr-fix rounds regardless" — all of which reads as "the halting round increments too". It does not: workflows/ticketmill.js:5027-5030 sets haltReason, retypes, and breaks, with no ctx.metrics.rebuttal_only_rounds++, so the counter can never exceed 1 per reviewAndMerge() call. That is intentional and pinned — tests/pr-review-gate.test.js:568-570 asserts 1 with the comment "Only the FIRST rebuttal-only round counts — the second is a halt, not a recorded round" — so the code and the test agree and only the doc is wrong. Nothing downstream breaks (the engine-internals.md:737-741 claim that rebuttal_only_rounds > 0 distinguishes route (c) still holds, since the first round already incremented), but a reader reconciling the counter against the round count will misread it.

Fix direction: correct the doc, not the code — one sentence in that section saying the halting second round is deliberately not counted (the counter tracks continuations, and the halt is already carried by haltReason and the needs_human status), and drop or reword the "at most two rebuttal-only pr-fix rounds" phrasing so it describes rounds rather than counter values.

Observed, deliberately not filed

The halting second rebuttal-only round also pushes no ctx.contested entries and no VERIFY_SKIPS line of its own. Neither is a gap in practice: the run halts to needs_human immediately after, so a contested entry would never be rendered to another reviewer, and the first round's rolled-up VERIFY_SKIPS line plus the haltReason string already reach the human.
rebuttal_only_rounds is absent from FRICTION_WEIGHTS and test-quality has no gate_findings key to retype. Adjudicated upstream at the plan-challenge gate and documented at gate-hygiene.md:733-743; not re-litigating it here.
Partial rebuttals (rebut some, fix others) still discard normRebuttals at all three sites. Settled upstream and pinned by tests/quality-loop.test.js:643; noted only so the next reader knows it was seen again and left alone.

[code-i2-1] The halting second pr-fix rebuttal-only round retypes the disposition and breaks without incrementing the counter (by design, pinned at tests/pr-review-gate.test.js:568-570), but the prose said the counter "bounds an issue to at most two rebuttal-only pr-fix rounds" without noting the halt itself is never counted. Clarify that the counter tracks continuations, not rounds, and that the halt is already carried by haltReason and the needs_human status.

aaddrick · 2026-07-28T20:10:51Z

PR Review Fix (iteration 2)

Commit: af03409

[code-i2-1] Corrected the rebuttal_only_rounds prose in docs/architecture/gate-hygiene.md: the halting second pr-fix rebuttal-only round retypes the disposition and breaks without incrementing the counter (matches the code and tests/pr-review-gate.test.js:568-570). Clarified that the counter tracks continuations, not rounds, and that the halt is already carried by haltReason and needs_human. No engine or test change.

aaddrick · 2026-07-28T20:12:42Z

Spec Review (Iteration 3)

Verdict: approved

Re-verified PR #185 against issue #167 at HEAD af03409 (branch issue-167-a-finding-is-a-hypothesis-at-the-evaluator-fed-fix) after the "PR Review Fix (iteration 2)" commit. node --test 781/781 green, node scripts/lint-engine.js clean (478,435/524,288 bytes), lockstep .claude/workflows/ticketmill.js byte-identical to workflows/ticketmill.js (cmp).

Iterations 1 and 2 already confirmed the PR meets every acceptance criterion in #167; this pass checks whether af03409 preserved that. af03409 touches exactly one file, docs/architecture/gate-hygiene.md, and is the doc-only fix for the single Code Review (Iteration 2) minor finding (code-i2-1): the prose previously implied the halting second rebuttal-only pr-fix round increments rebuttal_only_rounds. It does not — workflows/ticketmill.js's halting branch retypes the disposition and breaks without an increment, matching the pinned assertion at tests/pr-review-gate.test.js:568-570. The corrected text now states the counter tracks continuations rather than rounds, and that the halt itself is already carried by haltReason and the needs_human status. Diff-checked: no engine or test lines changed, no behavior change, no new claims introduced beyond what the code and existing tests already do.

All acceptance criteria from #167 still hold, unaffected by this doc-only round:

The three evaluator-fed stages (quality-fix, test-quality-fix, pr-fix) render FINDING_HYPOTHESIS_ASK; the two oracle-fed stages (test-fix, browser-fix) are untouched with their anti-rebuttal guards intact.
rebutted entries require non-blank finding_id/evidence via normalizeRebuttals, with ids restricted to the finding set actually rendered to that fixer.
The JS rule (normalizeRebuttals, contestedBlock, retypeGateDisposition) is unit-tested above the TICKETMILL-TEST-HARNESS-SPLIT marker.
A rebuttal-only round (rebuts something, fixes/changes nothing) never resolves as approved at any of the three gates.
A partial rebuttal (some rebutted, some fixed) proceeds normally — pinned by tests/quality-loop.test.js:643.
Omitting rebutted entirely is behavior-identical to today — pinned by tests/quality-loop.test.js:687, tests/test-loop.test.js:255.
Documented in docs/architecture/gate-hygiene.md, with its docs/architecture/index.md row present; the hash-frozen pipeline.md/metrics.md/failure-semantics.md files are untouched.
The explicitly-out-of-scope item (stripping reviewer attribution) remains untouched.

No scope creep. No changes requested.

aaddrick · 2026-07-28T20:14:57Z

Code Review (Iteration 3)

Verdict: approved

Re-reviewed at af03409 in .worktrees/issue-167 against base Batch_2026-07-28_134922 (b591d71).

Baseline clean: node --check workflows/ticketmill.js passes, cmp confirms the lockstep .claude/workflows/ticketmill.js is byte-identical, node scripts/lint-engine.js reports clean (workflows/ticketmill.js, 2 lockstep pairs in sync, 478,435/524,288 bytes) with no size warning, and node --test is 781/781 green. No sandbox violations in the engine diff (Date.now(), Math.random(), argless new Date(), require/process/fs, TS syntax — grepped the + side, zero hits), no agentType in any agent() call, no profile-shape change.

Iteration-2 finding, re-verified as fixed

code-i2-1 (rebuttal_only_rounds halt-round prose) — fixed at docs/architecture/gate-hygiene.md:722-739. The section now states plainly that a second rebuttal-only round in the same reviewAndMerge() call is a halt rather than a continuation, that the counter deliberately does not increment on it (it tracks continuations, and the halt is already carried by haltReason and the needs_human status), and reframes the "at most two rebuttal-only pr-fix rounds" bound as a property of rounds rather than of the counter value. Checked against the code: workflows/ticketmill.js:5026-5031 sets haltReason, calls retypeGateDisposition, and breaks with no ctx.metrics.rebuttal_only_rounds++; the increment lives only on the continuation path at :5038. The doc's "pr-fix runs at most at iterations 1 and 2 of MAX_PR_REVIEW_ITERATIONS = 3" is also true as written — capReached (:4956) breaks at :4987, above the fix stage. Doc-only commit; git show --stat confirms one file, 12 insertions / 7 deletions, no engine or test change.

Independently re-verified this pass (not just trusted from prior iterations)

Disposition retyping is always well-formed at pr-review. prReviewDisposition (:4957) is 're-litigated' by construction at both retypeGateDisposition call sites (:5029, :5033), since prReviewClean, bothNothingToFix, and capReached all break above the fix stage. retypeGateDisposition (:2094) is count-preserving when from === to, so the quality gate's final-iteration case (bookedDisposition === 'carried-unresolved' at :3038) is a genuine no-op rather than a lost count.
Every rebuttal-only exit pushes a VERIFY_SKIPS line. Quality (:3082-3091, rolled up via ctx.quality_rebuttals/quality_rebuttal_skip_index, mirroring the existing cap roll-up), test-quality (:3611, one line per round), pr-review (:5046, un-rolled and never rewritten). The quality cap-exhaustion branch is correctly gated to !approved && !degraded && !rebutted (:3121) so a contested round is never double-reported as a cap-out.
normalizeRebuttals (:2147) is pure and fails toward today. Non-array raw, blank finding_id/evidence, and any id absent from the exact findings array rendered to that fixer all drop silently. The returned summary is copied from the matched finding, never from the fixer, so a rebuttal cannot inject a summary for a finding it was never shown.
FIX_SCHEMA.rebutted (:476-478) is out of required, so a fixer that omits it is byte-identical to pre-A finding is a hypothesis at the evaluator-fed fix stages #167 behavior; contestedBlock (:1897) and FINDING_HYPOTHESIS_ASK (:1930) are pure and sit above the TICKETMILL-TEST-HARNESS-SPLIT marker.
Prompt/schema coherence. FINDING_HYPOTHESIS_ASK asks for rebutted entries with finding_id + concrete evidence; FIX_SCHEMA requires exactly those two keys per item. It renders only where structured findings exist (revFindings !== null at :3053, vFindings !== null at :3590, specFindings !== null || codeFindings !== null at :5005), so the prose-fallback path never asks for ids that were not rendered.

Observed, deliberately not filed

engine-internals.md:737-741 says a carried-unresolved pr-review tally with rebuttal_only_rounds > 0 identifies route (c). The counter is per-issue and shared across all three gates (:3077, :3613, :5038), so a quality-gate rebuttal plus a pr-review cap-out satisfies the same test. Not filing: findings_empty_exits has the identical three-site, run-wide shape and the pre-existing prose two lines up makes the same style of claim, and gate-hygiene.md:717-718 explicitly names this as "the same run-wide, not-per-gate shape findings_empty_exits already uses". Flagging it would be nitpicking an acknowledged pre-existing convention this PR follows accurately.
ctx.contested is not part of the durable gate-state payload (:4229 carries gate_budgets/settled only), so contested entries do not survive a mid-issue resume. Same as ctx.notes and pre-existing; not introduced here, and the "nothing closes a contested entry" gap is already documented at gate-hygiene.md:700-713 as a follow-up.
rebuttal_only_rounds absent from FRICTION_WEIGHTS, test-quality having no gate_findings key to retype, and partial rebuttals discarding normRebuttals — all adjudicated upstream and re-noted at iterations 1 and 2. Not re-litigating.

Nothing must be fixed. Approved for merge.

aaddrick · 2026-07-28T20:18:17Z

Implementation Complete

Branch issue-167-a-finding-is-a-hypothesis-at-the-evaluator-fed-fix has been squash-merged into Batch_2026-07-28_134922.

Spec review: passed
Code review: passed

Deferred Suggestions for Follow-up

Engine-owned guardrail: incidental change(s) to lockstep-installed path(s) — .claude/workflows/ticketmill.js — were NOT reverted (lockstep exemption); verify the lockstep pair stayed in sync (lint-engine) before merge.

aaddrick added 4 commits July 28, 2026 15:05

aaddrick mentioned this pull request Jul 28, 2026

A finding is a hypothesis at the evaluator-fed fix stages #167

Closed

aaddrick merged commit ff7448b into Batch_2026-07-28_134922 Jul 28, 2026
1 check passed

aaddrick deleted the issue-167-a-finding-is-a-hypothesis-at-the-evaluator-fed-fix branch July 28, 2026 20:18

Navigation Menu

feat(gate-findings): frame findings as hypotheses at evaluator-fed fix stages - #185

feat(gate-findings): frame findings as hypotheses at evaluator-fed fix stages#185
aaddrick merged 6 commits into
Batch_2026-07-28_134922from
issue-167-a-finding-is-a-hypothesis-at-the-evaluator-fed-fix

aaddrick commented Jul 28, 2026

Uh oh!

aaddrick commented Jul 28, 2026

Uh oh!

aaddrick commented Jul 28, 2026

Uh oh!

aaddrick commented Jul 28, 2026

Uh oh!

aaddrick commented Jul 28, 2026

Uh oh!

aaddrick commented Jul 28, 2026

Uh oh!

aaddrick commented Jul 28, 2026

Uh oh!

aaddrick commented Jul 28, 2026

Uh oh!

aaddrick commented Jul 28, 2026

Uh oh!

Uh oh!

aaddrick commented Jul 28, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

aaddrick commented Jul 28, 2026

Summary

Implementation

Key decisions

Testing

Uh oh!

aaddrick commented Jul 28, 2026

Spec Review (Iteration 1)

Uh oh!

aaddrick commented Jul 28, 2026

Code Review (Iteration 1)

Major

Minor

Observed, deliberately not filed

Uh oh!

aaddrick commented Jul 28, 2026

PR Review Fix (iteration 1)

Uh oh!

aaddrick commented Jul 28, 2026

Spec Review (Iteration 2)

Uh oh!

aaddrick commented Jul 28, 2026

Code Review (Iteration 2)

Iteration-1 findings, re-verified as fixed

Minor

Observed, deliberately not filed

Uh oh!

aaddrick commented Jul 28, 2026

PR Review Fix (iteration 2)

Uh oh!

aaddrick commented Jul 28, 2026

Spec Review (Iteration 3)

Uh oh!

aaddrick commented Jul 28, 2026

Code Review (Iteration 3)

Iteration-2 finding, re-verified as fixed

Independently re-verified this pass (not just trusted from prior iterations)

Observed, deliberately not filed

Uh oh!

Uh oh!

aaddrick commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Implementation Complete

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

aaddrick commented Jul 28, 2026 •

edited

Loading