diff --git a/CHANGELOG.md b/CHANGELOG.md index 7f50468..445f51a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- `--fail-on-warnings`: opt-in escape hatch that makes `--ci` exit `1` when any + check reports warnings. It is only meaningful together with `--ci` (clap + rejects it otherwise) and it restores the pre-change CI behaviour for teams + that want a warnings-clean trunk. `prview gate` is untouched — its exit codes + come from the verdict contract, not from this flag. - `00_summary/PROVENANCE.json` — a pack-level record of *what was analysed*, next to the per-check rows that record *where each gate ran*. It carries the `target_sha` the pack judges, the `base_sha` it diffed against — the merge @@ -78,8 +83,877 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 commit its analysis root was extracted from along with the scan's start and end times (all additive and optional). +### Changed + +- **`--ci` exit code for a warnings-only run: `1` → `0`.** Warning-level checks + no longer break `quality_pass`, and `--ci` still exits `1` only on `BLOCK` or a + broken quality gate — so a run whose worst signal is a warning now exits `0`. + Pass `--ci --fail-on-warnings` to keep the old exit. Runs with a real failure, + and every `prview gate` exit code, are unchanged. +- **BREAKING (behavioral): an unreadable `MERGE_GATE.json` is now an execution + error, not a guessed verdict.** `prview --json` / `--ci` used to fall back to + re-deriving the decision from the in-memory policy engine when the gate + artifact was missing or unparsable, publishing `allow_merge = recommendation + != block` — the only path in the codebase where `allow_merge: true` could + coexist with a `CONDITIONAL` verdict, contradicting the documented + `allow_merge == (verdict == "PASS")` invariant. That fallback is removed: + a missing, unparsable, or unknown-schema gate artifact now prints an error and + exits `3`, the same execution-error code `prview gate` already used. This also + applies to `--update` runs that re-read an earlier pack, so a truncated + previous run reports the failure instead of resurrecting a plausible verdict. +- **`MERGE_GATE.json` readers check `schema_version`.** A pack with an unknown or + unparsable MAJOR is rejected fail-loud (`exit 3` on the CLI, `storage_corrupt` + on the MCP surface), and so is a `schema_version` that is present but is not a + `MAJOR.MINOR` string — a number, an object, or an explicit `null` used to be + read as "field absent", i.e. as a legacy pack, which is the opposite of what it + means. A version with extra components (`2.1.3`) is rejected rather than + truncated to `2.1`, so "readable by prview" cannot drift away from the exact + set `tools/validate_merge_gate.py` accepts. A newer MINOR of a known MAJOR is + read and reported with a `schema_forward_compat:` caveat — on every known + MAJOR, so a `1.9` pack is now caveated instead of accepted in silence — and the + MCP surface marks that read `normalized: true`, as the documented contract + already promised. Version components must also be spelled canonically: + `u32::from_str` accepts leading zeros and a leading `+`, so `02.2`, `2.02` and + `+2.2` all parsed to the known `(2, 2)` and were read as the current schema + while the validator rejects those exact strings. An absent `schema_version` stays accepted: pre-2.1 packs + predate the field, and the documented `ALLOW`/`HOLD` verdict tolerance is + unchanged. +- **A versioned pack without a `decision` object is a corrupt artifact.** The CLI + reader fell back to treating the gate's ROOT as the decision, so a pack that + states `schema_version: "2.2"` and then carries no `decision` (or a non-object + one) normalized quietly to `BLOCK` / `allow_merge: false` with an + `unknown_verdict:` caveat — a verdict nothing in the pack ever stated. It now + exits `3`, matching `tools/validate_merge_gate.py` (which requires `decision` + at every version) and the `prview mcp` adapter (which already returned + `storage_corrupt`). A pack with NO `schema_version` predates the field and + keeps the legacy tolerance: its root is still read as the decision. +- **The legacy tolerance is now whole on both readers.** The `prview mcp` adapter + required a `decision` object unconditionally, so a genuine pre-2.1 pack — no + `schema_version`, signals at the root — was answered `storage_corrupt` by the + MCP surface while the CLI read the very same file and printed a verdict. One + artifact cannot be simultaneously readable and corrupt depending on which + surface asks. Both readers now select the decision object through a single + `gate::select_decision_object`: `decision` when it is an object, the root when + the pack states no `schema_version`, and fail-loud otherwise. The corruption + rule for versioned packs is unchanged; only the disagreement is gone. +- **A wrongly typed decision signal is a normalization, not an absent field.** + `verdict: "PASS"` beside `merge_recommendation: 7` used to collapse through + `as_str()` into "no recommendation", so the `prview mcp` adapter returned a + decision derived from the surviving signal with `normalized: false` and no + caveat — a field ignored in silence, which the MCP contract forbids. Each + decision signal now distinguishes absent from present-but-untypable and emits + an `unreadable_verdict:` / `unreadable_merge_recommendation:` / + `unreadable_allow_merge:` caveat with `normalized: true`. A pack with no + usable signal at all is still `storage_corrupt`. +- **The CLI reader names wrongly typed signals too, and refuses to approve on + them.** The `unreadable_*` discipline above shipped on the MCP surface only; + the CLI still went through `as_str()` / `as_bool()`, so `verdict: 7` was + reported as `unknown_verdict: … carries no verdict` (a claim about a field that + was in fact present), `merge_recommendation: 7` fell through to + `review_required`, and `allow_merge: "true"` silently became `false`. Worse, + a pack with a valid `verdict: "PASS"` beside a mistyped `merge_recommendation` + published a `PASS` derived from a decision block the reader had only partly + read. Both readers now share `gate::readable_signal`: a present-but-untypable + field emits the same `unreadable_:` caveat on `--json`, and — matching + the unknown-verdict rule already in place — forces every derived axis + conservative (`verdict: "BLOCK"`, `allow_merge: false`, + `merge_recommendation: block`, `--ci` exit `1`). A well-typed pack gains no + caveat and is unaffected. +- **Unknown verdicts are reported instead of silently absorbed.** The CLI still + collapses an unrecognized verdict to `BLOCK`, but now says so through a new + optional `caveats` array on the `--json` summary (`unknown_verdict: …`) — the + reader no longer presents a normalization as something it read. The MCP + `verdict` surface likewise reports `unknown_verdict` / + `unknown_merge_recommendation` and sets `normalized: true` instead of dropping + the unparsable field on the floor. The `--json` summary keeps + `schema_version: "cli-json/v1"`: `caveats` is additive and omitted when empty. + A verdict the CLI collapsed to `BLOCK` now also forces the axes derived beside + it: `allow_merge` is `false` and `merge_recommendation` is `Block` regardless + of what the same unreliable decision block claimed. A pack with an unreadable + verdict but `allow_merge: true` and `merge_recommendation: "approve"` used to + publish `verdict: "BLOCK"` next to an approval — breaking the + `allow_merge == (verdict == "PASS")` invariant — and, because + `compute_exit_code` keys off the recommendation, `--ci` exited `0` on it. +- Human stdout no longer prints "All checks passed!" when no gate artifact was + readable. The raw check tally is not a verdict; the summary now names the + missing truth. +- **`report.json` schema_version: `1.0` → `2.0`.** + `quality.coverage.heuristic_ratio` is `null` when nothing was measured + (previously a misleading `1.0`) and is accompanied by new `measured: bool` + and optional `not_measured_reason` fields; `quality.heuristics` omits its + counters on a skipped scan. No field was removed or renamed, but a field that + was always a number can now be `null` and counters can now be absent, so a + decoder written against `1.0` does not parse every pack — that is a MAJOR, not + an additive MINOR. Consumers reading `heuristic_ratio` must handle `null` — + the bundled dashboard PR-comment generator renders it as `not measured`, and + `history.rs` already treats a missing value as "no baseline". +- Bumped the bundled `loctree` structural-analysis crate from `0.8` to `0.13.0`. + The public API prview consumes (`analyzer::{cycles, dead_parrots, twins}`, + `snapshot::{Snapshot, project_cache_dir, run_init, SNAPSHOT_SCHEMA_VERSION}`, + `args::ParsedArgs`) is source-compatible — no call sites changed. The snapshot + schema version is now decoupled from the crate version (pinned at `0.11.0` + instead of tracking `CARGO_PKG_VERSION`); prview's `major.minor` schema gate + handles the transition, so stale `0.8`-era caches are re-scanned automatically. + loctree 0.13 also widens file-type coverage in the scan (markdown, shell, + config, and other non-source files now count toward the snapshot), so the + `LOCTREE` heuristics stats (`total_files`, `total_loc`, `by_language`) report + higher, broader numbers than under 0.8 for the same tree. + ### Fixed +- **The blocker flag and the blocker list are certified as one fact.** The + emitter computes `policy_allow_merge = blocking_issues.is_empty()` after the + last entry is pushed and writes both verbatim, but the contract validator used + that relation only in the harsher direction — a listed blocker raises the + verdict a pack must clear — which left the two halves free to contradict each + other outright. `policy_allow_merge: true` beside a listed blocker certified + clean, telling a reader that trusts the flag that policy let the merge through + while the list beside it named what blocked it. From schema 2.2, where both + fields are required, `tools/validate_merge_gate.py` enforces the equivalence in + both directions: `true` with blockers and `false` without them are both + rejected. This completes the reconciliation port rather than adding a rule to + it — same shape as the `quality_pass` / `quality_failure_details` equivalence, + and distinct from the older "no `allow_merge: true` beside a blocker" check, + which is about the merge verdict rather than the policy flag it derives from. A + test in `src/artifacts/merge_gate.rs` pins the flag to the list across the + emitted packs, so a second input to the flag fails the emitter instead of + making the validator reject prview's own output. Probed against every pack on + disk: no pack from a real run is rejected. +- **An `impl` owner is part of a declaration's site.** A `pub` associated item + moved between two impl blocks in one file — `pub const VALUE` leaving `impl A` + and appearing in `impl B` — matched on file, kind, name and text with an empty + scope on both sides, so the exact pairing consumed it and `A::VALUE` vanished + from the report entirely. Impl owners now ride the same stack as inline + modules, recorded as the header text with whitespace collapsed and nothing + parsed. The asymmetry is deliberate: two KNOWN and different owners never pair, + while an owner the hunk never showed stays unknown and pairs with anything, so + the accepted unseen-opener limit is untouched. Over 211 commits of this + repository the reports are identical before and after; over 708 crates.io + release pairs removals move 30,555 → 30,694 and signature changes 53,938 → + 53,805, i.e. mostly a reclassification of a real owner change. Recorded limit, + mirroring the `cfg` operand-ordering one: the same owner written with a + different path qualifier reads as two owners (40 of 2,784 blocked pairings, + all in one crate) — closing it means parsing types. +- **The contract validator now certifies the reconciliation, not just the + shape.** `tools/validate_merge_gate.py` checked each decision field on its own, + so a pack stating `verdict: "PASS"` beside `analysis_status: "incomplete"`, a + `block` recommendation and `policy_allow_merge: false` validated OK — while + every reader normalizes that same artifact to `BLOCK`. The readers were already + protected; the hole was in CERTIFICATION. From schema 2.2 the validator ports + their whole rule: it requires the remaining decision axes (`analysis_status`, + `merge_recommendation`, `policy_allow_merge`) with the vocabularies the typed + enums emit, and rejects a `verdict` milder than the most conservative axis + stated beside it. The rule is one-directional on purpose: a HARSHER verdict is + legal, because a semgrep scan that passes with parse errors writes `approve` + beside `degraded` and the contract turns that into `CONDITIONAL`. A test in + `src/policy/engine.rs` pins both enum spellings to the words the validator + lists. Probed against 3,547 real packs on disk (2,039 at schema 2.2): no + legitimate pack is rejected. +- **Bytes inside a literal now traverse the whole `cfg`-attribute pipeline + verbatim.** The accumulator glued an attribute's physical lines together with + nothing between them, and the caller trimmed each line before the tracker saw + it, so both the line break and a continuation's indentation vanished from + inside the value: `#[cfg(api = "a\nb")]` produced the same guard as + `#[cfg(api = "ab")]`, and a declaration that really left one configuration + paired with its re-add under another. This is the third finding of one shape, + after the delimiter count and the whitespace strip, so it is closed as an + invariant rather than patched again. The tracker now takes the raw line, joins + a physical break with `\n` exactly when a literal is open across it, and trims + nowhere — after the dense view there is no whitespace left outside a literal, + so a trim could only eat value. Layout outside a literal is still normalized: + re-indenting or re-wrapping a predicate is the same gate. Of 568,128 `cfg` + attributes in the local crates.io registry, 4 carry a literal spanning a line + break, 2 of them gate an item, and none collide. +- **An unreadable `checks` list is not an empty one.** `checks` present but not + an array left the warning tally at zero and fell back to the checks the run + itself executed — which on an unchanged `--update` run is none — so + `--ci --fail-on-warnings --update` exited `0` on a reused pack whose warning + list the reader could not read. It now counts as at least one warning and says + so in the existing `unreadable_checks:` caveat. This is the r27 rule one level + up, on the container instead of an entry, and no legacy carve-out applies: + `checks` has been emitted since schema 1.0 and `validate_merge_gate.py` has + always required an array there, so a non-array was never a valid shape. An + ABSENT `checks` keeps its tolerance — a pack that states no list may simply + predate this build. +- **Whitespace inside a `cfg` value is part of the gate.** The guard tracker + normalized an attribute by stripping whitespace from its whole text, literals + included, so `#[cfg(api = "a b")]` and `#[cfg(api = "ab")]` produced one guard: + a declaration that really left builds configured with `--cfg 'api="a b"'` + paired with its re-add under another value and produced no finding. The strip + is now `SourceScanner`'s own dense view, which removes spacing only where it + can see the spacing is outside every literal, so reformatting an attribute is + still not a different gate. A fix by construction rather than by frequency: of + 524,530 gating attributes in the local crates.io registry only 3 carry + whitespace inside a value literal, and none of them collide. +- **A check status outside the emitted vocabulary is unreadable, not clean.** + `checks[].status` is a closed, case-sensitive set — `passed`, `failed`, + `warnings`, `skipped`, `error` — but the CLI tallied warnings by comparing + against the single string `"warnings"`, so any other spelling counted as "not a + warning" and `--ci --fail-on-warnings --update` exited `0` on a reused pack + whose warning signal it could not read. `tools/validate_merge_gate.py` accepted + any non-empty string there, so such an artifact even passed the repository + gate. Both sides now name the vocabulary: the reader counts an unrecognized + status toward the tally and raises an `unreadable_check_status:` caveat naming + the checks, and the validator rejects the pack. Case is deliberately not + folded — normalizing `"WARNINGS"` silently would hide that the pack is + off-contract, and the tally is the same either way. The vocabulary lives as + `CheckStatus::EMITTED` next to `CheckStatus::as_str`, with a test pinning the + two together. +- **An attribute's delimiters are counted with its literals removed.** The + `cfg`-guard tracker resolved comments away with a carried scanner but counted + brackets with a literal state of its own, reset at every line — so a literal + opened on an earlier line was invisible to it. A `)` typed inside a multi-line + `#[doc = r#"…"#]` balanced the attribute early and the literal's remaining + lines then cleared the pending `cfg`; a `#[must_use = "… \` continued onto the + next line had its own closing quote read as an opener, swallowing the `]`, so + the attribute never closed and absorbed the real `#[cfg(…)]` below it. Either + way both diff sides came out unguarded, the identical declaration text paired, + and a configuration-specific removal produced no finding. The counter now runs + on a literal-free view from a second scanner walking the same lines, while the + guard text keeps its literals so `feature = "a"` and `feature = "b"` stay two + gates. Measured over the local crates.io registry: of 237,368 `cfg`-guarded + attribute runs reaching a public declaration, 8,793 wrap, 90 carry a literal + spanning the break, 13 a raw string, and 9 balanced wrongly. +- **Re-indenting the inside of a multi-line public constant is a value change + again.** Continuation lines reached the breaking-change accumulator already + trimmed, so whitespace at a line edge INSIDE a string literal — which is value, + not layout — never reached the comparison. Two literals differing only in their + indentation produced identical identities, and the exact-match pass consumed + the addition: a changed public value left no finding at all. The accumulator + now takes the raw line and normalizes per edge — the leading edge is kept when + the previous line left a literal open, the trailing edge when the line itself + does — so a reflow outside a literal stays the no-op it must be, and a trailing + comment's leading gap still contributes nothing. Measured over the local + crates.io registry, of 200,553 multi-line public declarations 640 continuation + lines sit at a literal edge and 272 carry whitespace the old view dropped. +- **`tools/validate_merge_gate.py` now requires a boolean `quality_pass` from + schema 2.2.** The validator checked the field's agreement with the failure + details but never its presence or type, so a 2.2 pack stating + `quality_pass: "false"` — or omitting it — was certified clean while both + decision readers normalize a present-but-unreadable signal to BLOCK. The + contract gate was therefore passing artifacts the CLI and MCP refuse to trust. + The 2.2 writer emits the field unconditionally as a boolean, so requiring it + there is safe; absence stays forgiven below 2.2, where readers derive the flag + instead. +- **A body-less test item can now end its own test context.** After a top-level + `=` an item states a value, but the perf tracker kept reading `<` as a generic + opener there, so `#[cfg(test)] const ENABLED: bool = 1<2;` left the signature's + bracket depth above zero — the very thing the `;` close tests. The item could + not end the context it opened, and every loop or query below it was recorded as + test-only and dropped from the signal. Angle tracking now stops at the item's + top-level `=`, the same rule the declaration scanner already applies. The + reported shape is a comparison, but the corpus idiom is the compact shift + (`const Reverse = 1<<8;`, as objc2 generates its bitflags): of 2,206,540 + single-line `const`/`static`/`type` declarations in the local crates.io + registry that end at their own `;`, 1,069 left the depth stuck open before this + change and 64 still do — and those 64 are an array type wrapping to the next + line, where holding the depth open is exactly right. +- **A turbofish return type no longer hides a changed public signature.** + `pub fn run() -> Buffer::<{` is a valid return type — rustc accepts + `Type::<…>` in type position — but its `<` follows a `:`, which the scanner did + not accept as opening a generic argument list. The list went uncounted, the + const block's `{` read as the item's body opener, and both diff sides + finalized at that identical prefix: they paired as an unchanged re-add and a + changed const argument, which is a changed public return type, produced no + finding at all. `:` now joins an identifier and a closing `>` as a predecessor + that opens a list; whitespace still does not, so a comparison is still not a + list. Verdict-neutral where it is not needed — over all 4,334,018 public + declaration lines in the local crates.io registry the old and new rules + disagree on none, because a turbofish that closes on its own line nets out + either way. What changes is a list left open at end of line. +- **`tools/validate_merge_gate.py` now rejects a `quality_pass` that + contradicts its own evidence.** The flag and `quality_failure_details` are one + fact written twice — the emitter sets `quality_pass` to + `!QualityFailureSummary::has_new_failures()` and serializes the very details + that answer it — but the validator checked each side's shape and never + compared them. `quality_pass: true` beside + `{"origin": "failure", "classification": "introduced"}` therefore certified + clean, and both decision readers trust the permissive scalar, so a + validator-clean pack could approve an explicitly introduced failure. The check + is an equivalence: `quality_pass` is true if and only if no detail has + `origin: "failure"` with a classification other than `pre-existing`. The + `pre-existing` carve-out is load-bearing — a failure that predates the diff is + emitted beside `quality_pass: true` on purpose, so the simpler one-way rule + would have rejected packs prview itself writes. Packs without the field are + untouched. +- **A compactly written comparison in a const argument no longer mutes + production code.** The perf tracker judged `<` a generic opener whenever it + followed an identifier, which reads `Buffer<{ 1 < 2 }>` correctly and the same + type written `Buffer<{1<2}>` wrongly — `<` after a digit looks exactly like `<` + after an identifier. The signature's bracket depth then stayed above zero, the + real body brace read as another type-level brace, the test context never + closed, and every loop or query below the test was recorded as test-only and + dropped from the signal. Spacing is formatting, so it can no longer decide the + verdict: bracket tracking is now frozen inside a brace opened within the + signature, where a const argument holds an expression and a destructured + parameter holds a pattern and `<`/`>` are operators in both. +- **A comparison inside a const argument no longer swallows the item body.** + `pub fn run() -> Buffer<{ 1 < 2 }> {` counted the comparison as another + generic opener, the argument list's own `>` closed only that phantom level, + and the depth was still above zero at the real body brace — which read as a + further const argument, absorbed the body, and turned a body-only rewrite into + a phantom `ChangedSignature`. Inside a const block `<` and `>` are operators, + so the generic depth is now frozen there. Nothing is lost: whatever such a + block states about generics closes what it opens — a turbofish + (`{ size_of::() }`) or a qualified path (`Uint<{ ::LIMBS / 2 }>`), + which are also the only shapes the local crates.io corpus carries. Those + survived the previous rule by cancellation, the block's stray `>` closing the + outer list; they now reach the same verdict by construction. +- **A signature edited in place is no longer swallowed by the context lines + around it.** A hunk interleaves two texts, and the scanner reconstructs both: + the before side is context ∪ removed lines, the after side is context ∪ added + lines. It used to end BOTH pending declarations at the first line from the + other side, so the everyday shape of an edited signature — `pub fn f(` + retouched on both sides, a shared `x: u8,`, then `-old: u16,` / `+new: u32,` + and a shared `) {` — finalized to two identical openers, paired as an + unchanged re-add, and reported the parameter change nowhere. A `-` line now + extends only the removed side, a `+` line only the added side, and a context + line extends whichever side still has a declaration open. Context lines only + CONTINUE a declaration and never start one: a `pub` item first seen on a + context line is unchanged by the patch. `MAX_DECL_CONTINUATION_LINES` (32) + still bounds growth and a hunk header still finalizes both sides, so the + reconstruction stays inside the hunk that emitted it. +- **Braces in a stacked test attribute no longer end the test context.** An + attribute's brackets belong to the attribute, never to the item it annotates, + but the brace scan read them as the annotated item's: with `#[rstest]` stacked + over a brace-bearing `#[case(…)]`, the attribute's `{` was taken as the body + opener and its `}` closed the context on the same line, so the test function + below was classified as production and a query in its loop surfaced as a + phantom regression. The scan now tracks attribute depth per character and + skips what is inside one. The plain `#[case(Case { id: 1 })]` was safe only by + accident — its `[` and `(` hold the signature depth above zero — while + `#[case(1 > 0, 2 > 1, Case { id: 1 })]` clamps that depth back to zero first + and reaches the bug; skipping attributes removes the class rather than the one + shape. +- **A legacy `PASS` pack no longer fails `--ci` on the CLI while the MCP adapter + approves it.** A decision written before `quality_pass` existed — + `{"verdict": "PASS", "merge_recommendation": "approve", "allow_merge": true}` + — reconciled correctly to `PASS`, because an absent field adds no rank, but the + summary then published `quality_pass: false` from a bare default, derived + `analysis_status: incomplete` from that, and exited `1` under `--ci`. The two + readers answered the same artifact differently. Ranking an absent field and + publishing one are separate questions: an absent axis is now derived from the + reconciled outcome, so a reconciled `PASS` — which the contract permits only + when quality passes and the analysis is complete — publishes both, and a + decision held below `PASS` stays conservative on both. The absent/mistyped + split is untouched: an unreadable value normalizes the decision to `BLOCK`, so + nothing can be inferred as passing from it. +- **An incomplete analysis or a stated blocker can no longer be published as an + approval.** The conservative reconciliation ranked `verdict`, + `merge_recommendation`, `allow_merge` and `quality_pass`, but read + `analysis_status` only afterwards for display and `blocking_issues` only for + passthrough — so a pack shaped `verdict: "PASS"`, `merge_recommendation: + "approve"`, `allow_merge: true`, `quality_pass: true` published a clean + approval even when it also stated `analysis_status: "incomplete"` or listed a + blocking issue, on the CLI and the MCP surface alike. The contract permits + `PASS` only when the analysis is `complete`, and an entry reaches + `blocking_issues` only from a check whose `merge_impact` is `Block`. Both now + rank: `degraded`/`incomplete` as `CONDITIONAL`, a non-empty `blocking_issues` + (and its restatement `policy_allow_merge: false`) as `BLOCK`, each named in the + `core_inconsistency:` caveat and typed through `gate::readable_signal` so a + mistyped one normalizes conservatively. `analysis_status: "complete"`, + `policy_allow_merge: true` and an empty `blocking_issues` state no rank — they + are preconditions of a `PASS`, not grants of one — and absence still states + nothing, so older packs read exactly as before. +- **The decision axes are now enumerated in the contract.** Every field the + `decision` object may carry has a row in the ranking table of + `docs/contracts/merge_gate.md` saying whether it ranks and why, under one rule: + an axis states a rank only when its value RULES OUT a more permissive outcome. + The deliberate exclusions are recorded with their reasons — `recommended_merge` + restates `merge_recommendation`, `recommended_label` has an open vocabulary, + the `quality_failures` arrays are populated by warning-origin entries that + never flip `quality_pass`, and `quality_failure_details` is the evidence behind + that axis rather than an axis of its own. A field added to `decision` without a + row is an unfinished change. +- **A `quality_pass` that cannot be typed is no longer read as absent.** Both + readers took that axis with a bare `as_bool()`, which returns nothing for a + present-but-mistyped value just as it does for a missing one — so a pack + stating `quality_pass: "false"` beside a clean approval was read as a pack + written before the field existed, and published `PASS` with `allow_merge: true` + and no caveat at all, on the CLI and the MCP surface alike. `quality_pass` now + goes through the same `gate::readable_signal` as `verdict`, + `merge_recommendation` and `allow_merge`: a stated-but-unreadable axis + normalizes to `BLOCK` and is named by an `unreadable_quality_pass:` caveat. An + absent `quality_pass` is still silent and still states no rank, so packs + written before the field are unaffected. +- **A failed quality axis can no longer be published as a `PASS`.** The + conservative reconciliation ranked `verdict`, `merge_recommendation` and + `allow_merge` but read `quality_pass` separately, afterwards — so a pack + shaped `verdict: "PASS"`, `merge_recommendation: "approve"`, + `allow_merge: true`, `quality_pass: false` published a clean approval with + `allow_merge: true`, on the CLI and on the MCP surface alike, where automation + could act on it. A stated `quality_pass: false` now ranks as `CONDITIONAL` on + both readers, exactly like `allow_merge: false`, and is named in the + `core_inconsistency:` caveat. `quality_pass: true` still states no rank — a + quality-clean run is held at `CONDITIONAL` by a breaking-change escalation, so + one axis may not soften a verdict the others agree on — and an ABSENT + `quality_pass` still states nothing, so packs written before the field are + read exactly as before. +- **A `|` in a declaration no longer breaks the `BREAKING_CHANGES.md` tables.** + Declaration text went into a markdown table verbatim, and Rust states bitwise + or, patterns and closures with the table's own delimiter — so a row reporting + `pub const MASK: u32 = READ | WRITE;` opened extra columns and rendered as + garbage exactly where the declaration mattered. Every cell carrying source + text now escapes `|` as `\|`, which is what GitHub's table parser needs: it + splits on unescaped pipes before any inline markup runs, so a code span was + never protection. The span is also fenced by a backtick run longer than any + inside the cell, so a declaration stating a backtick of its own — + `pub const TEMPLATE: &str = r#"`value`"#;` — no longer closes its own code + span partway through and renders the remainder as prose. +- **`#[cfg(not(test))]` no longer mutes a production performance finding.** The + perf tracker opened inline test context on the bare token `test` appearing + anywhere inside a `cfg` predicate, so a query-in-loop under + `#[cfg(not(test))]` — code compiled into every build EXCEPT the test one — was + recorded as test-only and dropped, and so was one under + `#[cfg(any(test, feature = "bench"))]`, which compiles outside the test build + whenever the feature is on, or under `#[cfg(feature = "__internal-test")]`, a + feature that merely has `test` in its name. This inverted the module's own + rule that ambiguity resolves toward production. Only a gate that provably + holds solely in a test build now opens the context: an exact `#[cfg(test)]`, + an `#[cfg(all(…))]` naming `test` among its operands, `#[test]` / + `#[tokio::test]` / `#[rstest]`, and `mod tests`. Measured over the local + registry (58,586 files), of the 11,030 attributes the old pattern read as test + context 83.62% are exactly `cfg(test)` and 6.76% are `all(…, test, …)` — the + remaining 9.62% are the ones it was getting wrong. `all` is commutative, so + the operand's position carries no meaning: `all(feature = "bench", test)` is + read exactly like `all(test, feature = "bench")`, where matching only the + first operand made the same predicate production or test context depending on + how it was written (72 further attributes over that registry, none lost). The + operand must be a direct one, so `all(not(test), …)` — which proves the + opposite — and `all(any(test, …), …)` stay production. The predicate is also + read as a whole attribute rather than per physical line: rustfmt wraps a long + one, and a `#[cfg(all(` / `feature = "bench",` / `test` / `))]` spread over + four lines matched on none of them, so its test-only item was read as + production and its query-in-loop surfaced as a phantom regression. The lines + of one attribute are now joined and matched once, on the line that closes it, + bounded to 8 lines so an attribute that never closes is dropped instead of + swallowing the rest of the hunk. The shape is rare — 10 occurrences over that + registry, every one a genuine `all(test, …)` gate. +- **A block or struct-literal initializer no longer hides a changed public + constant.** `pub const LIMIT: usize = {` and `pub const ZERO: Self = Self {` + had their `{` read as the item's body opener, so both diff sides finalized at + their identical first line, paired as an unchanged re-add, and a changed + expression inside the block produced no finding at all. After a top-level `=` + the item states a value and runs to its `;`, and a `;` inside the initializer + terminates a statement rather than the declaration. Only a top-level `=` + counts — inside a generic argument list one states a default + (`struct Foo`) or an associated type + (`impl Iterator`), both still followed by a real body brace. + Measured over the local registry (58,586 files, 1,960 crates, 4,334,320 public + declaration lines) this changes the verdict on 2,465 lines, every sampled one + a public constant with a multi-line struct-literal or block initializer. +- **A reflowed declaration is no longer reported as a changed signature.** The + comparison identity preserved every physical line break, so + `pub type Alias =` followed by `u32;` was a different declaration from + `pub type Alias = u32;` — a purely cosmetic rewrap produced a + `ChangedSignature` whose "before" and "after" printed as the same string, and + could escalate the verdict. A break is now kept only where the previous line + left a string literal open, which is where it is part of the value; elsewhere + the lines are joined with a space. For the same reason a line contributing no + code is still dropped from the identity except inside a literal, where a blank + line is a blank line in the value. +- **A const argument that is not the first one no longer hides a public type + change.** The breaking-change scanner recognized `Buffer<{ LIMIT }>` as + type-level syntax by the exact `<{` sequence, so `Buffer` — + where the brace follows a comma, which is where a const generic usually sits — + finalized the declaration at its opener. Both diff sides then held the same + prefix, paired as an unchanged re-add, and the changed const expression below + produced no finding. The scanner now tracks the generic argument list itself. + `<<` is consumed whole so a shifted public constant still terminates at its + `;`, and measured over the local registry (59,946 files, 2,025 crates, + 4,354,142 public declaration lines) the new rule and the one it replaces judge + zero lines differently. +- **The MCP adapter and the CLI now answer the same way about a decision they + cannot rank.** A pack that stated a signal outside the vocabulary — a + `verdict: "PROBABLY"`, or nothing but `allow_merge` — was read as a + conservative `BLOCK` summary by the CLI and refused as `storage_corrupt` by + `prview mcp`, one artifact with two answers. `storage_corrupt` is now reserved + for a decision block stating none of `verdict`, `merge_recommendation` and + `allow_merge`; a stated-but-unrankable signal is a decision the pack gave, and + the adapter normalizes it exactly as the CLI does, with a caveat and + `normalized: true`. The substitution governs the axes published beside it, so + an unreadable verdict beside `merge_recommendation: "approve"` no longer reads + as an approval on the MCP surface while the CLI blocks on the same bytes. +- **A self-consistent `BLOCK` pack no longer reports contradicting itself.** + Both readers compared `allow_merge` to the numeric rank of the winning + verdict, but `allow_merge` has two values and `false` ranks as `CONDITIONAL` — + so `verdict: "BLOCK"` beside `merge_recommendation: "block"` and + `allow_merge: false`, the shape every blocking run writes, raised a + `core_inconsistency:` caveat naming a disagreement that was not there. The + check now compares the textual axes to the published verdict and `allow_merge` + to the flag actually published. +- **`--ci` strictness no longer depends on which preset the run resolves to.** + `--update` outranks `--ci` when the execution preset is picked, so + `prview --ci --fail-on-warnings --update` published `execution_mode: "update"` + — and the exit code read its strictness off that label. Both `--ci` exits, the + `!quality_pass` one and the warning hardening clap insists on `--ci` for, were + therefore inert for exactly the combination CI jobs use. Strictness now follows + the flag the caller typed. On top of that, an `--update` run with no new + commits forced exit `0` outright: it reuses the previous pack and reports it, + so a second invocation turned a warning-carrying — or outright `BLOCK` — pack + green. Such a run now derives its exit from the pack it reused, like every + other run; `--soft-exit` stays the one deliberate way to ask for `0`. + (`output::compute_exit_code` takes the strictness explicitly as a result.) +- **A `MERGE_GATE.json` decision that states nothing is corrupt, not a BLOCK.** + A pack shaped `{"schema_version":"2.2","decision":{}}` passed the CLI's + structural check — the object is there and it is an object — and then + normalized to `BLOCK` and published a summary with `--ci` exit `1`, for an + artifact that never gave a verdict. The other three readers already refused + it: the MCP adapter with `storage_corrupt`, `prview gate` on deserialization, + and `tools/validate_merge_gate.py` on its required fields. The CLI now + requires at least one of `verdict`, `merge_recommendation` or `allow_merge` + and exits `3` without them, so the readers agree on the same pack. Presence is + the test, not recognizability: a stated `verdict: "PROBABLY"` is still read + and still collapses to `BLOCK` with its caveat. +- **A block comment no longer takes a `cfg` guard down with it.** The guard + tracker read `/** Configuration for the a build. */` standing between + `#[cfg(feature = "a")]` and the item it guards as a new item, so both sides of + a diff came out unguarded, the identical declaration text paired as an + unchanged re-add, and a struct that really disappeared for the `a` build + produced no finding at all. Comments are now resolved away before the tracker + reads a line, by the same per-side scanner the declaration accumulator uses: + the comment reaches it as the blank line it is, wrapped over as many lines as + it likes. The same resolution retires the recorded limit on the attribute's + delimiter counter — `/* ))) */` inside a wrapped `#[cfg(any(` predicate no + longer balances the attribute early. Literals stay in that view, because + `#[cfg(feature = "a")]` and `#[cfg(feature = "b")]` are different gates. +- **A const argument in a type no longer ends the declaration.** `pub type Alias + = Buffer<{` opens a const argument, but the accumulator read that `{` as the + item's body opener and finalized there. Both diff sides held the same + truncated prefix, paired as an unchanged re-add, and a changed const + expression on the lines below — a different public type — produced no finding. + A `{` directly after a `<` is now carried to its matching `}`. The rule is + that exact sequence rather than generic-argument tracking, because `<` is also + the shift operator: 4,666 public `const`/`static` declarations in the local + registry state a shift on their own line, against 6 that carry a `<{`. +- **A changed multi-line array constant surfaces again.** An array type states + its length with a `;` — `pub const TABLE: [u8; 2] = [` — and the declaration + accumulator accepted that `;` as the terminator. Both sides of a diff + finalized at their identical opener, paired as an unchanged re-add, and the + changed values below produced no finding at all. Square brackets are now + counted like parentheses before a `;` ends a declaration. +- **A literal spanning two lines is no longer the same value as one with a + space.** The comparison identity joined physical lines with a space, including + the lines a literal spans, so a rewritten public constant paired away as an + unchanged re-add. Lines are now separated by the boundary that separated them. +- **A raw-identifier module is its own scope.** The inline-module parser stopped + at the `#`, recording both `mod r#type` and `mod r#match` as `r`: two + namespaces looked like one, and a removal from the first was cancelled by an + unrelated addition in the second. +- **A comparison inside a const argument no longer holds a test context open.** + The perf tracker counted the `<` of `Buffer<{ 1 < 2 }>` as a generic opener, + leaving the signature depth stuck above zero so the real body brace was read + as another type-level brace. The context never closed and every production + loop and query after the test was muted. A `<` now opens a generic only where + one can be — directly after what it parameterises. +- **Rewording a comment inside a declaration is no longer a signature change.** + Declarations were compared on their verbatim text, comments and all, so a + remove+re-add of a byte-identical public signature whose internal comment had + been rewritten came out as a `ChangedSignature` — a breaking-change claim + about text no consumer can observe. Pairing now compares a comment-free view + of the same lines while `BREAKING_CHANGES.md` keeps showing the declaration as + written. String and char literals stay in that view: a literal is code, so a + changed `pub const GREETING: &str = "hello";` still surfaces. +- **A brace in a test function's signature no longer ends its test context.** + The perf tracker treated the first `{` after a test marker as the item's body + opener, but a brace in type or pattern position — `fn run() -> Buffer<{ LIMIT + }>`, or the extractor idiom `fn handler(Parameters(Req { field }): + Parameters)` — balances before any body exists. The next line then looked + like the item closing again, so the context ended at the signature and every + loop and query in the test body was reported as a production perf regression. + The body opener is now the first brace outside the signature's bracket + nesting. +- **`report.json` names the origin of every quality-failure detail.** + `gate.quality_failure_details[]` carried `name` + `classification` while + `MERGE_GATE.json` has carried `origin` (`"failure"` / `"warning"`) since + schema 2.2, so the two artifacts of ONE run disagreed about what "failure" + meant: a consumer reading `introduced_quality_failures: ["Rustfmt"]` next to + `quality_pass: true` in `report.json` had nothing to reconcile them with. The + field is additive and `report.json` stays `schema_version: "2.0"` — that major + is itself unreleased, so no consumer has ever seen a 2.0 without it. +- **A `cfg_attr` that applies a `cfg` is part of the guard.** The guard filter + recognized only the literal `#[cfg(` spelling, so + `#[cfg_attr(feature = "a", cfg(unix))]` — which gates the item exactly as a + `cfg` does — was dropped from BOTH sides' identity: the declaration text then + paired, and a symbol that really left one configuration produced no finding at + all. `cfg_attr` now joins the conjunction when it applies a `cfg`, and only + then: `#[cfg_attr(unix, derive(Debug))]` decides an attribute on the item, not + the item, and a gate invented there would split an ordinary re-add into a + phantom removal. +- **A trailing `//` no longer swallows the rest of a declaration.** Continuation + lines are joined with a space, and the joined text was then scanned as one + piece — so a comment on any continuation line commented out every line + appended after it. `declaration_complete` never saw the closing `)` or the + body `{`, the accumulator ran on into the body, and a body-only rewrite of a + commented multi-line signature was reported as a `ChangedSignature` that never + happened. Completeness is now decided on a separate view of the same lines, + read one physical line at a time, which ends a `//` where it really ends while + still carrying an open literal or `/* … */` across the lines. +- **Every risky-pattern needle is word-bounded, not just the plain words.** + Bounded matching was applied only to needles made entirely of identifier + characters, so `todo!(`, `dbg!(`, `println!(`, `console.log(`, `unsafe {` and + `as any` kept raw substring matching: `mytodo!(…)` was reported as a TODO + marker and every `has any` in a doc comment as a type cast — the exact + substring false positives bounded matching exists to exclude. Each side of a + needle is now bounded where the needle itself has an identifier edge, which + leaves `.unwrap()` matching `value.unwrap()` and `eslint-disable` matching + `eslint-disable-next-line`. `eprintln!`/`eprint!` are now listed explicitly: + they used to be caught only because `eprintln!(` contains `println!(`. +- **A test marker on a body-less item no longer mutes the rest of its hunk.** + The performance-regression tracker closed a test context only when its opening + brace balanced again, but `#[cfg(test)] mod tests;` and `#[cfg(test)] use + crate::helper;` never open one. The context stayed active for the remainder of + the hunk, so production loops and queries added below such a declaration were + recorded as test-only and disappeared from the signal. A context opened over an + item that ends at its `;` now closes there. +- **A long signature change is no longer swallowed by the accumulation cap.** + Declaration text stopped accumulating after eight continuation lines, which + cuts inside the real distribution of `pub` signatures: two long declarations + that agree on their opener and those eight lines finalized to the SAME + truncated text, so the exact-match pass paired them as an unchanged re-add and + a parameter, bound or return type changed on the ninth line or later produced + no finding at all. The bound is now 32 lines and is documented as what it is — + a runaway valve for static bodies and generated data tables, not a display + width. +- **A `cfg` predicate wrapped across lines still guards its declaration.** The + breaking-change pairing recorded only the opener of `#[cfg(any(`, and the first + continuation line then looked like a new item and cleared the guard: both sides + of the diff came out unguarded, so a `pub` item that really disappeared for one + configuration paired with its re-add under a different one and left no finding + at all — the exact false negative the guard was added to prevent. Attributes + are now accumulated to their balanced close, which also makes a wrapped + predicate compare equal to its single-line spelling, and a wrapped + `#[derive(…)]` no longer takes the `cfg` above it down with it. +- **One verdict vocabulary now answers for every reader surface.** The CLI + matched a stored verdict case-sensitively while the MCP adapter ranked it + through an uppercase fold, so a pack stating `verdict: "pass"` was a clean + `PASS` to MCP automation and an unknown verdict normalized to `BLOCK` on the + CLI — the same artifact approved by one reader and rejected by the other, which + is the divergence the shared reconciliation exists to prevent. `APPROVE` + diverged identically, case aside. A third surface was worse: `prview gate` + compared the folded summary verdict against the pack's RAW string, so any + legacy or non-canonical spelling (`ALLOW`, `HOLD`, `pass`) failed loud as a + "gate verdict mismatch" on a pack both other readers accept. The vocabulary + moved into `gate::canonical_verdict` and all three surfaces fold through it; + `rank_from_verdict` is now derived from it, so ranking and folding cannot drift + apart. `GateVerdict` stays a strict parser of canonical spellings and is fed + the folded value. +- **Raw C string literals are read as raw strings.** The diff scanner accepted + the `r` and `br` raw prefixes but not `cr` (Rust 1.77), so `cr#"…"#` was not + recognized as an opener: the prefix leaked into the code text and the first + interior `"` opened a phantom ordinary string, leaving every brace in the + literal's body to be counted as syntax — the same failure as an untracked + multi-line literal, which pops a `mod` scope early and can cancel a real API + removal. Unlike the raw forms, `b"…"` and `c"…"` escape exactly like an + ordinary string and were already blanked correctly. The construct is real + outside this tree: 38 `cr#"…"#` sites across 11 crates in a 2025-crate + crates.io sample, including `syn` and `proc-macro2`. +- **String literals are tracked across lines, like block comments already were.** + The diff scanner blanked a literal only on the line that opened it, so the tail + of a multi-line template or JSON fixture reached the delimiter trackers as + code: its closing `"` read as an OPENER and the `}` in front of it as syntax. + That popped `mod a` one level early, left a removed `a::Config` with an unknown + scope, and an unknown scope pairs with anything — so an unrelated `b::Config` + addition cancelled a real API removal. The construct is not exotic here: 241 + multi-line literals live in this tree and 168 carry a brace in their body, and + replaying the last 201 commits shows 29 hunk sides whose brace counting this + corrects (21 of them in the scope-popped-early direction, the one that HIDES a + breaking change). The scanner now carries an open normal or raw literal, with + the raw delimiter's own hash count, and forgets it at the same hunk boundary + where it forgets an open comment. The residual cost of carrying is a hunk that + STARTS mid-literal, measured at 1 in 872 over the same history, and it cannot + outlive the hunk. +- **The `cfg` guard of a declaration is the whole stack of attributes above it.** + Stacked `#[cfg(…)]` attributes are Rust's `AND`, but only the last one was + recorded, so `#[cfg(unix)] #[cfg(feature = "x")] pub struct Config;` replaced + by the same struct under `#[cfg(windows)] #[cfg(feature = "x")]` compared equal + on the shared feature alone: the removal paired with the re-add and the API + that disappeared for Unix builds was never reported. The guard is now the + complete conjunction, sorted — reordering two attributes gates the item + identically and is not an API change. +- **Contradictory decision signals are reconciled by conservativeness, not by + field order.** A gate stating `verdict: "BLOCK"` beside + `merge_recommendation: "approve"` is correctly typed and in vocabulary, so + none of the unreadable/unknown guards fired and the CLI simply believed each + field in turn — publishing a `BLOCK` verdict next to an `Approve` + recommendation and, because `compute_exit_code` keys off the recommendation, + exiting `0` on a gate whose own canonical artifact said BLOCK. Both readers now + rank every stated axis through the shared `gate::rank_from_verdict` / + `gate::rank_from_merge_rec` (1 = pass, 2 = hold, 3 = block), publish all axes + from the highest rank, and name the contradiction with a `core_inconsistency:` + caveat. `allow_merge: true` beside `review_required` no longer buys a `PASS` + either, which is the `allow_merge == (verdict == "PASS")` invariant holding on + contradictory packs too. A recommendation outside the vocabulary cannot rank, + so it is excluded and named with `unknown_merge_recommendation:` — the caveat + the MCP surface already emitted and the CLI did not. +- **A gate whose root is not a JSON object is corrupt on both readers.** The + legacy tolerance says WHERE a schema-less pack's decision sits, not that + anything parseable counts as one. A `MERGE_GATE.json` holding an array, a + scalar or `null` was read by the CLI as a decision with every signal missing, + which normalized to `BLOCK` and returned a successful summary — for an artifact + the MCP reader rejected as `storage_corrupt`. Both now fail loud (`exit 3` / + `storage_corrupt`) with a message that names the actual defect. +- **`--ci --fail-on-warnings` counts the warnings it promised to count.** The + flag read `Report.checks` — the list the CLI itself executed — while the + artifact run appends `public_api_diff`, `unsafe_audit`, `ghost_refs` and the + synthetic `heuristics_loctree` to the list `MERGE_GATE.json` is built from, and + none of those ever returns to the CLI. A run whose only warning came from one + of them exited `0` under a flag that promises to fail when any check warns. The + exit now keys off the pack's canonical `checks[]`, and the `--json` summary + states both numbers: `checks_summary.warned` (what the CLI ran) and the new + additive `checks_summary.warned_in_pack` (the complete count), which is never + smaller. A pack with no readable `checks` array falls back to the CLI tally and + says so with an `unreadable_checks:` caveat. +- A warning is no longer reported as a failed quality check. A baseline-signal + check that reports `Warnings` (cargo-audit raising an unmaintained-crate + advisory, `rustfmt`, `eslint`, `ruff`, `prettier`, `stylelint`, `semgrep`) is + admitted to the quality summary so the pre-existing downgrade can be computed + for it — but when it produced no locatable finding it classified as + `unclassified`, which flipped `quality_pass` to `false` and printed + "N quality checks failed" for output that never contained a failure. Warning + entries now carry their origin and are excluded from the failure gate whatever + they classify as: `quality_pass` stays `true`, `decision.analysis_status` stays + `complete` instead of being degraded, the dashboard hero reads + `ALLOW WITH REVIEW` instead of `HOLD`, and the gate reason gets a separate + honest sentence (`2 warning signals: 1 pre-existing, 1 introduced`). Real + failures (`Failed`/`Error`) are unchanged and still fail closed on + `introduced`, `mixed`, and `unclassified`. The origin is now stated on the + wire: `decision.quality_failure_details[]` carries `origin` + (`"failure"` / `"warning"`), which is what lets a reader make sense of + `introduced_quality_failures: ["Rustfmt"]` sitting next to + `quality_pass: true`. This is an additive field, so `MERGE_GATE.json` is + `schema_version: "2.2"` and `tools/validate_merge_gate.py` accepts it — and, + from 2.2, requires it: an entry that omits `origin`, mistypes it, or spells it + anything other than `failure` / `warning` now fails the contract validator, + because a consumer told to filter on `origin == "failure"` cannot do that on a + pack where the field is optional. The validator checks the whole entry, not + only the field that names the schema: `name` must be a non-empty string and + `classification` one of `introduced` / `pre-existing` / `mixed` / + `unclassified`, the vocabulary `QualityFailureClass::as_str` emits. Validating + `origin` alone let `{"origin": "failure"}` — a failure naming no check and + stating no provenance — pass its own contract gate, and let `classification` + drift to any string at all, including the `preexisting` spelling used by the + sibling count field rather than the `pre-existing` the emitter writes. +- Perf regression detection now resolves inline Rust test context (`#[cfg(test)]`, + `mod tests`, `#[test]`) **per hit line** instead of per hunk. A production hot + path that merely shared a hunk with a test module was classified as + `test_context_only` and silently dropped from the reviewer-facing signal + (`perf_regression_suspected` and the risk score both ignore test-only + suspects). Test context now opens at its marker and closes when the braces + opened after it balance out, commented-out markers no longer open it, and any + ambiguity resolves toward production — a false positive costs a reviewer a + glance, a false negative hides a real regression. The scope is read from the + patch's **target state** only: a `#[cfg(test)]` that the patch *deletes* no + longer opens test context over the added production code, and a renamed test + function no longer leaves the scope permanently open (its removed and added + declaration lines each contributed an opening brace while sharing one closing + brace). A hit is now also paired only with a nearby loop in the *same* + context, so a production statement cannot borrow a loop from an adjacent test + module — or the reverse. Trailing comments are stripped before both the marker + match and the brace tracking, so `let x = 1; // #[cfg(test)]` no longer opens + test context and a `{` inside a comment no longer shifts the scope; a `//` + inside a string literal is still code. String and char literals are blanked + for the same reason: `const CLOSE: &str = "}"` in a test module used to close + the scope early and report every later test hit as production, and an + unmatched `{` in a literal held it open and muted real production hits. + Normal, raw (`r#"…"#`) and byte-string literals as well as char literals + (`'}'`, `'\u{7b}'`) are recognised; lifetimes are not mistaken for char + literals. Block comments count too, and they are tracked ACROSS lines — + commenting a block of code out is exactly how an unbalanced brace ends up + inside a comment, and a `/* … } … */` spread over three lines closed the test + scope early (or, with a `{`, held it open and muted real production hits). A + `/*` inside a string literal stays data: `format!("{}/*.{}", dir, ext)` is a + glob pattern, and reading it as a comment opener would swallow the rest of the + hunk — a far more common line in real diffs than a block comment is. A *string* + literal spanning several diff lines is carried the same way a block comment is: + the scanner keeps one open across lines, so a brace inside a multi-line + template or JSON fixture never reaches a delimiter tracker as syntax. What ends + the carrying is the hunk boundary, where the text stops being contiguous. +- Breaking-change detection pairs duplicate declarations one-to-one. `cfg`-gated + variants share (file, kind, name), and the pairing search never consumed its + match, so every removal cancelled against the same unchanged re-add: the + addition that actually replaced one of them stayed unpaired and its signature + change was never reported, while a genuine removal could be cancelled by an + addition already spent on another. Exact matches are now claimed first, each + addition is consumed once, and one cancelled removal retires exactly one + finding. +- Breaking-change detection no longer loses a removal to a same-named symbol in + another inline module. `pub mod a { pub struct Config }` deleted while + `pub mod b { pub struct Config }` is added in the same file was cancelled as a + no-op remove+re-add; the pairing now also requires compatible inline-module + scopes (tracked per diff side, hunk-local — an unseen `mod` opener leaves the + scope unknown and pairs as before). That module tracker now reads code only: + a brace inside a comment or a string/char literal (`// }`, `"{"`, `'}'`) used + to open or close a module scope that does not exist, so a removal and its + unrelated same-named addition landed in the same phantom scope and cancelled + each other — the breaking change vanished from the report. The literal/comment + scanner is shared with perf-regression test-context tracking (`rust_source`), + so both brace trackers agree on what counts as syntax, block comments spanning + lines included. +- Breaking-change detection no longer cancels a removal against a re-add under a + DIFFERENT `cfg`. `#[cfg(feature = "a")] pub struct Config;` replaced by the + same struct under feature `b` is an exact text match, so the pairing dropped + the removal — but `Config` really did disappear for anyone building with + feature `a`. The guard standing above a declaration is now part of its pairing + identity (whitespace-insensitive, so a reformatted attribute is not a + different predicate). A guard the diff never showed on one side stays unknown + and pairs as before, the same tolerance an unseen `mod` opener gets: the + attribute often sits on a context line, and reading "not shown" as "no cfg" + would turn ordinary re-adds into phantom removals. +- A public declaration no longer ends at a delimiter inside its own literal. + `pub const TEMPLATE: &str = r#"{` opens a multi-line raw string, and reading + that `{` as the declaration's body opener finalized a truncated declaration — + identical on both diff sides, so the removal was cancelled and the literal + change the patch actually made produced no finding at all. Completion is now + judged on code only, and the accumulated text is scanned as a whole, so a + literal spanning continuation lines closes the declaration where it really + ends. +- Multi-line public declarations are compared in full. `pub struct Config<` with + a changed bound on the next line used to hide behind its identical opening + line, because only that line was compared. Continuation lines are now + accumulated on both diff sides — for every symbol kind, not just `pub fn` — + up to 8 lines, and `BREAKING_CHANGES.md` shows the full declaration. +- `BREAKING_CHANGES.md` no longer collapses two different symbol kinds into one + row. Changed signatures were grouped by (file, name); now that non-fn + declarations also produce signature changes, a `pub struct Limit` and a + `pub const Limit` in one file were rendered as one row plus a bogus + "feature-gated variant" note. The grouping key now carries the symbol kind. +- The pattern scan no longer reports an identifier as a TODO marker. Word + boundaries were read byte-wise over ASCII only, so `$` — an identifier + character in JavaScript/TypeScript and the macro metavariable sigil in Rust — + and every non-ASCII letter counted as a boundary: `const $TODO = false` and an + identifier abutting a Unicode letter were both reported, inflating `prod_hits` + and the risk score with exactly the false positives bounded matching exists to + exclude. Boundaries are now read per character over the union of identifier + characters the scanned languages accept. +- A skipped `semgrep` run keeps its diagnostic. The tool/config-error skip reason + was built from stderr alone, but under `--json` semgrep reports rule and config + failures in the stdout payload's `errors[]` and can leave stderr empty — so the + one explanation available was discarded and the policy engine received the bare + "semgrep exited 2 with no findings payload" sentence. The excerpt is now taken + from stderr, else the payload's `errors[]` (reading `message` / `long_msg` / + `short_msg` / `type`, whichever the semgrep version emits), else raw stdout, so + a crash traceback printed on stdout also survives. +- `report.json` distinguishes a disabled heuristics run from a broken scanner. + `--quick` and `--no-heuristics` short-circuit the scan to a default result + that the caller still passes on, so the report described the intentional skip + as `skip_reason: "loctree analysis unavailable"` — a tool failure that never + happened — and pointed `log_path` at a zero-filled stub, while the + `"heuristics not run"` reason was unreachable from the production path. A run + that never asked for heuristics now reads `heuristics not run` and omits both + `total_files` and `log_path`. No field changed shape, so `report.json` stays + `schema_version: "2.0"`. +- Coverage no longer reports an unmeasured scan as perfect. A diff with zero + changed source files produced `0/0 (100%)` in `AI_INDEX.md`, + `coverage-delta.txt`, and the dashboard; it now reads `not measured`, and the + coverage card/chip/section is omitted instead of showing a fabricated 100%. + A real `0/N` (N > 0) is still a genuine `0%` measurement. +- `report.json` no longer zero-fills skipped analysis. `quality.heuristics` now + carries `status` (`"measured"` / `"skipped"`), an optional `skip_reason`, and + `total_files`; a loctree run that scanned no files (or never ran) omits + `dead_exports`, `cycles`, `twins`, and `unused_symbols` instead of emitting + zeros indistinguishable from a clean scan. This matches the SKIP semantics + `MERGE_GATE.json` and `heuristics_loctree.result.json` already used. - Cached check results now carry provenance. A cache hit used to return `provenance: None`, so the fastest runs — the ones where every gate is served from cache — were the only ones with no audit trail at all: no command, no @@ -326,23 +1200,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 as `target_sha`, because sitting below `repo_root` was taken as proof. Such a directory is `foreign` wherever it sits. -### Changed - -- Bumped the bundled `loctree` structural-analysis crate from `0.8` to `0.13.0`. - The public API prview consumes (`analyzer::{cycles, dead_parrots, twins}`, - `snapshot::{Snapshot, project_cache_dir, run_init, SNAPSHOT_SCHEMA_VERSION}`, - `args::ParsedArgs`) is source-compatible — no call sites changed. The snapshot - schema version is now decoupled from the crate version (pinned at `0.11.0` - instead of tracking `CARGO_PKG_VERSION`); prview's `major.minor` schema gate - handles the transition, so stale `0.8`-era caches are re-scanned automatically. - loctree 0.13 also widens file-type coverage in the scan (markdown, shell, - config, and other non-source files now count toward the snapshot), so the - `LOCTREE` heuristics stats (`total_files`, `total_loc`, `by_language`) report - higher, broader numbers than under 0.8 for the same tree. - ### Security -- bump ammonia 4.1.3 → 4.1.4 (RUSTSEC-2026-0213: XSS via SVG `animate`/`set` attributes) +- bump ammonia 4.1.3 → 4.1.4 (RUSTSEC-2026-0213: XSS via SVG `animate`/`set` attributes) ## [0.6.0] - 2026-07-07 diff --git a/docs/architecture.md b/docs/architecture.md index 7d9c7d2..7e00afb 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -709,13 +709,433 @@ Types and functions used across multiple signal modules: Heuristic scan of diffs for API-breaking changes: - `BreakingRisk` enum (`High`, `Medium`, `Low`) — publicness heuristic based on file path depth and barrel/re-export file detection -- `BreakingFinding` struct with `BreakingKind` (`RemovedSymbol`, `ChangedSignature`, `NewEnvRequirement`) +- `BreakingFinding` struct with `BreakingKind` (`RemovedSymbol`, `RelocatedSymbol`, `ChangedSignature`, `NewEnvRequirement`) - `analyze_all_breaking_changes(patches)` — returns all findings from multiple patch texts - `write_breaking_changes(dir, findings)` — writes `BREAKING_CHANGES.md` if findings are non-empty Scans for removed `pub` symbols (fn, struct, enum, trait, type, const, static), -JS/TS `export` removals, signature changes (same function name with different params), -and new environment variable requirements. Only scans code files (not tests, config, docs). +JS/TS `export` removals, signature changes, and new environment variable +requirements. Only scans code files (not tests, config, docs). + +Remove + re-add pairing (applies to every `pub` symbol kind above, not just +functions): when a declaration is removed and re-added for the same name and +kind, an identical declaration line is a diff artifact and yields no finding, +while a changed one yields a single `ChangedSignature` — never a removal plus a +silent re-addition. A re-add in a *different* file is a module move and becomes +`RelocatedSymbol`, which is reported but deliberately excluded from breaking +escalation. + +**Accepted limit (deferred to 0.8).** The scan sees only the declaration LINES a +diff emitted. An enum variant, a trait method or a struct field removed below an +unchanged `pub enum` / `pub trait` / `pub struct` opener is a breaking change +prview does not report: the opener was never emitted as `-`/`+`, so nothing +enters pairing to begin with. Reporting it needs the item's body from BOTH +commits, which a diff-only scanner does not have — the fix is the repo-backed +breaking analysis planned for 0.8, and this limit was reviewed and accepted +deliberately rather than papered over with a deeper heuristic. + +Declarations are compared on their FULL text, continuation lines joined, up to +`MAX_DECL_CONTINUATION_LINES` (32) — a runaway bound for static bodies and +generated data tables, not a display width. The bound used to be eight lines, +which cut inside the real distribution of `pub` signatures: two long +declarations agreeing on their opener and first eight lines finalized to the +same truncated text and paired as an unchanged re-add, so a parameter, bound or +return type changed below the cut produced no finding at all. + +A hunk interleaves two texts, and the accumulators reconstruct both: the before +side is context ∪ removed lines, the after side is context ∪ added lines. So a +`-` line never touches the added accumulator, a `+` line never touches the +removed one, and a context line EXTENDS whichever side still has a declaration +open. Ending both accumulators at the first line from the other side truncated +every declaration a patch edits in place — `pub fn f(` and a shared `x: u8,` +followed by `-old: u16,` / `+new: u32,` finalized to two identical openers, +paired as an unchanged re-add, and the parameter change was reported nowhere. +Context lines only ever CONTINUE a declaration; a `pub` item that first appears +on one is unchanged by the patch and must not open an accumulator, which keeps +the reconstruction inside the hunk that emitted it. The bound is unchanged: +`MAX_DECL_CONTINUATION_LINES` still caps growth, and a hunk header or a new file +still finalizes both sides. + +Where a declaration ENDS is decided on a separate, comment-resolved view of the +same lines, fed through the pending declaration's own `SourceScanner` one +physical line at a time. The joined text has no line breaks, so scanning it as a +whole let a `//` on any continuation line comment out every line appended after +it: the closing `)` and the body `{` were never seen and the accumulator ran on +into the body, so a body-only rewrite came out as a phantom `ChangedSignature`. +Feeding line by line ends a `//` where it really ends while the scanner still +carries an open literal or `/* … */` across the continuation lines. + +`BREAKING_CHANGES.md` renders those declarations into markdown tables, so every +cell carrying source text has its `|` escaped as `\|`. Rust states bitwise or, +patterns and closures with that character, and a declaration like +`pub const MASK: u32 = READ | WRITE;` written verbatim opened new columns — +the row rendered as garbage exactly where the declaration was interesting. +GitHub's table parser splits on unescaped pipes before any inline markup runs, +so a code span is no protection. The span itself is fenced by a backtick run +LONGER than any inside the cell, because a declaration may state a backtick of +its own — `pub const TEMPLATE: &str = r#"`value`"#;` — and a single-backtick +span ends at the first interior one, leaving the rest of the declaration to +render as prose. When the content itself begins or ends with a backtick the +fence carries one space of padding, which CommonMark strips back off. + +Display text and comparison identity are separate. `BREAKING_CHANGES.md` and a +`ChangedSignature` show the declaration verbatim, comments included, because a +reader shown a change should see the source as written; pairing COMPARES a +comment-free view of the same lines. A comment inside a declaration is not part +of the API, and rewording one used to come out as a `ChangedSignature` — a +breaking-change claim about text no consumer can observe. That view keeps string +and char literals verbatim: a literal is code, so `pub const GREETING: &str = +"hello";` and the same line ending `"bye";` must stay different declarations. +`SourceScanner` therefore offers both resolutions — `code_only` for the +delimiter trackers, which want a brace inside a string silenced, and +`code_with_literals` for callers comparing source. + +The identity keeps a physical line break only where the break is part of the +value — that is, where the previous line left a string literal open. A literal +spanning two physical lines otherwise compared equal to the same literal +rewritten with a space in it, and a changed public constant paired away as an +unchanged re-add. Everywhere else the break is layout and the lines are joined +with a space: keeping it there made `pub type Alias =` followed by `u32;` a +different declaration from `pub type Alias = u32;`, so a purely cosmetic reflow +was reported as a `ChangedSignature` whose "before" and "after" were the same +string. By the same rule a line contributing no code is dropped from the +identity — a comment-only line says nothing about the API — unless a literal is +open, where a blank line is a blank line in the value. + +Whitespace at a line's edges follows the same rule, one edge at a time: the +leading edge is kept when the PREVIOUS line left a literal open, the trailing +edge when THIS line does. Lines reached the accumulator already trimmed, so +re-indenting the inside of a multi-line public constant produced two identical +identities and the changed value paired away as an unchanged re-add. Trimming +neither edge would be worse in the other direction — every reflow would become a +phantom `ChangedSignature`, and a trailing comment's leading gap would make `a: +u8, // x` a different declaration from `a: u8,// y`. The per-edge rule keeps +whitespace only where a literal is open across it, which is exactly where it is +part of the value: measured over the local crates.io registry, of 200,553 +multi-line public declarations only 640 continuation lines sit at a literal edge +at all, and 272 of those carry edge whitespace the old view dropped. No +formatter re-indents inside a string literal, because that changes the program. + +A declaration ends at a `;` or a body `{` outside its brackets. Square brackets +count for the same reason parentheses do: an array type states its length with a +`;`, as in `pub const TABLE: [u8; 2] = [`, and reading that as the terminator +finalized both diff sides at their identical opener — the changed values below +produced no finding at all. Measured over the local crates.io registry, 719 +public `const`/`static` declarations in 126 crates open a multi-line initializer +on a line whose type carries such a `;`. + +A `{` in type position is not that body brace. `pub type Alias = Buffer<{` opens +a const argument, and finalizing there truncated both diff sides to the same +prefix, so a changed const expression below — a different public type — paired +away as an unchanged re-add. The scanner tracks the generic argument list +itself, so a const argument that is not the first one — `Buffer`, or a `:`, `->` never +closes one, and `<<` is consumed whole, because `<` is also the shift operator +and the 4,666 public `const`/`static` declarations in the local registry that +state a shift on their own line must still terminate at their `;`. Measured over +that registry (59,946 files, 2,025 crates, 4,354,142 public declaration lines), +argument-list tracking and the narrower `<{` sequence rule it replaced judge zero +lines differently. + +The `:` in that rule admits the TURBOFISH spelling of the same construct. +`pub fn run() -> Buffer::<{` is a valid return type — rustc accepts `Type::<…>` +in type position without a warning — but its `<` follows a `:`, so the list went +uncounted, the const block's `{` read as the item's body opener, and both diff +sides finalized at that identical prefix. They paired as an unchanged re-add and +a changed const argument, which is a changed public return type, was reported +nowhere: the direction that HIDES a break. Only `:` joins the openers, never +whitespace, so a comparison is still not a list. The widening is verdict-neutral +where it is not needed: `::<` appears on 557 public declaration lines in the +registry and a `:` immediately before a `<` on exactly one — inside a string +literal, which the code-only view never shows this scanner — and running both +rules over all 4,334,018 public declaration lines produces zero disagreements, +because a turbofish that closes on its own line (`size_of::()`) nets to the +same depth counted or ignored. What changes is the list left OPEN at end of line, +which the accumulator carries into the next one; two public declarations in the +registry wrap a turbofish that way today. + +INSIDE that const argument the same characters are operators, so the generic +depth is frozen there. `pub fn run() -> Buffer<{ 1 < 2 }> {` counted the +comparison as another opener, the argument list's own `>` closed only that +phantom level, and the depth was still above zero at the item's real body brace +— which therefore read as a further const argument and swallowed the body, +turning a body-only rewrite into a phantom `ChangedSignature`. Freezing costs +nothing, because whatever a const block states about generics is balanced +against itself: a turbofish (`{ size_of::() }`) and a qualified path +(`Uint<{ ::LIMBS / 2 }>`, the shape crypto-bigint carries) close what they +open. Those are also the only shapes the corpus holds — across 58,614 files, +61 declaration or field lines put a const argument's braces around a `<` or `>`, +43 of them a turbofish and 18 a qualified path or a shift, and none a bare +comparison. The previous rule survived all of them by cancellation, the block's +stray `>` closing the outer list and the outer list's `>` then finding nothing +left; the freeze reaches the same verdict by construction instead. + +Nor is the `{` of an initializer that body brace. After a top-level `=` the item +states a VALUE and runs to its `;`, so `pub const LIMIT: usize = {` and +`pub const ZERO: Self = Self {` open an initializer, not an item body — and a +`;` inside it terminates a statement, not the declaration. Finalizing at that +brace truncated both diff sides to their identical first line, they paired as an +unchanged re-add, and a changed expression inside the block produced no finding. +Only a TOP-LEVEL `=` counts: inside a generic argument list one states a default +(`struct Foo`) or an associated type +(`impl Iterator`), and both are followed by a body brace that must +still end the declaration; `==`, `=>` and the compound assignments are excluded +too. This is the widest of the brace rules by frequency — measured over the +local registry (58,586 files, 1,960 crates, 4,334,320 public declaration lines), +2,465 lines change verdict, every sampled one a public constant whose +initializer is a struct literal or block spanning several lines. What such a +declaration accumulates is still bounded by `MAX_DECL_CONTINUATION_LINES`. + +Inline module names keep their raw-identifier prefix. `mod r#type` and +`mod r#match` were both recorded as `r`, so two namespaces looked like one and a +removal from the first was cancelled by an unrelated addition in the second. + +Pairing is scoped: two declarations pair only when their declaration site and +their `#[cfg(…)]` guard may be the same. The site is the inline `mod` path AND +the `impl` owner, tracked on one stack because they answer one question — which +namespace does this item belong to? An associated `pub const VALUE` moving from +`impl A` to `impl B` in the same file used to be an exact pairing on file, kind, +name and text, with an empty scope on both sides, so `A::VALUE` disappeared from +the report along with the removal. The owner is recorded as TEXT — everything +before the body brace, whitespace collapsed — and nothing about it is parsed. +The asymmetry is the whole rule: two KNOWN and different owners never pair, but +an owner the hunk did not show stays unknown and pairs with anything, which is +the accepted limit for an unseen opener and is not narrowed here. Over 211 +commits of this repository the reports are identical with and without the owner; +over 708 crates.io release pairs removals move 30,555 → 30,694 and signature +changes 53,938 → 53,805, so the change mostly reclassifies a real owner change +from "signature changed" to "`A::x` removed". Its limit is the mirror of the +`cfg` one: the same owner written with a different path qualifier reads as two +(40 of 2,784 blocked pairings, all in one crate), and resolving that would mean +parsing types. The guard tracker resolves comments away +before it reads anything, with one per-side scanner reset with the guard, so a +block comment is not syntax on either count: `/** … */` standing between the +`cfg` and the item it guards no longer reads as a new item and clears the guard, +and `/* ))) */` inside a wrapped predicate no longer balances the attribute +early. The guard TEXT keeps literals, because `#[cfg(feature = "a")]` and +`#[cfg(feature = "b")]` are different gates and a literal-dropping view would +make them one; the DELIMITER COUNT drops them, from a second scanner walking the +same lines in step. Counting a literal's brackets as syntax broke the tracker +both ways. A `)` typed inside a multi-line `#[doc = r#"…"#]` balanced the +attribute early, and the literal's remaining lines then read as ordinary items +and cleared the pending `cfg`. The reverse cost more: the counter carried its +literal state per LINE, so a literal opened earlier was forgotten and its own +closing quote read as an opener — `#[must_use = "… \` continued onto the next +line swallowed the `]` after `…"`, the attribute never closed, and everything +below it, the real `#[cfg(…)]` included, was absorbed as continuation. Measured +over the local crates.io registry, of 237,368 `cfg`-guarded attribute runs +reaching a public declaration, 8,793 wrap across lines, 90 carry a literal +spanning the break, 13 of those a raw string, and 9 balanced wrongly under the +per-line state — all 9 of the line-continuation shape, in `rustix` and +`wit-bindgen`. + +Spacing follows the same split. Whitespace outside the literals is formatting — +`#[cfg(feature="a")]`, `#[cfg(feature = "a")]` and the same predicate wrapped +across four lines are ONE guard, and reading them as three would report removals +that never happened. Whitespace inside a literal is value: `--cfg 'api="a b"'` +and `--cfg 'api="ab"'` are different configurations, so the two attributes are +different gates. The tracker stripped it from the whole attribute text, literals +included, which made those two one guard and paired a struct that really left +one configuration with its re-add under another. The strip is now the scanner's +own dense view, so it cannot reach inside a value while normalizing layout. +This one is a fix by construction rather than by frequency: of 524,530 gating +attributes in the local registry only 3 carry whitespace inside a value literal, +and none of them collide. Both directions still cost what they always did, and +the hiding one is not worth leaving open for a one-line rule. + +Those were three findings of one shape — a delimiter, a space, a line break — +so the rule is stated once as an INVARIANT rather than patched a fourth time: +**bytes inside a literal traverse the whole attribute pipeline verbatim.** It +holds by enumeration, not by testing shapes, because the pipeline alters text in +exactly two places and both defer to the same `ScanState`. The delimiter count +runs on `code_only`, a view with no literal bytes in it at all, so trimming or +counting there cannot reach a value. The guard text runs on +`code_with_literals_dense`, whose only subtractive rule sits in the one `scan` +arm reached solely outside every literal and comment; each literal is emitted as +an unmodified slice. Everything downstream — `gates_the_item`, the sort, the +dedup, `cfgs_may_pair` — compares whole strings and transforms nothing. So the +line's raw bytes now reach the tracker (trimming at the caller ate a +continuation's indentation before it could be asked whether it was inside a +value), no `.trim()` survives inside the tracker (after the dense view there is +no whitespace left outside a literal, so a trim there could only eat value), and +the physical break is joined with a `\n` exactly when the previous line left a +literal open — gluing it unconditionally made `#[cfg(api = "a\nb")]` the same +guard as `#[cfg(api = "ab")]`. Measured like its siblings and just as rare: of +568,128 `cfg` attributes in the local registry, 4 carry a literal spanning a line +break, 2 of them gate an item, and none collide. + +The guard is the WHOLE conjunction of +the attributes stacked above the declaration, sorted — `#[cfg(unix)] +#[cfg(feature = "x")]` and `#[cfg(windows)] #[cfg(feature = "x")]` are different +guards, while reordering the same two is not. An unseen scope or guard is +`None`, which pairs with anything: the diff may simply not have re-emitted the +context line on that side. + +An attribute is read to its balanced close, not to the end of its first line. A +predicate wrapped as `#[cfg(any(` + feature lines + `))]` is one guard, equal to +its single-line spelling — whitespace and line breaks are formatting, not a +different gate. Reading only the opener left the continuation line looking like a +new item, which cleared the guard and let a declaration that really disappeared +for one configuration pair with its re-add under another. Any other wrapped +attribute (`#[derive(…)]`) is carried the same way so it cannot take the `cfg` +above it down with it; an attribute that never closes within +`MAX_ATTRIBUTE_CONTINUATION_LINES` falls back to the tolerant `None`. + +`cfg_attr` counts as a guard exactly when it applies a `cfg`: +`#[cfg_attr(feature = "a", cfg(unix))]` gates the item as surely as `#[cfg(unix)]` +does, so it joins the conjunction, while `#[cfg_attr(unix, derive(Debug))]` decides +an attribute ON the item and stays out — inventing a gate there would split an +ordinary re-add into a phantom removal. The two families separate cleanly on the +whitespace-stripped substring `,cfg(`: of the 44,562 `cfg_attr` attributes in the +local crates.io registry, 189 apply a `cfg` (12 crates, the `portable-atomic` +idiom) and none of them carry that substring inside a string literal. + +Reordering operands INSIDE one predicate is an accepted limit. Stacked +attributes are sorted, so `#[cfg(unix)] #[cfg(feature = "x")]` pairs either way +round, but `#[cfg(any(unix, windows))]` rewritten as `#[cfg(any(windows, unix))]` +is compared as text, does not pair, and reports a phantom `RemovedSymbol` under +an untouched declaration. Normalizing it means canonicalizing arbitrarily nested +predicates — a `cfg` parser — and the measurement says the parser would not earn +its risk. Across 708 consecutive-version pairs in the local registry, whole +releases and so far wider than any diff this scanner reads, 32 `cfg` attributes +were reordered at all, in 2 crates; across the 393 patch-level bumps among them, +the closest available proxy for a PR-sized change, zero. Of the 32, only 6 are +reachable by sorting an attribute's direct operands: the dominant real shape is +`not(any(a, b, c))`, with the reorder one level down. A bounded sort would close +a fifth of an already absent class while looking complete, which is worse than a +limit written down — and the error direction here is the tolerable one, a +phantom removal being visible in review rather than a real removal pairing away +in silence. + +Both the module path and the perf tracker's test-context scope are counted over +CODE only, via the shared scanner in `src/rust_source.rs`. It resolves comments +and literals in ONE pass — `"http://x"` is a string and `format!("{}/*.{}")` is +a glob, not a comment — and carries an open `/* … */` **or an open string +literal** across lines, so a brace inside a multi-line template or JSON fixture +never reaches a delimiter tracker as syntax. Every raw form is recognized — +`r`, `br` and `cr`, with any hash count — because an unrecognized raw opener is +worse than an unknown token: its body is then read as code, and the first +interior `"` opens a phantom literal. That state is per side and per hunk: a +hunk boundary is where contiguity ends, and every consumer resets there. + +What OPENS that context has to be provable, not merely suggestive. The marker +set is an exact `#[cfg(test)]`, a `#[cfg(all(…))]` with `test` among its +operands — which cannot hold unless `test` does — `#[test]` / `#[tokio::test]` / +`#[rstest]`, and +`mod tests`. Reading the bare token `test` anywhere inside a `cfg` predicate +instead made `#[cfg(not(test))]` — code compiled into every build EXCEPT the +test one — open test context and silently drop the production hits beneath it, +and did the same for `#[cfg(any(test, feature = "bench"))]`, which compiles +outside the test build whenever the feature is on, and for +`#[cfg(feature = "__internal-test")]`, a feature that merely has `test` in its +name. Measured over the local registry (58,586 files): of the 11,030 attributes +the old pattern read as test context, 83.62% are exactly `cfg(test)` and 6.76% +are `all(…, test, …)`; the remaining 9.62% are the ones it got wrong. `all` is +commutative, so the operand's position carries no meaning and reading only the +first one made `all(feature = "bench", test)` production while +`all(test, feature = "bench")` was test context; accepting it anywhere adds 72 +attributes over that registry and removes none. The operand must be a DIRECT +one, so nothing before it may open a nested predicate — `all(not(test), …)` +proves the opposite of itself and `all(any(test, …), …)` proves nothing. That +also drops `all(not(windows), test)`, an under-detection kept deliberately +rather than growing a paren-matching parser. Everything +unproven is production, because the two errors are not symmetrical — an +unrecognized test context costs one extra finding a reader can dismiss, while a +claimed one that does not hold deletes a production finding nobody ever sees. + +The pattern describes a complete `#[…]`, so it is matched against a complete +one. Attributes wrap — rustfmt breaks a long predicate over several lines — and +running the pattern per physical line meant a wrapped +`#[cfg(all(` / `feature = "bench",` / `test` / `))]` matched on no line at all, +leaving a test-only item read as production. An `AttributeAccumulator` joins the +lines of one attribute and matches once, on the line that closes it, counting +brackets on the same comment- and literal-resolved view the rest of the scan +uses so a `]` inside a string or a trailing comment cannot close it early. It +only ever CONTINUES: a wrapped attribute is bounded by +`MAX_ATTRIBUTE_CONTINUATION_LINES` (8), and one that never closes is dropped +rather than allowed to swallow the rest of the hunk — nothing was proven, and an +unproven gate is production. The shape is rare: 10 occurrences over the same +58,614-file registry, all of them genuine `all(test, …)` gates. It is a P2 +because its error direction is the mild one — a phantom finding a reader can +dismiss, not a muted production hit. + +An attribute's brackets are its own, and the brace scan skips them by tracking +attribute depth per character. `#[rstest]` stacked with +`#[case(Case { id: 1 })]` carries braces that belong to the attribute, never to +the annotated item, and letting them through made the `{` a body opener whose +`}` closed the test context on the same line — the test function below then read +as production. The plain shape survived by luck, because the attribute's `[` and +`(` hold `sig_depth` above zero; two clamping `>` comparisons +(`#[case(1 > 0, 2 > 1, Case { id: 1 })]`) drive it back to zero first and the +brace lands where the opener is accepted. Skipping the attribute outright +removes the class instead of the one shape that reaches it. This is separate +from the line-level `AttributeAccumulator` above and deliberately so: that one +answers "is this LINE part of an unterminated attribute" for the marker match, +while the brace scan needs "is this CHARACTER inside one". The depth persists +across lines, since attributes wrap; literals are already resolved away, so a +`]` inside a string cannot close one early. + +The perf tracker's test context closes two ways, because not every test item has +a body. One that opens a brace closes when that brace balances again; one that +does not — `#[cfg(test)] mod tests;`, `#[cfg(test)] use crate::helper;` — closes +at the `;` ending the item the marker annotates. Waiting for a brace that never +comes left the context open for the rest of the hunk, and every production loop +and query below it was recorded as test-only and dropped from the signal. + +Which brace opens that body is decided against the signature's bracket nesting, +not by taking the first `{`. A brace in type or pattern position — +`fn run() -> Buffer<{ LIMIT }>`, or the extractor idiom +`fn handler(Parameters(Req { field }): Parameters)` — balances before any +body exists, so reading it as the opener made the very next line look like the +item closing again: the context ended at the signature and the whole test body +was classified as production. Inside a signature `<` is reliably a generic +opener — but only where one can be: a `<` counts as opening a generic when it +FOLLOWS what it parameterises (`Buffer<`, `Vec<`, `fn f<`, `::<`), and a `<` +after whitespace is a comparison. `->` is excluded so a return arrow is not read +as a closing angle bracket; closers stay unconditional and the depth is clamped +at zero, so a `<` this rule misjudges — like a hunk starting mid-signature — can +only end the context early, never hold it open and mute production code. +Measured over the local crates.io registry: of 1,697,077 `fn` signatures, 1,191 +carry a brace in that position and 715 place the body opener on a later line — +the shape that actually breaks the tracker — 59 of them test-annotated. + +Spacing is where that rule stops being a boundary, so the boundary is drawn +around it: signature tracking is FROZEN inside a brace opened within those +brackets. A const argument holds an expression (`Buffer<{ 1 < 2 }>`) and a +destructured parameter holds a pattern, and in both `<` and `>` are operators. +The spacing heuristic reads the spaced spelling correctly and the compact +`Buffer<{1<2}>` — the same type, formatted without spaces — wrongly, because `<` +after a digit is indistinguishable from `<` after an identifier. Counting that +comparison left the depth stuck above zero, the real body brace read as another +type-level brace, and the context never closed, muting every production hit +after the test — the direction that HIDES work. Freezing costs nothing, because +what such a brace states about generics closes what it opens. Measured over the +same registry, of the 618 `fn` signatures whose brackets hold a brace, 6 put a +`<` or `>` inside it, all of them the qualified path +`Uint<{ ::LIMBS / 2 }>` as crypto-bigint writes it, and none the compact +comparison. Those 6 reached the right verdict before only through the clamp — +the path's `>` closed the outer list, and the outer list's own `>` was then +clamped away — and now reach it by construction. + +The item's own top-level `=` is the second such boundary, and by frequency the +larger one. After it the item states a VALUE, so both angle characters are +operators, and tracking is frozen for the rest of the item. A body-less item ends +at its `;` — the close that tests whether the signature's brackets are balanced — +so a counted comparison there could not be undone by anything: the context stayed +open and every production hit below the test was recorded as test-only, which +over-detects test context and HIDES work. `#[cfg(test)] const ENABLED: bool = +1<2;` is the reported shape, but the corpus idiom is the compact SHIFT, because +this tracker (unlike the declaration scanner) has no rule consuming `<<` whole. +Measured over the local registry on the same code-only view the tracker reads, +excluding lines with lifetimes, which this model cannot lex: of 2,206,540 +single-line `const`/`static`/`type` declarations ending at their own `;`, 1,069 +left the bracket depth stuck open under the old rule — dominated by +`const Reverse = 1<<8;` as objc2 generates its bitflags — and 64 still do. Those +64 are not a residual bug but the protection working: an array type wrapping to +the next line (`pub static X: [[u16; N];`) must hold its depth open so that `;` +is not mistaken for the end of the item. #### signal/coverage.rs — coverage delta computation @@ -727,6 +1147,29 @@ Cross-references changed source files with test files to estimate test coverage: - `CoveragePair` struct — a matched (source file, test file) pair with the match strategy used - `compute_coverage_signal(diffs, repo_root, repo)` — the canonical computation function - `generate_coverage_delta(dir, signal)` — renders `coverage-delta.txt` from a pre-computed signal +- `format_coverage_pct(Option)` — the one renderer for the percentage; `None` becomes `not measured` + +**Unmeasured is not 100%.** `coverage_pct` / `CoverageDelta::pct` are +`Option` and are `None` whenever no changed source file was evaluated +(`total_source_files == 0`). Consumers must render that as "not measured" or +omit the coverage surface entirely — never as a percentage. A real `0/N` +(N > 0) stays a genuine `0%` measurement. In `report.json`, +`quality.coverage.heuristic_ratio` is `null` in the unmeasured case and is +paired with `measured: false` + `not_measured_reason`. That nullability — with +the loctree counters becoming omittable for the same reason — is why +`report.json` carries `schema_version: "2.0"`: a decoder written against `1.0`, +where the ratio was always a number, does not parse every pack. + +`report.json`'s `gate.quality_failure_details[]` mirrors `MERGE_GATE.json`'s +`decision.quality_failure_details[]` field for field — `name`, `classification` +and `origin` (`"failure"` or `"warning"`). The origin is what makes +`introduced_quality_failures: ["Rustfmt"]` and `quality_pass: true` readable +together: the arrays admit warning-level baseline signals so the pre-existing +downgrade can be computed for them, and only a `"failure"` origin can fail the +quality gate. Emitting it in the gate artifact but not in `report.json` left the +two artifacts of one run disagreeing about what "failure" meant. The field is +additive and `report.json` stays `schema_version: "2.0"` — that major is +unreleased, so no consumer has ever seen a 2.0 without it. Four-strategy filename heuristic matching: 1. Exact stem match: `foo.rs` <-> `foo_test.rs` / `test_foo.rs` / `foo.test.ts` @@ -789,10 +1232,22 @@ Scans added lines in diff patches for 11 risky patterns: - `generate_pattern_scan(dir, diffs, repo)` — produces `PATTERN_SCAN.json` with per-pattern aggregation, prod/test split counts, and sample contexts -Scanned patterns: `unwrap`, `println`/`print`, `dbg`, `todo`/`FIXME`/`HACK`/`XXX`, +Scanned patterns: `unwrap`, `println`/`print`/`eprintln`/`eprint`, `dbg`, +`todo`/`FIXME`/`HACK`/`XXX`, `@ts-ignore`/`@ts-expect-error`/`@ts-nocheck`, `eslint-disable`, `console.log`/`error`/`warn`, bare `catch`, `unsafe`, `#[allow(...)]`, `as unknown as`/`as any`. +Every needle is matched with word boundaries, and each side is bounded only +where the NEEDLE has an identifier edge — that is the only side a longer +identifier can swallow it from. `todo!(` is already right-bounded by its `(` +but must be left-bounded, so `mytodo!(…)` is not a TODO marker; `.unwrap()` +starts with `.` and must NOT be left-bounded, or `value.unwrap()` would stop +matching. Bounding only needles made entirely of identifier characters left +`todo!(`, `dbg!(`, `println!(`, `console.log(`, `unsafe {` and `as any` on raw +substring matching — the last of which reported every `has any` in a doc +comment as a type cast. The `eprint` family is listed explicitly because it used +to be caught by accident: `eprintln!(` CONTAINS `println!(`. + Full-file `#[cfg(test)]` / `#[test]` context seeding: reads the complete file at the target commit and builds a set of line numbers inside test blocks, using string/comment-aware brace counting. This correctly classifies additions inside pre-existing test modules @@ -880,6 +1335,26 @@ Structural code analysis: - `loctree.rs` — universal heuristic (works with any profile): cycles, dead exports, unused symbols, exact twins across Rust/JS/TS/Python +**A zero-file scan is a skip, not a clean run.** Loctree can report +`available: true` while `summary.total_files == 0`. Every consumer treats that +as SKIP: `MERGE_GATE.json` and `20_quality/heuristics_loctree.result.json` emit +status `skipped`/`SKIP`, and `report.json`'s `quality.heuristics` emits +`status: "skipped"` with a `skip_reason` and omits `dead_exports`, `cycles`, +`twins`, and `unused_symbols` rather than writing zeros that read as results. + +**A disabled scan is not a broken scanner.** `--quick` and `--no-heuristics` +short-circuit `heuristics::run_all` to a default result, which the caller still +passes on, so `report.json` used to describe an intentional skip as +`skip_reason: "loctree analysis unavailable"` — a tool failure that never +happened — and hand the reader a `log_path` pointing at a zero-filled stub. The +three skips are now distinguishable in `quality.heuristics`: + +| `skip_reason` | meaning | `total_files` | `log_path` | +|---|---|---|---| +| `heuristics not run` | not asked for (`--quick`, `--no-heuristics`) | absent | absent | +| `loctree analysis unavailable` | asked for, scanner failed | present | present | +| `loctree scanned no files` | ran, measured nothing | `0` | present | + ### cache/mod.rs Hash-based caching: diff --git a/docs/contracts/merge_gate.md b/docs/contracts/merge_gate.md index d54bf4a..e419446 100644 --- a/docs/contracts/merge_gate.md +++ b/docs/contracts/merge_gate.md @@ -1,4 +1,4 @@ -# MERGE_GATE Contract (schema 2.1) +# MERGE_GATE Contract (schema 2.2) `MERGE_GATE.json` is the policy-aware merge decision emitted at `00_summary/MERGE_GATE.json`. It is the single machine-readable verdict surface @@ -11,7 +11,7 @@ document disagree, the code is the contract and this document is the bug. | Field | Type | Notes | |---|---|---| -| `schema_version` | string | `"2.1"` | +| `schema_version` | string | `"2.2"` | | `generated_at` | string | RFC 3339 local datetime | | `bridge_stage` | integer | `0..4` | | `target` | string | Resolved target branch name (not raw CLI input) | @@ -55,7 +55,7 @@ Every element of `checks` is one policy evaluation record: |---|---|---| | `id` | string | Policy check id (`check_id`) | | `name` | string | Human-readable check name | -| `status` | string | Raw check status | +| `status` | string | `passed` \| `failed` \| `warnings` \| `skipped` \| `error` — lowercase, exactly | | `execution_state` | string | `executed` \| `skipped` \| `unavailable` \| `unknown` | | `outcome` | string | `passed` \| `findings_failed` \| `findings_warning` \| `system_error` \| `skipped` \| `unavailable` \| `unknown` | | `class` | string | `PASS` \| `SKIP` \| `FAIL` \| `INFO` | @@ -75,6 +75,29 @@ is `0.0`, `cached` is `null`, `log` is `null`, and `evidence` degrades to a non-empty placeholder. These are contract-valid placeholders, never `null` evidence — the artifact must not fail its own gate on a runner that lacks a tool. +`status` is a CLOSED, case-sensitive vocabulary: exactly the image of +`CheckStatus::as_str`, pinned as `CheckStatus::EMITTED` in `src/checks/mod.rs` +and mirrored as `VALID_CHECK_STATUSES` in `tools/validate_merge_gate.py`. The +CLI tallies warnings by comparing against it, so a status outside it is +UNREADABLE rather than clean: it counts toward the warning tally, raises an +`unreadable_check_status:` caveat naming the offending checks, and +`--ci --fail-on-warnings` fails on it. The validator rejects such a pack outright +— it used to accept any non-empty string, which certified an artifact +`--update` could reuse and the reader could not read. Case is deliberately NOT +folded here, unlike `inline_findings.status`, whose writer has shipped legacy +spellings: folding `WARNINGS` into a warning silently would hide that the pack +is off-contract, and the resulting tally is the same either way. + +The same rule governs the CONTAINER. `checks` must be an array — the validator +has required one since schema 1.0 — and a pack stating anything else is +unreadable, not empty: it counts as at least one warning and raises an +`unreadable_checks:` caveat. It used to fall back to "the checks this run +executed", which on an unchanged `--update` run is none at all, so +`--ci --fail-on-warnings` exited `0` on a pack whose warning list the reader +could not read. An ABSENT `checks` is the one tolerant case and stays so: a pack +that states no list may simply predate this build, and the CLI's own tally still +applies. Absent and present-but-unreadable are different questions. + ## `inline_findings` | Field | Type | Notes | @@ -112,7 +135,7 @@ authoritative axes — `analysis_status` (confidence) and `merge_recommendation` | `preexisting_quality_failures` | string[] | Pre-existing failures | | `mixed_quality_failures` | string[] | Mixed-provenance failures | | `unclassified_quality_failures` | string[] | Failures with unknown provenance | -| `quality_failure_details` | object[] | `[{ name, classification }]` | +| `quality_failure_details` | object[] | `[{ name, classification, origin }]` — `name` a non-empty check name, `classification` one of `introduced` \| `pre-existing` \| `mixed` \| `unclassified`, `origin` `"failure"` \| `"warning"` (schema 2.2) | | `decision_reason` | string | Human-readable reason for the verdict | | `review_caveats` | string[] | Non-blocking caveats requiring reviewer attention | | `blocking_issues` | string[] | Issues that block the merge | @@ -138,7 +161,25 @@ by `derive_decision` (`src/artifacts/verdict.rs`), which calls - **`derive_decision` is the single source** of `verdict`, `allow_merge`, and `recommended_merge`. No caller sets these fields independently. - **`policy_allow_merge` is a distinct axis** ("policy did not hard-block") and - is not conflated with `allow_merge` or the recommendation. + is not conflated with `allow_merge` or the recommendation. It is derived from + one input and set nowhere else: `policy_allow_merge == blocking_issues.is_empty()`, + computed after the last entry is pushed and emitted beside that list. The + contract validator enforces the equivalence in both directions from 2.2. +- **Only `origin: "failure"` entries may fail the quality gate.** Warning-level + checks enter `quality_failures` (and its classification arrays) so the + pre-existing downgrade can be computed for them, but they never flip + `quality_pass`. Reading `introduced_quality_failures` without `origin` is what + made `quality_pass: true` look like a contradiction; a consumer that wants + "what actually failed" filters `quality_failure_details` on + `origin == "failure"`. All three fields of the entry are validated, not just + the one that names the schema: `tools/validate_merge_gate.py` requires a + non-empty `name` and a `classification` from the emitted vocabulary, so + `{"origin": "failure"}` — a failure naming no check and stating no provenance + — is rejected rather than passed through as contract-clean. The + classification vocabulary is pinned to `QualityFailureClass::as_str` + (`src/artifacts/verdict.rs`); note that the value is `pre-existing` while the + sibling count field is `preexisting_quality_failures`, and an unvalidated + `classification` is exactly where that drift would hide. - **An executed check always carries its result artifact and log** (non-null `evidence` + `log`); a non-executed check carries non-null placeholders, never `null` evidence. @@ -147,6 +188,339 @@ by `derive_decision` (`src/artifacts/verdict.rs`), which calls them; the schema validator and the `prview mcp` adapter still tolerate them on read-back of older packs. +## Reader contract + +`MERGE_GATE.json` is the ONLY derivation of the verdict. No reader re-derives one +when the artifact cannot be read: `prview --json` / `--ci` exits `3` and the +`prview mcp` adapter returns `storage_corrupt`. The removed CLI fallback +(`fallback_merge_gate_summary`) re-derived `allow_merge = recommendation != block` +and was the single place where `allow_merge: true` could coexist with a +`CONDITIONAL` verdict, breaking the invariant above. + +Readers accept a pack by MAJOR version and say what they had to normalize: + +| `schema_version` on disk | Reader behavior | +|---|---| +| absent | Accepted silently — pre-2.1 packs predate the field, and their root object is read as the `decision` | +| known MAJOR (`1`, `2`), same-or-older MINOR | Accepted silently | +| known MAJOR, newer MINOR | Accepted with a `schema_forward_compat:` caveat; unknown fields ignored | +| unknown MAJOR, unparsable version, a non-canonical spelling (`02.2`, `+2.2`), or a non-string value | Fail loud | + +A pack that STATES a `schema_version` must also carry the `decision` object that +schema is built around. A missing or non-object `decision` there is a corrupt +artifact, not a normalization: the CLI exits `3` and the MCP adapter returns +`storage_corrupt`, matching `tools/validate_merge_gate.py`, which requires +`decision` at every version. Only a pack with NO `schema_version` keeps the +legacy tolerance of reading its root as the decision — and that tolerance is +whole: a legacy pack shaped `{"verdict": "ALLOW", "allow_merge": true}` is read +by BOTH readers, not accepted by one and called corrupt by the other. The rule +lives in one place (`gate::select_decision_object`) so the two surfaces cannot +answer it differently again. + +That tolerance is about WHERE the decision sits, not about what counts as one. A +schema-less pack whose root parses to an array, a scalar or `null` has no fields +to read at all: it is corrupt on both readers, not a decision with every signal +missing. Reading one as a decision produced a "successful" summary carrying a +normalized `BLOCK` for an artifact that never stated anything. + +A `decision` object that is present and states no decision falls under the same +rule. It must carry at least one of `verdict`, `merge_recommendation` or +`allow_merge`; a block carrying none of the three is corrupt on every surface — +the CLI exits `3`, the MCP adapter returns `storage_corrupt`, `prview gate` +cannot deserialize it, and `tools/validate_merge_gate.py` rejects it for the +required fields it is missing. The test is PRESENCE, not recognizability: a +stated `verdict: "PROBABLY"` is a decision this pack gave, and it collapses to +`BLOCK` with an `unknown_verdict:` caveat as described below. Absence stays +forgiven per FIELD — that is the shape of an older pack — but a decision block +with no signal at all is not an older pack, it is a truncated one. + +For the same reason the tolerance is a fallback, not a precedence rule: a +`decision` object, wherever it appears, is the decision. A schema-less pack that +carries one is read from it rather than from its root, because reading a plainly +stated decision as "every signal missing" would normalize to `BLOCK` and +fabricate a block the artifact never stated. No writer has ever produced that +shape — every generation back to the first public release emits `schema_version` +and `decision` together — so a schema-less pack that ALSO carries root-level +decision fields is undefined by this contract rather than resolved by it. + +One vocabulary answers "what verdict is this?" for every surface. The CLI +`--json` summary, the MCP adapter and `prview gate` all fold a stored spelling +through `gate::canonical_verdict`, which is case-insensitive and accepts the +retired synonyms (`ALLOW`/`APPROVE` → `PASS`, `HOLD` → `CONDITIONAL`). Case is +not meaning: a pack stating `verdict: "pass"` stated a pass, and reading it as a +block would fabricate a verdict the artifact never gave. Each surface owning its +own copy of this vocabulary is precisely how they came to read one file three +ways — MCP ranking `"pass"` as a clean `PASS`, the CLI calling it an unknown +verdict and normalizing to `BLOCK`, and `prview gate` refusing the pack as a +verdict mismatch. `GateVerdict` stays a strict parser of the canonical spellings; +it is fed the folded value, never the raw one. + +A verdict outside `PASS` / `CONDITIONAL` / `BLOCK` (and the legacy synonyms) is +never read as-is and never silently dropped: BOTH readers collapse it to `BLOCK` +with an `unknown_verdict:` caveat, and the MCP adapter additionally sets +`normalized: true`. A verdict a reader substituted this way also governs +everything derived beside it: `allow_merge` is forced `false` and +`merge_recommendation` to `block`, whatever the same decision block claimed. A +pack whose verdict could not be read is not a pack whose approval can be +trusted, and the exit code follows the recommendation — so the invariant +`allow_merge == (verdict == "PASS")` holds on the substituted verdict too. The +same holds for a verdict that is simply absent while another signal is stated: +the surviving `merge_recommendation: "approve"` does not buy a `PASS` on either +surface, because a pack that names no verdict has not approved anything. + +That is what keeps "the artifact is corrupt" and "the artifact said something +this reader cannot use" apart on every surface. `storage_corrupt` is reserved +for a decision block stating none of the three signals. A signal that is present +but unrankable — a verdict outside the vocabulary, a recommendation outside it, +a lone `allow_merge` — is a decision the pack DID give, so it is normalized +conservatively with a caveat instead of being called corrupt by one reader while +the other publishes a summary from it. `allow_merge` is itself a rankable +signal: `false` ranks as `CONDITIONAL` and `true` as `PASS`, and a pack stating +nothing but `allow_merge: true` therefore still reads as `BLOCK` on both +surfaces, because its missing verdict is substituted before its lone flag is +ranked. + +The accepted version set is exactly the one `tools/validate_merge_gate.py` +accepts, compared as written — a spelling that merely parses to a known tuple +(`02.2`, `2.02`, `+2.2`) is rejected, so "readable by prview" cannot drift away +from "valid per the contract validator". From schema 2.2 the validator also +requires every `quality_failure_details` entry to carry an `origin` of exactly +`failure` or `warning`: consumers are told to filter on it, which they cannot do +if a pack may omit or mistype it. + +From 2.2 it also REQUIRES `quality_pass` and requires it to be a boolean. The +2.2 writer emits the field unconditionally, from a single object literal, as a +Rust `bool` — so a pack that claims 2.2 and omits it, or states it as `"false"`, +is not an old pack but a broken one. Absence stays forgiven below 2.2, where the +readers derive the flag instead, and that carve-out is deliberate: tightening it +would reject every pack written before the field existed. Type-checking it here +is what puts the validator back in step with the readers, which normalize a +present-but-unreadable signal to BLOCK — without it the contract gate certified +an artifact the CLI and MCP both refuse to trust. + +From the same version it also cross-checks `quality_pass` against those details, +because the two are one fact written twice: the emitter sets the flag to +`!QualityFailureSummary::has_new_failures()` and then serializes the very +details that answer it. An entry gates the diff when its `origin` is `failure` +AND its `classification` is anything but `pre-existing`, so + +> `quality_pass` is true if and only if no `quality_failure_details` entry +> has `origin: "failure"` with a classification other than `pre-existing`. + +Both directions are checked. `quality_pass: true` beside +`{"origin": "failure", "classification": "introduced"}` is the combination that +matters: the emitter cannot produce it, both decision readers trust the +permissive scalar, and a validator-clean pack could therefore approve a failure +it also reports. `quality_pass: false` with nothing that could have failed it is +equally unemittable, and rejected too. The `pre-existing` half of the rule is +not a detail — a failure that predates the diff is published beside +`quality_pass: true` deliberately, so the simpler rule "a failure-origin entry +forces `quality_pass: false`" would reject a legitimate pack, and a validator +that cries wolf on genuine output gates nothing. A pack that omits +`quality_pass` entirely is left alone, per the absence rule below. + +The blocker axis is cross-checked the same way, and for the same reason. The +emitter computes `policy_allow_merge = blocking_issues.is_empty()` after the +last entry is pushed to that list and then writes both verbatim, so + +> `policy_allow_merge` is true if and only if `blocking_issues` is empty. + +Both directions are checked from 2.2, where both fields are required. +`policy_allow_merge: true` beside a listed blocker is the shape that matters: it +tells a reader trusting the flag that policy let the merge through while the list +beside it names what blocked it. `false` with nothing in the list is equally +unemittable and rejected too. This is not the same rule as the ranking below, +which asks only how conservative the pack is and is satisfied by either half — +ranking a pair does not check that the pair agrees. It is also not the +pre-existing "no `allow_merge: true` beside a blocker" rule: that one is about +the merge verdict, this one about the policy flag it is derived from. A test in +`src/artifacts/merge_gate.rs` pins the flag to the list across the emitted packs, +so the day the flag gains a second input the emitter fails rather than the +validator rejecting output prview still writes. + +### The reconciliation is certified, not only read + +From 2.2 the validator requires the remaining decision axes on the same +argument, and with the same vocabularies: `analysis_status` (`complete` / +`degraded` / `incomplete`), `merge_recommendation` (`approve` / +`review_required` / `block`) and a boolean `policy_allow_merge`. All three come +out of the same object literal as `quality_pass`, from the typed enums in +`src/policy/engine.rs`, so a 2.2 pack missing one is broken rather than old. The +two enum vocabularies are case-sensitive and canonical-only — like +`checks[].status`, and unlike the READERS, which fold case and still accept the +retired `hold` spelling when reading an artifact off disk. That tolerance exists +for packs already written; the validator certifies freshly emitted ones. A test +in `src/policy/engine.rs` pins each variant's wire spelling to the word the +validator lists, so a rename cannot silently drift the two apart. + +Requiring them is what makes the last certification rule possible: **the +validator rejects a `verdict` milder than the axes stated beside it.** The rank +table above is the readers' rule, and the emitter's `legacy_verdict` produces +exactly the same number from the other direction, so a healthy `verdict` IS the +maximum rank of its own axes. Until this was ported, the contract gate certified +packs no reader would honour — `verdict: "PASS"` beside +`analysis_status: "incomplete"`, `merge_recommendation: "block"` and +`policy_allow_merge: false` validated OK, while every reader normalized the same +artifact to `BLOCK`. Readers were already protected; the hole was in +CERTIFICATION, which is a different claim: that the artifact is what it says it +is. + +The rule is deliberately ONE-DIRECTIONAL. A verdict HARSHER than its other axes +is legal and must stay so: a semgrep scan that passes with parse errors writes +`merge_recommendation: "approve"` beside `analysis_status: "degraded"`, which the +contract turns into `CONDITIONAL`, so "the verdict equals the maximum of the +OTHER axes" would reject a pack the emitter really produces. A harsher verdict +also misleads no one — every reader publishes it as stated. It is the permissive +direction that certifies a permission the artifact never earned. (The readers +still NAME the harsher case with a `core_inconsistency:` caveat; that is a report +about a pack, not a rejection of it.) + +A decision signal present with the wrong JSON type (`merge_recommendation: 7`, +`allow_merge: "false"`) is not the same as an absent one. Absence is the state a +reader forgives, because it is the shape of an older pack; a field that is there +and cannot be typed is a field the reader FAILED to read, and saying nothing +about it publishes a confidence the read does not have. Both readers name it +with an `unreadable_:` caveat — every axis in the ranking table below. +The MCP adapter additionally sets `normalized: true`; the CLI +forces every decision axis conservative (`verdict: "BLOCK"`, +`allow_merge: false`, `merge_recommendation: block`, and therefore `--ci` +exit `1`), because a decision derived from a block this reader only partly read +is not one it may publish as an approval. + +Correctly typed signals that CONTRADICT each other are reconciled the same way, +by conservativeness rather than by field order. Each stated axis ranks +`PASS`/`approve`/`allow_merge: true` as 1, `CONDITIONAL`/`review_required`/ +`allow_merge: false` as 2 and `BLOCK`/`block` as 3; the highest rank the pack +states wins and every axis is published from it, with a `core_inconsistency:` +caveat naming the originals. So `verdict: "BLOCK"` beside +`merge_recommendation: "approve"` yields `block` on both surfaces — the CLI used +to believe each field in turn and exit `0` on it — and `allow_merge: true` +beside `review_required` never buys a `PASS`. Both readers rank through +`gate::rank_from_verdict` / `gate::rank_from_merge_rec`. A recommendation +outside the `approve` / `review_required` / `block` vocabulary cannot rank, so +it is excluded from the reconciliation and named with an +`unknown_merge_recommendation:` caveat rather than dropped in silence. + +`quality_pass` is one of those axes, because the contract permits `PASS` only +when quality passes. A stated `quality_pass: false` therefore ranks 2 — it says +"not a `PASS`", exactly as `allow_merge: false` does — so a pack shaped +`verdict: "PASS"`, `merge_recommendation: "approve"`, `allow_merge: true`, +`quality_pass: false` is published as `review_required` with +`allow_merge: false` on both surfaces, with the `core_inconsistency:` caveat +naming every original including this one. Leaving that axis out of the +reconciliation published the approval verbatim, so automation reading the MCP +surface approved a run whose own artifact said quality had failed. The +asymmetry is deliberate in both directions: `quality_pass: true` states no rank +at all, because a quality-clean run is still held at `CONDITIONAL` by a +breaking-change escalation and one axis may not soften a verdict the others +agree on; and an ABSENT `quality_pass` states nothing either, per the same +per-field tolerance that governs the other signals — reading it as `false` +would turn every pack written before the field into a `CONDITIONAL`. Those two +states leave a third between them, and `quality_pass` is typed through the same +`gate::readable_signal` as the other axes so it does not fall into it: a +`quality_pass` that is PRESENT but not a boolean is neither a stated `false` nor +an older pack. Read with a bare `as_bool()` it was indistinguishable from +absent, so the string `"false"` bought a silent approval on both surfaces — +the one shape that defeats the paragraph above. It now normalizes to `BLOCK` +with an `unreadable_quality_pass:` caveat, like any other signal the reader +could not type. + +### Which fields rank, and which deliberately do not + +One rule decides membership: **an axis states a rank only when its value RULES +OUT a more permissive outcome.** A value that merely fails to forbid something +states nothing — that is why `quality_pass: true` is silent, and it is the same +reason `analysis_status: "complete"` is. Both are PRECONDITIONS of `PASS`, not +grants of it: a quality-clean, fully-analysed run is still a `BLOCK` when policy +blocks it, and letting either speak in the permissive direction would let one +axis soften a verdict the others agree on. + +The decision object is closed, so every field it may carry is accounted for +here. This table is the contract; a field added to `decision` without a row is +an unfinished change. + +| Field | Rank | Rule | +|---|---|---| +| `verdict` | 1 / 2 / 3 | `PASS` \| `CONDITIONAL` \| `BLOCK`, via `gate::rank_from_verdict` | +| `merge_recommendation` | 1 / 2 / 3 | `approve` \| `review_required` \| `block`, via `gate::rank_from_merge_rec` | +| `allow_merge` | 1 / 2 | `false` rules out `PASS`; `true` ranks 1 and so never raises | +| `quality_pass` | 2 | Only `false` ranks — `PASS` requires quality to pass | +| `analysis_status` | 2 | Only `degraded` / `incomplete` rank — `PASS` requires `complete` | +| `blocking_issues` | 3 | Non-empty ranks — see below | +| `policy_allow_merge` | 3 | Only `false` ranks — the same fact as a non-empty `blocking_issues` | +| `recommended_merge` | — | Legacy restatement of `merge_recommendation == approve`; ranking it counts one axis twice | +| `recommended_label` | — | Human label with an open vocabulary (`e.g.` in its own row); nothing to rank against | +| `quality_failures` and its four classification arrays | — | Non-empty ≠ failed: warning-origin entries populate them without flipping `quality_pass`, which is precisely the false positive `origin` was added to prevent | +| `quality_failure_details` | — | The evidence BEHIND `quality_pass`, not an independent axis; ranking it would recompute that axis from parts and re-introduce the same warning/failure conflation | +| `decision_reason` | — | Prose | +| `review_caveats` | — | Non-blocking by definition | + +`blocking_issues` ranks 3 rather than 2 because a blocker is not a doubt: an +entry appears there only when a check reached `PolicyConclusion::Blocked`, whose +`merge_impact` is `Block`, so a pack listing one has already stated a `BLOCK` +whether or not its `verdict` field agrees. `policy_allow_merge` is the same fact +written twice — the emitter computes `policy_allow_merge = +blocking_issues.is_empty()` — so both are read and both rank, which costs +nothing when they agree and covers a pack that states only one. This does not +conflate `policy_allow_merge` with `allow_merge`: the two remain distinct axes, +and only the value that rules `PASS` out speaks. An empty `blocking_issues` and +`policy_allow_merge: true` state nothing at all, because "policy did not +hard-block" is not "merge is allowed". + +#### Ranking an absent field is not publishing one + +The table above says what an absent field contributes to the RANK: nothing. It +does not say what the CLI summary should then report for that field, and +conflating the two produced a reader split. A pre-`quality_pass` pack — +`{"verdict": "PASS", "merge_recommendation": "approve", "allow_merge": true}` — +is correctly reconciled to `PASS`, because absence adds no rank; but the summary +published `quality_pass: false` from a bare default, derived +`analysis_status: incomplete` from that, and exited `1` under `--ci`, while the +MCP adapter returned a clean approval for the same artifact. + +An absent field is therefore PUBLISHED from the reconciled outcome rather than +from a default. The contract permits `PASS` only when quality passes and the +analysis is complete, so a reconciled `PASS` implies both; a decision held below +`PASS` implies nothing about either axis specifically and both stay +conservative. The direction is one-way — the reconciled verdict can only ever +confirm what the contract already requires of a `PASS`, never soften a verdict. + +This does not reopen the absent/mistyped split. A field that is present but +unreadable normalizes the whole decision to `BLOCK`, so by the time the summary +is built the reconciled outcome is not a `PASS` and nothing can be inferred as +passing from it. Absence is forgiven; an unreadable value is not. + +Every ranking axis is typed through `gate::readable_signal`, so present-but- +unreadable is a third state distinct from both a stated value and an absent one: +`blocking_issues: "Clippy"` (a string, not an array) or `analysis_status: 7` +normalizes to `BLOCK` with an `unreadable_:` caveat instead of being +mistaken for a pack written before the field. A stated `analysis_status` outside +`complete` / `degraded` / `incomplete` cannot rank, so it is excluded and named +with an `unknown_analysis_status:` caveat — the rule already applied to +`merge_recommendation`. + +The `core_inconsistency:` caveat reports a disagreement the pack actually +states, so the comparison is made per axis rather than against the winning rank: +the two textual axes are compared to the published verdict, and `allow_merge` to +the `allow_merge` the readers publish. `allow_merge` has only two values and +`false` ranks as `CONDITIONAL`, so measuring it against a rank it can never +reach made every healthy `BLOCK` pack — `verdict: "BLOCK"`, +`merge_recommendation: "block"`, `allow_merge: false` — report a contradiction +it did not contain. `quality_pass` needs no comparison of its own: ranking 2 +when false, it makes any axis claiming 1 beside it disagree with the winning +rank already, and a healthy `BLOCK` or `CONDITIONAL` pack states it in agreement +with everything else. It is named in the caveat all the same, so a reader can +see which axis forced the downgrade. The same holds for `analysis_status`, +`blocking_issues` and `policy_allow_merge`: each ranks in one direction only, so +a healthy pack states them in agreement with the winning rank — a `BLOCK` pack +naming its blocker beside `policy_allow_merge: false` and a `complete` analysis +is exactly what this tool writes and reports no contradiction — while a pack +that states one of them AGAINST a permissive verdict is caught by the textual +axes disagreeing with the rank it forced. All three are named in the caveat. +A pack whose verdict was substituted +reports the substitution (`unknown_verdict:`, `unreadable_:`) and is not +additionally accused of contradicting itself. + ## Blocking rules Whether a check's `FAIL` blocks the merge depends on its policy severity: diff --git a/docs/gate-playbook.md b/docs/gate-playbook.md index ffc8f49..afad3e4 100644 --- a/docs/gate-playbook.md +++ b/docs/gate-playbook.md @@ -17,6 +17,12 @@ they must not parse stdout. Use `prview gate --json` when CI needs a machine-readable summary, artifact paths, or SARIF path discovery. Pass/fail still comes from the process exit code. +Exit `3` covers every way the run can end without a trustworthy verdict — the +review failing to execute, and the pack's `00_summary/MERGE_GATE.json` being +missing, unparsable, or stamped with a `schema_version` this build cannot read. +Plain `prview --ci` uses the same code for the same conditions: it never +re-derives a verdict when the gate artifact cannot be read. + ## Breaking-change escalation A genuine breaking API change in the diff — a removed public symbol, a changed @@ -43,7 +49,11 @@ which command you run. Two contract lines, deliberately distinct: hard failure (`BLOCK` or a broken quality gate); a `CONDITIONAL` verdict — including a breaking-only `CONDITIONAL` — exits `0`, exactly as it does for any other `CONDITIONAL` cause. This is the historical review contract and does not - change with breaking-change escalation. + change with breaking-change escalation. Warning-level checks are advisory and + do not break the quality gate, so a warnings-only run exits `0`; add + `--fail-on-warnings` to opt into exit `1` for them. Both `--ci` exits hold + whatever preset the run resolves to — `--ci --update` is still strict, and an + `--update` run with no new commits takes its exit from the pack it reused. * **`prview gate`** — the contractual enforcement path. `CONDITIONAL` exits `1`, and `prview gate --strict` exits `2` (see the exit-code contract above). diff --git a/docs/mcp.md b/docs/mcp.md index a26fb2f..2646620 100644 --- a/docs/mcp.md +++ b/docs/mcp.md @@ -200,10 +200,60 @@ The decision surface is normalized so callers read one vocabulary: clean `PASS`. A permissive flag on disk can never override a block/hold signal. If the stored gate emits contradictory signals (for example `allow_merge: true` -alongside a block recommendation), the most conservative signal wins and a -`core_inconsistency` note is appended to `caveats`. Legacy gate tokens (`ALLOW`, -`HOLD`) written by older cores are still recognized on read and folded into the -`PASS` / `CONDITIONAL` surface rather than failing loud. +alongside a block recommendation, or a clean approval alongside +`quality_pass: false` — the contract permits `PASS` only when quality passes), +the most conservative signal wins and a `core_inconsistency` note is appended to +`caveats`. The note reports a +disagreement the pack actually states — the textual axes against the published +verdict, `allow_merge` against the flag published — so a self-consistent +`BLOCK` pack (`verdict: "BLOCK"`, `merge_recommendation: "block"`, +`allow_merge: false`) raises no caveat at all. The CLI `--json` surface +reconciles the same way through the same ranking +(`gate::rank_from_verdict` / `gate::rank_from_merge_rec`), so the two surfaces +cannot disagree about a contradictory pack. Legacy gate tokens (`ALLOW`, +`APPROVE`, `HOLD`) written by older cores are still recognized on read and folded +into the `PASS` / `CONDITIONAL` surface rather than failing loud. That fold is +`gate::canonical_verdict`, shared by this adapter, the CLI summary and +`prview gate`, and it ignores case: a stored `"pass"` reads as `PASS` on every +surface instead of approving on one and normalizing to `BLOCK` on another. + +Anything the adapter could not read is named rather than dropped, and every such +case sets `normalized: true`: + +- `unknown_verdict:` / `unknown_merge_recommendation:` — the field was present + but outside the known vocabulary, so it was ignored when deriving the decision. + A verdict that could not be ranked — outside the vocabulary, or simply absent + while another signal is stated — is substituted with `BLOCK`, and that + substitution governs the axes published beside it: `merge_recommendation` + reads `block` and `allow_merge` `false`, whatever the pack claimed. This is + the CLI's rule, applied here so the two readers cannot answer the same bytes + differently. `storage_corrupt` is reserved for a decision block stating NONE + of `verdict`, `merge_recommendation` and `allow_merge`; a signal that is + present but unrankable — including a lone `allow_merge` — is a decision the + pack gave, and it is normalized with a caveat rather than called corrupt. +- `unreadable_:` — the field was present with the wrong JSON type + (`merge_recommendation: 7`, `allow_merge: "false"`, `quality_pass: "false"`, + `analysis_status: 7`, `blocking_issues: "Clippy"`). Emitted for every axis in + the ranking table of `docs/contracts/merge_gate.md`. A wrongly typed field is + not an absent one: it is ignored for ranking, but it is named, and the + decision is normalized conservatively around it. The pack is + `storage_corrupt` only when no signal was stated at all. +- `unknown_analysis_status:` — the field is a string outside + `complete` / `degraded` / `incomplete`. Like `unknown_merge_recommendation:`, + it cannot rank, so it is excluded from the reconciliation and named rather + than dropped in silence. +- `schema_forward_compat:` — the pack's `schema_version` is a newer MINOR of a + known MAJOR; it is read, and fields this build does not know are ignored. An + unknown MAJOR is `storage_corrupt`, and so is a `schema_version` that is + present but not a `MAJOR.MINOR` string. A pack with no `schema_version` at all + is pre-2.1 and is accepted silently, like the `ALLOW`/`HOLD` tokens — including + the pre-2.1 shape that carries its signals at the root instead of under + `decision`. A pack that STATES a `schema_version` and still has no `decision` + object is `storage_corrupt`, and so is a schema-less pack whose root is not an + object at all (an array, a scalar, `null`) — that root states no decision, it + is not a decision missing every field. Both readers apply those rules from one + place (`gate::select_decision_object`), so a pack the CLI reads is never one + the MCP adapter calls corrupt. Completed response: @@ -329,7 +379,7 @@ fields (e.g. `retry_after_ms`, `active_run_id`, `run_id`). | `artifact_missing` | The requested artifact does not exist within the run, is not UTF-8 text, or would escape the run directory. | | `tool_missing` | A required external tool is unavailable. | | `storage_locked` | Another review is already running for this repo branch. Carries `active_run_id` and `retry_after_ms`. | -| `storage_corrupt` | `MERGE_GATE.json` is missing, invalid, has no recognizable decision, or an explicit `run_id` is ambiguous in storage. | +| `storage_corrupt` | `MERGE_GATE.json` is missing, invalid, carries a `schema_version` with an unknown MAJOR, states no decision signal at all, or an explicit `run_id` is ambiguous in storage. | | `stale_run` | The run is still in progress or its process died before completing. Carries `retry_after_ms` while running. | ### Retrying diff --git a/docs/usage.md b/docs/usage.md index 0275ff7..a558816 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -290,6 +290,39 @@ introduced. | `--no-color` | Disable ANSI colors | | `--no-zip` | Skip ZIP creation | | `--no-dashboard` | Skip HTML dashboard generation | +| `--soft-exit` | Always exit 0, whatever the checks found | +| `--fail-on-warnings` | With `--ci`: also exit 1 when any check reports warnings | + +### `--ci` exit codes + +`--ci` is the strict variant of the plain review run: it exits `1` on a `BLOCK` +verdict or a broken quality gate, and `0` otherwise. Warning-level signals — a +formatter delta, an unmaintained-crate advisory, lint warnings — are advisory: +they keep the verdict at `CONDITIONAL` and surface as review caveats, but they +do not fail the process. Add `--fail-on-warnings` to opt into exit `1` for them; +the flag requires `--ci` and does not affect `prview gate`, whose exit codes come +from the gate contract (see `docs/gate-playbook.md`). + +`--fail-on-warnings` counts the artifact pack's check list, not the CLI's own. +The artifact run generates further checks — `public_api_diff`, `unsafe_audit`, +`ghost_refs` and the synthetic `heuristics_loctree` — which reach +`MERGE_GATE.json` and the dashboard but never the in-memory report the plain +tally is built from. The `--json` summary states both numbers: +`checks_summary.warned` is what the CLI ran, `checks_summary.warned_in_pack` is +the complete count the flag keys off, and it is always the larger of the two. + +Strictness follows the `--ci` you typed, not the preset label the run reports. +`--update` outranks `--ci` when the execution preset is resolved, so +`prview --ci --fail-on-warnings --update` publishes `mode.execution_mode: +"update"` — and reading strictness off that label made both `--ci` exits +(`!quality_pass` and the warning hardening) silently inert for exactly the +combination CI jobs use. + +An `--update` run that finds no new commits reuses the previous pack, and its +exit code is derived from that pack like any other run's: a reused `BLOCK` or a +reused warning under `--fail-on-warnings` exits non-zero rather than reporting a +green second invocation over an artifact nothing re-checked. `--soft-exit` +remains the one way to ask for `0` regardless. ## Examples @@ -338,6 +371,15 @@ it carries the verdict, `output_dir`, a short `checks_summary`, `top_failures`, on disk, especially the canonical `RUN.json` and `MERGE_GATE.json` pair, plus `PR_REVIEW.md`. +The verdict fields (`verdict`, `allow_merge`, `quality_pass`, +`merge_recommendation`, `analysis_status`) are read from the run's +`00_summary/MERGE_GATE.json` and from nowhere else. If that artifact is missing, +unparsable, or stamped with a `schema_version` this build cannot read, prview +reports an execution error and exits `3` instead of re-deriving a verdict — +including on `--update` runs that re-read an earlier pack. A newer MINOR schema +within a known MAJOR is accepted and reported through the optional `caveats` +array, which also carries any verdict the reader had to normalize. + ## Output Artifacts are written to `$PRVIEW_HOME/runs////` diff --git a/src/artifacts/ai_index.rs b/src/artifacts/ai_index.rs index f1a1402..4246f40 100644 --- a/src/artifacts/ai_index.rs +++ b/src/artifacts/ai_index.rs @@ -64,8 +64,10 @@ pub(crate) fn generate_ai_index( )?; writeln!( md, - "- Coverage signal: {}/{} changed code files ({}%)", - coverage.covered_count, coverage.total_source, coverage.pct + "- Coverage signal: {}/{} changed code files ({})", + coverage.covered_count, + coverage.total_source, + crate::artifacts::signal::format_coverage_pct(coverage.pct) )?; let gate_path = Path::new("00_summary/MERGE_GATE.json"); if dir.join(gate_path).exists() diff --git a/src/artifacts/dashboard/assets.rs b/src/artifacts/dashboard/assets.rs index 99b85d8..b53b85e 100644 --- a/src/artifacts/dashboard/assets.rs +++ b/src/artifacts/dashboard/assets.rs @@ -2759,8 +2759,13 @@ const JS_SUFFIX: &str = r##" } if (quality.coverage) { - var pct = Math.round(quality.coverage.heuristic_ratio * 100); - comment += '**Coverage heuristic:** ' + pct + '% (' + quality.coverage.matched + '/' + quality.coverage.total + ')\n\n'; + // heuristic_ratio is null when nothing was measured (0 changed + // source files) - do not round null into a 0%/100% claim. + var ratio = quality.coverage.heuristic_ratio; + var covLabel = (ratio === null || ratio === undefined) + ? 'not measured' + : Math.round(ratio * 100) + '%'; + comment += '**Coverage heuristic:** ' + covLabel + ' (' + quality.coverage.matched + '/' + quality.coverage.total + ')\n\n'; } var hotspots = (diff.files || []) diff --git a/src/artifacts/dashboard/mod.rs b/src/artifacts/dashboard/mod.rs index 300e4f8..5b26d0e 100644 --- a/src/artifacts/dashboard/mod.rs +++ b/src/artifacts/dashboard/mod.rs @@ -557,14 +557,15 @@ fn build_html(input: BuildHtmlInput<'_>) -> String { ) }; - let coverage_summary = if ctx.coverage.total_source > 0 { - i18n_template( + let coverage_summary = match ctx.coverage.pct { + Some(pct) => i18n_template( "summary.coveragePct", - &format!("{}% coverage", ctx.coverage.pct), - &[("pct", ctx.coverage.pct.to_string())], - ) - } else { - escape_html("N/A") + &format!("{}% coverage", pct), + &[("pct", pct.to_string())], + ), + // Nothing was measured (no changed source files) — say so, do not + // borrow a number the heuristic never produced. + None => escape_html("N/A"), }; let commit_count = diff.map(|d| d.commits.len()).unwrap_or(0); @@ -927,12 +928,12 @@ fn build_html(input: BuildHtmlInput<'_>) -> String { ); } } - if ctx.coverage.total_source > 0 { - if ctx.coverage.pct < 80 { + if let Some(pct) = ctx.coverage.pct { + if pct < 80 { let _ = write!( nav, "Coverage {}", - ctx.coverage.pct + pct ); } else { let _ = write!( diff --git a/src/artifacts/dashboard/sections.rs b/src/artifacts/dashboard/sections.rs index d9f6694..633ef89 100644 --- a/src/artifacts/dashboard/sections.rs +++ b/src/artifacts/dashboard/sections.rs @@ -1027,18 +1027,21 @@ pub(super) fn build_action_center( } let cov = &ctx.coverage; - if cov.total_source > 0 || cov.non_code_count > 0 { - if cov.pct < 80 { + // `pct == None` means no changed source files were evaluated. A diff of + // pure non-code changes used to fall through here and emit a "Coverage: + // 100%" chip out of a 0/0 ratio — emit nothing instead. + if let Some(cov_pct) = cov.pct { + if cov_pct < 80 { push_card( &mut cards, "#section-coverage", - if cov.pct < 50 { + if cov_pct < 50 { "alert-error" } else { "alert-warning" }, r#"Coverage"#.to_string(), - escape_html(&format!("{}%", cov.pct)), + escape_html(&format!("{}%", cov_pct)), i18n_template( "message.changedCodeWithoutMatchingTests", "Changed code without matching tests", @@ -1055,7 +1058,7 @@ pub(super) fn build_action_center( i18n_template( "chip.coverageOk", "Coverage: {pct}%", - &[("pct", cov.pct.to_string())], + &[("pct", cov_pct.to_string())], ) ), ); @@ -1974,13 +1977,14 @@ pub(super) fn build_breaking_section(ctx: &DashboardContext) -> String { pub(super) fn build_coverage_section(ctx: &DashboardContext) -> String { let cov = &ctx.coverage; - if cov.total_source == 0 { + // No changed source files => nothing measured => no coverage section at all. + let Some(cov_pct) = cov.pct else { return String::new(); - } + }; // Coverage % is a metric, not a verdict: keep color only for the negative // signal (below threshold = "what is wrong?"), neutralize the rest. - let pct_color = if cov.pct < 50 { + let pct_color = if cov_pct < 50 { "var(--warn)" } else { "var(--fg)" @@ -2049,7 +2053,7 @@ pub(super) fn build_coverage_section(ctx: &DashboardContext) -> String { "#, color = pct_color, - pct = cov.pct, + pct = cov_pct, coverage_detail = i18n_template( "message.coverageDetail", &format!( diff --git a/src/artifacts/dashboard/tests.rs b/src/artifacts/dashboard/tests.rs index 5ff78b8..1f10f60 100644 --- a/src/artifacts/dashboard/tests.rs +++ b/src/artifacts/dashboard/tests.rs @@ -123,7 +123,7 @@ fn mock_ctx() -> DashboardContext { check_gates: vec![], breaking: vec![], coverage: super::super::CoverageDelta { - pct: 100, + pct: Some(100), total_source: 1, covered_count: 1, uncovered: vec![], @@ -372,7 +372,7 @@ fn test_header_merge_chip_shows_review_caveat() { risk_level: BreakingRisk::High, }]; ctx.coverage = super::super::CoverageDelta { - pct: 11, + pct: Some(11), total_source: 43, covered_count: 5, uncovered: vec![], @@ -509,7 +509,7 @@ fn test_merge_decision_card_review_caveats() { risk_level: BreakingRisk::High, }]; ctx.coverage = super::super::CoverageDelta { - pct: 11, + pct: Some(11), total_source: 43, covered_count: 5, uncovered: vec![], diff --git a/src/artifacts/dashboard/trends_tests.rs b/src/artifacts/dashboard/trends_tests.rs index 35878a0..9bf090d 100644 --- a/src/artifacts/dashboard/trends_tests.rs +++ b/src/artifacts/dashboard/trends_tests.rs @@ -95,7 +95,7 @@ fn mock_ctx() -> DashboardContext { check_gates: vec![], breaking: vec![], coverage: super::super::CoverageDelta { - pct: 100, + pct: Some(100), total_source: 1, covered_count: 1, uncovered: vec![], diff --git a/src/artifacts/merge_gate.rs b/src/artifacts/merge_gate.rs index 7d676db..45ffcd9 100644 --- a/src/artifacts/merge_gate.rs +++ b/src/artifacts/merge_gate.rs @@ -245,7 +245,7 @@ pub(super) fn generate_merge_gate(input: MergeGateInput<'_>) -> Result<()> { .count(); let gate = json!({ - "schema_version": "2.1", + "schema_version": crate::gate::MERGE_GATE_SCHEMA_VERSION, "generated_at": chrono::Local::now().to_rfc3339(), "bridge_stage": config.bridge_stage, "target": resolved_target.name, @@ -285,6 +285,7 @@ pub(super) fn generate_merge_gate(input: MergeGateInput<'_>) -> Result<()> { "quality_failure_details": quality_failures.details.iter().map(|detail| json!({ "name": detail.name, "classification": detail.classification.as_str(), + "origin": detail.origin.as_str(), })).collect::>(), "decision_reason": decision.reason, "review_caveats": all_review_caveats, @@ -500,7 +501,7 @@ mod tests { CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: Vec::new(), covered: Vec::new(), non_code_count: 0, @@ -959,6 +960,46 @@ mod tests { ); } + #[test] + fn the_blocker_flag_is_the_blocker_list_written_twice() { + // `tools/validate_merge_gate.py` certifies + // `policy_allow_merge == blocking_issues.is_empty()` as an equivalence, + // rejecting a pack that states one without the other. That is only sound + // while the flag is derived from the list and from nothing else, as it is + // above. Should the flag ever gain a second input, this pin fails here + // — in the emitter that changed — instead of the validator silently + // rejecting packs prview itself still writes. + let packs = [ + run_gate_with_skipped_policy_check( + "cargo_audit", + "Cargo audit", + "security disabled", + crate::policy::PolicySeverity::Block, + ), + run_gate_with_skipped_policy_check( + "cargo_audit", + "Cargo audit", + "tool not installed (cargo-audit is missing)", + crate::policy::PolicySeverity::Block, + ), + run_gate_with_cargo_test_finding(false), + run_gate_with_semgrep_finding(false, false), + ]; + + for gate in packs { + let decision = &gate["decision"]; + let no_blockers = decision["blocking_issues"] + .as_array() + .expect("blocking_issues array") + .is_empty(); + assert_eq!( + decision["policy_allow_merge"].as_bool(), + Some(no_blockers), + "policy_allow_merge must mirror an empty blocking_issues: {decision}" + ); + } + } + #[test] fn preexisting_semgrep_finding_outside_diff_does_not_degrade_verdict() { let gate = run_gate_with_semgrep_finding(false, false); @@ -1132,6 +1173,45 @@ mod tests { ); } + #[test] + fn preexisting_only_rustfmt_keeps_strict_gate_exit_zero() { + // Regression guard for the warning→failure cut: the gate exit contract is + // unchanged. The pre-existing-only rustfmt pack is a PASS, and `prview + // gate --strict` must still exit 0 on it — the same artifact the adapter + // in `gate.rs` reads, run through the same verdict → exit mapping. + use crate::gate::{GateVerdict, gate_exit_code}; + + let gate = run_gate_with_rustfmt_warning(false); + let verdict = GateVerdict::try_from( + gate["decision"]["verdict"] + .as_str() + .expect("verdict is a string"), + ) + .expect("verdict is contract vocabulary"); + + assert_eq!(verdict, GateVerdict::Pass); + assert_eq!(gate_exit_code(verdict, true), 0); + assert_eq!(gate_exit_code(verdict, false), 0); + } + + #[test] + fn introduced_warning_keeps_strict_gate_exit_two() { + // The other half of the contract: an in-diff warning stays CONDITIONAL, + // so `--strict` still exits 2. Warnings became honest, not toothless. + use crate::gate::{GateVerdict, gate_exit_code}; + + let gate = run_gate_with_rustfmt_warning(true); + let verdict = GateVerdict::try_from( + gate["decision"]["verdict"] + .as_str() + .expect("verdict is a string"), + ) + .expect("verdict is contract vocabulary"); + + assert_eq!(verdict, GateVerdict::Conditional); + assert_eq!(gate_exit_code(verdict, true), 2); + } + #[test] fn introduced_rustfmt_warning_in_diff_is_not_downgraded() { // In-diff formatting warnings belong to the change: no downgrade. diff --git a/src/artifacts/pr_review.rs b/src/artifacts/pr_review.rs index f0fce05..89494d0 100644 --- a/src/artifacts/pr_review.rs +++ b/src/artifacts/pr_review.rs @@ -394,10 +394,12 @@ pub(crate) fn generate_pr_review( code_files, test_files )); } - if coverage.total_source > 0 && coverage.pct < 80 { + if let Some(pct) = coverage.pct + && pct < 80 + { warnings.push(format!( "Coverage review signal: {}% heuristic coverage ({}/{})", - coverage.pct, coverage.covered_count, coverage.total_source + pct, coverage.covered_count, coverage.total_source )); } else if code_files > 0 && test_files == 0 { warnings.push(format!( diff --git a/src/artifacts/report.rs b/src/artifacts/report.rs index 8737a0d..72d7b56 100644 --- a/src/artifacts/report.rs +++ b/src/artifacts/report.rs @@ -1,4 +1,4 @@ -//! report.json v1 generator +//! report.json v2 generator //! //! Single source of truth for the dashboard and external tooling. //! All data the dashboard needs is serialized here; the HTML renderer @@ -178,10 +178,20 @@ struct GateReason { message: String, } +/// One quality-summary entry, mirroring `MERGE_GATE.json`'s `decision`. +/// +/// `origin` carries the same truth here as it does there, and for the same +/// reason: the entry arrays mix hard failures with warning-level baseline +/// signals, so a consumer reading `introduced_quality_failures: ["Rustfmt"]` +/// next to `quality_pass: true` sees a pack that contradicts itself unless the +/// detail says which status produced the entry. Emitting it in one artifact and +/// not the other left the two readers of the same run disagreeing about what +/// "failure" means. #[derive(Serialize)] struct GateQualityFailureDetail { name: String, classification: &'static str, + origin: &'static str, } #[derive(Serialize)] @@ -458,6 +468,17 @@ struct Quality { #[derive(Serialize)] struct HeuristicsSection { available: bool, + /// `"measured"` only when loctree actually scanned at least one file. + /// Mirrors the `heuristics_loctree` gate status so a zero-file scan can + /// never be read as a clean scan (SKIP-AS-ZERO). + status: &'static str, + #[serde(skip_serializing_if = "Option::is_none")] + skip_reason: Option<&'static str>, + /// Files loctree actually scanned. `None` when heuristics never ran. + #[serde(skip_serializing_if = "Option::is_none")] + total_files: Option, + /// Counts are present only for a measured scan. `None` (field omitted) + /// means "not measured" — never a zero that pretends to be a result. #[serde(skip_serializing_if = "Option::is_none")] dead_exports: Option, #[serde(skip_serializing_if = "Option::is_none")] @@ -488,7 +509,12 @@ struct BreakingSection { #[derive(Serialize)] struct CoverageSection { - heuristic_ratio: f64, + /// `null` when there were no changed source files to evaluate. 0/0 is an + /// absence of measurement, not a perfect ratio. + heuristic_ratio: Option, + measured: bool, + #[serde(skip_serializing_if = "Option::is_none")] + not_measured_reason: Option<&'static str>, matched: usize, total: usize, txt_path: &'static str, @@ -686,6 +712,7 @@ fn build_report(input: &ReportInput<'_>) -> Report { .map(|detail| GateQualityFailureDetail { name: detail.name.clone(), classification: detail.classification.as_str(), + origin: detail.origin.as_str(), }) .collect(), policy_mode: ctx.policy_mode.to_string(), @@ -862,15 +889,39 @@ fn build_report(input: &ReportInput<'_>) -> Report { .filter(|b| matches!(b.kind, BreakingKind::ChangedSignature { .. })) .count(); - let heuristics_section = match input.heuristics { + // A disabled run is not a broken scanner. `heuristics::run_all` + // short-circuits to a DEFAULT result when `run_heuristics` is off, and the + // caller still passes it, so `Some(..)` alone cannot tell "loctree failed" + // from "loctree was never asked to run". Only the config knows. + let heuristics_input = input + .config + .run_heuristics + .then_some(input.heuristics) + .flatten(); + + let heuristics_section = match heuristics_input { Some(h) => { let loctree = h.loctree.as_ref(); + let available = loctree.map(|l| l.available).unwrap_or(false); + // Loctree can report success while having scanned nothing. Zero + // counts from a zero-file scan are "not measured", not "clean" — + // the same rule build_heuristics_gate_check applies to the gate. + let measured = available && h.summary.total_files > 0; HeuristicsSection { - available: loctree.map(|l| l.available).unwrap_or(false), - dead_exports: loctree.map(|l| l.dead_exports.len()), - cycles: loctree.map(|l| l.cycles.len()), - twins: loctree.map(|l| l.twins.exact_twins.len()), - dead_parrots: loctree.map(|l| l.twins.dead_parrots.len()), + available, + status: if measured { "measured" } else { "skipped" }, + skip_reason: if measured { + None + } else if available { + Some("loctree scanned no files") + } else { + Some("loctree analysis unavailable") + }, + total_files: Some(h.summary.total_files), + dead_exports: measured.then(|| loctree.map_or(0, |l| l.dead_exports.len())), + cycles: measured.then(|| loctree.map_or(0, |l| l.cycles.len())), + twins: measured.then(|| loctree.map_or(0, |l| l.twins.exact_twins.len())), + dead_parrots: measured.then(|| loctree.map_or(0, |l| l.twins.dead_parrots.len())), log_path: Some("20_quality/heuristics_loctree.log"), analysis_root: h.analysis_root.clone(), regression: h.regression.clone(), @@ -878,6 +929,9 @@ fn build_report(input: &ReportInput<'_>) -> Report { } None => HeuristicsSection { available: false, + status: "skipped", + skip_reason: Some("heuristics not run"), + total_files: None, dead_exports: None, cycles: None, twins: None, @@ -903,11 +957,13 @@ fn build_report(input: &ReportInput<'_>) -> Report { new_env_vars_count: new_env_vars, }, coverage: CoverageSection { - heuristic_ratio: if ctx.coverage.total_source > 0 { - ctx.coverage.covered_count as f64 / ctx.coverage.total_source as f64 - } else { - 1.0 - }, + // 0/0 must serialize as null + measured:false, never as a 1.0 that + // downstream renders as "100% coverage". + heuristic_ratio: (ctx.coverage.total_source > 0) + .then(|| ctx.coverage.covered_count as f64 / ctx.coverage.total_source as f64), + measured: ctx.coverage.total_source > 0, + not_measured_reason: (ctx.coverage.total_source == 0) + .then_some("no changed source files to evaluate"), matched: ctx.coverage.covered_count, total: ctx.coverage.total_source, txt_path: "20_quality/coverage-delta.txt", @@ -1053,7 +1109,11 @@ fn build_report(input: &ReportInput<'_>) -> Report { .collect(); Report { - schema_version: "1.0", + // 2.0, not 1.1: `quality.coverage.heuristic_ratio` became nullable and + // the loctree counters became omittable, so a decoder written against + // 1.0 no longer parses every pack. Calling that additive would repeat, + // at the schema level, the "0/0 is 100%" lie the change removed. + schema_version: "2.0", meta, gate, checks: check_entries, @@ -1260,7 +1320,7 @@ test result: FAILED. 0 passed; 1 failed coverage: CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -1321,6 +1381,7 @@ test result: FAILED. 0 passed; 1 failed use crate::artifacts::signal::CoverageDelta; use crate::artifacts::{ CheckGateEntry, DashboardContext, QualityFailureClass, QualityFailureDetail, + QualityFailureOrigin, }; use crate::cli::ExecutionMode; use crate::config::test_config; @@ -1349,6 +1410,7 @@ test result: FAILED. 0 passed; 1 failed quality_failure_details: vec![QualityFailureDetail { name: "ESLint".to_string(), classification: QualityFailureClass::Preexisting, + origin: QualityFailureOrigin::Failure, }], policy_mode: "warn", blocking_issues: vec![], @@ -1363,7 +1425,7 @@ test result: FAILED. 0 passed; 1 failed coverage: CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -1427,6 +1489,116 @@ test result: FAILED. 0 passed; 1 failed ); } + #[test] + fn report_gate_names_the_origin_of_every_quality_failure_detail() { + // Without the origin, `introduced_quality_failures: ["Rustfmt"]` next to + // `quality_pass: true` reads as a contradiction: the array says the + // check failed, the flag says it never gated. `MERGE_GATE.json` has + // named the producing status since schema 2.2; `report.json` carried the + // same array without it, so the two artifacts of ONE run disagreed about + // what "failure" meant. + use crate::artifacts::signal::CoverageDelta; + use crate::artifacts::{ + CheckGateEntry, DashboardContext, QualityFailureClass, QualityFailureDetail, + QualityFailureOrigin, + }; + use crate::cli::ExecutionMode; + use crate::config::test_config; + use crate::git::ResolvedRef; + + let mut config = test_config(); + config.execution_mode = ExecutionMode::Standard; + + let ctx = DashboardContext { + verdict: "CONDITIONAL", + analysis_status: crate::policy::engine::AnalysisStatus::Complete, + merge_recommendation: crate::policy::engine::MergeRecommendation::ReviewRequired, + allow_merge: true, + // A warning-origin entry never fails the quality gate. + quality_pass: true, + policy_allow_merge: true, + recommended_merge: true, + review_caveats: vec![], + quality_failures: vec!["Rustfmt".to_string()], + introduced_quality_failures: vec!["Rustfmt".to_string()], + preexisting_quality_failures: vec![], + mixed_quality_failures: vec![], + unclassified_quality_failures: vec![], + quality_failure_details: vec![QualityFailureDetail { + name: "Rustfmt".to_string(), + classification: QualityFailureClass::Introduced, + origin: QualityFailureOrigin::Warning, + }], + policy_mode: "warn", + blocking_issues: vec![], + check_gates: vec![CheckGateEntry { + name: "Rustfmt".to_string(), + id: "rustfmt".to_string(), + blocking: false, + class: "WARN", + severity: "warn", + }], + breaking: vec![], + coverage: CoverageDelta { + total_source: 0, + covered_count: 0, + pct: None, + uncovered: vec![], + covered: vec![], + non_code_count: 0, + ghost_tests: vec![], + }, + findings: vec![], + per_file_diff_files: vec![], + skipped_checks: vec![], + previous_run: None, + run_history: vec![], + flaky_scores: vec![], + lint_metrics: vec![], + ownership_map: vec![], + risk_scores: vec![], + i18n_delta: None, + }; + let target = ResolvedRef { + name: "feature/report".to_string(), + commit_id: "deadbeef".to_string(), + is_remote: false, + }; + let bases = vec![ResolvedRef { + name: "main".to_string(), + commit_id: "cafebabe".to_string(), + is_remote: false, + }]; + + let tmp = tempfile::tempdir().expect("tempdir"); + let input = ReportInput { + dir: tmp.path(), + config: &config, + diffs: &[], + checks: &[], + resolved_target: &target, + resolved_bases: &bases, + ctx: &ctx, + run_started_at: "2026-03-09T00:00:00Z", + heuristics: None, + regression: None, + }; + + let report = build_report(&input); + let json = serde_json::to_value(&report).expect("serialize report"); + + let detail = &json["gate"]["quality_failure_details"][0]; + assert_eq!(detail["name"].as_str(), Some("Rustfmt")); + assert_eq!(detail["classification"].as_str(), Some("introduced")); + assert_eq!( + detail["origin"].as_str(), + Some("warning"), + "report.json must name the status that produced the entry, got: {}", + json["gate"]["quality_failure_details"] + ); + assert_eq!(json["gate"]["quality_pass"].as_bool(), Some(true)); + } + #[test] fn report_tracks_signature_changes_in_breaking_summary_and_counts() { use crate::artifacts::signal::CoverageDelta; @@ -1492,7 +1664,7 @@ test result: FAILED. 0 passed; 1 failed coverage: CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -1559,6 +1731,9 @@ test result: FAILED. 0 passed; 1 failed let mut config = test_config(); config.execution_mode = ExecutionMode::Standard; + // The fixture below is a loctree run that measured 10 files, so the + // config must be one that actually asked for heuristics. + config.run_heuristics = true; let ctx = DashboardContext { verdict: "PASS", @@ -1588,7 +1763,7 @@ test result: FAILED. 0 passed; 1 failed coverage: crate::artifacts::signal::CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -1675,6 +1850,234 @@ test result: FAILED. 0 passed; 1 failed assert_eq!(heuristics_json["unused_symbols"].as_u64(), Some(3)); assert!(heuristics_json.get("dead_parrots").is_none()); + assert_eq!(heuristics_json["status"].as_str(), Some("measured")); + assert_eq!(heuristics_json["total_files"].as_u64(), Some(10)); + assert!(heuristics_json.get("skip_reason").is_none()); + } + + // ── SKIP-AS-ZERO regression guards (report.json) ───────────────────── + + /// Minimal DashboardContext for report.json shape assertions. + fn skip_as_zero_ctx( + coverage: crate::artifacts::signal::CoverageDelta, + ) -> crate::artifacts::DashboardContext { + crate::artifacts::DashboardContext { + verdict: "PASS", + analysis_status: crate::policy::engine::AnalysisStatus::Complete, + merge_recommendation: crate::policy::engine::MergeRecommendation::Approve, + allow_merge: true, + quality_pass: true, + policy_allow_merge: true, + recommended_merge: true, + review_caveats: vec![], + quality_failures: vec![], + introduced_quality_failures: vec![], + preexisting_quality_failures: vec![], + mixed_quality_failures: vec![], + unclassified_quality_failures: vec![], + quality_failure_details: vec![], + policy_mode: "warn", + blocking_issues: vec![], + check_gates: vec![], + breaking: vec![], + coverage, + findings: vec![], + per_file_diff_files: vec![], + skipped_checks: vec![], + previous_run: None, + run_history: vec![], + flaky_scores: vec![], + lint_metrics: vec![], + ownership_map: vec![], + risk_scores: vec![], + i18n_delta: None, + } + } + + /// `run_heuristics` is the config flag, not a property of `heuristics`: + /// a disabled run still hands the report a default result, and telling the + /// two apart is the whole point of the heuristics section's skip reason. + fn skip_as_zero_report( + ctx: &crate::artifacts::DashboardContext, + heuristics: Option<&crate::heuristics::HeuristicsResult>, + run_heuristics: bool, + ) -> serde_json::Value { + use crate::cli::ExecutionMode; + use crate::config::test_config; + use crate::git::ResolvedRef; + + let mut config = test_config(); + config.execution_mode = ExecutionMode::Standard; + config.run_heuristics = run_heuristics; + let target = ResolvedRef { + name: "feature/skip-as-zero".to_string(), + commit_id: "deadbeef".to_string(), + is_remote: false, + }; + let bases = vec![ResolvedRef { + name: "main".to_string(), + commit_id: "cafebabe".to_string(), + is_remote: false, + }]; + let tmp = tempfile::tempdir().expect("tempdir"); + let input = ReportInput { + dir: tmp.path(), + config: &config, + diffs: &[], + checks: &[], + resolved_target: &target, + resolved_bases: &bases, + ctx, + run_started_at: "2026-03-12T00:00:00Z", + heuristics, + regression: None, + }; + serde_json::to_value(build_report(&input)).expect("serialize report") + } + + fn coverage_delta( + total_source: usize, + covered_count: usize, + pct: Option, + ) -> crate::artifacts::signal::CoverageDelta { + crate::artifacts::signal::CoverageDelta { + total_source, + covered_count, + pct, + uncovered: vec![], + covered: vec![], + non_code_count: 0, + ghost_tests: vec![], + } + } + + #[test] + fn report_coverage_zero_of_zero_is_null_not_full_ratio() { + let ctx = skip_as_zero_ctx(coverage_delta(0, 0, None)); + let json = skip_as_zero_report(&ctx, None, false); + let cov = &json["quality"]["coverage"]; + + assert!( + cov["heuristic_ratio"].is_null(), + "0/0 must serialize as null, got {:?}", + cov["heuristic_ratio"] + ); + assert_eq!(cov["measured"].as_bool(), Some(false)); + assert!(cov["not_measured_reason"].as_str().is_some()); + // Schema compatibility: the counters stay present for existing readers. + assert_eq!(cov["matched"].as_u64(), Some(0)); + assert_eq!(cov["total"].as_u64(), Some(0)); + } + + #[test] + fn report_schema_version_states_the_nullable_shape() { + // The unmeasured cut changed `heuristic_ratio` from a plain number to a + // nullable one, and made the loctree counters omittable. Both are shape + // changes a strict 1.0 decoder cannot survive, so leaving the stamp at + // "1.0" makes report.json misdescribe itself — the same class of lie the + // cut was fixing one level down. MINOR would promise old decoders keep + // working, which is exactly what stopped being true, so this is a MAJOR. + let ctx = skip_as_zero_ctx(coverage_delta(0, 0, None)); + let json = skip_as_zero_report(&ctx, None, false); + + assert!( + json["quality"]["coverage"]["heuristic_ratio"].is_null(), + "precondition: the nullable shape is what the version must describe" + ); + assert_eq!( + json["schema_version"].as_str(), + Some("2.0"), + "a nullable field and omittable counters are not an additive change" + ); + } + + #[test] + fn report_coverage_zero_of_n_stays_a_real_zero_ratio() { + // 0/3 IS a measurement — it must not be downgraded to "not measured". + let ctx = skip_as_zero_ctx(coverage_delta(3, 0, Some(0))); + let json = skip_as_zero_report(&ctx, None, false); + let cov = &json["quality"]["coverage"]; + + assert_eq!(cov["heuristic_ratio"].as_f64(), Some(0.0)); + assert_eq!(cov["measured"].as_bool(), Some(true)); + assert!(cov.get("not_measured_reason").is_none()); + assert_eq!(cov["total"].as_u64(), Some(3)); + } + + #[test] + fn report_heuristics_zero_file_scan_is_marked_skipped_without_zero_counts() { + use crate::heuristics::{HeuristicsResult, HeuristicsSummary, LoctreeAnalysis}; + + // Loctree "succeeded" but scanned nothing: the gate already calls this + // SKIP, so report.json must not emit dead_exports/cycles/twins = 0. + let heuristics = HeuristicsResult { + loctree: Some(LoctreeAnalysis { + available: true, + ..Default::default() + }), + summary: HeuristicsSummary { + total_files: 0, + ..Default::default() + }, + ..Default::default() + }; + let ctx = skip_as_zero_ctx(coverage_delta(0, 0, None)); + let json = skip_as_zero_report(&ctx, Some(&heuristics), true); + let h = &json["quality"]["heuristics"]; + + assert_eq!(h["status"].as_str(), Some("skipped")); + assert_eq!(h["skip_reason"].as_str(), Some("loctree scanned no files")); + assert_eq!(h["total_files"].as_u64(), Some(0)); + for key in ["dead_exports", "cycles", "twins", "unused_symbols"] { + assert!( + h.get(key).is_none(), + "{key} must be absent for a zero-file scan, got {:?}", + h.get(key) + ); + } + } + + #[test] + fn report_heuristics_disabled_is_not_reported_as_an_unavailable_scanner() { + use crate::heuristics::HeuristicsResult; + + // What a `--quick` / `--no-heuristics` run actually produces: + // `heuristics::run_all` short-circuits to a default result and `App::run` + // still passes it, so the report saw `Some(..)` with no loctree and + // called the scanner unavailable — a tool failure that never happened. + // A consumer cannot tell an intentional skip from a broken scanner, and + // the log path it was handed points at a zero-filled stub. + let heuristics = HeuristicsResult::default(); + let ctx = skip_as_zero_ctx(coverage_delta(0, 0, None)); + let json = skip_as_zero_report(&ctx, Some(&heuristics), false); + let h = &json["quality"]["heuristics"]; + + assert_eq!(h["available"].as_bool(), Some(false)); + assert_eq!(h["status"].as_str(), Some("skipped")); + assert_eq!(h["skip_reason"].as_str(), Some("heuristics not run")); + assert!( + h.get("log_path").is_none(), + "a run that never invoked loctree has no loctree log, got {:?}", + h.get("log_path") + ); + assert!( + h.get("total_files").is_none(), + "a disabled scan measured nothing, got {:?}", + h.get("total_files") + ); + } + + #[test] + fn report_heuristics_not_run_is_marked_skipped() { + let ctx = skip_as_zero_ctx(coverage_delta(0, 0, None)); + let json = skip_as_zero_report(&ctx, None, false); + let h = &json["quality"]["heuristics"]; + + assert_eq!(h["available"].as_bool(), Some(false)); + assert_eq!(h["status"].as_str(), Some("skipped")); + assert_eq!(h["skip_reason"].as_str(), Some("heuristics not run")); + assert!(h.get("total_files").is_none()); + assert!(h.get("dead_exports").is_none()); } #[test] @@ -1725,7 +2128,7 @@ test result: FAILED. 0 passed; 1 failed coverage: crate::artifacts::signal::CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, diff --git a/src/artifacts/signal/breaking.rs b/src/artifacts/signal/breaking.rs index 96b5891..657b371 100644 --- a/src/artifacts/signal/breaking.rs +++ b/src/artifacts/signal/breaking.rs @@ -72,6 +72,15 @@ const PUB_SYMBOL_TYPES: &[(&str, &str)] = &[ ("pub static ", "static"), ]; +/// Symbol-type label of a `pub ` declaration line (mirrors +/// `PUB_SYMBOL_TYPES`). Distinguishes namespaces that may share an identifier. +fn symbol_kind(line: &str) -> Option<&'static str> { + PUB_SYMBOL_TYPES + .iter() + .find(|(prefix, _)| line.starts_with(prefix)) + .map(|(_, symbol_type)| *symbol_type) +} + /// Extract the identifier following a `pub ` prefix (best-effort, mirrors /// `PUB_SYMBOL_TYPES`). Returns the symbol name for move-pairing. fn symbol_name(line: &str) -> Option { @@ -89,6 +98,306 @@ fn symbol_name(line: &str) -> Option { None } +/// Classify a public declaration line as `(symbol_type, name)`. +/// +/// Covers every kind in `PUB_SYMBOL_TYPES`, `pub fn` included: all of them go +/// through the same multi-line accumulator, so the recorded text is the full +/// declaration rather than its truncated opening line. +fn classify_pub_declaration(line: &str) -> Option<(&'static str, String)> { + PUB_SYMBOL_TYPES + .iter() + .find(|(prefix, _)| line.starts_with(prefix)) + .and_then(|(_, symbol_type)| symbol_name(line).map(|name| (*symbol_type, name))) +} + +/// Which side of the unified diff a declaration came from. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum DiffSide { + Removed, + Added, +} + +/// Where a declaration sits: everything about its position that pairing needs, +/// tracked per diff side. +struct DeclSite<'a> { + file: &'a str, + scope: &'a ModScope, + /// The `#[cfg(…)]` conjunction currently standing above the next + /// declaration on this side, `None` when the diff has not shown one. + cfg_guard: Option<&'a [String]>, + side: DiffSide, +} + +/// A public declaration collected from one side of the diff. +#[derive(Debug)] +struct SymbolDecl { + file: String, + symbol_type: String, + name: String, + /// Full declaration text — continuation lines joined, not just the opener. + /// + /// Verbatim, comments included: this is what a reader sees in + /// `BREAKING_CHANGES.md` and in a `ChangedSignature`. Comparisons use + /// [`identity`](Self::identity) instead. + text: String, + /// The same declaration with its comments resolved away. + /// + /// What pairing COMPARES. A comment inside a declaration is not part of the + /// API: rewording one used to make a remove+re-add of a byte-identical + /// signature come out as a `ChangedSignature` — a breaking-change claim + /// about text no consumer can observe. Literals are kept, because a literal + /// IS code: `pub const GREETING: &str = "hello";` and the same line ending + /// `"bye";` are different declarations. + identity: String, + /// Hunk-local declaration site — inline modules and `impl` owners joined + /// into one path (`""` when the diff never showed the opener). + scope: String, + /// Every `#[cfg(…)]` predicate guarding this declaration, whitespace + /// removed and sorted. `None` means the diff never showed one for this + /// side — unknown, not "unguarded". + cfg_guard: Option>, + side: DiffSide, + /// Continuation lines absorbed so far, capped by + /// [`MAX_DECL_CONTINUATION_LINES`]. + continuation_lines: usize, +} + +/// Continuation lines a single declaration may absorb before it is finalized +/// as-is. Bounds runaway accumulation — a `Lazy::new(|| { .. })` static body or +/// a hundred-line `pub const WORDS: &[&str] = &[` table, where "the rest of the +/// declaration" is data, not signature. +/// +/// The bound is a safety valve, NOT a display width: what gets truncated here is +/// the text the pairing COMPARES. At eight lines it cut inside the real +/// distribution, so two long declarations agreeing on their opener and first +/// eight lines finalized to the same truncated text, paired as an unchanged +/// re-add and swallowed a parameter, bound or return type changed below the cut. +/// Measured over 2,970,120 `pub` declarations in the local crates.io registry: +/// 94.76% wrap over no continuation line at all, 4.96% over one to eight, and +/// 0.27% over more — of which this bound now covers everything up to 32 lines +/// (87% of that remainder). What is left beyond it is dominated by generated +/// data tables. A declaration longer than the bound is still compared on its +/// first 32 lines, so a change below the cut can still hide; widening it further +/// trades that for smearing whole static bodies into one "declaration". +const MAX_DECL_CONTINUATION_LINES: usize = 32; + +/// Declaration-site nesting for ONE side of a unified diff: inline modules and +/// `impl` owners, tracked the same way because they are the same question — +/// which namespace does the next declaration belong to? +/// +/// Context lines feed both sides, `-` lines only the "before" side and `+` lines +/// only the "after" side, so a rename or a moved block cannot unbalance the +/// tracker. State is hunk-local: it resets at every `@@` header because hunks +/// are not contiguous, and an unseen opener simply leaves the scope unknown +/// (`""`) rather than inventing one. +#[derive(Default)] +struct ModScope { + /// `(scope name, brace depth the scope was opened at)`. + stack: Vec<(String, i32)>, + depth: i32, + /// Carries a `/* … */` or a string literal left open by an earlier line of + /// this side. + scanner: crate::rust_source::SourceScanner, +} + +impl ModScope { + fn reset(&mut self) { + self.stack.clear(); + self.depth = 0; + self.scanner.reset(); + } + + /// Feed one diff payload line to this side's tracker. + /// + /// Only the CODE part is counted. A brace inside a literal or a comment is + /// data: `const CLOSE: &str = "}";` inside `mod a` used to pop the module, + /// leaving a later removal of `a::Config` with an unknown scope — which + /// pairs with anything, so an unrelated `b::Config` addition cancelled a + /// real API removal. Block comments AND string literals are tracked across + /// lines: commenting a block of code out is exactly how an unbalanced brace + /// ends up inside a comment, and a multi-line template or JSON fixture is + /// exactly how one ends up inside a literal. State is per side and per + /// hunk — see [`ModScope::reset`]. + fn feed(&mut self, payload: &str) { + let code = self.scanner.code_only(payload); + let trimmed = code.trim(); + let opened = mod_opening_name(trimmed).or_else(|| impl_opening_scope(trimmed)); + let start_depth = self.depth; + for ch in code.chars() { + match ch { + '{' => self.depth += 1, + '}' => self.depth -= 1, + _ => {} + } + } + if let Some(name) = opened + && self.depth > start_depth + { + self.stack.push((name, start_depth)); + } + while let Some((_, opened_at)) = self.stack.last() { + if self.depth <= *opened_at { + self.stack.pop(); + } else { + break; + } + } + } + + fn path(&self) -> String { + self.stack + .iter() + .map(|(name, _)| name.as_str()) + .collect::>() + .join("::") + } +} + +/// Name of the inline module opened by `mod name {` / `pub mod name {` / +/// `pub(crate) mod name {`, if this line opens one. +/// +/// The brace must sit on the SAME line, which is what rustfmt emits and what +/// `mod name;` (a file module, not a scope) is told apart by. A declaration +/// written `mod name` with `{` on the next line is not recorded, so lines under +/// it carry no scope — `None`, which pairs with anything, the same conservative +/// default the diff already produces when a hunk omits the context line. +/// Carrying a pending name across lines is deliberately not done: the style is +/// 20 sites in a single crate out of 2025 sampled from crates.io (0.12% of +/// module declarations, zero here), and the state it needs would itself be +/// heuristic at hunk boundaries. +fn mod_opening_name(trimmed: &str) -> Option { + if !trimmed.contains('{') { + return None; + } + let mut rest = trimmed; + if let Some(after_pub) = rest.strip_prefix("pub") { + match after_pub.chars().next() { + Some('(') => rest = &after_pub[after_pub.find(')')? + 1..], + Some(c) if c.is_whitespace() => rest = after_pub, + _ => return None, + } + } + let rest = rest.trim_start().strip_prefix("mod")?; + if !rest.starts_with(char::is_whitespace) { + return None; + } + let rest = rest.trim_start(); + // A module may be named with a keyword through a raw identifier. Stopping at + // the `#` recorded `r#type` and `r#match` both as `r`, so two different + // namespaces looked like one and a removal from the first paired away + // against an unrelated addition in the second. The prefix is kept in the + // name because it is part of how the path is written. + let (prefix, rest) = match rest.strip_prefix("r#") { + Some(after) => ("r#", after), + None => ("", rest), + }; + let name: String = rest + .chars() + .take_while(|c| c.is_alphanumeric() || *c == '_') + .collect(); + (!name.is_empty()).then(|| format!("{prefix}{name}")) +} + +/// Scope name of the `impl` block this line opens, if it opens one. +/// +/// An associated item belongs to its impl owner exactly as an item belongs to +/// its module: `A::VALUE` disappearing while `B::VALUE` appears is a removal, +/// not a no-op re-add, even though both sit in one file with byte-identical +/// text. Without the owner in the scope both sides carried the empty path and +/// the exact pairing consumed the removal silently. +/// +/// The opener is recorded AS TEXT — everything before the body brace, whitespace +/// runs collapsed — and nothing about it is parsed. Two sides that show the same +/// header produce the same string, which is all the pairing asks; a header +/// rewritten between the sides (`impl A` -> `impl A`) reads as a different +/// owner, which is the text-level answer this scanner is allowed to give. It is +/// deliberately NOT a type parser: normalizing generics, lifetimes or paths here +/// would be the same over-reach the `cfg` operand-ordering limit already refuses, +/// and the identity comparison downstream is likewise textual. +/// +/// The same-line brace rule is inherited from [`mod_opening_name`], and so is +/// what happens without it: a header whose `{` sits on the next line records +/// nothing, the declarations under it carry the unknown scope `""`, and unknown +/// pairs with anything. That is the accepted limit for an opener the diff never +/// showed, and this function does not narrow it — it only speaks when the opener +/// IS visible. +/// +/// MEASURED. Over 211 commits of this repository — PR-sized diffs, the shape +/// this scanner actually reads — the reports are byte-identical with and without +/// the owner in the scope: 3 removals and 6 signature changes either way. Over +/// 708 consecutive-version pairs in the local crates.io registry (649 of them +/// touching Rust; whole releases, far coarser than any diff this scanner sees) +/// removals move 30,555 -> 30,694 and signature changes 53,938 -> 53,805, with +/// the number of patches reporting anything unchanged at 277. The change is +/// therefore mostly a RECLASSIFICATION: a declaration whose owner really changed +/// now reads as the removal of `A::x` instead of a signature change of `x`. +/// +/// ACCEPTED LIMIT — the owner is text, so the SAME owner written differently +/// reads as two. Of the 2,784 pairings the rule blocks across that corpus, 40 +/// (1.4%, all in one crate) differ only in a path qualifier: +/// `impl IReference` against the same header +/// without the leading `::`. Normalizing that means resolving paths, which is a +/// type parser — the same over-reach, and the same verdict, as the `cfg` operand +/// ordering limit in [`cfgs_may_pair`]. The error direction is the tolerable +/// one: a phantom removal is visible in review and rejected there, unlike a real +/// removal that pairs away silently. +fn impl_opening_scope(trimmed: &str) -> Option { + let header = trimmed.split('{').next()?; + if header.len() == trimmed.len() { + // No body brace on this line. + return None; + } + // `unsafe impl Send for A {}` opens a scope too. Checking the following + // character keeps `unsafely_impl_something` out. + let candidate = match header.strip_prefix("unsafe") { + Some(rest) if rest.starts_with(char::is_whitespace) => rest.trim_start(), + _ => header, + }; + let rest = candidate.strip_prefix("impl")?; + // `impl` must be the keyword, not the head of an identifier. A generic impl + // may open with `<` immediately (`impl Trait for T`). + if !rest.starts_with(char::is_whitespace) && !rest.starts_with('<') { + return None; + } + let name = header.split_whitespace().collect::>().join(" "); + (!name.is_empty()).then_some(name) +} + +/// May a removal in `removed_scope` and an addition in `added_scope` describe +/// the same declaration site? +/// +/// Scopes are hunk-local and often unknown, so an unknown scope stays +/// compatible with anything — that keeps today's pairing everywhere the diff +/// does not show a boundary. Two *known and different* paths mean two different +/// namespaces: `a::Config` disappearing while `b::Config` appears is a real +/// removal, not a no-op re-add, and so is `A::VALUE` disappearing while +/// `B::VALUE` appears (see [`impl_opening_scope`] — an `impl` owner is a +/// namespace exactly as a module is). +/// +/// The ASYMMETRY is the whole rule: known-vs-known blocks, but unknown on +/// EITHER side still pairs. Only an opener the hunk actually showed can speak, +/// so this never narrows the accepted unknown-scope limit below; it only fills +/// in the case where the diff already told us the answer. +/// +/// The known gap — two empty scopes pair even when the file's real modules +/// differ — is deliberate and measured, not overlooked. Over 173 commits of this +/// repository the current rule reports 3 removals and 4 signature changes, all +/// genuine. Treating an unknown scope as incompatible instead reports 7 removals +/// and 0 signature changes: it invents removals of symbols that are alive today +/// (`build_cli_json_summary`, `compute_exit_code`, `generate_diffs`, `McpArgs`) +/// and erases every real signature change, because a symbol whose declaration +/// moved across a hunk boundary then looks deleted. Seeding the scope from the +/// `@@` section heading does not close the gap either: only 149 of 1022 hunk +/// headers name a module at all, and virtually all of them say `mod tests`. +/// Closing it honestly needs the module path of the declaration site in the +/// source and target files, which means reading those files at those revisions — +/// input `analyze_patch_for_breaking_changes(patch: &str)` does not have. Until +/// this analysis is given repo + revision access, the ambiguous case resolves +/// toward not fabricating a breaking change. +fn scopes_may_pair(removed_scope: &str, added_scope: &str) -> bool { + removed_scope.is_empty() || added_scope.is_empty() || removed_scope == added_scope +} + /// Collect added public symbols across a patch as `(file, type, name)`. fn collect_added_public_symbols(patch: &str) -> Vec<(String, String, String)> { let mut out = Vec::new(); @@ -207,19 +516,39 @@ fn analyze_patch_for_breaking_changes(patch: &str) -> Vec { let mut current_file = String::new(); let mut should_scan_current_file = false; - // Track removed/added public function lines for signature change detection - let mut removed_fns: Vec<(String, String, String)> = Vec::new(); // (file, name, full_line) - let mut added_fns: Vec<(String, String, String)> = Vec::new(); + // Track removed/added public symbol declarations (ALL kinds in + // `PUB_SYMBOL_TYPES`, not just `pub fn`) for remove+re-add pairing and + // signature change detection. + let mut removed_syms: Vec = Vec::new(); + let mut added_syms: Vec = Vec::new(); + + // A public declaration may span several diff lines — `pub fn name(` with the + // parameters below it (BUG-4 / TOOLING-15), but equally `pub struct Name<` + // with its bounds below it. Accumulate continuation lines on BOTH sides so + // remove+re-add pairing compares full declarations: a change confined to a + // continuation line used to hide behind an identical opening line. + let mut pending_removed: Option = None; + let mut pending_added: Option = None; - // When an added `pub fn` signature spans multiple diff lines, accumulate the - // continuation lines so the "After" is the FULL signature, not just the - // truncated opening `pub fn name(` line (BUG-4 / TOOLING-15). - // (file, name, accumulated signature so far) - let mut pending_added_fn: Option<(String, String, String)> = None; + // Inline-module nesting, tracked per diff side (see `ModScope`). + let mut before_scope = ModScope::default(); + let mut after_scope = ModScope::default(); + + // The `#[cfg(…)]` currently standing above the next declaration, per side. + // Context lines feed both, so an unchanged guard above a re-emitted + // declaration is KNOWN on both sides and the pair is not split by it. + let mut before_cfg = CfgGuard::default(); + let mut after_cfg = CfgGuard::default(); for line in patch.lines() { // Track current file from diff headers if let Some(rest) = line.strip_prefix("diff --git a/") { + finalize_decl(&mut pending_removed, &mut removed_syms, &mut findings); + finalize_decl(&mut pending_added, &mut added_syms, &mut findings); + before_scope.reset(); + after_scope.reset(); + before_cfg.reset(); + after_cfg.reset(); if let Some(space_idx) = rest.find(" b/") { current_file = rest[space_idx + 3..].to_string(); should_scan_current_file = should_scan_for_breaking_changes(¤t_file); @@ -231,44 +560,52 @@ fn analyze_patch_for_breaking_changes(patch: &str) -> Vec { continue; } - // A pending multi-line signature is finalized by any non-added line. - let is_added_line = line.starts_with('+') && !line.starts_with("+++"); - if !is_added_line && let Some((f, n, sig)) = pending_added_fn.take() { - added_fns.push((f, n, sig)); + // Hunks are not contiguous: a boundary ends any pending declaration and + // invalidates the brace depth both scope trackers were carrying. + if line.starts_with("@@") { + finalize_decl(&mut pending_removed, &mut removed_syms, &mut findings); + finalize_decl(&mut pending_added, &mut added_syms, &mut findings); + before_scope.reset(); + after_scope.reset(); + before_cfg.reset(); + after_cfg.reset(); + continue; } + let removed_content = if line.starts_with("---") { + None + } else { + line.strip_prefix('-') + }; + let added_content = if line.starts_with("+++") { + None + } else { + line.strip_prefix('+') + }; + // Removed lines - if let Some(content) = line.strip_prefix('-') { + if let Some(content) = removed_content { + // A `-` line is absent from the after text, so it neither extends + // nor ends whatever the added side has open: the two accumulators + // reconstruct two independent texts out of one interleaved hunk. let trimmed = content.trim(); - let pub_types = [ - ("pub fn ", "function"), - ("pub struct ", "struct"), - ("pub enum ", "enum"), - ("pub trait ", "trait"), - ("pub type ", "type alias"), - ("pub const ", "constant"), - ("pub static ", "static"), - ]; - - for (pattern, symbol_type) in &pub_types { - if trimmed.starts_with(pattern) { - if *pattern == "pub fn " - && let Some(name) = extract_fn_name(trimmed) - { - removed_fns.push((current_file.clone(), name, trimmed.to_string())); - } - findings.push(BreakingFinding { - file: current_file.clone(), - kind: BreakingKind::RemovedSymbol { - symbol_type: symbol_type.to_string(), - }, - line: trimmed.to_string(), - risk_level: compute_breaking_risk(¤t_file), - }); - break; - } - } + // Record EVERY public symbol kind for remove+re-add pairing, not + // only `pub fn` — a non-fn declaration re-emitted unchanged by the + // diff used to leak a phantom removal. + accumulate_decl( + &mut pending_removed, + &mut removed_syms, + &mut findings, + content, + &DeclSite { + file: ¤t_file, + scope: &before_scope, + cfg_guard: before_cfg.guard(), + side: DiffSide::Removed, + }, + ); + before_cfg.feed(content); // JS/TS exports if trimmed.starts_with("export ") || trimmed.starts_with("export default") { @@ -281,35 +618,30 @@ fn analyze_patch_for_breaking_changes(patch: &str) -> Vec { risk_level: compute_breaking_risk(¤t_file), }); } + + before_scope.feed(content); + continue; } - // Added lines — track public functions for signature comparison + env requirements - if let Some(content) = line.strip_prefix('+') - && !line.starts_with("+++") - { + // Added lines — track public declarations for signature comparison + env requirements + if let Some(content) = added_content { let trimmed = content.trim(); - // Continuation of a multi-line signature already in progress. - if let Some((_, _, sig)) = pending_added_fn.as_mut() { - if !sig.ends_with('(') && !trimmed.is_empty() { - sig.push(' '); - } - sig.push_str(trimmed); - if signature_complete(sig) - && let Some(done) = pending_added_fn.take() - { - added_fns.push(done); - } - } else if trimmed.starts_with("pub fn ") - && let Some(name) = extract_fn_name(trimmed) - { - if signature_complete(trimmed) { - added_fns.push((current_file.clone(), name, trimmed.to_string())); - } else { - // Signature spans multiple lines — start accumulating. - pending_added_fn = Some((current_file.clone(), name, trimmed.to_string())); - } - } + accumulate_decl( + &mut pending_added, + &mut added_syms, + &mut findings, + content, + &DeclSite { + file: ¤t_file, + scope: &after_scope, + cfg_guard: after_cfg.guard(), + side: DiffSide::Added, + }, + ); + after_cfg.feed(content); + + after_scope.feed(content); // New env requirements if trimmed.contains("REQUIRED_ENV") || trimmed.contains(".env") { @@ -346,46 +678,100 @@ fn analyze_patch_for_breaking_changes(patch: &str) -> Vec { } } } + continue; } + + // Context (or non-hunk) line: it belongs to BOTH sides, so it extends a + // declaration open on either of them. `pub fn f(` / shared `x: u8,` / + // `-old: u16,` / `+new: u32,` / shared `) {` is one signature change on + // each side; truncating at the first shared line paired the identical + // openers and swallowed the break entirely. + // Every reader on this branch now takes the line raw and normalizes what + // it alone is entitled to: the accumulator keeps a literal's edges, the + // guard tracker keeps a literal's bytes, and the scope tracker counts + // braces. A shared pre-trim would decide that for all three. + let content = line.strip_prefix(' ').unwrap_or(line); + continue_pending_decl( + &mut pending_removed, + &mut removed_syms, + &mut findings, + content, + ); + continue_pending_decl(&mut pending_added, &mut added_syms, &mut findings, content); + before_cfg.feed(content); + after_cfg.feed(content); + before_scope.feed(content); + after_scope.feed(content); } - // Finalize a signature still being accumulated at end of patch. - if let Some(done) = pending_added_fn.take() { - added_fns.push(done); + // Finalize declarations still being accumulated at end of patch. + finalize_decl(&mut pending_removed, &mut removed_syms, &mut findings); + finalize_decl(&mut pending_added, &mut added_syms, &mut findings); + + // Pair removed + added public symbols of the SAME kind and name in the same + // file — every kind in `PUB_SYMBOL_TYPES`, not just `pub fn` (P1-09/10): + // - identical declaration -> no-op remove+readd, drop the removal + // (e.g. a fn body rewritten to delegate, or a struct whose fields + // changed below an unchanged `pub struct` line, emitted as -/+ by the + // diff) + // - different declaration -> a signature change, not a removal + // + // Pairing additionally requires compatible inline-module scopes, so a + // removal in one module is not cancelled by an unrelated same-named + // declaration added in another module of the same file. + // + // Pairing is one-to-one: an addition is consumed once. `cfg`-gated variants + // share (file, kind, name), so a non-consuming search let every removal + // cancel against the same unchanged re-add — the addition that actually + // replaced one of them was left unpaired and its change went unreported. + // Exact matches are claimed first (pass 1) so an unchanged re-add is never + // spent on a removal that a different addition replaces. + // + // ACCEPTED LIMIT (deferred to 0.8, do not re-litigate). Pairing sees only + // the declaration LINES the diff emitted. An enum variant, a trait method or + // a struct field removed below an unchanged `pub enum` / `pub trait` / + // `pub struct` opener is a breaking change this scanner does not report: + // the opener was never emitted as -/+, so nothing enters `removed_syms` to + // pair at all. Closing it needs the item's body from BOTH commits, which a + // diff-only scanner does not have; the fix is the repo-backed breaking + // analysis planned for 0.8, not a deeper heuristic here. The limit was + // reviewed and accepted deliberately — widening it here would trade a known + // blind spot for guesses about text the scanner never saw. + let mut added_used = vec![false; added_syms.len()]; + let mut unpaired_removed = Vec::new(); + + for removed in &removed_syms { + match find_pairable_addition(&added_syms, &added_used, removed, true) { + Some(index) => { + added_used[index] = true; + drop_removal_finding(&mut findings, removed); + } + None => unpaired_removed.push(removed), + } } - // Pair removed + added public functions in the same file: - // - identical signature line -> no-op remove+readd, drop the removal (P1-10: - // e.g. a body rewritten to delegate, with the `pub fn` line unchanged but - // emitted as -/+ by the diff) - // - different signature line -> a signature change, not a removal - for (r_file, r_name, r_line) in &removed_fns { - let Some((_, _, a_line)) = added_fns - .iter() - .find(|(a_file, a_name, _)| a_file == r_file && a_name == r_name) - else { + for removed in unpaired_removed { + let Some(index) = find_pairable_addition(&added_syms, &added_used, removed, false) else { continue; }; + added_used[index] = true; + let added = &added_syms[index]; - // Either way the removed-symbol finding is a false positive: drop it. - findings.retain(|f| { - !(f.file == *r_file - && matches!( - &f.kind, - BreakingKind::RemovedSymbol { symbol_type } if symbol_type == "function" - ) - && f.line == *r_line) - }); + // The removed-symbol finding is a false positive either way: drop it. + drop_removal_finding(&mut findings, removed); - if a_line != r_line { + // Compared on the comment-free identity, REPORTED verbatim: a reworded + // comment is not a signature change, but a reader shown the change + // should see the declaration as it is actually written. + if added.identity != removed.identity { findings.push(BreakingFinding { - file: r_file.clone(), + file: removed.file.clone(), kind: BreakingKind::ChangedSignature { - before: r_line.clone(), - after: a_line.clone(), + before: removed.text.clone(), + after: added.text.clone(), }, line: String::new(), - risk_level: compute_breaking_risk(r_file), + risk_level: compute_breaking_risk(&removed.file), }); } } @@ -393,26 +779,642 @@ fn analyze_patch_for_breaking_changes(patch: &str) -> Vec { findings } +/// Index of the first not-yet-consumed addition that may pair with `removed`. +/// +/// `require_identical_code` restricts the search to a declaration re-emitted +/// unchanged, which is what makes the two-pass pairing stable when several +/// declarations share (file, kind, name). "Unchanged" is judged on +/// [`SymbolDecl::identity`], so a re-emission that only reworded a comment is +/// still the exact match it looks like to a compiler. +fn find_pairable_addition( + added_syms: &[SymbolDecl], + added_used: &[bool], + removed: &SymbolDecl, + require_identical_code: bool, +) -> Option { + added_syms.iter().enumerate().find_map(|(index, added)| { + (!added_used[index] + && added.file == removed.file + && added.symbol_type == removed.symbol_type + && added.name == removed.name + && scopes_may_pair(&removed.scope, &added.scope) + && cfgs_may_pair(&removed.cfg_guard, &added.cfg_guard) + && (!require_identical_code || added.identity == removed.identity)) + .then_some(index) + }) +} + +/// May a removal and an addition guarded by these `cfg` predicates be the same +/// declaration? +/// +/// Two KNOWN predicates that differ never pair. `#[cfg(feature = "a")] +/// pub struct Config;` replaced by the same struct under feature `b` is an +/// exact text match, so the pairing dropped the removal — but `Config` really +/// did disappear for anyone building with feature `a`, which is precisely the +/// breaking change the report exists to name. +/// +/// A guard is the WHOLE conjunction of the attributes above the declaration. +/// Keeping only the last one made `#[cfg(unix)] #[cfg(feature = "x")]` and +/// `#[cfg(windows)] #[cfg(feature = "x")]` compare equal on the shared feature +/// alone, so a removal that really happened on Unix paired with a Windows-only +/// re-add and vanished. +/// +/// An unknown guard (`None`) pairs with anything, the same tolerance +/// [`scopes_may_pair`] gives an unseen module opener: the attribute may simply +/// sit on a context line this hunk did not re-emit on that side, and treating +/// "not shown" as "no cfg" would turn ordinary re-adds into phantom removals. +/// +/// ACCEPTED LIMIT — predicates are compared as text, so reordering the operands +/// INSIDE one attribute does not pair. `#[cfg(any(unix, windows))]` rewritten as +/// `#[cfg(any(windows, unix))]` gates the item identically, but the two strings +/// differ, and an untouched declaration under it reports a phantom +/// `RemovedSymbol`. (Reordering whole STACKED attributes does pair — [`record`] +/// sorts the conjunction.) Normalizing this properly means canonicalizing +/// arbitrarily nested predicates, which is a `cfg` parser, and the measurements +/// say the parser would not earn its risk. Across 708 consecutive-version pairs +/// in the local crates.io registry — whole releases, far wider than any diff +/// this scanner reads — 32 `cfg` attributes were reordered at all, in 2 crates; +/// across the 393 of those pairs that are patch-level bumps, the closest +/// available proxy for a PR-sized change, ZERO. And of the 32, only 6 are +/// reachable by sorting an attribute's direct operands: the dominant real shape +/// is `not(any(a, b, c))`, where the reorder sits one level down. A bounded sort +/// would therefore close a fifth of an already absent class while LOOKING +/// complete, which is worse than a limit written down. The error direction is +/// the tolerable one — a phantom removal is visible in review and rejected +/// there, unlike a real removal that pairs away silently. +fn cfgs_may_pair(removed: &Option>, added: &Option>) -> bool { + match (removed, added) { + (Some(removed), Some(added)) => removed == added, + _ => true, + } +} + +/// Does this line end the run of attributes standing above a declaration? +/// +/// Attributes, doc comments and blank lines sit between a `cfg` and the item it +/// guards without breaking the link; anything else is a new item. The line +/// arrives with its comments already resolved away, so `/** … */` — the block +/// form of `///`, wrapped or not — reaches this as the blank line it is. +fn breaks_attribute_run(trimmed: &str) -> bool { + !trimmed.is_empty() && !trimmed.starts_with("#[") +} + +/// An attribute may wrap over this many lines before the tracker gives up on it. +/// +/// A diff shows attributes the same way it shows everything else — partially. An +/// opener whose close never arrives would otherwise swallow the rest of the hunk +/// as continuation lines and keep a stale guard standing over declarations it +/// does not gate. +const MAX_ATTRIBUTE_CONTINUATION_LINES: usize = 32; + +/// An attribute whose delimiters have not closed yet. +struct OpenAttribute { + /// Everything read so far, whitespace removed. + text: String, + /// How many delimiters are still open. + depth: usize, + /// How many lines it has absorbed. + lines: usize, +} + +/// One diff side's `#[cfg(…)]` conjunction standing above the next declaration. +/// +/// Attributes wrap. `#[cfg(any(` on its own line used to be recorded as the +/// whole predicate, and the very next line — `feature = "a",` — was then read as +/// a new item and cleared the guard: the declaration below it came out +/// unguarded, so a struct that really disappeared for one configuration paired +/// with its re-add under a different one and left no finding. An attribute is +/// therefore accumulated until its delimiters balance, and only the finished +/// text becomes a guard. +/// +/// Whitespace OUTSIDE the literals is dropped so `#[cfg(feature="a")]`, +/// `#[cfg(feature = "a")]` and the same predicate wrapped across four lines are +/// ONE predicate: reformatting an attribute is not a different gate, and reading +/// it as one would report a removal that never happened. Whitespace INSIDE a +/// literal is kept, because it is part of the value the compiler matches on: +/// dropping it too made `#[cfg(api = "a b")]` and `#[cfg(api = "ab")]` one +/// guard, and a struct that really left builds configured with +/// `--cfg 'api="a b"'` paired with its re-add under another value. +/// +/// Comments are resolved away before any of that, by one +/// [`SourceScanner`](crate::rust_source::SourceScanner) per side fed one +/// physical line at a time. A block comment is not syntax on either count: a +/// `/** … */` doc comment standing between the `cfg` and its item used to read +/// as a new item and clear the guard, and a `/* ))) */` inside a wrapped +/// predicate balanced the attribute early with the same result — both sides +/// unguarded, the identical declaration text paired, and a struct that really +/// left one configuration produced no finding. +/// +/// The two questions asked of a line want opposite treatments of its literals, +/// so each gets its own scanner, fed in step. The guard TEXT keeps them, because +/// `#[cfg(feature = "a")]` and `#[cfg(feature = "b")]` are different gates and a +/// view that dropped literal bodies would make them one. The DELIMITER COUNT +/// must not see them: a `)` typed inside a multi-line `#[doc = r#"…"#]` is text, +/// and reading it as syntax balanced the attribute early, after which the +/// remaining literal lines looked like ordinary items and cleared the pending +/// `cfg` — both sides unguarded again. +#[derive(Default)] +struct CfgGuard { + /// The accumulated conjunction, or `None` for "not known on this side". + guards: Option>, + open: Option, + /// Resolves comments away and keeps literals: the view the guard text is + /// built from. Carries an open `/* … */` or literal between lines, and is + /// reset with the guard, because the diff has jumped elsewhere. + text_scanner: crate::rust_source::SourceScanner, + /// The same walk with literal bodies dropped: the view the delimiters are + /// counted on. A separate scanner rather than a second call, because one + /// scanner reads each line exactly once. + depth_scanner: crate::rust_source::SourceScanner, +} + +impl CfgGuard { + /// The conjunction currently standing above the next declaration. + fn guard(&self) -> Option<&[String]> { + self.guards.as_deref() + } + + /// Forget everything: the diff has jumped somewhere else. + fn reset(&mut self) { + self.forget_attributes(); + self.text_scanner.reset(); + self.depth_scanner.reset(); + } + + /// Drop the guard and any attribute in flight, keeping the scanner state. + /// + /// Used when an attribute overruns its continuation cap: the diff has NOT + /// jumped anywhere, so a literal or `/* … */` still open on this side is + /// still open on the next line. + fn forget_attributes(&mut self) { + self.guards = None; + self.open = None; + } + + /// Advance this side past `raw`. + /// + /// Call it AFTER the line has been offered to the declaration accumulator: a + /// declaration is guarded by the attribute above it, not by one on its own + /// line. + /// + /// Takes the line RAW, with its indentation still on it. Trimming at the + /// caller ate a continuation line's leading whitespace before this tracker + /// could ask whether that whitespace was inside a value. + fn feed(&mut self, raw: &str) { + // Both scanners read every line, in step: they carry the same open + // constructs and differ only in what they emit. + let counted = self.depth_scanner.code_only(raw); + let counted = counted.trim().to_string(); + + // Read BEFORE this line is scanned: a literal the PREVIOUS line left + // open makes the break between the two — and this line's leading + // whitespace — part of the value rather than layout. + let continues_literal = self.text_scanner.carries_literal(); + // The dense view has ALREADY removed every byte of whitespace outside + // the literals, so what is left at this line's edges can only be inside + // one. That is why nothing here trims: a `.trim()` at this point cannot + // reach layout any more, only value. + let resolved = self.text_scanner.code_with_literals_dense(raw); + let line: &str = &resolved; + if let Some(open) = self.open.as_mut() { + // The physical break is layout everywhere but inside a literal, + // where it is a byte of the value: gluing the lines unconditionally + // made `#[cfg(api = "a\nb")]` the same guard as `#[cfg(api = "ab")]` + // and paired a configuration-specific removal away. + if continues_literal { + open.text.push('\n'); + } + open.text.push_str(line); + open.depth = delimiter_depth(&counted, open.depth); + open.lines += 1; + if open.depth == 0 { + let finished = self.open.take().expect("open attribute").text; + self.record(finished); + } else if open.lines >= MAX_ATTRIBUTE_CONTINUATION_LINES { + // Unknown pairs with anything, which is the tolerant direction: + // a guard invented from an unfinished attribute would fabricate + // removals out of ordinary re-adds. + self.forget_attributes(); + } + return; + } + + if line.starts_with("#[") { + let text = line.to_string(); + let depth = delimiter_depth(&counted, 0); + if depth == 0 { + self.record(text); + } else { + self.open = Some(OpenAttribute { + text, + depth, + lines: 1, + }); + } + return; + } + + if breaks_attribute_run(line) { + self.guards = None; + } + } + + /// Add one finished attribute to the conjunction. + /// + /// Consecutive `#[cfg(…)]` attributes ACCUMULATE — stacking them is Rust's + /// `AND` — and the accumulated set is sorted, because `#[cfg(a)] #[cfg(b)]` + /// and `#[cfg(b)] #[cfg(a)]` gate the item identically and a reorder is not + /// an API change. Any other attribute keeps the run alive but adds nothing: + /// a `#[derive(…)]` between the `cfg` and its item does not change the gate. + fn record(&mut self, attribute: String) { + if !gates_the_item(&attribute) { + return; + } + let guards = self.guards.get_or_insert_with(Vec::new); + guards.push(attribute); + guards.sort(); + guards.dedup(); + } +} + +/// Does this whitespace-stripped attribute decide whether the item exists? +/// +/// `#[cfg(…)]` obviously does. So does `#[cfg_attr(feature = "a", cfg(unix))]`: +/// it applies a `cfg` under a condition, so the item is gated just as surely — +/// reading only the literal `#[cfg(` spelling dropped BOTH sides' guards, the +/// identical declaration text then paired, and a struct that really left the +/// Unix build produced no finding at all. +/// +/// The rest of the `cfg_attr` family — `#[cfg_attr(unix, derive(Debug))]`, +/// `#[cfg_attr(docsrs, doc(cfg(…)))]` — decides an attribute ON the item, not +/// the item, and must stay out: a gate invented there would split an ordinary +/// re-add into a phantom removal, which is the error direction that costs +/// trust. `,cfg(` is the whole distinction, applied to text whitespace has +/// already been stripped from, and it separates the two families exactly across +/// the 44,562 `cfg_attr` attributes in the local registry: 189 apply a `cfg` +/// (12 crates, the `portable-atomic` idiom), and in none of them does the +/// substring fall inside a string literal. +fn gates_the_item(attribute: &str) -> bool { + attribute.starts_with("#[cfg(") + || (attribute.starts_with("#[cfg_attr(") && attribute.contains(",cfg(")) +} + +/// How many delimiters `line` leaves open, starting from `depth`. +/// +/// Takes a line neither comments nor literals have survived: [`CfgGuard::feed`] +/// resolves both away before the line gets here. Counting either as syntax +/// balanced an attribute early — `/* ))) */` inside a wrapped `#[cfg(…)]` +/// predicate, a `)` inside a multi-line `#[doc = r#"…"#]` — and the lines below +/// it then read as ordinary items and cleared the pending guard. +/// +/// A per-line literal state of its own is what this function used to carry, and +/// it could not see a raw string opened on an EARLIER line: `r#"` has no +/// closing delimiter until `"#`, so the bare `"` starting the last line read as +/// an opener and the `]` after it was swallowed as text. +fn delimiter_depth(line: &str, depth: usize) -> usize { + let mut depth = depth; + for c in line.chars() { + match c { + '(' | '[' | '{' => depth += 1, + ')' | ']' | '}' => depth = depth.saturating_sub(1), + _ => {} + } + } + depth +} + +/// Drop ONE removed-symbol finding matching `removed`. +/// +/// One removal cancelled by one addition retires exactly one finding: two +/// identical `cfg`-gated removals against a single re-add must leave the second +/// one reported. +fn drop_removal_finding(findings: &mut Vec, removed: &SymbolDecl) { + let matching = findings.iter().position(|f| { + f.file == removed.file + && matches!( + &f.kind, + BreakingKind::RemovedSymbol { symbol_type } if *symbol_type == removed.symbol_type + ) + && f.line == removed.text + }); + if let Some(index) = matching { + findings.remove(index); + } +} + +/// A declaration still absorbing continuation lines. +/// +/// `decl.text` is the verbatim join used for identity and for the +/// `BREAKING_CHANGES.md` row. Completeness is decided on `code` instead, which +/// is the same lines read through a [`SourceScanner`] — one call per PHYSICAL +/// line, which is what makes a `//` end where it really ends. Joining first and +/// scanning the result once cannot: the join has no line breaks, so a comment on +/// any continuation line commented out every line appended after it, the closing +/// `)` and body `{` were never seen, and the accumulator ran on into the body +/// until the cap — turning a body-only rewrite into a phantom signature change. +/// +/// The scanner is what keeps the other direction right too: it carries an open +/// literal or `/* … */` from one continuation line to the next, so a brace +/// inside a multi-line string still does not end the declaration. +struct PendingDecl { + decl: SymbolDecl, + code: String, + /// Feeds `code`: literal bodies dropped, because this view counts delimiters. + completeness: crate::rust_source::SourceScanner, + /// Feeds `decl.identity`: literals kept, because this view compares source. + /// Two scanners rather than one because the two views want different output + /// from the same resolution; both see the same lines in the same order, so + /// their carried state never diverges. + identity: crate::rust_source::SourceScanner, +} + +/// Start or continue accumulating a public declaration on one diff side. +/// +/// A declaration in progress absorbs `trimmed` as a continuation line; otherwise +/// `trimmed` may open a new one. Either way the declaration is emitted as soon +/// as it is complete (or once it has absorbed [`MAX_DECL_CONTINUATION_LINES`]). +/// Feed one more physical line into a declaration that is already accumulating +/// on this side, and report whether there was one. +/// +/// This is the *continuation-only* half of [`accumulate_decl`]: it never starts +/// a declaration. Context lines of a unified hunk belong to both the before and +/// the after text, so they have to extend whatever each side already has open — +/// but a `pub` item that first appears on a context line is unchanged by the +/// patch and must not become a declaration on either side. +/// +/// Takes the line RAW — with its indentation still on it. Everything but the +/// identity view wants it normalized, and normalizing is this function's job +/// rather than the caller's: the identity is the one reader for which a line's +/// edge whitespace can be value instead of layout, and a caller that has already +/// trimmed cannot tell the two apart any more. +fn continue_pending_decl( + pending: &mut Option, + collected: &mut Vec, + findings: &mut Vec, + raw: &str, +) -> bool { + let Some(open) = pending.as_mut() else { + return false; + }; + let trimmed = raw.trim(); + if !open.decl.text.ends_with('(') && !trimmed.is_empty() { + open.decl.text.push(' '); + } + open.decl.text.push_str(trimmed); + open.push_code(raw); + open.decl.continuation_lines += 1; + if declaration_complete(&open.code) + || open.decl.continuation_lines >= MAX_DECL_CONTINUATION_LINES + { + finalize_decl(pending, collected, findings); + } + true +} + +/// Takes the line RAW, for the reason given on [`continue_pending_decl`]. +fn accumulate_decl( + pending: &mut Option, + collected: &mut Vec, + findings: &mut Vec, + raw: &str, + site: &DeclSite<'_>, +) { + if continue_pending_decl(pending, collected, findings, raw) { + return; + } + + let trimmed = raw.trim(); + let Some((symbol_type, name)) = classify_pub_declaration(trimmed) else { + return; + }; + let decl = SymbolDecl { + file: site.file.to_string(), + symbol_type: symbol_type.to_string(), + name, + text: trimmed.to_string(), + identity: String::new(), + scope: site.scope.path(), + cfg_guard: site.cfg_guard.map(<[String]>::to_vec), + side: site.side, + continuation_lines: 0, + }; + let mut open = PendingDecl { + decl, + code: String::new(), + completeness: crate::rust_source::SourceScanner::default(), + identity: crate::rust_source::SourceScanner::default(), + }; + open.push_code(raw); + if declaration_complete(&open.code) { + emit_decl(open.decl, collected, findings); + } else { + *pending = Some(open); + } +} + +impl PendingDecl { + /// Read one more physical line, RAW, into both derived views. + fn push_code(&mut self, raw: &str) { + // The completeness view counts delimiters, so indentation is noise: it + // gets the normalized line. + let trimmed = raw.trim(); + self.code.push_str(&self.completeness.code_only(trimmed)); + // The line ended: whatever the scanner still carries is a literal or a + // block comment, never a line comment. + self.code.push(' '); + + // Read BEFORE this line is scanned: a literal the PREVIOUS line left + // open makes the break between the two — and the whitespace at THIS + // line's leading edge — part of the value rather than layout. + let continues_literal = self.identity.carries_literal(); + + // The identity is fed the raw line and normalized per edge, because the + // two edges are not the same question. Trimming both unconditionally + // made re-indenting the inside of a multiline constant a no-op, hiding a + // changed public value; keeping both would make every rustfmt pass a + // wall of phantom signature changes. + let scanned = self.identity.code_with_literals(raw); + // Read AFTER: the trailing edge is value only while the literal is still + // open at the end of the line. + let ends_in_literal = self.identity.carries_literal(); + let mut line: &str = &scanned; + if !continues_literal { + line = line.trim_start(); + } + // A line that is nothing but a comment contributes nothing to the + // identity, and a line whose code ends where its comment begins must not + // contribute the whitespace between them either — otherwise `a: u8, //x` + // and `a: u8,// y` would read as different declarations. Inside a + // literal there is no comment to strip and a blank line is a blank line + // in the value, so it is kept. + if !ends_in_literal { + line = line.trim_end(); + } + if line.is_empty() && !continues_literal { + return; + } + if !self.decl.identity.is_empty() { + // Physical boundaries are preserved only where they are part of the + // value. Joining a literal's lines with a space made a constant + // written across two lines compare equal to the same constant + // rewritten with a space in it, so a changed public value paired + // away as an unchanged re-add. Everywhere else the break is layout: + // preserving it there made `pub type Alias =` + `u32;` a different + // declaration from `pub type Alias = u32;`, and a purely cosmetic + // reflow was reported as a changed signature — with an identical + // "before" and "after", since those are joined with a space. + self.decl + .identity + .push(if continues_literal { '\n' } else { ' ' }); + } + self.decl.identity.push_str(line); + } +} + +/// Emit a declaration that is no longer accumulating, if any. +fn finalize_decl( + pending: &mut Option, + collected: &mut Vec, + findings: &mut Vec, +) { + if let Some(open) = pending.take() { + emit_decl(open.decl, collected, findings); + } +} + +/// Record a finished declaration; removals also become a `RemovedSymbol` +/// finding, which the pairing pass may later drop or upgrade. +fn emit_decl( + decl: SymbolDecl, + collected: &mut Vec, + findings: &mut Vec, +) { + if decl.side == DiffSide::Removed { + findings.push(BreakingFinding { + file: decl.file.clone(), + kind: BreakingKind::RemovedSymbol { + symbol_type: decl.symbol_type.clone(), + }, + line: decl.text.clone(), + risk_level: compute_breaking_risk(&decl.file), + }); + } + collected.push(decl); +} + fn should_scan_for_breaking_changes(path: &str) -> bool { matches!(classify_review_file(path), ReviewFileCategory::Code) } -/// Has this (possibly partial) `pub fn` signature reached its end? +/// Has this (possibly partial) public declaration reached its end? +/// +/// A declaration is complete once its parens are balanced and it has reached the +/// body opener `{` or a `;` (trait method, type alias, const, static). Used to +/// decide whether to keep accumulating continuation lines so both "Before" and +/// "After" are full declarations (BUG-4 / TOOLING-15). /// -/// A signature is complete once its parameter parens are balanced and it has -/// reached the body opener `{` or a `;` (trait method / declaration). Used to -/// decide whether to keep accumulating continuation lines for the "After" -/// reconstruction (BUG-4 / TOOLING-15). -fn signature_complete(sig: &str) -> bool { +/// Only real delimiters count, and `code` has already had them resolved: it is +/// the declaration's lines read through the pending declaration's own +/// [`SourceScanner`], one call per physical line. `pub const TEMPLATE: &str = +/// r#"{` opens a multi-line literal, and reading that `{` as the body opener +/// finalized a TRUNCATED declaration — identical on both diff sides, so the +/// removal was cancelled and the literal change the patch actually made went +/// unreported. The scanner carries that literal across the continuation lines, +/// and ends a `//` comment at the line that wrote it. +/// +/// A `{` in TYPE position is not a body opener either. `pub type Alias = +/// Buffer<{` states a const argument, and finalizing there truncated both diff +/// sides to the same prefix: they paired as an unchanged re-add and a changed +/// const expression — a different public type — produced no finding. +fn declaration_complete(code: &str) -> bool { let mut depth: i32 = 0; - for ch in sig.chars() { + // How deep inside a generic argument list the scan is. A `{` opened there — + // `Buffer<{ LIMIT * 2 }>`, `Buffer` — is type-level + // syntax, not the item's body opener. + let mut angle: i32 = 0; + // How deep inside a brace that is NOT the item's body the scan is: a const + // argument, or an initializer block after a top-level `=`. + let mut block: i32 = 0; + // Has a top-level `=` been seen? After one, the item states a VALUE and + // runs to its `;` — every `{` from there on opens the initializer, never + // the body of a struct, enum, trait or function. + let mut initializes = false; + let mut prev = '\0'; + let mut chars = code.chars().peekable(); + while let Some(ch) = chars.next() { + // `<<` is the shift operator, never a nesting of two argument lists in + // any declaration this scanner sees. Consuming both characters is what + // keeps the 4,666 public `const`/`static` declarations in the local + // registry that state a shift on their own line terminating at their + // `;` instead of accumulating past it. + if ch == '<' && chars.peek() == Some(&'<') { + chars.next(); + prev = '<'; + continue; + } match ch { - '(' => depth += 1, - ')' => depth -= 1, - '{' if depth <= 0 => return true, - ';' if depth <= 0 => return true, + // `<` opens an argument list only directly after an identifier, a + // closing `>`, or a `:` — `Buffer<`, `Vec>`, `Buffer::<` — which + // is where a type names its arguments and is not where a comparison + // puts it. The `:` admits the turbofish spelling, which rustc accepts + // in type position without a warning: `pub fn run() -> Buffer::<{` + // left the list uncounted, so the const block's `{` read as the + // item's body opener and BOTH diff sides finalized at that identical + // prefix — they paired as an unchanged re-add and a changed const + // argument, a different public return type, was reported nowhere. + // Whitespace is still not an opener, so a comparison stays one. + // + // Inside a const block the same characters are OPERATORS, so the + // depth is frozen there: `Buffer<{ 1 < 2 }>` counted the comparison + // as an opener, the argument list's `>` then closed only that phantom + // level, and `angle` was still above zero at the item's real body + // brace — which therefore read as another const block and swallowed + // the whole body, turning a body-only rewrite into a phantom + // `ChangedSignature`. A block's own generics (`size_of::()`) + // are balanced against themselves, so freezing loses nothing. + '<' if block == 0 + && (prev.is_alphanumeric() || prev == '_' || prev == '>' || prev == ':') => + { + angle += 1 + } + // `->` is a return arrow, not a closing bracket. + '>' if block == 0 && prev != '-' && angle > 0 => angle -= 1, + // Only a top-level `=` is an initializer. Inside a generic argument + // list it states a default (`struct Foo`) or an + // associated type (`impl Iterator`), and both of those + // are followed by a body brace that must still end the declaration. + // `==`, `=>` and the compound assignments are not initializers either. + '=' if depth <= 0 + && angle == 0 + && block == 0 + && !matches!(chars.peek(), Some(&('=' | '>'))) + && !"=!<>+-*/%&|^".contains(prev) => + { + initializes = true + } + // Tracking the whole argument list rather than the exact `<{` + // sequence is what catches a const argument that is not the FIRST + // one, where the `{` follows a comma. Measured against the local + // registry (59,946 files, 2,025 crates, 4,354,142 public + // declaration lines), the two rules judge zero lines differently. + '{' if angle > 0 || block > 0 || initializes => block += 1, + '}' if block > 0 => block -= 1, + // Square brackets are counted for the same reason parentheses are: + // an array type states its length with a `;` — `pub const TABLE: + // [u8; 2] = [` — and reading that as the terminator finalized the + // declaration at its opener. Both sides of a diff then held the same + // opener text, paired as an unchanged re-add, and a changed + // initializer below produced no finding at all. + '(' | '[' => depth += 1, + ')' | ']' => depth -= 1, + // A `;` inside an initializer block is a statement terminator, not + // the declaration's. + '{' | ';' if depth <= 0 && block == 0 => return true, _ => {} } + if !ch.is_whitespace() { + prev = ch; + } } false } @@ -427,7 +1429,46 @@ fn extract_fn_name(line: &str) -> Option { Some(name.trim().to_string()) } +/// Grouping identity of a `ChangedSignature` row: `(file, symbol kind, name)`. +/// The kind separates namespaces that may legally share an identifier. +type ChangedSignatureKey = (String, &'static str, String); + /// Format breaking changes as markdown. +/// Make one value safe to drop into a markdown table cell. +/// +/// A table row is delimited by `|`, and Rust states bitwise or, patterns and +/// closures with the same character: `pub const MASK: u32 = READ | WRITE;` in a +/// cell opened two new columns and the row rendered as garbage. GitHub's table +/// parser splits on UNESCAPED pipes before any inline markup runs, so `\|` is +/// the escape even inside a code span. +fn escape_table_cell(text: &str) -> std::borrow::Cow<'_, str> { + if text.contains('|') { + std::borrow::Cow::Owned(text.replace('|', r"\|")) + } else { + std::borrow::Cow::Borrowed(text) + } +} + +/// Render one value as a markdown code span inside a table cell. +/// +/// Pipe escaping alone is not enough: a declaration may hold a backtick of its +/// own — `pub const TEMPLATE: &str = r#"`value`"#;` — and a single-backtick span +/// ends at the first interior one, so the rest of the declaration rendered as +/// prose. CommonMark's answer is a fence LONGER than any backtick run in the +/// content, plus one space of padding when the content itself begins or ends +/// with a backtick (the renderer strips exactly that pair back off). +fn code_cell(text: &str) -> String { + let escaped = escape_table_cell(text); + let longest_run = escaped.split(|c| c != '`').map(str::len).max().unwrap_or(0); + let fence = "`".repeat(longest_run + 1); + let pad = if escaped.starts_with('`') || escaped.ends_with('`') { + " " + } else { + "" + }; + format!("{fence}{pad}{escaped}{pad}{fence}") +} + fn format_breaking_changes(findings: &[BreakingFinding]) -> String { let mut md = String::new(); @@ -469,7 +1510,13 @@ fn format_breaking_changes(findings: &[BreakingFinding]) -> String { md.push_str("|------|--------|------|\n"); for f in &removed { if let BreakingKind::RemovedSymbol { symbol_type } = &f.kind { - let _ = writeln!(md, "| {} | `{}` | {} |", f.file, f.line, symbol_type); + let _ = writeln!( + md, + "| {} | {} | {} |", + escape_table_cell(&f.file), + code_cell(&f.line), + symbol_type + ); } } md.push('\n'); @@ -481,7 +1528,13 @@ fn format_breaking_changes(findings: &[BreakingFinding]) -> String { md.push_str("|------|--------|------|\n"); for f in &relocated { if let BreakingKind::RelocatedSymbol { symbol_type } = &f.kind { - let _ = writeln!(md, "| {} | `{}` | {} |", f.file, f.line, symbol_type); + let _ = writeln!( + md, + "| {} | {} | {} |", + escape_table_cell(&f.file), + code_cell(&f.line), + symbol_type + ); } } md.push('\n'); @@ -493,17 +1546,24 @@ fn format_breaking_changes(findings: &[BreakingFinding]) -> String { md.push_str("|------|--------|-------|\n"); // Collapse feature-gated duplicates: the same logical signature change // is often emitted once per `#[cfg(feature = ...)]` variant. Group by - // (file, fn name), render one row, and note the variant count - // (BUG-4 / TOOLING-15). - let mut order: Vec<(String, String)> = Vec::new(); - let mut groups: std::collections::HashMap<(String, String), Vec<(&String, &String)>> = + // (file, symbol kind, name), render one row, and note the variant count + // (BUG-4 / TOOLING-15). The kind is part of the key because non-fn + // declarations also land here: `pub struct Limit` and `pub const Limit` + // live in different namespaces, so sharing an identifier must not + // collapse them into a single row. + let mut order: Vec = Vec::new(); + let mut groups: std::collections::HashMap> = std::collections::HashMap::new(); for f in &changed { if let BreakingKind::ChangedSignature { before, after } = &f.kind { let name = extract_fn_name(before) .or_else(|| extract_fn_name(after)) + .or_else(|| symbol_name(before)) .unwrap_or_else(|| before.clone()); - let key = (f.file.clone(), name); + let kind = symbol_kind(before) + .or_else(|| symbol_kind(after)) + .unwrap_or(""); + let key = (f.file.clone(), kind, name); if !groups.contains_key(&key) { order.push(key.clone()); } @@ -516,15 +1576,21 @@ fn format_breaking_changes(findings: &[BreakingFinding]) -> String { if variants.len() > 1 { let _ = writeln!( md, - "| {} | `{}` | `{}` _(+{} feature-gated variant{})_ |", - key.0, - before, - after, + "| {} | {} | {} _(+{} feature-gated variant{})_ |", + escape_table_cell(&key.0), + code_cell(before), + code_cell(after), variants.len() - 1, if variants.len() - 1 == 1 { "" } else { "s" } ); } else { - let _ = writeln!(md, "| {} | `{}` | `{}` |", key.0, before, after); + let _ = writeln!( + md, + "| {} | {} | {} |", + escape_table_cell(&key.0), + code_cell(before), + code_cell(after) + ); } } md.push('\n'); @@ -536,7 +1602,12 @@ fn format_breaking_changes(findings: &[BreakingFinding]) -> String { md.push_str("|------|----------|\n"); for f in &env_reqs { if let BreakingKind::NewEnvRequirement { variable } = &f.kind { - let _ = writeln!(md, "| {} | `{}` |", f.file, variable); + let _ = writeln!( + md, + "| {} | {} |", + escape_table_cell(&f.file), + code_cell(variable) + ); } } md.push('\n'); @@ -550,6 +1621,71 @@ mod tests { use super::*; use crate::regression::tests::is_test_file; + #[test] + fn a_pipe_in_a_declaration_does_not_break_the_table_columns() { + // Declaration text goes into a markdown table, and Rust states bitwise + // or, patterns and closures with `|`. Written verbatim it opened new + // columns, so every row carrying one rendered as garbage — the report + // stopped being readable exactly where the declaration was interesting. + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/limits.rs", + &[ + "-pub const MASK: u32 = READ | WRITE;", + "+pub const MASK: u32 = READ | WRITE | EXEC;", + ], + )]); + + let md = format_breaking_changes(&findings); + let row = md + .lines() + .find(|l| l.contains("MASK")) + .expect("the changed constant is reported"); + assert!( + row.contains(r"\|"), + "the pipe must be escaped for the table: {row}" + ); + // A GitHub table row is `| a | b | c |`: splitting on UNESCAPED pipes + // leaves the two empty ends plus one field per column. + let cells = row.replace(r"\|", "\u{0}").split('|').count(); + assert_eq!(cells, 5, "three columns, two empty ends: {row}"); + } + + #[test] + fn a_backtick_in_a_declaration_does_not_end_its_code_span_early() { + // The other half of the escaping: a declaration whose literal states a + // backtick — a doc snippet, a shell template — wrapped in a + // single-backtick span closes that span at the first interior backtick, + // and the rest of the declaration renders as prose instead of code. + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/limits.rs", + &["-pub const TEMPLATE: &str = r#\"`value`\"#;"], + )]); + + let md = format_breaking_changes(&findings); + let row = md + .lines() + .find(|l| l.contains("TEMPLATE")) + .expect("the removed constant is reported"); + let cell = row.split('|').nth(2).expect("the symbol column").trim(); + // The fence must be longer than the longest backtick run inside it, + // which is CommonMark's rule for a span containing backticks. + assert_eq!( + cell, "``pub const TEMPLATE: &str = r#\"`value`\"#;``", + "the declaration must survive verbatim inside a longer fence: {row}" + ); + } + + #[test] + fn a_cell_that_begins_with_a_backtick_is_padded() { + // CommonMark strips exactly one leading and one trailing space from a + // code span, which is how a span whose CONTENT starts or ends with a + // backtick keeps that backtick instead of lengthening the fence. + assert_eq!(code_cell("`x`"), "`` `x` ``"); + assert_eq!(code_cell("plain"), "`plain`"); + assert_eq!(code_cell("a ``b`` c"), "```a ``b`` c```"); + assert_eq!(code_cell("A | B"), r"`A \| B`"); + } + #[test] fn breaking_changes_detects_removed_pub_fn() { let patch = "diff --git a/src/lib.rs b/src/lib.rs\n\ @@ -776,11 +1912,690 @@ mod tests { ); } + /// Build a single-file patch from raw diff body lines (each already carrying + /// its `-`/`+`/` ` prefix). + fn one_file_patch(file: &str, body: &[&str]) -> String { + let mut patch = + format!("diff --git a/{file} b/{file}\n--- a/{file}\n+++ b/{file}\n@@ -1,2 +1,2 @@\n"); + for line in body { + patch.push_str(line); + patch.push('\n'); + } + patch + } + + fn removed_symbol_types(findings: &[BreakingFinding]) -> Vec { + findings + .iter() + .filter_map(|f| match &f.kind { + BreakingKind::RemovedSymbol { symbol_type } => Some(symbol_type.clone()), + _ => None, + }) + .collect() + } + #[test] - fn changed_signature_reconstructs_multiline_after() { - // The new signature is split across several lines in the diff. The - // "After" must be the FULL reconstructed signature, not just the - // truncated opening `pub fn query_index(` line (BUG-4 / TOOLING-15). + fn identical_remove_readd_non_fn_symbols_are_not_breaking() { + // P1-09/10 residual: the same-file remove+re-add pairing used to cover + // `pub fn` ONLY, so a struct/enum/trait/type/const/static whose + // declaration line was re-emitted unchanged by the diff (e.g. a body or + // field reordering below it) produced a phantom RemovedSymbol. + let cases: [(&str, &str, &str); 6] = [ + ("struct", "src/model.rs", "pub struct Config {"), + ("enum", "src/model.rs", "pub enum Mode {"), + ("trait", "src/model.rs", "pub trait Check {"), + ("type alias", "src/model.rs", "pub type Alias = u32;"), + ("constant", "src/model.rs", "pub const LIMIT: usize = 8;"), + ("static", "src/model.rs", "pub static NAME: &str = \"a\";"), + ]; + + for (label, file, decl) in cases { + let findings = analyze_all_breaking_changes(&[one_file_patch( + file, + &[ + &format!("-{decl}"), + "- old_detail: u8,", + &format!("+{decl}"), + "+ new_detail: u8,", + ], + )]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "identical remove+readd of {label} must produce no breaking finding, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + } + + #[test] + fn changed_non_fn_declaration_is_signature_change_not_removal() { + // A genuinely modified public declaration must surface as a signature + // change (one finding), never as removal + silent re-addition. + let cases: [(&str, &str, &str); 3] = [ + ( + "struct", + "pub struct Config {", + "pub struct Config {", + ), + ( + "type alias", + "pub type Alias = u32;", + "pub type Alias = u64;", + ), + ( + "constant", + "pub const LIMIT: usize = 8;", + "pub const LIMIT: u32 = 8;", + ), + ]; + + for (label, before, after) in cases { + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/model.rs", + &[&format!("-{before}"), &format!("+{after}")], + )]); + assert!( + findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::ChangedSignature { before: b, after: a } + if b == before && a == after + )), + "{label} change must be a ChangedSignature, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + assert!( + removed_symbol_types(&findings).is_empty(), + "{label} change must not also report a removal, got: {:?}", + removed_symbol_types(&findings) + ); + } + } + + #[test] + fn genuine_non_fn_removal_stays_removed() { + // Guard against the pairing over-reaching: with no re-add, real removals + // of every public symbol kind must still be breaking. + let cases: [(&str, &str); 6] = [ + ("struct", "pub struct Config {"), + ("enum", "pub enum Mode {"), + ("trait", "pub trait Check {"), + ("type alias", "pub type Alias = u32;"), + ("constant", "pub const LIMIT: usize = 8;"), + ("static", "pub static NAME: &str = \"a\";"), + ]; + + for (expected_type, decl) in cases { + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/model.rs", + &[&format!("-{decl}"), "- detail: u8,"], + )]); + assert!( + removed_symbol_types(&findings).contains(&expected_type.to_string()), + "real removal of {expected_type} must stay a RemovedSymbol, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + } + + #[test] + fn duplicate_declarations_pair_one_to_one() { + // cfg-gated variants share (file, kind, name). Pairing scanned the added + // side without consuming the match, so every removal cancelled against + // the SAME unchanged addition: the real `u32 -> u64` change paired with + // nothing and vanished, and the widening went unreported. + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/model.rs", + &[ + "-pub type Value = u32;", + "-pub type Value = u32;", + "+pub type Value = u32;", + "+pub type Value = u64;", + ], + )]); + + let changes: Vec<(&String, &String)> = findings + .iter() + .filter_map(|f| match &f.kind { + BreakingKind::ChangedSignature { before, after } => Some((before, after)), + _ => None, + }) + .collect(); + assert_eq!( + changes.len(), + 1, + "the second removal must pair with the leftover addition: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + assert_eq!(changes[0].0, "pub type Value = u32;"); + assert_eq!(changes[0].1, "pub type Value = u64;"); + } + + #[test] + fn a_const_block_in_a_generic_argument_is_not_the_body_opener() { + // `pub type Alias = Buffer<{` opens a const argument, not an item body. + // Finalizing there truncated BOTH sides to the same prefix, they paired + // as an unchanged re-add, and the changed const expression below — + // a different public type — produced no finding at all. + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/model.rs", + &[ + "-pub type Alias = Buffer<{", + "- LIMIT * 2", + "-}>;", + "+pub type Alias = Buffer<{", + "+ LIMIT * 3", + "+}>;", + ], + )]); + + let changes: Vec<(&String, &String)> = findings + .iter() + .filter_map(|f| match &f.kind { + BreakingKind::ChangedSignature { before, after } => Some((before, after)), + _ => None, + }) + .collect(); + assert_eq!( + changes.len(), + 1, + "a changed const argument is a changed public type: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + assert!(changes[0].0.contains("LIMIT * 2"), "{}", changes[0].0); + assert!(changes[0].1.contains("LIMIT * 3"), "{}", changes[0].1); + } + + #[test] + fn a_comparison_inside_a_const_block_does_not_swallow_the_item_body() { + // Inside a const argument the `<` and `>` are OPERATORS, not delimiters. + // Counting one as a generic opener left `angle` above zero when the + // argument list closed, so the item's real body brace read as another + // const block and the whole body was absorbed into the declaration — + // making this body-only rewrite a phantom `ChangedSignature`. + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/model.rs", + &[ + "-pub fn run() -> Buffer<{ 1 < 2 }> {", + "- compute(2)", + "-}", + "+pub fn run() -> Buffer<{ 1 < 2 }> {", + "+ compute(3)", + "+}", + ], + )]); + + assert!( + findings.is_empty(), + "a body-only rewrite is not a signature change: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_changed_const_comparison_is_still_a_signature_change() { + // The other direction: the const expression is part of the public type, + // so changing it must still be reported. Freezing the angle depth + // inside the block must not make the declaration end early. + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/model.rs", + &[ + "-pub fn run() -> Buffer<{ 1 < 2 }> {", + "+pub fn run() -> Buffer<{ 1 < 3 }> {", + ], + )]); + + let changes: Vec<(&String, &String)> = findings + .iter() + .filter_map(|f| match &f.kind { + BreakingKind::ChangedSignature { before, after } => Some((before, after)), + _ => None, + }) + .collect(); + assert_eq!( + changes.len(), + 1, + "a changed const argument is a changed signature: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + assert!(changes[0].0.contains("1 < 2"), "{}", changes[0].0); + assert!(changes[0].1.contains("1 < 3"), "{}", changes[0].1); + } + + #[test] + fn a_turbofish_inside_a_const_block_keeps_its_own_balance() { + // A const block may legitimately state generics of its own — + // `size_of::()`. Freezing the outer depth inside the block leaves + // that pair balanced against itself, so the declaration still ends at + // its real body brace. + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/model.rs", + &[ + "-pub fn run() -> Buffer<{ size_of::() }> {", + "- compute(2)", + "-}", + "+pub fn run() -> Buffer<{ size_of::() }> {", + "+ compute(3)", + "+}", + ], + )]); + + assert!( + findings.is_empty(), + "a body-only rewrite is not a signature change: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_qualified_path_inside_a_const_block_stays_balanced() { + // The shape the corpus actually carries (crypto-bigint): + // `Uint<{ ::LIMBS / 2 }>`. Its `<`/`>` pair is a qualified path, + // not an argument list, and the previous rule only survived it by + // cancellation — the path's `>` decremented the OUTER list, whose own + // `>` then found nothing left to close. Freezing keeps both honest. + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/model.rs", + &[ + "-pub fn split(&self) -> Uint<{ ::LIMBS / 2 }> {", + "- compute(2)", + "-}", + "+pub fn split(&self) -> Uint<{ ::LIMBS / 2 }> {", + "+ compute(3)", + "+}", + ], + )]); + + assert!( + findings.is_empty(), + "a body-only rewrite is not a signature change: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_const_block_in_a_later_generic_argument_is_not_the_body_opener_either() { + // The same construct one argument along: `Buffer<1, {` opens a const + // argument whose `{` follows a comma, not a `<`. A const generic is + // rarely the FIRST argument, so this is the shape the previous rule's + // exact `<{` sequence missed while covering the rarer one. + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/model.rs", + &[ + "-pub type Alias = Buffer;", + "+pub type Alias = Buffer;", + ], + )]); + + let changes: Vec<(&String, &String)> = findings + .iter() + .filter_map(|f| match &f.kind { + BreakingKind::ChangedSignature { before, after } => Some((before, after)), + _ => None, + }) + .collect(); + assert_eq!( + changes.len(), + 1, + "a changed const argument is a changed public type: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + assert!(changes[0].0.contains("LIMIT * 2"), "{}", changes[0].0); + assert!(changes[0].1.contains("LIMIT * 3"), "{}", changes[0].1); + } + + #[test] + fn a_shared_context_line_does_not_truncate_both_sides_of_a_declaration() { + // A context line belongs to BOTH versions, so it continues whatever each + // side was accumulating. Finalizing on it left both accumulators at the + // identical opener — only its comment was reworded — they paired as an + // unchanged re-add, and the changed parameter below was ignored because + // a continuation line does not start a declaration. The signature break + // vanished from the pack. + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/api.rs", + &[ + "-pub fn handler( // takes the old width", + "+pub fn handler( // takes the new width", + " first: u8,", + "- second: u16,", + "+ second: u32,", + " ) -> bool {", + ], + )]); + + let changes: Vec<(&String, &String)> = findings + .iter() + .filter_map(|f| match &f.kind { + BreakingKind::ChangedSignature { before, after } => Some((before, after)), + _ => None, + }) + .collect(); + assert_eq!( + changes.len(), + 1, + "the changed parameter is a signature change: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + assert!(changes[0].0.contains("u16"), "{}", changes[0].0); + assert!(changes[0].1.contains("u32"), "{}", changes[0].1); + } + + #[test] + fn an_opposite_side_line_does_not_truncate_a_pending_declaration() { + // The same rule for the other interleaving: a `-` line is not part of + // the added declaration and a `+` line is not part of the removed one, + // so neither ENDS the other — it simply is not fed to it. Finalizing + // there cut both declarations at their identical first line. + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/api.rs", + &[ + "-pub fn handler(", + "+pub fn handler(", + "- first: u8,", + "+ first: u8,", + "- second: u16,", + "+ second: u32,", + "-) -> bool {", + "+) -> bool {", + ], + )]); + + let changes: Vec<(&String, &String)> = findings + .iter() + .filter_map(|f| match &f.kind { + BreakingKind::ChangedSignature { before, after } => Some((before, after)), + _ => None, + }) + .collect(); + assert_eq!( + changes.len(), + 1, + "the changed parameter is a signature change: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + assert!(changes[0].0.contains("u16"), "{}", changes[0].0); + assert!(changes[0].1.contains("u32"), "{}", changes[0].1); + } + + #[test] + fn a_block_initializer_is_not_the_body_opener() { + // `pub const LIMIT: usize = {` opens an initializer, not an item body: + // the declaration runs to the `;` after the block's `}`. Finalizing at + // the `{` truncated both diff sides to the same first line, they paired + // as an unchanged re-add, and the changed expression inside the block + // produced no finding at all. + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/limits.rs", + &[ + "-pub const LIMIT: usize = {", + "- let base = 2;", + "- base * 3", + "-};", + "+pub const LIMIT: usize = {", + "+ let base = 2;", + "+ base * 4", + "+};", + ], + )]); + + let changes: Vec<(&String, &String)> = findings + .iter() + .filter_map(|f| match &f.kind { + BreakingKind::ChangedSignature { before, after } => Some((before, after)), + _ => None, + }) + .collect(); + assert_eq!( + changes.len(), + 1, + "a changed block-valued constant is a changed public value: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + assert!(changes[0].0.contains("base * 3"), "{}", changes[0].0); + assert!(changes[0].1.contains("base * 4"), "{}", changes[0].1); + } + + #[test] + fn a_body_brace_after_a_const_argument_still_ends_the_declaration() { + // Guard against over-reach: the const block closes on the same line and + // the NEXT brace is the real body. Swallowing it would run the + // accumulator into the body and turn a body-only rewrite into a phantom + // signature change. + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/model.rs", + &[ + "-pub fn to_hex(&self) -> ArrayString<{ 2 * OUT_LEN }> {", + "- self.old_body()", + "-}", + "+pub fn to_hex(&self) -> ArrayString<{ 2 * OUT_LEN }> {", + "+ self.new_body()", + "+}", + ], + )]); + + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "a body-only rewrite is not a signature change: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_turbofish_return_type_does_not_finalize_at_its_const_block() { + // `Buffer::<{` is a valid return type — the turbofish spelling of a + // const argument, which rustc accepts without a warning. Its `<` follows + // a `:`, so the argument list went uncounted, the const block's `{` read + // as the item's body opener, and BOTH sides finalized at that identical + // prefix. They paired as an unchanged re-add and the changed const + // expression below — a different public return type — was reported + // nowhere. This is the direction that HIDES a break. + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/model.rs", + &[ + "-pub fn run() -> Buffer::<{", + "- LIMIT * 2", + "-}> {", + "- compute()", + "-}", + "+pub fn run() -> Buffer::<{", + "+ LIMIT * 3", + "+}> {", + "+ compute()", + "+}", + ], + )]); + + let changes: Vec<(&String, &String)> = findings + .iter() + .filter_map(|f| match &f.kind { + BreakingKind::ChangedSignature { before, after } => Some((before, after)), + _ => None, + }) + .collect(); + assert_eq!( + changes.len(), + 1, + "a changed const argument is a changed return type: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + assert!( + changes[0].0.contains("LIMIT * 2") && changes[0].1.contains("LIMIT * 3"), + "the finding must carry the const expression that changed: {:?}", + changes[0] + ); + } + + #[test] + fn a_turbofish_return_type_re_emitted_verbatim_is_a_no_op() { + // The other direction of the same rule: counting the turbofish list must + // not turn an untouched signature into a phantom change when only the + // body below it was rewritten. + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/model.rs", + &[ + "-pub fn run() -> Buffer::<{", + "- LIMIT * 2", + "-}> {", + "- compute(2)", + "-}", + "+pub fn run() -> Buffer::<{", + "+ LIMIT * 2", + "+}> {", + "+ compute(3)", + "+}", + ], + )]); + + assert!( + findings.is_empty(), + "a body-only rewrite is not a signature change: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_comparison_after_whitespace_still_does_not_open_an_argument_list() { + // The guard on widening the opener rule. `:` joins identifier and `>` as + // a predecessor that can open a list, but whitespace must not: a `<` + // after a space is a comparison, and counting it would leave every such + // constant accumulating past its `;`. + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/model.rs", + &[ + "-pub const OK: bool = 1 < 2;", + "-pub struct Keep;", + "+pub const OK: bool = 1 < 3;", + "+pub struct Keep;", + ], + )]); + + let changes: Vec<(&String, &String)> = findings + .iter() + .filter_map(|f| match &f.kind { + BreakingKind::ChangedSignature { before, after } => Some((before, after)), + _ => None, + }) + .collect(); + assert_eq!( + changes.len(), + 1, + "the constant is one declaration, the struct below another: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + assert_eq!(changes[0].0, "pub const OK: bool = 1 < 2;"); + } + + #[test] + fn a_shifted_constant_still_terminates_at_its_semicolon() { + // `1 << 3` is why the argument-list tracker consumes `<<` whole: 4,666 + // public `const`/`static` declarations in the local registry state a + // shift on their own line, and a `<` counted as an opener there would + // leave every one of them accumulating past its `;`. + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/model.rs", + &[ + "-pub const MASK: u32 = 1 << 3;", + "-pub struct Keep;", + "+pub const MASK: u32 = 1 << 4;", + "+pub struct Keep;", + ], + )]); + + let changes: Vec<(&String, &String)> = findings + .iter() + .filter_map(|f| match &f.kind { + BreakingKind::ChangedSignature { before, after } => Some((before, after)), + _ => None, + }) + .collect(); + assert_eq!( + changes.len(), + 1, + "the constant is one declaration, the struct below another: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + assert_eq!(changes[0].0, "pub const MASK: u32 = 1 << 3;"); + assert_eq!(changes[0].1, "pub const MASK: u32 = 1 << 4;"); + } + + #[test] + fn unpaired_duplicate_removal_stays_a_removal() { + // Two removals, one addition: one removal is genuinely gone. Consuming + // the addition must leave the second removal reported, not silently + // cancelled by an addition already spent on the first. + let findings = analyze_all_breaking_changes(&[one_file_patch( + "src/model.rs", + &[ + "-pub type Value = u32;", + "-pub type Value = u16;", + "+pub type Value = u32;", + ], + )]); + + assert_eq!( + removed_symbol_types(&findings), + vec!["type alias".to_string()], + "exactly one removal survives: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + assert!( + !findings + .iter() + .any(|f| matches!(&f.kind, BreakingKind::ChangedSignature { .. })), + "the identical pair is a no-op, not a signature change: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn pub_use_reexport_is_not_a_tracked_public_symbol() { + // `pub use` is deliberately outside PUB_SYMBOL_TYPES: re-export lines + // churn constantly and were never emitted as RemovedSymbol, so neither + // an identical nor a changed remove+re-add may invent a breaking + // finding. Pins that contract so a future symbol-kind addition cannot + // reintroduce the phantom asymmetry unpaired. + let identical = analyze_all_breaking_changes(&[one_file_patch( + "src/lib.rs", + &[ + "-pub use crate::model::Config;", + "+pub use crate::model::Config;", + ], + )]); + let changed = analyze_all_breaking_changes(&[one_file_patch( + "src/lib.rs", + &[ + "-pub use crate::model::Config;", + "+pub use crate::model::Settings;", + ], + )]); + + for (label, findings) in [("identical", identical), ("changed", changed)] { + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "{label} pub use re-export must produce no breaking finding, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + } + + #[test] + fn changed_signature_reconstructs_multiline_after() { + // The new signature is split across several lines in the diff. The + // "After" must be the FULL reconstructed signature, not just the + // truncated opening `pub fn query_index(` line (BUG-4 / TOOLING-15). let patch = "diff --git a/src/vector_index.rs b/src/vector_index.rs\n\ --- a/src/vector_index.rs\n\ +++ b/src/vector_index.rs\n\ @@ -860,6 +2675,1616 @@ mod tests { ); } + #[test] + fn changed_signatures_of_different_kinds_do_not_collapse() { + // Once non-fn declarations produce ChangedSignature findings, a name can + // repeat across namespaces in one file (`pub struct Limit` + + // `pub const Limit`). Grouping by (file, name) alone rendered one of + // them as a "feature-gated variant" of the other and dropped its row. + let patch = one_file_patch( + "src/model.rs", + &[ + "-pub struct Limit {", + "+pub struct Limit {", + "-pub const Limit: usize = 8;", + "+pub const Limit: usize = 16;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + let md = format_breaking_changes(&findings); + + assert!( + md.contains("pub struct Limit {"), + "struct change must have its own row, got:\n{md}" + ); + assert!( + md.contains("pub const Limit: usize = 16;"), + "const change must not be collapsed into the struct row, got:\n{md}" + ); + assert!( + !md.contains("variant"), + "two different symbol kinds are not feature-gated variants, got:\n{md}" + ); + } + + #[test] + fn same_name_in_different_inline_modules_is_not_a_no_op_pair() { + // `a::Config` is deleted while a same-named `b::Config` is added in the + // same file. Pairing on (file, kind, name) alone cancelled a genuine + // removal: the `a::Config` path is gone for every downstream consumer. + let patch = one_file_patch( + "src/model.rs", + &[ + " pub mod a {", + "- pub struct Config {", + "- pub x: u32,", + "- }", + " }", + " pub mod b {", + "+ pub struct Config {", + "+ pub x: u32,", + "+ }", + " }", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "removal from mod a must survive an unrelated add in mod b, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn raw_identifier_modules_are_different_scopes() { + // A module may be named with a keyword through a raw identifier. The + // scope parser stopped at the `#`, so `r#type` and `r#match` were both + // recorded as `r`: two different namespaces looked like one, and the + // removal of `r#type::Config` was cancelled by the unrelated addition of + // `r#match::Config`. + let patch = one_file_patch( + "src/model.rs", + &[ + " pub mod r#type {", + "- pub struct Config {", + "- pub x: u32,", + "- }", + " }", + " pub mod r#match {", + "+ pub struct Config {", + "+ pub x: u32,", + "+ }", + " }", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "removal from mod r#type must survive an add in mod r#match, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn literal_and_comment_braces_do_not_pop_the_module_scope() { + // A brace inside a literal or a comment is data. Counting it popped + // `mod a` early, so the removal of `a::Config` carried an unknown scope + // and paired with the unrelated addition of `b::Config` — the real API + // removal disappeared from the report. + let patch = one_file_patch( + "src/model.rs", + &[ + " pub mod a {", + " const CLOSE: &str = \"}\";", + " // trailing brace in a comment }", + "- pub struct Config {", + "- pub x: u32,", + "- }", + " }", + " pub mod b {", + "+ pub struct Config {", + "+ pub x: u32,", + "+ }", + " }", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "a brace in a literal or comment must not merge two module scopes, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_literal_spanning_lines_does_not_pop_the_module_scope() { + // Same defect, one line further on: the literal OPENS on one line and + // closes on the next, so the tail of its body reached the tracker as + // code and its `}` popped `mod a`. The removal of `a::Config` then + // carried an unknown scope, paired with the unrelated `b::Config` + // addition, and the real API removal vanished from the report. Multi- + // line literals are not exotic here: 241 of them live in this tree and + // 168 carry a brace in their body. + let patch = one_file_patch( + "src/model.rs", + &[ + " pub mod a {", + " const TEMPLATE: &str = \"opens {", + " closes } here\";", + "- pub struct Config {", + "- pub x: u32,", + "- }", + " }", + " pub mod b {", + "+ pub struct Config {", + "+ pub x: u32,", + "+ }", + " }", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "a brace inside a multi-line literal must not merge two module scopes, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn literal_brace_does_not_invent_a_module_scope() { + // The mirror direction: an unmatched `{` in a literal used to deepen the + // tracked scope, so a later removal and addition in the SAME module + // looked like two different namespaces and a plain no-op re-add was + // reported as a breaking removal. + let patch = one_file_patch( + "src/model.rs", + &[ + " pub mod a {", + " const OPEN: &str = \"{\";", + "- pub struct Config {", + "+ pub struct Config {", + " }", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "an identical re-add in one module is not breaking, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn an_associated_item_moved_between_impl_blocks_is_a_removal() { + // `A::VALUE` disappearing while `B::VALUE` appears is the same defect + // `raw_identifier_modules_are_different_scopes` covers one namespace + // kind up: same file, same kind, same name, byte-identical text, and an + // empty module scope on both sides, so the exact pairing consumed the + // removal. Nothing named `A::VALUE` exists after this diff. + let patch = one_file_patch( + "src/model.rs", + &[ + " impl A {", + "- pub const VALUE: u32 = 1;", + " }", + " impl B {", + "+ pub const VALUE: u32 = 1;", + " }", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"constant".to_string()), + "a const moved from impl A to impl B must survive as a removal, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn the_same_impl_owner_still_pairs_a_moved_declaration() { + // The no-op direction of the same rule: an item shuffled WITHIN one impl + // block has the same owner on both sides, so it must stay unreported. + // A scope that made every impl-local move look breaking would be the + // phantom this analysis refuses to produce. + let patch = one_file_patch( + "src/model.rs", + &[ + " impl A {", + "- pub const VALUE: u32 = 1;", + " pub fn keep(&self) {}", + "+ pub const VALUE: u32 = 1;", + " }", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "a move inside one impl block is not breaking, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn an_impl_opener_the_hunk_never_showed_still_pairs() { + // The accepted unknown-scope limit, unchanged. When the diff does not + // show the owner, both sides carry the empty path and the re-add pairs — + // the same tolerance an unseen `mod` opener has always had. Narrowing + // this would turn ordinary re-adds into phantom removals wherever a hunk + // starts inside a block, which is most of them. + let patch = one_file_patch( + "src/model.rs", + &[ + "- pub const VALUE: u32 = 1;", + "+ pub const VALUE: u32 = 1;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "an unseen impl opener must keep the scope unknown, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn an_impl_owner_composes_with_the_module_it_sits_in() { + // Owners and modules are one stack, so `a::impl A` and `b::impl A` are + // different sites even though the impl header is identical. Reading only + // the innermost opener would merge them. + let patch = one_file_patch( + "src/model.rs", + &[ + " pub mod a {", + " impl A {", + "- pub const VALUE: u32 = 1;", + " }", + " }", + " pub mod b {", + " impl A {", + "+ pub const VALUE: u32 = 1;", + " }", + " }", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"constant".to_string()), + "the same impl header in two modules is two sites, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_returned_impl_trait_does_not_open_a_scope() { + // `pub fn f() -> impl Iterator {` contains the keyword but + // opens a function body, not an owner. Recording it would invent a scope + // for whatever follows and split a plain re-add into a phantom removal — + // the same "do not invent a scope" rule the literal-brace guard enforces. + let patch = one_file_patch( + "src/model.rs", + &[ + " pub fn rows() -> impl Iterator {", + "- pub const VALUE: u32 = 1;", + " }", + " pub fn cols() -> impl Iterator {", + "+ pub const VALUE: u32 = 1;", + " }", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "`-> impl Trait` must not open an owner scope, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_literal_delimiter_does_not_end_a_declaration_early() { + // `pub const T: &str = r#"{` carries a `{` inside the literal it opens. + // Reading it as the declaration's body opener finalized a TRUNCATED + // declaration on both sides, so the two truncations matched exactly, + // the removal was cancelled, and the changed literal — the whole point + // of the diff — produced no finding at all. + let patch = one_file_patch( + "src/model.rs", + &[ + "-pub const TEMPLATE: &str = r#\"{", + "- \"kind\": \"old\"", + "-}\"#;", + "+pub const TEMPLATE: &str = r#\"{", + "+ \"kind\": \"new\"", + "+}\"#;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + findings + .iter() + .any(|f| matches!(&f.kind, BreakingKind::ChangedSignature { .. })), + "a changed multi-line const body must be reported, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_declaration_ending_inside_a_literal_still_completes_at_the_real_end() { + // Guard the other direction: blanking literals must not make an + // ordinary single-line declaration look unfinished and swallow the + // lines after it as continuations. + let patch = one_file_patch( + "src/model.rs", + &[ + "-pub const SEP: &str = \";\";", + "+pub const SEP: &str = \",\";", + " pub fn untouched() {}", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + let changed: Vec<_> = findings + .iter() + .filter_map(|f| match &f.kind { + BreakingKind::ChangedSignature { before, after } => Some((before, after)), + _ => None, + }) + .collect(); + assert_eq!( + changed.len(), + 1, + "exactly one const changed, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + assert_eq!(changed[0].0, "pub const SEP: &str = \";\";"); + assert_eq!(changed[0].1, "pub const SEP: &str = \",\";"); + } + + #[test] + fn a_removal_is_not_cancelled_by_a_re_add_under_a_different_cfg() { + // `#[cfg(feature = "a")] pub struct Config;` replaced by the same + // struct under feature `b` is an exact text match, so the pairing + // dropped the removal — but `Config` really did disappear for anyone + // building with feature `a`. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(feature = \"a\")]", + "-pub struct Config;", + "+#[cfg(feature = \"b\")]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "a re-add under a different cfg must not cancel the removal, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_re_add_under_the_same_cfg_still_cancels() { + // Guard against over-reach: the same guard on both sides is the no-op + // remove+re-add the pairing exists for. Spelling differences in the + // attribute are formatting, not a different predicate. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(feature = \"a\")]", + "-pub struct Config;", + "+#[cfg(feature=\"a\")]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "an identical re-add under the same cfg is not breaking, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_stack_of_cfg_attributes_is_one_guard() { + // Stacked attributes are Rust's AND. Keeping only the last one made + // these two sides compare equal on the shared `feature = "x"` alone, so + // a struct that really disappeared for Unix builds paired with its + // Windows-only re-add and left no finding. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(unix)]", + "-#[cfg(feature = \"x\")]", + "-pub struct Config;", + "+#[cfg(windows)]", + "+#[cfg(feature = \"x\")]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "a re-add under a different cfg stack must not cancel the removal, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_reordered_cfg_stack_is_the_same_guard() { + // Guard the other direction: the conjunction is commutative, so moving + // one attribute above another is formatting, not an API change. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(unix)]", + "-#[cfg(feature = \"x\")]", + "-pub struct Config;", + "+#[cfg(feature = \"x\")]", + "+#[cfg(unix)]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "reordering a cfg stack is not breaking, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_multiline_cfg_predicate_still_guards_its_declaration() { + // A predicate wrapped across lines used to record only its opener, and + // the first continuation line then cleared the guard entirely. Both + // sides read as unguarded, the identical struct text paired, and a + // struct that really disappeared for `feature = "b"` builds left no + // finding at all. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(any(", + "- feature = \"a\",", + "- feature = \"b\"", + "-))]", + "-pub struct Config;", + "+#[cfg(any(", + "+ feature = \"a\",", + "+ feature = \"c\"", + "+))]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "a re-add under a different multiline cfg must not cancel the removal, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_rewrapped_cfg_predicate_is_the_same_guard() { + // Guard the other direction: wrapping one predicate across lines is + // formatting, exactly like the spacing inside it. The accumulated + // attribute must compare equal to its single-line spelling, or every + // `rustfmt` rewrap would report a removal that never happened. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(any(feature = \"a\", feature = \"b\"))]", + "-pub struct Config;", + "+#[cfg(any(", + "+ feature = \"a\",", + "+ feature = \"b\"", + "+))]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "rewrapping a cfg predicate is not breaking, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn whitespace_inside_a_cfg_value_is_part_of_the_gate() { + // Whitespace was stripped from the WHOLE attribute text, literals + // included, so `#[cfg(api = "a b")]` and `#[cfg(api = "ab")]` normalized + // to one guard. A struct that really left builds configured with + // `--cfg 'api="a b"'` paired with its re-add under a different value and + // produced no finding: the space is part of the value the compiler + // matches on, not layout. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(api = \"a b\")]", + "-pub struct Config;", + "+#[cfg(api = \"ab\")]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "two cfg values are two gates, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_line_break_inside_a_cfg_value_is_part_of_the_gate() { + // The accumulator glued an attribute's physical lines together with + // nothing between them, so a value written across two lines collapsed + // onto the same guard as the same value written with the break removed. + // `--cfg 'api="a\nb"'` and `--cfg 'api="ab"'` are different + // configurations, so the struct that really left the first one paired + // with its re-add under the second and produced no finding. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(api = \"a", + "-b\")]", + "-pub struct Config;", + "+#[cfg(api = \"ab\")]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "a break inside a value is value, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_line_break_inside_a_raw_cfg_value_is_part_of_the_gate_too() { + // The same through the raw-string form, which reaches the accumulator by + // a different branch of the scanner. A rule that held for one spelling of + // a literal and not the other would be a coincidence, not an invariant. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(api = r#\"a", + "-b\"#)]", + "-pub struct Config;", + "+#[cfg(api = r#\"ab\"#)]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "a break inside a raw value is value too, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn indentation_inside_a_cfg_value_is_part_of_the_gate() { + // The third byte the pipeline used to eat: the caller trimmed every line + // before the tracker saw it, so a continuation line's leading whitespace + // never reached the guard even though it sits inside the value. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(api = \"a", + "- b\")]", + "-pub struct Config;", + "+#[cfg(api = \"a", + "+b\")]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "indentation inside a value is value, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_multiline_cfg_value_reemitted_unchanged_is_still_a_no_op() { + // The tolerant direction of the same rule: keeping those bytes must not + // start inventing gates. The identical attribute re-emitted across the + // identical lines is one guard on both sides. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(api = \"a", + "-b\")]", + "-pub struct Config;", + "+#[cfg(api = \"a", + "+b\")]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "an unchanged re-emission is not breaking, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn re_indenting_a_wrapped_predicate_is_still_only_layout() { + // The guard on the widening: OUTSIDE a literal the indentation of a + // continuation line is formatting. Feeding raw lines without that split + // would make every rustfmt pass over a wrapped `cfg` a different gate, + // and split ordinary re-adds into phantom removals. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(any(", + "- feature = \"a\",", + "- feature = \"b\"", + "-))]", + "-pub struct Config;", + "+#[cfg(any(", + "+ feature = \"a\",", + "+ feature = \"b\"", + "+))]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "re-indenting a wrapped predicate is not a different gate, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn spacing_outside_a_cfg_value_is_still_only_layout() { + // The guard on that: whitespace OUTSIDE the literal stays formatting. + // Keeping it would make reformatting an attribute a different gate and + // split an ordinary re-add into a phantom removal — the error direction + // that costs trust. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(api = \"a b\")]", + "-pub struct Config;", + "+#[cfg( api=\"a b\" )]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "respacing an attribute is not a different gate, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_multiline_attribute_does_not_drop_the_cfg_above_it() { + // The same wrapping applies to any attribute standing between the + // `cfg` and its item: a wrapped `#[derive(…)]` used to break the + // attribute run on its continuation line and take the guard with it. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(unix)]", + "-#[derive(", + "- Debug,", + "-)]", + "-pub struct Config;", + "+#[cfg(windows)]", + "+#[derive(", + "+ Debug,", + "+)]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "a wrapped attribute must not drop the cfg above it, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_delimiter_inside_a_multiline_raw_string_attribute_does_not_end_it() { + // The delimiter counter carried its own per-LINE literal state, so a raw + // string opened on one line was forgotten on the next: a `)` typed inside + // a multi-line `#[doc = r#"…"#]` read as syntax and balanced the + // attribute early. The remaining literal lines then looked like ordinary + // items and cleared the pending `cfg`, so both sides came out unguarded, + // the identical struct text paired, and a struct that really left one + // configuration produced no finding. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(unix)]", + "-#[doc = r#\"a", + "-) more", + "-\"#]", + "-pub struct Config;", + "+#[cfg(windows)]", + "+#[doc = r#\"a", + "+) more", + "+\"#]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "a raw string's contents are not attribute syntax, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_multiline_raw_string_attribute_ends_at_its_own_delimiter() { + // The other half: the attribute must still CLOSE once the raw string + // does. Counting a literal's delimiters as syntax leaves the attribute + // permanently open, and a stale guard standing over the rest of the hunk + // fabricates removals out of ordinary re-adds — the error direction that + // costs trust. Here the two sides carry the SAME guard, so the re-add + // must cancel the removal. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(unix)]", + "-#[doc = r#\"a", + "-) more", + "-\"#]", + "-pub struct Config;", + "+#[cfg(unix)]", + "+#[doc = r#\"a", + "+) more", + "+\"#]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !removed_symbol_types(&findings).contains(&"struct".to_string()), + "an unchanged re-add under the same guard is not a removal, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_line_continued_string_attribute_does_not_swallow_the_cfg_below_it() { + // The shape the registry actually holds: `#[must_use = "… \` continues + // its literal onto the next line, and the per-line counter read the + // literal's own closing quote as an OPENER — so the `]` after it was + // swallowed as text and the attribute never closed. Everything below it, + // the real `#[cfg(…)]` included, was absorbed as continuation, both sides + // came out unguarded, and the struct that left one configuration paired + // with its re-add under another. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[must_use = \"a \\", + "- b\"]", + "-#[cfg(unix)]", + "-pub struct Config;", + "+#[must_use = \"a \\", + "+ b\"]", + "+#[cfg(windows)]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "an attribute must end at its own bracket, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_literal_still_tells_two_cfg_predicates_apart() { + // The guard TEXT keeps literals even though the delimiter counter no + // longer sees them: `feature = "a"` and `feature = "b"` are different + // gates, and a view that dropped literal bodies would make them one and + // pair a configuration-specific removal away. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(feature = \"a\")]", + "-pub struct Config;", + "+#[cfg(feature = \"b\")]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "two feature gates are two gates, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_cfg_attr_that_applies_a_cfg_is_part_of_the_guard() { + // `#[cfg_attr(feature = "a", cfg(unix))]` gates the item exactly like a + // `#[cfg(…)]` does — it just decides, per feature, whether to apply one. + // Recognizing only the literal `#[cfg(` spelling discarded both sides' + // guards, so the identical struct text paired and the struct that really + // disappeared from Unix builds with feature `a` left no finding. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg_attr(feature = \"a\", cfg(unix))]", + "-pub struct Config;", + "+#[cfg_attr(feature = \"a\", cfg(windows))]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "a re-add under a different cfg_attr guard must not cancel the removal, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_cfg_attr_that_applies_no_cfg_is_not_a_guard() { + // Guard the other direction: `#[cfg_attr(unix, derive(Debug))]` decides + // a derive, not whether the item exists. Reading it as a gate would + // split an ordinary re-add into a phantom removal. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg_attr(unix, derive(Debug))]", + "-pub struct Config;", + "+#[cfg_attr(windows, derive(Debug))]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "a cfg_attr applying a derive is not a gate, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn an_unseen_cfg_guard_pairs_as_before() { + // The attribute may sit on a context line the hunk never re-emitted on + // one side. Unknown must pair with anything, exactly as an unseen + // module opener does — inventing a mismatch there would turn every + // ordinary re-add into a phantom removal. + let patch = one_file_patch( + "src/model.rs", + &[ + " #[cfg(feature = \"a\")]", + "-pub struct Config;", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "an unchanged guard on a context line must not split the pair, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_block_comment_between_the_guard_and_its_item_does_not_drop_the_guard() { + // `/** … */` is the block form of `///`, and it sits between a `cfg` + // and the item it guards exactly as the line form does. Reading it as a + // new item cleared BOTH sides' guards, the identical declaration text + // then paired, and a struct that really left the `a` build produced no + // finding at all. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(feature = \"a\")]", + "-/** Configuration for the a build. */", + "-pub struct Config;", + "+#[cfg(feature = \"b\")]", + "+/** Configuration for the b build. */", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "a block comment must not break the attribute run, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_multiline_block_comment_between_the_guard_and_its_item_does_not_drop_the_guard() { + // The close arrives on a later line, so tolerating the OPENER alone + // would leave the body and the `*/` line reading as new items — a fix + // that looks complete and still drops the guard. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(feature = \"a\")]", + "-/**", + "- * Configuration for the a build.", + "- */", + "-pub struct Config;", + "+#[cfg(feature = \"b\")]", + "+/**", + "+ * Configuration for the b build.", + "+ */", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "a wrapped block comment must not break the attribute run, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_block_comment_inside_a_cfg_predicate_is_not_counted_as_syntax() { + // The delimiter counter used to read `/* ) */` as a real closer, so the + // attribute balanced early, its real continuation read as a new item, + // and the guard was gone by the time the declaration arrived. + let patch = one_file_patch( + "src/model.rs", + &[ + "-#[cfg(any(/* ))) */", + "- feature = \"a\",", + "- feature = \"c\"))]", + "-pub struct Config;", + "+#[cfg(any(/* ))) */", + "+ feature = \"b\",", + "+ feature = \"c\"))]", + "+pub struct Config;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + removed_symbol_types(&findings).contains(&"struct".to_string()), + "a comment inside the predicate must not balance it early, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn same_name_in_the_same_inline_module_still_pairs() { + // Guard against the module tracker over-reaching: a remove+re-add inside + // ONE module is still the phantom-removal case it always was. + let patch = one_file_patch( + "src/model.rs", + &[ + " pub mod a {", + "- pub struct Config {", + "- pub x: u32,", + "+ pub struct Config {", + "+ pub x: u64,", + " }", + " }", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "same-module remove+re-add must stay a no-op, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn multiline_non_fn_declaration_change_is_not_swallowed() { + // Pairing compared only the opening line, so a bound change on a + // continuation line vanished behind an identical `pub struct Config<`. + let patch = one_file_patch( + "src/model.rs", + &[ + "-pub struct Config<", + "- T: Clone,", + "-> {", + "+pub struct Config<", + "+ T: Clone + Send,", + "+> {", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::ChangedSignature { before, after } + if before.contains("T: Clone,") && after.contains("T: Clone + Send,") + )), + "a changed bound on a continuation line must surface, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + assert!( + removed_symbol_types(&findings).is_empty(), + "the change must not also report a removal, got: {:?}", + removed_symbol_types(&findings) + ); + } + + #[test] + fn multiline_non_fn_declaration_reemitted_unchanged_is_not_breaking() { + // Same accumulation, opposite direction: an unchanged multi-line + // declaration re-emitted by the diff stays a no-op. + let patch = one_file_patch( + "src/model.rs", + &[ + "-pub struct Config<", + "- T: Clone,", + "-> {", + "- old_detail: u8,", + "+pub struct Config<", + "+ T: Clone,", + "+> {", + "+ new_detail: u8,", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "identical multi-line remove+re-add must produce no finding, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_long_signature_change_below_the_old_cap_is_not_swallowed() { + // Accumulation used to stop after eight continuation lines. Two long + // declarations that agree on the opener and those eight lines then + // finalized to the SAME truncated text, so the exact-match pass paired + // them, consumed the addition and dropped the removal — the changed + // return type on the tenth line produced no finding at all. + let mut body = Vec::new(); + for (side, ret) in [("-", "u8"), ("+", "u16")] { + body.push(format!("{side}pub fn build(")); + for name in ["a", "b", "c", "d", "e", "f", "g", "h"] { + body.push(format!("{side} {name}: u8,")); + } + body.push(format!("{side}) -> {ret} {{")); + } + let refs: Vec<&str> = body.iter().map(String::as_str).collect(); + let patch = one_file_patch("src/model.rs", &refs); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::ChangedSignature { before, after } + if before.contains("-> u8") && after.contains("-> u16") + )), + "a return type changed past the eighth continuation line must surface, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + assert!( + removed_symbol_types(&findings).is_empty(), + "the change must not also report a removal, got: {:?}", + removed_symbol_types(&findings) + ); + } + + #[test] + fn a_trailing_comment_does_not_swallow_the_rest_of_a_declaration() { + // Continuation lines are joined with a space, so a `//` on one of them + // commented out everything appended after it: `declaration_complete` + // never saw the closing `)` or the body `{`, the accumulator ran on + // into the body, and a body-only rewrite came out as a phantom + // signature change. + let patch = one_file_patch( + "src/model.rs", + &[ + "-pub fn build(", + "- a: u8, // how many", + "- b: u8,", + "-) -> u8 {", + "- old_body();", + "+pub fn build(", + "+ a: u8, // how many", + "+ b: u8,", + "+) -> u8 {", + "+ new_body();", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "a body-only rewrite under a commented signature is not breaking, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_real_change_below_a_trailing_comment_still_surfaces() { + // The other direction: ending the comment at its own line must not cost + // the change that follows it. + let patch = one_file_patch( + "src/model.rs", + &[ + "-pub fn build(", + "- a: u8, // how many", + "- b: u8,", + "-) -> u8 {", + "+pub fn build(", + "+ a: u8, // how many", + "+ b: u16,", + "+) -> u8 {", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::ChangedSignature { before, after } + if before.contains("b: u8") && after.contains("b: u16") + )), + "a parameter change below a trailing comment must surface, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn rewording_a_comment_inside_a_declaration_is_not_a_signature_change() { + // The Rust API here is byte-identical; only a note to the next reader + // changed. Comparing the verbatim join made that a `ChangedSignature`, + // which is a breaking-change claim about text no consumer can observe. + let patch = one_file_patch( + "src/model.rs", + &[ + "-pub fn build(", + "- a: u8, // how many", + "- b: u8,", + "-) -> u8 {", + "+pub fn build(", + "+ a: u8, // how many of them", + "+ b: u8,", + "+) -> u8 {", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "rewording a comment is not an API change, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_changed_multiline_array_constant_surfaces() { + // `[u8; 2]` states a length with a `;`, inside the TYPE. Accepting that + // `;` as the declaration's terminator finalized both sides at their + // identical opener, the exact-match pass paired them, and the changed + // values below vanished. + let patch = one_file_patch( + "src/model.rs", + &[ + "-pub const TABLE: [u8; 2] = [", + "- 1, 2,", + "-];", + "+pub const TABLE: [u8; 2] = [", + "+ 3, 4,", + "+];", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::ChangedSignature { before, after } + if before.contains("1, 2") && after.contains("3, 4") + )), + "a changed multiline array constant must surface, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_reemitted_multiline_array_constant_is_still_a_no_op() { + // The tolerant direction: reading the whole initializer must not turn a + // verbatim re-emission into a removal. + let patch = one_file_patch( + "src/model.rs", + &[ + "-pub const TABLE: [u8; 2] = [", + "- 1, 2,", + "-];", + "+pub const TABLE: [u8; 2] = [", + "+ 1, 2,", + "+];", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "an unchanged re-emission is not breaking, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_newline_inside_a_literal_is_not_the_same_value_as_a_space() { + // The identity joined physical lines with a space, INCLUDING the ones a + // literal spans. A constant written across two lines therefore compared + // equal to the same constant rewritten with a space, and the exact-match + // pass consumed the addition: a changed public value left no finding. + let patch = one_file_patch( + "src/model.rs", + &[ + "-pub const BANNER: &str = \"a", + "-b\";", + "+pub const BANNER: &str = \"a b\";", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "collapsing a literal's newline into a space changes the value, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn reflowing_a_declaration_across_lines_is_not_a_signature_change() { + // The line break is preserved because a literal may span it. Preserving + // it unconditionally made a purely cosmetic reflow — the same alias + // rewritten onto one line — read as two different declarations, and the + // pairing pass reported a signature change for an API that did not move. + let patch = one_file_patch( + "src/model.rs", + &["-pub type Alias =", "- u32;", "+pub type Alias = u32;"], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "a reflow states the same API, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn reflowing_a_declaration_onto_more_lines_is_not_a_signature_change_either() { + // The same no-op in the other direction, so the rule is not a one-way + // tolerance that merely happens to hold for the shape above. + let patch = one_file_patch( + "src/model.rs", + &["-pub type Alias = u32;", "+pub type Alias =", "+ u32;"], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "a reflow states the same API, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_blank_line_inside_a_literal_is_part_of_the_value() { + // The other half of the same rule: a line contributing no code is + // dropped from the identity, but inside a literal an empty line IS the + // value. Dropping it made a constant with a blank line compare equal to + // the same constant without one. + let patch = one_file_patch( + "src/model.rs", + &[ + "-pub const BANNER: &str = \"a", + "-", + "-b\";", + "+pub const BANNER: &str = \"a", + "+b\";", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "dropping a literal's blank line changes the value, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_multiline_literal_reemitted_unchanged_is_still_a_no_op() { + // The tolerant direction: the same constant re-emitted across the same + // physical lines must stay a no-op. + let patch = one_file_patch( + "src/model.rs", + &[ + "-pub const BANNER: &str = \"a", + "-b\";", + "+pub const BANNER: &str = \"a", + "+b\";", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "an unchanged multiline literal is not breaking, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn re_indenting_a_literal_s_continuation_line_changes_the_value() { + // The identity trimmed every continuation line unconditionally, so the + // leading whitespace of a line INSIDE a literal — which is value, not + // layout — never reached the comparison. Re-indenting the second half of + // a public constant produced two identical identities and the exact-match + // pass consumed the addition: a changed public value left no finding. + let patch = one_file_patch( + "src/model.rs", + &[ + "-pub const BANNER: &str = \"a", + "- b\";", + "+pub const BANNER: &str = \"a", + "+ b\";", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "indentation inside a literal is part of the value, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn trailing_whitespace_inside_a_literal_changes_the_value_too() { + // The other edge of the same line. Whitespace before the break belongs to + // the value whenever the literal is still open at the end of the line, + // and trimming it collapsed a constant ending in a space onto one that + // does not. + let patch = one_file_patch( + "src/model.rs", + &[ + "-pub const BANNER: &str = \"a ", + "-b\";", + "+pub const BANNER: &str = \"a", + "+b\";", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "trailing whitespace inside a literal is part of the value, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn re_indenting_a_continuation_line_outside_a_literal_is_still_a_no_op() { + // The guard on the tolerance the trimming bought: outside a literal the + // whitespace at a line edge is layout, and re-indenting a continuation + // states the same API. Feeding the raw line everywhere would have turned + // every rustfmt pass into a wall of phantom signature changes. + let patch = one_file_patch( + "src/model.rs", + &[ + "-pub type Alias =", + "- u32;", + "+pub type Alias =", + "+ u32;", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "indentation outside a literal is layout, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn the_gap_before_a_trailing_comment_is_still_not_part_of_the_identity() { + // The second half of that guard: a line whose code ends where its comment + // begins must not contribute the whitespace between them either. The + // comment is dropped from the identity, so the line ends outside a + // literal and its trailing edge is normalized like any other layout. + let patch = one_file_patch( + "src/model.rs", + &[ + "-pub fn run(", + "- a: u8, // first", + "-) {", + "+pub fn run(", + "+ a: u8,// second", + "+) {", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "a comment is not the signature, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_changed_string_literal_is_still_a_signature_change() { + // The direction the comment-free identity must NOT buy: a literal is + // code. Comparing declarations on a view that drops literal bodies would + // pair a real value change away as an unchanged re-add. + let patch = one_file_patch( + "src/model.rs", + &[ + "-pub const GREETING: &str = \"hello\";", + "+pub const GREETING: &str = \"goodbye\";", + ], + ); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::ChangedSignature { before, after } + if before.contains("hello") && after.contains("goodbye") + )), + "a changed public constant must still surface, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + + #[test] + fn a_long_declaration_reemitted_unchanged_is_still_a_no_op() { + // The tolerant direction of the same accumulation: a long signature the + // diff re-emits verbatim must stay a no-op, not become a removal. + let mut body = Vec::new(); + for side in ["-", "+"] { + body.push(format!("{side}pub fn build(")); + for name in ["a", "b", "c", "d", "e", "f", "g", "h", "i", "j"] { + body.push(format!("{side} {name}: u8,")); + } + body.push(format!("{side}) -> u8 {{")); + } + let refs: Vec<&str> = body.iter().map(String::as_str).collect(); + let patch = one_file_patch("src/model.rs", &refs); + + let findings = analyze_all_breaking_changes(&[patch]); + assert!( + !findings.iter().any(|f| matches!( + &f.kind, + BreakingKind::RemovedSymbol { .. } | BreakingKind::ChangedSignature { .. } + )), + "an unchanged long declaration must produce no finding, got: {:?}", + findings.iter().map(|f| &f.kind).collect::>() + ); + } + // ── compute_breaking_risk tests ────────────────────────────────── #[test] diff --git a/src/artifacts/signal/coverage.rs b/src/artifacts/signal/coverage.rs index 3f8cc84..8692665 100644 --- a/src/artifacts/signal/coverage.rs +++ b/src/artifacts/signal/coverage.rs @@ -32,11 +32,27 @@ impl std::fmt::Display for CoverageMatchTier { // ── Coverage data structures ───────────────────────────────────────── +/// Rendered when there was nothing to measure (no changed source files). +/// A 0/0 ratio is an absence of data, never a perfect score. +pub const COVERAGE_NOT_MEASURED: &str = "not measured"; + +/// Format an optional coverage percentage for human-facing artifacts. +/// +/// `None` means the heuristic had no changed source files to evaluate, which +/// must never be presented as 100%. +pub fn format_coverage_pct(pct: Option) -> String { + match pct { + Some(p) => format!("{}%", p), + None => COVERAGE_NOT_MEASURED.to_string(), + } +} + #[derive(Debug, Clone)] pub struct CoverageDelta { pub total_source: usize, pub covered_count: usize, - pub pct: u32, + /// `None` when `total_source == 0` — nothing was measured. + pub pct: Option, pub uncovered: Vec, pub covered: Vec, pub non_code_count: usize, @@ -116,7 +132,9 @@ pub struct CoverageSignal { pub uncovered_files: Vec, pub total_source_files: usize, pub covered_count: usize, - pub coverage_pct: u32, + /// `None` when `total_source_files == 0`: the heuristic had nothing to + /// evaluate. Consumers must render this as "not measured", not as 100%. + pub coverage_pct: Option, pub non_code_count: usize, pub has_rust_inline_tests: bool, pub rust_uncovered_count: usize, @@ -275,10 +293,13 @@ pub fn compute_coverage_signal( let total = source_files.len(); let covered_count = covered.len() + inline_tested_count; + // 0/0 is "nothing was measured", not "everything is covered". Reporting 100% + // for an empty scan is the SKIP-AS-ZERO inversion the gate already avoids + // (see build_heuristics_gate_check); coverage must speak the same language. let pct = if total > 0 { - (covered_count as f64 / total as f64 * 100.0) as u32 + Some((covered_count as f64 / total as f64 * 100.0) as u32) } else { - 100 + None }; let has_rust = source_files.iter().any(|f| f.path.ends_with(".rs")); @@ -452,8 +473,10 @@ pub fn generate_coverage_delta(dir: &Path, signal: &CoverageSignal) -> Result<() let _ = writeln!( output, - "Summary: {}/{} changed code files have matching test changes ({}%)", - signal.covered_count, signal.total_source_files, pct + "Summary: {}/{} changed code files have matching test changes ({})", + signal.covered_count, + signal.total_source_files, + format_coverage_pct(pct) ); if signal.inline_tested_count > 0 { let _ = writeln!( @@ -820,7 +843,7 @@ mod tests { let signal = compute_coverage_signal(&[diff], None, Some(&repo)); assert_eq!(signal.covered_count, 1); - assert_eq!(signal.coverage_pct, 100); + assert_eq!(signal.coverage_pct, Some(100)); assert!(signal.uncovered_files.is_empty()); assert_eq!( signal.covered_files[0], @@ -850,7 +873,7 @@ mod tests { let signal = compute_coverage_signal(&[diff], None, Some(&repo)); assert_eq!(signal.uncovered_files, vec!["src/lib.rs"]); - assert!(signal.coverage_pct < 100); + assert!(signal.coverage_pct.is_some_and(|p| p < 100)); } #[test] @@ -900,7 +923,7 @@ mod tests { assert_eq!(signal.total_source_files, 1); assert_eq!(signal.covered_count, 1); assert_eq!(signal.non_code_count, 4); - assert_eq!(signal.coverage_pct, 100); + assert_eq!(signal.coverage_pct, Some(100)); } #[test] @@ -994,7 +1017,7 @@ mod tests { signal.uncovered_files.is_empty(), "inline-tested file must not be counted as uncovered" ); - assert_eq!(signal.coverage_pct, 100); + assert_eq!(signal.coverage_pct, Some(100)); assert!( signal .covered_files @@ -1029,20 +1052,56 @@ mod tests { "a cfg(test) import without #[test] must not count as inline-tested" ); assert_eq!(signal.uncovered_files, vec!["src/calc.rs"]); - assert!(signal.coverage_pct < 100); + assert!(signal.coverage_pct.is_some_and(|p| p < 100)); } #[test] - fn coverage_delta_empty() { + fn coverage_delta_empty_reports_not_measured_not_100() { let tmp = TempDir::new().unwrap(); let diff = mock_diff(vec![]); let signal = compute_coverage_signal(&[diff], None, None); + assert_eq!( + signal.coverage_pct, None, + "0/0 must be an absent measurement, not a percentage" + ); generate_coverage_delta(tmp.path(), &signal).unwrap(); let content = fs::read_to_string(tmp.path().join("coverage-delta.txt")).unwrap(); assert!(content.contains("0/0")); - assert!(content.contains("100%")); + assert!( + content.contains(COVERAGE_NOT_MEASURED), + "empty scan must be labelled '{COVERAGE_NOT_MEASURED}', got:\n{content}" + ); + assert!( + !content.contains("100%"), + "empty scan must never claim 100% coverage, got:\n{content}" + ); + } + + #[test] + fn coverage_zero_of_n_is_a_real_zero_percent_measurement() { + // 0/N (N > 0) IS a measurement: source changed, no test changed. + // It must stay 0%, not degrade into "not measured". + let diff = mock_diff(vec![ + mock_file_change("src/lib.rs", FileStatus::Modified, 10, 5), + mock_file_change("src/utils.rs", FileStatus::Added, 20, 0), + ]); + + let signal = compute_coverage_signal(&[diff], None, None); + assert_eq!(signal.total_source_files, 2); + assert_eq!(signal.covered_count, 0); + assert_eq!(signal.coverage_pct, Some(0)); + + let tmp = TempDir::new().unwrap(); + generate_coverage_delta(tmp.path(), &signal).unwrap(); + let content = fs::read_to_string(tmp.path().join("coverage-delta.txt")).unwrap(); + assert!(content.contains("0/2")); + assert!( + content.contains("(0%)"), + "0/N must render as a real 0%, got:\n{content}" + ); + assert!(!content.contains(COVERAGE_NOT_MEASURED)); } #[test] @@ -1071,9 +1130,9 @@ mod tests { text ); assert!( - text.contains(&format!("{}%", delta.pct)), - "Text artifact must show {}% but got:\n{}", - delta.pct, + text.contains(&format_coverage_pct(delta.pct)), + "Text artifact must show {} but got:\n{}", + format_coverage_pct(delta.pct), text ); assert!( diff --git a/src/artifacts/signal/patterns.rs b/src/artifacts/signal/patterns.rs index 1d136c4..6ec4447 100644 --- a/src/artifacts/signal/patterns.rs +++ b/src/artifacts/signal/patterns.rs @@ -35,10 +35,94 @@ fn is_cli_entry_point(path: &str) -> bool { ) || norm.ends_with("/src/main.rs") } +/// True if `c` can be part of an identifier in one of the scanned languages. +/// +/// Deliberately the UNION across languages rather than ASCII only: `$` forms +/// identifiers in JavaScript/TypeScript (and is the metavariable sigil in Rust +/// macros), and every scanned language admits non-ASCII letters and digits. +/// Reading either as a word boundary reported `const $TODO = false` and an +/// identifier abutting a Unicode letter as TODO markers — inflating `prod_hits` +/// and the risk score with the very false positives bounded matching exists to +/// exclude. +/// +/// Known and deliberate boundary: `char::is_alphanumeric` is the Unicode +/// Alphabetic + Numeric property, which excludes most combining marks (`U+0301` +/// and friends), while every scanned language admits them as identifier +/// continuations. `TODO` followed by a bare combining mark therefore still reads +/// as a standalone marker. Closing that needs an `XID_Continue` table — a new +/// dependency or a hand-rolled range set — and the case is not observable: a +/// 33.4M-line sample (crates.io, npm, site-packages) holds 181k combining marks +/// and NOT ONE of them adjacent to a `TODO`/`FIXME`/`HACK`/`XXX` occurrence. +/// The residual error also points at reporting a false marker, not at hiding a +/// real one. +fn is_word_char(c: char) -> bool { + c.is_alphanumeric() || c == '_' || c == '$' +} + +/// Match `needle` inside `haystack` respecting word boundaries. +/// +/// EACH side is checked only where the needle itself has an identifier edge, +/// because that is the only side on which a longer identifier can swallow it: +/// +/// - `"todo!("` ends in `(`, so it is already right-bounded and no trailing +/// check applies — but it STARTS with `t`, so `mytodo!(…)` must not match. +/// - `".unwrap()"` starts with `.`, so no leading check applies — requiring one +/// would reject `value.unwrap()`, which is every real occurrence. +/// - `"TODO"` has identifier edges on both sides, so both are checked. +/// +/// Deriving the rule from the needle rather than from a "is it a plain word" +/// test is what lets a punctuated needle be bounded on its one identifier side: +/// gating the whole helper on plain words left `todo!(`, `dbg!(` and +/// `println!(` on raw substring matching, so `mytodo!(…)` was reported as a +/// TODO marker. +/// +/// Prevents substring false positives like `XXX` matching inside +/// `mktemp fooXXXXXX`, or `TODO` matching inside `TODOS`/`todos_list`. +fn contains_word_bounded(haystack: &str, needle: &str) -> bool { + if needle.is_empty() { + return false; + } + let needs_left_boundary = needle.chars().next().is_some_and(is_word_char); + let needs_right_boundary = needle.chars().next_back().is_some_and(is_word_char); + + let mut search_from = 0; + while let Some(rel) = haystack[search_from..].find(needle) { + let start = search_from + rel; + let end = start + needle.len(); + + // Boundaries are read per CHARACTER, not per byte: the byte before a + // non-ASCII letter is a UTF-8 continuation byte, which is not + // alphanumeric and used to read as a boundary. + let left_ok = !needs_left_boundary + || !haystack[..start] + .chars() + .next_back() + .is_some_and(is_word_char); + let right_ok = + !needs_right_boundary || !haystack[end..].chars().next().is_some_and(is_word_char); + + if left_ok && right_ok { + return true; + } + search_from = start + 1; + } + false +} + /// Patterns to scan for in added lines of patches. +/// +/// Every needle is matched with [`contains_word_bounded`], which bounds each +/// side only where the needle has an identifier edge. The `eprint` family is +/// listed explicitly because it used to be caught by accident: `eprintln!(` +/// CONTAINS `println!(`, so raw substring matching reported it while also +/// reporting `myprintln!(`. Bounding the needle ends both, and the family it +/// was catching for real is named here instead. const SCAN_PATTERNS: &[(&str, &[&str])] = &[ ("unwrap", &[".unwrap()"]), - ("println", &["println!(", "print!("]), + ( + "println", + &["println!(", "print!(", "eprintln!(", "eprint!("], + ), ("dbg", &["dbg!("]), ("todo", &["todo!(", "TODO", "FIXME", "HACK", "XXX"]), ( @@ -226,7 +310,8 @@ pub fn generate_pattern_scan(dir: &Path, diffs: &[Diff], repo: &Repository) -> R let is_test_code = file_is_test || test_lines.contains(&line_no); for &(pattern_name, needles) in SCAN_PATTERNS { - if needles.iter().any(|n| content.contains(n)) { + let matched = needles.iter().any(|n| contains_word_bounded(content, n)); + if matched { // `println`/`print` hits in a CLI entry point are // intended output, not a debug leftover (P2-05). let cli_intended = @@ -833,4 +918,245 @@ mod tests { assert!(!set.contains(&3), "closing brace of prod fn"); assert!(set.contains(&7), "unwrap inside test module"); } + + // ── word-boundary pattern matching (PRV-TODO-XXX / PRV-PATTERN-SUBSTRING) ── + + fn scan_todo_pattern(new_content: &str) -> Option { + let (_tmp, repo, base_id, target_id) = + make_test_repo(&[("src/lib.rs", "fn prod() {}\n", new_content)]); + let out = TempDir::new().unwrap(); + let diff = make_diff_with_ids( + base_id, + target_id, + vec![mock_file_change("src/lib.rs", FileStatus::Modified, 1, 0)], + ); + + generate_pattern_scan(out.path(), &[diff], &repo).unwrap(); + + let path = out.path().join("PATTERN_SCAN.json"); + if !path.exists() { + return None; + } + let parsed: serde_json::Value = + serde_json::from_str(&fs::read_to_string(&path).unwrap()).unwrap(); + Some(parsed) + } + + fn pattern_present(scan: &Option, pattern: &str) -> bool { + scan.as_ref() + .and_then(|v| v["by_pattern"].as_array()) + .map(|arr| arr.iter().any(|e| e["pattern"] == pattern)) + .unwrap_or(false) + } + + fn todo_pattern_present(scan: &Option) -> bool { + pattern_present(scan, "todo") + } + + #[test] + fn pattern_scan_mktemp_template_is_not_todo() { + // XXX inside a mktemp-style template (fooXXXXXX) is a template + // placeholder, not a TODO marker — substring match falsely caught it. + let new = "fn run() {\n let path = \"/tmp/fooXXXXXX\";\n}\n"; + let scan = scan_todo_pattern(new); + assert!( + !todo_pattern_present(&scan), + "mktemp template XXXXXX must not be classified as todo pattern" + ); + } + + #[test] + fn pattern_scan_standalone_xxx_is_todo() { + // A standalone XXX (word-bounded on both sides) is a genuine TODO marker. + let new = "fn run() {\n // XXX this needs a real fix\n}\n"; + let scan = scan_todo_pattern(new); + assert!( + todo_pattern_present(&scan), + "standalone XXX marker must be classified as todo pattern" + ); + } + + #[test] + fn pattern_scan_todos_identifier_is_not_todo() { + // TODOS / todos_list are identifiers containing TODO as a substring, + // not the TODO marker itself. + let new = "fn run() {\n let TODOS = 1;\n let todos_list = vec![];\n}\n"; + let scan = scan_todo_pattern(new); + assert!( + !todo_pattern_present(&scan), + "TODOS/todos_list identifiers must not be classified as todo pattern" + ); + } + + #[test] + fn pattern_scan_todo_colon_is_todo() { + // `TODO:` is the classic marker form and must still be caught. + let new = "fn run() {\n // TODO: fix this\n}\n"; + let scan = scan_todo_pattern(new); + assert!( + todo_pattern_present(&scan), + "TODO: marker must be classified as todo pattern" + ); + } + + #[test] + fn pattern_scan_todo_macro_is_todo() { + // `todo!(` is the Rust macro invocation and must still be caught. + let new = "fn run() {\n todo!(\"not implemented\");\n}\n"; + let scan = scan_todo_pattern(new); + assert!( + todo_pattern_present(&scan), + "todo!( macro must be classified as todo pattern" + ); + } + + #[test] + fn pattern_scan_prefixed_todo_macro_is_not_todo() { + // The scanner reached `contains_word_bounded` only for needles made + // entirely of identifier characters, so `todo!(` kept plain substring + // matching and `mytodo!(…)` was reported as a TODO marker — the very + // substring false positive bounded matching exists to exclude. + let new = "fn run() {\n mytodo!(\"custom macro\");\n}\n"; + let scan = scan_todo_pattern(new); + assert!( + !todo_pattern_present(&scan), + "a macro whose name merely ends in todo! must not be a TODO marker" + ); + } + + #[test] + fn pattern_scan_prefixed_debug_macros_are_not_hits() { + // Same class, same needles list: `dbg!(` and `println!(` start with an + // identifier character, so a longer macro name ending in one of them + // used to be reported as a debug leftover. + let new = "fn run() {\n mydbg!(1);\n myprintln!(\"x\");\n}\n"; + let scan = scan_todo_pattern(new); + assert!( + !pattern_present(&scan, "dbg"), + "mydbg! must not be reported as a dbg leftover" + ); + assert!( + !pattern_present(&scan, "println"), + "myprintln! must not be reported as a println leftover" + ); + } + + #[test] + fn pattern_scan_still_flags_the_eprint_family() { + // Guard the coverage that used to ride on substring matching: + // `eprintln!(` contains `println!(`, so it was reported only by + // accident. Word-bounding the needle ends that accident, and the + // family is listed explicitly instead. + let new = "fn run() {\n eprintln!(\"x\");\n}\n"; + assert!( + pattern_present(&scan_todo_pattern(new), "println"), + "eprintln! must still be reported as debug output" + ); + + let new = "fn run() {\n eprint!(\"x\");\n}\n"; + assert!( + pattern_present(&scan_todo_pattern(new), "println"), + "eprint! must still be reported as debug output" + ); + } + + #[test] + fn word_boundaries_are_derived_from_each_needle_edge() { + // A needle is bounded on the side where it has an identifier edge, and + // only there — that is the only side a longer identifier can swallow it + // from. Asking for a boundary on the other side rejects every real + // occurrence instead. + assert!(contains_word_bounded("value.unwrap()", ".unwrap()")); + assert!(contains_word_bounded("a.b().unwrap();", ".unwrap()")); + assert!(!contains_word_bounded("mytodo!(\"x\")", "todo!(")); + assert!(contains_word_bounded("crate::todo!(\"x\")", "todo!(")); + assert!(!contains_word_bounded("", "TODO")); + assert!(!contains_word_bounded("TODO", "")); + } + + #[test] + fn every_scan_needle_keeps_its_real_occurrences() { + // The whole needle table, not just the reported three: bounding must + // not cost a single genuine hit. `eslint-disable-next-line` is the + // interesting one — `-` is not an identifier character, so the longer + // directive still matches the shorter needle. + for (haystack, needle) in [ + ("value.unwrap()", ".unwrap()"), + (" println!(\"x\");", "println!("), + (" eprintln!(\"x\");", "eprintln!("), + (" dbg!(x);", "dbg!("), + (" todo!(\"x\");", "todo!("), + ("// TODO: fix", "TODO"), + ("// @ts-ignore", "@ts-ignore"), + ("// eslint-disable-next-line no-console", "eslint-disable"), + ("window.console.log(x)", "console.log("), + ("} catch {", "catch {"), + ("promise.catch(() => {})", ".catch(() =>"), + ("pub unsafe fn raw() {}", "unsafe fn"), + (" unsafe { ptr.read() }", "unsafe {"), + ("#[allow(dead_code)]", "#[allow("), + ("#![allow(dead_code)]", "#![allow("), + ("const x = y as unknown as Z;", "as unknown as"), + ("const x = y as any;", "as any"), + ] { + assert!( + contains_word_bounded(haystack, needle), + "needle {needle:?} must still match {haystack:?}" + ); + } + } + + #[test] + fn every_identifier_edged_needle_rejects_a_longer_identifier() { + // The other direction, for every needle that starts or ends inside an + // identifier: a longer name containing it is not a hit. + for (haystack, needle) in [ + ("myprintln!(\"x\")", "println!("), + ("myprint!(\"x\")", "print!("), + ("mydbg!(1)", "dbg!("), + ("mytodo!(\"x\")", "todo!("), + ("let TODOS = 1;", "TODO"), + ("let myconsole.log(x)", "console.log("), + ("fn trycatch {", "catch {"), + ("myunsafe { }", "unsafe {"), + ("const x = y as anything;", "as any"), + ] { + assert!( + !contains_word_bounded(haystack, needle), + "needle {needle:?} must not match {haystack:?}" + ); + } + } + + #[test] + fn contains_word_bounded_requires_boundaries_on_alnum_needle() { + assert!(!contains_word_bounded("fooXXXXXX", "XXX")); + assert!(contains_word_bounded("// XXX fixme", "XXX")); + assert!(!contains_word_bounded("TODOS", "TODO")); + assert!(contains_word_bounded("TODO: fix", "TODO")); + } + + #[test] + fn contains_word_bounded_respects_dollar_and_unicode_identifiers() { + // `$` is an identifier character in JavaScript/TypeScript and a macro + // metavariable sigil in Rust; a non-ASCII letter is an identifier + // character in every language scanned here. Treating either as a word + // boundary reported a plain identifier as a TODO marker and inflated + // `prod_hits` — the opposite of what bounded matching is for. + assert!(!contains_word_bounded("const $TODO = false", "TODO")); + assert!(!contains_word_bounded("const TODO$ = false", "TODO")); + assert!(!contains_word_bounded("let żTODO = 1;", "TODO")); + assert!(!contains_word_bounded("let TODOż = 1;", "TODO")); + // A real marker next to non-identifier punctuation still matches. + assert!(contains_word_bounded("// TODO: fix ż", "TODO")); + assert!(contains_word_bounded("${TODO}", "TODO")); + } + + #[test] + fn contains_word_bounded_needle_ending_in_punctuation_skips_right_check() { + // "todo!(" already ends in punctuation, so no trailing-boundary + // check is required — only the left side must be word-bounded. + assert!(contains_word_bounded("todo!(\"x\")", "todo!(")); + assert!(!contains_word_bounded("mytodo!(\"x\")", "todo!(")); + } } diff --git a/src/artifacts/signal/risk.rs b/src/artifacts/signal/risk.rs index 6d00c4d..4734bba 100644 --- a/src/artifacts/signal/risk.rs +++ b/src/artifacts/signal/risk.rs @@ -328,7 +328,7 @@ mod tests { fn empty_coverage() -> CoverageDelta { CoverageDelta { - pct: 100, + pct: None, total_source: 0, covered_count: 0, uncovered: vec![], diff --git a/src/artifacts/tests.rs b/src/artifacts/tests.rs index 665ec78..47d7f78 100644 --- a/src/artifacts/tests.rs +++ b/src/artifacts/tests.rs @@ -440,7 +440,7 @@ fn merge_gate_blocks_failed_cargo_audit_in_warn_mode_when_severity_is_block() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -520,7 +520,7 @@ fn merge_gate_executed_cargo_check_carries_real_evidence_and_log() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -1152,7 +1152,7 @@ fn merge_gate_marks_heuristics_disabled_as_not_run() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -1202,7 +1202,7 @@ fn merge_gate_files_field_omits_inline_findings_path_when_no_findings() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -1259,7 +1259,7 @@ fn merge_gate_includes_inline_findings_path_when_sarif_exists() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -1327,7 +1327,7 @@ fn merge_gate_surfaces_review_caveats_when_merge_needs_review() { let coverage = CoverageDelta { total_source: 4, covered_count: 1, - pct: 25, + pct: Some(25), uncovered: vec![crate::artifacts::signal::CoverageFile { status: 'M', path: "src/lib.rs".to_string(), @@ -1387,7 +1387,7 @@ fn build_review_caveats_include_orphaned_test_candidates() { let coverage = CoverageDelta { total_source: 2, covered_count: 2, - pct: 100, + pct: Some(100), uncovered: vec![], covered: vec![], non_code_count: 0, @@ -1442,7 +1442,7 @@ fn merge_gate_splits_introduced_and_preexisting_inline_findings() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -1535,7 +1535,7 @@ fn merge_gate_splits_preexisting_quality_failures_from_inline_findings() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -1638,7 +1638,7 @@ fn merge_gate_reason_mentions_preexisting_failures_under_merge_with_review() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -1716,7 +1716,7 @@ fn merge_gate_marks_skipped_rust_quality_signals_as_review_caveats() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -1783,7 +1783,7 @@ fn merge_gate_surfaces_skipped_cargo_geiger_when_security_was_requested() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -1855,7 +1855,7 @@ fn merge_gate_surfaces_runtime_skipped_cargo_geiger() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -1888,10 +1888,14 @@ fn merge_gate_surfaces_runtime_skipped_cargo_geiger() { #[test] fn merge_gate_surfaces_cargo_audit_informational_warnings_as_review_caveat() { + // The status here used to be `Passed`, which was a fiction: a cargo-audit run + // carrying an unmaintained-crate advisory reports `Warnings`. The injected + // `Passed` kept the check out of the quality summary entirely and therefore + // masked the warning→failure bug this test now also guards. let config = create_test_config(PolicyConfig::default()); let checks = vec![CheckResult { name: "Cargo audit".to_string(), - status: CheckStatus::Passed, + status: CheckStatus::Warnings, duration: Duration::from_secs(1), output: r#"{"vulnerabilities":{"found":false,"count":0,"list":[]},"warnings":{"unmaintained":[{"kind":"unmaintained","package":{"name":"paste","version":"1.0.15"},"advisory":{"id":"RUSTSEC-2024-0436"}}]}}"#.to_string(), cached: false, @@ -1920,7 +1924,7 @@ fn merge_gate_surfaces_cargo_audit_informational_warnings_as_review_caveat() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -1948,6 +1952,134 @@ fn merge_gate_surfaces_cargo_audit_informational_warnings_as_review_caveat() { .as_str() .is_some_and(|text| text.contains("paste (unmaintained)")))) ); + + // A warning-level check that produced no locatable finding classifies as + // `Unclassified`, but it is a WARNING — it must not be counted as a failed + // quality check. Before the origin split this exact shape flipped + // `quality_pass` to false and printed "1 quality check failed". + assert_eq!( + gate["decision"]["quality_pass"].as_bool(), + Some(true), + "an unlocated warning is not a quality failure: {}", + gate["decision"] + ); + assert!( + !raw.contains("quality check failed") && !raw.contains("quality checks failed"), + "MERGE_GATE.json must not describe warnings as failed quality checks: {raw}" + ); + // The verdict itself is unchanged: Warnings still reach the policy engine as + // an advisory signal, so the run stays CONDITIONAL — only the label is honest. + assert_eq!( + gate["decision"]["verdict"].as_str(), + Some("CONDITIONAL"), + "warning-level advisory keeps the CONDITIONAL verdict" + ); + // …and the analysis is no longer degraded by a phantom quality failure. + assert_eq!( + gate["decision"]["analysis_status"].as_str(), + Some("complete"), + "no failed check means the analysis is complete, not degraded" + ); +} + +#[test] +fn merge_gate_names_the_origin_of_every_quality_failure_entry() { + // The origin split lives in memory only until the pack states it. Read from + // disk, `introduced_quality_failures: ["Rustfmt"]` next to `quality_pass: + // true` is a pack contradicting itself: the array claims a failure, the flag + // claims nothing failed, and nothing in the JSON explains which is right. + let config = create_test_config(PolicyConfig::default()); + let checks = vec![ + CheckResult { + name: "Rustfmt".to_string(), + status: CheckStatus::Warnings, + duration: Duration::from_secs(1), + output: "Diff in src/new.rs at line 1".to_string(), + cached: false, + provenance: None, + }, + CheckResult { + name: "Clippy".to_string(), + status: CheckStatus::Failed, + duration: Duration::from_secs(1), + output: "src/new.rs:1: error".to_string(), + cached: false, + provenance: None, + }, + ]; + let inline = InlineFindingsSummary { + status: "passed".to_string(), + findings_count: 0, + dashboard_findings: vec![], + }; + let resolved_target = ResolvedRef { + name: "main".to_string(), + commit_id: "abc1234abc1234abc1234abc1234abc1234ab".to_string(), + is_remote: false, + }; + let tmp = tempfile::tempdir().expect("tempdir"); + + generate_merge_gate_test!( + tmp.path(), + &config, + &checks, + None, + &inline, + &[], + &CoverageDelta { + total_source: 0, + covered_count: 0, + pct: None, + uncovered: vec![], + covered: vec![], + non_code_count: 0, + ghost_tests: vec![], + }, + &[], + &resolved_target, + &[], + ) + .expect("merge gate"); + + let raw = std::fs::read_to_string(tmp.path().join("MERGE_GATE.json")).expect("read gate"); + let gate: serde_json::Value = serde_json::from_str(&raw).expect("parse gate"); + + let details = gate["decision"]["quality_failure_details"] + .as_array() + .expect("details array"); + let origin_of = |name: &str| { + details + .iter() + .find(|detail| detail["name"] == name) + .unwrap_or_else(|| panic!("`{name}` missing from quality_failure_details: {details:?}")) + ["origin"] + .as_str() + .map(str::to_string) + }; + assert_eq!( + origin_of("Rustfmt"), + Some("warning".to_string()), + "a warning-level entry must say so on the wire: {gate}" + ); + assert_eq!( + origin_of("Clippy"), + Some("failure".to_string()), + "a hard failure must stay distinguishable from a warning: {gate}" + ); + + // A new readable field is a MINOR schema change; a pack that carries it and + // still claims 2.1 lies to `tools/validate_merge_gate.py` and to any reader + // deciding whether `origin` can be trusted to be present. + assert_eq!( + gate["schema_version"].as_str(), + Some(crate::gate::MERGE_GATE_SCHEMA_VERSION), + "the pack stamps the schema this build writes" + ); + assert_eq!( + crate::gate::MERGE_GATE_SCHEMA_VERSION, + "2.2", + "adding `origin` to quality_failure_details bumps the MINOR" + ); } #[test] @@ -1980,7 +2112,7 @@ fn merge_gate_does_not_fail_fast_remote_only_for_expected_rust_gaps() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -2063,7 +2195,7 @@ fn merge_gate_blocks_missing_rust_quality_signal_when_policy_sets_block() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -2128,7 +2260,7 @@ fn merge_gate_skipped_cargo_geiger_with_ignore_severity_produces_no_caveat() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -2668,7 +2800,7 @@ fn pr_review_summarizes_cargo_audit_without_dumping_json() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -2757,7 +2889,7 @@ fn pr_review_surfaces_cargo_audit_informational_warnings_when_check_passes() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -2905,6 +3037,70 @@ fn is_pathish_candidate_rejects_code_fragments() { assert!(!is_pathish_candidate("a/b c")); } +#[test] +fn ai_index_coverage_signal_says_not_measured_for_zero_of_zero() { + use crate::artifacts::signal::COVERAGE_NOT_MEASURED; + use crate::git::DiffStats; + + let tmp = tempfile::tempdir().expect("tempdir"); + let out = tmp.path(); + let config = create_test_config(PolicyConfig::default()); + let diffs = vec![Diff { + base: "main".to_string(), + target: "feat/x".to_string(), + base_commit_id: "aaa".to_string(), + target_commit_id: "bbb".to_string(), + files: vec![], + stats: DiffStats { + files_changed: 0, + additions: 0, + deletions: 0, + copied: 0, + }, + commits: vec![], + }]; + + let empty = CoverageDelta { + total_source: 0, + covered_count: 0, + pct: None, + uncovered: vec![], + covered: vec![], + non_code_count: 0, + ghost_tests: vec![], + }; + generate_ai_index(out, &config, &diffs, &[], &empty).expect("ai index"); + let index = std::fs::read_to_string(out.join("AI_INDEX.md")).expect("AI_INDEX.md"); + assert!( + index.contains(&format!( + "Coverage signal: 0/0 changed code files ({COVERAGE_NOT_MEASURED})" + )), + "0/0 must be labelled not-measured, got:\n{index}" + ); + assert!( + !index.contains("(100%)"), + "0/0 must never render as 100%, got:\n{index}" + ); + + // 0/N stays a real 0% measurement. + let measured = CoverageDelta { + total_source: 4, + covered_count: 0, + pct: Some(0), + uncovered: vec![], + covered: vec![], + non_code_count: 0, + ghost_tests: vec![], + }; + generate_ai_index(out, &config, &diffs, &[], &measured).expect("ai index"); + let index = std::fs::read_to_string(out.join("AI_INDEX.md")).expect("AI_INDEX.md"); + assert!( + index.contains("Coverage signal: 0/4 changed code files (0%)"), + "0/N must render as 0%, got:\n{index}" + ); + assert!(!index.contains(COVERAGE_NOT_MEASURED)); +} + #[test] fn generate_ai_index_writes_reading_order_and_verdict() { use crate::git::DiffStats; @@ -2939,7 +3135,7 @@ fn generate_ai_index_writes_reading_order_and_verdict() { let coverage = CoverageDelta { total_source: 0, covered_count: 0, - pct: 100, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -3120,7 +3316,7 @@ fn pr_review_counts_code_test_and_non_code_separately() { &CoverageDelta { total_source: 1, covered_count: 1, - pct: 100, + pct: Some(100), uncovered: vec![], covered: vec![], non_code_count: 0, @@ -3177,7 +3373,7 @@ fn pr_review_uses_coverage_delta_for_warning_summary() { &CoverageDelta { total_source: 4, covered_count: 1, - pct: 25, + pct: Some(25), uncovered: vec![crate::artifacts::signal::CoverageFile { status: 'M', path: "src/lib.rs".to_string(), @@ -3241,7 +3437,7 @@ fn pr_review_surfaces_quick_wins_for_rust_signal_gaps_and_cargo_audit() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -4124,6 +4320,7 @@ fn quality_failure_summary_has_new_failures_with_introduced() { &mut summary, "ESLint".to_string(), QualityFailureClass::Introduced, + QualityFailureOrigin::Failure, ); assert!(summary.has_new_failures()); } @@ -4135,6 +4332,7 @@ fn quality_failure_summary_has_new_failures_with_mixed() { &mut summary, "ESLint".to_string(), QualityFailureClass::Mixed, + QualityFailureOrigin::Failure, ); assert!(summary.has_new_failures()); } @@ -4146,6 +4344,7 @@ fn quality_failure_summary_has_new_failures_with_unclassified() { &mut summary, "cargo test".to_string(), QualityFailureClass::Unclassified, + QualityFailureOrigin::Failure, ); assert!(summary.has_new_failures()); } @@ -4157,11 +4356,13 @@ fn quality_failure_summary_no_new_failures_when_only_preexisting() { &mut summary, "Cargo audit".to_string(), QualityFailureClass::Preexisting, + QualityFailureOrigin::Failure, ); push_quality_failure( &mut summary, "ESLint".to_string(), QualityFailureClass::Preexisting, + QualityFailureOrigin::Failure, ); assert!(!summary.has_new_failures()); // quality_failures still lists them (backward compat) @@ -4249,7 +4450,7 @@ fn preexisting_failures_do_not_block_gate() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -4356,7 +4557,7 @@ fn introduced_failures_still_block_gate() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, @@ -4476,7 +4677,7 @@ fn mixed_failures_include_both_preexisting_and_introduced_in_output() { &CoverageDelta { total_source: 0, covered_count: 0, - pct: 0, + pct: None, uncovered: vec![], covered: vec![], non_code_count: 0, diff --git a/src/artifacts/verdict.rs b/src/artifacts/verdict.rs index f63c9e5..863a328 100644 --- a/src/artifacts/verdict.rs +++ b/src/artifacts/verdict.rs @@ -42,10 +42,42 @@ impl QualityFailureClass { } } +/// Which check status produced a quality-summary entry. +/// +/// The summary deliberately mixes two kinds of signal: hard failures +/// (`Failed`/`Error`) and warning-level baseline signals (`Warnings`) that are +/// admitted so the pre-existing downgrade can be computed for them. Only the +/// first kind may fail the quality gate — a warning is an advisory signal by +/// definition, and calling it a failure was the "warning→failure" lie. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum QualityFailureOrigin { + /// The check reported `Failed` or `Error`. + Failure, + /// The check reported `Warnings`. + Warning, +} + +impl QualityFailureOrigin { + /// Wire name used in `MERGE_GATE.json`. + /// + /// The origin is not an internal detail: without it a consumer reading + /// `introduced_quality_failures: ["Rustfmt"]` next to `quality_pass: true` + /// sees a self-contradicting pack, because the array says "failure" and the + /// flag says the entry never gated. Naming the origin is what makes the two + /// readable together. + pub fn as_str(self) -> &'static str { + match self { + Self::Failure => "failure", + Self::Warning => "warning", + } + } +} + #[derive(Debug, Clone)] pub(crate) struct QualityFailureDetail { pub name: String, pub classification: QualityFailureClass, + pub origin: QualityFailureOrigin, } #[derive(Debug, Clone, Default)] @@ -59,16 +91,29 @@ pub(crate) struct QualityFailureSummary { } impl QualityFailureSummary { - /// Returns true when there are failures that are new or indeterminate. + /// Returns true when there are FAILURES that are new or indeterminate. /// - /// Purely pre-existing failures do NOT count — they existed before this - /// diff and should not block the gate. Introduced, mixed, and - /// unclassified failures are all considered "new" because they either - /// definitely or possibly originate from the current change. + /// Two independent filters apply, and both are load-bearing: + /// + /// * **Origin.** Only entries whose check actually failed + /// (`QualityFailureOrigin::Failure`) can fail the quality gate. Entries + /// admitted from `Warnings` checks are here purely so the pre-existing + /// downgrade can be computed for them; a warning is advisory by + /// definition and must never be reported as a failed quality check — + /// regardless of how it classifies, `Unclassified` included. It still + /// reaches the verdict through the policy engine (Warnings → Advisory → + /// ReviewRequired), which keeps a CONDITIONAL verdict; what changes is + /// the truth of the label, not the verdict. + /// * **Classification.** Purely pre-existing failures do NOT count — they + /// existed before this diff and should not block the gate. Introduced, + /// mixed, and unclassified failures are all considered "new" because they + /// either definitely or possibly originate from the current change + /// (fail-closed). pub(crate) fn has_new_failures(&self) -> bool { - !self.introduced_quality_failures.is_empty() - || !self.mixed_quality_failures.is_empty() - || !self.unclassified_quality_failures.is_empty() + self.details.iter().any(|detail| { + detail.origin == QualityFailureOrigin::Failure + && !matches!(detail.classification, QualityFailureClass::Preexisting) + }) } } @@ -312,8 +357,10 @@ pub(crate) fn build_review_caveats( caveats.push(breaking_breakdown.summary_parts().join(" · ")); } - if coverage.total_source > 0 && coverage.pct < 80 { - let mut coverage_caveat = format!("{}% coverage heuristic", coverage.pct); + if let Some(pct) = coverage.pct + && pct < 80 + { + let mut coverage_caveat = format!("{}% coverage heuristic", pct); if coverage_has_rust_inline_test_blind_spot(coverage) { coverage_caveat.push_str(" (Rust inline #[cfg(test)] modules may be missed)"); } @@ -370,10 +417,15 @@ pub(crate) fn build_merge_decision_view( blocking_issues: &[String], review_caveats: Vec, ) -> MergeDecisionView { - let has_new_quality_failures = quality_failure_details - .iter() - .any(|detail| !matches!(detail.classification, QualityFailureClass::Preexisting)) - || (quality_failure_details.is_empty() && !quality_failures.is_empty()); + // Mirrors `QualityFailureSummary::has_new_failures`: only a real failure + // (not a warning-level signal) that is new or indeterminate holds the merge. + // Keeping the two predicates aligned is what stops the hero label from + // reading HOLD while `quality_pass` is true. + let has_new_quality_failures = quality_failure_details.iter().any(|detail| { + detail.origin == QualityFailureOrigin::Failure + && !matches!(detail.classification, QualityFailureClass::Preexisting) + }) || (quality_failure_details.is_empty() + && !quality_failures.is_empty()); let state = if !policy_allow_merge { MergeDecisionState::Block @@ -1233,11 +1285,13 @@ pub(crate) fn push_quality_failure( summary: &mut QualityFailureSummary, name: String, classification: QualityFailureClass, + origin: QualityFailureOrigin, ) { summary.quality_failures.push(name.clone()); summary.details.push(QualityFailureDetail { name: name.clone(), classification, + origin, }); match classification { @@ -1267,7 +1321,15 @@ pub(crate) fn build_quality_failure_summary( dashboard_findings, clean_comparison.applies_to(&check_id), ); - push_quality_failure(&mut summary, check.name.clone(), classification); + // The origin is recorded alongside the classification: warning-level + // entries take part in the pre-existing downgrade (that is why they are + // admitted at all) but never fail the gate — see `has_new_failures`. + let origin = if check.is_failure() { + QualityFailureOrigin::Failure + } else { + QualityFailureOrigin::Warning + }; + push_quality_failure(&mut summary, check.name.clone(), classification, origin); } summary @@ -1280,8 +1342,17 @@ pub(crate) fn build_quality_failure_summary( /// deltas surfaces as `Warnings`, and when every reported location lies outside /// the diff it is purely pre-existing debt that should get the same /// preexisting-only downgrade as a failure — otherwise the verdict stays -/// CONDITIONAL instead of PASS-with-caveat. An in-diff warning is classified -/// `Introduced` and keeps its review weight (no downgrade). +/// CONDITIONAL instead of PASS-with-caveat. +/// +/// Eligibility is NOT the same thing as gating (R2-13 re-adjudicated). Entering +/// the summary is what lets a warning be classified and downgraded; it never +/// makes the warning a failure. The origin recorded in +/// [`QualityFailureDetail::origin`] keeps every `Warnings` entry out of +/// [`QualityFailureSummary::has_new_failures`], whatever it classifies as — so +/// an in-diff warning is still reported as `Introduced` and keeps its review +/// weight through the policy engine, and a warning that produced no locatable +/// finding at all (`Unclassified`) no longer counterfeits "N quality checks +/// failed" and no longer flips `quality_pass` to false. fn quality_downgrade_eligible(check: &CheckResult) -> bool { check.is_failure() || (matches!(check.status, crate::checks::CheckStatus::Warnings) @@ -1300,12 +1371,61 @@ pub(crate) fn quality_failure_reason_text( return None; } + // Failures and warnings get SEPARATE sentences: only a check that actually + // failed may be described with the word "failed". A warning-level baseline + // signal is reported as what it is — a warning signal — so the gate text can + // no longer manufacture "N quality checks failed" out of advisory output. + let mut sentences = Vec::new(); + if let Some(breakdown) = classification_breakdown(quality_failure_details, |detail| { + detail.origin == QualityFailureOrigin::Failure + }) { + sentences.push(format!( + "{} quality check{} failed ({})", + breakdown.count, + if breakdown.count == 1 { "" } else { "s" }, + breakdown.parts.join(", ") + )); + } + if let Some(breakdown) = classification_breakdown(quality_failure_details, |detail| { + detail.origin == QualityFailureOrigin::Warning + }) { + sentences.push(format!( + "{} warning signal{}: {}", + breakdown.count, + if breakdown.count == 1 { "" } else { "s" }, + breakdown.parts.join(", ") + )); + } + + if sentences.is_empty() { + return None; + } + + Some(sentences.join("; ")) +} + +struct ClassificationBreakdown { + count: usize, + parts: Vec, +} + +/// Count the selected details per classification, rendering the same +/// `N introduced, M pre-existing, …` breakdown used by both sentences. +fn classification_breakdown( + quality_failure_details: &[QualityFailureDetail], + select: impl Fn(&QualityFailureDetail) -> bool, +) -> Option { let mut introduced = 0usize; let mut preexisting = 0usize; let mut mixed = 0usize; let mut unclassified = 0usize; + let mut count = 0usize; - for detail in quality_failure_details { + for detail in quality_failure_details + .iter() + .filter(|detail| select(detail)) + { + count += 1; match detail.classification { QualityFailureClass::Introduced => introduced += 1, QualityFailureClass::Preexisting => preexisting += 1, @@ -1314,6 +1434,10 @@ pub(crate) fn quality_failure_reason_text( } } + if count == 0 { + return None; + } + let mut parts = Vec::new(); if introduced > 0 { parts.push(format!("{} introduced", introduced)); @@ -1328,16 +1452,7 @@ pub(crate) fn quality_failure_reason_text( parts.push(format!("{} unclassified", unclassified)); } - if parts.is_empty() { - return None; - } - - Some(format!( - "{} quality check{} failed ({})", - quality_failures.len(), - if quality_failures.len() == 1 { "" } else { "s" }, - parts.join(", ") - )) + Some(ClassificationBreakdown { count, parts }) } #[cfg(test)] @@ -1460,6 +1575,7 @@ mod tests { &[QualityFailureDetail { name: "clippy".to_string(), classification: QualityFailureClass::Introduced, + origin: QualityFailureOrigin::Failure, }], &[], vec!["clippy returned warnings".to_string()], @@ -1478,6 +1594,7 @@ mod tests { &[QualityFailureDetail { name: "Semgrep scan".to_string(), classification: QualityFailureClass::Preexisting, + origin: QualityFailureOrigin::Failure, }], &[], vec!["Pre-existing quality failures (not from this diff): Semgrep scan".to_string()], @@ -1908,7 +2025,133 @@ mod tests { ); assert!(summary.preexisting_quality_failures.is_empty()); assert_eq!(summary.unclassified_quality_failures, vec!["Rustfmt"]); + // Deliberately updated with the origin split (re-adjudicates R2-13). + // What R5-21 protects is the CLASSIFICATION: a changed rustfmt config + // means the out-of-diff rows cannot be proven pre-existing, so they stay + // Unclassified and keep their review weight through the policy engine + // (Warnings → Advisory → ReviewRequired → CONDITIONAL). It never + // protected calling a formatter warning a *failed quality check* — + // Rustfmt reported `Warnings`, not `Failed`. The suppression is intact + // above; only the failure claim is gone. + assert!( + !summary.has_new_failures(), + "an unclassified WARNING is still not a failed quality check" + ); + + // The same shape from a check that genuinely failed still gates. + let failed_summary = build_quality_failure_summary( + &[failed_check("Rustfmt")], + &findings, + &CleanComparison::for_test_config_changed(&["rustfmt"]), + ); + assert!( + failed_summary.has_new_failures(), + "a failed check with the same unclassified rows still fails the gate" + ); + } + + #[test] + fn unlocated_warning_is_not_a_failed_quality_check() { + // P0 "warning→failure": a baseline-signal check that reports `Warnings` + // without producing a single locatable finding (cargo audit raising an + // unmaintained-crate advisory) classifies as Unclassified — there is + // nothing to place inside or outside the diff. It must NOT make + // `quality_pass` false, and the gate text must not say "failed". + let summary = build_quality_failure_summary( + &[warning_check("Cargo audit")], + &[], + &CleanComparison::for_test(true, true), + ); + assert_eq!(summary.unclassified_quality_failures, vec!["Cargo audit"]); + assert!( + !summary.has_new_failures(), + "a warning that produced no finding is not a new failure" + ); + + let reason = quality_failure_reason_text(&summary.quality_failures, &summary.details) + .expect("warning signals are still described"); + assert!( + !reason.contains("failed"), + "warning-only reason must not use the word 'failed': {reason}" + ); + assert_eq!(reason, "1 warning signal: 1 unclassified"); + } + + #[test] + fn unlocated_failure_still_fails_the_gate() { + // Fail-closed control for the test above: the SAME unlocated shape from + // a check that actually failed keeps gating. + let summary = build_quality_failure_summary( + &[failed_check("Cargo audit")], + &[], + &CleanComparison::for_test(true, true), + ); + assert!(summary.has_new_failures()); + assert_eq!( + quality_failure_reason_text(&summary.quality_failures, &summary.details).as_deref(), + Some("1 quality check failed (1 unclassified)") + ); + } + + #[test] + fn in_diff_warning_is_introduced_but_still_not_a_failure() { + // An introduced warning keeps its classification (and its review weight + // through the policy engine) but is still not a failed quality check. + let findings = [in_diff_finding("rustfmt")]; + let summary = build_quality_failure_summary( + &[warning_check("Rustfmt")], + &findings, + &CleanComparison::for_test(true, true), + ); + assert_eq!(summary.introduced_quality_failures, vec!["Rustfmt"]); + assert!(!summary.has_new_failures()); + assert_eq!( + quality_failure_reason_text(&summary.quality_failures, &summary.details).as_deref(), + Some("1 warning signal: 1 introduced") + ); + } + + #[test] + fn failure_and_warning_get_separate_sentences() { + let findings = [ + in_diff_finding("cargo_test"), + out_of_diff_finding("rustfmt"), + ]; + let summary = build_quality_failure_summary( + &[failed_check("cargo test"), warning_check("Rustfmt")], + &findings, + &CleanComparison::for_test(true, true), + ); assert!(summary.has_new_failures()); + assert_eq!( + quality_failure_reason_text(&summary.quality_failures, &summary.details).as_deref(), + Some("1 quality check failed (1 introduced); 1 warning signal: 1 pre-existing") + ); + } + + #[test] + fn warning_only_summary_is_allow_with_review_not_hold() { + // Blast radius of the origin split on the hero label: with no real + // failure left, a warnings-only run is "mergeable with advisories". + let view = build_merge_decision_view( + true, // policy_allow_merge + true, // quality_pass (no longer broken by the warning) + false, // recommended_merge — policy still says review required + &["Cargo audit".to_string()], + &[QualityFailureDetail { + name: "Cargo audit".to_string(), + classification: QualityFailureClass::Unclassified, + origin: QualityFailureOrigin::Warning, + }], + &[], + vec!["Cargo audit note: 1 informational advisory".to_string()], + ); + assert_eq!(view.state, MergeDecisionState::AllowWithReview); + assert!( + !view.reason.contains("failed"), + "decision reason must not claim a failure: {}", + view.reason + ); } #[test] diff --git a/src/checks/mod.rs b/src/checks/mod.rs index c7b62c2..a6f2c7b 100644 --- a/src/checks/mod.rs +++ b/src/checks/mod.rs @@ -357,6 +357,14 @@ impl CheckStatus { Self::Error => "error", } } + + /// Every spelling a check status can be written as in an artifact. + /// + /// `as_str` is total and this is its image, so a reader can tell "a status I + /// do not recognize" from "a status that is not a warning" — the difference + /// between counting an unreadable pack as clean and reporting it. Kept in + /// step with `as_str` by `every_emitted_status_is_in_the_vocabulary`. + pub const EMITTED: [&'static str; 5] = ["passed", "failed", "warnings", "skipped", "error"]; } /// Trait for implementing checks @@ -1382,6 +1390,39 @@ mod tests { use crate::config::{Config, test_config, test_rust_profile}; use std::time::Duration; + #[test] + fn every_emitted_status_is_in_the_vocabulary() { + // The vocabulary is what the CLI reader and the contract validator both + // measure a pack against, so it has to BE the writer's image. A variant + // added to the enum and forgotten here would be emitted into artifacts + // and then reported as unreadable by the very tool that wrote it. + let variants = [ + CheckStatus::Passed, + CheckStatus::Failed, + CheckStatus::Warnings, + CheckStatus::Skipped, + CheckStatus::Error, + ]; + for variant in variants { + assert!( + CheckStatus::EMITTED.contains(&variant.as_str()), + "{:?} is emitted but missing from the vocabulary", + variant + ); + } + for spelling in CheckStatus::EMITTED { + assert!( + variants.iter().any(|v| v.as_str() == spelling), + "{spelling} is in the vocabulary but nothing emits it" + ); + } + assert_eq!( + variants.len(), + CheckStatus::EMITTED.len(), + "one variant per spelling" + ); + } + fn rust_config(run_tests: bool, run_lint: bool, run_security: bool) -> Config { let mut config = test_config(); config.profile = test_rust_profile(true); diff --git a/src/checks/semgrep.rs b/src/checks/semgrep.rs index 86b799b..9f17f94 100644 --- a/src/checks/semgrep.rs +++ b/src/checks/semgrep.rs @@ -75,11 +75,22 @@ impl Check for SemgrepCheck { let status = classify_semgrep_status(output.status.success(), &stdout, &combined); + // A tool/config error (non-zero exit, no findings payload) is + // classified Skipped above; give it a reason-carrying output instead + // of the raw combined dump, so the gate's skip-reason plumbing + // (policy/engine.rs reads `result.output` verbatim as the reason) shows + // *why* it was skipped rather than a wall of stderr noise. + let output_text = if status == CheckStatus::Skipped && !output.status.success() { + format_tool_error_reason(output.status.code(), &stdout, &stderr) + } else { + combined.clone() + }; + Ok(CheckResult { name: self.name().to_string(), status, duration: start.elapsed(), - output: combined.clone(), + output: output_text, cached: false, provenance: Some( ProvenanceBuilder { @@ -113,11 +124,24 @@ impl Check for SemgrepCheck { /// /// Any non-empty `errors[]` (parse errors / partial parsing) downgrades a /// successful scan to `Warnings`, making degraded coverage a visible review -/// signal instead of a silent pass. A non-zero exit (real findings with -/// `--error`, or a tool crash) remains a `Failed`. +/// signal instead of a silent pass. +/// +/// A non-zero exit is ambiguous on its own: `--error` makes semgrep exit 1 +/// when it found real results, but semgrep also exits non-zero (commonly 2) +/// on a config/tool error — an invalid ruleset, a crash — where it never +/// produced a findings payload at all. Counting the latter as a code `Failed` +/// makes a broken scanner look like a regression in the PR's own code +/// (TOOL-VS-CODE). So a non-zero exit is only `Failed` when stdout carries a +/// parsable payload with at least one actual result; otherwise it is a tool +/// error and classifies `Skipped`, mirroring the missing-tool pattern already +/// used for ruff/mypy (`checks/python.rs`). fn classify_semgrep_status(command_succeeded: bool, stdout: &str, _combined: &str) -> CheckStatus { if !command_succeeded { - return CheckStatus::Failed; + return if output_has_findings_payload(stdout) { + CheckStatus::Failed + } else { + CheckStatus::Skipped + }; } if output_reports_scan_errors(stdout) { @@ -150,6 +174,106 @@ pub(crate) fn output_reports_scan_errors(output: &str) -> bool { .is_some_and(|errors| !errors.is_empty()) } +/// True when `stdout` carries a parsable JSON payload with at least one entry +/// in `results` — i.e. semgrep actually produced findings, as opposed to a +/// non-zero exit with no results at all (config/tool error). Used to +/// distinguish a genuine `--error` exit (real findings in the code, stays +/// `Failed`) from a tool/config error exit (no payload, classifies `Skipped` +/// as a tool error rather than a code failure). +fn output_has_findings_payload(stdout: &str) -> bool { + let Some(start) = stdout.find('{') else { + return false; + }; + let mut de = serde_json::Deserializer::from_str(&stdout[start..]); + let Ok(parsed) = serde_json::Value::deserialize(&mut de) else { + return false; + }; + parsed + .get("results") + .and_then(|results| results.as_array()) + .is_some_and(|results| !results.is_empty()) +} + +/// First excerpt in `candidates` that carries any text. +fn first_nonempty(candidates: [String; N]) -> String { + candidates + .into_iter() + .find(|candidate| !candidate.is_empty()) + .unwrap_or_default() +} + +/// Up to five non-empty lines of raw tool output, on one line. +fn text_excerpt(text: &str) -> String { + text.lines() + .map(str::trim) + .filter(|line| !line.is_empty()) + .take(5) + .collect::>() + .join(" | ") +} + +/// Messages from a semgrep `--json` payload's `errors[]`, if it carries any. +/// +/// Field naming varies across semgrep versions, so the first of +/// `message` / `long_msg` / `short_msg` / `type` that is present wins. +fn json_errors_excerpt(stdout: &str) -> String { + let Some(start) = stdout.find('{') else { + return String::new(); + }; + let mut de = serde_json::Deserializer::from_str(&stdout[start..]); + let Ok(parsed) = serde_json::Value::deserialize(&mut de) else { + return String::new(); + }; + let Some(errors) = parsed.get("errors").and_then(|e| e.as_array()) else { + return String::new(); + }; + + errors + .iter() + .filter_map(|error| { + ["message", "long_msg", "short_msg", "type"] + .iter() + .find_map(|field| error.get(field).and_then(|v| v.as_str())) + }) + .map(str::trim) + .filter(|message| !message.is_empty()) + .take(5) + .collect::>() + .join(" | ") +} + +/// Human-readable skip reason for a semgrep tool/config error: the exit code +/// plus a short excerpt of whatever diagnostic the run produced — stderr when +/// there is any, otherwise the stdout payload's `errors[]`, otherwise raw +/// stdout — so a reviewer (and the policy engine, which reads +/// `CheckResult.output` verbatim as the skip reason) sees why the check did not +/// run rather than a raw stdout/stderr dump or a generic sentence. +fn format_tool_error_reason(exit_code: Option, stdout: &str, stderr: &str) -> String { + // Under `--json` semgrep reports config/rule failures in the stdout payload's + // `errors[]` and can leave stderr completely empty, so reading stderr alone + // discarded the only diagnostic there was and the policy engine received the + // generic "no findings payload" sentence as its skip reason. + let excerpt = first_nonempty([ + text_excerpt(stderr), + json_errors_excerpt(stdout), + text_excerpt(stdout), + ]); + + let exit_label = exit_code + .map(|code| code.to_string()) + .unwrap_or_else(|| "unknown".to_string()); + + if excerpt.is_empty() { + format!( + "semgrep exited {exit_label} with no findings payload (tool/config error, not a code failure)" + ) + } else { + format!( + "semgrep exited {exit_label} with no findings payload (tool/config error, not a code failure): {excerpt}" + ) + } +} + /// Build the `semgrep scan` argument list. Excludes build/vendor artifacts — /// `target`, `node_modules`, minified bundles (`*.min.js`) and the generated /// `public_dist/` site — so the scan does not emit forever-red findings on @@ -437,8 +561,10 @@ mod tests { } #[test] - fn non_zero_exit_is_failed() { - let stdout = r#"{"version":"1.135.0","results":[],"errors":[]}"#; + fn non_zero_exit_with_findings_payload_is_failed() { + // `--error` makes semgrep exit 1 when it found real results: that is a + // genuine code failure, not a tool error. + let stdout = r#"{"version":"1.135.0","results":[{"check_id":"rust.lang.security.blah","path":"src/x.rs","start":{"line":1}}],"errors":[]}"#; let combined = format!("{stdout}\n"); assert_eq!( classify_semgrep_status(false, stdout, &combined), @@ -446,6 +572,32 @@ mod tests { ); } + #[test] + fn non_zero_exit_with_no_payload_is_skipped_as_tool_error() { + // exit 2 with no results at all (invalid ruleset / crash): a tool or + // config error, not a code regression — must not be Failed + // (TOOL-VS-CODE, verify-ledger claim #8). + let stdout = ""; + let combined = "Invalid configuration file\nsemgrep: error while validating rules\n"; + assert_eq!( + classify_semgrep_status(false, stdout, combined), + CheckStatus::Skipped + ); + } + + #[test] + fn non_zero_exit_with_empty_results_array_is_skipped_as_tool_error() { + // A JSON payload that parses but carries zero results is still "no + // findings" — a non-zero exit alongside it is a tool error, not a + // code failure hiding behind an empty result set. + let stdout = r#"{"version":"1.135.0","results":[],"errors":[]}"#; + let combined = format!("{stdout}\n"); + assert_eq!( + classify_semgrep_status(false, stdout, &combined), + CheckStatus::Skipped + ); + } + #[test] fn output_reports_scan_errors_detects_partial_parsing() { let with_errors = r#"{"results":[],"errors":[{"type":["PartialParsing",[]]}]}"#; @@ -464,6 +616,81 @@ mod tests { assert!(output_reports_scan_errors(combined)); } + #[test] + fn output_has_findings_payload_detects_nonempty_results() { + let with_results = r#"{"results":[{"check_id":"x"}],"errors":[]}"#; + let empty_results = r#"{"results":[],"errors":[]}"#; + assert!(output_has_findings_payload(with_results)); + assert!(!output_has_findings_payload(empty_results)); + assert!(!output_has_findings_payload("not json")); + assert!(!output_has_findings_payload("")); + } + + #[test] + fn format_tool_error_reason_includes_exit_code_and_stderr_excerpt() { + let reason = format_tool_error_reason( + Some(2), + "", + "Invalid configuration file\nsemgrep: error while validating rules\n", + ); + assert!(reason.contains('2'), "reason must surface the exit code"); + assert!( + reason.contains("Invalid configuration file"), + "reason must surface a stderr excerpt" + ); + assert!( + reason.contains("tool/config error"), + "reason must name it as a tool error, not a code failure" + ); + } + + #[test] + fn format_tool_error_reason_handles_missing_exit_code_and_empty_stderr() { + let reason = format_tool_error_reason(None, "", ""); + assert!(reason.contains("unknown")); + assert!(reason.contains("tool/config error")); + } + + #[test] + fn format_tool_error_reason_reads_json_errors_from_stdout() { + // With `--json`, semgrep puts its diagnostics in the stdout payload's + // `errors[]` and can leave stderr empty. Reading stderr alone threw the + // only explanation away and handed the policy engine the generic "no + // findings payload" line as the skip reason. + let stdout = r#"{"results":[],"errors":[ + {"type":"InvalidRuleSchemaError","message":"invalid rule: missing key `pattern`"}, + {"type":"SemgrepError","long_msg":"config auto is unreachable"} + ]}"#; + let reason = format_tool_error_reason(Some(2), stdout, ""); + + assert!( + reason.contains("invalid rule: missing key `pattern`"), + "the JSON error must reach the skip reason: {reason}" + ); + assert!( + reason.contains("config auto is unreachable"), + "a second error must not be dropped: {reason}" + ); + assert!(reason.contains("tool/config error"), "{reason}"); + } + + #[test] + fn format_tool_error_reason_falls_back_to_non_json_stdout() { + // A crashing semgrep can print a traceback on stdout with nothing on + // stderr; that text is still the only diagnostic there is. + let reason = format_tool_error_reason(Some(2), "Traceback (most recent call last)\n", ""); + assert!( + reason.contains("Traceback"), + "non-JSON stdout is still a diagnostic: {reason}" + ); + } + + #[test] + fn format_tool_error_reason_prefers_stderr_when_both_carry_text() { + let reason = format_tool_error_reason(Some(2), "{\"results\":[],\"errors\":[]}", "boom\n"); + assert!(reason.contains("boom"), "{reason}"); + } + #[test] fn test_semgrep_check_can_run() { let config = test_config(); diff --git a/src/cli/mod.rs b/src/cli/mod.rs index 446c9be..bcf824d 100644 --- a/src/cli/mod.rs +++ b/src/cli/mod.rs @@ -215,6 +215,19 @@ pub struct Cli { #[arg(long = "soft-exit")] pub soft_exit: bool, + /// In --ci mode, also exit 1 when any check reports warnings + #[arg( + long = "fail-on-warnings", + requires = "ci", + conflicts_with = "soft_exit", + long_help = "Make --ci exit 1 when any check reports warnings, not only on a hard \ + failure. Warning-level signals (rustfmt deltas, an unmaintained-crate \ + advisory, lint warnings) are advisory by default and exit 0. This flag \ + restores the stricter pre-0.7 CI behaviour for teams that want a \ + warnings-clean trunk." + )] + pub fail_on_warnings: bool, + /// Regex pattern to filter which tests to run (passed to the test runner, e.g. vitest --grep) #[arg(long = "tests-pattern", value_name = "REGEX")] pub tests_pattern: Option, @@ -607,6 +620,7 @@ mod tests { no_dashboard: false, current_only: false, soft_exit: false, + fail_on_warnings: false, tests_pattern: None, pr_url: None, policy_file: None, diff --git a/src/gate.rs b/src/gate.rs index f12e235..8ed4427 100644 --- a/src/gate.rs +++ b/src/gate.rs @@ -8,6 +8,352 @@ use std::path::Path; pub const GATE_EXECUTION_ERROR_EXIT_CODE: i32 = 3; +/// `schema_version` this build stamps into `MERGE_GATE.json`. +pub const MERGE_GATE_SCHEMA_VERSION: &str = "2.2"; + +/// `MERGE_GATE.json` schemas this build has actually seen, as `(MAJOR, MINOR)`. +/// +/// This is the SAME set `tools/validate_merge_gate.py` accepts verbatim +/// (`1.0` / `2.0` / `2.1` / `2.2`), so "readable by the CLI/MCP" and "valid per the +/// contract validator" cannot drift apart for a version in the set. The reader +/// is deliberately broader in exactly two documented directions — an absent +/// field and a newer MINOR of a known MAJOR — and both are announced rather +/// than silent. +const MERGE_GATE_KNOWN_SCHEMAS: &[(u32, u32)] = &[(1, 0), (2, 0), (2, 1), (2, 2)]; + +/// Parse a strict `MAJOR.MINOR` version. Anything else — a bare `2`, a trailing +/// dot, or a third component like `2.1.3` — is NOT this contract's version +/// shape and must not be silently truncated into one. +/// +/// Each component must also be spelled canonically. `u32::from_str` accepts a +/// leading `+` and leading zeros, so `02.02` and `+2.2` would parse to the same +/// `(2, 2)` as `2.2` and be read as a known schema — while +/// `tools/validate_merge_gate.py` compares the raw string and rejects them. The +/// accepted set must BE the validator's set, not a superset that happens to +/// parse into it. +fn parse_major_minor(version: &str) -> Option<(u32, u32)> { + let (major, minor) = version.split_once('.')?; + if minor.contains('.') { + return None; + } + Some((canonical_u32(major)?, canonical_u32(minor)?)) +} + +/// Parse a decimal component written exactly as the validator would compare it: +/// digits only, and no leading zero unless the component IS `0`. +fn canonical_u32(component: &str) -> Option { + if component.is_empty() || !component.bytes().all(|b| b.is_ascii_digit()) { + return None; + } + if component.len() > 1 && component.starts_with('0') { + return None; + } + component.parse().ok() +} + +/// Newest MINOR this build knows for `major`, if the MAJOR is known at all. +fn newest_known_minor(major: u32) -> Option { + MERGE_GATE_KNOWN_SCHEMAS + .iter() + .filter(|(known_major, _)| *known_major == major) + .map(|(_, minor)| *minor) + .max() +} + +/// Check a pack's `MERGE_GATE.json` `schema_version` against what this build can +/// read, so readers stop guessing at packs they do not understand. +/// +/// * absent — accepted silently; packs predating the field are the documented +/// legacy read-back surface (same safety net as the retired `ALLOW`/`HOLD` +/// verdict synonyms). +/// * known MAJOR, MINOR this build has seen — accepted silently. +/// * known MAJOR, newer MINOR — accepted with a caveat: the pack may carry +/// fields this build ignores, and the reader must say so. This holds on EVERY +/// known MAJOR, not just the current one: a `1.9` pack is as unseen as a +/// `2.9` one, and silence about it was a reader claiming a fidelity it does +/// not have. +/// * unknown MAJOR, or a version that is not `MAJOR.MINOR` at all — fail loud; +/// a reader that cannot name the schema cannot honestly name the verdict. +pub fn check_merge_gate_schema(raw: Option<&str>) -> Result> { + let Some(raw) = raw else { + return Ok(None); + }; + let Some((major, minor)) = parse_major_minor(raw) else { + bail!("unreadable MERGE_GATE.json schema_version `{raw}` (expected MAJOR.MINOR)"); + }; + let Some(newest_minor) = newest_known_minor(major) else { + let known: Vec = MERGE_GATE_KNOWN_SCHEMAS + .iter() + .map(|(major, minor)| format!("{major}.{minor}")) + .collect(); + bail!( + "unsupported MERGE_GATE.json schema_version `{raw}`: major {major} is not readable by \ + this build (known schemas: {}; current schema {MERGE_GATE_SCHEMA_VERSION})", + known.join(", ") + ); + }; + if minor > newest_minor { + return Ok(Some(format!( + "schema_forward_compat: MERGE_GATE.json schema_version `{raw}` is newer than the \ + newest `{major}.{newest_minor}` this build knows; unknown fields were ignored" + ))); + } + Ok(None) +} + +/// [`check_merge_gate_schema`] for a raw JSON field, distinguishing "absent" +/// from "present but not a string". +/// +/// Every reader used to reach the checker through `.and_then(Value::as_str)`, +/// which maps a number, an object, or an explicit `null` onto `None` — the one +/// input the checker accepts in silence, because an absent field means a +/// pre-2.1 pack. A pack that states a `schema_version` this build cannot even +/// type is the opposite of a legacy pack and must fail loud. +pub fn check_merge_gate_schema_field(field: Option<&serde_json::Value>) -> Result> { + match field { + None => check_merge_gate_schema(None), + Some(serde_json::Value::String(raw)) => check_merge_gate_schema(Some(raw)), + Some(other) => bail!( + "unreadable MERGE_GATE.json schema_version: expected a MAJOR.MINOR string, found {}", + match other { + serde_json::Value::Null => "null".to_string(), + serde_json::Value::Bool(_) => "a boolean".to_string(), + serde_json::Value::Number(n) => format!("the number {n}"), + serde_json::Value::Array(_) => "an array".to_string(), + serde_json::Value::Object(_) => "an object".to_string(), + serde_json::Value::String(_) => unreachable!("handled above"), + } + ), + } +} + +/// JSON type a decision signal is expected to carry. +#[derive(Clone, Copy)] +pub(crate) enum JsonKind { + String, + Boolean, + Array, +} + +impl JsonKind { + fn matches(self, value: &serde_json::Value) -> bool { + match self { + Self::String => value.is_string(), + Self::Boolean => value.is_boolean(), + Self::Array => value.is_array(), + } + } + + fn label(self) -> &'static str { + match self { + Self::String => "a string", + Self::Boolean => "a boolean", + Self::Array => "an array", + } + } +} + +/// Human-readable JSON type name, for saying what was found instead. +fn json_type_name(value: &serde_json::Value) -> &'static str { + match value { + serde_json::Value::Null => "null", + serde_json::Value::Bool(_) => "a boolean", + serde_json::Value::Number(_) => "a number", + serde_json::Value::String(_) => "a string", + serde_json::Value::Array(_) => "an array", + serde_json::Value::Object(_) => "an object", + } +} + +/// A decision signal, or `None` plus a caveat when it is present with the wrong +/// JSON type. +/// +/// Absence is the one state a reader accepts in silence — it is the documented +/// shape of an older pack. A field that IS there but cannot be typed is a +/// different thing entirely, and collapsing the two through `as_str()` lets a +/// reader ignore a signal while reporting a clean passthrough. +/// +/// Shared by both readers on purpose: the CLI and the MCP adapter answer the +/// same contract question about the same artifact, and the one that had this +/// rule while the other did not is how `merge_recommendation: 7` came back as +/// `storage_corrupt` from one surface and as `approve` from the other. +pub(crate) fn readable_signal<'v>( + field: &str, + value: Option<&'v serde_json::Value>, + want: JsonKind, + caveats: &mut Vec, +) -> Option<&'v serde_json::Value> { + let present = value?; + if want.matches(present) { + return Some(present); + } + caveats.push(format!( + "unreadable_{field}: MERGE_GATE.json {field} is {}, not {}; it was ignored when deriving \ + this decision", + json_type_name(present), + want.label() + )); + None +} + +/// Select the object a gate pack's decision is read from. +/// +/// A stated `decision` object always wins, and only then does the presence of +/// `schema_version` decide what an absent one means: +/// +/// * no `schema_version` — a pack predating the field. Its ROOT is the decision; +/// this is the legacy read-back surface every reader keeps. The tolerance +/// answers WHERE the decision sits when nothing else states it — it is not a +/// rule that the root outranks a `decision` object the pack did write. A +/// schema-less pack carrying one is read from it, because the alternative is +/// to read a plainly stated decision as a decision with every signal missing, +/// and every signal missing normalizes to BLOCK. +/// * `schema_version` stated — the `decision` object that schema is built around +/// is mandatory. Falling back to the root there would publish a verdict +/// nothing in the pack stated, which is a re-derivation wearing a reader's +/// clothes. `tools/validate_merge_gate.py` requires `decision` at every +/// version, so a reader that shrugs disagrees with the contract validator. +/// +/// The legacy tolerance is about WHERE the decision sits, not about whether the +/// pack is a decision at all: a root that is an array, a scalar or `null` has no +/// fields to read, and accepting it let the CLI answer a normalized BLOCK for an +/// artifact the MCP reader called corrupt. Both now reject it. +/// +/// `Err` describes which shape rule the pack broke; callers add their own +/// framing. +pub fn select_decision_object( + value: &serde_json::Value, +) -> Result<&serde_json::Value, DecisionShapeError> { + match value.get("decision") { + Some(decision) if decision.is_object() => Ok(decision), + _ if value.get("schema_version").is_some() => { + Err(DecisionShapeError::VersionedWithoutDecision( + value + .get("schema_version") + .and_then(serde_json::Value::as_str) + .unwrap_or("?") + .to_string(), + )) + } + _ if value.is_object() => Ok(value), + _ => Err(DecisionShapeError::NonObjectRoot(json_type_name(value))), + } +} + +/// Why a gate pack carries no readable decision. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum DecisionShapeError { + /// The pack names its schema and then omits the object that schema is built + /// around. + VersionedWithoutDecision(String), + /// The pack states no schema, so its root WOULD be the decision — but the + /// root is not an object. + NonObjectRoot(&'static str), +} + +impl DecisionShapeError { + /// The defect, as a clause a caller can put in its own sentence. + pub fn describe(&self) -> String { + match self { + Self::VersionedWithoutDecision(schema) => { + format!("states schema_version {schema} but carries no `decision` object") + } + Self::NonObjectRoot(kind) => { + format!("is {kind}, not a JSON object, so it states no decision at all") + } + } + } +} + +/// Conservativeness rank of one decision axis: 1 = clean pass, 2 = hold / +/// review required, 3 = block. +/// +/// Both readers reconcile a decision by taking the MAX rank across the axes the +/// pack states, then publishing every axis from that one number. A pack whose +/// `verdict` says BLOCK beside a `merge_recommendation` of `approve` is +/// contradictory, and a reader that simply believes each field in turn +/// publishes an approval the artifact never gave. The rule lives here so the +/// CLI and the MCP adapter cannot answer it differently. +pub fn rank_from_merge_rec(s: &str) -> Option { + match s.to_ascii_lowercase().as_str() { + "block" => Some(3), + "review_required" | "hold" => Some(2), + "approve" => Some(1), + _ => None, + } +} + +/// The canonical verdict a stored spelling means, or `None` when it is outside +/// the vocabulary entirely. +/// +/// This is THE verdict vocabulary. Every reader folds through it — the CLI +/// `--json` summary and the MCP adapter both — because two surfaces owning two +/// copies of one vocabulary is how they came to disagree about the same file: +/// the CLI matched the raw string case-sensitively while the adapter ranked it +/// through an uppercase fold, so `verdict: "pass"` was a clean PASS to MCP +/// automation and an unknown verdict normalized to BLOCK on the CLI. +/// +/// Case is not meaning. Neither is a retired synonym: `ALLOW`/`APPROVE` are the +/// pre-2.1 spellings of a clean pass and `HOLD` of `CONDITIONAL`, kept readable +/// so a legacy pack on disk still normalizes instead of failing loud. What a +/// pack states is what it stated — reading `"pass"` as a block would fabricate +/// a verdict the artifact never gave, which is the same defect in the other +/// direction. +pub fn canonical_verdict(raw: &str) -> Option<&'static str> { + match raw.to_ascii_uppercase().as_str() { + "BLOCK" => Some("BLOCK"), + "CONDITIONAL" | "HOLD" => Some("CONDITIONAL"), + "PASS" | "APPROVE" | "ALLOW" => Some("PASS"), + _ => None, + } +} + +pub fn rank_from_verdict(s: &str) -> Option { + canonical_verdict(s).map(|canonical| match canonical { + "BLOCK" => 3, + "CONDITIONAL" => 2, + _ => 1, + }) +} + +/// Conservativeness rank of a stated `analysis_status`, or `None` when the +/// value states nothing this contract can rank. +/// +/// `complete` is a PRECONDITION of `PASS`, not a grant of it: a complete +/// analysis still ends at `BLOCK` when policy blocks, so reading it as rank 1 +/// would let one axis soften a verdict the others agree on — the same asymmetry +/// as `quality_pass: true`. `degraded` and `incomplete` rule `PASS` out, so both +/// rank 2. Anything else is outside the vocabulary and cannot rank at all; +/// callers name it with an `unknown_analysis_status:` caveat rather than +/// letting it vanish, exactly as they do for `merge_recommendation`. +pub(crate) fn rank_from_analysis_status(s: &str) -> Option { + match s { + "degraded" | "incomplete" => Some(2), + _ => None, + } +} + +/// Whether a stated `analysis_status` is one this contract defines. +pub(crate) fn known_analysis_status(s: &str) -> bool { + matches!(s, "complete" | "degraded" | "incomplete") +} + +pub fn merge_rec_from_rank(rank: u8) -> &'static str { + match rank { + 3 => "block", + 2 => "review_required", + _ => "approve", + } +} + +pub fn verdict_from_rank(rank: u8) -> &'static str { + match rank { + 3 => "BLOCK", + 2 => "CONDITIONAL", + _ => "PASS", + } +} + #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] pub enum GateVerdict { #[serde(rename = "PASS")] @@ -95,7 +441,16 @@ pub fn build_gate_json_output( strict: bool, ) -> Result { let decision = read_merge_gate_decision(merge_gate_path)?; - if summary.verdict != decision.verdict { + // Fold the stored spelling before comparing. This check guards against the + // summary and the pack stating DIFFERENT decisions; a pack spelling its + // verdict `ALLOW`, `hold` or `pass` states the same decision the summary + // read, and firing here on that made `prview gate` reject an artifact both + // other readers accept. + let stated = match canonical_verdict(&decision.verdict) { + Some(canonical) => canonical, + None => bail!("unknown gate verdict `{}`", decision.verdict), + }; + if summary.verdict != stated { bail!( "gate verdict mismatch: CLI summary has `{}`, MERGE_GATE.json has `{}`", summary.verdict, @@ -103,7 +458,7 @@ pub fn build_gate_json_output( ); } - let verdict = GateVerdict::try_from(decision.verdict.as_str())?; + let verdict = GateVerdict::try_from(stated)?; let exit_code = gate_exit_code(verdict, strict); Ok(GateJsonOutput { @@ -148,7 +503,152 @@ mod tests { #[test] fn gate_verdict_parse_fails_loud_for_unknown_values() { + // `GateVerdict` is the TYPED parser for canonical values, so it stays + // strict. Legacy and non-canonical spellings are folded by + // `canonical_verdict` before they ever reach it. assert!(GateVerdict::try_from("HOLD").is_err()); assert!(GateVerdict::try_from("ALLOW").is_err()); + assert!(GateVerdict::try_from("pass").is_err()); + } + + #[test] + fn one_vocabulary_folds_every_spelling_the_readers_accept() { + for spelling in ["PASS", "pass", "Pass", "ALLOW", "allow", "APPROVE"] { + assert_eq!(canonical_verdict(spelling), Some("PASS"), "{spelling}"); + } + for spelling in ["CONDITIONAL", "conditional", "HOLD", "hold"] { + assert_eq!( + canonical_verdict(spelling), + Some("CONDITIONAL"), + "{spelling}" + ); + } + for spelling in ["BLOCK", "block", "Block"] { + assert_eq!(canonical_verdict(spelling), Some("BLOCK"), "{spelling}"); + } + // Outside the vocabulary stays outside it: folding is about spelling, + // not about inventing a reading. + for spelling in ["", "MAYBE", "pas", "approved"] { + assert_eq!(canonical_verdict(spelling), None, "{spelling}"); + } + } + + #[test] + fn the_rank_of_a_verdict_follows_its_canonical_form() { + // The ranking and the folding cannot drift apart, because the ranking + // is derived from the folding. + for spelling in ["PASS", "pass", "ALLOW", "APPROVE"] { + assert_eq!(rank_from_verdict(spelling), Some(1), "{spelling}"); + } + for spelling in ["CONDITIONAL", "hold", "HOLD"] { + assert_eq!(rank_from_verdict(spelling), Some(2), "{spelling}"); + } + for spelling in ["BLOCK", "block"] { + assert_eq!(rank_from_verdict(spelling), Some(3), "{spelling}"); + } + assert_eq!(rank_from_verdict("MAYBE"), None); + } + + #[test] + fn a_decision_object_wins_over_the_root_even_without_a_schema() { + // The precedence is deliberate and load-bearing, so it is asserted + // rather than left implicit. + // + // The legacy tolerance answers WHERE a pack's decision sits when + // nothing else states it — not "the root always wins". Preferring the + // root whenever `schema_version` is absent would read this pack, whose + // decision is stated plainly in a `decision` object, as a decision with + // every signal missing, and every signal missing normalizes to BLOCK: + // a fabricated block for an artifact that stated an approval. + let nested_only = serde_json::json!({ + "checks": [], + "decision": {"verdict": "PASS", "allow_merge": true}, + }); + let decision = + select_decision_object(&nested_only).expect("a stated decision object is readable"); + assert_eq!( + decision.get("verdict").and_then(|v| v.as_str()), + Some("PASS") + ); + + // With no `decision` object the root IS the decision, which is the + // whole of the legacy read-back surface. + let root_only = serde_json::json!({"verdict": "ALLOW", "allow_merge": true}); + let decision = select_decision_object(&root_only).expect("a legacy root is readable"); + assert_eq!( + decision.get("verdict").and_then(|v| v.as_str()), + Some("ALLOW") + ); + } + + #[test] + fn schema_check_accepts_absent_and_known_versions_silently() { + assert_eq!(check_merge_gate_schema(None).unwrap(), None); + assert_eq!(check_merge_gate_schema(Some("2.2")).unwrap(), None); + assert_eq!(check_merge_gate_schema(Some("2.1")).unwrap(), None); + assert_eq!(check_merge_gate_schema(Some("2.0")).unwrap(), None); + assert_eq!(check_merge_gate_schema(Some("1.0")).unwrap(), None); + } + + #[test] + fn schema_check_tolerates_newer_minor_with_caveat() { + let caveat = check_merge_gate_schema(Some("2.7")) + .expect("newer minor is tolerated") + .expect("newer minor emits a caveat"); + assert!(caveat.starts_with("schema_forward_compat:"), "{caveat}"); + assert!(caveat.contains("2.7"), "{caveat}"); + } + + #[test] + fn schema_check_rejects_versions_that_are_not_major_minor() { + // `tools/validate_merge_gate.py` accepts an exact string set, so a + // reader that silently truncates `2.1.3` to `2.1` calls a pack readable + // that the contract validator rejects. + for raw in ["2.1.3", "2", "2.", "2.1.", ".1", "2.1.0.0"] { + assert!( + check_merge_gate_schema(Some(raw)).is_err(), + "`{raw}` is not MAJOR.MINOR and must fail loud" + ); + } + } + + #[test] + fn schema_check_rejects_non_canonical_component_spelling() { + // `u32::from_str` accepts leading zeros and a leading `+`, so `02.02`, + // `2.02` and `+2.2` all normalized to the known `(2, 2)` and were read + // as the current schema — while the validator rejects those exact + // strings. The accepted set has to be the validator's set, not a + // superset that happens to parse. + for raw in ["02.2", "2.02", "+2.2", "2.+2", "02.02", " 2.2", "2.2 "] { + assert!( + check_merge_gate_schema(Some(raw)).is_err(), + "`{raw}` is not canonical MAJOR.MINOR and must fail loud" + ); + } + // The canonical spellings, including a genuine zero component, stay read. + assert_eq!(check_merge_gate_schema(Some("2.0")).unwrap(), None); + assert_eq!(check_merge_gate_schema(Some("1.0")).unwrap(), None); + } + + #[test] + fn schema_check_caveats_newer_minor_of_a_legacy_major() { + // A `1.x` pack newer than the only released `1.0` was accepted in total + // silence, while the same situation on the current major produced a + // caveat. Unknown minors carry unknown fields on every known major. + let caveat = check_merge_gate_schema(Some("1.9")) + .expect("a known major stays readable") + .expect("a minor this build never saw must be named"); + assert!(caveat.starts_with("schema_forward_compat:"), "{caveat}"); + assert!(caveat.contains("1.9"), "{caveat}"); + } + + #[test] + fn schema_check_fails_loud_on_unknown_major() { + let err = check_merge_gate_schema(Some("3.0")).expect_err("unknown major must fail loud"); + assert!(err.to_string().contains("unsupported"), "{err}"); + assert!( + check_merge_gate_schema(Some("not-a-version")).is_err(), + "unparsable schema_version must fail loud" + ); } } diff --git a/src/lib.rs b/src/lib.rs index 00b9fd7..01d8d54 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -19,6 +19,7 @@ pub mod paths; pub mod policy; pub mod proc; pub mod regression; +pub(crate) mod rust_source; pub mod scope; pub mod state; pub mod storage; diff --git a/src/main.rs b/src/main.rs index 532f1dc..50a1ed8 100644 --- a/src/main.rs +++ b/src/main.rs @@ -110,7 +110,16 @@ async fn run() -> Result<()> { // Normal run let report = app.run().await?; - let cli_summary = prview::output::build_cli_json_summary(&app.config, &report); + // The verdict comes from the pack's MERGE_GATE.json and nowhere else. If it + // cannot be read, prview cannot report a verdict — that is an execution + // error (exit 3, same contract as `prview gate`), never a guessed summary. + let cli_summary = match prview::output::build_cli_json_summary(&app.config, &report) { + Ok(summary) => summary, + Err(err) => { + display_error(&err); + std::process::exit(prview::gate::GATE_EXECUTION_ERROR_EXIT_CODE); + } + }; // JSON output mode. Human summaries are emitted by App::run(); do not // print them a second time here. @@ -118,15 +127,31 @@ async fn run() -> Result<()> { println!("{}", serde_json::to_string_pretty(&cli_summary)?); } - // Exit with appropriate code. An unchanged --update run re-checked nothing, - // so it exits 0 regardless of --json — previously the human path exited 0 - // while the JSON path derived its code from an empty gate. - let exit_code = if report.unchanged || cli.soft_exit { + // An unchanged `--update` run re-checked nothing, but it REPORTS the pack it + // reused, and the exit code follows the summary it just published — the same + // rule every other run obeys. Forcing 0 here made a second `--ci + // --fail-on-warnings` invocation turn green over a pack that still warned, + // and it swallowed a reused BLOCK just as quietly. The original reason for + // the shortcut is gone: the code was derived from an EMPTY gate back when a + // missing pack was re-derived, and an unreadable pack now exits 3 above. + // `--soft-exit` stays the one deliberate way to ask for 0. + let exit_code = if cli.soft_exit { 0 } else { - prview::output::compute_exit_code(&cli_summary) + prview::output::compute_exit_code(&cli_summary, cli.ci, cli.fail_on_warnings) }; + // A human unchanged run prints "Nothing to update." and no verdict, so say + // which verdict the exit code came from instead of failing wordlessly. + if report.unchanged && !cli.json && !cli.quiet && exit_code != 0 { + println!( + "{} Reused verdict: {} (exit {})", + "ℹ".blue(), + cli_summary.verdict, + exit_code + ); + } + std::process::exit(exit_code); } @@ -144,12 +169,15 @@ async fn run_gate_command(cli: &Cli, args: &GateArgs) -> Result { run_cli.quiet = true; run_cli.json = false; run_cli.soft_exit = false; + // `gate` forces `ci = false` and derives its exit from the gate contract, so + // the `--ci`-scoped warnings escape hatch must not leak into it. + run_cli.fail_on_warnings = false; let mut config = Config::from_cli(&run_cli)?; config.apply_gate_profile(); let app = App::from_config(config)?; let report = app.run().await.context("gate review run failed")?; - let cli_summary = prview::output::build_cli_json_summary(&app.config, &report); + let cli_summary = prview::output::build_cli_json_summary(&app.config, &report)?; let merge_gate_path = report .artifacts_dir .join("00_summary") diff --git a/src/mcp/read.rs b/src/mcp/read.rs index d954d52..217aa39 100644 --- a/src/mcp/read.rs +++ b/src/mcp/read.rs @@ -4,6 +4,10 @@ //! (`~/.prview/`) or a run's artifact pack. No review logic lives here — the //! MCP surface only reads truth the core already wrote. +use crate::gate::{ + JsonKind, merge_rec_from_rank, rank_from_merge_rec, rank_from_verdict, readable_signal, + verdict_from_rank, +}; use crate::mcp::types::{ToolError, error_class}; use crate::storage::{RunEntry, RunIndex}; use std::path::{Path, PathBuf}; @@ -623,48 +627,6 @@ pub struct NormalizedDecision { pub normalized: bool, } -/// Conservativeness rank: BLOCK(3) > HOLD/review_required(2) > APPROVE/PASS(1). -fn rank_from_merge_rec(s: &str) -> Option { - match s.to_ascii_lowercase().as_str() { - "block" => Some(3), - "review_required" | "hold" => Some(2), - "approve" => Some(1), - _ => None, - } -} - -fn rank_from_verdict(s: &str) -> Option { - match s.to_ascii_uppercase().as_str() { - "BLOCK" => Some(3), - // `CONDITIONAL` is the unified core vocabulary (PV-03/04); `HOLD` is the - // retired legacy synonym, still recognized so the adapter stays a safe - // read-back net for pre-2.1 runs on disk. - "CONDITIONAL" | "HOLD" => Some(2), - // `ALLOW` is the retired pre-2.1 verdict synonym for a clean pass (folded - // to `PASS` on the CLI `--json` surface in `output::read_merge_gate_summary`). - // The adapter recognizes it for the same reason it recognizes `HOLD`: - // a legacy gate on disk must still normalize instead of failing loud. - "PASS" | "APPROVE" | "ALLOW" => Some(1), - _ => None, - } -} - -fn merge_rec_from_rank(rank: u8) -> &'static str { - match rank { - 3 => "block", - 2 => "review_required", - _ => "approve", - } -} - -fn verdict_from_rank(rank: u8) -> &'static str { - match rank { - 3 => "BLOCK", - 2 => "CONDITIONAL", - _ => "PASS", - } -} - fn string_array(value: Option<&serde_json::Value>) -> Vec { value .and_then(|v| v.as_array()) @@ -692,62 +654,260 @@ pub fn read_decision(run_dir: &Path) -> Result { format!("MERGE_GATE.json is not valid JSON: {e}"), ) })?; - let decision = value.get("decision").ok_or_else(|| { + // A pack whose schema this build does not know cannot be normalized + // honestly: an unknown MAJOR is fail-loud, a newer MINOR is tolerated but + // carries a caveat. An absent `schema_version` is the documented pre-2.1 + // read-back surface and is accepted silently. + let schema_caveat = crate::gate::check_merge_gate_schema_field(value.get("schema_version")) + .map_err(|e| ToolError::new(error_class::STORAGE_CORRUPT, e.to_string()))?; + + // A pack with no `schema_version` predates the field and its ROOT is the + // decision — the same legacy tolerance the CLI reader and the contract keep. + // Demanding a nested `decision` at every version made the one shape the + // contract explicitly tolerates come back `storage_corrupt` from this + // surface while the CLI read it fine. + let decision = crate::gate::select_decision_object(&value).map_err(|shape| { ToolError::new( error_class::STORAGE_CORRUPT, - "MERGE_GATE.json missing `decision` object", + format!("MERGE_GATE.json {}", shape.describe()), ) })?; - let raw_merge = decision - .get("merge_recommendation") - .and_then(|v| v.as_str()) - .map(str::to_string); - let raw_verdict = decision - .get("verdict") - .and_then(|v| v.as_str()) - .map(str::to_string); - let raw_allow = decision.get("allow_merge").and_then(|v| v.as_bool()); + // A field present with the wrong JSON type is NOT an absent field. Reading + // it through `as_str()` / `as_bool()` collapsed the two, and "absent" is the + // one state the adapter accepts in silence — so `merge_recommendation: 7` + // beside a valid verdict produced a confident passthrough that had quietly + // ignored a signal. Keep the two apart and name what was ignored. + let mut unknown_signal_caveats = Vec::new(); + + let raw_merge = readable_signal( + "merge_recommendation", + decision.get("merge_recommendation"), + JsonKind::String, + &mut unknown_signal_caveats, + ) + .and_then(|v| v.as_str()) + .map(str::to_string); + let raw_verdict = readable_signal( + "verdict", + decision.get("verdict"), + JsonKind::String, + &mut unknown_signal_caveats, + ) + .and_then(|v| v.as_str()) + .map(str::to_string); + let raw_allow = readable_signal( + "allow_merge", + decision.get("allow_merge"), + JsonKind::Boolean, + &mut unknown_signal_caveats, + ) + .and_then(|v| v.as_bool()); + // Read here rather than next to its rank below, so that a mistyped value + // reaches `mistyped_signal` on the line after this one. A bare `as_bool()` + // read `quality_pass: "false"` as absent: no caveat, no rank, and the + // surface answered `approve` while the pack said the quality axis failed. + let raw_quality_pass = readable_signal( + "quality_pass", + decision.get("quality_pass"), + JsonKind::Boolean, + &mut unknown_signal_caveats, + ) + .and_then(|v| v.as_bool()); + // The confidence and blocker axes, mirroring the CLI. All three are read + // here so a mistyped one reaches `mistyped_signal` below; `blocking_issues` + // was previously read only at the very end, for passthrough, so a stated + // blocker never touched the decision this adapter returned. + let raw_analysis_status = readable_signal( + "analysis_status", + decision.get("analysis_status"), + JsonKind::String, + &mut unknown_signal_caveats, + ) + .and_then(|v| v.as_str().map(str::to_string)); + let raw_policy_allow_merge = readable_signal( + "policy_allow_merge", + decision.get("policy_allow_merge"), + JsonKind::Boolean, + &mut unknown_signal_caveats, + ) + .and_then(|v| v.as_bool()); + let raw_blocking_issues = readable_signal( + "blocking_issues", + decision.get("blocking_issues"), + JsonKind::Array, + &mut unknown_signal_caveats, + ) + .and_then(|v| v.as_array()) + .map(|issues| issues.len()); + + // Whether any signal was present and could not be TYPED. Captured before + // the vocabulary caveats below join the same list, because the two are + // different failures with the same consequence. + let mistyped_signal = !unknown_signal_caveats.is_empty(); let merge_rank = raw_merge.as_deref().and_then(rank_from_merge_rec); let verdict_rank = raw_verdict.as_deref().and_then(rank_from_verdict); - // Need at least one decision signal to build a truthful surface. - if merge_rank.is_none() && verdict_rank.is_none() { + // A present-but-unrecognized signal used to vanish into the `flatten()` + // below, so the caller saw a confident surface derived from the OTHER + // signal with no hint that a field had been dropped. Record it instead: + // the value is still not used to rank, but the reader stops pretending it + // read the pack cleanly. Legacy `ALLOW`/`HOLD` are recognized vocabulary, + // so they never land here. + if let Some(raw) = raw_verdict.as_deref() + && verdict_rank.is_none() + { + unknown_signal_caveats.push(format!( + "unknown_verdict: MERGE_GATE.json verdict `{raw}` is not in the \ + PASS/CONDITIONAL/BLOCK vocabulary; normalized to BLOCK" + )); + } + // An absent verdict is named too, exactly as the CLI names it: the decision + // this adapter returns is then the reader's substitution, not the pack's. + if raw_verdict.is_none() && decision.get("verdict").is_none() { + unknown_signal_caveats.push( + "unknown_verdict: MERGE_GATE.json decision carries no `verdict`; normalized to BLOCK" + .to_string(), + ); + } + if let Some(raw) = raw_merge.as_deref() + && merge_rank.is_none() + { + unknown_signal_caveats.push(format!( + "unknown_merge_recommendation: MERGE_GATE.json merge_recommendation `{raw}` is not in \ + the approve/review_required/block vocabulary; it was ignored when deriving this \ + decision" + )); + } + if let Some(raw) = raw_analysis_status.as_deref() + && !crate::gate::known_analysis_status(raw) + { + unknown_signal_caveats.push(format!( + "unknown_analysis_status: MERGE_GATE.json analysis_status `{raw}` is not in the \ + complete/degraded/incomplete vocabulary; it was ignored when deriving this decision" + )); + } + + // Corrupt means the decision states NOTHING — not that what it states is + // unrankable. The presence test is the CLI's, field for field: a pack that + // named a verdict outside the vocabulary, or nothing but `allow_merge`, DID + // state a decision, and calling it corrupt here while the CLI published a + // summary for it left the same artifact readable on one surface and broken + // on the other. + if !["verdict", "merge_recommendation", "allow_merge"] + .iter() + .any(|field| decision.get(*field).is_some()) + { return Err(ToolError::new( error_class::STORAGE_CORRUPT, - "MERGE_GATE.json decision has no recognizable merge_recommendation or verdict", + "MERGE_GATE.json decision states no verdict, merge_recommendation or allow_merge", )); } // allow_merge=false raises conservativeness to at least HOLD; allow=true // never lowers it (a permissive flag can't override a block/hold signal). let allow_rank = raw_allow.map(|allow| if allow { 1 } else { 2 }); - - let final_rank = [merge_rank, verdict_rank, allow_rank] - .into_iter() - .flatten() - .max() - .unwrap_or(2); + // `quality_pass: false` says "not a PASS" — the contract permits `PASS` only + // when quality passes — so it ranks 2, like `allow_merge: false`. `true` + // states no rank of its own: a quality-clean run is still held at + // CONDITIONAL by a breaking-change escalation. Absence states nothing + // either, so a pack written before the field reads exactly as it always did. + // This is the CLI's rule, mirrored, because a pack this adapter approved + // while the CLI held it is the same split the reader parity work closed. + // (Read above, with the other typed signals.) + let quality_rank = match raw_quality_pass { + Some(false) => Some(2), + _ => None, + }; + // Same rule on the confidence axis: only `degraded`/`incomplete` rule `PASS` + // out and therefore rank. `complete` is a precondition of `PASS`, not a + // grant of it, so it stays silent like `quality_pass: true`. + let analysis_rank = raw_analysis_status + .as_deref() + .and_then(crate::gate::rank_from_analysis_status); + // A stated blocker is a stated BLOCK: `blocking_issues` is non-empty only + // when a check reached `PolicyConclusion::Blocked`, whose `merge_impact` is + // `Block`. `policy_allow_merge: false` is the same fact — the emitter writes + // `policy_allow_merge = blocking_issues.is_empty()`. Neither says anything + // permissive: "policy did not hard-block" is explicitly NOT `allow_merge`. + let blocker_rank = (raw_policy_allow_merge == Some(false) + || raw_blocking_issues.is_some_and(|len| len > 0)) + .then_some(3); + + // A verdict this reader had to SUBSTITUTE — absent, outside the vocabulary, + // or present with the wrong JSON type — governs everything derived beside + // it, and so does any other signal that could not be typed. This is the + // CLI's `normalized_to_block` rule, mirrored: a decision derived from a + // block the reader only partly read is not one either surface may publish + // as permissive, and the two surfaces answering that differently is how one + // pack came to be a `PASS` for MCP automation and a `BLOCK` on the CLI. + let normalized_to_block = mistyped_signal || verdict_rank.is_none(); + + let stated_ranks: Vec = [ + merge_rank, + verdict_rank, + allow_rank, + quality_rank, + analysis_rank, + blocker_rank, + ] + .into_iter() + .flatten() + .collect(); + let final_rank = if normalized_to_block { + 3 + } else { + stated_ranks.iter().copied().max().unwrap_or(3) + }; let allow_merge = final_rank == 1; - // Inconsistent iff the raw signals disagree on conservativeness, or the raw - // allow_merge flag contradicts the final (derived) recommendation. - let signal_ranks: Vec = [merge_rank, verdict_rank].into_iter().flatten().collect(); - let signals_disagree = signal_ranks.iter().any(|&r| r != final_rank); - let allow_contradicts = raw_allow.map(|a| a != allow_merge).unwrap_or(false); - let normalized = signals_disagree || allow_contradicts; - - let mut caveats = Vec::new(); - if normalized { + // Only the PACK's own axes can be inconsistent with each other; a verdict + // this reader substituted is already named by its own caveat, and calling + // the substitution an inconsistency would blame the artifact for the + // reader's normalization. + // + // `allow_merge` raises the rank but is not compared as one: `false` says + // "not a PASS" — `>= 2`, never 3 — so treating it as an exact rank would + // call every healthy BLOCK pack inconsistent with itself. It contradicts the + // decision only when the DERIVED flag disagrees with the stated one. + let textual_ranks: Vec = [merge_rank, verdict_rank].into_iter().flatten().collect(); + let signals_disagree = + !normalized_to_block && textual_ranks.iter().any(|&rank| rank != final_rank); + let allow_contradicts = + !normalized_to_block && raw_allow.map(|a| a != allow_merge).unwrap_or(false); + // An ignored signal is itself a normalization: the returned decision is not + // a faithful passthrough of what the pack says. A forward schema is the same + // situation one level up — the pack was written by a build this one does not + // fully know, so the read is best-effort and the caveat must be backed by the + // flag consumers actually branch on. + let normalized = signals_disagree + || allow_contradicts + || !unknown_signal_caveats.is_empty() + || schema_caveat.is_some(); + + let mut caveats = schema_caveat.into_iter().collect::>(); + caveats.append(&mut unknown_signal_caveats); + if signals_disagree || allow_contradicts { caveats.push(format!( - "core_inconsistency: original allow_merge={}, merge_recommendation={}, verdict={}", + "core_inconsistency: original allow_merge={}, merge_recommendation={}, verdict={}, \ + quality_pass={}, analysis_status={}, blocking_issues={}, policy_allow_merge={}", raw_allow .map(|b| b.to_string()) .unwrap_or_else(|| "null".to_string()), raw_merge.as_deref().unwrap_or("null"), raw_verdict.as_deref().unwrap_or("null"), + raw_quality_pass + .map(|b| b.to_string()) + .unwrap_or_else(|| "null".to_string()), + raw_analysis_status.as_deref().unwrap_or("null"), + raw_blocking_issues + .map(|len| len.to_string()) + .unwrap_or_else(|| "null".to_string()), + raw_policy_allow_merge + .map(|b| b.to_string()) + .unwrap_or_else(|| "null".to_string()), )); } caveats.extend(string_array(decision.get("review_caveats"))); @@ -924,6 +1084,309 @@ mod tests { ); } + #[test] + fn unknown_verdict_is_reported_as_normalized_with_caveat() { + // An unrecognized verdict used to vanish into the rank `flatten()`: the + // decision was derived from `merge_recommendation` alone and returned as + // a clean passthrough, with nothing telling the caller a field had been + // dropped. It must surface as an explicit `unknown_verdict` caveat — and + // it must also govern the decision published beside it. Deriving a PASS + // from the surviving `approve` was this adapter reading one pack as an + // approval while the CLI, on the same bytes, substituted BLOCK. + let dir = tempfile::tempdir().unwrap(); + write_gate( + dir.path(), + &serde_json::json!({ + "bases": ["main"], + "decision": { + "merge_recommendation": "approve", + "verdict": "MAYBE", + "allow_merge": true + } + }), + ); + let d = read_decision(dir.path()).unwrap(); + assert_eq!( + d.verdict, "BLOCK", + "a substituted verdict governs every axis derived beside it" + ); + assert_eq!(d.merge_recommendation, "block"); + assert!( + !d.allow_merge, + "the pack's `allow_merge: true` does not stand" + ); + assert!(d.normalized, "an ignored signal is a normalization"); + let caveat = d + .caveats + .iter() + .find(|c| c.starts_with("unknown_verdict:")) + .expect("unknown_verdict caveat present"); + assert!(caveat.contains("MAYBE"), "{caveat}"); + } + + #[test] + fn unknown_merge_recommendation_is_reported_with_caveat() { + let dir = tempfile::tempdir().unwrap(); + write_gate( + dir.path(), + &serde_json::json!({ + "bases": ["main"], + "decision": { + "merge_recommendation": "probably_fine", + "verdict": "BLOCK", + "allow_merge": false + } + }), + ); + let d = read_decision(dir.path()).unwrap(); + assert_eq!(d.verdict, "BLOCK"); + assert!(d.normalized); + assert!( + d.caveats + .iter() + .any(|c| c.starts_with("unknown_merge_recommendation:")), + "caveats: {:?}", + d.caveats + ); + } + + #[test] + fn legacy_verdict_synonyms_raise_no_unknown_verdict_caveat() { + // The documented pre-2.1 tolerance is a safety net, not a hole: ALLOW and + // HOLD are recognized vocabulary and must never be reported as unknown. + for verdict in ["ALLOW", "HOLD"] { + let dir = tempfile::tempdir().unwrap(); + write_gate( + dir.path(), + &serde_json::json!({ + "bases": ["main"], + "decision": { "verdict": verdict, "allow_merge": verdict == "ALLOW" } + }), + ); + let d = read_decision(dir.path()).unwrap(); + assert!( + !d.caveats.iter().any(|c| c.starts_with("unknown_verdict:")), + "legacy `{verdict}` must stay tolerated: {:?}", + d.caveats + ); + } + } + + #[test] + fn unknown_schema_major_fails_loud() { + let dir = tempfile::tempdir().unwrap(); + write_gate( + dir.path(), + &serde_json::json!({ + "schema_version": "9.0", + "bases": ["main"], + "decision": { "verdict": "PASS", "allow_merge": true } + }), + ); + let err = read_decision(dir.path()).expect_err("unknown major must fail loud"); + assert_eq!(err.class, error_class::STORAGE_CORRUPT); + assert!(err.message.contains("9.0"), "{}", err.message); + } + + #[test] + fn newer_schema_minor_is_tolerated_with_caveat() { + let dir = tempfile::tempdir().unwrap(); + write_gate( + dir.path(), + &serde_json::json!({ + "schema_version": "2.9", + "bases": ["main"], + "decision": { "verdict": "PASS", "merge_recommendation": "approve", "allow_merge": true } + }), + ); + let d = read_decision(dir.path()).expect("newer minor is readable"); + assert_eq!(d.verdict, "PASS"); + assert!( + d.caveats + .iter() + .any(|c| c.starts_with("schema_forward_compat:")), + "caveats: {:?}", + d.caveats + ); + } + + #[test] + fn forward_schema_read_is_marked_normalized() { + // docs/mcp.md: "Anything the adapter could not read is named rather than + // dropped, and every such case sets `normalized: true`" — and it lists + // `schema_forward_compat:` among them. A caveat next to + // `normalized: false` tells the client the decision was passed through + // unchanged while simultaneously admitting fields were ignored. + let dir = tempfile::tempdir().unwrap(); + write_gate( + dir.path(), + &serde_json::json!({ + "schema_version": "2.9", + "bases": ["main"], + "decision": { "verdict": "PASS", "merge_recommendation": "approve", "allow_merge": true } + }), + ); + let d = read_decision(dir.path()).expect("newer minor is readable"); + assert!( + d.caveats + .iter() + .any(|c| c.starts_with("schema_forward_compat:")), + "caveats: {:?}", + d.caveats + ); + assert!( + d.normalized, + "a forward-schema read ignored unknown fields; that is a normalization" + ); + } + + #[test] + fn non_string_schema_version_is_storage_corrupt() { + // Same defect as the CLI reader: `as_str()` turned a present-but- + // wrongly-typed field into "absent", which is the silently-accepted + // legacy path. + for bad in [ + serde_json::json!(2.1), + serde_json::json!(null), + serde_json::json!({ "major": 2 }), + ] { + let dir = tempfile::tempdir().unwrap(); + write_gate( + dir.path(), + &serde_json::json!({ + "schema_version": bad, + "bases": ["main"], + "decision": { "verdict": "PASS", "allow_merge": true } + }), + ); + let err = read_decision(dir.path()).expect_err("non-string schema must fail loud"); + assert_eq!(err.class, error_class::STORAGE_CORRUPT); + assert!( + err.message.contains("schema_version"), + "{} for {bad}", + err.message + ); + } + } + + #[test] + fn a_legacy_root_shaped_pack_is_read_not_called_corrupt() { + // A pack with no `schema_version` predates the field, and reading its + // ROOT as the decision is the documented legacy read-back surface — the + // CLI reader and `docs/contracts/merge_gate.md` both keep it. This + // adapter demanded a nested `decision` object at every version, so the + // one pack shape the contract explicitly tolerates came back + // `storage_corrupt`. + let dir = tempfile::tempdir().unwrap(); + write_gate( + dir.path(), + &serde_json::json!({ "verdict": "ALLOW", "allow_merge": true }), + ); + + let d = read_decision(dir.path()).expect("a legacy root-shaped pack is readable"); + assert_eq!(d.verdict, "PASS"); + assert!(d.allow_merge, "{d:?}"); + } + + #[test] + fn a_non_object_gate_root_is_corrupt_on_both_readers() { + // The legacy root tolerance covers a pack whose decision fields sit at + // the root — not a pack that is an array, a scalar or `null`. Those + // carry no fields to read, and the two readers must agree they are + // corrupt rather than one of them inventing a normalized BLOCK. + for root in ["[1,2,3]", "\"BLOCK\"", "null", "7"] { + let dir = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(dir.path().join("00_summary")).unwrap(); + std::fs::write(dir.path().join("00_summary/MERGE_GATE.json"), root).unwrap(); + + let err = read_decision(dir.path()).expect_err("a non-object gate root is corrupt"); + assert_eq!(err.class, error_class::STORAGE_CORRUPT); + assert!(err.message.contains("not a JSON object"), "{}", err.message); + } + } + + #[test] + fn a_versioned_pack_without_a_decision_object_stays_corrupt() { + // The other half of the same rule: once a pack names its schema, the + // object that schema is built around is mandatory. Reading the root + // there would publish a verdict nothing in the pack stated. + for decision in [None, Some(serde_json::json!("PASS"))] { + let dir = tempfile::tempdir().unwrap(); + let mut gate = serde_json::json!({ + "schema_version": "2.2", + "verdict": "ALLOW", + "allow_merge": true + }); + if let Some(decision) = decision.clone() { + gate["decision"] = decision; + } + write_gate(dir.path(), &gate); + + let err = read_decision(dir.path()) + .expect_err("a versioned pack with no decision object is corrupt"); + assert_eq!(err.class, error_class::STORAGE_CORRUPT); + assert!(err.message.contains("decision"), "{}", err.message); + } + } + + #[test] + fn wrongly_typed_decision_signal_is_reported_not_dropped() { + // `verdict: "PASS"` with `merge_recommendation: 7`: `as_str()` mapped the + // malformed field onto "absent", so the unknown-signal branch never saw + // it and the adapter returned a clean `normalized: false` decision + // derived from the surviving signal — a pack field ignored in silence. + let dir = tempfile::tempdir().unwrap(); + write_gate( + dir.path(), + &serde_json::json!({ + "bases": ["main"], + "decision": { "verdict": "PASS", "merge_recommendation": 7, "allow_merge": true } + }), + ); + + let d = read_decision(dir.path()).expect("one readable signal keeps the pack readable"); + assert!( + d.caveats + .iter() + .any(|c| c.starts_with("unreadable_merge_recommendation:")), + "the ignored field must be named: {:?}", + d.caveats + ); + assert!( + d.normalized, + "a decision that ignored a field is not a passthrough" + ); + } + + #[test] + fn wrongly_typed_allow_merge_is_reported_not_dropped() { + // Same defect on the third signal: a non-boolean `allow_merge` was + // dropped by `as_bool()` and the conservativeness it should have raised + // simply disappeared. + let dir = tempfile::tempdir().unwrap(); + write_gate( + dir.path(), + &serde_json::json!({ + "bases": ["main"], + "decision": { + "verdict": "PASS", + "merge_recommendation": "approve", + "allow_merge": "false" + } + }), + ); + + let d = read_decision(dir.path()).expect("both ranked signals are readable"); + assert!( + d.caveats + .iter() + .any(|c| c.starts_with("unreadable_allow_merge:")), + "the ignored field must be named: {:?}", + d.caveats + ); + assert!(d.normalized); + } + #[test] fn healthy_conditional_core_is_passthrough_no_inconsistency() { // A self-consistent post-PV-03/04 core (CONDITIONAL verdict + diff --git a/src/output/mod.rs b/src/output/mod.rs index c0f6282..694ea72 100644 --- a/src/output/mod.rs +++ b/src/output/mod.rs @@ -55,6 +55,11 @@ pub struct CliJsonSummary { pub artifacts: CliJsonArtifacts, #[serde(skip_serializing_if = "Option::is_none")] pub why_blocked: Option, + /// Reader-side caveats raised while decoding `MERGE_GATE.json` — a verdict + /// this build had to normalize, or a pack schema newer than it understands. + /// Empty (and omitted from the wire) on a clean read. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub caveats: Vec, } #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] @@ -80,6 +85,16 @@ pub struct CliJsonChecksSummary { pub warned: usize, pub skipped: usize, pub cached: usize, + /// Warning-status checks in the artifact pack's canonical check list. + /// + /// `warned` counts only the checks the CLI itself ran. The artifact run + /// appends more — `public_api_diff`, `unsafe_audit`, `ghost_refs`, + /// `heuristics_loctree` — and those reach `MERGE_GATE.json` and the + /// dashboard but never the in-memory `Report`. This is the complete + /// number, so it is always `>= warned`, and it is what + /// `--ci --fail-on-warnings` keys off. + #[serde(default)] + pub warned_in_pack: usize, } #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] @@ -120,6 +135,9 @@ struct MergeGateSummary { allow_merge: bool, quality_pass: bool, reason: Option, + caveats: Vec, + /// Warning-status entries in the pack's canonical `checks[]` list. + warned_checks: usize, } mod duration_serde { @@ -169,12 +187,25 @@ fn failures_degraded_to_advisory(gate: &MergeGateSummary) -> bool { && gate.merge_recommendation == crate::policy::engine::MergeRecommendation::Approve } -pub fn build_cli_json_summary(config: &Config, report: &Report) -> CliJsonSummary { - let gate = read_merge_gate_summary(&report.artifacts_dir) - .unwrap_or_else(|| fallback_merge_gate_summary(config, report)); - let checks_summary = CliJsonChecksSummary::from_checks(&report.checks); - - CliJsonSummary { +/// Build the `--json` / gate summary from the pack's `MERGE_GATE.json`. +/// +/// The gate artifact is the ONLY derivation of the verdict. There is no +/// re-derivation from the in-memory policy engine when the artifact is missing +/// or unparsable: that fallback used to publish `allow_merge = rec != Block`, +/// the single place in the codebase where `allow_merge: true` could coexist with +/// a `CONDITIONAL` verdict, breaking the `allow_merge == (verdict == "PASS")` +/// invariant of `docs/contracts/merge_gate.md`. An unreadable pack is now an +/// execution error (exit 3), not a guess. +pub fn build_cli_json_summary(config: &Config, report: &Report) -> anyhow::Result { + let gate = read_merge_gate_summary(&report.artifacts_dir)?; + let mut checks_summary = CliJsonChecksSummary::from_checks(&report.checks); + // The pack's list is the canonical one; the CLI's own tally is a subset of + // it. Taking the larger keeps a legacy pack (or one whose `checks` this + // build could not read) from reporting FEWER warnings than the CLI already + // knows about. + checks_summary.warned_in_pack = gate.warned_checks.max(checks_summary.warned); + + Ok(CliJsonSummary { schema_version: "cli-json/v1", status: gate .merge_recommendation @@ -207,7 +238,8 @@ pub fn build_cli_json_summary(config: &Config, report: &Report) -> CliJsonSummar } else { None }, - } + caveats: gate.caveats, + }) } /// Process exit code, derived from the merge *recommendation* rather than the @@ -219,39 +251,35 @@ pub fn build_cli_json_summary(config: &Config, report: &Report) -> CliJsonSummar /// variant A — advisory-fail does not force `exit != 0`). /// - CI (`--ci`) is the explicit strict exception: it additionally fails when /// the analysis did not fully pass, matching the documented "strict exit -/// codes" contract of `--ci`. This preserves the historical CI behavior -/// (`block || !quality_pass → 1`) exactly. -pub fn compute_exit_code(summary: &CliJsonSummary) -> i32 { +/// codes" contract of `--ci` (`block || !quality_pass → 1`). +/// - `--ci --fail-on-warnings` is the opt-in escape hatch: warning-level checks +/// no longer break `quality_pass` (a warning is not a failure), so a team that +/// wants a warnings-clean trunk asks for that exit explicitly. It counts the +/// PACK's checks, not the CLI's own list: the signal checks the artifact run +/// generates (`public_api_diff`, `unsafe_audit`, `ghost_refs`, +/// `heuristics_loctree`) warn like any other check, and a flag that promises +/// to fail on any warning cannot be blind to four of them. +/// +/// `strict` is the INVOCATION's answer to "did the caller ask for `--ci`?", not +/// a property read back off the published summary. It used to be derived from +/// `mode.execution_mode == "ci"`, and that label is a preset name, not a +/// strictness flag: `--update` outranks `--ci` when the preset is resolved, so +/// `--ci --fail-on-warnings --update` published `execution_mode: "update"` and +/// silently ran lenient — the flag clap had just insisted on `--ci` for could +/// not fire, and neither could the `!quality_pass` exit `--ci` promises. +pub fn compute_exit_code(summary: &CliJsonSummary, strict: bool, fail_on_warnings: bool) -> i32 { use crate::policy::engine::MergeRecommendation; if summary.merge_recommendation == MergeRecommendation::Block { return 1; } - let strict = summary.mode.execution_mode == "ci"; if strict && !summary.quality_pass { return 1; } - 0 -} - -fn fallback_merge_gate_summary(config: &Config, report: &Report) -> MergeGateSummary { - let engine = crate::policy::engine::PolicyEngine::new(config); - let policy_summary = engine.evaluate_all(&report.checks, &[]); - let quality_pass = !report.has_failures(); - let allow_merge = - policy_summary.merge_recommendation != crate::policy::engine::MergeRecommendation::Block; - - MergeGateSummary { - verdict: policy_summary - .merge_recommendation - .legacy_verdict(policy_summary.analysis_status, quality_pass) - .to_string(), - analysis_status: policy_summary.analysis_status, - merge_recommendation: policy_summary.merge_recommendation, - allow_merge, - quality_pass, - reason: None, + if strict && fail_on_warnings && summary.checks_summary.warned_in_pack > 0 { + return 1; } + 0 } impl CliJsonChecksSummary { @@ -299,19 +327,183 @@ fn existing_relative_path(output_dir: &Path, relative: &str) -> Option { .then(|| relative.to_string()) } -fn read_merge_gate_summary(output_dir: &Path) -> Option { - let raw = - std::fs::read_to_string(output_dir.join("00_summary").join("MERGE_GATE.json")).ok()?; - let value: Value = serde_json::from_str(&raw).ok()?; - let decision = value.get("decision").unwrap_or(&value); - // Canonical verdict vocabulary (PV-03/04): PASS / CONDITIONAL / BLOCK. Legacy - // ALLOW/HOLD tokens from pre-2.1 runs are folded onto the unified set so the - // CLI `--json` surface speaks the same language as MERGE_GATE.json. - let verdict = match decision.get("verdict").and_then(Value::as_str) { - Some("PASS") | Some("ALLOW") => "PASS", - Some("CONDITIONAL") | Some("HOLD") => "CONDITIONAL", - Some("BLOCK") => "BLOCK", - _ => "BLOCK", +/// Decode a pack's `MERGE_GATE.json` into the CLI summary surface. +/// +/// Fail-loud: a missing, unreadable, or unparsable artifact is an error, and so +/// is a pack whose `schema_version` this build does not know. Everything the +/// reader had to normalize instead of read is reported back as a caveat. +fn read_merge_gate_summary(output_dir: &Path) -> anyhow::Result { + use anyhow::Context; + + let gate_path = output_dir.join("00_summary").join("MERGE_GATE.json"); + let raw = std::fs::read_to_string(&gate_path).with_context(|| { + format!( + "cannot read the merge gate artifact {} — the run produced no readable verdict", + gate_path.display() + ) + })?; + let value: Value = serde_json::from_str(&raw).with_context(|| { + format!( + "failed to parse merge gate artifact {}", + gate_path.display() + ) + })?; + + let mut caveats = Vec::new(); + if let Some(caveat) = crate::gate::check_merge_gate_schema_field(value.get("schema_version")) + .with_context(|| format!("merge gate artifact {}", gate_path.display()))? + { + caveats.push(caveat); + } + + // Legacy root-as-decision vs. mandatory `decision` object: one rule, shared + // with the MCP adapter, because the two readers answering it differently is + // exactly how the same pack became readable from one surface and corrupt + // from the other. + let decision = crate::gate::select_decision_object(&value).map_err(|shape| { + anyhow::anyhow!( + "merge gate artifact {} {} — the pack is corrupt and no verdict can be read from it", + gate_path.display(), + shape.describe(), + ) + })?; + // A decision object that states NO decision is not a decision with every + // signal missing — it is the same corrupt pack the other two readers + // already refuse. `tools/validate_merge_gate.py` rejects it for its missing + // required fields, the MCP adapter returns `storage_corrupt`, and + // `prview gate` cannot deserialize it; only this reader used to normalize + // it to BLOCK and publish a summary, so a truncated artifact came back as a + // clean `--ci` exit 1 with a verdict the pack never gave. Presence is the + // test, not recognizability: a stated verdict outside the vocabulary IS a + // decision, and the contract has it collapse to BLOCK with a caveat. + if !["verdict", "merge_recommendation", "allow_merge"] + .iter() + .any(|field| decision.get(*field).is_some()) + { + anyhow::bail!( + "merge gate artifact {} states no verdict, merge_recommendation or allow_merge — \ + the pack is corrupt and no verdict can be read from it", + gate_path.display(), + ); + } + + // A decision signal present with the WRONG JSON type is not an absent one. + // Reading each through `as_str()` / `as_bool()` collapsed the two, and + // "absent" is the state this reader forgives: `merge_recommendation: 7` + // became "no recommendation", the fallback below then reconstructed + // `Approve` from `allow_merge`, and `--ci` exited 0 on a pack whose + // decision this reader had silently failed to read. The MCP adapter has + // named such a field since the unreadable-signal contract landed; the CLI + // half of that contract was documented but never implemented. + let mut unreadable = Vec::new(); + let raw_verdict = crate::gate::readable_signal( + "verdict", + decision.get("verdict"), + crate::gate::JsonKind::String, + &mut unreadable, + ) + .and_then(Value::as_str); + let raw_allow_merge = crate::gate::readable_signal( + "allow_merge", + decision.get("allow_merge"), + crate::gate::JsonKind::Boolean, + &mut unreadable, + ) + .and_then(Value::as_bool); + let raw_recommendation = crate::gate::readable_signal( + "merge_recommendation", + decision.get("merge_recommendation"), + crate::gate::JsonKind::String, + &mut unreadable, + ) + .and_then(Value::as_str); + // Read as an OPTION, not through `unwrap_or(false)`: the reconciliation + // below has to tell a pack that states a failed quality axis from one + // written before the field existed. It goes through `readable_signal` for + // the third case those two hide between them — a `quality_pass` that is + // PRESENT but not a boolean. Reading it with a bare `as_bool()` made + // `"false"` indistinguishable from absent and published an approval with no + // caveat at all; a stated-but-unreadable axis now normalizes to BLOCK like + // every other one. + let raw_quality_pass = crate::gate::readable_signal( + "quality_pass", + decision.get("quality_pass"), + crate::gate::JsonKind::Boolean, + &mut unreadable, + ) + .and_then(Value::as_bool); + // The confidence axis. The contract permits `PASS` only when the analysis is + // `complete`, so this ranks beside the policy axes instead of being read + // afterwards for display — which is what let an `incomplete` run publish a + // clean approval. + let raw_analysis_status = crate::gate::readable_signal( + "analysis_status", + decision.get("analysis_status"), + crate::gate::JsonKind::String, + &mut unreadable, + ) + .and_then(Value::as_str); + // The blocker axis, stated twice by the emitter: + // `policy_allow_merge = blocking_issues.is_empty()`. A pack may carry either + // or both, so both are read; agreeing on the same rank costs nothing and a + // pack that states only one is still covered. + let raw_policy_allow_merge = crate::gate::readable_signal( + "policy_allow_merge", + decision.get("policy_allow_merge"), + crate::gate::JsonKind::Boolean, + &mut unreadable, + ) + .and_then(Value::as_bool); + let raw_blocking_issues = crate::gate::readable_signal( + "blocking_issues", + decision.get("blocking_issues"), + crate::gate::JsonKind::Array, + &mut unreadable, + ) + .and_then(Value::as_array); + // Whether the verdict below is what the pack said or what this reader had to + // substitute for it. A substituted verdict cannot leave the OTHER decision + // axes reading whatever the same unreliable decision block claimed: that + // published `verdict: "BLOCK"` beside `allow_merge: true` and an `approve` + // recommendation, breaking the `allow_merge == (verdict == "PASS")` + // invariant and letting `compute_exit_code` exit 0 on a BLOCK. An ignored + // signal anywhere in the block earns the same treatment: a decision derived + // from a partly unread block is not a decision this reader may publish as + // permissive. + let mut normalized_to_block = !unreadable.is_empty(); + let verdict_is_mistyped = decision.get("verdict").is_some() && raw_verdict.is_none(); + caveats.append(&mut unreadable); + // The vocabulary itself lives in `gate::canonical_verdict`, shared with the + // MCP adapter. This reader owning a second copy of it is exactly how the + // two surfaces came to read one pack two ways. + let verdict = match raw_verdict.map(|raw| (raw, crate::gate::canonical_verdict(raw))) { + Some((_, Some(canonical))) => canonical, + // Collapsing an unreadable verdict to BLOCK is the safe default, but it + // is a normalization, not a reading — say so instead of letting the + // caller mistake it for what the pack claimed. + Some((other, None)) => { + caveats.push(format!( + "unknown_verdict: MERGE_GATE.json verdict `{other}` is not in the \ + PASS/CONDITIONAL/BLOCK vocabulary; normalized to BLOCK" + )); + normalized_to_block = true; + "BLOCK" + } + None => { + // A verdict that IS there but could not be typed has already been + // named by its `unreadable_verdict:` caveat; saying the decision + // "carries no verdict" on top of that would be a second, false + // claim about the same field. + if !verdict_is_mistyped { + caveats.push( + "unknown_verdict: MERGE_GATE.json decision carries no `verdict`; \ + normalized to BLOCK" + .to_string(), + ); + } + normalized_to_block = true; + "BLOCK" + } }; let reason = decision @@ -320,36 +512,251 @@ fn read_merge_gate_summary(output_dir: &Path) -> Option { .and_then(Value::as_str) .map(|s| s.to_string()); - let allow_merge = decision - .get("allow_merge") - .and_then(Value::as_bool) - .unwrap_or(false); - let quality_pass = decision - .get("quality_pass") - .and_then(Value::as_bool) - .unwrap_or(false); + // An unrecognized recommendation is not an absent one either. It cannot rank, + // so it drops out of the reconciliation below — but it is named, exactly as + // the MCP adapter names it, instead of vanishing into a confident surface + // derived from the remaining signals. + let recommendation_rank = raw_recommendation.and_then(crate::gate::rank_from_merge_rec); + if let Some(raw) = raw_recommendation + && recommendation_rank.is_none() + { + caveats.push(format!( + "unknown_merge_recommendation: MERGE_GATE.json merge_recommendation `{raw}` is not in \ + the approve/review_required/block vocabulary; it was ignored when deriving this \ + decision" + )); + } + + // Conservativeness reconciliation, shared with the MCP adapter through + // `gate::rank_from_*`: the most conservative axis the pack states wins, and + // every axis is then published from that one rank. Believing each field in + // turn let a pack that says `verdict: "BLOCK"` beside + // `merge_recommendation: "approve"` publish an approval — and, because + // `compute_exit_code` keys off the recommendation, exit 0 on a gate whose + // own canonical artifact said BLOCK. + let allow_rank = raw_allow_merge.map(|allow| if allow { 1 } else { 2 }); + // `quality_pass` is a decision axis too, and only its FALSE is informative. + // `false` says "not a PASS" — the contract permits `PASS` only when quality + // passes — so it ranks 2, exactly like `allow_merge: false`. `true` states + // no rank: a quality-clean run is still held at CONDITIONAL by a + // breaking-change escalation, so reading it as an assertion that the gate + // passed would let one axis soften a verdict two others agree on. Absence + // states nothing either — that is the shape of a pack written before the + // field, and defaulting it to `false` would turn every one of them + // CONDITIONAL. + let quality_rank = match raw_quality_pass { + Some(false) => Some(2), + _ => None, + }; + // The confidence axis, ranked by the same rule: only the values that RULE + // OUT a more permissive outcome state a rank. `complete` rules nothing out + // — it is a precondition of `PASS`, not a grant of it — so like + // `quality_pass: true` it stays silent. + let analysis_rank = raw_analysis_status.and_then(crate::gate::rank_from_analysis_status); + if let Some(raw) = raw_analysis_status + && !crate::gate::known_analysis_status(raw) + { + caveats.push(format!( + "unknown_analysis_status: MERGE_GATE.json analysis_status `{raw}` is not in the \ + complete/degraded/incomplete vocabulary; it was ignored when deriving this decision" + )); + } + // The blocker axis. `blocking_issues` is non-empty only when a check reached + // `PolicyConclusion::Blocked`, whose `merge_impact` is `Block`, so a stated + // blocker is a stated BLOCK — rank 3. `policy_allow_merge: false` is the + // same fact by its own definition. Neither states anything in the permissive + // direction: an empty list and `policy_allow_merge: true` mean only "policy + // did not hard-block", which the contract is explicit is NOT the same as + // `allow_merge`. + let blocker_rank = (raw_policy_allow_merge == Some(false) + || raw_blocking_issues.is_some_and(|issues| !issues.is_empty())) + .then_some(3); + let stated_ranks: Vec = [ + crate::gate::rank_from_verdict(verdict), + recommendation_rank, + allow_rank, + quality_rank, + analysis_rank, + blocker_rank, + ] + .into_iter() + .flatten() + .collect(); + let final_rank = if normalized_to_block { + 3 + } else { + stated_ranks.iter().copied().max().unwrap_or(3) + }; + // Only the PACK's own axes can be inconsistent with each other. A verdict + // this reader had to substitute is already named by its own caveat, and + // calling the substitution an inconsistency would blame the artifact for + // the reader's normalization. + // + // `allow_merge` is deliberately NOT one of the axes compared here, though it + // still raises the rank above. `false` says "not a PASS" — it is `>= 2`, not + // `== 2`, and cannot reach 3 at all — so comparing it as an exact rank + // called every healthy BLOCK pack (`verdict: "BLOCK"`, + // `merge_recommendation: "block"`, `allow_merge: false`) inconsistent with + // itself, which is every BLOCK pack this tool writes. It contradicts the + // decision only when the derived `allow_merge` disagrees with the stated + // one, which is the test the MCP adapter already used. + // + // `quality_pass` needs no test of its own. It ranks 2 when false, so any + // axis claiming 1 beside it already disagrees with the winning rank and + // fires above; and a healthy BLOCK or CONDITIONAL pack states + // `quality_pass: false` in agreement with everything else. A separate + // `quality_pass == Some(false) && final_rank == 1` guard would be + // unreachable — the rank it contributes is what makes `final_rank == 1` + // impossible. It is still NAMED in the caveat, so a reader can see which + // axis forced the downgrade. + let textual_ranks = [crate::gate::rank_from_verdict(verdict), recommendation_rank]; + let axes_disagree = textual_ranks + .iter() + .flatten() + .any(|rank| *rank != final_rank) + || raw_allow_merge.is_some_and(|allow| allow != (final_rank == 1)); + if !normalized_to_block && axes_disagree { + caveats.push(format!( + "core_inconsistency: MERGE_GATE.json states verdict={verdict}, \ + merge_recommendation={}, allow_merge={}, quality_pass={}, analysis_status={}, \ + blocking_issues={}, policy_allow_merge={}; the most conservative signal wins", + raw_recommendation.unwrap_or("null"), + raw_allow_merge + .map(|b| b.to_string()) + .unwrap_or_else(|| "null".to_string()), + raw_quality_pass + .map(|b| b.to_string()) + .unwrap_or_else(|| "null".to_string()), + raw_analysis_status.unwrap_or("null"), + raw_blocking_issues + .map(|issues| issues.len().to_string()) + .unwrap_or_else(|| "null".to_string()), + raw_policy_allow_merge + .map(|b| b.to_string()) + .unwrap_or_else(|| "null".to_string()), + )); + } + + let verdict = crate::gate::verdict_from_rank(final_rank); + let allow_merge = final_rank == 1; + let merge_recommendation = match crate::gate::merge_rec_from_rank(final_rank) { + "approve" => crate::policy::engine::MergeRecommendation::Approve, + "review_required" => crate::policy::engine::MergeRecommendation::ReviewRequired, + _ => crate::policy::engine::MergeRecommendation::Block, + }; - let analysis_status = match decision.get("analysis_status").and_then(Value::as_str) { + // Ranking and PUBLISHING are different questions about the same absent + // field. Absence states no rank — that is what keeps a pre-`quality_pass` + // pack a PASS — but the summary still has to answer "did quality pass", and + // answering `false` asserted a failure the pack never claimed: it derived + // `analysis_status: Incomplete` from that, and `--ci` exited 1 on the same + // artifact the MCP adapter approved. + // + // So an absent axis is derived from the RECONCILED outcome instead. The + // contract permits `PASS` only when quality passes, so a reconciled `PASS` + // implies it; anything held below `PASS` implies nothing about quality + // specifically and stays conservative. This cannot launder a MISTYPED + // value: an unreadable signal normalizes the whole decision to `BLOCK`, so + // `allow_merge` is already false by the time it is read here. + let quality_pass = raw_quality_pass.unwrap_or(allow_merge); + + // Reuses the value typed above, so a mistyped `analysis_status` reaches this + // fallback as absent rather than being read a second time by a laxer rule. + // Its absent case follows the same rule: a reconciled `PASS` requires a + // complete analysis, and nothing below `PASS` implies one. + let analysis_status = match raw_analysis_status { Some("complete") => crate::policy::engine::AnalysisStatus::Complete, Some("degraded") => crate::policy::engine::AnalysisStatus::Degraded, Some("incomplete") => crate::policy::engine::AnalysisStatus::Incomplete, _ if allow_merge && quality_pass => crate::policy::engine::AnalysisStatus::Complete, _ => crate::policy::engine::AnalysisStatus::Incomplete, }; - let merge_recommendation = match decision.get("merge_recommendation").and_then(Value::as_str) { - Some("approve") => crate::policy::engine::MergeRecommendation::Approve, - Some("review_required") => crate::policy::engine::MergeRecommendation::ReviewRequired, - _ if allow_merge => crate::policy::engine::MergeRecommendation::Approve, - _ if verdict == "CONDITIONAL" => crate::policy::engine::MergeRecommendation::ReviewRequired, - _ => crate::policy::engine::MergeRecommendation::Block, + // `checks[]` sits at the pack ROOT, beside `decision`, and it is the only + // complete list of what ran: the artifact stage appends its own signal + // checks (`public_api_diff`, `unsafe_audit`, `ghost_refs`, + // `heuristics_loctree`) to the list the gate is built from, and none of + // them ever reaches the in-memory `Report` the CLI tallies. + // + // A status is matched against the vocabulary the writer emits, not against + // the single string `"warnings"`. Anything else is UNREADABLE, not clean: + // `"WARNINGS"` from another writer — or a stale pack `--update` reused + // unchanged — used to count as "not a warning", so `--ci + // --fail-on-warnings` exited 0 on an artifact whose warning signal this + // reader could not read. It is the same rule the decision axes follow: a + // present-but-untypeable signal normalizes conservatively and is reported, + // while an ABSENT one may legitimately mean a legacy pack. Case is not + // folded, deliberately — normalizing `"WARNINGS"` into a warning silently + // would hide that the pack is off-contract, and the tally is the same + // either way. + let warned_checks = match value.get("checks") { + Some(Value::Array(entries)) => { + let mut warned = 0usize; + let mut unreadable: Vec = Vec::new(); + for (index, entry) in entries.iter().enumerate() { + match entry.get("status").and_then(Value::as_str) { + Some("warnings") => warned += 1, + Some(status) if crate::checks::CheckStatus::EMITTED.contains(&status) => {} + _ => { + warned += 1; + unreadable.push( + entry + .get("id") + .and_then(Value::as_str) + .map(str::to_string) + .unwrap_or_else(|| format!("checks[{index}]")), + ); + } + } + } + if !unreadable.is_empty() { + caveats.push(format!( + "unreadable_check_status: MERGE_GATE.json states a status outside the emitted \ + vocabulary ({}) for {}; each one counts toward the warning tally", + crate::checks::CheckStatus::EMITTED.join(", "), + unreadable.join(", ") + )); + } + warned + } + // The same rule one level up, on the container instead of an entry. This + // used to fall back to "the checks this run executed", which on an + // unchanged `--update` run is none at all: `--ci --fail-on-warnings` + // exited 0 on a pack whose warning list the reader could not read. An + // unreadable list is not an empty one, so it counts as at least one + // warning. No legacy carve-out applies — `checks` has been emitted since + // schema 1.0 and the contract validator has always required an array + // there, so a non-array was never a valid shape. + Some(other) => { + caveats.push(format!( + "unreadable_checks: MERGE_GATE.json checks is {}, not an array; the warning tally \ + cannot be read and counts as at least one warning", + match other { + Value::Null => "null", + Value::Bool(_) => "a boolean", + Value::Number(_) => "a number", + Value::String(_) => "a string", + Value::Object(_) => "an object", + Value::Array(_) => unreachable!("matched above"), + } + )); + 1 + } + // ABSENT is the one tolerant case, and stays so: a pack that states no + // list may simply predate this build, and the CLI's own tally still + // applies through the `max` at the call site. Absent is not the same + // question as present-but-unreadable. + None => 0, }; - Some(MergeGateSummary { + + Ok(MergeGateSummary { verdict: verdict.to_string(), analysis_status, merge_recommendation, allow_merge, quality_pass, reason, + caveats, + warned_checks, }) } @@ -858,7 +1265,7 @@ pub fn print_summary(report: &Report) { } let gate = read_merge_gate_summary(&report.artifacts_dir); - if let Some(heading) = failure_summary_heading(report, gate.as_ref()) { + if let Some(heading) = failure_summary_heading(report, gate.as_ref().ok()) { println!(); println!("{} {heading}", "⚠".yellow()); for check in &report.checks { @@ -875,24 +1282,32 @@ pub fn print_summary(report: &Report) { // Final authoritative line: the merge-gate verdict, in the same vocabulary // as `--json` (PV-03). Never print a bare "all checks passed" that could // contradict a BLOCK/CONDITIONAL gate — the stdout summary must not lie. - if let Some(gate) = gate { - println!(); - let (icon, label) = match gate.verdict.as_str() { - "PASS" => ("✓".green(), "PASS".green().bold()), - "BLOCK" => ("🛑".red(), "BLOCK".red().bold()), - _ => ("⚠".yellow(), "CONDITIONAL".yellow().bold()), - }; - match gate.reason.as_deref() { - Some(reason) if !reason.trim().is_empty() => { - println!("{icon} Verdict: {label} — {reason}"); + match gate { + Ok(gate) => { + println!(); + let (icon, label) = match gate.verdict.as_str() { + "PASS" => ("✓".green(), "PASS".green().bold()), + "BLOCK" => ("🛑".red(), "BLOCK".red().bold()), + _ => ("⚠".yellow(), "CONDITIONAL".yellow().bold()), + }; + match gate.reason.as_deref() { + Some(reason) if !reason.trim().is_empty() => { + println!("{icon} Verdict: {label} — {reason}"); + } + _ => println!("{icon} Verdict: {label}"), + } + for caveat in &gate.caveats { + println!(" {} {caveat}", "⚠".yellow()); } - _ => println!("{icon} Verdict: {label}"), } - } else if !report.checks.is_empty() && !report.has_failures() { - // No gate artifact for this run (degenerate/minimal path) — fall back - // to the raw check tally rather than inventing a verdict. - println!(); - println!("{} All checks passed!", "✓".green()); + Err(err) => { + // No readable gate artifact. The raw check tally is NOT a verdict — + // announcing "all checks passed" here is exactly the guess this + // path used to make. Name the missing truth instead; the process + // exits 3 on the same condition. + println!(); + println!("{} No merge-gate verdict for this run: {err}", "⚠".yellow()); + } } println!(); @@ -919,6 +1334,41 @@ mod tests { use crate::cli::ExecutionMode; use crate::config::{test_config, test_rust_profile}; + /// Minimal artifact pack carrying one `MERGE_GATE.json` decision. The gate + /// artifact is now the ONLY source of the verdict, so every summary test + /// has to plant one instead of leaning on a re-derivation fallback. + /// Exit code the CLI would return for a pack, with no check of its own to + /// contribute — so the code reflects the gate artifact alone. + fn exit_code_for(pack: &tempfile::TempDir, strict: bool) -> i32 { + let mut config = test_config(); + if strict { + config.execution_mode = ExecutionMode::Ci; + } + let report = Report { + target: "feature/legacy".to_string(), + bases: vec!["main".to_string()], + diffs: vec![], + checks: vec![], + heuristics: None, + artifacts_dir: pack.path().to_path_buf(), + duration: Duration::from_secs(1), + unchanged: false, + }; + let summary = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); + compute_exit_code(&summary, strict, false) + } + + fn pack_with_gate(decision: &str) -> tempfile::TempDir { + let temp = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(temp.path().join("00_summary")).unwrap(); + std::fs::write( + temp.path().join("00_summary/MERGE_GATE.json"), + format!(r#"{{"schema_version":"2.1","decision":{decision}}}"#), + ) + .unwrap(); + temp + } + #[test] fn config_box_inner_width_fits_longest_line_and_title() { let rows = vec![ @@ -1272,7 +1722,7 @@ mod tests { unchanged: false, }; - let summary = build_cli_json_summary(&config, &report); + let summary = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); let value = serde_json::to_value(&summary).unwrap(); assert_eq!(summary.schema_version, "cli-json/v1"); @@ -1294,6 +1744,7 @@ mod tests { warned: 0, skipped: 0, cached: 1, + warned_in_pack: 0, } ); assert_eq!(summary.top_failures.len(), 2); @@ -1330,6 +1781,11 @@ mod tests { #[test] fn test_cli_json_summary_marks_warning_runs_without_failures() { let config = test_config(); + let pack = pack_with_gate( + r#"{"verdict":"CONDITIONAL","analysis_status":"degraded", + "merge_recommendation":"review_required","allow_merge":false, + "quality_pass":true}"#, + ); let report = Report { target: "main".to_string(), bases: vec!["develop".to_string()], @@ -1343,12 +1799,12 @@ mod tests { provenance: None, }], heuristics: None, - artifacts_dir: PathBuf::from("."), + artifacts_dir: pack.path().to_path_buf(), duration: Duration::from_secs(1), unchanged: false, }; - let summary = build_cli_json_summary(&config, &report); + let summary = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); assert_eq!(summary.verdict, "CONDITIONAL"); assert_eq!(summary.status, "fail"); @@ -1389,9 +1845,9 @@ mod tests { unchanged: false, }; - let summary = build_cli_json_summary(&config, &report); + let summary = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); assert_eq!(summary.status, "ok"); - assert_eq!(compute_exit_code(&summary), 0); + assert_eq!(compute_exit_code(&summary, false, false), 0); } #[test] @@ -1400,6 +1856,11 @@ mod tests { // failure is a review-required advisory — the status is still "fail", // but the process exits 0 because only a hard Block fails a non-CI run. let config = test_config(); + let pack = pack_with_gate( + r#"{"verdict":"CONDITIONAL","analysis_status":"complete", + "merge_recommendation":"review_required","allow_merge":false, + "quality_pass":false}"#, + ); let report = Report { target: "feature/broken".to_string(), bases: vec!["main".to_string()], @@ -1413,18 +1874,18 @@ mod tests { provenance: None, }], heuristics: None, - artifacts_dir: PathBuf::from("."), + artifacts_dir: pack.path().to_path_buf(), duration: Duration::from_secs(1), unchanged: false, }; - let summary = build_cli_json_summary(&config, &report); + let summary = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); assert_eq!(summary.status, "fail"); assert_eq!( summary.merge_recommendation, crate::policy::engine::MergeRecommendation::ReviewRequired ); - assert_eq!(compute_exit_code(&summary), 0); + assert_eq!(compute_exit_code(&summary, false, false), 0); } #[test] @@ -1433,6 +1894,11 @@ mod tests { // run tolerates fails the process under --ci. let mut config = test_config(); config.execution_mode = ExecutionMode::Ci; + let pack = pack_with_gate( + r#"{"verdict":"CONDITIONAL","analysis_status":"complete", + "merge_recommendation":"review_required","allow_merge":false, + "quality_pass":false}"#, + ); let report = Report { target: "feature/broken".to_string(), bases: vec!["main".to_string()], @@ -1446,14 +1912,133 @@ mod tests { provenance: None, }], heuristics: None, - artifacts_dir: PathBuf::from("."), + artifacts_dir: pack.path().to_path_buf(), + duration: Duration::from_secs(1), + unchanged: false, + }; + + let summary = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); + assert_eq!(summary.mode.execution_mode, "ci"); + assert_eq!(compute_exit_code(&summary, true, false), 1); + } + + #[test] + fn test_exit_code_ci_warnings_only_passes_unless_opted_in() { + // Warning→failure P0: a run whose only signal is a warning-level check + // has `quality_pass == true`, so --ci exits 0. `--fail-on-warnings` is + // the opt-in escape hatch that restores the old exit 1. + let mut config = test_config(); + config.execution_mode = ExecutionMode::Ci; + let temp = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(temp.path().join("00_summary")).unwrap(); + std::fs::write( + temp.path().join("00_summary/MERGE_GATE.json"), + r#"{"decision":{"verdict":"CONDITIONAL","merge_recommendation":"review_required","allow_merge":false,"quality_pass":true}}"#, + ) + .unwrap(); + + let report = Report { + target: "feature/warnings-only".to_string(), + bases: vec!["main".to_string()], + diffs: vec![], + checks: vec![CheckResult { + name: "Cargo audit".to_string(), + status: CheckStatus::Warnings, + duration: Duration::from_secs(1), + output: "1 unmaintained crate".to_string(), + cached: false, + provenance: None, + }], + heuristics: None, + artifacts_dir: temp.path().to_path_buf(), duration: Duration::from_secs(1), unchanged: false, }; - let summary = build_cli_json_summary(&config, &report); + let summary = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); assert_eq!(summary.mode.execution_mode, "ci"); - assert_eq!(compute_exit_code(&summary), 1); + assert_eq!(summary.checks_summary.warned, 1); + assert!(summary.quality_pass); + assert_eq!(compute_exit_code(&summary, true, false), 0); + assert_eq!(compute_exit_code(&summary, true, true), 1); + } + + #[test] + fn test_fail_on_warnings_is_scoped_to_ci() { + // Outside --ci the exit stays derived from the merge recommendation + // alone; the escape hatch never silently hardens a plain local run. + let config = test_config(); + let temp = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(temp.path().join("00_summary")).unwrap(); + std::fs::write( + temp.path().join("00_summary/MERGE_GATE.json"), + r#"{"decision":{"verdict":"CONDITIONAL","merge_recommendation":"review_required","allow_merge":false,"quality_pass":true}}"#, + ) + .unwrap(); + let report = Report { + target: "feature/warnings-only".to_string(), + bases: vec!["main".to_string()], + diffs: vec![], + checks: vec![CheckResult { + name: "Cargo audit".to_string(), + status: CheckStatus::Warnings, + duration: Duration::from_secs(1), + output: "1 unmaintained crate".to_string(), + cached: false, + provenance: None, + }], + heuristics: None, + artifacts_dir: temp.path().to_path_buf(), + duration: Duration::from_secs(1), + unchanged: false, + }; + + let summary = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); + assert_ne!(summary.mode.execution_mode, "ci"); + assert_eq!(compute_exit_code(&summary, false, true), 0); + } + + #[test] + fn the_preset_label_does_not_decide_ci_strictness() { + // `--update` outranks `--ci` when the preset is resolved, so a + // `--ci --fail-on-warnings --update` run publishes + // `execution_mode: "update"`. Deriving strictness from that label made + // the flag clap had just insisted on `--ci` for silently inert, and took + // the `!quality_pass` exit `--ci` promises down with it. + let mut config = test_config(); + config.execution_mode = ExecutionMode::Update; + let temp = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(temp.path().join("00_summary")).unwrap(); + std::fs::write( + temp.path().join("00_summary/MERGE_GATE.json"), + r#"{"decision":{"verdict":"CONDITIONAL","merge_recommendation":"review_required","allow_merge":false,"quality_pass":true}, + "checks":[{"id":"rustfmt","status":"warnings"}]}"#, + ) + .unwrap(); + let report = Report { + target: "feature/update-strictness".to_string(), + bases: vec!["main".to_string()], + diffs: vec![], + checks: vec![], + heuristics: None, + artifacts_dir: temp.path().to_path_buf(), + duration: Duration::from_secs(1), + unchanged: true, + }; + + let summary = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); + assert_eq!(summary.mode.execution_mode, "update"); + assert_eq!(summary.checks_summary.warned_in_pack, 1); + assert_eq!( + compute_exit_code(&summary, true, true), + 1, + "the caller asked for --ci, so the pack's warning fails the run" + ); + assert_eq!( + compute_exit_code(&summary, false, true), + 0, + "without --ci the escape hatch stays inert, preset or not" + ); } #[test] @@ -1479,17 +2064,21 @@ mod tests { unchanged: false, }; - let summary = build_cli_json_summary(&config, &report); + let summary = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); assert_eq!( summary.merge_recommendation, crate::policy::engine::MergeRecommendation::Block ); - assert_eq!(compute_exit_code(&summary), 1); + assert_eq!(compute_exit_code(&summary, false, false), 1); } #[test] fn test_cli_json_summary_canonicalizes_cargo_audit_failure_summary() { let config = test_config(); + let pack = pack_with_gate( + r#"{"verdict":"BLOCK","analysis_status":"complete", + "merge_recommendation":"block","allow_merge":false,"quality_pass":false}"#, + ); let report = Report { target: "feature/security".to_string(), bases: vec!["main".to_string()], @@ -1537,12 +2126,12 @@ mod tests { provenance: None, }], heuristics: None, - artifacts_dir: PathBuf::from("."), + artifacts_dir: pack.path().to_path_buf(), duration: Duration::from_secs(1), unchanged: false, }; - let summary = build_cli_json_summary(&config, &report); + let summary = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); let failure = &summary.top_failures[0]; assert_eq!(failure.id, "cargo_audit"); @@ -1556,6 +2145,10 @@ mod tests { #[test] fn test_cli_json_summary_canonicalizes_semgrep_failure_summary() { let config = test_config(); + let pack = pack_with_gate( + r#"{"verdict":"BLOCK","analysis_status":"complete", + "merge_recommendation":"block","allow_merge":false,"quality_pass":false}"#, + ); let report = Report { target: "feature/security".to_string(), bases: vec!["main".to_string()], @@ -1576,12 +2169,12 @@ api-router/app/core/cache.py provenance: None, }], heuristics: None, - artifacts_dir: PathBuf::from("."), + artifacts_dir: pack.path().to_path_buf(), duration: Duration::from_secs(1), unchanged: false, }; - let summary = build_cli_json_summary(&config, &report); + let summary = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); let failure = &summary.top_failures[0]; assert_eq!(failure.id, "semgrep_scan"); @@ -1615,6 +2208,1445 @@ api-router/app/core/cache.py ); } + #[test] + fn missing_merge_gate_is_an_error_not_a_re_derived_verdict() { + // The removed `fallback_merge_gate_summary` published + // `allow_merge = rec != Block`, so a re-derived CONDITIONAL run came back + // with `allow_merge: true` — the one place the + // `allow_merge == (verdict == "PASS")` invariant could be violated. An + // unreadable pack must now fail loud instead. + let config = test_config(); + let empty = tempfile::tempdir().unwrap(); + let report = Report { + target: "feature/no-gate".to_string(), + bases: vec!["main".to_string()], + diffs: vec![], + checks: vec![CheckResult { + name: "cargo test".to_string(), + status: CheckStatus::Failed, + duration: Duration::from_secs(1), + output: "failed".to_string(), + cached: false, + provenance: None, + }], + heuristics: None, + artifacts_dir: empty.path().to_path_buf(), + duration: Duration::from_secs(1), + unchanged: false, + }; + + let err = build_cli_json_summary(&config, &report) + .expect_err("a pack with no MERGE_GATE.json carries no verdict"); + assert!( + format!("{err:#}").contains("MERGE_GATE.json"), + "error must name the missing artifact: {err:#}" + ); + } + + #[test] + fn unparsable_merge_gate_is_an_error() { + let config = test_config(); + let temp = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(temp.path().join("00_summary")).unwrap(); + std::fs::write(temp.path().join("00_summary/MERGE_GATE.json"), "{not json").unwrap(); + let report = Report { + target: "feature/corrupt".to_string(), + bases: vec!["main".to_string()], + diffs: vec![], + checks: vec![], + heuristics: None, + artifacts_dir: temp.path().to_path_buf(), + duration: Duration::from_secs(1), + unchanged: false, + }; + + assert!(build_cli_json_summary(&config, &report).is_err()); + } + + #[test] + fn unknown_verdict_collapses_to_block_with_an_explicit_caveat() { + // Collapsing to BLOCK is safe, but the reader must say it normalized + // rather than let the caller read it as the pack's own verdict. + let pack = pack_with_gate(r#"{"verdict":"PROBABLY","allow_merge":false}"#); + let gate = read_merge_gate_summary(pack.path()).expect("gate is readable"); + + assert_eq!(gate.verdict, "BLOCK"); + let caveat = gate + .caveats + .iter() + .find(|c| c.starts_with("unknown_verdict:")) + .expect("unknown_verdict caveat present"); + assert!(caveat.contains("PROBABLY"), "{caveat}"); + } + + #[test] + fn legacy_verdict_synonyms_are_folded_without_a_caveat() { + // `allow_merge` matches each verdict here on purpose: `ALLOW` means a + // clean pass, so pairing it with `allow_merge: false` would be a + // contradictory pack, and a contradictory pack is supposed to earn a + // `core_inconsistency` caveat. What this test pins is the synonym + // folding, not the reconciliation. + for (legacy, unified, allow) in [("ALLOW", "PASS", true), ("HOLD", "CONDITIONAL", false)] { + let pack = pack_with_gate(&format!( + r#"{{"verdict":"{legacy}","allow_merge":{allow}}}"# + )); + let gate = read_merge_gate_summary(pack.path()).expect("gate is readable"); + assert_eq!(gate.verdict, unified); + assert!( + gate.caveats.is_empty(), + "legacy `{legacy}` is recognized vocabulary: {:?}", + gate.caveats + ); + } + } + + #[test] + fn both_readers_agree_on_every_verdict_spelling() { + // The two surfaces used to carry two vocabularies for one field: the CLI + // matched the raw string case-sensitively while the MCP adapter ranked + // it through an ASCII-uppercase fold. A pack saying `verdict: "pass"` + // was therefore a clean PASS to MCP automation and an unknown verdict + // normalized to BLOCK on the CLI — the same artifact, approved by one + // reader and rejected by the other. `APPROVE` diverged the same way, + // case aside: it ranked as a pass but was not in the CLI's fold. + for (spelling, expected) in [ + ("PASS", "PASS"), + ("pass", "PASS"), + ("Pass", "PASS"), + ("ALLOW", "PASS"), + ("allow", "PASS"), + ("APPROVE", "PASS"), + ("CONDITIONAL", "CONDITIONAL"), + ("conditional", "CONDITIONAL"), + ("HOLD", "CONDITIONAL"), + ("hold", "CONDITIONAL"), + ("BLOCK", "BLOCK"), + ("block", "BLOCK"), + ] { + let pack = pack_with_gate(&format!( + r#"{{"verdict":"{spelling}","merge_recommendation":"{rec}", + "allow_merge":{allow},"quality_pass":true, + "analysis_status":"complete"}}"#, + rec = match expected { + "PASS" => "approve", + "CONDITIONAL" => "review_required", + _ => "block", + }, + allow = expected == "PASS", + )); + + let cli = read_merge_gate_summary(pack.path()).expect("gate is readable"); + let mcp = crate::mcp::read::read_decision(pack.path()).expect("gate is readable"); + + assert_eq!( + cli.verdict, expected, + "CLI read `{spelling}` as {}: {:?}", + cli.verdict, cli.caveats + ); + assert_eq!( + mcp.verdict, expected, + "MCP read `{spelling}` as {}", + mcp.verdict + ); + assert_eq!( + cli.verdict, mcp.verdict, + "the two readers must not disagree about `{spelling}`" + ); + assert!( + !cli.caveats + .iter() + .any(|c| c.starts_with("unknown_verdict:")), + "`{spelling}` is recognized vocabulary, not an unknown verdict: {:?}", + cli.caveats + ); + } + } + + #[test] + fn unknown_schema_major_fails_loud_and_newer_minor_only_caveats() { + let temp = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(temp.path().join("00_summary")).unwrap(); + std::fs::write( + temp.path().join("00_summary/MERGE_GATE.json"), + r#"{"schema_version":"9.0","decision":{"verdict":"PASS","allow_merge":true}}"#, + ) + .unwrap(); + let err = read_merge_gate_summary(temp.path()).expect_err("unknown major must fail loud"); + assert!(format!("{err:#}").contains("9.0"), "{err:#}"); + + std::fs::write( + temp.path().join("00_summary/MERGE_GATE.json"), + r#"{"schema_version":"2.9","decision":{"verdict":"PASS","allow_merge":true,"quality_pass":true}}"#, + ) + .unwrap(); + let gate = read_merge_gate_summary(temp.path()).expect("newer minor is readable"); + assert_eq!(gate.verdict, "PASS"); + assert!( + gate.caveats + .iter() + .any(|c| c.starts_with("schema_forward_compat:")), + "caveats: {:?}", + gate.caveats + ); + } + + #[test] + fn non_string_schema_version_fails_loud_instead_of_reading_as_legacy() { + // `and_then(Value::as_str)` collapsed a present-but-wrongly-typed field + // to `None`, which the schema checker reads as "pre-2.1 pack, accept + // silently". A pack that states a version this reader cannot even type + // is exactly the case fail-loud exists for. + let temp = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(temp.path().join("00_summary")).unwrap(); + for bad in [ + r#"{"schema_version":2.1,"decision":{"verdict":"PASS","allow_merge":true}}"#, + r#"{"schema_version":null,"decision":{"verdict":"PASS","allow_merge":true}}"#, + r#"{"schema_version":{"major":2},"decision":{"verdict":"PASS","allow_merge":true}}"#, + r#"{"schema_version":["2.1"],"decision":{"verdict":"PASS","allow_merge":true}}"#, + ] { + std::fs::write(temp.path().join("00_summary/MERGE_GATE.json"), bad).unwrap(); + let err = read_merge_gate_summary(temp.path()) + .expect_err("a non-string schema_version must fail loud"); + assert!( + format!("{err:#}").contains("schema_version"), + "{err:#} for {bad}" + ); + } + } + + #[test] + fn verdict_normalized_to_block_forces_every_derived_axis_conservative() { + // A verdict collapsed to BLOCK while `allow_merge`/`merge_recommendation` + // stayed permissive published a decision that contradicted itself: the + // human surface said BLOCK, the machine surface said approve, and + // `compute_exit_code` keyed off the latter and let automation through. + // The documented invariant is `allow_merge == (verdict == "PASS")`. + let pack = pack_with_gate( + r#"{"verdict":"MAYBE","merge_recommendation":"approve", + "allow_merge":true,"quality_pass":true, + "analysis_status":"complete"}"#, + ); + + let summary = read_merge_gate_summary(pack.path()).expect("pack stays readable"); + assert_eq!(summary.verdict, "BLOCK"); + assert!( + !summary.allow_merge, + "a normalized BLOCK cannot keep allow_merge: {summary:?}" + ); + assert_eq!( + summary.merge_recommendation, + crate::policy::engine::MergeRecommendation::Block, + "the recommendation must follow the verdict it was normalized to: {summary:?}" + ); + assert!( + summary + .caveats + .iter() + .any(|c| c.starts_with("unknown_verdict:")), + "the normalization is still reported: {:?}", + summary.caveats + ); + + let config = test_config(); + let report = Report { + target: "feature/unknown-verdict".to_string(), + bases: vec!["main".to_string()], + diffs: vec![], + checks: vec![], + heuristics: None, + artifacts_dir: pack.path().to_path_buf(), + duration: Duration::from_secs(1), + unchanged: false, + }; + let cli = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); + assert_eq!( + compute_exit_code(&cli, false, false), + 1, + "a BLOCK verdict must not exit 0" + ); + } + + #[test] + fn a_gate_whose_root_is_not_an_object_is_corrupt_not_a_block() { + // The legacy tolerance says WHERE the decision sits, not that anything + // parseable is a decision. A pack that parses to an array, a scalar or + // `null` has no fields at all: the CLI read one as a decision with no + // signals and answered a normalized BLOCK — a successful summary for an + // artifact the MCP reader rejects as corrupt. + for root in ["[1,2,3]", "\"BLOCK\"", "null", "7"] { + let temp = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(temp.path().join("00_summary")).unwrap(); + std::fs::write(temp.path().join("00_summary/MERGE_GATE.json"), root).unwrap(); + + let err = read_merge_gate_summary(temp.path()) + .expect_err("a non-object gate root carries no decision"); + let message = format!("{err:#}"); + assert!( + message.contains("not a JSON object"), + "the error must name the real defect, got: {message}" + ); + } + } + + #[test] + fn an_explicit_block_verdict_overrides_an_approve_recommendation() { + // Every field here is present, correctly typed and in vocabulary, so + // none of the unreadable/unknown guards fire — and the reader simply + // believed each field in turn: it published the pack's `BLOCK` verdict + // beside an `Approve` recommendation, and `compute_exit_code` keys off + // the recommendation, so a gate whose own canonical artifact said BLOCK + // exited 0 outside CI. The MCP adapter has reconciled these axes by + // conservativeness since it was written. + let pack = pack_with_gate( + r#"{"verdict":"BLOCK","merge_recommendation":"approve", + "allow_merge":false,"quality_pass":true, + "analysis_status":"complete"}"#, + ); + + let summary = read_merge_gate_summary(pack.path()).expect("pack stays readable"); + assert_eq!(summary.verdict, "BLOCK"); + assert_eq!( + summary.merge_recommendation, + crate::policy::engine::MergeRecommendation::Block, + "an explicit BLOCK cannot leave an approve recommendation: {summary:?}" + ); + assert!(!summary.allow_merge, "{summary:?}"); + assert!( + summary + .caveats + .iter() + .any(|c| c.starts_with("core_inconsistency:")), + "the contradiction must be named, not silently resolved: {:?}", + summary.caveats + ); + + let config = test_config(); + let report = Report { + target: "feature/contradictory-gate".to_string(), + bases: vec!["main".to_string()], + diffs: vec![], + checks: vec![], + heuristics: None, + artifacts_dir: pack.path().to_path_buf(), + duration: Duration::from_secs(1), + unchanged: false, + }; + let cli = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); + assert_eq!( + compute_exit_code(&cli, false, false), + 1, + "a BLOCK gate must fail the process even outside CI" + ); + } + + #[test] + fn a_permissive_flag_never_lowers_a_stated_verdict() { + // The mirror direction of the same rule: `allow_merge: true` beside a + // `review_required` recommendation must not buy a PASS. + let pack = pack_with_gate( + r#"{"verdict":"CONDITIONAL","merge_recommendation":"review_required", + "allow_merge":true,"quality_pass":true, + "analysis_status":"complete"}"#, + ); + + let summary = read_merge_gate_summary(pack.path()).expect("pack stays readable"); + assert_eq!(summary.verdict, "CONDITIONAL"); + assert!( + !summary.allow_merge, + "allow_merge == (verdict == PASS) is the documented invariant: {summary:?}" + ); + assert_eq!( + summary.merge_recommendation, + crate::policy::engine::MergeRecommendation::ReviewRequired, + "{summary:?}" + ); + } + + #[test] + fn a_consistent_pack_earns_no_inconsistency_caveat() { + // Guard against over-reach: reconciliation must be silent when the + // axes already agree. + let pack = pack_with_gate( + r#"{"verdict":"PASS","merge_recommendation":"approve", + "allow_merge":true,"quality_pass":true, + "analysis_status":"complete"}"#, + ); + + let summary = read_merge_gate_summary(pack.path()).expect("pack stays readable"); + assert_eq!(summary.verdict, "PASS"); + assert!(summary.allow_merge, "{summary:?}"); + assert!( + summary.caveats.is_empty(), + "a consistent pack reads clean: {:?}", + summary.caveats + ); + } + + #[test] + fn fail_on_warnings_counts_the_checks_the_artifact_run_generated() { + // `--fail-on-warnings` promises to fail when ANY check warns, but it + // read `Report.checks` — the list the CLI itself executed. The artifact + // stage appends `public_api_diff`, `unsafe_audit`, `ghost_refs` and the + // synthetic heuristics check to the list `MERGE_GATE.json` is built + // from, and none of them ever returns to the CLI. A run whose only + // warning came from one of those exited 0 under the flag. + let temp = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(temp.path().join("00_summary")).unwrap(); + std::fs::write( + temp.path().join("00_summary/MERGE_GATE.json"), + r#"{"schema_version":"2.2", + "checks":[ + {"name":"Cargo check","status":"passed"}, + {"name":"public_api_diff","status":"warnings"} + ], + "decision":{"verdict":"PASS","merge_recommendation":"approve", + "allow_merge":true,"quality_pass":true, + "analysis_status":"complete"}}"#, + ) + .unwrap(); + + let mut config = test_config(); + config.execution_mode = ExecutionMode::Ci; + let report = Report { + target: "feature/generated-warning".to_string(), + bases: vec!["main".to_string()], + diffs: vec![], + checks: vec![CheckResult { + name: "Cargo check".to_string(), + status: CheckStatus::Passed, + duration: Duration::from_secs(1), + output: String::new(), + cached: false, + provenance: None, + }], + heuristics: None, + artifacts_dir: temp.path().to_path_buf(), + duration: Duration::from_secs(1), + unchanged: false, + }; + + let cli = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); + assert_eq!( + cli.checks_summary.warned, 0, + "the CLI's own list genuinely has no warning: {:?}", + cli.checks_summary + ); + assert_eq!( + cli.checks_summary.warned_in_pack, 1, + "the pack's canonical list has one: {:?}", + cli.checks_summary + ); + assert_eq!( + compute_exit_code(&cli, true, true), + 1, + "--ci --fail-on-warnings must fail on a warning only the pack knows about" + ); + assert_eq!( + compute_exit_code(&cli, true, false), + 0, + "without the flag a warning still does not fail the run" + ); + } + + #[test] + fn a_pack_without_a_checks_list_keeps_the_cli_warning_tally() { + // Guard the fallback: a legacy pack with no `checks` array must not + // report FEWER warnings than the CLI already counted itself. + let pack = pack_with_gate( + r#"{"verdict":"CONDITIONAL","merge_recommendation":"review_required", + "allow_merge":false,"quality_pass":true, + "analysis_status":"complete"}"#, + ); + + let mut config = test_config(); + config.execution_mode = ExecutionMode::Ci; + let report = Report { + target: "feature/legacy-pack".to_string(), + bases: vec!["main".to_string()], + diffs: vec![], + checks: vec![CheckResult { + name: "Semgrep scan".to_string(), + status: CheckStatus::Warnings, + duration: Duration::from_secs(1), + output: String::new(), + cached: false, + provenance: None, + }], + heuristics: None, + artifacts_dir: pack.path().to_path_buf(), + duration: Duration::from_secs(1), + unchanged: false, + }; + + let cli = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); + assert_eq!( + cli.checks_summary.warned_in_pack, 1, + "{:?}", + cli.checks_summary + ); + assert_eq!(compute_exit_code(&cli, true, true), 1); + } + + #[test] + fn a_reused_pack_with_an_unreadable_check_status_still_fails_on_warnings() { + // The tally compared against the exact string `"warnings"`, so a status + // this build does not emit — `"WARNINGS"` from another writer, a stale + // pack `--update` reused unchanged — counted as NOT a warning. The run + // exited 0 under `--ci --fail-on-warnings` on an artifact whose warning + // signal the reader could not read. Present-but-unreadable is not zero: + // it joins the tally and says so. + let pack = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(pack.path().join("00_summary")).unwrap(); + std::fs::write( + pack.path().join("00_summary/MERGE_GATE.json"), + r#"{"schema_version":"2.2", + "checks":[{"id":"semgrep","status":"WARNINGS"}], + "decision":{"verdict":"CONDITIONAL", + "merge_recommendation":"review_required", + "allow_merge":false,"quality_pass":true, + "analysis_status":"complete"}}"#, + ) + .unwrap(); + + let mut config = test_config(); + config.execution_mode = ExecutionMode::Ci; + let report = Report { + target: "feature/reused-pack".to_string(), + bases: vec!["main".to_string()], + diffs: vec![], + checks: vec![], + heuristics: None, + artifacts_dir: pack.path().to_path_buf(), + duration: Duration::from_secs(1), + unchanged: true, + }; + + let cli = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); + assert_eq!( + cli.checks_summary.warned_in_pack, 1, + "an unreadable status is not a clean one: {:?}", + cli.checks_summary + ); + assert_eq!( + compute_exit_code(&cli, true, true), + 1, + "--ci --fail-on-warnings must not pass a pack it cannot read" + ); + assert!( + cli.caveats + .iter() + .any(|caveat| caveat.starts_with("unreadable_check_status:")), + "the reader must say what it could not read, got: {:?}", + cli.caveats + ); + } + + #[test] + fn a_reused_pack_with_an_unreadable_checks_container_still_fails_on_warnings() { + // The same rule as the per-entry status, one level up: `checks` present + // but not an array left the tally at zero and fell back to the checks + // this run executed — which on an unchanged `--update` run is none at + // all. `--ci --fail-on-warnings` then exited 0 on a pack whose warning + // list the reader could not read. + // + // ABSENT `checks` keeps its tolerance and is covered by + // `a_pack_without_a_checks_list_keeps_the_cli_warning_tally`: a pack that + // states no list may simply be an old one. A pack that states something + // unreadable is not, and no legacy carve-out applies — `checks` has been + // emitted since 1.0, so a non-array there was never a valid shape. + for container in [ + r#"{"semgrep":"warnings"}"#, + r#""warnings""#, + "7", + "null", + "true", + ] { + let pack = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(pack.path().join("00_summary")).unwrap(); + std::fs::write( + pack.path().join("00_summary/MERGE_GATE.json"), + format!( + r#"{{"schema_version":"2.2", + "checks":{container}, + "decision":{{"verdict":"PASS","merge_recommendation":"approve", + "allow_merge":true,"quality_pass":true, + "analysis_status":"complete"}}}}"# + ), + ) + .unwrap(); + + let mut config = test_config(); + config.execution_mode = ExecutionMode::Ci; + let report = Report { + target: "feature/reused-pack".to_string(), + bases: vec!["main".to_string()], + diffs: vec![], + checks: vec![], + heuristics: None, + artifacts_dir: pack.path().to_path_buf(), + duration: Duration::from_secs(1), + unchanged: true, + }; + + let cli = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); + assert!( + cli.checks_summary.warned_in_pack >= 1, + "an unreadable checks list is not an empty one ({container}): {:?}", + cli.checks_summary + ); + assert_eq!( + compute_exit_code(&cli, true, true), + 1, + "--ci --fail-on-warnings must not pass a pack it cannot read ({container})" + ); + assert!( + cli.caveats + .iter() + .any(|caveat| caveat.starts_with("unreadable_checks:")), + "the reader must say what it could not read ({container}), got: {:?}", + cli.caveats + ); + } + } + + #[test] + fn a_canonical_check_status_is_not_reported_as_unreadable() { + // The guard on that widening: every status this build emits must stay + // readable, or the caveat becomes noise on every clean run and a + // `passed` check inflates the warning tally. + let pack = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(pack.path().join("00_summary")).unwrap(); + std::fs::write( + pack.path().join("00_summary/MERGE_GATE.json"), + r#"{"schema_version":"2.2", + "checks":[{"id":"a","status":"passed"},{"id":"b","status":"failed"}, + {"id":"c","status":"skipped"},{"id":"d","status":"error"}], + "decision":{"verdict":"PASS","merge_recommendation":"approve", + "allow_merge":true,"quality_pass":true, + "analysis_status":"complete"}}"#, + ) + .unwrap(); + + let mut config = test_config(); + config.execution_mode = ExecutionMode::Ci; + let report = Report { + target: "feature/clean-pack".to_string(), + bases: vec!["main".to_string()], + diffs: vec![], + checks: vec![], + heuristics: None, + artifacts_dir: pack.path().to_path_buf(), + duration: Duration::from_secs(1), + unchanged: true, + }; + + let cli = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); + assert_eq!( + cli.checks_summary.warned_in_pack, 0, + "no check in that pack warned: {:?}", + cli.checks_summary + ); + assert!( + !cli.caveats + .iter() + .any(|caveat| caveat.starts_with("unreadable_check_status:")), + "the emitted vocabulary is readable, got: {:?}", + cli.caveats + ); + } + + #[test] + fn a_mistyped_recommendation_is_not_read_as_an_absent_one() { + // `merge_recommendation: 7` collapsed through `as_str()` into "no + // recommendation", and the fallback then RECONSTRUCTED `Approve` from + // `allow_merge` — so a pack carrying a signal this reader could not + // read reported success, silently, and `--ci` exited 0. The documented + // contract says a mistyped signal normalizes conservatively and is + // reported; only the MCP surface actually did that. + let pack = pack_with_gate( + r#"{"verdict":"PASS","merge_recommendation":7, + "allow_merge":true,"quality_pass":true, + "analysis_status":"complete"}"#, + ); + + let summary = read_merge_gate_summary(pack.path()).expect("pack stays readable"); + assert!( + summary + .caveats + .iter() + .any(|c| c.starts_with("unreadable_merge_recommendation:")), + "an ignored signal must be named: {:?}", + summary.caveats + ); + assert_eq!( + summary.merge_recommendation, + crate::policy::engine::MergeRecommendation::Block, + "an unreadable signal cannot leave a permissive recommendation: {summary:?}" + ); + assert!( + !summary.allow_merge, + "an unreadable signal cannot leave allow_merge: {summary:?}" + ); + + let config = test_config(); + let report = Report { + target: "feature/mistyped-recommendation".to_string(), + bases: vec!["main".to_string()], + diffs: vec![], + checks: vec![], + heuristics: None, + artifacts_dir: pack.path().to_path_buf(), + duration: Duration::from_secs(1), + unchanged: false, + }; + let cli = build_cli_json_summary(&config, &report).expect("gate artifact is readable"); + assert_eq!( + compute_exit_code(&cli, false, false), + 1, + "a pack with an unreadable decision signal must not exit 0" + ); + } + + #[test] + fn a_mistyped_allow_merge_is_named_not_silently_defaulted() { + // `allow_merge: "false"` already defaulted to `false`, which is the safe + // direction — but silently. The reader ignored a field and reported a + // clean read, which is the same contract breach in a quieter costume. + let pack = pack_with_gate( + r#"{"verdict":"PASS","merge_recommendation":"approve", + "allow_merge":"true","quality_pass":true, + "analysis_status":"complete"}"#, + ); + + let summary = read_merge_gate_summary(pack.path()).expect("pack stays readable"); + assert!( + summary + .caveats + .iter() + .any(|c| c.starts_with("unreadable_allow_merge:")), + "an ignored signal must be named: {:?}", + summary.caveats + ); + assert!(!summary.allow_merge, "{summary:?}"); + assert_eq!( + summary.merge_recommendation, + crate::policy::engine::MergeRecommendation::Block, + "{summary:?}" + ); + } + + #[test] + fn a_mistyped_verdict_says_so_instead_of_claiming_none_was_present() { + // The `None` arm's message ("carries no `verdict`") is a lie for a + // verdict that IS present and merely untypable. + let pack = pack_with_gate( + r#"{"verdict":7,"merge_recommendation":"approve", + "allow_merge":true,"quality_pass":true}"#, + ); + + let summary = read_merge_gate_summary(pack.path()).expect("pack stays readable"); + assert_eq!(summary.verdict, "BLOCK"); + assert!( + summary + .caveats + .iter() + .any(|c| c.starts_with("unreadable_verdict:")), + "the mistyped verdict must be named: {:?}", + summary.caveats + ); + assert!( + !summary + .caveats + .iter() + .any(|c| c.contains("carries no `verdict`")), + "a present-but-untypable verdict is not an absent one: {:?}", + summary.caveats + ); + } + + #[test] + fn a_well_typed_pack_gains_no_unreadable_caveats() { + // Guard against over-reach: the conservative path must fire on + // mistyped signals only, never on an ordinary pack. + let pack = pack_with_gate( + r#"{"verdict":"PASS","merge_recommendation":"approve", + "allow_merge":true,"quality_pass":true, + "analysis_status":"complete"}"#, + ); + + let summary = read_merge_gate_summary(pack.path()).expect("pack stays readable"); + assert_eq!(summary.verdict, "PASS"); + assert!(summary.allow_merge, "{summary:?}"); + assert_eq!( + summary.merge_recommendation, + crate::policy::engine::MergeRecommendation::Approve, + "{summary:?}" + ); + assert!( + !summary.caveats.iter().any(|c| c.starts_with("unreadable_")), + "a well-typed pack carries no unreadable caveats: {:?}", + summary.caveats + ); + } + + #[test] + fn absent_verdict_normalized_to_block_is_equally_conservative() { + // Same collapse through the other arm: no `verdict` at all, with the + // remaining fields permissive. + let pack = pack_with_gate( + r#"{"merge_recommendation":"approve","allow_merge":true,"quality_pass":true}"#, + ); + + let summary = read_merge_gate_summary(pack.path()).expect("pack stays readable"); + assert_eq!(summary.verdict, "BLOCK"); + assert!(!summary.allow_merge); + assert_eq!( + summary.merge_recommendation, + crate::policy::engine::MergeRecommendation::Block + ); + } + + #[test] + fn versioned_pack_without_a_decision_object_is_an_error() { + // From 2.1 the pack states its schema, and that schema has a `decision` + // object — `tools/validate_merge_gate.py` requires one and the MCP + // reader errors without one. Treating the root as the decision instead + // let a structurally broken pack normalize quietly to BLOCK/false with + // a caveat, which is a re-derived verdict wearing a reader's clothes. + let temp = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(temp.path().join("00_summary")).unwrap(); + for bad in [ + r#"{"schema_version":"2.2"}"#, + r#"{"schema_version":"2.2","decision":null}"#, + r#"{"schema_version":"2.2","decision":[]}"#, + r#"{"schema_version":"2.2","decision":"PASS"}"#, + r#"{"schema_version":"1.0","verdict":"PASS","allow_merge":true}"#, + ] { + std::fs::write(temp.path().join("00_summary/MERGE_GATE.json"), bad).unwrap(); + let err = read_merge_gate_summary(temp.path()) + .expect_err("a versioned pack without a decision object must fail loud"); + assert!( + format!("{err:#}").contains("decision"), + "the error must name the missing object: {err:#} for {bad}" + ); + } + } + + #[test] + fn a_decision_stating_no_signal_is_corrupt_on_both_readers() { + // The object is THERE and it is an object, so the structural check + // above passes — and it states nothing. Normalizing that to BLOCK + // published a verdict for an artifact that never gave one, and did it + // on the one surface that mattered: `tools/validate_merge_gate.py` + // rejects the same pack for its missing required fields, the MCP + // adapter returns `storage_corrupt`, and `prview gate` cannot even + // deserialize it. Absence is forgiven per FIELD, because that is the + // shape of an older pack; a decision block with no signal at all is not + // an older pack, it is a truncated one. + let temp = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(temp.path().join("00_summary")).unwrap(); + for bad in [ + r#"{"schema_version":"2.2","decision":{}}"#, + r#"{"schema_version":"2.2","decision":{"quality_pass":true,"analysis_status":"complete"}}"#, + r#"{}"#, + ] { + std::fs::write(temp.path().join("00_summary/MERGE_GATE.json"), bad).unwrap(); + + let err = read_merge_gate_summary(temp.path()) + .expect_err("a decision that states nothing is not a BLOCK verdict"); + assert!( + format!("{err:#}").contains("corrupt"), + "the CLI must call it corrupt: {err:#} for {bad}" + ); + + let mcp_err = crate::mcp::read::read_decision(temp.path()) + .expect_err("the MCP adapter rejects the same pack"); + assert_eq!( + mcp_err.class, + crate::mcp::types::error_class::STORAGE_CORRUPT, + "the two readers must agree on {bad}" + ); + } + } + + #[test] + fn an_incomplete_analysis_cannot_be_published_as_a_pass() { + // The contract permits `PASS` only when `analysis_status == "complete"`, + // so an `incomplete` analysis beside a clean approval contradicts + // itself. The axis was read only AFTER the reconciliation, for + // reporting, so it never raised the rank and the approval published + // verbatim — a run that says it did not finish looking. + for status in ["incomplete", "degraded"] { + let pack = pack_with_gate(&format!( + r#"{{"verdict":"PASS","merge_recommendation":"approve","allow_merge":true,"quality_pass":true,"analysis_status":"{status}"}}"# + )); + + let cli = read_merge_gate_summary(pack.path()).expect("readable"); + assert_eq!( + cli.verdict, "CONDITIONAL", + "{status} analysis is not a PASS" + ); + assert!(!cli.allow_merge); + assert!( + cli.caveats.iter().any(|c| { + c.starts_with("core_inconsistency:") + && c.contains(&format!("analysis_status={status}")) + }), + "the contradiction must be named: {:?}", + cli.caveats + ); + + let mcp = crate::mcp::read::read_decision(pack.path()).expect("readable"); + assert_eq!( + mcp.verdict, cli.verdict, + "the two readers must not disagree" + ); + assert_eq!(mcp.merge_recommendation, "review_required"); + assert!(!mcp.allow_merge); + assert!(mcp.normalized); + assert!( + mcp.caveats.iter().any(|c| { + c.contains("core_inconsistency") + && c.contains(&format!("analysis_status={status}")) + }), + "the contradiction must be named: {:?}", + mcp.caveats + ); + } + } + + #[test] + fn a_stated_blocker_cannot_be_published_as_a_pass() { + // `blocking_issues` is non-empty only when a check reached + // `PolicyConclusion::Blocked`, which carries `merge_impact == Block`, so + // a pack that lists one beside a clean approval is stating a BLOCK it + // did not publish. `policy_allow_merge` is the same fact + // (`policy_allow_merge = blocking_issues.is_empty()`), so a pack may + // state either of them. + for decision in [ + r#"{"verdict":"PASS","merge_recommendation":"approve","allow_merge":true,"quality_pass":true,"blocking_issues":["Clippy (failed)"]}"#, + r#"{"verdict":"PASS","merge_recommendation":"approve","allow_merge":true,"quality_pass":true,"policy_allow_merge":false}"#, + ] { + let pack = pack_with_gate(decision); + + let cli = read_merge_gate_summary(pack.path()).expect("readable"); + assert_eq!(cli.verdict, "BLOCK", "a stated blocker is a BLOCK"); + assert!(!cli.allow_merge); + assert!( + cli.caveats + .iter() + .any(|c| c.starts_with("core_inconsistency:")), + "the contradiction must be named: {:?}", + cli.caveats + ); + + let mcp = crate::mcp::read::read_decision(pack.path()).expect("readable"); + assert_eq!( + mcp.verdict, cli.verdict, + "the two readers must not disagree" + ); + assert_eq!(mcp.merge_recommendation, "block"); + assert!(!mcp.allow_merge); + assert!(mcp.normalized); + } + } + + #[test] + fn a_healthy_block_pack_states_no_inconsistency() { + // The false positive the ranking must not create: a BLOCK pack states a + // blocker, `policy_allow_merge: false`, `quality_pass: false` and a + // COMPLETE analysis — every axis agreeing on rank 3 — and that is every + // BLOCK pack this tool writes. + let pack = pack_with_gate( + r#"{"verdict":"BLOCK","merge_recommendation":"block","allow_merge":false,"quality_pass":false,"analysis_status":"complete","policy_allow_merge":false,"blocking_issues":["Clippy (failed)"]}"#, + ); + + let cli = read_merge_gate_summary(pack.path()).expect("readable"); + assert_eq!(cli.verdict, "BLOCK"); + assert!(!cli.allow_merge); + assert!( + !cli.caveats + .iter() + .any(|c| c.starts_with("core_inconsistency:")), + "a pack whose axes all agree is not inconsistent: {:?}", + cli.caveats + ); + + let mcp = crate::mcp::read::read_decision(pack.path()).expect("readable"); + assert_eq!(mcp.verdict, "BLOCK"); + assert!( + !mcp.caveats.iter().any(|c| c.contains("core_inconsistency")), + "a pack whose axes all agree is not inconsistent: {:?}", + mcp.caveats + ); + assert!(!mcp.normalized, "a healthy BLOCK pack is a faithful read"); + } + + #[test] + fn a_healthy_conditional_pack_states_no_inconsistency() { + // The other side of the same false positive: a CONDITIONAL pack states + // a degraded analysis and a failed quality axis with NO blocker, and + // every axis agrees on rank 2. + let pack = pack_with_gate( + r#"{"verdict":"CONDITIONAL","merge_recommendation":"review_required","allow_merge":false,"quality_pass":false,"analysis_status":"degraded","policy_allow_merge":true,"blocking_issues":[]}"#, + ); + + let cli = read_merge_gate_summary(pack.path()).expect("readable"); + assert_eq!(cli.verdict, "CONDITIONAL"); + assert!( + !cli.caveats + .iter() + .any(|c| c.starts_with("core_inconsistency:")), + "a pack whose axes all agree is not inconsistent: {:?}", + cli.caveats + ); + + let mcp = crate::mcp::read::read_decision(pack.path()).expect("readable"); + assert_eq!(mcp.verdict, "CONDITIONAL"); + assert!( + !mcp.normalized, + "a healthy CONDITIONAL pack is a faithful read" + ); + } + + #[test] + fn a_pack_stating_none_of_the_new_axes_is_read_exactly_as_before() { + // Absence states nothing, on every axis. A pack written before + // `analysis_status`, `policy_allow_merge` or `blocking_issues` existed + // must not be dragged to CONDITIONAL by their mere omission. + let pack = pack_with_gate( + r#"{"verdict":"PASS","merge_recommendation":"approve","allow_merge":true,"quality_pass":true}"#, + ); + + let cli = read_merge_gate_summary(pack.path()).expect("readable"); + assert_eq!(cli.verdict, "PASS"); + assert!(cli.allow_merge); + assert!( + !cli.caveats + .iter() + .any(|c| c.starts_with("core_inconsistency:")), + "absence is not a contradiction: {:?}", + cli.caveats + ); + + let mcp = crate::mcp::read::read_decision(pack.path()).expect("readable"); + assert_eq!(mcp.verdict, "PASS"); + assert!(mcp.allow_merge); + assert!(!mcp.normalized); + } + + #[test] + fn the_new_axes_are_conservative_when_they_cannot_be_typed() { + // Same rule as `quality_pass`: present-but-unreadable is neither a + // stated value nor an absent one, on every axis that now ranks. + for (field, decision) in [ + ( + "analysis_status", + r#"{"verdict":"PASS","merge_recommendation":"approve","allow_merge":true,"quality_pass":true,"analysis_status":7}"#, + ), + ( + "policy_allow_merge", + r#"{"verdict":"PASS","merge_recommendation":"approve","allow_merge":true,"quality_pass":true,"policy_allow_merge":"false"}"#, + ), + ( + "blocking_issues", + r#"{"verdict":"PASS","merge_recommendation":"approve","allow_merge":true,"quality_pass":true,"blocking_issues":"Clippy"}"#, + ), + ] { + let pack = pack_with_gate(decision); + + let cli = read_merge_gate_summary(pack.path()).expect("readable"); + assert_eq!(cli.verdict, "BLOCK", "{field} cannot be typed"); + assert!(!cli.allow_merge); + assert!( + cli.caveats + .iter() + .any(|c| c.starts_with(&format!("unreadable_{field}:"))), + "the unreadable axis must be named: {:?}", + cli.caveats + ); + + let mcp = crate::mcp::read::read_decision(pack.path()).expect("readable"); + assert_eq!(mcp.verdict, "BLOCK", "{field} cannot be typed"); + assert!(mcp.normalized); + assert!( + mcp.caveats + .iter() + .any(|c| c.starts_with(&format!("unreadable_{field}:"))), + "the unreadable axis must be named: {:?}", + mcp.caveats + ); + } + } + + #[test] + fn an_analysis_status_outside_the_vocabulary_is_named_not_ranked() { + // Mirrors `unknown_merge_recommendation`: a value that IS a string but + // is not one this contract defines cannot rank, so it drops out of the + // reconciliation — and is named rather than vanishing into a confident + // surface derived from the remaining axes. + let pack = pack_with_gate( + r#"{"verdict":"PASS","merge_recommendation":"approve","allow_merge":true,"quality_pass":true,"analysis_status":"partial"}"#, + ); + + let cli = read_merge_gate_summary(pack.path()).expect("readable"); + assert!( + cli.caveats + .iter() + .any(|c| c.starts_with("unknown_analysis_status:")), + "the unrecognized value must be named: {:?}", + cli.caveats + ); + + let mcp = crate::mcp::read::read_decision(pack.path()).expect("readable"); + assert!( + mcp.caveats + .iter() + .any(|c| c.starts_with("unknown_analysis_status:")), + "the unrecognized value must be named: {:?}", + mcp.caveats + ); + } + + #[test] + fn a_quality_axis_that_cannot_be_typed_is_not_read_as_absent() { + // The gap between the two states the previous test relies on: a + // `quality_pass` that is PRESENT but not a boolean. `as_bool()` returned + // `None` for it, which is exactly what an OLDER pack looks like, so the + // string `"false"` bought a clean approval with no caveat at all — on + // both surfaces. A stated-but-unreadable axis is now a mistyped signal + // like every other one: BLOCK, and named. + let pack = pack_with_gate( + r#"{"verdict":"PASS","merge_recommendation":"approve","allow_merge":true,"quality_pass":"false"}"#, + ); + + let cli = read_merge_gate_summary(pack.path()).expect("readable"); + assert_eq!( + cli.verdict, "BLOCK", + "a signal that cannot be typed normalizes to BLOCK" + ); + assert!(!cli.allow_merge); + assert!( + cli.caveats + .iter() + .any(|c| c.starts_with("unreadable_quality_pass:")), + "the unreadable axis must be named: {:?}", + cli.caveats + ); + + let mcp = crate::mcp::read::read_decision(pack.path()).expect("readable"); + assert_eq!( + mcp.verdict, cli.verdict, + "the two readers must not disagree" + ); + assert_eq!(mcp.merge_recommendation, "block"); + assert!(!mcp.allow_merge); + assert!(mcp.normalized); + assert!( + mcp.caveats + .iter() + .any(|c| c.starts_with("unreadable_quality_pass:")), + "the unreadable axis must be named: {:?}", + mcp.caveats + ); + } + + #[test] + fn a_failed_quality_axis_cannot_be_published_as_a_pass() { + // The contract permits `PASS` only when quality passes, so a pack that + // states `quality_pass: false` beside a clean approval contradicts + // itself. Leaving that axis out of the reconciliation published the + // approval verbatim — with `allow_merge: true`, and on BOTH surfaces, so + // automation approved it too. + let pack = pack_with_gate( + r#"{"verdict":"PASS","merge_recommendation":"approve","allow_merge":true,"quality_pass":false}"#, + ); + + let cli = read_merge_gate_summary(pack.path()).expect("readable"); + assert_eq!( + cli.verdict, "CONDITIONAL", + "a failed quality axis is not a PASS" + ); + assert!(!cli.allow_merge); + assert!( + cli.caveats + .iter() + .any(|c| c.starts_with("core_inconsistency:") && c.contains("quality_pass=false")), + "the contradiction must be named: {:?}", + cli.caveats + ); + + let mcp = crate::mcp::read::read_decision(pack.path()).expect("readable"); + assert_eq!( + mcp.verdict, cli.verdict, + "the two readers must not disagree" + ); + assert_eq!(mcp.merge_recommendation, "review_required"); + assert!(!mcp.allow_merge); + assert!(mcp.normalized); + assert!( + mcp.caveats + .iter() + .any(|c| c.contains("core_inconsistency") && c.contains("quality_pass=false")), + "the contradiction must be named: {:?}", + mcp.caveats + ); + } + + #[test] + fn a_pack_that_states_no_quality_axis_is_read_exactly_as_before() { + // Absence stays forgiven per FIELD — that is the shape of an older pack. + // Only a STATED `quality_pass: false` ranks; defaulting an absent one to + // `false` would have turned every pre-quality_pass pack into a + // CONDITIONAL. + let pack = pack_with_gate( + r#"{"verdict":"PASS","merge_recommendation":"approve","allow_merge":true}"#, + ); + + let cli = read_merge_gate_summary(pack.path()).expect("readable"); + assert_eq!(cli.verdict, "PASS"); + assert!(cli.allow_merge); + assert!( + !cli.caveats + .iter() + .any(|c| c.starts_with("core_inconsistency:")), + "{:?}", + cli.caveats + ); + // The reconciliation kept the PASS, but the SUMMARY has to agree with + // it. Defaulting the absent axis to `false` published a failed quality + // gate the pack never claimed, derived `analysis_status: Incomplete` + // from that, and made `--ci` exit 1 — on the same artifact the MCP + // adapter approved. + assert!( + cli.quality_pass, + "a reconciled PASS implies the quality axis passed" + ); + assert_eq!( + cli.analysis_status, + crate::policy::engine::AnalysisStatus::Complete, + "a reconciled PASS implies a complete analysis" + ); + assert_eq!( + exit_code_for(&pack, true), + 0, + "a legacy PASS pack must not fail --ci" + ); + + let mcp = crate::mcp::read::read_decision(pack.path()).expect("readable"); + assert_eq!(mcp.verdict, "PASS"); + assert!(mcp.allow_merge); + assert!(!mcp.normalized); + assert_eq!( + mcp.allow_merge, cli.allow_merge, + "the two readers must not disagree on the same artifact" + ); + } + + #[test] + fn an_absent_quality_axis_stays_conservative_when_the_pack_is_not_a_pass() { + // The derivation runs off the RECONCILED outcome, so it only ever says + // "passed" where the contract already implies it. A legacy pack that is + // held at CONDITIONAL or BLOCK states no quality axis either, and + // inferring a pass for it would soften a verdict on no evidence. + for decision in [ + r#"{"verdict":"CONDITIONAL","merge_recommendation":"review_required","allow_merge":false}"#, + r#"{"verdict":"BLOCK","merge_recommendation":"block","allow_merge":false}"#, + ] { + let cli = read_merge_gate_summary(pack_with_gate(decision).path()).expect("readable"); + assert!(!cli.quality_pass, "{decision}"); + assert_eq!( + cli.analysis_status, + crate::policy::engine::AnalysisStatus::Incomplete, + "{decision}" + ); + } + } + + #[test] + fn a_mistyped_quality_axis_is_never_inferred_from_the_verdict() { + // The absent/mistyped split from round 20 is load-bearing here: a + // `quality_pass` that is PRESENT but unreadable normalizes the whole + // decision to BLOCK, so the derivation below can never read it back as + // a pass. Inferring from the verdict must not become a way around that. + let pack = pack_with_gate( + r#"{"verdict":"PASS","merge_recommendation":"approve","allow_merge":true,"quality_pass":"true"}"#, + ); + + let cli = read_merge_gate_summary(pack.path()).expect("readable"); + assert_eq!(cli.verdict, "BLOCK"); + assert!(!cli.allow_merge); + assert!( + !cli.quality_pass, + "a signal that cannot be read is not a pass" + ); + assert_eq!( + exit_code_for(&pack, true), + 1, + "an unreadable quality axis still fails --ci" + ); + } + + #[test] + fn a_quality_axis_that_passes_does_not_soften_a_conservative_verdict() { + // The asymmetry that keeps the healthy packs quiet: `quality_pass: true` + // does not assert the gate passed — a breaking-change escalation holds a + // quality-clean run at CONDITIONAL — so it states no rank of its own and + // never contradicts the verdict beside it. + let pack = pack_with_gate( + r#"{"verdict":"CONDITIONAL","merge_recommendation":"review_required","allow_merge":false,"quality_pass":true}"#, + ); + + let cli = read_merge_gate_summary(pack.path()).expect("readable"); + assert_eq!(cli.verdict, "CONDITIONAL"); + assert!( + !cli.caveats + .iter() + .any(|c| c.starts_with("core_inconsistency:")), + "{:?}", + cli.caveats + ); + + let mcp = crate::mcp::read::read_decision(pack.path()).expect("readable"); + assert_eq!(mcp.verdict, "CONDITIONAL"); + assert!(!mcp.caveats.iter().any(|c| c.contains("core_inconsistency"))); + } + + #[test] + fn a_self_consistent_pack_raises_no_inconsistency_on_either_surface() { + // `allow_merge` is a two-valued axis: `false` can never rank as high as + // BLOCK, so comparing it to the numeric rank of the winning verdict made + // every healthy BLOCK pack look self-contradictory. The consistency + // check compares the textual axes to each other and `allow_merge` to the + // flag actually published — nothing else. + for decision in [ + r#"{"merge_recommendation":"block","verdict":"BLOCK","allow_merge":false,"quality_pass":false}"#, + r#"{"merge_recommendation":"review_required","verdict":"CONDITIONAL","allow_merge":false,"quality_pass":true}"#, + r#"{"merge_recommendation":"approve","verdict":"PASS","allow_merge":true,"quality_pass":true}"#, + ] { + let pack = pack_with_gate(decision); + + let cli = read_merge_gate_summary(pack.path()).expect("readable"); + assert!( + !cli.caveats + .iter() + .any(|c| c.starts_with("core_inconsistency:")), + "the CLI invented a disagreement in {decision}: {:?}", + cli.caveats + ); + + let mcp = crate::mcp::read::read_decision(pack.path()).expect("readable"); + assert!( + !mcp.caveats.iter().any(|c| c.contains("core_inconsistency")), + "the MCP adapter invented a disagreement in {decision}: {:?}", + mcp.caveats + ); + assert!( + !mcp.normalized, + "a self-consistent pack is published as stated: {decision}" + ); + } + } + + #[test] + fn a_present_but_unrankable_decision_reads_the_same_on_both_surfaces() { + // The residual of the previous round: `storage_corrupt` is reserved for + // a decision that states NOTHING. A signal that is present but cannot + // rank — a verdict outside the vocabulary, a lone `allow_merge` — is a + // decision the pack gave, and both readers must normalize it the same + // conservative way instead of one publishing a summary while the other + // calls the identical artifact corrupt. + for decision in [ + r#"{"verdict":"PROBABLY","allow_merge":false}"#, + r#"{"allow_merge":false}"#, + r#"{"allow_merge":true}"#, + r#"{"merge_recommendation":"approve","verdict":"MAYBE","allow_merge":true}"#, + ] { + let pack = pack_with_gate(decision); + + let cli = read_merge_gate_summary(pack.path()).expect("the CLI reads a stated signal"); + let mcp = crate::mcp::read::read_decision(pack.path()) + .expect("the MCP adapter reads the same signal"); + + assert_eq!( + cli.verdict, "BLOCK", + "a decision this reader had to substitute is not an approval: {decision}" + ); + assert_eq!( + mcp.verdict, cli.verdict, + "the two readers must not disagree about {decision}" + ); + assert_eq!( + mcp.merge_recommendation, "block", + "every axis follows the substituted verdict: {decision}" + ); + assert!(!mcp.allow_merge, "{decision}"); + assert!(mcp.normalized, "a substituted verdict is a normalization"); + } + } + + #[test] + fn an_unversioned_allow_merge_only_pack_reads_the_same_on_both_surfaces() { + // The legacy root shape of the same case: a pre-2.1 pack whose root IS + // the decision and whose only correctly typed field is `allow_merge`. + let temp = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(temp.path().join("00_summary")).unwrap(); + for root in [r#"{"allow_merge":false}"#, r#"{"allow_merge":true}"#] { + std::fs::write(temp.path().join("00_summary/MERGE_GATE.json"), root).unwrap(); + + let cli = read_merge_gate_summary(temp.path()).expect("legacy pack stays readable"); + let mcp = crate::mcp::read::read_decision(temp.path()) + .expect("the MCP adapter reads the same legacy pack"); + + assert_eq!(cli.verdict, "BLOCK", "{root}"); + assert_eq!(mcp.verdict, cli.verdict, "the readers must agree on {root}"); + assert!(!mcp.allow_merge, "{root}"); + } + } + + #[test] + fn a_decision_stating_one_unreadable_signal_is_still_read() { + // The other direction, and the reason the rule counts PRESENCE rather + // than recognizability: a pack that states a verdict outside the + // vocabulary DID state a decision. The contract has it collapse to + // BLOCK with an `unknown_verdict:` caveat, and calling it corrupt + // instead would retire a documented read. + let pack = pack_with_gate(r#"{"verdict":"PROBABLY"}"#); + let summary = read_merge_gate_summary(pack.path()).expect("a stated verdict is a decision"); + assert_eq!(summary.verdict, "BLOCK"); + assert!( + summary + .caveats + .iter() + .any(|c| c.starts_with("unknown_verdict:")), + "{:?}", + summary.caveats + ); + } + + #[test] + fn unversioned_pack_still_reads_the_root_as_its_decision() { + // The other direction: a pack with NO `schema_version` predates the + // field and is the documented legacy read-back surface. Tightening the + // structural check must not retire it. + let temp = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(temp.path().join("00_summary")).unwrap(); + std::fs::write( + temp.path().join("00_summary/MERGE_GATE.json"), + r#"{"verdict":"ALLOW","allow_merge":true,"quality_pass":true}"#, + ) + .unwrap(); + + let summary = read_merge_gate_summary(temp.path()).expect("legacy pack stays readable"); + assert_eq!(summary.verdict, "PASS"); + assert!(summary.allow_merge); + assert!( + summary.caveats.is_empty(), + "a legacy pack read as intended raises no caveat: {:?}", + summary.caveats + ); + } + #[test] fn test_format_duration_zero() { assert_eq!(format_duration(Duration::from_secs(0)), "0s"); @@ -1651,6 +3683,8 @@ api-router/app/core/cache.py allow_merge: true, quality_pass: true, reason: Some("pre-existing findings outside the change".to_string()), + caveats: Vec::new(), + warned_checks: 0, }; let heading = failure_summary_heading(&report, Some(&gate)).expect("heading"); diff --git a/src/policy/engine.rs b/src/policy/engine.rs index ddbf309..51a579e 100644 --- a/src/policy/engine.rs +++ b/src/policy/engine.rs @@ -447,3 +447,88 @@ fn display_status_label(status: &str) -> String { None => status.to_string(), } } + +#[cfg(test)] +mod tests { + use super::*; + + /// The two enum decision axes reach MERGE_GATE.json through `serde`, and + /// `tools/validate_merge_gate.py` mirrors their spellings as a closed, + /// case-sensitive vocabulary — the same arrangement `CheckStatus::EMITTED` + /// has with `VALID_CHECK_STATUSES`. Renaming a variant without touching the + /// validator would leave the contract gate certifying a word no reader + /// ranks, so pin the wire spelling where the rename would happen. + #[test] + fn the_decision_axes_serialize_to_the_words_the_validator_knows() { + let analysis = [ + (AnalysisStatus::Complete, "complete"), + (AnalysisStatus::Degraded, "degraded"), + (AnalysisStatus::Incomplete, "incomplete"), + ]; + for (status, spelling) in analysis { + assert_eq!( + serde_json::to_value(status).expect("serialize analysis status"), + serde_json::Value::String(spelling.to_string()), + "VALID_ANALYSIS_STATUSES in tools/validate_merge_gate.py lists {spelling}" + ); + } + + let recommendations = [ + (MergeRecommendation::Approve, "approve"), + (MergeRecommendation::ReviewRequired, "review_required"), + (MergeRecommendation::Block, "block"), + ]; + for (recommendation, spelling) in recommendations { + assert_eq!( + serde_json::to_value(recommendation).expect("serialize recommendation"), + serde_json::Value::String(spelling.to_string()), + "VALID_MERGE_RECOMMENDATIONS in tools/validate_merge_gate.py lists {spelling}" + ); + } + } + + /// The rank table the validator ports. Every `verdict` the emitter derives + /// is the MAX rank of the axes it derived it from, which is what lets the + /// contract gate reject a verdict milder than its own axes. + #[test] + fn the_derived_verdict_is_the_most_conservative_axis() { + let rank = |verdict: &str| match verdict { + "PASS" => 1, + "CONDITIONAL" => 2, + _ => 3, + }; + for recommendation in [ + MergeRecommendation::Approve, + MergeRecommendation::ReviewRequired, + MergeRecommendation::Block, + ] { + for analysis in [ + AnalysisStatus::Complete, + AnalysisStatus::Degraded, + AnalysisStatus::Incomplete, + ] { + for quality_pass in [true, false] { + let recommendation_rank = match recommendation { + MergeRecommendation::Approve => 1, + MergeRecommendation::ReviewRequired => 2, + MergeRecommendation::Block => 3, + }; + // Only values that RULE OUT a milder outcome state a rank: + // `complete` and `quality_pass: true` are preconditions of + // PASS, not grants of it. + let analysis_rank = if analysis == AnalysisStatus::Complete { + 1 + } else { + 2 + }; + let quality_rank = if quality_pass { 1 } else { 2 }; + assert_eq!( + rank(recommendation.legacy_verdict(analysis, quality_pass)), + recommendation_rank.max(analysis_rank).max(quality_rank), + "{recommendation:?} + {analysis:?} + quality_pass={quality_pass}" + ); + } + } + } + } +} diff --git a/src/regression/perf.rs b/src/regression/perf.rs index 5e71962..b0fff3a 100644 --- a/src/regression/perf.rs +++ b/src/regression/perf.rs @@ -7,10 +7,25 @@ //! Non-code files (docs, scripts, configs, assets) are skipped entirely. //! Test/e2e files are also skipped — a query-in-loop in a Playwright test //! is not a production performance regression signal. +//! +//! Inline Rust test context (`#[cfg(test)]` / `mod tests` / `#[test]`) is +//! resolved **per hit line**, not per hunk: a production hot path that merely +//! shares a hunk with a trailing test module still counts as a production +//! signal. When the context of a hit is ambiguous it is classified as +//! production — a false positive costs a reviewer a glance, a false negative +//! hides a real regression. That rule governs the MARKERS too: only a gate that +//! provably holds solely in a test build opens test context, so +//! `#[cfg(not(test))]` and `#[cfg(any(test, …))]` are production. The context is read from the patch's **target +//! state** only (added and context lines); removed lines describe what the +//! patch replaces and never open or close a scope. A hit is paired only with a +//! nearby loop in the *same* context, so a production statement cannot borrow a +//! loop from an adjacent test module (or the reverse). use super::RegressionContext; +use crate::rust_source::SourceScanner; use regex::Regex; use serde::{Deserialize, Serialize}; +use std::borrow::Cow; use std::collections::{HashMap, HashSet}; use std::sync::LazyLock; @@ -52,13 +67,120 @@ static QUERY_PATTERN: LazyLock = LazyLock::new(|| { static CLONE_COLLECT_PATTERN: LazyLock = LazyLock::new(|| Regex::new(r"\.(clone|collect)\s*\(\s*\)").unwrap()); +/// Markers that PROVE the item below them exists only in a test build. +/// +/// The `cfg` alternatives are deliberately narrow. Matching the bare token +/// `test` anywhere inside a predicate read `#[cfg(not(test))]` — code compiled +/// into every build EXCEPT the test one — as test context and silently muted the +/// production hits under it, and did the same for +/// `#[cfg(any(test, feature = "bench"))]`, which compiles outside the test build +/// whenever the feature is on, and for `#[cfg(feature = "__internal-test")]`, +/// which is a feature that merely has `test` in its name. Only an exact +/// `cfg(test)` and an `all(…)` with `test` among its operands — which cannot +/// hold unless `test` does — are provable. Measured over the local registry +/// (58,586 files), of the 11,030 attributes the previous pattern read as test +/// context 83.62% are exactly `cfg(test)` and 6.76% are `all(…, test, …)`; the +/// remaining 9.62% — `any(test, …)`, `not(…)` and `test`-named features — are +/// the ones it was getting wrong. +/// +/// `all` is commutative, so the operand's POSITION carries no meaning: +/// `all(feature = "bench", test)` is as provably test-only as +/// `all(test, feature = "bench")`, and reading only the first operand made the +/// same predicate test context or production depending on how it was written. +/// Accepting the operand anywhere adds 72 attributes over the same 58,614-file +/// registry and removes none — `all(loom, test)`, `all(feature = "std", test)`, +/// `all(windows, test)` and the degenerate `all(test)` — every one of them a +/// direct `test` operand. +/// The operand must be a DIRECT one, which is why nothing before it may open a +/// nested predicate: `test` inside `all(not(test), …)` proves the opposite of +/// itself, and `all(any(test, …), …)` proves nothing. That exclusion also drops +/// `all(not(windows), test)`, where the `test` IS direct — an under-detection +/// kept on purpose, per the asymmetry below, rather than a paren-matching +/// parser this signal does not need. +/// +/// Anything unproven is production, because the two errors are not +/// symmetrical: failing to recognize test context costs one extra finding a +/// reader can dismiss, while claiming it where it does not hold deletes a +/// production finding nobody ever sees. static INLINE_RUST_TEST_CONTEXT_PATTERN: LazyLock = LazyLock::new(|| { Regex::new( - r"#\[\s*(?:cfg\s*\([^]]*\btest\b[^]]*\)|(?:[\w:]+::)*test|rstest)\s*\]|\bmod\s+tests\b", + r"#\[\s*(?:cfg\s*\(\s*(?:test|all\s*\(\s*[^()]*\btest\s*(?:,[^]]*)?\))\s*\)|(?:[\w:]+::)*test|rstest)\s*\]|\bmod\s+tests\b", ) .unwrap() }); +/// Runaway bound on how many physical lines one attribute may span. +/// +/// rustfmt wraps a long `cfg` predicate over a handful of lines; nothing +/// legitimate approaches this. The cap exists so a `#[` that never closes — a +/// truncated hunk, a macro fragment — cannot keep swallowing lines and silence +/// the rest of the file. +const MAX_ATTRIBUTE_CONTINUATION_LINES: usize = 8; + +/// Joins the physical lines of one attribute so the marker pattern sees the +/// whole predicate. +/// +/// The pattern is written against a complete `#[…]`, but it was applied per +/// physical line, so a wrapped +/// `#[cfg(all(\n feature = "bench",\n test\n))]` matched on no line at +/// all and its test-only item was read as production. Brackets are counted on +/// the comment- and literal-resolved view, so a `]` inside a string or a +/// trailing comment cannot close the attribute early. +#[derive(Default)] +struct AttributeAccumulator { + /// Joined code of the attribute so far, or `None` when none is open. + buffer: Option, + /// Unclosed `[` of the open attribute. + depth: i32, + lines: usize, +} + +impl AttributeAccumulator { + /// Feed one line of resolved code and return the text to match against, if + /// any: the line itself when no attribute is open and it needs none, the + /// joined attribute on the line that closes it, and nothing while one is + /// still accumulating. + fn feed<'a>(&mut self, code: &'a str) -> Option> { + let delta = bracket_delta(code); + + if let Some(buffer) = self.buffer.as_mut() { + buffer.push(' '); + buffer.push_str(code); + self.depth += delta; + self.lines += 1; + if self.depth <= 0 { + return self.buffer.take().map(Cow::Owned); + } + if self.lines >= MAX_ATTRIBUTE_CONTINUATION_LINES { + // Unterminated: nothing was proven, so drop it rather than + // letting a half-read predicate decide the context. + self.buffer = None; + self.depth = 0; + self.lines = 0; + } + return None; + } + + if delta > 0 && code.contains('#') { + self.buffer = Some(code.to_string()); + self.depth = delta; + self.lines = 1; + return None; + } + + Some(Cow::Borrowed(code)) + } +} + +/// Unclosed `[` in one line of resolved code. +fn bracket_delta(code: &str) -> i32 { + code.chars().fold(0, |depth, ch| match ch { + '[' => depth + 1, + ']' => depth - 1, + _ => depth, + }) +} + #[derive(Debug, Clone, Default, Serialize, Deserialize)] pub struct PerfRegression { pub perf_regression_suspected: bool, @@ -171,56 +293,30 @@ pub fn analyze(ctx: &RegressionContext) -> PerfRegression { .map(|l| &l[1..]) // strip leading '+' .collect(); + // Per-added-line inline test context, aligned index-by-index with + // `added_lines`, so each hit is classified by its own line. + let test_context = added_line_test_context(&file, &hunk); + // Proximity-based detection: patterns must appear within PROXIMITY_WINDOW // added lines of each other, not just anywhere in the same hunk. - let (has_query_near_loop, has_clone_near_loop) = check_proximity(&added_lines); - let is_inline_test_context = is_inline_test_context(&file, &hunk); + let hits = check_proximity(&added_lines, &test_context); - if has_query_near_loop { + if hits.query_prod || hits.query_test || hits.clone_prod || hits.clone_test { let reasons = file_reasons.entry(file.clone()).or_default(); - if is_inline_test_context { - if !reasons - .test_reasons - .iter() - .any(|r| r.contains("query in loop")) - { - reasons.test_reasons.push("query in loop".to_string()); - } - } else { + + if hits.query_prod { query_in_loop += 1; - if !reasons - .prod_reasons - .iter() - .any(|r| r.contains("query in loop")) - { - reasons.prod_reasons.push("query in loop".to_string()); - } + push_reason(&mut reasons.prod_reasons, "query in loop"); } - } - - if has_clone_near_loop { - let reasons = file_reasons.entry(file.clone()).or_default(); - if is_inline_test_context { - if !reasons - .test_reasons - .iter() - .any(|r| r.contains("clone/collect")) - { - reasons - .test_reasons - .push("clone/collect in loop".to_string()); - } - } else { + if hits.query_test { + push_reason(&mut reasons.test_reasons, "query in loop"); + } + if hits.clone_prod { clone_in_loop += 1; - if !reasons - .prod_reasons - .iter() - .any(|r| r.contains("clone/collect")) - { - reasons - .prod_reasons - .push("clone/collect in loop".to_string()); - } + push_reason(&mut reasons.prod_reasons, "clone/collect in loop"); + } + if hits.clone_test { + push_reason(&mut reasons.test_reasons, "clone/collect in loop"); } } } @@ -274,63 +370,401 @@ pub fn analyze(ctx: &RegressionContext) -> PerfRegression { } } +/// Append `reason` unless it is already recorded. +fn push_reason(reasons: &mut Vec, reason: &str) { + if !reasons.iter().any(|r| r == reason) { + reasons.push(reason.to_string()); + } +} + +/// Proximity hits of one hunk, split by the context of the hit itself. +#[derive(Debug, Default, Clone, Copy, PartialEq, Eq)] +struct ProximityHits { + query_prod: bool, + query_test: bool, + clone_prod: bool, + clone_test: bool, +} + +impl ProximityHits { + fn is_saturated(&self) -> bool { + self.query_prod && self.query_test && self.clone_prod && self.clone_test + } +} + /// Check if query/clone patterns appear within [`PROXIMITY_WINDOW`] added lines -/// of a loop pattern. Returns `(query_near_loop, clone_near_loop)`. -fn check_proximity(added_lines: &[&str]) -> (bool, bool) { - let loop_lines: Vec = added_lines +/// of a loop pattern, classifying **each hit** as production or test context. +/// +/// `test_context[i]` describes `added_lines[i]`. A hit counts as test context +/// only when **its own line** sits in test context; anything else — including a +/// missing or unknown classification — counts as production. +/// +/// The nearby loop must share the hit's context. A production statement sitting +/// just above a trailing `#[cfg(test)]` module is not "in" the loop of a test +/// that happens to be within [`PROXIMITY_WINDOW`] lines of it — pairing across +/// the boundary invents a loop that exists in neither context. Unknown context +/// still resolves to production on both sides, so ambiguity keeps pairing. +fn check_proximity(added_lines: &[&str], test_context: &[bool]) -> ProximityHits { + // Missing context data means "unknown" — treat it as production. + let context_of = |i: usize| test_context.get(i).copied().unwrap_or(false); + + let loop_lines: Vec<(usize, bool)> = added_lines .iter() .enumerate() .filter(|(_, l)| is_loop_line(l)) - .map(|(i, _)| i) + .map(|(i, _)| (i, context_of(i))) .collect(); + let mut hits = ProximityHits::default(); if loop_lines.is_empty() { - return (false, false); + return hits; } - let mut query_near_loop = false; - let mut clone_near_loop = false; - for (i, line) in added_lines.iter().enumerate() { + let is_query = QUERY_PATTERN.is_match(line); + let is_clone = CLONE_COLLECT_PATTERN.is_match(line); + if !is_query && !is_clone { + continue; + } + + let in_test = context_of(i); + let near_loop = loop_lines .iter() - .any(|&l| i.abs_diff(l) <= PROXIMITY_WINDOW); + .any(|&(l, loop_in_test)| loop_in_test == in_test && i.abs_diff(l) <= PROXIMITY_WINDOW); if !near_loop { continue; } - if !query_near_loop && QUERY_PATTERN.is_match(line) { - query_near_loop = true; + + if is_query { + hits.query_prod |= !in_test; + hits.query_test |= in_test; } - if !clone_near_loop && CLONE_COLLECT_PATTERN.is_match(line) { - clone_near_loop = true; + if is_clone { + hits.clone_prod |= !in_test; + hits.clone_test |= in_test; } - if query_near_loop && clone_near_loop { + if hits.is_saturated() { break; } } - (query_near_loop, clone_near_loop) + hits } fn is_loop_line(line: &str) -> bool { EXPLICIT_LOOP_PATTERN.is_match(line) || ITERATOR_LOOP_PATTERN.is_match(line) } -fn is_inline_test_context(file: &str, hunk: &str) -> bool { +/// Returns `true` for hunk lines that are diff bookkeeping rather than source. +fn is_diff_metadata_line(line: &str) -> bool { + line.starts_with("@@") + || line.starts_with("diff --git") + || line.starts_with("+++") + || line.starts_with("--- a/") + || line == "---" + || line.starts_with("index ") + || line.starts_with("similarity index ") + || line.starts_with("rename ") + || line.starts_with("new file mode ") + || line.starts_with("deleted file mode ") +} + +/// Map each added line of `hunk` to "is this line inside inline Rust test +/// context?", in the same order as the added-line vector used for detection. +/// +/// A test-context marker (`#[cfg(test)]`, `mod tests`, `#[test]`, `#[rstest]`) +/// opens the context; it closes again once the braces opened after that marker +/// balance out. Lines *before* the marker stay production — that is the whole +/// point of per-hit classification: a hot path sharing a hunk with a trailing +/// test module is still production code. +/// +/// Ambiguity resolves toward production: non-Rust files, unrecognised braces +/// and commented-out markers all leave the line classified as production. +/// +/// Only the **target state** shapes the scope: added (`+`) and context (` `) +/// lines. Removed (`-`) lines describe the state being replaced and are ignored +/// wholesale — both for markers and for brace tracking. A `#[cfg(test)]` deleted +/// by the patch does not exist afterwards, and a renamed declaration whose old +/// and new lines both open a brace would otherwise leave the test scope +/// permanently open and mute every production hit below it. +fn added_line_test_context(file: &str, hunk: &str) -> Vec { + let added_lines = hunk + .lines() + .filter(|l| l.starts_with('+') && !l.starts_with("+++")); + if !file.ends_with(".rs") { - return false; + return added_lines.map(|_| false).collect(); } - hunk.lines().any(|line| { - let trimmed = line + let mut flags = Vec::new(); + let mut in_test = false; + let mut depth: i32 = 0; + let mut seen_open = false; + // Bracket nesting of the annotated item's SIGNATURE, tracked only until its + // body opens. A brace inside `(…)`, `[…]` or `<…>` is not the body. + let mut sig_depth: i32 = 0; + // Braces open INSIDE those brackets — a const argument (`Buffer<{1<2}>`) or + // a destructured parameter (`Params(Req { field })`). Their contents are + // expression or pattern text, where `<` and `>` are operators, so signature + // tracking is frozen while one is open. + let mut sig_block: i32 = 0; + // Has the annotated item passed a top-level `=`? After one it states a + // VALUE, so `<` and `>` there are operators too. + let mut sig_initializes = false; + // Unclosed `[` of an attribute being scanned, and whether the previous + // character was the `#`/`#!` that opens one. Both persist across lines: an + // attribute may wrap, and its braces are never the item's body. + let mut attr_depth: i32 = 0; + let mut attr_sigil = false; + // Block comments and string literals span lines, so the reader is stateful + // for this hunk. Hunks are not contiguous, and this function is called per + // hunk, so the scanner starts clean here and is never carried past the + // hunk boundary. + let mut scanner = SourceScanner::default(); + // Attributes wrap across lines; the marker pattern needs the whole one. + // Per hunk, like the scanner beside it. + let mut attributes = AttributeAccumulator::default(); + + for line in hunk.lines() { + if is_diff_metadata_line(line) { + continue; + } + + // Removed lines are not part of the state this patch produces. + if line.starts_with('-') { + continue; + } + + let is_added = line.starts_with('+'); + let payload = line .strip_prefix('+') - .or_else(|| line.strip_prefix('-')) .or_else(|| line.strip_prefix(' ')) - .unwrap_or(line) - .trim(); + .unwrap_or(line); + + // Comments (whole-line, doc, trailing, or a `/* … */` still open from + // an earlier line) are not code: neither their markers nor their braces + // may move the scope. A commented-out line reduces to an empty slice + // here, which is inert on both counts. + let code = scanner.code_only(payload); + let trimmed = code.trim(); + + // A wrapped attribute is matched once, on the line that closes it; + // while one is still open there is nothing complete to judge. + let marker_text = attributes.feed(trimmed); + + // Only the outermost marker opens the context, so nested `#[test]` + // attributes do not reset the enclosing `mod tests` brace tracking. + if !in_test + && marker_text + .as_deref() + .is_some_and(|text| INLINE_RUST_TEST_CONTEXT_PATTERN.is_match(text)) + { + in_test = true; + depth = 0; + seen_open = false; + sig_depth = 0; + sig_block = 0; + sig_initializes = false; + attr_depth = 0; + attr_sigil = false; + } - INLINE_RUST_TEST_CONTEXT_PATTERN.is_match(trimmed) - }) + if is_added { + flags.push(in_test); + } + + if in_test { + let mut prev = '\0'; + let mut chars = code.chars().peekable(); + while let Some(ch) = chars.next() { + // An attribute's brackets are its own: `#[case(Case { id: 1 })]` + // stacked under a test marker carries braces that belong to the + // attribute, never to the annotated item. Letting them through + // made the `{` a body opener and the `}` close the context on + // the same line, so the test function below read as production + // and its query-in-loop became a phantom regression. The depth + // persists across lines because attributes wrap. + if attr_depth > 0 { + match ch { + '[' => attr_depth += 1, + ']' => attr_depth -= 1, + _ => {} + } + prev = ch; + continue; + } + if ch == '[' && attr_sigil { + attr_depth = 1; + prev = ch; + continue; + } + // `#` opens an attribute sigil, `#!` an inner one; anything else + // clears it, so a plain index `a[0]` is not mistaken for one. + attr_sigil = ch == '#' || (attr_sigil && ch == '!'); + + match ch { + '{' => { + depth += 1; + // Only a brace OUTSIDE the signature's brackets opens + // the body. `fn run() -> Buffer<{ LIMIT }>` and + // `fn run(Params(Req { field }): Params)` both + // balance a brace pair in type or pattern position, + // before any body exists; reading one as the opener made + // the very next line look like the item closing again, + // so the context ended at the signature and the whole + // test body was recorded as production. + seen_open |= sig_depth == 0; + // A brace that is NOT the body opener holds expression or + // pattern text, where `<` and `>` are operators. + if !seen_open { + sig_block += 1; + } + } + '}' => { + depth -= 1; + sig_block = (sig_block - 1).max(0); + } + // A top-level `=` ends the signature and starts a VALUE. The + // `<` after it is a comparison or a shift, never a generic + // opener, and counting one left `sig_depth` above zero — the + // very thing the `;` close tests — so a body-less item like + // `#[cfg(test)] const ENABLED: bool = 1<2;` could not end its + // own context and muted every production hit below it. Only a + // TOP-LEVEL `=` counts: inside brackets it states a default + // (`fn f`) or an associated type + // (`Iterator`), and `==`, `=>` and the compound + // assignments are not initializers at all. + '=' if !seen_open + && sig_depth == 0 + && sig_block == 0 + && !matches!(chars.peek(), Some(&('=' | '>'))) + && !"=!<>+-*/%&|^".contains(prev) => + { + sig_initializes = true; + } + _ if !seen_open && sig_block == 0 => { + track_signature_brackets(ch, prev, sig_initializes, &mut sig_depth); + } + _ => {} + } + prev = ch; + } + if seen_open && depth <= 0 { + in_test = false; + depth = 0; + seen_open = false; + sig_depth = 0; + sig_block = 0; + sig_initializes = false; + } else if !seen_open && sig_depth == 0 && ends_the_annotated_item(trimmed) { + // Not every test item has a body. `#[cfg(test)] mod tests;` and + // `#[cfg(test)] use crate::helper;` never open a brace, so the + // "balanced again" close above could not fire and the context + // stayed open over the rest of the hunk — every production loop + // and query below it was recorded as test-only and vanished from + // the signal. Such an item ends at its `;`, and so does the + // context it opened. A `;` still inside the signature's brackets + // (`fn f(x: [u8;\n N])`) is not that end. + in_test = false; + depth = 0; + sig_depth = 0; + sig_block = 0; + sig_initializes = false; + } + } + } + + flags +} + +/// Advance the signature's bracket nesting by one character. +/// +/// Called only before the annotated item's body opens, which is the one region +/// where `<` is reliably a generic opener rather than a comparison: signatures +/// do not compare. `->` is spelled with the same `>`, so a return arrow must not +/// be read as closing an angle bracket. The depth is clamped at zero because a +/// hunk may start mid-signature and hand the tracker a closer whose opener it +/// never saw; erring toward zero makes the next brace read as the body opener, +/// which ENDS the test context — a phantom perf signal costs a reviewer a +/// glance, while a context stuck open mutes real production code. +/// +/// Braces in type or pattern position are rare but real: across the local +/// crates.io registry (59,974 files, 1,697,077 `fn` signatures) 1,191 signatures +/// carry one, 715 of them with the body opener on a later line — the shape that +/// actually breaks the tracker — and 59 of those signatures are test-annotated. +/// The dominant idiom is not the const generic `Buffer<{ LIMIT }>` but the +/// destructured extractor parameter, `fn handler(Parameters(Req { field }): +/// Parameters)`, as written by `rmcp`, `leptos` and `sqlx`. +fn track_signature_brackets(ch: char, prev: char, initializes: bool, sig_depth: &mut i32) { + match ch { + '(' | '[' => *sig_depth += 1, + // A generic opener FOLLOWS the thing it parameterises — `Buffer<`, + // `Vec<`, `fn f<`, `::<`. A `<` after whitespace is a comparison. That + // spacing rule is a heuristic, not a boundary: it reads `Buffer<{ 1 < 2 + // }>` correctly and `Buffer<{1<2}>` — the same type, formatted compactly + // — wrongly, because `<` after a digit looks exactly like `<` after an + // identifier. The boundary is the caller's, which freezes this tracker + // inside a const argument's braces. What survives here is the ordinary + // signature, where a comparison cannot appear at all. Closers stay + // unconditional (minus the `->` arrow) and the depth is clamped, so a + // `<` this rule still misjudges can only end the context early, never + // hold it open. + // + // The second boundary is the item's top-level `=`. After one the item + // states a VALUE, so both angle characters are operators there — and a + // counted comparison in a body-less initializer + // (`#[cfg(test)] const ENABLED: bool = 1<2;`) left the depth above zero, + // which is precisely what the `;` close tests, so the item could not end + // its own context and muted every production hit below it. + '<' if !initializes + && (prev.is_alphanumeric() || prev == '_' || prev == '>' || prev == ':') => + { + *sig_depth += 1; + } + '>' if prev == '-' || initializes => {} + ')' | ']' | '>' => *sig_depth = (*sig_depth - 1).max(0), + _ => {} + } +} + +/// Does this line finish a body-less item that a test marker annotated? +/// +/// Only meaningful while the marker's item has not opened a brace: attributes +/// stack above their item, so a line that is nothing but attributes is not it +/// yet, and a line that opens a body is handled by the brace tracker instead. +fn ends_the_annotated_item(trimmed: &str) -> bool { + let item = item_after_attributes(trimmed); + !item.is_empty() && item.ends_with(';') +} + +/// The line with any leading `#[…]` attributes removed. +/// +/// `#[cfg(test)] mod tests;` states the marker and the item it annotates on one +/// line, so the item cannot be found by looking at how the line starts. +fn item_after_attributes(trimmed: &str) -> &str { + let mut rest = trimmed; + while rest.starts_with("#[") { + let mut depth = 0usize; + let mut end = None; + for (index, ch) in rest.char_indices() { + match ch { + '[' => depth += 1, + ']' => { + depth -= 1; + if depth == 0 { + end = Some(index + ch.len_utf8()); + break; + } + } + _ => {} + } + } + // An attribute whose `]` is on a later line annotates nothing here. + let Some(end) = end else { return "" }; + rest = rest[end..].trim_start(); + } + rest } /// Split patch text into hunks (each starting with @@ or diff --git). @@ -699,23 +1133,78 @@ diff --git a/tests/handler_test.rs b/tests/handler_test.rs } #[test] - fn test_inline_test_context_does_not_pollute_prod_perf_reasons() { + fn a_not_test_cfg_is_production_context() { + // `#[cfg(not(test))]` is the exact OPPOSITE of test context: the item + // exists in every build EXCEPT the test one. Matching the bare token + // `test` anywhere inside the predicate read it as test context and muted + // a production query-in-loop entirely. let patch = r#"diff --git a/src/portal.rs b/src/portal.rs +++ b/src/portal.rs -@@ -10,3 +10,6 @@ -+for user in users.iter() { -+ db.query("SELECT 1"); +@@ -20,3 +20,10 @@ ++#[cfg(not(test))] ++fn refresh(users: &[User]) { ++ for user in users.iter() { ++ db.query("SELECT 1"); ++ } +} -@@ -40,3 +43,9 @@ - #[cfg(test)] - mod tests { -+ #[test] -+ fn portal_roundtrip() { -+ for user in users.iter() { -+ let ids: Vec<_> = values.iter().collect(); -+ } +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "code excluded from the test build is production code" + ); + assert_eq!(result.query_in_loop_count, 1); + assert!(!result.suspected_files[0].test_context_only); + } + + #[test] + fn a_cfg_that_only_may_be_test_is_production_context() { + // `#[cfg(any(test, feature = "bench"))]` compiles into a non-test build + // whenever the feature is on, so it is not PROVABLY test-only. The + // tolerated direction is a production finding for test code — one extra + // row a reader can dismiss — never a production hit silently dropped. + let patch = r#"diff --git a/src/portal.rs b/src/portal.rs ++++ b/src/portal.rs +@@ -20,3 +20,10 @@ ++#[cfg(any(test, feature = "bench"))] ++fn refresh(users: &[User]) { ++ for user in users.iter() { ++ db.query("SELECT 1"); + } - } ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "a cfg that may compile outside the test build is production" + ); + assert_eq!(result.query_in_loop_count, 1); + assert!(!result.suspected_files[0].test_context_only); + } + + #[test] + fn a_feature_named_after_test_is_production_context() { + // `#[cfg(feature = "__internal-test")]` states a FEATURE whose name + // happens to contain `test`. It compiles into an ordinary build. + let patch = r#"diff --git a/src/portal.rs b/src/portal.rs ++++ b/src/portal.rs +@@ -20,3 +20,10 @@ ++#[cfg(feature = "__internal-test")] ++fn refresh(users: &[User]) { ++ for user in users.iter() { ++ db.query("SELECT 1"); ++ } ++} "#; let ctx = RegressionContext { patch_text: Some(patch.to_string()), @@ -725,13 +1214,1320 @@ diff --git a/tests/handler_test.rs b/tests/handler_test.rs let result = analyze(&ctx); assert!(result.perf_regression_suspected); assert_eq!(result.query_in_loop_count, 1); - assert_eq!(result.clone_collect_in_loop_count, 0); - assert_eq!(result.suspected_files.len(), 1); assert!(!result.suspected_files[0].test_context_only); - assert!(result.suspected_files[0].mixed_context); - assert_eq!( - result.suspected_files[0].reasons, - vec!["query in loop".to_string()] + } + + #[test] + fn an_all_test_cfg_is_still_test_context() { + // The narrowing must not swallow the shape it is allowed to keep: + // `all(test, …)` cannot hold unless `test` does, so it is provably + // test-only and stays muted. 6.76% of the registry's cfg-test gates are + // written this way. + let patch = r#"diff --git a/src/portal.rs b/src/portal.rs ++++ b/src/portal.rs +@@ -20,3 +20,10 @@ ++#[cfg(all(test, feature = "std"))] ++fn refresh(users: &[User]) { ++ for user in users.iter() { ++ db.query("SELECT 1"); ++ } ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!(!result.perf_regression_suspected); + assert_eq!(result.query_in_loop_count, 0); + assert!(result.suspected_files[0].test_context_only); + } + + #[test] + fn an_all_cfg_proves_test_context_whatever_the_operand_order() { + // `all` is commutative, so `all(feature = "bench", test)` holds exactly + // when `all(test, feature = "bench")` does. Reading only the FIRST + // operand made the same predicate test context or production depending + // on how it happened to be written, and the second spelling produced a + // phantom production finding. + let patch = r#"diff --git a/src/portal.rs b/src/portal.rs ++++ b/src/portal.rs +@@ -20,3 +20,10 @@ ++#[cfg(all(feature = "bench", test))] ++fn refresh(users: &[User]) { ++ for user in users.iter() { ++ db.query("SELECT 1"); ++ } ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!(!result.perf_regression_suspected); + assert_eq!(result.query_in_loop_count, 0); + assert!(result.suspected_files[0].test_context_only); + } + + #[test] + fn a_nested_not_test_inside_all_is_production_context() { + // The operand has to be a DIRECT one. `all(feature = "x", not(test))` + // compiles in every build except the test one, so the bare token `test` + // inside it proves the opposite of test context — muting the hits under + // it would delete a production finding nobody ever sees. + let patch = r#"diff --git a/src/portal.rs b/src/portal.rs ++++ b/src/portal.rs +@@ -20,3 +20,10 @@ ++#[cfg(all(feature = "x", not(test)))] ++fn refresh(users: &[User]) { ++ for user in users.iter() { ++ db.query("SELECT 1"); ++ } ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!(!result.suspected_files[0].test_context_only); + } + + #[test] + fn a_feature_named_after_test_inside_all_is_production_context() { + // The widened alternative scans the whole operand list, so it must not + // start matching `test` inside a feature NAME. `all(unix, feature = + // "test-utils")` states no `test` operand at all. + let patch = r#"diff --git a/src/portal.rs b/src/portal.rs ++++ b/src/portal.rs +@@ -20,3 +20,10 @@ ++#[cfg(all(unix, feature = "test-utils"))] ++fn refresh(users: &[User]) { ++ for user in users.iter() { ++ db.query("SELECT 1"); ++ } ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!(!result.suspected_files[0].test_context_only); + } + + #[test] + fn a_multiline_cfg_attribute_still_opens_test_context() { + // rustfmt wraps a long predicate, and the marker regex runs per + // physical line, so no line ever carried the whole attribute: a + // test-only helper read as production and its query-in-loop surfaced as + // a phantom regression. + let patch = r#"diff --git a/src/portal.rs b/src/portal.rs ++++ b/src/portal.rs +@@ -20,3 +20,12 @@ ++#[cfg(all( ++ feature = "bench", ++ test ++))] ++fn refresh(users: &[User]) { ++ for user in users.iter() { ++ db.query("SELECT 1"); ++ } ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!(!result.perf_regression_suspected); + assert_eq!(result.query_in_loop_count, 0); + assert!(result.suspected_files[0].test_context_only); + } + + #[test] + fn a_multiline_not_test_cfg_is_still_production_context() { + // The accumulation must not turn the predicate into a bag of tokens: + // wrapped across lines, `not(test)` still proves the opposite of test + // context, and muting the hits under it would delete a production + // finding nobody ever sees. + let patch = r#"diff --git a/src/portal.rs b/src/portal.rs ++++ b/src/portal.rs +@@ -20,3 +20,12 @@ ++#[cfg(not( ++ test ++))] ++fn refresh(users: &[User]) { ++ for user in users.iter() { ++ db.query("SELECT 1"); ++ } ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!(!result.suspected_files[0].test_context_only); + } + + #[test] + fn a_multiline_any_test_cfg_is_still_production_context() { + // The other unproven shape, wrapped: `any(test, …)` compiles outside the + // test build whenever the other operand holds. + let patch = r#"diff --git a/src/portal.rs b/src/portal.rs ++++ b/src/portal.rs +@@ -20,3 +20,12 @@ ++#[cfg(any( ++ test, ++ feature = "bench" ++))] ++fn refresh(users: &[User]) { ++ for user in users.iter() { ++ db.query("SELECT 1"); ++ } ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!(!result.suspected_files[0].test_context_only); + } + + #[test] + fn an_unterminated_attribute_does_not_swallow_the_rest_of_the_hunk() { + // The accumulator is bounded. A `#[` that never closes — a truncated + // hunk, a macro fragment — must not keep eating lines and silence + // everything after it. + let mut patch = String::from( + "diff --git a/src/portal.rs b/src/portal.rs\n+++ b/src/portal.rs\n@@ -20,3 +20,40 @@\n+#[cfg(all(\n", + ); + for _ in 0..MAX_ATTRIBUTE_CONTINUATION_LINES + 2 { + patch.push_str("+ feature = \"x\",\n"); + } + patch.push_str("+fn refresh(users: &[User]) {\n"); + patch.push_str("+ for user in users.iter() {\n"); + patch.push_str("+ db.query(\"SELECT 1\");\n"); + patch.push_str("+ }\n"); + patch.push_str("+}\n"); + + let ctx = RegressionContext { + patch_text: Some(patch), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + !result.suspected_files[0].test_context_only, + "an unterminated attribute proves nothing and must not mute the hit" + ); + } + + /// Build a patch that stacks `attr` under `#[rstest]` over a test function + /// whose body holds a query in a loop. + fn stacked_attribute_patch(attr: &str) -> String { + format!( + "diff --git a/src/portal.rs b/src/portal.rs\n+++ b/src/portal.rs\n@@ -20,3 +20,12 @@\n+#[rstest]\n+{attr}\n+fn checks(#[case] c: Case) {{\n+ for user in c.users.iter() {{\n+ db.query(\"SELECT 1\");\n+ }}\n+}}\n" + ) + } + + #[test] + fn braces_inside_a_stacked_attribute_do_not_open_the_item_body() { + // An attribute's braces belong to the attribute, never to the annotated + // item. `#[case(Case { id: 1 })]` alone is safe only because the `[` and + // `(` hold the signature depth above zero; two clamping `>` comparisons + // drive it back to zero first, and then the attribute's `{` was read as + // the body opener and its `}` closed the test context on the same line. + // The rstest function below was classified as production and its + // query-in-loop surfaced as a phantom regression. + for attr in [ + "#[case(1 > 0, 2 > 1, Case { id: 1 })]", + "#[case(a > b, c > d, e > f, Case { id: 1 })]", + ] { + let ctx = RegressionContext { + patch_text: Some(stacked_attribute_patch(attr)), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.suspected_files[0].test_context_only, + "the rstest function is test context: {attr}" + ); + assert!(!result.perf_regression_suspected, "{attr}"); + assert_eq!(result.query_in_loop_count, 0, "{attr}"); + } + } + + #[test] + fn ordinary_stacked_attributes_keep_their_test_context() { + // The shapes that already worked must keep working: the fix must not + // buy the case above by loosening the signature tracking. + for attr in [ + "#[case(Case { id: 1 })]", + "#[case::first(Case { id: 1 })]", + "#[case(Case { id: 1 }, Other { id: 2 })]", + "#[case(Case:: { id: 1 })]", + "#[should_panic(expected = \"Case { id: 1 }\")]", + "#[values(Cfg { a: 1 })]", + ] { + let ctx = RegressionContext { + patch_text: Some(stacked_attribute_patch(attr)), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!(result.suspected_files[0].test_context_only, "{attr}"); + } + } + + #[test] + fn an_attribute_does_not_hide_the_real_body_brace() { + // The other direction: skipping an attribute's brackets must not also + // skip the item's own body. A production function carrying an attribute + // with braces still ends its (absent) test context exactly where it did, + // so the hit inside it stays visible. + let patch = r#"diff --git a/src/portal.rs b/src/portal.rs ++++ b/src/portal.rs +@@ -20,3 +20,14 @@ ++#[cfg(test)] ++fn helper() { ++ let _ = 1; ++} ++ ++#[instrument(fields(ctx = Ctx { id: 1 }))] ++fn refresh(users: &[User]) { ++ for user in users.iter() { ++ db.query("SELECT 1"); ++ } ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + !result.suspected_files[0].test_context_only, + "the production function after the test helper is not test context" + ); + } + + #[test] + fn test_inline_test_context_does_not_pollute_prod_perf_reasons() { + let patch = r#"diff --git a/src/portal.rs b/src/portal.rs ++++ b/src/portal.rs +@@ -10,3 +10,6 @@ ++for user in users.iter() { ++ db.query("SELECT 1"); ++} +@@ -40,3 +43,9 @@ + #[cfg(test)] + mod tests { ++ #[test] ++ fn portal_roundtrip() { ++ for user in users.iter() { ++ let ids: Vec<_> = values.iter().collect(); ++ } ++ } + } +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!(result.perf_regression_suspected); + assert_eq!(result.query_in_loop_count, 1); + assert_eq!(result.clone_collect_in_loop_count, 0); + assert_eq!(result.suspected_files.len(), 1); + assert!(!result.suspected_files[0].test_context_only); + assert!(result.suspected_files[0].mixed_context); + assert_eq!( + result.suspected_files[0].reasons, + vec!["query in loop".to_string()] + ); + } + + // ---- per-hit (not per-hunk) test-context classification ---- + + #[test] + fn test_prod_hit_in_mixed_hunk_is_not_muted_by_trailing_test_module() { + // Single hunk: production hot path first, test module afterwards. + // Per-hunk classification marked the whole hunk as test context and + // hid the production signal entirely. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -10,4 +10,18 @@ ++for candidate in candidates.iter() { ++ let stored = db.query("SELECT hash FROM users WHERE id = ?"); ++ argon2.verify_password(password, &stored)?; ++} + + #[cfg(test)] + mod tests { ++ #[test] ++ fn verify_roundtrip() { ++ for candidate in candidates.iter() { ++ let ids: Vec<_> = candidate.ids.iter().collect(); ++ } ++ } + } +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "production hot path sharing a hunk with a test module must stay a signal" + ); + assert_eq!(result.query_in_loop_count, 1); + assert_eq!(result.suspected_files.len(), 1); + assert_eq!(result.suspected_files[0].file, "src/auth.rs"); + assert!(!result.suspected_files[0].test_context_only); + assert!(result.suspected_files[0].mixed_context); + assert_eq!( + result.suspected_files[0].reasons, + vec!["query in loop".to_string()] + ); + } + + #[test] + fn test_prod_hit_after_closed_test_module_in_same_hunk_is_prod() { + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -10,4 +10,20 @@ + #[cfg(test)] + mod tests { ++ #[test] ++ fn roundtrip() { ++ for candidate in candidates.iter() { ++ let ids: Vec<_> = candidate.ids.iter().collect(); ++ } ++ } + } ++ ++pub fn verify_all(candidates: &[Candidate]) { ++ for candidate in candidates.iter() { ++ let stored = db.query("SELECT hash FROM users WHERE id = ?"); ++ } ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "code after the test module closes is production again" + ); + assert_eq!(result.query_in_loop_count, 1); + assert!(!result.suspected_files[0].test_context_only); + assert!(result.suspected_files[0].mixed_context); + } + + #[test] + fn test_commented_test_marker_does_not_mute_prod_hit() { + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -10,4 +10,8 @@ ++// covered by #[cfg(test)] mod tests below ++for candidate in candidates.iter() { ++ let stored = db.query("SELECT hash FROM users WHERE id = ?"); ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "a comment mentioning test attributes is not test context" + ); + assert_eq!(result.query_in_loop_count, 1); + assert!(!result.suspected_files[0].test_context_only); + } + + #[test] + fn test_same_reason_in_both_contexts_counts_once_as_prod() { + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -10,4 +10,18 @@ ++for candidate in candidates.iter() { ++ let stored = db.query("SELECT hash FROM users WHERE id = ?"); ++} + + #[cfg(test)] + mod tests { ++ #[test] ++ fn roundtrip() { ++ for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++ } ++ } + } +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!(result.perf_regression_suspected); + assert_eq!( + result.query_in_loop_count, 1, + "the test-context hit must not inflate the production counter" + ); + assert!(!result.suspected_files[0].test_context_only); + assert!( + result.suspected_files[0].mixed_context, + "the same reason in both contexts is still a mixed-context file" + ); + assert_eq!( + result.suspected_files[0].reasons, + vec!["query in loop".to_string()] + ); + } + + #[test] + fn test_pure_test_hunk_stays_test_context_only() { + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -40,3 +40,10 @@ + #[cfg(test)] + mod tests { ++ #[test] ++ fn roundtrip() { ++ for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++ } ++ } + } +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!(!result.perf_regression_suspected); + assert_eq!(result.query_in_loop_count, 0); + assert_eq!(result.suspected_files.len(), 1); + assert!(result.suspected_files[0].test_context_only); + assert!(!result.suspected_files[0].mixed_context); + assert_eq!(result.skipped_test_hits_count, 1); + } + + #[test] + fn test_hit_in_rust_test_file_is_skipped_by_path() { + let patch = r#"diff --git a/src/auth_test.rs b/src/auth_test.rs ++++ b/src/auth_test.rs +@@ -1,1 +1,5 @@ ++pub fn helper(candidates: &[Candidate]) { ++ for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++ } ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + !result.perf_regression_suspected, + "a hit in a test file is not a production signal, even outside #[cfg(test)]" + ); + assert_eq!(result.query_in_loop_count, 0); + assert_eq!(result.skipped_test_hits_count, 1); + assert!(result.suspected_files.is_empty()); + } + + // ---- target-state only: removed lines never shape the scope ---- + + #[test] + fn test_removed_test_marker_does_not_open_test_context() { + // The diff DELETES `#[cfg(test)]`, promoting the module to production. + // A marker that exists only on a removed line describes the *before* + // state and must not classify added lines as test context. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,6 +1,10 @@ +-#[cfg(test)] + pub mod helpers { ++ pub fn verify_all(candidates: &[Candidate]) { ++ for candidate in candidates.iter() { ++ let stored = db.query("SELECT hash FROM users WHERE id = ?"); ++ } ++ } + } +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "a removed #[cfg(test)] marker must not mute the added production hit" + ); + assert_eq!(result.query_in_loop_count, 1); + assert_eq!(result.suspected_files.len(), 1); + assert!(!result.suspected_files[0].test_context_only); + } + + #[test] + fn test_removed_declaration_does_not_unbalance_test_scope() { + // Renaming a fn inside `mod tests` emits `-fn old_name() {` and + // `+fn new_name() {`. Counting braces on BOTH sides leaves one extra + // opening brace, so the test scope never closes and every production + // hit later in the hunk was muted. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,10 +1,14 @@ + #[cfg(test)] + mod tests { +- fn old_name() { ++ fn new_name() { + assert!(true); + } + } ++ ++pub fn verify_all(candidates: &[Candidate]) { ++ for candidate in candidates.iter() { ++ let stored = db.query("SELECT hash FROM users WHERE id = ?"); ++ } ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "a renamed test fn must not leave the test scope open over prod code" + ); + assert_eq!(result.query_in_loop_count, 1); + assert_eq!(result.suspected_files.len(), 1); + assert!(!result.suspected_files[0].test_context_only); + } + + // ---- proximity pairing respects the context of the loop ---- + + #[test] + fn test_prod_hit_is_not_paired_with_a_loop_inside_a_test_module() { + // The only loop in range lives in the trailing test module; pairing it + // with a production statement invented a query-in-loop that does not + // exist in either context. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,12 @@ ++let stored = db.query("SELECT hash FROM users WHERE id = ?"); ++ + #[cfg(test)] + mod tests { ++ #[test] ++ fn roundtrip() { ++ for candidate in candidates.iter() { ++ assert!(candidate.ok); ++ } ++ } + } +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + !result.perf_regression_suspected, + "a production statement must not borrow a loop from the test module" + ); + assert_eq!(result.query_in_loop_count, 0); + assert!(result.suspected_files.is_empty()); + } + + #[test] + fn test_test_hit_is_not_paired_with_a_production_loop() { + // Mirror direction: a `collect()` inside the test module has no test + // loop nearby, so the production loop above must not manufacture a + // test-context suspect either. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,12 @@ ++for candidate in candidates.iter() { ++ verify(candidate); ++} + #[cfg(test)] + mod tests { ++ #[test] ++ fn roundtrip() { ++ let ids: Vec<_> = candidate.ids.iter().collect(); ++ } + } +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!(!result.perf_regression_suspected); + assert_eq!(result.clone_collect_in_loop_count, 0); + assert!( + result.suspected_files.is_empty(), + "a test-context hit must not borrow a production loop, got: {:?}", + result.suspected_files + ); + assert_eq!(result.skipped_test_hits_count, 0); + } + + // ---- trailing `//` comments are not code ---- + + #[test] + fn test_inline_trailing_comment_marker_does_not_open_test_context() { + // Only FULL-LINE `//` comments were treated as non-code, and the marker + // pattern is unanchored — so a marker mentioned at the end of a real + // statement opened test context and muted the production hit below it. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,8 @@ ++let verify_all = true; // mirrored by #[cfg(test)] mod tests below ++for candidate in candidates.iter() { ++ let stored = db.query("SELECT hash FROM users WHERE id = ?"); ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "a marker inside a trailing comment is not test context" + ); + assert_eq!(result.query_in_loop_count, 1); + assert!(!result.suspected_files[0].test_context_only); + } + + #[test] + fn test_braces_in_trailing_comment_do_not_close_test_context() { + // The brace tracker counted braces inside a trailing comment, so a + // comment mentioning `}}` closed the test scope early and reported a + // genuine test-only hit as production. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,10 @@ + #[cfg(test)] + mod tests { ++ let n = 1; // not real braces: }} ++ for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++ } + } +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + !result.perf_regression_suspected, + "braces inside a comment must not leak a test hit into production" + ); + assert_eq!(result.query_in_loop_count, 0); + assert_eq!(result.suspected_files.len(), 1); + assert!(result.suspected_files[0].test_context_only); + } + + #[test] + fn test_cfg_gated_external_module_closes_its_test_context() { + // `#[cfg(test)] mod tests;` annotates an item with no body, so no brace + // ever opened and the "balanced again" close could not fire. The test + // context stayed open for the rest of the hunk and recorded the + // production query-in-loop below it as test-only, dropping it from the + // performance signal entirely. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,10 @@ + #[cfg(test)] + mod tests; ++for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "a body-less test item must not mute the production code below it" + ); + assert_eq!(result.query_in_loop_count, 1); + assert!(!result.suspected_files[0].test_context_only); + } + + #[test] + fn test_cfg_gated_import_closes_its_test_context() { + // The same shape with a `use`: the annotated item ends at its `;`, so + // the context it opened ends there too. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,10 @@ + #[cfg(test)] + use crate::helper; ++for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "a cfg-gated import must not mute the production code below it" + ); + assert_eq!(result.query_in_loop_count, 1); + assert!(!result.suspected_files[0].test_context_only); + } + + #[test] + fn test_a_comparison_in_a_bodyless_initializer_closes_its_test_context() { + // A body-less item may state a VALUE, and after its top-level `=` the + // `<` is a comparison, not a generic opener. Counting it left the + // signature's bracket depth above zero, which is exactly what the `;` + // close checks, so the item's own semicolon could not end the context — + // and every production hit below the test was recorded as test-only. + // Over-detection of test context is the direction that HIDES work. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,10 @@ + #[cfg(test)] + const ENABLED: bool = 1<2; ++for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "a body-less initializer must not mute the production code below it" + ); + assert_eq!(result.query_in_loop_count, 1); + assert!(!result.suspected_files[0].test_context_only); + } + + #[test] + fn test_a_generic_bodyless_initializer_still_closes_its_test_context() { + // The same item whose initializer genuinely names generics. A turbofish + // closes what it opens, so freezing the angle tracking after the `=` + // cannot leave this one stuck either. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,10 @@ + #[cfg(test)] + static NAMES: Vec = Vec::::new(); ++for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "a generic initializer must not mute the production code below it" + ); + assert_eq!(result.query_in_loop_count, 1); + assert!(!result.suspected_files[0].test_context_only); + } + + #[test] + fn test_a_generic_signature_is_still_tracked_before_any_equals() { + // The guard on the new rule: `=` only stops angle tracking AFTER it is + // seen at top level. A generic in an ordinary signature is untouched, so + // a const argument's brace still does not read as the body opener and + // the context still spans the whole test function. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,12 @@ + #[test] + fn run() -> Buffer<{ LIMIT }> + { ++ for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++ } + } +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + !result.perf_regression_suspected, + "a const-generic signature must still hold its test context open" + ); + assert_eq!(result.query_in_loop_count, 0); + assert!(result.suspected_files[0].test_context_only); + } + + #[test] + fn test_cfg_gated_module_with_a_body_still_holds_its_context() { + // The other direction: an item that DOES open a body keeps the context + // until its brace balances, exactly as before. Closing at the first + // `;` inside it would report genuine test-only work as production. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,10 @@ + #[cfg(test)] + mod tests { ++ let n = 1; ++ for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++ } + } +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + !result.perf_regression_suspected, + "a statement inside a test module must not close its context" + ); + assert_eq!(result.query_in_loop_count, 0); + assert!(result.suspected_files[0].test_context_only); + } + + #[test] + fn test_const_generic_braces_in_a_signature_do_not_open_the_body() { + // `Buffer<{ LIMIT }>` balances a brace pair in TYPE position, before the + // body exists. Counting it as the item's opener let the very next line + // look like the item closing again, so the test context ended at the + // signature and every loop in the body was recorded as production. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,12 @@ + #[test] + fn run() -> Buffer<{ LIMIT }> + { ++ for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++ } + } +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + !result.perf_regression_suspected, + "a const-generic brace in the signature must not close the test context" + ); + assert_eq!(result.query_in_loop_count, 0); + assert!(result.suspected_files[0].test_context_only); + } + + #[test] + fn test_a_destructured_parameter_does_not_open_the_body() { + // The idiom the corpus actually shows: an extractor parameter matched by + // pattern, as `rmcp`, `leptos` and `sqlx` write their handlers. The brace + // pair sits inside the parameter list, lines before the body exists. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,12 @@ + #[tokio::test] + async fn slow_tool( + Parameters(SlowToolRequest { sleep_ms }): Parameters, + ) -> Result<(), Error> { ++ for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++ } + } +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + !result.perf_regression_suspected, + "a destructured parameter must not close the test context" + ); + assert_eq!(result.query_in_loop_count, 0); + assert!(result.suspected_files[0].test_context_only); + } + + #[test] + fn test_a_comparison_inside_const_braces_does_not_hold_the_context_open() { + // A `<` inside a const argument is a comparison, not a generic opener. + // Counting it left the signature's bracket depth stuck above zero, the + // real body brace was then read as another type-level brace, and the + // context never closed — muting every production hit after the test, + // which is the error direction that HIDES work. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,12 @@ + #[test] + fn run() -> Buffer<{ 1 < 2 }> { + let n = 1; + } ++for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "production code after the test must not be muted by a comparison in a const argument" + ); + assert_eq!(result.query_in_loop_count, 1); + assert!(!result.suspected_files[0].test_context_only); + } + + #[test] + fn test_a_compact_comparison_inside_const_braces_does_not_hold_the_context_open() { + // The same comparison written without spaces. `<` after a DIGIT passes + // the "follows an identifier" test that catches the spaced spelling, so + // the depth was still stuck above zero, the real body brace read as + // another type-level brace, and the context never closed. Whitespace is + // formatting, not meaning: both spellings must judge the same. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,12 @@ + #[test] + fn run() -> Buffer<{1<2}> { + let n = 1; + } ++for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "production code after the test must not be muted by a compact comparison" + ); + assert_eq!(result.query_in_loop_count, 1); + assert!(!result.suspected_files[0].test_context_only); + } + + #[test] + fn test_a_turbofish_inside_const_braces_keeps_the_signature_balanced() { + // The shape the corpus actually carries in a const argument: a turbofish + // that opens and closes its own generic list. Freezing the signature's + // depth inside the braces must not break it — the pair is balanced + // against itself, so the item's real body opener is still found and the + // context still ends at the body's end. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,12 @@ + #[test] + fn run() -> Buffer<{size_of::()}> { + let n = 1; + } ++for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "a turbofish in a const argument must not mute production code below the test" + ); + assert_eq!(result.query_in_loop_count, 1); + assert!(!result.suspected_files[0].test_context_only); + } + + #[test] + fn test_a_qualified_path_inside_const_braces_keeps_the_signature_balanced() { + // The only shape the corpus actually carries here (crypto-bigint): + // `Uint<{ ::LIMBS / 2 }>`. Its trace CHANGES under the freeze — the + // path's `>` used to decrement the outer list, leaving the depth at zero + // one closer early, and only clamping kept the verdict right. Frozen, the + // outer `>` does that job itself. Same verdict, different arithmetic, so + // it is worth holding. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,12 @@ + #[test] + fn split(&self) -> Uint<{ ::LIMBS / 2 }> { + let n = 1; + } ++for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "a qualified path in a const argument must not mute production code below the test" + ); + assert_eq!(result.query_in_loop_count, 1); + assert!(!result.suspected_files[0].test_context_only); + } + + #[test] + fn test_a_const_generic_signature_still_closes_at_its_real_body_end() { + // The other direction, and the reason the fix cannot simply ignore + // braces in signatures: once the real body opener is found the context + // must still end where the body ends, or production code after the test + // is muted. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,12 @@ + #[test] + fn run() -> Buffer<{ LIMIT }> + { + let n = 1; + } ++for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "production code after the test body must not stay muted" + ); + assert_eq!(result.query_in_loop_count, 1); + assert!(!result.suspected_files[0].test_context_only); + } + + #[test] + fn test_braces_in_string_literals_do_not_move_test_scope() { + // A brace typed inside a literal is data, not syntax. Counting it closed + // the test scope early and reported the test-only query-in-loop below it + // as a production perf suspect. + let patch = r##"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,12 @@ + #[cfg(test)] + mod tests { ++ const CLOSE: &str = "}"; ++ const RAW: &str = r#"}"#; ++ const CH: char = '}'; ++ for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++ } + } +"##; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + !result.perf_regression_suspected, + "a brace inside a literal must not leak a test hit into production" + ); + assert_eq!(result.query_in_loop_count, 0); + assert_eq!(result.suspected_files.len(), 1); + assert!(result.suspected_files[0].test_context_only); + } + + #[test] + fn test_open_brace_in_string_literal_does_not_hold_test_scope_open() { + // The mirror failure: an unmatched `{` inside a literal kept the scope + // open past the end of the test module and muted the production hit + // that followed it. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,12 @@ + #[cfg(test)] + mod tests { ++ const OPEN: &str = "{"; ++} ++for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "a production query-in-loop after the test module must stay reported" + ); + assert_eq!(result.query_in_loop_count, 1); + } + + #[test] + fn test_braces_in_block_comments_do_not_move_test_scope() { + // Commenting out a block of code is what block comments are FOR, so a + // `}` inside one is ordinary. Counting it closed the test scope early + // and reported the test-only query-in-loop below it as production. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,12 @@ + #[cfg(test)] + mod tests { ++ /* removed the tail of the old case: ++ } ++ */ ++ for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++ } + } +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + !result.perf_regression_suspected, + "a brace inside a block comment must not leak a test hit into production" + ); + assert_eq!(result.query_in_loop_count, 0); + } + + #[test] + fn test_open_brace_in_block_comment_does_not_hold_test_scope_open() { + // The mirror failure: a `{` inside a block comment kept the test scope + // open past the end of the module and muted the production hit below. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,12 @@ + #[cfg(test)] + mod tests { ++ /* old shape: ++ fn f() { ++ */ ++} ++for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "a production query-in-loop after the test module must stay reported" + ); + assert_eq!(result.query_in_loop_count, 1); + } + + #[test] + fn test_glob_pattern_is_not_a_block_comment() { + // `format!("{}/*.{}")` carries `/*` inside a string literal. Reading it + // as a comment opener would swallow the rest of the hunk and mute every + // production hit after it — a worse failure than the one block-comment + // tracking fixes, and a far more common line in real diffs. + let patch = r#"diff --git a/src/auth.rs b/src/auth.rs ++++ b/src/auth.rs +@@ -1,4 +1,12 @@ ++let pattern = format!("{}/*.{}", dir, ext); ++for candidate in candidates.iter() { ++ let stored = db.query("SELECT 1"); ++} +"#; + let ctx = RegressionContext { + patch_text: Some(patch.to_string()), + ..Default::default() + }; + + let result = analyze(&ctx); + assert!( + result.perf_regression_suspected, + "a glob pattern must not open a block comment over the rest of the hunk" + ); + assert_eq!(result.query_in_loop_count, 1); + } + + #[test] + fn test_added_line_test_context_is_aligned_with_added_lines() { + let hunk = "@@ -1,4 +1,9 @@\n+let prod = 1;\n-let removed = 2;\n #[cfg(test)]\n mod tests {\n+ let inside = 3;\n }\n+let after = 4;\n"; + assert_eq!( + added_line_test_context("src/auth.rs", hunk), + vec![false, true, false], + "flags must line up with added lines only, in order" + ); + assert_eq!( + added_line_test_context("src/auth.ts", hunk), + vec![false, false, false], + "inline Rust test context does not apply to non-Rust files" ); } } diff --git a/src/rust_source.rs b/src/rust_source.rs new file mode 100644 index 0000000..5381c46 --- /dev/null +++ b/src/rust_source.rs @@ -0,0 +1,648 @@ +//! Reading Rust source out of a unified diff, one line at a time. +//! +//! Several trackers walk diff text counting delimiters to decide something: +//! the perf tracker (is this line inside inline `#[cfg(test)]` context?), the +//! breaking-change tracker (which inline `mod` is this line in?) and the +//! declaration accumulator (has this public declaration ended?). All three are +//! fooled by the same thing — a delimiter that is not syntax — and all three +//! need the same answer, so the scanner lives in one place rather than being +//! reimplemented per consumer. + +use std::borrow::Cow; + +/// Line-by-line source reader that remembers a construct left open. +/// +/// Block comments and string literals are the two things a per-line scanner +/// cannot resolve on its own: `/* } */` spread over three lines hides a brace +/// that never reaches the tracker as syntax, and so does +/// `const T: &str = "{\n}";`. Consumers that walk a hunk in order keep one +/// scanner for that walk and [`reset`](Self::reset) it at boundaries where the +/// text is no longer contiguous. +#[derive(Default)] +pub(crate) struct SourceScanner { + state: ScanState, +} + +impl SourceScanner { + /// The code part of `line`, continuing any construct still open. + /// + /// Literal BODIES are dropped along with the comments, because this view + /// exists for delimiter counting: a brace typed inside a string is text, not + /// structure. + pub(crate) fn code_only<'a>(&mut self, line: &'a str) -> Cow<'a, str> { + scan(line, &mut self.state, Literals::Drop) + } + + /// The same, with string and char literals kept verbatim. + /// + /// For callers that compare source rather than count delimiters. Dropping a + /// literal is right for a delimiter tracker and wrong for an identity: + /// `pub const GREETING: &str = "hello";` and the same line ending `"bye";` + /// are not the same declaration, and reading them as one would pair a real + /// value change away as an unchanged re-add. + pub(crate) fn code_with_literals<'a>(&mut self, line: &'a str) -> Cow<'a, str> { + scan(line, &mut self.state, Literals::Keep) + } + + /// The same again, with whitespace OUTSIDE the literals removed. + /// + /// For callers that treat spacing as formatting but a literal's contents as + /// a value: `#[cfg(api = "a b")]` and `#[cfg( api="a b" )]` are one gate, + /// while `#[cfg(api = "ab")]` is another. Stripping whitespace from the + /// whole line instead made those last two equal and paired a + /// configuration-specific removal away. + pub(crate) fn code_with_literals_dense<'a>(&mut self, line: &'a str) -> Cow<'a, str> { + scan(line, &mut self.state, Literals::KeepDense) + } + + /// Is a string literal still open at the point the last line ended? + /// + /// The line break that follows is then literal CONTENT, not layout. A caller + /// comparing source needs the difference: a break inside a literal is part + /// of the value, while one between two halves of a reflowed declaration says + /// nothing about the API. + pub(crate) fn carries_literal(&self) -> bool { + self.state.open_literal.is_some() + } + + /// Forget a comment or literal left open: the next line is not contiguous + /// with the last one (a new hunk, a new file). + /// + /// This is also the boundary at which carrying stops being sound. A hunk + /// may START in the middle of a literal, and then its closing delimiter + /// reads as an opener — measured at 1 hunk in 872 over this repo's history, + /// against 29 hunk sides in the same history whose brace counting the + /// carrying fixes. The residue never outlives the hunk. + pub(crate) fn reset(&mut self) { + self.state = ScanState::default(); + } +} + +/// What a scan does with the literals it resolves. +/// +/// Every view resolves comments and literals identically — only the output +/// differs, so a scanner's carried state advances the same way whichever one a +/// caller asks for. +#[derive(Clone, Copy)] +enum Literals { + /// Emit nothing for a literal: it is text, and its delimiters are not syntax. + Drop, + /// Emit the literal verbatim, opening and closing delimiters included. + Keep, + /// The same, and drop whitespace everywhere else. + /// + /// The only mode that changes what is emitted for NON-literal code, and it + /// changes only spacing. It exists so a caller normalizing formatting cannot + /// reach inside a value while doing it. + KeepDense, +} + +impl Literals { + fn emit(self, out: &mut String, literal: &str) { + if matches!(self, Literals::Keep | Literals::KeepDense) { + out.push_str(literal); + } + } + + /// Does whitespace outside a literal survive into the output? + fn keeps_spacing(self) -> bool { + !matches!(self, Literals::KeepDense) + } +} + +/// What an earlier line left open. +#[derive(Default)] +struct ScanState { + /// `/* … */` nesting carried in (Rust block comments nest). + block_comment_depth: u32, + /// A string literal whose closing delimiter has not been seen yet. + open_literal: Option, +} + +/// A string literal still waiting for its closing delimiter. +#[derive(Clone, Copy)] +enum OpenLiteral { + /// `"…` — closed by the first unescaped `"`. + Normal, + /// `r#"…` / `br##"…` — closed by `"` plus exactly this many `#`. No escapes. + Raw { hashes: usize }, +} + +/// One pass over `line`, dropping comments and treating literals per `literals`. +/// +/// `state` is what earlier lines left open — a nested block comment, a string +/// literal — and is updated in place. It advances identically for every +/// `literals` mode: the mode decides what is written out, never what is read. +/// +/// Comments and literals are resolved in the SAME pass, which is what keeps a +/// delimiter from being read in the wrong language: `"http://x"` is a string, +/// not a comment, and `format!("{}/*.{}", dir, ext)` is a glob pattern, not a +/// block comment swallowing the rest of the file. Normal strings (with `\` +/// escapes), raw strings (`r"…"`, `r#"…"#`, `br##"…"##`, `cr#"…"#`) and char literals +/// (including `'\u{7b}'`) are recognised; a `'` that does not close as a char +/// literal is a lifetime and is left alone. A char literal cannot span lines, +/// so only strings are carried. +fn scan<'a>(line: &'a str, state: &mut ScanState, literals: Literals) -> Cow<'a, str> { + let bytes = line.as_bytes(); + if literals.keeps_spacing() + && state.block_comment_depth == 0 + && state.open_literal.is_none() + && !bytes.iter().any(|b| matches!(b, b'"' | b'\'')) + && !line.contains("//") + && !line.contains("/*") + { + return Cow::Borrowed(line); + } + + let mut out = String::with_capacity(line.len()); + let mut i = 0; + // A literal opened on an earlier line owns the start of this one. + if let Some(open) = state.open_literal { + match literal_close(line, 0, open) { + Some(end) => { + state.open_literal = None; + literals.emit(&mut out, &line[..end]); + i = end; + } + None => { + literals.emit(&mut out, line); + return Cow::Owned(out); + } + } + } + + while i < bytes.len() { + if state.block_comment_depth > 0 { + if bytes[i] == b'*' && bytes.get(i + 1) == Some(&b'/') { + state.block_comment_depth -= 1; + i += 2; + } else if bytes[i] == b'/' && bytes.get(i + 1) == Some(&b'*') { + state.block_comment_depth += 1; + i += 2; + } else { + i += next_char_len(line, i); + } + continue; + } + + if let Some(raw) = raw_string_start(line, i) { + match literal_close(line, raw.body_start, raw.open) { + Some(end) => { + literals.emit(&mut out, &line[i..end]); + i = end; + } + None => { + state.open_literal = Some(raw.open); + literals.emit(&mut out, &line[i..]); + return Cow::Owned(out); + } + } + continue; + } + + match bytes[i] { + b'"' => match literal_close(line, i + 1, OpenLiteral::Normal) { + Some(end) => { + literals.emit(&mut out, &line[i..end]); + i = end; + } + None => { + state.open_literal = Some(OpenLiteral::Normal); + literals.emit(&mut out, &line[i..]); + return Cow::Owned(out); + } + }, + b'\'' => match char_literal_end(line, i) { + Some(end) => { + literals.emit(&mut out, &line[i..end]); + i = end; + } + None => { + out.push('\''); + i += 1; + } + }, + // The rest of the line is a `//` comment: nothing after it is code. + b'/' if bytes.get(i + 1) == Some(&b'/') => return Cow::Owned(out), + b'/' if bytes.get(i + 1) == Some(&b'*') => { + state.block_comment_depth += 1; + i += 2; + } + _ => { + let ch = line[i..] + .chars() + .next() + .expect("index sits on a char boundary"); + // The one place a mode may drop non-literal code, and it drops + // only spacing: this arm is reached exactly when `ch` is outside + // every literal and comment. + if literals.keeps_spacing() || !ch.is_whitespace() { + out.push(ch); + } + i += ch.len_utf8(); + } + } + } + Cow::Owned(out) +} + +/// Byte length of the char starting at `i`. +fn next_char_len(line: &str, i: usize) -> usize { + line[i..].chars().next().map_or(1, char::len_utf8) +} + +/// A raw string opener found in the text. +struct RawStringStart { + /// Index just past the opening `"`, where the literal body begins. + body_start: usize, + open: OpenLiteral, +} + +/// The raw string opening at `start`, or `None` if none opens there. +/// +/// Every raw form is accepted: `r"…"`, `br"…"` and `cr"…"`, each with any hash +/// count. A raw literal the scanner fails to recognize is worse than an unknown +/// token, because its body is then read as code: the first interior `"` opens a +/// phantom ordinary string, and the braces around it are counted as syntax. +fn raw_string_start(code: &str, start: usize) -> Option { + let bytes = code.as_bytes(); + // The prefix must be a token start, otherwise `bar"` would look like `b` + `"`. + if start > 0 && (bytes[start - 1].is_ascii_alphanumeric() || bytes[start - 1] == b'_') { + return None; + } + + let mut i = start; + // `b` (byte string, 1.0) and `c` (C string, 1.77) both take the raw form. + // Only the RAW forms need recognizing here: `b"…"` and `c"…"` escape exactly + // like an ordinary string, so the `"` arm already blanks them correctly. + if matches!(bytes.get(i), Some(&b'b') | Some(&b'c')) { + i += 1; + } + if bytes.get(i) != Some(&b'r') { + return None; + } + i += 1; + + let hash_start = i; + while bytes.get(i) == Some(&b'#') { + i += 1; + } + let hashes = i - hash_start; + if bytes.get(i) != Some(&b'"') { + return None; + } + Some(RawStringStart { + body_start: i + 1, + open: OpenLiteral::Raw { hashes }, + }) +} + +/// Index just past the closing delimiter of `open`, searching from `from`, or +/// `None` when the literal runs past the end of `code`. +/// +/// `None` is the whole point of carrying literal state: it says the literal is +/// still open, so the NEXT line's leading text is body, not code. +fn literal_close(code: &str, from: usize, open: OpenLiteral) -> Option { + let bytes = code.as_bytes(); + let mut i = from; + match open { + OpenLiteral::Normal => { + while i < bytes.len() { + match bytes[i] { + b'\\' => i += 2, + b'"' => return Some(i + 1), + _ => i += 1, + } + } + None + } + // Raw strings have no escapes: the only terminator is a quote followed + // by exactly as many hashes as the opener carried. + OpenLiteral::Raw { hashes } => { + while i < bytes.len() { + if bytes[i] == b'"' + && bytes.len() - (i + 1) >= hashes + && bytes[i + 1..].iter().take(hashes).all(|b| *b == b'#') + { + return Some(i + 1 + hashes); + } + i += 1; + } + None + } + } +} + +/// End index of the char literal opening at `start`, or `None` for a lifetime. +fn char_literal_end(code: &str, start: usize) -> Option { + let bytes = code.as_bytes(); + if bytes.get(start + 1) == Some(&b'\\') { + // `'\''`, `'\\'`, `'\u{7b}'`: skip the escaped byte, then find the close. + // Bounded so a stray backslash cannot swallow the rest of the line. + let mut i = start + 3; + let limit = (start + 12).min(bytes.len()); + while i < limit { + if bytes[i] == b'\'' { + return Some(i + 1); + } + i += 1; + } + return None; + } + + let ch = code.get(start + 1..)?.chars().next()?; + let close = start + 1 + ch.len_utf8(); + (bytes.get(close) == Some(&b'\'')).then_some(close + 1) +} + +#[cfg(test)] +mod tests { + use super::*; + + /// One line read by a scanner that carries nothing in from before it. + /// + /// Every consumer keeps a scanner across the lines of one hunk or one + /// declaration; these cases are about what a SINGLE line reduces to, so + /// each starts clean. What a scanner carries between lines is covered by + /// the `SourceScanner` cases below. + fn code_only(line: &str) -> Cow<'_, str> { + SourceScanner::default().code_only(line) + } + + #[test] + fn line_comments_are_not_code_but_a_url_is_not_a_comment() { + assert_eq!(code_only("// whole line {"), ""); + assert_eq!(code_only("let x = 1; // trailing {"), "let x = 1; "); + // Truncating at the `//` of a URL would drop the brace after it and + // corrupt the depth in the other direction. + assert_eq!( + code_only("let url = \"https://example.com\"; {"), + "let url = ; {" + ); + assert_eq!(code_only("let x = 1;"), "let x = 1;"); + } + + #[test] + fn literal_contents_are_blanked_but_lifetimes_survive() { + assert_eq!(code_only("let s = \"} {\";"), "let s = ;"); + assert_eq!(code_only("let c = '}';"), "let c = ;"); + assert_eq!(code_only("let e = '\\u{7b}';"), "let e = ;"); + assert_eq!(code_only("let q = \"\\\"}\";"), "let q = ;"); + assert_eq!(code_only("let r = r#\"}\"#;"), "let r = ;"); + assert_eq!(code_only("let b = br##\"}\"##;"), "let b = ;"); + // A lifetime is not a char literal and must survive untouched. + assert_eq!( + code_only("fn f<'a>(x: &'a str) {"), + "fn f<'a>(x: &'a str) {" + ); + assert_eq!(code_only("if depth > 0 {"), "if depth > 0 {"); + } + + #[test] + fn the_literal_keeping_view_drops_only_the_comments() { + fn kept(line: &str) -> String { + SourceScanner::default() + .code_with_literals(line) + .into_owned() + } + + // Same comment resolution as `code_only` … + assert_eq!(kept("let x = 1; // trailing {"), "let x = 1; "); + assert_eq!( + kept("let x = 1; /* mid */ let y = 2;"), + "let x = 1; let y = 2;" + ); + // … but the literal is source, not a delimiter to be silenced. + assert_eq!(kept("let s = \"} {\";"), "let s = \"} {\";"); + assert_eq!(kept("let c = '}';"), "let c = '}';"); + assert_eq!(kept("let r = r#\"}\"#;"), "let r = r#\"}\"#;"); + assert_eq!(kept("let b = br##\"}\"##;"), "let b = br##\"}\"##;"); + // A `//` inside a raw string is still literal text on either view. + assert_eq!( + kept("let s = r#\"a \" b // c\"#; {"), + "let s = r#\"a \" b // c\"#; {" + ); + // A lifetime is not a literal and is untouched by either view. + assert_eq!(kept("fn f<'a>(x: &'a str) {"), "fn f<'a>(x: &'a str) {"); + } + + #[test] + fn the_literal_keeping_view_carries_an_open_literal_across_lines() { + let mut scanner = SourceScanner::default(); + assert_eq!( + scanner.code_with_literals("let s = \"start {"), + "let s = \"start {" + ); + assert_eq!( + scanner.code_with_literals("still inside }"), + "still inside }" + ); + assert_eq!(scanner.code_with_literals("end\"; // tail"), "end\"; "); + } + + #[test] + fn a_raw_string_holding_a_quote_and_a_slash_slash_stays_one_literal() { + // The reason comments and literals are resolved in ONE pass: a + // comment-stripping pass that only knows `"…"` sees the interior quote + // of `r#"a " b // c"#` as closing the string, then reads the `//` as a + // real comment and truncates the line — dropping the `{` after it and + // corrupting the depth for both the perf and module-scope trackers. + assert_eq!(code_only("let s = r#\"a \" b // c\"#; {"), "let s = ; {"); + assert_eq!(code_only("let b = br##\"x \" // y\"##; }"), "let b = ; }"); + // A `//` genuinely after the literal still ends the code. + assert_eq!(code_only("let s = r#\"a\"#; // trailing {"), "let s = ; "); + } + + #[test] + fn block_comments_are_not_code() { + assert_eq!(code_only("let a = 5 /* } */ ;"), "let a = 5 ;"); + // Rust block comments nest. + assert_eq!(code_only("a /* x /* } */ y */ b"), "a b"); + assert_eq!(code_only("/** doc { */ fn f() {"), " fn f() {"); + } + + #[test] + fn a_block_comment_opener_inside_a_string_is_data() { + // Glob patterns carry `/*` far more often than Rust code carries a + // block comment. Reading one as a comment opener would swallow the + // rest of the line — and, with a scanner, the rest of the hunk. + assert_eq!( + code_only("let p = format!(\"{}/*.{}\", dir, ext);"), + "let p = format!(, dir, ext);" + ); + assert_eq!(code_only("let g = \"**/\"; {"), "let g = ; {"); + } + + #[test] + fn a_block_comment_stays_open_across_lines() { + let mut scanner = SourceScanner::default(); + assert_eq!(scanner.code_only("mod tests { /* start"), "mod tests { "); + assert_eq!(scanner.code_only(" } still commented"), ""); + assert_eq!( + scanner.code_only(" end */ let x = 1; {"), + " let x = 1; {" + ); + // Nothing is open any more, so the next line is ordinary code. + assert_eq!(scanner.code_only("}"), "}"); + } + + #[test] + fn reset_forgets_a_comment_left_open() { + let mut scanner = SourceScanner::default(); + assert_eq!(scanner.code_only("/* opened and never closed"), ""); + scanner.reset(); + assert_eq!( + scanner.code_only("pub struct Config {"), + "pub struct Config {" + ); + } + + #[test] + fn a_normal_string_stays_open_across_lines() { + // A string literal spans lines exactly like a block comment does, and + // its body is data on every one of them. Reading the tail as code made + // the closing `"` look like an OPENER and the `}` in front of it look + // like syntax — a brace that pops `mod inner` one level early, after + // which a removed `inner::Config` carries an unknown scope and pairs + // with any addition, hiding a real API removal. + let mut scanner = SourceScanner::default(); + assert_eq!(scanner.code_only("mod inner {"), "mod inner {"); + assert_eq!( + scanner.code_only(" const T: &str = \"{"), + " const T: &str = " + ); + assert_eq!(scanner.code_only("}\";"), ";"); + assert_eq!(scanner.code_only("}"), "}"); + } + + #[test] + fn a_raw_string_stays_open_across_lines_until_its_own_delimiter() { + // Multi-line raw strings are how JSON fixtures are written, so their + // bodies are full of braces. The closing delimiter is `"` plus exactly + // as many hashes as the opener carried: an interior `"#` with the wrong + // hash count does not end it. + let mut scanner = SourceScanner::default(); + assert_eq!(scanner.code_only("let j = br##\"{"), "let j = "); + assert_eq!(scanner.code_only(" \"a\": \"x\"#,"), ""); + assert_eq!(scanner.code_only("}\"##; {"), "; {"); + } + + #[test] + fn the_dense_view_normalizes_spacing_without_reaching_into_a_literal() { + // Two questions, one line: spacing outside a literal is formatting, and + // spacing inside one is value. A caller that answered the first with a + // blanket whitespace filter answered the second wrongly. + let mut scanner = SourceScanner::default(); + assert_eq!( + scanner.code_with_literals_dense("#[cfg( api = \"a b\" )]"), + "#[cfg(api=\"a b\")]" + ); + assert_eq!( + scanner.code_with_literals_dense("#[cfg(api=\"ab\")]"), + "#[cfg(api=\"ab\")]" + ); + // A comment is still resolved away, and the spacing it leaves behind + // with it. + assert_eq!( + scanner.code_with_literals_dense("#[cfg(unix)] // why"), + "#[cfg(unix)]" + ); + } + + #[test] + fn the_dense_view_keeps_a_carried_literal_verbatim() { + // The line break inside a multi-line literal is not this view's to + // normalize either: it only ever removes spacing it can see is outside + // every literal. + let mut scanner = SourceScanner::default(); + assert_eq!( + scanner.code_with_literals_dense("const T: &str = \"a b"), + "constT:&str=\"a b" + ); + assert_eq!(scanner.code_with_literals_dense(" c d\";"), " c d\";"); + } + + #[test] + fn the_other_views_still_keep_their_spacing() { + // Guard on the widening: only the dense mode drops non-literal + // whitespace, and the two established views are untouched by it. + let mut scanner = SourceScanner::default(); + assert_eq!( + scanner.code_with_literals("#[cfg( api = \"a b\" )]"), + "#[cfg( api = \"a b\" )]" + ); + let mut scanner = SourceScanner::default(); + assert_eq!( + scanner.code_only("#[cfg( api = \"a b\" )]"), + "#[cfg( api = )]" + ); + } + + #[test] + fn an_escaped_quote_does_not_close_a_carried_string() { + let mut scanner = SourceScanner::default(); + assert_eq!(scanner.code_only("let s = \"one {"), "let s = "); + assert_eq!(scanner.code_only("two \\\" still inside {"), ""); + assert_eq!(scanner.code_only("three\"; }"), "; }"); + } + + #[test] + fn a_raw_c_string_is_a_raw_string() { + // `cr"…"` / `cr#"…"#` (Rust 1.77) carry the same hash-counted delimiter + // as `r`/`br`. Reading only `r` and `br` left the `c` as code and the + // following `"` as an ORDINARY string opener, so an interior quote + // closed a literal that was still open and the braces after it were + // counted as syntax. + let mut scanner = SourceScanner::default(); + assert_eq!( + scanner.code_only(r####"let s = cr#"a " {"#; }"####), + "let s = ; }" + ); + // The same delimiter rule across lines: only `"#` with the opener's own + // hash count ends it. + let mut scanner = SourceScanner::default(); + assert_eq!(scanner.code_only("let s = cr#\"{"), "let s = "); + assert_eq!(scanner.code_only(" \"still inside\","), ""); + assert_eq!(scanner.code_only("}\"#; {"), "; {"); + } + + #[test] + fn a_c_string_is_still_an_ordinary_literal() { + // Guard the neighbour: `c"…"` is NOT raw, so it keeps normal escape + // handling — the escaped quote does not close it and the brace in its + // body never reaches the tracker. + // + // The `c` itself survives into the code text, exactly as `b"…"` has + // always left its `b`. That is left alone deliberately: a bare prefix + // letter is not a delimiter and cannot form a keyword, so no consumer + // reads it, and consuming it would change `b` handling for no measured + // defect. + let mut scanner = SourceScanner::default(); + assert_eq!( + scanner.code_only(r#"let s = c"a \" { b"; }"#), + "let s = c; }" + ); + assert_eq!( + scanner.code_only(r#"let s = b"a \" { b"; }"#), + "let s = b; }" + ); + } + + #[test] + fn reset_forgets_a_literal_left_open() { + // The hunk boundary is where carrying stops being deterministic: the + // next hunk may start anywhere, including outside the literal. Every + // consumer resets there, and the reset must clear the literal for the + // same reason it clears the comment. + let mut scanner = SourceScanner::default(); + assert_eq!( + scanner.code_only("let s = \"opened and never closed"), + "let s = " + ); + scanner.reset(); + assert_eq!( + scanner.code_only("pub struct Config {"), + "pub struct Config {" + ); + } +} diff --git a/tests/gate_exit_codes.rs b/tests/gate_exit_codes.rs index 17debc0..4892633 100644 --- a/tests/gate_exit_codes.rs +++ b/tests/gate_exit_codes.rs @@ -163,6 +163,77 @@ fn gate_exits_one_for_block_verdict() { prview_gate_command(repo).arg("gate").assert().code(1); } +/// A pack whose `MERGE_GATE.json` is gone carries no verdict. `prview --ci` used +/// to paper over that by re-deriving the decision from the in-memory policy +/// engine — the one path where `allow_merge: true` could sit beside a +/// `CONDITIONAL` verdict. The reader now fails loud with the same execution-error +/// exit code the gate uses, and the contract has to hold at the process boundary. +#[test] +fn ci_run_exits_three_when_pack_has_no_merge_gate() { + let temp = create_gate_fixture(); + let repo = temp.path(); + let home = tempfile::tempdir().expect("prview home"); + // Built once: the helper copies git into the fixture bin dir, which is not + // writable a second time. + let path = path_without_semgrep(repo); + + // 1. A real run, so the pack on disk is a genuine one (metadata included). + let assert = Command::new(assert_cmd::cargo::cargo_bin!("prview")) + .current_dir(repo) + .env("PATH", &path) + .env("PRVIEW_HOME", home.path()) + .args(["--ci", "--quiet", "--no-zip", "--no-heuristics"]) + .assert(); + let first_code = assert.get_output().status.code(); + assert!( + matches!(first_code, Some(0) | Some(1)), + "seeding run must produce a verdict, got exit {first_code:?}" + ); + + // 2. Amputate the verdict artifact, leaving an otherwise complete pack. + let mut removed = 0usize; + for gate in walk_merge_gate_json(home.path()) { + fs::remove_file(&gate).expect("remove MERGE_GATE.json"); + removed += 1; + } + assert_eq!( + removed, 1, + "seeding run must write exactly one MERGE_GATE.json" + ); + + // 3. `--update` re-reads that pack (HEAD is unchanged). No verdict is + // readable, so the process must report an execution error, not a guess. + Command::new(assert_cmd::cargo::cargo_bin!("prview")) + .current_dir(repo) + .env("PATH", &path) + .env("PRVIEW_HOME", home.path()) + .args(["--ci", "--update", "--quiet", "--no-zip", "--no-heuristics"]) + .assert() + .code(3); +} + +/// Every `00_summary/MERGE_GATE.json` under a prview home. +fn walk_merge_gate_json(root: &Path) -> Vec { + let mut found = Vec::new(); + let Ok(entries) = fs::read_dir(root) else { + return found; + }; + for entry in entries.flatten() { + let path = entry.path(); + // `latest` is a symlink to the newest run; following it would visit the + // same pack twice. + if path.is_symlink() { + continue; + } + if path.is_dir() { + found.extend(walk_merge_gate_json(&path)); + } else if path.file_name().is_some_and(|n| n == "MERGE_GATE.json") { + found.push(path); + } + } + found +} + #[test] fn gate_exits_three_when_it_cannot_execute() { // Outside a git repository the review cannot run, so the gate reports an diff --git a/tests/json_contract.rs b/tests/json_contract.rs index 5e24bfe..9e831a1 100644 --- a/tests/json_contract.rs +++ b/tests/json_contract.rs @@ -68,6 +68,24 @@ fn gate_help_documents_exit_code_contract() { .stdout(predicate::str::contains("3 = gate could not execute")); } +#[test] +fn fail_on_warnings_is_documented_and_scoped_to_ci() { + // The escape hatch for the warning→failure change: warnings no longer break + // `--ci` on their own, so a team that wants that exit asks for it. It is + // meaningless outside `--ci`, and clap says so loudly instead of no-opping. + Command::new(assert_cmd::cargo::cargo_bin!("prview")) + .arg("--help") + .assert() + .success() + .stdout(predicate::str::contains("--fail-on-warnings")); + + Command::new(assert_cmd::cargo::cargo_bin!("prview")) + .arg("--fail-on-warnings") + .assert() + .failure() + .stderr(predicate::str::contains("--ci")); +} + #[test] fn gate_json_emits_verdict_and_caveats_from_merge_gate() { let temp = create_fixture_repo(); @@ -284,6 +302,943 @@ fn generated_merge_gate_passes_repo_validator() { .success(); } +/// Accepting schema `2.2` without checking the fields that define it lets a pack +/// omit, mistype, or invent any of them and still pass its own contract gate. +/// Consumers are told to filter `quality_failure_details` on +/// `origin == "failure"`, so an unvalidated origin makes a real failure +/// indistinguishable from a warning; and an entry validated on `origin` alone +/// could be `{"origin": "failure"}` — a failure with no check name and no +/// classification, which no consumer can report or act on. +#[test] +fn validator_rejects_schema_two_two_without_a_usable_origin() { + let temp = create_fixture_repo(); + let repo = temp.path(); + + let payload = run_json_quiet(repo, &["feature/json-contract", "main"]); + let output_dir = Path::new( + payload["output_dir"] + .as_str() + .expect("output_dir should be a string"), + ); + let merge_gate = output_dir.join("00_summary/MERGE_GATE.json"); + let validator = Path::new(env!("CARGO_MANIFEST_DIR")).join("tools/validate_merge_gate.py"); + + let raw = std::fs::read_to_string(&merge_gate).expect("read gate"); + let original: serde_json::Value = serde_json::from_str(&raw).expect("parse gate"); + assert_eq!( + original["schema_version"].as_str(), + Some("2.2"), + "this test pins the schema that introduced `origin`" + ); + + let broken_details = [ + serde_json::json!([{ "name": "Clippy", "classification": "introduced" }]), + serde_json::json!([{ "name": "Clippy", "classification": "introduced", "origin": "failed" }]), + serde_json::json!([{ "name": "Clippy", "classification": "introduced", "origin": true }]), + serde_json::json!([{ "name": "Clippy", "classification": "introduced", "origin": null }]), + serde_json::json!(["Clippy"]), + // The entry the origin-only check waved through: a failure that names + // no check and states no classification. + serde_json::json!([{ "origin": "failure" }]), + // `name` present but useless. + serde_json::json!([{ "name": "", "classification": "introduced", "origin": "failure" }]), + serde_json::json!([{ "name": " ", "classification": "introduced", "origin": "failure" }]), + serde_json::json!([{ "name": 7, "classification": "introduced", "origin": "failure" }]), + // `classification` outside the emitted vocabulary. `preexisting` is the + // spelling of the sibling COUNT field, not of this value — the emitter + // writes `pre-existing`, and accepting any string hid that drift. + serde_json::json!([{ "name": "Clippy", "classification": "preexisting", "origin": "failure" }]), + serde_json::json!([{ "name": "Clippy", "classification": "", "origin": "failure" }]), + serde_json::json!([{ "name": "Clippy", "classification": true, "origin": "failure" }]), + serde_json::json!([{ "name": "Clippy", "origin": "failure" }]), + ]; + + for details in broken_details { + let mut gate = original.clone(); + gate["decision"]["quality_failure_details"] = details.clone(); + std::fs::write( + &merge_gate, + serde_json::to_string_pretty(&gate).expect("serialize gate"), + ) + .expect("write gate"); + + Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert() + .failure(); + } + + // Every classification the emitter can write must validate — the vocabulary + // is pinned to `QualityFailureClass::as_str`, so a validator that spelled + // one of them differently would reject a pack prview itself produced. The + // flag moves with the row because `quality_pass` is derived from these very + // details: only `pre-existing` leaves the gate passing, so pinning one flag + // across all four would test a pack the emitter cannot write. What this loop + // asserts is unchanged — all four spellings validate. + for classification in ["introduced", "pre-existing", "mixed", "unclassified"] { + let mut gate = original.clone(); + gate["decision"]["quality_failure_details"] = serde_json::json!([{ + "name": "Clippy", + "classification": classification, + "origin": "failure", + }]); + gate["decision"]["quality_pass"] = serde_json::json!(classification == "pre-existing"); + std::fs::write( + &merge_gate, + serde_json::to_string_pretty(&gate).expect("serialize gate"), + ) + .expect("write gate"); + + Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert() + .success(); + } + + // The shape the emitter actually writes still validates. + std::fs::write(&merge_gate, &raw).expect("restore gate"); + Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert() + .success(); +} + +/// `quality_pass` and `quality_failure_details` are ONE fact written twice: the +/// emitter sets the flag to `!QualityFailureSummary::has_new_failures()` and +/// serializes the very details that answer it. Validating each side's shape +/// while never comparing them let a pack claim `quality_pass: true` beside an +/// explicitly introduced failure, and both decision readers trust the permissive +/// scalar — so a validator-clean pack could approve a failure it also reports. +/// +/// The check is an equivalence, and the `pre-existing` row is why the obvious +/// one-way rule would be wrong: a failure that predates the diff is emitted +/// beside `quality_pass: true` on purpose. +#[test] +fn validator_rejects_quality_pass_contradicting_its_own_details() { + let temp = create_fixture_repo(); + let repo = temp.path(); + + let payload = run_json_quiet(repo, &["feature/json-contract", "main"]); + let output_dir = Path::new( + payload["output_dir"] + .as_str() + .expect("output_dir should be a string"), + ); + let merge_gate = output_dir.join("00_summary/MERGE_GATE.json"); + let validator = Path::new(env!("CARGO_MANIFEST_DIR")).join("tools/validate_merge_gate.py"); + + let raw = std::fs::read_to_string(&merge_gate).expect("read gate"); + let original: serde_json::Value = serde_json::from_str(&raw).expect("parse gate"); + + let detail = |classification: &str, origin: &str| serde_json::json!([{ "name": "Clippy", "classification": classification, "origin": origin }]); + + // (details, quality_pass, must_validate) + let cases: [(serde_json::Value, bool, bool); 10] = [ + // A gating failure beside a passing flag — the combination the emitter + // cannot produce, and the one that lets a clean-looking pack approve an + // introduced failure. + (detail("introduced", "failure"), true, false), + (detail("mixed", "failure"), true, false), + (detail("unclassified", "failure"), true, false), + // The same rows with the flag the emitter would actually write. + (detail("introduced", "failure"), false, true), + (detail("mixed", "failure"), false, true), + (detail("unclassified", "failure"), false, true), + // Pre-existing failures do NOT gate: this pack is legal and must not be + // rejected by a rule that keys on origin alone. + (detail("pre-existing", "failure"), true, true), + // Warnings never gate either, whatever they classify as. + (detail("introduced", "warning"), true, true), + // The other direction: a failing flag with nothing that could have + // failed it is equally unemittable. + (detail("pre-existing", "failure"), false, false), + (serde_json::json!([]), false, false), + ]; + + for (details, quality_pass, must_validate) in cases { + let mut gate = original.clone(); + gate["decision"]["quality_failure_details"] = details.clone(); + gate["decision"]["quality_pass"] = serde_json::json!(quality_pass); + std::fs::write( + &merge_gate, + serde_json::to_string_pretty(&gate).expect("serialize gate"), + ) + .expect("write gate"); + + let assertion = Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert(); + if must_validate { + assertion.success(); + } else { + assertion.failure(); + } + } + + // The shape the emitter actually writes still validates. + std::fs::write(&merge_gate, &raw).expect("restore gate"); + Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert() + .success(); +} + +/// `quality_pass` is a documented decision axis, and from schema 2.2 the writer +/// emits it unconditionally as a boolean. A 2.2 pack that omits it or states it +/// as a string is therefore not an old pack but a broken one — and both decision +/// readers normalize a present-but-unreadable signal to BLOCK, so a validator +/// that accepted it certified an artifact the CLI and MCP both refuse to trust. +/// +/// Absence stays forgiven BELOW 2.2, where readers derive the flag instead; that +/// carve-out is asserted here too, because tightening it would break every pack +/// written before the field existed. +#[test] +fn validator_requires_a_boolean_quality_pass_from_schema_two_two() { + let temp = create_fixture_repo(); + let repo = temp.path(); + + let payload = run_json_quiet(repo, &["feature/json-contract", "main"]); + let output_dir = Path::new( + payload["output_dir"] + .as_str() + .expect("output_dir should be a string"), + ); + let merge_gate = output_dir.join("00_summary/MERGE_GATE.json"); + let validator = Path::new(env!("CARGO_MANIFEST_DIR")).join("tools/validate_merge_gate.py"); + + let raw = std::fs::read_to_string(&merge_gate).expect("read gate"); + let original: serde_json::Value = serde_json::from_str(&raw).expect("parse gate"); + assert_eq!( + original["schema_version"].as_str(), + Some("2.2"), + "this test pins the schema that requires the field" + ); + assert!( + original["decision"]["quality_pass"].is_boolean(), + "the writer must emit a boolean for the requirement to be safe" + ); + + // Absent, and every non-boolean spelling of it. + let broken = [ + None, + Some(serde_json::json!("false")), + Some(serde_json::json!("true")), + Some(serde_json::json!(0)), + Some(serde_json::json!(1)), + Some(serde_json::json!(null)), + Some(serde_json::json!([])), + Some(serde_json::json!({})), + ]; + + for value in broken { + let mut gate = original.clone(); + match value { + Some(value) => gate["decision"]["quality_pass"] = value, + None => { + gate["decision"] + .as_object_mut() + .expect("decision object") + .remove("quality_pass"); + } + } + std::fs::write( + &merge_gate, + serde_json::to_string_pretty(&gate).expect("serialize gate"), + ) + .expect("write gate"); + + Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert() + .failure(); + } + + // The legacy carve-out: below 2.2 an absent `quality_pass` is an old pack, + // not a broken one, and the readers derive the flag rather than refusing it. + let mut legacy = original.clone(); + legacy["schema_version"] = serde_json::json!("2.1"); + legacy["decision"] + .as_object_mut() + .expect("decision object") + .remove("quality_pass"); + std::fs::write( + &merge_gate, + serde_json::to_string_pretty(&legacy).expect("serialize gate"), + ) + .expect("write gate"); + Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert() + .success(); + + // The shape the emitter actually writes still validates. + std::fs::write(&merge_gate, &raw).expect("restore gate"); + Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert() + .success(); +} + +/// Regression: the validator accepted any non-empty `checks[].status`, so a pack +/// spelling a status the emitter never writes — `WARNINGS` from another writer, +/// a stale artifact `--update` reused unchanged — passed the repository gate +/// while the CLI, which counts warnings against the emitted vocabulary, could not +/// read it. The contract now names that vocabulary, case included. +#[test] +fn validator_rejects_a_check_status_outside_the_emitted_vocabulary() { + let temp = create_fixture_repo(); + let repo = temp.path(); + + let payload = run_json_quiet(repo, &["feature/json-contract", "main"]); + let output_dir = Path::new( + payload["output_dir"] + .as_str() + .expect("output_dir should be a string"), + ); + let merge_gate = output_dir.join("00_summary/MERGE_GATE.json"); + let validator = Path::new(env!("CARGO_MANIFEST_DIR")).join("tools/validate_merge_gate.py"); + + let raw = std::fs::read_to_string(&merge_gate).expect("read gate"); + let original: serde_json::Value = serde_json::from_str(&raw).expect("parse gate"); + assert!( + original["checks"] + .as_array() + .expect("checks array") + .iter() + .all(|check| matches!( + check["status"].as_str(), + Some("passed" | "failed" | "warnings" | "skipped" | "error") + )), + "the emitter writes only the vocabulary this test pins: {:?}", + original["checks"] + ); + + // Recognizable-but-uncanonical spellings, plus the non-strings a bare + // "non-empty" rule never caught either. + let broken = [ + serde_json::json!("WARNINGS"), + serde_json::json!("Warnings"), + serde_json::json!("warning"), + serde_json::json!("warn"), + serde_json::json!("PASSED"), + serde_json::json!("ok"), + serde_json::json!(" warnings"), + serde_json::json!(true), + serde_json::json!(0), + serde_json::json!(null), + ]; + + for value in broken { + let mut gate = original.clone(); + gate["checks"][0]["status"] = value.clone(); + std::fs::write( + &merge_gate, + serde_json::to_string_pretty(&gate).expect("serialize gate"), + ) + .expect("write gate"); + + Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert() + .failure(); + } + + // Every canonical spelling stays accepted, so the vocabulary is a contract + // and not a single-value pin. + for spelling in ["passed", "failed", "warnings", "skipped", "error"] { + let mut gate = original.clone(); + gate["checks"][0]["status"] = serde_json::json!(spelling); + std::fs::write( + &merge_gate, + serde_json::to_string_pretty(&gate).expect("serialize gate"), + ) + .expect("write gate"); + + Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert() + .success(); + } + + // The shape the emitter actually writes still validates. + std::fs::write(&merge_gate, &raw).expect("restore gate"); + Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert() + .success(); +} + +/// The container half of the same contract. The validator already required +/// `checks` to be an array — this pins that, because the CLI now treats a +/// present-but-unreadable list as at least one warning and the two surfaces have +/// to agree on which packs are valid at all. Absence is a separate question and +/// is rejected here too: `checks` has been emitted since schema 1.0. +#[test] +fn validator_rejects_a_checks_list_that_is_not_an_array() { + let temp = create_fixture_repo(); + let repo = temp.path(); + + let payload = run_json_quiet(repo, &["feature/json-contract", "main"]); + let output_dir = Path::new( + payload["output_dir"] + .as_str() + .expect("output_dir should be a string"), + ); + let merge_gate = output_dir.join("00_summary/MERGE_GATE.json"); + let validator = Path::new(env!("CARGO_MANIFEST_DIR")).join("tools/validate_merge_gate.py"); + + let raw = std::fs::read_to_string(&merge_gate).expect("read gate"); + let original: serde_json::Value = serde_json::from_str(&raw).expect("parse gate"); + assert!( + original["checks"].is_array(), + "the emitter writes an array here" + ); + + let broken = [ + Some(serde_json::json!({"semgrep": "warnings"})), + Some(serde_json::json!("warnings")), + Some(serde_json::json!(7)), + Some(serde_json::json!(null)), + Some(serde_json::json!(true)), + None, + ]; + + for value in broken { + let mut gate = original.clone(); + match value { + Some(value) => gate["checks"] = value, + None => { + gate.as_object_mut().expect("gate object").remove("checks"); + } + } + std::fs::write( + &merge_gate, + serde_json::to_string_pretty(&gate).expect("serialize gate"), + ) + .expect("write gate"); + + Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert() + .failure(); + } + + // An empty list is a legitimate shape — a run in which nothing gated — and + // must not be swept up by the same rule. + let mut empty = original.clone(); + empty["checks"] = serde_json::json!([]); + std::fs::write( + &merge_gate, + serde_json::to_string_pretty(&empty).expect("serialize gate"), + ) + .expect("write gate"); + Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert() + .success(); + + // The shape the emitter actually writes still validates. + std::fs::write(&merge_gate, &raw).expect("restore gate"); + Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert() + .success(); +} + +/// The decision axes the 2.2 writer emits unconditionally, from the typed enums +/// in `src/policy/engine.rs`. A 2.2 pack missing one is broken rather than old — +/// and the reconciliation the next test pins can only compare axes that are +/// there and readable in the first place. +#[test] +fn validator_requires_the_decision_axes_schema_two_two_emits() { + let temp = create_fixture_repo(); + let repo = temp.path(); + + let payload = run_json_quiet(repo, &["feature/json-contract", "main"]); + let output_dir = Path::new( + payload["output_dir"] + .as_str() + .expect("output_dir should be a string"), + ); + let merge_gate = output_dir.join("00_summary/MERGE_GATE.json"); + let validator = Path::new(env!("CARGO_MANIFEST_DIR")).join("tools/validate_merge_gate.py"); + + let raw = std::fs::read_to_string(&merge_gate).expect("read gate"); + let original: serde_json::Value = serde_json::from_str(&raw).expect("parse gate"); + assert_eq!( + original["schema_version"].as_str(), + Some("2.2"), + "these requirements are scoped to 2.2" + ); + for axis in [ + "analysis_status", + "merge_recommendation", + "policy_allow_merge", + ] { + assert!( + original["decision"].get(axis).is_some(), + "the emitter writes {axis}: {:?}", + original["decision"] + ); + } + + // (axis, value, must_validate) — `None` removes the key entirely. + let cases: [(&str, Option, bool); 20] = [ + ("analysis_status", None, false), + ("merge_recommendation", None, false), + ("policy_allow_merge", None, false), + // Case is not a spelling the emitter has ever written, and neither is a + // word outside the enum. Same rule as `checks[].status`. + ( + "analysis_status", + Some(serde_json::json!("COMPLETE")), + false, + ), + ( + "analysis_status", + Some(serde_json::json!("Degraded")), + false, + ), + ("analysis_status", Some(serde_json::json!("partial")), false), + ("analysis_status", Some(serde_json::json!(7)), false), + ("analysis_status", Some(serde_json::json!(null)), false), + ( + "merge_recommendation", + Some(serde_json::json!("APPROVE")), + false, + ), + ( + "merge_recommendation", + Some(serde_json::json!("Review_Required")), + false, + ), + // The retired pre-2.1 synonym. Readers still fold it when reading a pack + // off disk; a freshly emitted one may not spell it that way. + ( + "merge_recommendation", + Some(serde_json::json!("hold")), + false, + ), + ("merge_recommendation", Some(serde_json::json!(true)), false), + ("policy_allow_merge", Some(serde_json::json!("true")), false), + ("policy_allow_merge", Some(serde_json::json!(1)), false), + ("policy_allow_merge", Some(serde_json::json!(null)), false), + // Every canonical spelling stays accepted, so this is a vocabulary and + // not a single-value pin. All three confidence values sit at or below + // the CONDITIONAL this pack states, so none of them trips the + // reconciliation rule the next test covers. + ("analysis_status", Some(serde_json::json!("complete")), true), + ("analysis_status", Some(serde_json::json!("degraded")), true), + ( + "analysis_status", + Some(serde_json::json!("incomplete")), + true, + ), + ( + "merge_recommendation", + Some(serde_json::json!("approve")), + true, + ), + // `block` is canonical too, but only beside a BLOCK verdict — the next + // test states it there. + ("policy_allow_merge", Some(serde_json::json!(true)), true), + ]; + + for (axis, value, must_validate) in cases { + let mut gate = original.clone(); + match value { + Some(value) => gate["decision"][axis] = value, + None => { + gate["decision"] + .as_object_mut() + .expect("decision object") + .remove(axis); + } + } + std::fs::write( + &merge_gate, + serde_json::to_string_pretty(&gate).expect("serialize gate"), + ) + .expect("write gate"); + + let assertion = Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert(); + if must_validate { + assertion.success(); + } else { + assertion.failure(); + } + } + + // The shape the emitter actually writes still validates. + std::fs::write(&merge_gate, &raw).expect("restore gate"); + Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert() + .success(); +} + +/// The reconciliation contract itself, ported from the readers into the +/// certification gate. +/// +/// Both readers derive a decision by taking the most conservative axis the pack +/// states, so a `verdict` milder than that maximum is one no consumer will +/// honour: the artifact certifies one outcome and everything downstream computes +/// another. The reported payload — `PASS` beside an `incomplete` analysis, a +/// `block` recommendation and `policy_allow_merge: false` — validated OK before +/// this rule existed. +/// +/// The opposite direction is deliberately still legal, and asserted below: a +/// semgrep scan that passes with parse errors writes `approve` beside `degraded` +/// and the contract turns that into `CONDITIONAL`, so a verdict harsher than its +/// neighbours is a pack the emitter really produces. +#[test] +fn validator_rejects_a_verdict_its_own_axes_contradict() { + let temp = create_fixture_repo(); + let repo = temp.path(); + + let payload = run_json_quiet(repo, &["feature/json-contract", "main"]); + let output_dir = Path::new( + payload["output_dir"] + .as_str() + .expect("output_dir should be a string"), + ); + let merge_gate = output_dir.join("00_summary/MERGE_GATE.json"); + let validator = Path::new(env!("CARGO_MANIFEST_DIR")).join("tools/validate_merge_gate.py"); + + let raw = std::fs::read_to_string(&merge_gate).expect("read gate"); + let original: serde_json::Value = serde_json::from_str(&raw).expect("parse gate"); + + let gating_detail = serde_json::json!([ + { "name": "Clippy", "classification": "introduced", "origin": "failure" } + ]); + + let with = |base: &serde_json::Value, patch: serde_json::Value| { + let mut decision = base.clone(); + for (key, value) in patch.as_object().expect("patch object") { + decision[key] = value.clone(); + } + decision + }; + // A clean pass, the shape every conflicting axis below is measured against. + // Built by patching the emitted decision so the fields this rule says + // nothing about — `decision_reason`, the legacy mirrors, the caveats — stay + // exactly as the writer left them. + let clean = with( + &original["decision"], + serde_json::json!({ + "verdict": "PASS", + "allow_merge": true, + "merge_recommendation": "approve", + "analysis_status": "complete", + "quality_pass": true, + "policy_allow_merge": true, + "blocking_issues": [], + "quality_failure_details": [], + }), + ); + + // (decision patch over `clean`, must_validate) + let cases: [(serde_json::Value, bool); 15] = [ + // The reported payload, verbatim. + ( + serde_json::json!({ + "analysis_status": "incomplete", + "merge_recommendation": "block", + "policy_allow_merge": false, + }), + false, + ), + // One axis at a time, so the rule is not passing on the strength of the + // others. Each of these rules `PASS` out by itself. + ( + serde_json::json!({ "merge_recommendation": "review_required" }), + false, + ), + ( + serde_json::json!({ "merge_recommendation": "block" }), + false, + ), + (serde_json::json!({ "analysis_status": "degraded" }), false), + ( + serde_json::json!({ "analysis_status": "incomplete" }), + false, + ), + ( + serde_json::json!({ + "quality_pass": false, + "quality_failure_details": gating_detail, + }), + false, + ), + (serde_json::json!({ "policy_allow_merge": false }), false), + // The blocker axis stated by its other half. `allow_merge` is already + // false here, so the pre-existing "no merge beside a blocking issue" + // rule is satisfied and only the reconciliation can reject this. + ( + serde_json::json!({ + "verdict": "CONDITIONAL", + "allow_merge": false, + "merge_recommendation": "review_required", + "blocking_issues": ["Semgrep (failed)"], + }), + false, + ), + // A CONDITIONAL that is still milder than a stated block. + ( + serde_json::json!({ + "verdict": "CONDITIONAL", + "allow_merge": false, + "merge_recommendation": "block", + "policy_allow_merge": false, + "blocking_issues": ["Semgrep (failed)"], + }), + false, + ), + // Legal shapes. The clean pass itself, untouched. + (serde_json::json!({}), true), + // CONDITIONAL because the recommendation says so. + ( + serde_json::json!({ + "verdict": "CONDITIONAL", + "allow_merge": false, + "merge_recommendation": "review_required", + }), + true, + ), + // CONDITIONAL because the analysis was degraded, while the recommendation + // still approves — the harsher-verdict shape a passing semgrep scan with + // parse errors writes. + ( + serde_json::json!({ + "verdict": "CONDITIONAL", + "allow_merge": false, + "analysis_status": "degraded", + }), + true, + ), + // CONDITIONAL because quality failed, recommendation still approving. + ( + serde_json::json!({ + "verdict": "CONDITIONAL", + "allow_merge": false, + "quality_pass": false, + "quality_failure_details": gating_detail, + }), + true, + ), + // A healthy BLOCK: every axis at the same rank. + ( + serde_json::json!({ + "verdict": "BLOCK", + "allow_merge": false, + "merge_recommendation": "block", + "analysis_status": "incomplete", + "policy_allow_merge": false, + "blocking_issues": ["Semgrep (failed)"], + }), + true, + ), + // A BLOCK whose blocker is stated ONLY as a policy flag. This case was + // asserted legal when the reconciliation rule landed, on the assumption + // that a pack may state either half of the blocker axis. The source says + // otherwise: the emitter computes + // `policy_allow_merge = blocking_issues.is_empty()` after the last push + // to that list, so the flag and the list are one fact written twice and + // `false` beside an empty list is unemittable. The correction is not a + // relaxation — this shape is now rejected, by the equivalence the test + // below pins. + ( + serde_json::json!({ + "verdict": "BLOCK", + "allow_merge": false, + "merge_recommendation": "block", + "policy_allow_merge": false, + }), + false, + ), + ]; + + for (patch, must_validate) in cases { + let mut gate = original.clone(); + gate["decision"] = with(&clean, patch.clone()); + std::fs::write( + &merge_gate, + serde_json::to_string_pretty(&gate).expect("serialize gate"), + ) + .expect("write gate"); + + let assertion = Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert(); + if must_validate { + assertion.success(); + } else { + assertion.failure(); + } + } + + // The shape the emitter actually writes still validates. + std::fs::write(&merge_gate, &raw).expect("restore gate"); + Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert() + .success(); +} + +/// The blocker axis is one fact written twice, and the gate certifies it as one. +/// +/// The emitter computes `policy_allow_merge = blocking_issues.is_empty()` after +/// the last push to that list, and then emits both verbatim, so the flag has no +/// input the list does not have. The reconciliation rule only used that in the +/// harsher direction — a non-empty list raises the required verdict — which left +/// the two fields free to contradict each other outright: a pack claiming +/// `policy_allow_merge: true` beside real blockers, or `false` beside none, +/// certified clean while every reader treats the pair as a single signal. +/// +/// Both illegal shapes below carry a `BLOCK` verdict, the most conservative one +/// there is, so the reconciliation cannot be what rejects them. Only the +/// equivalence can. +#[test] +fn validator_rejects_a_blocker_flag_its_blocking_issues_contradict() { + let temp = create_fixture_repo(); + let repo = temp.path(); + + let payload = run_json_quiet(repo, &["feature/json-contract", "main"]); + let output_dir = Path::new( + payload["output_dir"] + .as_str() + .expect("output_dir should be a string"), + ); + let merge_gate = output_dir.join("00_summary/MERGE_GATE.json"); + let validator = Path::new(env!("CARGO_MANIFEST_DIR")).join("tools/validate_merge_gate.py"); + + let raw = std::fs::read_to_string(&merge_gate).expect("read gate"); + let original: serde_json::Value = serde_json::from_str(&raw).expect("parse gate"); + + let with = |base: &serde_json::Value, patch: serde_json::Value| { + let mut decision = base.clone(); + for (key, value) in patch.as_object().expect("patch object") { + decision[key] = value.clone(); + } + decision + }; + let clean = with( + &original["decision"], + serde_json::json!({ + "verdict": "PASS", + "allow_merge": true, + "merge_recommendation": "approve", + "analysis_status": "complete", + "quality_pass": true, + "policy_allow_merge": true, + "blocking_issues": [], + "quality_failure_details": [], + }), + ); + + // (decision patch over `clean`, must_validate) + let cases: [(serde_json::Value, bool); 5] = [ + // The reported hole: blockers listed, yet the flag says policy let the + // merge through. + ( + serde_json::json!({ + "verdict": "BLOCK", + "allow_merge": false, + "merge_recommendation": "block", + "policy_allow_merge": true, + "blocking_issues": ["Semgrep (failed)"], + }), + false, + ), + // The other direction, unemittable for the same reason: policy is said to + // have blocked, but the list it is computed from is empty. + ( + serde_json::json!({ + "verdict": "BLOCK", + "allow_merge": false, + "merge_recommendation": "block", + "policy_allow_merge": false, + "blocking_issues": [], + }), + false, + ), + // Legal shapes. The clean pass itself: no blockers, flag agrees. + (serde_json::json!({}), true), + // A healthy BLOCK: blockers listed, flag agrees. + ( + serde_json::json!({ + "verdict": "BLOCK", + "allow_merge": false, + "merge_recommendation": "block", + "analysis_status": "incomplete", + "policy_allow_merge": false, + "blocking_issues": ["Semgrep (failed)"], + }), + true, + ), + // A CONDITIONAL nobody blocked: the equivalence says nothing about the + // axes that made it conditional. + ( + serde_json::json!({ + "verdict": "CONDITIONAL", + "allow_merge": false, + "analysis_status": "degraded", + "policy_allow_merge": true, + "blocking_issues": [], + }), + true, + ), + ]; + + for (patch, must_validate) in cases { + let mut gate = original.clone(); + gate["decision"] = with(&clean, patch.clone()); + std::fs::write( + &merge_gate, + serde_json::to_string_pretty(&gate).expect("serialize gate"), + ) + .expect("write gate"); + + let assertion = Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert(); + if must_validate { + assertion.success(); + } else { + assertion.failure(); + } + } + + // The shape the emitter actually writes still validates. + std::fs::write(&merge_gate, &raw).expect("restore gate"); + Command::new("python3") + .arg(&validator) + .arg(&merge_gate) + .assert() + .success(); +} + /// Resolve an executable by scanning `PATH` (test helper; no external crate). #[cfg(unix)] fn resolve_in_path(bin: &str) -> Option { @@ -736,3 +1691,102 @@ fn init_command_creates_policy_and_updates_gitignore() { "prview-artifacts already in .gitignore", )); } + +/// Recursively find the one `00_summary/MERGE_GATE.json` under `root`. +fn find_merge_gate(root: &Path) -> Option { + let entries = fs::read_dir(root).ok()?; + for entry in entries.filter_map(Result::ok) { + let path = entry.path(); + if path.is_dir() { + if let Some(found) = find_merge_gate(&path) { + return Some(found); + } + } else if path.ends_with("00_summary/MERGE_GATE.json") { + return Some(path); + } + } + None +} + +#[test] +fn an_unchanged_update_run_still_honors_fail_on_warnings() { + // `--update` with no new commits reuses the previous pack, and that pack is + // what the run reports. Forcing exit 0 there made a warnings-clean CI job + // turn green on its second invocation while the reused pack still carried + // warnings — the flag promises exit 1 whenever any pack check warns. + let temp = create_fixture_repo(); + let repo = temp.path(); + let home = tempfile::tempdir().expect("prview home"); + + // A first run produces the pack the update run will reuse. + Command::new(assert_cmd::cargo::cargo_bin!("prview")) + .current_dir(repo) + .env("PRVIEW_HOME", home.path()) + .args([ + "--quick", + "--quiet", + "--no-zip", + "--no-heuristics", + "--no-fetch", + "--local-only", + ]) + .output() + .expect("first run"); + + // Plant a decision that is clean on every axis EXCEPT one warning check, so + // the exit code can only come from the warning-hardening flag. + let gate = find_merge_gate(home.path()).expect("the first run wrote a pack"); + fs::write( + &gate, + r#"{ + "schema_version": "2.2", + "decision": { + "verdict": "PASS", + "merge_recommendation": "approve", + "allow_merge": true, + "quality_pass": true, + "analysis_status": "complete" + }, + "checks": [{"id": "rustfmt", "status": "warnings"}] +}"#, + ) + .expect("plant gate"); + + let update_args = [ + "--ci", + "--update", + "--quiet", + "--no-zip", + "--no-heuristics", + "--no-fetch", + "--local-only", + ]; + + // Without the flag the reused pack is advisory: warnings do not fail CI. + let lenient = Command::new(assert_cmd::cargo::cargo_bin!("prview")) + .current_dir(repo) + .env("PRVIEW_HOME", home.path()) + .args(update_args) + .output() + .expect("lenient update run"); + assert_eq!( + lenient.status.code(), + Some(0), + "an unchanged run over a non-blocking pack still exits 0: {}", + String::from_utf8_lossy(&lenient.stderr) + ); + + let strict = Command::new(assert_cmd::cargo::cargo_bin!("prview")) + .current_dir(repo) + .env("PRVIEW_HOME", home.path()) + .args(update_args) + .arg("--fail-on-warnings") + .output() + .expect("strict update run"); + assert_eq!( + strict.status.code(), + Some(1), + "the reused pack warns, so --fail-on-warnings must exit 1: {}", + String::from_utf8_lossy(&strict.stderr) + ); +} diff --git a/tools/validate_merge_gate.py b/tools/validate_merge_gate.py index 3b94fae..547c5fd 100755 --- a/tools/validate_merge_gate.py +++ b/tools/validate_merge_gate.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""Validate MERGE_GATE.json contract (schema 1.0/2.0/2.1).""" +"""Validate MERGE_GATE.json contract (schema 1.0/2.0/2.1/2.2).""" from __future__ import annotations @@ -18,6 +18,74 @@ # `HOLD` synonym is retired: legacy_verdict() now emits CONDITIONAL for every # review-required/degraded run, so a freshly generated gate never carries HOLD. VALID_VERDICTS = {"PASS", "CONDITIONAL", "BLOCK"} +# schema 2.2: quality_failure_details entries name which check status produced +# them. Only "failure" entries may fail the quality gate, so a consumer told to +# filter on origin == "failure" cannot do that if the field is absent, mistyped, +# or spelled something else. +VALID_QUALITY_FAILURE_ORIGINS = {"failure", "warning"} +# The other two fields of the same entry. `name` is what a consumer reports and +# what the per-classification arrays are keyed by, so an empty one names no +# check at all; `classification` decides whether the entry gated this diff, and +# only `introduced`/`mixed`/`unclassified` do. Mirrors +# `QualityFailureClass::as_str` in src/artifacts/verdict.rs -- note the hyphen in +# `pre-existing`, which the sibling COUNT field spells `preexisting_*`. +VALID_QUALITY_FAILURE_CLASSES = { + "introduced", + "pre-existing", + "mixed", + "unclassified", +} +# Every spelling a check status is emitted as. Mirrors `CheckStatus::EMITTED` in +# src/checks/mod.rs, which a test there pins to `CheckStatus::as_str`. +# +# Case-SENSITIVE, unlike inline_findings.status below. The CLI counts warnings by +# comparing this vocabulary exactly, so a pack spelling `WARNINGS` is one the +# reader cannot read; accepting it here would certify an artifact that makes +# `--ci --fail-on-warnings` report a warning it cannot attribute. The inline +# field folds case because its writer has shipped legacy spellings; this one has +# only ever emitted lowercase. +VALID_CHECK_STATUSES = {"passed", "failed", "warnings", "skipped", "error"} +# The two enum axes of `decision`, spelled exactly as serde writes them. Mirror +# `AnalysisStatus` and `MergeRecommendation` in src/policy/engine.rs -- both +# `#[serde(rename_all = "snake_case")]`, and a test there pins every variant to +# the spelling below. +# +# Case-SENSITIVE and canonical-only, like VALID_CHECK_STATUSES and unlike the +# READERS, which fold case and still accept the retired `hold` spelling when +# reading a pack off disk. That tolerance exists for artifacts already written; +# this file certifies freshly emitted ones, and the 2.2 emitter has only ever +# written these. +VALID_ANALYSIS_STATUSES = {"complete", "degraded", "incomplete"} +VALID_MERGE_RECOMMENDATIONS = {"approve", "review_required", "block"} +# Conservativeness rank of one decision axis: 1 = clean pass, 2 = held below a +# pass, 3 = blocked. Mirrors `rank_from_verdict`, `rank_from_merge_rec` and +# `rank_from_analysis_status` in src/gate.rs, which both readers reconcile +# through. +# +# MEMBERSHIP RULE: an axis ranks only when its value RULES OUT a milder outcome. +# `complete` and `quality_pass: true` are preconditions of PASS, not grants of +# it -- a complete, quality-clean run is still held at CONDITIONAL by a +# review-required recommendation -- so neither states a rank, and neither +# appears in these tables. +VERDICT_RANK = {"PASS": 1, "CONDITIONAL": 2, "BLOCK": 3} +MERGE_RECOMMENDATION_RANK = {"approve": 1, "review_required": 2, "block": 3} +ANALYSIS_STATUS_RANK = {"degraded": 2, "incomplete": 2} +# Mirrors `verdict_from_rank`: the word the readers publish for a rank. +VERDICT_FROM_RANK = {rank: verdict for verdict, rank in VERDICT_RANK.items()} + + +def schema_at_least(raw: Any, minimum: tuple[int, int]) -> bool: + """Whether `raw` is a canonical MAJOR.MINOR at or above `minimum`. + + Mirrors the reader in `src/gate.rs`: components are compared as written, so + a non-canonical spelling is never treated as a version at all. + """ + if not isinstance(raw, str): + return False + parts = raw.split(".") + if len(parts) != 2 or not all(p.isdigit() and (p == "0" or not p.startswith("0")) for p in parts): + return False + return (int(parts[0]), int(parts[1])) >= minimum def err(msg: str) -> None: @@ -49,6 +117,204 @@ def require_non_negative_integer(value: Any, ctx: str, issues: list[str]) -> Non issues.append(f"{ctx} must be a non-negative integer") +def check_quality_pass_agrees_with_details( + decision: dict[str, Any], details: list[Any] +) -> list[str]: + """Cross-check `quality_pass` against the details it is computed from. + + `quality_pass` is not an independent opinion. The writer sets it to + `!QualityFailureSummary::has_new_failures()` and serializes the very same + details 1:1 into `quality_failure_details`, so the two are one fact written + twice and the check is an EQUIVALENCE, enforced in both directions. + + An entry gates the diff when its origin is `failure` AND its classification + is anything but `pre-existing`. Both halves matter, and the second is the + reason the obvious rule -- "a failure-origin entry forces quality_pass + false" -- is WRONG: a purely pre-existing failure is emitted next to + `quality_pass: true` on purpose, because it predates the diff and must not + block the merge. `security_full_preexisting_semgrep_finding_is_advisory_only` + in src/artifacts/merge_gate.rs produces exactly that pack, and rejecting it + would make the validator cry wolf on a genuine one -- which costs more than + the hole it closes, because a validator nobody trusts gates nothing. + + Only entries whose own shape already validated are counted, so a malformed + detail is reported once as a shape error rather than twice. + """ + quality_pass = decision.get("quality_pass") + if not isinstance(quality_pass, bool): + # From 2.2 the caller has already required a boolean, so a non-boolean + # is reported once as a type error rather than twice. Below 2.2 this + # function is not reached at all: absence there is an old pack, not a + # contradiction, and there is no cross-field claim to check. + return [] + gating = [ + detail.get("name") + for detail in details + if isinstance(detail, dict) + and detail.get("origin") == "failure" + and detail.get("classification") in VALID_QUALITY_FAILURE_CLASSES + and detail.get("classification") != "pre-existing" + ] + if quality_pass and gating: + return [ + "decision.quality_pass must be false when a quality_failure_details " + "entry has origin 'failure' and a classification other than " + f"'pre-existing': got quality_pass=true with {sorted(map(str, gating))}" + ] + if not quality_pass and not gating: + return [ + "decision.quality_pass must be true when no quality_failure_details " + "entry has origin 'failure' with a classification other than " + "'pre-existing': got quality_pass=false with no such entry" + ] + return [] + + +def check_blocker_flag_agrees_with_blocking_issues(decision: dict[str, Any]) -> list[str]: + """Cross-check `policy_allow_merge` against the list it is computed from. + + Like `quality_pass`, this flag is not an independent opinion: the writer sets + `policy_allow_merge = blocking_issues.is_empty()` (src/artifacts/merge_gate.rs) + AFTER the last push to that list, and then serializes both verbatim into the + same `json!` literal. Nothing else writes the field -- the other computation + of the same formula, in src/artifacts/context.rs, feeds the dashboard, not + this artifact. So the two are one fact written twice and the check is an + EQUIVALENCE, enforced in both directions: + `policy_allow_merge: true` beside real blockers, and `false` beside none, + are equally unemittable. + + The reconciliation below already reads the pair -- but only in the harsher + direction, where either half raises the rank the verdict must clear. That + leaves the two halves free to contradict each other outright, which is the + hole this closes: a pack whose blockers say "blocked" and whose flag says + "policy let it through" certifies a state prview never produced, and readers + that trust one half read the opposite of readers that trust the other. + + Reached only from schema 2.2, where both fields are required; below it, + absence is an old pack rather than a contradiction and no cross-field claim + exists to check. + """ + policy_allow_merge = decision.get("policy_allow_merge") + blocking_issues = decision.get("blocking_issues") + if not isinstance(policy_allow_merge, bool) or not isinstance(blocking_issues, list): + # Type errors are reported once, by the shape checks that own them. + return [] + if policy_allow_merge and blocking_issues: + return [ + "decision.policy_allow_merge must be false when blocking_issues is " + "non-empty -- the writer derives the flag from that list: got " + f"policy_allow_merge=true with {len(blocking_issues)} blocking_issues" + ] + if not policy_allow_merge and not blocking_issues: + return [ + "decision.policy_allow_merge must be true when blocking_issues is " + "empty -- the writer derives the flag from that list: got " + "policy_allow_merge=false with no blocking_issues" + ] + return [] + + +def check_decision_axes_agree_on_the_verdict(decision: dict[str, Any]) -> list[str]: + """Reject a `verdict` milder than the axes stated beside it. + + Both readers reconcile a decision the same way: take the MAX rank across the + axes the pack states, then publish every axis from that one number. So a + verdict below that maximum is not a verdict any reader will honour -- the + pack certifies one outcome and every consumer of it computes another. The + reported hole was exactly that: `verdict: "PASS"` beside + `analysis_status: "incomplete"`, `merge_recommendation: "block"` and + `policy_allow_merge: false` validated OK, so an artifact both readers + normalize to BLOCK carried a green certification. + + The emitter cannot produce such a pack. It derives `verdict` through + `MergeRecommendation::legacy_verdict`, whose result is the same maximum: + + * `block` -> `BLOCK` : rank 3 + * `review_required` -> `CONDITIONAL`: rank 2 + * `approve` + complete + quality : rank 1 (`PASS`) + * `approve` + degraded/quality fail : rank 2 (`CONDITIONAL`) + + `allow_merge` is `verdict == "PASS"`, so it never exceeds the verdict it + sits beside; and `blocking_issues`/`policy_allow_merge` rank 3 because a + blocking issue is pushed only for a check whose `PolicyConclusion` is + `Blocked`, whose `merge_impact` is `Block` -- a stated blocker IS a stated + `block` recommendation. + + DELIBERATE LIMIT: only the permissive direction is rejected. A verdict + HARSHER than its other axes is legal -- a semgrep scan that passes with + parse errors leaves `merge_recommendation: "approve"` beside + `analysis_status: "degraded"`, which the contract turns into `CONDITIONAL` + -- so "verdict equals the max of the OTHER axes" would reject a pack the + emitter really writes. A harsher verdict also fools nobody: every reader + publishes it as stated. It is the milder direction that certifies a + permission the artifact never earned. + + Only axes whose own type and vocabulary already validated are ranked, so a + malformed axis is reported once as a shape error rather than twice. An axis + the pack does not state is not ranked either -- absence states nothing, and + from 2.2 the caller has already required every axis this reads. + """ + verdict = decision.get("verdict") + verdict_rank = VERDICT_RANK.get(verdict) if isinstance(verdict, str) else None + if verdict_rank is None: + return [] + + stated: list[tuple[str, int]] = [(f"verdict={verdict!r}", verdict_rank)] + + recommendation = decision.get("merge_recommendation") + if isinstance(recommendation, str) and recommendation in MERGE_RECOMMENDATION_RANK: + stated.append( + ( + f"merge_recommendation={recommendation!r}", + MERGE_RECOMMENDATION_RANK[recommendation], + ) + ) + + allow_merge = decision.get("allow_merge") + if isinstance(allow_merge, bool): + stated.append((f"allow_merge={allow_merge}", 1 if allow_merge else 2)) + + # Only the FALSE of these two states a rank -- see the membership rule above. + if decision.get("quality_pass") is False: + stated.append(("quality_pass=False", 2)) + + analysis_status = decision.get("analysis_status") + if isinstance(analysis_status, str) and analysis_status in ANALYSIS_STATUS_RANK: + stated.append( + ( + f"analysis_status={analysis_status!r}", + ANALYSIS_STATUS_RANK[analysis_status], + ) + ) + + # The blocker axis, stated twice by the emitter + # (`policy_allow_merge = blocking_issues.is_empty()`). It is ONE axis: a pack + # carrying both must not count it twice, and one carrying either is covered. + # That the two halves actually AGREE is not this rule's job -- ranking only + # asks how conservative the pack is -- it is enforced as an equivalence by + # `check_blocker_flag_agrees_with_blocking_issues`. + blocking_issues = decision.get("blocking_issues") + if decision.get("policy_allow_merge") is False: + stated.append(("policy_allow_merge=False", 3)) + elif isinstance(blocking_issues, list) and blocking_issues: + stated.append((f"{len(blocking_issues)} blocking_issues", 3)) + + # `stated` includes the verdict's own rank, so this is the milder-direction + # test and nothing else: the maximum can only exceed the verdict when some + # OTHER axis rules the verdict out. + final_rank = max(rank for _, rank in stated) + if verdict_rank >= final_rank: + return [] + + axes = ", ".join(name for name, _ in stated) + return [ + f"decision.verdict is milder than the axes beside it: {axes}. The most " + "conservative axis a pack states is the verdict it may publish (rank " + f"{final_rank}), and every reader reconciles this decision to " + f"{VERDICT_FROM_RANK[final_rank]!r}" + ] + + def require_iso_datetime(value: Any, ctx: str, issues: list[str]) -> None: if not isinstance(value, str) or not value.strip(): issues.append(f"{ctx} must be an ISO datetime string") @@ -105,8 +371,8 @@ def validate(path: Path) -> list[str]: if not isinstance(data["schema_version"], str): issues.append("schema_version must be a string") - elif data["schema_version"] not in ("1.0", "2.0", "2.1"): - issues.append("schema_version must be '1.0', '2.0', or '2.1'") + elif data["schema_version"] not in ("1.0", "2.0", "2.1", "2.2"): + issues.append("schema_version must be '1.0', '2.0', '2.1', or '2.2'") require_iso_datetime(data["generated_at"], "generated_at", issues) if ( isinstance(data["bridge_stage"], bool) @@ -175,7 +441,10 @@ def validate(path: Path) -> list[str]: ) require_non_empty_string(check.get("id"), f"{ctx}.id", issues) require_non_empty_string(check.get("name"), f"{ctx}.name", issues) - require_non_empty_string(check.get("status"), f"{ctx}.status", issues) + if check.get("status") not in VALID_CHECK_STATUSES: + issues.append( + f"{ctx}.status must be one of {sorted(VALID_CHECK_STATUSES)}" + ) if check.get("class") not in VALID_CLASSES: issues.append(f"{ctx}.class must be one of {sorted(VALID_CLASSES)}") if check.get("severity") not in VALID_SEVERITIES: @@ -247,6 +516,81 @@ def validate(path: Path) -> list[str]: "decision.allow_merge must equal (verdict == 'PASS'): " f"got allow_merge={allow_merge} with verdict={verdict!r}" ) + # From schema 2.2 the whole entry is part of the contract, not an extra. + # `origin` is the only thing that explains an entry in + # `introduced_quality_failures` sitting next to `quality_pass: true`; + # `name` is what a consumer reports; `classification` is what decides + # whether the entry gated this diff. Validating only one of the three let + # `{"origin": "failure"}` -- an anonymous, unclassified failure -- pass + # its own contract gate. + if schema_at_least(data.get("schema_version"), (2, 2)): + # `quality_pass` is a documented decision axis and the 2.2 writer + # emits it unconditionally, as a boolean, from a single `json!` + # literal -- so a 2.2 pack that omits it or states it as a string is + # not an old pack, it is a broken one. Absence stays forgiven BELOW + # 2.2, where readers derive the flag from the reconciled verdict; a + # 2.2 pack gets no such benefit of the doubt. Type-checking here also + # puts the validator back in step with the readers, which normalize a + # present-but-unreadable signal to BLOCK: without this the contract + # gate certified an artifact the CLI and MCP both refuse to trust. + issues.extend(ensure_keys(decision, ["quality_pass"], "decision")) + if "quality_pass" in decision: + require_boolean(decision.get("quality_pass"), "decision.quality_pass", issues) + # The remaining decision axes, on the same argument. All three come + # out of the same 2.2 `json!` literal, unconditionally and from the + # typed enums, so a 2.2 pack missing one is broken rather than old -- + # and the reconciliation below can only reject a verdict its axes + # contradict if the axes are actually there to read. Absence stays + # forgiven BELOW 2.2, where a reader derives what the pack omits. + issues.extend( + ensure_keys( + decision, + ["analysis_status", "merge_recommendation", "policy_allow_merge"], + "decision", + ) + ) + if "analysis_status" in decision: + if decision.get("analysis_status") not in VALID_ANALYSIS_STATUSES: + issues.append( + "decision.analysis_status must be one of " + f"{sorted(VALID_ANALYSIS_STATUSES)} (schema 2.2)" + ) + if "merge_recommendation" in decision: + if decision.get("merge_recommendation") not in VALID_MERGE_RECOMMENDATIONS: + issues.append( + "decision.merge_recommendation must be one of " + f"{sorted(VALID_MERGE_RECOMMENDATIONS)} (schema 2.2)" + ) + if "policy_allow_merge" in decision: + require_boolean( + decision.get("policy_allow_merge"), "decision.policy_allow_merge", issues + ) + issues.extend(check_blocker_flag_agrees_with_blocking_issues(decision)) + issues.extend(check_decision_axes_agree_on_the_verdict(decision)) + details = decision.get("quality_failure_details") + if not isinstance(details, list): + issues.append("decision.quality_failure_details must be an array") + else: + for idx, detail in enumerate(details): + ctx = f"decision.quality_failure_details[{idx}]" + if not isinstance(detail, dict): + issues.append(f"{ctx} must be an object") + continue + require_non_empty_string(detail.get("name"), f"{ctx}.name", issues) + classification = detail.get("classification") + if classification not in VALID_QUALITY_FAILURE_CLASSES: + issues.append( + f"{ctx}.classification must be one of " + f"{sorted(VALID_QUALITY_FAILURE_CLASSES)} (schema 2.2)" + ) + origin = detail.get("origin") + if origin not in VALID_QUALITY_FAILURE_ORIGINS: + issues.append( + f"{ctx}.origin must be one of " + f"{sorted(VALID_QUALITY_FAILURE_ORIGINS)} (schema 2.2)" + ) + issues.extend(check_quality_pass_agrees_with_details(decision, details)) + if not isinstance(decision.get("blocking_issues"), list): issues.append("decision.blocking_issues must be an array") else: