Skip to content

docs(review-rigor): V3 — prevention does not close a detection finding - #272

Open
ClintonSarkar wants to merge 1 commit into
benitogf:mainfrom
ClintonSarkar:docs/finding-closed-by-detection-not-prevention
Open

docs(review-rigor): V3 — prevention does not close a detection finding#272
ClintonSarkar wants to merge 1 commit into
benitogf:mainfrom
ClintonSarkar:docs/finding-closed-by-detection-not-prevention

Conversation

@ClintonSarkar

Copy link
Copy Markdown
Contributor

Closes #271

Adds V3 to core/review-rigorVerdict integrity: a finding about detection is not closed
by a change to prevention.

Why V1 doesn't already cover it

V1 bans speculative clears — it keys on hedge-words and unverified claims. The miss this comes from
had neither. A self-review pass correctly found "nothing gates this invariant against regrowth", the
author answered it with a declarative constraint that pins the invariant, and that constraint was
real, working, better code. Nothing was hedged; the item was marked resolved with confidence.

But the finding was about verification, and the answer was about likelihood. Declared is not
checked — deleting the declaration broke nothing, so the gap the finding named survived. Two further
review passes then inherited the disposition rather than re-litigating it, which is the documented
behaviour of Re-review continuity, and the human reviewer's first response was
CHANGES_REQUESTED on exactly that gap.

Self-review is especially exposed here, because the accepted resolution is frequently the reviewer's
own earlier suggestion — there is no independent party to notice the goalposts moved.

The rule as encoded

When the finding is that something is unverified — no test, no assertion, nothing fails if it
regresses — it closes ONLY when something FAILS on regression.

The test that settles it: if someone deletes this hardening, what turns red? "Nothing" means the
finding stands.

with the detection cue (the finding's own words are no test / nothing catches / unenforced / could
silently regress
, and the resolution changed a mechanism instead of adding an observer), the
self-review/re-review exposure, and a ✅/❌ pair.

It also carries one sub-rule learned expensively in the same incident: a red-then-green demo is only
evidence if the artifact under test actually moved.
Two negatives passed vacuously before a valid
one existed — the first widened a declarative allowlist whose symbols a different flag already
suppressed, so the built output was byte-identical; the second used a pattern that silently failed to
match because of line-ending differences, so the change was never applied and the build stayed green.
Both read exactly like a passing check. Editing the source that supposedly produces an artifact is not
the same as changing the artifact.

Scope

docs/core/review-rigor.md only, +10/−4:

  • V3 added after V2 in Verdict integrity; the section heading and the changes-verdict line updated
    from V1–V2 to V1–V3.
  • Re-review continuity — the verdict-integrity sentence now names V3 and says a resolution accepted
    in an earlier pass is inherited rather than re-litigated, so re-ask closed-vs-mitigated.
  • Tests — the "is there a regression test?" bullet now says hardening is not a substitute, pointing
    at V3.

Swept for stale references to the old V1–V2 range (none remain) and for Go-embedded doctrine
referencing these rules (none), so this is a docs-only writeback with no toolchain verification
required.


🤖 Generated with Claude Code

📚 Learned by detritus — absorb session/2026-08-06-export-surface-invariant

A self-review loop cleared a change three times after resolving "nothing gates
this against regrowth" with a declarative constraint. The human reviewer's
first response was CHANGES_REQUESTED: the invariant still had no committed
test, so deleting the constraint would silently restore the defect.

Both parties had named the same risk; they diverged on what closes it. A change
to the enforcement mechanism was accepted as answering a finding about missing
verification. V1 does not catch this — nothing was hedged, and the resolution
was real, working code that genuinely improved the change. Two later passes then
inherited the disposition instead of re-asking whether the finding was closed.

V3 states the rule: a finding that something is unverified closes only when
something FAILS on regression, and the settling question is what turns red if
the hardening is deleted. It carries the detection cue, the self-review
exposure, and the sub-rule that a red-then-green demo is only evidence if the
artifact under test actually moved — two vacuous negatives preceded a valid one
in the incident, one because a different flag already suppressed the symbols
being "exposed" and one because a line-ending mismatch meant the edit never
applied.

Also extends the Re-review continuity note to say V3 applies across passes, and
cross-references it from Tests so the regression-test check names hardening as a
non-substitute.

Closes benitogf#271

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

review-rigor: a finding about detection is not closed by a change to prevention

1 participant