Correct mutation-score explanation with a reduced-suite regression - #12
Merged
Merged
Conversation
noteflowai
added a commit
that referenced
this pull request
Sep 15, 2026
The margin set was built from detected controls only, so a surviving fault was excluded from it and an audit could report a weakest margin of 1 while a declared fault sat undetected. #12 fixed the filter as part of correcting the explanation; this stages the release, because the version on GitHub still misreports a number it advertises. Measured rather than argued, and re-run here: with cas-type-sensitivity removed from durable-kv at seed 17, boolean-equals-one survives, the audit reports 7 of 8 at 0.875, and the weakest margin is 0. Computed the way 0.10.0 does it, over killed rows only, the same run reports 1. Version bumped in both declared places, which the version-agreement test guards, and the changelog states the measurement rather than the intent. Full local gate run before pushing: ruff check, ruff format --check, 308 tests, a built wheel, and check_installed.py against a clean venv from outside the checkout. The tag and the release assets are deliberately not created here. The asset set on v0.10.0 includes audit-reports.zip, which no script in the repository produces, so the build is not reproducible from what is written down. Publishing a release missing an asset every previous one carried would be worse than leaving it staged. Whoever cuts it has the version and the notes ready; documenting the asset pipeline would remove the obstacle.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The methodology and HVE outreach draft claimed that removing a sole detector leaves the mutation score at 1.0. A fresh audit must lower the score; only an old report can retain the old result.
Correct the methodology, bilingual README and comments, and add a real reduced-suite regression: removing cas-type-sensitivity leaves boolean-equals-one undetected, producing 7/8 (0.875) and weakest margin zero. Revise the HVE note into a scoped question about mutation-checking the existing onset tests merged in upstream #2, acknowledging their clean-episode, zero-step, first-firing and reset coverage. No HVE instrumentation evaluation is claimed.
Validation: the new reduced-suite test and existing two-seed margin test both pass (35.06 seconds); Ruff passes. Runtime scoring logic is unchanged.
Preserve the outside submission of HVE #4 and its publication log. Correct that existing issue in place; the linked reduced-suite reproduction is tied to an immutable source commit.