Repository navigation
S2.6.4 — The two-surface mutation recorder - #332
Merged
dev-pmallapp merged 2 commits intoSep 10, 2026
Merged
dev-pmallapp merged 2 commits into
dev-pmallapp merged 2 commits into
Conversation
5 tasks
dev-pmallapp
marked this pull request as ready for review
September 9, 2026 12:58
dev-pmallapp
merged commit Sep 10, 2026
27ac716
into
story/293-s2-6-skill-execution-harness
7 checks passed
8 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Build target 5 of
docs/design/stories/293.md. AnchorsCLAIM-293.8.Verdict: PASS for case 4 and case 10.
CLAIM-293.8is met for the recorder; its per-skill threshold — at least one scenario per file-writing skill withrun1.tree_hash != pre.tree_hash— needs a transcription and a runner and closes at #323, which must not read this as having closed it.Evidence:
docs/evidence/321-20260909T125257Z.mdChain at
761bb1d987 tests / 973 pass / 0 fail / 14 skip / 3513
expect()· lint 0 across 109 files · typecheck clean · skill-lint 4 + 12, 0 errors · claim-lint 117 files, 0 errors, 1 warning (the expectedstories/9.mdone). The evidence artifact adds +2expect()on top, per #299.What it ships
packages/harness/src/mutation-recorder.ts— two surfaces, reported as two separately named lists, never one total. The forge surface reusesisMutatingGhArgv; there is no second argv classifier.git status,git hash-object, a--statdiff and an mtime comparison all sample effect, and the defining property of a byte-identical rewrite is that it leaves no effect. The seam is the worktree'screateRecordingPort.git commit --allow-emptyafter a byte-identical rewrite moves HEAD while leavingHEAD^{tree}byte-identical, so a tree-hash-only recorder cannot tell a committed identical-bytes rewrite from an uncommitted one.mutationEvidencereturns a list whose emptiness is the zero-mutation result, mirroringghOk([]). A boolean is the assertion shape under which every surface-unwiring mutation survives.attemptedMutationsis typedGhPlan— an argv list with no exit code — so&& c.exitCode === 0is a type error downstream rather than a review miss.Mutation testing — 14 injected, 12 killed, 2 survivors by design
M3 and M4 kill disjoint halves:
if (identical) return;kills only the two identical-bytes scenarios, andidentical := existedBeforekills only the differing-bytes ones. A suite without case 10 ships M3 green.M5/M6 is #320's M9/M9b one layer up: weakening the matrix assertion to a length check survives alone, and the kill for the paired defect comes from a different test. A weakened assertion is not detectable by the test it weakens.
M12's asymmetry: an exit-code filter is killed only by a lie that refuses. The
silent-no-oplie exits 0, so the property "records the attempt, not the success" is observable only through a double that says no.S2 survives — the fake made to enforce a duplicate leaves this suite green — while M12 dies. Same property, opposite sides.
Findings
story-designas zero mutations.The first-contact bill (#330) is still unpaid — this task answered every
ghcall from #320's fake. #331 is untouched: the transient mode was injected to prove a failed attempt is recorded, and no retry logic was built on it.Refs #321