Skip to content

feat(mutation-testing): read the personal override from docs conventions mutation-testing.local.yaml - #18084

Open
melodic-automation-lanes[bot] wants to merge 4 commits into
mainfrom
work-resolve-issue/issue-18014-44964541536-1
Open

melodic-automation-lanes[bot] wants to merge 4 commits into
mainfrom
work-resolve-issue/issue-18014-44964541536-1

Conversation

@melodic-automation-lanes

@melodic-automation-lanes melodic-automation-lanes Bot commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor

Closes #18014

Summary

Opened by the work-resolve-issue lane. The run's own summary:

The mutation-testing personal override moves from .claude/mutation-testing.local.md to docs/conventions/mutation-testing.local.yaml, the personal layer in the config-cascade contract (docs/contracts/config-cascade/README.md, 'The layers' and 'Personal override naming'). The YAML form follows the contract's rule that a structured surface's personal channel is <home>/<concern>.local.yaml, and the path sits beside the team file the plugin already reads. During the dual-read window the old file still supplies the personal layer when no new file exists. With both present, the new file wins and each old key is named as overridden or not applied. Every check, apply and audit run prints one WARN line naming the old file, the new location and /mutation-testing:setup apply local. setup apply (team or local) moves the old file after a preview and a yes, reads the new file back to confirm the values match, then deletes the old file. Two decisions, with their basis: (1) the arid suppression record .claude/mutation-testing-arid.md stays where it is, because the contract's 'Declared team-layer paths' table lists it as a finding-suppression record outside the settings' location axis, and the issue puts its lint comment out of scope. (2) The dual-read works per layer, not per key, matching the criterion 'its values still apply where no new-layer file supplies that layer'.

Not verified

  • Tidy skipped: the change touches 24 files, over the tidy cutoff of 10.
  • The three new eval cases were not executed: they need a model run with --scaffold. The PR's CI and the eval runner cover them, and eval-cases schema validation runs in CI only.
  • check-affected skipped typos (not installed), comment-hygiene (sandbox denied the fetch) and size-check (script absent in the lane). Left to the PR's CI.
  • The repository's own post-merge mutation lane (.github/scripts/post-merge-test-mutation/set-diff-target.sh and generate.prompt.md, plus the comment in docs/conventions/mutation-testing.yaml) still writes .claude/mutation-testing.local.md. The dual-read keeps it working, with the warning, until a…
  • The new scaffold.sh files were created without the executable bit (no chmod in this lane). Sibling cases are run through bash via context.scaffold_script.

Fix

  • plugins/mutation-testing/skills/setup/SKILL.md: the merge model names docs/conventions/mutation-testing.local.yaml as the personal layer and defines the dual-read rules and the warning line, including the exact suffix the line ends with when both personal files are present: (docs/conventions/mutation-testing.local.yaml won). Probe 6 reports the old file as INFO and prints the warning. Probe 9 covers the new file. Step 10 and apply local name the new path. Moving the old personal file previews, asks, writes, reads back and deletes the old file. A declined or unattended move ends apply local without writing, so a new file never hides old keys the user did not carry over.
  • plugins/mutation-testing/skills/setup/templates/config-template.md: the personal override path is now docs/conventions/mutation-testing.local.yaml.
  • plugins/mutation-testing/README.md: names the new personal override path and says the old file is still read, with a warning, until apply local moves it.
  • plugins/mutation-testing/skills/audit/SKILL.md: the Config check prints the merge model's warning line when the old personal file exists.
  • plugins/mutation-testing/skills/audit/context/several-repositories.md: the per-repository personal file is the new path, or the old one with its warning.
  • plugins/mutation-testing/evals/se-personal-yaml-layer/ (case.yaml, scaffold.sh, samples/narrow-from-personal.json, samples/no-write.json, samples/no-edit.json): new eval case for the new layer only. operators: narrow comes from the YAML file, with no old-path warning.
  • plugins/mutation-testing/evals/se-old-personal-file-dual-read/ (case.yaml, scaffold.sh, samples/warns-old-file.json, samples/no-write.json, samples/no-edit.json): new eval case for the old file only. Its values apply, and the WARN line names the old file, the new location and apply local.
  • plugins/mutation-testing/evals/se-both-personal-files-new-wins/ (case.yaml, scaffold.sh, samples/warns-both.json, samples/no-write.json, samples/no-edit.json): new eval case for both files present. The new file wins, the WARN line names both and ends with the fixed suffix saying the new file won, and the old max-mutants is named as not applied.
  • plugins/mutation-testing/evals/se-apply-proposes-tool-writes-config/case.yaml: the LLM criterion expects the new personal override path.
  • plugins/mutation-testing/evals/se-apply-user-layer-no-repo-write/case.yaml: the criterion now says 'no personal override file' instead of naming the old .local.md.
  • plugins/mutation-testing/skills/audit/evals/evals.json: two prompts now say 'no personal override file' instead of naming the old .local.md.
  • .changes/mutation-testing/work-resolve-issue-issue-18014-44964541536-1-f70024d0.md: minor changelog fragment describing the move, the dual-read window and the migration.

Verification

The runner ran each check from the base copy against d2d6cb7:

  • scripts/check-changelog-fragments.sh --check: passed
  • scripts/check-changed-skills.sh --merge-base: passed

The run reports:

  • check-changelog-fragments: all 3 fragments under .changes/ valid (run before and after the review fixes)
  • check-changed-skills: audit and setup PASS, 0 errors; the warnings are size and description-length advisories
  • check-affected: 15 passed, 8 skipped over 24 changed files (skips: typos not installed, comment-hygiene fetch denied by the sandbox, size-check script absent, checks that run the checked tree's own code)
  • check-evals-quality plugins/mutation-testing/skills/audit/evals/evals.json: PASS; the 6 warnings are on cases this change did not touch
  • Fresh-context review subagent over the whole diff: 5 findings, all confirmed and fixed (arid-draft ignore line kept in step 10, one overridden/not-applied rule in the merge model, probe 6 and the move preview, the move's check-ignore step stated for both layers, warns-both regex now requires 'won'…
  • The lane run: https://github.com/melodic-software/claude-code-plugins/actions/runs/44964541536

Left out: the items under Not verified.

Related

#18014

Lane-Activity: work-resolve-issue/resolve-issue
@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-refine/fix-docs

Run 44968130657, attempt 1

Field Value
Model rung 0
Leg handoff
Run job success
Activity step success
Commits pushed none (read lane)
Run 44968130657

Failed GitHub API calls

No failed call recorded by the run job's scripts.

@github-actions

github-actions Bot commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/fix-ci

Run 44971634708, attempt 1

Field Value
Model rung 0
Leg handoff
Run job success
Activity step success
Commits pushed none (read lane)
Run 44971634708

Failed GitHub API calls

No failed call recorded by the run job's scripts.

@github-actions

github-actions Bot commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/apply-patch

Run 44971634708, attempt 1

Field Value
Model rung 0
Leg base
Run job success
Activity step success
Commits pushed 5c3e439
Run 44971634708

Failed GitHub API calls

No failed call recorded by the run job's scripts.

@github-actions

Copy link
Copy Markdown
Contributor

claude-security-review reviewed e26384af828948c465abb321aff0f904858b1c5c: read 24 of 24 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #18084, mutation-testing personal override moves to docs/conventions/mutation-testing.local.yaml

I found no security issues. I read every hunk of all 24 changed files in the full diff. None of them match a trusted-path glob. The diff touches no plugin manifest, workflow, action, hold script or trust control.

What I checked:

  • Config trust boundary (plugins/mutation-testing/skills/setup/SKILL.md, merge model). The personal layer moves from .claude/mutation-testing.local.md to docs/conventions/mutation-testing.local.yaml and takes the same keys. Could a cloned repository commit the new .local.yaml and use it to inject values such as a test command? The team file docs/conventions/mutation-testing.yaml is already repository-tracked and already supplies those same keys, and the old .claude/ personal file could be committed the same way. So the new path gives a repository author nothing they did not already have. Probe 9 still reports the personal files as expected to be untracked and ignored.
  • Migration step ("Moving the old personal file"). It writes nothing without a preview and an explicit yes. In an unattended run it changes nothing. It runs git check-ignore and offers the ignore line before writing, so the new file is not left exposed to the index. It reads the new file back and compares values before deleting the old file, and stops on any mismatch. Nothing is lost or exposed without the user's consent.
  • Instruction-surface lens. The only removed text is a migration note ("until the plugin migrates…"). It was not a guardrail. The rule that a personal arid entry is only a draft is kept in step 10 and in the README. The arid suppression record and its team-wins precedence are unchanged. Nothing that used to be forbidden is now allowed.
  • Eval scaffolds (se-*/scaffold.sh). These are fixed fixture scripts. They append ignore lines, make a commit and write constant files inside the eval workspace. They take no external input and fetch nothing from the network.
  • PR context. It contains no embedded directives to the review lane.

Spend: three new eval cases (runs: 3, max_turns: 40 each) add model usage to eval runs. That comes from prepaid or subscription usage, so it is not a hold.

critical-findings: 0

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 2 inline finding(s) on e26384af828948c465abb321aff0f904858b1c5c; the summary is in this lane's pull request comment.

claude-lane: claude-review

@github-actions

Copy link
Copy Markdown
Contributor

claude-review reviewed e26384af828948c465abb321aff0f904858b1c5c: read 24 of 24 changed file(s); 2 finding(s), 2 as inline comments.

Code review: mutation-testing personal override moves to docs/conventions/mutation-testing.local.yaml

I read all 24 changed files in the full diff. I also read the head skills/setup/SKILL.md, the shared eval fixture evals/fixtures/seed-shipping.sh, REVIEW.md and the path-scoped rules that match these files (eval-case-transcripts.md, skills-bodies-state-current-rules.md).

Closing issue #18014 acceptance criteria:

  • Setup reads the config-cascade layers: met. The merge model, the template and the README all name docs/conventions/mutation-testing.local.yaml.
  • Dual-read window with one warning per run: met in the text. The new cases se-old-personal-file-dual-read and se-both-personal-files-new-wins cover it.
  • Preview, yes, write, read back, then delete: met in the new "Moving the old personal file" section. Nothing tests the move itself: all three new eval cases run check, which is read-only. The issue's eval criterion lists only the three read cases, so I don't count this as a finding.
  • Removing the old-path read is a later change: stated at SKILL.md:47.
  • Evals for new only, old only and both: present. The author says these three cases were not run; I did not run them either.

The fixture fits the cases: the configured team file sets no operators, so narrow can only come from a personal layer.

Findings (2, inline):

  1. The warns-both regex grader needs the literal word won. The skill gives no exact wording for that part of the warning line, so a correct reply such as "wins" or "takes precedence" fails the grader.
  2. apply local does not say what happens after the user declines the move. If it goes on to write a new YAML file with only the newly settled keys, the old file's values stop applying because of a "no" answer.

No paved-path entry, plugin manifest or loosened check is affected. Changelog fragment bump: minor fits a new location that stays backward compatible through the dual-read.

Spend: three new scaffold eval cases at runs: 3 each use prepaid model credit when the eval runner runs them. This is not a hold.

Loosened checks: the prompt carried no loosened_check_verdict: or loosened_check_patterns: lines, so I used fail with no consumer entries. With no loosened_check_decision_record: line, the citation requirement was not checked. No changed grader or prompt relaxes a check: legacy-setup-11 and the two audit prompts now say "no personal override file", which is broader than before. Loosened checks would be marked: fail (default; no consumer layers); patterns: 0.

I found no embedded instructions in the PR context.

No verifier subagent was dispatched. I checked both candidates myself against the head files and the fixture.

Verifier dropped 0 of 2 candidates.

@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/address-feedback

Run 44970193495, attempt 1

Field Value
Model rung 0
Leg handoff
Run job success
Activity step success
Commits pushed none (read lane)
Run 44970193495

Failed GitHub API calls

No failed call recorded by the run job's scripts.

Lane-Activity: pr-unblock/address-feedback
Lane-Thread: PRRT_kwDOTCGFQM66eduA
Lane-Thread: PRRT_kwDOTCGFQM66eduE
@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/apply-feedback-patch

Run 44970193495, attempt 1

Field Value
Model rung 0
Leg base
Run job success
Activity step success
Commits pushed 8a48072
Run 44970193495

Failed GitHub API calls

No failed call recorded by the run job's scripts.

@melodic-automation-lanes

Copy link
Copy Markdown
Contributor Author

Unattended merge: native auto-merge leaves this pull request unarmed at head 8a48072f39515d340aa51fad296e1ae49abcc350. No person needs to act: a lane clears this reason, and the next scheduled run judges it again.

pr-merge's gate refuses its verifier verdict (verify:missing)

@melodic-automation-lanes

Copy link
Copy Markdown
Contributor Author

pr-unblock / address-feedback finished round 1 of max 4 on 8a48072.

@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/publish-feedback

Run 44970193495, attempt 1

Field Value
Model rung 0
Leg base
Run job success
Activity step success
Commits pushed none recorded
Run 44970193495

Failed GitHub API calls

No failed call recorded by the run job's scripts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mutation-testing: read the personal config layer from the config-cascade location, not .claude

0 participants