diff --git a/.changes/mutation-testing/work-resolve-issue-issue-18014-44964541536-1-f70024d0.md b/.changes/mutation-testing/work-resolve-issue-issue-18014-44964541536-1-f70024d0.md new file mode 100644 index 00000000000..52407600651 --- /dev/null +++ b/.changes/mutation-testing/work-resolve-issue-issue-18014-44964541536-1-f70024d0.md @@ -0,0 +1,9 @@ +--- +bump: minor +--- + +### Changed + +- **The personal override moves to `docs/conventions/mutation-testing.local.yaml`**, beside the team file, where the config-cascade contract puts the personal layer. `setup apply local` writes it there and offers the `docs/conventions/**/*.local.*` ignore line. +- **An old `.claude/mutation-testing.local.md` is still read**: with no new file it supplies the personal layer as before, and with both present the new file wins and each old key is named as overridden or not applied. Every `check`, `apply` and audit run prints one warning line naming the old file, the new location and `/mutation-testing:setup apply local`. +- **`setup apply` moves the old file** after a preview and a yes: it writes the new file, checks it resolves the same values, then deletes the old one. New eval cases cover the new file only, the old file only and both present. diff --git a/plugins/mutation-testing/README.md b/plugins/mutation-testing/README.md index ae6ed2f5d03..e780d840c1a 100644 --- a/plugins/mutation-testing/README.md +++ b/plugins/mutation-testing/README.md @@ -69,7 +69,8 @@ about it, and a coverage report will call that file healthy. They are kept separate deliberately: a config diff reads as a policy change, a suppression diff reads as an accepted finding. **Their layering also differs.** The config layers per the config-cascade -convention (the user layer in `userConfig` → team → `.local.md` personal override, later wins). The +convention (the user layer in `userConfig` → team → the personal override +`docs/conventions/mutation-testing.local.yaml`, later wins). The suppression record sits in the cascade's policy-floor precedence-inversion class: on a conflict the **team** layer wins, and a personal-layer entry for an id the team layer does not carry **does not suppress at all**. It is reported `personal-only, not applied`. A personal layer is a draft @@ -78,8 +79,9 @@ surface there, so one developer cannot hide a finding the team never accepted. The user layer is this plugin's `userConfig`, not a file: two options that mean the same thing in every repository, the operator set and a mutant cap. A repository's team or local file overrides them. `/mutation-testing:setup apply user` settles them and names the route below for saving them; -it writes nothing itself. `apply local` writes the untracked `.local.md` personal override instead of the team -file. +it writes nothing itself. `apply local` writes the untracked personal override instead of the team +file. An old `.claude/mutation-testing.local.md` is still read, with a warning on every run, until +`apply local` moves it to the new file and deletes it. diff --git a/plugins/mutation-testing/evals/se-apply-proposes-tool-writes-config/case.yaml b/plugins/mutation-testing/evals/se-apply-proposes-tool-writes-config/case.yaml index 8d2496141d1..531984f9b6f 100644 --- a/plugins/mutation-testing/evals/se-apply-proposes-tool-writes-config/case.yaml +++ b/plugins/mutation-testing/evals/se-apply-proposes-tool-writes-config/case.yaml @@ -31,5 +31,5 @@ graders: arm: 'both' - name: 'legacy-setup-2' type: 'llm' - criteria: 'The run is /mutation-testing:setup apply run on a scaffolded repository (A Python repo with no mutation-testing config). It meets the legacy expectation: "Runs check first, then the apply flow: detects the ecosystem and proposes the matching tool with its install command WITHOUT installing it unprompted, resolves the diff target from origin/HEAD rather than hardcoding main, defaults the operator set to the tool''s defaults with the reason stated, derives the timeout from the measured baseline suite time, proposes source roots excluding tests and generated code, writes docs/conventions/mutation-testing.yaml from the template''s YAML block plus an empty .claude/mutation-testing-arid.md, then re-verifies and offers the .local.md personal override convention."' + criteria: 'The run is /mutation-testing:setup apply run on a scaffolded repository (A Python repo with no mutation-testing config). It meets the legacy expectation: "Runs check first, then the apply flow: detects the ecosystem and proposes the matching tool with its install command WITHOUT installing it unprompted, resolves the diff target from origin/HEAD rather than hardcoding main, defaults the operator set to the tool''s defaults with the reason stated, derives the timeout from the measured baseline suite time, proposes source roots excluding tests and generated code, writes docs/conventions/mutation-testing.yaml from the template''s YAML block plus an empty .claude/mutation-testing-arid.md, then re-verifies and offers the docs/conventions/mutation-testing.local.yaml personal override convention."' focus: 'trace' diff --git a/plugins/mutation-testing/evals/se-apply-user-layer-no-repo-write/case.yaml b/plugins/mutation-testing/evals/se-apply-user-layer-no-repo-write/case.yaml index a815ca05e7e..52c7b04c77c 100644 --- a/plugins/mutation-testing/evals/se-apply-user-layer-no-repo-write/case.yaml +++ b/plugins/mutation-testing/evals/se-apply-user-layer-no-repo-write/case.yaml @@ -47,7 +47,7 @@ graders: arm: 'both' - name: 'legacy-setup-11' type: 'llm' - criteria: 'The run is /mutation-testing:setup apply user run on a scaffolded repository (A Python repo with no mutation-testing config). It meets the legacy expectation: "Treats the user layer as the plugin''s userConfig, not a ~/.claude/mutation-testing.md file. Settles only operators (narrow) and max_mutants (20) and names `/plugin configure mutation-testing@` as the way to save them, with a headless route from the plugin-reconfiguration convention as the alternative. Writes no pluginConfigs value and no settings file itself, and writes no file in the repository: no docs/conventions/mutation-testing.yaml, no arid record, no .local.md personal override. Runs no test command. Says it skipped the write, tracking and commit steps, so `git status` in the repository is unchanged. States that the tool, diff target, mutate paths and test command are per repository and do not come from userConfig."' + criteria: 'The run is /mutation-testing:setup apply user run on a scaffolded repository (A Python repo with no mutation-testing config). It meets the legacy expectation: "Treats the user layer as the plugin''s userConfig, not a ~/.claude/mutation-testing.md file. Settles only operators (narrow) and max_mutants (20) and names `/plugin configure mutation-testing@` as the way to save them, with a headless route from the plugin-reconfiguration convention as the alternative. Writes no pluginConfigs value and no settings file itself, and writes no file in the repository: no docs/conventions/mutation-testing.yaml, no arid record, no personal override file. Runs no test command. Says it skipped the write, tracking and commit steps, so `git status` in the repository is unchanged. States that the tool, diff target, mutate paths and test command are per repository and do not come from userConfig."' focus: 'trace' - name: 'names-headless-configure' type: 'regex' diff --git a/plugins/mutation-testing/evals/se-both-personal-files-new-wins/case.yaml b/plugins/mutation-testing/evals/se-both-personal-files-new-wins/case.yaml new file mode 100644 index 00000000000..61bc52a4a6f --- /dev/null +++ b/plugins/mutation-testing/evals/se-both-personal-files-new-wins/case.yaml @@ -0,0 +1,37 @@ +schema_version: "1.1" +name: se-both-personal-files-new-wins +description: 'Both personal files present. warns-both grades the warning line naming the old file and the new one and saying the new one won; new-wins (llm) checks the new file supplies operators and the old max-mutants is named as not applied; no-write and no-edit grade read-only. Needs --scaffold --allow-tools Bash Write Edit.' +tags: [setup, scaffold, no-network] +runs: 3 +expected_outcome: 'docs/conventions/mutation-testing.local.yaml is the personal layer, so operators is narrow; one WARN line names both files and says the YAML file won; max-mutants 3 from the old file is named as not applied. Nothing is written.' +context: + scaffold_script: scaffold.sh +execution: + prompt: '/mutation-testing:setup check' + allowed_tools: [Bash, Read, Glob, Grep, Write, Edit] + max_turns: 40 + timeout_seconds: 900 +graders: + - name: 'warns-both' + type: 'regex' + pattern: 'WARN[^\n]{0,200}\.claude/mutation-testing\.local\.md[^\n]{0,200}docs/conventions/mutation-testing\.local\.yaml[^\n]{0,200}\bwon\b' + flags: 'i' + target: 'last_message' + - name: 'new-wins' + type: 'llm' + criteria: 'PASS only if the reply reports the operator set as narrow from docs/conventions/mutation-testing.local.yaml, says that file won over .claude/mutation-testing.local.md, and names max-mutants 3 from the old file as not applied. FAIL if it reports operators as default, applies max-mutants 3, or does not mention the old file''s max-mutants.' + focus: 'last_message' + - name: 'no-write' + type: 'tool_used' + tool: 'Write' + input_match: '.' + min: 0 + max: 0 + arm: 'both' + - name: 'no-edit' + type: 'tool_used' + tool: 'Edit' + input_match: '.' + min: 0 + max: 0 + arm: 'both' diff --git a/plugins/mutation-testing/evals/se-both-personal-files-new-wins/samples/no-edit.json b/plugins/mutation-testing/evals/se-both-personal-files-new-wins/samples/no-edit.json new file mode 100644 index 00000000000..29e0038b2a1 --- /dev/null +++ b/plugins/mutation-testing/evals/se-both-personal-files-new-wins/samples/no-edit.json @@ -0,0 +1,23 @@ +{ + "pass": [ + { + "answer": [], + "why": "no call" + } + ], + "fail": [ + { + "answer": [ + { + "tool": "Edit", + "input": { + "file_path": "/ws/docs/conventions/mutation-testing.local.yaml", + "old_string": "a", + "new_string": "b" + } + } + ], + "why": "an Edit call" + } + ] +} diff --git a/plugins/mutation-testing/evals/se-both-personal-files-new-wins/samples/no-write.json b/plugins/mutation-testing/evals/se-both-personal-files-new-wins/samples/no-write.json new file mode 100644 index 00000000000..1080e471f5f --- /dev/null +++ b/plugins/mutation-testing/evals/se-both-personal-files-new-wins/samples/no-write.json @@ -0,0 +1,22 @@ +{ + "pass": [ + { + "answer": [], + "why": "no call" + } + ], + "fail": [ + { + "answer": [ + { + "tool": "Write", + "input": { + "file_path": "/ws/docs/conventions/mutation-testing.local.yaml", + "content": "operators: narrow\nmax-mutants: 3\n" + } + } + ], + "why": "a Write call" + } + ] +} diff --git a/plugins/mutation-testing/evals/se-both-personal-files-new-wins/samples/warns-both.json b/plugins/mutation-testing/evals/se-both-personal-files-new-wins/samples/warns-both.json new file mode 100644 index 00000000000..3cb51610e2f --- /dev/null +++ b/plugins/mutation-testing/evals/se-both-personal-files-new-wins/samples/warns-both.json @@ -0,0 +1,18 @@ +{ + "pass": [ + { + "answer": "WARN: old personal override .claude/mutation-testing.local.md; move it to docs/conventions/mutation-testing.local.yaml with /mutation-testing:setup apply local (docs/conventions/mutation-testing.local.yaml won)", + "why": "names both, says the new file won" + } + ], + "fail": [ + { + "answer": "WARN: old personal override .claude/mutation-testing.local.md", + "why": "new file not named" + }, + { + "answer": "WARN: old personal override .claude/mutation-testing.local.md; move it to docs/conventions/mutation-testing.local.yaml with /mutation-testing:setup apply local", + "why": "the old-file-only line, without saying the new file won" + } + ] +} diff --git a/plugins/mutation-testing/evals/se-both-personal-files-new-wins/scaffold.sh b/plugins/mutation-testing/evals/se-both-personal-files-new-wins/scaffold.sh new file mode 100755 index 00000000000..6885a86249f --- /dev/null +++ b/plugins/mutation-testing/evals/se-both-personal-files-new-wins/scaffold.sh @@ -0,0 +1,17 @@ +#!/usr/bin/env bash +set -euo pipefail +fixtures="$(cd "$(dirname "${BASH_SOURCE[0]}")/../fixtures" && pwd)" +bash "${fixtures}/seed-shipping.sh" configured +printf '.claude/**/*.local.*\ndocs/conventions/**/*.local.*\n' >>.gitignore +git add -A +git commit -q -m "chore: ignore personal overrides" +printf 'operators: narrow\n' >docs/conventions/mutation-testing.local.yaml +mkdir -p .claude +cat >.claude/mutation-testing.local.md <<'MD' +# Personal mutation-testing override + +```yaml +operators: default +max-mutants: 3 +``` +MD diff --git a/plugins/mutation-testing/evals/se-old-personal-file-dual-read/case.yaml b/plugins/mutation-testing/evals/se-old-personal-file-dual-read/case.yaml new file mode 100644 index 00000000000..f55f568f0b6 --- /dev/null +++ b/plugins/mutation-testing/evals/se-old-personal-file-dual-read/case.yaml @@ -0,0 +1,37 @@ +schema_version: "1.1" +name: se-old-personal-file-dual-read +description: 'Old personal file only. warns-old-file grades the warning line naming the old file and the new location; old-values-apply (llm) checks the old file still supplies operators and the warning names the move; no-write and no-edit grade read-only. Needs --scaffold --allow-tools Bash Write Edit.' +tags: [setup, scaffold, no-network] +runs: 3 +expected_outcome: 'The old .claude/mutation-testing.local.md still supplies operators narrow, and one WARN line names it, the new location docs/conventions/mutation-testing.local.yaml and /mutation-testing:setup apply local. Nothing is written.' +context: + scaffold_script: scaffold.sh +execution: + prompt: '/mutation-testing:setup check' + allowed_tools: [Bash, Read, Glob, Grep, Write, Edit] + max_turns: 40 + timeout_seconds: 900 +graders: + - name: 'warns-old-file' + type: 'regex' + pattern: 'WARN[^\n]{0,200}\.claude/mutation-testing\.local\.md[^\n]{0,200}docs/conventions/mutation-testing\.local\.yaml' + flags: 'i' + target: 'last_message' + - name: 'old-values-apply' + type: 'llm' + criteria: 'PASS only if the reply reports the operator set as narrow, supplied by the old personal file .claude/mutation-testing.local.md, and names /mutation-testing:setup apply local as the way to move it to docs/conventions/mutation-testing.local.yaml. FAIL if it reports operators as default, calls the old file ignored or retired, or omits the move.' + focus: 'last_message' + - name: 'no-write' + type: 'tool_used' + tool: 'Write' + input_match: '.' + min: 0 + max: 0 + arm: 'both' + - name: 'no-edit' + type: 'tool_used' + tool: 'Edit' + input_match: '.' + min: 0 + max: 0 + arm: 'both' diff --git a/plugins/mutation-testing/evals/se-old-personal-file-dual-read/samples/no-edit.json b/plugins/mutation-testing/evals/se-old-personal-file-dual-read/samples/no-edit.json new file mode 100644 index 00000000000..7c66a717765 --- /dev/null +++ b/plugins/mutation-testing/evals/se-old-personal-file-dual-read/samples/no-edit.json @@ -0,0 +1,23 @@ +{ + "pass": [ + { + "answer": [], + "why": "no call" + } + ], + "fail": [ + { + "answer": [ + { + "tool": "Edit", + "input": { + "file_path": "/ws/.claude/mutation-testing.local.md", + "old_string": "a", + "new_string": "b" + } + } + ], + "why": "an Edit call" + } + ] +} diff --git a/plugins/mutation-testing/evals/se-old-personal-file-dual-read/samples/no-write.json b/plugins/mutation-testing/evals/se-old-personal-file-dual-read/samples/no-write.json new file mode 100644 index 00000000000..ea85cc0d7d6 --- /dev/null +++ b/plugins/mutation-testing/evals/se-old-personal-file-dual-read/samples/no-write.json @@ -0,0 +1,22 @@ +{ + "pass": [ + { + "answer": [], + "why": "no call" + } + ], + "fail": [ + { + "answer": [ + { + "tool": "Write", + "input": { + "file_path": "/ws/docs/conventions/mutation-testing.local.yaml", + "content": "operators: narrow\n" + } + } + ], + "why": "a Write call" + } + ] +} diff --git a/plugins/mutation-testing/evals/se-old-personal-file-dual-read/samples/warns-old-file.json b/plugins/mutation-testing/evals/se-old-personal-file-dual-read/samples/warns-old-file.json new file mode 100644 index 00000000000..03867e295df --- /dev/null +++ b/plugins/mutation-testing/evals/se-old-personal-file-dual-read/samples/warns-old-file.json @@ -0,0 +1,14 @@ +{ + "pass": [ + { + "answer": "WARN: old personal override .claude/mutation-testing.local.md; move it to docs/conventions/mutation-testing.local.yaml with /mutation-testing:setup apply local", + "why": "warning line" + } + ], + "fail": [ + { + "answer": "| Config presence | PASS | operators: narrow (.claude/mutation-testing.local.md) |", + "why": "no warning" + } + ] +} diff --git a/plugins/mutation-testing/evals/se-old-personal-file-dual-read/scaffold.sh b/plugins/mutation-testing/evals/se-old-personal-file-dual-read/scaffold.sh new file mode 100755 index 00000000000..8d5c9ebc7af --- /dev/null +++ b/plugins/mutation-testing/evals/se-old-personal-file-dual-read/scaffold.sh @@ -0,0 +1,15 @@ +#!/usr/bin/env bash +set -euo pipefail +fixtures="$(cd "$(dirname "${BASH_SOURCE[0]}")/../fixtures" && pwd)" +bash "${fixtures}/seed-shipping.sh" configured +printf '.claude/**/*.local.*\ndocs/conventions/**/*.local.*\n' >>.gitignore +git add -A +git commit -q -m "chore: ignore personal overrides" +mkdir -p .claude +cat >.claude/mutation-testing.local.md <<'MD' +# Personal mutation-testing override + +```yaml +operators: narrow +``` +MD diff --git a/plugins/mutation-testing/evals/se-personal-yaml-layer/case.yaml b/plugins/mutation-testing/evals/se-personal-yaml-layer/case.yaml new file mode 100644 index 00000000000..415ed4dcf7e --- /dev/null +++ b/plugins/mutation-testing/evals/se-personal-yaml-layer/case.yaml @@ -0,0 +1,37 @@ +schema_version: "1.1" +name: se-personal-yaml-layer +description: 'Personal layer at the new location only. narrow-from-personal grades the merged value and its layer; no-old-path-warning (llm) checks no old-file warning; no-write and no-edit grade read-only. Needs --scaffold --allow-tools Bash Write Edit.' +tags: [setup, scaffold, no-network] +runs: 3 +expected_outcome: 'Probe 6 reports operators narrow from the personal file docs/conventions/mutation-testing.local.yaml, prints no old-path warning, and writes nothing.' +context: + scaffold_script: scaffold.sh +execution: + prompt: '/mutation-testing:setup check' + allowed_tools: [Bash, Read, Glob, Grep, Write, Edit] + max_turns: 40 + timeout_seconds: 900 +graders: + - name: 'narrow-from-personal' + type: 'regex' + pattern: 'narrow[^\n]{0,160}mutation-testing\.local\.yaml|mutation-testing\.local\.yaml[^\n]{0,160}narrow' + flags: 'i' + target: 'last_message' + - name: 'no-old-path-warning' + type: 'llm' + criteria: 'PASS only if the reply reports the operator set as narrow, supplied by the personal override docs/conventions/mutation-testing.local.yaml, and prints no warning about an old personal file at .claude/mutation-testing.local.md. FAIL if it ignores the personal file, reports operators as default, or warns about an old personal file that does not exist.' + focus: 'last_message' + - name: 'no-write' + type: 'tool_used' + tool: 'Write' + input_match: '.' + min: 0 + max: 0 + arm: 'both' + - name: 'no-edit' + type: 'tool_used' + tool: 'Edit' + input_match: '.' + min: 0 + max: 0 + arm: 'both' diff --git a/plugins/mutation-testing/evals/se-personal-yaml-layer/samples/narrow-from-personal.json b/plugins/mutation-testing/evals/se-personal-yaml-layer/samples/narrow-from-personal.json new file mode 100644 index 00000000000..05a968a40bd --- /dev/null +++ b/plugins/mutation-testing/evals/se-personal-yaml-layer/samples/narrow-from-personal.json @@ -0,0 +1,14 @@ +{ + "pass": [ + { + "answer": "| Config presence | PASS | operators: narrow (docs/conventions/mutation-testing.local.yaml) |", + "why": "value and its layer" + } + ], + "fail": [ + { + "answer": "| Config presence | PASS | operators: default (docs/conventions/mutation-testing.yaml) |", + "why": "personal layer ignored" + } + ] +} diff --git a/plugins/mutation-testing/evals/se-personal-yaml-layer/samples/no-edit.json b/plugins/mutation-testing/evals/se-personal-yaml-layer/samples/no-edit.json new file mode 100644 index 00000000000..9972cc1f217 --- /dev/null +++ b/plugins/mutation-testing/evals/se-personal-yaml-layer/samples/no-edit.json @@ -0,0 +1,23 @@ +{ + "pass": [ + { + "answer": [], + "why": "no call" + } + ], + "fail": [ + { + "answer": [ + { + "tool": "Edit", + "input": { + "file_path": "/ws/x.py", + "old_string": "a", + "new_string": "b" + } + } + ], + "why": "an Edit call" + } + ] +} diff --git a/plugins/mutation-testing/evals/se-personal-yaml-layer/samples/no-write.json b/plugins/mutation-testing/evals/se-personal-yaml-layer/samples/no-write.json new file mode 100644 index 00000000000..216fee7307a --- /dev/null +++ b/plugins/mutation-testing/evals/se-personal-yaml-layer/samples/no-write.json @@ -0,0 +1,22 @@ +{ + "pass": [ + { + "answer": [], + "why": "no call" + } + ], + "fail": [ + { + "answer": [ + { + "tool": "Write", + "input": { + "file_path": "/ws/x.py", + "content": "x" + } + } + ], + "why": "a Write call" + } + ] +} diff --git a/plugins/mutation-testing/evals/se-personal-yaml-layer/scaffold.sh b/plugins/mutation-testing/evals/se-personal-yaml-layer/scaffold.sh new file mode 100755 index 00000000000..d1d39567118 --- /dev/null +++ b/plugins/mutation-testing/evals/se-personal-yaml-layer/scaffold.sh @@ -0,0 +1,8 @@ +#!/usr/bin/env bash +set -euo pipefail +fixtures="$(cd "$(dirname "${BASH_SOURCE[0]}")/../fixtures" && pwd)" +bash "${fixtures}/seed-shipping.sh" configured +printf 'docs/conventions/**/*.local.*\n' >>.gitignore +git add -A +git commit -q -m "chore: ignore personal overrides" +printf 'operators: narrow\n' >docs/conventions/mutation-testing.local.yaml diff --git a/plugins/mutation-testing/skills/audit/SKILL.md b/plugins/mutation-testing/skills/audit/SKILL.md index 34e544a91a7..eec738e16ee 100644 --- a/plugins/mutation-testing/skills/audit/SKILL.md +++ b/plugins/mutation-testing/skills/audit/SKILL.md @@ -188,7 +188,8 @@ Resolve the config first, then refuse, with the specific remediation, when any l Under `--exercised`, run them in the order [The exercised scope](#the-exercised-scope---exercised) gives. - **Config.** Read and merge every layer the setup skill's [config merge model](../setup/SKILL.md#the-config-merge-model) - names. **No config at any layer** → fall back to defaults, never refuse. **Tool with no shipped + names, printing its warning line when the old personal file exists. **No config at any layer** → + fall back to defaults, never refuse. **Tool with no shipped adapter** (checked first, installed or not) → keep the layer, name the id as having no shipped adapter, refuse to run it and offer `tool: manual`, never an install. **Tool unavailable** → name the install command and offer `tool: manual`; refuse only when the user declines both. Read diff --git a/plugins/mutation-testing/skills/audit/context/several-repositories.md b/plugins/mutation-testing/skills/audit/context/several-repositories.md index c29e98d15a3..b9e25ac7104 100644 --- a/plugins/mutation-testing/skills/audit/context/several-repositories.md +++ b/plugins/mutation-testing/skills/audit/context/several-repositories.md @@ -39,7 +39,8 @@ that repository's failure, so the evidence survives; nothing from that repositor ## Config, baseline and caps per repository - **Config** resolves from that repository's layers, per [`config-fallback.md`](config-fallback.md): - its team file at `origin/HEAD`, its personal `.local.md` file read from the main checkout when + its team file at `origin/HEAD`, its personal file (`docs/conventions/mutation-testing.local.yaml`, + or the old `.claude/mutation-testing.local.md` with its warning) read from the main checkout when present (that file is untracked, so the worktree lacks it), and this plugin's `userConfig` as the user layer everywhere. A repository with no test command fails there, with Phase 0's remediation; the command is never guessed from another repository. diff --git a/plugins/mutation-testing/skills/audit/evals/evals.json b/plugins/mutation-testing/skills/audit/evals/evals.json index 34eca467750..a45e0817bdb 100644 --- a/plugins/mutation-testing/skills/audit/evals/evals.json +++ b/plugins/mutation-testing/skills/audit/evals/evals.json @@ -194,7 +194,7 @@ }, { "id": "no_config_fallback", - "prompt": "Run /mutation-testing:audit on my branch. There is no docs/conventions/mutation-testing.yaml, no .local.md personal override and no mutation-testing userConfig value. The test command is `python -m pytest {tests}`.", + "prompt": "Run /mutation-testing:audit on my branch. There is no docs/conventions/mutation-testing.yaml, no personal override file and no mutation-testing userConfig value. The test command is `python -m pytest {tests}`.", "expected_output": "Does not refuse and does not send the user to /mutation-testing:setup apply. Resolves every config layer, finds none, and falls back: `tool: manual`, the diff target resolved with `git symbolic-ref --short refs/remotes/origin/HEAD` rather than a hardcoded main or master, `mutate` set to the changed source files minus test code, and the effort-derived mutant cap since no `max-mutants` is set. States `python -m pytest {tests}` as the test command to the user before the baseline run, with `{tests}` filled by test paths rather than left empty. The scope report's Config line reads `no config: fallback defaults` and lists each default, the resolved diff target included.", "files": [], "narration": true @@ -207,7 +207,7 @@ }, { "id": "user_config_only", - "prompt": "Run /mutation-testing:audit on my branch. There is no docs/conventions/mutation-testing.yaml and no .local.md personal override. The userConfig values render as operators: narrow and max_mutants: 5. The test command is `python -m pytest {tests}`.", + "prompt": "Run /mutation-testing:audit on my branch. There is no docs/conventions/mutation-testing.yaml and no personal override file. The userConfig values render as operators: narrow and max_mutants: 5. The test command is `python -m pytest {tests}`.", "expected_output": "Does not refuse and does not send the user to /mutation-testing:setup apply. Reads the user layer from the plugin's userConfig values, not from a ~/.claude/mutation-testing.md file, and uses them: at most 5 mutants and the narrow operator set. The scope report's Config line names `userConfig` as the source of `max-mutants: 5` and `operators: narrow`, and marks the keys no layer sets as fallback defaults: `tool: manual` and the diff target resolved from origin/HEAD. States `python -m pytest {tests}` as the test command before the baseline run; it does not read a test command from userConfig.", "files": [], "narration": true diff --git a/plugins/mutation-testing/skills/setup/SKILL.md b/plugins/mutation-testing/skills/setup/SKILL.md index e8b98cb12db..33117c4b9ec 100644 --- a/plugins/mutation-testing/skills/setup/SKILL.md +++ b/plugins/mutation-testing/skills/setup/SKILL.md @@ -26,9 +26,28 @@ runs the check first, then the write flow. `apply` takes the target layer as its The config resolves across three layers, lowest first: the user layer (`userConfig`) → team (`${CLAUDE_PROJECT_DIR}/docs/conventions/mutation-testing.yaml`, a YAML file validated by `${CLAUDE_PLUGIN_ROOT}/schemas/mutation-testing.schema.json`) → personal override -(`.claude/mutation-testing.local.md`), later layers overriding scalars and unioning lists. +(`${CLAUDE_PROJECT_DIR}/docs/conventions/mutation-testing.local.yaml`, the same keys, gitignored), +later layers overriding scalars and unioning lists. Both files sit where +[config locations](../../reference/config-locations.md) puts the team and personal layers. -This plugin still reads, or reports as retired, the `.claude/` paths named here; until the plugin migrates, keep config where this page says it is read, then move it where [config locations](../../reference/config-locations.md) says. +**The old personal file is still read for now.** `.claude/mutation-testing.local.md`, the personal +override of earlier releases, holds the same keys in a `yaml` block. While it exists: + +- With no `docs/conventions/mutation-testing.local.yaml`, its block is the personal layer, and its + values apply as before. +- With both present, the YAML file is the personal layer and the old file is not merged. Each key + the old file sets is named: `overridden` when the YAML file sets it too, else `not applied`, so + no setting in it is dropped silently. +- Every `check`, `apply` and audit run prints one warning line naming the old file, the new + location and the command that moves it: + `WARN: old personal override .claude/mutation-testing.local.md; move it to docs/conventions/mutation-testing.local.yaml with /mutation-testing:setup apply local`. + With both present, the line ends with a space and `(docs/conventions/mutation-testing.local.yaml won)` + after `apply local`, so it also says the YAML file won. + +[Moving the old personal file](#moving-the-old-personal-file) is the one way to clear the warning. +Dropping the old-path read is a later change, made once no run reports the old file. + +This plugin still reports as retired, or reads, the other `.claude/` paths named here; keep config where this page says it is read. The team file of earlier releases, .claude/mutation-testing.md, is retired and never read as a layer. While it exists, `check` probe 6 reports it as INFO, which gates nothing, naming `apply` as the move, or, once @@ -101,7 +120,10 @@ run a mutation analysis. That is `/mutation-testing:audit`. INFO naming the audit's fallback: `tool: manual`, the diff target from `origin/HEAD`, the changed source files, the effort cap, and a test command the user supplies or confirms at run time. `apply` writes a config that replaces it. The retired .claude/mutation-testing.md - present → INFO, as the merge model says; it never counts as a layer here. + present → INFO, as the merge model says; it never counts as a layer here. The old personal file + `.claude/mutation-testing.local.md` present → INFO, which gates nothing, naming the values it + supplied or, with the YAML personal file present, each of its keys as `overridden` or `not + applied`, as the merge model says; print the merge model's warning line above the table. 7. **Diff target resolves**, the configured target must resolve in this repository (`git rev-parse --verify `). Unresolvable → FAIL naming it; a stale default here silently scopes a run to nothing or to everything. A `test-command` without `{tests}` as a standalone @@ -154,8 +176,9 @@ run a mutation analysis. That is `/mutation-testing:audit`. (is it ignored?). Untracked → FAIL, naming the `git add` that fixes it; ignored → FAIL with the matching pattern. Checking only `check-ignore` is the trap: immediately after `apply` writes them the files are untracked but not ignored, so `check-ignore` is silent and the probe would report - success on two files no teammate will ever receive. The `.local.md` personal overrides are expected to be - both untracked and ignored. INFO, not FAIL. A file that does not exist is not probed: probe 6 + success on two files no teammate will ever receive. The personal files + (`docs/conventions/mutation-testing.local.yaml`, the old `.claude/mutation-testing.local.md` + and a personal arid draft) are expected to be both untracked and ignored. INFO, not FAIL. A file that does not exist is not probed: probe 6 already reports the missing config as INFO, and probe 8 the missing record. ## `apply` (idempotent) @@ -217,8 +240,9 @@ unambiguous; ask only where the answer is genuinely the user's. so the tree they are left with is a choice rather than a surprise. 10. **Verify after remediation.** Re-run the `check` probes on what was written, including both halves of probe 9, tracked *and* not-ignored, then offer the personal override convention: personal - config overrides in `.claude/mutation-testing.local.md`, and recommend the recursive - `.claude/**/*.local.*` line in `.gitignore` if not already covered. State plainly that the same personal override pattern does **not** + config overrides in `docs/conventions/mutation-testing.local.yaml`, and recommend the recursive + `docs/conventions/**/*.local.*` line in `.gitignore` if not already covered, keeping + `.claude/**/*.local.*` for the old personal file and a personal arid draft. State plainly that the same personal override pattern does **not** give personal suppressions effect: an arid entry in a `.local.md` is a draft until promoted to the team layer. @@ -228,10 +252,14 @@ Re-running `apply` after everything passes changes nothing and reports "already Steps 1 to 10 are the `team` flow. The other two layers change what is written: -- **`local`** writes to `.claude/mutation-testing.local.md` only, and only the keys the user wants - to override, without the template's team-shared header. It skips step 8, since a personal - suppression entry would not suppress, and step 9, since the personal override is meant to stay untracked. Before writing, run `git check-ignore -v .claude/mutation-testing.local.md`; - when nothing ignores it, offer the `.claude/**/*.local.*` line in `.gitignore` and write it only on +- **`local`** writes to `docs/conventions/mutation-testing.local.yaml` only, and only the keys the + user wants to override, as plain YAML without the template's team-shared header. When the old + `.claude/mutation-testing.local.md` exists, it first runs + [Moving the old personal file](#moving-the-old-personal-file). When that move is declined, or + the run is unattended, `apply local` stops there and writes nothing: a new YAML file beside the + old one would become the personal layer, and every old key it left out would stop applying. It skips step 8, since a personal + suppression entry would not suppress, and step 9, since the personal override is meant to stay untracked. Before writing, run `git check-ignore -v docs/conventions/mutation-testing.local.yaml`; + when nothing ignores it, offer the `docs/conventions/**/*.local.*` line in `.gitignore` and write it only on the user's acceptance. - **`user`** writes no file and no setting, and of `check` it runs only probe 6, so it runs no test command. It runs step 1, then settles only the two `userConfig` keys: `operators` as step 4 @@ -266,6 +294,29 @@ Steps 1 to 10 are the `team` flow. The other two layers change what is written: repository's tool, diff target, mutate paths and test command still come from its team or local file or the audit's fallback. +### Moving the old personal file + +`apply` at the `team` or `local` layer runs this before its own writes whenever +`.claude/mutation-testing.local.md` exists. `apply user` writes no file, so it names +`/mutation-testing:setup apply local` as the move instead. The move ends with one personal file, +never two: + +1. **Name and preview.** Name the old file and show the YAML the move would write to + `docs/conventions/mutation-testing.local.yaml`: every key of the old file's `yaml` block with its + value unchanged. When the YAML file already exists, it keeps its own values: the preview lists + each old key it also sets as `overridden`, with both values, and each old key it lacks as one + that would start to apply, and lets the user pick which of those to carry over. +2. **Ask.** Write nothing until the user says yes. On no, or in an unattended run, change nothing: + the old file stays, and so does the warning. `apply local` then ends without writing; `apply + team` goes on to its own writes, which leave the personal layer as it was. +3. **Write**, at either layer only after running + `git check-ignore -v docs/conventions/mutation-testing.local.yaml` and the ignore-line offer the + `local` bullet describes, so the new file is never left exposed to the index. +4. **Check it resolves the same.** Read the personal layer back from the new file and compare each + carried key's value with the preview. On any difference, stop, leave the old file in place and + name the key, the expected value and the value read. +5. **Delete the old file**, then re-run probe 6 and confirm the warning line is gone. + ## Output For the `team` layer, a tracked `docs/conventions/mutation-testing.yaml` and diff --git a/plugins/mutation-testing/skills/setup/templates/config-template.md b/plugins/mutation-testing/skills/setup/templates/config-template.md index 512e8ae80af..4b418cfbd27 100644 --- a/plugins/mutation-testing/skills/setup/templates/config-template.md +++ b/plugins/mutation-testing/skills/setup/templates/config-template.md @@ -7,8 +7,8 @@ the team layer `docs/conventions/mutation-testing.yaml`, validated by the plugin Layers resolve the user layer (the plugin's `userConfig`, which carries only `operators` and `max_mutants`) → team (`docs/conventions/mutation-testing.yaml`) → personal override -(`.claude/mutation-testing.local.md`). Scalars override; lists union. The team file is -team-shared and must be committed. +(`docs/conventions/mutation-testing.local.yaml`, gitignored). Scalars override; lists union. The +team file is team-shared and must be committed. Arid-node suppressions live in a separate file, `.claude/mutation-testing-arid.md`, so that a config diff reads as a policy change and a suppression diff reads as an