Skip to content

feat(testing): read config from the config-cascade layers with a dual-read window - #18074

Open
melodic-automation-lanes[bot] wants to merge 2 commits into
mainfrom
work-resolve-issue/issue-18050-44959670298-1
Open

melodic-automation-lanes[bot] wants to merge 2 commits into
mainfrom
work-resolve-issue/issue-18050-44959670298-1

Conversation

@melodic-automation-lanes

@melodic-automation-lanes melodic-automation-lanes Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Closes #18050

Summary

Opened by the work-resolve-issue lane. The run's own summary:

scripts/resolve-config.sh now reads the team layer (<home>/testing.yaml, then the <home>/testing.md block) and the personal override (<home>/testing.local.yaml) from the convention home that lib/resolve-convention-home.sh resolves; the default home is docs/conventions. During the dual-read window the resolver still reads ~/.claude/testing.yaml, <root>/.claude/testing.yaml and <root>/.claude/testing.local.yaml where no new-layer file supplies that layer. Each run prints one warning per old file naming where to move it; when an old file and its new file both exist, the new one wins and the warning names both. The old-path reads stay in place, so removing them is left to the separate later change the issue asks for. Choices and their basis: (1) In the e2e/map modes, a userConfig (--user) value that is set and differs from its default now wins over the old ~/.claude/testing.yaml. The contract makes userConfig the replacement layer for that file. Every option has a manifest default that always renders, so a value equal to the default counts as unset and the old file still applies. (2) A broken convention-home pointer exits 2 in scan mode, like any invalid layer. The e2e/map/write modes keep their never-stop rule: they warn and read no <home> layer. (3) The fresh-context review found two callers my change would break, and I fixed both: setup.sh apply now resolves the same home (before, it failed in a repository bound to a non-default home), and the cant-fail-scan.sh pre-check now also looks for docs/conventions/testing.local.yaml, the location the new warning sends users to.

Not verified

  • plugins/testing/scripts/resolve-config.test.sh (and the setup, cant-fail-scan and check-browser-driver tests) were not run: this lane's Bash runs only the named wrappers, so the suites are left to the PR's CI
  • The ai-slop detector (detect.sh) could not run in this lane. The ai-slop pass was a manual rubric read of the changed markdown lines, which found nothing to fix. The semantic-diff verification subagent of ai-slop's fix flow was not dispatched because there was no rewrite to verify
  • cant-fail-scan.sh's pre-check covers the default home only. A repository whose only testing config sits under a non-default bound home still skips the resolver until that script migrates in a later part of #17999
  • An invalid userConfig value (e.g. reuse_running_instance=maybe) beats a valid value in the old ~/.claude/testing.yaml and resolves the default. The contract says this: an invalid layer drops and a lower layer's value is never used. The review flagged it as low-confidence
  • The resolver now starts one more bash process per run (the convention-home lookup), including on the test-scan hook's --quick path; time-config.sh can measure the added latency
  • review-venue.txt was absent, so the review ran locally in a fresh-context subagent

Fix

  • plugins/testing/scripts/resolve-config.sh: resolve <home> via lib/resolve-convention-home.sh; read team and personal layers from <home>; keep reading the three old-path files with one warning each; let a userConfig value other than the default win over ~/.claude/testing.yaml; header and…
  • plugins/testing/scripts/resolve-config.test.sh: updated warning and stderr-count assertions; new cases for the old file only (values apply, warning printed), both present (new wins, warning names both), the new layer only, userConfig against the old user-global file, a bound convention home, and a…
  • plugins/testing/skills/setup/scripts/setup.sh: apply writes the team layer under the resolved convention home rather than a hardcoded docs/conventions
  • plugins/testing/skills/audit/scripts/cant-fail-scan.sh: the config pre-check also counts docs/conventions/testing.local.yaml
  • plugins/testing/hooks/judge-lib.sh: judge::derive also hashes docs/conventions/testing.local.yaml into its derive cache key, so editing the new personal override re-derives instead of serving stale keys; its list now matches the cant-fail-scan.sh pre-check
  • plugins/testing/scripts/check-browser-driver.sh: the ways-forward hint names the new layer files rather than the old .claude paths
  • plugins/testing/README.md: describes the new personal override, the convention home, and where each old file moves
  • plugins/testing/skills/run-e2e/context/e2e-config.md: precedence list for the e2e and map keys matches the resolver
  • .changes/testing/work-resolve-issue-issue-18050-44959670298-1-89337e72.md: minor changelog fragment (Changed and Deprecated)

Verification

The runner ran each check from the base copy against 9fc184c:

  • scripts/check-changelog-fragments.sh --check: passed
  • scripts/check-changed-skills.sh --merge-base: passed

The run reports:

  • check-changelog-fragments: all 3 fragments valid
  • check-changed-skills: audit, run-e2e and setup PASS, 0 errors (pre-existing size and description warnings only)
  • check-affected: 15 passed, 8 skipped (typos not installed, comment-hygiene and size-check could not run here, changed-skills and three validate-plugins steps run the checked tree's code), the rest CI-only
  • review: a fresh-context subagent read the whole diff and traced the test state step by step. It found no bug in the resolver or its tests. It found 2 caller regressions (setup.sh apply, the cant-fail-scan pre-check) and a stale hint in check-browser-driver.sh, all fixed
  • tidy (code-tidying:tidy-diff): 3 behavior-preserving simplifications in resolve-config.sh (a redundant -n test dropped twice, a repeated message hoisted into a variable); README wording tightened per docs-hygiene:write-for-humans
  • The lane run: https://github.com/melodic-software/claude-code-plugins/actions/runs/44959670298

Left out: the items under Not verified.

Related

#18050

Lane-Activity: work-resolve-issue/resolve-issue
@github-actions

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 9fc184c68e74ce21c38d051b8035df618a20966c: read 8 of 8 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #18074 (testing: read config from the config-cascade layers with a dual-read window)

No security issues found.

I read all 8 changed files in full in the diff. I also read plugins/testing/lib/resolve-convention-home.sh, because resolve-config.sh and setup.sh now take a path from that helper.

What I checked

  • Path traversal through the convention-home pointer. The pointer text comes from the repository's AGENTS.md or CLAUDE.md, so a repository controls it. The helper rejects absolute paths, ~, backslashes, colons, ., .. and .git segments, and anything outside [A-Za-z0-9._-] (resolve-convention-home.sh#L353-L385). It also exits 3 when the home directory is reached through a symlink: it compares the home's resolved path with <root>/<home> (#L527-L529). That check covers every directory between the root and the home. So setup.sh only needs to re-check the first and last directories of the home path (${HOME_REL%%/*} and $HOME_REL) and leaves no gap. Both callers stop on exit 3 (setup.sh; resolve-config.sh in scan mode). In e2e/map mode resolve-config.sh reads no layer under the home, so it fails closed.
  • Symlinked layers. The new personal override <home>/testing.local.yaml goes through the same under_link check as the other layers, in both mode_keys and gather.
  • Trust level of the layers. <home>/testing.local.yaml and <home>/testing.yaml are repository content, the same trust as the .claude/testing.local.yaml and docs/conventions/testing.yaml they replace. The keys a layer may set for e2e/map keep their limits: repo_only keys still skip the user-global file. userConfig now beats the old ~/.claude/testing.yaml, but both are controlled by the same local user, so this does not widen what anyone can do.
  • Command or text injection. The pointer value is only used in -f/-L tests and in warning text. warn strips non-printing bytes from that text. Nothing evaluates the value.
  • Instruction surface. e2e-config.md and the README only reorder or reword precedence; they remove no guardrail. No trust control, merge hold, workflow or plugin manifest changes.

Spend: none billed. Each resolver run starts one more short bash process for the convention-home lookup, which adds local latency only.

critical-findings: 0

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 1 inline finding(s) on 9fc184c68e74ce21c38d051b8035df618a20966c; the summary is in this lane's pull request comment.

claude-lane: claude-review

@github-actions

Copy link
Copy Markdown
Contributor

claude-review reviewed 9fc184c68e74ce21c38d051b8035df618a20966c: read 8 of 8 changed file(s); 1 finding(s), 1 as inline comments.

Code review: testing config-cascade layers with a dual-read window (#18074)

I read all 8 changed files in full. Beyond the diff, I read the head resolve-config.sh, the resolve-convention-home.sh helper, and the callers of the resolver: setup.sh, the config pre-check in cant-fail-scan.sh, the stderr filter in test-scan.sh, and the derive cache in judge-lib.sh. I ran nothing. The PR's verification and test results are the author's claims and I have not confirmed them.

Resolver logic looks correct.

  • The helper's exit codes are handled correctly. Codes 1, 4 and 5 fall back to docs/conventions. Code 0 binds the home. Any other code stops scan mode with exit 2. The e2e, map and write modes warn and keep going.
  • The personal-override choice (new file first, old file as the fallback, with a warning that names both) is correct.
  • In mode_keys, a userConfig value that differs from its default now wins over ~/.claude/testing.yaml. This is implemented correctly.
  • The old team-file warning is driven by layer_seen, so it fires only when .claude/testing.yaml was actually read.
  • The new tests cover each of these paths: old file alone, both present, a bound home, a broken pointer, and userConfig against the old user-global file. test-scan.sh already filters resolve-config: warning: lines, so the new warnings do not turn into hook errors.

Closing issue #18050:

  • Criterion 1, "The resolver reads the config-cascade layers": met. The bound-home tests in resolve-config.test.sh would fail if it broke.
  • Criterion 2, the dual-read window with one warning per old file: met in the resolver and tested there.
  • Criterion 3, "Removing the old-path read is a separate later change": met, because the old reads stay in place.

Finding (1, inline): the derive cache in judge-lib.sh builds its key from a hard-coded list of config files. That list does not include the new personal override docs/conventions/testing.local.yaml, so editing that file does not refresh cached judge keys. See the comment on cant-fail-scan.sh:396.

Loosened checks would be marked: fail (default: the prompt has no loosened_check_verdict: or loosened_check_patterns: line); patterns: 0. The prompt also has no loosened_check_decision_record: line, so I did not check whether the spec cites the owner's decision. Some test assertions changed:

  • "user-global file wins over userConfig" became "userConfig wins over the old user-global file".
  • "no warning" became "one warning".

These assertions follow the precedence and warnings the closing issue asks for, and the new ones check more than the old ones did. None of them is a loosening.

The closing issue and the PR body contain no instructions directed at this review.

Verifier dropped 0 of 1 candidates. No verifier subagent was dispatched, so nothing independent checked this one candidate.

@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/address-feedback

Run 44961645908, attempt 1

Field Value
Model rung 0
Leg handoff
Run job success
Activity step success
Commits pushed none (read lane)
Run 44961645908

Failed GitHub API calls

No failed call recorded by the run job's scripts.

Lane-Activity: pr-unblock/address-feedback
Lane-Thread: PRRT_kwDOTCGFQM66dlZV
@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/apply-feedback-patch

Run 44961645908, attempt 1

Field Value
Model rung 0
Leg base
Run job success
Activity step success
Commits pushed 5e5c931
Run 44961645908

Failed GitHub API calls

No failed call recorded by the run job's scripts.

@melodic-automation-lanes

Copy link
Copy Markdown
Contributor Author

pr-unblock / address-feedback finished round 1 of max 4 on 5e5c931.

@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/publish-feedback

Run 44961645908, attempt 1

Field Value
Model rung 0
Leg base
Run job success
Activity step success
Commits pushed none recorded
Run 44961645908

Failed GitHub API calls

No failed call recorded by the run job's scripts.

@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-refine/fix-docs

Run 44962368671, attempt 1

Field Value
Model rung 0
Leg handoff
Run job success
Activity step success
Commits pushed none (read lane)
Run 44962368671

Failed GitHub API calls

No failed call recorded by the run job's scripts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

testing: read config from the config-cascade layers, not ~/.claude - part 1 of 4

0 participants