You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Name every skipped file and unread value in toolchain, code-metrics, testing setup and code-tidying #15705
Refs #15703
Source: PLAN Phase 2, topic ecosystem-specifics
Depends on
none: on the frontier now
Outcome
A changed text source file that no ecosystem claims is named in the /toolchain:check and /toolchain:lint reports instead of being skipped silently, and four other tools name what they cannot read. Nothing is removed.
Demo: run /toolchain:lint on a diff that changes one .rs file in a repository with no Rust record; the report names the file.
What changes:
/toolchain:lint and /toolchain:check: an unclaimed text source file is named; binary and asset files stay quiet.
The code-metrics suppression audit and its scanner: the report names each file whose language the scanner has no kind for.
The testing plugin's setup script: its catch-all arm prints a line for any other detected language.
The three code-tidying file-scoping scripts read one shared extension list; the changed-files script gains the seven extensions it lacks, recorded as an intended change.
Method:
A characterization test first for each code-tidying caller (one fixture file per extension in the union, the old output asserted, then the named delta), and a negative-path test per script.
The two toolchain skill edits ship on a two-version compare of the toolchain check and lint cases plus one new case where a changed .rs file is named.
Run budget: 240 runs (18 cases plus 1 new, on the opus and sonnet aliases, three runs per side, after two measuring runs). Select cases with --case, use --runs 3 and a --max-cost-usd ceiling, and start no batch after a usage-limit error.
Eval batches run in the one local eval-runner session in the main checkout. Open the pull request with the do-not-merge label; the compare record releases it.
Decisions
Q26: every compare runs on the opus and sonnet aliases, three runs per side, the same count on both sides.
Owner, 2026-11-12: a compare runs only the changed skills' eval cases, not the plugin's whole suite.
Key interfaces
"Executed adapters, single-tool plugins and security floors stay; an adapter names what it cannot read instead of skipping it."
Dependency direction: skills depend on contracts and reach each other only by /plugin:skill.
Done when
AC4 (third clause): a source file outside every ecosystem is named, not skipped silently; the lint skill no longer says it silently skips.
AC8 (first clause): each tool touched here names a file or value it cannot read.
Each touched script's test file passes, including its negative-path test.
The changed-files script covers the seven added extensions, or reads the shared list.
The two-version compare is non-inferior and its record is in the pull request body, with this repository's dogfooding step.
AC7: nothing is cut here, so no control result is owed.
No work-resolve-issue run started in the 1 hour since agent-ready was last added. work-unblock removed and added agent-ready back to start a fresh run on main's workflow files. If this run ends the same way, it re-fires the issue again, waiting longer each time.
Size: Large
Complexity: Medium
Uncertainty: Known
Flags: none
Blocked by: none
Understood: /toolchain:lint and /toolchain:check should name changed text files that no ecosystem claims. The code-metrics scanner, the testing setup script and the three code-tidying scripts should also name what they cannot read. Nothing is removed.
Recorded decision (Q146, owner answer "go w/ recomemdnations"), quoted from its opening:
Decision recorded.
Answer: (a). Every changed text file no ecosystem claims is named, text assets and config files included; binary files stay quiet.
This settles the open question on which files count as unclaimed. It amends the Outcome line so that only binary files stay quiet, not asset files. A later recorded decision (Q150) concerns the compare on PR #16186 and leaves nothing open here.
The work touches several subsystems and needs an eval compare, so it is sized Large. No merged pull request in the recent list appears to already fix this, and I found no duplicate.
Not admitted to the issue lane: it carries host-run. Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).
The work-resolve-issue lane stopped on this issue (host-only): the run reports the work needs a host run the lane cannot make.
It added host-run and set Work status to confirmed: an agent run on a host builds it, and it waits on no decision from a person.
In the run's words:
host-only: the issue's done-when needs a two-version eval compare of the toolchain check and lint cases (18 cases plus 1 new, on the opus and sonnet aliases, three runs per side, 240-run budget), run in the one local eval-runner session in the main checkout, with its record in the PR body and the PR opened under the do-not-merge label that the compare record releases. This lane has no eval runner, cannot spend the model-run budget, and opens PRs ready without that label, so it cannot finish the issue as written.
What the run says it did not do:
The whole implementation: toolchain lint and check reports, the code-metrics suppression scanner, the testing setup script's catch-all arm, the shared code-tidying extension list, plus characterization and negative-path tests. Not started, because the eval compare the toolchain edits ship on can…
The two-version eval compare (opus and sonnet, three runs per side) and its record in the PR body. It needs the local eval-runner session.
Tidy, review, build and lint did not run, because the working tree has no changes.
Owner answer, 2026-11-14T16:21:44Z, in the GitHub Support hub session (transcript 851c547a), to the question below: "go w/ recomemdnations". Posted by the same hub lane after a session clear, from the transcript.
Question (Q146). In #15705, which changed files should /toolchain:lint and /toolchain:check name as unclaimed?
Answer: (a), the recommended option. Every changed text file no ecosystem claims is named, text assets and config files included (for example .svg, .css); binary files stay quiet. The options not taken were (b) only files whose extension is in a shared code-extension list, and (c) a new source-extension list inside toolchain.
This amends the issue's Outcome line that says asset files stay quiet: it now applies to binary files only.
Acceptance criteria for this decision:
/toolchain:lint and /toolchain:check name every changed text file that no resolved ecosystem claims, whatever its extension.
A changed binary file outside every ecosystem gets no mention in the reply.
An eval case covers each of the two behaviors above with a grader of its own.
Owner answer, 2026-11-14 (about 19:38Z), in the GitHub Support hub session, to the question below: "go w/ recommendations". Posted by that session.
Question (Q150). The third two-version compare of pull request #16186 (registration 3) read opus MET and sonnet NOT MET on its paired rule: change +0.00, 95% interval -0.06 to +0.07 against a 0.05 margin, with one case, ln-go-fix-filters-non-go, down from 1.00 to 0.67 on one run of three. Registration 3 forbids reruns. What happens to the pull request?
Answer: (a), the recommended option, an exception to the no-rerun clause. One diagnostic check of the dropped case only (ln-go-fix-filters-non-go, sonnet, 6 runs per side), with its bar written before any run (registration 4). Registration 3's sonnet result stays recorded as NOT MET whatever the check shows. If the after side scores no lower than the before side, the pull request is released with both records in its body; if lower, it stays held for a text fix and a fourth full compare. The options not taken were (b) a text change and a fourth full compare now, and (c) release with no further evidence.
This amends the issue's done-when item "The two-version compare is non-inferior" for this pull request: it is met by registration 3's record (opus non-inferior; sonnet rules 1 and 3 met, rule 2 missed on one case) together with a registration 4 diagnostic check that meets its bar.
Result of the check (re-derived from its two result files): both files VALID, before mean 1.00 (6 of 6 runs pass every grader), after mean 1.00 (6 of 6), bar met.
Acceptance criteria for this decision:
The pull request body carries registration 3's record with the sonnet rule 2 miss stated as NOT MET.
The pull request body carries registration 4's record, with the bar that was written before the run and its result.
Registration 4's result shows the after-side mean on ln-go-fix-filters-non-go at or above the before-side mean, both files VALID.
Activity
melodic-automation-lanes commentedon Nov 13, 2026
No work-resolve-issue run started in the 1 hour since
agent-readywas last added. work-unblock removed and addedagent-readyback to start a fresh run on main's workflow files. If this run ends the same way, it re-fires the issue again, waiting longer each time.github-actions commentedon Nov 13, 2026
Size: Large
Complexity: Medium
Uncertainty: Known
Flags: none
Blocked by: none
Understood:
/toolchain:lintand/toolchain:checkshould name changed text files that no ecosystem claims. The code-metrics scanner, the testing setup script and the three code-tidying scripts should also name what they cannot read. Nothing is removed.Recorded decision (Q146, owner answer "go w/ recomemdnations"), quoted from its opening:
This settles the open question on which files count as unclaimed. It amends the Outcome line so that only binary files stay quiet, not asset files. A later recorded decision (Q150) concerns the compare on PR #16186 and leaves nothing open here.
The work touches several subsystems and needs an eval compare, so it is sized Large. No merged pull request in the recent list appears to already fix this, and I found no duplicate.
Automated triage. A maintainer reviews it.
melodic-automation-lanes commentedon Nov 13, 2026
Not admitted to the issue lane: it carries host-run. Admission takes the issue once each of those is met, when intake triages it again (
.github/actions/resolve-config/admission-rules.json).2 remaining items
github-actions commentedon Nov 14, 2026
melodic-automation-lanes commentedon Nov 14, 2026
The work-resolve-issue lane stopped on this issue (
host-only): the run reports the work needs a host run the lane cannot make.It added
host-runand set Work status to confirmed: an agent run on a host builds it, and it waits on no decision from a person.In the run's words:
What the run says it did not do:
Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/43539931117
To hand it back to the lane, remove
host-runand set Work status to agent-ready again.kyle-sexton commentedon Nov 14, 2026
Decision recorded.
Owner answer, 2026-11-14T16:21:44Z, in the GitHub Support hub session (transcript 851c547a), to the question below: "go w/ recomemdnations". Posted by the same hub lane after a session clear, from the transcript.
Question (Q146). In #15705, which changed files should
/toolchain:lintand/toolchain:checkname as unclaimed?Answer: (a), the recommended option. Every changed text file no ecosystem claims is named, text assets and config files included (for example
.svg,.css); binary files stay quiet. The options not taken were (b) only files whose extension is in a shared code-extension list, and (c) a new source-extension list inside toolchain.This amends the issue's Outcome line that says asset files stay quiet: it now applies to binary files only.
Acceptance criteria for this decision:
/toolchain:lintand/toolchain:checkname every changed text file that no resolved ecosystem claims, whatever its extension.kyle-sexton commentedon Nov 14, 2026
Decision recorded.
Owner answer, 2026-11-14 (about 19:38Z), in the GitHub Support hub session, to the question below: "go w/ recommendations". Posted by that session.
Question (Q150). The third two-version compare of pull request #16186 (registration 3) read opus MET and sonnet NOT MET on its paired rule: change +0.00, 95% interval -0.06 to +0.07 against a 0.05 margin, with one case,
ln-go-fix-filters-non-go, down from 1.00 to 0.67 on one run of three. Registration 3 forbids reruns. What happens to the pull request?Answer: (a), the recommended option, an exception to the no-rerun clause. One diagnostic check of the dropped case only (
ln-go-fix-filters-non-go, sonnet, 6 runs per side), with its bar written before any run (registration 4). Registration 3's sonnet result stays recorded as NOT MET whatever the check shows. If the after side scores no lower than the before side, the pull request is released with both records in its body; if lower, it stays held for a text fix and a fourth full compare. The options not taken were (b) a text change and a fourth full compare now, and (c) release with no further evidence.This amends the issue's done-when item "The two-version compare is non-inferior" for this pull request: it is met by registration 3's record (opus non-inferior; sonnet rules 1 and 3 met, rule 2 missed on one case) together with a registration 4 diagnostic check that meets its bar.
Result of the check (re-derived from its two result files): both files VALID, before mean 1.00 (6 of 6 runs pass every grader), after mean 1.00 (6 of 6), bar met.
Acceptance criteria for this decision:
ln-go-fix-filters-non-goat or above the before-side mean, both files VALID.