Skip to content

feat(toolchain)!: delete the bundled ecosystem defaults and per-language check pages - #17926

Open
melodic-automation-lanes[bot] wants to merge 6 commits into
mainfrom
work-resolve-issue/issue-16032-44896998505-1
Open

melodic-automation-lanes[bot] wants to merge 6 commits into
mainfrom
work-resolve-issue/issue-16032-44896998505-1

Conversation

@melodic-automation-lanes

@melodic-automation-lanes melodic-automation-lanes Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Closes #16032

Summary

Opened by the work-resolve-issue lane. The run's own summary:

Deletes toolchain's nine bundled ecosystem files and /toolchain:check's six per-language context pages, so check and lint run only a recorded, stated, inferred or (attended) answered command and report anything else as skip (unresolved: ...). The resolution ladder now has three rungs plus an unresolved report; coverage comes from records, stated instructions and build-file inference, and in a repository with no ecosystem record of its own (none at the team or personal layer; a user-global record does not count) a changed file's type infers its id (chosen so the existing no-record evals still read unresolved while this repository, which has records, keeps its current command list). The check skill keeps the retry, seed and slow-test reporting rules in tool-neutral form, setup drafts from inference only, and lint states the workflow-security scan (zizmor, advisory) itself next to the secret scan that part 1 added, since the deleted yaml file was where that scan was mentioned.

Review round: the ladder and setup now carry the package-runner rule the deleted typescript files held, in tool-neutral form (name the package the manifest declares, such as @biomejs/biome, and use the runner's no-download option); the no-record condition reads the same in all three places in the ladder; and lint's zizmor section states our decisions with anchored pointers instead of restating zizmor's config names, offline behavior and exit-code table.

Not verified

  • The no-record fixture's before/after eval run (done-when item 2) and every other toolchain eval run: eval batches run in the owner's local eval-runner session, which this lane cannot reach. The change keeps the no-record cases reading skip (unresolved: ...) by rule, but no run shows it yet.
  • docs/contracts/ecosystem-commands/README.md still lists the bundled-defaults rung 4 (and examples/go.yaml mentions a rung-4 default). That is step 8 of the parent plan (contract major plus migration note), outside this part's done-when; the reviewer flagged it as a mismatch with the ladder in the…
  • plugins/docs-hygiene/context/derivability-route-followups.md rows 61-65 name the deleted context pages in a dated…
  • Other plugins still describe toolchain's bundled-default rung (plugins/autonomy/reference/prerequisite-resolution.md, plugins/verification/skills/confirm/SKILL.md): out of this item's scope, which names toolchain only.
  • Tidy skipped: the change is 31 files and touches scripts/, over the 10-file cutoff and on a listed shared path.
  • check-evals-quality does not apply to these case.yaml evals (it reads JSON eval sets and refused the YAML cases); eval-cases validation and evals-quality are left to the PR's CI.
  • Build and test: no build or test suite covers these markdown, YAML and eval-sample files locally; the PR's CI runs the suites.
  • Untracked dotfiles at the checkout root (.bash_profile, .bashrc, .gitconfig, .idea, .mcp.json, .profile, .ripgreprc, .vscode, .zprofile, .zshrc) were not created by this run; they look like sandbox artifacts and belong out of the patch.

Fix

  • docs/adr/0101: the Context example describes the bundled files without naming the deleted path (AC4). docs/architecture/landscape.json: the deleted check/context/python.md is dropped from the pytest-randomly and pytest-rerunfailures rows.
  • plugins/toolchain/reference/ecosystems/*.yaml (9 files): deleted (bash, cross-cutting, dotnet, go, markdown, powershell, python, typescript, yaml).
  • plugins/toolchain/skills/check/context/{bash,dotnet,go,powershell,python,typescript}.md: deleted.
  • plugins/toolchain/reference/resolution-ladder.md: three rungs; a 'Nothing resolved' unresolved report replaces the bundled fourth rung; covered ids are recorded, stated or inferred (file-type inference only in a repository with no ecosystem record at the team or personal layer, worded the same in all three places), with id naming pinned to the alias ids; rung 2 names a package-runner tool by the package the manifest declares and uses the runner's no-download option; resolution is per…
  • plugins/toolchain/skills/check/SKILL.md: description follows the mechanism (project's own commands only); ladder text has no rung 4; context list now points at gates.md and sarif.md only; retry/flaky, shuffle-seed and slowest-tests rules moved in as tool-neutral text; ecosystem-specific opt-in and…
  • plugins/toolchain/skills/check/context/gates.md: drops references to the bundled go.yaml gate and context/go.md; gates come from records only.
  • plugins/toolchain/skills/lint/SKILL.md: three-rung ladder, no bundled list; new 'Workflow-security scan' section (zizmor --offline ., advisory row, opt-in on a zizmor config), stating our decisions with anchored pointers to zizmor's config discovery, operating modes and exit codes, read at run time, and no as-of Basis line; security-scan opt-in case and record rule cover both scans; advisory cell added to the report rules; $EC_BIN resolution…
  • plugins/toolchain/skills/setup/SKILL.md: drafts from inference only; no bundled seed; a drafted package-runner command names the declared package and uses the no-download option; INFO wording for unconfigured ecosystems now says they infer, ask or read unresolved.
  • plugins/toolchain/README.md: describes resolution without bundled defaults.
  • plugins/toolchain/evals/ck-consumer-config-wins/{case.yaml,prompt.md}: llm criterion and description drop the rung-4 expectation.
  • plugins/toolchain/evals/{ck-unattended-inferred-runs,ck-unattended-stated-unresolved}/samples/logged-assumed.json, {ck-unattended-stated-unresolved,ck-unsure-run-is-unattended}/samples/no-record-write.json: fail samples no longer name the deleted folder; each still fails its grader regex.
  • scripts/org-repo-tokens-allow.tsv: removed the nine rows for the deleted YAML files.
  • scripts/shell-portability-skill-md-baseline.txt: removed the stale entry for the deleted check/context/powershell.md.
  • scripts/affected-tests-no-suite.txt: comment no longer names the deleted folder.
  • .changes/toolchain/work-resolve-issue-issue-16032-44896998505-1-d4f74506.md: major fragment with removal, migration note (typos, editorconfig-checker, lychee --offline and actionlint dropped on purpose; secret scan stays), coverage, the workflow-security scan and the check-skill changes.

Verification

The runner ran each check from the base copy against da2c7c1:

  • scripts/check-changelog-fragments.sh --check: passed
  • scripts/check-changed-skills.sh --merge-base: passed

The run reports:

  • check-changelog-fragments: all 7 fragments valid (first run failed on an unknown '### Migration' heading; moved the note under Removed).
  • check-changed-skills: check, lint and setup PASS, 0 errors (warnings only: body size, router cap, pre-existing missing Next/Gotchas on lint and setup).
  • check-affected: 14 passed, 9 skipped (typos not installed; comment-hygiene and size-check could not fetch or find their scripts; validate-plugins sub-steps and changed-skills run code from the checked tree), 0 failed.
  • Grep over the repo, changelogs excluded, for 'toolchain/reference/ecosystems' and 'reference/ecosystems/': only the kept fixture in scripts/affected-tests.test.sh remains (ADR 0101's Context line and two landscape.json rows were reworded in a follow-up commit).
  • Grep of plugins/toolchain for 'bundled', 'rung 4', 'four-rung': only 'bundled in one command string' phrases remain.
  • Fresh-context review subagent over the whole diff: found a coverage gap (no-record .sh/.md files reaching no ecosystem, breaking ck-unattended-stated-unresolved and ck-unsure-run-is-unattended), per-key vs per-ecosystem ladder wording, attended runs asking for keys they never run, a missing…
  • read https: //docs.zizmor.sh/usage/: zizmor takes a repository-root directory input, --offline disables all online actions including repository fetches, exit 0 means no findings, 11-14 mean findings by highest severity (10 retired in v1.14.0).
  • read https: //docs.zizmor.sh/configuration/: zizmor discovers .github/zizmor.yml, .github/zizmor.yaml, zizmor.yml or zizmor.yaml at the repository root.
  • The lane run: https://github.com/melodic-software/claude-code-plugins/actions/runs/44896998505

Left out: the items under Not verified.

Related

#16032

Lane-Activity: work-resolve-issue/resolve-issue
@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-refine/fix-docs

Run 44899164587, attempt 1

Field Value
Model rung 0
Leg handoff
Run job success
Activity step success
Commits pushed none (read lane)
Run 44899164587

Failed GitHub API calls

No failed call recorded by the run job's scripts.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-security-review: 1 inline finding(s) on da2c7c1e1482f8a7df917d159642a1519a612354; the summary is in this lane's pull request comment.

claude-lane: claude-security-review

Comment thread plugins/toolchain/skills/setup/SKILL.md Resolved
@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed ca1f09ba728c9428f1c0af919bfdbc5cbcff868e: read 18 of 18 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #17926 (full diff at ca1f09b)

No security issues found. This lane already reviewed an earlier head, so this review was blocking only: it reports only CRITICAL and IMPORTANT findings.

Scope. I read every hunk in .claude-lane/pr.diff. That covers all 18 listed files and the 15 files the PR deletes whole: nine plugins/toolchain/reference/ecosystems/*.yaml and six skills/check/context/*.md. No changed file matches a trusted path. The diff changes no plugin manifest, workflow, token request, kill switch, merge hold, hook or trusted-actor list. The new head is a merge from main, and the PR's own hunks match what the previous head reviewed.

Deleted instructions. I checked the deleted standing instructions against the protected classes in docs/contracts/instruction-exception-register/README.md:

  • Secret scan. Deleting the bundled cross-cutting file removes its gitleaks … --redact scan. Lint still runs its own secret scan in attended and unattended runs (lint SKILL.md:44).

  • Package runners. The scoped-package rule from the deleted typescript files now applies to any tool:

    Either way, a package-runner command names the package the manifest declares and passes the runner's no-download option. The deleted bundled defaults also ran bare npx tsc and npx markdownlint-cli2 with no such guard, so the deletion lowers the risk of fetching an undeclared package.

  • Code-changing fixes. The --code-fix confirmation gate, the scope fence and "never install a tool" are unchanged.

Trust boundary.

  • Unattended runs still take commands only from a record, the repository's own instructions, or its build files. They still copy no command text out of CI workflow files.
  • The new "Stated" coverage bullet adds no new command source: unattended runs could already run stated commands.
  • File-type inference in a repository with no ecosystem record of its own supplies only an id and its globs, never a command. Such a key reads skip (unresolved: …).

Earlier threads.

  • My earlier finding on the package-runner rule (discussion_r4652273769) is fixed, and its lines are unchanged since it was resolved.
  • The ladder's no-record condition reads the same in all three places, and the zizmor section is gone. Both settled threads stay settled.

PR text. The PR context contains no instruction aimed at this lane.

Spend. None.

critical-findings: 0

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 2 inline finding(s) on da2c7c1e1482f8a7df917d159642a1519a612354; the summary is in this lane's pull request comment.

claude-lane: claude-review

Comment thread plugins/toolchain/skills/lint/SKILL.md Resolved
@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed ca1f09ba728c9428f1c0af919bfdbc5cbcff868e: read 18 of 18 changed file(s); 1 finding(s), 0 as inline comments.

Code review (blocking only): toolchain drops its bundled ecosystem defaults (#17926)

Scope. I read every hunk of all 18 listed files in .claude-lane/pr.diff. I also read the hunks for the 9 deleted reference/ecosystems/*.yaml files and the 6 deleted check/context/*.md pages. This lane already reviewed an earlier head (9400abb7), so this review reports blocking findings only. The head ca1f09ba is a merge of main. I ran no tests or evals. The PR body's verification steps are the author's claims, and I did not check them.

What the change does. It deletes toolchain's 9 bundled ecosystem files and /toolchain:check's 6 per-language context pages. The resolution ladder goes from four rungs to three, followed by a "Nothing resolved → unresolved" report. An ecosystem counts as covered when it is recorded, stated or inferred. In a repository with no ecosystem record of its own, a changed file's type also infers its id. Check, lint and setup are reworded to match. The retry, seed and slow-test rules now live in check SKILL.md, written for any test runner. The package-runner rule (name the declared package, use npx --no) is back, worded for any tool. The changelog fragment is bump: major, with a migration note.

Closing issue #16032

  • AC4 (no bundled default command runs, the folder is gone, and nothing outside changelogs and the kept fixture names it): met, under the owner's option-B decision on the issue, which exempts accepted ADRs. ADR 0101 keeps its Context line and adds a dated amendment.
  • Ladder no longer describes a bundled default; shell-portability baseline drops its stale entry: met. The check/context/powershell.md entry is removed.
  • AC10 (no new enforcement-placing skill): met.
  • No-record after-run eval: moved to evals: changes merged without their eval while the gate is suspended (2026-11-21) #18018 by owner decision, so it is not judged here.

Blocking findings

No new blocking finding at this head.

Not raised again. These settled threads are resolved, and nothing at this head reopens them: the Biome package-runner rule, the no-record condition wording, the zizmor restatement, ADR 0101 and landscape.json, and the after-run eval.

Other notes

  • Loosened checks (kind 6, a check removed). The removed bundled commands (typos, editorconfig-checker, lychee, actionlint, shellcheck/shfmt, ruff/pyright, golangci-lint, the go-mod-tidy-drift gate and the rest) are removed checks.
    • The spec allows them: Delete toolchain's bundled ecosystem defaults - part 2 of 3 #16032 AC4 requires that "toolchain runs no bundled default command", and the bundled ecosystems folder holds those commands.
    • The allowlist and baseline removals only make checks stricter.
    • The prompt has no loosened_check_verdict: or loosened_check_patterns: line, so I used fail with no consumer entries.
    • It has no loosened_check_decision_record: line either, so I did not check the citation requirement.
    • Loosened checks would be marked: fail (plugin default); patterns: 0
  • Manifests: no .claude-plugin manifest changed, so the manifest checks do not apply.
  • Spend: none beyond prepaid usage. The change only edits skill text and eval samples.

Verifier dropped 0 of 0 candidates.

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/address-feedback

Run 44904495798, attempt 1

Field Value
Model rung 0
Leg handoff
Run job success
Activity step success
Commits pushed none (read lane)
Run 44904495798

Failed GitHub API calls

No failed call recorded by the run job's scripts.

@melodic-automation-lanes

melodic-automation-lanes Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor Author

Unattended merge: native auto-merge leaves this pull request unarmed at head ca1f09ba728c9428f1c0af919bfdbc5cbcff868e. No person needs to act: a lane clears this reason, and the next scheduled run judges it again.

a pr-unblock lane stopped its head (pr-unblock/address-feedback is failure), which clears when the
  lane runs again; its review rounds are not converging (feedback-not-converging)

Lane-Activity: pr-unblock/address-feedback
Lane-Thread: PRRT_kwDOTCGFQM66VxhH
Lane-Thread: PRRT_kwDOTCGFQM66VyQY
Lane-Thread: PRRT_kwDOTCGFQM66VyQh
@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/apply-feedback-patch

Run 44904495798, attempt 1

Field Value
Model rung 0
Leg base
Run job success
Activity step success
Commits pushed 61f95c0
Run 44904495798

Failed GitHub API calls

No failed call recorded by the run job's scripts.

@melodic-automation-lanes

Copy link
Copy Markdown
Contributor Author

pr-unblock / address-feedback finished round 1 of max 4 on dc8569b.

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/publish-feedback

Run 44904495798, attempt 1

Field Value
Model rung 0
Leg base
Run job success
Activity step success
Commits pushed none recorded
Run 44904495798

Failed GitHub API calls

No failed call recorded by the run job's scripts.

@melodic-automation-lanes

Copy link
Copy Markdown
Contributor Author

pr-unblock / address-feedback finished round 2 of max 4 on dc8569b.

Comment thread plugins/toolchain/skills/lint/SKILL.md Resolved
@melodic-automation-lanes

melodic-automation-lanes Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor Author

The pr-unblock lane stopped address-feedback on this pull request: review rounds are not converging: after the round cap and one fresh fixer round the code review still has open findings, so the fixer and the reviewer disagree on what the change should be; on a lanes App pull request the pr-merge lane then closes it and re-admits its issue for a fresh attempt. The lane sets no label: the run ends in a failing pr-unblock/address-feedback status on the head the lane gated.

The review rounds are not converging after the round cap and one fresh fixer round. Open review threads:

The pr-merge lane closes this pull request on its next run and re-admits its issue for a fresh attempt. To keep it open for one more round, first add needs-human, which holds the close but not a requested round, then reply on a thread, from an account with write access, starting PR-fix lane: please fix this finding. do-not-merge holds the close too, but it also stops the lane's rounds, so with it a person resolves the threads and dispatches the workflow below before removing it.

To run it again on this head now, dispatch the pr-unblock workflow:

gh workflow run pr-unblock.yml -R melodic-software/claude-code-plugins -f pr-number=17926 -f activity=address-feedback

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/report-escalation

Run 44960287669, attempt 1

Field Value
Model rung 0
Leg base
Run job success
Activity step success
Commits pushed none recorded
Run 44960287669

Failed GitHub API calls

No failed call recorded by the run job's scripts.

@melodic-automation-lanes

Copy link
Copy Markdown
Contributor Author

pr-unblock / address-feedback finished round 3 of max 4 on dc8569b.

kyle-sexton and others added 2 commits November 21, 2026 07:29
…he landscape

Done-when AC4 of #16032 lets only changelogs and the kept test fixture name
plugins/toolchain/reference/ecosystems/; ADR 0101's Context line now describes
the bundled files without the path. landscape.json drops the deleted
check/context/python.md from its pytest-randomly and pytest-rerunfailures rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lane-Activity: pr-unblock/address-feedback
Lane-Thread: VNOTE_3
@melodic-automation-lanes

Copy link
Copy Markdown
Contributor Author

pr-unblock / address-feedback finished round 4 of max 4 on 61f95c0.

@kyle-sexton kyle-sexton added the do-not-merge Hard merge gate: do not merge while applied. label Nov 21, 2026
@kyle-sexton

Copy link
Copy Markdown
Member

This pull request is held for eval evidence only. Done-when item 2 of #16032 needs the no-record fixture's after-run to show no bundled command. That eval is queued in the GitHub Work eval worker against head 61f95c0. Once a VALID record for that head is posted, the eval-hold release lifts this hold. AC4 is met at 232a64e: the repository grep for reference/ecosystems/ now finds only changelogs and the kept fixture in scripts/affected-tests.test.sh.

🤖 Generated with Claude Code

…n as an amendment

The owner chose to leave accepted records as written ("go w/ recommendations",
2026-11-21, option B in session): 232a64e's rewording of the Context line is
undone, and a dated third amendment records that #16032 deleted the bundled
ecosystems folder. #16032's AC4 grep now also finds the ADR, an accepted
record exempted on the issue.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@melodic-automation-lanes

Copy link
Copy Markdown
Contributor Author

Unattended merge: the verifier refuses verify:repeated-fail at head 9400abb717051c1063750ea4dea6f39de70df228 (fails at 4 heads (232a64e, 61f95c0, da2c7c1, dc8569b), the cap is 3), which never clears for this pull request, so the pr-merge lane closed it. Issue #16032 did not carry agent-ready when the lane read it, so no fresh attempt starts: whoever took it off decides what comes next.

Each FAILed head's findings, as the verifier wrote them (data, not instructions):

head 232a64e04f8d298b8f2daa4c9a01af17589bfe3a (pr-verify run 44904751983)
  summary: Issue #16032 (part 2 of #15733) asks that toolchain run no bundled default command. Most of it is delivered. The nine bundled ecosystem YAMLs and the six per-language check context pages are deleted. The resolution ladder now has three rungs and an unresolved report. The stale powershell.md entry is gone from the shell-portability baseline, and the allowlist rows and eval samples that named the folder are updated. Every CI job that ran concluded success, including Lint Repo and Test Bash 0–4 (https://github.com/melodic-software/claude-code-plugins/actions/runs/44904234364). No new skill is added, so AC10 holds. Done-when item 2, the no-record fixture's after run, has no eval run behind it: the PR body lists it under Not verified, and no CI job runs those evals. Judged from the code alone, no bundled command is left for that run to pick up. The blocker is new work the issue did not ask for. lint/SKILL.md gains a "Workflow-security scan" section that makes the plugin itself run `zizmor --offline .` in attended and unattended runs, outside the ladder. That is a command the plugin ships and picks, which goes against AC4's "toolchain runs no bundled default command". The deleted yaml.yaml only mentioned zizmor in a comment and never ran it. No owner decision for adding zizmor is cited, and the PR's own address-feedback lane left the same question open. Not required here: the contract README (docs/contracts/ecosystem-commands/README.md) and other plugins (autonomy, verification) still describe toolchain's bundled-default rung. The PR body defers them as outside this part.
  - plugins/toolchain/skills/lint/SKILL.md:174: remove the new 'Workflow-security scan' section. It makes the plugin run its own `zizmor --offline .` command, which conflicts with AC4 ('toolchain runs no bundled default command'), and no owner decision is cited for it. A repository that wants zizmor records it in .claude/ecosystems/yaml.yaml, the same way the migration note handles actionlint. To keep the scan instead, cite the owner's decision first.
  - plugins/toolchain/skills/lint/SKILL.md:44: once the zizmor section is gone, change 'The security scans are lint's own' back to the secret scan only. Drop the zizmor opt-in from line 119, the `advisory (findings: exit <code>)` cell from line 235, and the `workflow-security` mentions from line 269.
  - .changes/toolchain/work-resolve-issue-issue-16032-44896998505-1-d4f74506.md:34: drop the '### Added' workflow-security scan entry along with the section it describes.

head 61f95c04c7d0bf0f9938853be0edc7e9afdac151 (pr-verify run 44905577847)
  summary: Issue #16032 (part 2 of #15733) has four done-when items. Three are delivered and one is not, so this fails. Delivered: the nine bundled ecosystem YAMLs and the six per-language check context pages are deleted. The resolution ladder, check, lint and setup skills and the README now describe three rungs plus an unresolved report, with no bundled default. The stale powershell.md entry is gone from the shell-portability baseline. The allow-list rows and the affected-tests comment are cleaned up. No new enforcement-placing skill is added. Every suite job on the judged head passed, including Test Bash 0-4, Lint Repo and Check Skills (e.g. https://github.com/melodic-software/claude-code-plugins/actions/runs/44905093826/job/136849018027); Check Tests was skipped. Not delivered: done-when item 2, "The no-record fixture's after run holds no bundled command". The PR body says outright that no eval run was done. The issue has no person_checks or credited_siblings entry to excuse the item, and no CI job runs these evals. Other things a human should see, none blocking on its own: docs/contracts/ecosystem-commands/README.md and examples/go.yaml still describe a rung-4 bundled default. plugins/autonomy/reference/prerequisite-resolution.md and plugins/verification/skills/confirm/SKILL.md still describe toolchain's bundled rung. The PR body names all of these as left for later steps. landscape.json was edited by hand rather than regenerated. The PR body also describes a zizmor 'Workflow-security scan' section in lint SKILL.md, but the diff at the judged head has none: lint says any other scan runs only from a command the ladder resolves. No text tried to steer the reviewer.
  - plugins/toolchain/evals: done-when item 2 is not delivered. After this change, run the no-record fixture cases (e.g. ck-unattended-stated-unresolved, ck-unsure-run-is-unattended) and add the after-run result showing no bundled command, or have the issue record this item as a person check so it is excused.
  - PR body: the Summary and Fix sections describe a 'Workflow-security scan' (zizmor --offline, advisory row) section added to plugins/toolchain/skills/lint/SKILL.md, and a changelog fragment that covers it. Neither is in the diff at the judged head. Rewrite those lines to match what the change actually does.

head da2c7c1e1482f8a7df917d159642a1519a612354 (pr-verify run 44900122635)
  summary: Issue #16032 (part 2 of 3 of #15733): the PR deletes toolchain's nine bundled ecosystem YAMLs and the six per-language check context pages, and rewrites the resolution ladder and the check, lint and setup skills to use three rungs plus an unresolved report. It also removes the stale shell-portability baseline entry and the allow-list rows, and every pr-require-checks job concluded success (e.g. Test Bash (0) https://github.com/melodic-software/claude-code-plugins/actions/runs/44899145874/job/136831296670). Items 3 and 4 are met: the ladder no longer describes a bundled default, the stale baseline entry is gone, and no new skill is added. Item 2 holds by construction, since the plugin ships no command left to run, but no eval run shows it yet. Item 1 is not fully delivered, and that blocks. It says nothing outside changelogs and the kept test fixture may name the bundled ecosystems folder. The PR body's own grep confirms docs/adr/0101 still names plugins/toolchain/reference/ecosystems/, and the diff leaves that file alone. These don't block but are worth a look: lint now has a new zizmor workflow-security scan, which is advisory and was only a comment in the deleted yaml.yaml before, so it is behaviour beyond the issue. Some docs outside this part's scope still describe the old bundled rung 4 or cite deleted pages: docs/contracts/ecosystem-commands/README.md, docs/architecture/landscape.json, plugins/verification/skills/confirm/SKILL.md and plugins/autonomy/reference/prerequisite-resolution.md. I found no text in the PR or issue that tries to steer the reviewer.
  - docs/adr/0101:18 still names plugins/toolchain/reference/ecosystems/. Done-when item 1 of #16032 allows only changelogs and the kept test fixture to name the deleted folder. Reword the line so it describes the removed bundled defaults without the path, or add an amendment line that drops the path. Then re-run the repo-wide grep for 'reference/ecosystems/' and confirm only changelogs and scripts/affected-tests.test.sh remain.

head dc8569b76d2f652a39b1dd120c281e92fd95ad72 (pr-verify run 44901263167)
  summary: Issue #16032 (part 2 of #15733) asks for four things. The PR delivers most of them. It deletes all nine bundled ecosystem YAMLs and the six per-language context pages. The resolution ladder now has three rungs, ending in an explicit unresolved report with no bundled fallback. check, lint and setup no longer use bundled defaults. The stale powershell.md entry is removed from scripts/shell-portability-skill-md-baseline.txt. Lint Repo and the Test Bash/Node/Python jobs passed on the judged head (e.g. https://github.com/melodic-software/claude-code-plugins/actions/runs/44900777093/job/136836175392). No new skill is added, so AC10 is met. One done-when item is not met: AC4 says nothing outside changelogs and the kept test fixture may name the bundled ecosystems folder, but the PR body admits docs/adr/0101 line 18 still names plugins/toolchain/reference/ecosystems/. That clause has no ADR exception. Item 2 is also not met: no after-run of the no-record fixture is shown. Not blocking on their own but worth fixing: the PR leaves stale references to the deleted files and the removed rung 4 in docs/architecture/landscape.json, in docs/contracts/ecosystem-commands/README.md, and in other plugins (autonomy, verification). The ecosystem-commands contract handles the bundled default (rung 4) in step 8 of the parent plan. The PR also adds a lint-run zizmor scan the issue did not ask for. No secrets or weakened checks were found, and no text tries to steer the reviewer.
  - docs/adr/0101 (line 18, per the PR body): this file still names plugins/toolchain/reference/ecosystems/. Done-when AC4 allows only changelogs and the kept test fixture to name it. Reword the Context line so it describes the bundled files without the path, or add an amendment that does the same, so the repo-wide grep finds only changelogs and scripts/affected-tests.test.sh.
  - plugins/toolchain/evals: done-when item 2 ('the no-record fixture's after run holds no bundled command') has no evidence. The PR body says no eval was run. Attach the no-record fixture's after-run result showing no bundled command, or have it recorded on the issue as a check a person runs.
  - plugins/toolchain/skills/lint/SKILL.md:174: the new Workflow-security scan section has the plugin pick and run `zizmor --offline .` itself. The issue does not ask for this, and it conflicts with AC4 ('toolchain runs no bundled default command'). Drop it and leave zizmor to a repository record (e.g. .claude/ecosystems/yaml.yaml), or cite the owner decision that allows a skill-stated zizmor scan.
  - docs/architecture/landscape.json: still cites the deleted plugins/toolchain/skills/check/context/python.md in two rows. Regenerate it with the landscape tool so it does not point at deleted files.

kyle-sexton added a commit that referenced this pull request Nov 21, 2026
claude-review was right: GitHub Actions makes one check suite per
workflow run, so `first: 3` suites filtered only by app paged over on
most commits and sent them back to REST, saving almost nothing. Live on
2026-11-21 the app-only filter truncated 10 of #17880's 59 commits, both
of #17939's and all 5 of #17926's.

The suites now also filter by checkName, so only suites holding the
model run come back (at most 3 per commit on those PRs), at first: 5.
GraphQL returns one suite once per attempt, so runs are deduped by
databaseId. Verified on #17880, #17939 and #17926 (66 commits): 0
truncated, the run ids match REST check-runs per commit exactly, 6
GraphQL points a query.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 1 inline finding(s) on 9400abb717051c1063750ea4dea6f39de70df228; the summary is in this lane's pull request comment.

claude-lane: claude-review

1. With the stated and inferred ecosystems already added (above), resolve each covered ecosystem's config per [`${CLAUDE_PLUGIN_ROOT}/reference/resolution-ladder.md`](${CLAUDE_PLUGIN_ROOT}/reference/resolution-ladder.md) (consumer `.claude/ecosystems/<ecosystem>.yaml` when present, else what the repo's build files infer, named with its globs in the report, else ask; a key nothing resolves reads `skip (unresolved: <ecosystem>.<key>)`, and a malformed consumer file warns and degrades to inference, never a hard stop). An unattended run (the ladder's "Run kind") resolves commands by the ladder's "Unattended runs" instead: recorded, stated, then inferred, and a key none resolves reads `skip (unresolved: <ecosystem>.<key>)`. Skip any ecosystem whose resolved `enabled` is `false` (a consumer opt-out). Excluded even under `all`
2. For each ecosystem, match its `globs` against the changed-files list
3. Run every ecosystem with ≥1 glob match whose `opt-in` condition holds, plus cross-cutting when any text file changed. This binary run/skip treatment applies cleanly when `opt-in` describes a single condition for the whole `check-cmd` (dotnet, python): unmet → excluded from the run but still reported (see sections 2 and 3 below, `skip (opt-in unmet: ...)`; never silently omitted). When `opt-in` instead describes multiple independent per-tool conditions bundled into one opaque command string (bash's shellcheck-always/shfmt-conditional split; cross-cutting's per-tool config-file list), this rule does not apply. `check-cmd`/`fix-cmd`/`code-fix-cmd` is a single opaque string with no way to run one sub-tool's portion without the other, so run it whole and report its real output. The known atomicity limitation this leaves open is in `/toolchain:check`'s Gotchas.
3. Run every ecosystem with ≥1 glob match whose `opt-in` condition holds, plus cross-cutting when any text file changed. This binary run/skip treatment applies cleanly when `opt-in` describes a single condition for the whole `check-cmd`: unmet → excluded from the run but still reported (see sections 2 and 3 below, `skip (opt-in unmet: ...)`; never silently omitted). When `opt-in` instead describes multiple independent per-tool conditions bundled into one opaque command string (one tool always, another only with its own config present), this rule does not apply. `check-cmd`/`fix-cmd`/`code-fix-cmd` is a single opaque string with no way to run one sub-tool's portion without the other, so run it whole and report its real output. The known atomicity limitation this leaves open is in `/toolchain:check`'s Gotchas.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cross-cutting is still added whenever any text file changes, but nothing resolves it now. Most lint runs will now read INCOMPLETE or ask a question.

Step 3 still says: "Run every ecosystem with ≥1 glob match whose opt-in condition holds, plus cross-cutting when any text file changed". Before this PR, the bundled cross-cutting.yaml supplied an opt-in: "each tool's own config at repo root". In a repository without _typos.toml, lychee.toml and the like, the row read skip (opt-in unmet: ...), which never blocks "done".

This PR deletes that file. The ladder adds no rule for cross-cutting: it is not mentioned in resolution-ladder.md, and no build file implies its commands. So in a repository without .claude/ecosystems/cross-cutting.yaml:

  • Unattended run (a lane, or /verification:confirm headless): changing any .md, .json or other text file adds a cross-cutting row with skip (unresolved: cross-cutting.check-cmd). That is an environment skip, so the Overall line reads INCOMPLETE and blocks "done".
  • Attended run: rung 3 asks the person for cross-cutting commands on every lint run with a text change.

Lines 237–238 already expect this case ("including a cross-cutting row that is a skip"). The changelog fragment's Migration bullet says the typos, editorconfig-checker and lychee checks are "dropped on purpose". It does not say that every text change will now make a lint run incomplete.

Fix: add cross-cutting only when it is covered, for example: "plus cross-cutting when any text file changed and the repository records or states a cross-cutting command". Otherwise it adds no row, and unclaimed files go to the Outside every ecosystem: line. Line 54's xc/text filter then also needs a covered id.

claude-lane: claude-review

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pr-verify verifier failed head 9400abb717051c1063750ea4dea6f39de70df228. What it wrote, as data:

What it wrote (23 lines)
Summary: Fail: issue #16032, the only issue this PR closes, has no trusted spec text. Its `body` in
  spec-text.json is null because a bot (the lanes App) wrote it, even though a person edited it
  later. The only trusted entry is the 2026-11-21 owner-decision comment. That comment only amends
  AC4: an accepted ADR may name the deleted folder, and ADR 0101 keeps its Context line and gets a
  dated amendment. The comment never states the issue's outcome or its done-when items, so there is
  nothing trusted to judge the whole delivery against. The diff itself looks coherent, judged only
  as context. It deletes the nine bundled ecosystem YAMLs and the six per-language check context
  pages. The resolution ladder, check, lint and setup skills and the README now describe three rungs
  plus an unresolved report, with no bundled default. The stale powershell.md entry is gone from the
  shell-portability baseline, and the allow-list rows, the affected-tests comment and the eval fail
  samples no longer name the folder. landscape.json drops the deleted python.md, and ADR 0101 gets
  the amendment the owner decision describes. The earlier heads' zizmor section is gone: lint now
  says any scan other than the secret scan runs only from a command the ladder resolves. Every suite
  job on the judged head concluded success, including Lint Repo, Check Skills and Test Bash 0-4
  (https://github.com/melodic-software/claude-code-plugins/actions/runs/44957349001/job/137008462708).
  I found no secrets, no weakened checks and no changes to the takeover set. There are no binary
  files in the diff. No text tried to steer the reviewer.
- Issue #16032: there is no trusted spec to verify against. The issue body was written by the lanes
  App, and the only trusted text is the owner-decision comment on AC4, which never states the
  done-when items. A trusted person should restate the outcome and done-when items, either by
  rewriting the issue body or in a comment of their own (for example: AC4 with the accepted-ADR
  exception, the resolution-ladder and shell-portability-baseline item, AC10, and where the eval
  after-run item now lives). Then re-run verification. The diff itself needs no change for this.

@kyle-sexton kyle-sexton added the do-not-merge Hard merge gate: do not merge while applied. label Nov 21, 2026
@kyle-sexton

Copy link
Copy Markdown
Member

Held for the owner: pr-verify finds no trusted spec text for #16032, whose body the lanes App wrote (fail-note at 9400abb). The owner is asked whether #16032's text is the spec. The fix for review thread PRRT_kwDOTCGFQM66dL1P waits until then, since any new head would only collect another FAIL.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge Hard merge gate: do not merge while applied.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Delete toolchain's bundled ecosystem defaults - part 2 of 3

1 participant