Skip to content

Delete toolchain's bundled ecosystem defaults - part 2 of 3 #16032

Description

@melodic-automation-lanes

Part 2 of 3 of #15733, split at intake (owner Q93). The done-when below is copied from #15733 as written.

Done when

  • AC4 (first clause): toolchain runs no bundled default command; the bundled ecosystems folder is gone and nothing outside changelogs and the kept test fixture names it.

Moved to #18018 while evals are suspended (owner 2026-11-21 ~19:1xZ, D491): the no-record fixture's after run holding no bundled command is an eval result, so it is checked when evals return, not here.

  • The resolution-ladder reference no longer describes a bundled portable default; the shell-portability baseline passes its own stale-entry check.
  • AC10: no new enforcement-placing skill is added here.

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 13, 2026
  2. github-actions commented on Nov 13, 2026

    @github-actions
    Contributor

    Size: Medium
    Complexity: Medium
    Uncertainty: Known
    Flags: none
    Blocked by: none
    Blast radius: the toolchain plugin (bundled ecosystems folder, resolution-ladder reference, check/lint/setup skills), the shell-portability baseline, and the repo-wide grep for the deleted folder path; anyone running toolchain skills.
    Reversible: yes; revert the deletion commit.
    Open questions: none
    Understood: delete toolchain's bundled ecosystem defaults so no bundled command runs, update the ladder reference and baseline, and add no new enforcement skill. The owner note on this issue (option B: an accepted ADR may still name the deleted folder, with ADR 0101 amended) is not a **Decision recorded.** comment, so it is not treated as a recorded decision here. It also needs no decision, because the issue text and the repository files already settle the scope. The earlier attempt, PR #17926, was closed unmerged. Its verifier findings were a zizmor scan added to lint that the issue did not ask for, and no after-run evidence for the no-record fixture item. The after-run item is an eval run, so admission routes it.

    Automated triage. A maintainer reviews it.

  3. removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 13, 2026
  4. melodic-automation-lanes commented on Nov 13, 2026

    @melodic-automation-lanes
    ContributorAuthor

    Not admitted to the issue lane yet: blocker #16031 is not closed by a listed human or a merge.

  5. melodic-automation-lanes commented on Nov 21, 2026

    @melodic-automation-lanes
    ContributorAuthor

    {"v":1,"actor_id":337595936,"title_sha256":"5a9e463db54326f267704ecd3de2d6cc75e9aaa6679b6c5e59435fcae5e0138d","body_sha256":"dfcbf18964d6a56ef30c6e314dcbf2a0a7480c17268eb0cdee48933c95d949e2"}

  6. github-actions commented on Nov 21, 2026

    @github-actions
    Contributor
    No description provided.
  7. kyle-sexton commented on Nov 21, 2026

    @kyle-sexton
    Member

    Owner decision, 2026-11-21, in the GitHub PRs lane session, verbatim: "go w/ recommendations". That chose option B: accepted ADRs stay as written. AC4's grep therefore also allows an accepted ADR to name plugins/toolchain/reference/ecosystems/. ADR 0101 keeps its Context line and records the deletion in a dated amendment (#17926, 9400abb).

    🤖 Generated with Claude Code

  8. melodic-automation-lanes commented on Nov 21, 2026

    @melodic-automation-lanes
    ContributorAuthor

    Pull request #17926: Unattended merge: the verifier refuses verify:repeated-fail at head 9400abb717051c1063750ea4dea6f39de70df228 (fails at 4 heads (232a64e, 61f95c0, da2c7c1, dc8569b), the cap is 3), which never clears for this pull request, so the pr-merge lane closed it. Issue #16032 did not carry agent-ready when the lane read it, so no fresh attempt starts: whoever took it off decides what comes next.

    Each FAILed head's findings, as the verifier wrote them (data, not instructions):

    head 232a64e04f8d298b8f2daa4c9a01af17589bfe3a (pr-verify run 44904751983)
      summary: Issue #16032 (part 2 of #15733) asks that toolchain run no bundled default command. Most of it is delivered. The nine bundled ecosystem YAMLs and the six per-language check context pages are deleted. The resolution ladder now has three rungs and an unresolved report. The stale powershell.md entry is gone from the shell-portability baseline, and the allowlist rows and eval samples that named the folder are updated. Every CI job that ran concluded success, including Lint Repo and Test Bash 0–4 (https://github.com/melodic-software/claude-code-plugins/actions/runs/44904234364). No new skill is added, so AC10 holds. Done-when item 2, the no-record fixture's after run, has no eval run behind it: the PR body lists it under Not verified, and no CI job runs those evals. Judged from the code alone, no bundled command is left for that run to pick up. The blocker is new work the issue did not ask for. lint/SKILL.md gains a "Workflow-security scan" section that makes the plugin itself run `zizmor --offline .` in attended and unattended runs, outside the ladder. That is a command the plugin ships and picks, which goes against AC4's "toolchain runs no bundled default command". The deleted yaml.yaml only mentioned zizmor in a comment and never ran it. No owner decision for adding zizmor is cited, and the PR's own address-feedback lane left the same question open. Not required here: the contract README (docs/contracts/ecosystem-commands/README.md) and other plugins (autonomy, verification) still describe toolchain's bundled-default rung. The PR body defers them as outside this part.
      - plugins/toolchain/skills/lint/SKILL.md:174: remove the new 'Workflow-security scan' section. It makes the plugin run its own `zizmor --offline .` command, which conflicts with AC4 ('toolchain runs no bundled default command'), and no owner decision is cited for it. A repository that wants zizmor records it in .claude/ecosystems/yaml.yaml, the same way the migration note handles actionlint. To keep the scan instead, cite the owner's decision first.
      - plugins/toolchain/skills/lint/SKILL.md:44: once the zizmor section is gone, change 'The security scans are lint's own' back to the secret scan only. Drop the zizmor opt-in from line 119, the `advisory (findings: exit <code>)` cell from line 235, and the `workflow-security` mentions from line 269.
      - .changes/toolchain/work-resolve-issue-issue-16032-44896998505-1-d4f74506.md:34: drop the '### Added' workflow-security scan entry along with the section it describes.
    
    head 61f95c04c7d0bf0f9938853be0edc7e9afdac151 (pr-verify run 44905577847)
      summary: Issue #16032 (part 2 of #15733) has four done-when items. Three are delivered and one is not, so this fails. Delivered: the nine bundled ecosystem YAMLs and the six per-language check context pages are deleted. The resolution ladder, check, lint and setup skills and the README now describe three rungs plus an unresolved report, with no bundled default. The stale powershell.md entry is gone from the shell-portability baseline. The allow-list rows and the affected-tests comment are cleaned up. No new enforcement-placing skill is added. Every suite job on the judged head passed, including Test Bash 0-4, Lint Repo and Check Skills (e.g. https://github.com/melodic-software/claude-code-plugins/actions/runs/44905093826/job/136849018027); Check Tests was skipped. Not delivered: done-when item 2, "The no-record fixture's after run holds no bundled command". The PR body says outright that no eval run was done. The issue has no person_checks or credited_siblings entry to excuse the item, and no CI job runs these evals. Other things a human should see, none blocking on its own: docs/contracts/ecosystem-commands/README.md and examples/go.yaml still describe a rung-4 bundled default. plugins/autonomy/reference/prerequisite-resolution.md and plugins/verification/skills/confirm/SKILL.md still describe toolchain's bundled rung. The PR body names all of these as left for later steps. landscape.json was edited by hand rather than regenerated. The PR body also describes a zizmor 'Workflow-security scan' section in lint SKILL.md, but the diff at the judged head has none: lint says any other scan runs only from a command the ladder resolves. No text tried to steer the reviewer.
      - plugins/toolchain/evals: done-when item 2 is not delivered. After this change, run the no-record fixture cases (e.g. ck-unattended-stated-unresolved, ck-unsure-run-is-unattended) and add the after-run result showing no bundled command, or have the issue record this item as a person check so it is excused.
      - PR body: the Summary and Fix sections describe a 'Workflow-security scan' (zizmor --offline, advisory row) section added to plugins/toolchain/skills/lint/SKILL.md, and a changelog fragment that covers it. Neither is in the diff at the judged head. Rewrite those lines to match what the change actually does.
    
    head da2c7c1e1482f8a7df917d159642a1519a612354 (pr-verify run 44900122635)
      summary: Issue #16032 (part 2 of 3 of #15733): the PR deletes toolchain's nine bundled ecosystem YAMLs and the six per-language check context pages, and rewrites the resolution ladder and the check, lint and setup skills to use three rungs plus an unresolved report. It also removes the stale shell-portability baseline entry and the allow-list rows, and every pr-require-checks job concluded success (e.g. Test Bash (0) https://github.com/melodic-software/claude-code-plugins/actions/runs/44899145874/job/136831296670). Items 3 and 4 are met: the ladder no longer describes a bundled default, the stale baseline entry is gone, and no new skill is added. Item 2 holds by construction, since the plugin ships no command left to run, but no eval run shows it yet. Item 1 is not fully delivered, and that blocks. It says nothing outside changelogs and the kept test fixture may name the bundled ecosystems folder. The PR body's own grep confirms docs/adr/0101 still names plugins/toolchain/reference/ecosystems/, and the diff leaves that file alone. These don't block but are worth a look: lint now has a new zizmor workflow-security scan, which is advisory and was only a comment in the deleted yaml.yaml before, so it is behaviour beyond the issue. Some docs outside this part's scope still describe the old bundled rung 4 or cite deleted pages: docs/contracts/ecosystem-commands/README.md, docs/architecture/landscape.json, plugins/verification/skills/confirm/SKILL.md and plugins/autonomy/reference/prerequisite-resolution.md. I found no text in the PR or issue that tries to steer the reviewer.
      - docs/adr/0101:18 still names plugins/toolchain/reference/ecosystems/. Done-when item 1 of #16032 allows only changelogs and the kept test fixture to name the deleted folder. Reword the line so it describes the removed bundled defaults without the path, or add an amendment line that drops the path. Then re-run the repo-wide grep for 'reference/ecosystems/' and confirm only changelogs and scripts/affected-tests.test.sh remain.
    
    head dc8569b76d2f652a39b1dd120c281e92fd95ad72 (pr-verify run 44901263167)
      summary: Issue #16032 (part 2 of #15733) asks for four things. The PR delivers most of them. It deletes all nine bundled ecosystem YAMLs and the six per-language context pages. The resolution ladder now has three rungs, ending in an explicit unresolved report with no bundled fallback. check, lint and setup no longer use bundled defaults. The stale powershell.md entry is removed from scripts/shell-portability-skill-md-baseline.txt. Lint Repo and the Test Bash/Node/Python jobs passed on the judged head (e.g. https://github.com/melodic-software/claude-code-plugins/actions/runs/44900777093/job/136836175392). No new skill is added, so AC10 is met. One done-when item is not met: AC4 says nothing outside changelogs and the kept test fixture may name the bundled ecosystems folder, but the PR body admits docs/adr/0101 line 18 still names plugins/toolchain/reference/ecosystems/. That clause has no ADR exception. Item 2 is also not met: no after-run of the no-record fixture is shown. Not blocking on their own but worth fixing: the PR leaves stale references to the deleted files and the removed rung 4 in docs/architecture/landscape.json, in docs/contracts/ecosystem-commands/README.md, and in other plugins (autonomy, verification). The ecosystem-commands contract handles the bundled default (rung 4) in step 8 of the parent plan. The PR also adds a lint-run zizmor scan the issue did not ask for. No secrets or weakened checks were found, and no text tries to steer the reviewer.
      - docs/adr/0101 (line 18, per the PR body): this file still names plugins/toolchain/reference/ecosystems/. Done-when AC4 allows only changelogs and the kept test fixture to name it. Reword the Context line so it describes the bundled files without the path, or add an amendment that does the same, so the repo-wide grep finds only changelogs and scripts/affected-tests.test.sh.
      - plugins/toolchain/evals: done-when item 2 ('the no-record fixture's after run holds no bundled command') has no evidence. The PR body says no eval was run. Attach the no-record fixture's after-run result showing no bundled command, or have it recorded on the issue as a check a person runs.
      - plugins/toolchain/skills/lint/SKILL.md:174: the new Workflow-security scan section has the plugin pick and run `zizmor --offline .` itself. The issue does not ask for this, and it conflicts with AC4 ('toolchain runs no bundled default command'). Drop it and leave zizmor to a repository record (e.g. .claude/ecosystems/yaml.yaml), or cite the owner decision that allows a skill-stated zizmor scan.
      - docs/architecture/landscape.json: still cites the deleted plugins/toolchain/skills/check/context/python.md in two rows. Regenerate it with the landscape tool so it does not point at deleted files.
    
  9. melodic-automation-lanes commented on Nov 21, 2026

    @melodic-automation-lanes
    ContributorAuthor

    The lanes App's pull request #17926 for this issue was closed without merging. work-unblock removed status: ready and agent-ready, so the hourly triage sweep sends it back to intake triage, which decides what work is left and admits it again.

  10. github-actions commented on Nov 21, 2026

    @github-actions
    Contributor

    The triage sweep dispatched intake triage for this issue (dispatch 1; 2 runs that reach intake, or 6 dispatches, hand it to a maintainer).

  11. melodic-automation-lanes commented on Nov 21, 2026

    @melodic-automation-lanes
    ContributorAuthor

    {"v":1,"actor_id":337595936,"title_sha256":"5a9e463db54326f267704ecd3de2d6cc75e9aaa6679b6c5e59435fcae5e0138d","body_sha256":"dfcbf18964d6a56ef30c6e314dcbf2a0a7480c17268eb0cdee48933c95d949e2"}

  12. melodic-automation-lanes commented on Nov 21, 2026

    @melodic-automation-lanes
    ContributorAuthor

    This records a re-fire of an earlier stop, not a new one. The start gate stopped run 44946258537, the work-resolve-issue run that agent-ready started, before any model turn, because the GITHUB_TOKEN budget was under its start-gate share, or the window was projected to have under its start-project share of the header's limit left by the run's expected end (github-token start-gate, start-project and start-run-minutes in rate-limits.json; the run's start-gate notice names which); that run ended at 2026-11-21T20:28:10Z. work-unblock has now removed and added agent-ready back to start a fresh run. A budget stop is the lanes' load, not a fault in this issue: it never counts toward limits.limit-stops-per-issue or the kill-switch wait.

  13. github-actions commented on Nov 21, 2026

    @github-actions
    Contributor
    No description provided.
  14. melodic-automation-lanes commented on Nov 21, 2026

    @melodic-automation-lanes
    ContributorAuthor

    The work-resolve-issue lane stopped partway through this issue and needs a person.

    Reason: security

    Matched security word: secret. A match holds the stop whether or not the text names a risk, so read the word in context below.

    Why, in the run's own words:

    host-only: done-when item 2 ("The no-record fixture's after run holds no bundled command") needs the toolchain no-record eval cases run after this change (for example ck-unattended-stated-unresolved and ck-unsure-run-is-unattended, plus the ln-secret-scan-no-record floor case), and the parent plan puts eval batches in the one local eval-runner session in the main checkout. This lane cannot run evals, and the pr-merge verifier closed #17926 after four heads failed on exactly this missing after-run. The rest of the issue is done in the working tree: a host run can resume from this branch, run those cases, and attach the after-run result.

    What the run says it did not do:

    • Done-when item 2: no after-run of the no-record fixture eval cases; this lane cannot run evals (host-only)
    • Tidy not run: the change touches about 30 files and over 600 lines, past the 10-file and 300-line cutoffs, and includes scripts/, a listed shared path
    • CI-only checks (evals-quality, eval-cases, org-repo-tokens, purged-em-dashes, portability-lint, lychee, typos and others check-affected lists) left to the PR's CI
    • check-evals-quality wrapper not applicable: toolchain evals are case.yaml directories, not an evals.json file
    • docs/architecture/landscape.json still lists the deleted check/context/python.md in two rows; regenerate with /architecture:map-landscape
    • docs/contracts/ecosystem-commands and the autonomy and verification plugins still describe toolchain's bundled rung; outside this part (parent plan step 8)

    Partial work is on the branch work-resolve-issue/issue-16032-44949350380-1. A retry continues from it while it is not behind the default branch, and otherwise starts fresh from the default branch; delete the branch first to always start fresh.

    Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/44949350380

    To retry, set Work status to agent-ready again (clearing the needs-human hold).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    Work status

    needs-human

    Priority

    medium

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions