Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Comment thread
kyle-sexton marked this conversation as resolved.
Resolved

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pr-verify verifier failed head 9400abb717051c1063750ea4dea6f39de70df228. What it wrote, as data:

What it wrote (23 lines)
Summary: Fail: issue #16032, the only issue this PR closes, has no trusted spec text. Its `body` in
  spec-text.json is null because a bot (the lanes App) wrote it, even though a person edited it
  later. The only trusted entry is the 2026-11-21 owner-decision comment. That comment only amends
  AC4: an accepted ADR may name the deleted folder, and ADR 0101 keeps its Context line and gets a
  dated amendment. The comment never states the issue's outcome or its done-when items, so there is
  nothing trusted to judge the whole delivery against. The diff itself looks coherent, judged only
  as context. It deletes the nine bundled ecosystem YAMLs and the six per-language check context
  pages. The resolution ladder, check, lint and setup skills and the README now describe three rungs
  plus an unresolved report, with no bundled default. The stale powershell.md entry is gone from the
  shell-portability baseline, and the allow-list rows, the affected-tests comment and the eval fail
  samples no longer name the folder. landscape.json drops the deleted python.md, and ADR 0101 gets
  the amendment the owner decision describes. The earlier heads' zizmor section is gone: lint now
  says any scan other than the secret scan runs only from a command the ladder resolves. Every suite
  job on the judged head concluded success, including Lint Repo, Check Skills and Test Bash 0-4
  (https://github.com/melodic-software/claude-code-plugins/actions/runs/44957349001/job/137008462708).
  I found no secrets, no weakened checks and no changes to the takeover set. There are no binary
  files in the diff. No text tried to steer the reviewer.
- Issue #16032: there is no trusted spec to verify against. The issue body was written by the lanes
  App, and the only trusted text is the owner-decision comment on AC4, which never states the
  done-when items. A trusted person should restate the outcome and done-when items, either by
  rewriting the issue body or in a comment of their own (for example: AC4 with the accepted-ADR
  exception, the resolution-ladder and shell-portability-baseline item, AC10, and where the eval
  after-run item now lives). Then re-run verification. The diff itself needs no change for this.

Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
bump: major
---

### Removed

- **No bundled ecosystem defaults.** The nine bundled ecosystem files (`bash`, `cross-cutting`,
`dotnet`, `go`, `markdown`, `powershell`, `python`, `typescript`, `yaml`) are deleted, and so are
`/toolchain:check`'s six per-language context pages. `/toolchain:check` and `/toolchain:lint` run
only a command the repository records in `.claude/ecosystems/<id>.yaml`, one its build files
imply, or, in an attended run, one the person gives when asked. A key nothing resolves reads
`skip (unresolved: <ecosystem>.<key>)` and runs nothing. The resolution ladder has three rungs.
`/toolchain:setup apply` drafts records from the repository alone.
([#16032](https://github.com/melodic-software/claude-code-plugins/issues/16032))
- **Migration.** A repository with no ecosystem record used to get the bundled commands in an
attended run. It now gets inferred commands, a question, or an unresolved skip. To keep a
command, record it with `/toolchain:setup apply <ecosystem>`. Checks only the bundled files ran
are dropped on purpose: the spelling check (`typos`), the editorconfig check
(`editorconfig-checker`) and the offline link check (`lychee --offline`) from the cross-cutting
file, and the workflow lint (`actionlint`) from the yaml file. Record them in
`.claude/ecosystems/cross-cutting.yaml` or `.claude/ecosystems/yaml.yaml` to keep them. The secret
scan stays: `/toolchain:lint` runs it from the skill.
- **Coverage.** With no bundled file to supply globs, an ecosystem id is covered when it is
recorded, stated in the repository's instructions, or inferred from its build files. In a
repository with no ecosystem record, a changed file's own type also infers its id, so it reads
`skip (unresolved: ...)` rather than going unchecked in silence.

### Changed

- `/toolchain:check` keeps its retry, seed and slow-test reporting rules in the skill, written for
any test runner. The description now says what the skill runs: the project's own commands only.
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@

- Status: accepted; amended 2026-11-16 (source citations exclude one ecosystem's tool documentation
and examples, see the amendment below); amended 2026-11-17 (two kept classes: operating a kept
adapter's tool, and a measured fact; see the second amendment)
adapter's tool, and a measured fact; see the second amendment); amended 2026-11-21 (the bundled
ecosystems folder is deleted; see the third amendment)
- Date: 2026-11-13
- Refs: [#15703](https://github.com/melodic-software/claude-code-plugins/issues/15703) (spec),
[#15706](https://github.com/melodic-software/claude-code-plugins/issues/15706); owner decision of
Expand Down Expand Up @@ -128,3 +129,9 @@ wrong thing, and only while a targeted eval case shows the loss without it. Neit
default: no bundled default survives decision 2. A record of this marketplace's own files, keys,
rule ids, scripts or behavior, and the upstream source of a kept script's patterns, is not a
choice made for the consumer and is outside these classes.

## Amendment (2026-11-21): the bundled ecosystems folder is deleted

[#16032](https://github.com/melodic-software/claude-code-plugins/issues/16032) deleted the nine
default files under `plugins/toolchain/reference/ecosystems/`, and the Context section above still
describes `main` as it stood when this record was accepted.
4 changes: 2 additions & 2 deletions docs/architecture/landscape.json
Original file line number Diff line number Diff line change
Expand Up @@ -190,8 +190,8 @@
{"from":"claude-code-plugins","to":"primer/primitives","type":"cites","relation":"external","count":1,"files":["docs/upstream/user-interface-default-sources.md"]},
{"from":"claude-code-plugins","to":"priv-kweihmann/multimetric","type":"cites","relation":"external","count":2,"files":["plugins/code-metrics/reference/collectors.md","plugins/code-metrics/scripts/collectors/multimetric.py"]},
{"from":"claude-code-plugins","to":"pulumi/pulumi","type":"cites","relation":"external","count":1,"files":["docs/conventions/pr-pipeline/README.md"]},
{"from":"claude-code-plugins","to":"pytest-dev/pytest-randomly","type":"cites","relation":"external","count":4,"files":["plugins/debugging/skills/debug/reference/ecosystem-debugging.md","plugins/testing/skills/diagnose/context/investigate.md","plugins/testing/skills/write/context/write.md","plugins/toolchain/skills/check/context/python.md"]},
{"from":"claude-code-plugins","to":"pytest-dev/pytest-rerunfailures","type":"cites","relation":"external","count":3,"files":["plugins/testing/skills/audit/SKILL.md","plugins/testing/skills/audit/scripts/retry-config-scan.awk","plugins/toolchain/skills/check/context/python.md"]},
{"from":"claude-code-plugins","to":"pytest-dev/pytest-randomly","type":"cites","relation":"external","count":3,"files":["plugins/debugging/skills/debug/reference/ecosystem-debugging.md","plugins/testing/skills/diagnose/context/investigate.md","plugins/testing/skills/write/context/write.md"]},
{"from":"claude-code-plugins","to":"pytest-dev/pytest-rerunfailures","type":"cites","relation":"external","count":2,"files":["plugins/testing/skills/audit/SKILL.md","plugins/testing/skills/audit/scripts/retry-config-scan.awk"]},
{"from":"claude-code-plugins","to":"python/cpython","type":"cites","relation":"external","count":1,"files":["plugins/source-control/skills/babysit-loop/reference/promotion-evidence-bootstrap.md"]},
{"from":"claude-code-plugins","to":"python/tzdata","type":"cites","relation":"external","count":1,"files":["plugins/session-flow/skills/keep-going/scripts/vendor/README.md"]},
{"from":"claude-code-plugins","to":"radix-ui/themes","type":"cites","relation":"external","count":2,"files":["docs/upstream/user-interface-default-sources.md"]},
Expand Down
18 changes: 9 additions & 9 deletions plugins/toolchain/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

A Claude Code plugin for **polyglot build/test/lint verification**. Detect the
ecosystems a change touches and run the right build, test, and lint commands for
each, with the consuming project's own documented commands overriding portable
defaults. Four skills: three for mechanical verification of changed code, and
each, running only the commands the consuming project records or its build files
imply. Four skills: three for mechanical verification of changed code, and
`/toolchain:ratchet`, which holds a count at a checked-in CI ceiling.

| Skill | Role |
Expand All @@ -22,13 +22,13 @@ restatement here.
Command resolution follows the marketplace-wide ecosystem-commands convention
(`docs/contracts/ecosystem-commands/README.md`; schema `ecosystem.schema.json`):
your repo's tracked `.claude/ecosystems/<ecosystem>.yaml` files are authoritative
when present, and the plugin's bundled portable defaults are the last resort of
an attended run. An unattended run (`--unattended`, a headless or lane run, or
one that cannot tell whether a person is present) runs only a recorded, stated
or inferred command and reports any other key as `skip (unresolved: ...)`.
Testing structure and commit conventions still come from your own `CLAUDE.md` and
rules. All command tables and per-ecosystem context ship inside the plugin via
`${CLAUDE_PLUGIN_ROOT}`; state and artifacts go to your project's own tree.
when present; otherwise an attended run infers commands from your build files or
asks. The plugin ships no default command. An unattended run (`--unattended`, a
headless or lane run, or one that cannot tell whether a person is present) runs
only a recorded, stated or inferred command. Either kind of run reports a key
nothing resolves as `skip (unresolved: ...)`. Testing structure and commit
conventions still come from your own `CLAUDE.md` and rules. State and artifacts
go to your project's own tree.

## Install

Expand Down
2 changes: 1 addition & 1 deletion plugins/toolchain/evals/ck-consumer-config-wins/case.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@ graders:
arm: both
- name: meets-expectations
type: llm
criteria: 'The run checks a .NET repository whose tracked .claude/ecosystems/dotnet.yaml declares its own commands. It meets the legacy expectations: "Resolves commands via the ladder with the consumer''s `.claude/ecosystems/dotnet.yaml` taking precedence (rung 1)" "Uses the bundled portable default only as the rung-4 fallback when no consumer file is present" "Runs the consumer''s declared test-cmd, not the bundled default''s".'
criteria: 'The run checks a .NET repository whose tracked .claude/ecosystems/dotnet.yaml declares its own commands. It meets the legacy expectations: "Resolves commands via the ladder with the consumer''s `.claude/ecosystems/dotnet.yaml` taking precedence (rung 1)" "Runs the consumer''s declared test-cmd, not a plain `dotnet test` or `dotnet build` of its own".'
2 changes: 1 addition & 1 deletion plugins/toolchain/evals/ck-consumer-config-wins/prompt.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
description: "Legacy check 3. The tracked .claude/ecosystems/dotnet.yaml is rung 1 and authoritative: its project-specific test-cmd runs (it logs `dotnet test` to .calls.log through tools/run.sh), and the bundled `dotnet test` / `dotnet build` never run. ran-consumer-test reads .calls.log, no-bundled-dotnet the Bash calls; meets-expectations (llm) quotes the legacy expectations. Needs --scaffold --allow-tools Bash."
description: "Legacy check 3. The tracked .claude/ecosystems/dotnet.yaml is rung 1 and authoritative: its project-specific test-cmd runs (it logs `dotnet test` to .calls.log through tools/run.sh), and a plain `dotnet test` / `dotnet build` never runs. ran-consumer-test reads .calls.log, no-bundled-dotnet the Bash calls; meets-expectations (llm) quotes the legacy expectations. Needs --scaffold --allow-tools Bash."
tags: [toolchain, check, resolution-ladder, scaffold, no-network]
max_turns: 30
timeout_seconds: 600
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
],
"fail": [
{"answer": "assumed: typescript.test-cmd=npm test", "why": "no source named"},
{"answer": "assumed: typescript.test-cmd=npx vitest run (reference/ecosystems/typescript.yaml:13)", "why": "the bundled default, not the inferred script"},
{"answer": "assumed: typescript.test-cmd=npx vitest run (vitest.config.ts:1)", "why": "a runner chosen for the repo, not the inferred script"},
{"answer": "Inferred typescript: globs *.js, package.json", "why": "no assumed: line"}
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,6 @@
"fail": [
{"answer": "assumed: bash.check-cmd=bash tools/run.sh bash check <files>", "why": "no source named"},
{"answer": "Ran bash tools/run.sh bash check scripts/app.sh, stated in CLAUDE.md.", "why": "not an assumed: line"},
{"answer": "assumed: bash.check-cmd=shellcheck <files> (reference/ecosystems/bash.yaml:7)", "why": "sourced from the bundled default"}
{"answer": "assumed: bash.check-cmd=shellcheck <files> (.github/workflows/ci.yml:7)", "why": "copied from a CI workflow file, not stated"}
]
}
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"pass": [{"answer": [{"tool": "Bash", "input": {"command": "ls .claude/ecosystems/ 2>/dev/null"}}]}, {"answer": [{"tool": "Bash", "input": {"command": "cat /tmp/w/.claude/ecosystems/bash.yaml 2>&1"}}], "why": "a read, not a write"}], "fail": [{"answer": [{"tool": "Bash", "input": {"command": "printf 'globs: [\"*.sh\"]\\n' > .claude/ecosystems/bash.yaml"}}]}, {"answer": [{"tool": "Bash", "input": {"command": "echo x | tee -a /w/.claude/ecosystems/markdown.yaml"}}]}, {"answer": [{"tool": "Bash", "input": {"command": "cp plugin/reference/ecosystems/markdown.yaml .claude/ecosystems/markdown.yaml"}}]}]}
{"pass": [{"answer": [{"tool": "Bash", "input": {"command": "ls .claude/ecosystems/ 2>/dev/null"}}]}, {"answer": [{"tool": "Bash", "input": {"command": "cat /tmp/w/.claude/ecosystems/bash.yaml 2>&1"}}], "why": "a read, not a write"}], "fail": [{"answer": [{"tool": "Bash", "input": {"command": "printf 'globs: [\"*.sh\"]\\n' > .claude/ecosystems/bash.yaml"}}]}, {"answer": [{"tool": "Bash", "input": {"command": "echo x | tee -a /w/.claude/ecosystems/markdown.yaml"}}]}, {"answer": [{"tool": "Bash", "input": {"command": "cp /tmp/markdown.yaml .claude/ecosystems/markdown.yaml"}}]}]}
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"pass": [{"answer": [{"tool": "Bash", "input": {"command": "ls .claude/ecosystems/ 2>/dev/null"}}]}], "fail": [{"answer": [{"tool": "Bash", "input": {"command": "printf 'globs: [\"*.md\"]\\n' >> .claude/ecosystems/markdown.yaml"}}]}, {"answer": [{"tool": "Bash", "input": {"command": "cp plugin/reference/ecosystems/markdown.yaml .claude/ecosystems/"}}]}]}
{"pass": [{"answer": [{"tool": "Bash", "input": {"command": "ls .claude/ecosystems/ 2>/dev/null"}}]}], "fail": [{"answer": [{"tool": "Bash", "input": {"command": "printf 'globs: [\"*.md\"]\\n' >> .claude/ecosystems/markdown.yaml"}}]}, {"answer": [{"tool": "Bash", "input": {"command": "cp /tmp/markdown.yaml .claude/ecosystems/"}}]}]}
10 changes: 0 additions & 10 deletions plugins/toolchain/reference/ecosystems/bash.yaml

This file was deleted.

24 changes: 0 additions & 24 deletions plugins/toolchain/reference/ecosystems/cross-cutting.yaml

This file was deleted.

35 changes: 0 additions & 35 deletions plugins/toolchain/reference/ecosystems/dotnet.yaml

This file was deleted.

Loading
Loading