From 6db8556a400967123562cc3624815ae06045d698 Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Mon, 3 Aug 2026 16:31:07 +1000 Subject: [PATCH 1/2] Docs: the layout trees, the tag facts, and two pointers the release procedure left behind MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #28 rewrote how releases work without updating either picture of what a plugin contains. Both layout trees — README's and developing-skills' — still showed a plugin as manifest + skills + scripts + references, with no CHANGELOG.md, while CI now fails any PR that omits one. They are fixed together; leaving one right and one wrong is worse than both being stale, because a contributor cannot tell which to trust. `scripts/validate.py` in both trees becomes `scripts/`, since it is no longer the only thing in there. "Only SKILL.md is required" was true when written and is not now — the manifest and the changelog are both required — so it now says so rather than being quietly contradicted by the CI guard three sections down. § Tags is rewritten because creating the tags falsified its opening sentence. It now documents the scheme as adopted, and records why only current versions were tagged: several historical versions cannot be tagged honestly, since some name two different published trees — content shipped twice under one version before the guard existed — and others sit at commits that fail today's validation. The changelogs carry that ambiguity in prose, with PR numbers. Two stale pointers: developing-skills said issue #4 tracks the ${CLAUDE_PLUGIN_ROOT} pattern, which shipped and now has a worked example; and "which is the existing precedent" claimed a consistency the history does not have — adding qe did not move the catalogue version, adding audit did. The rule stands, so it is stated rather than inferred. Also: the audit tutorial is added to README's documentation table, where it was the only guide missing while being in AGENTS.md's doc map; and its Step 7 line gave one hardcoded filename as if it were the instruction, which now reads as the convention with run 1's file as the example. --- README.md | 6 +++++- docs/developing-skills.md | 19 ++++++++++++++----- docs/tutorial-run-an-audit.md | 2 +- 3 files changed, 20 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index bf4bf16..7883d69 100644 --- a/README.md +++ b/README.md @@ -26,6 +26,7 @@ Each plugin bundles one area of work — a skill (the instructions Claude follow | [docs/using-skills.md](docs/using-skills.md) | Authors/reviewers: setup, invoking skills, what to expect | | [docs/developing-skills.md](docs/developing-skills.md) | Contributors: layout, conventions, dev loop, testing locally, versioning and releases, PR flow | | [docs/tutorial-run-an-evaluation.md](docs/tutorial-run-an-evaluation.md) | Tutorial: the evaluation procedure by hand, with the ge_arrow validation run as the checkable example | +| [docs/tutorial-run-an-audit.md](docs/tutorial-run-an-audit.md) | Tutorial: a whole-tracker audit end to end, and how to review what it produces | | [benchmark/README.md](benchmark/README.md) | The evaluation skill: review mode, triage mode, report format, manual pipeline | | [audit/README.md](audit/README.md) | The audit family: what belongs in it, the shared method, running one | @@ -78,9 +79,10 @@ The official action accepts the marketplace and plugin directly: ``` .claude-plugin/marketplace.json # the marketplace catalogue -scripts/validate.py # manifest + frontmatter validation (run in CI) +scripts/ # manifest + frontmatter validation, version-bump guard (run in CI) qe/ # author-facing plugin .claude-plugin/plugin.json + CHANGELOG.md # what shipped in each version skills/check-style/SKILL.md # umbrella skill skills/check-/ # thin per-category sub-skills skills/copilot-review/SKILL.md # PR review-feedback workflow @@ -88,10 +90,12 @@ qe/ # author-facing plugin scripts/ # deterministic preflight checkers + fetch-copilot.sh benchmark/ # specialist plugin .claude-plugin/plugin.json + CHANGELOG.md skills/review-acceleration/SKILL.md scripts/ # supporting Python scripts the skill drives audit/ # maintainer-facing bulk-audit plugin .claude-plugin/plugin.json + CHANGELOG.md skills/issues/SKILL.md # one skill per audit subject references/ # shared doctrine, org context, reporting guidance scripts/ # deterministic snapshot + coverage machinery diff --git a/docs/developing-skills.md b/docs/developing-skills.md index e2365ee..f41bb92 100644 --- a/docs/developing-skills.md +++ b/docs/developing-skills.md @@ -6,17 +6,18 @@ For contributors adding or modifying plugins in this repo. (Using them: [using-s ``` .claude-plugin/marketplace.json # the catalogue — every plugin registers here -scripts/validate.py # manifest + frontmatter validation (CI runs this) +scripts/ # manifest + frontmatter validation, version-bump guard (CI runs these) docs/ # these guides / # one directory per plugin .claude-plugin/plugin.json # name, description, version + CHANGELOG.md # required: one entry per released version README.md # the plugin's user guide skills//SKILL.md # one directory per skill scripts/ # deterministic scripts the skills drive references/ # rule/rubric content the skills read ``` -**Only `SKILL.md` is required.** A skill that is purely a procedure — nothing deterministic to run, no long reference material to point at — is one file in one directory, and should stay that way. `scripts/` appears when there is something mechanical worth doing in code; `references/` when the skill needs more context than belongs in its body. Adding either before you need it just makes the skill harder to read. +**Beyond the manifest and the changelog, only `SKILL.md` is required.** A skill that is purely a procedure — nothing deterministic to run, no long reference material to point at — is one file in one directory, and should stay that way. `scripts/` appears when there is something mechanical worth doing in code; `references/` when the skill needs more context than belongs in its body. Adding either before you need it just makes the skill harder to read. The three live plugins show some of the range: `qe` (an umbrella skill plus thin per-category sub-skills sharing plugin-level rules and scripts, alongside an unrelated standalone procedure with one script of its own), `benchmark` (one skill driving a deterministic engine, with worked examples as its regression baseline), and `audit` (sibling procedures sharing a method document). None of these is the house style — they are what three problems happened to need. @@ -32,7 +33,7 @@ Guidance rather than gates. The repo is early, and most of what follows generali 2. *Skills that judge*: record judgement as discrete answers (true/false per criterion, each cited), not free prose — so it's checkable. 3. *Skills that score*: when multiple judgements aggregate into a verdict with stakes, use the benchmark plugin's evidence-file pattern — judgement lives only in an evidence file, a deterministic engine computes every score, and no score is ever typed by hand. Aggregation is where hand-waving hides; the engine eliminates it. - **Don't duplicate content across docs** — one canonical location, pointers elsewhere. Rule text, weights, and thresholds especially: restated copies drift. -- **Self-contained plugins** (load-bearing — this one is a hard constraint of how plugins install, not a preference): an installed plugin ships only its own directory. No relative links or paths that escape the plugin root; use absolute GitHub URLs for repo-level files, and anchor runtime paths for the installed context (issue #4 tracks the `${CLAUDE_PLUGIN_ROOT}` pattern). +- **Self-contained plugins** (load-bearing — this one is a hard constraint of how plugins install, not a preference): an installed plugin ships only its own directory. No relative links or paths that escape the plugin root; use absolute GitHub URLs for repo-level files, and anchor runtime paths for the installed context with `${CLAUDE_PLUGIN_ROOT}` — `benchmark/skills/review-acceleration/SKILL.md` is the worked example. ## Development loop @@ -95,7 +96,7 @@ The plugin is the released artifact and its version string is how a release is d - **A version bump is the delivery mechanism** (load-bearing — this is how the install cache works, not a convention we chose). An installed plugin lives at `~/.claude/plugins/cache////`, and `claude plugin update` compares version strings only: if the content changed and the version did not, it reports "already at the latest version" and refreshes nothing. Merged content with an unchanged version reaches nobody who has the plugin installed, and nothing warns you. This has already happened here three times — most recently [#27](https://github.com/QuantEcon/skills/pull/27), which edited a shipped `SKILL.md` under `qe/` without bumping `qe`. - **Therefore any change to any file under `/` bumps that plugin's version.** Every file in the directory ships; there is no non-shipping edit inside it, and an exception list is where a rule like this rots. A file *moved out* of a plugin counts too — what it ships changed either way. Repo-level files — `docs/`, `README.md`, `CATALOG.md`, CI, the marketplace manifest — ship to nobody and bump nothing. - **Choose the number for what a user of the plugin experiences.** A new skill, or a procedure that now does something materially different: minor (benchmark's scaffolding → working evaluation system was 0.1.0 → 0.3.0 — there was never a published 0.2.0). A correction that leaves the procedure as it was: patch. Nothing below 1.0.0 promises stability. -- **The catalogue's own top-level `version` in `marketplace.json`** moves only when a plugin is added or removed, which is the existing precedent. It is not a cache key and delivers nothing, so it gets no changelog entry. +- **The catalogue's own top-level `version` in `marketplace.json`** moves only when a plugin is added or removed. (Past practice is mixed — adding `qe` did not move it, adding `audit` did — so the rule is stated here rather than inferred.) It is not a cache key and delivers nothing, so it gets no changelog entry. ### The changelog @@ -119,7 +120,15 @@ A second job runs `claude plugin validate --strict` against each plugin and the ### Tags -The repo has no git tags today, so nothing outside each `CHANGELOG.md` maps content to a version number. `claude plugin tag ./` creates `{name}--v{version}` from `plugin.json`, refuses unless the marketplace entry agrees, and refuses on a dirty working tree so the tag points at the version you meant to release. Add `--push` to publish it. Adopting it is a separate decision; nothing installs from a tag — the marketplace serves `main` — so the merge is still the release. +Each release is tagged `{name}--v{version}`, so three independently-versioned plugins share one tag namespace. Tag from a clean checkout of `main` after the release merges: + +```bash +claude plugin tag ./ --push -m " %s" +``` + +It takes the version from `plugin.json`, refuses unless the marketplace entry agrees, refuses if the plugin fails validation, and refuses on a dirty working tree — so a tag cannot point at a version you did not mean to release. `--dry-run` prints what it would do. + +**Nothing installs from a tag.** The marketplace serves `main`, so the merge is still the release and the tag is a bookmark for human archaeology — which is also why only current versions were tagged when the scheme was adopted, rather than backfilling history. Several historical versions could not have been tagged honestly: some name two different published trees, because content shipped twice under one version before the guard existed, and others sit at commits that fail today's validation. Each `CHANGELOG.md` records that ambiguity in prose, with PR numbers, which is more than a tag can carry. ## PR flow diff --git a/docs/tutorial-run-an-audit.md b/docs/tutorial-run-an-audit.md index 249c607..73365d0 100644 --- a/docs/tutorial-run-an-audit.md +++ b/docs/tutorial-run-an-audit.md @@ -105,7 +105,7 @@ Then confirm the boundary held: `git status --short` shows nothing but your igno ## Step 7 — record the run -Findings belong in this repo; the bundle does not. Write `reviews/audit-run-action-translation-2026-07-28.md`, following the [ge_arrow validation run](../reviews/validation-run-ge_arrow-2026-07-22.md): +Findings belong in this repo; the bundle does not. Write `reviews/audit-run--.md` — run 1's is [audit-run-action-translation-2026-07-28.md](../reviews/audit-run-action-translation-2026-07-28.md) — following the shape of the [ge_arrow validation run](../reviews/validation-run-ge_arrow-2026-07-22.md): - **Setup** — repo, snapshot timestamp, `fetched_by`, item counts, unaccounted numbers, plugin version. - **Cost** — wall clock and rough token spend per phase. Run 1's figures are in [its record](https://github.com/QuantEcon/skills/blob/main/reviews/audit-run-action-translation-2026-07-28.md); a second data point at a different repo type is what turns one measurement into an estimate. From 1263c8f9a1515c09c76fb279be492b6237206680 Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Mon, 3 Aug 2026 19:32:34 +1000 Subject: [PATCH 2/2] Docs: separate the plugin-level requirements from the skill-level one MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot review on #31, and correct. The sentence opens a paragraph that is entirely about skills — "A skill that is purely a procedure … is one file in one directory" — so prefixing it with two plugin-level requirements read as though a skill needs a manifest and a changelog of its own. Both live at /, not under skills//. Naming the two levels in one line is clearer than either the original, which omitted the plugin requirements the CI guard now enforces, or the prefix, which smuggled them into a sentence about something else. --- docs/developing-skills.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developing-skills.md b/docs/developing-skills.md index f41bb92..b782447 100644 --- a/docs/developing-skills.md +++ b/docs/developing-skills.md @@ -17,7 +17,7 @@ docs/ # these guides references/ # rule/rubric content the skills read ``` -**Beyond the manifest and the changelog, only `SKILL.md` is required.** A skill that is purely a procedure — nothing deterministic to run, no long reference material to point at — is one file in one directory, and should stay that way. `scripts/` appears when there is something mechanical worth doing in code; `references/` when the skill needs more context than belongs in its body. Adding either before you need it just makes the skill harder to read. +**A plugin needs its manifest and its changelog; a skill needs only `SKILL.md`.** A skill that is purely a procedure — nothing deterministic to run, no long reference material to point at — is one file in one directory, and should stay that way. `scripts/` appears when there is something mechanical worth doing in code; `references/` when the skill needs more context than belongs in its body. Adding either before you need it just makes the skill harder to read. The three live plugins show some of the range: `qe` (an umbrella skill plus thin per-category sub-skills sharing plugin-level rules and scripts, alongside an unrelated standalone procedure with one script of its own), `benchmark` (one skill driving a deterministic engine, with worked examples as its regression baseline), and `audit` (sibling procedures sharing a method document). None of these is the house style — they are what three problems happened to need.