Repository navigation
Description
Parent
Refs #15703
Source: #16029, part 2 of #15729. The list recorded in #16602 gives /knowledge:course-digest the route "call /corrections:codify, presence-gated, with the strongest-first fallback".
Depends on
- none open. The knowledge cut (Cut the model-read ecosystem specifics from knowledge #16766) is closed, and its rows were elsewhere in
context/workflow.md(lines 143 and 340-351).
Outcome
Course-digest's Phase 5 sorts each recommendation into fixed buckets, and its action-items template sends every rule to a "CLAUDE.md / .claude/rules/ candidates" list with "Add to {target file}", which picks the instruction rung for every rule a course suggests. After this change a rule candidate is handed to /corrections:codify when it is available, which picks the strongest rung that can decide it, with the shared strongest-first fallback when it is not. Extraction, analysis and the other recommendation buckets do not change.
Demo: with a corrections stand-in plugin loaded and an already extracted fixture course, run Phase 5. analysis/action-items.md lists a course rule such as "validate request bodies at the boundary" as a rule candidate handed to /corrections:codify, not as a line to add to a named instruction file.
Where the text changes
Line numbers as read on main at a483a2c1; they may move.
| File and lines | Present wording | Change |
|---|---|---|
plugins/knowledge/skills/course-digest/context/workflow.md:273-280 (Phase 5, step 4) |
"Categorize recommendations ...: CLAUDE.md / .claude/rules/ rule candidates; Skill candidates ...; CI/CD improvements" |
The first bucket becomes "Rule candidates", each handed to /corrections:codify (gate plus shared fallback) |
plugins/knowledge/skills/course-digest/reference/analysis-template.md:56-61 (action-items.md template) |
"## CLAUDE.md / .claude/rules/ candidates. Rules or conventions worth codifying: - [ ] {Rule description} — Add to {target file}." |
Heading "Rule candidates"; each entry names the codify hand-off instead of a target file |
Call form: /corrections:codify "<the rule>". The digest names the call in its output; it does not run codify unasked.
Decisions
-
Design Q21 (Resolve ecosystem specifics from the consumer repository and align the enforcement ladder with the talk's pyramid #15703): a skill that carries no ladder copy reads the ladder by calling codify; only a skill that applies its own selection rule holds a synced copy and a Selection rules row.
-
docs(enforcement-ladder): add the architecture and retro Selection rules rows in ladder 3.2.0 #16602 recorded this skill under "call
/corrections:codify, presence-gated, with the strongest-first fallback (12; change, filed as follow-on items by part 2, List enforcement-placing skills and give each one route to the ladder - part 2 of 2 #16029)". -
The fallback wording is shared by all nine follow-ons of List enforcement-placing skills and give each one route to the ladder - part 2 of 2 #16029. After the gate sentence ("when
/corrections:codifyis among the available skills, hand it the choice of mechanism; it picks the strongest rung of the enforcement ladder that can decide the rule and proves the check"), the fallback is exactly:Otherwise, propose the strongest mechanism that can decide the rule: a change that makes the mistake impossible first, then a check that blocks it, then a test, and an instruction only when nothing a machine runs can decide it. Say that
/corrections:codifywas not available, so the proposal is unproven.
Key interfaces
- "A skill that carries no ladder copy calls
/corrections:codify, presence-gated, with a two-sentence strongest-first fallback. Only a skill that applies its own selection rule holds a synced copy and gets a row in the ladder's Selection rules table." (List enforcement-placing skills and give each one route to the ladder #15729) - The seam-phrasing convention: an availability gate, a stated fallback beside it, the plugin named and never the marketplace.
Done when
- Phase 5 and the action-items template hand each rule candidate to
/corrections:codifybehind the availability gate, followed by the shared fallback, and no longer send it to an instruction file by default. - The skill gains no ladder copy or Selection rules row, and no rung list is restated.
- The pull request body quotes the sentence that picked a mechanism from a private list before the change. If the recheck finds none, the skill is left unchanged and the finding is reported on List enforcement-placing skills and give each one route to the ladder - part 2 of 2 #16029.
- One new eval case loads a corrections stand-in plugin and a pre-extracted fixture course (no browser, no download), in the shape of session-flow's
re-mechanical-finding-check-proposalcase, and grades that a course rule lands as a rule candidate handed to/corrections:codify. - The pull request body names the existing case whose run reads the changed text, or says none does (the current
cd-*cases stop before Phase 5). The lane runs no eval. - A changelog fragment under
.changes/knowledge/describes the change. - AC10: no new enforcement-placing skill is added here.
- Open the pull request with
Closesthis issue in its body, then stop: the lane adds no label.
After the pull request opens (host session, not the lane)
The pull request stays held with do-not-merge for the eval; a host session runs only the eval and posts its record; the merge lane releases the hold when the record matches the head.
- Hold line: the hub's sweep (or a host session) adds
do-not-mergeand posts, from a listed human's account, a comment with the line<!-- pr-hold reason=eval -->on a line of its own. - Run budget. List enforcement-placing skills and give each one route to the ladder #15729 asks for "that skill's cases on the
opusandsonnetaliases, three runs per side, and no more than 240 runs per plugin". The owner decision of 2026-11-16 in the owner-decisions record (hop 4 Q15, which names List enforcement-placing skills and give each one route to the ladder #15729, amended by hop 7 Q3) replaces the run counts: per changed skill, one targeted before-and-after case onsonnetat 3 runs a side;opusruns only to confirm a sonnet loss; a new case gets a 1 to 3 run smoke first. Select cases with--case, use--runs 3and a--max-cost-usdceiling, and start no batch after a usage-limit error. Ceiling for this plugin: 1 skill x (3 smoke + 6 sonnet + 6 opus confirm) = 15 runs, under the parent's 240. - Control: the replaced bucket is a partial cut, so the pull request body carries a detection-control result. No paired no-loss compare (#15703 comment 6906800940).
- On a LOSS reading: a trace read of each dropped case and an
opusconfirm; only a loss traced to the changed text counts. - Record: the verdict and the control result go into the pull request body. When the run is VALID and shows no loss, the host session posts, from a listed human's account, a comment with the line
<!-- eval-pair verdict=no-loss validity=VALID head=<40-hex head SHA> -->naming the current head, keeping "do not merge" anddo-not-mergeout of that comment (docs/contracts/loop-lane/README.md, "Cross-lane PR hold"). A loss or a run that is not VALID leaves the hold in place; say so on the pull request. - Release: pr-merge's scheduled step "Release eval holds whose record matches the head" removes
do-not-mergeonce that record names the current head and no other hold stands. A push after the record needs a new record.
Out of scope
- Extraction, adapters, the other recommendation buckets, and every other knowledge skill.
Related
- Resolve ecosystem specifics from the consumer repository and align the enforcement ladder with the talk's pyramid #15703 (container)
- List enforcement-placing skills and give each one route to the ladder #15729
- List enforcement-placing skills and give each one route to the ladder - part 2 of 2 #16029
- docs(enforcement-ladder): add the architecture and retro Selection rules rows in ladder 3.2.0 #16602
Metadata
Metadata
Assignees
Labels
Type
Fields
Work status
Priority
Activity
github-actions commentedon Nov 18, 2026
Size: Small
Complexity: Low
Uncertainty: Known
Flags: none
Blocked by: none
Blast radius: course-digest's Phase 5 workflow text, its action-items template, one new eval case and a changelog fragment; read by anyone running /knowledge:course-digest.
Reversible: yes; revert the text edits and remove the eval case.
Open questions: none
I read this as: Phase 5 and the action-items template should hand each course rule candidate to
/corrections:codify, behind the availability gate and with the shared fallback, instead of naming an instruction file. The outcome and done-when are exact. The pull request body quote and the eval run need a person or host, which does not hold this issue. No open issue asks for the same thing; #17174, #17172, #17163, #17179, #17162, #17158, #17160 and #17159 are sibling follow-ons for other skills. No merged pull request in the list already fixes it.Automated triage. A maintainer reviews it.
melodic-automation-lanes commentedon Nov 18, 2026
Not admitted to the issue lane: its done-when asks for what an unattended lane run cannot perform: item 3 (tracker-output: "The pull request body quotes the sentence that picked a mechanism from a priv..."), item 4 (eval: "One new eval case loads a corrections stand-in plugin and a pre-extracted fix..."); recommended split: keep the other items here as the admissible issue and move these to a child issue for the host queue (
host-run). Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).melodic-automation-lanes commentedon Nov 18, 2026
Refinement. A read-only run explored this repository, searched its issues and pull requests, and read the allowed docs to draft what follows. On an issue a listed human filed, it counts as the issue's spec once posted, while unedited; it is still model text, so check it.
Intended outcome: Course-digest's Phase 5 and action-items template hand each rule candidate to /corrections:codify behind the availability gate with the shared fallback, instead of naming an instruction file.
Split. Done-when items 3, 4 of the issue body need a person: they are checks a person runs, listed below, and the lane does not do them. This issue keeps the rest of its done-when.
Not done by the lane: checks a person runs
External state: none
Dependencies
Related work
Sources read
.claude/rules/rule candidates' and still has it as of the base checkoutAutomated refinement by the lanes App. Intake triages the issue again with it.
melodic-automation-lanes commentedon Nov 18, 2026
{"v":1,"actor_id":337595936,"title_sha256":"f00eae579e176637fdeef23812e01a1b0c1f1ad1857879a65187e54f1aeca69d","body_sha256":"bfea3d39d8a3c093fc61d2050ed866f7f453156fd7b7590fa2261c216b199582"}
melodic-automation-lanes commentedon Nov 18, 2026
{"v":1,"actor_id":337595936,"title_sha256":"f00eae579e176637fdeef23812e01a1b0c1f1ad1857879a65187e54f1aeca69d","body_sha256":"bfea3d39d8a3c093fc61d2050ed866f7f453156fd7b7590fa2261c216b199582"}
github-actions commentedon Nov 18, 2026
test(knowledge): grade course rules handed to /corrections:codify
kyle-sexton commentedon Nov 19, 2026
Closed by #17216 (merged 2026-11-19 09:49 UTC). For the record, what the host eval did and did not run:
sonnet, 3 runs a side (the eval record on feat(knowledge): hand course-digest rule candidates to corrections:codify #17216, issuecomment-6997295013): PASS, 0.714 before, 1.000 after;rule-candidate-to-codifyandno-instruction-targetwent from 0 of 3 to 3 of 3. Thereference/analysis-template.mdrow was measured; thecontext/workflow.mdrow was not (opened in 0 of 3 runs before, 1 of 3 after).opusconfirmation and the detection control. The gate in force is sonnet at 3 runs a side.