Skip to content

feat(instruction-placement): hand a checkable instruction's mechanism to /corrections:codify #17163

Description

@kyle-sexton

Parent

Refs #15703
Source: #16029, part 2 of #15729. The list recorded in #16602 gives /instruction-placement:audit ("routing rubric's own ladder") and /instruction-placement:realign ("applies the audit's routing rubric") the route "call /corrections:codify, presence-gated, with the strongest-first fallback".

Depends on

Outcome

Today, when the routing rubric finds that a candidate is mechanically checkable (rung 2), it chooses the mechanism from its own list ("a compiler, linter, formatter, analyzer, or hook"). After this change the rubric hands that choice to /corrections:codify when it is available, and falls back to the shared strongest-first sentences when it is not. Both skills keep reporting the route without building it, and keep the prose until the mechanism exists. One rubric edit covers both skills, because audit and realign read the same file.

Demo: run /instruction-placement:audit on a repository whose CLAUDE.md holds a formatting rule a check could decide, with a corrections stand-in plugin loaded. The finding names /corrections:codify as the next step for that rule, does not pick a mechanism itself, and does not mark the prose for deletion.

Where the text changes

Line numbers as read on main at a483a2c1; they may move.

File and lines Present wording Change
plugins/instruction-placement/context/routing-rubric.md:83-86 (rung 2 of "The decision ladder") "A compiler, linter, formatter, analyzer, or hook could enforce it deterministically → route to that mechanism. Formatting rules, import ordering, naming patterns a regex can express. Never send an agent to do a linter's job. This plugin reports the routing; it does not author linters, and it does not delete the prose until the mechanism exists." Replace the mechanism list with the gated codify call plus the shared fallback (below); keep "reports the routing" and "does not delete the prose until the mechanism exists"
plugins/instruction-placement/context/routing-rubric.md:225 ("What this rubric does not decide") "Authoring the linter, hook, or skill that rung 2 and rung 3 route to." Say that choosing and proving rung 2's mechanism is /corrections:codify's, when available; authoring stays out of scope
plugins/instruction-placement/skills/realign/context/apply-recipes.md:185-187 ("Report-only outcomes") "Rung 2, mechanical enforcement. The remedy is a linter, formatter, analyzer, or hook. Report the routing and leave the prose in place." The blocked finding names the codify call (or the fallback) instead of a mechanism; the prose still stays in place
plugins/instruction-placement/skills/audit/context/routing-out.md:43-46 and plugins/instruction-placement/skills/realign/SKILL.md:258 "content a linter should enforce"; "A linter or skill routing has no destination this skill" Reword only where they name a mechanism for rung 2; no behavior change beyond the rubric's

The call form: /corrections:codify rules <the instruction file> for the candidate's own file, or /corrections:codify "<the rule>", whichever codify's argument hint fits; the audit names the call in the finding and never invokes it unasked (the audit is read-only).

Decisions

Key interfaces

  • "A skill that carries no ladder copy calls /corrections:codify, presence-gated, with a two-sentence strongest-first fallback. Only a skill that applies its own selection rule holds a synced copy and gets a row in the ladder's Selection rules table." (List enforcement-placing skills and give each one route to the ladder #15729)
  • The seam-phrasing convention: the gate is an availability condition ("when it is among the available skills"), the fallback is stated beside it, and the reference names the plugin, never the marketplace.
  • The fallback names no tool and no ladder file path: ADR 0101 forbids a shipped tool choice, and these skills carry no ladder copy.

Done when

  • Rung 2 of the routing rubric hands the mechanism choice to /corrections:codify behind the availability gate, followed by the shared fallback quoted under Decisions, and no longer lists mechanisms of its own.
  • Realign's "Report-only outcomes" names the same route for a rung 2 finding, still marks it blocked, and still leaves the prose in place.
  • Neither skill gains a ladder copy or a Selection rules row, and no rung list is restated.
  • The pull request body quotes, for each skill, the sentence that picked a mechanism from a private list before the change. If the recheck finds none for a skill, that skill is left unchanged and the finding is reported on List enforcement-placing skills and give each one route to the ladder - part 2 of 2 #16029.
  • One new eval case per changed skill (audit, realign) loads a corrections stand-in plugin, in the shape of session-flow's re-mechanical-finding-check-proposal case, and grades that the finding hands the checkable rule to /corrections:codify, picks no mechanism itself, and keeps the prose.
  • The pull request body names, for each changed skill, the existing case whose run reads the changed text, or says none does. The lane runs no eval.
  • A changelog fragment under .changes/instruction-placement/ describes the change.
  • AC10: no new enforcement-placing skill is added here.
  • Open the pull request with Closes this issue in its body, then stop: the lane adds no label.

After the pull request opens (host session, not the lane)

The pull request stays held with do-not-merge for the eval; a host session runs only the eval and posts its record; the merge lane releases the hold when the record matches the head.

  • Hold line: the hub's sweep (or a host session) adds do-not-merge and posts, from a listed human's account, a comment with the line <!-- pr-hold reason=eval --> on a line of its own.
  • Run budget. List enforcement-placing skills and give each one route to the ladder #15729 asks for "that skill's cases on the opus and sonnet aliases, three runs per side, and no more than 240 runs per plugin". The owner decision of 2026-11-16 in the owner-decisions record (hop 4 Q15, which names List enforcement-placing skills and give each one route to the ladder #15729, amended by hop 7 Q3) replaces the run counts: per changed skill, one targeted before-and-after case on sonnet at 3 runs a side; opus runs only to confirm a sonnet loss; a new case gets a 1 to 3 run smoke first. Select cases with --case, use --runs 3 and a --max-cost-usd ceiling, and start no batch after a usage-limit error. Ceiling for this plugin: 2 skills x (3 smoke + 6 sonnet + 6 opus confirm) = 30 runs, under the parent's 240.
  • Control: the removed mechanism list is a partial cut, so the pull request body carries a detection-control result. No paired no-loss compare (#15703 comment 6906800940).
  • On a LOSS reading: a trace read of each dropped case and an opus confirm; only a loss traced to the changed text counts.
  • Record: the verdict and the control result go into the pull request body. When the run is VALID and shows no loss, the host session posts, from a listed human's account, a comment with the line <!-- eval-pair verdict=no-loss validity=VALID head=<40-hex head SHA> --> naming the current head, keeping "do not merge" and do-not-merge out of that comment (docs/contracts/loop-lane/README.md, "Cross-lane PR hold"). A loss or a run that is not VALID leaves the hold in place; say so on the pull request.
  • Release: pr-merge's scheduled step "Release eval holds whose record matches the head" removes do-not-merge once that record names the current head and no other hold stands. A push after the record needs a new record.

Out of scope

  • Building any linter, hook or check the route names.
  • Rung 3 (skill) routing, the hard-deny gate, glob derivation and the promote lane.
  • Every other plugin's follow-on.

Related

Activity

  1. added
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Nov 18, 2026
  2. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 18, 2026
  3. github-actions commented on Nov 18, 2026

    @github-actions
    Contributor

    Size: Medium
    Complexity: Medium
    Uncertainty: Known
    Flags: none
    Blocked by: none
    Blast radius: instruction-placement's routing rubric, the realign apply recipes, audit and realign skill text, two new eval cases and a changelog fragment; read by anyone running /instruction-placement:audit or :realign.
    Reversible: yes; revert the text edits, eval cases and fragment.
    Open questions: none
    Understood: rung 2 of the routing rubric should hand the mechanism choice to /corrections:codify when it is available, with the shared strongest-first fallback otherwise. Realign's report-only outcome should name the same route, and the prose should stay in place. The issue gives exact wording, the files to change and the done-when list, so nothing is missing. The sibling follow-ons (#17179, #17175, #17174, #17172, #17162, #17160, #17159, #17158) cover other plugins, so none is a duplicate. I found no merged pull request that already does this.

    Automated triage. A maintainer reviews it.

  4. removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 18, 2026
  5. melodic-automation-lanes commented on Nov 18, 2026

    @melodic-automation-lanes
    Contributor

    Not admitted to the issue lane: its done-when asks for what an unattended lane run cannot perform: item 4 (tracker-output: "The pull request body quotes, for each skill, the sentence that picked a mech..."), item 5 (eval: "One new eval case per changed skill (audit, realign) loads a corrections stan..."); recommended split: keep the other items here as the admissible issue and move these to a child issue for the host queue (host-run). Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).

  6. melodic-automation-lanes commented on Nov 18, 2026

    @melodic-automation-lanes
    Contributor

    Refinement. A read-only run explored this repository, searched its issues and pull requests, and read the allowed docs to draft what follows. On an issue a listed human filed, it counts as the issue's spec once posted, while unedited; it is still model text, so check it.

    Intended outcome: Rung 2 of the instruction-placement routing rubric hands the mechanism choice to /corrections:codify behind an availability gate with the shared fallback, and realign's report-only outcome names that route while staying blocked and keeping the prose.

    Split. Done-when items 4, 5 of the issue body need a person: they are checks a person runs, listed below, and the lane does not do them. This issue keeps the rest of its done-when.

    Not done by the lane: checks a person runs

    External state: none

    Related work

    Sources read

    • plugins/instruction-placement/context/routing-rubric.md:83-86: rung 2 still lists 'compiler, linter, formatter, analyzer, or hook' and says the plugin reports routing and does not delete prose until the mechanism exists
    • plugins/instruction-placement/context/routing-rubric.md:225: 'Authoring the linter, hook, or skill that rung 2 and rung 3 route to' is out of scope
    • plugins/instruction-placement/skills/realign/context/apply-recipes.md:185: Rung 2 report-only outcome names 'a linter, formatter, analyzer, or hook'
    • plugins/instruction-placement/skills/audit/context/routing-out.md:43: names 'content a linter should' enforce
    • plugins/instruction-placement/skills/realign/SKILL.md:258: 'A linter or skill routing has no destination this skill' can apply

    Automated refinement by the lanes App. Intake triages the issue again with it.

  7. github-actions commented on Nov 18, 2026

    @github-actions
    Contributor

    The triage sweep dispatched intake triage for this issue (dispatch 1; 2 runs that reach intake, or 6 dispatches, hand it to a maintainer).

  8. melodic-automation-lanes commented on Nov 18, 2026

    @melodic-automation-lanes
    Contributor

    {"v":1,"actor_id":337595936,"title_sha256":"78db5de65a504899d35e45a3c645132a5264f9c80c266815b73602448164adc9","body_sha256":"e2a3e63b439da0ac1d445830e922bd42eb2bead4348d6c4efaa4186c3c9aca29"}

  9. melodic-automation-lanes commented on Nov 18, 2026

    @melodic-automation-lanes
    Contributor

    {"v":1,"actor_id":337595936,"title_sha256":"78db5de65a504899d35e45a3c645132a5264f9c80c266815b73602448164adc9","body_sha256":"e2a3e63b439da0ac1d445830e922bd42eb2bead4348d6c4efaa4186c3c9aca29"}

  10. github-actions commented on Nov 18, 2026

    @github-actions
    Contributor
    No description provided.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.

    Fields

    Work status

    needs-triage

    Priority

    medium

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions