Skip to content

feat(harness-config): hand mechanism choices in three instruction skills to /corrections:codify #17158

Description

@kyle-sexton

Parent

Refs #15703
Source: #16029, part 2 of #15729. The list recorded in #16602 gives three harness-config skills the route "call /corrections:codify, presence-gated, with the strongest-first fallback": /harness-config:audit-instructions ("runs its own rule-to-hook check on skill, agent and hook text"), /harness-config:audit-prompting-postures, and /harness-config:draft-auto-mode-rules ("sends a hard rule to permissions.deny or a PreToolUse hook from its own list").

Depends on

Outcome

Each of these three skills picks a mechanism for a rule from its own short list today: a hook or another blocking check (audit-instructions I5 and I15), a confirmation line or a gate (audit-prompting-postures P7), and permissions.deny or a PreToolUse hook (draft-auto-mode-rules). After this change each one hands that choice to /corrections:codify when it is available, and uses the shared strongest-first fallback when it is not. What each skill detects and reports does not change.

Demo: with a corrections stand-in plugin loaded, /harness-config:draft-auto-mode-rules asked for an entry that must never happen names /corrections:codify for the harness-level mechanism and still keeps the hard_deny entry as a second layer; /harness-config:audit-instructions on a skill body with a must-always rule and no check names /corrections:codify instead of proposing "a hook".

Where the text changes

Line numbers as read on main at a483a2c1; they may move.

Skill File and lines Present wording Change
audit-instructions plugins/harness-config/skills/audit-instructions/reference/criteria.md:415-437 (I5 "Rule-to-hook or delete", Remediate) "Convert a must-always rule with no such check, including one whose check only warns or reaches less than the prose, to a hook or another blocking check; once the check exists, the prose goes in the same change" The conversion target is handed to /corrections:codify (gate plus shared fallback); the deletion and hold rules stay as written
audit-instructions same file, :1220-1223 (I15, "Remediate by scope") "Where the same conflict keeps recurring, offer the mechanism route, a PreToolUse hook, a permissions.deny rule, or a skill's own disallowed-tools, since a mechanism outranks instruction text." The recurring-conflict route names /corrections:codify instead of the three mechanisms
audit-prompting-postures plugins/harness-config/skills/audit-prompting-postures/reference/postures.md:174-184 (P7) and SKILL.md:117-136 (Phase C) P7: "destructive actions require confirmation or an equivalent mechanical gate ... A deny-by-default hook or script gate satisfies this without any prose." Presence judging is unchanged (a gate still counts as PRESENT). A MISSING P7 proposal hands the choice between a gate and a confirmation line to /corrections:codify, with the shared fallback
draft-auto-mode-rules plugins/harness-config/skills/draft-auto-mode-rules/SKILL.md:111-116 (Phase 2, item 1) "When the answer is an action that must never happen ... say that it belongs in permissions.deny or a PreToolUse hook, which the harness applies every time, and keep the hard_deny entry as a second layer. For a Bash action prefer the hook" The harness-level mechanism is handed to /corrections:codify; keep "keep the hard_deny entry as a second layer" and the pointer to permission rule syntax

Call forms codify's argument hint offers: /corrections:codify findings <file> for a persisted audit report, /corrections:codify rules <files> for rules in instruction files, /corrections:codify "<the rule>" for one described rule. The audit skills are report-only: they name the call and never invoke it unasked.

Decisions

Key interfaces

  • "A skill that carries no ladder copy calls /corrections:codify, presence-gated, with a two-sentence strongest-first fallback. Only a skill that applies its own selection rule holds a synced copy and gets a row in the ladder's Selection rules table." (List enforcement-placing skills and give each one route to the ladder #15729)
  • The seam-phrasing convention: an availability gate, a stated fallback beside it, the plugin named and never the marketplace.
  • The fallback names no tool and no ladder file path (ADR 0101; these skills carry no ladder copy). /harness-config:audit-automation-gaps keeps its own copy and row and is not touched.

Done when

  • audit-instructions I5 and I15 hand the mechanism choice to /corrections:codify behind the availability gate, followed by the shared fallback, and list no mechanisms of their own.
  • audit-prompting-postures' proposal for a MISSING P7 does the same, and its rule that a gate counts as PRESENT is unchanged.
  • draft-auto-mode-rules' Phase 2 item 1 does the same and still keeps the hard_deny entry as a second layer.
  • None of the three gains a ladder copy or a Selection rules row, and no rung list is restated.
  • The pull request body names, for each changed skill, the existing case whose run reads the changed text, or says none does (candidates: ai-enforced-step-flagged, ai-cross-surface-conflict, pp-destructive-missing-gate, pp-script-gate-present, dr-compose-paste-ready, dr-uncheckable-condition). The lane runs no eval.
  • A changelog fragment under .changes/harness-config/ describes the change.
  • AC10: no new enforcement-placing skill is added here.
  • Open the pull request with Closes this issue in its body, then stop: the lane adds no label.

Host steps (eval case, tracker-output quote) are tracked in #17200.

After the pull request opens (host session, not the lane)

The pull request stays held with do-not-merge for the eval; a host session runs only the eval and posts its record; the merge lane releases the hold when the record matches the head.

  • Hold line: the hub's sweep (or a host session) adds do-not-merge and posts, from a listed human's account, a comment with the line <!-- pr-hold reason=eval --> on a line of its own.
  • Run budget. List enforcement-placing skills and give each one route to the ladder #15729 asks for "that skill's cases on the opus and sonnet aliases, three runs per side, and no more than 240 runs per plugin". The owner decision of 2026-11-16 in the owner-decisions record (hop 4 Q15, which names List enforcement-placing skills and give each one route to the ladder #15729, amended by hop 7 Q3) replaces the run counts: per changed skill, one targeted before-and-after case on sonnet at 3 runs a side; opus runs only to confirm a sonnet loss; a new case gets a 1 to 3 run smoke first. Select cases with --case, use --runs 3 and a --max-cost-usd ceiling, and start no batch after a usage-limit error. Ceiling for this plugin: 3 skills x (3 smoke + 6 sonnet + 6 opus confirm) = 45 runs, under the parent's 240.
  • Control: each removed mechanism list is a partial cut, so the pull request body carries a detection-control result. No paired no-loss compare (#15703 comment 6906800940).
  • On a LOSS reading: a trace read of each dropped case and an opus confirm; only a loss traced to the changed text counts.
  • Record: the verdict and the control result go into the pull request body. When the run is VALID and shows no loss, the host session posts, from a listed human's account, a comment with the line <!-- eval-pair verdict=no-loss validity=VALID head=<40-hex head SHA> --> naming the current head, keeping "do not merge" and do-not-merge out of that comment (docs/contracts/loop-lane/README.md, "Cross-lane PR hold"). A loss or a run that is not VALID leaves the hold in place; say so on the pull request.
  • Release: pr-merge's scheduled step "Release eval holds whose record matches the head" removes do-not-merge once that record names the current head and no other hold stands. A push after the record needs a new record.

Out of scope

Related

Activity

  1. added
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Nov 18, 2026
  2. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 18, 2026
  3. github-actions commented on Nov 18, 2026

    @github-actions
    Contributor

    Size: Medium
    Complexity: Medium
    Uncertainty: Known
    Flags: none
    Blocked by: none
    Blast radius: wording in three harness-config skills (audit-instructions criteria, audit-prompting-postures P7 and Phase C, draft-auto-mode-rules Phase 2), plus a changelog fragment. Plugin users who run those skills read it.
    Reversible: yes; revert the text edits.
    Open questions: none
    I read this as a bounded text change. The three skills hand their mechanism choice to /corrections:codify behind an availability gate, with the shared strongest-first fallback. Detection and reporting stay as they are. The issue gives exact files, wording and done-when items, so no decision is open. The eval and tracker-output steps need a host run and are tracked in #17200. They don't hold the lane-checkable items. The sibling follow-ons (#17159, #17160, #17162, #17174, #17175) cover other plugins, so none of them duplicates this issue.

    Automated triage. A maintainer reviews it.

  4. removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 18, 2026
  5. melodic-automation-lanes commented on Nov 18, 2026

    @melodic-automation-lanes
    Contributor

    Not admitted to the issue lane: its done-when asks for what an unattended lane run cannot perform: item 5 (tracker-output: "The pull request body quotes, for each skill, the sentence that picked a mech..."), item 6 (eval: "One new eval case per changed skill (three) loads a corrections stand-in plug..."); recommended split: keep the other items here as the admissible issue and move these to a child issue for the host queue (host-run). Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).

  6. melodic-automation-lanes commented on Nov 18, 2026

    @melodic-automation-lanes
    Contributor

    {"v":1,"actor_id":337595936,"title_sha256":"313bd1686200fdba09886e3e0554262d3dac41cd930e90642b68b78c68721101","body_sha256":"842e786a01b881897249c1daa1589fd4770c977cc345d8707389e94098397ed3"}

  7. melodic-automation-lanes commented on Nov 18, 2026

    @melodic-automation-lanes
    Contributor

    {"v":1,"actor_id":337595936,"title_sha256":"313bd1686200fdba09886e3e0554262d3dac41cd930e90642b68b78c68721101","body_sha256":"842e786a01b881897249c1daa1589fd4770c977cc345d8707389e94098397ed3"}

  8. github-actions commented on Nov 18, 2026

    @github-actions
    Contributor
    No description provided.
  9. github-actions commented on Nov 19, 2026

    @github-actions
    Contributor
    No description provided.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.

    Type

    Fields

    Work status

    needs-triage

    Priority

    medium

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions