Skip to content

Cut the model-read ecosystem specifics from harness-memory #16775

Description

@kyle-sexton

Parent

Refs #15703
Source: #15730 (blind classification under rules revision 5, #15730 comment 6950077480), cut item 21 of 47 in its filing order.

Depends on

Blocks #15732 (the portability gate widens only after every cut lands).

Outcome

harness-memory ships no model-read tool, library or command choice outside ADR 0101's named exceptions: the 4 rows below are cut (R11 4).

Demo: on main after the merge, each listed row's cited text is gone or replaced by generic text at a routing site, and the pull request body lists every row id with what happened to it.

Row count (bulk-mutation standard): 4 rows. The pull request states this number and lists each row.

Rows to cut

Paths are repository-relative; line numbers are as read at 5b66a7fa and may have moved. Kind is the inventory kind (K2 detection table, K3 bundled value, K4 tool guidance, K5 tool pointer, K7 example).

Row Rule Kind Cited at Fact the classifier matched Notes
F0214 R11 K7 plugins/harness-memory/skills/audit/context/audit.md:65-66 audit.md:66 still names global.json, .nvmrc, .python-version as model-read example pin files ("or ecosystem equivalents").
F0215 R11 K7 plugins/harness-memory/skills/audit/reference/criteria.md:276-277 criteria.md:281-282 still illustrates version checks with global.json and .nvmrc, a model-read K7 example.
F0216 R11 K7 plugins/harness-memory/skills/audit/reference/criteria.md:295-301 criteria.md:301 C8 still illustrates a compiler setting with "nullable, warnings-as-errors", a model-read K7 example.
F0217 R11 K7 plugins/harness-memory/skills/audit/reference/criteria.md:325-331 criteria.md:351-356 still lists package.json scripts, Makefile, *.csproj, pyproject.toml and "we use pytest" as model-read examples.

Kept rows, not touched here (count per rule): none.

Steps

  • List every item with its rule: the pull request body repeats the table above with each row's outcome (cut, replaced at a routing site, or kept with its reason).
  • Back every R11 call with a search showing whether a script reads the file, for example git grep -n -F '<file name>' -- plugins scripts .github, quoted in the pull request body. A row whose file a script reads is not cut here: report it on Classify the remaining guidance rows blind and count the cuts per plugin #15730.
  • Name the cases that read the cut text: in the pull request body, name for each changed skill the existing eval case whose run reads the cut text, or say that none does. The lane names them; it runs none.
  • Whole page or part: in the pull request body, say for each changed file whether the cut deletes the whole page or part of it.
  • Open the pull request with Closes this issue in its body, then stop: the lane adds no label. A sweep adds do-not-merge and the hold line when the pull request opens; the evidence below is a host session's work.

After the pull request opens (host session, not the lane)

The lane opens the pull request and stops there. The pull request stays held with do-not-merge for the eval; a host session runs only the eval and posts its record on the pull request; the merge lane releases the hold when the record matches the head; this issue closes with the pull request.

  • Hold line: the hub's sweep (or a host session) adds do-not-merge to the pull request and posts, from a listed human's account, a comment with the line <!-- pr-hold reason=eval --> on a line of its own.
  • Targeted case per changed skill (this replaces Classify the remaining guidance rows blind and count the cuts per plugin #15730's "no more than 400 runs per plugin (the changed skills' cases on the opus and sonnet aliases, three runs per side, plus control sides on one model)"): one before-and-after case, on sonnet at 3 runs a side; opus runs only to confirm a sonnet loss; a new case gets a 1 to 3 run smoke first (owner decision of 2026-11-16 in the owner-decisions record, superseding the run counts in #15703 comment 6906800940). Select cases with --case, use --runs 3 and a --max-cost-usd ceiling, and start no batch after a usage-limit error.
  • Controls: each whole-page deletion carries a sensitivity-control result, its predicted outcome written in the pull request body before the first batch; each partial cut carries a detection-control result. No paired no-loss compare per cut (#15703 comment 6906800940).
  • On a LOSS reading: a trace read of each dropped case and an opus confirm; only a loss traced to the changed text counts (#16672 comment 6935228111).
  • Record: the verdict and the control result go into the pull request body. When the run is VALID and shows no loss, the host session posts, from a listed human's account, a comment with the line <!-- eval-pair verdict=no-loss validity=VALID head=<40-hex head SHA> --> naming the pull request's current head, and keeps "do not merge" and do-not-merge out of that comment (docs/contracts/loop-lane/README.md, "Cross-lane PR hold"). A loss or a run that is not VALID leaves the hold in place; say so on the pull request.
  • Release: pr-merge's scheduled step "Release eval holds whose record matches the head" (.github/scripts/pr-merge/release-eval-hold.mjs) removes do-not-merge once that record is the newest since the label went on and names the current head, and no other hold stands. A push after the record needs a new record.
  • AC7 is met here: each whole-page deletion carries a sensitivity-control result and each partial cut a detection-control result, in the pull request body.

Decisions

  • D3: each plugin's cut is its own topic and pull request.
  • Interview Q32 (D4 of the Brief): bundled defaults that pick a tool, command or library are retired wherever they appear.
  • AC1's named exceptions are the kept classes of ADR 0101 (docs/adr/0101-resolve-ecosystem-specifics-from-the-consumer-repository.md, Decision and the 2026-11-17 amendment).
  • Bulk-mutation standard: count first, then act; the pull request states its row count.
  • The rules this item's rows fall under (rules revision 5):
    • R11 (shipped knowledge): everything else that is model-read K2, K3, K4, K5 or K7. Goes through the cutting route: the generic text replaces it only at a routing site where the skill still takes a language-specific step; anywhere else it is deleted with no replacement. An item kept only because a removed item consumes it goes in the same pull request.

Key interfaces

  • "The consumer's ecosystem (language, framework, tools): abstract now. Seam: the ecosystem-resolution contract and the consumer's own records."
  • Conventions followed: a cut ships with eval evidence (the skills-and-agents review rules).

Done when

  • AC1 (this plugin's share): harness-memory ships no model-read tool, library or command choice outside the named exceptions; each listed R11 row is cut.
  • The pull request body states the row count (4) and lists every row id with its rule and outcome.
  • Each R11 row's script-reader search is quoted in the pull request body.
  • The pull request body names, for each changed skill, the existing case that reads the cut text, or says none does.
  • The pull request body says, for each changed file, whether the cut deletes the whole page or part of it.
  • AC10: no new enforcement-placing skill is added here.

Out of scope

Activity

  1. added
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Nov 17, 2026
  2. added theissue type on Nov 17, 2026
  3. updated Work status ready and Priority medium on Nov 17, 2026
  4. github-actions commented on Nov 17, 2026

    @github-actions
    Contributor

    Size: Small
    Complexity: Low
    Uncertainty: Known
    Flags: none
    Blocked by: none
    Blast radius: two harness-memory audit skill files (context/audit.md and reference/criteria.md), read by the model when it runs the audit skill; the pull request body and eval evidence are also part of the work.
    Reversible: yes; revert the text edits.
    Open questions: none
    I read this as a bounded cut of 4 rows (F0214 to F0217). Each row is model-read ecosystem example text in the harness-memory audit skill. Each is deleted, or replaced with generic text where a routing site needs it. The pull request needs the row count, a script-reader search for each R11 row, and the targeted eval verdict. Nothing is missing, and the open-issues and merged-pulls lists showed no duplicate or merged fix. #16775 refs #15703 and #15730 and blocks #15732, so it is not blocked by any open issue.

    Automated triage. A maintainer reviews it.

  5. melodic-automation-lanes commented on Nov 17, 2026

    @melodic-automation-lanes
    Contributor

    {"v":1,"actor_id":337595936,"title_sha256":"0beb0162df5bfbc37733a2ce5c96a9414f2d0b69c0d5791212973d86147ba0dd","body_sha256":"9d84f6e4bdff315de87567511b2d01bf07a71f10fdb73836f0c5e6984553ded0"}

  6. added
    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.
    on Nov 17, 2026
  7. github-actions commented on Nov 17, 2026

    @github-actions
    Contributor
    No description provided.
  8. removed
    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.
    on Nov 17, 2026
  9. melodic-automation-lanes commented on Nov 17, 2026

    @melodic-automation-lanes
    Contributor

    The work-resolve-issue run that agent-ready started failed or was cancelled before it reported, or it stopped or the token broker refused it because its workflow files differ from main's. work-unblock removed and added agent-ready back to start a fresh run on main's workflow files. If this run ends the same way, it re-fires the issue again, waiting longer each time.

  10. github-actions commented on Nov 17, 2026

    @github-actions
    Contributor
    No description provided.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.

    Type

    Fields

    Work status

    needs-triage

    Priority

    medium

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions