Skip to content

Cut the model-read ecosystem specifics from review, starting with the ecosystem-specialist agent's generic defaults #16751

Closed

Description

@kyle-sexton

Parent

Refs #15703
Source: #15730 (blind classification under rules revision 5, #15730 comment 6950077480), cut item 2 of 47 in its filing order.

Depends on

Blocks #15732 (the portability gate widens only after every cut lands).

Outcome

review ships no model-read tool, library or command choice outside ADR 0101's named exceptions: the 6 rows below are cut (R11 5, R4 1).

#15730's filing order names this cut ahead of the rest: "the review ecosystem-specialist agent's generic defaults".

Pull request #16722 (merged 2026-11-17) edited plugins/review/agents/ecosystem-specialist.md after the classification input was read; re-read the cited lines there before cutting.

Demo: on main after the merge, each listed row's cited text is gone or replaced by generic text at a routing site, and the pull request body lists every row id with what happened to it.

Row count (bulk-mutation standard): 6 rows. The pull request states this number and lists each row.

Rows to cut

Paths are repository-relative; line numbers are as read at 5b66a7fa and may have moved. Kind is the inventory kind (K2 detection table, K3 bundled value, K4 tool guidance, K5 tool pointer, K7 example).

Row Rule Kind Cited at Fact the classifier matched Notes
F0347 R11 K2 plugins/review/agents/ecosystem-specialist.md:17 (path-to-ecosystem table) ecosystem-specialist.md:17 is a model-read path-to-ecosystem heuristic (.cs→.NET, .py→Python, and so on) for an agent whose description is a general multi-language capability (K2).
F0348 R4 K3 plugins/review/agents/ecosystem-specialist.md:30-38 (fallback build/test/lint commands) ecosystem-specialist.md's fallback commands (ruff check, eslint/biome check, pytest, golangci-lint, and others) pick tools for the consumer when the repository declares none.
F0349 R11 K7 plugins/review/agents/ecosystem-specialist.md:42-50 (report example) The report-format example (.NET block, Bash ShellCheck FAIL) illustrates an ecosystem-neutral report shape (K7).
F0356 R11 K7 plugins/review/agents/code-reviewer.md:79 (cast and debug-output examples) code-reviewer.md:82 gives unknown/any casts and console.log/print/dbg! as examples of a language-neutral review check (K7).
F0360 R11 K5 plugins/review/skills/quality-gate/context/security.md:49-50 "CodeQL or similar" is a model-read tool pointer in after-review advice, not a list or check that guards a harm, so nothing before R11 holds.
F0361 R11 K7 plugins/review/skills/quality-gate/context/downstream.md:149 (probe command example) python3 "<tmpdir>/probe.py" "<root>" is a one-language example inside the downstream-probe rule; the guard is the sentence around it, and an example inside a check is judged by later rules, none of which holds. Two readings (other: R2). Reader's note: F0361: R2 is the other reading, if the example counts as part of the "untrusted text stays data" guard.

Kept rows, not touched here (count per rule): R2 4, R6 2, R7 2. 1 more rows were answered GONE (cited content no longer on main) and need nothing.

Steps

  • List every item with its rule: the pull request body repeats the table above with each row's outcome (cut, replaced at a routing site, or kept with its reason).
  • Back every R11 call with a search showing whether a script reads the file, for example git grep -n -F '<file name>' -- plugins scripts .github, quoted in the pull request body. A row whose file a script reads is not cut here: report it on Classify the remaining guidance rows blind and count the cuts per plugin #15730.
  • Name the cases that read the cut text: in the pull request body, name for each changed skill the existing eval case whose run reads the cut text, or say that none does. The lane names them; it runs none.
  • Whole page or part: in the pull request body, say for each changed file whether the cut deletes the whole page or part of it.
  • R4 rows: the bundled default goes; the choice is resolved from the consumer through the ecosystem-resolution contract (docs/contracts/ecosystem-resolution/). A shipped default that picks a library goes with this cut.
  • Security floor check: the reader's note on F0361 mentions R2 (a security or safety floor). Confirm each is not a security floor before cutting it; narrowing or removing a security floor is the owner's decision (D2 of the Brief), so a row that is one is not cut here: report it on Resolve ecosystem specifics from the consumer repository and align the enforcement ladder with the talk's pyramid #15703.
  • Open the pull request with Closes this issue in its body, then stop: the lane adds no label. A sweep adds do-not-merge and the hold line when the pull request opens; the evidence below is a host session's work.

After the pull request opens (host session, not the lane)

The lane opens the pull request and stops there. The pull request stays held with do-not-merge for the eval; a host session runs only the eval and posts its record on the pull request; the merge lane releases the hold when the record matches the head; this issue closes with the pull request.

  • Hold line: the hub's sweep (or a host session) adds do-not-merge to the pull request and posts, from a listed human's account, a comment with the line <!-- pr-hold reason=eval --> on a line of its own.
  • Targeted case per changed skill (this replaces Classify the remaining guidance rows blind and count the cuts per plugin #15730's "no more than 400 runs per plugin (the changed skills' cases on the opus and sonnet aliases, three runs per side, plus control sides on one model)"): one before-and-after case, on sonnet at 3 runs a side; opus runs only to confirm a sonnet loss; a new case gets a 1 to 3 run smoke first (owner decision of 2026-11-16 in the owner-decisions record, superseding the run counts in #15703 comment 6906800940). Select cases with --case, use --runs 3 and a --max-cost-usd ceiling, and start no batch after a usage-limit error.
  • Controls: each whole-page deletion carries a sensitivity-control result, its predicted outcome written in the pull request body before the first batch; each partial cut carries a detection-control result. No paired no-loss compare per cut (#15703 comment 6906800940).
  • On a LOSS reading: a trace read of each dropped case and an opus confirm; only a loss traced to the changed text counts (#16672 comment 6935228111).
  • Record: the verdict and the control result go into the pull request body. When the run is VALID and shows no loss, the host session posts, from a listed human's account, a comment with the line <!-- eval-pair verdict=no-loss validity=VALID head=<40-hex head SHA> --> naming the pull request's current head, and keeps "do not merge" and do-not-merge out of that comment (docs/contracts/loop-lane/README.md, "Cross-lane PR hold"). A loss or a run that is not VALID leaves the hold in place; say so on the pull request.
  • Release: pr-merge's scheduled step "Release eval holds whose record matches the head" (.github/scripts/pr-merge/release-eval-hold.mjs) removes do-not-merge once that record is the newest since the label went on and names the current head, and no other hold stands. A push after the record needs a new record.
  • AC7 is met here: each whole-page deletion carries a sensitivity-control result and each partial cut a detection-control result, in the pull request body.

Decisions

  • D3: each plugin's cut is its own topic and pull request.
  • Interview Q32 (D4 of the Brief): bundled defaults that pick a tool, command or library are retired wherever they appear.
  • AC1's named exceptions are the kept classes of ADR 0101 (docs/adr/0101-resolve-ecosystem-specifics-from-the-consumer-repository.md, Decision and the 2026-11-17 amendment).
  • Bulk-mutation standard: count first, then act; the pull request states its row count.
  • The rules this item's rows fall under (rules revision 5):
    • R11 (shipped knowledge): everything else that is model-read K2, K3, K4, K5 or K7. Goes through the cutting route: the generic text replaces it only at a routing site where the skill still takes a language-specific step; anywhere else it is deleted with no replacement. An item kept only because a removed item consumes it goes in the same pull request.
    • R4 (consumer default): the value picks a tool or library for the consumer. Goes: no bundled default survives (ADR 0101 decision 2, "Ship no bundled default"); the choice is resolved from the consumer.

Key interfaces

  • "The consumer's ecosystem (language, framework, tools): abstract now. Seam: the ecosystem-resolution contract and the consumer's own records."
  • Conventions followed: a cut ships with eval evidence (the skills-and-agents review rules).

Done when

  • AC1 (this plugin's share): review ships no model-read tool, library or command choice outside the named exceptions; each listed R11 and R4 row is cut.
  • The pull request body states the row count (6) and lists every row id with its rule and outcome.
  • Each R11 row's script-reader search is quoted in the pull request body.
  • The pull request body names, for each changed skill, the existing case that reads the cut text, or says none does.
  • The pull request body says, for each changed file, whether the cut deletes the whole page or part of it.
  • AC10: no new enforcement-placing skill is added here.

Out of scope

Activity

  1. added
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Nov 17, 2026
  2. added theissue type on Nov 17, 2026
  3. updated Priority medium and Work status ready on Nov 17, 2026
  4. melodic-automation-lanes commented on Nov 17, 2026

    @melodic-automation-lanes
    Contributor

    {"v":1,"actor_id":337595936,"title_sha256":"840765f6733ff656cefa41508a9469b3c93b3864c63fa43cfecfc62458ee6cd7","body_sha256":"5e2964d540dd8347f0e9c17eb867eeda14d0ea50d11551cb0865d83b4f389dc7"}

  5. github-actions commented on Nov 17, 2026

    @github-actions
    Contributor
    No description provided.
  6. added
    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.
    on Nov 17, 2026
  7. kyle-sexton commented on Nov 18, 2026

    @kyle-sexton
    MemberAuthor

    Delivered by pull request 16807 (merged 2026-11-18): #16807. Eval record for the merged head: #16807 (comment). The wrong-text control result (AC7) was not run, by the owner's decision of 2026-11-17: cut pull requests are gated on the no-loss compare alone. Closed by an agent session.

  8. removed
    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.
    on Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.

    Type

    Fields

    Work status

    confirmed

    Priority

    medium

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions