Skip to content

review: the two reviewers share one context and one model, so the isolation SKILL.md requires is nominal #71

Description

@Alex1990

What

skills/review/SKILL.md requires two independent reviewers:

The implementing agent (maker) must not be the only judge. Spawn two reviewer
agents with isolation (separate worktree or equivalent).

The current loop does not do this. pr-review is a single stage
(docs/agents/ops.md, "Trigger → loop map"), executed as one GitHub Actions job by
one pi process (scripts/loop/shared/agent.mjs:207 runPi, one spawn), in one
session. Reviewer A and Reviewer B are two passes of the same context and the same
model
, not two independent readers.

pi's core has no sub-agent mechanism at all — @earendil-works/pi-coding-agent@0.85.1,
docs/usage.md, Design Principles:

It intentionally does not include built-in MCP, sub-agents, permission popups, plan
mode, to-dos, or background bash.

So the sentence in the skill cannot be satisfied by the current engine as written.

Why it matters

Independence is the entire point of the split. In Addy Osmani's four-reviewer
experiment (146 PRs, 679 findings), of 617 distinct flagged locations 93.4% were
caught by exactly one tool and 0% by all four
— heterogeneity is what produces
coverage. Two passes of one model over one context is "a single reviewer with a larger
invoice", and it silently breaks a norms-layer promise.

Options

  • A. Two runs in one job. Call runPi() twice per review stage with separate
    --session-dir, a different --model for the second lens, and a read-only tool
    allowlist (--tools read,grep,find,ls,bash). Smallest host change: runPi already
    accepts sessionDir and model; only --tools needs exposing.
  • B. Two jobs (strategy.matrix.reviewer: [A, B]). Adds filesystem and
    environment isolation on top of context isolation. Higher CI cost.
  • C. Install pi-subagents (npm:pi-subagents@0.70.1, third-party) and define
    .pi/agents/*.md. Most capability, but pulls an unaudited 10 MB dependency into CI,
    and .github/workflows/** is human-only (see ops.md), so C is not loop work.

Whichever is chosen, skills/review/SKILL.md must be reworded so the requirement
matches the mechanism (norms layer → proposal PR).

Acceptance

  • The two verdicts come from runs that cannot see each other, demonstrable from
    separate session files/logs.
  • The second lens runs on a different model.
  • Reviewer runs hold a read-only tool set.
  • skills/review/SKILL.md no longer instructs the agent to spawn sub-agents.

References

  • pi v0.85.1 packages/coding-agent/docs/usage.md (Design Principles); CLI flags
    --tools, --model, --skill
  • Addy Osmani, Agentic Code Review

Activity

  1. github-actions commented on Sep 21, 2026

    @github-actions
    Contributor

    Loop triage — feature / ready-for-agent (confidence: high)

    Confirmed the defect against the tree, not just the report:

    • skills/review/SKILL.md:9-10 requires two reviewer agents "with isolation (separate worktree or equivalent)".
    • scripts/loop/shared/agent.mjs runPi() builds one argv and performs one spawn, with no --tools flag.
    • scripts/loop/entry.mjs doRun() calls runPi() once, with a single sessionDir (LOOP_SESSION_DIR/<rid>) and a single model (LOOP_MODEL).
    • The most recent pr-review report (state/reports/r-20260921-164320-7u4k.md) is one run whose "Two-lens review" is two sections of the same report, not two independent verdicts.
    • pi here is 0.85.1: there is no sub-agent mechanism, so the skill sentence cannot be satisfied as written.

    Why this is loop work. Option A (two runPi() calls in one job) touches only scripts/loop/** and the norms-layer skills/review/SKILL.md — no .github/workflows/** change, so the job token can push the branch. docs/norms/ops.md explicitly allows host changes as loop PRs (owner's review is the checker) and norms-layer changes as proposal PRs. Option B (strategy.matrix in loop.yml) and option C (pi-subagents) both need workflow/config files the loop cannot push, so they are human work.

    Acceptance maps to A: separate --session-dir per reviewer → distinct session files/logs; a second --model from the available catalog (deepseek-v4-flash / -vision-exp / -pro) → a genuinely different lens; --tools read,grep,find,ls,bash → the edit/write tools are absent; and the SKILL.md reword removes the sub-agent instruction.

    One caveat to carry into the implementation: bash remains in that allowlist (the reviewer needs git/gh), so "read-only" must be stated as "no edit/write tools", not filesystem read-onlyness. Also give each reviewer its own result/report path, since the current contract assumes one <rid>.md / <rid>.result.json per run and the second pass would otherwise overwrite the first.

    Labelling ready-for-agent.

  2. added
    ready-for-humanRequires a human: implementation, merge of a loop PR, or a release call
    and removed
    ready-for-agentFully specified, an AFK agent may handle it unattended
    on Sep 21, 2026
  3. github-actions commented on Sep 21, 2026

    @github-actions
    Contributor

    Loop feature — PR opened for this.

    pr-review now runs two genuinely isolated reviewers instead of two passes of one context: two pi processes with separate --session-dir, a different --model for the second lens, and --tools read,grep,find,ls,bash, merged by the host. Both must approve (accepted); a missing verdict is never counted as approval (transient retry, otherwise failed). skills/review/SKILL.md was reworded to match the mechanism (norms proposal) and no longer tells the agent to spawn sub-agents.

    Verified with a fake-engine harness: exactly two invocations, distinct session dirs and models, both read-only, plus the accept / request-changes / inconclusive merge paths (32/32 checks). All repo gates green (pnpm lint, pnpm fmt:check, pnpm build, pnpm check:types, Azure + TOS oracles, headless browser suite 24 files / 255 passed / 26 skipped).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-humanRequires a human: implementation, merge of a loop PR, or a release call

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions