-
Notifications
You must be signed in to change notification settings - Fork 56
fix(agents): Explore agent type is *specifically* not to be used for code review #2098
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -312,8 +312,8 @@ For each selected sub-agent, assemble a context package containing: | |
| For each selected sub-agent: | ||
|
|
||
| 1. Read the sub-agent definition from `sub-agents/{name}.md` | ||
| 2. Extract the `model` from frontmatter | ||
| 3. Compose the spawn prompt from three parts: | ||
| 2. Extract `model` and `name` from frontmatter | ||
| 3. Compose the spawn prompt from: | ||
|
|
||
| **Part 1 — Sub-agent definition:** the full markdown body of the | ||
| sub-agent file (everything after the frontmatter) | ||
|
|
@@ -360,11 +360,13 @@ For each selected sub-agent: | |
| ``` | ||
|
|
||
| 4. Spawn via Agent tool with: | ||
| - `name`: from the sub-agent frontmatter | ||
| - `tools`: `Read, Grep, Glob` | ||
| - `model`: from the sub-agent frontmatter (any value accepted by | ||
|
ben-alkov marked this conversation as resolved.
|
||
| the Agent tool's `model` parameter) | ||
| - `subagent_type`: `Explore` (read-only — sub-agents do not write) | ||
| - `run_in_background`: `true` | ||
| - `prompt`: composed from parts 1–5 | ||
| the Agent tool's `model` parameter) | ||
| - `permissionMode`: `dontAsk` | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [important] Are these parameter names verified against a live Agent tool call? The schema I can see has Also — the challenger dispatch at step 6d (~line 495) still uses the old
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Verified in the Anthropic docs https://code.claude.com/docs/en/sub-agents#supported-frontmatter-fields EDIT: I keep forgetting that you guys don't necessarily read commit messages 😁
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Which schema?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updated challenger
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The Agent tool's own JSON schema — you can see it in any Claude Code session by looking at what parameters the Agent tool accepts. It takes The page you linked (code.claude.com/docs/en/sub-agents#supported-frontmatter-fields) lists frontmatter fields for agent definition So there are two interfaces here, and I think the PR is crossing them:
The old SKILL.md used valid Agent tool params ( ADR 0027 in this repo covers the distinction — If the goal is to restrict sub-agent tools to
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Agreed, but I'd take it a step further: IMO, all of that argument marshalling in the skill is wasted tokens anyway; Claude Code knows how to invoke subagents (using the Agent tool, which presumably Claude Code knows how to populate from the subagent definitions) - all we need to do is instruct it to build up the prompt the way we want. |
||
| - `background`: `true` | ||
| - `initialPrompt`: composed from parts 1–5 | ||
|
|
||
| **All sub-agents MUST be dispatched simultaneously** — include all | ||
| Agent calls in a single message so they run concurrently. This is the | ||
|
|
@@ -459,8 +461,9 @@ fresh context. The challenger has not seen the orchestrator's synthesis | |
| — it receives only the raw findings and the diff, preserving context | ||
| isolation. | ||
|
|
||
| 1. Read `sub-agents/challenger.md` for the sub-agent definition | ||
| 2. Compose the spawn prompt from: | ||
| 1. Read the sub-agent definition from `sub-agents/challenger.md` | ||
| 2. Extract `model` and `name` from frontmatter | ||
| 3. Compose the spawn prompt from: | ||
|
|
||
| **Part 1 — Sub-agent definition:** the full markdown body of the | ||
| challenger sub-agent file (everything after the frontmatter) | ||
|
|
@@ -494,10 +497,14 @@ isolation. | |
| REVIEW_SUB_AGENT_TRUE | ||
| ``` | ||
|
|
||
| 3. Spawn via Agent tool with: | ||
| - `model`: from the challenger sub-agent frontmatter (`opus`) | ||
| - `subagent_type`: `Explore` (read-only) | ||
| - `prompt`: composed from parts 1–4 | ||
| 4. Spawn via Agent tool with: | ||
| - `name`: from the challenger sub-agent frontmatter | ||
| - `tools`: `Read, Grep, Glob` | ||
| - `model`: from the sub-agent frontmatter (any value accepted by | ||
| the Agent tool's `model` parameter) | ||
| - `permissionMode`: `dontAsk` | ||
| - `background`: `true` | ||
| - `initialPrompt`: composed from parts 1–4 | ||
|
|
||
| **Prompt size guard:** If the combined context package (findings | ||
| JSON + diff + file list + PR metadata) exceeds 80 000 tokens, | ||
|
|
@@ -510,7 +517,7 @@ isolation. | |
| needs their findings as input), so it is dispatched sequentially, | ||
| not in the parallel batch from step 4. | ||
|
|
||
| 4. Consume the challenger's output. The challenger returns a **different | ||
| 5. Consume the challenger's output. The challenger returns a **different | ||
| format** from dimension sub-agents: an object with | ||
| `adjudicated_findings` and `removed_findings` arrays (not a flat | ||
| finding array). Parse accordingly: | ||
|
|
@@ -522,15 +529,15 @@ isolation. | |
| part of the standard finding schema. | ||
| - If `adjudicated_findings` is empty but the pre-challenger finding | ||
| set was non-empty, treat this as a challenger failure (fall back | ||
| per step 5 below). A legitimate challenger pass that removes all | ||
| findings is unlikely — an empty result more likely indicates a | ||
| parsing error or context truncation. | ||
| per the immediate next step below). A legitimate challenger pass | ||
| that removes all findings is unlikely — an empty result more likely | ||
| indicates a parsing error or context truncation. | ||
| - Otherwise, replace the merged finding set with the challenger's | ||
| `adjudicated_findings`. | ||
| - Log any `removed_findings` for transparency but do not include | ||
| them in the final review. | ||
|
|
||
| 5. If the challenger sub-agent fails (timeout, error, empty | ||
| 6. If the challenger sub-agent fails (timeout, error, empty | ||
| response), fall back to using the pre-challenger merged finding | ||
| set from steps 6a–6c. Record an **info**-level finding: | ||
|
|
||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.