Repository navigation
feat(external-agents): add the external-agents plugin to replace the official Codex plugin - #17954
kyle-sexton wants to merge 27 commits into
Conversation
…he no-bin gate New plugin external-agents: manifest, README, catalog entry, team config schema, the Codex prerequisite with its github latest-release source, shared copies of config-cascade, yaml-subset, cli-version-check and the prerequisites checker, and the bin/external-agents shim (usage only). scripts/check-plugin-manifest-presence.sh rule 3 now skips plugins/external-agents/bin, because review:council needs its command on PATH; any other plugin's bin/ still fails, with a test case for each. The plugin is left out of claude.ai organization sync (owner decision). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
…ecord, binary and job store - scripts/lib/config.mjs resolves external-agents.yaml through the config-cascade layers over the bundled defaults.json. - providers/codex.yaml is the Codex record, its flags checked against codex exec, exec review and exec resume --help on 0.168.0; scripts/lib/provider.mjs validates any record, including per-mode settings files, the json_array_nonempty and stderr_substring soft-deny kinds and notice fallback warnings the Antigravity record needs. - scripts/lib/binary.mjs resolves the binary: binary_path, then the record's per-platform candidates, then PATH (PATHEXT on Windows), and names what it tried when nothing is found. - scripts/lib/jobs.mjs keeps one job file per job under a repo key hashed from the git common dir (no-repo outside one), written atomically and pruned to jobs.keep, never pruning a running job. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
A committed team file could set <product>.binary_path and so choose the executable the runner spawns from any checkout of that repository. The value is now honored only from the personal override and only when it is an absolute path; a team or relative value is reported and ignored. The defaults test now compares against the defaults the schema documents instead of repeating the numbers, the binary_path precedence test puts a decoy in a candidate location and on PATH, and the README and the config-cascade Implementers row describe what exists now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
…h the done-vs-blocked rule and the breaker Every run passes the mode's sandbox and approval_policy never; a write request on a read-only use runs read-only and names the config key. A run is ok only when the CLI exits 0, its last turn completed, no error event arrived, the final message is non-empty and no command or file change was refused; any refusal records blocked-by-mode with the refused commands. MCP tool calls are counted and named on the first output line. Resume takes only thread ids in this plugin's job files. Each job gets its own folder for the prompt, final message, logs and review diff. cancel and the timeout kill the process tree after checking the pid's start time, and the runner keeps a cancelled status that cancel wrote first. The breaker opens on a missing, logged-out, auth-failed or usage-limited CLI and refuses with the next trial time. For a binary started through cmd.exe, every value that reaches argv must match a conservative character set, and a CLI that takes its prompt as an argument is refused, both pointing at binary_path. The provider record now rejects unknown keys and carries the prompt form, output kind, resume and review blocks; both output kinds, both prompt forms and the diff-file review are implemented. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
…el and review status A run the CLI rejects before any event now reports the CLI's own error line. Text on stderr alone no longer opens the breaker: only an error the CLI reported in its event stream or result does. A run that finishes well closes the breaker only when it was the trial of that same breaker state, so it cannot erase one a concurrent run opened. A cancel whose kill fails leaves the job running and cancellable. A review that was blocked and then failed is reported failed and relays no partial findings. The run timeout is capped at the largest delay a timer accepts. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
…ignal the CLI cannot ignore Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
…ind the data directory from the install path codex exec review refuses --base together with a prompt, so a review with a brief now runs the record's brief form: no --base, and a preface on stdin that names the base before the brief. A review without a brief keeps --base. Both forms are provider-record data. A command run through the Bash tool receives no CLAUDE_PLUGIN_DATA, so an installed copy derives the documented data directory from its own install path; a copy that is not installed still needs the variable. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
…tory binds The runner now resolves the home through the shared resolve-convention-home.sh helper, registered as a generated copy, and falls back to docs/conventions in an unbound repository. A pointer region the helper rejects is reported as a notice. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
binary_path is honored only from the personal layer when git does not track that file, and only when the file is named for the CLI; a tracked .local.yaml is reported and ignored, a wrongly named file is refused. Version and login queries run in the job's working directory. A cmd.exe binary whose own path is unsafe is refused before any query starts. Relative PATH entries are skipped, so a PATH-resolved binary is absolute. The warning that a reviewed change touches the CLI's project config is written before the CLI starts, and notices reach stderr on a refused run. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
…top a failed login check opening the breaker A review usage error, such as an unreadable brief file or an unknown option, now prints `<product>: failed (<reason>)` on stdout before the usage text on stderr. Text interpolated into the status line has its whitespace collapsed, so a denied command holding a newline cannot split it. A login check that cannot start or times out is a failed run and leaves the breaker alone; only a completed check that reports logged out opens it. A run refused after its job folder was created removes the folder. A convention-home helper that cannot run is named in a notice with the home read instead. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
… an unexpected error An error the runner did not anticipate, such as an unusable data directory or a corrupt breaker file, now ends a run or review with `<product>: failed (<reason>)` on stdout, the stack on stderr and exit 1, instead of a stack trace alone. The bin shim guards a rejected main() the same way for review. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
…cker typos reads the `nd` in `(st|nd|rd|th)` as a misspelling of `and`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
Council's Codex members and fanout's Codex surface now run the bare `external-agents review codex` command from the Bash PATH instead of locating the OpenAI Codex plugin's install path. `council-plan.mjs member` reads a script member's captured output and exit code as its council status and findings. It takes the first line shaped like the command's status line, because a background Bash capture merges stderr and the runner's notices print before it. A command the shell cannot find (exit 127, how a disabled external-agents plugin looks from Bash) is `skipped (external-agents not enabled)`. The tests build PATH from the stub folder and system folders only, so an external-agents enabled in the session running them is never the one found. Fanout's second, adversarial Codex run is dropped: the command has no adversarial mode, and a briefed Codex review is a council member with a brief. Deletes scripts/codex-review.sh and its test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
Fanout runs Codex twice again: plain, then with --brief-file naming its own bundled brief, skills/fanout/context/codex-adversarial-brief.md. Both runs are read through council-plan.mjs member, the parse council's Codex members use, and fanout's allowed-tools grant that verb and the bare external-agents review command. Eval cases 50 and 51 expect both runs and the new skip reason (case 51 still named the old one), and the no-reviewed-code grader samples cover both commands. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
`external-agents update <product>` reports the CLI's version and login state, captures the help commands the provider record lists into cli-facts/<product>/<version>/, runs the shared latest-release check with the configured update hint, HEAD-checks the docs index, claims the daily routing refresh, and checks that every config key the runner passes still appears in the CLI help or the record's config reference. It never installs anything; a present, logged-in CLI closes the breaker. Runner fixes: a job file that cannot be written once the CLI has started stops the CLI and reports failed; status, result and cancel turn an unexpected error into one stderr line; the data directory comes from the install path before CLAUDE_PLUGIN_DATA, which is used only when it is named for this plugin; a briefed review's preface names the merge base and the changed files, and the job records review_scope. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
…e config-path warning - tests/fixtures/providers/antigravity.yaml mirrors the antigravity record: `--disable-slash-commands` in always_args (a brief starting with `/` otherwise runs the user's agy slash commands) and `config_paths: [.agents/]` under review (agy loads workspace hooks and agents from there). - The config-path warning says "hooks and MCP servers", which fits both CLIs. - docs/upstream/t3dotgg-tstack.md points at `external-agents review codex` instead of the deleted codex-review.sh. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
/external-agents:codex drives the external-agents command: run a task or investigation in the foreground or background, resume a thread, review a branch, and read back status, result, cancel and update. The first reply line names the mode the job ran in; model and effort guidance is pointers to the Codex docs only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
agy prints its --help on stderr with an empty stdout, so update cached an empty file and still read ready. Help is now taken from stdout, or stderr when stdout is empty; a help command that prints nothing is a skipped fact that keeps line 1 at ready (check: cli facts). The agy fixture mirrors the record's help commands. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
The skill ships to every repository, so the prompt file goes to a gitignored scratch folder or the session scratchpad rather than an assumed .work/. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
…ly preface The codex skill's workflow-stage named a slug the cheat sheet does not know, so generate-cheatsheet failed; it now uses implement, as skills/antigravity does, and the sheet is regenerated. The agy fixture takes the read-only preface that let an investigate run finish on agy 1.5.14, and the envelope tests read the preface from the fixture instead of repeating it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
ADR 0117 replaces the official Codex plugin with one external-agents plugin and a skill per CLI. ADR 0118 ships its command in bin/ with a named exemption from the no-bin gate, superseding ADR 0021's bin/ verdict for this plugin only. ADR 0119 records the single runner over provider records, per-use sandbox ceilings, the done-versus-blocked rule and each CLI's stated limits. plugin-philosophy names the exception in the bin/ stance and uses the external-agents Codex skill as the cross-vendor advisor example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
…bs running A live Codex review of this branch found that codex.council.enabled was read by nothing: council's adapter now passes --council, and the runner reports the member skipped when that key is false. A status check could also mark a job failed while its runner was still recording the finished CLI; the job now records the runner's pid and stays running while it lives. PATH and POSIX candidate search skip a file without the execute bit. agy runs on its own default model when none is configured (owner decision of 2026-11-21): the fixture mirrors the record, and ADR 0119 says so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 277e5453e5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const env = { LOCALAPPDATA: local, APPDATA: appdata, PATH: onPath, PATHEXT: ".EXE;.CMD" }; | ||
| const found = resolveBinary(record, { platform: "win32", env }); | ||
| assert.equal(found.path, join(appdata, "npm", "demo.cmd")); |
There was a problem hiding this comment.
Isolate Windows filesystem assumptions in the resolution tests
On Linux, both Windows-resolution tests fail: their fixtures use native /tmp/... paths, but passing platform: "win32" makes resolveBinary normalize and join those paths with Windows separators before checking the real filesystem. The files therefore cannot be found, and found.path is null. Reproduced with the new foundations suite. Gate these cases to Windows or inject filesystem operations alongside the simulated platform; .github/workflows/pr-require-checks.yml runs their shell wrapper in the Ubuntu test-bash lane.
Useful? React with 👍 / 👎.
| const finished = await pending; | ||
| assert.equal(finished.job.status, "cancelled"); | ||
| assert.equal(readJob(w.dataDir, "no-repo", running.id).status, "cancelled"); | ||
| assert.notEqual(readJob(w.dataDir, "no-repo", running.id).exit_code, null, "the runner still records how the process ended"); |
There was a problem hiding this comment.
Accept signal termination in the cancellation assertion
On POSIX, killTree terminates the child with SIGKILL, so the child-process close callback receives a null exit code, which the runner correctly stores as exit_code: null. This assertion consequently fails even though cancellation succeeded and the job was recorded as cancelled. Reproduced by running this test alone on Linux. Assert the cancelled status and process termination, or record and assert the terminating signal instead of requiring a numeric exit code.
Useful? React with 👍 / 👎.
| assert.equal(readBreaker(w.dataDir, "codex"), null); | ||
| }); | ||
|
|
||
| test("a run refused after its job folder exists leaves no folder behind", async () => { |
There was a problem hiding this comment.
Gate the cmd.exe cleanup test to Windows
On Linux, this test launches fake-codex.cmd through the host shell during the login check, before reaching the unsafe job-path guard it intends to exercise. The login check fails, and run returns unavailable code 4 rather than the expected refusal code 2; this fails deterministically when run alone. Add the Windows-only guard used by the other executable .cmd tests, or provide a host-independent fixture that successfully answers the preliminary queries.
Useful? React with 👍 / 👎.
| - **Codex exiting 0 is not success.** Codex exits 0 when its sandbox refused the commands it | ||
| needed. The command reads Codex's event stream and records such a run `blocked-by-mode`, exits |
There was a problem hiding this comment.
Replace upstream behavior restatements with live pointers
AGENTS.md requires skill bodies to point at the live upstream source for volatile specifics instead of restating them. This paragraph nevertheless embeds Codex's exit behavior as a standing fact; the MCP/hooks paragraph below likewise restates upstream sandbox behavior, which an adjacent link does not cure under the referenced rule. Keep the plugin-owned instruction to report blocked-by-mode, and replace the upstream explanations with section-anchored pointers so future CLI changes do not leave stale behavioral guidance in the skill.
AGENTS.md reference: AGENTS.md:L204-L204
Useful? React with 👍 / 👎.
No related issue: owner-directed plan from a planning interview; no tracking issue was filed
Summary
Problem: the official Codex plugin (
codex@openai-codex) is behind Anthropic's skill and plugin guidance and behind its own CLI, and review:council reaches into its private script. Intent: "is that plugin up to date? Is it up to date even with its own CLI? Are there better approaches?"This PR adds the
external-agentsplugin, which replaces it with one skill per non-Anthropic coding-agent CLI behind one shared runner, and moves council's Codex member onto it. Upstream source, read as reference and not copied: openai/codex-plugin-cc. Decisions: ADR 0117 (replace, one plugin, a skill per CLI), ADR 0118 (bin/command and gate exemption; ADR 0021'sbin/verdict partly superseded), ADR 0119 (one runner over provider records, per-use sandbox ceilings, per-CLI limits).Fix
plugins/external-agents/: theexternal-agentscommand (bin/), the runner (run,review,status,result,cancel,update) over provider records (providers/codex.yaml), job files, CLI-facts cache, breaker, and/external-agents:codex. No hooks, agents orcommands/.okonly when nothing was denied; a denial isblocked-by-modeeven with exit 0. Every run passes its mode's sandbox andapproval_policy="never".binary_pathis read only from an untracked personal file.review:councilandreview:fanoutrunexternal-agents review codexas a bare command; council passes--council, socodex.council.enabled: falseskips the member. The install-path lookup is gone.scripts/check-plugin-manifest-presence.shrule 3 exemptsplugins/external-agents/binonly; claude.ai organization sync skips this plugin (org sync).docs/plugin-philosophy.md: thebin/stance names the exception, and the external-agents Codex skill is the cross-vendor advisor example.Verification
node --testonplugins/external-agents/tests/{runner,foundations,update}.test.mjs: 119 pass, 1 skipped (a POSIX-only execute-bit test that CI's Linux runner exercises);council-plan.test.sh: 46/46.validate-plugins.sh --only external-agents: passes except two themes bedroom sound tests (sound-mixer,sound-weather) that fail locally outside this PR's files.external-agents review codex --base origin/mainfrom this branch printedcodex: ran (reviewed 76 changed files against be755b31; 1 command blocked by read-only mode ...)with three P2 findings. Two are fixed here: the council opt-out was read by nothing, and astatuscould overwrite a finishing job.codex: ran.Related
/external-agents:antigravityand its record and defaults.codex@openai-codexand drops its marketplace entry.Known limits
--basewith a prompt. The runner pins the scope in the prompt (merge base and changed files) and prints the file count, but cannot prove that Codex read only the committed diff. Callers that run it already treat Codex as committed-only and name the dirty-tree gap.skills/codexhas noevals/evals.jsonyet (skill-quality WARN). Eval runs are paused until open issues and PRs reach zero (ADR 0116).Acceptance criteria
/external-agents:codexruns a task in the foreground or background, resumes a thread, and supportsstatus,result,cancelandupdate; it passes no model unless one is configured.reviewplugin locates another plugin's install path or names its internal scripts.commands/folder; each skill follows /playbooks:skill-authoring and/skill-quality:checkpasses.codex --helpand are cached by CLI version.docs/plugin-philosophy.mdnamesexternal-agentsas the cross-vendor advisor example.codex@openai-codexand drop its marketplace entry.update, with no fallback answer.--sandbox read-only; a request asking for write on a use whose configured ceiling is read-only stays read-only and names the config key.🤖 Generated with Claude Code
https://claude.ai/code/session_015XWwmfSVU9f2by5nqbM3Ak