feat(activation): drop the Claude-only early-stop restriction, pin coder-eval 0.9.1

[mohsen-uipath]

What

coder-eval 0.9.1 (UiPath/coder_eval#64) extends cooperative early stop to the codex and antigravity agents. Before it, an armed activation run on those harnesses hard-errored at resolution (EarlyStopConfigError), forcing unarmed runs at ~2-3x cost — or no run, hence no suite aggregates and no Activation: NN% avg recall line in their nightly reports.

File	Change
tests/.coder-eval-version	0.9.0 → 0.9.1 — only functional change; drives the CLI wheel AND the ghcr.io/uipath/coder-eval-agent image tag across the consuming workflows. 0.9.1 wheel + image confirmed published: the full smoke suite passed on this PR under the new pin
tests/experiments/activation.yaml	Drop stale HARNESS CONSTRAINT comment; comment version floor >= 0.8.10 → >= 0.9.1; parsed YAML byte-identical to main
tests/README.md	Version floor + Claude-only clause → >= 0.9.1
tests/tasks/activation/README.md	Drop "Claude Code agent only" instruction; version floor >= 0.8.10 → >= 0.9.1 + name the below-floor failure (EarlyStopConfigError); unarmed-run note → "optional for routine checks, required for authoritative precision"
tests/scripts/activation_gate.py	Comment-only: last remaining >= 0.8.10 floor → >= 0.9.1 (review finding)

Claude path untouched: stop_early: true, stop_when: auto, max_turns: 3 unchanged; experiment resolves to the identical task.

Verification

• Static: all 1,144 task YAMLs coder-eval plan-valid under 0.9.1; armed activation resolves on all three harnesses (hard error on 0.9.0); no tracked text still claims early stop is Claude-only.
• Live, armed activation on codex (24 rows, gpt-5.6-terra): zero EarlyStopConfigError, zero crashes/retries; 22 positives pass-stopped at turn 1 (score 1.0), 1 misfire fail-stopped correctly, negative ran to cap and passed; suite.json with per-skill P/R/F1 produced.
• Armed vs unarmed A/B on codex (24 rows each): armed = 61% fewer tool calls (36 vs 92), ~65-70% lower cost and wall-clock.
• Armed vs unarmed A/B on antigravity (24 rows each, gemini-3.5-flash): armed = 24/24 SUCCESS, all 23 positives pass-stopped at turn 1, 33 tool calls, ~$0.19, ~7.5s/row. Unarmed = 430 tool calls, ~$8.97, and 8 rows ERRORed on the experiment's 120s turn timeout — armed mode is what keeps the suite inside its limits on this harness, not just cheaper.
• Claude regression control (armed, 0.9.1): behavior identical to before — 22/23 pass-stops at turn 1, complete token accounting, zero crashes; the one miss was a model-level skill misfire (benchmark signal).
• Known coder_eval follow-up (not this PR): codex and antigravity rows stopped at tool-call 1 record total_token_usage: null (interrupt lands before the first usage-bearing message). Cost reporting only; verdicts/metrics unaffected. Claude records fully.

Nightly dry-run

UiPath.Coder_Eval.Nightly run 20260729.17: harness=codex, model=gpt-5.6-terra, dryRun=true, skillsRef = this branch (6eb85b42), pin → 0.9.1. Mirrors reference run 12799116 (claude-code) with only branch + harness changed.

• GATE: PASS — 22 skills-suite tasks (21 ok / 1 fail, 95% >= 90% floor), 0 errors.
• Activation ran armed on codex — zero EarlyStopConfigError. 24 rows: 21 pass-stopped at sdk_turn=1 tool_call=1, 2 fail-stopped at turn 1 on genuine misfires (uipath-functions-016, uipath-review-030, score 0.957), negative ran to cap. All three stop polarities exercised.
• Suite rollup produced: skill-activation pass_rate=91.7% (22/24) gate=PASS — the aggregates the Activation: line derives from now exist for codex.
• Pre-existing, unrelated: the runner's composed Slack line says Activation: 0% avg recall across 24 skills — the claude reference run shows the identical 0% next to a 100% rollup, so it is a composer bug in the external runner, independent of this change and harness. Flagged to the runner owner.

Notes

• The external nightly runner can now drop -D run_limits.stop_early=false for codex/antigravity activation (coordinated separately).
• Unarmed runs remain authoritative for precision (pass-stop can hide a later distractor misfire); guidance retained in docs.

[github-actions]

Claude finished @mohsen-uipath's task in 1m 55s —— View job

---

PR Review: feat(activation): drop the Claude-only early-stop restriction, pin coder-eval 0.9.1

• Read review criteria and project conventions
• Read full PR diff
• Read each changed file in full for context
• Check for cross-skill references, secrets, broken links
• Post review findings

---

Summary

Bumps tests/.coder-eval-version from 0.9.0 to 0.9.1 and removes documentation claiming cooperative early-stop is Claude Code-only, now that coder-eval 0.9.1 extends the seam to codex and antigravity agents. Four files changed, all in tests/ — version pin, experiment YAML comments, and two READMEs. No functional YAML changes (parsed experiment is byte-identical).

Change-by-Change Review

1. tests/.coder-eval-version

Severity: OK

0.9.0 → 0.9.1. Clean version bump, single line.

2. tests/README.md

Severity: OK

Version floor updated from >= 0.8.10 + Claude-only clause to >= 0.9.1. The Claude-only restriction (--type codex/antigravity must add -D run_limits.stop_early=false) is correctly removed. Resulting sentence reads cleanly.

3. tests/experiments/activation.yaml

Severity: Low

The stale HARNESS CONSTRAINT comment block (lines 24–29 on main) is correctly deleted. The retained note about unarmed runs being authoritative for precision is accurate and well-worded.

One residual mention at line 34: "a true negative, or an unarmed codex/antigravity run" — this parenthetical still singles out codex/antigravity as needing unarmed mode, which is the restriction this PR removes. Now that all harnesses support armed early-stop, this should read something like "a true negative, or any unarmed run" to avoid implying codex/antigravity still require special treatment.

Fix this →

4. tests/tasks/activation/README.md

Severity: OK

Both changes are correct:

• Line 39: "Claude Code agent only: for --type codex or --type antigravity add -D run_limits.stop_early=false" sentence removed cleanly.
• Line 49: "required for codex/antigravity and for authoritative precision" → "optional; authoritative for precision" — accurate rewording.

The remaining (coder_eval >= 0.8.10) parenthetical on line 39 refers to when stop_when: auto was introduced, not the overall minimum version, so it's technically correct. Could be bumped to >= 0.9.1 for consistency with tests/README.md, but not incorrect as-is.

What's Missing

• Stale codex/antigravity comment in tests/experiments/activation.yaml:34 — the only remaining text in the changed files that still implies codex/antigravity need unarmed mode. Minor, but inconsistent with the PR's stated goal of a repo-wide sweep.

Area Ratings

Area	Status	Notes
Frontmatter	N/A	No skill frontmatter changed
E2E Tests	OK	No test tasks modified — comment-only changes
Skill Body	N/A	No skills changed
References & Assets	OK	All relative links resolve (tasks/activation/README.md ✓)
Repo Hygiene	OK	No secrets, no cross-skill refs, changes scoped to tests/

Issues for Manual Review

None found. The PR is scoped to documentation and a version pin — no behavioral changes to verify beyond confirming coder-eval 0.9.1 supports early-stop on codex/antigravity (verified per the PR description's live-run evidence).

Conclusion

Clean, well-scoped PR. One Low nit: the comment at tests/experiments/activation.yaml:34 still singles out codex/antigravity for unarmed runs, which is the restriction being lifted. Otherwise, approve.

---

[uipreliga]

Code Review Summary

Reviewed with Gemini 3.1 Pro + GPT-5.3-Codex in parallel, plus an Opus sub-agent. The central claim was also verified directly against the coder_eval source.

Premise verified. coder_eval 0.9.1 (commit b849421, tag v0.9.1 pushed to origin) sets supports_cooperative_stop = True on both CodexAgent and AntigravityAgent, and early_stop.py:161 now enumerates (claude-code, codex, antigravity) in its EarlyStopConfigError. The removed "Claude-only" constraint is genuinely obsolete, not prematurely relaxed. noop_agent is the only remaining holdout, and no task uses it.

Critical / High

None.

Medium

• Stale >= 0.8.10 floor survives the commit that was supposed to remove it — tests/scripts/activation_gate.py:64-65. Flagged independently by all three reviewers. grep -rn '0\.8\.10' returns exactly one hit repo-wide, and it is in the one file commit 31522929b ("reconcile stale >= 0.8.10 floors with the 0.9.1 requirement") did not touch. The sentence is technically still true — the arming contract did land in 0.8.10 — but it contradicts the three sites this PR just rewrote, which is precisely the drift the commit set out to kill. No functional risk: the gate job always runs claude-code, never --type.

Low

• "optional; authoritative for precision" reads as self-contradictory — tests/tasks/activation/README.md:49. Flagged by Codex and Opus. The old clause was "required for codex/antigravity and for authoritative precision"; deleting the harness half left a dangling "optional" sitting next to a claim that you need it for real numbers. An operator can read this as "skip it" while still treating armed precision as definitive. Suggested wording: "optional for routine checks, but required if you need authoritative precision".
• Ragged rewrap left by the deletions — tests/experiments/activation.yaml:33-36 ("an unarmed codex/antigravity run" → "any unarmed run" shortened line 34 without reflowing the paragraph) and the same class at :23-25 (">= 0.8.10's any-engagement latch" → "the any-engagement latch"). Cosmetic, but these comment blocks are load-bearing documentation.
• The failure symptom is now undocumented in all three activation docs — activation.yaml:14, tests/README.md:181, tasks/activation/README.md:39. The deleted paragraph was the only place naming EarlyStopConfigError; all three now state a bare floor and nothing about what happens below it. Narrow but real trigger: run-coder-eval.yml:35-38 exposes a coder_eval_version override input, and local runs are wheel-choice-by-hand. Opus-only finding — both external models were content with the bare floor. Defensible to decline if the pin is treated as authoritative.
• Version literal hand-copied in four places (three docs + activation_gate.py) when tests/.coder-eval-version is already the single source of truth. This PR is itself the evidence the pattern does not hold. Note the semantics differ slightly: stop_when: auto landed in 0.8.10; only codex/antigravity support needs 0.9.1 — so a flat >= 0.9.1 over-specifies the claude-only case.
• tests/README.md:181 and tasks/activation/README.md:39 duplicate the same 4-sentence pass-stop/fail-stop/deferred-fail-stop explanation and were edited in lockstep here. Codex explicitly discarded this as non-actionable; the redundancy is probably tolerable for two audiences, but the lockstep edit is the tell.

Blast radius — worth a line in the PR description

tests/.coder-eval-version is repo-wide, consumed by five places, none activation-only. Beyond wheel installs it is also a docker image tag: smoke-skills.yml:345,356 pull ghcr.io/uipath/coder-eval-agent:0.9.1.

• Pre-merge check: confirm the 0.9.1 GHCR image is published and the wheel is on the feed. docker-publish.yml skips chore(release): commits, so that tag comes solely from release.yml — a partial failure there breaks every smoke/nightly run at docker pull, not at a task.
• smoke-skills.yml:130 matches ^tests/\.coder-eval-version$, so this PR fans out the full non-Windows smoke suite. That is the desired outcome — it is the only thing exercising the bump. Do not apply ci:skip-infra-trigger.
• Historically these bumps shipped standalone (dd97a3f33, 664bbe389, eefe56761). Bundling is defensible here since the doc change is false without 0.9.1, but it couples a docs revert to an engine revert.

Risk is low: 0.9.0 → 0.9.1 is one feature commit touching codex_agent.py, antigravity_agent.py, and one error string. Nothing on the claude-code path moves.

Positive observations

• Precisely-scoped version delta — 0.9.1 is the exact release that unblocks the change, no incidental behavior rides along.
• The removal is complete rather than partial: repo docs and the engine's own error message now agree on all three agents.
• Commit 2aedbb177 avoided orphaning the disarm flag — deleting the constraint block would have removed the only mention of -D run_limits.stop_early=false from activation.yaml; folding it into the Note preserved it.
• Genuinely subtractive diff: deferred-fail-stop semantics, the decided-is-invalid note, and the cost model all survived verbatim.
• No workflow edits needed — the pin file was already the single consumer point, and no workflow ever hardcoded the disarm flag.

Overall

Correct, well-scoped, and the premise checks out against upstream source. The one substantive defect is the >= 0.8.10 comment in activation_gate.py:64 that the reconcile commit was meant to catch; everything else is prose polish. Ship once that is fixed and the 0.9.1 GHCR image is confirmed published.

🤖 Generated with Claude Code

[uipreliga]

Fix what you agree with and 🚢

[mohsen-uipath]

Thanks for the thorough review

• Medium (activation_gate.py stale 0.8.10): fixed → 0.9.1; repo-wide grep now clean.
• Precision wording: reworded per your suggestion.
• Comment reflow: done; parsed YAML still identical to main.
• EarlyStopConfigError: now named in tasks/activation/README.md (kept to one site).
• Blast radius: PR description now notes the GHCR image-tag consumer. Wheel + image confirmed published — smoke passed on this PR under the 0.9.1 pin.

Declined as out of scope of this PR: version-literal templating (will file a follow-up) and the README dedup (two audiences; the Codex reviewer agreed).