fix(tests): unstarve three uipath-agents smoke tasks (max_turns -> 60) - #2413
fix(tests): unstarve three uipath-agents smoke tasks (max_turns -> 60)#2413mjnovice wants to merge 1 commit into
Conversation
… -> 60) Three smoke tasks keep hitting MAX_TURNS_EXHAUSTED with high partial scores and zero failed commands — pure turn starvation, same disease as #2349/#2234: - skill-agents-coded-is-smoke (max_turns: 30, below the smoke experiment default of 40): starved twice in a row on PR #1694's runs at score 0.84 with only the doc-read criterion left. - skill-agent-context-prompt-reference (max_turns: 30): starved on PR #1694's run AND on PR #1801's run — mid-flow at score 0.41 after creating agent.json + the context resource. - skill-agent-coded-conversational-langgraph (inherited 40): starved on PR #1694's rerun. Raise all three to expected_turns 40 / max_turns 60, matching the precedent set by #2349 for eval_custom_evaluator. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Claude finished @mjnovice's task in 2m 51s —— View job Coder-eval task lint (advisory)3 task YAMLs changed; verdict counts: 0 Critical, 0 High, 0 Medium, 0 Low, 3 OK. Rubric: .claude/commands/lint-task.md. This check is advisory and never blocks merge. Evidence of passing run❌ High — PR body does not claim the changed tasks have been run and passed with the new
(Honor-system — a verbal claim is sufficient.) Per-task lint
|
|
Claude finished @mjnovice's task in 1m 53s —— View job Reviewing PR #2413
SummaryRaises Change-by-Change Review1.
|
| Area | Status | Notes |
|---|---|---|
| Frontmatter | N/A | No skill frontmatter changed |
| E2E Tests | OK | Only run_limits tuned — tags, criteria, prompts untouched |
| Skill Body | N/A | No skill docs changed |
| References & Assets | N/A | No references or assets changed |
| Repo Hygiene | OK | No secrets, no unrelated changes, scoped to the three affected tasks |
Issues for Manual Review
None found.
Conclusion
Clean, well-scoped fix. The 40/60 values are consistent with the existing pattern across other uipath-agents tasks that needed similar relief (bindings_sync, edit_roundtrip). The starvation evidence in the PR description is thorough. Approve.
Why
The uipath-agents smoke suite is flaking below the 95% gate because three tasks keep hitting
MAX_TURNS_EXHAUSTEDwith high partial scores and zero failed commands — pure turn starvation, the same disease #2349 and #2234 fixed for other tasks.Evidence from PR #1694's three smoke runs (26/29, 27/29, 27/29 — a different failure set each time) plus PR #1801's run:
skill-agents-coded-is-smokeskill-agent-context-prompt-referenceskill-agent-coded-conversational-langgraphWhat
Raise all three to
expected_turns: 40/max_turns: 60, matching #2349's precedent foreval_custom_evaluator. No prompt or criteria changes.🤖 Generated with Claude Code