[feat] Open improvement PRs proven by a before/after benchmark #1249

Parent: #818
Related: #1137

Goal

Turn repeated scorer failures into a PR that changes the team's prompts, skills or agents, and prove in that PR that the change fixes those failures without breaking anything, on every tool the team uses. #818 lists improvement PRs in its design; #1085 and #1137 leave them out.

Example

review-quality failed 6 times for agent reviewer (labels from #1085 and #1178)

teamai digest --json            → 6 distinct failing subjects, no reasoning
workflow improve-agent          → edits agents/reviewer.md, opens PR (deliver: pr)
  regression tasks              → 6 new tasks in benchmarks/regressions/tasks/ (#1180)
  benchmark (#1181 arms)        → current assets vs this PR, one suite run per tool

PR body
  arm        tool     pass    cost
  current    claude   0/6     …
  proposed   claude   5/6     …
  current    codex    1/6     …
  proposed   codex    5/6     …
  regression suite (41 tasks): no new failures

Scope

• Read the failures. teamai digest gains --json, an option on an existing command, printing [feat] Show scorer results in the dashboard #1182's aggregation with its deduplication and its rubric, source and judge cohorts: failing labels with their subjects, never reasoning, benchmark labels left out.
• Open the PR. The guide ships an example improve-agent workflow, run from a checkout of the team repo and started from CI ([feat] Deliver workflow branches as pull requests #1090): a command step reads that output and picks a bounded set of distinct failures, a prompt step proposes changes to prompts/, skills/ or agents/ and lists the subjects it addresses, and deliver: pr ([feat] Deliver workflow branches as pull requests #1090) opens the PR. It checks open proposals for those subjects first and updates one rather than opening a duplicate. No scheduler and no new command.
• Say how far the scorer can be trusted. The PR body shows, for each failing scorer it addresses, the agreement of its model judge with hand labels from [feat] Compare scorers with hand labels, including on pushed sessions #1269's view: paired, agree, and both kinds of disagreement.
• Prove it. Each addressed failure becomes a regression task for [feat] Run benchmark suites as workflows #1180: from a session, its recorded start commit (feat(session): safe session reads and copies for workflows, drivers, scorers, and benchmarks (#593 follow-up) #1135); from a workflow step, the commit the pass started from. A task uses the scorer's check when [feat] Score what a workflow run delivered #1247 has added it and the scorer sets one, otherwise successCriteria. A session that started with tracked changes is refused as a task, as [Proposal] Score agent sessions and run benchmarks as workflows #1137's original proposal says. The workflow runs a benchmark with [feat] Compare team asset sets in benchmarks #1181 arms, the current team assets against the PR's, once per tool the team syncs to (an arm changes one key, [feat] Run benchmark suites as workflows #1180), over the new tasks and the existing regression suite. The PR body carries the before/after table, with cost per arm in the tool's own unit when known, and the regression result.
• The regression tasks are committed in the same PR, so every later change is measured against every failure already fixed.

Out of scope

Merging automatically, changing application code, scheduling, and improving from a single failure.

Dependencies

• Blocked by: [feat] Show scorer results in the dashboard #1182, [feat] Deliver workflow branches as pull requests #1090, [feat] Compare team asset sets in benchmarks #1181, [feat] Compare scorers with hand labels, including on pushed sessions #1269
• Unblocks: none

Acceptance criteria

• teamai digest --json lists failing labels per scorer and agent with their subjects, no reasoning, and no benchmark labels.
• Running the example twice on the same failures updates one PR instead of opening two.
• The PR adds one regression task per addressed failure, and refuses a session that started with tracked changes.
• The PR body has the before/after table for each team tool and the regression suite result; an arm with an unknown cost says so instead of showing zero.

Verification

Focused digest, task-from-failure and dedup tests; regenerate the commands reference (npx vitest run commands-reference -u); build/typecheck/lint; one real-CLI run of the example on a fixture team repo with a fake provider and two tools. Bilingual guide with the example.