Evals: automatically compare runs against prior or selected baseline #1385

Context

AgentV should make eval comparisons first-class. During WTG Braintrust/Phoenix UX benchmarking, Braintrust automatically compared a new experiment against the prior compatible experiment, and Phoenix exposes experiment/job status and comparison-oriented experiment tables.

Tracking Bead: av-2s7.10
Parent research Bead: av-2s7
Related WTG Braintrust Bead: av-2s7.9

Primary references:

• Braintrust experiments are immutable snapshots meant to compare over time: https://www.braintrust.dev/docs/evaluate/run-evaluations
• Braintrust SDK supports explicit comparison experiment IDs and experiment tags: https://www.braintrust.dev/docs/reference/sdks/typescript/3.9.0/typescript
• Phoenix background experiment jobs expose lifecycle status, progress, stop, and resume behavior: https://arize.com/docs/phoenix/datasets-and-experiments/how-to-experiments/run-experiments-in-background

Problem

AgentV can run evals and inspect results, but users need an obvious comparison workflow:

• After a run completes, compare against a relevant prior run automatically when possible.
• Allow manual comparison against any compatible run.
• Surface improvements/regressions in aggregate and per test case.
• Preserve tags/metadata so comparisons can be grouped and filtered by eval source, target, branch, CI/manual origin, and feature area.

Without this, users must remember which prior run matters and manually invoke comparison tooling, which is weaker than the Braintrust/Phoenix UX benchmark.

Proposed Behavior

Default comparison:

• On completed run detail, auto-select a compatible prior baseline if one exists.
• Baseline selection should consider eval identity, target/model, dataset/case IDs, branch/git ancestry, tags/metadata, and timestamp fallback.
• Show score deltas, improvements/regressions, duration/token/cost deltas, and comparison experiment/run identity.

Manual comparison:

• Add Compare against... from run detail and recent runs.
• Let users choose any compatible run, with warnings for incompatible case sets/targets.
• Show row-level diffs and aggregate changes.

Run metadata:

• Persist tags/metadata on runs.
• Make tags filterable in recent runs and comparison flows.
• Recommended tags include framework/source, target, branch, CI/manual, feature, and eval family.

Relationship to jobs UX:

• Recent Runs should include in-progress runs with status.
• A dedicated Jobs page should wait until AgentV has durable queue/background worker semantics: queued/running/stopped/resumable/cancelled/retryable jobs.

Acceptance Criteria

• Completed eval run automatically shows comparison against a prior compatible run when one exists.
• Run detail offers Compare against... for any compatible run.
• Recent runs list exposes baseline identity and delta summary.
• Row-level improvements/regressions are visible.
• Tags/metadata are persisted and filterable.
• UX docs or notes link to the Braintrust/Phoenix benchmarking evidence from av-2s7.9.