This was written agentically; verify its assertions:
Slice 7 of epic #152. Design: docs/superpowers/specs/2026-08-13-token-benchmark-design.md, section "Staged plan" stage 1.
Why
Per-trial cost (USD, wall-clock, GraphQL) is unknown until measured; the design stages all spend behind go/no-go gates. The pilot is that measurement.
What
- Run 2 trials:
old@high (06d18cf), new@high (53e7e8c), alternating arm order per the design.
- Outputs: real per-trial cost, wall clock, GraphQL consumption; sets the trial timeout for later stages; validates harness, oracle injection, and rollout parsing end to end on real data.
- Go/no-go record: a short committed note (with ledger references) deciding whether stage 2 proceeds and at what budget.
Acceptance
🤖 Co-authored by Claude Fable 5.
This was written agentically; verify its assertions:
Slice 7 of epic #152. Design:
docs/superpowers/specs/2026-08-13-token-benchmark-design.md, section "Staged plan" stage 1.Why
Per-trial cost (USD, wall-clock, GraphQL) is unknown until measured; the design stages all spend behind go/no-go gates. The pilot is that measurement.
What
old@high(06d18cf),new@high(53e7e8c), alternating arm order per the design.Acceptance
is_drift_controlhandling exercised🤖 Co-authored by Claude Fable 5.