Make routine CI validate benchmarks without automatic full timing runs #321

Summary

Target v0.5.3. Keep comprehensive correctness checks, benchmark compilation and input/configuration validation automatic, while moving full Criterion acquisition out of ordinary PR/main runs into explicit performance investigations and the release workflow.

Current State

The main CI workflow runs just ci, including benchmark compilation without sampling. Separately, .github/workflows/benchmarks.yml automatically runs the unfiltered exact target on relevant PRs/main pushes with a 90-minute job limit, including diagnostic workloads excluded from the focused release selection. It compares PR results against a main artifact collected in an earlier hosted session.

Criterion warns that cloud-runner noise can produce apparent performance changes: official CI guidance. Such comparisons must not become authoritative release speedup claims.

Related: #301, #316 and the existing release contract in #311. This is future work; do not interrupt or invalidate the active v0.5.2 acquisition.

Proposed Changes

• Inventory all validation currently supplied by the automatic benchmark job before changing its triggers. Ensure native benchmark compilation and every required correctness/input/configuration gate remain automatic on supported features/platforms.
• Make full timing explicitly requested (for example, manual dispatch for a reviewed ref) and preserve the deliberate release acquisition path. Keep broad diagnostics separately opt-in.
• Define what the optional comparison can claim. For performance decisions, collect both compared revisions with equivalent inputs/harness/toolchain in a compatible measurement environment; do not present a previous hosted runner's artifact as a controlled baseline.
• Preserve workflow permissions, untrusted-PR boundaries, required status checks and publication evidence. Reconcile any required-check references so changing a trigger cannot leave merges waiting indefinitely.
• Document commands, triggers, outputs and costs in the owning guides. Do not add a new reduced-sampling policy merely to make the old automatic job shorter.

Benefits

Routine feedback stops paying for long diagnostic timing runs while retaining the checks that establish code and benchmark correctness.

Implementation Notes and Acceptance

• Record a guarantee-to-validator map for every removed automatic action; correctness tests and input gates are neither deleted nor weakened.
• Focused workflow/consumer tests show ordinary PR/main events perform no Criterion acquisition, explicit requests still work, and required statuses/release evidence remain valid.
• Validate applicable workflow, shell, Python and documentation surfaces through repository-required checks; measure any future runtime savings only with comparable completed runs.
• Astra checkpoint: routine trigger/recipe wiring does not require Astra. Request Astra before changing scientific comparison eligibility, required evidence or publication trust.
• No upstream feature is required to implement this consumer policy. Keep generic engine changes in separately linked upstream issues if a concrete gap emerges.

This issue can proceed independently of shared progress, resume and evidence-reuse implementation. It does not authorize running a benchmark campaign or dispatching CI during planning.

Workstream tracker and PR/dependency plan: #326.