Skip to content

ci: merge bench-compile-gate + bench-report into single bench job#395

Merged
dryotta merged 1 commit into
mainfrom
chore/merge-bench-ci-jobs
May 16, 2026
Merged

ci: merge bench-compile-gate + bench-report into single bench job#395
dryotta merged 1 commit into
mainfrom
chore/merge-bench-ci-jobs

Conversation

@dryotta

@dryotta dryotta commented May 16, 2026

Copy link
Copy Markdown
Owner

What

Replaces bench-compile-gate + bench-report with a single bench job, and fixes the post-merge tree-hash skip for bench work.

Why

1. Compile work was duplicated per PR

The two jobs each ran cargo bench --profile release-ci under independent Swatinem prefix-keys (bench-compile-gate and bench-report), compiling the six [[bench]] targets twice per PR. The release-ci profile is fast (lto=off, codegen-units=16), but a cold compile is still ~8-12 min. Two independent caches also mean two independent cache-state miss windows.

2. Post-merge tree-hash skip was bypassed

ci-gate-inputs computes a skip boolean that is true when the post-squash tree on main is byte-identical to a PR head whose ci.yml run already concluded success. Every heavy job's if: carries code == 'true' && skip != 'true' except bench-compile-gate and bench-report, which only checked code == 'true'. Result: every push-to-main re-ran the bench compile even on tree-identical squash-merges.

How

Single bench job structure:

Step continue-on-error Notes
Setup (checkout, toolchain, cache, apt, stage CLI placeholder)
cargo bench --no-run --profile release-ci none Blocking compile gate. Failure here fails the job and the merge gate.
Generate bench fixtures true
cargo bench ... --output-format bencher true Reuses target/release-ci binaries no recompile.
Write perf summary true + if: always()
Upload bench output true + if: always()

Step-level continue-on-error: true (instead of job-level) means a slow or budget-busting bench shows as a red step inside an otherwise-green job. This is strictly more visible than the prior job-level continue-on-error, which hid regressions behind a job-level "advisory" label.

test-gate's needs: list and result check were updated from bench-compile-gate to bench. No other workflow callers reference the old names.

Trade-offs considered

  • Two jobs with needs: chain + shared prefix-key. Cleaner separation, but Swatinem only saves on main (save-if: github.ref == 'refs/heads/main'), so on PR runs bench-report would still do a full release-ci recompile defeating the goal. Flipping save-if: true would write to the shared cache on every PR run (size churn + eviction risk).

  • Critical-path impact. Negligible. Both former bench jobs were strictly under the slowest of {build matrix, native-e2e, installer-e2e}, so neither was on the critical path. The merged job's compile-then-run sequence (~3 min compile warm + ~1 min bench + ~10 s summary) is still well under critical-path jobs.

Cache migration

The old bench-compile-gate and bench-report Swatinem caches are now orphaned and will LRU-evict naturally. The new prefix-key: bench starts cold for the first main-branch run.

Two related CI optimizations:

1. Compile work happens once. The former bench-compile-gate and
   bench-report jobs each ran cargo bench in the release-ci profile
   under independent Swatinem prefix-keys (�ench-compile-gate vs
   �ench-report), compiling the six bench targets twice per PR.
   Replaced with a single �ench job that:

   - Runs cargo bench --no-run --profile release-ci as a BLOCKING gate
     step (no continue-on-error)  failure here fails the job and the
     merge gate, preserving the bench-API-drift signal.
   - Then runs fixtures + cargo bench + perf-summary + artifact-upload
     as steps marked continue-on-error: true, reusing the just-compiled
     target/release-ci binaries without recompile. A slow or regressed
     bench shows as a red X step inside a green job rather than the
     previous job-level continue-on-error (which hid regressions behind
     an advisory label).

   Net effect: one fewer release-ci compile per PR, with strictly more
   visible bench-budget regressions.

2. Post-merge tree-hash skip now applies. The former bench jobs only
   checked code == true in their if: clauses, so squash-merges to
   main re-ran the bench compile even when the post-squash tree was
   byte-identical to a passing PR head. The unified bench job now
   carries the standard code == true && skip != true clause used by
   every other heavy job, restoring the push-to-main tree-hash skip
   for bench work.

test-gate
eeds: list and result-check updated from �ench-compile-gate
to �ench. No other workflow callers reference either old name.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@dryotta dryotta merged commit 3be43cf into main May 16, 2026
28 of 30 checks passed
@dryotta dryotta deleted the chore/merge-bench-ci-jobs branch May 16, 2026 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant