ci: merge bench-compile-gate + bench-report into single bench job#395
Merged
Conversation
Two related CI optimizations:
1. Compile work happens once. The former bench-compile-gate and
bench-report jobs each ran cargo bench in the release-ci profile
under independent Swatinem prefix-keys (�ench-compile-gate vs
�ench-report), compiling the six bench targets twice per PR.
Replaced with a single �ench job that:
- Runs cargo bench --no-run --profile release-ci as a BLOCKING gate
step (no continue-on-error) failure here fails the job and the
merge gate, preserving the bench-API-drift signal.
- Then runs fixtures + cargo bench + perf-summary + artifact-upload
as steps marked continue-on-error: true, reusing the just-compiled
target/release-ci binaries without recompile. A slow or regressed
bench shows as a red X step inside a green job rather than the
previous job-level continue-on-error (which hid regressions behind
an advisory label).
Net effect: one fewer release-ci compile per PR, with strictly more
visible bench-budget regressions.
2. Post-merge tree-hash skip now applies. The former bench jobs only
checked code == true in their if: clauses, so squash-merges to
main re-ran the bench compile even when the post-squash tree was
byte-identical to a passing PR head. The unified bench job now
carries the standard code == true && skip != true clause used by
every other heavy job, restoring the push-to-main tree-hash skip
for bench work.
test-gate
eeds: list and result-check updated from �ench-compile-gate
to �ench. No other workflow callers reference either old name.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Replaces
bench-compile-gate+bench-reportwith a singlebenchjob, and fixes the post-merge tree-hash skip for bench work.Why
1. Compile work was duplicated per PR
The two jobs each ran
cargo bench --profile release-ciunder independent Swatinemprefix-keys (bench-compile-gateandbench-report), compiling the six[[bench]]targets twice per PR. The release-ci profile is fast (lto=off, codegen-units=16), but a cold compile is still ~8-12 min. Two independent caches also mean two independent cache-state miss windows.2. Post-merge tree-hash skip was bypassed
ci-gate-inputscomputes askipboolean that istruewhen the post-squash tree onmainis byte-identical to a PR head whose ci.yml run already concludedsuccess. Every heavy job'sif:carriescode == 'true' && skip != 'true'exceptbench-compile-gateandbench-report, which only checkedcode == 'true'. Result: every push-to-main re-ran the bench compile even on tree-identical squash-merges.How
Single
benchjob structure:continue-on-errorcargo bench --no-run --profile release-citruecargo bench ... --output-format benchertruetrue+if: always()true+if: always()Step-level
continue-on-error: true(instead of job-level) means a slow or budget-busting bench shows as a red step inside an otherwise-green job. This is strictly more visible than the prior job-level continue-on-error, which hid regressions behind a job-level "advisory" label.test-gate'sneeds:list and result check were updated frombench-compile-gatetobench. No other workflow callers reference the old names.Trade-offs considered
Two jobs with
needs:chain + sharedprefix-key. Cleaner separation, but Swatinem only saves onmain(save-if: github.ref == 'refs/heads/main'), so on PR runsbench-reportwould still do a full release-ci recompile defeating the goal. Flippingsave-if: truewould write to the shared cache on every PR run (size churn + eviction risk).Critical-path impact. Negligible. Both former bench jobs were strictly under the slowest of
{build matrix, native-e2e, installer-e2e}, so neither was on the critical path. The merged job's compile-then-run sequence (~3 min compile warm + ~1 min bench + ~10 s summary) is still well under critical-path jobs.Cache migration
The old
bench-compile-gateandbench-reportSwatinem caches are now orphaned and will LRU-evict naturally. The newprefix-key: benchstarts cold for the first main-branch run.