ci: 👷 aggregate matrix results into a stable check - #336
Merged
Conversation
The `package-tests` and `flake` matrices only ever report per-combination check names, so branch protection has to be re-edited whenever a lane moves. Add an `if: always()` aggregation job to each workflow that publishes a fixed name and fails unless the matrix succeeded or was skipped. Assisted-by: GitHub Copilot:claude-opus-5
Panadestein
marked this pull request as ready for review
September 3, 2026 12:33
Panadestein
requested review from
diagonal-hamiltonian,
fpietra,
ludmilaasb and
robertodr
as code owners
September 3, 2026 12:33
|
Docs preview: https://pr-336.monoprop-docs.pages.dev |
robertodr
reviewed
Sep 3, 2026
robertodr
reviewed
Sep 3, 2026
robertodr
reviewed
Sep 3, 2026
Member
Author
Exactly this. Then we can change the set of jobs and rule won't break. |
Co-authored-by: Roberto Di Remigio Eikås <robertodr@users.noreply.github.com> Signed-off-by: Ramón L. Panadés-Barrueta <rpana92@gmail.com>
Co-authored-by: Roberto Di Remigio Eikås <robertodr@users.noreply.github.com> Signed-off-by: Ramón L. Panadés-Barrueta <rpana92@gmail.com>
robertodr
approved these changes
Sep 3, 2026
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #336 +/- ##
=======================================
Coverage 97.70% 97.70%
=======================================
Files 14 14
Lines 742 742
Branches 98 98
=======================================
Hits 725 725
Misses 12 12
Partials 5 5
Flags with carried forward coverage won't be shown. Click here to find out more. |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



🤖 AI text below 🤖
Summary
A matrix job reports one check run per combination, so its check names encode the matrix
dimensions (
Test [mpi:off] ubuntu-26.04 py3.11 default,Flake on macos-15, …). Anyruleset that requires those checks has to be re-edited whenever a lane is added, dropped
or renamed, and a required check that no longer exists silently stops gating.
This adds the aggregation pattern: each matrix workflow ends with a tiny job that runs
with
if: always(),needsthe matrix, and fails unless the matrix result wassuccessor
skipped. The result is a single stable check name per workflow that branchprotection can require, independent of the matrix dimensions.
Changes
test.yml: newtestsjob, check name All tests, gatingpackage-tests.nix.yml: newflake-resultjob, check name All flake checks, gatingflake.ubuntu-slimand onlyecho+case, socheck-workflow-commandsissatisfied without a new recipe.
docs/content/docs/testing.mdx: documents the pattern and the two names.Follow-up needed on the repository settings
The ruleset must be updated to require All tests and All flake checks, and the
per-combination entries removed — otherwise this only adds two green checks.
Two caveats worth a look before this leaves draft:
nix.ymlhas apaths:filter onpull_request. When the filter does not match, thewhole workflow is skipped and All flake checks never reports, which a ruleset treats
as pending. Either leave that one non-required, or drop the path filter.
flakesetscontinue-on-errorformacos-15, so a macOS failure still yields a jobresult of
success. That behaviour is unchanged by this PR, but the aggregate inherits it.Not touched:
qa-analysis.yml'scoverage-runmatrix already feedscode-coverage,which has a stable name, and
deploy.yml's wheel matrix does not gate PRs.Checklist
docs/,CONTRIBUTING.md) if neededCHANGELOG/ release notes updated if applicable — n/aAI/LLM disclosure