Skip to content

ci: 👷 aggregate matrix results into a stable check - #336

Merged
robertodr merged 3 commits into
mainfrom
ci/matrix-result-gates
Sep 3, 2026
Merged

ci: 👷 aggregate matrix results into a stable check#336
robertodr merged 3 commits into
mainfrom
ci/matrix-result-gates

Conversation

@Panadestein

Copy link
Copy Markdown
Member

🤖 AI text below 🤖

Summary

A matrix job reports one check run per combination, so its check names encode the matrix
dimensions (Test [mpi:off] ubuntu-26.04 py3.11 default, Flake on macos-15, …). Any
ruleset that requires those checks has to be re-edited whenever a lane is added, dropped
or renamed, and a required check that no longer exists silently stops gating.

This adds the aggregation pattern: each matrix workflow ends with a tiny job that runs
with if: always(), needs the matrix, and fails unless the matrix result was success
or skipped. The result is a single stable check name per workflow that branch
protection can require, independent of the matrix dimensions.

Changes

  • test.yml: new tests job, check name All tests, gating package-tests.
  • nix.yml: new flake-result job, check name All flake checks, gating flake.
  • Both run on ubuntu-slim and only echo + case, so check-workflow-commands is
    satisfied without a new recipe.
  • docs/content/docs/testing.mdx: documents the pattern and the two names.

Follow-up needed on the repository settings

The ruleset must be updated to require All tests and All flake checks, and the
per-combination entries removed — otherwise this only adds two green checks.

Two caveats worth a look before this leaves draft:

  • nix.yml has a paths: filter on pull_request. When the filter does not match, the
    whole workflow is skipped and All flake checks never reports, which a ruleset treats
    as pending. Either leave that one non-required, or drop the path filter.
  • flake sets continue-on-error for macos-15, so a macOS failure still yields a job
    result of success. That behaviour is unchanged by this PR, but the aggregate inherits it.

Not touched: qa-analysis.yml's coverage-run matrix already feeds code-coverage,
which has a stable name, and deploy.yml's wheel matrix does not gate PRs.

Checklist

  • Tests added or updated to cover the changes — n/a, CI wiring only
  • Documentation updated (docstrings, docs/, CONTRIBUTING.md) if needed
  • CHANGELOG / release notes updated if applicable — n/a

AI/LLM disclosure

  • I used the following tool to help write this PR description: GitHub Copilot (Claude Opus 5)
  • I used the following tool to generate or modify code: GitHub Copilot (Claude Opus 5)

The `package-tests` and `flake` matrices only ever report per-combination check
names, so branch protection has to be re-edited whenever a lane moves. Add an
`if: always()` aggregation job to each workflow that publishes a fixed name and
fails unless the matrix succeeded or was skipped.

Assisted-by: GitHub Copilot:claude-opus-5
@github-actions github-actions Bot added documentation Improvements or additions to documentation ci labels Sep 3, 2026
@Panadestein
Panadestein marked this pull request as ready for review September 3, 2026 12:33
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

Docs preview: https://pr-336.monoprop-docs.pages.dev

Comment thread .github/workflows/nix.yml Outdated
Comment thread .github/workflows/test.yml Outdated

@robertodr robertodr left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the idea is: use a single proxy that requires a set of jobs to pass and then change the ruleset to require that proxy to pass?

Meaning: we have to edit the ruleset once more after merging this PR, but then we can live happily ever after?

@Panadestein

Copy link
Copy Markdown
Member Author

So the idea is: use a single proxy that requires a set of jobs to pass and then change the ruleset to require that proxy to pass?

Meaning: we have to edit the ruleset once more after merging this PR, but then we can live happily ever after?

Exactly this. Then we can change the set of jobs and rule won't break.

Panadestein and others added 2 commits September 3, 2026 14:51
Co-authored-by: Roberto Di Remigio Eikås <robertodr@users.noreply.github.com>
Signed-off-by: Ramón L. Panadés-Barrueta <rpana92@gmail.com>
Co-authored-by: Roberto Di Remigio Eikås <robertodr@users.noreply.github.com>
Signed-off-by: Ramón L. Panadés-Barrueta <rpana92@gmail.com>
@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.70%. Comparing base (9129f92) to head (21f5742).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #336   +/-   ##
=======================================
  Coverage   97.70%   97.70%           
=======================================
  Files          14       14           
  Lines         742      742           
  Branches       98       98           
=======================================
  Hits          725      725           
  Misses         12       12           
  Partials        5        5           
Flag Coverage Δ
cpp 97.70% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

@sonarqubecloud

sonarqubecloud Bot commented Sep 3, 2026

Copy link
Copy Markdown

@robertodr
robertodr merged commit 5061c79 into main Sep 3, 2026
35 checks passed
@robertodr
robertodr deleted the ci/matrix-result-gates branch September 3, 2026 13:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants