Skip to content

Declare the test matrices in one place - #967

Merged
d-chambers merged 1 commit into
devfrom
ci-matrix-config
Aug 21, 2026
Merged

Declare the test matrices in one place#967
d-chambers merged 1 commit into
devfrom
ci-matrix-config

Conversation

@d-chambers

@d-chambers d-chambers commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Description

The test matrices are now declared once, in .github/actions/load-shared-vars/action.yml, and nowhere else. Changing what CI runs is an edit to that one file; no workflow job gains a condition of its own.

Two changes make that possible.

Each matrix is emitted as a whole matrix object rather than two lists. The consuming job becomes one line:

strategy:
  fail-fast: false
  matrix: ${{ fromJson(needs.setup.outputs.test-matrix) }}

That is less than it replaced (two fromJson lines plus their comments), and it allows include, which a cross product of an OS list and a Python list cannot express.

The pull-request/push split lives in a single if in that same file. Everything a maintainer would want to change sits above a marked line; below it is assembly only.

What runs where

pull request push to dev / master
test_code 5 — ubuntu × {3.12, 3.13, 3.14}, windows × 3.13, macos × 3.13 9 — 3 OS × 3 Python
test_code_min_deps 2 — ubuntu × {3.11, 3.14} 6 — 3 OS × {3.11, 3.14}
network_tests 1 — ubuntu 3 — one per OS
setup, conda_env unchanged unchanged

Every Python version and every OS is still exercised on a pull request; only the redundant cross product is dropped. The full grid still runs, on the push to dev that follows the merge — and since #965 a newer push supersedes it rather than stacking.

Measured against medians from the last 25 runs, a pull request drops from ~28 jobs / ~145 min of runner time to ~15 jobs / ~72 min. The org is capped at 20 concurrent jobs, so this is roughly the difference between one pull request saturating CI and three sharing it.

Restoring the old behavior is a one-line edit: set pr_test_matrix to {"os":'"$all_os"',"python-version":'"$full_py"'}, and likewise for the other two.

Verification

The script was executed directly with GITHUB_EVENT_NAME set both ways and the emitted JSON expanded into job lists; the counts above are that output, not an estimate.

Changelog

none

Checklist

I have:

  • filled in the Changelog section above (see docs/contributing/general_guidelines.qmd).

Each matrix is emitted whole from load-shared-vars, so the consuming job
is a single fromJson line and the pull-request/push split lives in one
if. Emitting an object rather than two lists also allows include, which
a cross product cannot express.
@coderabbitai

coderabbitai Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@d-chambers, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 27 minutes

Limit details: You’ve used all 2 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 05e6e0bb-a883-4289-8cd2-0ba6372fb035

📥 Commits

Reviewing files that changed from the base of the PR and between 6ad7db8 and daf9a29.

📒 Files selected for processing (3)
  • .github/actions/load-shared-vars/action.yml
  • .github/workflows/run_min_dep_tests.yml
  • .github/workflows/runtests.yml

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@d-chambers
d-chambers merged commit d1a4a88 into dev Aug 21, 2026
9 of 18 checks passed
@d-chambers
d-chambers deleted the ci-matrix-config branch August 21, 2026 11:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant