Skip to content

Declare the test matrices in one place #967

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
d-chambers merged 1 commit into from
Aug 21, 2026
Merged

Declare the test matrices in one place #967

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 41 additions & 15 deletions .github/actions/load-shared-vars/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
# This is where global configurations are added.
#
# The test matrices live here, and only here. Each is emitted as a whole
# GitHub matrix object so the consuming job is one `matrix: fromJson(...)`
# line with no conditions of its own, and so a matrix can use `include`
# (a cross product of two lists cannot express "every python on one OS,
# plus a single job on the others").

name: "Load Shared Variables"
description: "Loads shared variables and sets them as step outputs"
Expand All @@ -7,15 +13,15 @@ outputs:
python-default:
description: "Default python version"
value: ${{ steps.load.outputs.python-default }}
python-test-matrix:
description: "Python version matrix for full tests"
value: ${{ steps.load.outputs.python-test-matrix }}
python-min-deps-matrix:
description: "Python version matrix for minimum dependency tests"
value: ${{ steps.load.outputs.python-min-deps-matrix }}
test-os-matrix:
description: "OS matrix for test workflows"
value: ${{ steps.load.outputs.test-os-matrix }}
test-matrix:
description: "Matrix object for the full test job"
value: ${{ steps.load.outputs.test-matrix }}
min-deps-matrix:
description: "Matrix object for the minimum dependency test job"
value: ${{ steps.load.outputs.min-deps-matrix }}
network-os-matrix:
description: "OS list for the network test job"
value: ${{ steps.load.outputs.network-os-matrix }}

runs:
using: "composite"
Expand All @@ -25,13 +31,33 @@ runs:
shell: bash
run: |
python_default="3.13"
all_os='["ubuntu-latest","macos-latest","windows-latest"]'
# Min-deps covers the oldest supported version, so every version
# pyproject claims is tested somewhere without widening the full
# matrix (each entry there costs one environment cache per OS).
python_min_deps_matrix='["3.11","3.14"]'
python_test_matrix='["3.12","3.13","3.14"]'
test_os_matrix='["ubuntu-latest","macos-latest","windows-latest"]'
full_py='["3.12","3.13","3.14"]'
min_deps_py='["3.11","3.14"]'

# What a pull request runs. Every python version and every OS is
# still covered; only the redundant cross product is dropped. The
# full grid runs on the push to dev that follows the merge. Set any
# of these to its full_* counterpart to restore the old behavior.
pr_test_matrix='{"os":["ubuntu-latest"],"python-version":["3.12","3.13","3.14"],"include":[{"os":"windows-latest","python-version":"3.13"},{"os":"macos-latest","python-version":"3.13"}]}'
pr_min_deps_matrix='{"os":["ubuntu-latest"],"python-version":["3.11","3.14"]}'
pr_network_os='["ubuntu-latest"]'

# ---- nothing below here needs editing ----------------------
if [ "$GITHUB_EVENT_NAME" = "pull_request" ]; then
test_matrix="$pr_test_matrix"
min_deps_matrix="$pr_min_deps_matrix"
network_os="$pr_network_os"
else
test_matrix="{\"os\":${all_os},\"python-version\":${full_py}}"
min_deps_matrix="{\"os\":${all_os},\"python-version\":${min_deps_py}}"
network_os="$all_os"
fi

echo "python-default=$python_default" >> "$GITHUB_OUTPUT"
echo "python-test-matrix=$python_test_matrix" >> "$GITHUB_OUTPUT"
echo "python-min-deps-matrix=$python_min_deps_matrix" >> "$GITHUB_OUTPUT"
echo "test-os-matrix=$test_os_matrix" >> "$GITHUB_OUTPUT"
echo "test-matrix=$test_matrix" >> "$GITHUB_OUTPUT"
echo "min-deps-matrix=$min_deps_matrix" >> "$GITHUB_OUTPUT"
echo "network-os-matrix=$network_os" >> "$GITHUB_OUTPUT"
10 changes: 3 additions & 7 deletions .github/workflows/run_min_dep_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,7 @@ jobs:
runs-on: ubuntu-latest
outputs:
# Shared values live in .github/actions/load-shared-vars/action.yml
python-matrix: ${{ steps.load-vars.outputs.python-min-deps-matrix }}
os-matrix: ${{ steps.load-vars.outputs.test-os-matrix }}
min-deps-matrix: ${{ steps.load-vars.outputs.min-deps-matrix }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
Expand All @@ -64,11 +63,8 @@ jobs:
# One job failing used to cancel the rest, which both hid the other
# results and wasted the work those jobs had already done.
fail-fast: false
matrix:
# Defined in .github/actions/load-shared-vars/action.yml
os: ${{ fromJson(needs.setup.outputs.os-matrix) }}
# Defined in .github/actions/load-shared-vars/action.yml
python-version: ${{ fromJson(needs.setup.outputs.python-matrix) }}
# Defined in .github/actions/load-shared-vars/action.yml
matrix: ${{ fromJson(needs.setup.outputs.min-deps-matrix) }}

# only run if CI isn't turned off
if: github.event_name == 'push' || !contains(github.event.pull_request.labels.*.name, 'no_ci')
Expand Down
14 changes: 6 additions & 8 deletions .github/workflows/runtests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,8 @@ jobs:
outputs:
# Shared values live in .github/actions/load-shared-vars/action.yml
python-default: ${{ steps.load-vars.outputs.python-default }}
python-matrix: ${{ steps.load-vars.outputs.python-test-matrix }}
os-matrix: ${{ steps.load-vars.outputs.test-os-matrix }}
test-matrix: ${{ steps.load-vars.outputs.test-matrix }}
network-os-matrix: ${{ steps.load-vars.outputs.network-os-matrix }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
Expand All @@ -72,11 +72,8 @@ jobs:
# One job failing used to cancel the rest, which both hid the other
# results and wasted the work those jobs had already done.
fail-fast: false
matrix:
# Defined in .github/actions/load-shared-vars/action.yml
os: ${{ fromJson(needs.setup.outputs.os-matrix) }}
# Defined in .github/actions/load-shared-vars/action.yml
python-version: ${{ fromJson(needs.setup.outputs.python-matrix) }}
# Defined in .github/actions/load-shared-vars/action.yml
matrix: ${{ fromJson(needs.setup.outputs.test-matrix) }}

# only run if CI isn't turned off
if: github.event_name == 'push' || !contains(github.event.pull_request.labels.*.name, 'no_ci')
Expand Down Expand Up @@ -154,7 +151,8 @@ jobs:
strategy:
fail-fast: false
matrix:
os: ${{ fromJson(needs.setup.outputs.os-matrix) }}
# Defined in .github/actions/load-shared-vars/action.yml
os: ${{ fromJson(needs.setup.outputs.network-os-matrix) }}

# Keep remote-IO coverage visible without blocking unrelated changes.
if: github.event_name == 'push' || !contains(github.event.pull_request.labels.*.name, 'no_ci')
Expand Down
Loading