Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 29 additions & 2 deletions .github/workflows/build-wheels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,16 @@ jobs:
# `~/.cache/uv` a second writer alongside ci.yml's tool cache for a job that
# would save less time than the archive costs to move.
build-pure-wheel:
runs-on: ubicloud-standard-2
# Reached from ci.yml on every pull request, forks included, and from
# release.yml on a tag. A called workflow inherits its caller's `github`
# context, so this expression reads the caller's event: on the tag caller
# `github.event.pull_request` is null and the Ubicloud arm is selected,
# and on a fork's pull request the hosted arm is, because a fork cannot
# obtain an Ubicloud runner. Keep the continuation at this indent; a
# more-indented one would put a newline inside the expression.
runs-on: >-
${{ github.event.pull_request.head.repo.fork
&& 'ubuntu-latest' || 'ubicloud-standard-2' }}
timeout-minutes: 20
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
Expand All @@ -37,7 +46,16 @@ jobs:
artifact-name: wheels-pure

build-native-wheels:
# Unchanged placement. GitHub spells matrix values into the check name, so
# both `ubuntu-latest` legs are named verbatim in the `main-required-checks`
# ruleset; moving either renames a required context, and nine steps here
# are guarded by `startsWith(matrix.os, 'ubuntu')`, which an expression
# would switch off. That is a decision for the repository owner, not a
# placement change.
runs-on: ${{ matrix.os }}
# Measured at 29 to 107 s across the five legs; the Linux ones build in a
# container and the aarch64 leg emulates under QEMU.
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
Expand Down Expand Up @@ -156,7 +174,16 @@ jobs:
# Also uncached: it installs published wheels into a throwaway virtual
# environment and imports them. There is nothing expensive to keep.
verify-wheel-install:
runs-on: ubicloud-standard-2
# Reached from ci.yml on every pull request, forks included, and from
# release.yml on a tag. A called workflow inherits its caller's `github`
# context, so this expression reads the caller's event: on the tag caller
# `github.event.pull_request` is null and the Ubicloud arm is selected,
# and on a fork's pull request the hosted arm is, because a fork cannot
# obtain an Ubicloud runner. Keep the continuation at this indent; a
# more-indented one would put a newline inside the expression.
runs-on: >-
${{ github.event.pull_request.head.repo.fork
&& 'ubuntu-latest' || 'ubicloud-standard-2' }}
timeout-minutes: 20
needs:
- build-pure-wheel
Expand Down
122 changes: 94 additions & 28 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,23 @@ env:

jobs:
lint-test:
runs-on: ubuntu-latest
# A pull request from a fork cannot obtain an Ubicloud runner, so the fork
# arm falls back to a GitHub-hosted label and only forks take it. Keep the
# continuation at this indent: a more-indented continuation in a folded
# scalar keeps its line break and puts a newline inside the expression.
# GitHub evaluates the broken value regardless, so a green run is not
# evidence; `test_placement_expressions_parse_to_one_line` is.
runs-on: >-
${{ github.event.pull_request.head.repo.fork
&& 'ubuntu-latest' || 'ubicloud-standard-2' }}
# Compile-bound on two toolchains plus a Windows cross-target check,
# measured at 198 to 292 s over six runs on four GitHub-hosted vCPU.
# Ubicloud vCPU are not GitHub vCPU, so the Ubicloud figure is expected to
# be higher and the arm is judged on the third run. The ceiling matches the
# interpreter-suite lanes: a regression fails visibly instead of quietly
# costing, and `ubicloud-standard-4` needs measured evidence this
# repository does not have yet.
timeout-minutes: 45
env:
MBAKE_VERSION: '1.4.6'
MDTABLEFIX_VERSION: '0.6.0'
Expand All @@ -72,6 +88,12 @@ jobs:
# token must not stay in .git/config where that code could reach it.
persist-credentials: false

- name: Start resource sampler
uses: ./.github/actions/resource-sampler
with:
mode: start
vcpus: ${{ env.LINUX_RUNNER_VCPUS }}

- name: Compute cache keys
uses: ./.github/actions/cache-keys
with:
Expand All @@ -80,10 +102,14 @@ jobs:
ubuntu-release: ${{ env.UBUNTU_RELEASE }}
compiler-shape: lint

# This job is GitHub-hosted, so these archives live in GitHub's cache
# service rather than Ubicloud's. Every key carries `runner.environment`,
# so the two lanes cannot restore each other's entries even though they
# share a family name, and each lane has its own single writer.
# Every `CARGO_CACHE_KEY` and `SCCACHE_CACHE_KEY` family carries
# `runner.environment`, so the owned arm and the fork arm cannot restore
# each other's entries even though they share a family name. Nothing
# writes the fork arm's scope of those two families: their saves require
# a push to `refs/heads/main`, which a fork can never produce, so a
# fork's pull request restores nothing from them and compiles cold. The
# plain `actions/cache` tool caches further down are not key families:
# they save on every event under GitHub's own branch scoping.
- name: Restore the Cargo registry
id: cargo-cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
Expand Down Expand Up @@ -328,26 +354,19 @@ jobs:
printf '%s\n' "${fence}"
} >> "${GITHUB_STEP_SUMMARY}"

# Single writer for the GitHub-hosted lane, mirroring `typecheck-test`
# on the Ubicloud lane. It is the only Linux gate here and the only one
# that compiles, so there is nothing to race with.
- name: Save the Cargo registry
if: >-
github.event_name == 'push' && github.ref == 'refs/heads/main' &&
steps.cargo-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
~/.cargo/registry
~/.cargo/git
key: ${{ env.CARGO_CACHE_KEY }}

# This job restores the Cargo registry and no longer saves it. On the
# owned arm it now runs on the Ubicloud lane, where `runner.environment`
# renders `self-hosted` and the key it would publish is the very family
# `extension-tests` writes. Two writers of one family is the collision
# `test_each_cache_family_has_exactly_one_writer` exists to refuse, and
# the registry holds the resolved dependency graph, which is the same
# content either job would archive.
- name: Save the compiler cache
# The only GitHub-hosted job that compiles, so it owns the whole
# `github-hosted` compiler-cache scope; its Cranelift lint build is
# also a shape no other job produces. Its archive lands in GitHub's
# cache rather than Ubicloud's, which is the other reason the lane is
# named in the key.
# The `lint` build shape is produced by no other job, so this remains
# the sole writer of its compiler-cache family. Only the lane in the
# rendered key changed: the fork arm cannot reach this step, because
# the guard below requires a push to `refs/heads/main` and a fork's
# pull request is neither.
#
# The entry is a rolling generation named after the run, so there is
# no cache-hit guard to skip: the key is new every time, and a run
Expand All @@ -362,10 +381,25 @@ jobs:
path: ~/.cache/sccache
key: ${{ env.SCCACHE_CACHE_KEY }}

- name: Report peak resource use
if: always()
uses: ./.github/actions/resource-sampler
with:
mode: report
vcpus: ${{ env.LINUX_RUNNER_VCPUS }}

typecheck-test:
name: Typecheck and test (Python ${{ matrix.python-label }})
continue-on-error: ${{ matrix.experimental }}
runs-on: ubicloud-standard-2
# A pull request from a fork cannot obtain an Ubicloud runner, so the fork
# arm falls back to a GitHub-hosted label and only forks take it. Keep the
# continuation at this indent: a more-indented continuation in a folded
# scalar keeps its line break and puts a newline inside the expression.
# GitHub evaluates the broken value regardless, so a green run is not
# evidence; `test_placement_expressions_parse_to_one_line` is.
runs-on: >-
${{ github.event.pull_request.head.repo.fork
&& 'ubuntu-latest' || 'ubicloud-standard-2' }}
timeout-minutes: 45
env:
MDTABLEFIX_VERSION: '0.6.0'
Expand Down Expand Up @@ -617,7 +651,15 @@ jobs:

extension-tests:
name: Extension-gated tests (Python/Rust boundary)
runs-on: ubicloud-standard-2
# A pull request from a fork cannot obtain an Ubicloud runner, so the fork
# arm falls back to a GitHub-hosted label and only forks take it. Keep the
# continuation at this indent: a more-indented continuation in a folded
# scalar keeps its line break and puts a newline inside the expression.
# GitHub evaluates the broken value regardless, so a green run is not
# evidence; `test_placement_expressions_parse_to_one_line` is.
runs-on: >-
${{ github.event.pull_request.head.repo.fork
&& 'ubuntu-latest' || 'ubicloud-standard-2' }}
timeout-minutes: 45
steps:
- name: Check out repository
Expand Down Expand Up @@ -822,6 +864,11 @@ jobs:
extension-tests-windows:
name: Extension-gated tests (Windows Python/Rust boundary)
runs-on: windows-2022
# Measured at 67 to 72 s. A ceiling belongs on every job that declares
# steps, not only on the paid ones: once a label is event-dependent,
# "an Ubicloud lane" is a property of the event, so a rule keyed on the
# label would stop applying on exactly the arm that hangs.
timeout-minutes: 20
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
Expand Down Expand Up @@ -873,7 +920,15 @@ jobs:
# ratchet baseline, so this lane only compares its coverage locally
# against the trusted baseline without generating coverage twice on main.
if: github.event_name == 'pull_request'
runs-on: ubicloud-standard-2
# A pull request from a fork cannot obtain an Ubicloud runner, so the fork
# arm falls back to a GitHub-hosted label and only forks take it. Keep the
# continuation at this indent: a more-indented continuation in a folded
# scalar keeps its line break and puts a newline inside the expression.
# GitHub evaluates the broken value regardless, so a green run is not
# evidence; `test_placement_expressions_parse_to_one_line` is.
runs-on: >-
${{ github.event.pull_request.head.repo.fork
&& 'ubuntu-latest' || 'ubicloud-standard-2' }}
# Above the 2,700 s cargo watchdog set below plus the work either side
# of the coverage step, which reached 43 s on run 34071469378. Fifty
# minutes is the requirement, and the estate's rule is fifteen above
Expand Down Expand Up @@ -1128,7 +1183,10 @@ jobs:
# `needs` edge never leaves benchmark-ratchet skipped on pushes to
# main, where the baseline artefact must stay fresh.
changes:
# Stays GitHub-hosted: a five-second paths-filter call is administrative,
# API-bound work that gains nothing from a metered build slot.
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
# paths-filter reads pull-request changed files through GitHub's API.
Expand Down Expand Up @@ -1426,7 +1484,15 @@ jobs:
# ratio between the baseline and the candidate, so runner speed cancels out
# of the comparison and the job does not need a fixed or larger shape. It
# therefore starts at the estate default like every other Linux gate.
runs-on: ubicloud-standard-2
# A pull request from a fork cannot obtain an Ubicloud runner, so the fork
# arm falls back to a GitHub-hosted label and only forks take it. Keep the
# continuation at this indent: a more-indented continuation in a folded
# scalar keeps its line break and puts a newline inside the expression.
# GitHub evaluates the broken value regardless, so a green run is not
# evidence; `test_placement_expressions_parse_to_one_line` is.
runs-on: >-
${{ github.event.pull_request.head.repo.fork
&& 'ubuntu-latest' || 'ubicloud-standard-2' }}
timeout-minutes: 60
needs:
- lint-test
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/delayed-pr-comment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ jobs:
# Deliberately stays on GitHub-hosted Ubuntu: this sleeps while holding a
# runner and must not consume a metered Ubicloud build slot.
runs-on: ubuntu-latest
# A ceiling far above any plausible delay, so a runaway `delay_minutes`
# cannot hold a runner for GitHub's six-hour default. A delay longer than
# this belongs in a scheduled workflow, not in a sleeping job.
timeout-minutes: 180
permissions:
pull-requests: write # needed to post comments
steps:
Expand Down
12 changes: 11 additions & 1 deletion docs/ci-cache-ownership.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,6 @@ the family without saving it.
| Family | Writer |
| ------------------------------------- | ------------------------------------- |
| Cargo registry, Ubicloud | `ci.yml` `extension-tests` |
| Cargo registry, GitHub-hosted | `ci.yml` `lint-test` |
| Tools, per interpreter | `ci.yml` `typecheck-test`, that leg |
| Compiler, 3.13 unoptimized | `ci.yml` `extension-tests` |
| Compiler, 3.12/3.14/3.15a unoptimized | `ci.yml` `typecheck-test`, that leg |
Expand All @@ -111,6 +110,17 @@ the family without saving it.
| Compiler, Cranelift lint | `ci.yml` `lint-test` |
| Compiler, Loom model | `loom.yml` `loom` |

`lint-test` moved to the Ubicloud lane with the fork fallback, taking its
Cranelift lint family with it, and stopped saving the Cargo registry: on its
owned arm it renders the same family `extension-tests` writes, and one writer
per family is the invariant. The Loom family stays on the GitHub-hosted lane,
because its writer is a scheduled job that stays there too.

The fork arm renders `runner.environment` as `github-hosted`, so a fork's pull
request reads a scope whose only writer is that scheduled Loom job and runs
cold for everything else. Every save is guarded on a push to `refs/heads/main`,
which a fork can never produce, so the fork arm restores and never publishes.

_Table 2: The single job that publishes each family under the writer trigger
described above._

Expand Down
Loading
Loading