Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/actionlint.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
self-hosted-runner:
labels:
- ubicloud-standard-8
- ubicloud-standard-4
37 changes: 26 additions & 11 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -66,16 +66,26 @@ jobs:
runs-on: >-
${{ (github.event.pull_request.head.repo.fork
|| github.event.pull_request.user.login == 'dependabot[bot]')
&& 'ubuntu-latest' || 'ubicloud-standard-8' }}
# 90, which is this job's own figure and no longer the coverage jobs':
# both of those went to 120 in #486 to clear the 5,400 s cargo watchdog,
# and this job has no watchdog to clear. A cold sccache pays about 1.4 s
# to store each miss, and a run with no trunk generation to restore
# misses almost everything, so the first run on a new generation is the
# longest this job ever gets. 60 cancelled it mid compile-fail suite with
# every step passing, which loses the generation the next run would have
# read.
timeout-minutes: 90
&& 'ubuntu-latest' || 'ubicloud-standard-4' }}
# Four vCPU, not eight, on measurement. Sampled every 10 s on 2026-10-17
# (https://github.com/leynos/wildside/actions/runs/39038438142), a warm
# run (99.78% sccache hits) took 21m03s on `ubicloud-standard-4` at 1.41
# busy cores on average (p95 2.20, peak memory 2.4 GiB), and a cold run
# with no cache to restore
# (https://github.com/leynos/wildside/actions/runs/39061927189) took
# 86m46s at 0.88 busy cores (7.81% hits, peak memory 5.7 GiB of 16). A
# job that keeps fewer than half of four cores busy gains little from
# eight, and Ubicloud bills per vCPU.
#
# 120, because the cold run above took 86m46s: a run with no trunk
# generation to restore misses almost everything and pays about 1.4 s to
# store each miss, so the first run on a new generation is the longest
# this job ever gets. 90 would have left four minutes; 60 cancelled it
# mid compile-fail suite with every step passing, which loses the
# generation the next run would have read. This job has no cargo
# watchdog to clear, and nextest's own 60 minute global timeout bounds
# the test run inside it.
timeout-minutes: 120
permissions:
contents: read
steps:
Expand Down Expand Up @@ -683,10 +693,15 @@ jobs:
coverage:
if: github.actor != 'dependabot[bot]' && github.event_name != 'push'
# The same fork fallback as `build`; see the comment there.
#
# Four vCPU on measurement (see `build`): a warm run took 8m15s at 1.70
# busy cores on average (peak memory 4.0 GiB;
# https://github.com/leynos/wildside/actions/runs/39038438142) and a
# cold one 17m02s at 1.46 (https://github.com/leynos/wildside/actions/runs/39061927189).
runs-on: >-
${{ (github.event.pull_request.head.repo.fork
|| github.event.pull_request.user.login == 'dependabot[bot]')
&& 'ubuntu-latest' || 'ubicloud-standard-8' }}
&& 'ubuntu-latest' || 'ubicloud-standard-4' }}
# Above the 5,400 s cargo watchdog below plus the work either side of
# the coverage step, which reached 59 s here and 522 s on the trunk
# lane. At 90 minutes the ceiling equalled the watchdog exactly, so a
Expand Down
8 changes: 7 additions & 1 deletion .github/workflows/coverage-main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,13 @@ permissions:

jobs:
coverage-upload:
runs-on: ubicloud-standard-8
# Four vCPU, as for `ci.yml`'s `coverage` job, whose work this lane
# repeats on a push to `main`: 8m15s warm and 17m02s cold at 1.46 to
# 1.70 busy cores on average, peak memory 4.0 GiB (runs 39038438142 and
# 39061927189 in leynos/wildside). This lane adds the upload, which
# reached 522 s of work outside the coverage step on run 33921794186; the
# 120 minute ceiling and 5,400 s watchdog below are unchanged.
runs-on: ubicloud-standard-4
# The CodeScene token lives in this environment, whose deployment policy
# admits `main` alone; the codescene environment contract holds it to
# this job.
Expand Down
35 changes: 27 additions & 8 deletions docs/developers-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -235,7 +235,7 @@ run through `make test-workflow-contracts`.

### Job placement

Build and test work runs on the managed `ubicloud-standard-8` label: `ci.yml`'s
Build and test work runs on the managed `ubicloud-standard-4` label: `ci.yml`'s
`build` and `coverage` jobs, and `coverage-main.yml`'s `coverage-upload` job.
Every managed job declares `timeout-minutes`.

Expand All @@ -250,7 +250,7 @@ managed one for everything else:
runs-on: >-
${{ (github.event.pull_request.head.repo.fork
|| github.event.pull_request.user.login == 'dependabot[bot]')
&& 'ubuntu-latest' || 'ubicloud-standard-8' }}
&& 'ubuntu-latest' || 'ubicloud-standard-4' }}
```

The Dependabot test reads the pull request's author, not `github.actor`,
Expand Down Expand Up @@ -279,11 +279,30 @@ leaves a newline inside the expression. GitHub evaluates the broken value
regardless, so a green run is not evidence that the label is well formed. Keep
the continuation at the same indent as the line it continues.

That `-8` shape is inherited rather than measured. It predates the managed
runner work and no job here has yet been sampled for peak memory or disk, so
there is no evidence for or against a smaller shape. Add a sampler to the two
Rust jobs and read it before proposing a change; the default elsewhere in the
estate is `ubicloud-standard-2`, with `-4` only on measured disk evidence.
The shape is `-4` on measurement, taken on 2026-10-17 with a sampler that read
`/proc/stat` every 10 seconds on a measurement branch (the sampler is not
merged). The jobs were previously on `ubicloud-standard-8`, a shape that was
inherited rather than measured.

| Job | Cache state | Wall time | Busy cores (avg / p95 / max) | Peak memory | sccache hits |
| ---------- | -------------------------- | --------- | ---------------------------- | ----------- | ------------ |
| `build` | warm | 21m03s | 1.41 / 2.20 / 2.85 of 4 | 2,423 MiB | 99.78% |
| `build` | cold (no cache to restore) | 86m46s | 0.88 / 1.74 / 3.88 of 4 | 5,650 MiB | 7.81% |
| `coverage` | warm | 8m15s | 1.70 / 3.99 / 4.00 of 4 | 4,060 MiB | 99.87% |
| `coverage` | cold | 17m02s | 1.46 / 2.64 / 3.49 of 4 | 3,394 MiB | 0.38% |

*Table: `ubicloud-standard-4` trial runs, warm
([39038438142](https://github.com/leynos/wildside/actions/runs/39038438142))
and cold
([39061927189](https://github.com/leynos/wildside/actions/runs/39061927189)).*

Ubicloud bills per vCPU, so a job that keeps fewer than half of its cores busy
gains little from a larger shape; none of these did. Minimum free disk was at
least 65 GiB throughout. The cold `build` is the longest run the job will ever
have, because the first run on a new generation misses almost everything, so its
`timeout-minutes` is 120 rather than the 90 it had on eight vCPU. The cold
`coverage` run is far inside its 120 minutes and its 5,400 second cargo
watchdog.

Everything else stays on GitHub-hosted `ubuntu-latest`. Scheduled, delayed,
metadata, label, review-bot, and release-orchestration work sits off the
Expand Down Expand Up @@ -2472,7 +2491,7 @@ until this was written the outermost one was set wrong.
| Per-test `slow-timeout` | one test | `.config/nextest.toml` | 60 s default; 300 s for the database-backed overrides |
| nextest `global-timeout` | the whole test run | `.config/nextest.toml` | 3,600 s (60 m) |
| Cargo watchdog | one `cargo` invocation, wall clock | `RUN_RUST_CARGO_WAIT_TIMEOUT`, read from the step, then the job, then the workflow; set at job level here | 5,400 s (90 m) |
| Job `timeout-minutes` | the whole job | job level in `ci.yml` and `coverage-main.yml` | 120 m |
| Job `timeout-minutes` | the whole job | job level in `ci.yml` and `coverage-main.yml` | 120 m (`build` too, since its cold run took 86m46s) |

*Table: the timers that can end a run, innermost first.*

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
`main`, on a push or a dispatch alike.

Running the workflow itself, under `act` or on a runner, would need the
CodeScene token and the `ubicloud-standard-8` label; the upload's real proof
CodeScene token and the `ubicloud-standard-4` label; the upload's real proof
is the publisher run on the merge commit.

Run via ``make test-workflow-contracts``.
Expand Down
16 changes: 16 additions & 0 deletions tests/workflow_contracts/tool_installation_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -354,6 +354,22 @@ def test_build_jobs_keep_their_reviewed_label_and_a_timeout(
)


def test_the_build_ceiling_clears_the_cold_standard_4_run() -> None:
"""The first run on a new cache generation is the longest the job has.

Measured on ``ubicloud-standard-4`` with no cache to restore (7.81% of
sccache requests hit), ``build`` took 86m46s: leynos/wildside run
39061927189. A ceiling of 90 leaves four minutes, and a cancelled job
discards the generation the next run would have read, so the ceiling is
held to 120 minutes, which is the cold run plus a margin.
"""
job = inv.load_workflow("ci.yml")["jobs"]["build"]
assert job.get("timeout-minutes", 0) >= 120, (
"ci.yml:build must allow 120 minutes; its cold run on standard-4 "
f"took 86m46s and it declares {job.get('timeout-minutes')!r}"
)


def test_the_coverage_tool_cache_key_tracks_the_shared_action_pin() -> None:
"""The coverage lane's executables are chosen by the pinned action.

Expand Down
2 changes: 1 addition & 1 deletion tests/workflow_contracts/workflow_inventory.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@

#: Managed-runner labels this repository intends to use. Every label here must
#: also be registered with actionlint.
MANAGED_RUNNER_LABELS = frozenset({"ubicloud-standard-8"})
MANAGED_RUNNER_LABELS = frozenset({"ubicloud-standard-4"})
GITHUB_HOSTED_LABELS = frozenset({"ubuntu-latest"})

#: Jobs that build or test the product. Everything else is administrative,
Expand Down
Loading