Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 7 additions & 15 deletions .github/workflows/fast-checks.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,11 @@
# Every push to develop and every pull request. No gate, so the run is
# exactly what a maintainer sees locally: contracts, validation, wiring,
# the product surface, and the tight numerical unit guards that catch a
# shifted constant in one run. Budget: <= 15 min of tests in THIS job,
# which keeps the whole job near half its 30-minute timeout.
#
# Read the budget off this job's own test phase. The tiering audit's
# "~90 s" is a different measurement -- an OPTIMIZED build on a developer
# machine -- and nothing this job prints can be compared against it.
# devtools::test() compiles a debug build (pkgbuild::compiler_flags(debug
# = TRUE) gives "-UNDEBUG -Wall -pedantic -g -O0"), and this job keeps it
# that way on purpose (see the extra-packages note below). Measured on
# identical content that build costs 2.38x, and this 2-core runner a
# further ~2.31x over one developer machine. So quote CI minutes here, or
# say which build a local number came from; a bare seconds figure is not
# comparable across the two and let this tier drift unnoticed once.
# shifted constant in one run. The budget is this job's own test phase:
# 25 min on the October 2026 ubuntu-latest runner (run 39602904192),
# against timeout-minutes: 45. A local devtools::test() wall time is
# not comparable: it compiles a debug build (-O0) on a different
# machine. Measure from a run of this job that finished.
#
# T1 nightly .github/workflows/nightly-validation.yaml. Daily 03:00 UTC
# on develop, BGMS_RUN_SLOW_TESTS=true, <= 60 min, timeout 90.
Expand All @@ -48,9 +40,9 @@ on:

jobs:
fast-tests:
name: T0 every-run tier (~90 s of tests)
name: T0 every-run tier
runs-on: ubuntu-latest
timeout-minutes: 30
timeout-minutes: 45
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
R_KEEP_PKG_SOURCE: yes
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/nightly-validation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
#
# T0 every run local devtools::test() + .github/workflows/fast-checks.yaml
# Contracts, validation, wiring, the product surface, the tight numerical
# unit guards. No gate. Budget ~90 s.
# unit guards. No gate; budget stated in fast-checks.yaml.
#
# T1 nightly THIS FILE. Daily 03:00 UTC on develop.
# BGMS_RUN_SLOW_TESTS=true.
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/weekly-certification.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
#
# T0 every run local devtools::test() + .github/workflows/fast-checks.yaml
# Contracts, validation, wiring, the product surface, the tight numerical
# unit guards. No gate. Budget ~90 s.
# unit guards. No gate; budget stated in fast-checks.yaml.
#
# T1 nightly .github/workflows/nightly-validation.yaml. Daily 03:00 UTC.
# BGMS_RUN_SLOW_TESTS=true.
Expand Down
10 changes: 6 additions & 4 deletions dev/review-2026-08/MAINTAINERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ be T1. Budget is the tie-breaker, never the criterion.

| Tier | Cadence | Content, by bug class | Budget | Gate |
|---|---|---|---|---|
| **T0** every run | local `devtools::test()`; every push to develop and every PR (`fast-checks.yaml`) | contracts, validation, wiring, unit guards, product surface | **≤ 15 min of tests** in the `fast-checks` job (`timeout-minutes: 30`) | none |
| **T0** every run | local `devtools::test()`; every push to develop and every PR (`fast-checks.yaml`) | contracts, validation, wiring, unit guards, product surface | the `fast-checks` job's own test phase: 25 min on the October 2026 runner (`timeout-minutes: 45`) | none |
| **T1** nightly heartbeat | daily 03:00 UTC on **develop** (`nightly-validation.yaml`) | curated calibration subset — graph-law and prior-chain identities, gauge detector, single surface-vs-gold cells, RB saturation, concordance smokes. "Does the settled math still hold tonight" | **≤ 60 min** on the 2-core runner; `timeout-minutes: 90` | `BGMS_RUN_SLOW_TESTS=true` |
| **T2** weekly certification | Sunday 03:00 UTC on **develop** (`weekly-certification.yaml`) | the heavy Monte-Carlo machinery — SBC suites, full parameter-recovery sweeps, full NUTS-vs-MH condition grids, n = 2e6 MC channels, refit cross-validations (incl. the F-049 gate) | `timeout-minutes: 360` | `BGMS_RUN_CERTIFICATION=true` (the T2 workflow sets **both** vars, so a weekly run also carries T1) |

Expand All @@ -233,9 +233,11 @@ machine and landed 89.7 s; `devtools::test()` and every CI tier compile a
*debug* build (`pkgbuild::compiler_flags(debug = TRUE)` is `-UNDEBUG -Wall
-pedantic -g -O0`). Measured on byte-identical content the build is worth
**2.38x**, and the 2-core runner a further **~2.31x** over one developer
machine (`dev/review-2026-08/reports/40-t0-retier.md`). So the audit's 89.7 s
and anything CI reports were never comparable, and T0 drifted for five weeks
without the drift being visible to either number. The reference instrument is
machine (`dev/review-2026-08/reports/40-t0-retier.md`). The audit's 89.7 s
and anything CI reports are not comparable. The runner itself also moves:
between the August and October 2026 images of `ubuntu-latest` every test file
ran 1.4-2x slower (fast-checks runs 31265365814 and 39602904192), which took
the every-run tier from 14 to 25 min of tests. The reference instrument is
the `fast-checks` job's own test phase. When re-tiering, measure from a run
that FINISHED, in that job, and quote minutes.

Expand Down
12 changes: 4 additions & 8 deletions tests/testthat/helper-tiers.R
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,10 @@
# T0 every run (local devtools::test(), and every push/PR)
# Contracts, validation, wiring, the product surface, and the tight
# numerical unit guards that catch a shifted constant in one run.
# No gate. Budget: <= 15 min of tests in the fast-checks job.
#
# State which build any local number came from. devtools::test()
# compiles a debug build (-O0), the tiering audit measured an optimized
# one, and the two differ by 2.38x on identical content -- so the
# audit's "~90 s" cannot be compared with a devtools::test() wall time
# or with what CI prints. The fast-checks job is the reference
# instrument; see .github/workflows/fast-checks.yaml.
# No gate. Budget: the fast-checks job's own test phase (25 min on the
# October 2026 runner, timeout 45 min); see
# .github/workflows/fast-checks.yaml. A local devtools::test() wall
# time is a debug build on a different machine and is not comparable.
#
# CRAN runs a SUBSET of T0: the heavy internal numerical guard files
# are excluded there via skip_heavy_guard_on_cran() below (maintainer
Expand Down
Loading