diff --git a/.github/workflows/fast-checks.yaml b/.github/workflows/fast-checks.yaml index 714529a9..e92812ea 100644 --- a/.github/workflows/fast-checks.yaml +++ b/.github/workflows/fast-checks.yaml @@ -10,19 +10,11 @@ # Every push to develop and every pull request. No gate, so the run is # exactly what a maintainer sees locally: contracts, validation, wiring, # the product surface, and the tight numerical unit guards that catch a -# shifted constant in one run. Budget: <= 15 min of tests in THIS job, -# which keeps the whole job near half its 30-minute timeout. -# -# Read the budget off this job's own test phase. The tiering audit's -# "~90 s" is a different measurement -- an OPTIMIZED build on a developer -# machine -- and nothing this job prints can be compared against it. -# devtools::test() compiles a debug build (pkgbuild::compiler_flags(debug -# = TRUE) gives "-UNDEBUG -Wall -pedantic -g -O0"), and this job keeps it -# that way on purpose (see the extra-packages note below). Measured on -# identical content that build costs 2.38x, and this 2-core runner a -# further ~2.31x over one developer machine. So quote CI minutes here, or -# say which build a local number came from; a bare seconds figure is not -# comparable across the two and let this tier drift unnoticed once. +# shifted constant in one run. The budget is this job's own test phase: +# 25 min on the October 2026 ubuntu-latest runner (run 39602904192), +# against timeout-minutes: 45. A local devtools::test() wall time is +# not comparable: it compiles a debug build (-O0) on a different +# machine. Measure from a run of this job that finished. # # T1 nightly .github/workflows/nightly-validation.yaml. Daily 03:00 UTC # on develop, BGMS_RUN_SLOW_TESTS=true, <= 60 min, timeout 90. @@ -48,9 +40,9 @@ on: jobs: fast-tests: - name: T0 every-run tier (~90 s of tests) + name: T0 every-run tier runs-on: ubuntu-latest - timeout-minutes: 30 + timeout-minutes: 45 env: GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} R_KEEP_PKG_SOURCE: yes diff --git a/.github/workflows/nightly-validation.yaml b/.github/workflows/nightly-validation.yaml index 065d0c8c..88d3fa88 100644 --- a/.github/workflows/nightly-validation.yaml +++ b/.github/workflows/nightly-validation.yaml @@ -8,7 +8,7 @@ # # T0 every run local devtools::test() + .github/workflows/fast-checks.yaml # Contracts, validation, wiring, the product surface, the tight numerical -# unit guards. No gate. Budget ~90 s. +# unit guards. No gate; budget stated in fast-checks.yaml. # # T1 nightly THIS FILE. Daily 03:00 UTC on develop. # BGMS_RUN_SLOW_TESTS=true. diff --git a/.github/workflows/weekly-certification.yaml b/.github/workflows/weekly-certification.yaml index 52736b4b..f96d4d0c 100644 --- a/.github/workflows/weekly-certification.yaml +++ b/.github/workflows/weekly-certification.yaml @@ -8,7 +8,7 @@ # # T0 every run local devtools::test() + .github/workflows/fast-checks.yaml # Contracts, validation, wiring, the product surface, the tight numerical -# unit guards. No gate. Budget ~90 s. +# unit guards. No gate; budget stated in fast-checks.yaml. # # T1 nightly .github/workflows/nightly-validation.yaml. Daily 03:00 UTC. # BGMS_RUN_SLOW_TESTS=true. diff --git a/dev/review-2026-08/MAINTAINERS.md b/dev/review-2026-08/MAINTAINERS.md index 299dae5e..6e3ad876 100644 --- a/dev/review-2026-08/MAINTAINERS.md +++ b/dev/review-2026-08/MAINTAINERS.md @@ -218,7 +218,7 @@ be T1. Budget is the tie-breaker, never the criterion. | Tier | Cadence | Content, by bug class | Budget | Gate | |---|---|---|---|---| -| **T0** every run | local `devtools::test()`; every push to develop and every PR (`fast-checks.yaml`) | contracts, validation, wiring, unit guards, product surface | **≤ 15 min of tests** in the `fast-checks` job (`timeout-minutes: 30`) | none | +| **T0** every run | local `devtools::test()`; every push to develop and every PR (`fast-checks.yaml`) | contracts, validation, wiring, unit guards, product surface | the `fast-checks` job's own test phase: 25 min on the October 2026 runner (`timeout-minutes: 45`) | none | | **T1** nightly heartbeat | daily 03:00 UTC on **develop** (`nightly-validation.yaml`) | curated calibration subset — graph-law and prior-chain identities, gauge detector, single surface-vs-gold cells, RB saturation, concordance smokes. "Does the settled math still hold tonight" | **≤ 60 min** on the 2-core runner; `timeout-minutes: 90` | `BGMS_RUN_SLOW_TESTS=true` | | **T2** weekly certification | Sunday 03:00 UTC on **develop** (`weekly-certification.yaml`) | the heavy Monte-Carlo machinery — SBC suites, full parameter-recovery sweeps, full NUTS-vs-MH condition grids, n = 2e6 MC channels, refit cross-validations (incl. the F-049 gate) | `timeout-minutes: 360` | `BGMS_RUN_CERTIFICATION=true` (the T2 workflow sets **both** vars, so a weekly run also carries T1) | @@ -233,9 +233,11 @@ machine and landed 89.7 s; `devtools::test()` and every CI tier compile a *debug* build (`pkgbuild::compiler_flags(debug = TRUE)` is `-UNDEBUG -Wall -pedantic -g -O0`). Measured on byte-identical content the build is worth **2.38x**, and the 2-core runner a further **~2.31x** over one developer -machine (`dev/review-2026-08/reports/40-t0-retier.md`). So the audit's 89.7 s -and anything CI reports were never comparable, and T0 drifted for five weeks -without the drift being visible to either number. The reference instrument is +machine (`dev/review-2026-08/reports/40-t0-retier.md`). The audit's 89.7 s +and anything CI reports are not comparable. The runner itself also moves: +between the August and October 2026 images of `ubuntu-latest` every test file +ran 1.4-2x slower (fast-checks runs 31265365814 and 39602904192), which took +the every-run tier from 14 to 25 min of tests. The reference instrument is the `fast-checks` job's own test phase. When re-tiering, measure from a run that FINISHED, in that job, and quote minutes. diff --git a/tests/testthat/helper-tiers.R b/tests/testthat/helper-tiers.R index c3ee89a6..983e89b6 100644 --- a/tests/testthat/helper-tiers.R +++ b/tests/testthat/helper-tiers.R @@ -7,14 +7,10 @@ # T0 every run (local devtools::test(), and every push/PR) # Contracts, validation, wiring, the product surface, and the tight # numerical unit guards that catch a shifted constant in one run. -# No gate. Budget: <= 15 min of tests in the fast-checks job. -# -# State which build any local number came from. devtools::test() -# compiles a debug build (-O0), the tiering audit measured an optimized -# one, and the two differ by 2.38x on identical content -- so the -# audit's "~90 s" cannot be compared with a devtools::test() wall time -# or with what CI prints. The fast-checks job is the reference -# instrument; see .github/workflows/fast-checks.yaml. +# No gate. Budget: the fast-checks job's own test phase (25 min on the +# October 2026 runner, timeout 45 min); see +# .github/workflows/fast-checks.yaml. A local devtools::test() wall +# time is a debug build on a different machine and is not comparable. # # CRAN runs a SUBSET of T0: the heavy internal numerical guard files # are excluded there via skip_heavy_guard_on_cran() below (maintainer