ci: split the test archive by crate - #586
Merged
Merged
Conversation
This was referenced Sep 25, 2026
build-tests made one archive holding every crate's test binaries and one holding the reference emulator's release ROM suites, one after the other, and every test job downloaded both. A simulation group needs only the native crate's suites. build-tests is now a matrix of two archives built side by side: native.tar.zst with the native crate's suites and workspace.tar.zst with every other crate's. The ROM suites archive moves to build-refemu, which already builds the reference emulator in release and is off the path to the test jobs. Each group downloads only what scripts/test-group.sh runs it from: the simulation groups the native archive, the emulator group the workspace and ROM suites archives, and native-rest both, since it runs the native crate's other suites and the driver's native_* suites. native-rest runs them as two nextest invocations, one per archive, at the same two threads. Measured on the throwaway run 36122853256 with a warm cache, no debug info and the native crate's suites at opt-level 0, the native archive built in 48 s (119 MB) and the other crates' in 94 s (72 MB), against 2 m 16 s (206 MB) for the single archive under the same settings. Each group selects the same tests as before: listing every group's filter over the workspace before the split and over the two archives after gives the same test lists for all eight groups, 953 tests in all, none in two groups.
MarcusKainth
force-pushed
the
ci/split-test-archives
branch
from
September 25, 2026 11:21
0ac7839 to
712e4eb
Compare
MarcusKainth
marked this pull request as ready for review
September 25, 2026 11:31
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #584 (
ci/archive-no-debuginfo); the diff against that branch is this one commit.What this changes, and why
Every test job downloaded one archive holding every crate's test binaries, and
build-testsbuilt that archive and the ROM suites archive one after the other. A simulation group needs only the native crate's suites.build-testsis a matrix of two archives built side by side:native.tar.zst(the native crate's suites) andworkspace.tar.zst(every other crate's). Each leg has its own rust-cache key.build-refemu, which already builds the reference emulator in release and isn't on the path to the test jobs.scripts/test-group.shruns it from: the simulation groups get the native archive,emulatorgets the workspace and ROM suites archives, andnative-restgets both. It runs the native crate's other suites and the driver'snative_*suites as two nextest invocations, one per archive, still at two threads each.native_connections_livestays inemulatorat--test-threads 1.ci-passedstill needsbuild-tests, which now reports the result of both legs together.Evidence
Each group selects the same tests
Every group's filter over the whole workspace before the split, against the split filters over the two archives after (
nextest list --message-format oneline, sorted, compared withcmp), with local archives built the way CI builds them:This PR's CI run 36124399424
Each build leg has a new rust-cache key, so this run compiled the dependencies too. The warm figures for the same split come from the measurement run 36122853256, with the native crate at opt-level 0 (#585): 48 s for the native archive and 94 s for the rest.
emulatortest-archives191,650,427 Bnative-tests101,225,013 B,workspace-tests70,556,324 B,rom-suites5,064,897 BThe nextest summaries show every group running what it ran before:
The whole run took 9 min 15 s from the first job's start (10:31:02) to
ci-passed(10:40:17).native-restran in 220 s as two invocations, against 356 s in #584's attempt 2 and 276 s in #584's attempt 1.Invariants
None. CI layout only; the same tests run, from different archives.
Spec impact
Checks
make gates: not run locally. CI's lint job ranmake lintand every test group passedWritten mostly by Claude Opus 5.5.