Skip to content

test: build the native crate's suites unoptimised - #585

Merged
MarcusKainth merged 1 commit into
mainfrom
test/native-suites-unoptimised
Sep 25, 2026
Merged

MarcusKainth merged 1 commit into
mainfrom
test/native-suites-unoptimised

Conversation

@MarcusKainth

@MarcusKainth MarcusKainth commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

What this changes, and why

[profile.test] sets opt-level 2 for the emulator's fixture and trace suites. The native crate got it too, and its test binaries are the largest compile cost in build-tests: 876 CPU seconds of the workspace's own units at opt-level 2, against 200 at opt-level 0 (cargo --timings from the measurement run 36121287631). The native crate builds SQL text and its suites wait on the server, so this gives it opt-level 0 through [profile.test.package.clickdoom-native]. Every other crate keeps opt-level 2, refemu included.

Evidence

The generated SQL is byte-identical

A scratch test, not committed, hashing everything the crate generates:

let (a, b) = tick::resident_statements("nat");            // xxh64 of each
tick::run_statement("nat", &[Input { tic: 7, source: 1, keys: 3, mouse: (1, -2) }]);
tick::demo_statement("nat", 1, 40);                         // sql, body, settings
load::plan + sql::level_statements("nat","E1M1","demo3") + sql::render_statements("nat","SKY1");
sql::parity::first_divergence("nat") + sql::parity::field_summary("nat");

Run under each profile, with the level confirmed from cargo test -v (-C opt-level=2, =1, and no flag for 0):

$ bash cmphash.sh; echo "exit=$?"
hash-o2        9 lines 84e10ba884417051
hash-o1        9 lines 84e10ba884417051
hash-o0        9 lines 84e10ba884417051
identical across opt-level 2, 1, 0
exit=0

stage1 2179678 ddbc3c3eca51187b
stage2 203311 1040c854b1969203
load+level+render e5990d653f124421
parity 90f73aa513156ae4

Rerun on a Linux runner in the throwaway run 36123894500. cargo test -v showed opt-level=2 for the first build and no flag for the second:

opt-level 2: 0ad7d300a3f5c0a9b57e373676c8ecbaaeb2a274bec5255a56fa921f985122cc  -
stage1 2179678 ddbc3c3eca51187b
stage2 203311 1040c854b1969203
opt-level 0: 0ad7d300a3f5c0a9b57e373676c8ecbaaeb2a274bec5255a56fa921f985122cc  -
stage1 2179678 ddbc3c3eca51187b
stage2 203311 1040c854b1969203
identical at opt-level 2 and 0

Compile time, from this PR's CI run 36123100020

build-tests restored the same rust-cache key main uses (full match: true), so only the workspace crates compiled:

before: run 36116301376 (PR #580) this PR
Finished test profile 6 m 17 s 2 m 52 s
Archive the suites (with the ROM suites archive) 448 s 235 s

cargo --timings in the measurement run: the native crate's test units took 876 CPU s at opt-level 2, 607 at opt-level 1 and 200 at opt-level 0.

Suite times

The emulator group doesn't regress. These are test-seconds summed from each run's nextest PASS lines. Main's run and #584's are on this PR's base commit, and #580's is on an earlier commit. Together they show the spread from one runner to the next:

emulator 36116301376 (#580) 36122388569 (main) 36122982586 (#584) this PR
sqlcpu_live the_sql_cpu_passes_its_own_suite 135.9 101.3 130.1 134.2
demo3_parity 29.0 22.2 29.0 29.0
run_live 15.9 12.0 15.8 15.6
group sum 250.5 197.2 243.8 251.1

refemu's own suites, which this PR can't touch, moved from 36.8 to 48.5 test-seconds between main's run and this one. That is the runner-to-runner spread.

native-rest's live suites match run 36116301376 within a few seconds each (group sum 653.1 there, 685.7 here). The pure-Rust statement tests get slower at opt-level 0: every_cut_builds_a_statement_the_server_could_run goes from 3.2 to 5.0 s to 15.3 s, and no_cut_statement_holds_a_comparison_operand from 1.6 to 2.8 s to 8.2 s. About 15 test-seconds in all, in a group that runs two at a time. The simulation groups' sums fall inside the spread between runs (sim-f: 908.8 main, 969.7 here).

Every group passed and ci-passed succeeded.

Invariants

None. Only the test profile of one crate changes. opt-level doesn't change what safe Rust computes, and the hashes above show the SQL text is unchanged.

Spec impact

  • None. No contract in SPEC.md is touched

Checks

  • make gates: not run locally. CI's lint job ran make lint and every test group passed
  • No AI attribution trailers in the commits

Written mostly by Claude Opus 5.5.

The test profile's opt-level 2 is there for the emulator's fixture and
trace suites, which run millions of instructions in Rust. The native
crate builds SQL text and its suites spend their time waiting on the
server, so it gets opt-level 0 through a package override. Every other
crate keeps opt-level 2, the reference emulator included.

On a standard runner the native crate's 57 test binaries took 876 CPU
seconds to compile at opt-level 2 and 200 at opt-level 0 (cargo
--timings, CI run 36121287631). Its non-live suites ran in 0.87 s at
opt-level 2 and 1.98 s at opt-level 0 locally.

opt-level does not change what safe Rust computes. The generated text is
byte-identical anyway: xxh64 of both resident statements, run_statement,
demo_statement, the load, level and render plans and the parity queries
match at opt-level 2, 1 and 0.
@MarcusKainth
MarcusKainth force-pushed the test/native-suites-unoptimised branch from 247ad37 to 97b2940 Compare September 25, 2026 11:31
@MarcusKainth
MarcusKainth marked this pull request as ready for review September 25, 2026 11:44
@MarcusKainth
MarcusKainth merged commit 3e336a7 into main Sep 25, 2026
21 checks passed
@MarcusKainth
MarcusKainth deleted the test/native-suites-unoptimised branch September 25, 2026 11:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci Workflows, the Makefile, and the scripts they run

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant