Skip to content

Qualify the 0.24.0 test-suite speedup — 2.6x was an idle-machine number #21

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cfdude merged 1 commit into from
Jul 30, 2026
Merged

Qualify the 0.24.0 test-suite speedup — 2.6x was an idle-machine number #21

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion changelog.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ All notable changes to the `pm` plugin, mirroring [CHANGELOG.md](https://github.

### Changed

- **The test suite is now `scripts/test/*.test.mjs` (11 files) instead of one `scripts/conductor.test.mjs`, cutting a full run from ~118s to ~46s.** Contributor-facing: the command is `node --test scripts/test/*.test.mjs`. Measured rather than guessed — one engine spawn costs ~73ms and the suite makes 676 of them, so ~49s was pure `node` startup, serialized in a single file on a 16-core machine; `node --test` parallelizes across _files_ only. The split is verbatim, verified by test-name-set equality (255 → 255). Note `node --test scripts/test` (a directory) does **not** work — Node treats the argument as a module to execute and dies with `MODULE_NOT_FOUND` while reporting "1 test".
- **The test suite is now `scripts/test/*.test.mjs` (11 files) instead of one `scripts/conductor.test.mjs`, so it runs in parallel.** Contributor-facing: the command is `node --test scripts/test/*.test.mjs`. Measured rather than guessed — one engine spawn costs ~73ms and the suite makes 676 of them, so ~49s was pure `node` startup, serialized in a single file on a 16-core machine; `node --test` parallelizes across _files_ only. The split is verbatim, verified by test-name-set equality (255 → 255). Note `node --test scripts/test` (a directory) does **not** work — Node treats the argument as a module to execute and dies with `MODULE_NOT_FOUND` while reporting "1 test".
**How much faster depends entirely on free cores, so treat any single headline number with salt.** Measured on this project: an otherwise-idle 16-core workstation went ~118s → ~46s (2.6×), but the _same_ machine under load (~12) went ~146s → ~122s — only 1.2×, because parallel execution is exactly what suffers when cores are already contended. On CI, whole-workflow time went from ~52–58s to ~38–48s (~1.3×). On a busy laptop or a few-core runner, expect the low end.
- **`.githooks/pre-commit` now aborts if the suite runs fewer tests than are declared.** The glob makes a partial-suite pass possible in a way the single file never did: if a file stops matching, everything still goes green, just on a subset. The hook cross-checks the runner's count against `grep -Hc '^test('` across the files — self-maintaining, with no constant to bump as tests are added.

### Fixed
Expand Down