Skip to content

Run the unit suites in CI, which nothing in this repository ran - #26

Merged
systemslibrarian merged 2 commits into
mainfrom
ci/run-unit-tests
Oct 1, 2026
Merged

systemslibrarian merged 2 commits into
mainfrom
ci/run-unit-tests

Conversation

@systemslibrarian

@systemslibrarian systemslibrarian commented Oct 1, 2026 •

Copy link
Copy Markdown
Owner

Closes #24. Reported fleet-wide as crypto-lab#39.

package.json has declared "test": "vitest run" for as long as the suites have existed, and no workflow ever called it. The publishing path waited on npm run test:a11y and nothing else, so 5 test files and 44 cases sat in the repository where anyone would look for them and never executed.

Four of the five assert that the script engine rejects what it should reject — the opcode disable list, malformed pushes, inspection of invalid scripts. Before this step, a change that broke a reject path reached the live site with every gate green. That is worse than having no tests, because the tests are visible and read as coverage.

Adding the step found a second defect

On the pinned node 20 these suites cannot run at all. The first CI run on this branch failed: three of the five test files died starting their worker with webidl.util.markAsUncloneable is not a function. jsdom requires undici, whose CacheStorage calls it, and node 20.20.2 does not have it. Two files and 23 cases passed; the other three never executed.

npm ci had said so on that same run, two steps earlier:

npm warn EBADENGINE   package: '@asamuzakjp/css-color@7.0.1',
npm warn EBADENGINE   required: { node: '^22.22.2 || ^24.15.0 || >=26.0.0' },
npm warn EBADENGINE   current: { node: 'v20.20.2', npm: '10.8.2' }

Those warnings scrolled past unread because nothing downstream of them ran. So the suites were not merely uncalled — they had rotted to unrunnable, and the uncalled-ness is what hid it.

The gate now pins node 24: setup-node resolves 24 to a release at or above the 24.15.0 those packages require, and 36 labs in the fleet are already there, including crypto-lab-padding-oracle after the same class of fix.

The step

      - run: npm ci
      - name: Unit tests (vitest)
        run: npm test
      - run: npm run build

Before the build, so a unit failure ends the run in seconds rather than after a Chromium download.

It cannot pass by running nothing. vitest run exits 1 when it matches no files — verified by pointing it at a directory that does not exist. That is the failure mode behind padding-oracle#15, where node --test on node 20 found no TypeScript test files and exited 0 with the gate green.

Measured on this branch

Unit (vitest) 5 files, 44 cases, pass
Accessibility and claims e2e (playwright) 43 pass
Build job 1m05s

It also makes two sentences in this file true

The auto-merge comments say a bump merges only if "this lab's own tests, build and axe/e2e run went green", and that the gate is "unit tests, build, axe WCAG and the e2e claims". The e2e half was already accurate: playwright.config.ts sets testDir: './e2e', so claims.spec.ts runs alongside a11y.spec.ts. The unit half was not, until this step.

Both deploy and dependabot-auto-merge already name build in needs:, so the unit tests gate the deploy and the Dependabot merge with no further wiring.

🤖 Generated with Claude Code

systemslibrarian and others added 2 commits October 1, 2026 18:59
`package.json` has declared `"test": "vitest run"` for as long as the suites
have existed, and no workflow ever called it. The publishing path waited on
`npm run test:a11y` and nothing else, so five test files and 44 cases sat in
the repository where anyone would look for them and never executed. Four of the
five assert that the script engine REJECTS what it should reject -- the opcode
disable list, malformed pushes, inspection of invalid scripts -- so a change
that broke a reject path could reach the live site with every gate green.

They pass as they stand: 5 files, 44 cases, 1.45s. Nothing about them needed
fixing; they needed calling.

The step goes before `npm run build` so a unit failure ends the run in seconds
rather than after a Chromium download. `vitest run` exits 1 when it matches no
files -- verified here by pointing it at a directory that does not exist -- so
this step cannot pass by silently running nothing, which is the failure mode
that made crypto-lab-padding-oracle#15 necessary.

It also makes two sentences in this file true that were not. The auto-merge
comments say a bump merges only if "this lab's own tests, build and axe/e2e run
went green" and that the gate is "unit tests, build, axe WCAG and the e2e
claims". The e2e half was accurate -- `playwright.config.ts` has
`testDir: './e2e'`, so `claims.spec.ts` runs alongside `a11y.spec.ts`. The unit
half was not, until this step.

Both `deploy` and `dependabot-auto-merge` already name `build` in `needs:`, so
the unit tests now gate the deploy and the Dependabot merge with no further
wiring.

Found by `tools/test-invocation.js`, reported as crypto-lab#39 and
crypto-lab-bitcoin-script#24.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adding the unit step revealed more than an uncalled suite: on the pinned node
20 the suites cannot run at all. Three of the five test files failed to start
their worker with "webidl.util.markAsUncloneable is not a function" -- jsdom
requires undici, whose CacheStorage calls it, and node 20.20.2 does not have it.
Two files and 23 cases passed; the other three never executed, and vitest exited
1 with "3 errors".

`npm ci` said so in advance on the same run: @asamuzakjp/css-color@7.0.1 and
@asamuzakjp/dom-selector@9.2.1 both declare
`node: ^22.22.2 || ^24.15.0 || >=26.0.0`, and the warnings scrolled past
unread because nothing downstream of them ran.

24 rather than 22 because setup-node resolves 24 to a release at or above the
24.15.0 these packages require, and 36 labs in the fleet are already there,
including crypto-lab-padding-oracle after the same class of fix.

This is the cost of a suite with no caller stated precisely: it was not merely
unrun, it had rotted to unrunnable, and the one signal that would have said so
was a warning on a step whose output nobody had reason to read.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unit suites do not run in CI, so the publishing path is gated on the accessibility scan alone

1 participant