Repository navigation
Test files that exist but no workflow runs: fleet sweep #39
Description
Generated by node tools/test-invocation.js (PR #38) over an export of every lab's remote default branch, taken today.
The question, and why it is not the zero-test sweep
Issue #4 reads runs that happened and asks whether any reported nothing. This asks whether the run happens at all.
A suite with no caller produces no run, so it produces no zero either. Nothing reports it, every file-comparing checker here stays green, and the lab reads as tested because the tests are sitting in the repository where anyone would look for them. That is the gap: a lab can carry a complete suite, including the cases that check its reject paths, and never execute one of them in CI.
Findings — module labs first
| Lab | In a module | Test files present that nothing runs | Runners the workflows do invoke |
|---|---|---|---|
| bitcoin-script | no | unit (5) | browser |
No lab taught by a course module appears here. That is the ordering doing its job rather than an absence worth celebrating — the sweep was built expecting module labs at the top of it.
crypto-lab-bitcoin-script
package.json declares "test": "vitest run", and there are suites under tests/ — engine, inspect, features, ui, a11y. Four of them contain negative assertions: the cases that check the script engine refuses what it should refuse.
No workflow in that repository runs npm test or vitest. The publishing path waits for npm run test:a11y and nothing else, so a change that breaks a reject path can reach the live site with every gate green. Filed on the lab as crypto-lab-bitcoin-script#24; nothing in that repository was changed.
Labs this sweep could not determine
| Lab | Why this tool could not decide |
|---|
| Scope | Labs |
|---|---|
| Exported default branches read | 211 |
| Test invocation resolved | 211 |
| Could not determine | 0 |
That figure started much larger and came down by reading rather than by relaxing the bar:
- Workflows resolve
npm testandnpm run Xthrough everypackage.jsonin the lab, not only the root one — the older labs keep theirs underdemos/<slug>/andcdinto it. - Local reusable workflows (
uses: ./.github/workflows/…) are followed, because a gate reached through one is still a gate. - A hand-written harness is followed one hop when it is in the repository.
padding-oracle'sscripts/test.mjsdrivesnode:test's own runner oversrc/**, and its header says it exists becausenode --testonce exited 0 having loaded nothing. The lab that had already fixed this class was the one the tool could not read. - A suite counts as invoked when a command names it, whatever the runner is called:
jevilrunstsx scripts/core.test.ts.
Where the chain still left the tool's sight, the lab would be reported here rather than called clean — that is the rule, and today nothing meets it.
What this does not establish
It asks whether a runner is invoked at all. It does not ask whether that runner's own configuration then includes every file: a Playwright project scoped by testMatch, or a testIgnore, can still exclude a spec the runner was told to run. A lab absent from the findings has a caller for each kind of test it carries, and nothing stronger than that.
It also says nothing about whether the tests that do run are good ones. Depth of evidence is the other audit.
Activity
systemslibrarian commentedon Oct 1, 2026
Fixed and closing. The one finding is gated, and fixing it exposed a second defect underneath it.
crypto-lab-bitcoin-script#26 (merged
b065d5c) adds the unit step to the publishing path. The suites pass as they stood — 5 files, 44 cases — so nothing about them needed fixing; they needed calling. Bothdeployanddependabot-auto-mergealready namebuildinneeds:, so they now gate the deploy and the Dependabot merge with no further wiring.The sweep understated the problem. On the pinned node 20 those suites could not run at all. The first CI run on the branch failed: three of the five test files died starting their worker with
webidl.util.markAsUncloneable is not a function, because jsdom requires undici and node 20.20.2 lacks it.npm cihad said so two steps earlier on the same run —@asamuzakjp/css-colorand@asamuzakjp/dom-selectorboth declarenode: ^22.22.2 || ^24.15.0 || >=26.0.0— and those warnings scrolled past unread because nothing downstream of them ran.So the suite was not merely uncalled; it had rotted to unrunnable, and being uncalled is what hid it. The gate now pins node 24, as
crypto-lab-padding-oracledid for the same class of fix. Measured on the merged branch: unit 5 files / 44 cases pass, playwright 43 pass, build job 1m05s.What this does not close.
tools/test-invocation.jsstays in the fleet job, and the limits in the original report stand: it asks whether a runner is invoked, not whether that runner's owntestMatchortestIgnorethen includes every file, and not whether the tests that run are good ones. Depth is #31, which is open.Re-running the sweep today still resolves test invocation for every exported default branch with nothing undetermined, and
bitcoin-scriptis no longer among the findings.