Skip to content

ci(test): run the suite and the typechecker, not just the build (MX-222) - #19

Merged
MGrin merged 1 commit into
mainfrom
mx-222-ci-tests
Aug 21, 2026
Merged

ci(test): run the suite and the typechecker, not just the build (MX-222)#19
MGrin merged 1 commit into
mainfrom
mx-222-ci-tests

Conversation

@MGrin

@MGrin MGrin commented Aug 21, 2026

Copy link
Copy Markdown
Owner

The only CI job here was managed-install. It proves the plugin builds from a git install with runtime deps only — a real, non-obvious check that a runtime import parked in devDependencies would fail. It stays untouched.

It is not a check on whether the code works. bb plugin build bundles without typechecking, and npm test ran only when somebody typed it, so a change breaking all 269 tests still got a green check on its PR.

This adds a second job, deliberately separate rather than more steps on the first: managed-install installs --omit=dev, so typescript and @types/* are gone by the time it ends, and a build failure and a test failure should stay distinguishable at a glance.

Verified locally at this commit: npm ci, npm run typecheck, npm test — all exit 0, 269 pass / 0 fail.

What a green Test run does NOT mean. No test imports server.ts (1678 lines), so the plugin wiring, handlers and schedules are typechecked and never executed. That is the same hole MX-219 measured in bb-plugin-bus, where deleting a call left 23/23 passing. The workflow comment says so, in the file, so the next reader of a green check is not misled by it.

Swept across the siblings in the same pass: bb-plugin-bus already had this (MX-228) and gets only an npm installnpm ci alignment; bb-plugin-system has no tests at all and gets a Typecheck job named for what it actually checks.

The only CI job here was `managed-install`, which proves the plugin builds
from a git install with runtime deps only. That is a real check and it stays.
It is not a check on whether the code works: `bb plugin build` bundles without
typechecking, and `npm test` ran only when somebody typed it, so a change that
broke all 269 tests still got a green check.

Separate job, not extra steps: managed-install installs `--omit=dev`, so
typescript and @types/* are absent by the time it ends, and a build failure
and a test failure should stay distinguishable at a glance.

Verified locally on this commit: `npm ci` + `npm run typecheck` + `npm test`
all exit 0, 269 pass / 0 fail.

What it does not cover, stated in the workflow: no test imports server.ts
(1678 lines), so the wiring, handlers and schedules are typechecked and never
executed — the same hole MX-219 measured in bb-plugin-bus.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@MGrin

MGrin commented Aug 21, 2026

Copy link
Copy Markdown
Owner Author

Negative control — the new job was observed going red, then the branch was deleted.

A gate that has never failed is not known to be a gate. #20 (now closed, branch gone) added one deliberately failing assertion to lib.test.ts and nothing else:

job result run
Test — unit tests and typecheck fail 32442705108not ok 270, # fail 1, exit 1
Managed install pass 32442705101

That row is the whole ticket: with a broken test in the tree, the old job is green. It is not lying — it is answering a different question — but a reader who saw only that check would have merged it.

This branch, unbroken: Test 32442564874# tests 269, # pass 269, # fail 0 — and Managed install 32442564853, both pass.

@MGrin
MGrin merged commit 825884b into main Aug 21, 2026
2 checks passed
@MGrin
MGrin deleted the mx-222-ci-tests branch August 21, 2026 03:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant