ci(test): run the suite and the typechecker, not just the build (MX-222) - #19
Merged
Conversation
The only CI job here was `managed-install`, which proves the plugin builds from a git install with runtime deps only. That is a real check and it stays. It is not a check on whether the code works: `bb plugin build` bundles without typechecking, and `npm test` ran only when somebody typed it, so a change that broke all 269 tests still got a green check. Separate job, not extra steps: managed-install installs `--omit=dev`, so typescript and @types/* are absent by the time it ends, and a build failure and a test failure should stay distinguishable at a glance. Verified locally on this commit: `npm ci` + `npm run typecheck` + `npm test` all exit 0, 269 pass / 0 fail. What it does not cover, stated in the workflow: no test imports server.ts (1678 lines), so the wiring, handlers and schedules are typechecked and never executed — the same hole MX-219 measured in bb-plugin-bus. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Owner
Author
|
Negative control — the new job was observed going red, then the branch was deleted. A gate that has never failed is not known to be a gate. #20 (now closed, branch gone) added one deliberately failing assertion to
That row is the whole ticket: with a broken test in the tree, the old job is green. It is not lying — it is answering a different question — but a reader who saw only that check would have merged it. This branch, unbroken: Test 32442564874 — |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The only CI job here was
managed-install. It proves the plugin builds from a git install with runtime deps only — a real, non-obvious check that a runtime import parked in devDependencies would fail. It stays untouched.It is not a check on whether the code works.
bb plugin buildbundles without typechecking, andnpm testran only when somebody typed it, so a change breaking all 269 tests still got a green check on its PR.This adds a second job, deliberately separate rather than more steps on the first:
managed-installinstalls--omit=dev, so typescript and@types/*are gone by the time it ends, and a build failure and a test failure should stay distinguishable at a glance.Verified locally at this commit:
npm ci,npm run typecheck,npm test— all exit 0, 269 pass / 0 fail.What a green Test run does NOT mean. No test imports
server.ts(1678 lines), so the plugin wiring, handlers and schedules are typechecked and never executed. That is the same hole MX-219 measured in bb-plugin-bus, where deleting a call left 23/23 passing. The workflow comment says so, in the file, so the next reader of a green check is not misled by it.Swept across the siblings in the same pass: bb-plugin-bus already had this (MX-228) and gets only an
npm install→npm cialignment; bb-plugin-system has no tests at all and gets a Typecheck job named for what it actually checks.