Unblock first-plugin authoring: testable scaffold, correct SDK example, visible npm failure - #2550
Open
t1mdurden wants to merge 3 commits into
Open
Unblock first-plugin authoring: testable scaffold, correct SDK example, visible npm failure#2550t1mdurden wants to merge 3 commits into
t1mdurden wants to merge 3 commits into
Conversation
added 3 commits
August 27, 2026 19:00
`@get-bb/plugin-sdk/testing` — the backend harness the bb-plugin-authoring skill tells plugin authors to test with — imports better-sqlite3, cron-parser, hono and zod. All four are OPTIONAL peers of the SDK, so npm installs none of them; the scaffold has to declare them itself. It declared three. A fresh scaffold's first backend test therefore died on `Cannot find package 'cron-parser'`, pointing at a file inside node_modules with nothing in the guide to explain it. The regression test derives the rule from the harness's own imports and the SDK's peer list rather than restating it, so a new import in the harness cannot leave it stale. testing/app is deliberately out of scope: React, Testing Library and jsdom are the author's to add, and the skill says so.
`bb.sdk.threads.list` resolves to ThreadListResponse — a plain array. The
skill showed `const { threads } = await bb.sdk.threads.list(...)`, so every
reader who copied it got `undefined` and no error, then went looking through
the 700KB of bundled declarations to find out why.
plugin-authoring-docs.test.ts already proves the skill mentions every API
member; nothing proved its examples compile. The new test generates a probe
from SKILL.md and runs tsc over it: indexing the SDK type proves each
documented `bb.sdk.<path>(` exists, and each destructured await is checked
against what that method really returns. Arguments are not reconstructed —
only what the example claims to call and receive is under test.
The scaffold install is best-effort, but the catch swallowed npm's output and printed only "Could not run npm install". npm's own message is the whole diagnosis — an unwritable cache, a refused proxy, a missing platform binary — so the author had to rerun the same command by hand to read what the CLI already had. Quote the tail of npm's output under the warning. `npm` missing from PATH still prints nothing extra, since that failure produces no output.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Human comments
What was wrong
Three independent defects, all of which land on the same person: someone writing their first bb plugin by following the bundled
bb-plugin-authoringskill.@get-bb/plugin-sdk/testingimportsbetter-sqlite3,cron-parser,honoandzod. All four are optional peers of the SDK, so npm installs none of them and the scaffold has to declare them itself — it declared three. A fresh scaffold's firstcreateFakePluginHosttest dies onCannot find package 'cron-parser', pointing insidenode_moduleswith nothing in the guide to explain it. (bb plugin new scaffolds a plugin that cannot import @get-bb/plugin-sdk/testing: cron-parser is undeclared #2546)bb.sdk.threads.listexample is wrong. It showsconst { threads } = await bb.sdk.threads.list(...); the method resolves toThreadListResponse, a plain array. The call succeeds, nothing throws, and the reader getsundefined. (bb-plugin-authoring skill documents bb.sdk.threads.list as returning { threads }; it returns an array #2547)bb plugin newthrows away npm's failure output. The catch printed onlyCould not run npm install …, so the author had to rerun the same command by hand to read the reason the CLI already had — an unwritable cache, a refused proxy, a missing platform binary. (bb plugin new discards npm's failure output, so a failed scaffold install gives no reason #2548)What changed
packages/templates/src/plugin-scaffold.ts— declarescron-parseralongsidebetter-sqlite3andhonoin the scaffold'sdevDependencies, and the comment above the block now says what that group is for.apps/cli/src/__tests__/plugin-scaffold-dependencies.test.ts— new test: every optional peer the backend harness imports must be declared in the scaffold manifest. It reads the harness's own imports and the SDK'speerDependenciesrather than restating a list, so a new import in the harness cannot leave it stale.testing/appis deliberately out of scope — React, Testing Library and jsdom stay the author's to add, as the skill says..../builtin-skills/bb-plugin-authoring/SKILL.md— one line:const threads = await bb.sdk.threads.list(...).apps/server/test/services/plugins/plugin-authoring-doc-examples.test.ts(new) — the durable half.plugin-authoring-docs.test.tsalready proves the skill mentions every API member; nothing proved its examples compile. This generates a probe fromSKILL.mdand runstscover it: indexing the SDK type proves each documentedbb.sdk.<path>(exists and is callable, and each destructured await is checked against what that method really returns. Arguments are not reconstructed — only what the example claims to call and receive is under test, which is exactly the class of bug that slipped through. The probe is generated at run time, so a new example is covered the moment it is written.apps/cli/src/commands/plugin.ts— quotes the tail of npm's own output under the existing warning.npmmissing from PATH still prints nothing extra, since that failure produces no output.apps/cli/src/__tests__/helpers/fake-npm.ts— newBB_TEST_NPM_INSTALL=failmode: exits non-zero with npm's own explanation on stderr, standing in for the installs that die before touching the tree.No wire, protocol, CLI-surface or guide changes beyond the corrected line.
How you verified
Every test fails before its fix and passes after. Base commit
ad79bbb.cron-parser, with the scaffold line reverted:threads.list, with the doc line reverted:npm's reason, with the detail suppressed:
Suites, after:
apps/cli—Test Files 50 passed (50) / Tests 498 passed (498)apps/servertest/services/plugins/plugin-authoring-docs.test.ts+ the new file —Test Files 2 passed (2) / Tests 16 passed (16)tsc --noEmitclean inapps/cli,packages/templates,apps/serveroxfmtrun over every touched fileOne caveat, stated plainly:
packages/templates' own suite fails 11 tests withERR_REQUIRE_ESMon my machine, and it fails identically on a cleangit stashofmainatad79bbb— same 3 files, same 11 tests. Pre-existing in my environment (Node v20.13.1, macOS arm64), not caused by this change, and I did not chase it.Fixes #2546
Fixes #2547
Fixes #2548