Skip to content

Unblock first-plugin authoring: testable scaffold, correct SDK example, visible npm failure - #2550

Open
t1mdurden wants to merge 3 commits into
get-bb:mainfrom
t1mdurden:plugin-authoring-friction
Open

Unblock first-plugin authoring: testable scaffold, correct SDK example, visible npm failure#2550
t1mdurden wants to merge 3 commits into
get-bb:mainfrom
t1mdurden:plugin-authoring-friction

Conversation

@t1mdurden

Copy link
Copy Markdown

Human comments

What was wrong

Three independent defects, all of which land on the same person: someone writing their first bb plugin by following the bundled bb-plugin-authoring skill.

  1. The scaffold cannot import the test harness the guide recommends. @get-bb/plugin-sdk/testing imports better-sqlite3, cron-parser, hono and zod. All four are optional peers of the SDK, so npm installs none of them and the scaffold has to declare them itself — it declared three. A fresh scaffold's first createFakePluginHost test dies on Cannot find package 'cron-parser', pointing inside node_modules with nothing in the guide to explain it. (bb plugin new scaffolds a plugin that cannot import @get-bb/plugin-sdk/testing: cron-parser is undeclared #2546)
  2. The guide's bb.sdk.threads.list example is wrong. It shows const { threads } = await bb.sdk.threads.list(...); the method resolves to ThreadListResponse, a plain array. The call succeeds, nothing throws, and the reader gets undefined. (bb-plugin-authoring skill documents bb.sdk.threads.list as returning { threads }; it returns an array #2547)
  3. bb plugin new throws away npm's failure output. The catch printed only Could not run npm install …, so the author had to rerun the same command by hand to read the reason the CLI already had — an unwritable cache, a refused proxy, a missing platform binary. (bb plugin new discards npm's failure output, so a failed scaffold install gives no reason #2548)

What changed

packages/templates/src/plugin-scaffold.ts — declares cron-parser alongside better-sqlite3 and hono in the scaffold's devDependencies, and the comment above the block now says what that group is for.

apps/cli/src/__tests__/plugin-scaffold-dependencies.test.ts — new test: every optional peer the backend harness imports must be declared in the scaffold manifest. It reads the harness's own imports and the SDK's peerDependencies rather than restating a list, so a new import in the harness cannot leave it stale. testing/app is deliberately out of scope — React, Testing Library and jsdom stay the author's to add, as the skill says.

.../builtin-skills/bb-plugin-authoring/SKILL.md — one line: const threads = await bb.sdk.threads.list(...).

apps/server/test/services/plugins/plugin-authoring-doc-examples.test.ts (new) — the durable half. plugin-authoring-docs.test.ts already proves the skill mentions every API member; nothing proved its examples compile. This generates a probe from SKILL.md and runs tsc over it: indexing the SDK type proves each documented bb.sdk.<path>( exists and is callable, and each destructured await is checked against what that method really returns. Arguments are not reconstructed — only what the example claims to call and receive is under test, which is exactly the class of bug that slipped through. The probe is generated at run time, so a new example is covered the moment it is written.

apps/cli/src/commands/plugin.ts — quotes the tail of npm's own output under the existing warning. npm missing from PATH still prints nothing extra, since that failure produces no output.

apps/cli/src/__tests__/helpers/fake-npm.ts — new BB_TEST_NPM_INSTALL=fail mode: exits non-zero with npm's own explanation on stderr, standing in for the installs that die before touching the tree.

No wire, protocol, CLI-surface or guide changes beyond the corrected line.

How you verified

Every test fails before its fix and passes after. Base commit ad79bbb.

cron-parser, with the scaffold line reverted:

× declares every optional peer the backend test harness imports
  AssertionError: expected [ 'cron-parser' ] to deeply equal []

threads.list, with the doc line reverted:

× calls and destructures bb.sdk the way the SDK actually declares it
  probe.ts(24,9): error TS2339: Property 'threads' does not exist on type
    '{ id: string; projectId: string; environmentId: string | null; … }'.

npm's reason, with the detail suppressed:

× passes npm's own reason through when the install fails
  AssertionError: expected 'Could not run npm install — run it in…'
    to contain 'npm error code EPERM'

Suites, after:

  • apps/cliTest Files 50 passed (50) / Tests 498 passed (498)
  • apps/server test/services/plugins/plugin-authoring-docs.test.ts + the new file — Test Files 2 passed (2) / Tests 16 passed (16)
  • tsc --noEmit clean in apps/cli, packages/templates, apps/server
  • oxfmt run over every touched file

One caveat, stated plainly: packages/templates' own suite fails 11 tests with ERR_REQUIRE_ESM on my machine, and it fails identically on a clean git stash of main at ad79bbb — same 3 files, same 11 tests. Pre-existing in my environment (Node v20.13.1, macOS arm64), not caused by this change, and I did not chase it.

Fixes #2546
Fixes #2547
Fixes #2548

Timmy-Lane added 3 commits August 27, 2026 19:00
`@get-bb/plugin-sdk/testing` — the backend harness the bb-plugin-authoring
skill tells plugin authors to test with — imports better-sqlite3, cron-parser,
hono and zod. All four are OPTIONAL peers of the SDK, so npm installs none of
them; the scaffold has to declare them itself. It declared three.

A fresh scaffold's first backend test therefore died on `Cannot find package
'cron-parser'`, pointing at a file inside node_modules with nothing in the
guide to explain it.

The regression test derives the rule from the harness's own imports and the
SDK's peer list rather than restating it, so a new import in the harness cannot
leave it stale. testing/app is deliberately out of scope: React, Testing
Library and jsdom are the author's to add, and the skill says so.
`bb.sdk.threads.list` resolves to ThreadListResponse — a plain array. The
skill showed `const { threads } = await bb.sdk.threads.list(...)`, so every
reader who copied it got `undefined` and no error, then went looking through
the 700KB of bundled declarations to find out why.

plugin-authoring-docs.test.ts already proves the skill mentions every API
member; nothing proved its examples compile. The new test generates a probe
from SKILL.md and runs tsc over it: indexing the SDK type proves each
documented `bb.sdk.<path>(` exists, and each destructured await is checked
against what that method really returns. Arguments are not reconstructed —
only what the example claims to call and receive is under test.
The scaffold install is best-effort, but the catch swallowed npm's output and
printed only "Could not run npm install". npm's own message is the whole
diagnosis — an unwritable cache, a refused proxy, a missing platform binary —
so the author had to rerun the same command by hand to read what the CLI
already had.

Quote the tail of npm's output under the warning. `npm` missing from PATH
still prints nothing extra, since that failure produces no output.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant