Context
The current alpha test suite is credible: it covers compiler discovery, MCP runtime methods, auth, pagination, widget bundling, platform-specific files, diagnostics, package metadata, and npm install/create smoke tests. Before treating Sidecar as stable, we should raise the bar around generated artifacts and real host compatibility.
Current coverage
- Unit/runtime tests for tools, resources, prompts, auth, proxy, pagination, schema enforcement, and the MCP Apps bridge.
- Compiler tests for reserved files, folder-derived ids/URIs, widget resources, generated clients, platform-specific file selection, and diagnostics.
- E2E build tests for plain MCP, ChatGPT target, Claude plugin output, widget URI cache busting, path containment, and plugin filename/frontmatter sanitization.
- Packaging checks for exports/bins and smoke tests for
npm install sidecar-ai and npm create sidecar-app@latest.
Work to add
- Add golden artifact snapshots for every generated manifest/plugin file.
- Validate emitted MCP/plugin artifacts against official JSON Schemas where available.
- Add live host acceptance tests for ChatGPT and Claude where automation is possible.
- Add browser visual regression tests for native components across light/dark mode and host recipes.
- Add CI smoke tests that install from npm, scaffold an app, run
sidecar check, and run sidecar build.
- Add more adversarial/fuzz tests for JSON-RPC request handling, auth, proxy middleware, and path handling.
Acceptance criteria
- CI has a job that validates generated artifacts structurally, not only through hand-written expectations.
- CI has a published-package smoke test path that exercises
sidecar-ai and create-sidecar-app from npm.
- We have a documented distinction between fast unit tests, build artifact tests, visual tests, and live host acceptance tests.
- Failures identify whether the breakage is compiler output, MCP protocol behavior, host-specific behavior, packaging, or visual/native UI parity.
Context
The current alpha test suite is credible: it covers compiler discovery, MCP runtime methods, auth, pagination, widget bundling, platform-specific files, diagnostics, package metadata, and npm install/create smoke tests. Before treating Sidecar as stable, we should raise the bar around generated artifacts and real host compatibility.
Current coverage
npm install sidecar-aiandnpm create sidecar-app@latest.Work to add
sidecar check, and runsidecar build.Acceptance criteria
sidecar-aiandcreate-sidecar-appfrom npm.