diff --git a/CHANGELOG.md b/CHANGELOG.md index 93ab9e3..8dfe509 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,18 @@ Format: [Keep a Changelog](https://keepachangelog.com). Versioning: semver — for skills *and* for this CLI, breaking prompt changes are breaking changes. +## [0.20.0] — 2026-08-09 + +The token-cost argument, applied to MCP — the largest standing-context line item Kitbash was not measuring. + +A skill body's cost is statically countable, which is why `context.budget` can be enforced at compile time. An MCP server's real cost is not: the tokens it adds are the JSON schema of every tool it exposes, and knowing those means asking the server — which for a stdio server means **executing it**. That is exactly what the install gate exists to prevent, so Kitbash does not do it, and does not print an estimate it cannot derive. + +What it can count exactly is the thing the manifest already states. Every tool in a declared allowlist is a tool definition the agent carries for the whole session, so the allowlist is an exact **floor** on the cost. + +### Added +- **MCP tool budget, reported on every compile and by `doctor`** — `MCP tool budget: 8 tool(s) across 2 server(s); cap 100`, alongside an explicit statement that the token cost is unmeasured and why. A server declaring `tools = ["*"]` has no countable bound, so the total renders as `8+ (1 server(s) declare "*")` and warns that it cannot be bounded, rather than folding into a total that would be wrong. +- **`max_mcp_tools` in `[policy]`** — caps declared MCP tools across all skills. A breach warns and fails `--strict`. The default ceiling is **100**, the one primary-source numeric limit found across the clients (Windsurf documents it as a hard cap past which tools are dropped). A widely-repeated "40 tool" Cursor limit is deliberately **not** encoded: it appears only in forum posts and blogs, never in Cursor's own docs, and a tool whose pitch is measurement must not ship a folklore number. + ## [0.19.1] — 2026-08-09 Follow-through on 0.19.0: policy gets a say over MCP servers, and the support matrix becomes something you can read before you compile. diff --git a/README.md b/README.md index fae5391..9ec58f9 100644 --- a/README.md +++ b/README.md @@ -168,6 +168,7 @@ deny_write = true # refuse skills declaring write access max_budget = 6000 # cap per-skill context budget # deny_mcp = true # refuse skills declaring any MCP server # allow_mcp_servers = ["https://mcp.your-org.com/*"] # globs; matched against a server's command or url +# max_mcp_tools = 100 # cap declared MCP tools across all skills # deny_remote_exec = false # opt out of the curl|sh body lint (default: on) ``` @@ -190,6 +191,12 @@ Three targets have a project-scoped MCP config file, and those are the three Kit The other eight targets get a warning naming the specific reason — `no-mcp-surface` (aider, AGENTS.md have no configuration mechanism), `no-project-scope` (Cline and Windsurf are user-global only), `needs-shared-file-merge` (Zed and Gemini keep servers in a settings file full of unrelated user config) — and **no file**. A config the client never reads would look configured and do nothing, which is the failure mode this project exists to prevent. +### What it costs you + +`compile` and `doctor` report the tool budget: `MCP tool budget: 8 tool(s) across 2 server(s); cap 100`. Every tool in a declared allowlist is a tool definition the agent carries for the whole session, so the allowlist is an exact floor on the cost — and `max_mcp_tools` in `[policy]` caps it, warning on breach and failing `--strict`. The default ceiling is 100, the one limit any client documents (Windsurf, past which tools are dropped). + +The *real* token cost is reported as unmeasured, and stays that way on purpose. It is the JSON schema of every tool a server exposes, which is only knowable by asking the server — and asking a stdio server means executing it, which is precisely what the install gate exists to prevent. A floor Kitbash can prove beats an estimate it cannot. + A declared MCP server is third-party code that will run with your agent's permissions, so its lints are non-bypassable, like the four safety lints: shell strings, unpinned versions, literal credentials, plain-`http` URLs off loopback, and a missing `tools` allowlist all block install, `--yes` included. Secrets may only appear as `${VAR}`; where a format defines no credential reference and expands no variables, the server is omitted with a warning rather than emitted broken. ## In CI diff --git a/packages/cli/package.json b/packages/cli/package.json index b2dddfa..0ebe390 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -1,6 +1,6 @@ { "name": "kitbash", - "version": "0.19.1", + "version": "0.20.0", "description": "The package manager and compiler for AI agent skills — write once, run in every coding agent", "license": "Apache-2.0", "author": "Harsh Singh", diff --git a/packages/cli/scripts/test.mjs b/packages/cli/scripts/test.mjs index 46e0e2d..891f277 100644 --- a/packages/cli/scripts/test.mjs +++ b/packages/cli/scripts/test.mjs @@ -1632,6 +1632,53 @@ try { rmSync(polTmp, { recursive: true, force: true }); } +// ── MCP tool budget ────────────────────────────────────────────────────────── +// The standing-cost argument applied to MCP. Counted from the declared allowlist +// (an exact floor), never estimated — measuring the real cost means executing the +// server, which is what the install gate exists to prevent. +const budTmp = mkdtempSync(join(tmpdir(), "kitbash-budget-")); +try { + const bs = join(budTmp, "src"); + mkdirSync(bs, { recursive: true }); + writeFileSync( + join(bs, "skill.toml"), + '[skill]\nname = "toolheavy"\nversion = "1.0.0"\ndescription = "Declares several MCP tools across two servers"\n[context]\nbudget = 1500\n\n[mcp.servers.a]\ntransport = "stdio"\ncommand = "npx"\nargs = ["-y", "@acme/a@1.0.0"]\ntools = ["t1","t2","t3","t4","t5"]\n\n[mcp.servers.b]\ntransport = "streamable-http"\nurl = "https://b.example.com/mcp"\ntools = ["u1","u2","u3"]\n', + ); + writeFileSync(join(bs, "SKILL.md"), "# Tool heavy\n\nBody.\n"); + writeFileSync(join(budTmp, "kitbash.toml"), '[project]\ntargets = ["claude-code"]\n'); + run(["install", `file:${bs}`, "--yes"], budTmp); + + const def = run(["compile"], budTmp); + check("budget: the tool count is reported on every compile", def.out.includes("MCP tool budget: 8 tool(s) across 2 server(s)"), def.out); + check("budget: token cost is reported as unmeasured, not guessed", def.out.includes("unmeasured"), def.out); + check("budget: under the cap is a note, not a warning", def.status === 0 && !def.out.includes("budget exceeded"), def.out); + + writeFileSync(join(budTmp, "kitbash.toml"), '[project]\ntargets = ["claude-code"]\n[policy]\nmax_mcp_tools = 5\n'); + const over = run(["compile"], budTmp); + check("budget: a breach of max_mcp_tools warns", over.out.includes("MCP tool budget exceeded"), over.out); + check("budget: a breach still compiles by default", over.status === 0, over.out); + const strictBudget = run(["compile", "--strict"], budTmp); + check("budget: a breach fails --strict", strictBudget.status === 1, strictBudget.out); + + // A wildcard server cannot be bounded — say so rather than fold it into a total. + const ws = join(budTmp, "wild"); + mkdirSync(ws, { recursive: true }); + writeFileSync( + join(ws, "skill.toml"), + '[skill]\nname = "wildy"\nversion = "1.0.0"\ndescription = "Declares a wildcard MCP tool allowlist"\n[context]\nbudget = 1500\n\n[mcp.servers.wild]\ntransport = "streamable-http"\nurl = "https://w.example.com/mcp"\ntools = ["*"]\n', + ); + writeFileSync(join(ws, "SKILL.md"), "# Wild\n\nBody.\n"); + writeFileSync(join(budTmp, "kitbash.toml"), '[project]\ntargets = ["claude-code"]\n'); + run(["install", `file:${ws}`, "--yes"], budTmp); + const wild = run(["compile"], budTmp); + check("budget: a wildcard server makes the total a floor, marked +", wild.out.includes('8+ (1 server(s) declare "*")'), wild.out); + check("budget: and says it cannot be bounded", wild.out.includes("cannot be bounded"), wild.out); + const wdoc = run(["doctor"], budTmp); + check("budget: doctor reports the same budget line", wdoc.out.includes("MCP tool budget:"), wdoc.out); +} finally { + rmSync(budTmp, { recursive: true, force: true }); +} + if (failures) { console.error(`\n${failures} test(s) failed`); process.exit(1); diff --git a/packages/cli/src/commands.ts b/packages/cli/src/commands.ts index f7af5cb..256e9f9 100644 --- a/packages/cli/src/commands.ts +++ b/packages/cli/src/commands.ts @@ -11,7 +11,7 @@ import { dropLock, integrityOf, readLock, upsertLock, walk, LOCK_FILE } from "./ import { fileChanges, manifestDelta, textOf, unifiedDiff } from "./diff.js"; import { collectImports, driftGroups, type ImportedSource } from "./importers.js"; import { estimateTokens, loadInstalledSkills, loadInstalledSkillsSafe, loadSkill, resolveBody, schemaLints, standingStub, COMMAND_RE, NAME_RE, SKILLS_DIR, type LoadedSkill } from "./ksf.js"; -import { collectServers, emitMcp, mcpLints, mcpSupportMatrix, mcpWarnings } from "./mcp.js"; +import { collectServers, DEFAULT_MAX_TOOLS, emitMcp, mcpLints, mcpSupportMatrix, mcpWarnings, toolBudget, toolBudgetReport } from "./mcp.js"; import { toSarif, type SarifFinding } from "./sarif.js"; import { parseToml } from "./toml.js"; @@ -42,6 +42,7 @@ const INIT_CONFIG = `# kitbash project configuration — https://github.com/sing # max_budget = 6000 # refuse skills with a larger context budget # deny_remote_exec = false # opt OUT of the download-and-execute body lint (default: on) # deny_mcp = true # refuse skills declaring any MCP server +# max_mcp_tools = 100 # cap declared MCP tools across all skills (default: 100) # allow_mcp_servers = ["https://mcp.your-org.com/*"] # globs; matched against a server's command or url `; @@ -641,6 +642,7 @@ export async function cmdDoctor(): Promise { const declared = collectServers(skills).servers; if (declared.length) { console.log(`\nMCP servers declared: ${declared.map((s) => s.name).join(", ")}`); + console.log(` ${toolBudgetReport(toolBudget(declared), loadPolicy(root)?.maxMcpTools ?? DEFAULT_MAX_TOOLS).line}`); for (const line of mcpSupportMatrix()) console.log(` ${line}`); } @@ -721,6 +723,8 @@ interface Policy { denyRemoteExec: boolean; /** Refuse any skill that declares an MCP server at all. */ denyMcp: boolean; + /** Cap on declared MCP tools across all skills. Absent uses the documented 100-tool ceiling. */ + maxMcpTools?: number | undefined; /** * Globs an MCP server's command (stdio) or url (remote) must match. Empty means * unrestricted — the same shape as allow_sources, so an org can pin servers to @@ -747,6 +751,7 @@ function loadPolicy(root: string): Policy | null { // Absent means true — you must opt OUT of the remote-exec block explicitly. denyRemoteExec: tbl["deny_remote_exec"] !== false, denyMcp: tbl["deny_mcp"] === true, + maxMcpTools: typeof tbl["max_mcp_tools"] === "number" ? (tbl["max_mcp_tools"] as number) : undefined, allowMcpServers: Array.isArray(tbl["allow_mcp_servers"]) ? (tbl["allow_mcp_servers"] as unknown[]).filter((x): x is string => typeof x === "string") : [], @@ -957,6 +962,13 @@ export async function cmdCompile(args: string[]): Promise { return 1; } if (mcpServers.length) { + // The standing-cost argument, applied to MCP. Counted from the declared + // allowlist — an exact floor, never an estimate — and reported as a note so + // the number shows on every compile, with a breach as a real warning. + const report = toolBudgetReport(toolBudget(mcpServers), loadPolicy(root)?.maxMcpTools ?? DEFAULT_MAX_TOOLS); + notes.push(report.line); + warnings.push(...report.warnings); + const unsupported: string[] = []; for (const adapter of adapters) { const out = emitMcp(adapter.id, mcpServers, AGENT_PLUGIN_DIR); diff --git a/packages/cli/src/mcp.ts b/packages/cli/src/mcp.ts index 8853f56..d5703f7 100644 --- a/packages/cli/src/mcp.ts +++ b/packages/cli/src/mcp.ts @@ -394,6 +394,69 @@ function emitCopilot(servers: McpServer[]): McpEmit { return { files: [{ path: ".github/mcp.json", content: json({ mcpServers: out }) }], warnings }; } +// ── tool budget ────────────────────────────────────────────────────────────── + +/** + * The tool budget: how many MCP tools a repo's skills put in front of the model. + * + * This is the standing-cost argument applied to MCP. A skill body's cost is + * statically countable, which is why Kitbash can enforce `context.budget`. An + * MCP server's real cost — the JSON schema of every tool it exposes — is not + * knowable without asking the server, and asking a stdio server means executing + * it. That is precisely what the install gate exists to prevent, so Kitbash does + * not do it, and does not estimate a number it cannot derive. + * + * What it does count is the thing the manifest already states: the declared tool + * allowlist. Every one of those tools is a tool definition the agent carries, so + * the count is a real floor on the cost, and it is exact whenever the allowlist + * is exact. A server declaring `["*"]` has no countable bound — that is reported + * as unbounded rather than folded into a total that would then be wrong. + */ +export interface ToolBudget { + /** Tools named explicitly across all servers — an exact floor on what is loaded. */ + declared: number; + /** Servers declaring `["*"]`: real cost unknown without asking the server. */ + unbounded: string[]; + servers: number; +} + +/** + * Windsurf documents a hard cap of 100 tools total; past it, tools are dropped. + * It is the only primary-source numeric limit found across the clients, so it is + * the default ceiling. (A widely-repeated Cursor "40 tool" limit appears only in + * forum posts and blogs, never in Cursor's own docs, so it is deliberately not + * encoded here — a tool whose pitch is measurement must not ship a folklore number.) + */ +export const DEFAULT_MAX_TOOLS = 100; + +export function toolBudget(servers: McpServer[]): ToolBudget { + let declared = 0; + const unbounded: string[] = []; + for (const s of servers) { + if (s.tools.includes("*")) unbounded.push(s.name); + else declared += s.tools.length; + } + return { declared, unbounded, servers: servers.length }; +} + +/** + * Report the budget, and fail when a countable total breaches the cap. Servers + * declaring `["*"]` can never be proven under a cap, so they are surfaced as the + * reason the total is a floor rather than silently passing. + */ +export function toolBudgetReport(b: ToolBudget, max: number): { line: string; warnings: string[]; over: boolean } { + const bound = b.unbounded.length ? `${b.declared}+ (${b.unbounded.length} server(s) declare "*")` : `${b.declared}`; + const line = `MCP tool budget: ${bound} tool(s) across ${b.servers} server(s); cap ${max}. Real token cost is unmeasured — it needs the server's tool schemas, and reading those means running it.`; + const warnings: string[] = []; + const over = b.declared > max; + if (over) { + warnings.push(`MCP tool budget exceeded: ${b.declared} declared tools against a cap of ${max}. Windsurf documents 100 total tools as a hard cap, past which tools are dropped — and every tool definition is carried in context for the whole session.`); + } else if (b.unbounded.length) { + warnings.push(`MCP tool budget cannot be bounded: ${b.unbounded.join(", ")} declare tools = ["*"], so the total is at least ${b.declared} and may exceed the cap of ${max}. Name the tools to make it countable.`); + } + return { line, warnings, over }; +} + /** Targets that can carry an MCP declaration today. */ const MCP_TARGETS = ["agent-plugins", "claude-code", "copilot"]; diff --git a/site/changelog.html b/site/changelog.html index 8264471..dab8f39 100644 --- a/site/changelog.html +++ b/site/changelog.html @@ -91,7 +91,7 @@

Changelog

Releases follow Keep a Changelog and semver — for skills and for this CLI, breaking prompt changes are breaking changes. The CLI is published to npm as kitbash and to Homebrew via singhharsh1708/tap. Tagged builds are on the GitHub releases page.

-
v0.19.1Current CLI version
+
v0.20.0Current CLI version
8Compile targets
Apache-2.0License
@@ -105,10 +105,22 @@

Changelog

Confirm with kitbash --version, which reads the installed package.json. Install and uninstall routes are covered on the installation page.

+
+
+

v0.20.0

+ 2026-08-09latest +
+

The token-cost argument, applied to MCP — the largest standing-context line item Kitbash was not measuring.

+

A skill body's cost is statically countable, which is why context.budget can be enforced at compile time. An MCP server's real cost is not: the tokens it adds are the JSON schema of every tool it exposes, and knowing those means asking the server — which for a stdio server means executing it. That is exactly what the install gate exists to prevent, so Kitbash does not do it, and does not print an estimate it cannot derive.

+

What it can count exactly is the thing the manifest already states. Every tool in a declared allowlist is a tool definition the agent carries for the whole session, so the allowlist is an exact floor on the cost.

+

Added

+
  • MCP tool budget, reported on every compile and by doctor — MCP tool budget: 8 tool(s) across 2 server(s); cap 100, alongside an explicit statement that the token cost is unmeasured and why. A server declaring tools = ["*"] has no countable bound, so the total renders as 8+ (1 server(s) declare "*") and warns that it cannot be bounded, rather than folding into a total that would be wrong.
  • max_mcp_tools in [policy] — caps declared MCP tools across all skills. A breach warns and fails --strict. The default ceiling is 100, the one primary-source numeric limit found across the clients (Windsurf documents it as a hard cap past which tools are dropped). A widely-repeated "40 tool" Cursor limit is deliberately not encoded: it appears only in forum posts and blogs, never in Cursor's own docs, and a tool whose pitch is measurement must not ship a folklore number.
+
+

v0.19.1

- 2026-08-09latest + 2026-08-09

Follow-through on 0.19.0: policy gets a say over MCP servers, and the support matrix becomes something you can read before you compile.

Added

diff --git a/site/index.html b/site/index.html index 210bd4a..3506688 100644 --- a/site/index.html +++ b/site/index.html @@ -151,7 +151,7 @@ -

Open format for AI agent skills · v0.19.1 · stable spec (RFC 0002)

+

Open format for AI agent skills · v0.20.0 · stable spec (RFC 0002)

Write an agent skill once. Run it everywhere.

Get started