From 44079d81333a754ba4bfd6f49082b4dd4129f204 Mon Sep 17 00:00:00 2001 From: Gabriel Bauman <967743+gabrielbauman@users.noreply.github.com> Date: Tue, 2 Jun 2026 23:43:33 -0700 Subject: [PATCH 1/2] Add regenerate command and auto-regenerate stale codegen on startup Re-running an API's code generation after an anyapi-mcp upgrade previously required re-adding it, which destroys stored credentials: a bearer API re-added without --token resolves to auth:none, and its keystore token is then deleted. Add a codegen-only path that preserves auth instead. - regenerate [id ...] [--stale-only]: re-fetch each API's saved source and rebuild its typed client + operation index in place. Auth, baseUrl, and hosts are preserved; no secret is written or deleted (a bearer token is only read, to re-introspect a source that needs it). Each API is independent, so one unreachable source never blocks the rest. - Stamp generated artifacts with CODEGEN_VERSION. serve regenerates any entry with an older or missing stamp on startup, before connecting, with all logging on stderr. A re-fetch failure is logged and keeps the old code. --- CLAUDE.md | 22 ++++--- README.md | 21 ++++++- src/commands/regenerate.ts | 84 +++++++++++++++++++++++++++ src/commands/serve.ts | 39 ++++++++++++- src/main.ts | 5 ++ src/register.ts | 115 ++++++++++++++++++++++++++++++++++++- src/registry.ts | 7 +++ 7 files changed, 283 insertions(+), 10 deletions(-) create mode 100644 src/commands/regenerate.ts diff --git a/CLAUDE.md b/CLAUDE.md index 42b68f7..9566b7f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -7,7 +7,7 @@ drives by writing code. See [README.md](README.md) for the user-facing pitch. ## Commands ```sh -deno task dev # run from source (add | list | login | logout | remove | serve | install) +deno task dev # run from source (add | list | regenerate | login | logout | remove | serve | install) deno task compile # build the self-contained ./anyapi-mcp binary deno task check # type-check (deno check src/main.ts) deno task lint # deno lint src/ @@ -26,11 +26,13 @@ The core (registry, search, execute sandbox) is **protocol-agnostic**; each protocol plugs in through one in-tree adapter object. - `src/main.ts`: argv dispatch to the subcommands. -- `src/commands/{add,install,list,login,logout,remove,serve}.ts`: one file per - subcommand. `serve.ts` is the MCP server exposing `search`, `execute`, - `authenticate`, `configure_oauth`, `add_api`, `list_apis`, `remove_api`; - `install.ts` registers anyapi-mcp with Claude Code / Desktop; `login`/`logout` - manage OAuth sessions. +- `src/commands/{add,install,list,regenerate,login,logout,remove,serve}.ts`: one + file per subcommand. `serve.ts` is the MCP server exposing `search`, + `execute`, `authenticate`, `configure_oauth`, `add_api`, `list_apis`, + `remove_api`; `install.ts` registers anyapi-mcp with Claude Code / Desktop; + `login`/`logout` manage OAuth sessions; `regenerate.ts` rebuilds generated + code from saved sources (credentials untouched) — and `serve` runs the same + pass on stale entries at startup (see `register.ts`). - `src/adapter.ts`: the `ProtocolAdapter` seam. `prepare()` turns a source into base URL + hosts + operation index + generated client types (+ optional discovered OAuth config); `buildHarness()` builds the `execute` preamble that @@ -68,7 +70,13 @@ protocol plugs in through one in-tree adapter object. upserts in place (fresh `addedAt` invalidates serve's ops cache; same-id keystore accounts are preserved, so an OAuth API stays logged in across a re-register). `unregisterApi` (shared by `remove` and `remove_api`) deletes - the entry, its secrets, and its cached artifacts. + the entry, its secrets, and its cached artifacts. `regenerateApi`/ + `regenerateApis` re-run only codegen (re-fetch source → rewrite types + ops, + bump `addedAt`) while preserving the entry's auth/baseUrl/hosts — no secret is + written or deleted (a bearer token is only read, to re-introspect). Each + generated artifact is stamped with `CODEGEN_VERSION`; bump that constant when + a generator change makes old artifacts stale, and `serve` regenerates anything + older on startup (an entry missing the stamp counts as stale). - `src/operation.ts`: operation index + keyword `search` (no embeddings). Params carry optional `description`/`enum`, surfaced in search (`clampDescription`/ `applyEnum` bound their size; `enum` holds only real values — an over-cap diff --git a/README.md b/README.md index 569bc2e..e4bed51 100644 --- a/README.md +++ b/README.md @@ -129,6 +129,21 @@ Lists registered APIs with id, name, base URL, operation count, and auth kind. For OAuth APIs it also shows live login status and token expiry (e.g. `oauth2 (logged in, expires in 5h)` or `oauth2 (not logged in)`). +### `regenerate [id ...]` + +Rebuilds the generated client code (typed `.d.ts`/client module + operation +index) for registered APIs by re-fetching each one's saved source. With no id it +does all of them; pass ids to target specific APIs, or `--stale-only` to rebuild +just those whose code predates the current build. + +This only refreshes generated code: **saved credentials and OAuth config are +left untouched** (bearer tokens, OAuth client credentials, and login state all +survive; baseUrl and the host allowlist are preserved too). Run it after +upgrading `anyapi-mcp` so cached code matches the new generators - though +`serve` also does this automatically (see below), so you rarely need to run it +by hand. Each API is independent: if one source can't be re-fetched, the rest +still regenerate and that API keeps its existing (working) code. + ### `login [options]` Authenticates an OAuth 2.0 API in the browser (see [OAuth](#oauth-apis)). Stores @@ -162,7 +177,11 @@ token, or OAuth client credentials + tokens), and cleans up cached files. ### `serve` Runs the stdio MCP server. It re-reads the registry on each call (so newly -registered APIs are picked up without a restart) and exposes: +registered APIs are picked up without a restart). On startup, if the build's +codegen version has changed since an API's code was generated (i.e. you upgraded +`anyapi-mcp`), it regenerates that API's code first - the same work as +`regenerate`, limited to stale entries, with credentials preserved and any +re-fetch failure logged but non-fatal. It exposes: - **`search`** - `{ query, api? }` → compact operation matches (`api`, `method`, `path`, `operationId`, `summary`, `params`, `requestBodyHint`). Each param diff --git a/src/commands/regenerate.ts b/src/commands/regenerate.ts new file mode 100644 index 0000000..d2446c4 --- /dev/null +++ b/src/commands/regenerate.ts @@ -0,0 +1,84 @@ +// `anyapi-mcp regenerate` - rebuild generated client code for registered APIs +// from their saved sources, without touching stored credentials or OAuth config. +// Run it after upgrading anyapi-mcp so cached types/ops match the new generators +// (serve also does this automatically for stale entries on startup). + +import { parseArgs } from "@std/cli/parse-args"; +import { readRegistry } from "../registry.ts"; +import { regenerateApis } from "../register.ts"; + +const HELP = + `anyapi-mcp regenerate - rebuild generated client code for registered APIs + +Usage: + anyapi-mcp regenerate [id ...] [options] + +Re-fetches each API's source (OpenAPI spec, GraphQL schema, or WSDL) and rebuilds +its typed client + operation index in place. Saved credentials and OAuth config +are left untouched - this only refreshes generated code, so run it after a new +anyapi-mcp version changes how that code is generated. With no id, regenerates +every registered API. + +Options: + --stale-only Only regenerate APIs whose code predates this anyapi-mcp build + -h, --help Show this help`; + +export async function runRegenerate(args: string[]): Promise { + const flags = parseArgs(args, { + boolean: ["help", "stale-only"], + alias: { h: "help" }, + }); + if (flags.help) { + console.log(HELP); + return; + } + + const ids = flags._.map(String).filter(Boolean); + const entries = await readRegistry(); + if (entries.length === 0) { + console.log( + "No APIs registered yet. Add one with: anyapi-mcp add ", + ); + return; + } + + // Reject unknown ids up front so a typo is a hard error, not a silent no-op. + if (ids.length > 0) { + const known = new Set(entries.map((e) => e.id)); + const unknown = ids.filter((id) => !known.has(id)); + if (unknown.length > 0) { + console.error( + `anyapi-mcp regenerate: unknown id(s): ${unknown.join(", ")}`, + ); + console.error(`Registered ids: ${[...known].join(", ")}`); + Deno.exit(1); + } + } + + const results = await regenerateApis({ + ids: ids.length > 0 ? ids : undefined, + staleOnly: flags["stale-only"], + onProgress: (m) => console.error(m), // progress to stderr; report to stdout + }); + + let ok = 0; + let failed = 0; + let skipped = 0; + for (const r of results) { + if (r.skipped) { + skipped++; + } else if (r.ok) { + ok++; + console.log(` ok ${r.id} (${r.operationCount} operations)`); + } else { + failed++; + console.log(` FAIL ${r.id} - ${r.error}`); + } + } + + const summary = [`${ok} regenerated`]; + if (skipped > 0) summary.push(`${skipped} already current`); + if (failed > 0) summary.push(`${failed} failed`); + console.log(summary.join(", ") + "."); + if (failed > 0) Deno.exit(1); +} diff --git a/src/commands/serve.ts b/src/commands/serve.ts index 0d92c1f..ee0029f 100644 --- a/src/commands/serve.ts +++ b/src/commands/serve.ts @@ -20,7 +20,12 @@ import { import { isUrl } from "../openapi.ts"; import { type OperationInfo, readOpsIndex } from "../operation.ts"; import { getSecret } from "../keystore.ts"; -import { registerApi, unregisterApi } from "../register.ts"; +import { + CODEGEN_VERSION, + regenerateApis, + registerApi, + unregisterApi, +} from "../register.ts"; import { getAdapter } from "../adapters.ts"; import { runSandboxed } from "../execute/run.ts"; import { @@ -634,6 +639,38 @@ export async function runServe(_args: string[]): Promise { : " (use the add_api tool or `anyapi-mcp add` to register one)"), ); + // After an anyapi-mcp upgrade the cached client code may predate the current + // generators. Rebuild any entry whose codegenVersion is older before serving + // (one source re-fetch per stale API). Resilient: a failure logs and keeps the + // old, still-working code. All output stays on stderr - stdout is the MCP + // frame stream. The bumped addedAt makes the tools reload the fresh ops index. + const stale = startup.entries.filter( + (e) => e.codegenVersion !== CODEGEN_VERSION, + ); + if (stale.length > 0) { + console.error( + `anyapi-mcp serve: codegen version changed; regenerating ${stale.length} ` + + `API(s) (${stale.map((e) => e.id).join(", ")}) ...`, + ); + const results = await regenerateApis({ + ids: stale.map((e) => e.id), + onProgress: (m) => console.error(m), + }); + for (const r of results) { + if (!r.ok) { + console.error( + `anyapi-mcp serve: regeneration failed for ${r.id}: ${r.error} ` + + `(keeping existing code).`, + ); + } + } + console.error( + `anyapi-mcp serve: regenerated ${ + results.filter((r) => r.ok).length + }/${stale.length} API(s).`, + ); + } + const server = new McpServer( { name: "anyapi-mcp", version: "0.1.0" }, { instructions: buildInstructions(startup.entries) }, diff --git a/src/main.ts b/src/main.ts index 99139fd..a8420ea 100644 --- a/src/main.ts +++ b/src/main.ts @@ -9,6 +9,7 @@ import { runInstall } from "./commands/install.ts"; import { runList } from "./commands/list.ts"; import { runLogin } from "./commands/login.ts"; import { runLogout } from "./commands/logout.ts"; +import { runRegenerate } from "./commands/regenerate.ts"; import { runRemove } from "./commands/remove.ts"; import { runServe } from "./commands/serve.ts"; @@ -17,6 +18,7 @@ const USAGE = `anyapi-mcp - code-mode MCP server for any API Usage: anyapi-mcp add [options] Register an API from an OpenAPI spec anyapi-mcp list List registered APIs + anyapi-mcp regenerate [id ...] Rebuild generated code (keeps credentials) anyapi-mcp login [options] Authenticate an OAuth API (opens a browser) anyapi-mcp logout Remove an OAuth API's stored tokens anyapi-mcp remove Remove a registered API and its token @@ -34,6 +36,9 @@ async function main(): Promise { case "list": await runList(rest); break; + case "regenerate": + await runRegenerate(rest); + break; case "login": await runLogin(rest); break; diff --git a/src/register.ts b/src/register.ts index cebd40e..47dca88 100644 --- a/src/register.ts +++ b/src/register.ts @@ -10,12 +10,13 @@ import { type Auth, findEntry, type OAuth2Auth, + readRegistry, type RegistryEntry, removeEntry, updateEntry, } from "./registry.ts"; import { writeOpsIndex } from "./operation.ts"; -import { deleteSecret, setSecret } from "./keystore.ts"; +import { deleteSecret, getSecret, setSecret } from "./keystore.ts"; import { opsPathFor, typesPathFor } from "./paths.ts"; import { clearOAuthSecrets, @@ -24,6 +25,20 @@ import { quirkForAuthUrl, } from "./oauth.ts"; +/** + * Version of the generated-code contract. Bump this whenever a change would make + * previously-generated artifacts stale relative to the current build: the + * openapi-typescript output or its post-processing (openapi-sanitize.ts), the + * GraphQL/SOAP type/client emitters, the SOAP runtime deps baked into the client + * module, or the operation-index shape (operation.ts). Each registry entry + * records the version its artifacts were built under; `serve` regenerates any + * stale entry on startup and `anyapi-mcp regenerate` rebuilds on demand. + * + * It is a deliberate counter, not the git hash: tying regeneration to commits + * would re-fetch every spec on releases that never touched codegen. + */ +export const CODEGEN_VERSION = 1; + /** * Build a reverse-DNS id from a base URL: the host reversed, then the base-path * segments. "https://petstore3.swagger.io/api/v3" -> "io.swagger.petstore3.api.v3"; @@ -248,6 +263,7 @@ export async function registerApi( hosts, auth, typesPath, + codegenVersion: CODEGEN_VERSION, addedAt: new Date().toISOString(), }; if (opts.docsUrl) entry.docsUrl = opts.docsUrl; @@ -277,3 +293,100 @@ export async function registerApi( overwritten: existing !== undefined, }; } + +export interface RegenerateResult { + id: string; + /** True when the code was rebuilt, or (with `skipped`) was already current. */ + ok: boolean; + /** Operation count on the refreshed source (present on a successful rebuild). */ + operationCount?: number; + /** Set when `staleOnly` skipped an entry already at CODEGEN_VERSION. */ + skipped?: boolean; + /** Failure reason (present when `ok` is false). */ + error?: string; +} + +/** + * Rebuild ONLY the generated code for an already-registered API: re-fetch its + * source, regenerate the typed client + operation index in place, and bump the + * entry's `codegenVersion` and `addedAt` (the latter busts serve's ops cache). + * + * The entry's auth, baseUrl, and hosts are preserved verbatim - this is a codegen + * refresh, not a re-registration. No secret is written or deleted; a bearer token + * is only READ, to re-inspect a source that needs auth (e.g. GraphQL + * introspection). On any failure the existing artifacts are left in place and the + * error is returned, never thrown - a stale-but-working client beats a broken one. + */ +export async function regenerateApi( + entry: RegistryEntry, + onProgress?: (message: string) => void, +): Promise { + const log = onProgress ?? (() => {}); + try { + const adapter = getAdapter(entry.kind); + const token = entry.auth.kind === "bearer" + ? await getSecret(entry.auth.tokenKey) + : undefined; + const prepared = await adapter.prepare(entry.specSource, { + // Pin the registered base URL so a codegen refresh never moves the base or + // re-derives the host allowlist - that is a re-registration concern. + baseUrlOverride: entry.baseUrl, + token, + onProgress: log, + }); + await prepared.writeTypes(entry.typesPath); + await writeOpsIndex(entry.id, prepared.operations); + const updated: RegistryEntry = { + ...entry, + codegenVersion: CODEGEN_VERSION, + addedAt: new Date().toISOString(), + }; + if (!(await updateEntry(updated))) { + return { id: entry.id, ok: false, error: "no longer registered" }; + } + return { + id: entry.id, + ok: true, + operationCount: prepared.operations.length, + }; + } catch (err) { + return { + id: entry.id, + ok: false, + error: err instanceof Error ? err.message : String(err), + }; + } +} + +/** + * Regenerate code for many registered APIs. With `ids`, only those entries are + * considered; otherwise every registered API. With `staleOnly`, entries already + * at CODEGEN_VERSION are skipped (serve's post-upgrade startup pass uses this). + * Each API is independent - one failure never aborts the rest. + */ +export async function regenerateApis( + opts: { + ids?: string[]; + staleOnly?: boolean; + onProgress?: (message: string) => void; + } = {}, +): Promise { + const log = opts.onProgress ?? (() => {}); + let entries = await readRegistry(); + if (opts.ids) { + const want = new Set(opts.ids); + entries = entries.filter((e) => want.has(e.id)); + } + const results: RegenerateResult[] = []; + for (const entry of entries) { + if (opts.staleOnly && entry.codegenVersion === CODEGEN_VERSION) { + results.push({ id: entry.id, ok: true, skipped: true }); + continue; + } + log( + `Regenerating ${entry.id} (${entry.kind}) from ${entry.specSource} ...`, + ); + results.push(await regenerateApi(entry, log)); + } + return results; +} diff --git a/src/registry.ts b/src/registry.ts index e8f0615..7f621f5 100644 --- a/src/registry.ts +++ b/src/registry.ts @@ -58,6 +58,13 @@ export interface RegistryEntry { auth: Auth; /** Absolute path to the generated .d.ts in the cache dir. */ typesPath: string; + /** + * The `CODEGEN_VERSION` the cached artifacts (types + ops index) were built + * under. Absent on entries written before this field existed; a missing or + * older value is treated as stale, so `serve` regenerates it on startup (and + * `anyapi-mcp regenerate` rebuilds on demand). See register.ts. + */ + codegenVersion?: number; addedAt: string; } From 620ac9a383b23f42f5b59dbc8a32f94eed9a34d8 Mon Sep 17 00:00:00 2001 From: Gabriel Bauman <967743+gabrielbauman@users.noreply.github.com> Date: Wed, 3 Jun 2026 00:06:17 -0700 Subject: [PATCH 2/2] Correct typesPath and CODEGEN_VERSION doc wording Address review: typesPath is a .d.ts only for OpenAPI/GraphQL (SOAP emits a .ts runtime module), and the version stamp lives on the registry entry's codegenVersion field, not embedded in the generated artifacts themselves. --- CLAUDE.md | 9 +++++---- src/registry.ts | 5 ++++- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 9566b7f..4e8a667 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -73,10 +73,11 @@ protocol plugs in through one in-tree adapter object. the entry, its secrets, and its cached artifacts. `regenerateApi`/ `regenerateApis` re-run only codegen (re-fetch source → rewrite types + ops, bump `addedAt`) while preserving the entry's auth/baseUrl/hosts — no secret is - written or deleted (a bearer token is only read, to re-introspect). Each - generated artifact is stamped with `CODEGEN_VERSION`; bump that constant when - a generator change makes old artifacts stale, and `serve` regenerates anything - older on startup (an entry missing the stamp counts as stale). + written or deleted (a bearer token is only read, to re-introspect). Each entry + records (in `codegenVersion`) the `CODEGEN_VERSION` its artifacts were built + under; bump that constant when a generator change makes old artifacts stale, + and `serve` regenerates anything older on startup (an entry missing the stamp + counts as stale). - `src/operation.ts`: operation index + keyword `search` (no embeddings). Params carry optional `description`/`enum`, surfaced in search (`clampDescription`/ `applyEnum` bound their size; `enum` holds only real values — an over-cap diff --git a/src/registry.ts b/src/registry.ts index 7f621f5..fdd3533 100644 --- a/src/registry.ts +++ b/src/registry.ts @@ -56,7 +56,10 @@ export interface RegistryEntry { /** Documentation URL: stored and surfaced, never parsed. */ docsUrl?: string; auth: Auth; - /** Absolute path to the generated .d.ts in the cache dir. */ + /** + * Absolute path to the generated types/client file in the cache dir: a `.d.ts` + * for type-only outputs (OpenAPI, GraphQL), a `.ts` runtime module for SOAP. + */ typesPath: string; /** * The `CODEGEN_VERSION` the cached artifacts (types + ops index) were built