From 523155d37c55c2a14c744d2e6841dbeab6b1e865 Mon Sep 17 00:00:00 2001 From: yellowgram Date: Thu, 24 Sep 2026 18:15:17 -0400 Subject: [PATCH] SurfacePin v1.4.0: hash tool annotations + outputSchema, HINT_FLIP taxonomy MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Complete the agent-visible tool descriptor (Lockfile Spec v1.4): - Hash annotations (drop title; keep unknown keys; omit if empty) - Hash outputSchema (absent ≠ {}; non-object → exit 2) - Keep 1.3 coercions for description/inputSchema; no MCP default materialization - Diff taxonomy: COMPATIBLE | BREAKING | HINT_FLIP (kinds array) - Existing goldens unchanged for tools without annotations/outputSchema - Action unchanged; algorithm id stays surfacepin-jcs-v1; lock write v3 - Re-lock after upgrade to >=1.4 when tools had annotations/outputSchema Forbidden this cut: resourceTemplates, initialize_instructions, signed locks, streamable_http, probes/proxy/quarantine/policy_engine, action changes. --- README.md | 22 +- SPEC.md | 96 +++- STATUS.md | 25 +- package-lock.json | 4 +- package.json | 4 +- src/cli.ts | 2 +- src/hash.ts | 8 +- src/index.ts | 2 + src/lock.ts | 5 +- src/normalize.ts | 37 +- src/schema-diff.ts | 165 ++++++- src/test/schema-diff.test.ts | 40 +- src/test/spec-1.4.test.ts | 592 +++++++++++++++++++++++ src/types.ts | 10 +- src/verify.ts | 28 +- testdata/annotations-output.tools.json | 40 ++ testdata/annotations-output.v3.lock.json | 41 ++ 17 files changed, 1043 insertions(+), 78 deletions(-) create mode 100644 src/test/spec-1.4.test.ts create mode 100644 testdata/annotations-output.tools.json create mode 100644 testdata/annotations-output.v3.lock.json diff --git a/README.md b/README.md index ac480bc..e7f83d0 100644 --- a/README.md +++ b/README.md @@ -2,15 +2,15 @@ Lock **exact hashes** of an MCP server’s list surfaces and fail CI when they drift silently: -- **tools** — `name`, `description`, `inputSchema` +- **tools** — `name`, `description`, `inputSchema`, `annotations`, `outputSchema` - **resources** — `uri`, `name`, `description`, `mimeType` - **prompts** — `name`, `description`, `arguments` This is deterministic byte-level hashing — **not** semantic / LLM / embedding drift detection. If a description character changes, the digest changes. -On mismatch, **lockfile v3** also explains *what* changed with a deterministic JSON Schema field-diff (breaking vs non-breaking). Pass/fail remains digest equality. +On mismatch, **lockfile v3** also explains *what* changed with a deterministic field-diff (`COMPATIBLE` | `BREAKING` | `HINT_FLIP`). Pass/fail remains digest equality. Re-lock after upgrading to ≥1.4. -Spec: [SPEC.md](./SPEC.md) (canonicalization `surfacepin-jcs-v1` + lockfile v1/v2/v3). +Spec: [SPEC.md](./SPEC.md) (Lockfile Spec v1.4; canonicalization `surfacepin-jcs-v1` + lockfile v1/v2/v3). ## 60-second start @@ -54,14 +54,16 @@ When a tool’s `inputSchema` drifts under a v3 lock: DRIFT: surface does not match lockfile CHANGED echo f35d75b2… -> a6fdd0e9… - CHANGED description (non-breaking) + CHANGED description (COMPATIBLE) "Echo text back" -> "Echo text back (updated)" - ADDED inputSchema.properties.lang (non-breaking) + ADDED inputSchema.properties.lang (COMPATIBLE) + {"type":"string"} CHANGED inputSchema.properties.text.type (BREAKING) "string" -> "number" ADDED inputSchema.required["lang"] (BREAKING) + "lang" + CHANGED annotations.readOnlyHint (HINT_FLIP) + true -> false ``` Machine output: `surfacepin diff … --json`. @@ -89,16 +91,16 @@ Composite action under [`action/`](./action/). File-based tools verify (CI usual Or call the CLI yourself after `npm install surfacepin`. -## What v1.3 does / does not +## What v1.4 does / does not | Does | Does not | |------|----------| -| Hash tools + resources + prompts | Semantic similarity gates | +| Hash tools (+ annotations, outputSchema) + resources + prompts | Semantic similarity gates | | Lockfile v3 (embedded surfaces) + verify v1/v2/v3 | Streamable HTTP / SSE (stdio only) | -| Deterministic schema field-diff on `diff` | LLM / fuzzy matching | +| Deterministic field-diff: COMPATIBLE / BREAKING / HINT_FLIP | LLM / fuzzy matching; safety verdicts from hints | | Offline verify from JSON files | Hosted service, telemetry, signed locks | -| Live list* via MCP stdio (`--stdio -- …`) | Resource templates list | -| Breaking vs non-breaking path classification | | +| Live list* via MCP stdio (`--stdio -- …`) | Resource templates / initialize.instructions | +| Ignore tool title / icons / _meta | Materialize MCP annotation defaults into hashes | ## License diff --git a/SPEC.md b/SPEC.md index 051ff52..a614f00 100644 --- a/SPEC.md +++ b/SPEC.md @@ -3,7 +3,7 @@ Canonical specification for SurfacePin surface locking. CLI and Action are reference clients of this document. -**Status:** v1.3 (2026-09-24) +**Status:** Lockfile Spec v1.4 (2026-09-24) **Scope:** MCP `tools/list`, `resources/list`, and `prompts/list` surfaces. **Wire formats:** lockfile **v1** (tools digests), **v2** (multi-surface digests), **v3** (multi-surface + embedded hashed payloads for structured field-diff). **Not in wire format:** signed lockfiles, semantic/embedding drift, LLM-assisted diff. @@ -24,10 +24,14 @@ CLI and Action are reference clients of this document. | Kind | MCP method | Identity key | Hashed fields | |------|------------|--------------|---------------| -| `tools` | `tools/list` | `name` | `name`, `description`, `inputSchema` | +| `tools` | `tools/list` | `name` | `name`, `description`, `inputSchema`, `annotations`, `outputSchema` | | `resources` | `resources/list` | `uri` | `uri`, `name`, `description`, `mimeType` | | `prompts` | `prompts/list` | `name` | `name`, `description`, `arguments` | +**Field rule:** Hash a tools field only if an MCP client or host surfaces it for invocation, structured I/O, or client hints. Skip decorative and volatile fields. Do not inject MCP defaults into new hashed fields. + +Aligned to MCP schema revision **2026-07-28** as a documentation reference only (not a runtime fetch). + Canonicalization algorithm id (unchanged): `"surfacepin-jcs-v1"`. ### 2.1 Input shapes @@ -45,23 +49,33 @@ SurfacePin accepts a combined document: or List*Result-shaped objects that include those keys (other top-level fields ignored). Selected kinds with a missing key are treated as an **empty array**. -### 2.2 Tools (unchanged from v1) +### 2.2 Tools | Field | Required | Notes | |-------|----------|-------| | `name` | yes | Non-empty string. Primary identity. | -| `description` | no | If absent, treated as `""`. | -| `inputSchema` | no | If absent, treated as `{}`. Must be a JSON object when present. | +| `description` | no | If absent, treated as `""` (1.3 coercion unchanged). | +| `inputSchema` | no | If absent, treated as `{}`. Must be a JSON object when present (1.3 coercion unchanged). | +| `annotations` | no | Source: `tool.annotations`. Drop only `title`. Keep all other keys including unknown. If absent or non-object → omit `annotations` key entirely. If empty after dropping `title` → omit key. **Do not** materialize MCP defaults into hashed fields. | +| `outputSchema` | no | Absent or `null` → omit key. Empty object `{}` → include as empty object. Present non-object → usage error (exit 2). **Do not** coerce absent → `{}` (absent ≠ `{}`). | -All other tool fields (`annotations`, `outputSchema`, etc.) are **ignored**. +Still ignored (not hashed, never appear in diff paths): `title`, `icons`, `_meta`. Duplicate `name` values are a **usage error** (exit 2). -**Tool descriptor** (hashed): +**Tool descriptor** (hashed; optional keys omitted when absent): ```json -{ "description": "", "inputSchema": { }, "name": "" } +{ + "annotations": { "readOnlyHint": true }, + "description": "", + "inputSchema": { }, + "name": "", + "outputSchema": { } +} ``` +Known annotation hints: `readOnlyHint`, `destructiveHint`, `idempotentHint`, `openWorldHint`. Unknown keys are still hashed. + ### 2.3 Resources MCP Resource shape fields that matter for drift. **Ignored** (volatile / non-identity): `size`, `annotations`, `title`, `icons`, `_meta`, and any other fields. @@ -347,7 +361,7 @@ Exit codes same as verify (0 match, 1 drift, 2 error). ## 6. Structured field-diff (explanatory) -Deterministic only — **no** LLM, embeddings, or fuzzy similarity. Classification does **not** affect exit codes. +Deterministic only — **no** LLM, embeddings, or fuzzy similarity. Classification does **not** affect exit codes (0 match / 1 drift / 2 usage). Pass/fail is **digest equality only**. ### 6.1 Scope @@ -355,24 +369,35 @@ When a lock entry embeds `surface` and the item digest changed, compare old `sur | Kind | Diff engine | |------|-------------| -| tools | descriptor `description` + recursive JSON Schema diff on `inputSchema` | +| tools | `description` + recursive JSON Schema diff on `inputSchema` / `outputSchema` + annotation hint diff | | resources | `uri` / `name` / `description` / `mimeType` field compares | | prompts | `description` + arguments matched **by name** | -### 6.2 Output lines +Never appear in diff paths: `annotations.title`, `tool.title`, `tool.icons`, `tool._meta`. -Stable sorted by `path`, then `kind`. Each change: +### 6.2 Taxonomy: COMPATIBLE | BREAKING | HINT_FLIP -``` -ADDED|REMOVED|CHANGED (BREAKING|non-breaking) - -> # or + value / - value -``` +JSON field changes carry a `kinds` **array** of taxonomy labels. Human lines print all kinds on the same path line. + +**Hint vs trust:** MCP `ToolAnnotations` are hints. Clients must not make tool-use decisions from untrusted servers. `HINT_FLIP` is explanatory client-hint drift, **not** a safety verdict. It is reserved for scarier or lost safety-leaning hints. Safer or newly asserted hints are `COMPATIBLE`. `outputSchema` is a structured I/O contract → classified `COMPATIBLE`|`BREAKING` only. **Hash is not safety.** + +#### HINT_FLIP when -Paths use dotted JSON-ish form, e.g. `inputSchema.properties.text.type`, `inputSchema.required["lang"]`, `arguments["who"].required`. +- `readOnlyHint`: `true → false` or `true → absent` +- `destructiveHint`: `false → true` or `false → absent` +- `idempotentHint`: `true → false` or `true → absent` +- `openWorldHint`: `false → true` or `false → absent` +- unknown annotation key: added, removed, or changed -### 6.3 Breaking vs non-breaking (deterministic rules) +Known-hint transitions **not** in that list → `COMPATIBLE` (still a digest change). Examples: `readOnlyHint` `false → true` / `absent → true`; `destructiveHint` `true → false`; `idempotentHint` `false → true`; `openWorldHint` `true → false`. -**Breaking** (examples): +Explicit value equal to an MCP default then omitted still changes the digest and stays `HINT_FLIP` this cut when the transition is in the list above (do not special-case default-equivalent omit as `COMPATIBLE`). + +#### outputSchema + +Classify with existing JSON Schema rules (`COMPATIBLE`|`BREAKING` only; **never** `HINT_FLIP`). Add or remove of `outputSchema` = `BREAKING`. Unknown schema keyword = `BREAKING`. + +#### Schema rules (inputSchema / outputSchema) — BREAKING examples - Remove a `properties` entry - Add a name to `required` @@ -383,7 +408,7 @@ Paths use dotted JSON-ish form, e.g. `inputSchema.properties.text.type`, `inputS - Resource `mimeType` / `name` / `uri` change - Prompt argument removed; new required argument; `required: false → true` -**Non-breaking** (examples): +#### COMPATIBLE examples (schema / descriptor) - Add an optional property (not listed in `required`) - Remove a name from `required` @@ -392,8 +417,22 @@ Paths use dotted JSON-ish form, e.g. `inputSchema.properties.text.type`, `inputS - `description` / `title` / `$comment` / `$id` / `examples` / `default` changes - Prompt argument description change; `required: true → false`; argument reorder only - Add optional prompt argument (`required: false`) +- Safer / newly asserted known annotation hints (not in HINT_FLIP list) + +Unknown keyword value changes default to **BREAKING** (fail-closed). Meta keys listed above are always `COMPATIBLE`. -Unknown keyword value changes default to **breaking** (fail-closed classification). Meta keys listed above are always non-breaking. +`HINT_FLIP` may also be `BREAKING` on the same path → label both. Example `kinds`: `["HINT_FLIP"]`, `["BREAKING"]`, `["COMPATIBLE"]`, `["HINT_FLIP", "BREAKING"]`. + +### 6.3 Output lines + +Stable sorted by `path`, then change-kind. Each change: + +``` +ADDED|REMOVED|CHANGED (HINT_FLIP|BREAKING|COMPATIBLE …) + -> # or + value / - value +``` + +Paths use dotted JSON-ish form, e.g. `inputSchema.properties.text.type`, `annotations.readOnlyHint`, `outputSchema.properties.id.type`, `arguments["who"].required`. --- @@ -420,13 +459,22 @@ treats that surface as **empty** and prints a clear stderr note. Streamable HTTP / SSE live fetch is not required for lockfile conformance. -## 9. Explicitly out of scope +## 9. Migration (1.3 → 1.4) + +- Old lockfiles (v1 / v2 / v3) still verify. +- Digests **change** when a tool had kept annotation keys or `outputSchema`. +- Digests stay **stable** when a tool had neither and only `title` / `icons` / `_meta` changed or were present. +- **Re-lock after upgrading to ≥1.4** so embedded surfaces and digests include annotations / outputSchema. + +## 10. Explicitly out of scope - Signed lockfiles / provenance - Semantic or embedding “similarity” gates -- Resource templates (`resources/templates/list`) — not pinned in 1.3 +- Resource templates (`resources/templates/list`) — not pinned +- `initialize.instructions` - LLM / fuzzy schema matching +- Materializing MCP annotation defaults into hashes -## 10. Conformance +## 11. Conformance Golden vectors under `testdata/` are normative for digests. Implementations MUST match those digests for the given inputs. Structured field-diff classification rules in §6 are normative for the reference CLI’s explanatory output. diff --git a/STATUS.md b/STATUS.md index 8004b58..089da75 100644 --- a/STATUS.md +++ b/STATUS.md @@ -61,16 +61,29 @@ Exit: 0 match, 1 drift, 2 usage/error. https://github.com/yellowgram/surfacepin -## Known gaps (intentional) +### v1.4.0 -- No Streamable HTTP / SSE live fetch yet (stdio is enough this cut) -- Lockfiles are not signed -- Action stays file-based tools-only (multi-surface / live stdio / schema diff are local/CLI) +- [x] Complete agent-visible tool descriptor: hash `annotations` (drop `title`) + `outputSchema` +- [x] Keep 1.3 coercions for `description` / `inputSchema`; do not materialize MCP annotation defaults +- [x] Diff taxonomy: `COMPATIBLE` | `BREAKING` | `HINT_FLIP` (`kinds` array); pass/fail still digest-only +- [x] SPEC / STATUS / README + golden tests; lockfile write remains v3; algorithm id unchanged +- [x] Action unchanged; no new runtime deps +- [ ] npm publish / git tag — founder-gated + +## Known gaps (intentional — document, do not implement this cut) + +- `initialize.instructions` not pinned - Resource templates (`resources/templates/list`) not pinned +- `tool.title` / `tool.icons` / `tool._meta` ignored (decorative) +- `resource.annotations` not hashed +- `prompt.title` / `prompt.icons` / `prompt._meta` ignored +- Action stays tools-file-only (multi-surface / live stdio are CLI/local) +- No Streamable HTTP / SSE live fetch yet (stdio is enough) +- Lockfiles are not signed ## Suggested next -1. Optional Streamable HTTP live fetch (if cheap) -2. Action multi-surface file inputs (if users ask) +1. Action multi-surface file inputs (if users ask) — separate cut +2. Optional Streamable HTTP live fetch (if cheap) 3. Conformance suite packaged for external implementations 4. Optional signed lockfiles (minisign / sigstore) — still exact-hash underneath diff --git a/package-lock.json b/package-lock.json index cec5c4b..4362c20 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "surfacepin", - "version": "1.0.0", + "version": "1.4.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "surfacepin", - "version": "1.0.0", + "version": "1.4.0", "license": "MIT", "dependencies": { "@modelcontextprotocol/sdk": "^1.30.1", diff --git a/package.json b/package.json index 958811c..d689bd7 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "surfacepin", - "version": "1.3.0", - "description": "Lock exact hashes of MCP tools/resources/prompts list surfaces; fail CI on silent drift. Offline files or live --stdio. Lockfile v3 structured schema field-diff.", + "version": "1.4.0", + "description": "Lock exact hashes of MCP tools/resources/prompts list surfaces; fail CI on silent drift. Offline files or live --stdio. Lockfile v3 structured field-diff (COMPATIBLE|BREAKING|HINT_FLIP).", "license": "MIT", "author": "yellowgram", "type": "module", diff --git a/src/cli.ts b/src/cli.ts index d4121c4..7dd1625 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -36,7 +36,7 @@ Live --stdio calls tools/list, resources/list, prompts/list as selected. Servers lacking a capability → empty list + stderr note. diff prints ADDED/REMOVED/CHANGED ids; for CHANGED tools/resources/prompts with -a v3 lock, also prints path-level schema/field changes (BREAKING / non-breaking). +a v3 lock, also prints path-level field changes (COMPATIBLE / BREAKING / HINT_FLIP). --json emits the DiffResult object. Exit codes: 0 match/ok, 1 drift, 2 usage/error diff --git a/src/hash.ts b/src/hash.ts index fd35397..85472d8 100644 --- a/src/hash.ts +++ b/src/hash.ts @@ -13,11 +13,17 @@ export function sha256Hex(data: Buffer | string): string { /** Per-tool digest over the surface descriptor object. */ export function toolDigest(descriptor: ToolDescriptor): string { - const payload = { + const payload: Record = { description: descriptor.description, inputSchema: descriptor.inputSchema, name: descriptor.name, }; + if (descriptor.annotations !== undefined) { + payload.annotations = descriptor.annotations; + } + if (descriptor.outputSchema !== undefined) { + payload.outputSchema = descriptor.outputSchema; + } return sha256Hex(canonicalizeToBytes(payload)); } diff --git a/src/index.ts b/src/index.ts index 7a7087c..d2e2b8b 100644 --- a/src/index.ts +++ b/src/index.ts @@ -38,7 +38,9 @@ export type { FieldChange, ChangeSeverity, ChangeKind, + DiffTaxonomyKind, } from "./schema-diff.js"; +export { normalizeKinds } from "./schema-diff.js"; export { fetchToolsViaStdio, fetchSurfacesViaStdio } from "./mcp-stdio.js"; export type { StdioFetchOptions, ToolsListDoc } from "./mcp-stdio.js"; export type { diff --git a/src/lock.ts b/src/lock.ts index 1a2f70f..783b1c8 100644 --- a/src/lock.ts +++ b/src/lock.ts @@ -48,11 +48,14 @@ export interface ComputedSurface { /** Exact hashed tool payload (matches toolDigest input). */ function toolSurfacePayload(d: ToolDescriptor): ToolDescriptor { - return { + const payload: ToolDescriptor = { description: d.description, inputSchema: d.inputSchema, name: d.name, }; + if (d.annotations !== undefined) payload.annotations = d.annotations; + if (d.outputSchema !== undefined) payload.outputSchema = d.outputSchema; + return payload; } /** Exact hashed resource payload (matches resourceDigest input). */ diff --git a/src/normalize.ts b/src/normalize.ts index 30c29eb..f30c589 100644 --- a/src/normalize.ts +++ b/src/normalize.ts @@ -88,7 +88,42 @@ export function normalizeTools(raw: ToolSurface[]): ToolDescriptor[] { inputSchema = t.inputSchema as Record; } - out.push({ name: t.name, description, inputSchema }); + // annotations: omit if absent/non-object; drop title; omit if empty after drop. + // Do not materialize MCP defaults into hashed fields. + let annotations: Record | undefined; + const rawAnn = t.annotations; + if (rawAnn !== undefined && rawAnn !== null) { + if (!isPlainObject(rawAnn)) { + // Non-object annotations → omit key (same as absent) + annotations = undefined; + } else { + const kept: Record = {}; + for (const [k, v] of Object.entries(rawAnn)) { + if (k === "title") continue; + kept[k] = v; + } + if (Object.keys(kept).length > 0) { + annotations = kept; + } + } + } + + // outputSchema: absent/null → omit; {} → include; non-object → usage error. + let outputSchema: Record | undefined; + const rawOut = t.outputSchema; + if (rawOut !== undefined && rawOut !== null) { + if (!isPlainObject(rawOut)) { + throw new SurfacePinError( + `tools[${i}].outputSchema must be an object when present`, + ); + } + outputSchema = rawOut as Record; + } + + const desc: ToolDescriptor = { name: t.name, description, inputSchema }; + if (annotations !== undefined) desc.annotations = annotations; + if (outputSchema !== undefined) desc.outputSchema = outputSchema; + out.push(desc); } out.sort((a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0)); diff --git a/src/schema-diff.ts b/src/schema-diff.ts index abfdaaa..325e13c 100644 --- a/src/schema-diff.ts +++ b/src/schema-diff.ts @@ -3,6 +3,9 @@ * Exact structural comparison only — no LLM, embeddings, or fuzzy similarity. */ +/** Taxonomy labels for explanatory field-diff (never affect exit codes). */ +export type DiffTaxonomyKind = "COMPATIBLE" | "BREAKING" | "HINT_FLIP"; +/** @deprecated Use DiffTaxonomyKind / FieldChange.kinds */ export type ChangeSeverity = "breaking" | "non-breaking"; export type ChangeKind = "added" | "removed" | "changed"; @@ -10,12 +13,28 @@ export interface FieldChange { /** Dot-path from descriptor root, e.g. "inputSchema.properties.text.type" */ path: string; kind: ChangeKind; - severity: ChangeSeverity; + /** + * Taxonomy labels. Usually one of COMPATIBLE | BREAKING | HINT_FLIP. + * HINT_FLIP may co-occur with BREAKING on the same path → both listed. + */ + kinds: DiffTaxonomyKind[]; /** Compact JSON for display (optional). */ oldValue?: string; newValue?: string; } +/** Stable display order for kinds on one path line. */ +const KINDS_ORDER: DiffTaxonomyKind[] = ["HINT_FLIP", "BREAKING", "COMPATIBLE"]; + +export function normalizeKinds(kinds: DiffTaxonomyKind[]): DiffTaxonomyKind[] { + const set = new Set(kinds); + return KINDS_ORDER.filter((k) => set.has(k)); +} + +function sevToKinds(severity: ChangeSeverity): DiffTaxonomyKind[] { + return severity === "breaking" ? ["BREAKING"] : ["COMPATIBLE"]; +} + const META_KEYS = new Set([ "description", "title", @@ -76,11 +95,16 @@ function push( out: FieldChange[], path: string, kind: ChangeKind, - severity: ChangeSeverity, + severityOrKinds: ChangeSeverity | DiffTaxonomyKind[], oldValue?: unknown, newValue?: unknown, ): void { - const c: FieldChange = { path, kind, severity }; + const kinds = normalizeKinds( + Array.isArray(severityOrKinds) + ? severityOrKinds + : sevToKinds(severityOrKinds), + ); + const c: FieldChange = { path, kind, kinds }; if (oldValue !== undefined) c.oldValue = compactJson(oldValue); if (newValue !== undefined) c.newValue = compactJson(newValue); out.push(c); @@ -468,10 +492,133 @@ function diffAdditionalProperties( // --- Descriptor-level diffs (tool / resource / prompt) --- -/** Diff two tool descriptors (name/description/inputSchema). */ +/** Known MCP ToolAnnotations hint keys. */ +const KNOWN_HINTS = new Set([ + "readOnlyHint", + "destructiveHint", + "idempotentHint", + "openWorldHint", +]); + +/** + * HINT_FLIP transitions (scarier / lost safety-leaning hints). + * Safer or newly asserted known hints → COMPATIBLE. + * Unknown annotation keys: any add/remove/change → HINT_FLIP. + */ +function isHintFlip( + key: string, + oldV: unknown | undefined, + newV: unknown | undefined, +): boolean { + if (!KNOWN_HINTS.has(key)) { + // unknown key: added, removed, or changed + return true; + } + const present = (v: unknown | undefined) => v !== undefined; + // readOnlyHint: true→false or true→absent + if (key === "readOnlyHint") { + return oldV === true && (newV === false || !present(newV)); + } + // destructiveHint: false→true or false→absent + if (key === "destructiveHint") { + return oldV === false && (newV === true || !present(newV)); + } + // idempotentHint: true→false or true→absent + if (key === "idempotentHint") { + return oldV === true && (newV === false || !present(newV)); + } + // openWorldHint: false→true or false→absent + if (key === "openWorldHint") { + return oldV === false && (newV === true || !present(newV)); + } + return false; +} + +/** Diff tool.annotations (never emits annotations.title). */ +function diffAnnotations( + out: FieldChange[], + oldAnn: Record | undefined, + newAnn: Record | undefined, +): void { + const oldO = oldAnn ?? {}; + const newO = newAnn ?? {}; + // If both absent/empty, nothing to report + const oldKeys = new Set(Object.keys(oldO)); + const newKeys = new Set(Object.keys(newO)); + if (oldKeys.size === 0 && newKeys.size === 0) return; + + const keys = new Set([...oldKeys, ...newKeys]); + for (const key of [...keys].sort(cmpStr)) { + // title is never hashed / never in diff paths + if (key === "title") continue; + const ov = oldO[key]; + const nv = newO[key]; + const oldPresent = oldKeys.has(key); + const newPresent = newKeys.has(key); + if (oldPresent && newPresent && deepEqual(ov, nv)) continue; + + const path = `annotations.${key}`; + let changeKind: ChangeKind; + if (!oldPresent && newPresent) changeKind = "added"; + else if (oldPresent && !newPresent) changeKind = "removed"; + else changeKind = "changed"; + + const flip = isHintFlip( + key, + oldPresent ? ov : undefined, + newPresent ? nv : undefined, + ); + const taxonomy: DiffTaxonomyKind[] = flip ? ["HINT_FLIP"] : ["COMPATIBLE"]; + push( + out, + path, + changeKind, + taxonomy, + oldPresent ? ov : undefined, + newPresent ? nv : undefined, + ); + } +} + +/** Diff tool.outputSchema with existing schema rules; never HINT_FLIP. */ +function diffOutputSchema( + out: FieldChange[], + oldS: Record | undefined, + newS: Record | undefined, +): void { + const oldPresent = oldS !== undefined; + const newPresent = newS !== undefined; + if (!oldPresent && !newPresent) return; + if (!oldPresent && newPresent) { + // add = BREAKING + push(out, "outputSchema", "added", "breaking", undefined, newS); + return; + } + if (oldPresent && !newPresent) { + // remove = BREAKING + push(out, "outputSchema", "removed", "breaking", oldS, undefined); + return; + } + // both present: recursive schema diff under outputSchema path + out.push(...diffJsonSchema(oldS, newS, "outputSchema")); +} + +/** Diff two tool descriptors (hashed fields). */ export function diffToolDescriptors( - oldD: { name: string; description: string; inputSchema: Record }, - newD: { name: string; description: string; inputSchema: Record }, + oldD: { + name: string; + description: string; + inputSchema: Record; + annotations?: Record; + outputSchema?: Record; + }, + newD: { + name: string; + description: string; + inputSchema: Record; + annotations?: Record; + outputSchema?: Record; + }, ): FieldChange[] { const out: FieldChange[] = []; if (oldD.name !== newD.name) { @@ -481,6 +628,8 @@ export function diffToolDescriptors( push(out, "description", "changed", "non-breaking", oldD.description, newD.description); } out.push(...diffJsonSchema(oldD.inputSchema, newD.inputSchema, "inputSchema")); + diffAnnotations(out, oldD.annotations, newD.annotations); + diffOutputSchema(out, oldD.outputSchema, newD.outputSchema); out.sort((a, b) => cmpStr(a.path, b.path) || cmpStr(a.kind, b.kind)); return out; } @@ -596,8 +745,8 @@ export function diffPromptDescriptors( export function formatFieldChanges(changes: FieldChange[], indent = " "): string[] { const lines: string[] = []; for (const c of changes) { - const tag = c.severity === "breaking" ? "BREAKING" : "non-breaking"; - let line = `${indent}${c.kind.toUpperCase().padEnd(7)} ${c.path} (${tag})`; + const tag = normalizeKinds(c.kinds).join(" "); + const line = `${indent}${c.kind.toUpperCase().padEnd(7)} ${c.path} (${tag})`; if (c.kind === "changed" && c.oldValue !== undefined && c.newValue !== undefined) { lines.push(line); lines.push(`${indent} ${c.oldValue} -> ${c.newValue}`); diff --git a/src/test/schema-diff.test.ts b/src/test/schema-diff.test.ts index a7da59e..70c6fe5 100644 --- a/src/test/schema-diff.test.ts +++ b/src/test/schema-diff.test.ts @@ -28,7 +28,7 @@ describe("diffJsonSchema breaking / non-breaking", () => { assert.equal(changes.length, 1); assert.equal(changes[0].path, "properties.b"); assert.equal(changes[0].kind, "added"); - assert.equal(changes[0].severity, "non-breaking"); + assert.deepEqual(changes[0].kinds, ["COMPATIBLE"]); }); it("removing property is breaking", () => { @@ -41,7 +41,7 @@ describe("diffJsonSchema breaking / non-breaking", () => { properties: { a: { type: "string" } }, }; const changes = diffJsonSchema(oldS, newS); - assert.ok(changes.some((c) => c.path === "properties.b" && c.kind === "removed" && c.severity === "breaking")); + assert.ok(changes.some((c) => c.path === "properties.b" && c.kind === "removed" && c.kinds.includes("BREAKING"))); }); it("adding to required is breaking", () => { @@ -61,7 +61,7 @@ describe("diffJsonSchema breaking / non-breaking", () => { (c) => c.path.includes("required") && c.kind === "added" && - c.severity === "breaking", + c.kinds.includes("BREAKING"), ), ); }); @@ -83,7 +83,7 @@ describe("diffJsonSchema breaking / non-breaking", () => { (c) => c.path.includes("required") && c.kind === "removed" && - c.severity === "non-breaking", + c.kinds.includes("COMPATIBLE"), ), ); }); @@ -95,7 +95,7 @@ describe("diffJsonSchema breaking / non-breaking", () => { ); assert.ok( narrow.some( - (c) => c.path === "type" && c.severity === "breaking", + (c) => c.path === "type" && c.kinds.includes("BREAKING"), ), ); const widen = diffJsonSchema( @@ -104,7 +104,7 @@ describe("diffJsonSchema breaking / non-breaking", () => { ); assert.ok( widen.some( - (c) => c.path === "type" && c.severity === "non-breaking", + (c) => c.path === "type" && c.kinds.includes("COMPATIBLE"), ), ); }); @@ -116,7 +116,7 @@ describe("diffJsonSchema breaking / non-breaking", () => { ); assert.equal(changes.length, 1); assert.equal(changes[0].path, "description"); - assert.equal(changes[0].severity, "non-breaking"); + assert.deepEqual(changes[0].kinds, ["COMPATIBLE"]); }); it("enum value removal is breaking; add is non-breaking", () => { @@ -124,12 +124,12 @@ describe("diffJsonSchema breaking / non-breaking", () => { { enum: ["a", "b"] }, { enum: ["a"] }, ); - assert.ok(rem.some((c) => c.kind === "removed" && c.severity === "breaking")); + assert.ok(rem.some((c) => c.kind === "removed" && c.kinds.includes("BREAKING"))); const add = diffJsonSchema( { enum: ["a"] }, { enum: ["a", "b"] }, ); - assert.ok(add.some((c) => c.kind === "added" && c.severity === "non-breaking")); + assert.ok(add.some((c) => c.kind === "added" && c.kinds.includes("COMPATIBLE"))); }); it("paths are sorted stably", () => { @@ -172,14 +172,14 @@ describe("diffToolDescriptors / prompts", () => { const changes = diffToolDescriptors(oldD, newD); assert.ok( changes.some( - (c) => c.path === "description" && c.severity === "non-breaking", + (c) => c.path === "description" && c.kinds.includes("COMPATIBLE"), ), ); assert.ok( changes.some( (c) => c.path === "inputSchema.properties.x.type" && - c.severity === "breaking", + c.kinds.includes("BREAKING"), ), ); }); @@ -201,7 +201,7 @@ describe("diffToolDescriptors / prompts", () => { (c) => c.path.includes("required") && c.kind === "changed" && - c.severity === "breaking", + c.kinds.includes("BREAKING"), ), ); }); @@ -253,7 +253,7 @@ describe("lockfile v3 structured diff integration", () => { fields.some( (f) => f.path === "inputSchema.properties.text.type" && - f.severity === "breaking", + f.kinds.includes("BREAKING"), ), ); assert.ok( @@ -261,7 +261,7 @@ describe("lockfile v3 structured diff integration", () => { (f) => f.path === "inputSchema.properties.lang" && f.kind === "added" && - f.severity === "non-breaking", + f.kinds.includes("COMPATIBLE"), ), ); assert.ok( @@ -269,19 +269,19 @@ describe("lockfile v3 structured diff integration", () => { (f) => f.path.includes("required") && f.kind === "added" && - f.severity === "breaking", + f.kinds.includes("BREAKING"), ), ); assert.ok( fields.some( - (f) => f.path === "description" && f.severity === "non-breaking", + (f) => f.path === "description" && f.kinds.includes("COMPATIBLE"), ), ); const text = formatDiff(d); assert.match(text, /BREAKING/); assert.match(text, /inputSchema\.properties\.text\.type/); - assert.match(text, /non-breaking/); + assert.match(text, /COMPATIBLE/); }); it("v1 lock still verifies by digest without field paths", () => { @@ -344,18 +344,18 @@ describe("lockfile v3 structured diff integration", () => { { path: "inputSchema.properties.a", kind: "removed", - severity: "breaking", + kinds: ["BREAKING"], oldValue: '{"type":"string"}', }, { path: "description", kind: "changed", - severity: "non-breaking", + kinds: ["COMPATIBLE"], oldValue: '"x"', newValue: '"y"', }, ]); assert.ok(lines.some((l) => l.includes("BREAKING"))); - assert.ok(lines.some((l) => l.includes("non-breaking"))); + assert.ok(lines.some((l) => l.includes("COMPATIBLE"))); }); }); diff --git a/src/test/spec-1.4.test.ts b/src/test/spec-1.4.test.ts new file mode 100644 index 0000000..43ef139 --- /dev/null +++ b/src/test/spec-1.4.test.ts @@ -0,0 +1,592 @@ +import assert from "node:assert/strict"; +import { describe, it } from "node:test"; +import { canonicalize } from "../canonicalize.js"; +import { sha256Hex, toolDigest } from "../hash.js"; +import { computeSurface } from "../lock.js"; +import { diffToolDescriptors } from "../schema-diff.js"; +import { normalizeTools } from "../normalize.js"; +import { diffSurface, parseLockfile } from "../verify.js"; +import { SurfacePinError, type LockfileV3 } from "../types.js"; +import { readFileSync } from "node:fs"; +import { fileURLToPath } from "node:url"; +import { dirname, join } from "node:path"; + +const here = dirname(fileURLToPath(import.meta.url)); +const testdata = join(here, "..", "..", "testdata"); + +describe("spec 1.4 — annotations + outputSchema hashing", () => { + it("existing goldens unchanged for tools without kept annotations/outputSchema", () => { + const tools = JSON.parse( + readFileSync(join(testdata, "basic.tools.json"), "utf8"), + ); + const golden = JSON.parse( + readFileSync(join(testdata, "basic.v3.lock.json"), "utf8"), + ); + const { lockfile } = computeSurface(tools); + assert.deepEqual(lockfile, golden); + }); + + it("annotations+outputSchema golden lockfile v3", () => { + const tools = JSON.parse( + readFileSync(join(testdata, "annotations-output.tools.json"), "utf8"), + ); + const golden = JSON.parse( + readFileSync(join(testdata, "annotations-output.v3.lock.json"), "utf8"), + ); + const { lockfile } = computeSurface(tools); + assert.deepEqual(lockfile, golden); + const v3 = lockfile as LockfileV3; + const surf = v3.tools!.entries[0].surface as { + annotations?: Record; + outputSchema?: Record; + title?: unknown; + }; + assert.ok(surf.annotations); + assert.equal(surf.annotations!.title, undefined); + assert.equal(surf.annotations!.readOnlyHint, true); + assert.ok(surf.outputSchema); + assert.equal(surf.title, undefined); + }); + + it("title / icons / _meta do not change digest", () => { + const base = { + tools: [ + { + name: "t", + description: "d", + inputSchema: { type: "object" }, + }, + ], + }; + const decorated = { + tools: [ + { + name: "t", + description: "d", + inputSchema: { type: "object" }, + title: "Pretty Title", + icons: [{ src: "https://example.com/i.png" }], + _meta: { "io.modelcontextprotocol/foo": 1 }, + }, + ], + }; + assert.equal(computeSurface(base).root, computeSurface(decorated).root); + assert.equal( + computeSurface(base).tools!.entries[0].digest, + computeSurface(decorated).tools!.entries[0].digest, + ); + }); + + it("title / icons / _meta absent from diff paths", () => { + const base = { + tools: [ + { + name: "t", + description: "d", + inputSchema: { type: "object", properties: { a: { type: "string" } } }, + }, + ], + }; + const { lockfile } = computeSurface(base); + const drifted = { + tools: [ + { + name: "t", + description: "d2", + title: "X", + icons: [], + _meta: { x: 1 }, + inputSchema: { + type: "object", + properties: { a: { type: "number" } }, + }, + }, + ], + }; + const d = diffSurface(drifted, lockfile); + assert.equal(d.match, false); + const paths = (d.changed[0].fields ?? []).map((f) => f.path); + for (const p of paths) { + assert.ok(!p.includes("title"), p); + assert.ok(!p.includes("icons"), p); + assert.ok(!p.includes("_meta"), p); + } + }); + + it("annotations.title dropped → same digest as no title", () => { + const a = { + tools: [ + { + name: "t", + description: "", + inputSchema: {}, + annotations: { title: "Ignored", readOnlyHint: true }, + }, + ], + }; + const b = { + tools: [ + { + name: "t", + description: "", + inputSchema: {}, + annotations: { readOnlyHint: true }, + }, + ], + }; + assert.equal( + computeSurface(a).tools!.entries[0].digest, + computeSurface(b).tools!.entries[0].digest, + ); + }); + + it("annotations unknown key changes digest", () => { + const a = { + tools: [ + { + name: "t", + description: "", + inputSchema: {}, + annotations: { customHint: 1 }, + }, + ], + }; + const b = { + tools: [ + { + name: "t", + description: "", + inputSchema: {}, + annotations: { customHint: 2 }, + }, + ], + }; + assert.notEqual( + computeSurface(a).tools!.entries[0].digest, + computeSurface(b).tools!.entries[0].digest, + ); + }); + + it("empty or title-only annotations omits key", () => { + const empty = normalizeTools([ + { name: "t", annotations: {} }, + ]); + assert.equal(empty[0].annotations, undefined); + + const titleOnly = normalizeTools([ + { name: "t", annotations: { title: "Only" } }, + ]); + assert.equal(titleOnly[0].annotations, undefined); + + const none = normalizeTools([{ name: "t" }]); + assert.equal(none[0].annotations, undefined); + + // Digests match across empty / title-only / absent + assert.equal(toolDigest(empty[0]), toolDigest(titleOnly[0])); + assert.equal(toolDigest(empty[0]), toolDigest(none[0])); + }); + + it("absent annotations ≠ explicit MCP defaults", () => { + const absent = { + tools: [{ name: "t", description: "", inputSchema: {} }], + }; + const explicitDefaults = { + tools: [ + { + name: "t", + description: "", + inputSchema: {}, + annotations: { + readOnlyHint: false, + destructiveHint: false, + idempotentHint: false, + openWorldHint: true, + }, + }, + ], + }; + assert.notEqual( + computeSurface(absent).tools!.entries[0].digest, + computeSurface(explicitDefaults).tools!.entries[0].digest, + ); + }); + + it("outputSchema absent ≠ empty object", () => { + const absent = { + tools: [{ name: "t", description: "", inputSchema: {} }], + }; + const emptyObj = { + tools: [ + { + name: "t", + description: "", + inputSchema: {}, + outputSchema: {}, + }, + ], + }; + assert.notEqual( + computeSurface(absent).tools!.entries[0].digest, + computeSurface(emptyObj).tools!.entries[0].digest, + ); + const desc = normalizeTools(emptyObj.tools as never); + assert.deepEqual(desc[0].outputSchema, {}); + assert.equal(normalizeTools(absent.tools as never)[0].outputSchema, undefined); + }); + + it("outputSchema null is omitted; non-object is usage error", () => { + const withNull = normalizeTools([ + { name: "t", outputSchema: null }, + ]); + assert.equal(withNull[0].outputSchema, undefined); + + assert.throws( + () => normalizeTools([{ name: "t", outputSchema: "nope" }]), + (e: unknown) => + e instanceof SurfacePinError && + e.exitCode === 2 && + /outputSchema/.test(e.message), + ); + }); + + it("outputSchema add/remove/change never HINT_FLIP", () => { + const oldD = { + name: "t", + description: "", + inputSchema: {}, + outputSchema: { type: "object", properties: { a: { type: "string" } } }, + }; + const removed = diffToolDescriptors(oldD, { + name: "t", + description: "", + inputSchema: {}, + }); + assert.ok(removed.some((c) => c.path === "outputSchema")); + for (const c of removed.filter((c) => c.path.startsWith("outputSchema"))) { + assert.ok(!c.kinds.includes("HINT_FLIP"), JSON.stringify(c)); + assert.ok(c.kinds.includes("BREAKING") || c.kinds.includes("COMPATIBLE")); + } + + const added = diffToolDescriptors( + { name: "t", description: "", inputSchema: {} }, + oldD, + ); + assert.ok( + added.some( + (c) => + c.path === "outputSchema" && + c.kind === "added" && + c.kinds.includes("BREAKING") && + !c.kinds.includes("HINT_FLIP"), + ), + ); + + const narrowed = diffToolDescriptors(oldD, { + name: "t", + description: "", + inputSchema: {}, + outputSchema: { type: "object", properties: { a: { type: "number" } } }, + }); + assert.ok( + narrowed.some( + (c) => + c.path === "outputSchema.properties.a.type" && + c.kinds.includes("BREAKING") && + !c.kinds.includes("HINT_FLIP"), + ), + ); + }); +}); + +describe("spec 1.4 — HINT_FLIP taxonomy", () => { + function tool( + ann?: Record, + extra: Record = {}, + ) { + return { + name: "t", + description: "", + inputSchema: {}, + ...(ann ? { annotations: ann } : {}), + ...extra, + }; + } + + it("readOnlyHint true→false is HINT_FLIP", () => { + const changes = diffToolDescriptors( + tool({ readOnlyHint: true }), + tool({ readOnlyHint: false }), + ); + const c = changes.find((x) => x.path === "annotations.readOnlyHint"); + assert.ok(c); + assert.deepEqual(c!.kinds, ["HINT_FLIP"]); + }); + + it("destructiveHint false→true is HINT_FLIP", () => { + const changes = diffToolDescriptors( + tool({ destructiveHint: false }), + tool({ destructiveHint: true }), + ); + const c = changes.find((x) => x.path === "annotations.destructiveHint"); + assert.ok(c); + assert.deepEqual(c!.kinds, ["HINT_FLIP"]); + }); + + it("idempotentHint true→false is HINT_FLIP", () => { + const changes = diffToolDescriptors( + tool({ idempotentHint: true }), + tool({ idempotentHint: false }), + ); + const c = changes.find((x) => x.path === "annotations.idempotentHint"); + assert.ok(c); + assert.deepEqual(c!.kinds, ["HINT_FLIP"]); + }); + + it("openWorldHint false→true is HINT_FLIP", () => { + const changes = diffToolDescriptors( + tool({ openWorldHint: false }), + tool({ openWorldHint: true }), + ); + const c = changes.find((x) => x.path === "annotations.openWorldHint"); + assert.ok(c); + assert.deepEqual(c!.kinds, ["HINT_FLIP"]); + }); + + it("explicit default then absent still HINT_FLIP (destructiveHint false→absent)", () => { + const changes = diffToolDescriptors( + tool({ destructiveHint: false }), + tool(), // absent + ); + const c = changes.find((x) => x.path === "annotations.destructiveHint"); + assert.ok(c); + assert.equal(c!.kind, "removed"); + assert.deepEqual(c!.kinds, ["HINT_FLIP"]); + }); + + it("safer / newly asserted known hints are COMPATIBLE not HINT_FLIP", () => { + const cases: Array<[Record | undefined, Record | undefined, string]> = [ + [tool({ readOnlyHint: false }).annotations, tool({ readOnlyHint: true }).annotations, "annotations.readOnlyHint"], + [undefined, { readOnlyHint: true }, "annotations.readOnlyHint"], + [{ destructiveHint: true }, { destructiveHint: false }, "annotations.destructiveHint"], + [undefined, { destructiveHint: false }, "annotations.destructiveHint"], + [{ idempotentHint: false }, { idempotentHint: true }, "annotations.idempotentHint"], + [{ openWorldHint: true }, { openWorldHint: false }, "annotations.openWorldHint"], + ]; + for (const [oldA, newA, path] of cases) { + const changes = diffToolDescriptors( + { name: "t", description: "", inputSchema: {}, ...(oldA ? { annotations: oldA } : {}) }, + { name: "t", description: "", inputSchema: {}, ...(newA ? { annotations: newA } : {}) }, + ); + const c = changes.find((x) => x.path === path); + assert.ok(c, `missing ${path}`); + assert.ok(c!.kinds.includes("COMPATIBLE"), JSON.stringify(c)); + assert.ok(!c!.kinds.includes("HINT_FLIP"), JSON.stringify(c)); + } + }); + + it("unknown annotation key add/remove/change is HINT_FLIP", () => { + const add = diffToolDescriptors(tool(), tool({ customX: 1 })); + assert.ok( + add.some( + (c) => + c.path === "annotations.customX" && + c.kind === "added" && + c.kinds.includes("HINT_FLIP"), + ), + ); + const rem = diffToolDescriptors(tool({ customX: 1 }), tool()); + assert.ok( + rem.some( + (c) => + c.path === "annotations.customX" && + c.kind === "removed" && + c.kinds.includes("HINT_FLIP"), + ), + ); + const ch = diffToolDescriptors(tool({ customX: 1 }), tool({ customX: 2 })); + assert.ok( + ch.some( + (c) => + c.path === "annotations.customX" && + c.kind === "changed" && + c.kinds.includes("HINT_FLIP"), + ), + ); + }); + +}); + +describe("spec 1.4 — dual kinds + invariants", () => { + it("dual kinds array on same FieldChange is representable and printed", async () => { + const { formatFieldChanges, normalizeKinds } = await import("../schema-diff.js"); + assert.deepEqual(normalizeKinds(["BREAKING", "HINT_FLIP", "COMPATIBLE"]), [ + "HINT_FLIP", + "BREAKING", + "COMPATIBLE", + ]); + const lines = formatFieldChanges([ + { + path: "annotations.customX", + kind: "changed", + kinds: ["HINT_FLIP", "BREAKING"], + oldValue: "1", + newValue: "2", + }, + ]); + assert.ok(lines[0].includes("HINT_FLIP BREAKING")); + assert.ok(lines[0].includes("annotations.customX")); + }); + + it("key order invariant (JCS) — annotations/outputSchema key order irrelevant", () => { + const a = { + tools: [ + { + name: "t", + description: "d", + inputSchema: { type: "object", properties: { z: { type: "string" }, a: { type: "string" } } }, + annotations: { openWorldHint: false, readOnlyHint: true }, + outputSchema: { type: "object", properties: { b: { type: "number" }, a: { type: "string" } } }, + }, + ], + }; + const b = { + tools: [ + { + outputSchema: { properties: { a: { type: "string" }, b: { type: "number" } }, type: "object" }, + annotations: { readOnlyHint: true, openWorldHint: false }, + inputSchema: { properties: { a: { type: "string" }, z: { type: "string" } }, type: "object" }, + description: "d", + name: "t", + }, + ], + }; + assert.equal( + computeSurface(a).tools!.entries[0].digest, + computeSurface(b).tools!.entries[0].digest, + ); + }); + + it("v1 / v2 / v3 verify compat for tools without annotations/outputSchema", () => { + const tools = JSON.parse( + readFileSync(join(testdata, "basic.tools.json"), "utf8"), + ); + const v1 = parseLockfile( + JSON.parse(readFileSync(join(testdata, "basic.lock.json"), "utf8")), + ); + const v3 = parseLockfile( + JSON.parse(readFileSync(join(testdata, "basic.v3.lock.json"), "utf8")), + ); + assert.equal(diffSurface(tools, v1).match, true); + assert.equal(diffSurface(tools, v3).match, true); + + // Synthesize v2 from current compute (digests identical) + const computed = computeSurface(tools); + const v2 = parseLockfile({ + version: 2, + algorithm: "sha256", + canonicalization: "surfacepin-jcs-v1", + root: computed.root, + tools: { + root: computed.tools!.root, + entries: computed.tools!.entries.map(({ name, digest }) => ({ + name, + digest, + })), + }, + }); + assert.equal(diffSurface(tools, v2).match, true); + }); + + it("digest changes when tool had kept annotation keys or outputSchema", () => { + const plain = { + tools: [{ name: "t", description: "", inputSchema: {} }], + }; + const withAnn = { + tools: [ + { + name: "t", + description: "", + inputSchema: {}, + annotations: { readOnlyHint: true }, + }, + ], + }; + const withOut = { + tools: [ + { + name: "t", + description: "", + inputSchema: {}, + outputSchema: { type: "object" }, + }, + ], + }; + assert.notEqual( + computeSurface(plain).tools!.entries[0].digest, + computeSurface(withAnn).tools!.entries[0].digest, + ); + assert.notEqual( + computeSurface(plain).tools!.entries[0].digest, + computeSurface(withOut).tools!.entries[0].digest, + ); + }); + + it("toolDigest payload includes annotations/outputSchema when present", () => { + const d = { + name: "t", + description: "", + inputSchema: {}, + annotations: { readOnlyHint: true }, + outputSchema: {}, + }; + const expected = sha256Hex( + Buffer.from( + canonicalize({ + annotations: { readOnlyHint: true }, + description: "", + inputSchema: {}, + name: "t", + outputSchema: {}, + }), + "utf8", + ), + ); + assert.equal(toolDigest(d), expected); + }); + + it("integration: HINT_FLIP appears in diff -- structured fields", () => { + const base = { + tools: [ + { + name: "echo", + description: "d", + inputSchema: { type: "object" }, + annotations: { readOnlyHint: true }, + }, + ], + }; + const { lockfile } = computeSurface(base); + const drifted = { + tools: [ + { + name: "echo", + description: "d", + inputSchema: { type: "object" }, + annotations: { readOnlyHint: false }, + }, + ], + }; + const d = diffSurface(drifted, lockfile); + assert.equal(d.match, false); + const fields = d.changed[0].fields!; + assert.ok( + fields.some( + (f) => + f.path === "annotations.readOnlyHint" && + f.kinds.includes("HINT_FLIP"), + ), + ); + // never leak title + assert.ok(!fields.some((f) => f.path.includes("title"))); + }); +}); diff --git a/src/types.ts b/src/types.ts index 36c0384..d8d7b62 100644 --- a/src/types.ts +++ b/src/types.ts @@ -9,11 +9,15 @@ export const ALL_SURFACE_KINDS: readonly SurfaceKind[] = [ "prompts", ] as const; -/** MCP Tool fields used by SurfacePin. */ +/** MCP Tool fields used by SurfacePin (title/icons/_meta ignored). */ export interface ToolSurface { name: string; description?: string; inputSchema?: Record; + /** Client hints; title dropped; absent/non-object omitted from hash. */ + annotations?: unknown; + /** Structured output contract; absent/null omitted; {} kept; non-object → error. */ + outputSchema?: unknown; } /** Normalized descriptor hashed for a single tool. */ @@ -21,6 +25,10 @@ export interface ToolDescriptor { name: string; description: string; inputSchema: Record; + /** Present only when tool.annotations is an object with keys after dropping title. */ + annotations?: Record; + /** Present only when tool.outputSchema is a (possibly empty) object. */ + outputSchema?: Record; } /** MCP Resource fields used by SurfacePin (volatile fields ignored). */ diff --git a/src/verify.ts b/src/verify.ts index 88b3ece..9afa8e2 100644 --- a/src/verify.ts +++ b/src/verify.ts @@ -42,11 +42,37 @@ function parseToolSurface(raw: unknown, label: string): ToolDescriptor | undefin if (!isPlainObject(raw.inputSchema)) { throw new SurfacePinError(`${label}.surface.inputSchema must be an object`); } - return { + const desc: ToolDescriptor = { name: raw.name, description: raw.description, inputSchema: raw.inputSchema as Record, }; + if (raw.annotations !== undefined) { + if (!isPlainObject(raw.annotations)) { + throw new SurfacePinError( + `${label}.surface.annotations must be an object when present`, + ); + } + // Drop title if somehow present in an old/corrupt lock; never hash/diff it. + const kept: Record = {}; + for (const [k, v] of Object.entries(raw.annotations)) { + if (k === "title") continue; + kept[k] = v; + } + if (Object.keys(kept).length > 0) desc.annotations = kept; + } + if (raw.outputSchema !== undefined) { + if (raw.outputSchema === null) { + // treat null as omit (should not appear in well-formed locks) + } else if (!isPlainObject(raw.outputSchema)) { + throw new SurfacePinError( + `${label}.surface.outputSchema must be an object when present`, + ); + } else { + desc.outputSchema = raw.outputSchema as Record; + } + } + return desc; } function parseResourceSurface( diff --git a/testdata/annotations-output.tools.json b/testdata/annotations-output.tools.json new file mode 100644 index 0000000..954e76d --- /dev/null +++ b/testdata/annotations-output.tools.json @@ -0,0 +1,40 @@ +{ + "tools": [ + { + "name": "echo", + "description": "Echo text back", + "inputSchema": { + "type": "object", + "properties": { + "text": { + "type": "string" + } + }, + "required": [ + "text" + ] + }, + "annotations": { + "readOnlyHint": true, + "title": "ignored" + }, + "outputSchema": { + "type": "object", + "properties": { + "text": { + "type": "string" + } + } + }, + "title": "Echo Tool", + "icons": [ + { + "src": "https://example.com/e.png" + } + ], + "_meta": { + "note": "ignored" + } + } + ] +} diff --git a/testdata/annotations-output.v3.lock.json b/testdata/annotations-output.v3.lock.json new file mode 100644 index 0000000..eb293ed --- /dev/null +++ b/testdata/annotations-output.v3.lock.json @@ -0,0 +1,41 @@ +{ + "version": 3, + "algorithm": "sha256", + "canonicalization": "surfacepin-jcs-v1", + "root": "eb3d367190d9d62f77225e211b90da7090ad11152b541190a35dda25b637dcfa", + "tools": { + "root": "600629ad9aa40f4266cbe3c1bf37b0f2c9164259100cc31ae0eec35afcdea7d1", + "entries": [ + { + "name": "echo", + "digest": "16c078cc8c9a59f0a1fef7b6067b995f2f805249f5e3c3123a27a66816d56057", + "surface": { + "description": "Echo text back", + "inputSchema": { + "type": "object", + "properties": { + "text": { + "type": "string" + } + }, + "required": [ + "text" + ] + }, + "name": "echo", + "annotations": { + "readOnlyHint": true + }, + "outputSchema": { + "type": "object", + "properties": { + "text": { + "type": "string" + } + } + } + } + } + ] + } +}