Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 12 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@

Lock **exact hashes** of an MCP server’s list surfaces and fail CI when they drift silently:

- **tools** — `name`, `description`, `inputSchema`
- **tools** — `name`, `description`, `inputSchema`, `annotations`, `outputSchema`
- **resources** — `uri`, `name`, `description`, `mimeType`
- **prompts** — `name`, `description`, `arguments`

This is deterministic byte-level hashing — **not** semantic / LLM / embedding drift detection. If a description character changes, the digest changes.

On mismatch, **lockfile v3** also explains *what* changed with a deterministic JSON Schema field-diff (breaking vs non-breaking). Pass/fail remains digest equality.
On mismatch, **lockfile v3** also explains *what* changed with a deterministic field-diff (`COMPATIBLE` | `BREAKING` | `HINT_FLIP`). Pass/fail remains digest equality. Re-lock after upgrading to ≥1.4.

Spec: [SPEC.md](./SPEC.md) (canonicalization `surfacepin-jcs-v1` + lockfile v1/v2/v3).
Spec: [SPEC.md](./SPEC.md) (Lockfile Spec v1.4; canonicalization `surfacepin-jcs-v1` + lockfile v1/v2/v3).

## 60-second start

Expand Down Expand Up @@ -54,14 +54,16 @@ When a tool’s `inputSchema` drifts under a v3 lock:
DRIFT: surface does not match lockfile
CHANGED echo
f35d75b2… -> a6fdd0e9…
CHANGED description (non-breaking)
CHANGED description (COMPATIBLE)
"Echo text back" -> "Echo text back (updated)"
ADDED inputSchema.properties.lang (non-breaking)
ADDED inputSchema.properties.lang (COMPATIBLE)
+ {"type":"string"}
CHANGED inputSchema.properties.text.type (BREAKING)
"string" -> "number"
ADDED inputSchema.required["lang"] (BREAKING)
+ "lang"
CHANGED annotations.readOnlyHint (HINT_FLIP)
true -> false
```

Machine output: `surfacepin diff … --json`.
Expand Down Expand Up @@ -89,16 +91,16 @@ Composite action under [`action/`](./action/). File-based tools verify (CI usual

Or call the CLI yourself after `npm install surfacepin`.

## What v1.3 does / does not
## What v1.4 does / does not

| Does | Does not |
|------|----------|
| Hash tools + resources + prompts | Semantic similarity gates |
| Hash tools (+ annotations, outputSchema) + resources + prompts | Semantic similarity gates |
| Lockfile v3 (embedded surfaces) + verify v1/v2/v3 | Streamable HTTP / SSE (stdio only) |
| Deterministic schema field-diff on `diff` | LLM / fuzzy matching |
| Deterministic field-diff: COMPATIBLE / BREAKING / HINT_FLIP | LLM / fuzzy matching; safety verdicts from hints |
| Offline verify from JSON files | Hosted service, telemetry, signed locks |
| Live list* via MCP stdio (`--stdio -- …`) | Resource templates list |
| Breaking vs non-breaking path classification | |
| Live list* via MCP stdio (`--stdio -- …`) | Resource templates / initialize.instructions |
| Ignore tool title / icons / _meta | Materialize MCP annotation defaults into hashes |

## License

Expand Down
96 changes: 72 additions & 24 deletions SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Canonical specification for SurfacePin surface locking.
CLI and Action are reference clients of this document.

**Status:** v1.3 (2026-09-24)
**Status:** Lockfile Spec v1.4 (2026-09-24)
**Scope:** MCP `tools/list`, `resources/list`, and `prompts/list` surfaces.
**Wire formats:** lockfile **v1** (tools digests), **v2** (multi-surface digests), **v3** (multi-surface + embedded hashed payloads for structured field-diff).
**Not in wire format:** signed lockfiles, semantic/embedding drift, LLM-assisted diff.
Expand All @@ -24,10 +24,14 @@ CLI and Action are reference clients of this document.

| Kind | MCP method | Identity key | Hashed fields |
|------|------------|--------------|---------------|
| `tools` | `tools/list` | `name` | `name`, `description`, `inputSchema` |
| `tools` | `tools/list` | `name` | `name`, `description`, `inputSchema`, `annotations`, `outputSchema` |
| `resources` | `resources/list` | `uri` | `uri`, `name`, `description`, `mimeType` |
| `prompts` | `prompts/list` | `name` | `name`, `description`, `arguments` |

**Field rule:** Hash a tools field only if an MCP client or host surfaces it for invocation, structured I/O, or client hints. Skip decorative and volatile fields. Do not inject MCP defaults into new hashed fields.

Aligned to MCP schema revision **2026-07-28** as a documentation reference only (not a runtime fetch).

Canonicalization algorithm id (unchanged): `"surfacepin-jcs-v1"`.

### 2.1 Input shapes
Expand All @@ -45,23 +49,33 @@ SurfacePin accepts a combined document:
or List*Result-shaped objects that include those keys (other top-level fields ignored).
Selected kinds with a missing key are treated as an **empty array**.

### 2.2 Tools (unchanged from v1)
### 2.2 Tools

| Field | Required | Notes |
|-------|----------|-------|
| `name` | yes | Non-empty string. Primary identity. |
| `description` | no | If absent, treated as `""`. |
| `inputSchema` | no | If absent, treated as `{}`. Must be a JSON object when present. |
| `description` | no | If absent, treated as `""` (1.3 coercion unchanged). |
| `inputSchema` | no | If absent, treated as `{}`. Must be a JSON object when present (1.3 coercion unchanged). |
| `annotations` | no | Source: `tool.annotations`. Drop only `title`. Keep all other keys including unknown. If absent or non-object → omit `annotations` key entirely. If empty after dropping `title` → omit key. **Do not** materialize MCP defaults into hashed fields. |
| `outputSchema` | no | Absent or `null` → omit key. Empty object `{}` → include as empty object. Present non-object → usage error (exit 2). **Do not** coerce absent → `{}` (absent ≠ `{}`). |

All other tool fields (`annotations`, `outputSchema`, etc.) are **ignored**.
Still ignored (not hashed, never appear in diff paths): `title`, `icons`, `_meta`.
Duplicate `name` values are a **usage error** (exit 2).

**Tool descriptor** (hashed):
**Tool descriptor** (hashed; optional keys omitted when absent):

```json
{ "description": "<string>", "inputSchema": { }, "name": "<string>" }
{
"annotations": { "readOnlyHint": true },
"description": "<string>",
"inputSchema": { },
"name": "<string>",
"outputSchema": { }
}
```

Known annotation hints: `readOnlyHint`, `destructiveHint`, `idempotentHint`, `openWorldHint`. Unknown keys are still hashed.

### 2.3 Resources

MCP Resource shape fields that matter for drift. **Ignored** (volatile / non-identity): `size`, `annotations`, `title`, `icons`, `_meta`, and any other fields.
Expand Down Expand Up @@ -347,32 +361,43 @@ Exit codes same as verify (0 match, 1 drift, 2 error).

## 6. Structured field-diff (explanatory)

Deterministic only — **no** LLM, embeddings, or fuzzy similarity. Classification does **not** affect exit codes.
Deterministic only — **no** LLM, embeddings, or fuzzy similarity. Classification does **not** affect exit codes (0 match / 1 drift / 2 usage). Pass/fail is **digest equality only**.

### 6.1 Scope

When a lock entry embeds `surface` and the item digest changed, compare old `surface` vs the newly computed descriptor:

| Kind | Diff engine |
|------|-------------|
| tools | descriptor `description` + recursive JSON Schema diff on `inputSchema` |
| tools | `description` + recursive JSON Schema diff on `inputSchema` / `outputSchema` + annotation hint diff |
| resources | `uri` / `name` / `description` / `mimeType` field compares |
| prompts | `description` + arguments matched **by name** |

### 6.2 Output lines
Never appear in diff paths: `annotations.title`, `tool.title`, `tool.icons`, `tool._meta`.

Stable sorted by `path`, then `kind`. Each change:
### 6.2 Taxonomy: COMPATIBLE | BREAKING | HINT_FLIP

```
ADDED|REMOVED|CHANGED <path> (BREAKING|non-breaking)
<old> -> <new> # or + value / - value
```
JSON field changes carry a `kinds` **array** of taxonomy labels. Human lines print all kinds on the same path line.

**Hint vs trust:** MCP `ToolAnnotations` are hints. Clients must not make tool-use decisions from untrusted servers. `HINT_FLIP` is explanatory client-hint drift, **not** a safety verdict. It is reserved for scarier or lost safety-leaning hints. Safer or newly asserted hints are `COMPATIBLE`. `outputSchema` is a structured I/O contract → classified `COMPATIBLE`|`BREAKING` only. **Hash is not safety.**

#### HINT_FLIP when

Paths use dotted JSON-ish form, e.g. `inputSchema.properties.text.type`, `inputSchema.required["lang"]`, `arguments["who"].required`.
- `readOnlyHint`: `true → false` or `true → absent`
- `destructiveHint`: `false → true` or `false → absent`
- `idempotentHint`: `true → false` or `true → absent`
- `openWorldHint`: `false → true` or `false → absent`
- unknown annotation key: added, removed, or changed

### 6.3 Breaking vs non-breaking (deterministic rules)
Known-hint transitions **not** in that list → `COMPATIBLE` (still a digest change). Examples: `readOnlyHint` `false → true` / `absent → true`; `destructiveHint` `true → false`; `idempotentHint` `false → true`; `openWorldHint` `true → false`.

**Breaking** (examples):
Explicit value equal to an MCP default then omitted still changes the digest and stays `HINT_FLIP` this cut when the transition is in the list above (do not special-case default-equivalent omit as `COMPATIBLE`).

#### outputSchema

Classify with existing JSON Schema rules (`COMPATIBLE`|`BREAKING` only; **never** `HINT_FLIP`). Add or remove of `outputSchema` = `BREAKING`. Unknown schema keyword = `BREAKING`.

#### Schema rules (inputSchema / outputSchema) — BREAKING examples

- Remove a `properties` entry
- Add a name to `required`
Expand All @@ -383,7 +408,7 @@ Paths use dotted JSON-ish form, e.g. `inputSchema.properties.text.type`, `inputS
- Resource `mimeType` / `name` / `uri` change
- Prompt argument removed; new required argument; `required: false → true`

**Non-breaking** (examples):
#### COMPATIBLE examples (schema / descriptor)

- Add an optional property (not listed in `required`)
- Remove a name from `required`
Expand All @@ -392,8 +417,22 @@ Paths use dotted JSON-ish form, e.g. `inputSchema.properties.text.type`, `inputS
- `description` / `title` / `$comment` / `$id` / `examples` / `default` changes
- Prompt argument description change; `required: true → false`; argument reorder only
- Add optional prompt argument (`required: false`)
- Safer / newly asserted known annotation hints (not in HINT_FLIP list)

Unknown keyword value changes default to **BREAKING** (fail-closed). Meta keys listed above are always `COMPATIBLE`.

Unknown keyword value changes default to **breaking** (fail-closed classification). Meta keys listed above are always non-breaking.
`HINT_FLIP` may also be `BREAKING` on the same path → label both. Example `kinds`: `["HINT_FLIP"]`, `["BREAKING"]`, `["COMPATIBLE"]`, `["HINT_FLIP", "BREAKING"]`.

### 6.3 Output lines

Stable sorted by `path`, then change-kind. Each change:

```
ADDED|REMOVED|CHANGED <path> (HINT_FLIP|BREAKING|COMPATIBLE …)
<old> -> <new> # or + value / - value
```

Paths use dotted JSON-ish form, e.g. `inputSchema.properties.text.type`, `annotations.readOnlyHint`, `outputSchema.properties.id.type`, `arguments["who"].required`.

---

Expand All @@ -420,13 +459,22 @@ treats that surface as **empty** and prints a clear stderr note.

Streamable HTTP / SSE live fetch is not required for lockfile conformance.

## 9. Explicitly out of scope
## 9. Migration (1.3 → 1.4)

- Old lockfiles (v1 / v2 / v3) still verify.
- Digests **change** when a tool had kept annotation keys or `outputSchema`.
- Digests stay **stable** when a tool had neither and only `title` / `icons` / `_meta` changed or were present.
- **Re-lock after upgrading to ≥1.4** so embedded surfaces and digests include annotations / outputSchema.

## 10. Explicitly out of scope

- Signed lockfiles / provenance
- Semantic or embedding “similarity” gates
- Resource templates (`resources/templates/list`) — not pinned in 1.3
- Resource templates (`resources/templates/list`) — not pinned
- `initialize.instructions`
- LLM / fuzzy schema matching
- Materializing MCP annotation defaults into hashes

## 10. Conformance
## 11. Conformance

Golden vectors under `testdata/` are normative for digests. Implementations MUST match those digests for the given inputs. Structured field-diff classification rules in §6 are normative for the reference CLI’s explanatory output.
25 changes: 19 additions & 6 deletions STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,16 +61,29 @@ Exit: 0 match, 1 drift, 2 usage/error.

https://github.com/yellowgram/surfacepin

## Known gaps (intentional)
### v1.4.0

- No Streamable HTTP / SSE live fetch yet (stdio is enough this cut)
- Lockfiles are not signed
- Action stays file-based tools-only (multi-surface / live stdio / schema diff are local/CLI)
- [x] Complete agent-visible tool descriptor: hash `annotations` (drop `title`) + `outputSchema`
- [x] Keep 1.3 coercions for `description` / `inputSchema`; do not materialize MCP annotation defaults
- [x] Diff taxonomy: `COMPATIBLE` | `BREAKING` | `HINT_FLIP` (`kinds` array); pass/fail still digest-only
- [x] SPEC / STATUS / README + golden tests; lockfile write remains v3; algorithm id unchanged
- [x] Action unchanged; no new runtime deps
- [ ] npm publish / git tag — founder-gated

## Known gaps (intentional — document, do not implement this cut)

- `initialize.instructions` not pinned
- Resource templates (`resources/templates/list`) not pinned
- `tool.title` / `tool.icons` / `tool._meta` ignored (decorative)
- `resource.annotations` not hashed
- `prompt.title` / `prompt.icons` / `prompt._meta` ignored
- Action stays tools-file-only (multi-surface / live stdio are CLI/local)
- No Streamable HTTP / SSE live fetch yet (stdio is enough)
- Lockfiles are not signed

## Suggested next

1. Optional Streamable HTTP live fetch (if cheap)
2. Action multi-surface file inputs (if users ask)
1. Action multi-surface file inputs (if users ask) — separate cut
2. Optional Streamable HTTP live fetch (if cheap)
3. Conformance suite packaged for external implementations
4. Optional signed lockfiles (minisign / sigstore) — still exact-hash underneath
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "surfacepin",
"version": "1.3.0",
"description": "Lock exact hashes of MCP tools/resources/prompts list surfaces; fail CI on silent drift. Offline files or live --stdio. Lockfile v3 structured schema field-diff.",
"version": "1.4.0",
"description": "Lock exact hashes of MCP tools/resources/prompts list surfaces; fail CI on silent drift. Offline files or live --stdio. Lockfile v3 structured field-diff (COMPATIBLE|BREAKING|HINT_FLIP).",
"license": "MIT",
"author": "yellowgram",
"type": "module",
Expand Down
2 changes: 1 addition & 1 deletion src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Live --stdio calls tools/list, resources/list, prompts/list as selected.
Servers lacking a capability → empty list + stderr note.

diff prints ADDED/REMOVED/CHANGED ids; for CHANGED tools/resources/prompts with
a v3 lock, also prints path-level schema/field changes (BREAKING / non-breaking).
a v3 lock, also prints path-level field changes (COMPATIBLE / BREAKING / HINT_FLIP).
--json emits the DiffResult object.

Exit codes: 0 match/ok, 1 drift, 2 usage/error
Expand Down
8 changes: 7 additions & 1 deletion src/hash.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,17 @@ export function sha256Hex(data: Buffer | string): string {

/** Per-tool digest over the surface descriptor object. */
export function toolDigest(descriptor: ToolDescriptor): string {
const payload = {
const payload: Record<string, unknown> = {
description: descriptor.description,
inputSchema: descriptor.inputSchema,
name: descriptor.name,
};
if (descriptor.annotations !== undefined) {
payload.annotations = descriptor.annotations;
}
if (descriptor.outputSchema !== undefined) {
payload.outputSchema = descriptor.outputSchema;
}
return sha256Hex(canonicalizeToBytes(payload));
}

Expand Down
2 changes: 2 additions & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,9 @@ export type {
FieldChange,
ChangeSeverity,
ChangeKind,
DiffTaxonomyKind,
} from "./schema-diff.js";
export { normalizeKinds } from "./schema-diff.js";
export { fetchToolsViaStdio, fetchSurfacesViaStdio } from "./mcp-stdio.js";
export type { StdioFetchOptions, ToolsListDoc } from "./mcp-stdio.js";
export type {
Expand Down
5 changes: 4 additions & 1 deletion src/lock.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,14 @@ export interface ComputedSurface {

/** Exact hashed tool payload (matches toolDigest input). */
function toolSurfacePayload(d: ToolDescriptor): ToolDescriptor {
return {
const payload: ToolDescriptor = {
description: d.description,
inputSchema: d.inputSchema,
name: d.name,
};
if (d.annotations !== undefined) payload.annotations = d.annotations;
if (d.outputSchema !== undefined) payload.outputSchema = d.outputSchema;
return payload;
}

/** Exact hashed resource payload (matches resourceDigest input). */
Expand Down
37 changes: 36 additions & 1 deletion src/normalize.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,42 @@ export function normalizeTools(raw: ToolSurface[]): ToolDescriptor[] {
inputSchema = t.inputSchema as Record<string, unknown>;
}

out.push({ name: t.name, description, inputSchema });
// annotations: omit if absent/non-object; drop title; omit if empty after drop.
// Do not materialize MCP defaults into hashed fields.
let annotations: Record<string, unknown> | undefined;
const rawAnn = t.annotations;
if (rawAnn !== undefined && rawAnn !== null) {
if (!isPlainObject(rawAnn)) {
// Non-object annotations → omit key (same as absent)
annotations = undefined;
} else {
const kept: Record<string, unknown> = {};
for (const [k, v] of Object.entries(rawAnn)) {
if (k === "title") continue;
kept[k] = v;
}
if (Object.keys(kept).length > 0) {
annotations = kept;
}
}
}

// outputSchema: absent/null → omit; {} → include; non-object → usage error.
let outputSchema: Record<string, unknown> | undefined;
const rawOut = t.outputSchema;
if (rawOut !== undefined && rawOut !== null) {
if (!isPlainObject(rawOut)) {
throw new SurfacePinError(
`tools[${i}].outputSchema must be an object when present`,
);
}
outputSchema = rawOut as Record<string, unknown>;
}

const desc: ToolDescriptor = { name: t.name, description, inputSchema };
if (annotations !== undefined) desc.annotations = annotations;
if (outputSchema !== undefined) desc.outputSchema = outputSchema;
out.push(desc);
}

out.sort((a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0));
Expand Down
Loading
Loading