feat(sep-2640): skills server conformance — index, manifest, directory scenarios - #330
Open
panyam wants to merge 11 commits into
Open
feat(sep-2640): skills server conformance — index, manifest, directory scenarios#330panyam wants to merge 11 commits into
panyam wants to merge 11 commits into
Conversation
…rce-template) Re-extract sep-2640.yaml against current SEP-2640 HEAD. The SEP removed the `mcp-resource-template` index entry type in two commits on 2026-06-04 (fd50cc91 "Remove mcp-resource-template entries from skill index", 556154c0 "Remove remaining resource template mentions from SDK and rationale sections"), after this extraction was first captured on 2026-06-03 at b77fdfe8. Provenance comment bumped accordingly. Resulting changes: - sep-2640-index-entry-type-enum: drop `"mcp-resource-template"` from the enum. - sep-2640-index-name-required: drop the "is Required for ... Omitted for mcp-resource-template" conditional language; the column is now unconditional Required=Yes at SEP HEAD. - sep-2640-index-digest-required: same simplification; also align "hexadecimal" -> "hex" with the HEAD table text. - sep-2640-template-resource-template-registered: removed. The SEP no longer defines a template entry type, so the SHOULD on registering an MCP resource template is gone. - Excluded "Hosts SHOULD surface template entries..." excerpt: removed. The sentence no longer exists in the SEP. Sweep verified: zero "template" mentions in the SEP at HEAD. Security Implications section sentences referenced by the remaining excluded entries are intact. No other check rows touched.
…y/read
Adds the conformance scenario for the SEP-2640 directoryRead surface that
landed in spec commit 2e04c48d (2026-06-09). Per AGENTS.md "fewer scenarios,
more checks", a single ResourcesDirectoryReadScenario emits 6
ConformanceChecks, one per new sep-2640.yaml requirement row.
Class named for the wire method (resources/directory/read), matching the
existing ResourcesListScenario / ResourcesReadTextScenario / etc. family in
src/scenarios/server/resources.ts. The runner-facing name field stays as
'sep-2640-skills' (umbrella) so mcpkit's conformance/Makefile entry
--scenario sep-2640-skills keeps working without a cross-repo race.
src/seps/sep-2640.yaml
- File-level provenance held at 556154c (the existing PR 330 baseline).
- 6 new check rows appended for the directoryRead additions, each
carrying a verbatim, grep-F-searchable excerpt from the SEP at
2e04c48da90224000e750ffd54a3611f2824fbc0:
- sep-2640-capability-directory-read-flag
- sep-2640-directory-read-method-registered
- sep-2640-directory-read-subdir-mimetype
- sep-2640-directory-read-result-resources-shape
- sep-2640-directory-read-invalid-params
- sep-2640-directory-read-pagination
- forward_reference header comment notes the asymmetry: PR 97 schema
rewrite (360123d0, 2026-06-08) made 3 existing rows stale and drifted
~11 others' wording; full re-extraction at SEP HEAD is mcpkit#780's
lifecycle. Provenance deliberately holds at 556154c until that lands.
src/types.ts
- Adds io.modelcontextprotocol/skills to EXTENSION_IDS so the scenario's
source: { extensionId: ... } tag type-checks.
src/scenarios/server/directory.ts (new)
- Capability discovery via wire-observable signal: -32601 method-not-found
is the only definitive "server did not declare directoryRead" signal.
- Fixture assumption: server exposes skill://acme/billing/refunds/templates
with at least one subdirectory child (mcpkit examples/skills layout).
No skill:// resources at all -> every check emits SKIPPED so the
scenario stays green against upstream's everything-server fixture.
- 6 checks (1:1 with the YAML rows above):
1. directoryRead declared (derived from method registration)
2. method registered (happy-path call succeeds)
3. result.resources shape matches resources/list
4. subdir child carries mimeType: "inode/directory"
5. non-directory URI returns -32602
6. nextCursor round-trips (single-page is conformant)
src/scenarios/index.ts
- Imports + registers ResourcesDirectoryReadScenario in
allClientScenariosList, matching the registration name 'sep-2640-skills'
that mcpkit's conformance/Makefile already passes to --scenario.
Verified end-to-end against mcpkit examples/skills:
cd ~/newstack/mcpkit/main && \
MCPCONFORMANCE_SKILLS_PATH=~/newstack/mcpkit/conf-skills \
make -C conformance testconf-skills
-> 6/6 SUCCESS, 0 failed, 0 warnings.
Out of scope:
- PR 97 schema rewrite YAML refresh (separate ticket, mcpkit#780 lifecycle).
- Negative-capability fixture (server without directoryRead) - mcpkit
examples/skills has no flag for it; ext/skills/client_directory_test.go
already covers the SDK pre-call guard.
feat(sep-2640): ResourcesDirectoryReadScenario for resources/directory/read (mcpkit#784)
panyam
marked this pull request as ready for review
June 16, 2026 21:34
panyam
marked this pull request as draft
June 16, 2026 21:34
# Conflicts: # src/types.ts
commit: |
# Conflicts: # src/types.ts
…s, drift fixes
Builds on the directory-read scenario with the server-observable skills surface
and brings the whole set up to the current framework conventions.
New scenarios (src/scenarios/server/skills/):
- SkillsIndexScenario (sep-2640-skills-index): skill://index.json is exposed and
readable; every skills[].type is in the SEP enum; every entry has a non-empty
name; a present digest matches sha256:{64hex}; entry URLs use skill://.
- SkillsManifestScenario (sep-2640-skills-manifest): a skill-md SKILL.md is
markdown-typed; frontmatter name/description surface on the resource; the
final skill-path segment equals the frontmatter name; skill _meta keys use the
reverse-domain prefix.
directory.ts (moved into skills/): capability is now read from server/discover
instead of inferred from -32601; an undeclared optional capability SKIPs rather
than FAILs; the untestable.ts helpers back the not-exercisable paths; all
mcpkit-specific fixture URIs are gone — the directory to exercise is discovered
dynamically, so the scenario is brand-neutral.
Capability gating accepts directoryRead at either extensions[id].directoryRead
(SEP-2640's inline example) or extensions[id].config.directoryRead (the SEP-2133
negotiation envelope SEP-2640 defers to); the two SEPs are inconsistent on
nesting, so the check accepts either rather than privileging one reading.
All three scenarios verified green against a live SEP-2640 server (6/6, 6/6, 7/7);
typecheck, eslint, prettier, and the 504-test vitest suite pass. sep-2640.yaml
gains a header note mapping each scenario to the rows it now backs.
panyam
changed the title
panyam
marked this pull request as ready for review
August 4, 2026 04:38
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
src/seps/sep-2640.yaml(the Skills Extension requirement-traceability file) and the first runnable server scenarios for it. ThreeClientScenarios undersrc/scenarios/server/skills/exercise the server-observable SEP-2640 surface, discovered dynamically so they pass against any conformant server rather than a specific fixture.sep-2640-skills-index(index.ts) -skill://index.jsonis exposed and readable; everyskills[].typeis in the enum (skill-md|archive); every entry has a non-emptyname; a presentdigestissha256:{64 hex}; entry URLs useskill://.sep-2640-skills-manifest(manifest.ts) - a discoveredskill-mdSKILL.mdis markdown-typed; frontmattername/descriptionsurface on the resource; the final skill-path segment equals the frontmattername; skill_metakeys use theio.modelcontextprotocol.skills/prefix.sep-2640-skills-directory(directory.ts, moved intoskills/) - theresources/directory/readscenario, now reading the capability fromserver/discoverinstead of inferring it from a-32601, SKIPping whendirectoryReadis undeclared, using the shared untestable helpers, and discovering the directory to exercise dynamically (no hardcoded fixture URIs).Motivation and Context
The traceability YAML floated the requirement model for WG review ahead of scenario design. With that model settled, this PR lands the scenarios that assert the server-observable half of it. Discovery is dynamic and brand-neutral throughout, so the scenarios certify any SEP-2640 server, not one implementation's fixture.
The host-side security MUSTs (digest verification, resource-fetch bound, cross-origin scheme rejection) are client decisions and not server-observable, so they stay traceability-only here; they belong in a separate client scenario (a proposal is in preparation).
How Has This Been Tested?
sep-2640-skills-index6/6,sep-2640-skills-manifest6/6,sep-2640-skills-directory7/7 (all SUCCESS).npm run check(typecheck + eslint + prettier) passes; the vitest suite (504 tests) passes; lefthook pre-push (Test,Code Formatting) passes.Breaking Changes
None. Additive scenarios plus a within-repo file move (
src/scenarios/server/directory.ts→src/scenarios/server/skills/directory.ts, git-tracked rename; the class is renamedResourcesDirectoryReadScenario→SkillsDirectoryReadScenario). All three scenarios are registered in both the active and pending lists and SKIP cleanly against a server that does not declare the skills extension.Types of changes
Checklist
Additional context
A spec inconsistency to flag for the WG. The capability gate accepts
directoryRead: trueat either nesting, because two SEPs disagree on where it lives:capabilities.extensions["io.modelcontextprotocol/skills"].directoryRead.{ specVersion, stability, config }envelope:capabilities.extensions["io.modelcontextprotocol/skills"].config.directoryRead.Implementations already differ (a live SEP-2640 server tested here uses the SEP-2133 envelope). Rather than privilege one reading, the check accepts both; it collapses to the single location once the SEPs agree. Which nesting is canonical?
The
sep-2640.yamlheader carries a note mapping each scenario to the rows it now backs, and confirming which rows stay traceability-only (host-internal / off-wire: host load-by-uri, digest verification, byte-budget / archive-unpack safety, no-empty-index assumption).