Skip to content

Enforce MCP spec compliance by default (0.3.0) - #9

Merged
akriaueno merged 37 commits into
developfrom
feature/p0-spec-compliance
Jun 8, 2026
Merged

akriaueno merged 37 commits into
developfrom
feature/p0-spec-compliance

Conversation

@akriaueno

@akriaueno akriaueno commented Jun 6, 2026

Copy link
Copy Markdown
Member

Summary

This makes the server enforce the MCP 2025-11-25 specification by default. Behaviors that
were previously absent, lenient, or gated behind opt-in options are now always on, and the
server validates the requests the spec requires. The library is pre-1.0, so these breaking
changes replace the earlier opt-in options rather than adding more. The aim is to match the
spec exactly — neither more lenient nor stricter.

Targets 0.3.0.

Behavior changes (breaking)

All of the following are enforced by default, with no opt-out:

Tool-call arguments are validated

A DSL tool's tools/call arguments are validated against its input_schema before the
handler runs; a mismatch is a CallToolResult with isError: true so the model can
self-correct. A tool that declares no schema defaults to an object that accepts no properties
(additionalProperties: false, the spec's recommendation for parameterless tools) — declare an
explicit input_schema to accept arbitrary fields. A non-object arguments value remains a
JSON-RPC invalid_params error. Servers that implement call_tool/3 by hand validate their
own arguments.

Tool input/output schemas must be object schemas

A tool's input_schema and output_schema, when present, must be JSON Schema objects whose
root type is "object" (per the MCP tools spec); the DSL rejects a non-conforming schema
(e.g. type: "array", or a non-map) at compile time, so tools/list cannot advertise a
non-spec schema.

Tool errors are returned as tool results

A tools/call handler's {:error, "message"} (string) is surfaced as a CallToolResult with
isError: true. A protocol-level {:error, %Urchin.Error{}} — or a raised Urchin.Error
stays a JSON-RPC error; any other raised exception becomes an isError result.

The initialization lifecycle is enforced

Operation requests received before the client sends notifications/initialized are rejected
with invalid_request; only ping is allowed pre-init (the lifecycle's pings-and-logging
exception is for the server's own requests, not the client's logging/setLevel). The
notification is committed synchronously, so the next request observes the initialized state.

initialize and serverInfo are validated

initialize requires protocolVersion (string), capabilities (object) and clientInfo
(with a string name and version); a missing or mistyped field is invalid_params. The
server's serverInfo must carry a string name and version.

Duplicate literal tool names are rejected at compile time

Declaring two tools with the same literal name within a server fails compilation (previously
the last declaration silently won); non-literal names (a variable or expression) cannot be
compared statically and are not checked. No tool-name pattern is enforced — the MCP schema
imposes none, though servers should still follow its naming recommendations.

completion/complete is validated and capped at 100 values

Request params are validated per spec — ref as a ref/prompt/ref/resource union,
argument.name/value as strings, context.arguments values as strings — and malformed
params return invalid_params. Results are capped at the top 100 (already ranked) with
hasMore set; a non-conforming result shape (non-string values, etc.) is an internal error.

MCP-Protocol-Version is validated on DELETE

Matching POST and GET.

Other changes

  • logging/setLevel is a library builtin: when the server advertises the logging capability
    it succeeds and updates the session log level even without a set_log_level/2 callback; an
    exported callback still runs as a hook, the level is validated, and the session level changes
    only after the hook succeeds. Servers without the capability return method_not_found.
  • :sse_buffer_limit sizes the per-session GET (SSE) replay buffer used for resumption
    (default nil keeps the built-in default of 100).
  • README clarifies that only the GET SSE stream is replayed on reconnect.
  • Version bumped to 0.3.0.

Removed options

These transport/server options are removed; the behaviors they gated are now always on, and
they were never part of a release: :tool_errors, :validate_arguments, :enforce_initialized,
and validate_tool_names (the latter's pattern check is dropped entirely rather than made
default). Security options (:expose_internal_errors, :auth) are unchanged.

Migration from 0.2.0

  • Complete the lifecycle handshake (notifications/initialized) before sending operation
    requests.
  • Send a spec-conforming initialize (protocolVersion, capabilities, clientInfo).
  • A tools/call string error now arrives as an isError result, not a JSON-RPC error — read
    CallToolResult.isError.
  • Give parameterless DSL tools an explicit input_schema if they must accept extra fields, and
    ensure any input_schema/output_schema is an object schema (type: "object").
  • Return at most 100 completion values (or rely on the automatic cap).

Testing

mix test (run mode), mix format --check-formatted, and mix compile --warnings-as-errors
all pass (228 tests, 0 failures).

akriaueno and others added 6 commits June 6, 2026 09:17
Advertising the logging capability (use Urchin.Server, logging: true) now makes
logging/setLevel succeed and apply the level to the session even when the server
does not export set_log_level/2; an exported set_log_level/2 is still called as a hook.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Surface the session's initialized flag and add an opt-in :enforce_initialized
transport option (default false) that rejects operation requests received before
notifications/initialized with invalid_request; ping and logging/setLevel are
always allowed. Default preserves current behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a :tool_errors transport option (:json_rpc default | :result). With :result,
a tools/call handler's {:error, binary} is returned as an isError CallToolResult
so the model can self-correct, instead of a JSON-RPC internal error. Protocol
errors via {:error, %Urchin.Error{}} always remain JSON-RPC errors. Default
preserves current behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Duplicate tool names within a server are now always rejected at compile time
(a silently shadowed duplicate was previously accepted). Add an opt-in
validate_tool_names: true option to additionally enforce that every literal tool
name matches ~r/^[a-zA-Z0-9_.-]{1,128}$/ (default false).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Expose the per-session GET-stream replay buffer size as a :sse_buffer_limit
transport option, forwarded to Session.start only when set (default nil keeps the
Session default of 100). Validated as a positive integer at startup.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The README bullet implied every SSE stream is resumable; only the GET (general)
stream supports Last-Event-ID replay. POST request streams are not replayable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 6, 2026

Copy link
Copy Markdown

Need an answer fast? Review this PR in Change Stack to ask focused questions about the PR or a changed range.

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Urchin adds Context fields for initialization and tool-error handling, enforces optional pre-initialization request gating, shapes tool failures into result objects when configured, enhances logging/setLevel to update session state and call optional hooks, validates tool names at compile time (duplicates always rejected, pattern opt-in), forwards SSE buffer limits into sessions, and adds tests and docs.

Changes

Request Lifecycle and Tool Handling Enhancements

Layer / File(s) Summary
Context and session API updates
lib/urchin/context.ex, lib/urchin/session.ex
Urchin.Context adds initialized, enforce_initialized, and tool_errors; Session.mark_initialized/1 added, set_log_level/2 is synchronous, and Session.snapshot/1 includes initialized.
Request initialization gating
lib/urchin/dispatcher.ex, lib/urchin/transport/streamable_http.ex
Dispatcher calls check_initialized/2 to reject operation requests before notifications/initialized when enforce_initialized is true; transport handles notifications/initialized by calling Session.mark_initialized/1.
Tool error result shaping
lib/urchin/dispatcher.ex, test/support/echo_server.ex
When Context.tool_errors == :result, {:error, binary} tool failures are converted into tool result objects with isError: true; protocol errors remain JSON-RPC errors.
Enhanced logging/setLevel handler
lib/urchin/dispatcher.ex, test/urchin/dispatcher_test.exs
logging/setLevel requires advertised logging, validates level, optionally invokes exported set_log_level/2, updates session min_log_level via set_session_log_level/2, and returns appropriate JSON-RPC errors when invalid or not advertised.
Compile-time tool name validation
lib/urchin/server.ex, test/urchin/server_test.exs
Adds @tool_name_pattern and compile-time validators in __before_compile__/1 to reject duplicate tool names and (with validate_tool_names: true) enforce literal-name regex checks; tests verify opt-in behavior.
SSE replay buffer limit forwarding
lib/urchin/transport/streamable_http.ex, lib/urchin/session.ex, test/urchin/session_lifecycle_test.exs
Parses :sse_buffer_limit, validates it, conditionally forwards :buffer_limit to Session.start/1 when set, and tests confirm replay capping behavior.
Transport option parsing & context wiring
lib/urchin/transport/streamable_http.ex
Adds and validates :enforce_initialized, :tool_errors, and :sse_buffer_limit options; per-request Context includes initialized, enforce_initialized, and tool_errors.
Test fixtures and feature tests
test/support/echo_server.ex, test/urchin/dispatcher_test.exs, test/urchin/transport/streamable_http_test.exs
Adds failing/protocol_error echo tools, inline test servers for logging hooks, and tests for logging/setLevel semantics, initialization gating, tool_errors modes, and sse_buffer_limit validation/forwarding.
Documentation and changelog
CHANGELOG.md, README.md
CHANGELOG documents Added/Changed/Fixed entries for the new options and behaviors; README clarifies that resumable SSE resumption applies to the GET SSE stream.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • urth-inc/urchin#4: Both PRs modify lib/urchin/dispatcher.ex’s core request/error flow—especially handle_request/4 and normalize_error/2—to change how tool/handler failures are shaped or redacted based on Urchin.Context options.

Poem

🐰 I hopped through code at break of dawn,

I checked the gates and nudged sessions awake,
I coaxed tool-errors into tidy guise,
I capped the stream so replays stay small,
And signed the changelog with a tiny hop.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 62.50% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title "Enforce MCP spec compliance by default (0.3.0)" is partially related to the changeset. It mentions spec compliance and version, but the PR objectives indicate changes are opt-in and non-breaking (not enforced by default), and multiple spec-compliance items (C1-C10) are addressed—not just defaults. Clarify the title to more accurately reflect that this PR implements multiple opt-in, non-breaking spec-compliance fixes rather than enforcement-by-default changes.
✅ Passed checks (3 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@CHANGELOG.md`:
- Around line 45-46: Changelog incorrectly states the transport option
:sse_buffer_limit has a default of 100 while the implementation sets it to nil
(so it falls back to Urchin.Session's internal default of 100); update the
CHANGELOG entry for :sse_buffer_limit to state the transport option's default is
nil (which preserves Urchin.Session’s internal default of 100) or reword to
clarify that leaving :sse_buffer_limit unset preserves Urchin.Session’s default
buffer size of 100 so there is no config-contract confusion.

In `@lib/urchin/transport/streamable_http.ex`:
- Around line 45-46: The docs incorrectly state that :sse_buffer_limit defaults
to 100; update the documentation text to reflect that the transport option
defaults to nil (so init/1 and maybe_put_buffer_limit/2 can preserve the
session’s internal default), and clarify that a positive integer overrides the
session default while nil leaves the session's own fallback (e.g., 100) intact;
reference the :sse_buffer_limit option, the init/1 function, and
maybe_put_buffer_limit/2 in the change.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: dc1c966a-0202-4e71-8f6b-9ec95eb8d770

📥 Commits

Reviewing files that changed from the base of the PR and between 7f11234 and cd93951.

📒 Files selected for processing (12)
  • CHANGELOG.md
  • README.md
  • lib/urchin/context.ex
  • lib/urchin/dispatcher.ex
  • lib/urchin/server.ex
  • lib/urchin/session.ex
  • lib/urchin/transport/streamable_http.ex
  • test/support/echo_server.ex
  • test/urchin/dispatcher_test.exs
  • test/urchin/server_test.exs
  • test/urchin/session_lifecycle_test.exs
  • test/urchin/transport/streamable_http_test.exs

Comment thread CHANGELOG.md Outdated
Comment thread lib/urchin/transport/streamable_http.ex Outdated
akriaueno and others added 3 commits June 6, 2026 09:44
The transport option defaults to nil; an unset value preserves Urchin.Session's
internal default buffer of 100. The previous wording (default 100) conflated the
transport option default with the effective session fallback.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses review of the P0 PR:
- Validate the requested log level against the MCP levels; reject unknown values
  with invalid_params (-32602) instead of storing them on the session.
- Offer logging/setLevel only when the server advertises the logging capability;
  otherwise return method_not_found (-32601).
- Update the session level only after the optional set_log_level/2 hook succeeds,
  so a failed hook leaves no state change.
- Allow only ping before notifications/initialized (client-side logging/setLevel is
  no longer pre-init exempt); fix the related comment and docs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an integration test that initializes a session through the transport with
sse_buffer_limit: 1 and confirms the replay buffer is capped, proving the option
is forwarded from init/1 to Session.start.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
test/urchin/transport/streamable_http_test.exs (1)

381-412: ⚡ Quick win

Wrap session cleanup in on_exit for reliability.

The test calls Session.terminate(pid) on line 411, but if any assertion fails before that line (e.g., line 409), the session will not be terminated and may leak resources.

Other tests in dispatcher_test.exs consistently use on_exit(fn -> Session.terminate(pid) end) immediately after session creation to ensure cleanup even when assertions fail (see lines 319, 348, 378 in the dispatcher test file).

♻️ Proposed fix to add on_exit cleanup
       assert conn.status == 200
       [session_id] = get_resp_header(conn, "mcp-session-id")
       pid = Urchin.Session.whereis(session_id)
+      on_exit(fn -> Session.terminate(pid) end)
 
       Urchin.Session.notify(pid, "notifications/message", %{"n" => 1})
       Urchin.Session.notify(pid, "notifications/message", %{"n" => 2})
 
       # With the buffer capped at 1, only the most recent event is available to replay.
       {:ok, "g0", replay} = Urchin.Session.register_general_stream(pid, self(), {"g0", 0})
       assert length(replay) == 1
-
-      Urchin.Session.terminate(pid)
     end
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/urchin/transport/streamable_http_test.exs` around lines 381 - 412, After
creating the session (captured as pid from Urchin.Session.whereis), ensure the
session is always cleaned up by registering an on_exit callback that calls
Urchin.Session.terminate(pid) immediately after pid is obtained; replace the
lone Urchin.Session.terminate(pid) call at the end of the test with an
on_exit(fn -> Urchin.Session.terminate(pid) end) so the session is terminated
even if assertions around Urchin.Session.notify or
Urchin.Session.register_general_stream fail.
test/urchin/dispatcher_test.exs (1)

346-360: ⚡ Quick win

Capture the initial state before asserting it's unchanged.

The assertion on line 359 only verifies that min_log_level is not set to "verbose", but doesn't confirm the session remains at its original value. A valid level other than "verbose" could theoretically be set, and the test would still pass.

Compare this to the test at lines 374-390, which correctly captures the initial state (before = Session.snapshot(pid).min_log_level) and then asserts == before.

♻️ Proposed fix to capture initial state
     test "rejects an invalid level with -32602 and leaves the session unchanged" do
       {:ok, _id, pid} = Session.start(server: EchoServer, protocol_version: "2025-11-25")
       on_exit(fn -> Session.terminate(pid) end)
+      before = Session.snapshot(pid).min_log_level
 
       assert {:error, error} =
                Dispatcher.handle_request(
                  EchoServer,
                  "logging/setLevel",
                  %{"level" => "verbose"},
                  %Context{session: pid}
                )
 
       assert error.code == -32_602
-      assert Session.snapshot(pid).min_log_level != "verbose"
+      assert Session.snapshot(pid).min_log_level == before
     end
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/urchin/dispatcher_test.exs` around lines 346 - 360, The test should
capture the session's initial min_log_level before calling
Dispatcher.handle_request and then assert it remains unchanged; modify the test
by assigning before = Session.snapshot(pid).min_log_level prior to calling
Dispatcher.handle_request (the call to Dispatcher.handle_request with
EchoServer, "logging/setLevel", %{"level" => "verbose"}, %Context{session: pid})
and replace the final assertion assert Session.snapshot(pid).min_log_level !=
"verbose" with assert Session.snapshot(pid).min_log_level == before to ensure
the session state is preserved.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/urchin/dispatcher_test.exs`:
- Around line 346-360: The test should capture the session's initial
min_log_level before calling Dispatcher.handle_request and then assert it
remains unchanged; modify the test by assigning before =
Session.snapshot(pid).min_log_level prior to calling Dispatcher.handle_request
(the call to Dispatcher.handle_request with EchoServer, "logging/setLevel",
%{"level" => "verbose"}, %Context{session: pid}) and replace the final assertion
assert Session.snapshot(pid).min_log_level != "verbose" with assert
Session.snapshot(pid).min_log_level == before to ensure the session state is
preserved.

In `@test/urchin/transport/streamable_http_test.exs`:
- Around line 381-412: After creating the session (captured as pid from
Urchin.Session.whereis), ensure the session is always cleaned up by registering
an on_exit callback that calls Urchin.Session.terminate(pid) immediately after
pid is obtained; replace the lone Urchin.Session.terminate(pid) call at the end
of the test with an on_exit(fn -> Urchin.Session.terminate(pid) end) so the
session is terminated even if assertions around Urchin.Session.notify or
Urchin.Session.register_general_stream fail.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: fe9cb462-92c0-4357-a5c2-8195d5ef99ba

📥 Commits

Reviewing files that changed from the base of the PR and between 326b5c2 and 0d30892.

📒 Files selected for processing (6)
  • CHANGELOG.md
  • lib/urchin/context.ex
  • lib/urchin/dispatcher.ex
  • lib/urchin/transport/streamable_http.ex
  • test/urchin/dispatcher_test.exs
  • test/urchin/transport/streamable_http_test.exs
✅ Files skipped from review due to trivial changes (1)
  • CHANGELOG.md
🚧 Files skipped from review as they are similar to previous changes (3)
  • lib/urchin/context.ex
  • lib/urchin/transport/streamable_http.ex
  • lib/urchin/dispatcher.ex

akriaueno and others added 4 commits June 6, 2026 10:56
…onously

Make notifications/initialized a synchronous Session.mark_initialized/1 call and
turn Session.set_log_level/2 into a call, so once the client has the 202 ack the
next request always observes the committed state in the snapshot. Removes any
reliance on mailbox-ordering reasoning for the enforce_initialized gate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A hand-written capabilities/0 may return JSON-shaped string keys; logging_advertised?
now checks both :logging and "logging".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Compile-time duplicate detection only inspects DSL declarations; reword the
moduledoc and CHANGELOG so it does not imply hand-written list_tools/2 is covered.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… nil

Session.start(buffer_limit: nil) previously stored nil, crashing push_general via
Enum.take(_, nil). The transport already guarded against this; this hardens the
public Session API itself.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@akriaueno akriaueno changed the title P0 spec-compliance fixes (C1-C10) — non-breaking, opt-in P0 spec-compliance fixes (C1-C10) Jun 6, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/urchin/session_lifecycle_test.exs`:
- Around line 89-95: The current test only sends one notification so it can’t
distinguish between a nil fallback and an accidental effective limit of 1;
change the test to send multiple notifications (e.g., 3) using Session.notify,
read the configured default buffer size via Session.default_buffer_limit()/0 (or
the actual module function that returns the default) and then call
Session.register_general_stream to get replay and assert that length(replay) ==
min(Session.default_buffer_limit(), 3) (and that Session.default_buffer_limit()
> 1) so the test fails if nil does not correctly fall back to the real default.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: e5f43076-4cd1-4561-adaa-25c70de0d5b5

📥 Commits

Reviewing files that changed from the base of the PR and between 453320a and a0c393d.

📒 Files selected for processing (2)
  • lib/urchin/session.ex
  • test/urchin/session_lifecycle_test.exs
🚧 Files skipped from review as they are similar to previous changes (1)
  • lib/urchin/session.ex

Comment on lines +89 to +95
test "buffer_limit: nil falls back to the default replay buffer" do
{:ok, _id, pid} = start(buffer_limit: nil)
Session.notify(pid, "notifications/message", %{"n" => 1})

{:ok, "g0", replay} = Session.register_general_stream(pid, self(), {"g0", 0})
assert length(replay) == 1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Test does not actually verify default-buffer fallback semantics.

With only one notified message, length(replay) == 1 passes for both correct fallback-to-default and incorrect effective limit 1, so this test can’t detect the key regression it claims to cover.

Suggested tightening
 test "buffer_limit: nil falls back to the default replay buffer" do
   {:ok, _id, pid} = start(buffer_limit: nil)
-  Session.notify(pid, "notifications/message", %{"n" => 1})
+  Session.notify(pid, "notifications/message", %{"n" => 1})
+  Session.notify(pid, "notifications/message", %{"n" => 2})

   {:ok, "g0", replay} = Session.register_general_stream(pid, self(), {"g0", 0})
-  assert length(replay) == 1
+  assert length(replay) == 2
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
test "buffer_limit: nil falls back to the default replay buffer" do
{:ok, _id, pid} = start(buffer_limit: nil)
Session.notify(pid, "notifications/message", %{"n" => 1})
{:ok, "g0", replay} = Session.register_general_stream(pid, self(), {"g0", 0})
assert length(replay) == 1
test "buffer_limit: nil falls back to the default replay buffer" do
{:ok, _id, pid} = start(buffer_limit: nil)
Session.notify(pid, "notifications/message", %{"n" => 1})
Session.notify(pid, "notifications/message", %{"n" => 2})
{:ok, "g0", replay} = Session.register_general_stream(pid, self(), {"g0", 0})
assert length(replay) == 2
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/urchin/session_lifecycle_test.exs` around lines 89 - 95, The current
test only sends one notification so it can’t distinguish between a nil fallback
and an accidental effective limit of 1; change the test to send multiple
notifications (e.g., 3) using Session.notify, read the configured default buffer
size via Session.default_buffer_limit()/0 (or the actual module function that
returns the default) and then call Session.register_general_stream to get replay
and assert that length(replay) == min(Session.default_buffer_limit(), 3) (and
that Session.default_buffer_limit() > 1) so the test fails if nil does not
correctly fall back to the real default.

akriaueno and others added 5 commits June 6, 2026 13:46
Per review: a tools/call handler's {:error, binary} is now returned as an isError
CallToolResult by default (the MCP-spec behavior) so models can self-correct. Set
tool_errors: :json_rpc for the legacy JSON-RPC-error behavior. Protocol errors via
{:error, %Urchin.Error{}} still surface as JSON-RPC errors. This changes the prior
default where such a handler error became a JSON-RPC error.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
^...$ matches before a trailing newline, so a tool name like "abc\n" passed
validation; \A...\z requires a full-string match. Adds a regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
set_session_log_level now catches the GenServer.call exit from a dead session and
returns invalid_request instead of crashing the handler with a generic error.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… wiring

Integration tests through StreamableHTTP.init/1 -> Context -> Dispatcher: default
tool_errors :result returns an isError result, logging/setLevel updates the session
min_log_level, and client notifications are accepted (202) before initialized.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…replay

Sync the tool-name pattern wording to \A...\z, note that enforce_initialized: true
is required for strict MCP lifecycle compliance, and clarify that only the GET general
stream is replayed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@akriaueno akriaueno changed the title P0 spec-compliance fixes (C1-C10) Spec-compliance fixes: tool errors, lifecycle gating, logging and tool-name validation Jun 6, 2026
akriaueno and others added 6 commits June 6, 2026 16:05
The :tool_errors default is :result, but several docs still described the
legacy :json_rpc behavior or omitted the new transport options:

- server.ex moduledoc: for tools/call an {:error, binary} (and a raised
  exception) is an isError CallToolResult by default, not a JSON-RPC error;
  only {:error, %Urchin.Error{}} and the other callbacks stay JSON-RPC errors.
- streamable_http.ex: fix the tool_errors_opt!/1 comment that still claimed
  the default is :json_rpc.
- README: add the :enforce_initialized, :tool_errors and :sse_buffer_limit
  rows to the options table, and make the manual scope-check example return an
  Urchin.Error so it stays a JSON-RPC error like the declarative scopes path.
- CHANGELOG / SECURITY: separate error redaction from the tools/call response
  envelope, which is governed by :tool_errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add transport-level coverage for tool_errors: a :json_rpc handler error
surfaces as a JSON-RPC error end-to-end, and StreamableHTTP.init validates the
option (:result default, :json_rpc accepted, any other value raises).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nitialized

This PR made notifications/initialized commit synchronously via a GenServer.call so
the next request observes initialized: true. lookup_session/2 only proves the session
was alive a moment earlier, so a session that terminates in between would make the
call exit and crash the Plug process. Catch the exit and return a clean 404 "Session
not found", mirroring lookup_session/2 and the existing logging/setLevel guard.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rors

Per the MCP tool error semantics, input validation is a tool-execution error, not a
protocol error. With tool_errors: :result (the default) a tools/call input_schema
mismatch is now an isError CallToolResult so the model can self-correct; under
:json_rpc it stays a JSON-RPC invalid_params error (the legacy behavior). Malformed
requests (missing/invalid name, non-object params), unknown tools, and scope denials
remain JSON-RPC errors.

Also fix the Urchin.Server moduledoc, which wrongly implied :tool_errors governs
raised tool exceptions: a raising tool is always reported as an isError CallToolResult,
regardless of :tool_errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…error

A non-object CallToolRequestParams.arguments violates the request shape, so
it must be a JSON-RPC invalid_params error regardless of :tool_errors, not an
isError tool result. Input-schema validation now only runs on object arguments.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the previously opt-in spec-compliance behaviors the default, with no
opt-out, and validate the requests the spec requires:

- Validate tools/call arguments against each tool's input schema; a mismatch is
  an isError CallToolResult. Parameterless tools default to a closed object
  schema (additionalProperties: false).
- Reject operation requests received before notifications/initialized; only ping
  and logging/setLevel are allowed before initialization.
- Surface a tools/call handler's {:error, binary} as an isError CallToolResult;
  protocol errors stay JSON-RPC errors.
- Validate literal tool names against the spec pattern at compile time.
- Require protocolVersion, capabilities and clientInfo on initialize, and a
  serverInfo carrying a string name and version.
- Validate the MCP-Protocol-Version header on DELETE, matching POST and GET.
- Cap completion/complete results at 100 values, setting hasMore when truncated.

Removes the :tool_errors, :validate_arguments and :enforce_initialized transport
options and the validate_tool_names server option; the behaviors they gated are
now always on. These options were never part of a release.

BREAKING CHANGE: spec-compliance behaviors are always on and several previously
lenient defaults now reject non-conforming input.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
akriaueno and others added 3 commits June 7, 2026 01:09
Rewrite the unreleased CHANGELOG entries to describe the now always-on
behaviors, drop the removed transport/server options from the README options
table and SECURITY notes, and bump the version to 0.3.0 (a breaking change
under pre-1.0 semantic versioning).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…/call

call_tool_result/4 rescued every exception into an isError CallToolResult, so a
handler raising Urchin.Error became a tool result while a returned
{:error, %Urchin.Error{}} became a JSON-RPC error — an asymmetry that also
contradicted the Urchin.Error docs. Rescue Urchin.Error first and surface it as
a JSON-RPC error; any other exception still becomes an isError result.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e convention

Automatic input-schema validation lives in the DSL-generated call_tool/3, so a
hand-written Urchin.Server validates its own arguments; say so in the README,
SECURITY notes, transport moduledoc and CHANGELOG. Also note that the tool-name
pattern is a tool-calling-SDK convention Urchin enforces, not an MCP-mandated
pattern.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@akriaueno akriaueno changed the title Spec-compliance fixes: tool errors, lifecycle gating, logging and tool-name validation Enforce MCP spec compliance by default (0.3.0) Jun 6, 2026
akriaueno and others added 10 commits June 8, 2026 10:56
The MCP schema imposes no pattern on tool names, so rejecting non-conforming
literal names at compile time was stricter than the spec required. Drop the
pattern check entirely; only duplicate names within a server are still rejected,
which prevents an unambiguous bug rather than enforcing a style.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
notifications/initialized was committed synchronously by the transport through
mark_initialized/1 and also set initialized: true in the async handle_client
notification path. The transport never routes the notification to the async
path, so that clause was dead and risked drifting from the synchronous one.
Drop it and let mark_initialized/1 be the single source.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Urchin enforces no tool-name pattern, but the MCP spec still recommends a
conservative charset, a length bound and no whitespace. Say servers should
follow those recommendations rather than implying names are unconstrained.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
logging/setLevel is allowed before initialization, so a direct test with
initialized: false read ambiguously. Set initialized: true on the normal-path
logging tests; the pre-init case is covered explicitly in the gating describe.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A tool that raises Urchin.Error is surfaced as a JSON-RPC error, like a returned
{:error, %Urchin.Error{}}; only other exceptions become an isError result. The
moduledoc read as if any raise became isError.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Only ping is exempt from the lifecycle gate. The MCP lifecycle's
pings-and-logging exception covers the server's own requests/notifications, not
the client's logging/setLevel, so a pre-init logging/setLevel is now rejected
with invalid_request like any other operation request.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… wording

Duplicate-name rejection only compares literal names (non-literal names cannot
be compared at compile time), so say "duplicate literal tool names". Also
correct the lifecycle wording so only ping is described as exempt before
initialization.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Previously only ref and argument were checked for being maps. Validate the
CompleteRequestParams shape per spec: ref as a ref/prompt (name) or ref/resource
(uri) union, argument.name/value as required strings, and context.arguments
values as strings, returning invalid_params when malformed. The result is also
checked (values as strings, total a number, hasMore a boolean); a non-conforming
result is an internal error rather than a shipped, non-spec response.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tool.new/1 kept input_schema/output_schema as given, so a non-object schema
(e.g. type: array, or a non-map) could be advertised in tools/list and skew the
now-always-on argument validation. Validate that each, when present, is a JSON
Schema object with root type "object", per the MCP tools spec; the DSL rejects a
non-conforming schema at compile time.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@akriaueno
akriaueno merged commit 886a724 into develop Jun 8, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant