Repository navigation
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
betegon
force-pushed
the
bt/code-mode-http
branch
from
October 27, 2026 13:28
2fbe908 to
c74cfe6
Compare
betegon
force-pushed
the
bt/code-mode-http-evals
branch
from
October 27, 2026 13:28
8d7e92d to
954c0cd
Compare
betegon
added a commit
that referenced
this pull request
Oct 27, 2026
The Cloudflare test plugin still used an older Miniflare/workerd than Wrangler, so updating the direct dependency alone left different runtimes in local and integration tests. Switch to `@cloudflare/vitest-plugin` 1.4.6 and align Miniflare/workerd with Wrangler. Miniflare 5 removed `fetchMock`; an Undici outbound service preserves the existing fixtures and `disableNetConnect()` policy. This is the prerequisite for #1670 (HTTP Code Mode), followed by #1675 (agent evals). Only seven test/dependency files move here; the broker, permissions and production integration remain in #1670. Validated independently on Node 24: **443 Cloudflare tests across 34 files**, root typechecks, lint and frozen install. The earlier full workspace run had unrelated CLI fixture/platform failures; CLI fixes are excluded. No deployment performed.
betegon
force-pushed
the
bt/code-mode-http-evals
branch
from
October 27, 2026 13:44
954c0cd to
d518be2
Compare
betegon
force-pushed
the
bt/code-mode-http
branch
from
October 27, 2026 13:44
c74cfe6 to
9be4926
Compare
betegon
force-pushed
the
bt/code-mode-http-evals
branch
from
October 27, 2026 17:11
d518be2 to
6399954
Compare
betegon
force-pushed
the
bt/code-mode-http
branch
from
October 30, 2026 16:07
645c16d to
4e625e9
Compare
betegon
force-pushed
the
bt/code-mode-http-evals
branch
from
October 30, 2026 17:54
6399954 to
c9a0202
Compare
# Conflicts: # packages/mcp-cloudflare/src/code-mode/tsconfig.json
The eval worker now makes the same buildServer() and registerCodeModeTools() calls as the hosted handler, so ApiAuthorization applies: only operations in the shipped schema pass, regional lookups run and API Read/Write come from the connection's skills. Each surface is a connection grant on one worker: `/mcp` (catalog + API Write), `/catalog/mcp` (catalog) and `/code-mode/mcp` (API Write only). Fixtures answer the organization lookup the policy makes before organization-scoped calls, without a region URL, so requests stay on the loopback fixture instead of failing that lookup on every call. This removes the eval-only OPENAPI_URL and CODE_MODE_ALLOW_WRITES settings, the dashboard fixture's private OpenAPI subset and the standalone worker entrypoint. The worker test moves next to the evals and runs on Wrangler, and reports hash the files that now define Code Mode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017M6odW3kt5i65zATxfz9RX
All tasks pass on the hosted registration and ApiAuthorization with the shipped @sentry/api 0.428.0 schema (35/35 runs with claude-opus-5-5); the catalog-only control fails as intended. No run reached the operation policy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017M6odW3kt5i65zATxfz9RX
The eval worker now bounds request bodies with main's shared boundMcpRequestBody() and MAX_MCP_REQUEST_BODY_BYTES, like the hosted handler, since the Code Mode limiter and constants it used are gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017M6odW3kt5i65zATxfz9RX
CI lints before building mcp-core, so the tool definitions' type is unresolved there and the type-aware rule requires a comparator. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017M6odW3kt5i65zATxfz9RX
Brings in the Code Mode cleanup and the latest main. The eval worker drops the removed `configuredUrl` and `sdkVersion` and uses `isCodeModeSkill()`; the tool-list test leaves Code Mode out of the catalog names now that `toolDefinitions.json` includes it. The runner declares the MCP SDK client it imports, which main no longer hoists through mcp-core. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017M6odW3kt5i65zATxfz9RX
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017M6odW3kt5i65zATxfz9RX
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017M6odW3kt5i65zATxfz9RX
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017M6odW3kt5i65zATxfz9RX
This branch is pending to be deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Prerequisite #1676 is merged. Stack: #1670 (HTTP Code Mode) → this PR. Merge #1670 before retargeting this PR to
main.Tests whether agents discover HTTP Code Mode themselves and complete API tasks missing from the normal MCP catalog. Real Claude/Codex clients use local Dynamic Workers and stateful API fixtures; independent verifiers check the resulting state, including collateral writes, rather than trusting the agent's answer.
The harness covers dashboard copying, releases/deploys, repository pagination, alerts, monitors, partial-write recovery and permission failures. A catalog-only control measures the existing capability gap. Reports retain model usage, tool/API calls and provenance. Runtime, permissions and their safety tests stay in #1670; SDK comparisons remain future work.
The evals run what ships
mcp-cloudflare/src/code-mode/eval-worker.tsmakes the samebuildServer()andregisterCodeModeTools()calls as the hosted handler, soApiAuthorizationapplies as in production: only operations in the shipped schema pass, regional lookups run, and API Read/Write come from the connection's skills. Only the context differs: a dummy token and a loopback fixture. OAuth is not exercised.catalog-and-code/mcpcatalog/catalog/mcpcode-mode/code-mode/mcpAll surfaces search the shipped
@sentry/api0.428.0 schema. Fixtures answer the policy's organization lookup without a region URL, so requests stay on the fixture. There are no eval-onlyOPENAPI_URL/CODE_MODE_ALLOW_WRITESsettings and no private fixture schema any more. The worker test lives inmcp-server-evalsand runs on Wrangler withsrc/code-mode/wrangler.jsonc.Results on the hosted composition
Claude Code 2.1.316,
--model opus(claude-opus-5-5), medium effort:catalog-and-codecode-modecatalog(control)catalog-and-codecatalog-and-codeNo run requested an operation outside the shipped schema, so the policy refused nothing. Codex was not run. Details are in the evaluation guide.
The model runs above predate the #1670 cleanup, which this branch now merges. The cleanup changed how Code Mode is built, not what it allows: authorization now reads the generated operation list, and requests go through
SentryApiService.After the merge, these pass on
1bce8b1:The eval worker follows #1670: no
configuredUrl,buildServer()withoutsdkVersion, andisCodeModeSkill(). The runner now declares the@modelcontextprotocol/sdkclient it imports, which main no longer hoists through mcp-core. The branch merges #1670 instead of rebasing, so the original eval commit is unchanged. Fixtures do not establish production authorization or hosted CPU enforcement.https://claude.ai/code/session_017M6odW3kt5i65zATxfz9RX
Generated by Claude Code