Skip to content

feat(fetch): return binary response bodies and headers for hosted file downloads - #461

Merged
chrischall merged 1 commit into
mainfrom
feat/fetch-binary-base64
Nov 7, 2026
Merged

chrischall merged 1 commit into
mainfrom
feat/fetch-binary-base64

Conversation

@chrischall

Copy link
Copy Markdown
Owner

Adds an opt-in binary mode to fetch, so a hosted MCP can read a file's bytes through the user's browser. It covers files on a signed, CORS-less host outside the MCP's egress allowlist. Additive inside protocol 4: no version bump and no new capability. Nothing a signature covers changes.

Closes #460

Wire

  • Request: FetchInit.responseType: 'base64'. When it is absent the request is text, and the frame is byte-for-byte unchanged.
  • Response: body holds standard padded base64, marked bodyEncoding: 'base64', plus a relayed headers map (content-type, content-length, content-disposition, lower-cased, each present only when the response had it).
    • Reusing body keeps every published validator accepting the frame.
    • The validator requires strict base64 under the marker, because Node's decoder silently skips bad characters.
  • Capability: the extension hello has an unsigned, advisory features: ['fetch_base64'], with the same bounds and unknown-name tolerance as unavailableCapabilities.
  • Size: MAX_BINARY_RESPONSE_BYTES = 16 MiB. This is derived from transport limits rather than picked:
    • a binary frame is about 16/9 of the body on the wire, so the worst case is about 28.5 MiB sealed;
    • that clears the hosted relay's 32 MiB Durable Object WebSocket message limit (HOSTED_RELAY_MESSAGE_BYTES) and MAX_FRAME_BYTES (42 MiB). A test holds both;
    • 20 MiB would be 35.6 MiB on the wire and would close a hosted link.
  • Size refusal: code: 'response_too_large', with a fixed wording that starts response body too large:. Older servers already classify that prefix as body_too_large.

Server (@fetchproxy/server)

  • requestBytes(method, path, opts) / getBytes(path, opts) → { status, url, headers, bytes: Uint8Array }. They share URL resolution and the declared-domain guard with request(), which is refactored onto the same helper with unchanged behaviour. inPage and viaTab are not accepted.
  • FetchproxyExtensionUpdateRequiredError is thrown in two cases:
    • locally, with no round trip, when the hello lacks fetch_base64;
    • on any answer without the bodyEncoding marker. That is an old extension answering in text, and its text is never decoded as bytes.
  • fetch() returns kind: 'extension_update_required' instead of throwing.
  • FetchproxyResponseTooLargeError (.limitBytes, .declaredBytes) is built from the code or the wording. The envelope now carries the wire code.
  • The final URL is checked against the declared domains again before the bytes are returned.
  • bridgeHealth().session.features reports what the extension advertised.
  • cli: fpx get|request <url> -o <file>. test-helpers: requestBytes / getBytes spies.
  • Docs: PROTOCOL.md §Binary fetch (with a compatibility table) and SECURITY.md §T-binary-fetch, which covers why a background, CORS-free fetch does not widen what the extension can reach.

Extension

The extension half is nullnet-app/contextmint-bridge: a new background fetch-binary.ts handler and the hello flag. It depends on @fetchproxy/protocol@^3.7.0, i.e. this PR's release.

Release order: merge this → release fetchproxy (3.7.0) → the extension PR's CI goes green → ship ContextMint Bridge. Until the extension ships, requestBytes() refuses cleanly with the update hint.

Tests

  • protocol: validator for responseType, bodyEncoding, headers and features; strict base64; the size wording round-trip; the worst-case frame fits both transport limits.
  • server, against a mock extension speaking the real wire:
    • all 256 byte values round-trip exactly;
    • headers are relayed;
    • the redirect final URL is reported;
    • an off-domain final URL is refused;
    • an off-domain request URL is refused before sending;
    • expectStatus is honoured;
    • text mode is unchanged on the wire;
    • local refusal with an old extension (no frames sent);
    • an unmarked answer is refused;
    • early and streaming size refusals are typed.
  • cli and test-helpers coverage.

npm run build, npm run typecheck and npm test pass: 149 files, 2214 tests.

🤖 Generated with Claude Code

https://claude.ai/code/session_012aZHZ6pnrdzrECCUaQW3gY

…e downloads

Adds an opt-in binary mode to the `fetch` verb so a hosted MCP can get a
file's bytes through the user's browser (#460).

- protocol: `FetchInit.responseType: 'base64'`; the answer reuses `body`
  (standard padded base64) marked `bodyEncoding: 'base64'`, plus a relayed
  `headers` subset (content-type / content-length / content-disposition).
  Text-mode frames are byte-for-byte unchanged. The extension hello gains an
  unsigned, advisory `features` list (`fetch_base64`). Size refusals carry
  `code: 'response_too_large'` and a fixed wording that starts
  `response body too large:`, which every published server already files as
  `body_too_large`. `MAX_BINARY_RESPONSE_BYTES` = 16 MiB, derived so the
  worst-case sealed frame (~28.5 MiB) clears the hosted relay's 32 MiB
  WebSocket message limit and MAX_FRAME_BYTES; a test holds it there.
- server: `requestBytes()` / `getBytes()` resolve
  `{ status, url, headers, bytes: Uint8Array }`. Refused locally with
  `FetchproxyExtensionUpdateRequiredError` when the extension hello does not
  list `fetch_base64`, and on any answer without the `bodyEncoding` marker
  (an older extension answering in text). The final URL is re-checked
  against the declared domains. `FetchproxyResponseTooLargeError` maps the
  size refusal. `bridgeHealth().session.features` reports the list.
- cli: `fpx get|request <url> -o <file>` writes the bytes.
- test-helpers: `requestBytes` / `getBytes` spies.
- docs: PROTOCOL.md §Binary fetch, SECURITY.md §T-binary-fetch.

Closes #460

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aZHZ6pnrdzrECCUaQW3gY
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

✅ Auto-review verdict: pass — The opt-in binary fetch mode is additive and carefully built. Bytes are only decoded when the answer is marked as base64, the final URL is checked against the declared domains, the base64 is validated strictly, and the error types are covered by tests. I found no issues.

Reviewed commit 7944d80.


No issues found.

What I checked

Protocol (binary-fetch.ts, validate.ts, frames.ts)

  • A text request stays byte-for-byte unchanged: responseType, bodyEncoding, headers and features are all optional and only added when set.
  • The validator only accepts responseType: 'base64', and a body marked bodyEncoding: 'base64' must be strict padded base64 (isBase64Body).
  • The hello's features list has the same bounds and unknown-name tolerance as unavailableCapabilities.
  • The size-refusal wording keeps the response body too large: prefix that older servers already treat as body_too_large, and its parser regexes cap digit counts.

Server (ws-server.ts, session-info.ts, error-kind.ts)

  • request() was split into resolveRequestInit, throwFetchFailure and assertExpectedStatus. The logic is unchanged; the only difference is that error messages now name the calling method (request or requestBytes).
  • requestBytes() only decodes bytes when the answer carries the bodyEncoding marker. It doesn't rely on the unsigned features list for that.
  • It checks the final URL against this.opts.domains, re-checks the base64 before calling Buffer.from, and handles expectStatus with an empty-body HttpResponse rather than pretending a binary body is text.
  • refuseIfUnavailable now also refuses locally when the extension's hello didn't list the feature, before any request is sent. When there's no hello to check (info === null), the request goes out and the marker decides, as documented.
  • fetch() turns the local refusal into kind: 'extension_update_required', and _errorForResult returns the typed error from cause.
  • protocolErrorFrom maps a size refusal, by its code or its wording, to FetchproxyResponseTooLargeError. This check is separate from the capability-unavailable check and can't catch it by mistake.

CLI

  • -o/--output is refused together with --in-page or --via-tab before the server factory is called.
  • The bytes go through an injectable writer, so tests don't touch the disk, and a non-2xx status still exits with EXIT.HTTP.
  • The new errors extend FetchproxyHintedError, which bridge-errors.ts:96 (if (err instanceof FetchproxyHintedError)) already prints with its hint, so they don't fall into the generic version-mismatch advice.

API contract
Every change is additive:

  • new optional fields FetchResultError.code, FetchResult.bodyEncoding and FetchResult.headers;
  • a new required features field on the BridgeSessionState output;
  • a new FetchErrorKind value, extension_update_required;
  • new methods and exports.

No field or value was renamed or removed, and no input was renamed or made required.

Tests cover all 256 byte values round-tripping, the headers, the final URL after a redirect, off-domain request and final-URL refusals, a refusal for an old extension (no hello flag, or an answer without the marker), typed size refusals, unchanged text frames, the worst-case frame fitting under both transport limits, and the CLI paths. I didn't run the test suite myself.

Open question (not a finding)
The docs say responseType: 'base64' can't be combined with inPage: true. The protocol validator doesn't reject that pair; the docs leave it to the extension, and the typed server API (BinaryRequestOpts omits inPage) and the CLI both block it. A JavaScript caller passing inPage anyway would rely on the extension in the contextmint-bridge repo, which I can't check from here.

@chrischall chrischall added the ready-to-merge Arms auto-merge — added by the pipeline on a pass/warn verdict, never by hand label Nov 7, 2026
@chrischall
chrischall enabled auto-merge (squash) November 7, 2026 12:44
@chrischall
chrischall merged commit 2955db3 into main Nov 7, 2026
19 checks passed
@chrischall
chrischall deleted the feat/fetch-binary-base64 branch November 7, 2026 12:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-review Auto-review pipeline is handling this PR enhancement Release notes: Features ready-to-merge Arms auto-merge — added by the pipeline on a pass/warn verdict, never by hand

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fetch: return binary response bodies (base64) and response headers, for hosted file downloads

1 participant