feat(fetch): return binary response bodies and headers for hosted file downloads

CLAUDE.md

@@ -33,7 +33,7 @@ says why. All packages stay in lockstep on one version (see root
 | Package | What it does |
 |---|---|
 | `@fetchproxy/protocol` | Wire format: frame validators, crypto wrappers (X25519, Ed25519, HKDF, AES-GCM, SHA-256), mcp-id parsing, pair-code derivation, JSON-pointer evaluator. Pure functions, no I/O. Smallest dep surface — every other workspace depends on it. |
-| `@fetchproxy/server` | `request()` accepts `viaTab` to name the tab that relays a call — needed for API-only hosts (`api.example.com` serves no page, so its implied tab can never exist; route through the signed-in `www` tab instead). Guarded against the declared domains: it widens which tab performs the fetch, never which origins are reachable. Throws `FetchproxyScopeError` (with `.hint`) for gate-#2 scope rejections, so consumers that re-wrap bridge errors can still surface the re-pair remedy — build extension errors with `protocolErrorFrom()`, never `new FetchproxyProtocolError()` directly. MCP-side WebSocket bridge. `FetchproxyServer` class with `listen()`, `request()`, `fetch()`, `readCookies()`, `readLocalStorage()`, `readSessionStorage()`, `captureRequestHeader()`, `readIndexedDb()`, `graphqlQuery()`, `writeCookies()` (1.12+, the only write verb — see `docs/SECURITY.md` §T-cookie-write). Handles concentrator role-election (host vs peer), identity loading, session-key derivation. Persists per-MCP identity to `~/.fetchproxy/identity/<server-name>.json`. |
+| `@fetchproxy/server` | `request()` accepts `viaTab` to name the tab that relays a call — needed for API-only hosts (`api.example.com` serves no page, so its implied tab can never exist; route through the signed-in `www` tab instead). Guarded against the declared domains: it widens which tab performs the fetch, never which origins are reachable. Throws `FetchproxyScopeError` (with `.hint`) for gate-#2 scope rejections, so consumers that re-wrap bridge errors can still surface the re-pair remedy — build extension errors with `protocolErrorFrom()`, never `new FetchproxyProtocolError()` directly. MCP-side WebSocket bridge. `FetchproxyServer` class with `listen()`, `request()`, `requestBytes()`/`getBytes()` (#460: binary-safe body + relayed headers, run from the extension's background worker, 16 MiB cap derived from the hosted relay's 32 MiB message limit, gated on the extension hello's advisory `features: ['fetch_base64']` and trusted only on the answer's `bodyEncoding` marker), `fetch()`, `readCookies()`, `readLocalStorage()`, `readSessionStorage()`, `captureRequestHeader()`, `readIndexedDb()`, `graphqlQuery()`, `writeCookies()` (1.12+, the only write verb — see `docs/SECURITY.md` §T-cookie-write). Handles concentrator role-election (host vs peer), identity loading, session-key derivation. Persists per-MCP identity to `~/.fetchproxy/identity/<server-name>.json`. |
 | `@fetchproxy/bootstrap` | `createSessionLifter(opts)` returns a **repeatable** lift (declare scope → spin up `FetchproxyServer` → read everything → close, per call) — use it whenever the session can expire, wiring it straight into a session manager's `login`. `bootstrap(opts)` is one invocation of that lifter, kept for genuinely one-shot callers (a user-invoked `capture_session` tool that persists the token). Used by Pattern A MCPs (HoneyBook, OFW, Resy auth-refresh path) that just need a session blob then operate from Node. `storageDomain` selector for multi-domain MCPs. Returns `missing.{cookies,localStorage,sessionStorage}` — declared keys the browser did not return, so a **partial** lift can't masquerade as a clean one (reading the apex when the cookies live on `www` is the classic way to get a half-populated session that fails later somewhere unrelated). |
 | `@fetchproxy/cli` | `fpx` — a one-shot CLI over the same bridge: authenticated fetches and session reads through the user's signed-in browser tab, scoped by per-service **profiles** (each profile connects as `fpx-<name>` with its own identity, so ten services look like ten MCPs to the extension). Published, and it ships v4 changes like any other consumer — `src/bridge-errors.ts` is what names which half of the bridge is behind on a version mismatch. It is also how an operator debugs a straggler during a protocol rollout, without standing an MCP up. |
 | `@fetchproxy/test-helpers` | Published vitest mock helpers for consumers of `@fetchproxy/server` — a drop-in `FetchproxyServer` mock that captures constructor opts and exposes spy-able `request`/`fetch`/`captureRequestHeader`/`bridgeHealth`. Lets cohort MCPs unit-test their fetchproxy usage without a live bridge. |

docs/PROTOCOL.md

@@ -280,10 +280,17 @@ is also what every server hello is minted in answer to.
   "identityX25519Pub": "<base64 raw 32B>",   // 0.4.0+: long-term
   "identityEd25519Pub": "<base64 raw 32B>",  // 0.4.0+: signs the ready
   "sessionNonce": "<base64 raw 32B, fresh per connection>",
-  "unavailableCapabilities": ["download"]    // #418, optional — see below
+  "unavailableCapabilities": ["download"],   // #418, optional — see below
+  "features": ["fetch_base64"]               // #460, optional — see below
 }
 ```
 
+`features` (#460, optional) lists behaviours this extension build supports
+beyond protocol 4's base. Today one: `fetch_base64`, the binary `fetch` mode.
+Sent only when non-empty, sorted and de-duplicated; same validator bounds and
+unknown-name tolerance as `unavailableCapabilities`, and equally unsigned and
+advisory. See [§Binary fetch](#binary-fetch-460).
+
 `unavailableCapabilities` (#418, optional) lists the capabilities **this
 browser** cannot serve, found by runtime API detection — Safari has no
 `chrome.downloads`, for one. It is sent only when non-empty, sorted and
@@ -773,7 +780,9 @@ Semantics:
 
   Use it only where the isolated world genuinely fails. Some edge bot-managers accept a request from the page and reject the byte-identical one from the isolated world: on opentable.com a GraphQL **mutation** POST 403s from the isolated world and returns 200 from the page, while GraphQL queries and REST writes pass from either. The cost is that page script can see and patch `window.fetch`, so a flagged request loses the isolated world's tamper resistance — see `T-in-page-fetch` in [SECURITY.md](./SECURITY.md). Flag individual calls, never everything.
 
-**Body size caps:** request body ≤ 1 MB, response body ≤ 5 MB. Larger bodies are rejected with `ok: false` — including on the `inPage` path, which re-checks the response cap in the content script before the body leaves it.
+- `responseType`: optional, `"base64"` or absent (#460). Absent is text, unchanged. `"base64"` asks for the body's BYTES and changes where and how the request runs — see [§Binary fetch](#binary-fetch-460). Cannot be combined with `inPage: true`.
+
+**Body size caps:** request body ≤ 1 MB, response body ≤ 5 MB. Larger bodies are rejected with `ok: false` — including on the `inPage` path, which re-checks the response cap in the content script before the body leaves it. A binary (`responseType: "base64"`) response has its own cap, `MAX_BINARY_RESPONSE_BYTES` = 16 MiB.
 
 ##### `op: "read_cookies"`
 
@@ -965,6 +974,98 @@ break. So the list is advisory: it can only make an MCP believe *fewer*
 capabilities are available, never gain one. The analysis, and the optional
 tamper-evident follow-up, are in `docs/SECURITY.md` §T-unavailable-caps.
 
+## Binary fetch (#460)
+
+`fetch` decodes the body with `response.text()`, which is lossy for anything
+that is not UTF-8 text: invalid sequences become U+FFFD and cannot be
+recovered. A PDF, a zip-based office file or an image came back corrupted, and
+a hosted MCP whose files live on a host outside its egress allowlist had no
+route to the bytes. Binary fetch is an opt-in mode of the same verb, additive
+inside protocol 4: no `PROTOCOL_VERSION` bump, no new capability, and nothing
+any signature covers changes.
+
+**Request.** `init.responseType: "base64"`. Everything else is an ordinary
+`fetch` init; `tabUrl` is still required and still domain-checked.
+
+**Response.**
+
+```jsonc
+{
+  "type": "response", "id": 7, "ok": true, "op": "fetch",
+  "status": 200,
+  "url": "https://files.example.com/signed/7?sig=…",   // FINAL url, after redirects
+  "body": "JVBERi0xLjcK…",                             // standard padded base64 of the bytes
+  "bodyEncoding": "base64",
+  "headers": {                                         // relayed subset, lower-cased, each only if present
+    "content-type": "application/pdf",
+    "content-length": "48213",
+    "content-disposition": "attachment; filename=\"report.pdf\""
+  }
+}
+```
+
+The bytes ride in the existing `body` field rather than a new one, so every
+published validator still accepts the frame (they require `body` to be a
+string and ignore the rest); `bodyEncoding` is what makes it unambiguous. A
+text answer carries neither `bodyEncoding` nor `headers` and is byte-for-byte
+what it always was. The validator requires a `bodyEncoding: "base64"` body to
+be strict standard padded base64 (Node's decoder silently skips characters it
+does not understand), and `headers` to be a string map.
+
+**Where it runs.** In the extension's background service worker (event page on
+Safari), not a tab: the extension's `host_permissions` make CORS irrelevant
+there, so a signed file host that sends no CORS headers is readable, and
+`redirect: "follow"` reaches it. Cookies are the browser's for the URL
+(`credentials` is honoured; default `include`).
+
+**Scope — it reaches nothing a text fetch could not be pointed at.** The
+request URL must be on the MCP's declared `domains`, exactly as for text, and
+so must the **final** URL after redirects: an answer whose `response.url`
+left the declared domains is refused before a byte of its body is read. To
+read a signed CDN host, declare it. Intermediate redirect hops are not
+inspectable from `fetch()` (a `manual` redirect is opaque), but they reveal
+nothing to the MCP — their bodies are never read — and a page-level `fetch` has
+always followed the same hops. See `docs/SECURITY.md` §T-binary-fetch.
+
+**Size.** `MAX_BINARY_RESPONSE_BYTES` = 16 MiB of BYTES, derived from the
+transports rather than picked: a binary frame is about 16/9 of the body on the
+wire, so 16 MiB is ~28.5 MiB sealed — under the hosted relay's 32 MiB
+WebSocket message limit (`HOSTED_RELAY_MESSAGE_BYTES`) and `MAX_FRAME_BYTES`
+(42 MiB). The extension refuses early when `content-length` exceeds the cap,
+and again while streaming when the length is absent or lies. The refusal is
+`{ ok: false, op: "fetch", code: "response_too_large", error }` with the fixed
+wording from `responseTooLargeMessage()`, which starts `response body too
+large:` — the prefix every published server already classifies as
+`body_too_large`. The extension's per-frame size gate still applies on top.
+
+**Capability negotiation.** The extension lists `fetch_base64` in its hello
+`features`. A server (3.7.0+):
+
+- refuses a binary request **locally, with no round trip**, when the session's
+  extension hello did not list it — `FetchproxyExtensionUpdateRequiredError`
+  (`.feature`, `.hint`: update ContextMint Bridge); `fetch()` returns the
+  envelope `{ ok: false, kind: "extension_update_required" }`;
+- refuses any answer to a binary request that lacks `bodyEncoding: "base64"`
+  with the same error. This, not the advisory hello list, is what the bytes are
+  trusted on: an older extension ignores `responseType` (its validator predates
+  it) and answers in text, and that text is never decoded as bytes;
+- maps `code: "response_too_large"` / the fixed wording to
+  `FetchproxyResponseTooLargeError` (`.limitBytes`, `.declaredBytes`);
+- re-checks the final URL against its declared `domains` before returning.
+
+`requestBytes(method, path, opts)` / `getBytes(path, opts)` on
+`FetchproxyServer` resolve `{ status, url, headers, bytes: Uint8Array }`. The
+`fpx` CLI exposes it as `fpx get|request <url> -o <file>`.
+
+### Compatibility
+
+| MCP | Extension | What happens |
+|---|---|---|
+| < 3.7.0 | new | Never sends `responseType`, so nothing changes. The extra hello field is ignored (`return raw`). |
+| 3.7.0+ | old (no `features`) | `requestBytes()` refuses locally: update the extension. Text fetches are unaffected. |
+| 3.7.0+ | old, but a relay added `features` | The old extension ignores `responseType` and answers in text; the answer has no marker and is refused with the same update error. |
+| new peer | old host | An old host relays the extension hello as the parsed object, so `features` reaches the peer intact. A host too old to relay the extension hello at all leaves the peer with no list: the request is sent and the marker decides. |
+
 ## The ephemeral's lifetime
 
 Everything above says "the ephemeral this process currently holds" as though

docs/SECURITY.md

@@ -474,6 +474,21 @@ Names the MCP does not know are dropped, and the validator bounds the list (≤
 
 **Residual risk:** a party already positioned to relay the extension's frames can make an MCP believe *fewer* capabilities are available than really are, and make its error text blame the browser for it. That is a downgrade to refusal, never an escalation. If the list ever needs to be tamper-evident, the additive fix is an encrypted `session-info` frame (extension → MCP, inside the session, sent only to servers that list it in `accepts`) carrying the same list; this change does not add it.
 
+### T-binary-fetch — A background fetch that ignores CORS (#460)
+
+`fetch` with `responseType: "base64"` runs from the extension's **background worker**, not a tab ([PROTOCOL.md §Binary fetch](PROTOCOL.md#binary-fetch-460)). The manifest's `host_permissions` (`<all_urls>`) mean CORS does not apply there — that is the point, since signed file hosts send no CORS headers — so the extension's own code is the only scope gate, and it has to be at least as tight as the tab path's.
+
+**Defence, in the extension (`background/handlers/fetch-binary.ts` in contextmint-bridge):**
+
+- the request URL must be on the MCP's declared `domains` (the same `isUrlAllowedForAnyDomain` the text path uses), checked before any request is made;
+- the **final** URL after redirects (`response.url`) must be on them too, checked before a byte of the body is read; otherwise the body is cancelled and the request refused. So a declared host with an open redirect cannot be used to READ an undeclared, cookie-authenticated resource (the user's mail, their bank) without CORS standing in the way;
+- `inPage` is refused in this mode, and no tab is involved, so no CSRF token is injected;
+- the body is capped at `MAX_BINARY_RESPONSE_BYTES` early on `content-length` and again while streaming.
+
+**What is not narrowed, and why that is acceptable.** Intermediate redirect hops cannot be inspected from `fetch()` — a `manual` redirect is an opaque response with no `Location` — so a hop may pass through an undeclared host, carrying that host's cookies. Its response body is never read and nothing about it reaches the MCP, and a page-level `fetch()` (and `download`) has always followed the same hops; the browser sends that request either way. What changes relative to the tab path is that the browser may attach `SameSite=Strict`/`Lax` cookies to a request from an extension with host permission where a cross-site tab fetch would not, and that a CORS-less declared host becomes readable. Both are within the scope the user approved: the domains are what they paired the MCP to act on as them.
+
+**The capability flag is advisory.** `features: ["fetch_base64"]` on the extension hello is unsigned, like `unavailableCapabilities` ([§T-unavailable-caps](#t-unavailable-caps--tampering-with-this-browser-cannot-serve-x-418)). Stripping it makes the MCP refuse locally (a relay could already drop frames). Adding it makes the MCP send a binary request to an extension that does not understand it; that extension answers in text, and the server refuses any answer lacking the authenticated `bodyEncoding: "base64"` marker rather than decode text as bytes. It feeds nothing but that local refusal and the health report — never trust, pinning, the pair code, key derivation or a grant.
+
 ## What protocol 4 does not fix
 
 3.0.0 is the largest change to this document's crypto since 0.2.0, which makes it exactly the version whose residuals are worth writing down before somebody infers them away. v4 gives forward secrecy for past sessions and binds every frame to its `mcpId`, its ordinal and its direction. That is all it gives. Each of the following is true the day it ships:

packages/cli/README.md

@@ -79,6 +79,7 @@ Every verb takes `-p`/`--profile <name>` (except the `profile` subcommands thems
 | `fpx pair -p <name> [--domain <apex>] [--subdomain <label>]` | Force a pairing round-trip against one declared domain (defaults to the only one, if there's exactly one). `--subdomain` aims at the host your signed-in tab is actually on — the fetch tab-matcher is strict-prefix, so an apex-only profile won't match a `www.` tab. | `fpx pair -p opentable --subdomain www` |
 | `fpx health -p <name>` | Print the bridge's health snapshot (host/peer role, connection state). | `fpx health -p opentable` |
 | `fpx get <url> -p <name> [--json] [-H 'K: V']…` | GET a URL through the profile's tab. | `fpx get https://www.opentable.com/ -p opentable` |
+| `fpx get <url> -p <name> -o <file>` | Download the response **bytes** to a file (binary-safe; PDFs, office files, images). Fetched by the extension's background worker, so a CORS-less file host works; the final URL must be on the profile's domains; capped at 16 MiB. Needs a ContextMint Bridge with binary fetch. Also on `fpx request`. | `fpx get https://files.example.com/a.pdf -p files -o a.pdf` |
 | `fpx post-json <url> <body\|@file> -p <name> [--json] [-H 'K: V']…` | POST a JSON body (literal or `@file`); sets `Content-Type: application/json` unless overridden. | `fpx post-json https://www.opentable.com/dapi/fe/gql '{"operationName":"Autocomplete"}' -p opentable` |
 | `fpx request <url> -p <name> [-X METHOD] [-H 'K: V']… [-d body\|@file] [--json]` | Arbitrary method/headers/body. | `fpx request https://www.opentable.com/x -p opentable -X DELETE` |
 | `fpx cookies [keys…] -p <name>` | Read declared cookies (all declared keys if none given). | `fpx cookies -p opentable` |