Skip to content

feat(uploads): proxy bytes into a service instead of handing out its credential - #590

Merged
angel-manuel merged 1 commit into
devfrom
feat/proxy-uploads
Aug 31, 2026
Merged

feat(uploads): proxy bytes into a service instead of handing out its credential#590
angel-manuel merged 1 commit into
devfrom
feat/proxy-uploads

Conversation

@angel-manuel

Copy link
Copy Markdown
Contributor

What and why

WhatsApp media only moved one way through the gateway. download_media wraps the container's GET /media/<sha> behind a capability URL; POST /media was unreachable, so send_file could only forward a reference some earlier download produced. An agent could re-send a file someone sent it and could not send a file it made. services/whatsapp.yaml said so outright — originating bytes was "out-of-band work for whoever operates the container".

The direct alternative — hand the agent the container's URL and token — is the one thing the vault exists to prevent. There is no narrower credential to mint: the container has a single static AUTH_TOKEN with no scopes, audience or expiry, and it authorizes /mcp, which is every WhatsApp write. Proxying also buys what the direct route structurally cannot: the gateway sees the descriptor, so it can say what a later approval is actually about.

How

An MCP action may declare x-overslash-upload (bare alias upload:), naming a plain-HTTP byte route on the instance's own origin. Such an action is served by the gateway — never forwarded — and returns a one-shot upload_url on POST|PUT /v1/uploads/{token}. Redemption streams the request body through with the bearer re-resolved from the vault at that moment.

The organizing invariant: everything the reviewer approved is fixed at mint time; the anonymous redemption leg contributes only bytes. Filename, route, ceiling and declared content all come off the token; the redeemer contributes bytes and at most a Content-Type hint.

Three places this is deliberately not a mirror of the download path

  • Single-use. download_tokens is multi-use on purpose so a dropped transfer can resume with curl -C -. Redeeming a download twice re-fetches the same bytes; redeeming an upload twice stores two different payloads under one authorization. Hence a separate table whose claim consumes — bolting a consume onto a table with the opposite contract invites a later change to the shared claim to quietly make uploads replayable.
  • Two dispatch sites, and the second is the one that matters. upload_media is risk: write, so for any gated agent the first call is replayed from a stored payload after approval, through stored_call::run_mcp — and replay resolves nothing: it holds a URL and a tool name, not an action key. Intercepting only in call_mcp::dispatch would have worked perfectly until a human said yes, then answered -32603 Unknown tool. The spec therefore rides on StoredMcpCall, the same answer D56 gave timeouts and D75 gave pagination.
  • A hash can be detected, not prevented. It is not known until the last byte, which is after the upstream has them. A mismatch answers 422 and withholds the stored reference, so nothing downstream can name those bytes — but they do linger upstream as an unreferenced orphan, and the module doc says so rather than implying the check is preventive. Size is prevented: the meter cuts the transfer mid-stream.

Approvals stop showing a bare hash

The gateway sees a full descriptor at exactly two moments, and media_descriptors records both. The download-side write is deliberately not gated on deliver: "url" — the template's own guidance is to call download_media without it when forwarding, which is the common case, so gating would have left exactly the calls that produce references unrecorded.

A new non-tool resolve: {source: media} target reads that ledger, riding inside the existing x-overslash-resolve rather than growing the extension vocabulary. A send_file approval's primary row now reads invoice.pdf (application/pdf, 240912 bytes). scope: is refused alongside source: — collapsing every file sharing a filename onto one grant is a silent authorization widening.

Honest limitation: bytes pushed to the container out of band were never seen here, stay un-enriched, and fall back to the raw path. The fallback is lossless — the reviewer still sees exactly the string the call will send — so a miss is less helpful, never misleading.

Surface

  • Migration 116: upload_tokens (declared/stored column split) + media_descriptors.
  • Config: UPLOAD_TOKEN_TTL_SECS (900), UPLOAD_MAX_BYTES (100 MiB).
  • http_caller::call_streaming_upload — the first streamed request body in the repo. No timeout parameter, deliberately: on an upload send() waits for the last byte, so a deadline is a cap on transfer duration; liveness is bounded per chunk instead.
  • Audit action.uploaded carries declared and measured, since a divergence is the whole signal. Dashboard shows both.
  • resolve_same_origin moved from routes/actions/deferred.rs into services/deferred_download.rs with its test module unchanged — passing after the move is the evidence the upload path inherited the download path's refusals rather than a re-derived approximation.

Testing

  • New crates/overslash-api/tests/uploads.rs, 16 tests, registered in tests/api.rs. Each covers a way this can fail silently: the gated path (mint → approve → replay mints, and the stub records zero tool calls), single-use redemption, unknown/expired/consumed being indistinguishable, sha256 mismatch withholding the reference, mid-stream size cut, fail-closed on a deleted secret, off-origin refusal, and the disclosure enrichment including the un-enriched fallback.
  • Full workspace suite: 2727/2741. The 14 failures are oversla-sh::integration, which need a Valkey on :6380 that isn't running locally — a crate this branch does not touch.
  • cargo fmt, clippy --all-targets -D warnings, sqlx prepare --check, dashboard svelte-check (0 errors) and build:strict all pass.

Notes for review

  • SCHEMA.sql is regenerated, which also picks up pre-existing drift (magic_link_tokens, idx_service_templates_extends) the gate had been skipping whenever a committer's database was unreachable. Happy to split that out if you'd rather.
  • A failed push burns its token. Re-arming on failure would let a caller who can make the upstream fail re-offer different bytes indefinitely against one approval. The declared/stored split makes a started-and-abandoned push legible, and re-minting costs one action call.
  • Follow-up: OAuth-authenticated MCP services are refused at mint, same as deliver: "url" — the redemption cannot re-mint an OAuth bearer.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NYNCPycSzfq5YSq7dq2yWk

…credential

WhatsApp media moved one way through the gateway. `download_media` wrapped the
container's `GET /media/<sha>` behind a capability URL; `POST /media` was
unreachable, so `send_file` could only forward a reference some earlier download
produced. An agent could re-send a file someone sent it and could not send a
file it made — the template said so outright, and called originating bytes
out-of-band work for whoever operates the container.

The direct alternative is the one thing the vault exists to prevent: the
container has a single static AUTH_TOKEN with no scopes, audience or expiry, and
it authorizes /mcp — every WhatsApp write, not just an upload.

An MCP action may now declare `x-overslash-upload`, naming a plain-HTTP byte
route on the instance's own origin. Such an action is served by the gateway
rather than forwarded, and returns a one-shot `upload_url`. Redeeming it streams
the request body through to that route with the bearer re-resolved from the
vault at that moment. Everything the reviewer approved is fixed at mint time;
the anonymous redemption leg contributes only bytes.

Optional sha256/size_bytes bind a capability to one file. Size is cut
mid-stream, so the upstream never sees the overage. A hash can only be detected
— it is not known until the last byte — so a mismatch answers 422 and withholds
the stored reference, leaving nothing downstream able to name those bytes.

Intercepted at *both* dispatch sites. `upload_media` is risk: write, so for any
gated agent the first call is replayed from a stored payload after approval, and
replay resolves nothing: it holds a URL and a tool name, not an action key. An
interception present only on the inline path would have worked perfectly until a
human said yes.

`upload_tokens` is a separate table from `download_tokens` because their
redemption rules are opposites: a download is deliberately multi-use so a
dropped transfer can resume, while a second upload under one authorization
stores different bytes than were approved.

A `media_descriptors` ledger records every descriptor the gateway sees, in both
directions and regardless of `deliver: "url"` — the template's guidance is to
call download_media without it when forwarding, which is the common case. A new
non-tool `resolve: {source: media}` target reads that ledger, so a send_file
approval's primary row names the file instead of hashing it. References the
gateway never handled stay un-enriched and fall back to the raw path.

SCHEMA.sql is regenerated, which also picks up pre-existing drift
(magic_link_tokens, idx_service_templates_extends) the gate had been skipping
whenever a committer's database was unreachable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NYNCPycSzfq5YSq7dq2yWk
@vercel

vercel Bot commented Aug 31, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
overslash Ready Ready Preview Aug 31, 2026 2:13pm

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

Code diff size

+2653 / −117 across 31 files (net +2536)

Area Files Added Removed Net
overslash-api 19 +1895 −106 +1789
overslash-core 7 +361 −8 +353
overslash-db 3 +328 −0 +328
dashboard 2 +69 −3 +66

Source files under src/ only (.cjs, .go, .js, .jsx, .mjs, .py, .rs, .svelte, .ts, .tsx); test files and inline #[cfg(test)] modules excluded. 22 other changed files not counted.

@angel-manuel
angel-manuel merged commit 2397428 into dev Aug 31, 2026
15 checks passed
@angel-manuel
angel-manuel deleted the feat/proxy-uploads branch August 31, 2026 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant