feat(engine): MCP result-by-reference protocol — IndexedDB cache + URI substitution (Units 1-4) - #36
Merged
Merged
Conversation
Plan: docs/plans/2026-05-18-002-feat-mcp-result-by-reference-protocol-plan.md
Unit 1.
Foundation for the MCP result-by-reference protocol. Caches oversized
tool results in IndexedDB and mints `mcp+cache://<conv-id>/<8-byte-b64>`
URIs the LLM can pass forward in subsequent tool calls. Unit 2 will
instrument the engine's tool-result write site to use this. Unit 3
will add the substitution layer that resolves URIs before dispatch.
Storage choice rationale:
- IndexedDB (not in-memory Map): persists across page reload, browser-
managed eviction, gigabyte-scale quota. Survives the "user refreshed
mid-conversation" failure mode that an in-memory Map can't.
- One DB per chatbox-core installation. One object store keyed by URI.
- Secondary indexes on convId (for clearConversation) and addedAt
(for age-based eviction).
Heuristic auto-cache:
- Caches only payloads whose JSON-serialized size exceeds the threshold
(default 4 KB to match v0.6.4's MAX_TOOL_RESULT_CHARS). Smaller
payloads aren't worth the cache write.
- No producer-side opt-in needed. Every tool result over the threshold
becomes a candidate.
URI format: `mcp+cache://<sanitized-conv-id>/<11-char-base64url>`.
Conv-id is path-traversal-sanitized (`[^A-Za-z0-9_-]` → `_`); falls
back to "default" if sanitization yields an empty string. 8-byte
(64-bit) random token from `crypto.getRandomValues` — unguessable
inside a conversation that holds at most thousands of entries.
Public surface:
- `cacheToolResult({ payload, convId, sourceToolName, threshold })`
returns the minted URI or null (below threshold / no IndexedDB).
- `readCachedPayload(uri)` returns the original payload or null.
- `clearConversation(convId)` drops all entries for a conv-id (host
calls this on dashboard switch / chat reset).
- `evictOlderThan({ maxAgeMs })` age-based eviction.
- `hasIndexedDB()` runtime check — module is a graceful no-op in
test environments without an IndexedDB shim.
Tests: fake-indexeddb (new dev dep) shims the real API in Node so
the same code path that runs in browsers executes here. 16 new tests
cover URI minting + collision safety + sanitization, threshold
heuristic, write/read round-trip, cache miss, clearConversation
scoping (drops named conv-id only), and evictOlderThan no-op + keep-
fresh semantics.
Full suite: 463 → 479 passed.
…ns through runChatSession
Plan: docs/plans/2026-05-18-002-feat-mcp-result-by-reference-protocol-plan.md
Unit 2.
Instruments the tool-result write site at engine/index.js to call
`cacheToolResult()` from Unit 1's cache module and inject `_cache_uri`
into the LLM-visible envelope when (a) the host has opted in via the
new `enableResultCache` runChatSession parameter, and (b) the
serialized payload exceeds the threshold (4 KB default, matching
v0.6.4's truncation cap).
Three changes:
1. `runChatSession` signature gains two parameters:
- `enableResultCache: boolean` (default false — npm consumers
that don't opt in see no behavior change)
- `conversationId: string` (default "default" — used as the conv-id
segment of minted cache URIs; tethysapp-tethys_dash will pass the
dashboard UUID once Unit 4 wires the host prop)
These are composed into a `cacheOptions = { enabled, conversationId }`
object threaded into both call sites of `processToolCalls`.
2. `processToolCalls` signature accepts `cacheOptions` in its options
bag. At the tool-result write site (around the existing
`_engine_dispatched` injection), when cache is enabled AND the
result is object-shaped, calls `cacheToolResult({payload, convId,
sourceToolName})` and injects the returned URI as `_cache_uri` on
the LLM-visible envelope.
3. The cache write happens BEFORE the truncation pass, so the cached
payload is always the ORIGINAL pre-truncation result. The
truncation summary block now also preserves `_cache_uri` so the
LLM gets the reference even when the bulk payload was dropped.
This is exactly the case where the URI is most valuable — large
responses that overflow the cap can still be passed forward by
reference.
Unit 3 (next) will add the substitution layer in `processToolCalls`
that resolves URIs back to inline data before dispatch. Until Unit 3
lands, the `_cache_uri` is informational only — the LLM sees it but
chatbox-core doesn't act on it.
Tests: 5 new tests in cache-instrumentation.test.js covering:
- default-off: no `_cache_uri` when cacheOptions is absent
- enabled + oversized: `_cache_uri` injected with correct conv-id prefix
- enabled + small: threshold heuristic skips the cache write
- enabled + huge (truncation fires): `_cache_uri` preserved into summary
- disabled + huge: truncation summary has no `_cache_uri`
Full suite: 479 → 484 passed.
…_uri args from cache before dispatch
Plan: docs/plans/2026-05-18-002-feat-mcp-result-by-reference-protocol-plan.md
Unit 3.
The substitution layer is the load-bearing piece of the result-by-
reference protocol. When the LLM emits a tool call with a `*_uri` arg
matching the `mcp+cache://` scheme, chatbox-core resolves the URI
from IndexedDB (Unit 1), substitutes the cached payload into the
corresponding non-`_uri` arg, drops the `_uri` arg, and dispatches
the call with inline data. The server tool sees the resolved data
and runs unchanged — no MCP wire-contract changes for receivers that
don't opt in.
Three integration choices:
1. Substitution runs BEFORE the existing `_substituteLastUuidPlaceholders`
walk in processToolCalls. The two layers cover different patterns
(open-vocabulary `*_uri` vs closed-vocabulary `{{last_<type>_uuid}}`)
and ordering is explicit so a future URI value that happens to
match a placeholder shape gets resolved as URI first.
2. Substitution is gated on `cacheOptions.enabled` (Unit 2's host
prop). When disabled, the URI passes through verbatim and the
receiving tool's own validation rejects it — no half-resolved
state when the feature is off.
3. Cache miss short-circuits dispatch. The tool is NOT called; an
`invalid_args` envelope with `_missing_uris` + `fix_hint` is
pushed as the tool result so the LLM gets a recoverable error
matching the existing input-validation-middleware shape. No
auto-retry — the LLM must surface to the user (per the v1 cache-
miss policy locked in plan refinement).
Conflict resolution: if the LLM passes BOTH `data` AND `data_uri`,
URI wins, inline is dropped, console.info fires for telemetry.
Treats both-set as an LLM bug worth surfacing in metrics but not
worth failing the call.
Array URIs supported: `layers_uri: [u1, u2, u3]` resolves each URI
and substitutes `layers: [p1, p2, p3]`. Any miss aborts the
substitution with all missing URIs named in the envelope's
`_missing_uris` list.
Tests: 11 new tests in uri-substitution.test.js — 8 direct unit
tests on substituteCacheUris (happy path scalar + array, conflict,
miss scalar + array, non-cache URI scheme ignored, non-object args
graceful) plus 3 integration tests through processToolCalls
(resolved-arg dispatch, miss short-circuit, disabled pass-through).
Full suite: 484 → 495 passed.
…sult-by-reference protocol
Plan: docs/plans/2026-05-18-002-feat-mcp-result-by-reference-protocol-plan.md
Unit 4.
Threads the host-side opt-in for the MCP result-by-reference protocol
through Chatbox.jsx to runChatSession. Both props default off so npm
consumers of @aquaveo/chatbox-core that don't know about the feature
inherit no behavior change — the cache write site (Unit 2) and the
URI substitution layer (Unit 3) stay fully dormant.
New props:
- `enableResultCache: boolean` (default false) — when true, oversized
tool results are auto-cached in IndexedDB (Unit 1) and the LLM
receives `_cache_uri` markers it can pass forward as `*_uri` args.
- `conversationId: string` (default "default") — used as the conv-id
segment of minted cache URIs and as the scope for clearConversation
(called by the host on dashboard switch / chat reset).
Wiring:
- Props declared in the Chatbox component destructure
- Forwarded into the runChatSession call alongside the existing
engineExtensions spread
- Added to the useCallback dependency array so the callback re-creates
when either prop changes (e.g., user switches dashboards and the
host passes a new conversationId)
tethysapp-tethys_dash's <ChatSidebar> will set
`enableResultCache={true} conversationId={dashboardUuid}` in a
follow-up PR — that's the host-side activation. Until then, the
chatbox-core feature ships fully dormant on the npm package.
No new tests in this unit — the prop wiring is mechanical and is
covered end-to-end by Unit 3's integration tests (which exercise
processToolCalls with cacheOptions). Full suite: 495 passed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements Units 1-4 of the MCP result-by-reference protocol per plan `docs/plans/2026-05-18-002-feat-mcp-result-by-reference-protocol-plan.md` (firoh workspace).
Eliminates the LLM transcription bottleneck observed in production 2026-05-18: a 240-row time-series array took ~127s of LLM output tokens to regenerate between two MCP servers (nrds_mcps query → tethysdash_mcps create_plotly_chart). With this protocol, oversized tool results are cached in IndexedDB client-side and the LLM emits a short `mcp+cache://...` URI in the next tool call instead of inlining the data.
Architecture
Four chatbox-core units in this PR:
`engine/cache.js` (Unit 1) — IndexedDB cache module. Mints `mcp+cache:///<8-byte-base64url>` URIs, writes payloads keyed by URI, heuristic size threshold (4 KB, matching v0.6.4's truncation cap). Public surface: `cacheToolResult`, `readCachedPayload`, `clearConversation`, `evictOlderThan`, `mintCacheUri`, `estimateSize`, `hasIndexedDB`.
Tool-result instrumentation (Unit 2) — at `engine/index.js:978` (the tool-result write site), if `cacheOptions.enabled` AND the result is object-shaped, calls `cacheToolResult` and injects `_cache_uri` into the LLM-visible envelope. Cache write happens BEFORE the truncation pass, so the cached payload is always the ORIGINAL pre-truncation result. The truncation summary preserves `_cache_uri` so the LLM still gets the reference even when the bulk payload was dropped.
`engine/uri-substitution.js` (Unit 3) — substitution layer in `processToolCalls`. Runs BEFORE the existing `_substituteLastUuidPlaceholders` walk. Detects `*_uri` args matching `mcp+cache://`, resolves from cache, substitutes corresponding non-`_uri` arg, drops the `_uri` arg. Handles scalar + array URIs. Conflict resolution: URI wins, inline dropped, `console.info` logged. Cache miss: `invalid_args` envelope short-circuits dispatch.
Host prop wiring (Unit 4) — `` + `` flow through `runChatSession` → `processToolCalls` as a `cacheOptions` object. Defaults to `{enabled: false, conversationId: "default"}` so npm consumers that don't opt in inherit zero behavior change. Generic-engine identity preserved.
Why this works across providers
The substitution happens at the chatbox-core wire layer — BEFORE the tool call is dispatched. The receiving server tool sees inline data exactly as if the LLM had passed it directly. No MCP wire-contract change needed for receiving tools that don't opt in.
For receiving tools that DO opt in (declare a `*_uri` arg), the LLM's tool-call output drops from 4000+ tokens (a 240-row JSON array) to ~10 tokens (a short URI). Tool-call generation latency for large data flows drops from ~60-127s to milliseconds.
Companion PR
`Aquaveo/tethysdash_mcps#7` — receiving side. Adds `data_uri` opt-in to `create_plotly_chart`, `create_data_table`, `create_card`. Independent ship safety: each PR is additive and backward-compatible. Mediated end-to-end behavior unlocks once both ship.
Test plan
Plan + workspace
Ship sequence
This PR ships first. Then `Aquaveo/tethysdash_mcps#7`. Then `tethysapp-tethys_dash` PR (pending) sets `` — the host-side activation gate.
Once published as v0.7.0, the tethysapp-tethys_dash devcontainer alias bumps to `@aquaveo/chatbox-core@0.7.0`.