Skip to content

feat: incremental tile streaming — per-tool dispatch + streaming-state events - #45

Merged
romer8 merged 4 commits into
mainfrom
feat/incremental-tile-streaming
May 28, 2026
Merged

feat: incremental tile streaming — per-tool dispatch + streaming-state events#45
romer8 merged 4 commits into
mainfrom
feat/incremental-tile-streaming

Conversation

@romer8

@romer8 romer8 commented May 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

Phase 1 of docs/plans/2026-05-28-002-feat-chatbox-core-incremental-tile-streaming-plan.md (origin: docs/brainstorms/2026-05-28-incremental-tile-streaming-requirements.md).

Adds incremental per-tool dispatch to chatbox-core so tiles appear on the dashboard as each MCP tool returns, instead of all at once at end-of-turn. On local Ollama at ~10s per round, a 5-tool turn goes from "blank dashboard for 50s, then 5 tiles" to "first tile at ~10s, last at ~50s" — same total wall-clock, dramatically different perceived latency. Also lays the lifecycle groundwork for tethysdash (Phase 2) to gate per-tile edit/delete affordances during streaming.

What's in this PR (Phase 1)

Unit 1 — engine onToolEnvelope callback (engine/index.js)

  • New optional callback parameter on processToolCalls and runChatSession options bags.
  • Fires once per visualization / layer_update / patch_update push with the per-call delta of dispatchedUuids (NOT cumulative).
  • Wrapped in fireToolEnvelope helper mirroring the fireStatus pattern: try/catch + console.warn, never aborts the engine loop.
  • Threads signal into the processToolCalls options bag too (previously only forwarded to executeTool).
  • 10 new tests in engine/onToolEnvelope.test.js — happy paths per envelope kind, multi-shape single result, multi-tool ordering + per-call delta isolation, host throw containment, signal.aborted gate, undefined no-op backward compat.

Unit 2 — Chatbox.jsx per-tool dispatch + end-of-turn no-op sentinel (merged from Units 2+4 per scope-guardian review)

  • Chatbox.jsx supplies the engine callback; translates each invocation into the existing window.dispatchEvent shapes:
    • visualizationtethysdash:add-visualization {batch:true, panels:[panel]} (per-panel construction inlined from end-of-turn block at lines 928-955, including resolveVisualizationUrl + client_custom_remote + inlineData branches).
    • layer_updatetethysdash:update-visualization flat shape {uuid: map_uuid, operation: 'append_layers', layers: [layer]} matching the existing handler. NOT batched (handler does not look at detail.updates).
    • patch_updatetethysdash:update-visualization {batch:true, operation:'apply_patch', patches:[entry]}.
  • Defense-in-depth gates: signal.aborted check + capturedTurnId freshness check. No requestAnimationFrame per the raf-timing-race solution doc.
  • streamedDispatchFiredRef sentinel: reset at turn start, set on first callback invocation; the three end-of-turn dispatch sites (lines ~928, ~983, ~1072) check it and skip dispatch when streaming already fired (R10). Backward-compat: legacy hosts (no onToolEnvelope) see sentinel stay false and the existing end-of-turn dispatch runs unchanged.

Unit 3 — tethysdash:turn-start / tethysdash:turn-end window events

  • turn-start fires immediately AFTER abortRef.current = controller is assigned. Inline INVARIANT comment names the sequencing so a future refactor doesn't reorder.
  • turn-end fires from the shared finally block. Single site covers success / thrown error / user-initiated abort / /clear uniformly.
  • Events carry no payload; listeners (added in Phase 2 on tethysdash) toggle a boolean.

Unit 5 — Stop UX contract documentation

  • The existing result.aborted ? (accumulatedContent || "(Stopped)") : (result.assistantText ...) shape already implements R8a because the engine returns {aborted: true} rather than throwing on abort, so the stop path routes through setMessages not ChatErrorPanel. Added comment block naming the contract so the behavior survives future refactors.

API surface delta

Surface Before After
<Chatbox> component props unchanged unchanged
runChatSession options (existing) + onToolEnvelope (optional, default null)
processToolCalls options (existing) + onToolEnvelope, + signal
Window events emitted from chatbox-core tethysdash:add-visualization, tethysdash:update-visualization, tethysdash:patch-rejected + tethysdash:turn-start, + tethysdash:turn-end

Additive only. Passive consumers see no behavior change.

Phase 2 (deferred to a follow-up PR on tethysapp-tethys_dash)

Lands after this beta is published. Adds a StreamingContext to Contexts.js, has DashboardLoader.js listen for the new window events and provide an isStreaming flag, and gates DashboardItem's edit/delete/reorder affordances on it with aria-disabled="true" + opacity + tooltip.

Test plan

  • npx vitest run → 679/679 + 10 new = 689/689 pass
  • npm run build → dist rebuilt clean
  • After publish to beta dist-tag, smoke against tethysdash dev (Phase 2 PR will exercise the events end-to-end)
  • Plan deferred questions: planning needs to verify /clear interleaving with turn-end timing under real conditions

Release notes

CHANGELOG updated under ## [0.16.0-beta.0] — 2026-05-28. package.json bumped from 0.15.30.16.0-beta.0. Adjust before publishing if a different version is preferred.

Plan: docs/plans/2026-05-28-002-feat-chatbox-core-incremental-tile-streaming-plan.md
Origin: docs/brainstorms/2026-05-28-incremental-tile-streaming-requirements.md

romer8 added 4 commits May 28, 2026 15:35
Engine grows an optional `onToolEnvelope({kind, envelope, dispatchedUuids})`
callback in the processToolCalls + runChatSession options bags. Fires once
per `visualization` / `layer_update` / `patch_update` push with the per-call
delta (just-pushed UUID, NOT cumulative). Wrapped in a fireToolEnvelope
helper that mirrors the existing fireStatus pattern: try/catch + console.warn,
never aborts the engine loop on host bugs.

Also threads `signal` into the processToolCalls options bag (previously only
forwarded to executeTool). The fireToolEnvelope wrapper gates on
`signal?.aborted` so a user-initiated Stop between two tool dispatches
prevents further tile envelopes from reaching the host — preserves tiles
already dispatched while halting downstream side effects.

Undefined onToolEnvelope is a no-op: legacy consumers and tests inherit
end-of-turn-only dispatch behavior.

10 new tests in engine/onToolEnvelope.test.js covering:
- per-shape happy paths (visualization/layer_update/patch_update)
- multi-shape single result (visualization + layer_update fires twice)
- data-only tool (no envelope → no fire)
- multi-tool turn ordering + per-call delta isolation
- host throw containment via console.warn
- subsequent tools process after host throw
- signal.aborted gate suppresses post-abort fires
- undefined callback is a clean no-op

Phase 1 of docs/plans/2026-05-28-002-feat-chatbox-core-incremental-tile-streaming-plan.md
Unit 1.
…inel

Chatbox.jsx supplies onToolEnvelope to runChatSession. The callback translates
each per-tool envelope into the existing window.dispatchEvent shapes — same
event names, same payload contracts — so tiles render as each MCP tool
returns instead of in one end-of-turn batch.

Per-envelope payload shapes (mirrors existing handlers, no DashboardLayout
changes):
- visualization → tethysdash:add-visualization {batch:true, panels:[panel]}
  with the per-panel construction inlined from the end-of-turn block at
  lines 928-955 (resolveVisualizationUrl, client_custom_remote initialData,
  inlineData branch). Single-item batch preserves the stale-ref-doc boundary.
- layer_update → tethysdash:update-visualization FLAT shape
  {uuid: map_uuid, operation: 'append_layers', layers: [layer]}. Matches the
  existing single-event dispatch at lines 988-994 and the handler's
  append_layers branch in DashboardLayout.js which reads detail.uuid and
  detail.layers directly. NOT batched (handler does not look at detail.updates).
- patch_update → tethysdash:update-visualization
  {batch:true, operation:'apply_patch', patches:[{uuid, ops}]}. Mirrors the
  existing end-of-turn patch dispatch at lines 1066-1088.

Defense-in-depth gates inside the callback:
- controller.signal.aborted check (engine also gates, but a stale awaited
  result could land here mid-abort).
- capturedTurnId !== turnIdRef.current check (user submitted a new turn
  while this one's tools were in flight).
- No requestAnimationFrame — synchronous dispatch only per
  raf-timing-race-layer-dispatch-stale-ref solution doc.

End-of-turn no-op sentinel (streamedDispatchFiredRef):
- Reset to false at turn start (alongside setLoading(true)).
- Set to true by the callback on first invocation.
- Read by the three end-of-turn dispatch sites at lines ~928, ~983, ~1072 to
  skip dispatch when streaming already fired (R10). pendingVisualizations[],
  pendingLayerUpdates[], pendingPatches[] still flow through for
  dispatchBanner / whitelistWarning / onResult — only the dispatch is
  suppressed.

Backward compatibility: legacy hosts that don't supply onToolEnvelope
(undefined) see streamedDispatchFiredRef stay false and the existing
end-of-turn dispatch fires unchanged.

Phase 1 of docs/plans/2026-05-28-002-feat-chatbox-core-incremental-tile-streaming-plan.md
Unit 2 (Units 2+4 merged per scope-guardian review pass 2).

Note: a Chatbox.streaming.test.jsx file was drafted to drive the send flow
and assert window.dispatchEvent shapes, but the existing component-test
harness doesn't support the send-flow path (per the comment at the top of
Chatbox.test.jsx). The contract is covered at the engine layer by Unit 1's
onToolEnvelope.test.js (10 tests) and by manual smoke at PR time. A
follow-up could extract the per-tool dispatch logic to a helper that's
unit-testable in isolation.
…ract

Unit 3: Dispatches two new window CustomEvents around the chatbox turn
lifecycle so a host (DashboardLoader in tethysdash) can flip an isStreaming
flag and gate per-tile edit/delete affordances during streaming:

- tethysdash:turn-start fires immediately AFTER abortRef.current = controller
  is assigned. Inline INVARIANT comment names this sequencing so a future
  refactor doesn't silently move the dispatch above the assignment — a Stop
  click landing in the dispatch tick must reach a live controller, not null.
- tethysdash:turn-end fires from the shared finally block after
  setLoading(false). Single dispatch site covers success, thrown error,
  user-initiated abort, and /clear paths uniformly because they all converge
  in the same finally.

Events carry no payload; listeners toggle a boolean. Additive observable
surface — no existing event behavior changes, no chatbox-core public
component-prop changes.

Unit 5: Documents the existing Stop UX contract (R8a). The current
`result.aborted ? (accumulatedContent || "(Stopped)") : (result.assistantText ...)`
shape at the message-append site already implements the brainstorm's R8a
because the engine returns `{aborted: true, ...}` rather than throwing on
abort, so the stop path routes through the success branch and naturally
appends via setMessages. setError stays inert → ChatErrorPanel does NOT
render on Stop. Real errors land in the catch branch which calls setError
and surfaces via ChatErrorPanel — the two surfaces are visually distinct as
required by R8a. Added a comment block at the content-shaping site naming
the contract so the existing behavior is recognized as intentional rather
than incidental and survives future refactors.

Phase 1 of docs/plans/2026-05-28-002-feat-chatbox-core-incremental-tile-streaming-plan.md
Units 3 + 5.

679/679 tests pass. Dist rebuilt clean.
CHANGELOG entry documents the two new observable surfaces (onToolEnvelope
engine callback option + tethysdash:turn-start/turn-end window events) plus
the streamedDispatchFiredRef sentinel that prevents double-dispatch.

Bumps to 0.16.0-beta.0 on the beta dist-tag — minor bump appropriate for
additive observable API surface with no breaking changes. Adjust before
publishing if a different version is preferred.
@romer8
romer8 merged commit 023d648 into main May 28, 2026
2 checks passed
@romer8
romer8 deleted the feat/incremental-tile-streaming branch May 28, 2026 21:47
romer8 added a commit to Aquaveo/ciroh_devcon_2026_workshop that referenced this pull request May 28, 2026
…aming

The v2026.05.6 image (built by publish-image.yml on tag push of
v2026.05.6, ghcr.io/aquaveo/ciroh-devcon-2026:v2026.05.6) bakes
tethysdash @ tip of feature/tethysdash-mcp-server which carries:

  * @aquaveo/chatbox-core 0.16.0-beta.0 — adds the per-tool
    onToolEnvelope engine callback + tethysdash:turn-start /
    tethysdash:turn-end window CustomEvents (Aquaveo/chatbox-core#45).
  * tethysdash StreamingContext + DashboardLoader listener +
    DashboardItem edit/delete/reorder gates
    (tethysplatform/tethysapp-tethys_dash#134).

User-visible behavior changes for workshop participants:
  - Chatbox-created tiles appear incrementally as each MCP tool returns
    (instead of all at once at end-of-turn).
  - Per-tile edit/delete/order dropdown items are disabled during chatbox
    turns with a "Editing disabled while dashboard is updating" tooltip.
  - Layout operations (resize / drag-to-move) stay enabled during turns.
  - Stop button preserves successfully-rendered tiles.

Definitions.md adds a "What you'll notice in the chatbox (image v2026.05.6+)"
section pre-emptively explaining the two visible behaviors so participants
don't read them as bugs.

Participants run `bash scripts/update.sh` on their VMs to pull the new
image. update.sh's existing auto-sync logic propagates the new IMAGE_TAG
from .env.example into their .env, then `docker compose pull` fetches
v2026.05.6 from ghcr and `docker compose up -d --force-recreate` restarts
the stack against the new image. Hard-refresh the browser to load the
new React bundle.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant