feat: backend devcontainer + experimental MCP server + chatbox-core integration - #115
Draft
romer8 wants to merge 217 commits into
Draft
feat: backend devcontainer + experimental MCP server + chatbox-core integration#115romer8 wants to merge 217 commits into
romer8 wants to merge 217 commits into
Conversation
- Update .gitignore to include generated client plugin files - Enhance package.json with pre-collect, pre-start, and pre-build scripts - Implement dynamic panel creation in DashboardLayout for embedded plugins - Introduce ClientModuleLoader for loading React components from npm-installed client plugins - Modify AppLoader to merge npm-installed client plugins into the visualization list - Create collectClientPlugins script to discover and register client plugins - Update utilities for handling client custom visualizations
…ity for panel positioning
…panel positioning
- Added babel configuration for JSX syntax support. - Updated package dependencies, including @babel/preset-react to version 7.28.5. - Integrated @chatbox/core component into ChatSidebar, replacing dynamic federated component loading. - Modified Header component to conditionally render chatbox configuration. - Updated Tethys app settings to use new chatbox Ollama host and key settings. - Implemented Ollama proxy endpoints to handle requests to the chatbox API.
- Updated package.json to include @huggingface/transformers and fixed chatbox dependency path. - Modified DashboardLayout.js to initialize panel positions correctly and added checks for grid item results. - Updated ChatSidebar.js to clarify LLM provider configuration management. - Changed runtimePluginRegistry.json to update plugin ID and description. - Removed obsolete chatbox Ollama host settings from app.py and adjusted chatbox configuration in controllers.py. - Enhanced MCP server with CORS support and added new visualization tools with detailed descriptions. - Updated render functions to improve plugin rendering and discovery processes.
- Remove source="nwmps" example from parameter description and docstring - Remove source="my_plugin" example from create_variable_input docstring - LLMs copied these example values verbatim instead of calling list_intake_plugins
…atch events Add validGridItems deduplication layer to prevent duplicate i keys from corrupting react-grid-layout's compact() function. Coerce layout values to proper types with minH/minW minimums. Use gridItemsUpdated ref in updateLayout to avoid stale closures during React 18 batched state updates. Extend batch event format to support per-panel source with backward-compatible fallback. Bump create_variable_input defaults to w=25, h=12 for better sizing.
MCP tools create_text, create_custom_image, and create_map_visualization used synthetic "Inline *" source names that don't exist in the Default visualization registry. This prevented the edit modal from resolving the visualization type — findVisualizationBySource returned null. Change source names to match the Default group entries: - "Inline Text" → "Text" - "Inline Image" → "Custom Image" - "Inline Map" → "Map" This follows the pattern create_variable_input already uses correctly with source "Variable Input". Rendering is unaffected — Base.js routes by args.inlineData/vizType, not source name.
…eoJSON Map.js passed raw lon/lat degrees to an EPSG:3857 view, centering maps in Africa. Now detects EPSG:4326 coordinates via range heuristic and transforms via fromLonLat/transformExtent before setCenter/fit. loadGeoJSON returned early for object inputs, skipping CRS assignment. ModuleLoader then crashed accessing crs.properties.name on undefined. Now runs checkForCRS for all inputs regardless of type.
… drawing, variables
Redesign the create_map_visualization MCP tool for better LLM ergonomics:
- Smart defaults: streets base map, zoom 12, layer control off
- High-level markers param: [{lon, lat, label}] auto-builds GeoJSON
VectorLayer with correct CRS (EPSG:4326)
- Nominatim geocoding: optional 'center' param accepts place names
(e.g., "Salt Lake City") and geocodes to coordinates
- Drawing tools: drawing_tools, drawing_limit, drawing_variable params
for interactive geometry capture with dashboard variable publishing
- Variable support: extent_variable for map extent publishing,
${var} syntax passthrough in layer args documented
- Expanded base maps: added terrain and ocean shorthands (7 total)
- All extent coordinates include projection metadata for the frontend
- Auto-centers on first marker when no extent or center is provided
…tom Image
The frontend has dedicated rendering short-circuits for source "Map",
"Text", and "Custom Image" (utilities.js:99-123) that read args at the
top level. The inlineData wrapper bypassed these paths, making the edit
modal unable to read or modify the visualization config.
- create_text: inlineData → args: {text}
- create_custom_image: inlineData → args: {image_source} (matches
frontend's itemData.args.image_source at utilities.js:119)
- create_map_visualization: inlineData → args: {baseMap, layers, ...}
- Added create_map_visualization to BM25SearchTransform always_visible
so the LLM can discover it without calling search_tools first
Plotly, Table, and Card keep inlineData — they have no frontend
short-circuit and require the inline rendering path.
The edit modal's checkAllInputs treats null values as "unfilled" and blocks save with "All arguments must be filled out". mapDrawing=null and map_extent=null triggered this when editing chatbox-created maps. Build the args dict conditionally — only include map_extent and mapDrawing when they have actual values. Remove mapConfig entirely (not a registry arg — frontend generates it automatically). Matches the format the manual UI stores when creating maps.
Split create_map_visualization into two focused tools:
- create_map_visualization: simplified (center, markers, drawing, base map).
Generates UUID for cross-tool referencing. No raw layers param.
- add_map_service_layer: constructs layer configs from flat params with
required field validation per source type (WMS, ESRI, GeoJSON, KML, etc.).
Supports queryable + attributeVariables for click-to-update workflows.
Add tethysdash:update-visualization DOM event in DashboardLayout so layers
can be appended to any existing map grid item by UUID — works for maps
created in the same conversation or from previous sessions.
UUID flows through the dispatch chain: MCP generates it, Chatbox passes
it through panels, DashboardLayout uses panel.uuid || uuidv4().
Both tools added to BM25SearchTransform always_visible list.
render_plugin description emphasizes ${variable_name} linkage syntax.
handleUpdateVisualization now accepts layers (array) via operation "append_layers" instead of a single layer. All layers for the same map are pushed in one updateTab call, avoiding the stale-ref problem where multiple sequential events read outdated gridItemsUpdated.current. Same class of fix as the batch dispatch solution for add-visualization.
…rcion Three fixes: - Resolve ESRI service layer name for attributeVariables key so the click-time lookup matches the /identify response (matches manual UI) - Update gridItemsUpdated ref immediately in handleAddVisualization and handleUpdateVisualization as defense-in-depth against rAF timing races - Coerce JSON string arguments to dicts for params, geojson, and attribute_variables (some LLM providers serialize objects as strings)
…p-server # Conflicts: # package-lock.json # package.json # reactapp/components/dashboard/DashboardLayout.js # reactapp/components/visualizations/Base.js # reactapp/components/visualizations/utilities.js
…rcion Three changes to add_map_service_layer: - Place GeoJSON data at source.geojson (top-level) instead of source.props.geojson, matching the manual UI format that loadLayerJSONs and ModuleLoader.loadGeoJSON both expect - Add geojson_url parameter for externally-hosted GeoJSON files — the frontend fetches the URL at render time via the existing loadGeoJSON URL branch in utilities.js - Coerce JSON string arguments to dicts for params, geojson, and attribute_variables (some LLM providers serialize objects as strings)
Extends create_variable_input to support all frontend variable input types: dropdown (with options array), slider (with min/max/step/ outputFormat metadata), date-range, and csv-uploader. Includes LLM string coercion for the options parameter and safe float parsing for slider initial values.
…tions Adds documentation for: - MCP server architecture, tools table, and visualization data flow - All 7 MCP data contract rules (inlineData vs args, source names, tool split, per-type validation, ESRI attr key, GeoJSON placement, dict coercion) - ChatSidebar integration with @chatbox/core engine - docs/solutions/ knowledge store reference - docs/plans/ and docs/brainstorms/ workflow artifacts - Key conventions: no early returns, batch dispatch, no example values
167 tests covering: - 8 viz-creating tools (rendering path, source name, args shape) - 9 map layer source types (config structure, GeoJSON placement, ESRI key) - 8 variable input subtypes (dotted key metadata, slider outputFormat) - Dict parameter coercion (string and dict inputs) - Error paths (missing required fields, invalid types) Validates the 7 data contract rules documented in docs/solutions/. Runs in <1s with no DB or server dependency.
- playwright.config.js: webServer starts Django, workers:1, mocked/integration projects - helpers/db.js: better-sqlite3 wrapper for SQLite fixtures (createDashboard, truncateAll) - helpers/dashboards.js: composable fixture builders for all viz types and map layers - helpers/mocks.js: route interception for tiles, API responses, auth endpoints - setup-test-db.py: one-time setup for Tethys SQLite persistent store - visualizations.spec.js: first E2E test (Text panel rendering) - package.json: added @playwright/test, better-sqlite3, Jest testPathIgnorePatterns for e2e/ Requires: npx playwright install-deps chromium (system libraries)
…lause
Debug session 2026-05-21 turn 2 with gemini-flash: asked to 'Enable the
Custom Popup Modal on the China Flowlines layer...' (a FIRST-TIME
popup-modal setup), the LLM routed to patch_visualization instead of
configure_popup_modal_layer, then fabricated:
- wrong path: /args/layers/0/configuration/props/popup
(canonical is /args/layers/0/popupConfig)
- wrong value shape: {content, title, type}
(canonical is {mode, position, titleTemplate, gridItems})
Root cause: the PRIORITY clause in beforeFirstMessage's system content
sends 'modify / add to / update an existing tile' → patch_visualization.
'Enable popup on an existing layer' reads as modify-existing. The new
configure_popup_modal_layer tool had no carve-out, so it lost to
patch_visualization in the LLM's routing.
Fix: extend the existing 'add_*_layer for new layers on existing maps'
exception into a list of TWO exceptions:
(a) adding a new layer to an existing map → add_*_layer (unchanged)
(b) configuring a custom popup modal on an existing map layer for the
FIRST TIME (the layer exists, but the layer has no popupConfig
yet) → configure_popup_modal_layer, never patch_visualization
Plus an explicit boundary: patch_visualization is correct ONLY for
PARTIAL EDITS to an already-existing popupConfig (e.g., changing just
the titleTemplate via /args/layers/N/popupConfig/titleTemplate).
The tool-side fix is in tethysdash_mcps PR #10 (rewrites the
configure_popup_modal_layer description to lead with positive
first-time-setup framing). Both sides must land for the routing to
work end-to-end.
New test: PRIORITY rule preserves the configure_popup_modal_layer
exception for first-time popup-modal setup. Pins the wording so future
edits can't silently drop the carve-out.
Jest suite: 19/19 ChatSidebar tests pass.
…when args contain ${feature.*} tokens
Closes the deferred Unit 5 UX bug from the 2026-05-21 popup-modal debug
arc. When the Edit Visualization modal is opened on a popup gridItem
whose args carry unresolved `${feature.<key>}` tokens (no feature is
selected in the preview context), the preview pane was firing
getVisualization with the raw literal token. The plugin rejected the
unresolvable arg and the preview rendered "Failed to retrieve data" —
a confusing UX that looked like a configuration error when actually
the preview is waiting for runtime feature selection.
Base.js already had this short-circuit (Base.js:441-460, shipped with
the popup-modal feature): findUnresolvedFeatureTokens detects literal
${feature.*} tokens in resolved args and sets vizType="featurePending",
which Base.js's Visualization component renders as a friendly
"Awaiting feature selection" placeholder. VisualizationPane.js — the
parallel Edit Visualization preview path — was missing the same gate.
This commit ports the identical short-circuit into VisualizationPane.js
just before the getVisualization call:
- Imports findUnresolvedFeatureTokens from components/visualizations/utilities
- After resolving variable-input substitutions, scans the resolved
args for unresolved ${feature.*} tokens
- If any are present: setVizType("featurePending") + setVizData with
source (display label) + pendingTokens, then return without firing
getVisualization
- DataViewer's preview pane reuses the same <Visualization> component
from Base.js (DataViewer.js:22, 400), so the existing featurePending
shell renders automatically — no UI work needed in DataViewer.
source field in setVizData uses selectedVizTypeOption["source"] (the
display label) to match Base.js's gridItemSource pattern, so the
pending hint renders "Custom Image renders when a feature is clicked
on the map" or similar.
Test: VisualizationPane Pane short-circuits to featurePending when args
contain ${feature.*} tokens. Uses Custom Image source with
image_source containing a literal "${feature.comid}" token. Asserts:
(1) setVizType called with "featurePending" before getVisualization
would have fired, (2) setVizData includes pendingTokens listing
"feature.comid", (3) negative: setVizType never called with "image"
(the success path that would have fired if getVisualization had run).
Jest suite: 20/20 VisualizationPane tests pass (19 existing + 1 new).
No regression to other DataViewer paths.
The CIROH DevCon 2026 workshop image (ghcr.io/aquaveo/ciroh-devcon-2026:...) ships the tethysdash app, and the workshop demos depend on four FIRO-Tethys plugins being available out-of-the-box so participants don't have to pip install anything during the 2-hour session. Added one RUN layer after the main tethysdash install that pip-installs all four plugins from git, branch-pinned to @main per the workshop's branch-track convention (rebuilding the image close to event day picks up plugin fixes without re-pinning): - FIRO-Tethys/ciroh_plugins - FIRO-Tethys/tethysdash_plugin_cnrfc - FIRO-Tethys/tethysdash_examples - FIRO-Tethys/RFC-Plugins Each is a tethysdash plugin that brings its own intake driver via setup.py entry-points; no separate intake / intake-* install needed. Next step (manual, outside this commit): rebuild + push ghcr.io/aquaveo/ciroh-devcon-2026:<next-tag> so the workshop compose service picks up the new plugin catalog.
…in image" This reverts commit f56fa45.
Cascades the chatbox-core denylist-lift release (Aquaveo/chatbox-core@0.15.0, commits 59e0fc0 + 568bf68, tagged v0.15.0, CI-published 2026-05-21) into the tethysdash workshop image build. Two changes: 1. .devcontainer/Dockerfile sed-swap pin bumped from @aquaveo/chatbox-core@0.6.4 → @aquaveo/chatbox-core@0.15.0. This is a forward upgrade through the stable line (0.6.4 → 0.7.0 → 0.8.0 → ... → 0.14.0 → 0.15.0). Workshop image rebuild will pull the new version from npm; per project_tethysdash_devcontainer_multistage_ shipped.md the sed-swap fires before npm install in the multi-stage build, so the build resolves @aquaveo/chatbox-core@0.15.0 from npm instead of the local file: link. 2. tethysapp/tethysdash/public/frontend/ rebuilt by webpack against the local file: link (which symlinks to ../lib/chatbox-core, currently at v0.15.0). Old hashed bundle main.0f9d130a8a4f0652bd62.js replaced by main.c60861e5afe3620bdd0d.js; manifest.json updated to match. The committed bundle is the dev artifact — when participants build the workshop image, the Dockerfile rebuilds the bundle inside the image from sed-swapped npm-installed source (different hash, equivalent behavior). Net behavior change in the workshop image after this commit + a workshop image rebuild: Ollama Cloud catalog filter no longer hides qwen / deepseek / glm / chatglm / yi / baichuan / ernie / hunyuan / minicpm / xverse / internlm / skywork / kimi / minimax families. Operator-policy carve-out shape preserved (BLOCKED_MODEL_PREFIXES is frozen-empty in chatbox-core 0.15.0).
…rature drop) Cascades the chatbox-core fix (Aquaveo/chatbox-core@0.15.1, commit e13ef0c + bump 4735b5f, tagged v0.15.1, CI-published 2026-05-22) into the tethysdash workshop image build. Two changes: 1. .devcontainer/Dockerfile sed-swap pin bumped from @aquaveo/chatbox-core@0.15.0 → @aquaveo/chatbox-core@0.15.1. Workshop image rebuild will pull the new version from npm; the sed- swap fires before npm install in the multi-stage build, so the build resolves @aquaveo/chatbox-core@0.15.1 from npm instead of the local file: link. 2. tethysapp/tethysdash/public/frontend/ rebuilt by webpack against the local file: link (which symlinks to ../lib/chatbox-core, currently at v0.15.1 source with the anthropic temperature drop). Old hashed bundle main.c60861e5afe3620bdd0d.js replaced by main.fed9c822e1af905e559b.js; manifest.json updated to match. Net behavior change after this commit + a workshop image rebuild: Anthropic Claude 4.x models (claude-opus-4-7 and siblings) stop returning HTTP 400 `temperature is deprecated for this model` on every request. The chatbox-core Anthropic adapter no longer emits a temperature key in streamParams; older Claude 3.x models that still accept temperature ignore it on reasoning-style calls anyway. Next: tag a new workshop image version (e.g., v2026.05.2) on Aquaveo/ciroh_devcon_2026_workshop so the workshop image rebuilds against the new chatbox-core. The two-step CI build will pick up 0.15.1 via the sed-swap automatically.
…sponses-API graceful-block)
Cascades the chatbox-core fix (Aquaveo/chatbox-core@0.15.2, commit
8cdfdcc + bump c6ddc7b, tagged v0.15.2, CI-published 2026-05-22) into
the tethysdash workshop image build.
Two changes:
1. .devcontainer/Dockerfile sed-swap pin bumped from
@aquaveo/chatbox-core@0.15.1 → @aquaveo/chatbox-core@0.15.2.
2. tethysapp/tethysdash/public/frontend/ rebuilt by webpack against
the local file: link (which symlinks to ../lib/chatbox-core at
v0.15.2 source). Old hashed bundle main.fed9c822e1af905e559b.js
replaced by main.27f5be4ec7eaf653a657.js; manifest.json updated
to match.
Net behavior change after workshop image rebuild:
- OpenAI gpt-5.4-mini (and other GPT-5.x): no longer 400s with
`max_tokens is deprecated` - request uses `max_completion_tokens`
now and temperature is no longer sent.
- OpenAI gpt-5.1-codex-mini (and codex/o1-pro/o3-pro variants): the
model picker dropdown now disables these with ⚠️ prefix and
"(Responses API only)" suffix; tooltip explains the limitation.
If a participant has one persisted from before, the adapter
short-circuits with a structured "responses_api_required" error
instead of letting the SDK 404 with the raw API message.
Next: tag v2026.05.3 on Aquaveo/ciroh_devcon_2026_workshop to rebuild
the workshop image against the new chatbox-core. The two-step CI
build will pick up 0.15.2 via the sed-swap automatically.
… rebuild bundle - New llm_proxy_google endpoint: forwards arbitrary paths under /apps/tethysdash/llm-proxy/google/<rest>/ to https://generativelanguage.googleapis.com/<rest>. Reads X-Goog-API-Key from request headers. Uses regex='.+' so colon and slash in paths (e.g., 'v1beta/models/gemini-2.5-flash:streamGenerateContent') match. - Rebuilt React bundle with @google/genai SDK-based Gemini adapter.
…-compat (#133) * feat(controllers): proxy route for Ollama /v1/chat/completions OpenAI-compat Companion to chatbox-core feat/ollama-local-openai-compat-preset. Adds the Django proxy route the new "Ollama (local)" preset depends on: /apps/tethysdash/ollama-proxy/v1/chat/completions/ → ${X-Ollama-Host}/v1/chat/completions The new route mirrors the existing /ollama-proxy/api/{tags,show,chat}/ routes (same _proxy_to_ollama helper, same X-Ollama-Host/X-Ollama-Key header contract), but forwards to Ollama's OpenAI-compatibility endpoint instead of /api/chat. Why a new endpoint instead of swapping /api/chat: newer Ollama versions rejected the existing chat path's tool_call argument format mid-turn ("cannot unmarshal object into Go struct field ... of type string"), while older versions reject the OpenAI-spec string format. The two versions can't be satisfied by a single static wire format on the native endpoint. The OpenAI-compat endpoint follows OpenAI spec stably across versions. See chatbox-core PR #23 follow-up note for the architectural rationale. The Ollama Cloud preset (existing /ollama-proxy/api/chat/ route) is unchanged — Cloud's wire format is stable. Includes test_ollama_v1_chat_completions_proxy_forwards_to_openai_compat_path asserting the upstream URL ends with /v1/chat/completions and respects the X-Ollama-Host header. * chore(devcontainer): bump @aquaveo/chatbox-core 0.15.2 → 0.15.3 0.15.3 ships the new `ollama_local` provider preset (Aquaveo/chatbox-core#44 merged 2026-05-28). The companion proxy controller for /ollama-proxy/v1/chat/completions/ is in the previous commit in this PR. Without bumping the npm pin, the devcontainer base image (which the workshop image layers on top of) ships with chatbox-core 0.15.2 and the new "Ollama (local)" preset is missing from the chatbox provider dropdown.
…er gates (#134) * feat(loader): StreamingContext bridges chatbox-core turn events Add a dedicated StreamingContext (Contexts.js) for the chatbox-driven isStreaming flag. DashboardLoader.js owns the state, listens for the new tethysdash:turn-start / tethysdash:turn-end window events from @aquaveo/chatbox-core@>=0.16.0-beta.0, and provides via the new context. isStreaming lives in its own useState — separate from disabledEditingMovement — so the existing reset effect at lines 75-79 (which zeroes disabledEditingMovement when isEditing flips false) does NOT zero isStreaming spuriously. Dedicated context (rather than extending DisabledEditingMovementContext) keeps the re-render footprint to DashboardItem consumers only. The existing 4+ consumers of DisabledEditingMovementContext (Header, DashboardLayout, PopupLayoutEditor, DashboardItem) do NOT re-render on every turn boundary — addresses the fan-out concern flagged in the brainstorm/plan reviews. Test additions: - DashboardLoader.streaming.test.js (6 scenarios): initial-mount default false, turn-start flips true, turn-end flips back, idempotent duplicate starts, independence from DisabledEditingMovementContext, listener cleanup on unmount. - customRender.js adds StreamingPComponent helper for asserting context value in tests. Phase 2 of docs/plans/2026-05-28-002-feat-chatbox-core-incremental-tile-streaming-plan.md Unit 6. * feat(dashboard): gate per-tile edit/delete/reorder during chatbox streaming DashboardItem reads isStreaming from the new StreamingContext (Unit 6) and short-circuits the three config-mutating handlers when true: - editGridItem: guard fires before setShowDataViewerModal — the edit modal does not open at all while the chatbox is mid-turn. - deleteGridItem: guard fires BEFORE confirm() — the delete confirmation modal does not appear (rather than appearing and then no-op'ing on acceptance, which would be confusing UX). - updateGridItemOrder: guard fires before mutating gridItems. Covers the per-tile reorder affordances (Bring to Front / Forward / Send to Back / Backward in the dropdown) which all delegate here. The react-grid-layout drag-to-reorder gesture is NOT routed through this function, so move / resize / drag remain enabled per R6. DashboardItemDropdown propagates isStreaming to its menu items: - Edit / Delete / Order entries: render with the Bootstrap `disabled` prop (rendered-but-disabled, NOT hidden — keeps affordance location stable) plus a `title` tooltip ("Editing disabled while dashboard is updating"). - Copy / Export remain enabled — they don't mutate tile config and don't conflict with chatbox patch_visualization. Open edit modals at stream start are NOT auto-closed (accepted v1 UX cost per Scope Boundaries). Tests: DashboardItem.streaming.test.js (5 scenarios) covers the disabled- class + tooltip + handler no-op contract for Edit and Delete, the enabled state of Copy/Export under streaming, the false→true→false transition, and normal (non-streaming) baseline behavior. Note on a11y: react-bootstrap may or may not apply `aria-disabled='true'` depending on version; the `disabled` class is the jsdom-stable assertion. Production a11y verification (screen reader announces "Edit, disabled" with the documented reason) is worth checking after PR lands. Phase 2 of docs/plans/2026-05-28-002-feat-chatbox-core-incremental-tile-streaming-plan.md Unit 7. * chore: bump @aquaveo/chatbox-core 0.15.3 → 0.16.0-beta.0 + rebuild bundle Devcontainer Dockerfile sed-swap updated so the multi-stage build pulls the new beta from npm. The workshop image (built from this Dockerfile) will pick up the new onToolEnvelope engine option and the two new tethysdash:turn-start / tethysdash:turn-end window events that Phase 2 listens for. React bundle rebuilt against the current chatbox-core file: link (which itself is at 0.16.0-beta.0). The committed bundle keeps the non-Docker / dev-server / Jest paths in sync with the source. The Docker multi-stage build overwrites this bundle at image-build time anyway — both paths produce the same code now.
…uild bundle (#135) Picks up chatbox-core PR #46 (available_cache_uris hint on error envelopes). When an MCP tool returns {error: ...}, the engine now appends available_cache_uris: [{tool_name, cache_uri}, ...] so small models don't re-query NRDS after a chart-tool failure — the URI is in immediate context. Files: - .devcontainer/Dockerfile sed-swap pin: 0.16.0-beta.0 → 0.16.1-beta.0. - React bundle rebuilt: main.215c... → main.2314ae... - manifest.json updated.
romer8
marked this pull request as draft
July 10, 2026 16:56
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three additions, scoped intentionally. Two production-grade (devcontainer + chatbox-core wiring) and one experimental / work-in-progress (the MCP server). The experimental piece is gated behind opt-in editor permissions and a separately-running localhost service; it does not change any default user-facing behavior.
1. Backend-only devcontainer (
.devcontainer/)A self-contained dev environment for Python / Django / Tethys work. Pip-only (no conda), SQLite (no Postgres), no Node / Webpack. The frontend is served from the bundled
tethysapp/tethysdash/public/frontend/assets committed in the repo.Files:
.devcontainer/Dockerfile— Python base + system deps for Tethys..devcontainer/devcontainer.json— VS Code Dev Containers / Codex config; declarespostCreateCommand+postStartCommand..devcontainer/init.sh— idempotent first-boot orchestrator. Calls the two scripts in order, gates on a marker file under${TETHYS_PERSIST}..devcontainer/scripts/tethyscore.sh— generatesportal_config.yml, runstethys db migrate, creates the defaultadmin/adminsuperuser. Mirrors the role oftethys_portal_firo/docker/tethyscore.sls..devcontainer/scripts/devcontainer-app.sh— provisions the SQLite persistent store, links it totethysdash:ps_database:primary_db, runstethys syncstores tethysdash. Mirrorstethys_portal_firo/docker/salt/init_apps.sls..devcontainer/run.sh— convenience wrapper fortethys manage start -p 0.0.0.0:8000..devcontainer/README.md— how to use it + the rationale for choosing bash idempotence over Salt (pip-distributed Salt has incompleteinstall_requireson slim Python images, and the SaltProject apt repo is not reliably reachable from all build environments).If you need to do React work, the existing conda-based workflow in the top-level
README.mdis unchanged.2. Experimental MCP server (
tethysapp/tethysdash/mcp/) — WIPA FastMCP-based Model Context Protocol server that exposes dashboard-creation, visualization-rendering, and tile-modification capabilities as 25 typed tools + 25 slash-command prompt templates. The intent is to let an LLM (configured in the chatbox sidebar) build / edit dashboards via tool calls instead of point-and-click — for users who want it.
Areas added:
tethysapp/tethysdash/mcp/tethysdash_mcp_server.py— the FastMCP server (25@mcp.tool+ 25@mcp.promptdefinitions). Streamable HTTP transport at/mcpis the default; legacy SSE at/sseis opt-in viaMCP_TRANSPORT=ssefor migrating users. Host is127.0.0.1by default; setMCP_HOST=0.0.0.0only when fronting with an authenticated reverse proxy. CORS is env-driven (ALLOWED_ORIGINS, default*;ALLOW_CREDENTIALSis auto-derived to avoid thecredentials:true + origin:*spec violation).tethysapp/tethysdash/mcp/_input_validation_middleware.py— converts PydanticValidationErrors on tool input into a typed envelope withexpected_kwargsand afix_hint, so an LLM gets a single-retry recovery path instead of an opaque trace.tethysapp/tethysdash/mcp/_observability_middleware.py— per-tool-call structured logs.tethysapp/tethysdash/tests/mcp/— ~140 contract tests across 14 files covering tool output shapes, prompt-to-tool routing, validation envelopes, transport / CORS, and per-source-type layer-tool inputs.pyproject.toml:pydantic,python-dateutil,pytz,jsonpatch==1.33. Optional test extra:fastmcp.The MCP tool surface intentionally covers the same capabilities as the existing Edit Visualization modal — it does not unlock any privileged operations. Authorization piggybacks on the editor-only sidebar mount and the localhost-bound transport.
3. Chatbox-core library reference (
@chatbox/core)The in-app chat sidebar (
reactapp/components/sidebar/ChatSidebar.js) consumes the generic @aquaveo/chatbox-core engine, which lives outside this repo. The engine handles the LLM tool-use conversation loop, MCP wire transport, model capability detection, slash-prompt popover, and per-dashboard chat persistence.package.json:The
file:link is the workspace-aware dev path (chatbox-core sits as a sibling subrepo so tethysdash and chatbox-core can co-evolve). For stable downstream consumption an npm install of@aquaveo/chatbox-core(currently on thebetadist-tag) is the equivalent — the import paths under@chatbox/core/...move to@aquaveo/chatbox-core/....ChatSidebar.jsuses a generic system prompt and passes noengineExtensions, so the engine runs with its defaults. Variable input values published by the sidebar feedVariableInputsContextso MCP-created variable inputs integrate cleanly with the existing dashboard interactivity system.Playwright E2E suite — archived (not in this PR)
The Playwright E2E suite (36 mocked tests + 4 skipped) was archived to the
aquaveoremote at brancharchive/playwright-suite-2026-05-11and removed from the working tree before opening this PR. Rationale: the suite was paying more in UI-churn maintenance + perceived flake risk than it returned in caught bugs (0 documented Playwright-caught bugs that Jest missed). The full state of the suite — every spec, helper, the playwright.config.js, and the SQLite fixture script — is preserved at the exact state it would have been if we'd merged the suite upstream. To restore:git checkout aquaveo/archive/playwright-suite-2026-05-11 -- reactapp/__tests__/e2e/ reactapp/playwright.config.jsand re-add the three devDeps (@playwright/test,better-sqlite3,pg).The decision-making framework for whether and how to re-introduce Playwright lives at
docs/brainstorms/2026-05-11-tethysdash-playwright-smoke-charter-requirements.mdin the firoh workspace. Future Playwright additions should clear the charter's R2.1 unique-coverage criteria + R2.2 stability bar + R4 forward-going reviewer gate before merging. Reviewers who want a Playwright surface upstream are explicitly invited to flag that here — the trim direction is being communicated up front, not imposed.What is intentionally NOT in this PR
Test plan
pytest tethysapp/tethysdash/tests/mcp/— ~140 MCP contract tests pass.pytest tethysapp/tethysdash/tests/unit_tests/— existing unit tests + newtest_plugin_helpers,test_cli,test_plugin_registry_loaderpass.npm test— Jest component suite passes (~800 tests).bash .devcontainer/run.sh, log in asadmin/admin, confirm dashboard list loads at/apps/tethysdash/.Status framing for reviewers