feat: backend devcontainer + experimental MCP server + chatbox-core integration

.claude/skills/test-all/SKILL.md

@@ -0,0 +1,55 @@
+---
+name: test-all
+description: Run the full TethysDash test sweep in a self-contained `.venv-test/` — the full Python suite (includes MCP contracts) plus Jest. Rebuilds the React bundle first. Writes a pass/fail report. First run with a cold venv takes 2-5 minutes; subsequent runs are faster.
+argument-hint: (none)
+---
+
+Run the full TethysDash test sweep. Write results to `test-results/reports/<ISO-timestamp>-test-all.md`.
+
+On preflight, bootstrap, or bundle-build failure, STOP and tell the user exactly what to fix. On per-suite failure, continue to the next suite and aggregate results.
+
+1. **System preflight**. Verify these are on PATH: `python3`, `node`, `npm`. If any is missing, stop with remediation text naming the missing binary.
+
+2. **Venv bootstrap**. From the repo root:
+   - Compute `sha256sum pyproject.toml | awk '{print $1}'`.
+   - If `.venv-test/.pyproject-hash` matches, reuse. Otherwise recreate: `python3 -m venv .venv-test`, `.venv-test/bin/pip install --upgrade pip --quiet`, `.venv-test/bin/pip install -e ".[test]" --quiet` (the `[test]` extras pull in `fastmcp` so the MCP server imports cleanly), then write the new hash.
+   - On `pip install -e ".[test]"` failure: stop, surface stderr, exit non-zero.
+
+3. **Node deps**. From the repo root, if `package-lock.json` exists use `npm ci`; otherwise `npm install`.
+
+4. **Build the bundle**. `npm run build`. Failure here is fatal — stop.
+
+5. **Start report file** at `test-results/reports/<ISO-timestamp>-test-all.md`:
+   ```
+   # test-all run — <ISO-timestamp>
+
+   **Commit:** <git rev-parse HEAD>
+   **Python:** <.venv-test/bin/python --version> (.venv-test)
+   **Node:** <node --version>
+
+   ```
+
+6. **Full Python suite**. `.venv-test/bin/pytest --reuse-db --no-cov -q tethysapp/tethysdash/tests/`. Capture exit code and duration. This sweep covers MCP contracts, integrated tests, and unit tests.
+
+7. **Jest**. From the repo root: `npm test`. Capture exit code and duration.
+
+8. **Finalize report**. Append a summary table and, for any failing suite, the last ~50 lines of its output:
+   ```
+   | Suite        | Result | Duration | Count                 |
+   |--------------|--------|----------|-----------------------|
+   | python-suite | <pass|FAIL> | <d> | <N passed / M failed> |
+   | jest         | <pass|FAIL> | <d> | <N passed / M failed> |
+
+   **Result:** <PASS | FAIL (N of 2 suites failed)>
+   ```
+   Note: MCP contracts are implicit in `python-suite`; they are not surfaced as a separate row because the sweep runs them via the single pytest invocation.
+
+9. **Tell the user**: one-sentence summary plus report file path.
+
+10. **Exit**. Non-zero if any suite failed; zero otherwise.
+
+## Notes for maintainers
+
+- First run on a cold venv is 2–5 minutes; subsequent runs on a warm venv are roughly `pytest time + npm run build + npm test` (~1.5–3 min).
+- For day-to-day iteration on Python-only changes, prefer `test-backend` (fastest). For React-only iteration, run `npm test` directly — there's no dedicated front-end skill (Jest needs neither a venv nor a running Tethys server).
+- The Playwright E2E suite was archived in May 2026; see `CLAUDE.md` → "Playwright suite — archived" for the rationale and the `aquaveo` archive branch. If Playwright is reintroduced, this skill grows back a Playwright step gated by the charter at `docs/brainstorms/2026-05-11-tethysdash-playwright-smoke-charter-requirements.md` (firoh workspace).

.claude/skills/test-backend/SKILL.md

@@ -0,0 +1,49 @@
+---
+name: test-backend
+description: Run the full TethysDash Python pytest suite (MCP contracts + integrated + unit) in a self-contained `.venv-test/`. Python-only — no Node, no bundle, no server. Fastest skill. Writes a pass/fail report.
+argument-hint: (none)
+---
+
+Run the full Python test suite. Write results to `test-results/reports/<ISO-timestamp>-test-backend.md`.
+
+On preflight or bootstrap failure, STOP and tell the user exactly what to fix. On per-suite failure, continue to report finalization and exit non-zero.
+
+1. **System preflight**. Verify `python3` is on PATH (`command -v python3`). If missing, stop with: "ERROR: python3 not found on PATH. Install Python 3.10+ and retry." Do NOT check node/npm/tethys — this skill is Python-only.
+
+2. **Venv bootstrap**. From the repo root:
+   - Compute the target hash: `sha256sum pyproject.toml | awk '{print $1}'`.
+   - If `.venv-test/.pyproject-hash` exists and matches, reuse the venv (fast path).
+   - Otherwise: remove `.venv-test/` if present, create fresh with `python3 -m venv .venv-test`, then `.venv-test/bin/pip install --upgrade pip --quiet` and `.venv-test/bin/pip install -e ".[test]" --quiet`. The `[test]` extras pull in `fastmcp` (declared in `pyproject.toml` under `[project.optional-dependencies].test`) so the MCP contract tests can import. Write the new hash to `.venv-test/.pyproject-hash`.
+   - If `pip install -e ".[test]"` fails, stop and surface stderr to the user. Do not attempt any test run.
+
+3. **Start report file**. Create `test-results/reports/` if it does not exist. Write the report file with this header:
+   ```
+   # test-backend run — <ISO-timestamp>
+
+   **Commit:** <git rev-parse HEAD>
+   **Python:** <.venv-test/bin/python --version> (.venv-test)
+
+   ```
+
+4. **Python suite**. Run `.venv-test/bin/pytest --reuse-db --no-cov -q tethysapp/tethysdash/tests/` from the repo root. Capture exit code, wall-clock duration, and the full output.
+
+5. **Finalize report**. Append a summary table and, if the suite failed, the last ~50 lines of output:
+   ```
+   | Suite        | Result | Duration | Count            |
+   |--------------|--------|----------|------------------|
+   | python-suite | <pass|FAIL> | <duration> | <N passed / M failed> |
+
+   **Result:** <PASS | FAIL>
+   ```
+   Parse the pass/fail count from pytest's final summary line (`=== N passed, M failed in Xs ===`). If the suite failed, append a `## Failure details` section with the last 50 lines of output.
+
+6. **Tell the user**: one sentence summary plus the report file path.
+
+7. **Exit**. Non-zero if the suite failed; zero otherwise.
+
+## Notes for maintainers
+
+- The venv is cached. If someone edits `pyproject.toml` (adds/removes a dep), the next run will automatically recreate `.venv-test/`.
+- To force a rebuild without touching `pyproject.toml`, delete `.venv-test/` and re-run.
+- This skill covers `tests/integrated_tests/` and `tests/unit_tests/` — everything under `tethysapp/tethysdash/tests/`. (The MCP contract suite that used to live at `tests/mcp/` was moved to the standalone repo `Aquaveo/tethysdash_mcps` when the embedded MCP server was extracted.)
+- On a warm venv, a full green run today is ~25–60 seconds. Cold first-run with `pip install -e .` is 60–120 seconds plus suite time.

.devcontainer/Dockerfile

@@ -0,0 +1,63 @@
+# TethysDash devcontainer. Multi-stage: node:20-slim builds the React bundle;
+# python:3.11-slim runtime pulls it in. Node lives only in stage 1.
+
+# Stage 1: frontend bundle
+FROM node:20-slim AS frontend-builder
+
+WORKDIR /build
+
+COPY package.json package-lock.json ./
+
+# Swap `@chatbox/core` file: link → published npm alias; the sibling source
+# isn't in this build context. Bump version when chatbox-core releases.
+RUN sed -i \
+        's|"@chatbox/core": "file:\.\./lib/chatbox-core"|"@chatbox/core": "npm:@aquaveo/chatbox-core@0.16.1-beta.0"|' \
+        package.json \
+    && rm package-lock.json \
+    && npm install --no-audit --no-fund
+
+# Explicit COPYs (not `COPY . .`) so Python-only changes don't bust the npm layer.
+COPY babel.config.json jsconfig.json ./
+COPY reactapp/ ./reactapp/
+COPY tethysapp/tethysdash/public/ ./tethysapp/tethysdash/public/
+
+RUN npm run build
+
+# Stage 2: runtime (python:3.11-slim, pip-only)
+FROM python:3.11-slim
+
+ENV TETHYS_HOME=/root/.tethys
+ENV TETHYS_PERSIST=/root/.tethys
+ENV VIRTUAL_ENV=/opt/venv
+ENV PATH="${VIRTUAL_ENV}/bin:${PATH}"
+ENV PYTHONDONTWRITEBYTECODE=1
+ENV PIP_DISABLE_PIP_VERSION_CHECK=1
+
+# build-essential + libpq-dev for psycopg2-binary source fallback; rest is dev hygiene.
+RUN apt-get update \
+    && apt-get install -y --no-install-recommends \
+        build-essential \
+        libpq-dev \
+        git \
+        curl \
+        ca-certificates \
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*
+
+RUN python -m venv "${VIRTUAL_ENV}"
+
+WORKDIR /workspaces/tethysapp-tethys_dash
+
+COPY . .
+
+RUN pip install --no-cache-dir -e ".[test]"
+
+# Overlay the fresh bundle from stage 1, clobbering any stale main.js from `COPY . .`.
+COPY --from=frontend-builder \
+     /build/tethysapp/tethysdash/public/frontend/ \
+     /workspaces/tethysapp-tethys_dash/tethysapp/tethysdash/public/frontend/
+
+EXPOSE 8000
+
+# No CMD: devcontainer host keeps the container alive via overrideCommand;
+# contributor runs `bash .devcontainer/run.sh` to start the server.

.devcontainer/README.md

@@ -0,0 +1,121 @@
+# Backend-only devcontainer for TethysDash
+
+A self-contained dev environment for working on the Python / Django / Tethys side of TethysDash. Pip-only (no conda), SQLite (no Postgres), no Node / Webpack. The frontend is served from the bundled `tethysapp/tethysdash/public/frontend/` assets that are committed to the repo.
+
+If you need to do React work, use the conda-based workflow described in the top-level `README.md` instead — that path runs `npm start` and `tethys manage start` together.
+
+## Open in dev container
+
+**VS Code**: install the *Dev Containers* extension, open the `tethysapp-tethys_dash/` folder, then run **Dev Containers: Reopen in Container** from the command palette.
+
+**Codex / other devcontainer-aware editors**: open the same folder and accept the rebuild prompt.
+
+The first build runs `apt install`, creates an `/opt/venv` Python venv, `pip install -e .[test]`, and runs the MCP contract suite as a build-time gate (so dep-resolution regressions surface before you ever open a shell). Expect a few minutes on first build; subsequent rebuilds are fast.
+
+## What `postCreateCommand` does
+
+After the image builds, the host fires `bash .devcontainer/init.sh`, which runs two bash scripts in order:
+
+- `scripts/tethyscore.sh` — generates `portal_config.yml`, sets dev-mode settings (`DEBUG=True`, `ALLOWED_HOSTS=['*']`, SQLite ENGINE), runs `tethys db migrate`, and creates the default superuser. Mirrors the role of `tethys_portal_firo/docker/tethyscore.sls`.
+- `scripts/devcontainer-app.sh` — provisions the SQLite persistent store at the tethysdash app workspace, links it to `tethysdash:ps_database:primary_db`, runs `tethys syncstores tethysdash`, and finally touches the marker file. Mirrors the role of `tethys_portal_firo/docker/salt/init_apps.sls`.
+
+Both scripts gate on `${TETHYS_PERSIST}/tethysdash_devcontainer_setup_complete`. If the marker exists, they exit without doing anything. The marker is touched only after `devcontainer-app.sh` reaches its end, so a partial failure leaves the marker absent and `init.sh` reruns cleanly.
+
+**Why bash instead of Salt?** The `.sls` pattern is the canonical Aquaveo provisioning style, and we initially planned to use it here. In practice, pip-distributed Salt has incomplete install_requires on slim Python base images (missing `looseversion`, `distro`, and others), and the SaltProject apt repo isn't reliably reachable from all build environments. Bash gives us identical idempotence semantics (marker-file gating, ordered execution, env-var-driven settings) without any of the install fragility — at the cost of losing Salt's declarative DSL. The two-tier split (portal vs app) and the marker-file pattern are preserved 1:1 from the `.sls` precedent.
+
+## Start the server
+
+After `postCreateCommand` finishes, open a terminal in the container and:
+
+```bash
+bash .devcontainer/run.sh
+```
+
+That runs `tethys manage start -p 0.0.0.0:8000`. The forwarded port lands at `http://localhost:8000/` on your host. Log in with `admin` / `admin`, then the app is at:
+
+```
+http://localhost:8000/apps/tethysdash/
+```
+
+(Tethys mounts all apps under `/apps/<app_name>/`. Bare `/tethysdash/` will 404.)
+
+## Edit-and-reload
+
+The app is installed editable (`pip install -e .[test]`). Editing any `.py` file under `tethysapp/tethysdash/` triggers Django's autoreload — just save and refresh the browser.
+
+If you change `pyproject.toml` (e.g., add a Python dep), run `pip install -e .[test]` again in the container terminal to refresh the venv.
+
+## Run tests
+
+(The MCP contract suite was moved out with the embedded MCP server — it now
+lives in the standalone repo `Aquaveo/tethysdash_mcps`. Run it from that repo
+via `./scripts/setup-mcp.sh --setup && .venv-mcp/bin/python -m pytest test_mcp/`.)
+
+Backend tests:
+
+```bash
+python -m pytest --reuse-db
+```
+
+The Jest / Playwright frontend suites are NOT run from this container by design — see "Out of scope" below.
+
+## Out of scope
+
+This devcontainer intentionally does **not**:
+
+- Install Node, npm, or Webpack — frontend rebuilds happen on your host.
+- Provision Postgres, Redis, GeoServer, or any other service container.
+- Mount your host's `~/.tethys/` directory — container state is isolated.
+- Persist `~/.tethys/` or the SQLite store across `Rebuild Container Without Cache`. (The SQLite store is under `tethysapp/tethysdash/workspaces/`, which IS bind-mounted, so it does survive plain `Rebuild Container`. The portal_config and the marker file live in the container's `/root/.tethys/` and get wiped on full rebuilds.)
+
+If you need any of these, fall back to the conda-based workflow in the top-level `README.md`.
+
+## Frontend bundle staleness
+
+The dev server serves whatever is in `tethysapp/tethysdash/public/frontend/` — the bundle committed to the repo. If you're reviewing a frontend PR, that bundle may not reflect the React changes.
+
+Quick check on your host (NOT inside the container):
+
+```bash
+git log -1 --format=%h -- tethysapp/tethysdash/public/frontend/
+git log -1 --format=%h -- reactapp/
+```
+
+If the second commit is newer than the first, run `npm run build` on your host to refresh the bundle before opening the devcontainer.
+
+The devcontainer cannot detect or fix this for you — it's intentionally backend-only.
+
+## Troubleshooting
+
+### Re-bootstrap from a half-broken state
+
+If something inside Tethys gets into an inconsistent state (schema mismatch, deleted persistent store, etc.):
+
+```bash
+rm -f $TETHYS_PERSIST/tethysdash_devcontainer_setup_complete
+bash .devcontainer/init.sh
+```
+
+That removes the marker and re-runs all Salt states. Note this re-creates the SQLite store from scratch — any dashboards you saved will be gone.
+
+### `DisallowedHost (400)` after `bash run.sh`
+
+The Salt state's `tethys settings --set ALLOWED_HOSTS "['*']"` must have run. If you started the server before `init.sh` finished, restart the server.
+
+### `tethys gen portal_config --overwrite` prompt
+
+Should not happen — `--overwrite` is the explicit flag for this case. If you somehow hit a prompt (Ctrl-C and re-run), the marker-gate prevents this state from re-running anyway.
+
+### Don't run `git clean -fdx` casually
+
+The bundled frontend (`tethysapp/tethysdash/public/frontend/`) is committed and ignored by `.gitignore` patterns (it's exempted), and the SQLite dev store sits under `tethysapp/tethysdash/workspaces/`. A wide `git clean -fdx` can nuke either; prefer `git clean -fd` and review what's about to be removed.
+
+### Don't push this image to a registry
+
+The image bakes in `admin / admin` superuser credentials. It is dev-only. Pushing to a registry leaks those defaults — and the image is otherwise unhardened (DEBUG=True, ALLOWED_HOSTS=['*'], remoteUser=root).
+
+## Reference
+
+- Plan: `docs/plans/2026-05-06-001-feat-backend-only-devcontainer-plan.md` (workspace-level, outside this subrepo).
+- Brainstorm: `docs/brainstorms/2026-05-06-backend-only-devcontainer-requirements.md`.
+- Pattern source: `tethysapp-agwa/.devcontainer/` (sibling Aquaveo repo, conda-based; we adopted the file layout but diverged on env mgmt).

.devcontainer/devcontainer.json

@@ -0,0 +1,24 @@
+{
+    "name": "TethysDash Backend Devcontainer",
+    "build": {
+        "dockerfile": "Dockerfile",
+        "context": ".."
+    },
+    "workspaceFolder": "/workspaces/tethysapp-tethys_dash",
+    "appPort": [8000],
+    "forwardPorts": [8000],
+    "postCreateCommand": "bash .devcontainer/init.sh",
+    "remoteUser": "root",
+    "customizations": {
+        "vscode": {
+            "extensions": [
+                "ms-python.python",
+                "donjayamanne.python-extension-pack"
+            ],
+            "settings": {
+                "python.defaultInterpreterPath": "/opt/venv/bin/python",
+                "python.terminal.activateEnvironment": false
+            }
+        }
+    }
+}

.devcontainer/init.sh

@@ -0,0 +1,22 @@
+#!/usr/bin/env bash
+# Devcontainer postCreateCommand. Runs the two-stage idempotent setup:
+#   scripts/tethyscore.sh       — portal-level Tethys setup
+#   scripts/devcontainer-app.sh — tethysdash app-specific setup
+#
+# The two-script split mirrors the workspace pattern at
+# tethys_portal_firo/docker/tethyscore.sls + ./salt/init_apps.sls —
+# implemented in bash because pip-installed Salt has incomplete
+# transitive deps and the SaltProject apt repo isn't reliably reachable
+# from all build environments.
+#
+# Idempotence: every step is gated by
+# ${TETHYS_PERSIST}/tethysdash_devcontainer_setup_complete. Re-running
+# after first-time setup is a no-op until the marker is removed (see
+# .devcontainer/README.md > Troubleshooting).
+
+set -euo pipefail
+
+cd "$(dirname "$0")"
+
+bash scripts/tethyscore.sh
+bash scripts/devcontainer-app.sh

.devcontainer/run.sh

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+# Start the TethysDash dev server inside the devcontainer.
+# Bind to 0.0.0.0 so the host's forwarded port reaches it.
+
+set -euo pipefail
+
+echo "Starting Tethys dev server on 0.0.0.0:8000 (DEBUG=True)..."
+exec tethys manage start -p 0.0.0.0:8000