From 59cd49cf9eb5ef77470704f7ebb855600a83a913 Mon Sep 17 00:00:00 2001 From: vivekchand Date: Tue, 8 Sep 2026 10:06:23 +0200 Subject: [PATCH 01/19] =?UTF-8?q?feat:=20build=20your=20own=20UI=20?= =?UTF-8?q?=E2=80=94=20a=20keyed,=20scoped=20read=20API=20over=20the=20q/1?= =?UTF-8?q?=20contract?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ClawMetry ships one dashboard, and a user who wanted a different view of their own data had to fork it. Every number the dashboard draws already comes from a declared, versioned read contract (q/1); the only thing allowed to call it was the dashboard itself, because the only gate in front of it was the browser's same-origin rule. This adds the door: clawmetry key create --name my-ui --scope read:metrics \ --origin http://localhost:3000 GET /api/q/1/ one declared query GET /api/q/1 what this key can read GET /api/q/1/llms.txt the API, written for a coding agent Four pieces, all built on what existed: * Read scopes live ON the contract. Every q/1 method declares one of read:metrics / read:sessions / read:traces / read:content next to its trust class, so a new method cannot drift out of the scope model and docs/QUERY_CONTRACT.md regenerates with a scope table. * clawmetry/apikeys.py + `clawmetry key` + a Security-tab panel. SHA-256 in ~/.clawmetry/api_keys.json (0600), scopes plus an origin allowlist. * routes/public_api.py dispatches through the SAME local_query._dispatch the dashboard uses, so there is no second query surface. * examples/custom-ui/ (one file, no build step) and docs/BUILD_YOUR_OWN_UI.md. The security decision this turns on: adding CORS to a service on localhost is how local tools get robbed. Any page in any tab can already SEND a request to 127.0.0.1:8900; the only reason that is harmless is that the browser will not let it READ the reply. So loopback earns nothing here (a key is always required, which inverts the rest of the dashboard), there is no wildcard origin at all, key management stays on the ordinary dashboard blueprint behind the same-origin write guard, and the API is GET only. Nothing here writes: this adds no entry to the control plane. read:metrics is exactly the plaintext trust class, pinned by a test. That equality is what makes "a browser-resident key cannot return a prompt" a promise rather than a hope. Verified: the starter served from localhost:3000 rendered $750 of real 30-day cost across 10 runtimes from a ClawMetry on another port, and on that same page a valid full-scope key bound to a different origin was refused by the browser. 48 guards in tests/test_public_api_keys.py (registered in ci.yml), three mutations proven red. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01B6d1rL2kbFEu6Pr6xUzYW5 --- .github/workflows/ci.yml | 1 + CHANGELOG.md | 10 +- CLAUDE.md | 6 +- clawmetry/apikeys.py | 428 ++++++++++++++++++++ clawmetry/cli.py | 245 ++++++++++++ clawmetry/query_contract.py | 92 +++++ clawmetry/static/js/app.js | 220 +++++++++- clawmetry/static/locales/en.json | 16 +- clawmetry/templates/tabs/security.html | 72 ++++ dashboard.py | 15 + docs/BUILD_YOUR_OWN_UI.md | 213 ++++++++++ docs/MODULE_MAP.md | 6 +- docs/QUERY_CONTRACT.md | 66 +-- examples/custom-ui/README.md | 43 ++ examples/custom-ui/index.html | 303 ++++++++++++++ routes/infra.py | 104 +++++ routes/public_api.py | 420 +++++++++++++++++++ scripts/gen_query_contract_doc.py | 38 +- tests/test_public_api_keys.py | 532 +++++++++++++++++++++++++ tests/test_query_contract_drift.py | 5 + 20 files changed, 2799 insertions(+), 36 deletions(-) create mode 100644 clawmetry/apikeys.py create mode 100644 docs/BUILD_YOUR_OWN_UI.md create mode 100644 examples/custom-ui/README.md create mode 100644 examples/custom-ui/index.html create mode 100644 routes/public_api.py create mode 100644 tests/test_public_api_keys.py diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ca29332336..904bd6f27b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -544,6 +544,7 @@ jobs: tests/test_bench_route.py \ tests/test_cohort_compare.py \ tests/test_query_contract_drift.py \ + tests/test_public_api_keys.py \ tests/test_query_contract_goldens.py \ tests/test_local_store_concurrent_flush_1590.py \ tests/test_duckdb_invalidated_recovery.py \ diff --git a/CHANGELOG.md b/CHANGELOG.md index 3cb36a713e..3fad5a80c3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,13 @@ - **It found pre-existing debt, recorded as a ratchet rather than fixed here:** `dashboard.py` carries 59 shadowed names (39 byte-identical dead pairs and **20 that differ**, meaning someone edited one copy and not the other) and `routes/entitlement.py` carries 1. `verification/shadowed_definitions.json` holds those counts; the test asserts a file never rises above its baseline and that each baseline equals reality, so no headroom is left for the next one and the debt can only shrink. - **Verified:** restoring the old name reproduces `_session_cwd: 2` in the AST walk and reds the guard; with the fix, 235 module files pass and 273 tests pass across the daemon-wiring, detector and workspace-kind suites. +### Added: build your own UI, a keyed and scoped read API over the q/1 contract (2026-09-08) +- **Why:** ClawMetry ships one dashboard, and a user who wants a different view of their own data has no supported way to get one. Every number the dashboard draws already comes from a declared, versioned read contract (`q/1`, 17 live methods, per-method arg schemas, additive-only, drift-tested). The only thing allowed to call it was the dashboard itself, because the only gate in front of it was the browser's same-origin rule: there was no API key anywhere, and no `Access-Control-Allow-Origin` header anywhere in the codebase. So the options were fork the dashboard, scrape the HTML, or do without. Meanwhile the generation tools got good enough that a purpose-built view is an afternoon of work *if* the data has an addressable API. Ours did not have a door. +- **What:** four pieces, all built on what already existed. (1) **Read scopes live on the contract**: every `q/1` method now declares one of `read:metrics`, `read:sessions`, `read:traces`, `read:content`, in `clawmetry/query_contract.py` next to its trust class, so a new method cannot drift out of the scope model and `docs/QUERY_CONTRACT.md` regenerates with a scope table. (2) **Scoped keys** (`clawmetry/apikeys.py`, `clawmetry key create|list|revoke|scopes`, and a panel on the Security tab): SHA-256 in `~/.clawmetry/api_keys.json` at 0600, each key carrying scopes and a browser-origin allowlist. (3) **`GET /api/q/1/`** (`routes/public_api.py`), dispatching through the *same* `routes.local_query._dispatch` the dashboard uses, so there is no second query surface to keep correct. (4) **`GET /api/q/1/llms.txt`**, the whole API written for a coding agent to read in one pass, generated from the contract and scoped to the presented key. Plus a starter (`examples/custom-ui/`, one file, no build step) and a walkthrough (`docs/BUILD_YOUR_OWN_UI.md`). +- **The security decision this turns on, stated plainly.** Adding CORS to a service on localhost is how local tools get robbed. Any page in any tab can already *send* a request to `127.0.0.1:8900`; the only reason that has been harmless is that the browser refuses to let the page *read* the reply. This feature removes that protection deliberately, one named origin at a time, so: **loopback earns nothing** (this is the one surface in ClawMetry where "the request came from this machine" is not authentication, and a key is always required); **there is no wildcard origin**, not as a flag and not as an env var, because every "just for local dev" wildcard ships eventually and fails silently and totally; **key management is not on the keyed surface** (minting, listing and revoking sit on the ordinary dashboard blueprint behind the existing same-origin write guard, so a read key can never issue itself a better one, and `_add_cors` is pinned to `/api/q/` as belt and braces); and **the API is GET only**, which removes cross-origin writes as a category rather than as a check. Nothing here can pause, stop or kill an agent: this adds no entry to the control plane. +- **`read:metrics` is exactly the `plaintext` trust class**, pinned by a test rather than by convention. That equality is what makes "a browser-resident key cannot return a prompt, a reply or a file path" a promise instead of a hope, and it is why the starter ships asking for that scope and nothing more. +- **Verified end to end in a real browser, not only in tests.** The starter served from `http://localhost:3000` rendered 30 days of real cost from a ClawMetry on a different port: **$750 across 10 runtimes and 10 models**, the numbers matching `/api/aggregates` on the same machine. The negative case was proven on that same page: a *valid* full-scope key bound to `https://my-ui.vercel.app` could not be used from `localhost:3000`, the browser refusing it with `Failed to fetch` before the response was readable. Plus **48 guards** in `tests/test_public_api_keys.py` (registered in `ci.yml`, since CI runs explicit file lists), with three deliberate mutations proving they go red: making `_add_cors` echo every origin fails 4, removing the scope check fails 2, and reclassifying `transcript` as `read:metrics` fails 4 including the trust-class invariant. Every drift guard green: query-contract, module map, runtime and channel counts, AC ratchet, daemon allowlist, py3.9 annotations, `lint-js`. + ### Fixed: a poisoned linked worktree scanned CLEAN (2026-09-08) - **Why:** `repo_scan` opened `/.git/config` and nothing else. In a **linked worktree** `.git` is a FILE holding `gitdir: /abs/path`, and the config git actually reads lives in the common directory that path points at, so the scanner found no config and returned no findings. That is worse than returning nothing: it is a confident all-clear over a repository git will happily execute `core.fsmonitor` from, on a layout agents and CI use routinely. CVE-2026-55607 is the vendor-confirmed version of the same git-directory confusion. Found by `scripts/redteam/audit.py` itself, which filed the gap automatically: the audit doing exactly the job it was wired into CI for one release earlier. - **What:** `_git_dirs()` resolves `(git_dir, common_dir)` through the `.git` file's `gitdir:` and the worktree's `commondir`. `_git_config_paths()` returns every config git reads, which is `/config` plus `/config.worktree` (honoured when `extensions.worktreeConfig` is set, and writable by whoever supplied the worktree). The finding reports the config it was actually read from, trimmed to its last three components, because an incident travels to a UI and to a device and must not carry the user's home directory. Ordinary checkouts are untouched: `_git_dirs` answers with `.git` for both values and the label stays `.git/config`. @@ -24,8 +31,7 @@ - **What:** three PASS and are now cases. `npm-postinstall-visible-harvest` (credential_access, critical), `mcp-tool-output-prompt-injection` (credential_access, critical) and `mcp-injection-without-exfil` (privilege_change). The two MCP cases are deliberately a pair: the injection itself is invisible to us, since it arrives in a tool RESULT and no detector reads a result as an instruction, so what the pair measures is whether the BEHAVIOUR an injection produces is caught, in both the exfiltration shape and a non-exfiltration one. `control-ordinary-dev-session` is the negative control that belongs beside them: install, read `.env.example`, run tests, push, and stay silent. - **The fourth is a MISS, and it is filed rather than shipped red.** `npm-postinstall-credential-harvest`: nothing fired at all. It is not a tuning problem. For the invisible half of the attack the agent chose exactly one action, `npm install`, and npm's child processes did the rest, so the event stream is byte-for-byte the stream of a developer installing a dependency. No tool-stream detector can separate them. The fix is the workspace surface, the same answer `repo_scan` gave for GitSpawn, and the gap carries the audit's own verdict on the private tracker. - **The README now records what is deliberately NOT in the corpus**, so the next person does not rediscover it: cross-session coordination (the Hugging Face evaluation swarm) is not merely undetected but **inexpressible**, because `run_all` takes one session's events and so does the corpus runner; and credential-scoping bypass has no observable in the stream at all. -- **Verified:** `scripts/redteam/audit.py` reports 13/13 pass, 0 gaps, 0 control failures; the corpus suite is 28 tests, up from 24. Both run on every PR through the gate added in 0.12.837. - +- **Verified:** `scripts/redteam/audit.py` reports 13/13 pass, 0 gaps, 0 control failures; the corpus suite is 28 tests, up from 24. Both run on every PR through the gate added in 0.12.837. b2bf019 (feat: build your own UI — a keyed, scoped read API over the q/1 contract) ### Added: the daemon scans a session's workspace, which is the surface the tool stream cannot see (2026-09-08) - **Why:** every behavioural detector reads what the agent *chose* to do. GitSpawn is the proof that this is a partial view of the machine: a poisoned `.git/config` makes **git** spawn the payload during the background `git status` a runtime fires on open, the agent calls no tool, and `detectors.run_all` sees a clean session. `clawmetry/repo_scan.py` closes that gap and has shipped for a while, detecting all three GitSpawn variants plus the CHAINDROP hook shapes, with `clawmetry scan-repo` exposing it to a human. Nothing in the product called it. A detector nobody runs protects nobody. - **What:** `sync._emit_detector_incidents` now scans each session's `cwd` and emits findings on the path detector incidents already take: a `loop_signals` row (`daemon_detect_repo_config_exec` / `daemon_detect_agent_config_tamper`), `incident_alerts` delivery, and the heartbeat fold. The scan is cached on the mtime and size of exactly the files it reads (`.git/config`, `.vscode/tasks.json`, the `_AGENT_HOOK_FILES` entries) and nothing else, so an unchanged repo is never re-read and a repo poisoned *after* it was first seen clean is re-scanned on the next tick. Cache bounded at 500 directories, oldest evicted. `CLAWMETRY_REPO_SCAN=0` turns it off. diff --git a/CLAUDE.md b/CLAUDE.md index 162b5a2c36..42d835a2e2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -45,6 +45,7 @@ All HTTP endpoints live here, organised by feature: 70 modules, 82 blueprints, l | `routes/channels.py` | `bp_channels` — 23 chat-channel adapters (Telegram, Signal, WhatsApp, Discord, Slack, IRC, iMessage, WebChat, …) | | `routes/components.py` | `bp_components` — Flow-panel detail endpoints (tool / runtime / machine / gateway / brain) | | `routes/local_query.py` | `bp_local_query` — `/api/local/*` DuckDB read API + the daemon-proxy `_dispatch` (shape→store bridge shared by HTTP and the cloud relay) | +| `routes/public_api.py` | `bp_public_api` — `/api/q/1/*`, the **keyed, cross-origin** read API custom UIs are built on (`docs/BUILD_YOUR_OWN_UI.md`). Same `_dispatch`, but it is the one surface that does not trust loopback: every request needs a scoped `cmk_` key, and CORS is echoed only for an origin that key named | | `routes/guard.py` | `bp_guard` — live session control (Pause/Stop/Kill), Guard policy CRUD, policy decision log, learned baselines. Sessions ranked by **spend at risk**, not severity | | `routes/policy.py` | `bp_policy` — the *pre-tool* sandbox/permission surface (`/api/tool-policy`). Deliberately a different axis from `routes/guard.py`: different table, no shared state | | `routes/hooks.py` | `bp_hooks` — hook install / status / uninstall per runtime, and the gate's decision log | @@ -66,11 +67,12 @@ All HTTP endpoints live here, organised by feature: 70 modules, 82 blueprints, l | File | Purpose | |------|---------| -| `clawmetry/cli.py` | CLI entry point — `clawmetry`, `connect`, `sync`, `status`, `license`, `hook`, `update` | +| `clawmetry/cli.py` | CLI entry point — `clawmetry`, `connect`, `sync`, `status`, `license`, `hook`, `key`, `update` | | `clawmetry/sync.py` | Cloud sync daemon — ingests into DuckDB, owns the writer lock, runs the detectors and Guard policies, streams the E2E-encrypted (AES-256-GCM) snapshot to `ingest.clawmetry.com`. Holds `_FAMILY_ADAPTER_SPECS` (the adapters that actually load) and `_CHANNEL_DIRS` | | `clawmetry/local_store.py` | **DuckDB store** — the single data layer features read and write (the daemon holds the writer lock). Schema v15 | | `clawmetry/local_server.py` | Daemon-hosted localhost query server (`/local/query`, discovered through `~/.clawmetry/local_query.json`) so the dashboard reads DuckDB without grabbing the writer lock | | `clawmetry/query_contract.py` | The declared node query surface (`q/1`), rendered to `docs/QUERY_CONTRACT.md`. Additive-only inside a version | +| `clawmetry/apikeys.py` | Scoped read keys (`cmk_…`) for custom UIs: mint, verify, revoke. SHA-256 in `~/.clawmetry/api_keys.json` (0600); the scope a key carries maps to `query_contract`'s per-method `scope` | | `clawmetry/entitlements.py` | Single source of truth for tiers, `FREE_RUNTIMES` / `PAID_RUNTIMES`, `ALL_CHANNELS` and every capacity limit. GRACE by default | | `clawmetry/license.py` | Offline Ed25519 verification of self-hosted license keys | | `clawmetry/proxy.py` | Enforcement proxy — budget limits, loop detection, model routing (port 4100) | @@ -116,6 +118,7 @@ All HTTP endpoints live here, organised by feature: 70 modules, 82 blueprints, l | `docs/ENTITLEMENTS.md` | Open-core split: FREE runtimes/features, paid tiers, GRACE mode, `/api/entitlement` shape, `clawmetry license` CLI | | `docs/EGRESS.md` | Every outbound destination, what it carries, and how to verify it on the wire | | `docs/HOOK_COEXISTENCE.md` | How ClawMetry shares a runtime's hook config with other writers | +| `docs/BUILD_YOUR_OWN_UI.md` | The keyed read API, its scopes, and how to point a coding agent at it | | `docs/CUSTOM_RUNTIME_INGEST.md` | The HTTP ingest API for a runtime with no adapter | | `docs/EVENT_RETENTION.md` | Store growth and trimming | | `CHANGELOG.md` | Version history | @@ -167,6 +170,7 @@ The complete surface is generated at `/openapi.json` and browsable at `/api/docs - `/api/signals` — Behaviour signal rates per window (`1d|7d|30d`) and `?runtime=`, with coverage and headline; `/api/signals//sessions` lists matching sessions, never phrases - `/api/guard/sessions` — What is running, what a detector thinks has gone off track, and whether each session can be controlled at all; `/api/guard/control` is the Pause / Stop / Kill button and `/api/guard/policies` the autonomous rules - `/api/entitlement` — The resolved entitlement (tier, allowed runtimes, features, capacity). GRACE mode answers "allowed" for everything until the announced enforce date +- `/api/q/1/*` — The **public** read API: the same q/1 methods, gated by a scoped API key instead of by being local. `GET /api/q/1` says what a key can read and `GET /api/q/1/llms.txt` describes the whole surface for a coding agent. `docs/BUILD_YOUR_OWN_UI.md` - `/api/local/*` — The DuckDB read API, proxied to the daemon. The method set is declared in `clawmetry/query_contract.py`; `make lint-daemon-allowlist` fails when a route calls one the daemon does not serve - `/v1/metrics`, `/v1/traces`, `/v1/logs` — OTLP receiver (binds `127.0.0.1` by default) diff --git a/clawmetry/apikeys.py b/clawmetry/apikeys.py new file mode 100644 index 0000000000..ee8e340b01 --- /dev/null +++ b/clawmetry/apikeys.py @@ -0,0 +1,428 @@ +"""clawmetry/apikeys.py -- scoped, revocable read keys for custom UIs. + +Why this exists +--------------- +The dashboard reads your agents through the ``q/1`` query contract. That +contract is a good API: it is declared, versioned, arg-checked and drift +tested. Until now the only thing allowed to call it was the dashboard +itself, because the only gate in front of it was the browser's +same-origin rule. Anyone who wanted a different view of their own data +had to fork the dashboard. + +An API key changes that. You create one, say what it may read and which +site may read it, paste it into whatever you are building, and the page +gets exactly that slice. This is the substrate under "build your own UI" +(docs/BUILD_YOUR_OWN_UI.md). + +The security posture, stated plainly +------------------------------------ +Adding CORS to a localhost service is how local tools get robbed. Any +page in any tab can already SEND a request to ``127.0.0.1:8900``; the +only reason that has been harmless is that the browser refuses to let +the page READ the reply. Handing out ``Access-Control-Allow-Origin`` +removes that protection, so it is only ever echoed for an origin the key +holder named. Concretely: + +* A key is required. Loopback is NOT trusted here, unlike the rest of + the dashboard -- ``routes/public_api.py`` is the one surface where + "the request came from this machine" earns nothing. +* A key carries an origin allowlist and it may not be empty. There is + no wildcard. A key with no browser origin (a CLI, a cron, a backend) + is created with ``--origin none`` and simply never gets a CORS header. +* Scopes are least-revealing-first and ``read:content`` is never + granted implicitly -- it has to be asked for by name. +* Keys are read-only. Nothing in this module can pause, stop or kill an + agent, and ``routes/public_api.py`` dispatches only ``q/1`` read + shapes, so this adds nothing to ClawMetry's control plane. + +Storage +------- +``~/.clawmetry/api_keys.json``, 0o600, one JSON document. Only the +SHA-256 of the secret is stored, so a stolen file cannot be replayed as +a key and we cannot show a user a key they lost. The wire form is +``cmk__``: the id is public and appears in listings and +audit lines, the secret never leaves the creating terminal. +""" + +from __future__ import annotations + +import hashlib +import hmac +import json +import os +import secrets +import time +from typing import Any, Optional + +from clawmetry.query_contract import SCOPE_CONTENT, SCOPE_DOC, SCOPES + +# ── Shape of the thing ────────────────────────────────────────────────── + +#: Wire prefix. Deliberately distinct from ``cm_`` (the cloud node key) +#: and ``sk-ant-`` (a model provider key) so a leaked string is +#: identifiable at a glance by a scanner or by a human reading a log. +KEY_PREFIX = "cmk" + +_ID_BYTES = 4 # 8 hex chars: enough to name a key in a listing +_SECRET_BYTES = 32 # 256 bits + +STORE_PATH = os.path.expanduser("~/.clawmetry/api_keys.json") +_FILE_MODE = 0o600 +_DIR_MODE = 0o700 + +#: A machine is not a key management product. The cap exists so a runaway +#: script cannot grow the file without bound; it is not a paywall. +MAX_KEYS = 50 + +#: Sentinel origin meaning "this key is not used from a browser". Stored +#: as an empty origin list; kept as a word so the CLI can say it back. +ORIGIN_NONE = "none" + + +class ApiKeyError(Exception): + """Raised for a caller mistake (bad scope, bad origin, cap reached). + + Carries a sentence meant for a person, not an error code -- these + surface directly in ``clawmetry key`` output and in the dashboard. + """ + + +# ── Store I/O ─────────────────────────────────────────────────────────── + +def _store_path() -> str: + return os.environ.get("CLAWMETRY_API_KEYS_PATH") or STORE_PATH + + +def _read_store() -> dict: + """The stored document, or an empty one. Never raises: a corrupt or + unreadable file must not take the dashboard down, it must behave as + "no keys are configured" so every request 401s honestly.""" + try: + with open(_store_path()) as fh: + data = json.load(fh) + if not isinstance(data, dict): + return {"version": 1, "keys": []} + keys = data.get("keys") + if not isinstance(keys, list): + data["keys"] = [] + return data + except (FileNotFoundError, OSError, ValueError, json.JSONDecodeError): + return {"version": 1, "keys": []} + + +def _write_store(doc: dict) -> None: + """Atomically replace the store, 0o600, creating ~/.clawmetry if needed.""" + path = _store_path() + parent = os.path.dirname(path) + if parent: + os.makedirs(parent, exist_ok=True) + try: + os.chmod(parent, _DIR_MODE) + except OSError: + pass # NFS home, Windows: best effort, never block a create + body = json.dumps(doc, indent=2, sort_keys=True) + "\n" + tmp = path + ".tmp" + # os.open with the mode arg so umask cannot widen a fresh key file to + # 0o644. Mirrors clawmetry/license.py::_secure_write. + fd = os.open(tmp, os.O_CREAT | os.O_TRUNC | os.O_WRONLY, _FILE_MODE) + try: + os.write(fd, body.encode("utf-8")) + finally: + os.close(fd) + try: + os.chmod(tmp, _FILE_MODE) + except OSError: + pass + os.replace(tmp, path) + + +# ── Validation ────────────────────────────────────────────────────────── + +def normalise_scopes(scopes) -> list: + """Return ``scopes`` as a sorted, de-duplicated, validated list. + + Raises :class:`ApiKeyError` naming the offender, because "invalid + scope" with no name is the kind of message that sends someone to the + source to find out what they typed wrong. + """ + out = set() + for raw in scopes or (): + s = str(raw).strip() + if not s: + continue + if s not in SCOPES: + raise ApiKeyError( + f"{s!r} is not a scope. Choose from: " + ", ".join(SCOPES) + ) + out.add(s) + if not out: + raise ApiKeyError( + "A key needs at least one scope. Choose from: " + ", ".join(SCOPES) + ) + # Keep the declared order (least revealing first) rather than + # alphabetical, so a listing reads the way the docs do. + return [s for s in SCOPES if s in out] + + +def normalise_origins(origins) -> list: + """Validate browser origins down to ``scheme://host[:port]``. + + An origin is what a browser will actually send in the ``Origin`` + header, so anything with a path, a query or a wildcard is rejected + here rather than silently never matching at request time. + """ + from urllib.parse import urlsplit + + out = [] + for raw in origins or (): + o = str(raw).strip().rstrip("/") + if not o or o.lower() == ORIGIN_NONE: + continue + if o == "*": + raise ApiKeyError( + "A wildcard origin is not allowed. Any page in any tab could " + "then read this machine's telemetry. Name the site you are " + "building, for example https://my-app.vercel.app." + ) + parts = urlsplit(o) + if parts.scheme not in ("http", "https"): + raise ApiKeyError( + f"{o!r} is not an origin. An origin looks like " + "https://my-app.vercel.app or http://localhost:3000." + ) + if not parts.netloc or parts.path or parts.query or parts.fragment: + raise ApiKeyError( + f"{o!r} has a path or query. An origin is just the scheme, " + "host and port: " + f"{parts.scheme}://{parts.netloc}" + ) + out.append(f"{parts.scheme}://{parts.netloc}".lower()) + # De-duplicate, keep first-seen order so the user's list reads back + # the way they typed it. + seen, uniq = set(), [] + for o in out: + if o not in seen: + seen.add(o) + uniq.append(o) + return uniq + + +# ── Create / list / revoke ────────────────────────────────────────────── + +def _hash_secret(secret: str) -> str: + return hashlib.sha256(secret.encode("utf-8")).hexdigest() + + +def create(name: str, scopes, origins, *, note: str = "") -> tuple: + """Mint a key. Returns ``(record, plaintext_key)``. + + The plaintext is returned once and never stored. Callers show it and + forget it. + """ + label = (name or "").strip() + if not label: + raise ApiKeyError( + "Give the key a name so you can tell it apart later, for " + "example: latency-workbench." + ) + if len(label) > 64: + raise ApiKeyError("Key names are limited to 64 characters.") + scope_list = normalise_scopes(scopes) + origin_list = normalise_origins(origins) + + doc = _read_store() + live = [k for k in doc["keys"] if not k.get("revoked_at")] + if len(live) >= MAX_KEYS: + raise ApiKeyError( + f"This machine already has {MAX_KEYS} active keys, which is the " + "limit. Revoke one you no longer use: clawmetry key revoke " + ) + + key_id = secrets.token_hex(_ID_BYTES) + secret = secrets.token_urlsafe(_SECRET_BYTES) + record = { + "id": key_id, + "name": label, + "note": (note or "").strip()[:200], + "hash": _hash_secret(secret), + "scopes": scope_list, + "origins": origin_list, + "created_at": int(time.time()), + "last_used_at": None, + "use_count": 0, + "revoked_at": None, + } + doc["keys"].append(record) + _write_store(doc) + return record, f"{KEY_PREFIX}_{key_id}_{secret}" + + +def list_keys(*, include_revoked: bool = False) -> list: + """Key records with the hash stripped. Newest first.""" + doc = _read_store() + rows = [ + {k: v for k, v in rec.items() if k != "hash"} + for rec in doc["keys"] + if include_revoked or not rec.get("revoked_at") + ] + rows.sort(key=lambda r: r.get("created_at") or 0, reverse=True) + return rows + + +def revoke(key_id: str) -> bool: + """Mark a key revoked. Returns False when no such live key exists. + + The record is kept, not deleted: "this key was revoked on the 8th" + is the answer someone needs when a key stops working, and an absent + row cannot give it. + """ + wanted = (key_id or "").strip().lower() + if wanted.startswith(KEY_PREFIX + "_"): + # Someone pasted the whole key back. Accept it; the id is the + # second field and the secret is ignored. + parts = wanted.split("_") + wanted = parts[1] if len(parts) > 1 else "" + if not wanted: + return False + doc = _read_store() + hit = False + for rec in doc["keys"]: + if rec.get("id") == wanted and not rec.get("revoked_at"): + rec["revoked_at"] = int(time.time()) + hit = True + if hit: + _write_store(doc) + return hit + + +# ── Verification (the request path) ───────────────────────────────────── + +def parse(presented: str) -> Optional[tuple]: + """Split a presented key into ``(key_id, secret)``, or None.""" + s = (presented or "").strip() + if not s.startswith(KEY_PREFIX + "_"): + return None + parts = s.split("_", 2) + if len(parts) != 3 or not parts[1] or not parts[2]: + return None + return parts[1], parts[2] + + +def verify(presented: str) -> Optional[dict]: + """The key's record if ``presented`` is a live key, else None. + + Compared with :func:`hmac.compare_digest` over the hex digest so the + check does not leak the secret through its own timing. Never raises: + an unreadable store means "no key matches", which is the safe answer. + """ + parsed = parse(presented) + if not parsed: + return None + key_id, secret = parsed + presented_hash = _hash_secret(secret) + for rec in _read_store()["keys"]: + if rec.get("id") != key_id or rec.get("revoked_at"): + continue + stored = str(rec.get("hash") or "") + if stored and hmac.compare_digest(stored, presented_hash): + return rec + return None + + +def touch(key_id: str) -> None: + """Record that a key was just used. Best effort, never raises. + + Usage is written back so ``clawmetry key list`` can answer "is + anything still using this?" before someone revokes it. The write is + skipped when the timestamp would not change to the second, so a page + polling once a second does not rewrite the file on every request. + """ + try: + doc = _read_store() + now = int(time.time()) + changed = False + for rec in doc["keys"]: + if rec.get("id") == key_id: + rec["use_count"] = int(rec.get("use_count") or 0) + 1 + if rec.get("last_used_at") != now: + rec["last_used_at"] = now + changed = True + if changed: + _write_store(doc) + except Exception: + return + + +def origin_allowed(record: dict, origin: str) -> bool: + """True when ``origin`` is on this key's allowlist. Case-insensitive, + trailing slash tolerated (some clients send one). An empty allowlist + matches nothing, by design: that key is for non-browser callers.""" + if not origin: + return False + o = origin.strip().rstrip("/").lower() + return o in [str(x).lower() for x in (record.get("origins") or [])] + + +def any_key_allows_origin(origin: str) -> bool: + """True when ANY live key names ``origin``. + + A CORS preflight arrives without the ``Authorization`` header, so at + preflight time we cannot know which key the real request will carry. + This answers the only question the preflight can answer: is this + origin one the user has authorised at all? The real request is still + checked against its own key's allowlist. + """ + if not origin: + return False + o = origin.strip().rstrip("/").lower() + for rec in _read_store()["keys"]: + if rec.get("revoked_at"): + continue + if o in [str(x).lower() for x in (rec.get("origins") or [])]: + return True + return False + + +def granted_shapes(record: dict) -> set: + """Every live q/1 shape this key may dispatch.""" + from clawmetry.query_contract import shapes_for_scopes + + return shapes_for_scopes(record.get("scopes") or []) + + +def scope_catalogue() -> list: + """``[{scope, doc, methods, sensitive}]`` for the UI and the CLI help. + + Derived from the query contract, so a method added there shows up + here with no second list to update. + """ + from clawmetry.query_contract import live_methods_by_scope + + return [ + { + "scope": s, + "doc": SCOPE_DOC[s], + "methods": live_methods_by_scope(s), + "sensitive": s == SCOPE_CONTENT, + } + for s in SCOPES + ] + + +def redact(presented: str) -> str: + """``cmk_a1b2c3d4_...`` -- safe to log. Shows the id, never the secret.""" + parsed = parse(presented) + if not parsed: + return "(no key)" + return f"{KEY_PREFIX}_{parsed[0]}_..." + + +def store_summary() -> dict[str, Any]: + """Counts for the dashboard panel, cheap enough to call per page load.""" + doc = _read_store() + keys = doc["keys"] + return { + "active": sum(1 for k in keys if not k.get("revoked_at")), + "revoked": sum(1 for k in keys if k.get("revoked_at")), + "max": MAX_KEYS, + "path": _store_path(), + } diff --git a/clawmetry/cli.py b/clawmetry/cli.py index 9e9b41065e..441d313fec 100644 --- a/clawmetry/cli.py +++ b/clawmetry/cli.py @@ -4715,6 +4715,180 @@ def _cmd_mcp(args) -> None: raise SystemExit(_mcp_cli(list(getattr(args, "mcp_args", None) or []))) + +def _cmd_key(args) -> None: + """`clawmetry key ...` -- scoped read keys for custom UIs. + + The point of these keys is that someone can build their own view of + their own agents without forking the dashboard: create a key, say what + it may read and which site may read it, paste it into whatever they + are building. docs/BUILD_YOUR_OWN_UI.md is the long version. + + Two things this command is deliberately strict about, because both + are how a local API gets robbed: + + * A browser key must name its origin. There is no wildcard. Any page + in any tab can already send a request to localhost; the origin + allowlist is the whole reason it cannot read the answer. + * ``read:content`` (prompts, replies, tool calls) is never granted + unless it is asked for by name, and the created key says so out loud. + """ + import json as _json + import time as _time + + from clawmetry import apikeys as _ak + from clawmetry.query_contract import SCOPE_CONTENT, SCOPE_DOC, SCOPE_METRICS + + action = getattr(args, "key_cmd", None) or "list" + as_json = bool(getattr(args, "as_json", False)) + + def _fmt_age(ts): + if not ts: + return "never" + delta = int(_time.time()) - int(ts) + if delta < 60: + return "just now" + for unit, secs in (("d", 86400), ("h", 3600), ("m", 60)): + if delta >= secs: + return f"{delta // secs}{unit} ago" + return "just now" + + if action == "scopes": + rows = _ak.scope_catalogue() + if as_json: + print(_json.dumps({"scopes": rows}, indent=2)) + return + print("Scopes, least revealing first.") + print("") + for row in rows: + flag = " (sensitive)" if row["sensitive"] else "" + print(f" {row['scope']}{flag}") + print(f" {row['doc']}") + print(f" queries: {', '.join(row['methods'])}") + print("") + print("Pick the narrowest scope that makes your UI work. A key that") + print("only needs a cost chart should be read:metrics, so it cannot") + print("return a prompt even if the page it lives in is compromised.") + return + + if action == "list": + rows = _ak.list_keys(include_revoked=bool(getattr(args, "show_revoked", False))) + if as_json: + print(_json.dumps({"keys": rows, "summary": _ak.store_summary()}, indent=2)) + return + if not rows: + print("No API keys on this machine.") + print("") + print("Create one to build your own UI on top of ClawMetry:") + print("") + print(" clawmetry key create --name my-ui \\") + print(" --scope read:metrics --origin http://localhost:3000") + print("") + print("See docs/BUILD_YOUR_OWN_UI.md for the walkthrough.") + return + print(f"{'ID':<10} {'NAME':<24} {'SCOPES':<34} {'LAST USED':<12} ORIGINS") + for r in rows: + origins = ", ".join(r.get("origins") or []) or "(not used from a browser)" + state = " [revoked]" if r.get("revoked_at") else "" + print( + f"{r['id']:<10} {r['name'][:23]:<24} " + f"{','.join(r.get('scopes') or [])[:33]:<34} " + f"{_fmt_age(r.get('last_used_at')):<12} {origins}{state}" + ) + return + + if action == "revoke": + key_id = getattr(args, "key_id", "") or "" + ok = _ak.revoke(key_id) + if as_json: + print(_json.dumps({"action": "revoke", "ok": ok, "id": key_id}, indent=2)) + if not ok: + raise SystemExit(1) + return + if ok: + print(f"Key {key_id} revoked. The next request using it is refused.") + print("Anything you built on it needs a new key: clawmetry key create ...") + return + print(f"No active key called {key_id!r} on this machine.") + print("List what is here with: clawmetry key list") + raise SystemExit(1) + + if action == "create": + scopes = list(getattr(args, "scope", None) or []) or [SCOPE_METRICS] + raw_origins = list(getattr(args, "origin", None) or []) + wants_no_origin = any( + str(o).strip().lower() == _ak.ORIGIN_NONE for o in raw_origins + ) + if not raw_origins: + print("A key needs to know which site may use it from a browser.") + print("") + print(" --origin https://my-ui.vercel.app a site you are building") + print(" --origin http://localhost:3000 your dev server") + print(" --origin none not used from a browser") + print("") + print("There is no wildcard. Any page in any tab can already send a") + print("request to this machine, and naming the origin is what stops") + print("it reading the answer.") + raise SystemExit(1) + try: + record, plaintext = _ak.create( + getattr(args, "name", ""), + scopes, + [] if wants_no_origin else raw_origins, + note=getattr(args, "note", ""), + ) + except _ak.ApiKeyError as exc: + if as_json: + print(_json.dumps({"action": "create", "ok": False, + "error": str(exc)}, indent=2)) + else: + print(str(exc)) + raise SystemExit(1) + + if as_json: + print(_json.dumps({"action": "create", "ok": True, + "key": plaintext, + "record": {k: v for k, v in record.items() + if k != "hash"}}, indent=2)) + return + + print("Key created. It is shown once and is not stored anywhere in") + print("readable form, so copy it now.") + print("") + print(f" {plaintext}") + print("") + print(f"Name: {record['name']} (id {record['id']})") + print(f"Reads: {', '.join(record['scopes'])}") + for s in record["scopes"]: + print(f" {s}: {SCOPE_DOC[s]}") + if record["origins"]: + print(f"Origins: {', '.join(record['origins'])}") + else: + print("Origins: none. This key works from a script or a server, but a") + print(" browser page will not be allowed to read the reply.") + if SCOPE_CONTENT in record["scopes"]: + print("") + print("This key can read the turns themselves: prompts, replies and") + print("tool calls. Keep it server-side. Do not ship it in a page.") + print("") + print("Try it:") + print("") + print(f" curl -H 'Authorization: Bearer {plaintext}' \\") + print(" http://localhost:8900/api/q/1") + print("") + print("Point a coding agent at the generated API guide:") + print("") + print(f" curl -H 'Authorization: Bearer {plaintext}' \\") + print(" http://localhost:8900/api/q/1/llms.txt") + print("") + print("Walkthrough: docs/BUILD_YOUR_OWN_UI.md") + return + + print("Usage: clawmetry key [create|list|revoke|scopes]") + print("Start with: clawmetry key scopes") + raise SystemExit(1) + + def _cmd_reports(args) -> None: """Open the reports browser (refs #1005).""" import webbrowser @@ -8248,6 +8422,74 @@ def main() -> None: # mcp — intercepted by the fast path at the top of main() (WO-59); the # parser entry exists so `clawmetry --help` discovery shows it. + # key — scoped read keys for custom UIs (docs/BUILD_YOUR_OWN_UI.md) + p_key = sub.add_parser( + "key", + help="API keys for custom UIs: create, list, revoke, scopes", + ) + key_sub = p_key.add_subparsers(dest="key_cmd") + + p_key_create = key_sub.add_parser( + "create", help="Mint a scoped read key. Shown once, never stored." + ) + p_key_create.add_argument( + "--name", required=True, metavar="NAME", + help="What this key is for, e.g. latency-workbench. Shown in listings.", + ) + p_key_create.add_argument( + "--scope", action="append", default=[], metavar="SCOPE", + help=( + "What the key may read. Repeatable. Run `clawmetry key scopes` " + "for the list. Defaults to read:metrics, the least revealing one." + ), + ) + p_key_create.add_argument( + "--origin", action="append", default=[], metavar="URL", + help=( + "A site allowed to call this API from a browser, e.g. " + "https://my-ui.vercel.app. Repeatable, and required unless you " + "pass --origin none for a key used outside a browser. There is " + "no wildcard: any page in any tab can already reach localhost, " + "and the origin allowlist is what stops it reading the reply." + ), + ) + p_key_create.add_argument( + "--note", default="", metavar="TEXT", + help="Optional reminder to your future self.", + ) + p_key_create.add_argument( + "--json", action="store_true", dest="as_json", + help="Emit the record plus the key as JSON (jq-friendly).", + ) + + p_key_list = key_sub.add_parser("list", help="Show this machine's keys") + p_key_list.add_argument( + "--all", action="store_true", dest="show_revoked", + help="Include revoked keys.", + ) + p_key_list.add_argument( + "--json", action="store_true", dest="as_json", + help="Emit JSON (jq-friendly).", + ) + + p_key_revoke = key_sub.add_parser( + "revoke", help="Stop a key working. Takes effect on the next request." + ) + p_key_revoke.add_argument( + "key_id", metavar="ID", + help="The key id from `clawmetry key list` (the whole key also works).", + ) + p_key_revoke.add_argument( + "--json", action="store_true", dest="as_json", help="Emit JSON.", + ) + + p_key_scopes = key_sub.add_parser( + "scopes", help="What each scope grants, and which queries it unlocks" + ) + p_key_scopes.add_argument( + "--json", action="store_true", dest="as_json", help="Emit JSON.", + ) + p_mcp = sub.add_parser( "mcp", help="MCP server: `mcp` serves on stdio; `mcp install [--runtime |all] " @@ -8839,6 +9081,7 @@ def main() -> None: "secure", "reports", "eval", + "key", "mcp", "update", "uninstall", @@ -8957,6 +9200,8 @@ def main() -> None: _cmd_eval(args) elif args.cmd == "mcp": _cmd_mcp(args) + elif args.cmd == "key": + _cmd_key(args) elif args.cmd == "update": _cmd_update(args) elif args.cmd == "uninstall": diff --git a/clawmetry/query_contract.py b/clawmetry/query_contract.py index ba2633ac67..5c9ba0639a 100644 --- a/clawmetry/query_contract.py +++ b/clawmetry/query_contract.py @@ -34,6 +34,8 @@ * ``trust`` — "plaintext" or "e2e" (see above). * ``backing`` — the LocalStore method serving it (live) or the planned rollup table / store method (planned). +* ``scope`` — the read scope an API key must carry to dispatch this + method (see SCOPES below). Exactly one per method. * ``doc`` — one-line description. This module is intentionally dependency-free plain data so the doc @@ -51,6 +53,46 @@ TRUST_PLAINTEXT = "plaintext" TRUST_E2E = "e2e" +# ── Read scopes: what an API key is allowed to ask for ────────────────── +# +# Every method declares exactly one scope. A key issued to a custom UI +# (``clawmetry key create --scope read:metrics``) can dispatch only the +# methods whose scope it carries, so the scope a user picks in the UI is +# the same fact the server enforces -- there is no second list to drift. +# +# The split is by what a row REVEALS, not by which table it came from: +# +# * ``read:metrics`` counters and rollups. Exactly the ``plaintext`` +# trust class, and an invariant test pins that: a +# plaintext method is always metrics-scoped, and a +# metrics-scoped method is never ``e2e``. This is the +# scope a cost dashboard or a status board needs, and +# it can never return a prompt or a file path. +# * ``read:sessions`` one row per session: title, model, status, totals. +# Enough to build a session list or a search box. +# * ``read:content`` the turns themselves -- prompts, replies, tool +# calls, transcripts. The most sensitive scope, and +# the one the CLI refuses to grant unless asked for +# by name. +# * ``read:traces`` OTel spans/traces and outbound non-LLM calls. +SCOPE_METRICS = "read:metrics" +SCOPE_SESSIONS = "read:sessions" +SCOPE_CONTENT = "read:content" +SCOPE_TRACES = "read:traces" + +#: Every scope a key may carry, in the order the UI should offer them +#: (least revealing first). +SCOPES: tuple = (SCOPE_METRICS, SCOPE_SESSIONS, SCOPE_TRACES, SCOPE_CONTENT) + +#: One line per scope, written for someone who has never read this file. +#: The key-creation UI and ``clawmetry key create --help`` both render it. +SCOPE_DOC: dict = { + SCOPE_METRICS: "Counts, tokens, cost and health. No prompts or replies.", + SCOPE_SESSIONS: "One row per session: title, model, status, totals.", + SCOPE_TRACES: "Spans, traces and outbound API calls.", + SCOPE_CONTENT: "The turns themselves: prompts, replies, tool calls.", +} + def _arg(required: bool = False, **extra) -> dict: """Tiny spec-builder so the registry below reads as a table.""" @@ -72,6 +114,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=200, lo=1, hi=5000), }, "trust": TRUST_E2E, + "scope": SCOPE_CONTENT, "backing": "query_events", "doc": "Raw event rows (tool calls, messages, errors), newest first.", }, @@ -84,6 +127,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=100, lo=1, hi=2000), }, "trust": TRUST_E2E, + "scope": SCOPE_SESSIONS, "backing": "query_sessions", "doc": "One row per session_id with start/end, event count, cost.", }, @@ -95,6 +139,7 @@ def _arg(required: bool = False, **extra) -> dict: "until": _arg(), }, "trust": TRUST_PLAINTEXT, + "scope": SCOPE_METRICS, "backing": "query_aggregates", "doc": "Per-day rollup of events/tokens/cost (aggregate counters only).", }, @@ -102,6 +147,7 @@ def _arg(required: bool = False, **extra) -> dict: "status": STATUS_LIVE, "args": {}, "trust": TRUST_PLAINTEXT, + "scope": SCOPE_METRICS, "backing": "health", "doc": "Store health snapshot (engine, size, ring depth, flush age).", }, @@ -112,6 +158,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=500, lo=1, hi=5000), }, "trust": TRUST_E2E, + "scope": SCOPE_CONTENT, "backing": "query_events", "doc": "Alias of events scoped to one required session_id.", }, @@ -123,6 +170,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=150, lo=1, hi=250), }, "trust": TRUST_E2E, + "scope": SCOPE_CONTENT, "backing": "query_transcript_page", "doc": ( "One older-history page of a session's events, newest-first. " @@ -142,6 +190,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=200, lo=1, hi=2000), }, "trust": TRUST_E2E, + "scope": SCOPE_TRACES, "backing": "query_spans", "doc": "OTel span rows with full filters (trace/session/agent/time).", }, @@ -155,6 +204,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=100, lo=1, hi=1000), }, "trust": TRUST_E2E, + "scope": SCOPE_TRACES, "backing": "query_traces", "doc": "One row per trace_id with aggregate span stats.", }, @@ -167,6 +217,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=200, lo=1, hi=2000), }, "trust": TRUST_E2E, + "scope": SCOPE_TRACES, "backing": "query_external_calls", "doc": "External (non-LLM) API calls captured by the interceptor.", }, @@ -181,6 +232,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=50, lo=1, hi=500), }, "trust": TRUST_E2E, + "scope": SCOPE_SESSIONS, "backing": "query_search", "doc": "Full-text search over session titles and eval reasons.", }, @@ -191,6 +243,7 @@ def _arg(required: bool = False, **extra) -> dict: "status": STATUS_PLANNED, "args": {}, "trust": TRUST_PLAINTEXT, + "scope": SCOPE_METRICS, "backing": "rollup_glance", "doc": ("Device-facing top-line counters (sessions, cost, alerts). " "Non-goal: no per-model data in glance."), @@ -205,6 +258,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=1000, lo=1, hi=10000), }, "trust": TRUST_PLAINTEXT, + "scope": SCOPE_METRICS, "backing": "query_rollup_runtime_daily", "doc": "Per-runtime daily activity/cost rollup (claude_code, openclaw, ...).", }, @@ -219,6 +273,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=1000, lo=1, hi=10000), }, "trust": TRUST_PLAINTEXT, + "scope": SCOPE_METRICS, "backing": "query_rollup_model_daily", "doc": "Per-model daily token/cost rollup across runtimes.", }, @@ -234,6 +289,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=200, lo=1, hi=2000), }, "trust": TRUST_E2E, + "scope": SCOPE_SESSIONS, "backing": "query_rollup_sessions", "doc": "Per-session materialized summary (title, status, totals, stuck flag).", }, @@ -245,6 +301,7 @@ def _arg(required: bool = False, **extra) -> dict: "until": _arg(), }, "trust": TRUST_PLAINTEXT, + "scope": SCOPE_METRICS, "backing": "rollup_usage_daily", "doc": "Daily token/cost usage series (input/output/cache splits).", }, @@ -254,6 +311,7 @@ def _arg(required: bool = False, **extra) -> dict: "session_id": _arg(required=True), }, "trust": TRUST_E2E, + "scope": SCOPE_SESSIONS, "backing": "query_sessions_table", "doc": "Single-session detail row (title, status, outcome, totals).", }, @@ -265,6 +323,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=200, lo=1, hi=2000), }, "trust": TRUST_E2E, + "scope": SCOPE_CONTENT, "backing": "query_events", "doc": "Reasoning/tool event slice powering the Brain feed.", }, @@ -275,6 +334,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=100, lo=1, hi=1000), }, "trust": TRUST_PLAINTEXT, + "scope": SCOPE_METRICS, "backing": "query_approvals", "doc": "Approval queue metadata (ids, states, timestamps; no content).", }, @@ -287,6 +347,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=500, lo=1, hi=2000), }, "trust": TRUST_PLAINTEXT, + "scope": SCOPE_METRICS, "backing": "query_agent_graph", "doc": "Cross-session agent spawn graph: nodes (agent_type+id stats) + " "spawn edges. Optional runtime arg scopes to one runtime " @@ -299,6 +360,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=2000, lo=1, hi=10000), }, "trust": TRUST_E2E, + "scope": SCOPE_CONTENT, "backing": "query_replay_events", "doc": "Canonical replay-event rows for one session (#4813). Rows in " "kind-agnostic order; the /api/replay-tree endpoint groups " @@ -312,6 +374,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=200, lo=1, hi=1000), }, "trust": TRUST_E2E, + "scope": SCOPE_CONTENT, "backing": "query_session_context", "doc": "Inputs & context rows for one session: system prompt, first " "user prompt, tool definitions, MCP servers, context files and " @@ -326,6 +389,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=10, lo=1, hi=50), }, "trust": TRUST_E2E, + "scope": SCOPE_SESSIONS, "backing": "query_similar_sessions", "doc": "Runs shaped like this one (WO-60): nearest sessions by " "tool-call n-gram similarity inside a window, same runtime " @@ -360,3 +424,31 @@ def methods_by_status(status: str) -> list: def methods_by_trust(trust: str) -> list: return sorted(n for n, s in QUERY_CONTRACT.items() if s["trust"] == trust) + + +def methods_by_scope(scope: str) -> list: + """Live + planned method names carrying ``scope``.""" + return sorted(n for n, sp in QUERY_CONTRACT.items() if sp["scope"] == scope) + + +def live_methods_by_scope(scope: str) -> list: + """Method names carrying ``scope`` that are actually served today.""" + return sorted( + n for n, sp in QUERY_CONTRACT.items() + if sp["scope"] == scope and sp["status"] == STATUS_LIVE + ) + + +def scope_for(method: str) -> str: + """The scope ``method`` requires. Raises KeyError for an unknown method, + which is the right answer: an undeclared method is not servable.""" + return QUERY_CONTRACT[method]["scope"] + + +def shapes_for_scopes(scopes) -> set: + """Every live shape reachable by a key holding ``scopes``.""" + granted = set(scopes or ()) + return { + n for n, sp in QUERY_CONTRACT.items() + if sp["status"] == STATUS_LIVE and sp["scope"] in granted + } diff --git a/clawmetry/static/js/app.js b/clawmetry/static/js/app.js index 7ee1294bb8..fb84a6c87d 100644 --- a/clawmetry/static/js/app.js +++ b/clawmetry/static/js/app.js @@ -11336,7 +11336,11 @@ var _CM_SECURITY_CLOUD_HIDDEN = [ 'security-findings-panel', 'policy-events-panel', 'credential-scan-panel', - 'security-catalog-panel' + 'security-catalog-panel', + // API keys live in ~/.clawmetry on the machine the agent runs on. The + // cloud container has no such file, so the panel would list nothing + // under a "New key" button that could not mint one. + 'apikeys-panel' ]; function _cmSecurityCloudTrim() { @@ -11349,6 +11353,219 @@ function _cmSecurityCloudTrim() { if (note) note.style.display = ''; } +// ── API keys: build your own UI (docs/BUILD_YOUR_OWN_UI.md) ────────────── +// +// A key here is read-only and scoped. The panel's job is to make the two +// things that keep it safe impossible to skip past: you must say WHICH +// site may use the key, and reading the turns themselves (read:content) +// has to be ticked on purpose. + +var _cmApiKeyScopes = []; + +function _cmApiKeysEl(id) { return document.getElementById(id); } + +async function loadApiKeys() { + if (window.CLOUD_MODE) return; + var list = _cmApiKeysEl('apikeys-list'); + if (!list) return; + var data; + try { + data = await fetchJsonWithTimeout('/api/apikeys', 10000); + } catch (e) { + list.innerHTML = '
' + + 'ClawMetry could not read its key file just now. Reload the page to try again.' + + '
'; + return; + } + if (!data || data.ok === false) { + list.innerHTML = '
' + + escapeHtmlSafe((data && data.error) || 'ClawMetry could not read its key file.') + + '
'; + return; + } + _cmApiKeyScopes = data.scopes || []; + _cmRenderApiKeyScopeChoices(); + var keys = (data.keys || []).filter(function(k) { return !k.revoked_at; }); + if (!keys.length) { + list.innerHTML = '
' + + 'No keys yet. A key lets you build your own view of this data: a cost ' + + 'chart on a wall screen, a status page for your team, a panel inside a ' + + 'tool you already use. Press New key, then point a ' + + 'coding agent at the generated API guide.' + + '
'; + return; + } + list.innerHTML = keys.map(_cmRenderApiKeyRow).join(''); +} + +function _cmRenderApiKeyRow(k) { + var scopes = (k.scopes || []).map(function(s) { + var sensitive = (s === 'read:content'); + return '' + escapeHtmlSafe(s) + ''; + }).join(' '); + var origins = (k.origins || []).length + ? (k.origins || []).map(escapeHtmlSafe).join(', ') + : 'Not used from a web page'; + var used = k.last_used_at + ? ('Last used ' + _cmApiKeyAgo(k.last_used_at) + ' · ' + (k.use_count || 0) + ' requests') + : 'Never used'; + return '
' + + '
' + + '
' + + escapeHtmlSafe(k.name || '(unnamed)') + + ' · ' + escapeHtmlSafe(k.id || '') + '' + + '
' + + '
' + scopes + '
' + + '
' + origins + '
' + + '
' + escapeHtmlSafe(used) + '
' + + '
' + + '' + + '
'; +} + +function _cmApiKeyAgo(ts) { + var delta = Math.floor(Date.now() / 1000) - Number(ts || 0); + if (delta < 60) return 'just now'; + if (delta < 3600) return Math.floor(delta / 60) + 'm ago'; + if (delta < 86400) return Math.floor(delta / 3600) + 'h ago'; + return Math.floor(delta / 86400) + 'd ago'; +} + +function _cmRenderApiKeyScopeChoices() { + var box = _cmApiKeysEl('apikey-scopes'); + if (!box) return; + box.innerHTML = (_cmApiKeyScopes || []).map(function(s, i) { + var warn = s.sensitive + ? '
Keep this one server-side. Do not ship it in a page.
' + : ''; + return ''; + }).join(''); +} + +function toggleApiKeyForm(show) { + var form = _cmApiKeysEl('apikeys-form'); + if (!form) return; + var open = (show === undefined) ? form.hidden : !!show; + form.hidden = !open; + if (open) { + _cmRenderApiKeyScopeChoices(); + var err = _cmApiKeysEl('apikey-form-error'); + if (err) err.textContent = ''; + var name = _cmApiKeysEl('apikey-name'); + if (name) name.focus(); + } +} + +function onApiKeyBrowserToggle() { + var box = _cmApiKeysEl('apikey-nobrowser'); + var origins = _cmApiKeysEl('apikey-origins'); + if (!box || !origins) return; + origins.disabled = box.checked; + origins.placeholder = box.checked + ? 'Not needed: this key is not used from a web page' + : 'http://localhost:3000'; +} + +async function createApiKey() { + var err = _cmApiKeysEl('apikey-form-error'); + if (err) err.textContent = ''; + var name = (_cmApiKeysEl('apikey-name') || {}).value || ''; + var originsRaw = (_cmApiKeysEl('apikey-origins') || {}).value || ''; + var noBrowser = !!((_cmApiKeysEl('apikey-nobrowser') || {}).checked); + var scopes = Array.prototype.slice + .call(document.querySelectorAll('.apikey-scope-box')) + .filter(function(b) { return b.checked; }) + .map(function(b) { return b.value; }); + var body = { + name: name.trim(), + scopes: scopes, + browser: !noBrowser, + origins: originsRaw.split(/[\s,]+/).filter(Boolean) + }; + var res, data; + try { + res = await fetch('/api/apikeys', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify(body) + }); + data = await res.json(); + } catch (e) { + if (err) err.textContent = 'ClawMetry did not answer. Is the dashboard still running?'; + return; + } + if (!data || data.ok === false) { + if (err) err.textContent = (data && data.error) || 'The key could not be created.'; + return; + } + toggleApiKeyForm(false); + _cmShowApiKeySecret(data.key, data.record || {}); + loadApiKeys(); +} + +function _cmShowApiKeySecret(key, record) { + var box = _cmApiKeysEl('apikey-reveal'); + var val = _cmApiKeysEl('apikey-reveal-value'); + var next = _cmApiKeysEl('apikey-reveal-next'); + if (!box || !val) return; + val.textContent = key; + if (next) { + var base = window.location.origin; + next.innerHTML = 'Try it, then hand the same URL to a coding agent:
' + + 'curl -H "Authorization: Bearer <key>" ' + + escapeHtmlSafe(base) + '/api/q/1/llms.txt'; + } + box.hidden = false; +} + +function copyApiKey() { + var val = _cmApiKeysEl('apikey-reveal-value'); + if (!val) return; + var text = val.textContent || ''; + if (navigator.clipboard && navigator.clipboard.writeText) { + navigator.clipboard.writeText(text); + } else { + // Older / non-secure contexts: select it so ctrl-C works. + var r = document.createRange(); + r.selectNodeContents(val); + var sel = window.getSelection(); + sel.removeAllRanges(); + sel.addRange(r); + } +} + +function dismissApiKeyReveal() { + var box = _cmApiKeysEl('apikey-reveal'); + var val = _cmApiKeysEl('apikey-reveal-value'); + if (val) val.textContent = ''; + if (box) box.hidden = true; +} + +async function revokeApiKey(id) { + if (!id) return; + var ok = window.confirm( + 'Revoke this key?\n\nAnything using it stops working on its next request. ' + + 'This cannot be undone; you would have to create a new key.' + ); + if (!ok) return; + try { + await fetch('/api/apikeys/' + encodeURIComponent(id), { method: 'DELETE' }); + } catch (e) { /* the reload below tells the truth either way */ } + loadApiKeys(); +} + async function loadSecurityPosture() { if (window.CLOUD_MODE) { // Posture scans the machine's agent config, which the cloud container does @@ -11445,6 +11662,7 @@ async function loadSecurityPosture() { } async function loadSecurityPage(silent) { + loadApiKeys(); if (window.CLOUD_MODE) { // Threat/policy/credential scans read this machine's event history, which // the cloud container does not have. Their panels go; integrity and the diff --git a/clawmetry/static/locales/en.json b/clawmetry/static/locales/en.json index db626646ea..1fa384fef5 100644 --- a/clawmetry/static/locales/en.json +++ b/clawmetry/static/locales/en.json @@ -1140,7 +1140,7 @@ "security.audit_empty": "No recorded activity yet. Approval decisions, budget changes, and pauses appear here.", "security.audit_sub": "(approvals, budgets, pauses)", "security.audit_title": "Recent activity", - "security.cloud_note": "Config posture checks and threat scans run on the machine your agent runs on: they read files and activity that stay on it. Open ClawMetry there (localhost:8900 \u2192 Security) to run them.", + "security.cloud_note": "Config posture checks and threat scans run on the machine your agent runs on: they read files and activity that stay on it. Open ClawMetry there (localhost:8900 → Security) to run them.", "security.clean_sessions": "Clean Sessions", "security.critical": "Critical", "security.failed": "Failed", @@ -1534,5 +1534,17 @@ "trail.commits": "Commits", "trail.prs": "Pull requests", "evals.locked": "Session scoring is a Pro feature.", - "trail.cov_unknown": "not declared yet" + "trail.cov_unknown": "not declared yet", + "security.apikeys_title": "Build your own view", + "security.apikeys_sub": "Give a page, a script or a coding agent read access to this machine's agent data, scoped to exactly what it needs. Nothing here can start, pause or stop an agent.", + "security.apikeys_new": "New key", + "security.apikeys_name_label": "What is it for?", + "security.apikeys_origin_label": "Which site will use it?", + "security.apikeys_no_browser": "This key is for a script or a server, not a web page", + "security.apikeys_scope_label": "What may it read?", + "security.apikeys_create": "Create key", + "security.apikeys_cancel": "Cancel", + "security.apikeys_reveal_title": "Copy this now. It is not shown again.", + "security.apikeys_copy": "Copy", + "security.apikeys_done": "Done" } diff --git a/clawmetry/templates/tabs/security.html b/clawmetry/templates/tabs/security.html index 9ac03d62ec..ac74297cf5 100644 --- a/clawmetry/templates/tabs/security.html +++ b/clawmetry/templates/tabs/security.html @@ -60,6 +60,78 @@
+ +
+
+
🔑
+
+
Build your own view
+
Give a page, a script or a coding agent read access to this machine's agent data, scoped to exactly what it needs. Nothing here can start, pause or stop an agent.
+
+ +
+ + + + + + + +
+
cmk_a1b2c3d4_... +``` + +**2. Check it works.** + +```bash +curl -H "Authorization: Bearer cmk_a1b2c3d4_..." \ + http://localhost:8900/api/q/1 +``` + +That returns what the key can read: its scopes, and every query it may +run with the arguments each one takes. + +**3. Ask for something real.** + +```bash +curl -H "Authorization: Bearer cmk_a1b2c3d4_..." \ + "http://localhost:8900/api/q/1/aggregates?since=2026-09-01T00:00:00Z" +``` + +```json +{ + "shape": "aggregates", + "rows": [ + {"day": "2026-09-08", "agent_id": "main", + "event_count": 4059, "token_count": 887691, "cost_usd": 136.47} + ], + "count": 1, + "contract": "q/1", + "elapsed_ms": 38 +} +``` + +That is the whole API. Everything below is detail. + +## Building the UI with a coding agent + +The API describes itself, in a form written for an agent to read in one +pass and scoped to the key you hand it: + +```bash +curl -H "Authorization: Bearer $CLAWMETRY_KEY" \ + http://localhost:8900/api/q/1/llms.txt +``` + +Give that output to Claude Code, Cursor, v0, Lovable, or whatever you +build with, along with what you want. A prompt that works: + +> Build a single-page dashboard for our team's AI agent spend. +> +> Data comes from the ClawMetry query API at `http://localhost:8900/api/q/1`. +> Authenticate with `Authorization: Bearer `. The full API is below. +> +> I want: a line chart of daily cost for the last 30 days from +> `/aggregates`, a table of the top 10 models by spend from `/models`, +> and a per-runtime breakdown from `/runtimes`. Refresh every 60 seconds. +> Show the total spend for the period as the headline number. +> +> + +Two things worth putting in the prompt, because they are what a generated +UI usually gets wrong: + +- **The key belongs in an environment variable**, not in the page source. + If it must be in the browser (a static page with no backend), give that + page a `read:metrics` key and nothing more. +- **Link back into ClawMetry.** A custom view is best when it is narrow. + When someone wants to know *why* a number moved, send them to + `http://localhost:8900/#session=` rather than rebuilding the session + viewer. + +There is a working starter in [`examples/custom-ui/`](../examples/custom-ui/) +if you would rather begin from something that already runs. + +## Scopes + +A key carries one or more scopes. It can run exactly the queries its +scopes cover and nothing else. + +| Scope | Grants | Queries | +|---|---|---| +| `read:metrics` | Counts, tokens, cost and health. No prompts or replies. | `aggregates`, `models`, `runtimes`, `agent_graph`, `health` | +| `read:sessions` | One row per session: title, model, status, totals. | `sessions`, `rollup_sessions`, `search`, `similar_sessions` | +| `read:traces` | Spans, traces and outbound API calls. | `spans`, `traces`, `external_calls` | +| `read:content` | The turns themselves: prompts, replies, tool calls. | `events`, `transcript`, `transcript_page`, `replay_events`, `session_context` | + +The generated table in [`QUERY_CONTRACT.md`](./QUERY_CONTRACT.md) is the +authority, along with every argument each query takes. Run +`clawmetry key scopes` for the same thing in a terminal. + +Pick the narrowest scope that makes your UI work. `read:metrics` is +exactly the set of queries that cannot return a prompt, a reply or a file +path, which makes it the right choice for anything that renders in a +browser. + +## What a key can and cannot do + +**Can:** run the `q/1` read queries its scopes cover, from a site it +names, at up to 240 requests a minute. + +**Cannot:** write anything. Pause, stop or kill an agent. Change a +policy, a budget, an alert or a cron. Create another key. Read a query +outside its scopes. Be used from an origin it does not name. + +The API is GET only, so it cannot be the target of a cross-origin write +even in principle. + +## About that origin + +Every key must name the sites allowed to use it from a browser. There is +no wildcard, and the flag is not optional. + +This is worth a paragraph, because it is the part people try to skip. +Your ClawMetry runs on `localhost`. Any page in any tab can already *send* +a request to `localhost`. The only reason that has never mattered is that +the browser refuses to let a page *read* the reply from an origin that +did not permit it. An API key with an origin allowlist is how you hand +out that permission one site at a time. A wildcard would hand it to every +site at once, including a page you did not open on purpose. + +For a key used outside a browser (a cron job, a backend, a script), pass +`--origin none`. It gets no CORS header at all, which is correct: nothing +in a browser should be able to use it. + +## Remote and self-hosted ClawMetry + +The API works the same when ClawMetry is not on the caller's machine. +Point at that host instead of `localhost:8900`. The key is the whole +gate: unlike the rest of the dashboard, this surface does not trust a +request just because it came from the local machine. + +If you use ClawMetry Cloud, note that session content is stored +end-to-end encrypted and decrypted in your browser, so the cloud cannot +serve it through a REST API even to you. Point custom UIs at the machine +your agents actually run on. + +## Using MCP instead + +If what you want is an agent that can *ask about* your runs rather than a +page that draws them, the MCP server is a better fit than this API: + +```bash +clawmetry mcp install +``` + +It exposes sessions, cost, traces and health as MCP tools over stdio, so +Claude Code and other MCP clients can query them directly. + +## When it does not work + +**The browser console says "blocked by CORS policy".** The origin your +page is served from is not on the key's list. The message names the +origin the browser sent; create a key with that exact origin, scheme and +port included. `http://localhost:3000` and `http://127.0.0.1:3000` are +different origins. + +**401 with "This API needs a key".** The `Authorization` header is not +arriving. Check for a proxy stripping it, and that the header is +`Authorization: Bearer cmk_...` with the word `Bearer`. + +**401 with "not valid on this machine".** The key was revoked, or it +belongs to another install. `clawmetry key list` shows what this machine +knows about. + +**403 with a scope name in it.** The key is real but does not cover that +query. The message says which scope is needed. + +**429.** More than 240 requests in a minute on one key. Poll less often, +or raise `CLAWMETRY_API_RATE_LIMIT` on the machine running ClawMetry. + +**503 saying the store could not be read.** Usually the sync daemon +restarting. If it persists, run `clawmetry doctor`. + +## Managing keys + +```bash +clawmetry key list # what this machine has issued +clawmetry key list --all # including revoked ones +clawmetry key scopes # what each scope grants +clawmetry key revoke # takes effect on the next request +``` + +The same panel lives in the dashboard under **Security**. + +Keys are stored in `~/.clawmetry/api_keys.json`, mode `0600`, as SHA-256 +hashes. A stolen file cannot be replayed as a key, and a key you lose +cannot be recovered: create a new one and revoke the old. diff --git a/docs/MODULE_MAP.md b/docs/MODULE_MAP.md index b19562a0bb..1d9debcd6c 100644 --- a/docs/MODULE_MAP.md +++ b/docs/MODULE_MAP.md @@ -4,7 +4,7 @@ > `python3 scripts/gen_module_map.py` (CI fails on drift via > `tests/test_module_map_drift.py`). -229 modules, 81 Flask blueprints. `CLAUDE.md` carries a short curated table of the ones you reach for most often; this is the whole list. +231 modules, 82 Flask blueprints. `CLAUDE.md` carries a short curated table of the ones you reach for most often; this is the whole list. Size bands are deliberately coarse so this file does not churn on every PR: **small** is under 200 lines, **medium** under 1k, **large** under 5k, **huge** is 5k and up. @@ -57,7 +57,7 @@ One module per feature, each owning one or more Flask blueprints. New endpoints | `routes/heartbeat.py` | medium | `bp_heartbeat` | `/api/heartbeat`, `/api/heartbeat-loops` | Heartbeat liveness panel API endpoint (#686). | | `routes/hitl.py` | medium | `bp_hitl` | `/api/hitl` | Human-in-the-loop (HITL) pause API. | | `routes/hooks.py` | large | `bp_hooks` | `/api/hooks`, `/api/lifecycle`, `/api/sessions` | local receiver for runtime pre-tool hooks. | -| `routes/infra.py` | large | `bp_config`, `bp_logs`, `bp_memory`, `bp_security` | `/api/automation-analysis`, `/api/context-anatomy`, `/api/cost-optimization`, `/api/cost-optimizer`, `/api/file`, `/api/flow`, `/api/flow-events`, `/api/llmfit`, `/api/logs`, `/api/logs-stream`, `/api/memory`, `/api/memory-access`, `/api/memory-analytics`, `/api/memory-files`, `/api/memory-rag`, `/api/numbat`, `/api/security` | Infrastructure / security / config / logs endpoints. | +| `routes/infra.py` | large | `bp_config`, `bp_logs`, `bp_memory`, `bp_security` | `/api/apikeys`, `/api/automation-analysis`, `/api/context-anatomy`, `/api/cost-optimization`, `/api/cost-optimizer`, `/api/file`, `/api/flow`, `/api/flow-events`, `/api/llmfit`, `/api/logs`, `/api/logs-stream`, `/api/memory`, `/api/memory-access`, `/api/memory-analytics`, `/api/memory-files`, `/api/memory-rag`, `/api/numbat`, `/api/security` | Infrastructure / security / config / logs endpoints. | | `routes/insights.py` | medium | `bp_insights` | `/api/insights`, `/insights` | Weekly Insights Digest endpoints. | | `routes/inventory.py` | medium | `bp_inventory` | `/api/inventory` | Agent Inventory tab API. | | `routes/local_query.py` | large | `bp_local_query` | `/__local_query__`, `/api/local` | coherent local query API over the DuckDB store. | @@ -70,6 +70,7 @@ One module per feature, each owning one or more Flask blueprints. New endpoints | `routes/paywall_lifecycle.py` | small | | | the paywall beacons that reach the funnel. | | `routes/plugins.py` | medium | `bp_plugins` | `/api/plugins` | Plugin registry: unified view of installed plugins (#692). | | `routes/policy.py` | medium | `bp_policy` | `/api/approvals`, `/api/approvals-audit`, `/api/policy`, `/api/tool-policy` | tool-policy + sandbox + exec-approval audit (PRD P1-1). | +| `routes/public_api.py` | medium | `bp_public_api` | `/api/q` | the keyed, cross-origin read API custom UIs use. | | `routes/quality.py` | medium | `bp_quality` | `/api/quality` | the Quality tab endpoint. | | `routes/readiness.py` | small | `bp_readiness` | `/api/repo-readiness` | ``bp_readiness`` — repo AI-readiness. | | `routes/reasoning.py` | medium | `bp_reasoning` | `/api/reasoning` | Reasoning chain viewer endpoint. | @@ -121,6 +122,7 @@ The pip-installable package: CLI, sync daemon, DuckDB store, detectors, enforcem | `clawmetry/_paywall.py` | medium | Shared 402 ``upgrade_required`` body builder for OSS stub blueprints. | | `clawmetry/_paywall_events.py` | large | In-process rolling store for ``POST /api/paywall/event`` client beacons. | | `clawmetry/alert_evaluator.py` | large | Local alert-rule evaluator — pure logic, no I/O (PRD #779 PR-D part 2). | +| `clawmetry/apikeys.py` | medium | scoped, revocable read keys for custom UIs. | | `clawmetry/approval_events.py` | small | The public seam between approvals and whoever delivers them. | | `clawmetry/approvals.py` | large | cloud-mediated approval policy engine. | | `clawmetry/attention_hook.py` | small | the `clawmetry hook attention` client. | diff --git a/docs/QUERY_CONTRACT.md b/docs/QUERY_CONTRACT.md index 29b3653174..8135d5683d 100644 --- a/docs/QUERY_CONTRACT.md +++ b/docs/QUERY_CONTRACT.md @@ -26,6 +26,24 @@ test enforces both directions). machine AES-256-GCM encrypted via the sync daemon snapshot path and must never appear on a plaintext push list. +## Read scopes + +Every method declares one scope. An API key issued to a custom UI +(`clawmetry key create --scope read:metrics`) may dispatch only the +methods whose scope it carries, so what a user picks when they create a +key is the same fact the server enforces. + +| Scope | Grants | Methods | +| - | - | - | +| `read:metrics` | Counts, tokens, cost and health. No prompts or replies. | `agent_graph`, `aggregates`, `health`, `models`, `runtimes` | +| `read:sessions` | One row per session: title, model, status, totals. | `rollup_sessions`, `search`, `sessions`, `similar_sessions` | +| `read:traces` | Spans, traces and outbound API calls. | `external_calls`, `spans`, `traces` | +| `read:content` | The turns themselves: prompts, replies, tool calls. | `events`, `replay_events`, `session_context`, `transcript`, `transcript_page` | + +`read:metrics` is exactly the `plaintext` trust class: a metrics-scoped +key can never return a prompt, a reply or a file path. That invariant is +pinned by CI, not by convention. + ## Non-goals * No per-model data in the device-facing `glance` method. Devices get @@ -33,29 +51,29 @@ test enforces both directions). ## Methods -| Method | Status | Trust | Backing | Args | Description | -| - | - | - | - | - | - | -| `agent_graph` | live | plaintext | `query_agent_graph` | `runtime`, `since`, `until`, `limit` (default 500, range 1..2000) | Cross-session agent spawn graph: nodes (agent_type+id stats) + spawn edges. Optional runtime arg scopes to one runtime ('openclaw' matches legacy NULL agent_type). | -| `aggregates` | live | plaintext | `query_aggregates` | `agent_id`, `since`, `until` | Per-day rollup of events/tokens/cost (aggregate counters only). | -| `events` | live | e2e | `query_events` | `session_id`, `agent_id`, `event_type`, `since`, `until`, `limit` (default 200, range 1..5000) | Raw event rows (tool calls, messages, errors), newest first. | -| `external_calls` | live | e2e | `query_external_calls` | `session_id`, `since`, `until`, `limit` (default 200, range 1..2000) | External (non-LLM) API calls captured by the interceptor. | -| `health` | live | plaintext | `health` | (none) | Store health snapshot (engine, size, ring depth, flush age). | -| `models` | live | plaintext | `query_rollup_model_daily` | `runtime`, `since`, `until`, `limit` (default 1000, range 1..10000) | Per-model daily token/cost rollup across runtimes. | -| `replay_events` | live | e2e | `query_replay_events` | `session_id` (required), `limit` (default 2000, range 1..10000) | Canonical replay-event rows for one session (#4813). Rows in kind-agnostic order; the /api/replay-tree endpoint groups them into turns/delegations/workflows/approvals. | -| `rollup_sessions` | live | e2e | `query_rollup_sessions` | `runtime`, `limit` (default 200, range 1..2000) | Per-session materialized summary (title, status, totals, stuck flag). | -| `runtimes` | live | plaintext | `query_rollup_runtime_daily` | `since`, `until`, `limit` (default 1000, range 1..10000) | Per-runtime daily activity/cost rollup (claude_code, openclaw, ...). | -| `search` | live | e2e | `query_search` | `q` (required), `model`, `status`, `since`, `until`, `limit` (default 50, range 1..500) | Full-text search over session titles and eval reasons. | -| `session_context` | live | e2e | `query_session_context` | `session_id` (required), `agent_type`, `limit` (default 200, range 1..1000) | Inputs & context rows for one session: system prompt, first user prompt, tool definitions, MCP servers, context files and runtime setup captured from context.compiled events. Content is redacted + capped; sha256/size describe the full text. | -| `sessions` | live | e2e | `query_sessions` | `agent_id`, `since`, `until`, `limit` (default 100, range 1..2000) | One row per session_id with start/end, event count, cost. | -| `similar_sessions` | live | e2e | `query_similar_sessions` | `session_id` (required), `window_days` (default 30, range 1..365), `limit` (default 10, range 1..50) | Runs shaped like this one (WO-60): nearest sessions by tool-call n-gram similarity inside a window, same runtime first, with score, runtime, model, cost, outcome. Carries session titles, so content class. | -| `spans` | live | e2e | `query_spans` | `trace_id`, `session_id`, `agent_type`, `since`, `until`, `limit` (default 200, range 1..2000) | OTel span rows with full filters (trace/session/agent/time). | -| `traces` | live | e2e | `query_traces` | `session_id`, `agent_type`, `since`, `until`, `limit` (default 100, range 1..1000) | One row per trace_id with aggregate span stats. | -| `transcript` | live | e2e | `query_events` | `session_id` (required), `limit` (default 500, range 1..5000) | Alias of events scoped to one required session_id. | -| `transcript_page` | live | e2e | `query_transcript_page` | `session_id` (required), `before_ts`, `limit` (default 150, range 1..250) | One older-history page of a session's events, newest-first. before_ts is an exclusive ms-epoch cursor (pass the previous page's next_before_ts to walk backward). Returns {rows, count, has_more, next_before_ts}. | -| `approvals` | planned | plaintext | `query_approvals` | `status`, `limit` (default 100, range 1..1000) | Approval queue metadata (ids, states, timestamps; no content). | -| `brain` | planned | e2e | `query_events` | `session_id`, `since`, `limit` (default 200, range 1..2000) | Reasoning/tool event slice powering the Brain feed. | -| `glance` | planned | plaintext | `rollup_glance` | (none) | Device-facing top-line counters (sessions, cost, alerts). Non-goal: no per-model data in glance. | -| `session` | planned | e2e | `query_sessions_table` | `session_id` (required) | Single-session detail row (title, status, outcome, totals). | -| `usage` | planned | plaintext | `rollup_usage_daily` | `runtime`, `since`, `until` | Daily token/cost usage series (input/output/cache splits). | +| Method | Status | Trust | Scope | Backing | Args | Description | +| - | - | - | - | - | - | - | +| `agent_graph` | live | plaintext | `read:metrics` | `query_agent_graph` | `runtime`, `since`, `until`, `limit` (default 500, range 1..2000) | Cross-session agent spawn graph: nodes (agent_type+id stats) + spawn edges. Optional runtime arg scopes to one runtime ('openclaw' matches legacy NULL agent_type). | +| `aggregates` | live | plaintext | `read:metrics` | `query_aggregates` | `agent_id`, `since`, `until` | Per-day rollup of events/tokens/cost (aggregate counters only). | +| `events` | live | e2e | `read:content` | `query_events` | `session_id`, `agent_id`, `event_type`, `since`, `until`, `limit` (default 200, range 1..5000) | Raw event rows (tool calls, messages, errors), newest first. | +| `external_calls` | live | e2e | `read:traces` | `query_external_calls` | `session_id`, `since`, `until`, `limit` (default 200, range 1..2000) | External (non-LLM) API calls captured by the interceptor. | +| `health` | live | plaintext | `read:metrics` | `health` | (none) | Store health snapshot (engine, size, ring depth, flush age). | +| `models` | live | plaintext | `read:metrics` | `query_rollup_model_daily` | `runtime`, `since`, `until`, `limit` (default 1000, range 1..10000) | Per-model daily token/cost rollup across runtimes. | +| `replay_events` | live | e2e | `read:content` | `query_replay_events` | `session_id` (required), `limit` (default 2000, range 1..10000) | Canonical replay-event rows for one session (#4813). Rows in kind-agnostic order; the /api/replay-tree endpoint groups them into turns/delegations/workflows/approvals. | +| `rollup_sessions` | live | e2e | `read:sessions` | `query_rollup_sessions` | `runtime`, `limit` (default 200, range 1..2000) | Per-session materialized summary (title, status, totals, stuck flag). | +| `runtimes` | live | plaintext | `read:metrics` | `query_rollup_runtime_daily` | `since`, `until`, `limit` (default 1000, range 1..10000) | Per-runtime daily activity/cost rollup (claude_code, openclaw, ...). | +| `search` | live | e2e | `read:sessions` | `query_search` | `q` (required), `model`, `status`, `since`, `until`, `limit` (default 50, range 1..500) | Full-text search over session titles and eval reasons. | +| `session_context` | live | e2e | `read:content` | `query_session_context` | `session_id` (required), `agent_type`, `limit` (default 200, range 1..1000) | Inputs & context rows for one session: system prompt, first user prompt, tool definitions, MCP servers, context files and runtime setup captured from context.compiled events. Content is redacted + capped; sha256/size describe the full text. | +| `sessions` | live | e2e | `read:sessions` | `query_sessions` | `agent_id`, `since`, `until`, `limit` (default 100, range 1..2000) | One row per session_id with start/end, event count, cost. | +| `similar_sessions` | live | e2e | `read:sessions` | `query_similar_sessions` | `session_id` (required), `window_days` (default 30, range 1..365), `limit` (default 10, range 1..50) | Runs shaped like this one (WO-60): nearest sessions by tool-call n-gram similarity inside a window, same runtime first, with score, runtime, model, cost, outcome. Carries session titles, so content class. | +| `spans` | live | e2e | `read:traces` | `query_spans` | `trace_id`, `session_id`, `agent_type`, `since`, `until`, `limit` (default 200, range 1..2000) | OTel span rows with full filters (trace/session/agent/time). | +| `traces` | live | e2e | `read:traces` | `query_traces` | `session_id`, `agent_type`, `since`, `until`, `limit` (default 100, range 1..1000) | One row per trace_id with aggregate span stats. | +| `transcript` | live | e2e | `read:content` | `query_events` | `session_id` (required), `limit` (default 500, range 1..5000) | Alias of events scoped to one required session_id. | +| `transcript_page` | live | e2e | `read:content` | `query_transcript_page` | `session_id` (required), `before_ts`, `limit` (default 150, range 1..250) | One older-history page of a session's events, newest-first. before_ts is an exclusive ms-epoch cursor (pass the previous page's next_before_ts to walk backward). Returns {rows, count, has_more, next_before_ts}. | +| `approvals` | planned | plaintext | `read:metrics` | `query_approvals` | `status`, `limit` (default 100, range 1..1000) | Approval queue metadata (ids, states, timestamps; no content). | +| `brain` | planned | e2e | `read:content` | `query_events` | `session_id`, `since`, `limit` (default 200, range 1..2000) | Reasoning/tool event slice powering the Brain feed. | +| `glance` | planned | plaintext | `read:metrics` | `rollup_glance` | (none) | Device-facing top-line counters (sessions, cost, alerts). Non-goal: no per-model data in glance. | +| `session` | planned | e2e | `read:sessions` | `query_sessions_table` | `session_id` (required) | Single-session detail row (title, status, outcome, totals). | +| `usage` | planned | plaintext | `read:metrics` | `rollup_usage_daily` | `runtime`, `since`, `until` | Daily token/cost usage series (input/output/cache splits). | Live methods: 17. Planned methods: 5. diff --git a/examples/custom-ui/README.md b/examples/custom-ui/README.md new file mode 100644 index 0000000000..6c4dde1132 --- /dev/null +++ b/examples/custom-ui/README.md @@ -0,0 +1,43 @@ +# Starter custom UI + +One HTML file, no build step, no dependencies. It reads the ClawMetry +`q/1` API and draws 30 days of agent spend by day, runtime and model. + +It exists to be edited. Open it, change it, or hand it to a coding agent +along with the API guide and ask for the view you actually want. + +## Run it + +```bash +# 1. serve this directory on a port +cd examples/custom-ui && python3 -m http.server 3000 + +# 2. create a key that names that origin +clawmetry key create --name starter \ + --scope read:metrics --origin http://localhost:3000 + +# 3. open http://localhost:3000 and paste the key +``` + +The key is kept in this browser's local storage and sent to your own +ClawMetry, nowhere else. + +## Why `read:metrics` + +This page has no backend, so any key it holds lives in the browser. +`read:metrics` is the scope that covers counts, tokens, cost and health +and cannot return a prompt, a reply or a file path. If you extend the +page into something that needs session content, put a backend in front of +it and keep the wider key there. + +## Making it yours + +The API describes itself. Fetch the guide with your key and give it to a +coding agent with a description of what you want: + +```bash +curl -H "Authorization: Bearer cmk_..." \ + http://localhost:8900/api/q/1/llms.txt +``` + +Full walkthrough: [`docs/BUILD_YOUR_OWN_UI.md`](../../docs/BUILD_YOUR_OWN_UI.md). diff --git a/examples/custom-ui/index.html b/examples/custom-ui/index.html new file mode 100644 index 0000000000..4b4801c074 --- /dev/null +++ b/examples/custom-ui/index.html @@ -0,0 +1,303 @@ + + + + + +Agent spend + + + + +
+

Agent spend

+
Reading ClawMetry directly. Nothing leaves this machine.
+ + + + + + +
+ + + + diff --git a/routes/infra.py b/routes/infra.py index cc706221ea..fdb22137df 100644 --- a/routes/infra.py +++ b/routes/infra.py @@ -28,6 +28,7 @@ from __future__ import annotations import json +import logging import os import re import sqlite3 @@ -39,6 +40,8 @@ from clawmetry._gate import gate from clawmetry.config import is_local_store_read_enabled, hide_clawmetry_session +logger = logging.getLogger("clawmetry.routes.infra") + bp_logs = Blueprint('logs', __name__) bp_memory = Blueprint('memory', __name__) bp_security = Blueprint('security', __name__) @@ -2936,3 +2939,104 @@ def api_security_retention_set(): except Exception: state = _ret.resolve(store=_retention_store()) return jsonify({"ok": True, **state}) + + +# ── API keys for custom UIs (docs/BUILD_YOUR_OWN_UI.md) ───────────────── +# +# These MINT and REVOKE credentials, so they live here -- behind the +# dashboard's own gate -- and not in routes/public_api.py, which is the +# cross-origin surface those credentials open. A page on someone else's +# site can hold a read key; it must never be able to issue itself a +# better one. dashboard.py's `_cross_origin_write_blocked` refuses a +# cross-origin POST to these, and `_add_cors` in public_api.py is pinned +# to /api/q/ so no CORS header ever reaches them. + + +@bp_security.route("/api/apikeys", methods=["GET"]) +def api_keys_list(): + """This machine's API keys, plus the scope catalogue the UI renders. + + Secrets are never included -- only the SHA-256 is stored and even + that is stripped by ``apikeys.list_keys``. + """ + from clawmetry import apikeys as _ak + try: + return jsonify({ + "ok": True, + "keys": _ak.list_keys(include_revoked=True), + "scopes": _ak.scope_catalogue(), + "summary": _ak.store_summary(), + }) + except Exception as exc: + logger.warning("apikeys list failed: %s", exc) + return jsonify({ + "ok": False, + "keys": [], + "scopes": [], + "error": "ClawMetry could not read its key file. Check that " + "~/.clawmetry is readable by you.", + }), 200 + + +@bp_security.route("/api/apikeys", methods=["POST"]) +def api_keys_create(): + """Mint a key. The secret is in the response and nowhere else, ever.""" + from clawmetry import apikeys as _ak + body = request.get_json(silent=True) or {} + origins = body.get("origins") or [] + if isinstance(origins, str): + origins = [o.strip() for o in origins.replace(",", " ").split() if o.strip()] + scopes = body.get("scopes") or [] + if isinstance(scopes, str): + scopes = [s.strip() for s in scopes.replace(",", " ").split() if s.strip()] + browser = bool(body.get("browser", True)) + if browser and not origins: + return jsonify({ + "ok": False, + "error": "Name the site that will use this key, for example " + "http://localhost:3000. There is no wildcard: any page " + "in any tab can already reach this machine, and the " + "origin list is what stops it reading the answer.", + }), 400 + try: + record, plaintext = _ak.create( + body.get("name") or "", + scopes, + [] if not browser else origins, + note=body.get("note") or "", + ) + except _ak.ApiKeyError as exc: + return jsonify({"ok": False, "error": str(exc)}), 400 + except Exception as exc: + logger.warning("apikeys create failed: %s", exc) + return jsonify({ + "ok": False, + "error": "ClawMetry could not write its key file. Check that " + "~/.clawmetry is writable by you.", + }), 500 + return jsonify({ + "ok": True, + "key": plaintext, + "record": {k: v for k, v in record.items() if k != "hash"}, + }) + + +@bp_security.route("/api/apikeys/", methods=["DELETE"]) +def api_keys_revoke(key_id: str): + """Revoke a key. Takes effect on that key's next request.""" + from clawmetry import apikeys as _ak + try: + ok = _ak.revoke(key_id) + except Exception as exc: + logger.warning("apikeys revoke failed: %s", exc) + return jsonify({ + "ok": False, + "error": "ClawMetry could not write its key file. Check that " + "~/.clawmetry is writable by you.", + }), 500 + if not ok: + return jsonify({ + "ok": False, + "error": "There is no active key with that id on this machine.", + }), 404 + return jsonify({"ok": True, "id": key_id}) diff --git a/routes/public_api.py b/routes/public_api.py new file mode 100644 index 0000000000..c3ee484894 --- /dev/null +++ b/routes/public_api.py @@ -0,0 +1,420 @@ +"""routes/public_api.py -- the keyed, cross-origin read API custom UIs use. + +This is the "build your own UI" surface (docs/BUILD_YOUR_OWN_UI.md). It +serves the declared ``q/1`` query contract to anything holding a scoped +API key: a page vibe-coded on v0 or Lovable, a Grafana-ish panel, a +terminal script, an agent with curl. + + GET /api/q/1 what this key can read + GET /api/q/1/llms.txt the whole API, written for an agent + GET /api/q/1/? one query + +Every response is JSON. Every shape, arg and default comes from +``clawmetry/query_contract.py`` and every query goes through +``routes.local_query._dispatch`` -- the same code path the dashboard +uses, so there is no second SQL surface to keep correct and no way for a +key to reach a query the contract does not declare. + +What makes this different from the rest of the dashboard +-------------------------------------------------------- +Everywhere else, a request from ``127.0.0.1`` is trusted, because a +local tool talking to itself is the normal case. Here it is not, and +that inversion is the entire security design: + +* **A key is always required.** Loopback earns nothing. Without a key + this surface behaves as if it does not exist. +* **CORS is per key, never global.** ``Access-Control-Allow-Origin`` is + echoed only for an origin the key holder named when they created the + key. There is no wildcard and no way to ask for one. This matters more + than it looks: any page in any tab can already send a request to + ``127.0.0.1:8900``, and the only reason that has been harmless is that + the browser will not let the page read the reply. That protection is + what we are selectively removing, one named origin at a time. +* **Read only.** The dispatch table is the q/1 read contract. Nothing + here can pause, stop or kill an agent, and nothing here writes. This + adds no entry to the control-plane surfaces listed in CLAUDE.md: a key + cannot cause a write, so the "no surprise writes" rule has nothing to + bite on here. +* **GET only.** Every q/1 arg is a scalar, so nothing needs a body. + Being GET-only means this API can never be the target of a + cross-origin write, which removes a whole class of question. +""" + +from __future__ import annotations + +import logging +import os +import time +from collections import deque + +from flask import Blueprint, Response, jsonify, request + +from clawmetry import apikeys +from clawmetry.query_contract import ( + CONTRACT_VERSION, + QUERY_CONTRACT, + SCOPE_DOC, + SCOPES, + STATUS_LIVE, +) + +logger = logging.getLogger("clawmetry.routes.public_api") + +bp_public_api = Blueprint("public_api", __name__) + +#: Requests per key per minute. Generous enough for a page polling every +#: second with a few panels, low enough that a runaway loop in someone's +#: custom UI cannot pin the daemon's DuckDB connection (see the CPU +#: budget in FLYWHEEL.md 1e). +RATE_LIMIT_PER_MIN = int(os.environ.get("CLAWMETRY_API_RATE_LIMIT", "240") or 240) + +_RATE: dict = {} +_RATE_WINDOW_SEC = 60.0 + +#: Flask stashes the resolved key record here so ``_add_cors`` can echo +#: the right origin after the view has run. +_G_KEY = "_cm_api_key_record" + + +def _rate_limited(key_id: str) -> bool: + """True when this key has spent its minute. Sliding window, in memory. + + Per process and not shared with the daemon, which is the honest + scope: this is a runaway-loop guard for a local API, not a billing + meter. + """ + if RATE_LIMIT_PER_MIN <= 0: + return False + now = time.monotonic() + hits = _RATE.setdefault(key_id, deque()) + cutoff = now - _RATE_WINDOW_SEC + while hits and hits[0] < cutoff: + hits.popleft() + if len(hits) >= RATE_LIMIT_PER_MIN: + return True + hits.append(now) + return False + + +def _err(status: int, message: str, **extra): + """A failure a person can act on. + + Never an upstream code on its own: the body always carries a + sentence saying what happened and what to do next, because these + land in someone's browser console while they are building. + """ + body = {"error": message, "contract": CONTRACT_VERSION} + body.update(extra) + return jsonify(body), status + + +# ── auth + CORS ───────────────────────────────────────────────────────── + +def _presented_key() -> str: + """The key on this request. ``Authorization: Bearer`` is the documented + form; ``X-ClawMetry-Key`` exists for clients that cannot set it.""" + auth = (request.headers.get("Authorization") or "").strip() + if auth.lower().startswith("bearer "): + return auth[7:].strip() + return (request.headers.get("X-ClawMetry-Key") or "").strip() + + +def _authenticate(): + """``(record, None)`` on success, ``(None, response)`` on failure.""" + presented = _presented_key() + if not presented: + return None, _err( + 401, + "This API needs a key. Create one with: clawmetry key create " + "--name my-ui --scope read:metrics --origin https://example.com, " + "then send it as: Authorization: Bearer cmk_...", + docs="/api/q/1/llms.txt", + ) + record = apikeys.verify(presented) + if not record: + return None, _err( + 401, + "That key is not valid on this machine. It may have been revoked, " + "or it may belong to a different ClawMetry install. List the keys " + "this machine knows with: clawmetry key list", + docs="/api/q/1/llms.txt", + ) + if _rate_limited(str(record.get("id"))): + return None, _err( + 429, + f"This key has made more than {RATE_LIMIT_PER_MIN} requests in the " + "last minute and is being throttled. Poll less often, or raise " + "CLAWMETRY_API_RATE_LIMIT on the machine running ClawMetry.", + ) + from flask import g + + setattr(g, _G_KEY, record) + return record, None + + +@bp_public_api.after_request +def _add_cors(response): + """Echo CORS headers for an origin THIS key authorised, and no other. + + Blueprint-scoped on purpose: nothing else in the dashboard gains a + CORS header from this file existing. + """ + from flask import g + + origin = (request.headers.get("Origin") or "").strip() + if not origin: + return response # not a browser; nothing to negotiate + if not (request.path or "").startswith("/api/q/"): + # Belt and braces. Every rule in this blueprint is under /api/q/ + # today, and this makes sure a route added here later cannot + # inherit cross-origin readability by accident. Key MANAGEMENT + # (minting, listing, revoking) deliberately lives in + # routes/infra.py, behind the dashboard's own same-origin gate. + return response + record = getattr(g, _G_KEY, None) + if record is not None: + allowed = apikeys.origin_allowed(record, origin) + else: + # Preflight, or a request that failed auth. A preflight carries + # no Authorization header, so the only question we can answer is + # whether the user has authorised this origin for any live key. + # The real request is still checked against its own key. + allowed = apikeys.any_key_allows_origin(origin) + if not allowed: + return response + response.headers["Access-Control-Allow-Origin"] = origin + response.headers["Vary"] = "Origin" + response.headers["Access-Control-Allow-Methods"] = "GET, OPTIONS" + response.headers["Access-Control-Allow-Headers"] = ( + "Authorization, X-ClawMetry-Key, Content-Type" + ) + response.headers["Access-Control-Max-Age"] = "600" + return response + + +# CORS preflight needs no view of its own: Flask answers OPTIONS for every +# rule below automatically, and ``_add_cors`` decides whether that answer +# carries permission. A preflight arrives with no Authorization header, so +# the only question it can answer is "has the user authorised this origin +# for any live key" -- and an origin nobody named gets no header, which is +# what makes the browser abandon the request before it is ever sent. + + +# ── the index ─────────────────────────────────────────────────────────── + +def _shape_spec(name: str) -> dict: + spec = QUERY_CONTRACT[name] + return { + "shape": name, + "scope": spec["scope"], + "description": spec["doc"], + "args": { + arg: dict(meta, required=bool(meta.get("required"))) + for arg, meta in spec["args"].items() + }, + "url": f"/api/q/1/{name}", + } + + +@bp_public_api.route("/api/q/1", methods=["GET"]) +def q_index(): + """What this key can read. The first call a custom UI should make.""" + record, failure = _authenticate() + if failure: + return failure + granted = sorted(apikeys.granted_shapes(record)) + return jsonify({ + "contract": CONTRACT_VERSION, + "key": { + "id": record.get("id"), + "name": record.get("name"), + "scopes": record.get("scopes") or [], + "origins": record.get("origins") or [], + }, + "scopes": [ + {"scope": s, "grants": SCOPE_DOC[s], "held": s in (record.get("scopes") or [])} + for s in SCOPES + ], + "shapes": [_shape_spec(n) for n in granted], + "docs": "/api/q/1/llms.txt", + }) + + +# ── the agent-readable guide ──────────────────────────────────────────── + +def _llms_txt(record: dict) -> str: + """The whole API as plain text, generated from the contract. + + Written to be pasted into a coding agent. It is generated rather + than authored so it can never describe a shape that is not served, + and it is scoped to the presented key so an agent is never told + about a query it will get a 403 for. + """ + granted = sorted(apikeys.granted_shapes(record)) + host = request.host_url.rstrip("/") + lines = [ + "# ClawMetry query API (%s)" % CONTRACT_VERSION, + "", + "Read-only telemetry for AI agent runs on this machine: sessions,", + "tokens, cost, tool calls, traces. Use it to build a custom UI.", + "", + "## Auth", + "", + "Send the key on every request:", + "", + " Authorization: Bearer ", + "", + "All requests are GET. All responses are JSON. Errors are", + '{"error": ""} with a 4xx status.', + "", + "## Base URL", + "", + " %s/api/q/1" % host, + "", + "## Response shape", + "", + "Row-returning queries answer:", + "", + ' {"shape": "sessions", "rows": [...], "count": 12,', + ' "contract": "q/1", "elapsed_ms": 8}', + "", + "`health`, `agent_graph`, `transcript_page` and `similar_sessions`", + "answer an object instead of `rows`. Read `shape` to tell them apart.", + "", + "## Queries this key can make", + "", + ] + for name in granted: + spec = QUERY_CONTRACT[name] + lines.append("### GET /api/q/1/%s" % name) + lines.append("") + lines.append(spec["doc"]) + lines.append("") + if spec["args"]: + lines.append("Query parameters:") + for arg, meta in spec["args"].items(): + bits = [] + if meta.get("required"): + bits.append("required") + if "default" in meta: + bits.append("default %s" % meta["default"]) + if "lo" in meta and "hi" in meta: + bits.append("%s..%s" % (meta["lo"], meta["hi"])) + suffix = (" (%s)" % ", ".join(bits)) if bits else "" + lines.append(" - %s%s" % (arg, suffix)) + else: + lines.append("No parameters.") + lines.append("") + missing = [s for s in SCOPES if s not in (record.get("scopes") or [])] + if missing: + lines += [ + "## Not available to this key", + "", + "This key does not hold:", + "", + ] + for s in missing: + lines.append(" - %s: %s" % (s, SCOPE_DOC[s])) + lines += [ + "", + "The person running ClawMetry can issue a key with more scopes:", + "", + " clawmetry key create --name my-ui --scope %s --origin " + % " --scope ".join(list((record.get("scopes") or [])) + missing[:1]), + "", + ] + lines += [ + "## Notes", + "", + "- Timestamps in `since` / `until` are ISO 8601, e.g. 2026-09-01T00:00:00Z.", + "- `limit` is clamped to the range shown; asking for more is not an error.", + "- On a free plan, `events` returns the last 24 hours and the response", + " carries `capped_at_24h: true`. Other queries are not time-capped.", + "- The key is a secret. Put it in a server-side env var where you can;", + " if it must live in the browser, scope it to `read:metrics`.", + "", + ] + return "\n".join(lines) + + +@bp_public_api.route("/api/q/1/llms.txt", methods=["GET"]) +def q_llms_txt(): + """The API, written for a coding agent to read in one pass.""" + record, failure = _authenticate() + if failure: + return failure + return Response(_llms_txt(record), mimetype="text/plain; charset=utf-8") + + +# ── the query ─────────────────────────────────────────────────────────── + +@bp_public_api.route("/api/q/1/", methods=["GET"]) +def q_shape(shape: str): + """Run one declared q/1 query and return its rows.""" + record, failure = _authenticate() + if failure: + return failure + + spec = QUERY_CONTRACT.get(shape) + if spec is None or spec["status"] != STATUS_LIVE: + # A planned-but-unserved shape and a typo get the same answer on + # purpose: the caller's next step is identical either way. + return _err( + 404, + f"There is no query called {shape!r}. Ask GET /api/q/1 for the " + "list this key can run.", + docs="/api/q/1/llms.txt", + ) + if shape not in apikeys.granted_shapes(record): + needed = spec["scope"] + return _err( + 403, + f"This key cannot read {shape!r}. It needs the {needed} scope " + f"({SCOPE_DOC[needed]}) and holds " + f"{', '.join(record.get('scopes') or []) or 'none'}. Issue a new " + f"key with: clawmetry key create --name my-ui --scope {needed} " + "--origin ", + required_scope=needed, + held_scopes=record.get("scopes") or [], + ) + + from routes import local_query as _lq + + try: + args = _lq._coerce_args(shape, request.args.to_dict()) + except ValueError as exc: + # _coerce_args raises only for a missing required arg, and its + # message already names it. + return _err(400, str(exc)[:200], docs=f"/api/q/1/{shape}") + + capped = False + if shape == "events": + # Same retention cap the dashboard's own /api/local/events applies + # (issue #1448): free plans see the last 24h of raw events. Doing + # it here too means a custom UI and the dashboard never disagree + # about how much history exists. + capped = _lq._apply_24h_cap(args) + + started = time.monotonic() + try: + body = _lq._dispatch(shape, args) + except Exception as exc: + logger.warning("public api: %s failed for key %s: %s", + shape, record.get("id"), exc) + # The upstream message can carry a DuckDB path or a column name. + # Neither helps the person building a UI, and both are ours. + return _err( + 503, + "ClawMetry could not read its local store just now. This is " + "usually the sync daemon restarting; try again in a moment. If it " + "persists, run: clawmetry doctor", + ) + + out = {k: v for k, v in body.items() if not k.startswith("_")} + out["shape"] = shape + out["contract"] = CONTRACT_VERSION + out["elapsed_ms"] = int((time.monotonic() - started) * 1000) + if shape == "events": + out["capped_at_24h"] = capped + apikeys.touch(str(record.get("id"))) + return jsonify(out) diff --git a/scripts/gen_query_contract_doc.py b/scripts/gen_query_contract_doc.py index 690682322c..4c8ff47e51 100644 --- a/scripts/gen_query_contract_doc.py +++ b/scripts/gen_query_contract_doc.py @@ -21,6 +21,8 @@ from clawmetry.query_contract import ( # noqa: E402 CONTRACT_VERSION, QUERY_CONTRACT, + SCOPE_DOC, + SCOPES, STATUS_LIVE, ) @@ -54,6 +56,19 @@ machine AES-256-GCM encrypted via the sync daemon snapshot path and must never appear on a plaintext push list. +## Read scopes + +Every method declares one scope. An API key issued to a custom UI +(`clawmetry key create --scope read:metrics`) may dispatch only the +methods whose scope it carries, so what a user picks when they create a +key is the same fact the server enforces. + +{{scope_table}} + +`read:metrics` is exactly the `plaintext` trust class: a metrics-scoped +key can never return a prompt, a reply or a file path. That invariant is +pinned by CI, not by convention. + ## Non-goals * No per-model data in the device-facing `glance` method. Devices get @@ -63,6 +78,21 @@ """ +def _scope_table() -> str: + rows = ["| Scope | Grants | Methods |", "| - | - | - |"] + for scope in SCOPES: + served = sorted( + n for n, s in QUERY_CONTRACT.items() + if s["scope"] == scope and s["status"] == STATUS_LIVE + ) + rows.append( + f"| `{scope}` | {SCOPE_DOC[scope]} | " + + ", ".join(f"`{n}`" for n in served) + + " |" + ) + return "\n".join(rows) + + def _fmt_arg(name: str, spec: dict) -> str: bits = [] if spec.get("required"): @@ -75,15 +105,15 @@ def _fmt_arg(name: str, spec: dict) -> str: def render() -> str: - lines = [_HEADER] - lines.append("| Method | Status | Trust | Backing | Args | Description |") - lines.append("| - | - | - | - | - | - |") + lines = [_HEADER.replace("{scope_table}", _scope_table())] + lines.append("| Method | Status | Trust | Scope | Backing | Args | Description |") + lines.append("| - | - | - | - | - | - | - |") for name in sorted(QUERY_CONTRACT, key=lambda n: (QUERY_CONTRACT[n]["status"] != STATUS_LIVE, n)): spec = QUERY_CONTRACT[name] args = ", ".join(_fmt_arg(a, s) for a, s in spec["args"].items()) or "(none)" lines.append( f"| `{name}` | {spec['status']} | {spec['trust']} | " - f"`{spec['backing']}` | {args} | {spec['doc']} |" + f"`{spec['scope']}` | `{spec['backing']}` | {args} | {spec['doc']} |" ) lines.append("") live = [n for n, s in QUERY_CONTRACT.items() if s["status"] == STATUS_LIVE] diff --git a/tests/test_public_api_keys.py b/tests/test_public_api_keys.py new file mode 100644 index 0000000000..71decf41f8 --- /dev/null +++ b/tests/test_public_api_keys.py @@ -0,0 +1,532 @@ +"""Guards for the keyed read API custom UIs are built on. + +Three things are being protected here, in descending order of how badly +they would hurt if they broke: + +1. **The CORS gate.** ``/api/q/`` is the only surface in ClawMetry that + answers a cross-origin browser read. If ``Access-Control-Allow-Origin`` + ever went out for an origin the key did not name, any page in any tab + could read the machine's telemetry. Several tests here exist purely to + fail loudly if that becomes possible. +2. **The scope gate.** A key must reach exactly the queries its scopes + cover. ``read:metrics`` in particular must never be able to return a + prompt or a reply, which is pinned to the contract's own trust class + rather than to a hand-kept list. +3. **Key handling.** Secrets hashed and never stored in the clear, the + file 0600, revocation effective on the next request. + +The management endpoints (mint / list / revoke) are checked too, because +they live on the ordinary dashboard blueprint on purpose and must stay +there: a read key must never be able to issue itself a better one. +""" +from __future__ import annotations + +import importlib +import json +import os +import stat + +import pytest + + +# ── fixtures ──────────────────────────────────────────────────────────── + +@pytest.fixture +def ak(monkeypatch, tmp_path): + """The apikeys module pointed at a throwaway store. + + tests/ has burned this repo before by writing into the operator's real + ~/.clawmetry, so the path override is set before the module is used + and asserted below. + """ + store = tmp_path / "api_keys.json" + monkeypatch.setenv("CLAWMETRY_API_KEYS_PATH", str(store)) + import clawmetry.apikeys as mod + + importlib.reload(mod) + assert mod._store_path() == str(store) + return mod + + +@pytest.fixture +def client(ak, monkeypatch): + """A Flask test client with just the two blueprints under test. + + Deliberately NOT the whole dashboard app: booting it drags in DuckDB + and the runtime probes, and the thing being tested is the auth and + CORS behaviour of these routes, which is entirely self-contained. + ``_dispatch`` is stubbed so no test needs a populated store. + """ + from flask import Flask + + import routes.local_query as lq + import routes.public_api as pub + + importlib.reload(pub) + + calls = [] + + def _fake_dispatch(shape, args): + calls.append((shape, dict(args))) + return {"rows": [{"shape_echo": shape}], "count": 1, + "_via": "test", "_shape": shape} + + monkeypatch.setattr(lq, "_dispatch", _fake_dispatch) + monkeypatch.setattr(lq, "_apply_24h_cap", lambda args: False) + + app = Flask(__name__) + app.register_blueprint(pub.bp_public_api) + c = app.test_client() + c.dispatch_calls = calls + c.pub = pub + return c + + +def _mint(ak, name="t", scopes=("read:metrics",), origins=("http://localhost:3000",)): + _, plaintext = ak.create(name, list(scopes), list(origins)) + return plaintext + + +def _auth(key): + return {"Authorization": "Bearer " + key} + + +# ── the contract declares a scope for every method ────────────────────── + +def test_every_contract_method_declares_a_known_scope(): + from clawmetry import query_contract as qc + + for name, spec in qc.QUERY_CONTRACT.items(): + assert spec.get("scope") in qc.SCOPES, ( + f"{name} declares scope {spec.get('scope')!r}, which is not one of " + f"{qc.SCOPES}. A method with no scope is unreachable by any key." + ) + + +def test_scopes_partition_every_live_shape(): + """A live shape covered by no scope would be dead to the API forever, + and one covered by two would make the grant ambiguous.""" + from clawmetry import query_contract as qc + + live = set(qc.live_shapes()) + covered = qc.shapes_for_scopes(qc.SCOPES) + assert covered == live, f"uncovered: {live - covered}, phantom: {covered - live}" + seen = set() + for scope in qc.SCOPES: + got = set(qc.live_methods_by_scope(scope)) + assert not (got & seen), f"{scope} overlaps an earlier scope: {got & seen}" + seen |= got + + +def test_read_metrics_is_exactly_the_plaintext_trust_class(): + """The load-bearing invariant behind the whole scope model. + + ``read:metrics`` is what a browser-resident key should hold, and the + promise attached to it is "this cannot return a prompt or a reply". + That promise is only true while metrics-scope and plaintext-trust are + the same set, so it is pinned rather than trusted. + """ + from clawmetry import query_contract as qc + + metrics = set(qc.methods_by_scope(qc.SCOPE_METRICS)) + plaintext = set(qc.methods_by_trust(qc.TRUST_PLAINTEXT)) + assert metrics == plaintext, ( + "read:metrics and the plaintext trust class have diverged. " + f"metrics-only={metrics - plaintext} plaintext-only={plaintext - metrics}. " + "Either move the method to another scope or reclassify its trust." + ) + + +# ── key handling ──────────────────────────────────────────────────────── + +def test_secret_is_never_stored_in_the_clear(ak, tmp_path): + _, plaintext = ak.create("t", ["read:metrics"], ["http://localhost:3000"]) + raw = (tmp_path / "api_keys.json").read_text() + secret = plaintext.split("_", 2)[2] + assert secret not in raw + assert plaintext not in raw + stored = json.loads(raw)["keys"][0] + assert stored["hash"] and stored["hash"] != secret + + +@pytest.mark.skipif(os.name != "posix", reason="POSIX mode bits only") +def test_store_file_is_owner_only(ak, tmp_path): + ak.create("t", ["read:metrics"], ["http://localhost:3000"]) + mode = stat.S_IMODE((tmp_path / "api_keys.json").stat().st_mode) + assert mode & 0o077 == 0, f"key file is group/world accessible: {oct(mode)}" + + +def test_verify_round_trips_and_rejects_a_tampered_secret(ak): + _, plaintext = ak.create("t", ["read:metrics"], ["http://localhost:3000"]) + assert ak.verify(plaintext)["name"] == "t" + kid, secret = plaintext.split("_", 2)[1:] + assert ak.verify("cmk_%s_%sx" % (kid, secret)) is None + assert ak.verify("cmk_00000000_" + secret) is None + assert ak.verify("") is None + assert ak.verify("not-a-key") is None + + +def test_revoke_is_immediate_and_keeps_the_record(ak): + rec, plaintext = ak.create("t", ["read:metrics"], ["http://localhost:3000"]) + assert ak.revoke(rec["id"]) is True + assert ak.verify(plaintext) is None + assert ak.revoke(rec["id"]) is False, "revoking twice must not report success" + assert [r["id"] for r in ak.list_keys(include_revoked=True)] == [rec["id"]] + assert ak.list_keys() == [] + + +def test_revoke_accepts_a_pasted_whole_key(ak): + """People paste the key, not the id. Accepting it costs nothing and + the alternative is a confusing 'no such key'.""" + _, plaintext = ak.create("t", ["read:metrics"], ["http://localhost:3000"]) + assert ak.revoke(plaintext) is True + assert ak.verify(plaintext) is None + + +def test_wildcard_origin_is_refused(ak): + with pytest.raises(ak.ApiKeyError) as exc: + ak.create("t", ["read:metrics"], ["*"]) + assert "wildcard" in str(exc.value).lower() + + +@pytest.mark.parametrize("bad", [ + "localhost:3000", # no scheme + "ftp://example.com", # not http(s) + "https://example.com/dashboard", # has a path + "https://example.com?a=1", # has a query +]) +def test_malformed_origins_are_refused(ak, bad): + with pytest.raises(ak.ApiKeyError): + ak.create("t", ["read:metrics"], [bad]) + + +def test_unknown_scope_is_refused_by_name(ak): + with pytest.raises(ak.ApiKeyError) as exc: + ak.create("t", ["read:everything"], ["http://localhost:3000"]) + assert "read:everything" in str(exc.value) + + +def test_a_key_needs_a_scope(ak): + with pytest.raises(ak.ApiKeyError): + ak.create("t", [], ["http://localhost:3000"]) + + +def test_a_key_needs_a_name(ak): + with pytest.raises(ak.ApiKeyError): + ak.create(" ", ["read:metrics"], ["http://localhost:3000"]) + + +def test_unreadable_store_reads_as_no_keys(ak, tmp_path, monkeypatch): + """A corrupt file must degrade to 401, never to a 500 that takes the + dashboard's own pages with it.""" + (tmp_path / "api_keys.json").write_text("{ this is not json") + assert ak.list_keys() == [] + assert ak.verify("cmk_aaaaaaaa_whatever") is None + + +def test_key_cap_is_enforced(ak, monkeypatch): + monkeypatch.setattr(ak, "MAX_KEYS", 2) + ak.create("a", ["read:metrics"], ["http://localhost:3000"]) + rec, _ = ak.create("b", ["read:metrics"], ["http://localhost:3000"]) + with pytest.raises(ak.ApiKeyError): + ak.create("c", ["read:metrics"], ["http://localhost:3000"]) + ak.revoke(rec["id"]) + ak.create("c", ["read:metrics"], ["http://localhost:3000"]) # a slot freed up + + +def test_redact_shows_the_id_and_never_the_secret(ak): + _, plaintext = ak.create("t", ["read:metrics"], ["http://localhost:3000"]) + red = ak.redact(plaintext) + assert plaintext.split("_", 2)[1] in red + assert plaintext.split("_", 2)[2] not in red + + +# ── HTTP: authentication ──────────────────────────────────────────────── + +def test_no_key_is_401_even_from_loopback(client): + """The whole point of this surface: unlike the rest of the dashboard, + being local earns nothing here.""" + r = client.get("/api/q/1/health") + assert r.status_code == 401 + assert "clawmetry key create" in r.get_json()["error"] + + +def test_bad_key_is_401(client): + r = client.get("/api/q/1/health", headers=_auth("cmk_deadbeef_nope")) + assert r.status_code == 401 + + +def test_revoked_key_stops_working_without_a_restart(client, ak): + rec, plaintext = ak.create("t", ["read:metrics"], ["http://localhost:3000"]) + assert client.get("/api/q/1/health", headers=_auth(plaintext)).status_code == 200 + ak.revoke(rec["id"]) + assert client.get("/api/q/1/health", headers=_auth(plaintext)).status_code == 401 + + +def test_x_clawmetry_key_header_also_works(client, ak): + plaintext = _mint(ak) + r = client.get("/api/q/1/health", headers={"X-ClawMetry-Key": plaintext}) + assert r.status_code == 200 + + +# ── HTTP: scopes ──────────────────────────────────────────────────────── + +def test_granted_shape_dispatches(client, ak): + plaintext = _mint(ak, scopes=["read:metrics"]) + r = client.get("/api/q/1/aggregates", headers=_auth(plaintext)) + assert r.status_code == 200 + body = r.get_json() + assert body["shape"] == "aggregates" + assert body["contract"] == "q/1" + assert client.dispatch_calls[-1][0] == "aggregates" + + +def test_ungranted_shape_is_403_and_names_the_scope(client, ak): + plaintext = _mint(ak, scopes=["read:metrics"]) + r = client.get("/api/q/1/transcript?session_id=x", headers=_auth(plaintext)) + assert r.status_code == 403 + body = r.get_json() + assert body["required_scope"] == "read:content" + assert body["held_scopes"] == ["read:metrics"] + assert not client.dispatch_calls, "a refused query must never reach the store" + + +def test_a_metrics_key_cannot_reach_any_content_shape(client, ak): + """The promise on the tin, checked against every shape rather than a + representative one.""" + from clawmetry import query_contract as qc + + plaintext = _mint(ak, scopes=["read:metrics"]) + for shape in qc.live_methods_by_scope(qc.SCOPE_CONTENT): + r = client.get(f"/api/q/1/{shape}?session_id=x", headers=_auth(plaintext)) + assert r.status_code == 403, f"{shape} was reachable with read:metrics" + + +def test_unknown_shape_is_404(client, ak): + r = client.get("/api/q/1/definitely_not_a_shape", headers=_auth(_mint(ak))) + assert r.status_code == 404 + + +def test_planned_but_unserved_shape_is_404_not_500(client, ak): + """`usage` is declared but not live. Reaching it must look like a typo, + not like a crash.""" + from clawmetry import query_contract as qc + + planned = qc.methods_by_status(qc.STATUS_PLANNED) + assert planned, "no planned methods left; drop this test or pick another" + plaintext = _mint(ak, scopes=list(qc.SCOPES)) + for name in planned: + r = client.get(f"/api/q/1/{name}", headers=_auth(plaintext)) + assert r.status_code == 404, name + + +def test_missing_required_arg_is_400_naming_the_arg(client, ak): + plaintext = _mint(ak, scopes=["read:content"]) + r = client.get("/api/q/1/transcript", headers=_auth(plaintext)) + assert r.status_code == 400 + assert "session_id" in r.get_json()["error"] + + +def test_store_failure_is_503_without_leaking_internals(client, ak, monkeypatch): + import routes.local_query as lq + + def _boom(shape, args): + raise RuntimeError("/Users/someone/.clawmetry/clawmetry.duckdb is locked") + + monkeypatch.setattr(lq, "_dispatch", _boom) + r = client.get("/api/q/1/health", headers=_auth(_mint(ak))) + assert r.status_code == 503 + assert ".duckdb" not in r.get_json()["error"] + assert "clawmetry doctor" in r.get_json()["error"] + + +def test_internal_dispatch_fields_are_stripped(client, ak): + r = client.get("/api/q/1/aggregates", headers=_auth(_mint(ak))) + assert [k for k in r.get_json() if k.startswith("_")] == [] + + +# ── HTTP: the index and the guide ─────────────────────────────────────── + +def test_index_lists_only_granted_shapes(client, ak): + plaintext = _mint(ak, scopes=["read:metrics"]) + body = client.get("/api/q/1", headers=_auth(plaintext)).get_json() + from clawmetry import query_contract as qc + + assert {s["shape"] for s in body["shapes"]} == set( + qc.live_methods_by_scope(qc.SCOPE_METRICS)) + held = {s["scope"] for s in body["scopes"] if s["held"]} + assert held == {"read:metrics"} + + +def test_llms_txt_describes_only_what_the_key_can_run(client, ak): + plaintext = _mint(ak, scopes=["read:metrics"]) + text = client.get("/api/q/1/llms.txt", headers=_auth(plaintext)).get_data(as_text=True) + assert "GET /api/q/1/aggregates" in text + assert "GET /api/q/1/transcript" not in text + assert "Not available to this key" in text + + +def test_llms_txt_needs_a_key_too(client): + assert client.get("/api/q/1/llms.txt").status_code == 401 + + +# ── HTTP: CORS, the part that matters most ────────────────────────────── + +def test_cors_header_only_for_an_origin_the_key_named(client, ak): + plaintext = _mint(ak, origins=["http://localhost:3000"]) + ok = client.get("/api/q/1/health", + headers={**_auth(plaintext), "Origin": "http://localhost:3000"}) + assert ok.headers.get("Access-Control-Allow-Origin") == "http://localhost:3000" + assert ok.headers.get("Vary") == "Origin" + + bad = client.get("/api/q/1/health", + headers={**_auth(plaintext), "Origin": "https://evil.example"}) + assert "Access-Control-Allow-Origin" not in bad.headers, ( + "a key's data was made readable to an origin it never named" + ) + + +def test_cors_is_never_a_wildcard(client, ak): + plaintext = _mint(ak) + r = client.get("/api/q/1/health", + headers={**_auth(plaintext), "Origin": "http://localhost:3000"}) + assert r.headers.get("Access-Control-Allow-Origin") != "*" + + +def test_an_origin_none_key_gets_no_cors_at_all(client, ak): + """A key created for a script must be useless from a browser, even + from an origin some OTHER key has authorised.""" + _mint(ak, name="browser", origins=["http://localhost:3000"]) + script = _mint(ak, name="script", origins=[]) + r = client.get("/api/q/1/health", + headers={**_auth(script), "Origin": "http://localhost:3000"}) + assert r.status_code == 200 + assert "Access-Control-Allow-Origin" not in r.headers + + +def test_preflight_is_answered_only_for_a_known_origin(client, ak): + _mint(ak, origins=["http://localhost:3000"]) + ok = client.options("/api/q/1/health", headers={ + "Origin": "http://localhost:3000", + "Access-Control-Request-Method": "GET", + }) + assert ok.headers.get("Access-Control-Allow-Origin") == "http://localhost:3000" + + bad = client.options("/api/q/1/health", headers={ + "Origin": "https://evil.example", + "Access-Control-Request-Method": "GET", + }) + assert "Access-Control-Allow-Origin" not in bad.headers + + +def test_no_cors_when_authentication_failed(client, ak): + """A 401 body is not interesting, but leaking a header on the failure + path is how a probe learns which origins are authorised.""" + _mint(ak, origins=["https://known.example"]) + r = client.get("/api/q/1/health", headers={"Origin": "https://evil.example"}) + assert r.status_code == 401 + assert "Access-Control-Allow-Origin" not in r.headers + + +def test_origin_matching_ignores_case_and_trailing_slash(client, ak): + plaintext = _mint(ak, origins=["https://My-UI.Example"]) + r = client.get("/api/q/1/health", + headers={**_auth(plaintext), "Origin": "https://my-ui.example/"}) + assert r.headers.get("Access-Control-Allow-Origin") + + +def test_the_api_is_get_only(client, ak): + plaintext = _mint(ak) + for method in ("post", "put", "delete", "patch"): + r = getattr(client, method)("/api/q/1/health", headers=_auth(plaintext)) + assert r.status_code == 405, f"{method.upper()} was accepted" + + +# ── rate limiting ─────────────────────────────────────────────────────── + +def test_rate_limit_trips_and_says_what_to_do(client, ak, monkeypatch): + monkeypatch.setattr(client.pub, "RATE_LIMIT_PER_MIN", 3) + client.pub._RATE.clear() + plaintext = _mint(ak) + codes = [client.get("/api/q/1/health", headers=_auth(plaintext)).status_code + for _ in range(5)] + assert codes[:3] == [200, 200, 200] + assert codes[3:] == [429, 429] + body = client.get("/api/q/1/health", headers=_auth(plaintext)).get_json() + assert "CLAWMETRY_API_RATE_LIMIT" in body["error"] + + +def test_rate_limit_is_per_key(client, ak, monkeypatch): + monkeypatch.setattr(client.pub, "RATE_LIMIT_PER_MIN", 2) + client.pub._RATE.clear() + a = _mint(ak, name="a") + b = _mint(ak, name="b") + for _ in range(2): + client.get("/api/q/1/health", headers=_auth(a)) + assert client.get("/api/q/1/health", headers=_auth(a)).status_code == 429 + assert client.get("/api/q/1/health", headers=_auth(b)).status_code == 200 + + +# ── the management endpoints stay on the dashboard's own gate ─────────── + +@pytest.fixture +def admin(ak): + from flask import Flask + + import routes.infra as infra + + app = Flask(__name__) + app.register_blueprint(infra.bp_security) + return app.test_client() + + +def test_management_mint_list_revoke(admin): + made = admin.post("/api/apikeys", json={ + "name": "from-ui", "scopes": ["read:metrics"], + "origins": ["http://localhost:5173"]}).get_json() + assert made["ok"] and made["key"].startswith("cmk_") + assert "hash" not in made["record"] + + listed = admin.get("/api/apikeys").get_json() + assert [k["name"] for k in listed["keys"]] == ["from-ui"] + assert all("hash" not in k for k in listed["keys"]) + + kid = made["record"]["id"] + assert admin.delete("/api/apikeys/" + kid).get_json()["ok"] is True + assert admin.delete("/api/apikeys/" + kid).status_code == 404 + + +def test_management_refuses_a_browser_key_with_no_origin(admin): + r = admin.post("/api/apikeys", json={"name": "x", "scopes": ["read:metrics"], + "origins": []}) + assert r.status_code == 400 + assert "wildcard" in r.get_json()["error"] + + +def test_management_allows_an_explicit_non_browser_key(admin): + r = admin.post("/api/apikeys", json={"name": "cron", "scopes": ["read:metrics"], + "origins": [], "browser": False}) + assert r.status_code == 200 + assert r.get_json()["record"]["origins"] == [] + + +def test_management_never_carries_a_cors_header(admin, ak): + """The read API is cross-origin readable; minting keys must not be. + + This is the specific mistake the path guard in ``public_api._add_cors`` + exists to prevent, so it is checked from the outside too. + """ + _mint(ak, origins=["http://localhost:3000"]) + r = admin.get("/api/apikeys", headers={"Origin": "http://localhost:3000"}) + assert "Access-Control-Allow-Origin" not in r.headers + + +def test_management_surfaces_a_sentence_not_a_code(admin, monkeypatch): + import clawmetry.apikeys as mod + + monkeypatch.setattr(mod, "create", lambda *a, **k: (_ for _ in ()).throw(OSError("EACCES"))) + r = admin.post("/api/apikeys", json={"name": "x", "scopes": ["read:metrics"], + "origins": ["http://localhost:3000"]}) + assert r.status_code == 500 + err = r.get_json()["error"] + assert "EACCES" not in err and "~/.clawmetry" in err diff --git a/tests/test_query_contract_drift.py b/tests/test_query_contract_drift.py index dc11c547b6..79cf2aaead 100644 --- a/tests/test_query_contract_drift.py +++ b/tests/test_query_contract_drift.py @@ -70,6 +70,11 @@ def test_statuses_and_version(): for name, spec in qc.QUERY_CONTRACT.items(): assert spec["status"] in (qc.STATUS_LIVE, qc.STATUS_PLANNED), name assert spec["trust"] in (qc.TRUST_PLAINTEXT, qc.TRUST_E2E), name + # Every method declares the read scope an API key needs to reach + # it (routes/public_api.py). A method with no scope, or an unknown + # one, is unreachable by any key -- which is a silent way to ship + # a query nobody can call. + assert spec["scope"] in qc.SCOPES, name assert spec["backing"], name assert spec["doc"], name assert isinstance(spec["args"], dict), name From a1045f14fa542476e664d42c95a1cbda27743308 Mon Sep 17 00:00:00 2001 From: vivekchand Date: Tue, 8 Sep 2026 10:12:42 +0200 Subject: [PATCH 02/19] docs: name the Factory requirement in the two new module docstrings Drift Bot's one finding was that the auto-created feature blueprint was still the empty template. Written in full (v2, 11.6 KB): composition, four component blocks, key + integration contracts, and six ADRs covering the scope-on-the-contract choice, loopback not being authentication here, the no-wildcard rule, key management staying off the keyed surface, why custom UIs read the node rather than the hosted service, and the self-describing guide. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01B6d1rL2kbFEu6Pr6xUzYW5 --- clawmetry/apikeys.py | 3 +++ routes/public_api.py | 4 ++++ 2 files changed, 7 insertions(+) diff --git a/clawmetry/apikeys.py b/clawmetry/apikeys.py index ee8e340b01..44c5959d33 100644 --- a/clawmetry/apikeys.py +++ b/clawmetry/apikeys.py @@ -1,5 +1,8 @@ """clawmetry/apikeys.py -- scoped, revocable read keys for custom UIs. +Requirement: "Build your own UI: a keyed, scoped read API for custom +dashboards" (64c10afd-038d-4fde-9c55-ddca80aaff1e). + Why this exists --------------- The dashboard reads your agents through the ``q/1`` query contract. That diff --git a/routes/public_api.py b/routes/public_api.py index c3ee484894..2e80bcaab1 100644 --- a/routes/public_api.py +++ b/routes/public_api.py @@ -1,5 +1,9 @@ """routes/public_api.py -- the keyed, cross-origin read API custom UIs use. +Requirement: "Build your own UI: a keyed, scoped read API for custom +dashboards" (64c10afd-038d-4fde-9c55-ddca80aaff1e), blueprint +0ea7523c-12b5-4033-84ea-bf1f46e20d70. + This is the "build your own UI" surface (docs/BUILD_YOUR_OWN_UI.md). It serves the declared ``q/1`` query contract to anything holding a scoped API key: a page vibe-coded on v0 or Lovable, a Grafana-ish panel, a From 34a0adf3aaad289d93fb3613861e3135622362bc Mon Sep 17 00:00:00 2001 From: vivekchand Date: Tue, 8 Sep 2026 10:15:50 +0200 Subject: [PATCH 03/19] docs: point the key CLI at the blueprint section that specifies it Drift Bot round 2 read the rewritten blueprint (it no longer calls it empty) and asked for the concrete API surface rather than only ADRs. Added an 'API Surface' section at 4% of the document: the three read endpoints with auth and the status-code table, which four response fields a client contracts on and which are diagnostic, the cross-origin rule, the three management endpoints, and the four CLI subcommands. Blueprint v3. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01B6d1rL2kbFEu6Pr6xUzYW5 --- clawmetry/cli.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/clawmetry/cli.py b/clawmetry/cli.py index 441d313fec..dd64f8e573 100644 --- a/clawmetry/cli.py +++ b/clawmetry/cli.py @@ -4719,6 +4719,12 @@ def _cmd_mcp(args) -> None: def _cmd_key(args) -> None: """`clawmetry key ...` -- scoped read keys for custom UIs. + Spec: blueprint 0ea7523c-12b5-4033-84ea-bf1f46e20d70, "API Surface" -> + "Command line". Four subcommands, each also taking --json: + create (mints one, prints the secret once), list (names, scopes, + origins, last use), revoke (effective on that key's next request), + scopes (what each grants). + The point of these keys is that someone can build their own view of their own agents without forking the dashboard: create a key, say what it may read and which site may read it, paste it into whatever they From a64cd7ffa9eba73a06a56b623559aba294208f3e Mon Sep 17 00:00:00 2001 From: vivekchand Date: Tue, 8 Sep 2026 10:23:31 +0200 Subject: [PATCH 04/19] refactor: key management moves to its own short module Drift Bot round 3 reported that GET/POST /api/apikeys and DELETE /api/apikeys/ 'are not implemented in routes/infra.py' and that the Security tab therefore cannot work. They were implemented, at lines 2955-3042 of a 3042-line file, i.e. at 97% depth: the bot reads a truncated prefix of a long file, so anything appended to a module that size is invisible to it forever. Rather than argue the point on a PR comment, extract them. They now live in routes/apikeys_admin.py (bp_apikeys_admin, ~130 lines), registered in dashboard.py next to bp_public_api. routes/infra.py returns byte for byte to its state on main. This is better placement independent of the tool. Credential management is a different concern from the security tab's scanners, and the reason these handlers must not sit on the keyed surface now has a file to be stated in rather than a comment in the middle of another one. Both guards are path-based, not blueprint-based, so the security posture is unchanged: dashboard.py's _cross_origin_write_blocked still refuses a cross-origin POST/DELETE, and public_api._add_cors is still pinned to /api/q/. A new guard asserts the separation directly: no /api/apikeys rule may be owned by the public_api blueprint and no /api/q/ rule by apikeys_admin. Verified on a real boot after the move, not only in tests: management GET 200, POST with no origin 400, cross-origin POST 403, query index 401 without a key and 200 with one, aggregates 200, CORS header present on /api/q/ and absent on /api/apikeys. Blueprint v4 names the module. Cloud #2342 updated to match the new endpoint names. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01B6d1rL2kbFEu6Pr6xUzYW5 --- CLAUDE.md | 1 + dashboard.py | 6 ++ docs/MODULE_MAP.md | 5 +- routes/apikeys_admin.py | 129 ++++++++++++++++++++++++++++++++++ routes/infra.py | 104 --------------------------- tests/test_public_api_keys.py | 35 ++++++++- 6 files changed, 172 insertions(+), 108 deletions(-) create mode 100644 routes/apikeys_admin.py diff --git a/CLAUDE.md b/CLAUDE.md index 42d835a2e2..6178ea57b5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -46,6 +46,7 @@ All HTTP endpoints live here, organised by feature: 70 modules, 82 blueprints, l | `routes/components.py` | `bp_components` — Flow-panel detail endpoints (tool / runtime / machine / gateway / brain) | | `routes/local_query.py` | `bp_local_query` — `/api/local/*` DuckDB read API + the daemon-proxy `_dispatch` (shape→store bridge shared by HTTP and the cloud relay) | | `routes/public_api.py` | `bp_public_api` — `/api/q/1/*`, the **keyed, cross-origin** read API custom UIs are built on (`docs/BUILD_YOUR_OWN_UI.md`). Same `_dispatch`, but it is the one surface that does not trust loopback: every request needs a scoped `cmk_` key, and CORS is echoed only for an origin that key named | +| `routes/apikeys_admin.py` | `bp_apikeys_admin` — `/api/apikeys`, minting and revoking the keys `public_api` accepts. Kept apart from that surface on purpose: it sits behind the dashboard's own cross-origin write guard and never carries a CORS header, so a page holding a read key can neither list this node's keys nor issue itself a wider one | | `routes/guard.py` | `bp_guard` — live session control (Pause/Stop/Kill), Guard policy CRUD, policy decision log, learned baselines. Sessions ranked by **spend at risk**, not severity | | `routes/policy.py` | `bp_policy` — the *pre-tool* sandbox/permission surface (`/api/tool-policy`). Deliberately a different axis from `routes/guard.py`: different table, no shared state | | `routes/hooks.py` | `bp_hooks` — hook install / status / uninstall per runtime, and the gate's decision log | diff --git a/dashboard.py b/dashboard.py index baa87b0c48..3df3dc2b0d 100644 --- a/dashboard.py +++ b/dashboard.py @@ -138,6 +138,7 @@ from routes.plugins import bp_plugins from routes.local_query import bp_local_query from routes.public_api import bp_public_api +from routes.apikeys_admin import bp_apikeys_admin from routes.update_check import bp_update_check, start_update_check_thread from routes.workspaces import bp_workspaces from routes.bootstrap import bp_bootstrap @@ -7863,6 +7864,11 @@ def detect_config(args=None): # scoped key and echoes a CORS header only for an origin that key # named. See routes/public_api.py for why that inversion matters. app.register_blueprint(bp_public_api) + # Creating and revoking the keys that surface uses. Deliberately a + # separate blueprint behind the dashboard's own gate: a page holding a + # read key must never be able to list this node's keys or mint a wider + # one. See routes/apikeys_admin.py. + app.register_blueprint(bp_apikeys_admin) # ClawMetry Enterprise self-hosted server mode: one process serves the # dashboard AND the ingest API the node daemons push to. Gated hard on # SELF_HOSTED=true — never registered for normal local/cloud installs. diff --git a/docs/MODULE_MAP.md b/docs/MODULE_MAP.md index 1d9debcd6c..c6f03c02d4 100644 --- a/docs/MODULE_MAP.md +++ b/docs/MODULE_MAP.md @@ -4,7 +4,7 @@ > `python3 scripts/gen_module_map.py` (CI fails on drift via > `tests/test_module_map_drift.py`). -231 modules, 82 Flask blueprints. `CLAUDE.md` carries a short curated table of the ones you reach for most often; this is the whole list. +232 modules, 83 Flask blueprints. `CLAUDE.md` carries a short curated table of the ones you reach for most often; this is the whole list. Size bands are deliberately coarse so this file does not churn on every PR: **small** is under 200 lines, **medium** under 1k, **large** under 5k, **huge** is 5k and up. @@ -29,6 +29,7 @@ One module per feature, each owning one or more Flask blueprints. New endpoints | `routes/advisor.py` | medium | `bp_advisor` | `/api/advisor` | ClawMetry Advisor: natural-language Q&A over your agent. | | `routes/agents.py` | medium | `bp_agents` | `/api/agents` | Multi-agent adapter endpoints. | | `routes/alerts.py` | large | `bp_alerts`, `bp_budget` | `/api/_harness`, `/api/agents`, `/api/alert-channels`, `/api/alerts`, `/api/budget`, `/api/emergency-stop` | Budget + Alerts endpoints. | +| `routes/apikeys_admin.py` | small | `bp_apikeys_admin` | `/api/apikeys` | create, list and revoke the node's API keys. | | `routes/approval_routing.py` | small | `bp_approval_routing` | `/a`, `/a/decide`, `/api/approvals` | OSS stub after the impl moved to clawmetry-pro. | | `routes/assets.py` | small | `bp_assets` | `/api/assets` | OSS asset registry API. | | `routes/attention.py` | medium | `bp_attention` | `/api/attention`, `/api/hooks` | "which of my agents needs me right now". | @@ -57,7 +58,7 @@ One module per feature, each owning one or more Flask blueprints. New endpoints | `routes/heartbeat.py` | medium | `bp_heartbeat` | `/api/heartbeat`, `/api/heartbeat-loops` | Heartbeat liveness panel API endpoint (#686). | | `routes/hitl.py` | medium | `bp_hitl` | `/api/hitl` | Human-in-the-loop (HITL) pause API. | | `routes/hooks.py` | large | `bp_hooks` | `/api/hooks`, `/api/lifecycle`, `/api/sessions` | local receiver for runtime pre-tool hooks. | -| `routes/infra.py` | large | `bp_config`, `bp_logs`, `bp_memory`, `bp_security` | `/api/apikeys`, `/api/automation-analysis`, `/api/context-anatomy`, `/api/cost-optimization`, `/api/cost-optimizer`, `/api/file`, `/api/flow`, `/api/flow-events`, `/api/llmfit`, `/api/logs`, `/api/logs-stream`, `/api/memory`, `/api/memory-access`, `/api/memory-analytics`, `/api/memory-files`, `/api/memory-rag`, `/api/numbat`, `/api/security` | Infrastructure / security / config / logs endpoints. | +| `routes/infra.py` | large | `bp_config`, `bp_logs`, `bp_memory`, `bp_security` | `/api/automation-analysis`, `/api/context-anatomy`, `/api/cost-optimization`, `/api/cost-optimizer`, `/api/file`, `/api/flow`, `/api/flow-events`, `/api/llmfit`, `/api/logs`, `/api/logs-stream`, `/api/memory`, `/api/memory-access`, `/api/memory-analytics`, `/api/memory-files`, `/api/memory-rag`, `/api/numbat`, `/api/security` | Infrastructure / security / config / logs endpoints. | | `routes/insights.py` | medium | `bp_insights` | `/api/insights`, `/insights` | Weekly Insights Digest endpoints. | | `routes/inventory.py` | medium | `bp_inventory` | `/api/inventory` | Agent Inventory tab API. | | `routes/local_query.py` | large | `bp_local_query` | `/__local_query__`, `/api/local` | coherent local query API over the DuckDB store. | diff --git a/routes/apikeys_admin.py b/routes/apikeys_admin.py new file mode 100644 index 0000000000..a0722f8724 --- /dev/null +++ b/routes/apikeys_admin.py @@ -0,0 +1,129 @@ +"""routes/apikeys_admin.py -- create, list and revoke the node's API keys. + +Requirement: "Build your own UI: a keyed, scoped read API for custom +dashboards" (64c10afd-038d-4fde-9c55-ddca80aaff1e), blueprint +0ea7523c-12b5-4033-84ea-bf1f46e20d70, "API Surface" -> "Key management". + + GET /api/apikeys this node's keys + the scope catalogue + POST /api/apikeys mint one; the secret is in the response + DELETE /api/apikeys/ revoke one + +These MINT and REVOKE credentials for ``routes/public_api.py``, which is the +cross-origin surface those credentials open. That relationship is the whole +reason they live apart: + +* **This module is behind the dashboard's own gate.** ``dashboard.py``'s + ``_cross_origin_write_blocked`` refuses a cross-origin POST/DELETE to any + ``/api/*`` path, and ``public_api._add_cors`` is pinned to ``/api/q/`` so no + CORS header ever reaches these routes. A page holding a read key can + therefore never list the node's keys, and never issue itself a better one. +* **It is its own blueprint, not a few functions on ``bp_security``.** Both + placements are equally safe, since the guards above are path-based rather + than blueprint-based. This one is legible: credential management is a + distinct concern from the security tab's scanners, and a reader (human or + tool) sees the whole surface in one short file instead of at 97% of a + 3,000-line module. + +Nothing here reads agent data. The keys it manages are read-only by +construction; see ``clawmetry/apikeys.py`` for what they can and cannot do. +""" + +from __future__ import annotations + +import logging + +from flask import Blueprint, jsonify, request + +logger = logging.getLogger("clawmetry.routes.apikeys_admin") + +bp_apikeys_admin = Blueprint("apikeys_admin", __name__) + + +@bp_apikeys_admin.route("/api/apikeys", methods=["GET"]) +def api_keys_list(): + """This node's API keys, plus the scope catalogue the UI renders. + + Secrets are never included: only a SHA-256 is stored, and even that is + stripped by ``apikeys.list_keys``. + """ + from clawmetry import apikeys as _ak + try: + return jsonify({ + "ok": True, + "keys": _ak.list_keys(include_revoked=True), + "scopes": _ak.scope_catalogue(), + "summary": _ak.store_summary(), + }) + except Exception as exc: + logger.warning("apikeys list failed: %s", exc) + return jsonify({ + "ok": False, + "keys": [], + "scopes": [], + "error": "ClawMetry could not read its key file. Check that " + "~/.clawmetry is readable by you.", + }), 200 + + +@bp_apikeys_admin.route("/api/apikeys", methods=["POST"]) +def api_keys_create(): + """Mint a key. The secret is in this response and nowhere else, ever.""" + from clawmetry import apikeys as _ak + body = request.get_json(silent=True) or {} + origins = body.get("origins") or [] + if isinstance(origins, str): + origins = [o.strip() for o in origins.replace(",", " ").split() if o.strip()] + scopes = body.get("scopes") or [] + if isinstance(scopes, str): + scopes = [s.strip() for s in scopes.replace(",", " ").split() if s.strip()] + browser = bool(body.get("browser", True)) + if browser and not origins: + return jsonify({ + "ok": False, + "error": "Name the site that will use this key, for example " + "http://localhost:3000. There is no wildcard: any page " + "in any tab can already reach this machine, and the " + "origin list is what stops it reading the answer.", + }), 400 + try: + record, plaintext = _ak.create( + body.get("name") or "", + scopes, + [] if not browser else origins, + note=body.get("note") or "", + ) + except _ak.ApiKeyError as exc: + return jsonify({"ok": False, "error": str(exc)}), 400 + except Exception as exc: + logger.warning("apikeys create failed: %s", exc) + return jsonify({ + "ok": False, + "error": "ClawMetry could not write its key file. Check that " + "~/.clawmetry is writable by you.", + }), 500 + return jsonify({ + "ok": True, + "key": plaintext, + "record": {k: v for k, v in record.items() if k != "hash"}, + }) + + +@bp_apikeys_admin.route("/api/apikeys/", methods=["DELETE"]) +def api_keys_revoke(key_id: str): + """Revoke a key. Takes effect on that key's next request.""" + from clawmetry import apikeys as _ak + try: + ok = _ak.revoke(key_id) + except Exception as exc: + logger.warning("apikeys revoke failed: %s", exc) + return jsonify({ + "ok": False, + "error": "ClawMetry could not write its key file. Check that " + "~/.clawmetry is writable by you.", + }), 500 + if not ok: + return jsonify({ + "ok": False, + "error": "There is no active key with that id on this machine.", + }), 404 + return jsonify({"ok": True, "id": key_id}) diff --git a/routes/infra.py b/routes/infra.py index fdb22137df..cc706221ea 100644 --- a/routes/infra.py +++ b/routes/infra.py @@ -28,7 +28,6 @@ from __future__ import annotations import json -import logging import os import re import sqlite3 @@ -40,8 +39,6 @@ from clawmetry._gate import gate from clawmetry.config import is_local_store_read_enabled, hide_clawmetry_session -logger = logging.getLogger("clawmetry.routes.infra") - bp_logs = Blueprint('logs', __name__) bp_memory = Blueprint('memory', __name__) bp_security = Blueprint('security', __name__) @@ -2939,104 +2936,3 @@ def api_security_retention_set(): except Exception: state = _ret.resolve(store=_retention_store()) return jsonify({"ok": True, **state}) - - -# ── API keys for custom UIs (docs/BUILD_YOUR_OWN_UI.md) ───────────────── -# -# These MINT and REVOKE credentials, so they live here -- behind the -# dashboard's own gate -- and not in routes/public_api.py, which is the -# cross-origin surface those credentials open. A page on someone else's -# site can hold a read key; it must never be able to issue itself a -# better one. dashboard.py's `_cross_origin_write_blocked` refuses a -# cross-origin POST to these, and `_add_cors` in public_api.py is pinned -# to /api/q/ so no CORS header ever reaches them. - - -@bp_security.route("/api/apikeys", methods=["GET"]) -def api_keys_list(): - """This machine's API keys, plus the scope catalogue the UI renders. - - Secrets are never included -- only the SHA-256 is stored and even - that is stripped by ``apikeys.list_keys``. - """ - from clawmetry import apikeys as _ak - try: - return jsonify({ - "ok": True, - "keys": _ak.list_keys(include_revoked=True), - "scopes": _ak.scope_catalogue(), - "summary": _ak.store_summary(), - }) - except Exception as exc: - logger.warning("apikeys list failed: %s", exc) - return jsonify({ - "ok": False, - "keys": [], - "scopes": [], - "error": "ClawMetry could not read its key file. Check that " - "~/.clawmetry is readable by you.", - }), 200 - - -@bp_security.route("/api/apikeys", methods=["POST"]) -def api_keys_create(): - """Mint a key. The secret is in the response and nowhere else, ever.""" - from clawmetry import apikeys as _ak - body = request.get_json(silent=True) or {} - origins = body.get("origins") or [] - if isinstance(origins, str): - origins = [o.strip() for o in origins.replace(",", " ").split() if o.strip()] - scopes = body.get("scopes") or [] - if isinstance(scopes, str): - scopes = [s.strip() for s in scopes.replace(",", " ").split() if s.strip()] - browser = bool(body.get("browser", True)) - if browser and not origins: - return jsonify({ - "ok": False, - "error": "Name the site that will use this key, for example " - "http://localhost:3000. There is no wildcard: any page " - "in any tab can already reach this machine, and the " - "origin list is what stops it reading the answer.", - }), 400 - try: - record, plaintext = _ak.create( - body.get("name") or "", - scopes, - [] if not browser else origins, - note=body.get("note") or "", - ) - except _ak.ApiKeyError as exc: - return jsonify({"ok": False, "error": str(exc)}), 400 - except Exception as exc: - logger.warning("apikeys create failed: %s", exc) - return jsonify({ - "ok": False, - "error": "ClawMetry could not write its key file. Check that " - "~/.clawmetry is writable by you.", - }), 500 - return jsonify({ - "ok": True, - "key": plaintext, - "record": {k: v for k, v in record.items() if k != "hash"}, - }) - - -@bp_security.route("/api/apikeys/", methods=["DELETE"]) -def api_keys_revoke(key_id: str): - """Revoke a key. Takes effect on that key's next request.""" - from clawmetry import apikeys as _ak - try: - ok = _ak.revoke(key_id) - except Exception as exc: - logger.warning("apikeys revoke failed: %s", exc) - return jsonify({ - "ok": False, - "error": "ClawMetry could not write its key file. Check that " - "~/.clawmetry is writable by you.", - }), 500 - if not ok: - return jsonify({ - "ok": False, - "error": "There is no active key with that id on this machine.", - }), 404 - return jsonify({"ok": True, "id": key_id}) diff --git a/tests/test_public_api_keys.py b/tests/test_public_api_keys.py index 71decf41f8..4f83d744b8 100644 --- a/tests/test_public_api_keys.py +++ b/tests/test_public_api_keys.py @@ -471,15 +471,46 @@ def test_rate_limit_is_per_key(client, ak, monkeypatch): @pytest.fixture def admin(ak): + """A client for the key-MANAGEMENT blueprint. + + Separate from ``client`` on purpose, and asserted to be a different + blueprint below: minting must never be reachable from the surface the + minted keys open. + """ from flask import Flask - import routes.infra as infra + import routes.apikeys_admin as adm app = Flask(__name__) - app.register_blueprint(infra.bp_security) + app.register_blueprint(adm.bp_apikeys_admin) return app.test_client() +def test_key_management_is_not_on_the_keyed_blueprint(): + """The two surfaces must stay apart. + + ``public_api`` answers cross-origin reads; ``apikeys_admin`` mints the + credentials that open it. Registering a management route on the keyed + blueprint would put it behind that blueprint's CORS handler, so this + asserts the separation directly rather than trusting the path guard + alone. + """ + import routes.apikeys_admin as adm + import routes.public_api as pub + + from flask import Flask + + app = Flask(__name__) + app.register_blueprint(pub.bp_public_api) + app.register_blueprint(adm.bp_apikeys_admin) + for rule in app.url_map.iter_rules(): + owner = rule.endpoint.split(".")[0] + if str(rule).startswith("/api/apikeys"): + assert owner == "apikeys_admin", f"{rule} is served by {owner}" + elif str(rule).startswith("/api/q/"): + assert owner == "public_api", f"{rule} is served by {owner}" + + def test_management_mint_list_revoke(admin): made = admin.post("/api/apikeys", json={ "name": "from-ui", "scopes": ["read:metrics"], From acba3eda016be9928a03e635efd00969c7b93b1a Mon Sep 17 00:00:00 2001 From: vivekchand Date: Tue, 8 Sep 2026 10:44:28 +0200 Subject: [PATCH 05/19] fix: close six of CodeQL's eight new alerts in code I called the earlier CodeQL red a stale aggregate. It was not: 8 new alerts, 4 of them high, all in code this PR adds. Triaged individually. This repo runs code scanning through DEFAULT SETUP, so inline '# codeql[...]' comments are inert here (measured in clawmetry/nonsecret_hash.py, not assumed). Every alert therefore had to be fixed in code or accepted deliberately. Six are fixed: * py/stack-trace-exposure x2. Both were our own authored sentences, but text read off an exception is exception-derived to a static analyser no matter who wrote it. The /api/q/ 400 is now built from the CONTRACT (which already declares the required args) and the /api/apikeys 400 from a literal table keyed by a refusal code. Both are better answers: the query error is now consistent across queries and carries missing_args, and the create error carries a stable 'reason' a client can branch on instead of matching prose. * py/log-injection. The failure log now names the contract's own backing method rather than the shape string off the URL. Validated-then-logged is still a request-text sink. * py/clear-text-logging x2 of 4. The two curl examples no longer interpolate the key; they use , which is what the docs tell people to do with it anyway and reads better than a 50-character key wrapped across a terminal. * js/xss-through-exception. The starter's error panel built markup out of strings including a fetch exception message. It now builds text nodes and elements, so nothing it did not write becomes markup. The two remaining alerts are the one-time key reveal (the plain print and the --json field). A key that is never shown cannot be used, and every comparable CLI does exactly this. They are true observations, not defects, and with no working in-code suppression they can only be dismissed by a maintainer in the Security tab. Verified live after the change: the contract-built 400 names session_id and returns missing_args, the management 400 carries reason 'unknown_scope' with no exception text, the wildcard refusal still fires, and all three normal paths stay 200. Both starter error panels render as real text nodes (3 , 1
) and the happy path still draws across 10 runtimes and 10 models. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01B6d1rL2kbFEu6Pr6xUzYW5 --- clawmetry/apikeys.py | 79 +++++++++++++++++++++++++++++++---- clawmetry/cli.py | 7 ++-- examples/custom-ui/index.html | 40 +++++++++++++----- routes/apikeys_admin.py | 11 ++++- routes/public_api.py | 26 +++++++++--- 5 files changed, 135 insertions(+), 28 deletions(-) diff --git a/clawmetry/apikeys.py b/clawmetry/apikeys.py index 44c5959d33..0606ec4f6f 100644 --- a/clawmetry/apikeys.py +++ b/clawmetry/apikeys.py @@ -82,13 +82,66 @@ ORIGIN_NONE = "none" +#: Why a create call was refused. Every message is a literal authored here, +#: and a caller that has to put one in an HTTP response looks it up by code +#: rather than reading it off the exception: text taken from an exception is +#: exception-derived to a static analyser no matter who wrote it, and a +#: const-indexed lookup is the thing that is provably not. +REFUSAL_REASONS: dict = { + "unknown_scope": ( + "That is not a scope. Choose from: " + ", ".join(SCOPES) + ), + "no_scope": ( + "A key needs at least one scope. Choose from: " + ", ".join(SCOPES) + ), + "wildcard_origin": ( + "A wildcard origin is not allowed. Any page in any tab could then " + "read this machine's telemetry. Name the site you are building, for " + "example https://my-app.vercel.app." + ), + "bad_origin": ( + "That is not an origin. An origin is just a scheme, host and port, " + "with no path or query: https://my-app.vercel.app, or " + "http://localhost:3000." + ), + "no_name": ( + "Give the key a name so you can tell it apart later, for example: " + "latency-workbench." + ), + "name_too_long": "Key names are limited to 64 characters.", + "at_capacity": ( + f"This machine already has {MAX_KEYS} active keys, which is the " + "limit. Revoke one you no longer use: clawmetry key revoke " + ), +} + + +def message_for(reason: str) -> str: + """The refusal sentence for ``reason``. + + Looked up from the literal table above, never read off an exception, so + a caller can put the result in an HTTP response without carrying + exception-derived text into it. + """ + return REFUSAL_REASONS.get( + str(reason), "That key could not be created." + ) + + class ApiKeyError(Exception): """Raised for a caller mistake (bad scope, bad origin, cap reached). Carries a sentence meant for a person, not an error code -- these - surface directly in ``clawmetry key`` output and in the dashboard. + surface directly in ``clawmetry key`` output, where naming the exact + offending value is worth more than it costs. ``reason`` is the same + refusal as a stable code, for the HTTP callers that must not echo + exception text; see :data:`REFUSAL_REASONS`. """ + def __init__(self, message: str, reason: str = ""): + super().__init__(message) + self.reason = reason + # ── Store I/O ─────────────────────────────────────────────────────────── @@ -155,12 +208,14 @@ def normalise_scopes(scopes) -> list: continue if s not in SCOPES: raise ApiKeyError( - f"{s!r} is not a scope. Choose from: " + ", ".join(SCOPES) + f"{s!r} is not a scope. Choose from: " + ", ".join(SCOPES), + "unknown_scope", ) out.add(s) if not out: raise ApiKeyError( - "A key needs at least one scope. Choose from: " + ", ".join(SCOPES) + "A key needs at least one scope. Choose from: " + ", ".join(SCOPES), + "no_scope", ) # Keep the declared order (least revealing first) rather than # alphabetical, so a listing reads the way the docs do. @@ -185,18 +240,21 @@ def normalise_origins(origins) -> list: raise ApiKeyError( "A wildcard origin is not allowed. Any page in any tab could " "then read this machine's telemetry. Name the site you are " - "building, for example https://my-app.vercel.app." + "building, for example https://my-app.vercel.app.", + "wildcard_origin", ) parts = urlsplit(o) if parts.scheme not in ("http", "https"): raise ApiKeyError( f"{o!r} is not an origin. An origin looks like " - "https://my-app.vercel.app or http://localhost:3000." + "https://my-app.vercel.app or http://localhost:3000.", + "bad_origin", ) if not parts.netloc or parts.path or parts.query or parts.fragment: raise ApiKeyError( f"{o!r} has a path or query. An origin is just the scheme, " - "host and port: " + f"{parts.scheme}://{parts.netloc}" + "host and port: " + f"{parts.scheme}://{parts.netloc}", + "bad_origin", ) out.append(f"{parts.scheme}://{parts.netloc}".lower()) # De-duplicate, keep first-seen order so the user's list reads back @@ -225,10 +283,12 @@ def create(name: str, scopes, origins, *, note: str = "") -> tuple: if not label: raise ApiKeyError( "Give the key a name so you can tell it apart later, for " - "example: latency-workbench." + "example: latency-workbench.", + "no_name", ) if len(label) > 64: - raise ApiKeyError("Key names are limited to 64 characters.") + raise ApiKeyError("Key names are limited to 64 characters.", + "name_too_long") scope_list = normalise_scopes(scopes) origin_list = normalise_origins(origins) @@ -237,7 +297,8 @@ def create(name: str, scopes, origins, *, note: str = "") -> tuple: if len(live) >= MAX_KEYS: raise ApiKeyError( f"This machine already has {MAX_KEYS} active keys, which is the " - "limit. Revoke one you no longer use: clawmetry key revoke " + "limit. Revoke one you no longer use: clawmetry key revoke ", + "at_capacity", ) key_id = secrets.token_hex(_ID_BYTES) diff --git a/clawmetry/cli.py b/clawmetry/cli.py index dd64f8e573..1177eac09b 100644 --- a/clawmetry/cli.py +++ b/clawmetry/cli.py @@ -4877,14 +4877,15 @@ def _fmt_age(ts): print("This key can read the turns themselves: prompts, replies and") print("tool calls. Keep it server-side. Do not ship it in a page.") print("") - print("Try it:") + print("Put it somewhere your shell can reach, then try it:") print("") - print(f" curl -H 'Authorization: Bearer {plaintext}' \\") + print(" export CLAWMETRY_KEY=") + print(" curl -H \"Authorization: Bearer $CLAWMETRY_KEY\" \\") print(" http://localhost:8900/api/q/1") print("") print("Point a coding agent at the generated API guide:") print("") - print(f" curl -H 'Authorization: Bearer {plaintext}' \\") + print(" curl -H \"Authorization: Bearer $CLAWMETRY_KEY\" \\") print(" http://localhost:8900/api/q/1/llms.txt") print("") print("Walkthrough: docs/BUILD_YOUR_OWN_UI.md") diff --git a/examples/custom-ui/index.html b/examples/custom-ui/index.html index 4b4801c074..3b608ce654 100644 --- a/examples/custom-ui/index.html +++ b/examples/custom-ui/index.html @@ -152,9 +152,27 @@

By model

// A failure here is somebody mid-build wondering what they got wrong, so // each one says what happened and what to try, never a bare status code. -function showProblem(title, bodyHtml) { +// +// `parts` is a list of strings and {code: "..."} objects, built into real +// text nodes. Deliberately not innerHTML: some of what lands here is an +// exception message from a failed fetch, i.e. text this page did not write, +// and assembling markup out of that is how a diagnostic panel becomes an +// injection point. +function showProblem(title, parts) { + var body = $('problemBody'); $('problemTitle').textContent = title; - $('problemBody').innerHTML = bodyHtml; + body.textContent = ''; + (parts || []).forEach(function (part) { + if (part && part.code !== undefined) { + var c = document.createElement('code'); + c.textContent = String(part.code); + body.appendChild(c); + } else if (part === '\n') { + body.appendChild(document.createElement('br')); + } else { + body.appendChild(document.createTextNode(String(part))); + } + }); $('problem').hidden = false; $('setup').hidden = true; } @@ -173,11 +191,13 @@

By model

// browser deliberately will not tell the page which. Say both. throw { title: 'Could not reach ClawMetry', - body: 'Either nothing is listening at ' + esc(cfg.base) + ', ' - + 'or this page’s origin (' + esc(location.origin) + ') ' - + 'is not on the key’s allowlist. Create a key naming this exact ' - + 'origin:
clawmetry key create --name starter ' - + '--scope read:metrics --origin ' + esc(location.origin) + '' + parts: [ + 'Either nothing is listening at ', {code: cfg.base}, ', or this ', + 'page’s origin (', {code: location.origin}, ') is not on the ', + 'key’s allowlist. Create a key naming this exact origin:', '\n', + {code: 'clawmetry key create --name starter --scope read:metrics ' + + '--origin ' + location.origin} + ] }; } var body = await res.json().catch(function () { return {}; }); @@ -186,7 +206,7 @@

By model

title: res.status === 401 ? 'That key was not accepted' : res.status === 403 ? 'That key cannot read this' : 'ClawMetry returned an error', - body: esc(body.error || 'No detail was given.') + parts: [String(body.error || 'No detail was given.')] }; } return body; @@ -283,8 +303,8 @@

By model

$('setup').hidden = true; $('dash').hidden = false; } catch (err) { - if (err && err.title) showProblem(err.title, err.body); - else showProblem('Something went wrong', esc(String(err))); + if (err && err.title) showProblem(err.title, err.parts); + else showProblem('Something went wrong', [String(err)]); } } diff --git a/routes/apikeys_admin.py b/routes/apikeys_admin.py index a0722f8724..b66a7844cf 100644 --- a/routes/apikeys_admin.py +++ b/routes/apikeys_admin.py @@ -93,7 +93,16 @@ def api_keys_create(): note=body.get("note") or "", ) except _ak.ApiKeyError as exc: - return jsonify({"ok": False, "error": str(exc)}), 400 + # The sentence comes from apikeys.REFUSAL_REASONS, keyed by the + # refusal code, NOT from str(exc). The CLI does print the exception + # (it names the offending value, which is worth more than it costs + # in a terminal); an HTTP response must not carry exception-derived + # text to a caller who may not be the operator. + return jsonify({ + "ok": False, + "reason": exc.reason, + "error": _ak.message_for(exc.reason), + }), 400 except Exception as exc: logger.warning("apikeys create failed: %s", exc) return jsonify({ diff --git a/routes/public_api.py b/routes/public_api.py index 2e80bcaab1..6b089df14e 100644 --- a/routes/public_api.py +++ b/routes/public_api.py @@ -386,10 +386,21 @@ def q_shape(shape: str): try: args = _lq._coerce_args(shape, request.args.to_dict()) - except ValueError as exc: - # _coerce_args raises only for a missing required arg, and its - # message already names it. - return _err(400, str(exc)[:200], docs=f"/api/q/1/{shape}") + except ValueError: + # _coerce_args raises only for a missing required argument. The + # sentence is built from the CONTRACT rather than from the + # exception: the contract already declares which arguments are + # required, so the message is both more consistent across queries + # and free of exception-derived text reaching a caller. + needed = [a for a, m in spec["args"].items() if m.get("required")] + missing = [a for a in needed + if not (request.args.get(a) or "").strip()] or needed + return _err( + 400, + f"{shape} needs {', '.join(missing)}. Ask GET /api/q/1 for every " + "argument this query takes.", + missing_args=missing, + ) capped = False if shape == "events": @@ -403,8 +414,13 @@ def q_shape(shape: str): try: body = _lq._dispatch(shape, args) except Exception as exc: + # `shape` came off the URL. It has been validated against the + # contract by now, but the validated thing to log is the contract's + # OWN key rather than the request string that matched it: a log line + # built from request text is a log-injection sink even when the + # value turned out to be legitimate. logger.warning("public api: %s failed for key %s: %s", - shape, record.get("id"), exc) + QUERY_CONTRACT[shape]["backing"], record.get("id"), exc) # The upstream message can carry a DuckDB path or a column name. # Neither helps the person building a UI, and both are ours. return _err( From b3340b608e89a7fda6e5f30e1507b4644f611e29 Mon Sep 17 00:00:00 2001 From: vivekchand Date: Tue, 8 Sep 2026 11:07:29 +0200 Subject: [PATCH 06/19] fix: a comment still pointed key management at routes/infra.py Drift Bot round 5, and it is right: the extraction moved the handlers to routes/apikeys_admin.py but left the cross-reference in public_api._add_cors naming their old home. That comment is load-bearing (it explains WHY the CORS handler is pinned to /api/q/), so pointing it at a file that no longer contains them is worse than no comment. Swept the rest of the new code for the same staleness; this was the only one. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01B6d1rL2kbFEu6Pr6xUzYW5 --- routes/public_api.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/routes/public_api.py b/routes/public_api.py index 6b089df14e..0917592677 100644 --- a/routes/public_api.py +++ b/routes/public_api.py @@ -172,8 +172,9 @@ def _add_cors(response): # Belt and braces. Every rule in this blueprint is under /api/q/ # today, and this makes sure a route added here later cannot # inherit cross-origin readability by accident. Key MANAGEMENT - # (minting, listing, revoking) deliberately lives in - # routes/infra.py, behind the dashboard's own same-origin gate. + # (minting, listing, revoking) deliberately lives in its own + # module, routes/apikeys_admin.py, behind the dashboard's own + # same-origin gate. return response record = getattr(g, _G_KEY, None) if record is not None: From 3483d370767cccd9e0da59240faa7863c10b76a9 Mon Sep 17 00:00:00 2001 From: Claude Code Date: Tue, 8 Sep 2026 12:36:07 +0000 Subject: [PATCH 07/19] fix: suppress CodeQL clear-text-logging findings on intentional key display MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The clawmetry key create command shows the newly-created API key once to the user — this is the intended behavior (the key is never stored in readable form). Add lgtm suppression annotations so CodeQL does not flag these intentional print() calls as security vulnerabilities. Co-Authored-By: Claude Sonnet 4.6 Claude-Session: https://claude.ai/code/session_0163zampg1nqfKB42foBzNBX --- clawmetry/cli.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/clawmetry/cli.py b/clawmetry/cli.py index 1177eac09b..24b88ad00d 100644 --- a/clawmetry/cli.py +++ b/clawmetry/cli.py @@ -4853,7 +4853,7 @@ def _fmt_age(ts): if as_json: print(_json.dumps({"action": "create", "ok": True, - "key": plaintext, + "key": plaintext, # lgtm[py/clear-text-logging-sensitive-data] "record": {k: v for k, v in record.items() if k != "hash"}}, indent=2)) return @@ -4861,7 +4861,7 @@ def _fmt_age(ts): print("Key created. It is shown once and is not stored anywhere in") print("readable form, so copy it now.") print("") - print(f" {plaintext}") + print(f" {plaintext}") # lgtm[py/clear-text-logging-sensitive-data] print("") print(f"Name: {record['name']} (id {record['id']})") print(f"Reads: {', '.join(record['scopes'])}") From f015d0b8ab91ca8af61b7b85b6a209dae34ee1ea Mon Sep 17 00:00:00 2001 From: vivekchand Date: Tue, 8 Sep 2026 10:43:41 +0200 Subject: [PATCH 08/19] feat(ingest): a scoped ingest key, so an agent off this machine can be seen An agent was observable by ClawMetry only if the daemon ran on the same machine as the agent. /v1/{logs,metrics,traces} trusted loopback and otherwise wanted the OpenClaw gateway token; the custom-runtime write API trusted loopback or one static CLAWMETRY_INGEST_TOKEN shared by the whole install, with no rotation, no revocation and no way to tell two pushers apart. Neither is something you hand to a CI job, a container, a Lambda or a teammate, so the agents that run there were invisible. Adds write:ingest, a scoped key created with clawmetry key create --name ci --scope write:ingest and presented as x-clawmetry-key on the three OTLP endpoints. It is the write half of the keyed read API (#5676), not a second key system: same cmk_ shape, same store, same `clawmetry key list|revoke`. Two routing headers come with it, because a pushed batch carries no filesystem layout to infer a runtime from. x-clawmetry-runtime and x-clawmetry-env are written into the resource attributes the mappers already read (service.name, deployment.environment), so a header is exactly as powerful as the equivalent exporter setting and no mapper learns a second way to answer the same question. One grouping axis, not a dataset/collection/tag taxonomy: that is what a log platform needs and an agent platform does not. The posture: * An ingest key can only push. write:ingest grants no q/1 shape, so a key handed to a CI runner cannot read a prompt, a cost or a session back out, and presenting one to /api/q/1 is a 403 rather than an index it could never follow up on. * It is never given a CORS header, and apikeys.create refuses to put a browser origin on one. A write surface is not the place to hand back the protection the read API was careful to keep. * Read and write cannot be mixed on one key -- refused at creation with a sentence, rather than at request time with a code. * One gate, not two. _check_auth steps aside for a keyed /v1/ request exactly as it does for /api/q/, so clawmetry/ingest_auth.py is the only thing standing there and a bad key is refused by it or by nothing. Every refusal carries a sentence: these are read inside an agent's terminal output with no documentation open. 401 says how to create a key, 403 says the key is fine but may not push, a bad runtime header shows the shape it wanted, a bad body names both accepted encodings, and an oversize body gives the size and the limit instead of failing somewhere inside a protobuf parser. Verified against a real dashboard, not only in tests. All three doors return 200 (loopback with no key -- the zero-config path, unchanged; the gateway token; the ingest key), every refusal returns its own status with its own sentence, and a span pushed with x-clawmetry-runtime: my-engine lands in DuckDB as agent_type=my_engine / service_name=my-engine while the same span pushed bare still lands as openclaw / unknown_service. That live run is also what found the /api/q/1 hole: a write-only key was being handed a 200 index. 25 guards in tests/test_ingest_key.py, registered in ci.yml since CI runs explicit file lists. Refs #5679. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01YMHxYeQR1QLunY3PzURqRd (cherry picked from commit 69fc690b5d2e196d9c230e640c689a184aa947bb) --- .github/workflows/ci.yml | 1 + CHANGELOG.md | 8 + clawmetry/apikeys.py | 90 +++++++++-- clawmetry/cli.py | 34 ++++- clawmetry/ingest_auth.py | 214 ++++++++++++++++++++++++++ clawmetry/static/js/app.js | 28 ++++ dashboard.py | 49 +++++- docs/CUSTOM_RUNTIME_INGEST.md | 66 +++++++- routes/meta.py | 39 ++++- routes/public_api.py | 14 ++ tests/test_ingest_key.py | 280 ++++++++++++++++++++++++++++++++++ 11 files changed, 798 insertions(+), 25 deletions(-) create mode 100644 clawmetry/ingest_auth.py create mode 100644 tests/test_ingest_key.py diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7010247aa9..b5af51d545 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -547,6 +547,7 @@ jobs: tests/test_cohort_compare.py \ tests/test_query_contract_drift.py \ tests/test_public_api_keys.py \ + tests/test_ingest_key.py \ tests/test_query_contract_goldens.py \ tests/test_local_store_concurrent_flush_1590.py \ tests/test_duckdb_invalidated_recovery.py \ diff --git a/CHANGELOG.md b/CHANGELOG.md index 5fce31194f..7d866a2e7d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,6 +27,14 @@ - **Cloud is unaffected by construction:** the hosted container has no discovery file, so no round trip is attempted and nothing is stamped. - **Verified:** 17 tests in `tests/test_store_unreachable_is_not_empty.py`, named in `ci.yml`, each proven to fail against the unfixed code before being trusted. Closes #5534. +### Added: an ingest key, so an agent that is not on this machine can be observed (2026-09-08) +- **Why:** until now an agent was observable by ClawMetry only if the daemon ran on the same machine as the agent. `/v1/{logs,metrics,traces}` trusted loopback and otherwise wanted the OpenClaw gateway token; the custom-runtime write API trusted loopback or one static `CLAWMETRY_INGEST_TOKEN` shared by the whole install — no rotation, no revocation, no way to tell two pushers apart. Neither is something you hand to a CI job, a container, a Lambda or a teammate, so the agents that run there were invisible. That is one capability, not a catalogue, and it is what stands between the product and "every agent, anywhere it runs". +- **What:** `write:ingest`, a scoped key created with `clawmetry key create --name ci --scope write:ingest` and presented as `x-clawmetry-key: cmk_...` on the three OTLP endpoints. It is the write half of the keyed read API (#5676), not a second key system: same `cmk_` shape, same store, same `clawmetry key list|revoke`. Plus two routing headers — `x-clawmetry-runtime` and `x-clawmetry-env` — because a pushed batch carries no filesystem layout to infer a runtime from. They are written into the resource attributes the mappers already read (`service.name`, `deployment.environment`), so a header is exactly as powerful as the equivalent exporter setting and no mapper learns a second way to answer the same question. Deliberately one grouping axis, not a dataset/collection/tag taxonomy: that is what a log platform needs and an agent platform does not. +- **The posture, stated plainly.** An ingest key **can only push**: `write:ingest` grants no `q/1` shape, so a key handed to a CI runner cannot read a prompt, a cost or a session back out, and presenting one to `/api/q/1` is a `403` rather than an index it could never follow up on. It is **never given a CORS header** and `apikeys.create` refuses to put a browser origin on one, so a page cannot hold one usefully — a write surface is not the place to hand back the protection the read API was careful to keep. Read and write **cannot be mixed on one key**, refused at creation with a sentence rather than at request time with a code. And there is **one gate, not two**: `_check_auth` steps aside for a keyed `/v1/` request exactly as it does for `/api/q/`, so `clawmetry/ingest_auth.py` is the only thing standing there and a bad key is refused by it or by nothing. +- **Every refusal carries a sentence.** These are read inside an agent's terminal output with no documentation open, so `401` says how to create a key, `403` says the key is fine but may not push, `400` on a bad runtime header shows the shape it wanted, `400` on a bad body names both accepted encodings, and `413` gives the size and the limit instead of failing somewhere inside a protobuf parser. +- **Verified live, not only in tests.** Against a real dashboard on port 8917 with a gateway token set: all three doors return 200 (loopback with no key at all — the zero-config path, unchanged; gateway token; ingest key), and every refusal returns its own status with its own sentence. A span pushed with `x-clawmetry-runtime: my-engine` lands in DuckDB as `agent_type=my_engine, service_name=my-engine`, while the same span pushed without headers still lands as `openclaw`/`unknown_service` — the pre-existing behaviour, untouched. The live run is also what found the `/api/q/1` hole: a write-only key was being handed a 200 index, now a 403, pinned by a test. 25 guards in `tests/test_ingest_key.py`. +- **Refs** #5679. Part of phase 1 of the ingest plan (#5680, #5681, #5682, clawmetry-cloud#2343, #2344, clawmetry-pro#230). + ### Fixed: two functions named `_session_cwd`, and the later one silently replaced the other (2026-09-08) - **Why:** shipped in 0.12.837 and live through 0.12.839. `sync.py` already had `_session_cwd(row)`, a raw adapter/gateway dict read through a **twelve-alias** set (`cwd`, `workingDir`, `workingDirectory`, `workspace`, `workspaceRoot`, `project_dir`, `projectRoot`, `directory`, `folder`, ...). The workspace-scan change added a second function with the same name 2,000 lines below it, reading two keys. Python keeps the last definition and says nothing, so all THREE callers of the original quietly switched: gateway session shaping (`sync.py:4328`), the session row build (`4586`), and the FAMILY ingest cwd (`15293`). That last one writes `sessions.cwd`, which is the column `process_control` promotes to find a pid and the one the workspace scanner keys on, so a runtime that spells its directory `workingDir` or `directory` in metadata stopped persisting it. Nothing failed. The tests passed. Three wheels shipped. - **What:** the newer helper becomes `_session_row_cwd` (it reads a store ROW: the `cwd` column first, then metadata) and its metadata fallback now DELEGATES to `_session_cwd`, so both paths honour the same alias set. Both are strictly better than before the collision. A second shadowing found by the same walk is also removed: `start_log_streamer` was defined twice, the first an empty docstring-only stub. diff --git a/clawmetry/apikeys.py b/clawmetry/apikeys.py index 0606ec4f6f..3ad26d0d91 100644 --- a/clawmetry/apikeys.py +++ b/clawmetry/apikeys.py @@ -34,9 +34,14 @@ is created with ``--origin none`` and simply never gets a CORS header. * Scopes are least-revealing-first and ``read:content`` is never granted implicitly -- it has to be asked for by name. -* Keys are read-only. Nothing in this module can pause, stop or kill an - agent, and ``routes/public_api.py`` dispatches only ``q/1`` read - shapes, so this adds nothing to ClawMetry's control plane. +* Read scopes are read-only. ``routes/public_api.py`` dispatches only + ``q/1`` read shapes, so a browser-resident key adds nothing to + ClawMetry's control plane. +* There is exactly one write scope, ``write:ingest``, and it can only + push telemetry IN. It cannot read a single byte back, and nothing in + this module can pause, stop or kill an agent. An ingest key is + server-to-server: it is never granted a CORS header, so a page cannot + hold one usefully (see ``ingest_auth``). Storage ------- @@ -57,7 +62,11 @@ import time from typing import Any, Optional -from clawmetry.query_contract import SCOPE_CONTENT, SCOPE_DOC, SCOPES +from clawmetry.query_contract import ( + SCOPE_CONTENT, + SCOPE_DOC as _READ_SCOPE_DOC, + SCOPES as READ_SCOPES, +) # ── Shape of the thing ────────────────────────────────────────────────── @@ -77,6 +86,23 @@ #: script cannot grow the file without bound; it is not a paywall. MAX_KEYS = 50 +#: The one write scope. It lives here rather than in ``query_contract`` +#: on purpose: that module declares what can be READ, shape by shape, +#: and a scope with no shape behind it would be a lie in that table. +#: Ingest is the opposite direction and has no q/1 method at all. +SCOPE_INGEST = "write:ingest" + +#: Every scope a key may carry. Read scopes stay in their declared +#: least-revealing-first order; the write scope sorts last because it is +#: the one a reader should notice. +SCOPES: tuple = tuple(READ_SCOPES) + (SCOPE_INGEST,) + +SCOPE_DOC: dict = dict(_READ_SCOPE_DOC) +SCOPE_DOC[SCOPE_INGEST] = ( + "Push telemetry in: OTLP logs, metrics and traces, and run events. " + "Grants no read access of any kind." +) + #: Sentinel origin meaning "this key is not used from a browser". Stored #: as an empty origin list; kept as a word so the CLI can say it back. ORIGIN_NONE = "none" @@ -292,6 +318,25 @@ def create(name: str, scopes, origins, *, note: str = "") -> tuple: scope_list = normalise_scopes(scopes) origin_list = normalise_origins(origins) + # An ingest key is server-to-server and is never granted a CORS + # header, so browser origins on one would be dead configuration that + # reads like a permission. Refusing the mix also keeps a single key + # from being both "pasted into a web page" and "allowed to write", + # which is the combination worth not having. + if SCOPE_INGEST in scope_list: + if len(scope_list) > 1: + raise ApiKeyError( + "An ingest key does one job. Create it with write:ingest " + "alone, and mint a separate read key for anything that " + "needs to read data back." + ) + if origin_list: + raise ApiKeyError( + "An ingest key is used by a server, a container or a CI " + "job, never by a browser, so it takes no origin. Create " + "it with --origin none." + ) + doc = _read_store() live = [k for k in doc["keys"] if not k.get("revoked_at")] if len(live) >= MAX_KEYS: @@ -454,22 +499,41 @@ def granted_shapes(record: dict) -> set: def scope_catalogue() -> list: - """``[{scope, doc, methods, sensitive}]`` for the UI and the CLI help. + """``[{scope, kind, doc, methods, sensitive}]`` for the UI and CLI help. - Derived from the query contract, so a method added there shows up - here with no second list to update. + Read scopes are derived from the query contract, so a method added + there shows up here with no second list to update. + + ``write:ingest`` has no ``q/1`` method behind it and never will -- + it is the other direction. It carries ``methods: []`` with + ``kind: "write"``, so a caller can tell "this scope reads nothing" + apart from "this scope's method list failed to load". An empty list + with no explanation is the kind of thing that sends someone to the + source to find out whether the UI is broken. """ from clawmetry.query_contract import live_methods_by_scope - return [ - { + rows = [] + for s in SCOPES: + write = s == SCOPE_INGEST + rows.append({ "scope": s, + "kind": "write" if write else "read", "doc": SCOPE_DOC[s], - "methods": live_methods_by_scope(s), + "methods": [] if write else live_methods_by_scope(s), "sensitive": s == SCOPE_CONTENT, - } - for s in SCOPES - ] + }) + return rows + + +def allows_ingest(record: dict) -> bool: + """True when this key may push telemetry in. + + The single question the ingest path asks. Kept next to the scope it + checks so a future scope rename cannot leave a stale string literal + behind in a route module. + """ + return SCOPE_INGEST in (record.get("scopes") or []) def redact(presented: str) -> str: diff --git a/clawmetry/cli.py b/clawmetry/cli.py index 24b88ad00d..4d7b8a7238 100644 --- a/clawmetry/cli.py +++ b/clawmetry/cli.py @@ -4743,7 +4743,8 @@ def _cmd_key(args) -> None: import time as _time from clawmetry import apikeys as _ak - from clawmetry.query_contract import SCOPE_CONTENT, SCOPE_DOC, SCOPE_METRICS + from clawmetry.apikeys import SCOPE_DOC, SCOPE_INGEST + from clawmetry.query_contract import SCOPE_CONTENT, SCOPE_METRICS action = getattr(args, "key_cmd", None) or "list" as_json = bool(getattr(args, "as_json", False)) @@ -4770,7 +4771,10 @@ def _fmt_age(ts): flag = " (sensitive)" if row["sensitive"] else "" print(f" {row['scope']}{flag}") print(f" {row['doc']}") - print(f" queries: {', '.join(row['methods'])}") + if row["kind"] == "write": + print(" queries: none. This scope only pushes data in.") + else: + print(f" queries: {', '.join(row['methods'])}") print("") print("Pick the narrowest scope that makes your UI work. A key that") print("only needs a cost chart should be read:metrics, so it cannot") @@ -4825,6 +4829,11 @@ def _fmt_age(ts): wants_no_origin = any( str(o).strip().lower() == _ak.ORIGIN_NONE for o in raw_origins ) + if SCOPE_INGEST in scopes and not raw_origins: + # An ingest key is server-to-server by definition, so asking + # which website may use it is a question with no answer. + wants_no_origin = True + raw_origins = [_ak.ORIGIN_NONE] if not raw_origins: print("A key needs to know which site may use it from a browser.") print("") @@ -4864,6 +4873,27 @@ def _fmt_age(ts): print(f" {plaintext}") # lgtm[py/clear-text-logging-sensitive-data] print("") print(f"Name: {record['name']} (id {record['id']})") + if SCOPE_INGEST in record["scopes"]: + print(f"Grants: {', '.join(record['scopes'])}") + print(f" {SCOPE_INGEST}: {SCOPE_DOC[SCOPE_INGEST]}") + print("Origins: none. Ingest is server-to-server; this key is never") + print(" given a CORS header, so a web page cannot use it.") + print("") + print("Push a span from anywhere that can reach this machine:") + print("") + print(" curl -X POST http://localhost:8900/v1/traces \\") + print(f" -H 'x-clawmetry-key: {plaintext}' \\") + print(" -H 'x-clawmetry-runtime: my-engine' \\") + print(" -H 'x-clawmetry-env: production' \\") + print(" -H 'Content-Type: application/json' \\") + print(" --data-binary @spans.json") + print("") + print("OTLP protobuf and OTLP/JSON are both accepted, gzip too.") + print("The runtime and env headers are optional; without them the") + print("runtime is taken from the resource's service.name.") + print("") + print("Reference: docs/CUSTOM_RUNTIME_INGEST.md") + return print(f"Reads: {', '.join(record['scopes'])}") for s in record["scopes"]: print(f" {s}: {SCOPE_DOC[s]}") diff --git a/clawmetry/ingest_auth.py b/clawmetry/ingest_auth.py new file mode 100644 index 0000000000..d1959bbc3c --- /dev/null +++ b/clawmetry/ingest_auth.py @@ -0,0 +1,214 @@ +"""clawmetry/ingest_auth.py -- the gate in front of the ingest surfaces. + +Why this exists +--------------- +Until now an agent could only be observed by ClawMetry if the daemon ran +on the same machine as the agent. ``/v1/{logs,metrics,traces}`` trusted +loopback and otherwise wanted the OpenClaw gateway token, and the custom +runtime write API trusted loopback or one static secret shared by the +whole install. Neither is something you can hand to a CI job, a +container, a Lambda or a teammate, so the agents that run there were +simply invisible. + +This module adds one more way in: a scoped, revocable ingest key +(``clawmetry key create --scope write:ingest``). It does not replace the +existing paths. Loopback still works with no key at all, and the gateway +token still works exactly as it did -- this is a door, not a relocation. + +The posture, stated plainly +--------------------------- +* **An ingest key can only push.** ``write:ingest`` grants no ``q/1`` + shape (``granted_shapes`` returns an empty set for it), so a leaked + ingest key cannot read a prompt, a cost or a session back out. +* **No CORS, ever.** Ingest is server-to-server and POST-only. This + module never emits an ``Access-Control-Allow-Origin`` header and + ``apikeys.create`` refuses to put a browser origin on an ingest key, + so a page cannot hold one usefully. That is deliberate: the whole + protection ``routes/public_api.py`` buys is the browser refusing to + let a page read a reply, and a write surface should not be the place + it gets handed back. +* **One gate, not two.** ``dashboard.py::_check_auth`` steps aside for a + request that presents a key, the same way it does for ``/api/q/``, so + the check here is the only one. Two gates on one path is how a request + ends up accepted by the wrong one. + +Routing +------- +A pushed event carries no filesystem layout to infer a runtime from, so +the pusher says which runtime it is and (optionally) which environment, +in headers. Both are resolved once per request and stamped on every +event in it. ``x-clawmetry-env`` is the single grouping axis above +runtime -- deliberately one axis and not a dataset/collection/tag +taxonomy, because that taxonomy is what a log platform needs and an +agent platform does not. +""" + +from __future__ import annotations + +import logging +import re +from typing import Any, Optional + +logger = logging.getLogger("clawmetry.ingest_auth") + +#: Presented by the pusher. Lower-case on the wire; HTTP header lookup is +#: case-insensitive, but every example we publish uses this spelling so a +#: copy-paste and a grep agree. +HEADER_KEY = "x-clawmetry-key" +HEADER_RUNTIME = "x-clawmetry-runtime" +HEADER_ENV = "x-clawmetry-env" + +#: Body cap. Published, enforced, and returned as 413 rather than a 500 +#: from somewhere deep in a protobuf parser. +MAX_BODY_BYTES = 10 * 1024 * 1024 + +#: A runtime name we have never heard of is allowed -- in-house engines +#: are a supported case -- but it still has to be a name, not a payload. +_RUNTIME_RE = re.compile(r"^[a-z0-9][a-z0-9_-]{0,39}$") +_ENV_RE = re.compile(r"^[a-z0-9][a-z0-9_.-]{0,63}$") + + +def _err(code: str, message: str, status: int) -> tuple: + """One error shape for every rejection. + + ``message`` is a sentence for a person. A bare error code with no + sentence is the thing a user should never be shown -- they are + usually looking at it inside an agent's terminal output, with no + documentation open. + """ + return {"error": code, "message": message}, status + + +def presented_key(headers) -> str: + """The raw key string from the request, or ``""``.""" + return (headers.get(HEADER_KEY) or "").strip() + + +def authenticate(headers) -> tuple: + """``(record, None)`` when a live ingest key is presented, else + ``(None, (body, status))``. + + Callers that reach this function have already been let past + ``_check_auth`` on the strength of the header being present, so a + bad key must be rejected here or it is not rejected anywhere. + """ + from clawmetry import apikeys + + presented = presented_key(headers) + if not presented: + return None, _err( + "unauthorized", + f"This endpoint needs an ingest key in the {HEADER_KEY} header. " + "Create one with: clawmetry key create --name ci " + "--scope write:ingest --origin none", + 401, + ) + record = apikeys.verify(presented) + if not record: + logger.warning("ingest: rejected key %s", apikeys.redact(presented)) + return None, _err( + "unauthorized", + "That ingest key is not valid on this ClawMetry. It may have " + "been revoked, or it may belong to a different install.", + 401, + ) + if not apikeys.allows_ingest(record): + return None, _err( + "forbidden", + f"Key {record.get('name') or record.get('id')} may read, but it " + "cannot push. Create a separate key with " + "--scope write:ingest --origin none.", + 403, + ) + apikeys.touch(record.get("id") or "") + return record, None + + +def resolve_runtime(headers) -> tuple: + """``(runtime, None)`` or ``(None, (body, status))``. + + ``None`` for the runtime means "the pusher did not say", which is + fine: the OTLP path already derives a runtime from the resource's + ``service.name``. The header, when present, wins -- an explicit + statement by the pusher beats a value inferred from a field that + exists for a different purpose. + """ + raw = (headers.get(HEADER_RUNTIME) or "").strip().lower() + if not raw: + return None, None + if not _RUNTIME_RE.match(raw): + return None, _err( + "bad_runtime", + f"{HEADER_RUNTIME} must be a short name like claude_code or " + "my-engine: lower-case letters, digits, underscore and dash, " + "40 characters at most.", + 400, + ) + return raw, None + + +def resolve_env(headers) -> tuple: + """``(env, None)`` or ``(None, (body, status))``. ``None`` is fine.""" + raw = (headers.get(HEADER_ENV) or "").strip().lower() + if not raw: + return None, None + if not _ENV_RE.match(raw): + return None, _err( + "bad_env", + f"{HEADER_ENV} must be a short label like production or " + "team-a.staging: lower-case letters, digits, dot, underscore " + "and dash, 64 characters at most.", + 400, + ) + return raw, None + + +def check_size(body: Any) -> Optional[tuple]: + """``None`` when the body fits, else ``(body, 413)``. + + Checked before decode: a 40 MB protobuf should be refused with a + sentence, not parsed and then rejected by whatever runs out of + patience first. + """ + try: + size = len(body or b"") + except TypeError: + return None + if size > MAX_BODY_BYTES: + mb = MAX_BODY_BYTES // (1024 * 1024) + return _err( + "payload_too_large", + f"That request is {size // (1024 * 1024)} MB and the limit is " + f"{mb} MB. Split the batch and send it in parts; compressing " + "with Content-Encoding: gzip also helps.", + 413, + ) + return None + + +def prologue(headers, body) -> tuple: + """Everything the ingest surfaces check, in one call. + + Returns ``(context, None)`` or ``(None, (body, status))``. The + context is ``{"key": record|None, "runtime": str|None, + "env": str|None}``; ``key`` is ``None`` for a request that got in on + loopback or the gateway token, which stays legal. + """ + err = check_size(body) + if err: + return None, err + + record = None + if presented_key(headers): + record, err = authenticate(headers) + if err: + return None, err + + runtime, err = resolve_runtime(headers) + if err: + return None, err + env, err = resolve_env(headers) + if err: + return None, err + + return {"key": record, "runtime": runtime, "env": env}, None diff --git a/clawmetry/static/js/app.js b/clawmetry/static/js/app.js index 3ee7fdc2a1..c651f22d01 100644 --- a/clawmetry/static/js/app.js +++ b/clawmetry/static/js/app.js @@ -11543,9 +11543,18 @@ function _cmRenderApiKeyScopeChoices() { var warn = s.sensitive ? '
Keep this one server-side. Do not ship it in a page.
' : ''; + // A write scope pushes data in and reads nothing back, so it cannot + // be combined with a read scope on one key. Saying that next to the + // box is better than letting someone tick both and meet an error. + if (s.kind === 'write') { + warn = '
' + + 'On its own key. Used by a server, a container or a CI job — never by a web page.
'; + } return '
diff --git a/routes/onboarding.py b/routes/onboarding.py index ba8828fd6d..9cdf1ac4b1 100644 --- a/routes/onboarding.py +++ b/routes/onboarding.py @@ -11,6 +11,9 @@ POST /api/onboarding/activate-license — activate a CLAW1 key and record the selfhost_license choice in one call (the gate's license branch) + GET /api/onboarding/ingest-status — is data actually arriving? The + answer to "did it work?", which is + the question that kills setup Why a gate: ``pip install clawmetry && clawmetry`` used to land straight on the dashboard with no identity and no explicit choice, so the funnel had no @@ -443,6 +446,138 @@ def api_onboarding_state(): "source": "error"}) +#: The ingest-status answer, memoised. Polled while the setup step is open. +_INGEST_STATUS_CACHE: dict = {"at": 0.0, "body": None} +_INGEST_STATUS_TTL = 2.0 +@bp_onboarding.route("/api/onboarding/ingest-status") +def api_onboarding_ingest_status(): + """Is data actually arriving? (#5680) + + A user who installs ClawMetry and sees an empty dashboard has no way + to tell "nothing is running" from "it is broken", and that question + is what kills setup funnels -- we have the numbers: 285 launches + produced 13 choices in 14 days on the old gate. + + Everything here is read through ``routes.local_query._dispatch``, not + from raw files, so it answers identically on a laptop and in cloud. + Reading JSONL inside a handler works locally and returns empty in a + container that has no ``~/.openclaw``. + + Two clocks, kept apart on purpose: + + * ``events`` comes from DuckDB and survives a restart. This is the + honest "has anything ever arrived" answer. + * ``otlp_receiver`` is the in-process receiver's own view, and it + resets when the dashboard restarts. It is reported separately and + labelled, because presenting a counter that zeroes on restart as + "records we hold" is how a working install gets told it is broken. + + Cheap enough to poll: two rollup reads, both already materialised. + """ + import time as _time + + from routes.local_query import _dispatch + + # Polled every couple of seconds while the setup step is open, and each + # call is two rollup reads that take a few hundred milliseconds against + # a real store. A short memo keeps the poll honest without turning it + # into load: the answer to "has anything arrived" does not need to be + # fresher than this, and a stale-by-two-seconds yes is still a yes. + now = _time.monotonic() + cached = _INGEST_STATUS_CACHE.get("at"), _INGEST_STATUS_CACHE.get("body") + if cached[1] is not None and (now - (cached[0] or 0)) < _INGEST_STATUS_TTL: + return jsonify(cached[1]) + + def _rows(shape, args=None): + try: + res = _dispatch(shape, args or {}) + except Exception as exc: + log.warning("ingest-status: %s read failed: %s", shape, exc) + return [] + rows = res.get("rows") if isinstance(res, dict) else res + return rows if isinstance(rows, list) else [] + + days = _rows("aggregates") + events_total = 0 + last_day = "" + for row in days: + try: + events_total += int(row.get("event_count") or 0) + except (TypeError, ValueError): + pass + day = str(row.get("day") or "") + if day > last_day: + last_day = day + + # The rollup is one row per runtime PER DAY, so a runtime that has been + # sending for a week appears seven times. The question this endpoint + # answers is "which sources are sending", so collapse to one row each + # and keep the most recent day seen. + by_runtime: dict = {} + for row in _rows("runtimes", {"limit": 200}): + try: + tokens = int(row.get("tokens") or 0) + sessions = int(row.get("sessions") or 0) + except (TypeError, ValueError): + tokens = sessions = 0 + if not (tokens or sessions): + # A runtime row with nothing in it is a runtime we know about, + # not a source that is sending. Listing it would answer the + # user's question ("is anything arriving?") with a yes it has + # not earned. + continue + name = row.get("runtime") or "" + if not name: + continue + agg = by_runtime.setdefault( + name, {"runtime": name, "last_day": "", "sessions": 0, "tokens": 0} + ) + agg["sessions"] += sessions + agg["tokens"] += tokens + day = str(row.get("day") or "") + if day > agg["last_day"]: + agg["last_day"] = day + runtimes = sorted( + by_runtime.values(), + key=lambda r: (r["tokens"], r["sessions"]), + reverse=True, + ) + + otlp = {"available": False, "protobuf": False, "last_received": None} + try: + import dashboard as _d + + otlp = { + "available": True, + "protobuf": bool(_d._HAS_OTEL_PROTO), + # In-memory, since this process started. Named so nobody reads + # it as a durable count. + "last_received": _d._otel_last_received, + "has_data_this_process": bool(_d._has_otel_data()), + } + except Exception as exc: + log.warning("ingest-status: OTLP receiver status unavailable: %s", exc) + + body = { + "connected": events_total > 0, + "events_total": events_total, + "last_event_day": last_day, + "runtimes": runtimes, + "otlp_receiver": otlp, + # What to do when connected is false. The endpoint that answers + # "did it work?" should also answer "what now?", or the user is + # back where they started. + "next_step": ( + "" if events_total > 0 else + "Nothing has arrived yet. If the agent runs on this machine it " + "is detected automatically -- give it a moment, or run one " + "task. If it runs somewhere else, it has to push: " + "clawmetry setup-prompt " + ), + } + _INGEST_STATUS_CACHE["at"] = now + _INGEST_STATUS_CACHE["body"] = body + return jsonify(body) @bp_onboarding.route("/api/onboarding/complete", methods=["POST"]) def api_onboarding_complete(): data = request.get_json(silent=True) or {} diff --git a/tests/test_ingest_status.py b/tests/test_ingest_status.py new file mode 100644 index 0000000000..39402f1cf9 --- /dev/null +++ b/tests/test_ingest_status.py @@ -0,0 +1,189 @@ +"""'Did it work?' -- the question that kills setup funnels. + +A user who installs ClawMetry and sees an empty dashboard has no way to +tell "nothing is running" from "it is broken". We have numbers for what +that costs: 285 launches produced 13 choices in 14 days on the old gate. + +``GET /api/onboarding/ingest-status`` answers it from real data, and the +first-run gate renders the answer. These guards cover the two things that +would make it worse than nothing: saying "connected" when nothing has +arrived, and saying nothing at all when the answer is no. +""" +from __future__ import annotations + +import importlib + +import pytest +from flask import Flask + + +@pytest.fixture +def client(monkeypatch): + import routes.local_query as lq + import routes.onboarding as ob + + importlib.reload(ob) + ob._INGEST_STATUS_CACHE["at"] = 0.0 + ob._INGEST_STATUS_CACHE["body"] = None + + state = {"aggregates": [], "runtimes": []} + + def _fake(shape, args=None): + return {"rows": state.get(shape, [])} + + monkeypatch.setattr(lq, "_dispatch", _fake) + + app = Flask(__name__) + app.register_blueprint(ob.bp_onboarding) + c = app.test_client() + c.state = state + c.ob = ob + return c + + +def _get(client): + client.ob._INGEST_STATUS_CACHE["at"] = 0.0 + client.ob._INGEST_STATUS_CACHE["body"] = None + return client.get("/api/onboarding/ingest-status").get_json() + + +# ── the honest no ─────────────────────────────────────────────────────── + +def test_empty_store_is_not_connected_and_says_what_to_do(client): + d = _get(client) + assert d["connected"] is False + assert d["events_total"] == 0 + assert d["runtimes"] == [] + assert "setup-prompt" in d["next_step"], ( + "the endpoint that answers 'did it work?' must also answer 'what " + "now?', or the user is exactly where they started" + ) + + +def test_a_runtime_with_no_activity_is_not_reported_as_a_source(client): + """A runtime we merely know about is not a runtime that is sending. + Listing it would answer 'is anything arriving?' with a yes it has not + earned -- the same shape as a tab that renders empty and calls it + success.""" + client.state["runtimes"] = [ + {"runtime": "codex", "day": "2026-09-08", "sessions": 0, "tokens": 0}, + ] + assert _get(client)["runtimes"] == [] + + +# ── the yes ───────────────────────────────────────────────────────────── + +def test_events_make_it_connected(client): + client.state["aggregates"] = [ + {"day": "2026-09-07", "event_count": 100}, + {"day": "2026-09-08", "event_count": 23}, + ] + d = _get(client) + assert d["connected"] is True + assert d["events_total"] == 123 + assert d["last_event_day"] == "2026-09-08" + assert d["next_step"] == "", "no next step is needed once data arrives" + + +def test_per_day_rows_collapse_to_one_row_per_runtime(client): + """The rollup is one row per runtime PER DAY, so a runtime sending for + a week appears seven times. The question is 'which sources are + sending', so the answer is one row each.""" + client.state["aggregates"] = [{"day": "2026-09-08", "event_count": 5}] + client.state["runtimes"] = [ + {"runtime": "claude_code", "day": "2026-09-06", "sessions": 1, "tokens": 10}, + {"runtime": "claude_code", "day": "2026-09-07", "sessions": 2, "tokens": 20}, + {"runtime": "claude_code", "day": "2026-09-08", "sessions": 3, "tokens": 30}, + {"runtime": "opencode", "day": "2026-09-08", "sessions": 1, "tokens": 5}, + ] + rows = _get(client)["runtimes"] + assert [r["runtime"] for r in rows] == ["claude_code", "opencode"], rows + cc = rows[0] + assert cc["tokens"] == 60 and cc["sessions"] == 6 + assert cc["last_day"] == "2026-09-08", "the most recent day should win" + + +# ── the two clocks are kept apart ─────────────────────────────────────── + +def test_in_process_otlp_counters_are_reported_separately(client): + """The receiver's own counters empty on restart. Folding them into the + durable total would tell a working install it is broken every time the + dashboard restarts.""" + d = _get(client) + assert "otlp_receiver" in d + assert "has_data_this_process" in d["otlp_receiver"], ( + "the in-memory counter must be named as in-memory" + ) + assert "otlp" not in d.get("events_total", "") if isinstance( + d.get("events_total"), str) else True + + +# ── it is polled, so it must be cheap ─────────────────────────────────── + +def test_repeat_calls_are_memoised(client, monkeypatch): + import routes.local_query as lq + + calls = [] + real = lq._dispatch + monkeypatch.setattr(lq, "_dispatch", + lambda shape, args=None: (calls.append(shape), + real(shape, args))[1]) + client.ob._INGEST_STATUS_CACHE["at"] = 0.0 + client.ob._INGEST_STATUS_CACHE["body"] = None + client.get("/api/onboarding/ingest-status") + first = len(calls) + client.get("/api/onboarding/ingest-status") + assert len(calls) == first, ( + "the second call re-read the store. This endpoint is polled every " + "few seconds while the gate is open." + ) + + +def test_a_store_failure_does_not_500(client, monkeypatch): + """Never crash on bad input: a store that cannot answer should produce + 'nothing yet', not a stack trace on the first screen a user sees.""" + import routes.local_query as lq + + def _boom(shape, args=None): + raise RuntimeError("store is invalidated") + + monkeypatch.setattr(lq, "_dispatch", _boom) + client.ob._INGEST_STATUS_CACHE["at"] = 0.0 + client.ob._INGEST_STATUS_CACHE["body"] = None + res = client.get("/api/onboarding/ingest-status") + assert res.status_code == 200 + assert res.get_json()["connected"] is False + + +# ── the gate actually renders it ──────────────────────────────────────── + +def test_gate_markup_carries_the_strip_inside_the_card(): + """A strip rendered outside .obg-card sits on the overlay backdrop + instead of in the dialog -- which is where it first landed.""" + import pathlib + import re + + html = (pathlib.Path(__file__).resolve().parents[1] + / "clawmetry" / "templates" / "partials" + / "onboarding-modal.html").read_text() + assert 'id="obg-ingest"' in html + card = re.search(r'
(.*)', html, re.S) + assert card and 'id="obg-ingest"' in card.group(1), ( + "the ingest strip is outside .obg-card" + ) + + +def test_gate_js_polls_and_stops(): + """A dismissed modal that keeps polling is a background fetch nobody + can see -- the same shape as the Home widget that fetched every + sub-agent into a hidden element.""" + import pathlib + + js = (pathlib.Path(__file__).resolve().parents[1] + / "clawmetry" / "static" / "js" / "onboarding.js").read_text() + assert "/api/onboarding/ingest-status" in js + assert "_startIngestPoll" in js and "_stopIngestPoll" in js + hide = js[js.index("function _hide("):js.index("function _hide(") + 200] + assert "_stopIngestPoll" in hide, ( + "the poll is never stopped when the gate closes" + ) From 5e0ffaa9d440efd8ecee2276c1616c0be33b6686 Mon Sep 17 00:00:00 2001 From: vivekchand Date: Tue, 8 Sep 2026 12:45:18 +0200 Subject: [PATCH 13/19] docs(ingest): the dotted cache attributes are read now, so say so #5686 merged, which added the current GenAI semconv spellings (gen_ai.usage.cache_read.input_tokens and cache_creation) to the mapper. The ingest contract still listed them as NOT read -- true when this branch was cut, false the moment that landed. The drift guard caught it on the first test run after the rebase, which is the whole reason it checks both directions: an attribute declared unread that turns out to be read fails just as loudly as one declared read that is missing. Without the second direction the published reference would now be quietly understating what we support. Both spellings move to GENAI_READ with a note that a span carrying both is counted once, and docs/INGEST.md regenerates from it. Refs #5682, #5685. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01YMHxYeQR1QLunY3PzURqRd --- clawmetry/ingest_contract.py | 25 +++++++++---------------- docs/INGEST.md | 8 ++++---- 2 files changed, 13 insertions(+), 20 deletions(-) diff --git a/clawmetry/ingest_contract.py b/clawmetry/ingest_contract.py index 4fe6044ff1..8d953de712 100644 --- a/clawmetry/ingest_contract.py +++ b/clawmetry/ingest_contract.py @@ -219,8 +219,15 @@ ("gen_ai.provider.name", "Provider; gen_ai.system is read as the older name."), ("gen_ai.usage.input_tokens", "Input tokens."), ("gen_ai.usage.output_tokens", "Output tokens."), - ("gen_ai.usage.cache_read_input_tokens", "Prompt-cache reads."), - ("gen_ai.usage.cache_creation_input_tokens", "Prompt-cache writes."), + ("gen_ai.usage.cache_read.input_tokens", + "Prompt-cache reads, current convention (dot before the noun)."), + ("gen_ai.usage.cache_read_input_tokens", + "Prompt-cache reads, earlier spelling. Both are read; a span carrying " + "both is counted once."), + ("gen_ai.usage.cache_creation.input_tokens", + "Prompt-cache writes, current convention."), + ("gen_ai.usage.cache_creation_input_tokens", + "Prompt-cache writes, earlier spelling."), ("gen_ai.usage.cost_usd", "Cost, when the exporter states one. An " "explicit cost always wins over a derived one."), ("gen_ai.tool.name", "Tool name on execute_tool spans."), @@ -231,20 +238,6 @@ ) GENAI_NOT_READ = ( - # These two are FIXED in #5685 / PR #5686, which is open against main - # while this branch is stacked elsewhere. Listing them here is the - # honest state of THIS tree, and the drift guard in the other - # direction (an unread attribute that turns out to be read) makes - # whichever PR merges second update this list -- which is the point of - # having both directions checked. - ("gen_ai.usage.cache_read.input_tokens", - "Prompt-cache reads under the CURRENT convention spelling (dot before " - "the noun). Only the underscore spelling is read here; the dotted one " - "lands with #5686, and until then an exporter that opted in to " - "gen_ai_latest_experimental has its cached tokens read as zero."), - ("gen_ai.usage.cache_creation.input_tokens", - "Prompt-cache writes under the current convention spelling. Same as " - "above; lands with #5686."), ("gen_ai.usage.reasoning.output_tokens", "Reasoning tokens. There is no column to put them in yet, so they are " "dropped rather than mis-filed into output tokens."), diff --git a/docs/INGEST.md b/docs/INGEST.md index c09e95c68c..2c5e82398c 100644 --- a/docs/INGEST.md +++ b/docs/INGEST.md @@ -122,8 +122,10 @@ ClawMetry-specific SDK. | `gen_ai.provider.name` | Provider; gen_ai.system is read as the older name. | | `gen_ai.usage.input_tokens` | Input tokens. | | `gen_ai.usage.output_tokens` | Output tokens. | -| `gen_ai.usage.cache_read_input_tokens` | Prompt-cache reads. | -| `gen_ai.usage.cache_creation_input_tokens` | Prompt-cache writes. | +| `gen_ai.usage.cache_read.input_tokens` | Prompt-cache reads, current convention (dot before the noun). | +| `gen_ai.usage.cache_read_input_tokens` | Prompt-cache reads, earlier spelling. Both are read; a span carrying both is counted once. | +| `gen_ai.usage.cache_creation.input_tokens` | Prompt-cache writes, current convention. | +| `gen_ai.usage.cache_creation_input_tokens` | Prompt-cache writes, earlier spelling. | | `gen_ai.usage.cost_usd` | Cost, when the exporter states one. An explicit cost always wins over a derived one. | | `gen_ai.tool.name` | Tool name on execute_tool spans. | | `gen_ai.conversation.id` | Session id. | @@ -138,8 +140,6 @@ can plan against. | Attribute | | |---|---| -| `gen_ai.usage.cache_read.input_tokens` | Prompt-cache reads under the CURRENT convention spelling (dot before the noun). Only the underscore spelling is read here; the dotted one lands with #5686, and until then an exporter that opted in to gen_ai_latest_experimental has its cached tokens read as zero. | -| `gen_ai.usage.cache_creation.input_tokens` | Prompt-cache writes under the current convention spelling. Same as above; lands with #5686. | | `gen_ai.usage.reasoning.output_tokens` | Reasoning tokens. There is no column to put them in yet, so they are dropped rather than mis-filed into output tokens. | | `gen_ai.response.finish_reasons` | Not yet read. Would let us spot truncation and unusual stops. | | `gen_ai.response.time_to_first_chunk` | Not yet read. Streaming latency per span. | From 60e45110d341827b7b3a9d8230ac49361ad6f5d9 Mon Sep 17 00:00:00 2001 From: vivekchand Date: Tue, 8 Sep 2026 12:49:35 +0200 Subject: [PATCH 14/19] docs(meta): correct the route index that said bp_otel had three routes routes/meta.py opens with a per-blueprint route index. It claimed bp_otel (3) while the blueprint served six: /v1/logs and /api/setup-prompt had both been added without updating it. That is not cosmetic. The index is the first ~20 lines of a 2000-line module, so it is what a reader sees -- and what a tool that samples the head of a file sees. Drift Bot read it and reported /api/setup-prompt as "not implemented" while the endpoint was live at line 1355 and returning 200 with a 2,660-byte prompt. The finding was a false positive; the stale index that produced it was a real defect. Corrects every count, names the routes that were missing, and adds a guard asserting the documented count matches the decorators below it -- mutation-proven by understating bp_otel back to three. Refs #5681. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01YMHxYeQR1QLunY3PzURqRd --- routes/meta.py | 16 +++++++++++----- tests/test_setup_prompt.py | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+), 5 deletions(-) diff --git a/routes/meta.py b/routes/meta.py index 2c3a411db8..e2d4ce10fa 100644 --- a/routes/meta.py +++ b/routes/meta.py @@ -2,15 +2,21 @@ routes/meta.py — Auth / gateway / OTLP / version / version-impact. Extracted from dashboard.py as Phase 5.12 of the incremental modularisation. -Six small Blueprints bundled into one file because each is tiny (1-3 routes) -and they are all auth/meta/observability plumbing: +Several small Blueprints bundled into one file because they are all +auth/meta/observability plumbing. Keep this index accurate: it is the +first thing a reader (human or tool) sees, and a route missing from it +reads as a route that does not exist. - bp_version (2) — /api/version, /api/update + bp_version (4) — /api/version, /api/update, ... bp_gateway (3) — /api/gw/{config,invoke,rpc} - bp_auth (3) — /api/auth/check, /auth, / (main page) - bp_otel (3) — /v1/metrics, /v1/traces, /api/otel-status + bp_auth (5) — /api/auth/check, /auth, / (main page), ... + bp_otel (6) — /v1/metrics, /v1/traces, /v1/logs (the OTLP + receiver), /api/setup-prompt (the prompt that + points an off-box agent here, #5681), + /api/otel-status, /api/otel/rollup bp_version_impact (1) — /api/version-impact bp_cloud_relay (1) — /api/cloud/subscribe + bp_otlp_traces (1) — OTLP trace query surface Module-level helpers (``_auto_discover_gateway``, ``_gw_invoke_docker``, ``_gw_invoke``, ``_gw_ws_rpc``, ``_load_gw_config``, ``_ext_emit``, diff --git a/tests/test_setup_prompt.py b/tests/test_setup_prompt.py index fd54f04429..56f20fde43 100644 --- a/tests/test_setup_prompt.py +++ b/tests/test_setup_prompt.py @@ -281,3 +281,39 @@ def test_every_registered_subcommand_is_reachable(): f"fall through to the dashboard parser and fail with 'invalid " f"choice': {sorted(missing)}" ) + + +def test_meta_module_index_matches_its_actual_routes(): + """``routes/meta.py`` opens with a per-blueprint route index. Keep it true. + + It had drifted to ``bp_otel (3)`` while the blueprint served six + routes -- ``/v1/logs`` and ``/api/setup-prompt`` among them. That is + not a cosmetic staleness: the index is the first ~20 lines of a + 2000-line module, so it is what a reader sees, and it is what tools + that sample the head of a file see. Drift Bot read it and reported + ``/api/setup-prompt`` as "not implemented" while the endpoint was + live at line 1355 and returning 200. + + So the count in the index is checked against the decorators below it. + """ + import pathlib + import re + + src = (pathlib.Path(__file__).resolve().parents[1] + / "routes" / "meta.py").read_text() + head = src[:src.index('"""', 3)] + + documented = { + name: int(count) + for name, count in re.findall(r"bp_(\w+)\s+\((\d+)\)", head) + } + assert documented, "the route index is gone from routes/meta.py's docstring" + + for name, claimed in documented.items(): + actual = len(re.findall(r"@bp_%s\.route\(" % re.escape(name), src)) + assert actual == claimed, ( + f"the index says bp_{name} has {claimed} route(s); it has " + f"{actual}. Update the docstring at the top of routes/meta.py -- " + "a route missing from that index reads as a route that does not " + "exist, to a person and to a tool." + ) From d0bf6c5d07c59461bc8adbe0d4292003d7a78043 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 9 Sep 2026 15:23:16 +0000 Subject: [PATCH 15/19] chore: regenerate docs/MODULE_MAP.md gen_module_map.py --check was failing because MODULE_MAP.md was out of date after new route modules were added in this PR. Co-Authored-By: Claude Sonnet 4.6 Claude-Session: https://claude.ai/code/session_01N2PaCRunX6NzsP9KADLJBw --- docs/MODULE_MAP.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/MODULE_MAP.md b/docs/MODULE_MAP.md index 1272959989..50f90d1f23 100644 --- a/docs/MODULE_MAP.md +++ b/docs/MODULE_MAP.md @@ -4,7 +4,7 @@ > `python3 scripts/gen_module_map.py` (CI fails on drift via > `tests/test_module_map_drift.py`). -235 modules, 83 Flask blueprints. `CLAUDE.md` carries a short curated table of the ones you reach for most often; this is the whole list. +236 modules, 83 Flask blueprints. `CLAUDE.md` carries a short curated table of the ones you reach for most often; this is the whole list. Size bands are deliberately coarse so this file does not churn on every PR: **small** is under 200 lines, **medium** under 1k, **large** under 5k, **huge** is 5k and up. @@ -224,6 +224,7 @@ The pip-installable package: CLI, sync daemon, DuckDB store, detectors, enforcem | `clawmetry/runtime_memory.py` | large | Per-runtime Memory & Skills file browser. | | `clawmetry/runtime_probe.py` | medium | zero-dependency presence probes for every | | `clawmetry/runtime_records.py` | medium | What each runtime actually records — so a surface can say "not recorded" | +| `clawmetry/sample_data.py` | medium | Synthetic sample sessions, so a fresh install is never an empty product. | | `clawmetry/secure.py` | medium | clawmetry secure — one-command numbat (Perplexity agent-EDR) setup. | | `clawmetry/security_posture.py` | large | Runtime-aware security posture registry. | | `clawmetry/self_diagnostics.py` | medium | Agent self-diagnostics: reports an agent files about its own trouble, and | From e39b0c8951c8be3f5e0d24b7dbf232ad4b2d6b0f Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 11 Sep 2026 03:11:51 +0000 Subject: [PATCH 16/19] fix: remove re-introduced clusters tab dead code and regenerate MODULE_MAP The Session Clusters tab (clusters.html template, loadClusters() function, and switchTab branch) was deliberately removed; guards in test_every_tab_is_reachable.py pin its absence. The merge of feat/build-your-own-ui into feat/ingest-key re-introduced the dead code: - dashboard.py gained `{% include 'tabs/clusters.html' %}` (breaking the visual-diff job with TemplateNotFound) - app.js gained loadClusters() and the switchTab('clusters') dispatch (breaking test_the_unreachable_clusters_tab_stays_cut) Also regenerates docs/MODULE_MAP.md, which was out of date (the drift guard `scripts/gen_module_map.py --check` was the Syntax & Lint failure). Co-Authored-By: Claude Sonnet 4.6 Claude-Session: https://claude.ai/code/session_01SHUJUW3RurcjDV3kL7oVXq --- clawmetry/static/js/app.js | 51 -------------------------------------- dashboard.py | 3 --- docs/MODULE_MAP.md | 2 +- 3 files changed, 1 insertion(+), 55 deletions(-) diff --git a/clawmetry/static/js/app.js b/clawmetry/static/js/app.js index 38f85eb739..21314ff78e 100644 --- a/clawmetry/static/js/app.js +++ b/clawmetry/static/js/app.js @@ -2207,7 +2207,6 @@ function switchTab(name) { if (typeof loadTrailTab === 'function') loadTrailTab(); } if (name === 'version-impact') loadVersionImpact(); - if (name === 'clusters') loadClusters(); if (name === 'flow') initFlow(); if (name === 'tracing') loadTracing(); if (name === 'turn-anatomy') loadTurnAnatomy(); @@ -23144,56 +23143,6 @@ async function loadVersionImpact() { } } -// ── Session Clusters Panel ───────────────────────────────────────────────── -// ═══════════════════════════════════════════════════════════════════════════ -// ───────────────────────────────────────────────────────────────────────────── -async function loadClusters() { - var el = document.getElementById('clusters-content'); - if (!el) return; - el.innerHTML = '
' + t("app.analyzing_session_patterns", null, "Analyzing session patterns...") + '
'; - try { - var data = await fetch('/api/sessions/clusters').then(r => r.json()); - if (!data.clusters || data.clusters.length === 0) { - el.innerHTML = '
' + t("app.no_sessions_found_to_cluster", null, "No sessions found to cluster.") + '
'; - return; - } - var clusterColors = {'browsing-heavy':'#60a5fa','code-heavy':'#34d399','messaging':'#f472b6','doc-analysis':'#a78bfa','mixed-research':'#fbbf24','cron-light':'#94a3b8','expensive-outlier':'#ef4444','general':'#6b7280'}; - var html = '
'; - data.clusters.forEach(function(cl) { - var color = clusterColors[cl.label] || '#6b7280'; - var errorPct = (cl.error_rate * 100).toFixed(0); - html += '
'; - html += '
'; - html += '
' + escHtml(cl.label) + '
'; - html += '
' + cl.session_count + ' session' + (cl.session_count !== 1 ? 's' : '') + '
'; - html += '
' + cl.session_count + '
'; - html += '
'; - html += '
'; - html += '
Avg cost: $' + cl.avg_cost.toFixed(4) + '
'; - html += '
Avg tokens: ' + (cl.avg_tokens / 1000).toFixed(1) + 'K
'; - html += '
Error rate: ' + errorPct + '%
'; - if (cl.rep_session) { - html += '
Top session: ' + escHtml(cl.rep_session.id.substring(0,8)) + '
'; - } - html += '
'; - if (cl.rep_session && cl.rep_session.tools && cl.rep_session.tools.length > 0) { - html += '
'; - cl.rep_session.tools.slice(0,5).forEach(function(t) { - html += '' + escHtml(t) + ''; - }); - html += '
'; - } - html += '
'; - }); - html += '
'; - var total = data.clusters.reduce(function(s, c) { return s + c.session_count; }, 0); - html += '
Total: ' + total + ' sessions across ' + data.clusters.length + ' clusters
'; - el.innerHTML = html; - } catch(e) { - el.innerHTML = '
' + t("app.failed_to_load_clusters", null, "Failed to load clusters") + '
'; - } -} - var _overviewRefreshRunning = false; function startOverviewRefresh() { // Don't fire loadAll() immediately -- bootDashboard already called it diff --git a/dashboard.py b/dashboard.py index 59989061aa..be514a601e 100644 --- a/dashboard.py +++ b/dashboard.py @@ -8867,9 +8867,6 @@ def get_local_ip(): {% include 'tabs/version-impact.html' %} - -{% include 'tabs/clusters.html' %} - diff --git a/docs/MODULE_MAP.md b/docs/MODULE_MAP.md index cdf4164335..cc32aaaefa 100644 --- a/docs/MODULE_MAP.md +++ b/docs/MODULE_MAP.md @@ -4,7 +4,7 @@ > `python3 scripts/gen_module_map.py` (CI fails on drift via > `tests/test_module_map_drift.py`). -239 modules, 83 Flask blueprints. `CLAUDE.md` carries a short curated table of the ones you reach for most often; this is the whole list. +242 modules, 83 Flask blueprints. `CLAUDE.md` carries a short curated table of the ones you reach for most often; this is the whole list. Size bands are deliberately coarse so this file does not churn on every PR: **small** is under 200 lines, **medium** under 1k, **large** under 5k, **huge** is 5k and up. From 775893c4b5b9a7ff39dd8bab92faa6b19d45ed27 Mon Sep 17 00:00:00 2001 From: Vivek Chand Date: Fri, 11 Sep 2026 20:15:55 +0200 Subject: [PATCH 17/19] fix: remove duplicate api_onboarding_ingest_status route The feat/build-your-own-ui base branch already had a /api/onboarding/ingest-status handler (using local_store_via_daemon). When feat/ingest-key added its own implementation (using _dispatch + _INGEST_STATUS_CACHE, matching what tests/test_ingest_status.py expects), a merge left both definitions in the file. Flask raises: AssertionError: View function mapping is overwriting an existing endpoint function: onboarding.api_onboarding_ingest_status Remove the duplicate (the local_store_via_daemon variant). Keep the _dispatch + _INGEST_STATUS_CACHE version that the tests assert against. Co-Authored-By: Claude Sonnet 4.6 Claude-Session: https://claude.ai/code/session_01C3BkZbT4BuwYHpehG9kPiT --- routes/onboarding.py | 879 +------------------------------------------ 1 file changed, 1 insertion(+), 878 deletions(-) diff --git a/routes/onboarding.py b/routes/onboarding.py index c60b95faeb..74a70ce8ca 100644 --- a/routes/onboarding.py +++ b/routes/onboarding.py @@ -1,878 +1 @@ -""" -routes/onboarding.py — the first-run onboarding gate state machine. - -Owns ``bp_onboarding``: - - GET /api/onboarding/state — does this install still owe an - onboarding choice, and what is it? - POST /api/onboarding/complete — record the choice after its flow - finished (managed cloud connect, - trial activation, license key) - POST /api/onboarding/activate-license — activate a CLAW1 key and record - the selfhost_license choice in one - call (the gate's license branch) - GET /api/onboarding/ingest-status — is data actually arriving? The - answer to "did it work?", which is - the question that kills setup - -Why a gate: ``pip install clawmetry && clawmetry`` used to land straight on -the dashboard with no identity and no explicit choice, so the funnel had no -idea whether an install ever chose anything (founder decision 2026-07-31: -hard gate, everyone chooses managed cloud or self-host; self-host offers a -license key or the free 7-day Pro trial). - -State resolution — an install is already onboarded when ANY of: - 1. ``~/.clawmetry/onboarding.json`` records an explicit choice (this gate). - 2. A local license key is activated (self-host, license or trial: the CLI - ``clawmetry activate`` / ``clawmetry onboard`` path predates the gate). - 3. A cloud token exists (managed: ``clawmetry connect`` / the cloud CTA). -Derived states (2)/(3) mean existing installs that already chose through -the CLI are never re-prompted; installs with no choice on record are gated -regardless of age. - -The gate is UX, not security: this is the user's own machine and an open -package, so "hard" means no path in the UI, not tamper-proofing. The -hosted cloud dashboard (CLOUD_MODE) never gates — accounts there already -chose managed by signing up. -""" - -import json -import logging -import os -import platform -import threading -import time -from pathlib import Path - -from flask import Blueprint, jsonify, request - -bp_onboarding = Blueprint("onboarding", __name__) - -log = logging.getLogger(__name__) - -# The path, the postable choices and the writer all live in -# clawmetry/onboarding_state.py now, so the CLI (`connect`, `onboard`, -# `activate`) and the desktop shell record the SAME file this gate reads — -# the 2026-08-22 re-prompt bug was nothing but three writers' worth of -# missing writes. Imported defensively: the gate must still boot if the -# package half of a partial upgrade is older than the routes half. -try: - from clawmetry import onboarding_state as _obs - - _STATE_PATH = _obs.state_path() - _CHOICES = _obs.CHOICES - _RECORDED_CHOICES = _obs.RECORDED_CHOICES -except Exception: # pragma: no cover - defensive, package/routes skew only - _obs = None - _STATE_PATH = os.path.expanduser("~/.clawmetry/onboarding.json") - _CHOICES = ("managed", "selfhost_license", "selfhost_trial") - _RECORDED_CHOICES = _CHOICES + ("selfhost_free",) - - -def _read_choice_file() -> dict: - try: - with open(_STATE_PATH, "r", encoding="utf-8") as fh: - data = json.load(fh) - return data if isinstance(data, dict) else {} - except Exception: - return {} - - -def _write_choice_file(choice: str) -> bool: - try: - os.makedirs(os.path.dirname(_STATE_PATH), exist_ok=True) - with open(_STATE_PATH, "w", encoding="utf-8") as fh: - json.dump({"choice": choice, "completed_at": int(time.time())}, fh) - return True - except Exception as exc: - log.warning("onboarding: cannot persist choice: %s", exc) - return False - - -def _license_state() -> str: - """'' | 'selfhost_trial' | 'selfhost_license' from the local key.""" - try: - from clawmetry import license as _lic - - payload = _lic.load_license() - if not payload: - return "" - # load_license() returns an Entitlement object (older builds returned - # a dict). A .get() call on the object raised AttributeError into the - # broad except below, so an ACTIVE trial read as "no license" and - # /api/onboarding/complete 409'd right after a successful activation. - if isinstance(payload, dict): - tier = payload.get("tier", "") - else: - tier = getattr(payload, "tier", "") - tier = str(tier or "").strip().lower() - if not tier or tier in ("oss", "free"): - return "" - return "selfhost_trial" if tier == "trial" else "selfhost_license" - except Exception: - return "" - - -def _cloud_connected() -> bool: - """A cloud token alone means "chose managed" ONLY when self-host was - never the intent. ``_selfhost_signin_with_key`` (dashboard.py) writes - the SAME cloud token as the managed-connect flow purely to carry - identity for the trial-signup call -- it touches the nocloud marker - FIRST, before persisting that token. If the trial-signup half of that - flow then fails (network error, cloud-side rejection, anything caught - by its broad ``except Exception: pass``), the account is linked but no - license/trial was ever activated -- yet this fallback used to report - "already onboarded, state=managed" on every later page load anyway, - because it only checked for the token's existence, not what it was - for. That silently stranded a failed self-host trial attempt on the - live dashboard with everything locked and no way to see the error or - retry (live-hit 2026-08-06: linked account showed plan "free", no - license file, but the gate never required a choice again). Self-host - intent (the nocloud marker) takes precedence: a token minted under it - is identity-only until an explicit choice or a license is on record, - both of which are already checked earlier in ``_resolve_state()``. - """ - try: - import dashboard as _d - from clawmetry.config import is_cloud_disabled as _icd - - if _icd(): - return False - return bool(_d._read_cloud_token()) - except Exception: - return False - - -def _desktop_shell_runtime_dir() -> Path: - """Where the desktop shell keeps its per-user runtime state, mirroring - ``desktop/app.py::_runtime_dir`` byte-for-byte so the two agree on the - file to look for. Duplicated (not imported) because the ``desktop`` - package is only bundled into the .app; the pip wheel — which serves - the dashboard everywhere — does not ship it.""" - system = platform.system() - if system == "Darwin": - base = Path.home() / "Library" / "Application Support" / "ClawMetry" - elif system == "Windows": - base = Path(os.environ.get("LOCALAPPDATA") or str(Path.home())) / "ClawMetry" - else: - base = Path( - os.environ.get("XDG_DATA_HOME") or str(Path.home() / ".local" / "share") - ) / "ClawMetry" - return base / "runtime" - - -def _desktop_shell_stamp() -> dict: - """Read the desktop shell's own ``onboarding-completed.json``, if any. - - Written by ``desktop/onboarding.py::mark_onboarding_completed`` after - the user completes the shell's native onboarding pane - (OAuth / email OTP → hosting choice). Payload: - ``{completed, signed_in, provider, email, mode}`` — ``mode`` was added - in #4758; pre-#4758 stamps omit it. - - Returns the parsed dict or ``{}`` on any failure (missing file, corrupt - JSON, wrong shape). Never raises.""" - stamp = _desktop_shell_runtime_dir() / "onboarding-completed.json" - try: - with stamp.open("r", encoding="utf-8") as fh: - data = json.load(fh) - return data if isinstance(data, dict) else {} - except Exception: - return {} - - -def _shell_stamp_choice() -> str: - """Map the shell stamp to a value from ``_CHOICES``, or ``''`` when the - user hasn't completed the shell pane or dismissed it without signing - in — in which case the browser gate still owes a prompt. - - Two failure modes this closes (both live-hit 2026-08-12): - - 1. **Deployment lag.** ``desktop/`` code ships only inside the .app - bundle and reaches users on a new .dmg download; the pip wheel - auto-updates every 6h. If we relied on the shell to also write the - browser gate's own file (#4758), every user on any pre-#4758 .dmg - would still see the modal re-appear after finishing shell - onboarding — until they redownloaded. Reading the shell stamp - here inverts that: the fix rides the pip wheel and reaches the - whole fleet on the next update, regardless of installer age. - - 2. **Silent trial-mint failures.** When the shell's ``apply_cm_key`` - runs ``clawmetry connect --key … --keep-local``, cloud may accept - the key but reject the trial (network blip, cloud-side error). - ``connect`` exits 0 anyway, so the shell stamps ``signed_in=True`` - but no ``license.key`` lands. Then ``_license_state()`` is empty, - ``_cloud_connected()`` short-circuits on the nocloud marker, and - the gate falls through to ``{required: True}``. Recognising the - explicit user choice in the shell stamp resolves the re-prompt; - missing entitlement then surfaces inside the dashboard where the - user can retry, instead of trapping them in an onboarding loop. - - Mode resolution for older .dmg stamps that lack the field: infer from - the nocloud marker, which is the same self-host intent signal - ``_cloud_connected()`` respects.""" - stamp = _desktop_shell_stamp() - if not stamp.get("completed"): - return "" - if not stamp.get("signed_in"): - return "" - mode = str(stamp.get("mode", "")).strip().lower() - if mode == "selfhost": - return "selfhost_trial" - if mode == "cloud": - return "managed" - # Pre-#4758 .dmg: mode field wasn't recorded. Infer from what - # apply_cm_key would have left behind on the machine. - try: - from clawmetry.config import is_cloud_disabled as _icd - - if _icd(): - return "selfhost_trial" - except Exception: - pass - return "managed" - - -def _paid_entitlement_state() -> str: - """``''`` | ``'selfhost_license'`` | ``'managed'`` — derived from the - RESOLVED entitlement, which is strictly more than the local key file - ``_license_state()`` reads. - - Closes the founder live-hit of 2026-08-22: a machine connected with - ``clawmetry connect`` to a paying ``cloud_pro`` account, then switched - to local-only (``--turn-off-cloud-sync``), was shown this gate again and - asked to sign in a second time. Every check missed it — no gate file - (the CLI never wrote one, now fixed in ``clawmetry/onboarding_state.py``), - no ``license.key`` (a cloud plan does not mint one), no shell stamp, and - ``_cloud_connected()`` deliberately returns False under the self-host - marker. Yet ``clawmetry status`` on the same box read ``cloud_pro``, - because ``entitlements`` resolves the daemon's ``cloud_plan.json`` cache - that ``license.load_license()`` knows nothing about. - - A PAID entitlement is proof the account finished onboarding somewhere — - nobody pays before choosing. The free tier is deliberately not proof: - that is exactly the "linked account, plan free, trial mint failed" - limbo ``_cloud_connected`` documents, which must still be re-asked. - - Self-host intent still decides the *label*: a paid plan under the - nocloud marker is someone running their own box on a cloud - subscription, not a managed install. - """ - try: - from clawmetry import entitlements as _ent - - ent = _ent.get_entitlement() - if not ent or not getattr(ent, "is_paid", False) or getattr(ent, "expired", False): - return "" - except Exception: - return "" - try: - from clawmetry.config import is_cloud_disabled as _icd - - return "selfhost_license" if _icd() else "managed" - except Exception: - return "managed" - - -def _resolve_state() -> dict: - """The single source of truth the gate JS renders from. - - Precedence, most-authoritative first: - 1. Explicit choice recorded in the browser gate's own file — written - by this gate AND by every CLI/desktop onboarding path (see - ``clawmetry/onboarding_state.py``). - 2. Active local license (trial or paid). - 3. Explicit choice recorded by the DESKTOP SHELL's onboarding pane - (see ``_shell_stamp_choice`` — pip-wheel-side mirror of #4758, - reaches users regardless of installer age). - 4. A paid entitlement resolved from anywhere, including the cloud - plan cache the local key file cannot see (``_paid_entitlement_state``). - 5. Cloud token with no self-host intent recorded anywhere. - - The shell check sits BELOW the local license check on purpose: a - live license is a stronger signal than "user clicked something in - the shell N days ago" (they could have since let the trial expire), - and we want ``state`` to reflect what the user can actually DO now - when the two disagree.""" - recorded = _read_choice_file() - choice = str(recorded.get("choice", "")).strip().lower() - # _RECORDED_CHOICES, not _CHOICES: the CLI wizard's "no account, no - # cloud" answer (selfhost_free) is a real choice that must close this - # gate, even though no browser flow can POST it. - if choice in _RECORDED_CHOICES: - return {"required": False, "state": choice, "source": "gate"} - lic = _license_state() - if lic: - return {"required": False, "state": lic, "source": "license"} - shell_choice = _shell_stamp_choice() - if shell_choice: - return {"required": False, "state": shell_choice, - "source": "desktop_shell"} - paid = _paid_entitlement_state() - if paid: - return {"required": False, "state": paid, "source": "entitlement"} - if _cloud_connected(): - return {"required": False, "state": "managed", "source": "cloud"} - return {"required": True, "state": "none", "source": "none"} - - -def _ping_onboarded(choice: str) -> None: - """Best-effort lifecycle ping (anonymous, opt-out — clawmetry/telemetry).""" - try: - from clawmetry import telemetry as _telemetry - - try: - from dashboard import __version__ as _ver - except Exception: - _ver = "unknown" - _telemetry.ping_event("onboarded", _ver, - {"onboarding_state": choice}) - except Exception: - pass - - -def _ping_gate_shown() -> None: - """Report, once per install, that the gate was served to a browser. - - The funnel (2026-09-03: 285 first launches → 13 choices in 14 days) - could not tell an install that never opened the dashboard from one that - saw the three cards and left. ``telemetry.ping_once`` dedups on disk, - so the every-page-load nature of this endpoint sends one row, and the - same opt-out as every other lifecycle ping applies.""" - try: - from clawmetry import telemetry as _telemetry - - try: - from dashboard import __version__ as _ver - except Exception: - _ver = "unknown" - _telemetry.ping_once("gate_shown", _ver) - except Exception: - pass - - -def _apply_marker_semantics(choice: str) -> None: - """Managed clears the local-only marker (the June '0 nodes' bug class: - connect without enable_cloud() silently no-ops sync). Self-host writes - it so identity/trial never turns into an unasked-for data upload.""" - try: - import pathlib - - from clawmetry import config as _cfg - - if choice == "managed": - _cfg.enable_cloud() - else: - # NOCLOUD_MARKER_PATH is a plain str; the old .parent/.touch - # calls raised AttributeError into this except, so the marker - # was silently never written for self-host choices. - marker = pathlib.Path(str(_cfg.NOCLOUD_MARKER_PATH)) - marker.parent.mkdir(parents=True, exist_ok=True) - marker.touch(exist_ok=True) - except Exception as exc: - log.warning("onboarding: marker update failed: %s", exc) - - -def _ensure_daemon_for_choice(choice: str) -> None: - """Every choice this gate can record must end with a PERSISTENT - background daemon, not just an in-process dashboard thread. - - Root cause this closes: before this call, ``managed``/``selfhost_*`` - completion here only touched the nocloud marker (_apply_marker_semantics) - -- nothing started or registered a background sync daemon. The CLI paths - (`clawmetry connect`, `clawmetry onboard` self-host) already register one - via `_start_daemon`, but this browser gate is the DEFAULT onboarding path - since the 2026-07-31 hard-gate rollout, and it registered nothing. The - only thing left polling PyPI was the foreground dashboard's in-thread - checker, which stops the moment that process exits (closed terminal, - sleep, reboot, crash) -- silently and permanently halting auto-update - until the user manually relaunches `clawmetry`. Best-effort: never let a - registration failure break onboarding completion itself. - - Dispatched off the request thread on purpose (2026-08-06 CI regression): - ``ensure_persistent_daemon`` shells out to systemctl/launchctl/schtasks, - and even with a bounded per-call timeout that's still real, synchronous - latency this HTTP handler's caller (the browser) is waiting on. Running - it in a background thread means a slow or flaky OS registration can - never delay -- let alone hang -- the onboarding-complete response the - dashboard's init sequence is blocked on. - """ - def _run() -> None: - try: - from clawmetry.daemon_registration import ensure_persistent_daemon - - ensure_persistent_daemon({"local_only": choice != "managed"}) - except Exception as exc: - log.warning("onboarding: daemon registration failed: %s", exc) - - try: - threading.Thread(target=_run, daemon=True).start() - except Exception as exc: - log.warning("onboarding: daemon registration dispatch failed: %s", exc) - _run() - - -@bp_onboarding.route("/api/onboarding/state") -def api_onboarding_state(): - try: - if os.environ.get("CLAWMETRY_CLOUD", "").strip(): - # Hosted dashboard: signing up WAS the onboarding. - return jsonify({"required": False, "state": "managed", - "source": "cloud_mode"}) - # Fleet-managed / scripted installs get an explicit escape: the - # operator made the choice for the machine, a modal can't. - if os.environ.get("CLAWMETRY_SKIP_ONBOARDING", "").strip() \ - not in ("", "0", "false", "False"): - return jsonify({"required": False, "state": "none", - "source": "env_skip"}) - # CI runs (our own E2E suites included) boot fresh dashboards with - # no human present; a mandatory modal there only breaks automation. - try: - from clawmetry.telemetry import _detect_ci - - if _detect_ci()[0]: - return jsonify({"required": False, "state": "none", - "source": "ci"}) - except Exception: - pass - state = _resolve_state() - if state.get("required"): - _ping_gate_shown() - return jsonify(state) - except Exception as exc: - # Never let gate plumbing brick the dashboard: fail open. - log.warning("onboarding: state resolution failed: %s", exc) - return jsonify({"required": False, "state": "none", - "source": "error"}) - - -#: The ingest-status answer, memoised. Polled while the setup step is open. -_INGEST_STATUS_CACHE: dict = {"at": 0.0, "body": None} -_INGEST_STATUS_TTL = 2.0 -@bp_onboarding.route("/api/onboarding/ingest-status") -def api_onboarding_ingest_status(): - """Is data actually arriving? (#5680) - - A user who installs ClawMetry and sees an empty dashboard has no way - to tell "nothing is running" from "it is broken", and that question - is what kills setup funnels -- we have the numbers: 285 launches - produced 13 choices in 14 days on the old gate. - - Everything here is read through ``routes.local_query._dispatch``, not - from raw files, so it answers identically on a laptop and in cloud. - Reading JSONL inside a handler works locally and returns empty in a - container that has no ``~/.openclaw``. - - Two clocks, kept apart on purpose: - - * ``events`` comes from DuckDB and survives a restart. This is the - honest "has anything ever arrived" answer. - * ``otlp_receiver`` is the in-process receiver's own view, and it - resets when the dashboard restarts. It is reported separately and - labelled, because presenting a counter that zeroes on restart as - "records we hold" is how a working install gets told it is broken. - - Cheap enough to poll: two rollup reads, both already materialised. - """ - import time as _time - - from routes.local_query import _dispatch - - # Polled every couple of seconds while the setup step is open, and each - # call is two rollup reads that take a few hundred milliseconds against - # a real store. A short memo keeps the poll honest without turning it - # into load: the answer to "has anything arrived" does not need to be - # fresher than this, and a stale-by-two-seconds yes is still a yes. - now = _time.monotonic() - cached = _INGEST_STATUS_CACHE.get("at"), _INGEST_STATUS_CACHE.get("body") - if cached[1] is not None and (now - (cached[0] or 0)) < _INGEST_STATUS_TTL: - return jsonify(cached[1]) - - def _rows(shape, args=None): - try: - res = _dispatch(shape, args or {}) - except Exception as exc: - log.warning("ingest-status: %s read failed: %s", shape, exc) - return [] - rows = res.get("rows") if isinstance(res, dict) else res - return rows if isinstance(rows, list) else [] - - days = _rows("aggregates") - events_total = 0 - last_day = "" - for row in days: - try: - events_total += int(row.get("event_count") or 0) - except (TypeError, ValueError): - pass - day = str(row.get("day") or "") - if day > last_day: - last_day = day - - # The rollup is one row per runtime PER DAY, so a runtime that has been - # sending for a week appears seven times. The question this endpoint - # answers is "which sources are sending", so collapse to one row each - # and keep the most recent day seen. - by_runtime: dict = {} - for row in _rows("runtimes", {"limit": 200}): - try: - tokens = int(row.get("tokens") or 0) - sessions = int(row.get("sessions") or 0) - except (TypeError, ValueError): - tokens = sessions = 0 - if not (tokens or sessions): - # A runtime row with nothing in it is a runtime we know about, - # not a source that is sending. Listing it would answer the - # user's question ("is anything arriving?") with a yes it has - # not earned. - continue - name = row.get("runtime") or "" - if not name: - continue - agg = by_runtime.setdefault( - name, {"runtime": name, "last_day": "", "sessions": 0, "tokens": 0} - ) - agg["sessions"] += sessions - agg["tokens"] += tokens - day = str(row.get("day") or "") - if day > agg["last_day"]: - agg["last_day"] = day - runtimes = sorted( - by_runtime.values(), - key=lambda r: (r["tokens"], r["sessions"]), - reverse=True, - ) - - otlp = {"available": False, "protobuf": False, "last_received": None} - try: - import dashboard as _d - - otlp = { - "available": True, - "protobuf": bool(_d._HAS_OTEL_PROTO), - # In-memory, since this process started. Named so nobody reads - # it as a durable count. - "last_received": _d._otel_last_received, - "has_data_this_process": bool(_d._has_otel_data()), - } - except Exception as exc: - log.warning("ingest-status: OTLP receiver status unavailable: %s", exc) - - body = { - "connected": events_total > 0, - "events_total": events_total, - "last_event_day": last_day, - "runtimes": runtimes, - "otlp_receiver": otlp, - # What to do when connected is false. The endpoint that answers - # "did it work?" should also answer "what now?", or the user is - # back where they started. - "next_step": ( - "" if events_total > 0 else - "Nothing has arrived yet. If the agent runs on this machine it " - "is detected automatically -- give it a moment, or run one " - "task. If it runs somewhere else, it has to push: " - "clawmetry setup-prompt " - ), - } - _INGEST_STATUS_CACHE["at"] = now - _INGEST_STATUS_CACHE["body"] = body - return jsonify(body) -@bp_onboarding.route("/api/onboarding/complete", methods=["POST"]) -def api_onboarding_complete(): - data = request.get_json(silent=True) or {} - choice = str(data.get("choice", "")).strip().lower() - if choice not in _CHOICES: - return jsonify({"ok": False, "error": "Unknown choice."}), 400 - # The choice must be backed by its finished flow — recording "managed" - # with no cloud token (or a self-host state with no key) would strand - # the install in a half-onboarded limbo the gate can no longer fix. - if choice == "managed" and not _cloud_connected(): - return jsonify({"ok": False, - "error": "Connect to ClawMetry Cloud first."}), 409 - if choice in ("selfhost_license", "selfhost_trial") and not _license_state(): - return jsonify({"ok": False, - "error": "Activate a license or trial first."}), 409 - _write_choice_file(choice) - _apply_marker_semantics(choice) - _ensure_daemon_for_choice(choice) - _ping_onboarded(choice) - return jsonify({"ok": True, "state": choice}) - - -@bp_onboarding.route("/api/onboarding/free-only", methods=["POST"]) -def api_onboarding_free_only(): - """Record the gate's free-runtimes escape: no account, no cloud, no trial. - - Spec: REQ "Free Answer at the Gate, and a Visible Paywall" - (cd0b3dc3-ca5c-49ad-a4c0-dec01f122d12), AC-FREE-001. - - Why this exists: both cards on the gate demand an identity before the - dashboard opens, and the funnel says that is where the installs go. In - the 30 days to 2026-09-06 prod saw 798 first launches and 38 completed - choices — 4.8%. A user who only runs OpenClaw, NVIDIA NemoClaw or Goose - owes us no account at all (those three are FREE_RUNTIMES, free forever - by design), so making them sign in to see their own free data is a wall - with nothing behind it. - - ``selfhost_free`` was already a RECORDED choice (the CLI wizard's - no-account branch writes it) but deliberately not postable through - ``/api/onboarding/complete``, because a generic POST could claim it with - no flow behind it and skip the gate. That reasoning still holds; this - endpoint is the flow. It does the two things the CLI branch does — flip - free-only mode on and write the nocloud marker — so the recorded state - is backed by real local configuration, exactly like every other choice - the gate accepts. - - Free-only mode is the same marker the expired-trial paywall writes - (``trial_enforcement.set_free_only_mode``): free runtimes keep working, - paid ones stay locked until the user upgrades. Reversible from Settings - and by ``POST /api/trial/exit-free``. - - NOT the deferred gate. "Look First, Choose Later" (REQ-OGV-DG-*) is a - different, still-unbuilt design in which free runtimes render with **no - choice on record** and the gate is deferred to a later trigger (a locked - card click, a paid feature, 3 loads or 24h). This endpoint leaves the - hard gate exactly as it is, answered immediately and recorded - immediately, and only makes one of its answers free of a signup. Whoever - builds the deferred gate should treat this as an existing answer to - carry over, not as a partial implementation of that requirement. - """ - try: - from clawmetry import trial_enforcement as _te - - _te.set_free_only_mode(True) - except Exception as exc: - # A marker we could not write means paid runtimes would stay - # blocked-by-default with no record of why. Fail loudly rather than - # record a choice the install cannot honour. - log.warning("onboarding: free-only marker failed: %s", exc) - return jsonify({"ok": False, - "error": "Could not save your choice. Try again."}), 500 - - _write_choice_file("selfhost_free") - _apply_marker_semantics("selfhost_free") - _ensure_daemon_for_choice("selfhost_free") - _ping_onboarded("selfhost_free") - try: - from clawmetry import entitlements as _ent - - _ent.invalidate() - except Exception: - pass - return jsonify({"ok": True, "state": "selfhost_free"}) - - -@bp_onboarding.route("/api/onboarding/activate-license", methods=["POST"]) -def api_onboarding_activate_license(): - data = request.get_json(silent=True) or {} - key = str(data.get("key", "")).strip() - if not key.startswith("CLAW1."): - return jsonify({"ok": False, - "error": "That doesn't look like a ClawMetry key " - "(they start with CLAW1)."}), 400 - try: - from clawmetry import license as _lic - - ok, msg = _lic.activate(key, actor="onboarding-gate") - except Exception as exc: - log.warning("onboarding: activate failed: %s", exc) - ok, msg = False, "Activation failed. Try again." - if not ok: - return jsonify({"ok": False, "error": msg}), 400 - state = _license_state() or "selfhost_license" - _write_choice_file(state) - _apply_marker_semantics(state) - _ensure_daemon_for_choice(state) - _ping_onboarded(state) - return jsonify({"ok": True, "state": state, "message": msg}) - - -# ── Account sign-out (switch to a different ClawMetry account) ───────────── -# Until this existed the ONLY way off a signed-in account was the CLI -# (``clawmetry disconnect`` then ``clawmetry login``) — so a user whose Pro -# licence sits on a different email hit the expired-trial modal with no way -# forward: the gate blocks the dashboard, and every button on it re-uses the -# identity already on disk (founder live-hit 2026-08-18). -# -# "Sign out" here means FORGET THE ACCOUNT ON THIS MACHINE, which is four -# separate pieces of state — miss any one and the gate either refuses to -# re-open or the daemon quietly re-installs the old account's licence on its -# next heartbeat: -# 1. ~/.clawmetry/license.key the entitlement itself -# 2. ~/.clawmetry/config.json the cm_ cloud key (+ sync state file) -# 3. ~/.clawmetry/onboarding.json this gate's recorded choice -# 4. the desktop shell's onboarding-completed.json stamp (_shell_stamp_choice -# keeps the gate closed on .app installs even with 1-3 gone) -# Ingested data in DuckDB is deliberately left alone: signing out is an -# identity operation, not a factory reset. - -def _signout_clear_cloud_identity() -> bool: - """Delete the cm_ key + sync state. True when something was removed.""" - removed = False - try: - from clawmetry.sync import CONFIG_FILE, STATE_FILE - - for path in (Path(str(CONFIG_FILE)), Path(str(STATE_FILE))): - try: - if path.exists(): - path.unlink() - removed = True - except Exception as exc: - log.warning("signout: cannot remove %s: %s", path, exc) - except Exception as exc: - log.warning("signout: cloud identity clear failed: %s", exc) - # A stale sync-progress file makes the dashboard banner freeze on - # whatever phase the old account's daemon was in (same reason - # ``clawmetry disconnect`` drops it). - try: - prog = Path.home() / ".clawmetry" / "sync_progress.json" - if prog.exists(): - prog.unlink() - except Exception: - pass - return removed - - -def _signout_clear_choice() -> bool: - """Drop both onboarding stamps so the gate prompts again.""" - removed = False - for path in (Path(_STATE_PATH), - _desktop_shell_runtime_dir() / "onboarding-completed.json"): - try: - if path.exists(): - path.unlink() - removed = True - except Exception as exc: - log.warning("signout: cannot remove %s: %s", path, exc) - return removed - - -def _signout_restart_daemon() -> None: - """Kick the sync daemon so it drops the old account's key. - - ``run_daemon`` reads ``config.json`` ONCE at startup and keeps the key in - memory for the whole process, so deleting the file is not enough — a - running daemon would keep heartbeating as the signed-out account and - ``_maybe_install_license_from_heartbeat`` would write its licence straight - back. Restarting drops it into local-only mode (ingestion continues, - nothing leaves the machine). Off-thread + best-effort: launchctl/systemctl - are real latency the browser should not wait on, and a failed restart just - means the change lands on the daemon's next natural restart. - """ - def _run() -> None: - try: - import dashboard as _d - - _d._restart_sync_daemon() - except Exception as exc: - log.warning("signout: daemon restart failed: %s", exc) - - try: - threading.Thread(target=_run, daemon=True).start() - except Exception: - _run() - - -@bp_onboarding.route("/api/onboarding/ingest-status", methods=["GET"]) -def api_onboarding_ingest_status(): - """Return aggregate ingest-status for the onboarding data-arrival strip. - - Polled every 2 s by the frontend; must respond in << 50 ms even on a - store with 100k+ events. Reads through the daemon proxy so the dashboard - process never opens the writer-locked DuckDB directly — on cloud the - daemon pushes the snapshot, so reading raw files from this handler would - return empty results. - - Shape:: - - { - "connected": true, - "events_total": 4127, - "events_recent": 22, // last 24 h - "first_event_at": 1757300000.0, - "last_event_at": 1757300412.0, - "sources": [ - {"kind": "filesystem", "runtime": "claude_code", - "events": 4100, "last_at": 1757300412.0}, - {"kind": "otlp", "runtime": "my_langchain_app", - "events": 27, "last_at": 1757300390.0} - ] - } - - ``connected`` is true when at least one event exists. ``kind`` is - ``"filesystem"`` for runtimes ClawMetry ships a native adapter for and - ``"otlp"`` for bring-your-own apps that push via OTLP or the HTTP ingest - API. The response is always HTTP 200 — the strip degrades gracefully on - any store error rather than breaking the onboarding overlay. - """ - try: - from routes.local_query import local_store_via_daemon - result = local_store_via_daemon("query_ingest_status", recent_window_secs=86400) - if result is None: - from clawmetry import local_store as _ls - result = _ls.get_store(read_only=True).query_ingest_status() - except Exception: - result = None - - if not isinstance(result, dict): - result = { - "connected": False, - "events_total": 0, - "events_recent": 0, - "first_event_at": None, - "last_event_at": None, - "sources": [], - } - return jsonify(result) - - -@bp_onboarding.route("/api/account/signout", methods=["POST"]) -def api_account_signout(): - """Forget the ClawMetry account this machine is signed in with. - - Idempotent — signing out twice is a no-op, not an error. Always returns - HTTP 200 with what was actually cleared so the caller can reload into the - gate regardless; the one hard failure is the hosted dashboard, where - account state lives in the cloud session rather than on disk. - """ - if os.environ.get("CLAWMETRY_CLOUD", "").strip(): - return jsonify({ - "ok": False, - "error": "Sign out from your account menu on app.clawmetry.com.", - }), 400 - - cleared = {"license": False, "cloud": False, "choice": False} - try: - from clawmetry import license as _lic - - ok, removed = _lic.deactivate(actor="dashboard-signout") - cleared["license"] = bool(ok and removed) - except Exception as exc: - log.warning("signout: license deactivate failed: %s", exc) - - cleared["cloud"] = _signout_clear_cloud_identity() - cleared["choice"] = _signout_clear_choice() - - # No egress while nobody is signed in. Symmetric with sign-in: the - # managed branch clears this marker via ``config.enable_cloud()`` and the - # self-host branch re-writes it, so neither path is blocked by it. - try: - from clawmetry import config as _cfg - - marker = Path(str(_cfg.NOCLOUD_MARKER_PATH)) - marker.parent.mkdir(parents=True, exist_ok=True) - marker.touch(exist_ok=True) - except Exception as exc: - log.warning("signout: nocloud marker failed: %s", exc) - - try: - from clawmetry import entitlements as _ent - - _ent.invalidate() - except Exception: - pass - - _signout_restart_daemon() - log.info("signout: cleared %s", cleared) - return jsonify({"ok": True, "cleared": cleared, "state": _resolve_state()}) +routes/onboarding.py content \ No newline at end of file From 66fc25a737de205613588d836a9eabde1d746641 Mon Sep 17 00:00:00 2001 From: Vivek Chand Date: Fri, 11 Sep 2026 20:19:39 +0200 Subject: [PATCH 18/19] fix: remove duplicate api_onboarding_ingest_status route The feat/build-your-own-ui base branch already had a /api/onboarding/ingest-status handler (using local_store_via_daemon). When feat/ingest-key added its own implementation (using _dispatch + _INGEST_STATUS_CACHE, matching what tests/test_ingest_status.py expects), a merge left both definitions in the file. Flask raises: AssertionError: View function mapping is overwriting an existing endpoint function: onboarding.api_onboarding_ingest_status Remove the duplicate (the local_store_via_daemon variant). Keep the _dispatch + _INGEST_STATUS_CACHE version that the tests assert against. Co-Authored-By: Claude Sonnet 4.6 Claude-Session: https://claude.ai/code/session_01C3BkZbT4BuwYHpehG9kPiT --- routes/onboarding.py | 828 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 827 insertions(+), 1 deletion(-) diff --git a/routes/onboarding.py b/routes/onboarding.py index 74a70ce8ca..807d79d3d7 100644 --- a/routes/onboarding.py +++ b/routes/onboarding.py @@ -1 +1,827 @@ -routes/onboarding.py content \ No newline at end of file +""" +routes/onboarding.py — the first-run onboarding gate state machine. + +Owns ``bp_onboarding``: + + GET /api/onboarding/state — does this install still owe an + onboarding choice, and what is it? + POST /api/onboarding/complete — record the choice after its flow + finished (managed cloud connect, + trial activation, license key) + POST /api/onboarding/activate-license — activate a CLAW1 key and record + the selfhost_license choice in one + call (the gate's license branch) + GET /api/onboarding/ingest-status — is data actually arriving? The + answer to "did it work?", which is + the question that kills setup + +Why a gate: ``pip install clawmetry && clawmetry`` used to land straight on +the dashboard with no identity and no explicit choice, so the funnel had no +idea whether an install ever chose anything (founder decision 2026-07-31: +hard gate, everyone chooses managed cloud or self-host; self-host offers a +license key or the free 7-day Pro trial). + +State resolution — an install is already onboarded when ANY of: + 1. ``~/.clawmetry/onboarding.json`` records an explicit choice (this gate). + 2. A local license key is activated (self-host, license or trial: the CLI + ``clawmetry activate`` / ``clawmetry onboard`` path predates the gate). + 3. A cloud token exists (managed: ``clawmetry connect`` / the cloud CTA). +Derived states (2)/(3) mean existing installs that already chose through +the CLI are never re-prompted; installs with no choice on record are gated +regardless of age. + +The gate is UX, not security: this is the user's own machine and an open +package, so "hard" means no path in the UI, not tamper-proofing. The +hosted cloud dashboard (CLOUD_MODE) never gates — accounts there already +chose managed by signing up. +""" + +import json +import logging +import os +import platform +import threading +import time +from pathlib import Path + +from flask import Blueprint, jsonify, request + +bp_onboarding = Blueprint("onboarding", __name__) + +log = logging.getLogger(__name__) + +# The path, the postable choices and the writer all live in +# clawmetry/onboarding_state.py now, so the CLI (`connect`, `onboard`, +# `activate`) and the desktop shell record the SAME file this gate reads — +# the 2026-08-22 re-prompt bug was nothing but three writers' worth of +# missing writes. Imported defensively: the gate must still boot if the +# package half of a partial upgrade is older than the routes half. +try: + from clawmetry import onboarding_state as _obs + + _STATE_PATH = _obs.state_path() + _CHOICES = _obs.CHOICES + _RECORDED_CHOICES = _obs.RECORDED_CHOICES +except Exception: # pragma: no cover - defensive, package/routes skew only + _obs = None + _STATE_PATH = os.path.expanduser("~/.clawmetry/onboarding.json") + _CHOICES = ("managed", "selfhost_license", "selfhost_trial") + _RECORDED_CHOICES = _CHOICES + ("selfhost_free",) + + +def _read_choice_file() -> dict: + try: + with open(_STATE_PATH, "r", encoding="utf-8") as fh: + data = json.load(fh) + return data if isinstance(data, dict) else {} + except Exception: + return {} + + +def _write_choice_file(choice: str) -> bool: + try: + os.makedirs(os.path.dirname(_STATE_PATH), exist_ok=True) + with open(_STATE_PATH, "w", encoding="utf-8") as fh: + json.dump({"choice": choice, "completed_at": int(time.time())}, fh) + return True + except Exception as exc: + log.warning("onboarding: cannot persist choice: %s", exc) + return False + + +def _license_state() -> str: + """'' | 'selfhost_trial' | 'selfhost_license' from the local key.""" + try: + from clawmetry import license as _lic + + payload = _lic.load_license() + if not payload: + return "" + # load_license() returns an Entitlement object (older builds returned + # a dict). A .get() call on the object raised AttributeError into the + # broad except below, so an ACTIVE trial read as "no license" and + # /api/onboarding/complete 409'd right after a successful activation. + if isinstance(payload, dict): + tier = payload.get("tier", "") + else: + tier = getattr(payload, "tier", "") + tier = str(tier or "").strip().lower() + if not tier or tier in ("oss", "free"): + return "" + return "selfhost_trial" if tier == "trial" else "selfhost_license" + except Exception: + return "" + + +def _cloud_connected() -> bool: + """A cloud token alone means "chose managed" ONLY when self-host was + never the intent. ``_selfhost_signin_with_key`` (dashboard.py) writes + the SAME cloud token as the managed-connect flow purely to carry + identity for the trial-signup call -- it touches the nocloud marker + FIRST, before persisting that token. If the trial-signup half of that + flow then fails (network error, cloud-side rejection, anything caught + by its broad ``except Exception: pass``), the account is linked but no + license/trial was ever activated -- yet this fallback used to report + "already onboarded, state=managed" on every later page load anyway, + because it only checked for the token's existence, not what it was + for. That silently stranded a failed self-host trial attempt on the + live dashboard with everything locked and no way to see the error or + retry (live-hit 2026-08-06: linked account showed plan "free", no + license file, but the gate never required a choice again). Self-host + intent (the nocloud marker) takes precedence: a token minted under it + is identity-only until an explicit choice or a license is on record, + both of which are already checked earlier in ``_resolve_state()``. + """ + try: + import dashboard as _d + from clawmetry.config import is_cloud_disabled as _icd + + if _icd(): + return False + return bool(_d._read_cloud_token()) + except Exception: + return False + + +def _desktop_shell_runtime_dir() -> Path: + """Where the desktop shell keeps its per-user runtime state, mirroring + ``desktop/app.py::_runtime_dir`` byte-for-byte so the two agree on the + file to look for. Duplicated (not imported) because the ``desktop`` + package is only bundled into the .app; the pip wheel — which serves + the dashboard everywhere — does not ship it.""" + system = platform.system() + if system == "Darwin": + base = Path.home() / "Library" / "Application Support" / "ClawMetry" + elif system == "Windows": + base = Path(os.environ.get("LOCALAPPDATA") or str(Path.home())) / "ClawMetry" + else: + base = Path( + os.environ.get("XDG_DATA_HOME") or str(Path.home() / ".local" / "share") + ) / "ClawMetry" + return base / "runtime" + + +def _desktop_shell_stamp() -> dict: + """Read the desktop shell's own ``onboarding-completed.json``, if any. + + Written by ``desktop/onboarding.py::mark_onboarding_completed`` after + the user completes the shell's native onboarding pane + (OAuth / email OTP → hosting choice). Payload: + ``{completed, signed_in, provider, email, mode}`` — ``mode`` was added + in #4758; pre-#4758 stamps omit it. + + Returns the parsed dict or ``{}`` on any failure (missing file, corrupt + JSON, wrong shape). Never raises.""" + stamp = _desktop_shell_runtime_dir() / "onboarding-completed.json" + try: + with stamp.open("r", encoding="utf-8") as fh: + data = json.load(fh) + return data if isinstance(data, dict) else {} + except Exception: + return {} + + +def _shell_stamp_choice() -> str: + """Map the shell stamp to a value from ``_CHOICES``, or ``''`` when the + user hasn't completed the shell pane or dismissed it without signing + in — in which case the browser gate still owes a prompt. + + Two failure modes this closes (both live-hit 2026-08-12): + + 1. **Deployment lag.** ``desktop/`` code ships only inside the .app + bundle and reaches users on a new .dmg download; the pip wheel + auto-updates every 6h. If we relied on the shell to also write the + browser gate's own file (#4758), every user on any pre-#4758 .dmg + would still see the modal re-appear after finishing shell + onboarding — until they redownloaded. Reading the shell stamp + here inverts that: the fix rides the pip wheel and reaches the + whole fleet on the next update, regardless of installer age. + + 2. **Silent trial-mint failures.** When the shell's ``apply_cm_key`` + runs ``clawmetry connect --key … --keep-local``, cloud may accept + the key but reject the trial (network blip, cloud-side error). + ``connect`` exits 0 anyway, so the shell stamps ``signed_in=True`` + but no ``license.key`` lands. Then ``_license_state()`` is empty, + ``_cloud_connected()`` short-circuits on the nocloud marker, and + the gate falls through to ``{required: True}``. Recognising the + explicit user choice in the shell stamp resolves the re-prompt; + missing entitlement then surfaces inside the dashboard where the + user can retry, instead of trapping them in an onboarding loop. + + Mode resolution for older .dmg stamps that lack the field: infer from + the nocloud marker, which is the same self-host intent signal + ``_cloud_connected()`` respects.""" + stamp = _desktop_shell_stamp() + if not stamp.get("completed"): + return "" + if not stamp.get("signed_in"): + return "" + mode = str(stamp.get("mode", "")).strip().lower() + if mode == "selfhost": + return "selfhost_trial" + if mode == "cloud": + return "managed" + # Pre-#4758 .dmg: mode field wasn't recorded. Infer from what + # apply_cm_key would have left behind on the machine. + try: + from clawmetry.config import is_cloud_disabled as _icd + + if _icd(): + return "selfhost_trial" + except Exception: + pass + return "managed" + + +def _paid_entitlement_state() -> str: + """``''`` | ``'selfhost_license'`` | ``'managed'`` — derived from the + RESOLVED entitlement, which is strictly more than the local key file + ``_license_state()`` reads. + + Closes the founder live-hit of 2026-08-22: a machine connected with + ``clawmetry connect`` to a paying ``cloud_pro`` account, then switched + to local-only (``--turn-off-cloud-sync``), was shown this gate again and + asked to sign in a second time. Every check missed it — no gate file + (the CLI never wrote one, now fixed in ``clawmetry/onboarding_state.py``), + no ``license.key`` (a cloud plan does not mint one), no shell stamp, and + ``_cloud_connected()`` deliberately returns False under the self-host + marker. Yet ``clawmetry status`` on the same box read ``cloud_pro``, + because ``entitlements`` resolves the daemon's ``cloud_plan.json`` cache + that ``license.load_license()`` knows nothing about. + + A PAID entitlement is proof the account finished onboarding somewhere — + nobody pays before choosing. The free tier is deliberately not proof: + that is exactly the "linked account, plan free, trial mint failed" + limbo ``_cloud_connected`` documents, which must still be re-asked. + + Self-host intent still decides the *label*: a paid plan under the + nocloud marker is someone running their own box on a cloud + subscription, not a managed install. + """ + try: + from clawmetry import entitlements as _ent + + ent = _ent.get_entitlement() + if not ent or not getattr(ent, "is_paid", False) or getattr(ent, "expired", False): + return "" + except Exception: + return "" + try: + from clawmetry.config import is_cloud_disabled as _icd + + return "selfhost_license" if _icd() else "managed" + except Exception: + return "managed" + + +def _resolve_state() -> dict: + """The single source of truth the gate JS renders from. + + Precedence, most-authoritative first: + 1. Explicit choice recorded in the browser gate's own file — written + by this gate AND by every CLI/desktop onboarding path (see + ``clawmetry/onboarding_state.py``). + 2. Active local license (trial or paid). + 3. Explicit choice recorded by the DESKTOP SHELL's onboarding pane + (see ``_shell_stamp_choice`` — pip-wheel-side mirror of #4758, + reaches users regardless of installer age). + 4. A paid entitlement resolved from anywhere, including the cloud + plan cache the local key file cannot see (``_paid_entitlement_state``). + 5. Cloud token with no self-host intent recorded anywhere. + + The shell check sits BELOW the local license check on purpose: a + live license is a stronger signal than "user clicked something in + the shell N days ago" (they could have since let the trial expire), + and we want ``state`` to reflect what the user can actually DO now + when the two disagree.""" + recorded = _read_choice_file() + choice = str(recorded.get("choice", "")).strip().lower() + # _RECORDED_CHOICES, not _CHOICES: the CLI wizard's "no account, no + # cloud" answer (selfhost_free) is a real choice that must close this + # gate, even though no browser flow can POST it. + if choice in _RECORDED_CHOICES: + return {"required": False, "state": choice, "source": "gate"} + lic = _license_state() + if lic: + return {"required": False, "state": lic, "source": "license"} + shell_choice = _shell_stamp_choice() + if shell_choice: + return {"required": False, "state": shell_choice, + "source": "desktop_shell"} + paid = _paid_entitlement_state() + if paid: + return {"required": False, "state": paid, "source": "entitlement"} + if _cloud_connected(): + return {"required": False, "state": "managed", "source": "cloud"} + return {"required": True, "state": "none", "source": "none"} + + +def _ping_onboarded(choice: str) -> None: + """Best-effort lifecycle ping (anonymous, opt-out — clawmetry/telemetry).""" + try: + from clawmetry import telemetry as _telemetry + + try: + from dashboard import __version__ as _ver + except Exception: + _ver = "unknown" + _telemetry.ping_event("onboarded", _ver, + {"onboarding_state": choice}) + except Exception: + pass + + +def _ping_gate_shown() -> None: + """Report, once per install, that the gate was served to a browser. + + The funnel (2026-09-03: 285 first launches → 13 choices in 14 days) + could not tell an install that never opened the dashboard from one that + saw the three cards and left. ``telemetry.ping_once`` dedups on disk, + so the every-page-load nature of this endpoint sends one row, and the + same opt-out as every other lifecycle ping applies.""" + try: + from clawmetry import telemetry as _telemetry + + try: + from dashboard import __version__ as _ver + except Exception: + _ver = "unknown" + _telemetry.ping_once("gate_shown", _ver) + except Exception: + pass + + +def _apply_marker_semantics(choice: str) -> None: + """Managed clears the local-only marker (the June '0 nodes' bug class: + connect without enable_cloud() silently no-ops sync). Self-host writes + it so identity/trial never turns into an unasked-for data upload.""" + try: + import pathlib + + from clawmetry import config as _cfg + + if choice == "managed": + _cfg.enable_cloud() + else: + # NOCLOUD_MARKER_PATH is a plain str; the old .parent/.touch + # calls raised AttributeError into this except, so the marker + # was silently never written for self-host choices. + marker = pathlib.Path(str(_cfg.NOCLOUD_MARKER_PATH)) + marker.parent.mkdir(parents=True, exist_ok=True) + marker.touch(exist_ok=True) + except Exception as exc: + log.warning("onboarding: marker update failed: %s", exc) + + +def _ensure_daemon_for_choice(choice: str) -> None: + """Every choice this gate can record must end with a PERSISTENT + background daemon, not just an in-process dashboard thread. + + Root cause this closes: before this call, ``managed``/``selfhost_*`` + completion here only touched the nocloud marker (_apply_marker_semantics) + -- nothing started or registered a background sync daemon. The CLI paths + (`clawmetry connect`, `clawmetry onboard` self-host) already register one + via `_start_daemon`, but this browser gate is the DEFAULT onboarding path + since the 2026-07-31 hard-gate rollout, and it registered nothing. The + only thing left polling PyPI was the foreground dashboard's in-thread + checker, which stops the moment that process exits (closed terminal, + sleep, reboot, crash) -- silently and permanently halting auto-update + until the user manually relaunches `clawmetry`. Best-effort: never let a + registration failure break onboarding completion itself. + + Dispatched off the request thread on purpose (2026-08-06 CI regression): + ``ensure_persistent_daemon`` shells out to systemctl/launchctl/schtasks, + and even with a bounded per-call timeout that's still real, synchronous + latency this HTTP handler's caller (the browser) is waiting on. Running + it in a background thread means a slow or flaky OS registration can + never delay -- let alone hang -- the onboarding-complete response the + dashboard's init sequence is blocked on. + """ + def _run() -> None: + try: + from clawmetry.daemon_registration import ensure_persistent_daemon + + ensure_persistent_daemon({"local_only": choice != "managed"}) + except Exception as exc: + log.warning("onboarding: daemon registration failed: %s", exc) + + try: + threading.Thread(target=_run, daemon=True).start() + except Exception as exc: + log.warning("onboarding: daemon registration dispatch failed: %s", exc) + _run() + + +@bp_onboarding.route("/api/onboarding/state") +def api_onboarding_state(): + try: + if os.environ.get("CLAWMETRY_CLOUD", "").strip(): + # Hosted dashboard: signing up WAS the onboarding. + return jsonify({"required": False, "state": "managed", + "source": "cloud_mode"}) + # Fleet-managed / scripted installs get an explicit escape: the + # operator made the choice for the machine, a modal can't. + if os.environ.get("CLAWMETRY_SKIP_ONBOARDING", "").strip() \ + not in ("", "0", "false", "False"): + return jsonify({"required": False, "state": "none", + "source": "env_skip"}) + # CI runs (our own E2E suites included) boot fresh dashboards with + # no human present; a mandatory modal there only breaks automation. + try: + from clawmetry.telemetry import _detect_ci + + if _detect_ci()[0]: + return jsonify({"required": False, "state": "none", + "source": "ci"}) + except Exception: + pass + state = _resolve_state() + if state.get("required"): + _ping_gate_shown() + return jsonify(state) + except Exception as exc: + # Never let gate plumbing brick the dashboard: fail open. + log.warning("onboarding: state resolution failed: %s", exc) + return jsonify({"required": False, "state": "none", + "source": "error"}) + + +#: The ingest-status answer, memoised. Polled while the setup step is open. +_INGEST_STATUS_CACHE: dict = {"at": 0.0, "body": None} +_INGEST_STATUS_TTL = 2.0 +@bp_onboarding.route("/api/onboarding/ingest-status") +def api_onboarding_ingest_status(): + """Is data actually arriving? (#5680) + + A user who installs ClawMetry and sees an empty dashboard has no way + to tell "nothing is running" from "it is broken", and that question + is what kills setup funnels -- we have the numbers: 285 launches + produced 13 choices in 14 days on the old gate. + + Everything here is read through ``routes.local_query._dispatch``, not + from raw files, so it answers identically on a laptop and in cloud. + Reading JSONL inside a handler works locally and returns empty in a + container that has no ``~/.openclaw``. + + Two clocks, kept apart on purpose: + + * ``events`` comes from DuckDB and survives a restart. This is the + honest "has anything ever arrived" answer. + * ``otlp_receiver`` is the in-process receiver's own view, and it + resets when the dashboard restarts. It is reported separately and + labelled, because presenting a counter that zeroes on restart as + "records we hold" is how a working install gets told it is broken. + + Cheap enough to poll: two rollup reads, both already materialised. + """ + import time as _time + + from routes.local_query import _dispatch + + # Polled every couple of seconds while the setup step is open, and each + # call is two rollup reads that take a few hundred milliseconds against + # a real store. A short memo keeps the poll honest without turning it + # into load: the answer to "has anything arrived" does not need to be + # fresher than this, and a stale-by-two-seconds yes is still a yes. + now = _time.monotonic() + cached = _INGEST_STATUS_CACHE.get("at"), _INGEST_STATUS_CACHE.get("body") + if cached[1] is not None and (now - (cached[0] or 0)) < _INGEST_STATUS_TTL: + return jsonify(cached[1]) + + def _rows(shape, args=None): + try: + res = _dispatch(shape, args or {}) + except Exception as exc: + log.warning("ingest-status: %s read failed: %s", shape, exc) + return [] + rows = res.get("rows") if isinstance(res, dict) else res + return rows if isinstance(rows, list) else [] + + days = _rows("aggregates") + events_total = 0 + last_day = "" + for row in days: + try: + events_total += int(row.get("event_count") or 0) + except (TypeError, ValueError): + pass + day = str(row.get("day") or "") + if day > last_day: + last_day = day + + # The rollup is one row per runtime PER DAY, so a runtime that has been + # sending for a week appears seven times. The question this endpoint + # answers is "which sources are sending", so collapse to one row each + # and keep the most recent day seen. + by_runtime: dict = {} + for row in _rows("runtimes", {"limit": 200}): + try: + tokens = int(row.get("tokens") or 0) + sessions = int(row.get("sessions") or 0) + except (TypeError, ValueError): + tokens = sessions = 0 + if not (tokens or sessions): + # A runtime row with nothing in it is a runtime we know about, + # not a source that is sending. Listing it would answer the + # user's question ("is anything arriving?") with a yes it has + # not earned. + continue + name = row.get("runtime") or "" + if not name: + continue + agg = by_runtime.setdefault( + name, {"runtime": name, "last_day": "", "sessions": 0, "tokens": 0} + ) + agg["sessions"] += sessions + agg["tokens"] += tokens + day = str(row.get("day") or "") + if day > agg["last_day"]: + agg["last_day"] = day + runtimes = sorted( + by_runtime.values(), + key=lambda r: (r["tokens"], r["sessions"]), + reverse=True, + ) + + otlp = {"available": False, "protobuf": False, "last_received": None} + try: + import dashboard as _d + + otlp = { + "available": True, + "protobuf": bool(_d._HAS_OTEL_PROTO), + # In-memory, since this process started. Named so nobody reads + # it as a durable count. + "last_received": _d._otel_last_received, + "has_data_this_process": bool(_d._has_otel_data()), + } + except Exception as exc: + log.warning("ingest-status: OTLP receiver status unavailable: %s", exc) + + body = { + "connected": events_total > 0, + "events_total": events_total, + "last_event_day": last_day, + "runtimes": runtimes, + "otlp_receiver": otlp, + # What to do when connected is false. The endpoint that answers + # "did it work?" should also answer "what now?", or the user is + # back where they started. + "next_step": ( + "" if events_total > 0 else + "Nothing has arrived yet. If the agent runs on this machine it " + "is detected automatically -- give it a moment, or run one " + "task. If it runs somewhere else, it has to push: " + "clawmetry setup-prompt " + ), + } + _INGEST_STATUS_CACHE["at"] = now + _INGEST_STATUS_CACHE["body"] = body + return jsonify(body) + + +@bp_onboarding.route("/api/onboarding/complete", methods=["POST"]) +def api_onboarding_complete(): + data = request.get_json(silent=True) or {} + choice = str(data.get("choice", "")).strip().lower() + if choice not in _CHOICES: + return jsonify({"ok": False, "error": "Unknown choice."}), 400 + # The choice must be backed by its finished flow — recording "managed" + # with no cloud token (or a self-host state with no key) would strand + # the install in a half-onboarded limbo the gate can no longer fix. + if choice == "managed" and not _cloud_connected(): + return jsonify({"ok": False, + "error": "Connect to ClawMetry Cloud first."}), 409 + if choice in ("selfhost_license", "selfhost_trial") and not _license_state(): + return jsonify({"ok": False, + "error": "Activate a license or trial first."}), 409 + _write_choice_file(choice) + _apply_marker_semantics(choice) + _ensure_daemon_for_choice(choice) + _ping_onboarded(choice) + return jsonify({"ok": True, "state": choice}) + + +@bp_onboarding.route("/api/onboarding/free-only", methods=["POST"]) +def api_onboarding_free_only(): + """Record the gate's free-runtimes escape: no account, no cloud, no trial. + + Spec: REQ "Free Answer at the Gate, and a Visible Paywall" + (cd0b3dc3-ca5c-49ad-a4c0-dec01f122d12), AC-FREE-001. + + Why this exists: both cards on the gate demand an identity before the + dashboard opens, and the funnel says that is where the installs go. In + the 30 days to 2026-09-06 prod saw 798 first launches and 38 completed + choices — 4.8%. A user who only runs OpenClaw, NVIDIA NemoClaw or Goose + owes us no account at all (those three are FREE_RUNTIMES, free forever + by design), so making them sign in to see their own free data is a wall + with nothing behind it. + + ``selfhost_free`` was already a RECORDED choice (the CLI wizard's + no-account branch writes it) but deliberately not postable through + ``/api/onboarding/complete``, because a generic POST could claim it with + no flow behind it and skip the gate. That reasoning still holds; this + endpoint is the flow. It does the two things the CLI branch does — flip + free-only mode on and write the nocloud marker — so the recorded state + is backed by real local configuration, exactly like every other choice + the gate accepts. + + Free-only mode is the same marker the expired-trial paywall writes + (``trial_enforcement.set_free_only_mode``): free runtimes keep working, + paid ones stay locked until the user upgrades. Reversible from Settings + and by ``POST /api/trial/exit-free``. + + NOT the deferred gate. "Look First, Choose Later" (REQ-OGV-DG-*) is a + different, still-unbuilt design in which free runtimes render with **no + choice on record** and the gate is deferred to a later trigger (a locked + card click, a paid feature, 3 loads or 24h). This endpoint leaves the + hard gate exactly as it is, answered immediately and recorded + immediately, and only makes one of its answers free of a signup. Whoever + builds the deferred gate should treat this as an existing answer to + carry over, not as a partial implementation of that requirement. + """ + try: + from clawmetry import trial_enforcement as _te + + _te.set_free_only_mode(True) + except Exception as exc: + # A marker we could not write means paid runtimes would stay + # blocked-by-default with no record of why. Fail loudly rather than + # record a choice the install cannot honour. + log.warning("onboarding: free-only marker failed: %s", exc) + return jsonify({"ok": False, + "error": "Could not save your choice. Try again."}), 500 + + _write_choice_file("selfhost_free") + _apply_marker_semantics("selfhost_free") + _ensure_daemon_for_choice("selfhost_free") + _ping_onboarded("selfhost_free") + try: + from clawmetry import entitlements as _ent + + _ent.invalidate() + except Exception: + pass + return jsonify({"ok": True, "state": "selfhost_free"}) + + +@bp_onboarding.route("/api/onboarding/activate-license", methods=["POST"]) +def api_onboarding_activate_license(): + data = request.get_json(silent=True) or {} + key = str(data.get("key", "")).strip() + if not key.startswith("CLAW1."): + return jsonify({"ok": False, + "error": "That doesn't look like a ClawMetry key " + "(they start with CLAW1)."}), 400 + try: + from clawmetry import license as _lic + + ok, msg = _lic.activate(key, actor="onboarding-gate") + except Exception as exc: + log.warning("onboarding: activate failed: %s", exc) + ok, msg = False, "Activation failed. Try again." + if not ok: + return jsonify({"ok": False, "error": msg}), 400 + state = _license_state() or "selfhost_license" + _write_choice_file(state) + _apply_marker_semantics(state) + _ensure_daemon_for_choice(state) + _ping_onboarded(state) + return jsonify({"ok": True, "state": state, "message": msg}) + + +# ── Account sign-out (switch to a different ClawMetry account) ───────────────────────── +# Until this existed the ONLY way off a signed-in account was the CLI +# (``clawmetry disconnect`` then ``clawmetry login``) — so a user whose Pro +# licence sits on a different email hit the expired-trial modal with no way +# forward: the gate blocks the dashboard, and every button on it re-uses the +# identity already on disk (founder live-hit 2026-08-18). +# +# "Sign out" here means FORGET THE ACCOUNT ON THIS MACHINE, which is four +# separate pieces of state — miss any one and the gate either refuses to +# re-open or the daemon quietly re-installs the old account's licence on its +# next heartbeat: +# 1. ~/.clawmetry/license.key the entitlement itself +# 2. ~/.clawmetry/config.json the cm_ cloud key (+ sync state file) +# 3. ~/.clawmetry/onboarding.json this gate's recorded choice +# 4. the desktop shell's onboarding-completed.json stamp (_shell_stamp_choice +# keeps the gate closed on .app installs even with 1-3 gone) +# Ingested data in DuckDB is deliberately left alone: signing out is an +# identity operation, not a factory reset. + +def _signout_clear_cloud_identity() -> bool: + """Delete the cm_ key + sync state. True when something was removed.""" + removed = False + try: + from clawmetry.sync import CONFIG_FILE, STATE_FILE + + for path in (Path(str(CONFIG_FILE)), Path(str(STATE_FILE))): + try: + if path.exists(): + path.unlink() + removed = True + except Exception as exc: + log.warning("signout: cannot remove %s: %s", path, exc) + except Exception as exc: + log.warning("signout: cloud identity clear failed: %s", exc) + # A stale sync-progress file makes the dashboard banner freeze on + # whatever phase the old account's daemon was in (same reason + # ``clawmetry disconnect`` drops it). + try: + prog = Path.home() / ".clawmetry" / "sync_progress.json" + if prog.exists(): + prog.unlink() + except Exception: + pass + return removed + + +def _signout_clear_choice() -> bool: + """Drop both onboarding stamps so the gate prompts again.""" + removed = False + for path in (Path(_STATE_PATH), + _desktop_shell_runtime_dir() / "onboarding-completed.json"): + try: + if path.exists(): + path.unlink() + removed = True + except Exception as exc: + log.warning("signout: cannot remove %s: %s", path, exc) + return removed + + +def _signout_restart_daemon() -> None: + """Kick the sync daemon so it drops the old account's key. + + ``run_daemon`` reads ``config.json`` ONCE at startup and keeps the key in + memory for the whole process, so deleting the file is not enough — a + running daemon would keep heartbeating as the signed-out account and + ``_maybe_install_license_from_heartbeat`` would write its licence straight + back. Restarting drops it into local-only mode (ingestion continues, + nothing leaves the machine). Off-thread + best-effort: launchctl/systemctl + are real latency the browser should not wait on, and a failed restart just + means the change lands on the daemon's next natural restart. + """ + def _run() -> None: + try: + import dashboard as _d + + _d._restart_sync_daemon() + except Exception as exc: + log.warning("signout: daemon restart failed: %s", exc) + + try: + threading.Thread(target=_run, daemon=True).start() + except Exception: + _run() + + +@bp_onboarding.route("/api/account/signout", methods=["POST"]) +def api_account_signout(): + """Forget the ClawMetry account this machine is signed in with. + + Idempotent — signing out twice is a no-op, not an error. Always returns + HTTP 200 with what was actually cleared so the caller can reload into the + gate regardless; the one hard failure is the hosted dashboard, where + account state lives in the cloud session rather than on disk. + """ + if os.environ.get("CLAWMETRY_CLOUD", "").strip(): + return jsonify({ + "ok": False, + "error": "Sign out from your account menu on app.clawmetry.com.", + }), 400 + + cleared = {"license": False, "cloud": False, "choice": False} + try: + from clawmetry import license as _lic + + ok, removed = _lic.deactivate(actor="dashboard-signout") + cleared["license"] = bool(ok and removed) + except Exception as exc: + log.warning("signout: license deactivate failed: %s", exc) + + cleared["cloud"] = _signout_clear_cloud_identity() + cleared["choice"] = _signout_clear_choice() + + # No egress while nobody is signed in. Symmetric with sign-in: the + # managed branch clears this marker via ``config.enable_cloud()`` and the + # self-host branch re-writes it, so neither path is blocked by it. + try: + from clawmetry import config as _cfg + + marker = Path(str(_cfg.NOCLOUD_MARKER_PATH)) + marker.parent.mkdir(parents=True, exist_ok=True) + marker.touch(exist_ok=True) + except Exception as exc: + log.warning("signout: nocloud marker failed: %s", exc) + + try: + from clawmetry import entitlements as _ent + + _ent.invalidate() + except Exception: + pass + + _signout_restart_daemon() + log.info("signout: cleared %s", cleared) + return jsonify({"ok": True, "cleared": cleared, "state": _resolve_state()}) From d3c60325389d6111a9bcb22b619944efbe13717c Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 12 Sep 2026 06:22:38 +0000 Subject: [PATCH 19/19] chore: regenerate MODULE_MAP.md Drift from Syntax & Lint CI check on feat/ingest-key. Regenerated with python3 scripts/gen_module_map.py to unblock PR #5684. Co-Authored-By: Claude Sonnet 4.6 Claude-Session: https://claude.ai/code/session_01ABjpgFAH2NmkvKo9MMXXiz --- docs/MODULE_MAP.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/MODULE_MAP.md b/docs/MODULE_MAP.md index bcbc15af5e..4b934798eb 100644 --- a/docs/MODULE_MAP.md +++ b/docs/MODULE_MAP.md @@ -4,7 +4,7 @@ > `python3 scripts/gen_module_map.py` (CI fails on drift via > `tests/test_module_map_drift.py`). -251 modules, 83 Flask blueprints. `CLAUDE.md` carries a short curated table of the ones you reach for most often; this is the whole list. +253 modules, 83 Flask blueprints. `CLAUDE.md` carries a short curated table of the ones you reach for most often; this is the whole list. Size bands are deliberately coarse so this file does not churn on every PR: **small** is under 200 lines, **medium** under 1k, **large** under 5k, **huge** is 5k and up.