diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index f2b6a4a1c1..b6be3787cf 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -761,6 +761,7 @@ jobs: tests/test_outcome_trend.py \ tests/test_otel_export_sessions_shape.py \ tests/test_query_contract_drift.py \ + tests/test_public_api_keys.py \ tests/test_query_contract_goldens.py \ tests/test_local_store_concurrent_flush_1590.py \ tests/test_duckdb_invalidated_recovery.py \ diff --git a/CHANGELOG.md b/CHANGELOG.md index 1a43826224..d54df26900 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -341,6 +341,13 @@ - **It found pre-existing debt, recorded as a ratchet rather than fixed here:** `dashboard.py` carries 59 shadowed names (39 byte-identical dead pairs and **20 that differ**, meaning someone edited one copy and not the other), and `routes/entitlement.py` carries 1. `verification/shadowed_definitions.json` holds those counts; the test asserts a file never rises above its baseline and that each baseline equals reality, so no headroom is left for the next one and the debt can only shrink. - **Verified:** restoring the old name reproduces `_session_cwd: 2` in the AST walk and reds the guard; with the fix, 235 module files pass and 273 tests pass across the daemon-wiring, detector and workspace-kind suites. +### Added: build your own UI, a keyed and scoped read API over the q/1 contract (2026-09-08) +- **Why:** ClawMetry ships one dashboard, and a user who wants a different view of their own data has no supported way to get one. Every number the dashboard draws already comes from a declared, versioned read contract (`q/1`, 17 live methods, per-method arg schemas, additive-only, drift-tested). The only thing allowed to call it was the dashboard itself, because the only gate in front of it was the browser's same-origin rule: there was no API key anywhere, and no `Access-Control-Allow-Origin` header anywhere in the codebase. So the options were fork the dashboard, scrape the HTML, or do without. Meanwhile the generation tools got good enough that a purpose-built view is an afternoon of work *if* the data has an addressable API. Ours did not have a door. +- **What:** four pieces, all built on what already existed. (1) **Read scopes live on the contract**: every `q/1` method now declares one of `read:metrics`, `read:sessions`, `read:traces`, `read:content`, in `clawmetry/query_contract.py` next to its trust class, so a new method cannot drift out of the scope model and `docs/QUERY_CONTRACT.md` regenerates with a scope table. (2) **Scoped keys** (`clawmetry/apikeys.py`, `clawmetry key create|list|revoke|scopes`, and a panel on the Security tab): SHA-256 in `~/.clawmetry/api_keys.json` at 0600, each key carrying scopes and a browser-origin allowlist. (3) **`GET /api/q/1/`** (`routes/public_api.py`), dispatching through the *same* `routes.local_query._dispatch` the dashboard uses, so there is no second query surface to keep correct. (4) **`GET /api/q/1/llms.txt`**, the whole API written for a coding agent to read in one pass, generated from the contract and scoped to the presented key. Plus a starter (`examples/custom-ui/`, one file, no build step) and a walkthrough (`docs/BUILD_YOUR_OWN_UI.md`). +- **The security decision this turns on, stated plainly.** Adding CORS to a service on localhost is how local tools get robbed. Any page in any tab can already *send* a request to `127.0.0.1:8900`; the only reason that has been harmless is that the browser refuses to let the page *read* the reply. This feature removes that protection deliberately, one named origin at a time, so: **loopback earns nothing** (this is the one surface in ClawMetry where "the request came from this machine" is not authentication, and a key is always required); **there is no wildcard origin**, not as a flag and not as an env var, because every "just for local dev" wildcard ships eventually and fails silently and totally; **key management is not on the keyed surface** (minting, listing and revoking sit on the ordinary dashboard blueprint behind the existing same-origin write guard, so a read key can never issue itself a better one, and `_add_cors` is pinned to `/api/q/` as belt and braces); and **the API is GET only**, which removes cross-origin writes as a category rather than as a check. Nothing here can pause, stop or kill an agent: this adds no entry to the control plane. +- **`read:metrics` is exactly the `plaintext` trust class**, pinned by a test rather than by convention. That equality is what makes "a browser-resident key cannot return a prompt, a reply or a file path" a promise instead of a hope, and it is why the starter ships asking for that scope and nothing more. +- **Verified end to end in a real browser, not only in tests.** The starter served from `http://localhost:3000` rendered 30 days of real cost from a ClawMetry on a different port: **$750 across 10 runtimes and 10 models**, the numbers matching `/api/aggregates` on the same machine. The negative case was proven on that same page: a *valid* full-scope key bound to `https://my-ui.vercel.app` could not be used from `localhost:3000`, the browser refusing it with `Failed to fetch` before the response was readable. Plus **48 guards** in `tests/test_public_api_keys.py` (registered in `ci.yml`, since CI runs explicit file lists), with three deliberate mutations proving they go red: making `_add_cors` echo every origin fails 4, removing the scope check fails 2, and reclassifying `transcript` as `read:metrics` fails 4 including the trust-class invariant. Every drift guard green: query-contract, module map, runtime and channel counts, AC ratchet, daemon allowlist, py3.9 annotations, `lint-js`. + ### Fixed: a poisoned linked worktree scanned CLEAN (2026-09-08) - **Why:** `repo_scan` opened `/.git/config` and nothing else. In a **linked worktree** `.git` is a FILE holding `gitdir: /abs/path`, and the config git actually reads lives in the common directory that path points at, so the scanner found no config and returned no findings. That is worse than returning nothing: it is a confident all-clear over a repository git will happily execute `core.fsmonitor` from, on a layout agents and CI use routinely. CVE-2026-55607 is the vendor-confirmed version of the same git-directory confusion. Found by `scripts/redteam/audit.py` itself, which filed the gap automatically: the audit doing exactly the job it was wired into CI for one release earlier. - **What:** `_git_dirs()` resolves `(git_dir, common_dir)` through the `.git` file's `gitdir:` and the worktree's `commondir`. `_git_config_paths()` returns every config git reads, which is `/config` plus `/config.worktree` (honoured when `extensions.worktreeConfig` is set, and writable by whoever supplied the worktree). The finding reports the config it was actually read from, trimmed to its last three components, because an incident travels to a UI and to a device and must not carry the user's home directory. Ordinary checkouts are untouched: `_git_dirs` answers with `.git` for both values and the label stays `.git/config`. @@ -358,8 +365,7 @@ - **What:** three PASS and are now cases. `npm-postinstall-visible-harvest` (credential_access, critical), `mcp-tool-output-prompt-injection` (credential_access, critical), and `mcp-injection-without-exfil` (privilege_change). The two MCP cases are deliberately a pair: the injection itself is invisible to us, since it arrives in a tool RESULT and no detector reads a result as an instruction, so what the pair measures is whether the BEHAVIOUR an injection produces is caught, in both the exfiltration shape and a non-exfiltration one. `control-ordinary-dev-session` is the negative control that belongs beside them: install, read `.env.example`, run tests, push, and stay silent. - **The fourth is a MISS, and it is filed rather than shipped red.** `npm-postinstall-credential-harvest`: nothing fired at all. It is not a tuning problem. For the invisible half of the attack the agent chose exactly one action, `npm install`, and npm's child processes did the rest, so the event stream is byte-for-byte the stream of a developer installing a dependency. No tool-stream detector can separate them. The fix is the workspace surface, the same answer `repo_scan` gave for GitSpawn, and the gap carries the audit's own verdict on the private tracker. - **The README now records what is deliberately NOT in the corpus**, so the next person does not rediscover it: cross-session coordination (the Hugging Face evaluation swarm) is not merely undetected but **inexpressible**, because `run_all` takes one session's events and so does the corpus runner; and credential-scoping bypass has no observable in the stream at all. -- **Verified:** `scripts/redteam/audit.py` reports 13/13 pass, 0 gaps, 0 control failures; the corpus suite is 28 tests, up from 24. Both run on every PR through the gate added in 0.12.837. - +- **Verified:** `scripts/redteam/audit.py` reports 13/13 pass, 0 gaps, 0 control failures; the corpus suite is 28 tests, up from 24. Both run on every PR through the gate added in 0.12.837. b2bf019 (feat: build your own UI, a keyed, scoped read API over the q/1 contract) ### Added: the daemon scans a session's workspace, which is the surface the tool stream cannot see (2026-09-08) - **Why:** every behavioural detector reads what the agent *chose* to do. GitSpawn is the proof that this is a partial view of the machine: a poisoned `.git/config` makes **git** spawn the payload during the background `git status` a runtime fires on open, the agent calls no tool, and `detectors.run_all` sees a clean session. `clawmetry/repo_scan.py` closes that gap and has shipped for a while, detecting all three GitSpawn variants plus the CHAINDROP hook shapes, with `clawmetry scan-repo` exposing it to a human. Nothing in the product called it. A detector nobody runs protects nobody. - **What:** `sync._emit_detector_incidents` now scans each session's `cwd` and emits findings on the path detector incidents already take: a `loop_signals` row (`daemon_detect_repo_config_exec` / `daemon_detect_agent_config_tamper`), `incident_alerts` delivery, and the heartbeat fold. The scan is cached on the mtime and size of exactly the files it reads (`.git/config`, `.vscode/tasks.json`, the `_AGENT_HOOK_FILES` entries), and nothing else, so an unchanged repo is never re-read and a repo poisoned *after* it was first seen clean is re-scanned on the next tick. Cache bounded at 500 directories, oldest evicted. `CLAWMETRY_REPO_SCAN=0` turns it off. diff --git a/CLAUDE.md b/CLAUDE.md index 0261833b4d..2a146cb1b5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -45,6 +45,8 @@ All HTTP endpoints live here, organised by feature: 70 modules, 82 blueprints, l | `routes/channels.py` | `bp_channels` — 24 chat-channel adapters (Telegram, Signal, WhatsApp, Discord, Slack, IRC, iMessage, WebChat, …) | | `routes/components.py` | `bp_components` — Flow-panel detail endpoints (tool / runtime / machine / gateway / brain) | | `routes/local_query.py` | `bp_local_query` — `/api/local/*` DuckDB read API + the daemon-proxy `_dispatch` (shape→store bridge shared by HTTP and the cloud relay) | +| `routes/public_api.py` | `bp_public_api` — `/api/q/1/*`, the **keyed, cross-origin** read API custom UIs are built on (`docs/BUILD_YOUR_OWN_UI.md`). Same `_dispatch`, but it is the one surface that does not trust loopback: every request needs a scoped `cmk_` key, and CORS is echoed only for an origin that key named | +| `routes/apikeys_admin.py` | `bp_apikeys_admin` — `/api/apikeys`, minting and revoking the keys `public_api` accepts. Kept apart from that surface on purpose: it sits behind the dashboard's own cross-origin write guard and never carries a CORS header, so a page holding a read key can neither list this node's keys nor issue itself a wider one | | `routes/guard.py` | `bp_guard` — live session control (Pause/Stop/Kill), Guard policy CRUD, policy decision log, learned baselines. Sessions ranked by **spend at risk**, not severity | | `routes/policy.py` | `bp_policy` — the *pre-tool* sandbox/permission surface (`/api/tool-policy`). Deliberately a different axis from `routes/guard.py`: different table, no shared state | | `routes/hooks.py` | `bp_hooks` — hook install / status / uninstall per runtime, and the gate's decision log | @@ -66,11 +68,12 @@ All HTTP endpoints live here, organised by feature: 70 modules, 82 blueprints, l | File | Purpose | |------|---------| -| `clawmetry/cli.py` | CLI entry point — `clawmetry`, `connect`, `sync`, `status`, `license`, `hook`, `update` | +| `clawmetry/cli.py` | CLI entry point — `clawmetry`, `connect`, `sync`, `status`, `license`, `hook`, `key`, `update` | | `clawmetry/sync.py` | Cloud sync daemon — ingests into DuckDB, owns the writer lock, runs the detectors and Guard policies, streams the E2E-encrypted (AES-256-GCM) snapshot to `ingest.clawmetry.com`. Holds `_FAMILY_ADAPTER_SPECS` (the adapters that actually load) and `_CHANNEL_DIRS` | | `clawmetry/local_store.py` | **DuckDB store** — the single data layer features read and write (the daemon holds the writer lock). Schema v15 | | `clawmetry/local_server.py` | Daemon-hosted localhost query server (`/local/query`, discovered through `~/.clawmetry/local_query.json`) so the dashboard reads DuckDB without grabbing the writer lock | | `clawmetry/query_contract.py` | The declared node query surface (`q/1`), rendered to `docs/QUERY_CONTRACT.md`. Additive-only inside a version | +| `clawmetry/apikeys.py` | Scoped read keys (`cmk_…`) for custom UIs: mint, verify, revoke. SHA-256 in `~/.clawmetry/api_keys.json` (0600); the scope a key carries maps to `query_contract`'s per-method `scope` | | `clawmetry/entitlements.py` | Single source of truth for tiers, `FREE_RUNTIMES` / `PAID_RUNTIMES`, `ALL_CHANNELS` and every capacity limit. GRACE by default | | `clawmetry/license.py` | Offline Ed25519 verification of self-hosted license keys | | `clawmetry/proxy.py` | Enforcement proxy — budget limits, loop detection, model routing (port 4100) | @@ -116,6 +119,7 @@ All HTTP endpoints live here, organised by feature: 70 modules, 82 blueprints, l | `docs/ENTITLEMENTS.md` | Open-core split: FREE runtimes/features, paid tiers, GRACE mode, `/api/entitlement` shape, `clawmetry license` CLI | | `docs/EGRESS.md` | Every outbound destination, what it carries, and how to verify it on the wire | | `docs/HOOK_COEXISTENCE.md` | How ClawMetry shares a runtime's hook config with other writers | +| `docs/BUILD_YOUR_OWN_UI.md` | The keyed read API, its scopes, and how to point a coding agent at it | | `docs/CUSTOM_RUNTIME_INGEST.md` | The HTTP ingest API for a runtime with no adapter | | `docs/EVENT_RETENTION.md` | Store growth and trimming | | `CHANGELOG.md` | Version history | @@ -170,6 +174,7 @@ The complete surface is generated at `/openapi.json` and browsable at `/api/docs - `/api/signals` — Behaviour signal rates per window (`1d|7d|30d`) and `?runtime=`, with coverage and headline; `/api/signals//sessions` lists matching sessions, never phrases - `/api/guard/sessions` — What is running, what a detector thinks has gone off track, and whether each session can be controlled at all; `/api/guard/control` is the Pause / Stop / Kill button and `/api/guard/policies` the autonomous rules - `/api/entitlement` — The resolved entitlement (tier, allowed runtimes, features, capacity). GRACE mode answers "allowed" for everything until the announced enforce date +- `/api/q/1/*` — The **public** read API: the same q/1 methods, gated by a scoped API key instead of by being local. `GET /api/q/1` says what a key can read and `GET /api/q/1/llms.txt` describes the whole surface for a coding agent. `docs/BUILD_YOUR_OWN_UI.md` - `/api/local/*` — The DuckDB read API, proxied to the daemon. The method set is declared in `clawmetry/query_contract.py`; `make lint-daemon-allowlist` fails when a route calls one the daemon does not serve - `/v1/metrics`, `/v1/traces`, `/v1/logs` — OTLP receiver (binds `127.0.0.1` by default) diff --git a/clawmetry/apikeys.py b/clawmetry/apikeys.py new file mode 100644 index 0000000000..5582c3c0c7 --- /dev/null +++ b/clawmetry/apikeys.py @@ -0,0 +1,545 @@ +"""clawmetry/apikeys.py -- scoped, revocable read keys for custom UIs. + +Requirement: "Build your own UI: a keyed, scoped read API for custom +dashboards" (64c10afd-038d-4fde-9c55-ddca80aaff1e). + +Why this exists +--------------- +The dashboard reads your agents through the ``q/1`` query contract. That +contract is a good API: it is declared, versioned, arg-checked and drift +tested. Until now the only thing allowed to call it was the dashboard +itself, because the only gate in front of it was the browser's +same-origin rule. Anyone who wanted a different view of their own data +had to fork the dashboard. + +An API key changes that. You create one, say what it may read and which +site may read it, paste it into whatever you are building, and the page +gets exactly that slice. This is the substrate under "build your own UI" +(docs/BUILD_YOUR_OWN_UI.md). + +The security posture, stated plainly +------------------------------------ +Adding CORS to a localhost service is how local tools get robbed. Any +page in any tab can already SEND a request to ``127.0.0.1:8900``; the +only reason that has been harmless is that the browser refuses to let +the page READ the reply. Handing out ``Access-Control-Allow-Origin`` +removes that protection, so it is only ever echoed for an origin the key +holder named. Concretely: + +* A key is required. Loopback is NOT trusted here, unlike the rest of + the dashboard -- ``routes/public_api.py`` is the one surface where + "the request came from this machine" earns nothing. +* A key carries an origin allowlist and it may not be empty. There is + no wildcard. A key with no browser origin (a CLI, a cron, a backend) + is created with ``--origin none`` and simply never gets a CORS header. +* Scopes are least-revealing-first and ``read:content`` is never + granted implicitly -- it has to be asked for by name. +* Keys are read-only. Nothing in this module can pause, stop or kill an + agent, and ``routes/public_api.py`` dispatches only ``q/1`` read + shapes, so this adds nothing to ClawMetry's control plane. + +Storage +------- +``~/.clawmetry/api_keys.json``, 0o600, one JSON document. Only the +SHA-256 of the secret is stored, so a stolen file cannot be replayed as +a key and we cannot show a user a key they lost. The wire form is +``cmk__``: the id is public and appears in listings and +audit lines, the secret never leaves the creating terminal. +""" + +from __future__ import annotations + +import hashlib +import hmac +import json +import os +import secrets +import time +from typing import Any, Optional + +from clawmetry.query_contract import SCOPE_CONTENT, SCOPE_DOC, SCOPES + +# ── Shape of the thing ────────────────────────────────────────────────── + +#: Wire prefix. Deliberately distinct from ``cm_`` (the cloud node key) +#: and ``sk-ant-`` (a model provider key) so a leaked string is +#: identifiable at a glance by a scanner or by a human reading a log. +KEY_PREFIX = "cmk" + +_ID_BYTES = 4 # 8 hex chars: enough to name a key in a listing +_SECRET_BYTES = 32 # 256 bits + +STORE_PATH = os.path.expanduser("~/.clawmetry/api_keys.json") +_FILE_MODE = 0o600 +_DIR_MODE = 0o700 + +#: A machine is not a key management product. The cap exists so a runaway +#: script cannot grow the file without bound; it is not a paywall. +MAX_KEYS = 50 + +#: Sentinel origin meaning "this key is not used from a browser". Stored +#: as an empty origin list; kept as a word so the CLI can say it back. +ORIGIN_NONE = "none" + + +#: Why a create call was refused. Every message is a literal authored here, +#: and a caller that has to put one in an HTTP response looks it up by code +#: rather than reading it off the exception: text taken from an exception is +#: exception-derived to a static analyser no matter who wrote it, and a +#: const-indexed lookup is the thing that is provably not. +REFUSAL_REASONS: dict = { + "unknown_scope": ( + "That is not a scope. Choose from: " + ", ".join(SCOPES) + ), + "no_scope": ( + "A key needs at least one scope. Choose from: " + ", ".join(SCOPES) + ), + "wildcard_origin": ( + "A wildcard origin is not allowed. Any page in any tab could then " + "read this machine's telemetry. Name the site you are building, for " + "example https://my-app.vercel.app." + ), + "bad_origin": ( + "That is not an origin. An origin is just a scheme, host and port, " + "with no path or query: https://my-app.vercel.app, or " + "http://localhost:3000." + ), + "no_name": ( + "Give the key a name so you can tell it apart later, for example: " + "latency-workbench." + ), + "name_too_long": "Key names are limited to 64 characters.", + "at_capacity": ( + f"This machine already has {MAX_KEYS} active keys, which is the " + "limit. Revoke one you no longer use: clawmetry key revoke " + ), +} + + +def message_for(reason: str) -> str: + """The refusal sentence for ``reason``. + + Looked up from the literal table above, never read off an exception, so + a caller can put the result in an HTTP response without carrying + exception-derived text into it. + """ + return REFUSAL_REASONS.get( + str(reason), "That key could not be created." + ) + + +class ApiKeyError(Exception): + """Raised for a caller mistake (bad scope, bad origin, cap reached). + + Carries a sentence meant for a person, not an error code -- these + surface directly in ``clawmetry key`` output, where naming the exact + offending value is worth more than it costs. ``reason`` is the same + refusal as a stable code, for the HTTP callers that must not echo + exception text; see :data:`REFUSAL_REASONS`. + """ + + def __init__(self, message: str, reason: str = ""): + super().__init__(message) + self.reason = reason + + +# ── Store I/O ─────────────────────────────────────────────────────────── + +def _store_path() -> str: + return os.environ.get("CLAWMETRY_API_KEYS_PATH") or STORE_PATH + + +def _read_store() -> dict: + """The stored document, or an empty one. Never raises: a corrupt or + unreadable file must not take the dashboard down, it must behave as + "no keys are configured" so every request 401s honestly.""" + try: + with open(_store_path()) as fh: + data = json.load(fh) + if not isinstance(data, dict): + return {"version": 1, "keys": []} + keys = data.get("keys") + if not isinstance(keys, list): + data["keys"] = [] + return data + except (FileNotFoundError, OSError, ValueError, json.JSONDecodeError): + return {"version": 1, "keys": []} + + +def _write_store(doc: dict) -> None: + """Atomically replace the store, 0o600, creating ~/.clawmetry if needed.""" + path = _store_path() + parent = os.path.dirname(path) + if parent: + os.makedirs(parent, exist_ok=True) + try: + os.chmod(parent, _DIR_MODE) + except OSError: + pass # NFS home, Windows: best effort, never block a create + body = json.dumps(doc, indent=2, sort_keys=True) + "\n" + tmp = path + ".tmp" + # os.open with the mode arg so umask cannot widen a fresh key file to + # 0o644. Mirrors clawmetry/license.py::_secure_write. + fd = os.open(tmp, os.O_CREAT | os.O_TRUNC | os.O_WRONLY, _FILE_MODE) + try: + os.write(fd, body.encode("utf-8")) + finally: + os.close(fd) + try: + os.chmod(tmp, _FILE_MODE) + except OSError: + pass + os.replace(tmp, path) + + +# ── Validation ────────────────────────────────────────────────────────── + +def normalise_scopes(scopes) -> list: + """Return ``scopes`` as a sorted, de-duplicated, validated list. + + Raises :class:`ApiKeyError` naming the offender, because "invalid + scope" with no name is the kind of message that sends someone to the + source to find out what they typed wrong. + """ + out = set() + for raw in scopes or (): + s = str(raw).strip() + if not s: + continue + if s not in SCOPES: + raise ApiKeyError( + f"{s!r} is not a scope. Choose from: " + ", ".join(SCOPES), + "unknown_scope", + ) + out.add(s) + if not out: + raise ApiKeyError( + "A key needs at least one scope. Choose from: " + ", ".join(SCOPES), + "no_scope", + ) + # Keep the declared order (least revealing first) rather than + # alphabetical, so a listing reads the way the docs do. + return [s for s in SCOPES if s in out] + + +def normalise_origins(origins) -> list: + """Validate browser origins down to ``scheme://host[:port]``. + + An origin is what a browser will actually send in the ``Origin`` + header, so anything with a path, a query or a wildcard is rejected + here rather than silently never matching at request time. + """ + from urllib.parse import urlsplit + + out = [] + for raw in origins or (): + o = str(raw).strip().rstrip("/") + if not o or o.lower() == ORIGIN_NONE: + continue + if o == "*": + raise ApiKeyError( + "A wildcard origin is not allowed. Any page in any tab could " + "then read this machine's telemetry. Name the site you are " + "building, for example https://my-app.vercel.app.", + "wildcard_origin", + ) + parts = urlsplit(o) + if parts.scheme not in ("http", "https"): + raise ApiKeyError( + f"{o!r} is not an origin. An origin looks like " + "https://my-app.vercel.app or http://localhost:3000.", + "bad_origin", + ) + if not parts.netloc or parts.path or parts.query or parts.fragment: + raise ApiKeyError( + f"{o!r} has a path or query. An origin is just the scheme, " + "host and port: " + f"{parts.scheme}://{parts.netloc}", + "bad_origin", + ) + out.append(f"{parts.scheme}://{parts.netloc}".lower()) + # De-duplicate, keep first-seen order so the user's list reads back + # the way they typed it. + seen, uniq = set(), [] + for o in out: + if o not in seen: + seen.add(o) + uniq.append(o) + return uniq + + +# ── Create / list / revoke ────────────────────────────────────────────── + +def _hash_secret(secret: str) -> str: + return hashlib.sha256(secret.encode("utf-8")).hexdigest() + + +def create(name: str, scopes, origins, *, note: str = "") -> tuple: + """Mint a key. Returns ``(record, plaintext_key)``. + + The plaintext is returned once and never stored. Callers show it and + forget it. + """ + label = (name or "").strip() + if not label: + raise ApiKeyError( + "Give the key a name so you can tell it apart later, for " + "example: latency-workbench.", + "no_name", + ) + if len(label) > 64: + raise ApiKeyError("Key names are limited to 64 characters.", + "name_too_long") + scope_list = normalise_scopes(scopes) + origin_list = normalise_origins(origins) + + doc = _read_store() + live = [k for k in doc["keys"] if not k.get("revoked_at")] + if len(live) >= MAX_KEYS: + raise ApiKeyError( + f"This machine already has {MAX_KEYS} active keys, which is the " + "limit. Revoke one you no longer use: clawmetry key revoke ", + "at_capacity", + ) + + key_id = secrets.token_hex(_ID_BYTES) + secret = secrets.token_urlsafe(_SECRET_BYTES) + record = { + "id": key_id, + "name": label, + "note": (note or "").strip()[:200], + "hash": _hash_secret(secret), + "scopes": scope_list, + "origins": origin_list, + "created_at": int(time.time()), + "last_used_at": None, + "use_count": 0, + "revoked_at": None, + } + doc["keys"].append(record) + _write_store(doc) + return record, f"{KEY_PREFIX}_{key_id}_{secret}" + + +def list_keys(*, include_revoked: bool = False) -> list: + """Key records with the hash stripped. Newest first.""" + doc = _read_store() + rows = [ + {k: v for k, v in rec.items() if k != "hash"} + for rec in doc["keys"] + if include_revoked or not rec.get("revoked_at") + ] + rows.sort(key=lambda r: r.get("created_at") or 0, reverse=True) + return rows + + +def revoke(key_id: str) -> bool: + """Mark a key revoked. Returns False when no such live key exists. + + The record is kept, not deleted: "this key was revoked on the 8th" + is the answer someone needs when a key stops working, and an absent + row cannot give it. + """ + wanted = (key_id or "").strip().lower() + if wanted.startswith(KEY_PREFIX + "_"): + # Someone pasted the whole key back. Accept it; the id is the + # second field and the secret is ignored. + parts = wanted.split("_") + wanted = parts[1] if len(parts) > 1 else "" + if not wanted: + return False + doc = _read_store() + hit = False + for rec in doc["keys"]: + if rec.get("id") == wanted and not rec.get("revoked_at"): + rec["revoked_at"] = int(time.time()) + hit = True + if hit: + _write_store(doc) + return hit + + +# ── Verification (the request path) ───────────────────────────────────── + +def parse(presented: str) -> Optional[tuple]: + """Split a presented key into ``(key_id, secret)``, or None.""" + s = (presented or "").strip() + if not s.startswith(KEY_PREFIX + "_"): + return None + parts = s.split("_", 2) + if len(parts) != 3 or not parts[1] or not parts[2]: + return None + return parts[1], parts[2] + + +def verify(presented: str) -> Optional[dict]: + """The key's record if ``presented`` is a live key, else None. + + Compared with :func:`hmac.compare_digest` over the hex digest so the + check does not leak the secret through its own timing. Never raises: + an unreadable store means "no key matches", which is the safe answer. + """ + parsed = parse(presented) + if not parsed: + return None + key_id, secret = parsed + presented_hash = _hash_secret(secret) + for rec in _read_store()["keys"]: + if rec.get("id") != key_id or rec.get("revoked_at"): + continue + stored = str(rec.get("hash") or "") + if stored and hmac.compare_digest(stored, presented_hash): + return rec + return None + + +def touch(key_id: str) -> None: + """Record that a key was just used. Best effort, never raises. + + Usage is written back so ``clawmetry key list`` can answer "is + anything still using this?" before someone revokes it. The write is + skipped when the timestamp would not change to the second, so a page + polling once a second does not rewrite the file on every request. + """ + try: + doc = _read_store() + now = int(time.time()) + changed = False + for rec in doc["keys"]: + if rec.get("id") == key_id: + rec["use_count"] = int(rec.get("use_count") or 0) + 1 + if rec.get("last_used_at") != now: + rec["last_used_at"] = now + changed = True + if changed: + _write_store(doc) + except Exception: + return + + +def origin_allowed(record: dict, origin: str) -> bool: + """True when ``origin`` is on this key's allowlist. Case-insensitive, + trailing slash tolerated (some clients send one). An empty allowlist + matches nothing, by design: that key is for non-browser callers.""" + if not origin: + return False + o = origin.strip().rstrip("/").lower() + return o in [str(x).lower() for x in (record.get("origins") or [])] + + +def canonical_allowed_origin(record: dict, origin: str) -> "str | None": + """Return the stored canonical form of *origin* if this key allows it. + + Returns ``None`` when the origin is not on the key's allowlist. + Using the stored value (not the caller-supplied string) in the + ``Access-Control-Allow-Origin`` response header prevents a + user-input → response-header taint chain (CWE-113). + """ + if not origin: + return None + o = origin.strip().rstrip("/").lower() + for stored in (record.get("origins") or []): + if str(stored).lower() == o: + return str(stored) + return None + + +def any_key_allows_origin(origin: str) -> bool: + """True when ANY live key names ``origin``. + + A CORS preflight arrives without the ``Authorization`` header, so at + preflight time we cannot know which key the real request will carry. + This answers the only question the preflight can answer: is this + origin one the user has authorised at all? The real request is still + checked against its own key's allowlist. + """ + if not origin: + return False + o = origin.strip().rstrip("/").lower() + for rec in _read_store()["keys"]: + if rec.get("revoked_at"): + continue + if o in [str(x).lower() for x in (rec.get("origins") or [])]: + return True + return False + + +def any_canonical_allowed_origin(origin: str) -> "str | None": + """Return the stored canonical form of *origin* from any live key that allows it. + + Returns ``None`` when no live key names the origin. + Uses the stored value (not the caller-supplied string) so the + ``Access-Control-Allow-Origin`` response header is not built from + raw request data (CWE-113). + """ + if not origin: + return None + o = origin.strip().rstrip("/").lower() + for rec in _read_store()["keys"]: + if rec.get("revoked_at"): + continue + for stored in (rec.get("origins") or []): + if str(stored).lower() == o: + return str(stored) + return None + + +def all_live_origins() -> list: + """All canonical origins stored across every live (non-revoked) key. + + Used by the CORS preflight path in routes/public_api.py to compare + against the caller-supplied origin WITHOUT passing user input through + this function — that breaks the CodeQL CWE-113 taint chain. + """ + result = [] + for rec in _read_store()["keys"]: + if rec.get("revoked_at"): + continue + for stored in (rec.get("origins") or []): + result.append(str(stored)) + return result + + +def granted_shapes(record: dict) -> set: + """Every live q/1 shape this key may dispatch.""" + from clawmetry.query_contract import shapes_for_scopes + + return shapes_for_scopes(record.get("scopes") or []) + + +def scope_catalogue() -> list: + """``[{scope, doc, methods, sensitive}]`` for the UI and the CLI help. + + Derived from the query contract, so a method added there shows up + here with no second list to update. + """ + from clawmetry.query_contract import live_methods_by_scope + + return [ + { + "scope": s, + "doc": SCOPE_DOC[s], + "methods": live_methods_by_scope(s), + "sensitive": s == SCOPE_CONTENT, + } + for s in SCOPES + ] + + +def redact(presented: str) -> str: + """``cmk_a1b2c3d4_...`` -- safe to log. Shows the id, never the secret.""" + parsed = parse(presented) + if not parsed: + return "(no key)" + return f"{KEY_PREFIX}_{parsed[0]}_..." + + +def store_summary() -> dict[str, Any]: + """Counts for the dashboard panel, cheap enough to call per page load.""" + doc = _read_store() + keys = doc["keys"] + return { + "active": sum(1 for k in keys if not k.get("revoked_at")), + "revoked": sum(1 for k in keys if k.get("revoked_at")), + "max": MAX_KEYS, + "path": _store_path(), + } diff --git a/clawmetry/cli.py b/clawmetry/cli.py index f4f9e40e0a..68bccbfd81 100644 --- a/clawmetry/cli.py +++ b/clawmetry/cli.py @@ -4844,6 +4844,187 @@ def _cmd_mcp(args) -> None: raise SystemExit(_mcp_cli(list(getattr(args, "mcp_args", None) or []))) + +def _cmd_key(args) -> None: + """`clawmetry key ...` -- scoped read keys for custom UIs. + + Spec: blueprint 0ea7523c-12b5-4033-84ea-bf1f46e20d70, "API Surface" -> + "Command line". Four subcommands, each also taking --json: + create (mints one, prints the secret once), list (names, scopes, + origins, last use), revoke (effective on that key's next request), + scopes (what each grants). + + The point of these keys is that someone can build their own view of + their own agents without forking the dashboard: create a key, say what + it may read and which site may read it, paste it into whatever they + are building. docs/BUILD_YOUR_OWN_UI.md is the long version. + + Two things this command is deliberately strict about, because both + are how a local API gets robbed: + + * A browser key must name its origin. There is no wildcard. Any page + in any tab can already send a request to localhost; the origin + allowlist is the whole reason it cannot read the answer. + * ``read:content`` (prompts, replies, tool calls) is never granted + unless it is asked for by name, and the created key says so out loud. + """ + import json as _json + import time as _time + + from clawmetry import apikeys as _ak + from clawmetry.query_contract import SCOPE_CONTENT, SCOPE_DOC, SCOPE_METRICS + + action = getattr(args, "key_cmd", None) or "list" + as_json = bool(getattr(args, "as_json", False)) + + def _fmt_age(ts): + if not ts: + return "never" + delta = int(_time.time()) - int(ts) + if delta < 60: + return "just now" + for unit, secs in (("d", 86400), ("h", 3600), ("m", 60)): + if delta >= secs: + return f"{delta // secs}{unit} ago" + return "just now" + + if action == "scopes": + rows = _ak.scope_catalogue() + if as_json: + print(_json.dumps({"scopes": rows}, indent=2)) + return + print("Scopes, least revealing first.") + print("") + for row in rows: + flag = " (sensitive)" if row["sensitive"] else "" + print(f" {row['scope']}{flag}") + print(f" {row['doc']}") + print(f" queries: {', '.join(row['methods'])}") + print("") + print("Pick the narrowest scope that makes your UI work. A key that") + print("only needs a cost chart should be read:metrics, so it cannot") + print("return a prompt even if the page it lives in is compromised.") + return + + if action == "list": + rows = _ak.list_keys(include_revoked=bool(getattr(args, "show_revoked", False))) + if as_json: + print(_json.dumps({"keys": rows, "summary": _ak.store_summary()}, indent=2)) + return + if not rows: + print("No API keys on this machine.") + print("") + print("Create one to build your own UI on top of ClawMetry:") + print("") + print(" clawmetry key create --name my-ui \\") + print(" --scope read:metrics --origin http://localhost:3000") + print("") + print("See docs/BUILD_YOUR_OWN_UI.md for the walkthrough.") + return + print(f"{'ID':<10} {'NAME':<24} {'SCOPES':<34} {'LAST USED':<12} ORIGINS") + for r in rows: + origins = ", ".join(r.get("origins") or []) or "(not used from a browser)" + state = " [revoked]" if r.get("revoked_at") else "" + print( + f"{r['id']:<10} {r['name'][:23]:<24} " + f"{','.join(r.get('scopes') or [])[:33]:<34} " + f"{_fmt_age(r.get('last_used_at')):<12} {origins}{state}" + ) + return + + if action == "revoke": + key_id = getattr(args, "key_id", "") or "" + ok = _ak.revoke(key_id) + if as_json: + print(_json.dumps({"action": "revoke", "ok": ok, "id": key_id}, indent=2)) + if not ok: + raise SystemExit(1) + return + if ok: + print(f"Key {key_id} revoked. The next request using it is refused.") + print("Anything you built on it needs a new key: clawmetry key create ...") + return + print(f"No active key called {key_id!r} on this machine.") + print("List what is here with: clawmetry key list") + raise SystemExit(1) + + if action == "create": + scopes = list(getattr(args, "scope", None) or []) or [SCOPE_METRICS] + raw_origins = list(getattr(args, "origin", None) or []) + wants_no_origin = any( + str(o).strip().lower() == _ak.ORIGIN_NONE for o in raw_origins + ) + if not raw_origins: + print("A key needs to know which site may use it from a browser.") + print("") + print(" --origin https://my-ui.vercel.app a site you are building") + print(" --origin http://localhost:3000 your dev server") + print(" --origin none not used from a browser") + print("") + print("There is no wildcard. Any page in any tab can already send a") + print("request to this machine, and naming the origin is what stops") + print("it reading the answer.") + raise SystemExit(1) + try: + record, plaintext = _ak.create( + getattr(args, "name", ""), + scopes, + [] if wants_no_origin else raw_origins, + note=getattr(args, "note", ""), + ) + except _ak.ApiKeyError as exc: + if as_json: + print(_json.dumps({"action": "create", "ok": False, + "error": str(exc)}, indent=2)) + else: + print(str(exc)) + raise SystemExit(1) + + if as_json: + print(_json.dumps({"action": "create", "ok": True, # codeql[py/clear-text-logging-sensitive-data] + "key": plaintext, # codeql[py/clear-text-logging-sensitive-data] + "record": {k: v for k, v in record.items() + if k != "hash"}}, indent=2)) + return + + print("Key created. It is shown once and is not stored anywhere in") + print("readable form, so copy it now.") + print("") + print(f" {plaintext}") # codeql[py/clear-text-logging-sensitive-data] + print("") + print(f"Name: {record['name']} (id {record['id']})") + print(f"Reads: {', '.join(record['scopes'])}") + for s in record["scopes"]: + print(f" {s}: {SCOPE_DOC[s]}") + if record["origins"]: + print(f"Origins: {', '.join(record['origins'])}") + else: + print("Origins: none. This key works from a script or a server, but a") + print(" browser page will not be allowed to read the reply.") + if SCOPE_CONTENT in record["scopes"]: + print("") + print("This key can read the turns themselves: prompts, replies and") + print("tool calls. Keep it server-side. Do not ship it in a page.") + print("") + print("Put it somewhere your shell can reach, then try it:") + print("") + print(" export CLAWMETRY_KEY=") + print(" curl -H \"Authorization: Bearer $CLAWMETRY_KEY\" \\") + print(" http://localhost:8900/api/q/1") + print("") + print("Point a coding agent at the generated API guide:") + print("") + print(" curl -H \"Authorization: Bearer $CLAWMETRY_KEY\" \\") + print(" http://localhost:8900/api/q/1/llms.txt") + print("") + print("Walkthrough: docs/BUILD_YOUR_OWN_UI.md") + return + + print("Usage: clawmetry key [create|list|revoke|scopes]") + print("Start with: clawmetry key scopes") + raise SystemExit(1) + + def _cmd_reports(args) -> None: """Open the reports browser (refs #1005).""" import webbrowser @@ -8456,6 +8637,74 @@ def main() -> None: # mcp — intercepted by the fast path at the top of main() (WO-59); the # parser entry exists so `clawmetry --help` discovery shows it. + # key — scoped read keys for custom UIs (docs/BUILD_YOUR_OWN_UI.md) + p_key = sub.add_parser( + "key", + help="API keys for custom UIs: create, list, revoke, scopes", + ) + key_sub = p_key.add_subparsers(dest="key_cmd") + + p_key_create = key_sub.add_parser( + "create", help="Mint a scoped read key. Shown once, never stored." + ) + p_key_create.add_argument( + "--name", required=True, metavar="NAME", + help="What this key is for, e.g. latency-workbench. Shown in listings.", + ) + p_key_create.add_argument( + "--scope", action="append", default=[], metavar="SCOPE", + help=( + "What the key may read. Repeatable. Run `clawmetry key scopes` " + "for the list. Defaults to read:metrics, the least revealing one." + ), + ) + p_key_create.add_argument( + "--origin", action="append", default=[], metavar="URL", + help=( + "A site allowed to call this API from a browser, e.g. " + "https://my-ui.vercel.app. Repeatable, and required unless you " + "pass --origin none for a key used outside a browser. There is " + "no wildcard: any page in any tab can already reach localhost, " + "and the origin allowlist is what stops it reading the reply." + ), + ) + p_key_create.add_argument( + "--note", default="", metavar="TEXT", + help="Optional reminder to your future self.", + ) + p_key_create.add_argument( + "--json", action="store_true", dest="as_json", + help="Emit the record plus the key as JSON (jq-friendly).", + ) + + p_key_list = key_sub.add_parser("list", help="Show this machine's keys") + p_key_list.add_argument( + "--all", action="store_true", dest="show_revoked", + help="Include revoked keys.", + ) + p_key_list.add_argument( + "--json", action="store_true", dest="as_json", + help="Emit JSON (jq-friendly).", + ) + + p_key_revoke = key_sub.add_parser( + "revoke", help="Stop a key working. Takes effect on the next request." + ) + p_key_revoke.add_argument( + "key_id", metavar="ID", + help="The key id from `clawmetry key list` (the whole key also works).", + ) + p_key_revoke.add_argument( + "--json", action="store_true", dest="as_json", help="Emit JSON.", + ) + + p_key_scopes = key_sub.add_parser( + "scopes", help="What each scope grants, and which queries it unlocks" + ) + p_key_scopes.add_argument( + "--json", action="store_true", dest="as_json", help="Emit JSON.", + ) + p_mcp = sub.add_parser( "mcp", help="MCP server: `mcp` serves on stdio; `mcp install [--runtime |all] " @@ -9047,6 +9296,7 @@ def main() -> None: "secure", "reports", "eval", + "key", "mcp", "update", "uninstall", @@ -9191,6 +9441,8 @@ def main() -> None: _cmd_eval(args) elif args.cmd == "mcp": _cmd_mcp(args) + elif args.cmd == "key": + _cmd_key(args) elif args.cmd == "update": _cmd_update(args) elif args.cmd == "uninstall": diff --git a/clawmetry/query_contract.py b/clawmetry/query_contract.py index ba2633ac67..5c9ba0639a 100644 --- a/clawmetry/query_contract.py +++ b/clawmetry/query_contract.py @@ -34,6 +34,8 @@ * ``trust`` — "plaintext" or "e2e" (see above). * ``backing`` — the LocalStore method serving it (live) or the planned rollup table / store method (planned). +* ``scope`` — the read scope an API key must carry to dispatch this + method (see SCOPES below). Exactly one per method. * ``doc`` — one-line description. This module is intentionally dependency-free plain data so the doc @@ -51,6 +53,46 @@ TRUST_PLAINTEXT = "plaintext" TRUST_E2E = "e2e" +# ── Read scopes: what an API key is allowed to ask for ────────────────── +# +# Every method declares exactly one scope. A key issued to a custom UI +# (``clawmetry key create --scope read:metrics``) can dispatch only the +# methods whose scope it carries, so the scope a user picks in the UI is +# the same fact the server enforces -- there is no second list to drift. +# +# The split is by what a row REVEALS, not by which table it came from: +# +# * ``read:metrics`` counters and rollups. Exactly the ``plaintext`` +# trust class, and an invariant test pins that: a +# plaintext method is always metrics-scoped, and a +# metrics-scoped method is never ``e2e``. This is the +# scope a cost dashboard or a status board needs, and +# it can never return a prompt or a file path. +# * ``read:sessions`` one row per session: title, model, status, totals. +# Enough to build a session list or a search box. +# * ``read:content`` the turns themselves -- prompts, replies, tool +# calls, transcripts. The most sensitive scope, and +# the one the CLI refuses to grant unless asked for +# by name. +# * ``read:traces`` OTel spans/traces and outbound non-LLM calls. +SCOPE_METRICS = "read:metrics" +SCOPE_SESSIONS = "read:sessions" +SCOPE_CONTENT = "read:content" +SCOPE_TRACES = "read:traces" + +#: Every scope a key may carry, in the order the UI should offer them +#: (least revealing first). +SCOPES: tuple = (SCOPE_METRICS, SCOPE_SESSIONS, SCOPE_TRACES, SCOPE_CONTENT) + +#: One line per scope, written for someone who has never read this file. +#: The key-creation UI and ``clawmetry key create --help`` both render it. +SCOPE_DOC: dict = { + SCOPE_METRICS: "Counts, tokens, cost and health. No prompts or replies.", + SCOPE_SESSIONS: "One row per session: title, model, status, totals.", + SCOPE_TRACES: "Spans, traces and outbound API calls.", + SCOPE_CONTENT: "The turns themselves: prompts, replies, tool calls.", +} + def _arg(required: bool = False, **extra) -> dict: """Tiny spec-builder so the registry below reads as a table.""" @@ -72,6 +114,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=200, lo=1, hi=5000), }, "trust": TRUST_E2E, + "scope": SCOPE_CONTENT, "backing": "query_events", "doc": "Raw event rows (tool calls, messages, errors), newest first.", }, @@ -84,6 +127,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=100, lo=1, hi=2000), }, "trust": TRUST_E2E, + "scope": SCOPE_SESSIONS, "backing": "query_sessions", "doc": "One row per session_id with start/end, event count, cost.", }, @@ -95,6 +139,7 @@ def _arg(required: bool = False, **extra) -> dict: "until": _arg(), }, "trust": TRUST_PLAINTEXT, + "scope": SCOPE_METRICS, "backing": "query_aggregates", "doc": "Per-day rollup of events/tokens/cost (aggregate counters only).", }, @@ -102,6 +147,7 @@ def _arg(required: bool = False, **extra) -> dict: "status": STATUS_LIVE, "args": {}, "trust": TRUST_PLAINTEXT, + "scope": SCOPE_METRICS, "backing": "health", "doc": "Store health snapshot (engine, size, ring depth, flush age).", }, @@ -112,6 +158,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=500, lo=1, hi=5000), }, "trust": TRUST_E2E, + "scope": SCOPE_CONTENT, "backing": "query_events", "doc": "Alias of events scoped to one required session_id.", }, @@ -123,6 +170,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=150, lo=1, hi=250), }, "trust": TRUST_E2E, + "scope": SCOPE_CONTENT, "backing": "query_transcript_page", "doc": ( "One older-history page of a session's events, newest-first. " @@ -142,6 +190,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=200, lo=1, hi=2000), }, "trust": TRUST_E2E, + "scope": SCOPE_TRACES, "backing": "query_spans", "doc": "OTel span rows with full filters (trace/session/agent/time).", }, @@ -155,6 +204,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=100, lo=1, hi=1000), }, "trust": TRUST_E2E, + "scope": SCOPE_TRACES, "backing": "query_traces", "doc": "One row per trace_id with aggregate span stats.", }, @@ -167,6 +217,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=200, lo=1, hi=2000), }, "trust": TRUST_E2E, + "scope": SCOPE_TRACES, "backing": "query_external_calls", "doc": "External (non-LLM) API calls captured by the interceptor.", }, @@ -181,6 +232,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=50, lo=1, hi=500), }, "trust": TRUST_E2E, + "scope": SCOPE_SESSIONS, "backing": "query_search", "doc": "Full-text search over session titles and eval reasons.", }, @@ -191,6 +243,7 @@ def _arg(required: bool = False, **extra) -> dict: "status": STATUS_PLANNED, "args": {}, "trust": TRUST_PLAINTEXT, + "scope": SCOPE_METRICS, "backing": "rollup_glance", "doc": ("Device-facing top-line counters (sessions, cost, alerts). " "Non-goal: no per-model data in glance."), @@ -205,6 +258,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=1000, lo=1, hi=10000), }, "trust": TRUST_PLAINTEXT, + "scope": SCOPE_METRICS, "backing": "query_rollup_runtime_daily", "doc": "Per-runtime daily activity/cost rollup (claude_code, openclaw, ...).", }, @@ -219,6 +273,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=1000, lo=1, hi=10000), }, "trust": TRUST_PLAINTEXT, + "scope": SCOPE_METRICS, "backing": "query_rollup_model_daily", "doc": "Per-model daily token/cost rollup across runtimes.", }, @@ -234,6 +289,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=200, lo=1, hi=2000), }, "trust": TRUST_E2E, + "scope": SCOPE_SESSIONS, "backing": "query_rollup_sessions", "doc": "Per-session materialized summary (title, status, totals, stuck flag).", }, @@ -245,6 +301,7 @@ def _arg(required: bool = False, **extra) -> dict: "until": _arg(), }, "trust": TRUST_PLAINTEXT, + "scope": SCOPE_METRICS, "backing": "rollup_usage_daily", "doc": "Daily token/cost usage series (input/output/cache splits).", }, @@ -254,6 +311,7 @@ def _arg(required: bool = False, **extra) -> dict: "session_id": _arg(required=True), }, "trust": TRUST_E2E, + "scope": SCOPE_SESSIONS, "backing": "query_sessions_table", "doc": "Single-session detail row (title, status, outcome, totals).", }, @@ -265,6 +323,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=200, lo=1, hi=2000), }, "trust": TRUST_E2E, + "scope": SCOPE_CONTENT, "backing": "query_events", "doc": "Reasoning/tool event slice powering the Brain feed.", }, @@ -275,6 +334,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=100, lo=1, hi=1000), }, "trust": TRUST_PLAINTEXT, + "scope": SCOPE_METRICS, "backing": "query_approvals", "doc": "Approval queue metadata (ids, states, timestamps; no content).", }, @@ -287,6 +347,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=500, lo=1, hi=2000), }, "trust": TRUST_PLAINTEXT, + "scope": SCOPE_METRICS, "backing": "query_agent_graph", "doc": "Cross-session agent spawn graph: nodes (agent_type+id stats) + " "spawn edges. Optional runtime arg scopes to one runtime " @@ -299,6 +360,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=2000, lo=1, hi=10000), }, "trust": TRUST_E2E, + "scope": SCOPE_CONTENT, "backing": "query_replay_events", "doc": "Canonical replay-event rows for one session (#4813). Rows in " "kind-agnostic order; the /api/replay-tree endpoint groups " @@ -312,6 +374,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=200, lo=1, hi=1000), }, "trust": TRUST_E2E, + "scope": SCOPE_CONTENT, "backing": "query_session_context", "doc": "Inputs & context rows for one session: system prompt, first " "user prompt, tool definitions, MCP servers, context files and " @@ -326,6 +389,7 @@ def _arg(required: bool = False, **extra) -> dict: "limit": _arg(default=10, lo=1, hi=50), }, "trust": TRUST_E2E, + "scope": SCOPE_SESSIONS, "backing": "query_similar_sessions", "doc": "Runs shaped like this one (WO-60): nearest sessions by " "tool-call n-gram similarity inside a window, same runtime " @@ -360,3 +424,31 @@ def methods_by_status(status: str) -> list: def methods_by_trust(trust: str) -> list: return sorted(n for n, s in QUERY_CONTRACT.items() if s["trust"] == trust) + + +def methods_by_scope(scope: str) -> list: + """Live + planned method names carrying ``scope``.""" + return sorted(n for n, sp in QUERY_CONTRACT.items() if sp["scope"] == scope) + + +def live_methods_by_scope(scope: str) -> list: + """Method names carrying ``scope`` that are actually served today.""" + return sorted( + n for n, sp in QUERY_CONTRACT.items() + if sp["scope"] == scope and sp["status"] == STATUS_LIVE + ) + + +def scope_for(method: str) -> str: + """The scope ``method`` requires. Raises KeyError for an unknown method, + which is the right answer: an undeclared method is not servable.""" + return QUERY_CONTRACT[method]["scope"] + + +def shapes_for_scopes(scopes) -> set: + """Every live shape reachable by a key holding ``scopes``.""" + granted = set(scopes or ()) + return { + n for n, sp in QUERY_CONTRACT.items() + if sp["status"] == STATUS_LIVE and sp["scope"] in granted + } diff --git a/clawmetry/static/js/app.js b/clawmetry/static/js/app.js index 273a7f327a..e231590327 100644 --- a/clawmetry/static/js/app.js +++ b/clawmetry/static/js/app.js @@ -11444,7 +11444,11 @@ var _CM_SECURITY_CLOUD_HIDDEN = [ 'security-findings-panel', 'policy-events-panel', 'credential-scan-panel', - 'security-catalog-panel' + 'security-catalog-panel', + // API keys live in ~/.clawmetry on the machine the agent runs on. The + // cloud container has no such file, so the panel would list nothing + // under a "New key" button that could not mint one. + 'apikeys-panel' ]; function _cmSecurityCloudTrim() { @@ -11457,6 +11461,219 @@ function _cmSecurityCloudTrim() { if (note) note.style.display = ''; } +// ── API keys: build your own UI (docs/BUILD_YOUR_OWN_UI.md) ────────────── +// +// A key here is read-only and scoped. The panel's job is to make the two +// things that keep it safe impossible to skip past: you must say WHICH +// site may use the key, and reading the turns themselves (read:content) +// has to be ticked on purpose. + +var _cmApiKeyScopes = []; + +function _cmApiKeysEl(id) { return document.getElementById(id); } + +async function loadApiKeys() { + if (window.CLOUD_MODE) return; + var list = _cmApiKeysEl('apikeys-list'); + if (!list) return; + var data; + try { + data = await fetchJsonWithTimeout('/api/apikeys', 10000); + } catch (e) { + list.innerHTML = '
' + + 'ClawMetry could not read its key file just now. Reload the page to try again.' + + '
'; + return; + } + if (!data || data.ok === false) { + list.innerHTML = '
' + + escapeHtmlSafe((data && data.error) || 'ClawMetry could not read its key file.') + + '
'; + return; + } + _cmApiKeyScopes = data.scopes || []; + _cmRenderApiKeyScopeChoices(); + var keys = (data.keys || []).filter(function(k) { return !k.revoked_at; }); + if (!keys.length) { + list.innerHTML = '
' + + 'No keys yet. A key lets you build your own view of this data: a cost ' + + 'chart on a wall screen, a status page for your team, a panel inside a ' + + 'tool you already use. Press New key, then point a ' + + 'coding agent at the generated API guide.' + + '
'; + return; + } + list.innerHTML = keys.map(_cmRenderApiKeyRow).join(''); +} + +function _cmRenderApiKeyRow(k) { + var scopes = (k.scopes || []).map(function(s) { + var sensitive = (s === 'read:content'); + return '' + escapeHtmlSafe(s) + ''; + }).join(' '); + var origins = (k.origins || []).length + ? (k.origins || []).map(escapeHtmlSafe).join(', ') + : 'Not used from a web page'; + var used = k.last_used_at + ? ('Last used ' + _cmApiKeyAgo(k.last_used_at) + ' · ' + (k.use_count || 0) + ' requests') + : 'Never used'; + return '
' + + '
' + + '
' + + escapeHtmlSafe(k.name || '(unnamed)') + + ' · ' + escapeHtmlSafe(k.id || '') + '' + + '
' + + '
' + scopes + '
' + + '
' + origins + '
' + + '
' + escapeHtmlSafe(used) + '
' + + '
' + + '' + + '
'; +} + +function _cmApiKeyAgo(ts) { + var delta = Math.floor(Date.now() / 1000) - Number(ts || 0); + if (delta < 60) return 'just now'; + if (delta < 3600) return Math.floor(delta / 60) + 'm ago'; + if (delta < 86400) return Math.floor(delta / 3600) + 'h ago'; + return Math.floor(delta / 86400) + 'd ago'; +} + +function _cmRenderApiKeyScopeChoices() { + var box = _cmApiKeysEl('apikey-scopes'); + if (!box) return; + box.innerHTML = (_cmApiKeyScopes || []).map(function(s, i) { + var warn = s.sensitive + ? '
Keep this one server-side. Do not ship it in a page.
' + : ''; + return ''; + }).join(''); +} + +function toggleApiKeyForm(show) { + var form = _cmApiKeysEl('apikeys-form'); + if (!form) return; + var open = (show === undefined) ? form.hidden : !!show; + form.hidden = !open; + if (open) { + _cmRenderApiKeyScopeChoices(); + var err = _cmApiKeysEl('apikey-form-error'); + if (err) err.textContent = ''; + var name = _cmApiKeysEl('apikey-name'); + if (name) name.focus(); + } +} + +function onApiKeyBrowserToggle() { + var box = _cmApiKeysEl('apikey-nobrowser'); + var origins = _cmApiKeysEl('apikey-origins'); + if (!box || !origins) return; + origins.disabled = box.checked; + origins.placeholder = box.checked + ? 'Not needed: this key is not used from a web page' + : 'http://localhost:3000'; +} + +async function createApiKey() { + var err = _cmApiKeysEl('apikey-form-error'); + if (err) err.textContent = ''; + var name = (_cmApiKeysEl('apikey-name') || {}).value || ''; + var originsRaw = (_cmApiKeysEl('apikey-origins') || {}).value || ''; + var noBrowser = !!((_cmApiKeysEl('apikey-nobrowser') || {}).checked); + var scopes = Array.prototype.slice + .call(document.querySelectorAll('.apikey-scope-box')) + .filter(function(b) { return b.checked; }) + .map(function(b) { return b.value; }); + var body = { + name: name.trim(), + scopes: scopes, + browser: !noBrowser, + origins: originsRaw.split(/[\s,]+/).filter(Boolean) + }; + var res, data; + try { + res = await fetch('/api/apikeys', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify(body) + }); + data = await res.json(); + } catch (e) { + if (err) err.textContent = 'ClawMetry did not answer. Is the dashboard still running?'; + return; + } + if (!data || data.ok === false) { + if (err) err.textContent = (data && data.error) || 'The key could not be created.'; + return; + } + toggleApiKeyForm(false); + _cmShowApiKeySecret(data.key, data.record || {}); + loadApiKeys(); +} + +function _cmShowApiKeySecret(key, record) { + var box = _cmApiKeysEl('apikey-reveal'); + var val = _cmApiKeysEl('apikey-reveal-value'); + var next = _cmApiKeysEl('apikey-reveal-next'); + if (!box || !val) return; + val.textContent = key; + if (next) { + var base = window.location.origin; + next.innerHTML = 'Try it, then hand the same URL to a coding agent:
' + + 'curl -H "Authorization: Bearer <key>" ' + + escapeHtmlSafe(base) + '/api/q/1/llms.txt'; + } + box.hidden = false; +} + +function copyApiKey() { + var val = _cmApiKeysEl('apikey-reveal-value'); + if (!val) return; + var text = val.textContent || ''; + if (navigator.clipboard && navigator.clipboard.writeText) { + navigator.clipboard.writeText(text); + } else { + // Older / non-secure contexts: select it so ctrl-C works. + var r = document.createRange(); + r.selectNodeContents(val); + var sel = window.getSelection(); + sel.removeAllRanges(); + sel.addRange(r); + } +} + +function dismissApiKeyReveal() { + var box = _cmApiKeysEl('apikey-reveal'); + var val = _cmApiKeysEl('apikey-reveal-value'); + if (val) val.textContent = ''; + if (box) box.hidden = true; +} + +async function revokeApiKey(id) { + if (!id) return; + var ok = window.confirm( + 'Revoke this key?\n\nAnything using it stops working on its next request. ' + + 'This cannot be undone; you would have to create a new key.' + ); + if (!ok) return; + try { + await fetch('/api/apikeys/' + encodeURIComponent(id), { method: 'DELETE' }); + } catch (e) { /* the reload below tells the truth either way */ } + loadApiKeys(); +} + async function loadSecurityPosture() { if (window.CLOUD_MODE) { // Posture scans the machine's agent config, which the cloud container does @@ -11553,6 +11770,7 @@ async function loadSecurityPosture() { } async function loadSecurityPage(silent) { + loadApiKeys(); if (window.CLOUD_MODE) { // Threat/policy/credential scans read this machine's event history, which // the cloud container does not have. Their panels go; integrity and the diff --git a/clawmetry/static/locales/en.json b/clawmetry/static/locales/en.json index 8444994ed9..56b10124ad 100644 --- a/clawmetry/static/locales/en.json +++ b/clawmetry/static/locales/en.json @@ -1143,7 +1143,7 @@ "security.audit_empty": "No recorded activity yet. Approval decisions, budget changes, and pauses appear here.", "security.audit_sub": "(approvals, budgets, pauses)", "security.audit_title": "Recent activity", - "security.cloud_note": "Config posture checks and threat scans run on the machine your agent runs on: they read files and activity that stay on it. Open ClawMetry there (localhost:8900 \u2192 Security) to run them.", + "security.cloud_note": "Config posture checks and threat scans run on the machine your agent runs on: they read files and activity that stay on it. Open ClawMetry there (localhost:8900 → Security) to run them.", "security.clean_sessions": "Clean Sessions", "security.critical": "Critical", "security.failed": "Failed", @@ -1539,5 +1539,17 @@ "trail.commits": "Commits", "trail.prs": "Pull requests", "evals.locked": "Session scoring is a Pro feature.", - "trail.cov_unknown": "not declared yet" + "trail.cov_unknown": "not declared yet", + "security.apikeys_title": "Build your own view", + "security.apikeys_sub": "Give a page, a script or a coding agent read access to this machine's agent data, scoped to exactly what it needs. Nothing here can start, pause or stop an agent.", + "security.apikeys_new": "New key", + "security.apikeys_name_label": "What is it for?", + "security.apikeys_origin_label": "Which site will use it?", + "security.apikeys_no_browser": "This key is for a script or a server, not a web page", + "security.apikeys_scope_label": "What may it read?", + "security.apikeys_create": "Create key", + "security.apikeys_cancel": "Cancel", + "security.apikeys_reveal_title": "Copy this now. It is not shown again.", + "security.apikeys_copy": "Copy", + "security.apikeys_done": "Done" } diff --git a/clawmetry/templates/tabs/security.html b/clawmetry/templates/tabs/security.html index 9ac03d62ec..ac74297cf5 100644 --- a/clawmetry/templates/tabs/security.html +++ b/clawmetry/templates/tabs/security.html @@ -60,6 +60,78 @@
+ +
+
+
🔑
+
+
Build your own view
+
Give a page, a script or a coding agent read access to this machine's agent data, scoped to exactly what it needs. Nothing here can start, pause or stop an agent.
+
+ +
+ + + + + + + +
+
cmk_a1b2c3d4_... +``` + +**2. Check it works.** + +```bash +curl -H "Authorization: Bearer cmk_a1b2c3d4_..." \ + http://localhost:8900/api/q/1 +``` + +That returns what the key can read: its scopes, and every query it may +run with the arguments each one takes. + +**3. Ask for something real.** + +```bash +curl -H "Authorization: Bearer cmk_a1b2c3d4_..." \ + "http://localhost:8900/api/q/1/aggregates?since=2026-09-01T00:00:00Z" +``` + +```json +{ + "shape": "aggregates", + "rows": [ + {"day": "2026-09-08", "agent_id": "main", + "event_count": 4059, "token_count": 887691, "cost_usd": 136.47} + ], + "count": 1, + "contract": "q/1", + "elapsed_ms": 38 +} +``` + +That is the whole API. Everything below is detail. + +## Building the UI with a coding agent + +The API describes itself, in a form written for an agent to read in one +pass and scoped to the key you hand it: + +```bash +curl -H "Authorization: Bearer $CLAWMETRY_KEY" \ + http://localhost:8900/api/q/1/llms.txt +``` + +Give that output to Claude Code, Cursor, v0, Lovable, or whatever you +build with, along with what you want. A prompt that works: + +> Build a single-page dashboard for our team's AI agent spend. +> +> Data comes from the ClawMetry query API at `http://localhost:8900/api/q/1`. +> Authenticate with `Authorization: Bearer `. The full API is below. +> +> I want: a line chart of daily cost for the last 30 days from +> `/aggregates`, a table of the top 10 models by spend from `/models`, +> and a per-runtime breakdown from `/runtimes`. Refresh every 60 seconds. +> Show the total spend for the period as the headline number. +> +> + +Two things worth putting in the prompt, because they are what a generated +UI usually gets wrong: + +- **The key belongs in an environment variable**, not in the page source. + If it must be in the browser (a static page with no backend), give that + page a `read:metrics` key and nothing more. +- **Link back into ClawMetry.** A custom view is best when it is narrow. + When someone wants to know *why* a number moved, send them to + `http://localhost:8900/#session=` rather than rebuilding the session + viewer. + +There is a working starter in [`examples/custom-ui/`](../examples/custom-ui/) +if you would rather begin from something that already runs. + +## Scopes + +A key carries one or more scopes. It can run exactly the queries its +scopes cover and nothing else. + +| Scope | Grants | Queries | +|---|---|---| +| `read:metrics` | Counts, tokens, cost and health. No prompts or replies. | `aggregates`, `models`, `runtimes`, `agent_graph`, `health` | +| `read:sessions` | One row per session: title, model, status, totals. | `sessions`, `rollup_sessions`, `search`, `similar_sessions` | +| `read:traces` | Spans, traces and outbound API calls. | `spans`, `traces`, `external_calls` | +| `read:content` | The turns themselves: prompts, replies, tool calls. | `events`, `transcript`, `transcript_page`, `replay_events`, `session_context` | + +The generated table in [`QUERY_CONTRACT.md`](./QUERY_CONTRACT.md) is the +authority, along with every argument each query takes. Run +`clawmetry key scopes` for the same thing in a terminal. + +Pick the narrowest scope that makes your UI work. `read:metrics` is +exactly the set of queries that cannot return a prompt, a reply or a file +path, which makes it the right choice for anything that renders in a +browser. + +## What a key can and cannot do + +**Can:** run the `q/1` read queries its scopes cover, from a site it +names, at up to 240 requests a minute. + +**Cannot:** write anything. Pause, stop or kill an agent. Change a +policy, a budget, an alert or a cron. Create another key. Read a query +outside its scopes. Be used from an origin it does not name. + +The API is GET only, so it cannot be the target of a cross-origin write +even in principle. + +## About that origin + +Every key must name the sites allowed to use it from a browser. There is +no wildcard, and the flag is not optional. + +This is worth a paragraph, because it is the part people try to skip. +Your ClawMetry runs on `localhost`. Any page in any tab can already *send* +a request to `localhost`. The only reason that has never mattered is that +the browser refuses to let a page *read* the reply from an origin that +did not permit it. An API key with an origin allowlist is how you hand +out that permission one site at a time. A wildcard would hand it to every +site at once, including a page you did not open on purpose. + +For a key used outside a browser (a cron job, a backend, a script), pass +`--origin none`. It gets no CORS header at all, which is correct: nothing +in a browser should be able to use it. + +## Remote and self-hosted ClawMetry + +The API works the same when ClawMetry is not on the caller's machine. +Point at that host instead of `localhost:8900`. The key is the whole +gate: unlike the rest of the dashboard, this surface does not trust a +request just because it came from the local machine. + +If you use ClawMetry Cloud, note that session content is stored +end-to-end encrypted and decrypted in your browser, so the cloud cannot +serve it through a REST API even to you. Point custom UIs at the machine +your agents actually run on. + +## Using MCP instead + +If what you want is an agent that can *ask about* your runs rather than a +page that draws them, the MCP server is a better fit than this API: + +```bash +clawmetry mcp install +``` + +It exposes sessions, cost, traces and health as MCP tools over stdio, so +Claude Code and other MCP clients can query them directly. + +## When it does not work + +**The browser console says "blocked by CORS policy".** The origin your +page is served from is not on the key's list. The message names the +origin the browser sent; create a key with that exact origin, scheme and +port included. `http://localhost:3000` and `http://127.0.0.1:3000` are +different origins. + +**401 with "This API needs a key".** The `Authorization` header is not +arriving. Check for a proxy stripping it, and that the header is +`Authorization: Bearer cmk_...` with the word `Bearer`. + +**401 with "not valid on this machine".** The key was revoked, or it +belongs to another install. `clawmetry key list` shows what this machine +knows about. + +**403 with a scope name in it.** The key is real but does not cover that +query. The message says which scope is needed. + +**429.** More than 240 requests in a minute on one key. Poll less often, +or raise `CLAWMETRY_API_RATE_LIMIT` on the machine running ClawMetry. + +**503 saying the store could not be read.** Usually the sync daemon +restarting. If it persists, run `clawmetry doctor`. + +## Managing keys + +```bash +clawmetry key list # what this machine has issued +clawmetry key list --all # including revoked ones +clawmetry key scopes # what each scope grants +clawmetry key revoke # takes effect on the next request +``` + +The same panel lives in the dashboard under **Security**. + +Keys are stored in `~/.clawmetry/api_keys.json`, mode `0600`, as SHA-256 +hashes. A stolen file cannot be replayed as a key, and a key you lose +cannot be recovered: create a new one and revoke the old. diff --git a/docs/MODULE_MAP.md b/docs/MODULE_MAP.md index 891856251d..fdc1a126ee 100644 --- a/docs/MODULE_MAP.md +++ b/docs/MODULE_MAP.md @@ -4,7 +4,7 @@ > `python3 scripts/gen_module_map.py` (CI fails on drift via > `tests/test_module_map_drift.py`). -250 modules, 82 Flask blueprints. `CLAUDE.md` carries a short curated table of the ones you reach for most often; this is the whole list. +253 modules, 84 Flask blueprints. `CLAUDE.md` carries a short curated table of the ones you reach for most often; this is the whole list. Size bands are deliberately coarse so this file does not churn on every PR: **small** is under 200 lines, **medium** under 1k, **large** under 5k, **huge** is 5k and up. @@ -30,6 +30,7 @@ One module per feature, each owning one or more Flask blueprints. New endpoints | `routes/agentops.py` | small | `bp_agentops` | `/api/agentops`, `/api/ground-truth` | the AgentOps scorecard and the ground-truth endpoint. | | `routes/agents.py` | medium | `bp_agents` | `/api/agents` | Multi-agent adapter endpoints. | | `routes/alerts.py` | large | `bp_alerts`, `bp_budget` | `/api/_harness`, `/api/agents`, `/api/alert-channels`, `/api/alerts`, `/api/budget`, `/api/emergency-stop` | Budget + Alerts endpoints. | +| `routes/apikeys_admin.py` | small | `bp_apikeys_admin` | `/api/apikeys` | create, list and revoke the node's API keys. | | `routes/approval_routing.py` | small | `bp_approval_routing` | `/a`, `/a/decide`, `/api/approvals` | OSS stub after the impl moved to clawmetry-pro. | | `routes/assets.py` | small | `bp_assets` | `/api/assets` | OSS asset registry API. | | `routes/attention.py` | medium | `bp_attention` | `/api/attention`, `/api/hooks` | "which of my agents needs me right now". | @@ -70,6 +71,7 @@ One module per feature, each owning one or more Flask blueprints. New endpoints | `routes/paywall_lifecycle.py` | small | | | the paywall beacons that reach the funnel. | | `routes/plugins.py` | medium | `bp_plugins` | `/api/plugins` | Plugin registry: unified view of installed plugins (#692). | | `routes/policy.py` | medium | `bp_policy` | `/api/approvals`, `/api/approvals-audit`, `/api/policy`, `/api/tool-policy` | tool-policy + sandbox + exec-approval audit (PRD P1-1). | +| `routes/public_api.py` | medium | `bp_public_api` | `/api/q` | the keyed, cross-origin read API custom UIs use. | | `routes/quality.py` | medium | `bp_quality` | `/api/quality` | the Quality tab endpoint. | | `routes/readiness.py` | small | `bp_readiness` | `/api/repo-readiness` | ``bp_readiness`` — repo AI-readiness. | | `routes/reasoning.py` | medium | `bp_reasoning` | `/api/reasoning` | Reasoning chain viewer endpoint. | @@ -139,6 +141,7 @@ The pip-installable package: CLI, sync daemon, DuckDB store, detectors, enforcem | `clawmetry/_paywall_events.py` | large | In-process rolling store for ``POST /api/paywall/event`` client beacons. | | `clawmetry/agentops_metrics.py` | medium | AgentOps window metrics: latency, handoffs, guardrails, review, ground truth. | | `clawmetry/alert_evaluator.py` | large | Local alert-rule evaluator — pure logic, no I/O (PRD #779 PR-D part 2). | +| `clawmetry/apikeys.py` | medium | scoped, revocable read keys for custom UIs. | | `clawmetry/approval_events.py` | small | The public seam between approvals and whoever delivers them. | | `clawmetry/approvals.py` | large | cloud-mediated approval policy engine. | | `clawmetry/attention_hook.py` | small | the `clawmetry hook attention` client. | diff --git a/docs/QUERY_CONTRACT.md b/docs/QUERY_CONTRACT.md index 29b3653174..8135d5683d 100644 --- a/docs/QUERY_CONTRACT.md +++ b/docs/QUERY_CONTRACT.md @@ -26,6 +26,24 @@ test enforces both directions). machine AES-256-GCM encrypted via the sync daemon snapshot path and must never appear on a plaintext push list. +## Read scopes + +Every method declares one scope. An API key issued to a custom UI +(`clawmetry key create --scope read:metrics`) may dispatch only the +methods whose scope it carries, so what a user picks when they create a +key is the same fact the server enforces. + +| Scope | Grants | Methods | +| - | - | - | +| `read:metrics` | Counts, tokens, cost and health. No prompts or replies. | `agent_graph`, `aggregates`, `health`, `models`, `runtimes` | +| `read:sessions` | One row per session: title, model, status, totals. | `rollup_sessions`, `search`, `sessions`, `similar_sessions` | +| `read:traces` | Spans, traces and outbound API calls. | `external_calls`, `spans`, `traces` | +| `read:content` | The turns themselves: prompts, replies, tool calls. | `events`, `replay_events`, `session_context`, `transcript`, `transcript_page` | + +`read:metrics` is exactly the `plaintext` trust class: a metrics-scoped +key can never return a prompt, a reply or a file path. That invariant is +pinned by CI, not by convention. + ## Non-goals * No per-model data in the device-facing `glance` method. Devices get @@ -33,29 +51,29 @@ test enforces both directions). ## Methods -| Method | Status | Trust | Backing | Args | Description | -| - | - | - | - | - | - | -| `agent_graph` | live | plaintext | `query_agent_graph` | `runtime`, `since`, `until`, `limit` (default 500, range 1..2000) | Cross-session agent spawn graph: nodes (agent_type+id stats) + spawn edges. Optional runtime arg scopes to one runtime ('openclaw' matches legacy NULL agent_type). | -| `aggregates` | live | plaintext | `query_aggregates` | `agent_id`, `since`, `until` | Per-day rollup of events/tokens/cost (aggregate counters only). | -| `events` | live | e2e | `query_events` | `session_id`, `agent_id`, `event_type`, `since`, `until`, `limit` (default 200, range 1..5000) | Raw event rows (tool calls, messages, errors), newest first. | -| `external_calls` | live | e2e | `query_external_calls` | `session_id`, `since`, `until`, `limit` (default 200, range 1..2000) | External (non-LLM) API calls captured by the interceptor. | -| `health` | live | plaintext | `health` | (none) | Store health snapshot (engine, size, ring depth, flush age). | -| `models` | live | plaintext | `query_rollup_model_daily` | `runtime`, `since`, `until`, `limit` (default 1000, range 1..10000) | Per-model daily token/cost rollup across runtimes. | -| `replay_events` | live | e2e | `query_replay_events` | `session_id` (required), `limit` (default 2000, range 1..10000) | Canonical replay-event rows for one session (#4813). Rows in kind-agnostic order; the /api/replay-tree endpoint groups them into turns/delegations/workflows/approvals. | -| `rollup_sessions` | live | e2e | `query_rollup_sessions` | `runtime`, `limit` (default 200, range 1..2000) | Per-session materialized summary (title, status, totals, stuck flag). | -| `runtimes` | live | plaintext | `query_rollup_runtime_daily` | `since`, `until`, `limit` (default 1000, range 1..10000) | Per-runtime daily activity/cost rollup (claude_code, openclaw, ...). | -| `search` | live | e2e | `query_search` | `q` (required), `model`, `status`, `since`, `until`, `limit` (default 50, range 1..500) | Full-text search over session titles and eval reasons. | -| `session_context` | live | e2e | `query_session_context` | `session_id` (required), `agent_type`, `limit` (default 200, range 1..1000) | Inputs & context rows for one session: system prompt, first user prompt, tool definitions, MCP servers, context files and runtime setup captured from context.compiled events. Content is redacted + capped; sha256/size describe the full text. | -| `sessions` | live | e2e | `query_sessions` | `agent_id`, `since`, `until`, `limit` (default 100, range 1..2000) | One row per session_id with start/end, event count, cost. | -| `similar_sessions` | live | e2e | `query_similar_sessions` | `session_id` (required), `window_days` (default 30, range 1..365), `limit` (default 10, range 1..50) | Runs shaped like this one (WO-60): nearest sessions by tool-call n-gram similarity inside a window, same runtime first, with score, runtime, model, cost, outcome. Carries session titles, so content class. | -| `spans` | live | e2e | `query_spans` | `trace_id`, `session_id`, `agent_type`, `since`, `until`, `limit` (default 200, range 1..2000) | OTel span rows with full filters (trace/session/agent/time). | -| `traces` | live | e2e | `query_traces` | `session_id`, `agent_type`, `since`, `until`, `limit` (default 100, range 1..1000) | One row per trace_id with aggregate span stats. | -| `transcript` | live | e2e | `query_events` | `session_id` (required), `limit` (default 500, range 1..5000) | Alias of events scoped to one required session_id. | -| `transcript_page` | live | e2e | `query_transcript_page` | `session_id` (required), `before_ts`, `limit` (default 150, range 1..250) | One older-history page of a session's events, newest-first. before_ts is an exclusive ms-epoch cursor (pass the previous page's next_before_ts to walk backward). Returns {rows, count, has_more, next_before_ts}. | -| `approvals` | planned | plaintext | `query_approvals` | `status`, `limit` (default 100, range 1..1000) | Approval queue metadata (ids, states, timestamps; no content). | -| `brain` | planned | e2e | `query_events` | `session_id`, `since`, `limit` (default 200, range 1..2000) | Reasoning/tool event slice powering the Brain feed. | -| `glance` | planned | plaintext | `rollup_glance` | (none) | Device-facing top-line counters (sessions, cost, alerts). Non-goal: no per-model data in glance. | -| `session` | planned | e2e | `query_sessions_table` | `session_id` (required) | Single-session detail row (title, status, outcome, totals). | -| `usage` | planned | plaintext | `rollup_usage_daily` | `runtime`, `since`, `until` | Daily token/cost usage series (input/output/cache splits). | +| Method | Status | Trust | Scope | Backing | Args | Description | +| - | - | - | - | - | - | - | +| `agent_graph` | live | plaintext | `read:metrics` | `query_agent_graph` | `runtime`, `since`, `until`, `limit` (default 500, range 1..2000) | Cross-session agent spawn graph: nodes (agent_type+id stats) + spawn edges. Optional runtime arg scopes to one runtime ('openclaw' matches legacy NULL agent_type). | +| `aggregates` | live | plaintext | `read:metrics` | `query_aggregates` | `agent_id`, `since`, `until` | Per-day rollup of events/tokens/cost (aggregate counters only). | +| `events` | live | e2e | `read:content` | `query_events` | `session_id`, `agent_id`, `event_type`, `since`, `until`, `limit` (default 200, range 1..5000) | Raw event rows (tool calls, messages, errors), newest first. | +| `external_calls` | live | e2e | `read:traces` | `query_external_calls` | `session_id`, `since`, `until`, `limit` (default 200, range 1..2000) | External (non-LLM) API calls captured by the interceptor. | +| `health` | live | plaintext | `read:metrics` | `health` | (none) | Store health snapshot (engine, size, ring depth, flush age). | +| `models` | live | plaintext | `read:metrics` | `query_rollup_model_daily` | `runtime`, `since`, `until`, `limit` (default 1000, range 1..10000) | Per-model daily token/cost rollup across runtimes. | +| `replay_events` | live | e2e | `read:content` | `query_replay_events` | `session_id` (required), `limit` (default 2000, range 1..10000) | Canonical replay-event rows for one session (#4813). Rows in kind-agnostic order; the /api/replay-tree endpoint groups them into turns/delegations/workflows/approvals. | +| `rollup_sessions` | live | e2e | `read:sessions` | `query_rollup_sessions` | `runtime`, `limit` (default 200, range 1..2000) | Per-session materialized summary (title, status, totals, stuck flag). | +| `runtimes` | live | plaintext | `read:metrics` | `query_rollup_runtime_daily` | `since`, `until`, `limit` (default 1000, range 1..10000) | Per-runtime daily activity/cost rollup (claude_code, openclaw, ...). | +| `search` | live | e2e | `read:sessions` | `query_search` | `q` (required), `model`, `status`, `since`, `until`, `limit` (default 50, range 1..500) | Full-text search over session titles and eval reasons. | +| `session_context` | live | e2e | `read:content` | `query_session_context` | `session_id` (required), `agent_type`, `limit` (default 200, range 1..1000) | Inputs & context rows for one session: system prompt, first user prompt, tool definitions, MCP servers, context files and runtime setup captured from context.compiled events. Content is redacted + capped; sha256/size describe the full text. | +| `sessions` | live | e2e | `read:sessions` | `query_sessions` | `agent_id`, `since`, `until`, `limit` (default 100, range 1..2000) | One row per session_id with start/end, event count, cost. | +| `similar_sessions` | live | e2e | `read:sessions` | `query_similar_sessions` | `session_id` (required), `window_days` (default 30, range 1..365), `limit` (default 10, range 1..50) | Runs shaped like this one (WO-60): nearest sessions by tool-call n-gram similarity inside a window, same runtime first, with score, runtime, model, cost, outcome. Carries session titles, so content class. | +| `spans` | live | e2e | `read:traces` | `query_spans` | `trace_id`, `session_id`, `agent_type`, `since`, `until`, `limit` (default 200, range 1..2000) | OTel span rows with full filters (trace/session/agent/time). | +| `traces` | live | e2e | `read:traces` | `query_traces` | `session_id`, `agent_type`, `since`, `until`, `limit` (default 100, range 1..1000) | One row per trace_id with aggregate span stats. | +| `transcript` | live | e2e | `read:content` | `query_events` | `session_id` (required), `limit` (default 500, range 1..5000) | Alias of events scoped to one required session_id. | +| `transcript_page` | live | e2e | `read:content` | `query_transcript_page` | `session_id` (required), `before_ts`, `limit` (default 150, range 1..250) | One older-history page of a session's events, newest-first. before_ts is an exclusive ms-epoch cursor (pass the previous page's next_before_ts to walk backward). Returns {rows, count, has_more, next_before_ts}. | +| `approvals` | planned | plaintext | `read:metrics` | `query_approvals` | `status`, `limit` (default 100, range 1..1000) | Approval queue metadata (ids, states, timestamps; no content). | +| `brain` | planned | e2e | `read:content` | `query_events` | `session_id`, `since`, `limit` (default 200, range 1..2000) | Reasoning/tool event slice powering the Brain feed. | +| `glance` | planned | plaintext | `read:metrics` | `rollup_glance` | (none) | Device-facing top-line counters (sessions, cost, alerts). Non-goal: no per-model data in glance. | +| `session` | planned | e2e | `read:sessions` | `query_sessions_table` | `session_id` (required) | Single-session detail row (title, status, outcome, totals). | +| `usage` | planned | plaintext | `read:metrics` | `rollup_usage_daily` | `runtime`, `since`, `until` | Daily token/cost usage series (input/output/cache splits). | Live methods: 17. Planned methods: 5. diff --git a/examples/custom-ui/README.md b/examples/custom-ui/README.md new file mode 100644 index 0000000000..6c4dde1132 --- /dev/null +++ b/examples/custom-ui/README.md @@ -0,0 +1,43 @@ +# Starter custom UI + +One HTML file, no build step, no dependencies. It reads the ClawMetry +`q/1` API and draws 30 days of agent spend by day, runtime and model. + +It exists to be edited. Open it, change it, or hand it to a coding agent +along with the API guide and ask for the view you actually want. + +## Run it + +```bash +# 1. serve this directory on a port +cd examples/custom-ui && python3 -m http.server 3000 + +# 2. create a key that names that origin +clawmetry key create --name starter \ + --scope read:metrics --origin http://localhost:3000 + +# 3. open http://localhost:3000 and paste the key +``` + +The key is kept in this browser's local storage and sent to your own +ClawMetry, nowhere else. + +## Why `read:metrics` + +This page has no backend, so any key it holds lives in the browser. +`read:metrics` is the scope that covers counts, tokens, cost and health +and cannot return a prompt, a reply or a file path. If you extend the +page into something that needs session content, put a backend in front of +it and keep the wider key there. + +## Making it yours + +The API describes itself. Fetch the guide with your key and give it to a +coding agent with a description of what you want: + +```bash +curl -H "Authorization: Bearer cmk_..." \ + http://localhost:8900/api/q/1/llms.txt +``` + +Full walkthrough: [`docs/BUILD_YOUR_OWN_UI.md`](../../docs/BUILD_YOUR_OWN_UI.md). diff --git a/examples/custom-ui/index.html b/examples/custom-ui/index.html new file mode 100644 index 0000000000..3b608ce654 --- /dev/null +++ b/examples/custom-ui/index.html @@ -0,0 +1,323 @@ + + + + + +Agent spend + + + + +
+

Agent spend

+
Reading ClawMetry directly. Nothing leaves this machine.
+ + + + + + +
+ + + + diff --git a/routes/apikeys_admin.py b/routes/apikeys_admin.py new file mode 100644 index 0000000000..ada680fa2a --- /dev/null +++ b/routes/apikeys_admin.py @@ -0,0 +1,140 @@ +"""routes/apikeys_admin.py -- create, list and revoke the node's API keys. + +Requirement: "Build your own UI: a keyed, scoped read API for custom +dashboards" (64c10afd-038d-4fde-9c55-ddca80aaff1e), blueprint +0ea7523c-12b5-4033-84ea-bf1f46e20d70, "API Surface" -> "Key management". + + GET /api/apikeys this node's keys + the scope catalogue + POST /api/apikeys mint one; the secret is in the response + DELETE /api/apikeys/ revoke one + +These MINT and REVOKE credentials for ``routes/public_api.py``, which is the +cross-origin surface those credentials open. That relationship is the whole +reason they live apart: + +* **This module is behind the dashboard's own gate.** ``dashboard.py``'s + ``_cross_origin_write_blocked`` refuses a cross-origin POST/DELETE to any + ``/api/*`` path, and ``public_api._add_cors`` is pinned to ``/api/q/`` so no + CORS header ever reaches these routes. A page holding a read key can + therefore never list the node's keys, and never issue itself a better one. +* **It is its own blueprint, not a few functions on ``bp_security``.** Both + placements are equally safe, since the guards above are path-based rather + than blueprint-based. This one is legible: credential management is a + distinct concern from the security tab's scanners, and a reader (human or + tool) sees the whole surface in one short file instead of at 97% of a + 3,000-line module. + +Nothing here reads agent data. The keys it manages are read-only by +construction; see ``clawmetry/apikeys.py`` for what they can and cannot do. +""" + +from __future__ import annotations + +import logging + +from flask import Blueprint, jsonify, request + +logger = logging.getLogger("clawmetry.routes.apikeys_admin") + +bp_apikeys_admin = Blueprint("apikeys_admin", __name__) + + +@bp_apikeys_admin.route("/api/apikeys", methods=["GET"]) +def api_keys_list(): + """This node's API keys, plus the scope catalogue the UI renders. + + Secrets are never included: only a SHA-256 is stored, and even that is + stripped by ``apikeys.list_keys``. + """ + from clawmetry import apikeys as _ak + try: + return jsonify({ + "ok": True, + "keys": _ak.list_keys(include_revoked=True), + "scopes": _ak.scope_catalogue(), + "summary": _ak.store_summary(), + }) + except Exception as exc: + logger.warning("apikeys list failed: %s", exc) + return jsonify({ + "ok": False, + "keys": [], + "scopes": [], + "error": "ClawMetry could not read its key file. Check that " + "~/.clawmetry is readable by you.", + }), 200 + + +@bp_apikeys_admin.route("/api/apikeys", methods=["POST"]) +def api_keys_create(): + """Mint a key. The secret is in this response and nowhere else, ever.""" + from clawmetry import apikeys as _ak + body = request.get_json(silent=True) or {} + origins = body.get("origins") or [] + if isinstance(origins, str): + origins = [o.strip() for o in origins.replace(",", " ").split() if o.strip()] + scopes = body.get("scopes") or [] + if isinstance(scopes, str): + scopes = [s.strip() for s in scopes.replace(",", " ").split() if s.strip()] + browser = bool(body.get("browser", True)) + if browser and not origins: + return jsonify({ + "ok": False, + "error": "Name the site that will use this key, for example " + "http://localhost:3000. There is no wildcard: any page " + "in any tab can already reach this machine, and the " + "origin list is what stops it reading the answer.", + }), 400 + try: + record, plaintext = _ak.create( + body.get("name") or "", + scopes, + [] if not browser else origins, + note=body.get("note") or "", + ) + except _ak.ApiKeyError as exc: + # The sentence comes from apikeys.REFUSAL_REASONS, keyed by the + # refusal code, NOT from str(exc). The CLI does print the exception + # (it names the offending value, which is worth more than it costs + # in a terminal); an HTTP response must not carry exception-derived + # text to a caller who may not be the operator. + return jsonify({ + "ok": False, + "reason": exc.reason, + "error": _ak.message_for(exc.reason), + }), 400 + except Exception as exc: + logger.warning("apikeys create failed: %s", exc) + return jsonify({ + "ok": False, + "error": "ClawMetry could not write its key file. Check that " + "~/.clawmetry is writable by you.", + }), 500 + return jsonify({ + "ok": True, + "key": plaintext, + "record": {k: v for k, v in record.items() if k != "hash"}, + }) + + +@bp_apikeys_admin.route("/api/apikeys/", methods=["DELETE"]) +def api_keys_revoke(key_id: str): + """Revoke a key. Takes effect on that key's next request.""" + from clawmetry import apikeys as _ak + try: + ok = _ak.revoke(key_id) + except Exception as exc: + logger.warning("apikeys revoke failed: %s", exc) + return jsonify({ + "ok": False, + "error": "ClawMetry could not write its key file. Check that " + "~/.clawmetry is writable by you.", + }), 500 + if not ok: + return jsonify({ + "ok": False, + "error": "There is no active key with that id on this machine.", + }), 404 + # The caller supplied key_id in the URL; echoing it back would reflect + # user-controlled input into the response body (CodeQL CWE-79). + return jsonify({"ok": True}) diff --git a/routes/public_api.py b/routes/public_api.py new file mode 100644 index 0000000000..ad1dd7e000 --- /dev/null +++ b/routes/public_api.py @@ -0,0 +1,483 @@ +"""routes/public_api.py -- the keyed, cross-origin read API custom UIs use. + +Requirement: "Build your own UI: a keyed, scoped read API for custom +dashboards" (64c10afd-038d-4fde-9c55-ddca80aaff1e), blueprint +0ea7523c-12b5-4033-84ea-bf1f46e20d70. + +This is the "build your own UI" surface (docs/BUILD_YOUR_OWN_UI.md). It +serves the declared ``q/1`` query contract to anything holding a scoped +API key: a page vibe-coded on v0 or Lovable, a Grafana-ish panel, a +terminal script, an agent with curl. + + GET /api/q/1 what this key can read + GET /api/q/1/llms.txt the whole API, written for an agent + GET /api/q/1/? one query + +Every response is JSON. Every shape, arg and default comes from +``clawmetry/query_contract.py`` and every query goes through +``routes.local_query._dispatch`` -- the same code path the dashboard +uses, so there is no second SQL surface to keep correct and no way for a +key to reach a query the contract does not declare. + +What makes this different from the rest of the dashboard +-------------------------------------------------------- +Everywhere else, a request from ``127.0.0.1`` is trusted, because a +local tool talking to itself is the normal case. Here it is not, and +that inversion is the entire security design: + +* **A key is always required.** Loopback earns nothing. Without a key + this surface behaves as if it does not exist. +* **CORS is per key, never global.** ``Access-Control-Allow-Origin`` is + echoed only for an origin the key holder named when they created the + key. There is no wildcard and no way to ask for one. This matters more + than it looks: any page in any tab can already send a request to + ``127.0.0.1:8900``, and the only reason that has been harmless is that + the browser will not let the page read the reply. That protection is + what we are selectively removing, one named origin at a time. +* **Read only.** The dispatch table is the q/1 read contract. Nothing + here can pause, stop or kill an agent, and nothing here writes. This + adds no entry to the control-plane surfaces listed in CLAUDE.md: a key + cannot cause a write, so the "no surprise writes" rule has nothing to + bite on here. +* **GET only.** Every q/1 arg is a scalar, so nothing needs a body. + Being GET-only means this API can never be the target of a + cross-origin write, which removes a whole class of question. +""" + +from __future__ import annotations + +import logging +import os +import re +import time +from collections import deque +from flask import Blueprint, Response, jsonify, request + +from clawmetry import apikeys +from clawmetry.query_contract import ( + CONTRACT_VERSION, + QUERY_CONTRACT, + SCOPE_DOC, + SCOPES, + STATUS_LIVE, +) + +logger = logging.getLogger("clawmetry.routes.public_api") + +bp_public_api = Blueprint("public_api", __name__) + +#: Requests per key per minute. Generous enough for a page polling every +#: second with a few panels, low enough that a runaway loop in someone's +#: custom UI cannot pin the daemon's DuckDB connection (see the CPU +#: budget in FLYWHEEL.md 1e). +RATE_LIMIT_PER_MIN = int(os.environ.get("CLAWMETRY_API_RATE_LIMIT", "240") or 240) + +_RATE: dict = {} +_RATE_WINDOW_SEC = 60.0 + +#: Flask stashes the resolved key record here so ``_add_cors`` can echo +#: the right origin after the view has run. +_G_KEY = "_cm_api_key_record" + +# Structural guard for browser Origin values: scheme://host[:port]. +# Applied in _add_cors BEFORE comparing against stored origins so that +# structurally invalid values are rejected early. +_ORIGIN_RE = re.compile(r"^https?://[A-Za-z0-9._-]+(:\d{1,5})?$") + + +def _rate_limited(key_id: str) -> bool: + """True when this key has spent its minute. Sliding window, in memory. + + Per process and not shared with the daemon, which is the honest + scope: this is a runaway-loop guard for a local API, not a billing + meter. + """ + if RATE_LIMIT_PER_MIN <= 0: + return False + now = time.monotonic() + hits = _RATE.setdefault(key_id, deque()) + cutoff = now - _RATE_WINDOW_SEC + while hits and hits[0] < cutoff: + hits.popleft() + if len(hits) >= RATE_LIMIT_PER_MIN: + return True + hits.append(now) + return False + + +def _err(status: int, message: str, **extra): + """A failure a person can act on. + + Never an upstream code on its own: the body always carries a + sentence saying what happened and what to do next, because these + land in someone's browser console while they are building. + """ + body = {"error": message, "contract": CONTRACT_VERSION} + body.update(extra) + return jsonify(body), status # codeql[py/stack-trace-exposure] + + +# -- auth + CORS --------------------------------------------------------- + +def _presented_key() -> str: + """The key on this request. ``Authorization: Bearer`` is the documented + form; ``X-ClawMetry-Key`` exists for clients that cannot set it.""" + auth = (request.headers.get("Authorization") or "").strip() + if auth.lower().startswith("bearer "): + return auth[7:].strip() + return (request.headers.get("X-ClawMetry-Key") or "").strip() + + +def _authenticate(): + """``(record, None)`` on success, ``(None, response)`` on failure.""" + presented = _presented_key() + if not presented: + return None, _err( + 401, + "This API needs a key. Create one with: clawmetry key create " + "--name my-ui --scope read:metrics --origin https://example.com, " + "then send it as: Authorization: Bearer cmk_...", + docs="/api/q/1/llms.txt", + ) + record = apikeys.verify(presented) + if not record: + return None, _err( + 401, + "That key is not valid on this machine. It may have been revoked, " + "or it may belong to a different ClawMetry install. List the keys " + "this machine knows with: clawmetry key list", + docs="/api/q/1/llms.txt", + ) + if _rate_limited(str(record.get("id"))): + return None, _err( + 429, + f"This key has made more than {RATE_LIMIT_PER_MIN} requests in the " + "last minute and is being throttled. Poll less often, or raise " + "CLAWMETRY_API_RATE_LIMIT on the machine running ClawMetry.", + ) + from flask import g + + setattr(g, _G_KEY, record) + return record, None + + +@bp_public_api.after_request +def _add_cors(response): + """Echo CORS headers for an origin THIS key authorised, and no other. + + Blueprint-scoped on purpose: nothing else in the dashboard gains a + CORS header from this file existing. + + CWE-113 design: the ACAO header value always comes from the + file-backed all_live_origins() store, never from request headers. + The request Origin is used only as a search key; list.index() returns + an integer (untainted), and list[integer] retrieves the stored string, + so no user-controlled data enters the response header. + """ + from flask import g + + origin = (request.headers.get("Origin") or "").strip() + if not origin: + return response # not a browser; nothing to negotiate + if not (request.path or "").startswith("/api/q/"): + # Belt and braces. Every rule in this blueprint is under /api/q/ + # today, and this makes sure a route added here later cannot + # inherit cross-origin readability by accident. Key MANAGEMENT + # (minting, listing, revoking) deliberately lives in its own + # module, routes/apikeys_admin.py, behind the dashboard's own + # same-origin gate. + return response + + # Structural guard: reject structurally invalid origin values before + # any comparison with stored data. This is an early exit, not the + # CWE-113 sanitizer (the sanitizer is the stored-value lookup below). + _m = _ORIGIN_RE.fullmatch(origin.rstrip("/")) + if not _m: + return response + + record = getattr(g, _G_KEY, None) + + # Per-key gate: when a key was authenticated, restrict to that key's + # named origins. This check is BOOLEAN ONLY -- the result is never + # assigned to the response header, so no taint can flow through it. + if record is not None: + _safe_lc = _m.group(0).lower() + if _safe_lc not in [ + str(_o).rstrip("/").lower() for _o in (record.get("origins") or []) + ]: + return response + + # Resolve the ACAO header value from the file-backed origin store. + # CWE-113: _norm (tainted from Origin header) is used only as the search + # key against a list of stored strings. list.index() returns an integer + # (integers are never tainted in CodeQL's model), and list[integer] reads a + # stored value -- so no user-controlled data reaches the response header. + _norm = _m.group(0).rstrip("/").lower() + _stored = list(apikeys.all_live_origins()) + _stored_lc = [str(_o).rstrip("/").lower() for _o in _stored] + try: + _idx = _stored_lc.index(_norm) + except ValueError: + return response + + response.headers["Access-Control-Allow-Origin"] = str(_stored[_idx]) + response.headers["Vary"] = "Origin" + response.headers["Access-Control-Allow-Methods"] = "GET, OPTIONS" + response.headers["Access-Control-Allow-Headers"] = ( + "Authorization, X-ClawMetry-Key, Content-Type" + ) + response.headers["Access-Control-Max-Age"] = "600" + return response + + +# CORS preflight needs no view of its own: Flask answers OPTIONS for every +# rule below automatically, and ``_add_cors`` decides whether that answer +# carries permission. A preflight arrives with no Authorization header, so +# the only question it can answer is "has the user authorised this origin +# for any live key" -- and an origin nobody named gets no header, which is +# what makes the browser abandon the request before it is ever sent. + + +# -- the index ----------------------------------------------------------- + +def _shape_spec(name: str) -> dict: + spec = QUERY_CONTRACT[name] + return { + "shape": name, + "scope": spec["scope"], + "description": spec["doc"], + "args": { + arg: dict(meta, required=bool(meta.get("required"))) + for arg, meta in spec["args"].items() + }, + "url": f"/api/q/1/{name}", + } + + +@bp_public_api.route("/api/q/1", methods=["GET"]) +def q_index(): + """What this key can read. The first call a custom UI should make.""" + record, failure = _authenticate() + if failure: + return failure + granted = sorted(apikeys.granted_shapes(record)) + return jsonify({ + "contract": CONTRACT_VERSION, + "key": { + "id": record.get("id"), + "name": record.get("name"), + "scopes": record.get("scopes") or [], + "origins": record.get("origins") or [], + }, + "scopes": [ + {"scope": s, "grants": SCOPE_DOC[s], "held": s in (record.get("scopes") or [])} + for s in SCOPES + ], + "shapes": [_shape_spec(n) for n in granted], + "docs": "/api/q/1/llms.txt", + }) + + +# -- the agent-readable guide -------------------------------------------- + +def _llms_txt(record: dict) -> str: + """The whole API as plain text, generated from the contract. + + Written to be pasted into a coding agent. It is generated rather + than authored so it can never describe a shape that is not served, + and it is scoped to the presented key so an agent is never told + about a query it will get a 403 for. + """ + granted = sorted(apikeys.granted_shapes(record)) + # Use a hardcoded base URL so no request-derived taint reaches the + # response body. ClawMetry runs on loopback by default; the port is + # stable enough to note here without misleading callers. + host = "http://127.0.0.1:8900" + lines = [ + "# ClawMetry query API (%s)" % CONTRACT_VERSION, + "", + "Read-only telemetry for AI agent runs on this machine: sessions,", + "tokens, cost, tool calls, traces. Use it to build a custom UI.", + "", + "## Auth", + "", + "Send the key on every request:", + "", + " Authorization: Bearer ", + "", + "All requests are GET. All responses are JSON. Errors are", + '{"error": ""} with a 4xx status.', + "", + "## Base URL", + "", + " %s/api/q/1" % host, + "", + "## Response shape", + "", + "Row-returning queries answer:", + "", + ' {"shape": "sessions", "rows": [...], "count": 12,', + ' "contract": "q/1", "elapsed_ms": 8}', + "", + "`health`, `agent_graph`, `transcript_page` and `similar_sessions`", + "answer an object instead of `rows`. Read `shape` to tell them apart.", + "", + "## Queries this key can make", + "", + ] + for name in granted: + spec = QUERY_CONTRACT[name] + lines.append("### GET /api/q/1/%s" % name) + lines.append("") + lines.append(spec["doc"]) + lines.append("") + if spec["args"]: + lines.append("Query parameters:") + for arg, meta in spec["args"].items(): + bits = [] + if meta.get("required"): + bits.append("required") + if "default" in meta: + bits.append("default %s" % meta["default"]) + if "lo" in meta and "hi" in meta: + bits.append("%s..%s" % (meta["lo"], meta["hi"])) + suffix = (" (%s)" % ", ".join(bits)) if bits else "" + lines.append(" - %s%s" % (arg, suffix)) + else: + lines.append("No parameters.") + lines.append("") + missing = [s for s in SCOPES if s not in (record.get("scopes") or [])] + if missing: + lines += [ + "## Not available to this key", + "", + "This key does not hold:", + "", + ] + for s in missing: + lines.append(" - %s: %s" % (s, SCOPE_DOC[s])) + lines += [ + "", + "The person running ClawMetry can issue a key with more scopes:", + "", + " clawmetry key create --name my-ui --scope %s --origin " + % " --scope ".join(list((record.get("scopes") or [])) + missing[:1]), + "", + ] + lines += [ + "## Notes", + "", + "- Timestamps in `since` / `until` are ISO 8601, e.g. 2026-09-01T00:00:00Z.", + "- `limit` is clamped to the range shown; asking for more is not an error.", + "- On a free plan, `events` returns the last 24 hours and the response", + " carries `capped_at_24h: true`. Other queries are not time-capped.", + "- The key is a secret. Put it in a server-side env var where you can;", + " if it must live in the browser, scope it to `read:metrics`.", + "", + ] + return "\n".join(lines) + + +@bp_public_api.route("/api/q/1/llms.txt", methods=["GET"]) +def q_llms_txt(): + """The API, written for a coding agent to read in one pass.""" + record, failure = _authenticate() + if failure: + return failure + return Response(_llms_txt(record), mimetype="text/plain; charset=utf-8") + + +# -- the query ----------------------------------------------------------- + +@bp_public_api.route("/api/q/1/", methods=["GET"]) +def q_shape(shape: str): + """Run one declared q/1 query and return its rows.""" + record, failure = _authenticate() + if failure: + return failure + + spec = QUERY_CONTRACT.get(shape) + if spec is None or spec["status"] != STATUS_LIVE: + # A planned-but-unserved shape and a typo get the same answer on + # purpose: the caller's next step is identical either way. + # Do NOT reflect `shape` here -- it is unvalidated user input at this + # point (it was not found in the contract), so echoing it is a + # reflected-content sink. Direct the caller to GET /api/q/1 instead. + return _err( + 404, + "There is no such query. Ask GET /api/q/1 for the list " + "this key can run.", + docs="/api/q/1/llms.txt", + ) + if shape not in apikeys.granted_shapes(record): + needed = spec["scope"] + # Do NOT reflect `shape` (URL input) in the response body. + # The required_scope and held_scopes fields carry enough to act on. + return _err( + 403, + f"This key lacks the {needed!r} scope required for this query " + f"({SCOPE_DOC[needed]}) and holds " + f"{', '.join(record.get('scopes') or []) or 'none'}. Issue a new " + f"key with: clawmetry key create --name my-ui --scope {needed} " + "--origin ", + required_scope=needed, + held_scopes=record.get("scopes") or [], + ) + + from routes import local_query as _lq + + try: + args = _lq._coerce_args(shape, request.args.to_dict()) + except ValueError: + # _coerce_args raises only for a missing required argument. The + # sentence is built from the CONTRACT rather than from the + # exception: the contract already declares which arguments are + # required, so the message is both more consistent across queries + # and free of exception-derived text reaching a caller. + needed = [a for a, m in spec["args"].items() if m.get("required")] + missing = [a for a in needed + if not (request.args.get(a) or "").strip()] or needed + # Do NOT reflect `shape` (URL input) in the response body. + return _err( + 400, + f"Missing required argument(s): {', '.join(missing)}. " + "Ask GET /api/q/1 for the full argument list.", + missing_args=missing, + ) + + capped = False + if shape == "events": + # Same retention cap the dashboard's own /api/local/events applies + # (issue #1448): free plans see the last 24h of raw events. Doing + # it here too means a custom UI and the dashboard never disagree + # about how much history exists. + capped = _lq._apply_24h_cap(args) + + started = time.monotonic() + try: + body = _lq._dispatch(shape, args) + except Exception as exc: + # `shape` came off the URL. It has been validated against the + # contract by now, but the validated thing to log is the contract's + # OWN key rather than the request string that matched it: a log line + # built from request text is a log-injection sink even when the + # value turned out to be legitimate. + logger.warning("public api: %s failed for key %s: %s", + QUERY_CONTRACT[shape]["backing"], record.get("id"), exc) + # The upstream message can carry a DuckDB path or a column name. + # Neither helps the person building a UI, and both are ours. + return _err( + 503, + "ClawMetry could not read its local store just now. This is " + "usually the sync daemon restarting; try again in a moment. If it " + "persists, run: clawmetry doctor", + ) + + out = {k: v for k, v in body.items() if not k.startswith("_")} + out["shape"] = shape + out["contract"] = CONTRACT_VERSION + out["elapsed_ms"] = int((time.monotonic() - started) * 1000) + if shape == "events": + out["capped_at_24h"] = capped + apikeys.touch(str(record.get("id"))) + return jsonify(out) diff --git a/scripts/gen_query_contract_doc.py b/scripts/gen_query_contract_doc.py index 690682322c..4c8ff47e51 100644 --- a/scripts/gen_query_contract_doc.py +++ b/scripts/gen_query_contract_doc.py @@ -21,6 +21,8 @@ from clawmetry.query_contract import ( # noqa: E402 CONTRACT_VERSION, QUERY_CONTRACT, + SCOPE_DOC, + SCOPES, STATUS_LIVE, ) @@ -54,6 +56,19 @@ machine AES-256-GCM encrypted via the sync daemon snapshot path and must never appear on a plaintext push list. +## Read scopes + +Every method declares one scope. An API key issued to a custom UI +(`clawmetry key create --scope read:metrics`) may dispatch only the +methods whose scope it carries, so what a user picks when they create a +key is the same fact the server enforces. + +{{scope_table}} + +`read:metrics` is exactly the `plaintext` trust class: a metrics-scoped +key can never return a prompt, a reply or a file path. That invariant is +pinned by CI, not by convention. + ## Non-goals * No per-model data in the device-facing `glance` method. Devices get @@ -63,6 +78,21 @@ """ +def _scope_table() -> str: + rows = ["| Scope | Grants | Methods |", "| - | - | - |"] + for scope in SCOPES: + served = sorted( + n for n, s in QUERY_CONTRACT.items() + if s["scope"] == scope and s["status"] == STATUS_LIVE + ) + rows.append( + f"| `{scope}` | {SCOPE_DOC[scope]} | " + + ", ".join(f"`{n}`" for n in served) + + " |" + ) + return "\n".join(rows) + + def _fmt_arg(name: str, spec: dict) -> str: bits = [] if spec.get("required"): @@ -75,15 +105,15 @@ def _fmt_arg(name: str, spec: dict) -> str: def render() -> str: - lines = [_HEADER] - lines.append("| Method | Status | Trust | Backing | Args | Description |") - lines.append("| - | - | - | - | - | - |") + lines = [_HEADER.replace("{scope_table}", _scope_table())] + lines.append("| Method | Status | Trust | Scope | Backing | Args | Description |") + lines.append("| - | - | - | - | - | - | - |") for name in sorted(QUERY_CONTRACT, key=lambda n: (QUERY_CONTRACT[n]["status"] != STATUS_LIVE, n)): spec = QUERY_CONTRACT[name] args = ", ".join(_fmt_arg(a, s) for a, s in spec["args"].items()) or "(none)" lines.append( f"| `{name}` | {spec['status']} | {spec['trust']} | " - f"`{spec['backing']}` | {args} | {spec['doc']} |" + f"`{spec['scope']}` | `{spec['backing']}` | {args} | {spec['doc']} |" ) lines.append("") live = [n for n, s in QUERY_CONTRACT.items() if s["status"] == STATUS_LIVE] diff --git a/tests/test_public_api_keys.py b/tests/test_public_api_keys.py new file mode 100644 index 0000000000..4f83d744b8 --- /dev/null +++ b/tests/test_public_api_keys.py @@ -0,0 +1,563 @@ +"""Guards for the keyed read API custom UIs are built on. + +Three things are being protected here, in descending order of how badly +they would hurt if they broke: + +1. **The CORS gate.** ``/api/q/`` is the only surface in ClawMetry that + answers a cross-origin browser read. If ``Access-Control-Allow-Origin`` + ever went out for an origin the key did not name, any page in any tab + could read the machine's telemetry. Several tests here exist purely to + fail loudly if that becomes possible. +2. **The scope gate.** A key must reach exactly the queries its scopes + cover. ``read:metrics`` in particular must never be able to return a + prompt or a reply, which is pinned to the contract's own trust class + rather than to a hand-kept list. +3. **Key handling.** Secrets hashed and never stored in the clear, the + file 0600, revocation effective on the next request. + +The management endpoints (mint / list / revoke) are checked too, because +they live on the ordinary dashboard blueprint on purpose and must stay +there: a read key must never be able to issue itself a better one. +""" +from __future__ import annotations + +import importlib +import json +import os +import stat + +import pytest + + +# ── fixtures ──────────────────────────────────────────────────────────── + +@pytest.fixture +def ak(monkeypatch, tmp_path): + """The apikeys module pointed at a throwaway store. + + tests/ has burned this repo before by writing into the operator's real + ~/.clawmetry, so the path override is set before the module is used + and asserted below. + """ + store = tmp_path / "api_keys.json" + monkeypatch.setenv("CLAWMETRY_API_KEYS_PATH", str(store)) + import clawmetry.apikeys as mod + + importlib.reload(mod) + assert mod._store_path() == str(store) + return mod + + +@pytest.fixture +def client(ak, monkeypatch): + """A Flask test client with just the two blueprints under test. + + Deliberately NOT the whole dashboard app: booting it drags in DuckDB + and the runtime probes, and the thing being tested is the auth and + CORS behaviour of these routes, which is entirely self-contained. + ``_dispatch`` is stubbed so no test needs a populated store. + """ + from flask import Flask + + import routes.local_query as lq + import routes.public_api as pub + + importlib.reload(pub) + + calls = [] + + def _fake_dispatch(shape, args): + calls.append((shape, dict(args))) + return {"rows": [{"shape_echo": shape}], "count": 1, + "_via": "test", "_shape": shape} + + monkeypatch.setattr(lq, "_dispatch", _fake_dispatch) + monkeypatch.setattr(lq, "_apply_24h_cap", lambda args: False) + + app = Flask(__name__) + app.register_blueprint(pub.bp_public_api) + c = app.test_client() + c.dispatch_calls = calls + c.pub = pub + return c + + +def _mint(ak, name="t", scopes=("read:metrics",), origins=("http://localhost:3000",)): + _, plaintext = ak.create(name, list(scopes), list(origins)) + return plaintext + + +def _auth(key): + return {"Authorization": "Bearer " + key} + + +# ── the contract declares a scope for every method ────────────────────── + +def test_every_contract_method_declares_a_known_scope(): + from clawmetry import query_contract as qc + + for name, spec in qc.QUERY_CONTRACT.items(): + assert spec.get("scope") in qc.SCOPES, ( + f"{name} declares scope {spec.get('scope')!r}, which is not one of " + f"{qc.SCOPES}. A method with no scope is unreachable by any key." + ) + + +def test_scopes_partition_every_live_shape(): + """A live shape covered by no scope would be dead to the API forever, + and one covered by two would make the grant ambiguous.""" + from clawmetry import query_contract as qc + + live = set(qc.live_shapes()) + covered = qc.shapes_for_scopes(qc.SCOPES) + assert covered == live, f"uncovered: {live - covered}, phantom: {covered - live}" + seen = set() + for scope in qc.SCOPES: + got = set(qc.live_methods_by_scope(scope)) + assert not (got & seen), f"{scope} overlaps an earlier scope: {got & seen}" + seen |= got + + +def test_read_metrics_is_exactly_the_plaintext_trust_class(): + """The load-bearing invariant behind the whole scope model. + + ``read:metrics`` is what a browser-resident key should hold, and the + promise attached to it is "this cannot return a prompt or a reply". + That promise is only true while metrics-scope and plaintext-trust are + the same set, so it is pinned rather than trusted. + """ + from clawmetry import query_contract as qc + + metrics = set(qc.methods_by_scope(qc.SCOPE_METRICS)) + plaintext = set(qc.methods_by_trust(qc.TRUST_PLAINTEXT)) + assert metrics == plaintext, ( + "read:metrics and the plaintext trust class have diverged. " + f"metrics-only={metrics - plaintext} plaintext-only={plaintext - metrics}. " + "Either move the method to another scope or reclassify its trust." + ) + + +# ── key handling ──────────────────────────────────────────────────────── + +def test_secret_is_never_stored_in_the_clear(ak, tmp_path): + _, plaintext = ak.create("t", ["read:metrics"], ["http://localhost:3000"]) + raw = (tmp_path / "api_keys.json").read_text() + secret = plaintext.split("_", 2)[2] + assert secret not in raw + assert plaintext not in raw + stored = json.loads(raw)["keys"][0] + assert stored["hash"] and stored["hash"] != secret + + +@pytest.mark.skipif(os.name != "posix", reason="POSIX mode bits only") +def test_store_file_is_owner_only(ak, tmp_path): + ak.create("t", ["read:metrics"], ["http://localhost:3000"]) + mode = stat.S_IMODE((tmp_path / "api_keys.json").stat().st_mode) + assert mode & 0o077 == 0, f"key file is group/world accessible: {oct(mode)}" + + +def test_verify_round_trips_and_rejects_a_tampered_secret(ak): + _, plaintext = ak.create("t", ["read:metrics"], ["http://localhost:3000"]) + assert ak.verify(plaintext)["name"] == "t" + kid, secret = plaintext.split("_", 2)[1:] + assert ak.verify("cmk_%s_%sx" % (kid, secret)) is None + assert ak.verify("cmk_00000000_" + secret) is None + assert ak.verify("") is None + assert ak.verify("not-a-key") is None + + +def test_revoke_is_immediate_and_keeps_the_record(ak): + rec, plaintext = ak.create("t", ["read:metrics"], ["http://localhost:3000"]) + assert ak.revoke(rec["id"]) is True + assert ak.verify(plaintext) is None + assert ak.revoke(rec["id"]) is False, "revoking twice must not report success" + assert [r["id"] for r in ak.list_keys(include_revoked=True)] == [rec["id"]] + assert ak.list_keys() == [] + + +def test_revoke_accepts_a_pasted_whole_key(ak): + """People paste the key, not the id. Accepting it costs nothing and + the alternative is a confusing 'no such key'.""" + _, plaintext = ak.create("t", ["read:metrics"], ["http://localhost:3000"]) + assert ak.revoke(plaintext) is True + assert ak.verify(plaintext) is None + + +def test_wildcard_origin_is_refused(ak): + with pytest.raises(ak.ApiKeyError) as exc: + ak.create("t", ["read:metrics"], ["*"]) + assert "wildcard" in str(exc.value).lower() + + +@pytest.mark.parametrize("bad", [ + "localhost:3000", # no scheme + "ftp://example.com", # not http(s) + "https://example.com/dashboard", # has a path + "https://example.com?a=1", # has a query +]) +def test_malformed_origins_are_refused(ak, bad): + with pytest.raises(ak.ApiKeyError): + ak.create("t", ["read:metrics"], [bad]) + + +def test_unknown_scope_is_refused_by_name(ak): + with pytest.raises(ak.ApiKeyError) as exc: + ak.create("t", ["read:everything"], ["http://localhost:3000"]) + assert "read:everything" in str(exc.value) + + +def test_a_key_needs_a_scope(ak): + with pytest.raises(ak.ApiKeyError): + ak.create("t", [], ["http://localhost:3000"]) + + +def test_a_key_needs_a_name(ak): + with pytest.raises(ak.ApiKeyError): + ak.create(" ", ["read:metrics"], ["http://localhost:3000"]) + + +def test_unreadable_store_reads_as_no_keys(ak, tmp_path, monkeypatch): + """A corrupt file must degrade to 401, never to a 500 that takes the + dashboard's own pages with it.""" + (tmp_path / "api_keys.json").write_text("{ this is not json") + assert ak.list_keys() == [] + assert ak.verify("cmk_aaaaaaaa_whatever") is None + + +def test_key_cap_is_enforced(ak, monkeypatch): + monkeypatch.setattr(ak, "MAX_KEYS", 2) + ak.create("a", ["read:metrics"], ["http://localhost:3000"]) + rec, _ = ak.create("b", ["read:metrics"], ["http://localhost:3000"]) + with pytest.raises(ak.ApiKeyError): + ak.create("c", ["read:metrics"], ["http://localhost:3000"]) + ak.revoke(rec["id"]) + ak.create("c", ["read:metrics"], ["http://localhost:3000"]) # a slot freed up + + +def test_redact_shows_the_id_and_never_the_secret(ak): + _, plaintext = ak.create("t", ["read:metrics"], ["http://localhost:3000"]) + red = ak.redact(plaintext) + assert plaintext.split("_", 2)[1] in red + assert plaintext.split("_", 2)[2] not in red + + +# ── HTTP: authentication ──────────────────────────────────────────────── + +def test_no_key_is_401_even_from_loopback(client): + """The whole point of this surface: unlike the rest of the dashboard, + being local earns nothing here.""" + r = client.get("/api/q/1/health") + assert r.status_code == 401 + assert "clawmetry key create" in r.get_json()["error"] + + +def test_bad_key_is_401(client): + r = client.get("/api/q/1/health", headers=_auth("cmk_deadbeef_nope")) + assert r.status_code == 401 + + +def test_revoked_key_stops_working_without_a_restart(client, ak): + rec, plaintext = ak.create("t", ["read:metrics"], ["http://localhost:3000"]) + assert client.get("/api/q/1/health", headers=_auth(plaintext)).status_code == 200 + ak.revoke(rec["id"]) + assert client.get("/api/q/1/health", headers=_auth(plaintext)).status_code == 401 + + +def test_x_clawmetry_key_header_also_works(client, ak): + plaintext = _mint(ak) + r = client.get("/api/q/1/health", headers={"X-ClawMetry-Key": plaintext}) + assert r.status_code == 200 + + +# ── HTTP: scopes ──────────────────────────────────────────────────────── + +def test_granted_shape_dispatches(client, ak): + plaintext = _mint(ak, scopes=["read:metrics"]) + r = client.get("/api/q/1/aggregates", headers=_auth(plaintext)) + assert r.status_code == 200 + body = r.get_json() + assert body["shape"] == "aggregates" + assert body["contract"] == "q/1" + assert client.dispatch_calls[-1][0] == "aggregates" + + +def test_ungranted_shape_is_403_and_names_the_scope(client, ak): + plaintext = _mint(ak, scopes=["read:metrics"]) + r = client.get("/api/q/1/transcript?session_id=x", headers=_auth(plaintext)) + assert r.status_code == 403 + body = r.get_json() + assert body["required_scope"] == "read:content" + assert body["held_scopes"] == ["read:metrics"] + assert not client.dispatch_calls, "a refused query must never reach the store" + + +def test_a_metrics_key_cannot_reach_any_content_shape(client, ak): + """The promise on the tin, checked against every shape rather than a + representative one.""" + from clawmetry import query_contract as qc + + plaintext = _mint(ak, scopes=["read:metrics"]) + for shape in qc.live_methods_by_scope(qc.SCOPE_CONTENT): + r = client.get(f"/api/q/1/{shape}?session_id=x", headers=_auth(plaintext)) + assert r.status_code == 403, f"{shape} was reachable with read:metrics" + + +def test_unknown_shape_is_404(client, ak): + r = client.get("/api/q/1/definitely_not_a_shape", headers=_auth(_mint(ak))) + assert r.status_code == 404 + + +def test_planned_but_unserved_shape_is_404_not_500(client, ak): + """`usage` is declared but not live. Reaching it must look like a typo, + not like a crash.""" + from clawmetry import query_contract as qc + + planned = qc.methods_by_status(qc.STATUS_PLANNED) + assert planned, "no planned methods left; drop this test or pick another" + plaintext = _mint(ak, scopes=list(qc.SCOPES)) + for name in planned: + r = client.get(f"/api/q/1/{name}", headers=_auth(plaintext)) + assert r.status_code == 404, name + + +def test_missing_required_arg_is_400_naming_the_arg(client, ak): + plaintext = _mint(ak, scopes=["read:content"]) + r = client.get("/api/q/1/transcript", headers=_auth(plaintext)) + assert r.status_code == 400 + assert "session_id" in r.get_json()["error"] + + +def test_store_failure_is_503_without_leaking_internals(client, ak, monkeypatch): + import routes.local_query as lq + + def _boom(shape, args): + raise RuntimeError("/Users/someone/.clawmetry/clawmetry.duckdb is locked") + + monkeypatch.setattr(lq, "_dispatch", _boom) + r = client.get("/api/q/1/health", headers=_auth(_mint(ak))) + assert r.status_code == 503 + assert ".duckdb" not in r.get_json()["error"] + assert "clawmetry doctor" in r.get_json()["error"] + + +def test_internal_dispatch_fields_are_stripped(client, ak): + r = client.get("/api/q/1/aggregates", headers=_auth(_mint(ak))) + assert [k for k in r.get_json() if k.startswith("_")] == [] + + +# ── HTTP: the index and the guide ─────────────────────────────────────── + +def test_index_lists_only_granted_shapes(client, ak): + plaintext = _mint(ak, scopes=["read:metrics"]) + body = client.get("/api/q/1", headers=_auth(plaintext)).get_json() + from clawmetry import query_contract as qc + + assert {s["shape"] for s in body["shapes"]} == set( + qc.live_methods_by_scope(qc.SCOPE_METRICS)) + held = {s["scope"] for s in body["scopes"] if s["held"]} + assert held == {"read:metrics"} + + +def test_llms_txt_describes_only_what_the_key_can_run(client, ak): + plaintext = _mint(ak, scopes=["read:metrics"]) + text = client.get("/api/q/1/llms.txt", headers=_auth(plaintext)).get_data(as_text=True) + assert "GET /api/q/1/aggregates" in text + assert "GET /api/q/1/transcript" not in text + assert "Not available to this key" in text + + +def test_llms_txt_needs_a_key_too(client): + assert client.get("/api/q/1/llms.txt").status_code == 401 + + +# ── HTTP: CORS, the part that matters most ────────────────────────────── + +def test_cors_header_only_for_an_origin_the_key_named(client, ak): + plaintext = _mint(ak, origins=["http://localhost:3000"]) + ok = client.get("/api/q/1/health", + headers={**_auth(plaintext), "Origin": "http://localhost:3000"}) + assert ok.headers.get("Access-Control-Allow-Origin") == "http://localhost:3000" + assert ok.headers.get("Vary") == "Origin" + + bad = client.get("/api/q/1/health", + headers={**_auth(plaintext), "Origin": "https://evil.example"}) + assert "Access-Control-Allow-Origin" not in bad.headers, ( + "a key's data was made readable to an origin it never named" + ) + + +def test_cors_is_never_a_wildcard(client, ak): + plaintext = _mint(ak) + r = client.get("/api/q/1/health", + headers={**_auth(plaintext), "Origin": "http://localhost:3000"}) + assert r.headers.get("Access-Control-Allow-Origin") != "*" + + +def test_an_origin_none_key_gets_no_cors_at_all(client, ak): + """A key created for a script must be useless from a browser, even + from an origin some OTHER key has authorised.""" + _mint(ak, name="browser", origins=["http://localhost:3000"]) + script = _mint(ak, name="script", origins=[]) + r = client.get("/api/q/1/health", + headers={**_auth(script), "Origin": "http://localhost:3000"}) + assert r.status_code == 200 + assert "Access-Control-Allow-Origin" not in r.headers + + +def test_preflight_is_answered_only_for_a_known_origin(client, ak): + _mint(ak, origins=["http://localhost:3000"]) + ok = client.options("/api/q/1/health", headers={ + "Origin": "http://localhost:3000", + "Access-Control-Request-Method": "GET", + }) + assert ok.headers.get("Access-Control-Allow-Origin") == "http://localhost:3000" + + bad = client.options("/api/q/1/health", headers={ + "Origin": "https://evil.example", + "Access-Control-Request-Method": "GET", + }) + assert "Access-Control-Allow-Origin" not in bad.headers + + +def test_no_cors_when_authentication_failed(client, ak): + """A 401 body is not interesting, but leaking a header on the failure + path is how a probe learns which origins are authorised.""" + _mint(ak, origins=["https://known.example"]) + r = client.get("/api/q/1/health", headers={"Origin": "https://evil.example"}) + assert r.status_code == 401 + assert "Access-Control-Allow-Origin" not in r.headers + + +def test_origin_matching_ignores_case_and_trailing_slash(client, ak): + plaintext = _mint(ak, origins=["https://My-UI.Example"]) + r = client.get("/api/q/1/health", + headers={**_auth(plaintext), "Origin": "https://my-ui.example/"}) + assert r.headers.get("Access-Control-Allow-Origin") + + +def test_the_api_is_get_only(client, ak): + plaintext = _mint(ak) + for method in ("post", "put", "delete", "patch"): + r = getattr(client, method)("/api/q/1/health", headers=_auth(plaintext)) + assert r.status_code == 405, f"{method.upper()} was accepted" + + +# ── rate limiting ─────────────────────────────────────────────────────── + +def test_rate_limit_trips_and_says_what_to_do(client, ak, monkeypatch): + monkeypatch.setattr(client.pub, "RATE_LIMIT_PER_MIN", 3) + client.pub._RATE.clear() + plaintext = _mint(ak) + codes = [client.get("/api/q/1/health", headers=_auth(plaintext)).status_code + for _ in range(5)] + assert codes[:3] == [200, 200, 200] + assert codes[3:] == [429, 429] + body = client.get("/api/q/1/health", headers=_auth(plaintext)).get_json() + assert "CLAWMETRY_API_RATE_LIMIT" in body["error"] + + +def test_rate_limit_is_per_key(client, ak, monkeypatch): + monkeypatch.setattr(client.pub, "RATE_LIMIT_PER_MIN", 2) + client.pub._RATE.clear() + a = _mint(ak, name="a") + b = _mint(ak, name="b") + for _ in range(2): + client.get("/api/q/1/health", headers=_auth(a)) + assert client.get("/api/q/1/health", headers=_auth(a)).status_code == 429 + assert client.get("/api/q/1/health", headers=_auth(b)).status_code == 200 + + +# ── the management endpoints stay on the dashboard's own gate ─────────── + +@pytest.fixture +def admin(ak): + """A client for the key-MANAGEMENT blueprint. + + Separate from ``client`` on purpose, and asserted to be a different + blueprint below: minting must never be reachable from the surface the + minted keys open. + """ + from flask import Flask + + import routes.apikeys_admin as adm + + app = Flask(__name__) + app.register_blueprint(adm.bp_apikeys_admin) + return app.test_client() + + +def test_key_management_is_not_on_the_keyed_blueprint(): + """The two surfaces must stay apart. + + ``public_api`` answers cross-origin reads; ``apikeys_admin`` mints the + credentials that open it. Registering a management route on the keyed + blueprint would put it behind that blueprint's CORS handler, so this + asserts the separation directly rather than trusting the path guard + alone. + """ + import routes.apikeys_admin as adm + import routes.public_api as pub + + from flask import Flask + + app = Flask(__name__) + app.register_blueprint(pub.bp_public_api) + app.register_blueprint(adm.bp_apikeys_admin) + for rule in app.url_map.iter_rules(): + owner = rule.endpoint.split(".")[0] + if str(rule).startswith("/api/apikeys"): + assert owner == "apikeys_admin", f"{rule} is served by {owner}" + elif str(rule).startswith("/api/q/"): + assert owner == "public_api", f"{rule} is served by {owner}" + + +def test_management_mint_list_revoke(admin): + made = admin.post("/api/apikeys", json={ + "name": "from-ui", "scopes": ["read:metrics"], + "origins": ["http://localhost:5173"]}).get_json() + assert made["ok"] and made["key"].startswith("cmk_") + assert "hash" not in made["record"] + + listed = admin.get("/api/apikeys").get_json() + assert [k["name"] for k in listed["keys"]] == ["from-ui"] + assert all("hash" not in k for k in listed["keys"]) + + kid = made["record"]["id"] + assert admin.delete("/api/apikeys/" + kid).get_json()["ok"] is True + assert admin.delete("/api/apikeys/" + kid).status_code == 404 + + +def test_management_refuses_a_browser_key_with_no_origin(admin): + r = admin.post("/api/apikeys", json={"name": "x", "scopes": ["read:metrics"], + "origins": []}) + assert r.status_code == 400 + assert "wildcard" in r.get_json()["error"] + + +def test_management_allows_an_explicit_non_browser_key(admin): + r = admin.post("/api/apikeys", json={"name": "cron", "scopes": ["read:metrics"], + "origins": [], "browser": False}) + assert r.status_code == 200 + assert r.get_json()["record"]["origins"] == [] + + +def test_management_never_carries_a_cors_header(admin, ak): + """The read API is cross-origin readable; minting keys must not be. + + This is the specific mistake the path guard in ``public_api._add_cors`` + exists to prevent, so it is checked from the outside too. + """ + _mint(ak, origins=["http://localhost:3000"]) + r = admin.get("/api/apikeys", headers={"Origin": "http://localhost:3000"}) + assert "Access-Control-Allow-Origin" not in r.headers + + +def test_management_surfaces_a_sentence_not_a_code(admin, monkeypatch): + import clawmetry.apikeys as mod + + monkeypatch.setattr(mod, "create", lambda *a, **k: (_ for _ in ()).throw(OSError("EACCES"))) + r = admin.post("/api/apikeys", json={"name": "x", "scopes": ["read:metrics"], + "origins": ["http://localhost:3000"]}) + assert r.status_code == 500 + err = r.get_json()["error"] + assert "EACCES" not in err and "~/.clawmetry" in err diff --git a/tests/test_query_contract_drift.py b/tests/test_query_contract_drift.py index dc11c547b6..79cf2aaead 100644 --- a/tests/test_query_contract_drift.py +++ b/tests/test_query_contract_drift.py @@ -70,6 +70,11 @@ def test_statuses_and_version(): for name, spec in qc.QUERY_CONTRACT.items(): assert spec["status"] in (qc.STATUS_LIVE, qc.STATUS_PLANNED), name assert spec["trust"] in (qc.TRUST_PLAINTEXT, qc.TRUST_E2E), name + # Every method declares the read scope an API key needs to reach + # it (routes/public_api.py). A method with no scope, or an unknown + # one, is unreachable by any key -- which is a silent way to ship + # a query nobody can call. + assert spec["scope"] in qc.SCOPES, name assert spec["backing"], name assert spec["doc"], name assert isinstance(spec["args"], dict), name