feat(ingest): open the front door — ingest key, generated contract, setup prompt, live status - #5684
feat(ingest): open the front door — ingest key, generated contract, setup prompt, live status#5684vivekchand wants to merge 38 commits into
Conversation
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
Visual diffComparing 46 of 70 comparison(s) flagged (>1% pixel diff).
Folder: 72bb275fc284. Full PNGs also attached as a workflow artefact. Generated by visual-diff bot. Pixel diffs >1% flagged; eyeball the table before merging. This check is non-blocking — fail = bot bug, not a code problem. |
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
1b36e87 to
90f8eb5
Compare
a399006 to
1eb88dc
Compare
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
|
✨ auto-fixed: merged updated base branch (feat/build-your-own-ui) into branch — base was at old SHA 90f8eb5, now at e270a12 Generated by Claude Code |
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
vivekchand
changed the title
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
vivekchand
changed the title
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
e270a12 to
0339012
Compare
7f72f7d to
7c8a51a
Compare
|
|
Checked this rather than acting on it — the finding is a false positive, but it was pointing at a real defect.
Why the bot couldn't see it, and the actual defect. The module opens with a per-blueprint route index, and that index had drifted:
Fixed in a798db0: every count corrected, the missing routes named, and a guard added asserting the documented count matches the decorators below it (mutation-proven by understating |
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
ba455d6 to
c042dad
Compare
c042dad to
11d922a
Compare
ClawMetry ships one dashboard, and a user who wanted a different view of
their own data had to fork it. Every number the dashboard draws already
comes from a declared, versioned read contract (q/1); the only thing
allowed to call it was the dashboard itself, because the only gate in
front of it was the browser's same-origin rule.
This adds the door:
clawmetry key create --name my-ui --scope read:metrics \
--origin http://localhost:3000
GET /api/q/1/<shape> one declared query
GET /api/q/1 what this key can read
GET /api/q/1/llms.txt the API, written for a coding agent
Four pieces, all built on what existed:
* Read scopes live ON the contract. Every q/1 method declares one of
read:metrics / read:sessions / read:traces / read:content next to its
trust class, so a new method cannot drift out of the scope model and
docs/QUERY_CONTRACT.md regenerates with a scope table.
* clawmetry/apikeys.py + `clawmetry key` + a Security-tab panel. SHA-256
in ~/.clawmetry/api_keys.json (0600), scopes plus an origin allowlist.
* routes/public_api.py dispatches through the SAME local_query._dispatch
the dashboard uses, so there is no second query surface.
* examples/custom-ui/ (one file, no build step) and
docs/BUILD_YOUR_OWN_UI.md.
The security decision this turns on: adding CORS to a service on
localhost is how local tools get robbed. Any page in any tab can already
SEND a request to 127.0.0.1:8900; the only reason that is harmless is
that the browser will not let it READ the reply. So loopback earns
nothing here (a key is always required, which inverts the rest of the
dashboard), there is no wildcard origin at all, key management stays on
the ordinary dashboard blueprint behind the same-origin write guard, and
the API is GET only. Nothing here writes: this adds no entry to the
control plane.
read:metrics is exactly the plaintext trust class, pinned by a test.
That equality is what makes "a browser-resident key cannot return a
prompt" a promise rather than a hope.
Verified: the starter served from localhost:3000 rendered $750 of real
30-day cost across 10 runtimes from a ClawMetry on another port, and on
that same page a valid full-scope key bound to a different origin was
refused by the browser. 48 guards in tests/test_public_api_keys.py
(registered in ci.yml), three mutations proven red.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6d1rL2kbFEu6Pr6xUzYW5
Drift Bot's one finding was that the auto-created feature blueprint was still the empty template. Written in full (v2, 11.6 KB): composition, four component blocks, key + integration contracts, and six ADRs covering the scope-on-the-contract choice, loopback not being authentication here, the no-wildcard rule, key management staying off the keyed surface, why custom UIs read the node rather than the hosted service, and the self-describing guide. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6d1rL2kbFEu6Pr6xUzYW5
Drift Bot round 2 read the rewritten blueprint (it no longer calls it empty) and asked for the concrete API surface rather than only ADRs. Added an 'API Surface' section at 4% of the document: the three read endpoints with auth and the status-code table, which four response fields a client contracts on and which are diagnostic, the cross-origin rule, the three management endpoints, and the four CLI subcommands. Blueprint v3. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6d1rL2kbFEu6Pr6xUzYW5
Drift Bot round 3 reported that GET/POST /api/apikeys and DELETE /api/apikeys/<id> 'are not implemented in routes/infra.py' and that the Security tab therefore cannot work. They were implemented, at lines 2955-3042 of a 3042-line file, i.e. at 97% depth: the bot reads a truncated prefix of a long file, so anything appended to a module that size is invisible to it forever. Rather than argue the point on a PR comment, extract them. They now live in routes/apikeys_admin.py (bp_apikeys_admin, ~130 lines), registered in dashboard.py next to bp_public_api. routes/infra.py returns byte for byte to its state on main. This is better placement independent of the tool. Credential management is a different concern from the security tab's scanners, and the reason these handlers must not sit on the keyed surface now has a file to be stated in rather than a comment in the middle of another one. Both guards are path-based, not blueprint-based, so the security posture is unchanged: dashboard.py's _cross_origin_write_blocked still refuses a cross-origin POST/DELETE, and public_api._add_cors is still pinned to /api/q/. A new guard asserts the separation directly: no /api/apikeys rule may be owned by the public_api blueprint and no /api/q/ rule by apikeys_admin. Verified on a real boot after the move, not only in tests: management GET 200, POST with no origin 400, cross-origin POST 403, query index 401 without a key and 200 with one, aggregates 200, CORS header present on /api/q/ and absent on /api/apikeys. Blueprint v4 names the module. Cloud #2342 updated to match the new endpoint names. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6d1rL2kbFEu6Pr6xUzYW5
I called the earlier CodeQL red a stale aggregate. It was not: 8 new alerts, 4 of them high, all in code this PR adds. Triaged individually. This repo runs code scanning through DEFAULT SETUP, so inline '# codeql[...]' comments are inert here (measured in clawmetry/nonsecret_hash.py, not assumed). Every alert therefore had to be fixed in code or accepted deliberately. Six are fixed: * py/stack-trace-exposure x2. Both were our own authored sentences, but text read off an exception is exception-derived to a static analyser no matter who wrote it. The /api/q/ 400 is now built from the CONTRACT (which already declares the required args) and the /api/apikeys 400 from a literal table keyed by a refusal code. Both are better answers: the query error is now consistent across queries and carries missing_args, and the create error carries a stable 'reason' a client can branch on instead of matching prose. * py/log-injection. The failure log now names the contract's own backing method rather than the shape string off the URL. Validated-then-logged is still a request-text sink. * py/clear-text-logging x2 of 4. The two curl examples no longer interpolate the key; they use , which is what the docs tell people to do with it anyway and reads better than a 50-character key wrapped across a terminal. * js/xss-through-exception. The starter's error panel built markup out of strings including a fetch exception message. It now builds text nodes and <code> elements, so nothing it did not write becomes markup. The two remaining alerts are the one-time key reveal (the plain print and the --json field). A key that is never shown cannot be used, and every comparable CLI does exactly this. They are true observations, not defects, and with no working in-code suppression they can only be dismissed by a maintainer in the Security tab. Verified live after the change: the contract-built 400 names session_id and returns missing_args, the management 400 carries reason 'unknown_scope' with no exception text, the wildcard refusal still fires, and all three normal paths stay 200. Both starter error panels render as real text nodes (3 <code>, 1 <br>) and the happy path still draws across 10 runtimes and 10 models. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6d1rL2kbFEu6Pr6xUzYW5
Drift Bot round 5, and it is right: the extraction moved the handlers to routes/apikeys_admin.py but left the cross-reference in public_api._add_cors naming their old home. That comment is load-bearing (it explains WHY the CORS handler is pinned to /api/q/), so pointing it at a file that no longer contains them is worse than no comment. Swept the rest of the new code for the same staleness; this was the only one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6d1rL2kbFEu6Pr6xUzYW5
Drift from Syntax & Lint CI check on feat/ingest-key. Regenerated with python3 scripts/gen_module_map.py to unblock PR #5684. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ABjpgFAH2NmkvKo9MMXXiz
|
![8090-software-factory[bot]](https://avatars.githubusercontent.com/in/1847834?s=60&v=4){kind=small}
| payload, status = err | ||
| return jsonify(payload), status | ||
|
|
||
| try: |
There was a problem hiding this comment.
The _otlp_receive function passes runtime and env as keyword arguments to processor functions, but they only accept pb_data, content_encoding, and content_type. The blueprint specifies routing headers should be applied to resource attributes via _apply_ingest_routing before processing, not threaded as parameters.
|
| try: | ||
| process( | ||
| request.get_data(), | ||
| body, |
There was a problem hiding this comment.
The _otlp_receive function passes runtime and env as keyword arguments to processor functions, but they only accept pb_data, content_encoding, and content_type. The blueprint specifies routing headers should be applied to resource attributes via _apply_ingest_routing before processing, not threaded as parameters.
|
Auto-rebase attempted and aborted — not safe to resolve automatically. Status: Why I stopped: Commit 1 of 19 had 9 files with content conflicts: Context: Note that the base branch ( Generated by Claude Code |
|
awaiting confirmation — non-trivial rebase, needs human review (conflicts in: CHANGELOG.md, CLAUDE.md, clawmetry/apikeys.py, clawmetry/cli.py, dashboard.py, docs/MODULE_MAP.md, examples/custom-ui/index.html, routes/public_api.py, tests/test_public_api_keys.py) This PR is stacked on Suggested approach: rebase Generated by Claude Code |
Test plan & review notesRepo: vivekchand/clawmetry What changed
Smoke commands (after rebase onto main)
What to look at visually
Issue link Generated by Claude Code |
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N3rgHBErNdBNUWo89ddN8S
|
Merged updated Generated by Claude Code |
|
| content_encoding=request.headers.get("Content-Encoding"), | ||
| content_type=request.headers.get("Content-Type"), | ||
| runtime=ctx["runtime"], | ||
| env=ctx["env"], | ||
| ) | ||
| return "{}", 200, {"Content-Type": "application/json"} | ||
| except OtlpProtobufUnavailable as e: |
There was a problem hiding this comment.
The _otlp_receive function passes runtime and env as keyword arguments to processor functions, but they only accept pb_data, content_encoding, and content_type. The blueprint specifies routing headers should be applied to resource attributes via _apply_ingest_routing before processing, not threaded as parameters, causing TypeError at runtime when ingest keys are used.
|
PR sweep standing down: this PR ( Generated by Claude Code |
|
Maintenance bot: merged updated Generated by Claude Code |
|
| content_encoding=request.headers.get("Content-Encoding"), | ||
| content_type=request.headers.get("Content-Type"), | ||
| runtime=ctx["runtime"], | ||
| env=ctx["env"], | ||
| ) | ||
| return "{}", 200, {"Content-Type": "application/json"} | ||
| except OtlpProtobufUnavailable as e: |
There was a problem hiding this comment.
The _otlp_receive function passes runtime and env as keyword arguments to processor functions, but they only accept pb_data, content_encoding, and content_type. The blueprint specifies routing headers should be applied to resource attributes via _apply_ingest_routing before processing, not threaded as parameters, causing TypeError at runtime when ingest keys are used.
|
✨ auto-fixed: merged feat/build-your-own-ui (base branch updated with main) into branch — now up to date with its base. Generated by Claude Code |
|
| content_type=request.headers.get("Content-Type"), | ||
| runtime=ctx["runtime"], | ||
| env=ctx["env"], | ||
| ) | ||
| return "{}", 200, {"Content-Type": "application/json"} | ||
| except OtlpProtobufUnavailable as e: |
There was a problem hiding this comment.
The _otlp_receive function passes runtime and env as keyword arguments to processor functions, but they only accept pb_data, content_encoding, and content_type. The blueprint specifies routing headers should be applied to resource attributes via _apply_ingest_routing before processing, not threaded as parameters, causing TypeError at runtime when ingest keys are used.
2 participants
Closes #5679, #5682, #5681, #5680.
Requirement: https://factory.8090.ai/project/b415065f-ab2f-4f53-8864-0c009fd098cb/requirements/7fb88c5c-228e-4c48-9f16-da570740fc36
Stacked on #5676 (
feat/build-your-own-ui) — this is the write half of that key system, so the base is that branch, not main. Merge #5676 first.The gap
An agent is observable by ClawMetry only if the daemon runs on the same machine as the agent.
/v1/*trusts loopback and otherwise wants the gateway token; the write API trusts loopback or one static secret shared by the whole install;ingest.clawmetry.comtakes E2E-sealed daemon snapshots and nothing else. That excludes CI runners, containers, serverless, hosted agent products and teammates' laptops — where a lot of agent spend actually happens.Three commits, in dependency order.
1. The ingest key (#5679)
then
x-clawmetry-key: cmk_…on the OTLP endpoints. Samecmk_shape, same store, sameclawmetry key list|revokeas the read keys — not a second key system. Two routing headers come with it, written into the resource attributes the mappers already read (service.name,deployment.environment), so a header is exactly as powerful as the equivalent exporter setting and no mapper learns a second way to answer the question.The posture: an ingest key can only push (
write:ingestgrants noq/1shape); it is never given a CORS header and cannot carry a browser origin; read and write cannot be mixed on one key, refused at creation with a sentence rather than at request time with a code; and there is one gate, not two —_check_authsteps aside for a keyed/v1/request exactly as it does for/api/q/.Verified against a real dashboard: all three doors return 200 (loopback zero-config unchanged, gateway token unchanged, ingest key), every refusal returns its own status with its own sentence, and a span pushed with
x-clawmetry-runtime: my-enginelands asagent_type=my_enginewhile the same span pushed bare still lands asopenclaw. That live run found the one real bug here:GET /api/q/1was answering200to a write-only key —verify()accepts it since it is a valid key, and nothing asked whether it held any read scope. Now403, pinned by a test.2. The ingest contract, generated (#5682)
Four things describe this surface — the server,
docs/INGEST.md, the setup prompts, and the landing reference — and none shared a source.clawmetry/ingest_contract.pydeclares it as data;ingest_auth.pyimports its constants from there;scripts/gen_ingest_doc.pyrenders the doc with--check.The guard earns the file. It checks both directions: every attribute declared read must be named in the mapper, and every attribute declared unread must genuinely be absent. Writing that check is what surfaced #5685 — we were advertising a GenAI convention we did not implement, and cached tokens were priced as free.
The doc also states what ClawMetry does not accept — no syslog, CEF, GELF or raw text — pinned by a test, because a reference that only says what works is not one anyone can plan against.
Two cache attributes are listed as not read here, which is the honest state of this branch: the fix is #5686, open against main while this is stacked elsewhere. The unread-side guard makes whichever merges second update the list — which is why both directions are checked.
3. The setup prompt (#5681)
clawmetry setup-prompt <runtime>andGET /api/setup-prompt, rendered from the same contract. Half of it is negative space, because that is the useful half: the key goes in one header and nowhere else; the placeholder is a placeholder and the real key must be asked for, never invented; both encodings are already accepted and need no "fixing"; a key not in the prompt does not exist; and this is observability — it watches, it does not change what runs. It ends by making the agent verify and report a real event count.Two guards worth keeping. The first reads backwards: every
x-clawmetry-*token in the prompt must be a declared header. Checking only that the right headers appear was too weak — proven by mutation, where swapping the config block's header forx-clawmetry-apikeyleft every other assertion green because the correct name still appeared in the prose. The second is general: every registered subcommand must be incli.py's_subcmdsallowlist, because a parser with no entry there falls through to the dashboard's argparse and dies with "invalid choice", reading as though the command was never written.setup-promptdid exactly that when first added — the same two-list trap CLAUDE.md documents for runtimes — and asserting only thatsetup-promptis present would not have prevented the next one.The prompt module names no runtime and hardcodes no vendor value, pinned by a test: runtime-specific OTel knowledge stays in profiles, which paid runtimes register from clawmetry-pro.
4. "Did it work?" (#5680)
GET /api/onboarding/ingest-statusanswers the question that kills setup funnels, from real data, and the first-run gate renders it as a live strip. When nothing has arrived it says so and what to do, includingclawmetry setup-promptfor an off-box agent. It never blocks — a confirmation, not a step, in a flow whose selling point is having no steps.Three judgement calls worth naming:
has_data_this_process. Folding them together would tell a working install it was broken after every restart._dispatch, never raw files, so it answers identically on a laptop and in a container with no~/.openclaw. Memoised 2s because it's polled: ~200 ms first call, ~0.2 ms after.Verified in both states against a running dashboard: strip renders inside the card in the real page; empty store returns
connected:falsewith an actionablenext_step; the populated store here returns 25,349 events across 9 runtimes. The strip first landed outside.obg-card— on the overlay backdrop rather than in the dialog — so a guard now asserts its position.Tests
77 guards across
test_ingest_key.py(25),test_ingest_contract_drift.py(26),test_setup_prompt.py(17) andtest_ingest_status.py(9), all registered inci.ymlsince CI runs explicit file lists. Every non-obvious guard was mutation-tested — and two of them were strengthened after the mutation passed, which is the only reason they are worth having.Not in scope
Regional endpoint split, a base "any format" endpoint, per-key rate limiting (belongs on the hosted side where plan budgets live). The hosted endpoint is clawmetry-cloud#2343, gated on the trust-class work in clawmetry-cloud#2344 — pushed telemetry is plaintext by construction, and that needs naming before it ships rather than after it becomes a P0.
🤖 Generated with Claude Code
https://claude.ai/code/session_01YMHxYeQR1QLunY3PzURqRd