feat(mcp): self-diagnostics report tool, operator read tools, multi-runtime installer, honesty rollup (WO-59) - #5481
Conversation
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| except Exception as e: # noqa: BLE001 | ||
| log.debug("mcp support matrix failed: %s", e) | ||
| rows = [] | ||
| return jsonify({"runtimes": rows, "count": len(rows)}) |
Visual diffComparing 33 of 72 comparison(s) flagged (>1% pixel diff).
Folder: b30b911c1f6d. Full PNGs also attached as a workflow artefact. Generated by visual-diff bot. Pixel diffs >1% flagged; eyeball the table before merging. This check is non-blocking — fail = bot bug, not a code problem. |
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
9aea97b to
c6eb640
Compare
|
Drift-bot: clean. Test plan for review. This PR adds MCP self-diagnostics (WO-59): a report tool, operator read tools, multi-runtime installer, and honesty rollup. Suggested test checklist before merge: MCP report tool
Operator read tools
Multi-runtime installer
Honesty rollup
Regression
Generated by Claude Code |
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
c6eb640 to
797a80c
Compare
|
![8090-software-factory[bot]](https://avatars.githubusercontent.com/in/1847834?s=60&v=4){kind=small}
| @@ -1055,6 +1055,18 @@ def http_query(): | |||
| "query_signal_coverage", | |||
There was a problem hiding this comment.
The blueprint specifies that tools answer through the daemon and every tool should succeed or fail honestly. The get_guard_status MCP tool calls get_session_location through the daemon, but this method is not in the allowlist of exposed methods (_DAEMON_METHODS), causing the call to fail with a "method not allowed" error misidentified as a daemon upgrade issue.
|
blocked on author decision — skipping (auto-mergeability sweep) All CI checks pass. PR is BLOCKED by a required reviewer approval — mechanical sweep cannot approve; needs human review. Generated by Claude Code |
|
Drift-bot triage: the one finding is a base misread. |
|
| "query_self_report_honesty", | ||
| "query_guard_incidents", | ||
| "query_session_denials", | ||
| "find_session_by_cwd", |
There was a problem hiding this comment.
The allowlist in routes/local_query.py references methods that do not exist in the LocalStore: ingest_self_report, query_self_reports, query_self_report_counts, query_self_report_honesty, query_guard_incidents, query_session_denials, find_session_by_cwd. These methods are specified in the blueprint's SelfReportStore component and called by MCP tools, but are not implemented.
|
Drift-bot triage, second run (3 findings, all base misreads): on HEAD 38d4af0, |
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
4107e2a to
f636469
Compare
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
…untime installer, honesty rollup (WO-59) An agent can tell its operator what got in the way; ClawMetry checks whether the tool stream agrees. - MCP server: `report_to_operator` (six operator-extendable categories, 500-char summary, session inferred from env or cwd) plus read tools `list_incidents`, `get_guard_status`, `get_signal_rates` (honest "not available on this daemon version"), `list_self_reports`. No tool acts on a process; every tool returns an honest error when the daemon is down, busy, or too old. - Store: additive `agent_self_reports` table; summary redacted before storage; writes only through the daemon's local query server. - Daemon tick: corroborate reports against detector incidents and permission denials within CORROBORATION_WINDOW_SECS (600, env override); honesty per (runtime, model) withheld under MIN_INCIDENTS. - CLI: `clawmetry mcp install|uninstall|status` for Claude Code, Cursor, Codex, Gemini CLI, OpenCode, Windsurf (formats verified against vendor docs; Codex against `codex mcp add`), merge-only, never deletes a foreign entry, uninstall removes only what the marker says we wrote, guidance snippet offered and written only with --write-guidance. - Routes /api/self-reports, /honesty, /support; transcript panel "What the agent reported"; Guard tab "Agent reported" card; `selfReports` snapshot slice. - tests/test_self_diagnostics.py (49) wired into moat-tests + Makefile. Requirement: c2329afe-0337-4d09-97b6-b6b7c7e260a1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ERBVJarDT4Vw1hJox5efBB
… exception text served)
- parse_window_secs strips and bounds the input before an unambiguous
regex (`^(\d{1,12})([smhdw]?)$`), so a long run of spaces cannot make
the old `\s*...\s*` pair backtrack.
- support_matrix (served by /api/self-reports/support) replaces an OSError
message with a fixed sentence; the CLI still prints the real error.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ERBVJarDT4Vw1hJox5efBB
…tore; Guard card links to the Signals tab - The MCP get_signal_rates tool now calls query_signal_grouped (WO-58, merged in #5485) and shapes the rates with behaviour_signals.shape_rates, the same arithmetic the Signals tab uses. A daemon that predates the signals store still gets "signals not available on this daemon version". - The Guard tab "Agent reported" card links to the Signals tab. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ERBVJarDT4Vw1hJox5efBB
…on_location is allowlisted) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ERBVJarDT4Vw1hJox5efBB
…HEAD) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ERBVJarDT4Vw1hJox5efBB
f636469 to
5daca7b
Compare
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ERBVJarDT4Vw1hJox5efBB
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
2 participants
Product record: https://factory.8090.ai/project/b415065f-ab2f-4f53-8864-0c009fd098cb/requirements/c2329afe-0337-4d09-97b6-b6b7c7e260a1 (Self-diagnostics and operator tools over MCP, REQ-SELF-001 to 004; WO-59).
Blueprint: https://factory.8090.ai/project/b415065f-ab2f-4f53-8864-0c009fd098cb/blueprints/04b881ea-5b10-4fbe-95cc-71dca015793f (Components: Self-Diagnostics and Operator Tools over MCP).
Why
Reasoning models are good at introspection when asked plainly. Given a reporting tool framed as notes to the people who run them, an agent will say that a tool kept failing, that it lacked context or a permission, that it could not finish, or that it worked around a block. ClawMetry already watches the same session from outside, so every such note can sit next to what the detectors and the approval hooks recorded on their own, and the operator can see when the agent stayed quiet about something the detectors caught. The same MCP server is where a developer's editor can ask ClawMetry questions from where they already are.
What
MCP server (
clawmetry/mcp_server.py)report_to_operator— category (six defaults, operator-extendable via~/.clawmetry/config.json→self_diagnostics.categories; the enum is resolved attools/listtime), summary capped at 500 chars,session_idinferred from env (CLAUDE_SESSION_ID& co.) or a working-directory match through the daemon when absent,runtimeinferred from the session prefix or env. Described as feedback to the people who run the agent, never as confession.list_incidents(window/runtime/session),get_guard_status(per-session verdict fromprocess_control.runtime_control_support, policy decisions, open incidents,read_only: true),get_signal_rates(returns"signals not available on this daemon version"when the daemon lacks the method rather than failing),list_self_reports.Store (
clawmetry/local_store.py) — additiveagent_self_reportstable (CREATE TABLE IF NOT EXISTS, no schema bump).ingest_self_reportredacts the summary withclawmetry.redactionbefore the row is written; the raw text is never stored. Read methods:query_self_reports,query_self_report_counts,query_self_report_honesty,query_guard_incidents(loop_signals + session model),query_session_denials(approvals withstatus='denied'),find_session_by_cwd. All named in_DAEMON_METHODS; the MCP process writes only via/__local_query__/ingest_self_report(the daemon keeps the writer lock).Daemon (
clawmetry/sync.py,clawmetry/self_diagnostics.py) — on the detector tick,corroborate_pendingmarks a report corroborated when a detector incident's stretch or a permission denial lies withinCORROBORATION_WINDOW_SECS(600;CLAWMETRY_SELFDIAG_WINDOW_SECS) of it, with the nearest incident winning so abypassed_blockreport links to the block it worked around. Honesty per (runtime, model) = incidents the agent also reported / incidents; withheld with a reason underMIN_INCIDENTS(5;CLAWMETRY_SELFDIAG_MIN_INCIDENTS). Snapshot sliceselfReports={window_secs, total, corroborated, byRuntime{runtime:{category:n}}, honesty[], min_incidents, corroboration_window_secs}; no summaries leave the node. Try/except guarded.CLI (
clawmetry/mcp_install.py, fast path incli.py, no dashboard import) —clawmetry mcp install [--runtime <id>|all] [--dry-run] [--write-guidance] [--json],clawmetry mcp uninstall,clawmetry mcp status. Per runtime:registered|already present|no MCP support|unknown config format(+not installed,would register,removed,left in place). Merge only; a foreign entry is never deleted; a marker file (~/.clawmetry/mcp_installed.json) records what we wrote and uninstall removes only that; an entry namedclawmetrywe did not write is left in place; a JSONC file is reported and never rewritten. The guidance snippet for CLAUDE.md / AGENTS.md / GEMINI.md is printed and written only with--write-guidance.Installer formats, each verified before writing:
~/.claude.json→mcpServers(type: stdio)~/.cursor/mcp.json→mcpServers~/.codex/config.toml→[mcp_servers.clawmetry]codex mcp addinto a scratchCODEX_HOMEand read what it wrote (the docs site redirects to a host WebFetch cannot verify)~/.gemini/settings.json→mcpServers~/.config/opencode/opencode.json→mcp(type: local,command: [..],enabled)~/.codeium/windsurf/mcp_config.json→mcpServersNine runtimes are declared
no MCP supportwith a one-line reason; every other observed runtime reportsunknown config formatand its file is never touched.Routes (
routes/selfdiag.py,bp_selfdiag) —GET /api/self-reports?window=&runtime=&category=&session=,GET /api/self-reports/honesty?window=&runtime=,GET /api/self-reports/support. Reads go through the daemon proxy.UI — transcript view gains "What the agent reported" (time order, category chip, summary, Corroborated / Uncorroborated with the hover text "No independent evidence was found for this report. That is not the same as false."). Guard tab gains an "Agent reported" card: per-category counts per runtime, the honesty table with the withheld reason, and honest empty states including "this runtime has no MCP support" and "not registered yet, run
clawmetry mcp install". No em dashes in user-facing copy. The Signals tab is being built in parallel; this lands on Guard as agreed.Verification
tests/test_self_diagnostics.py: 49 tests. Tool schema and framing; no actuating tool; honest error for every tool when the daemon is down, busy, or too old; redaction + cap on the write path; idempotency; session inference from env then cwd; inclusive window logic; corroboration against realloop_signalsandapprovalsrows viacorroborate_pending; honesty withheld/computed and the env floor; installer merge / never-delete / uninstall-only-ours / hand-written-left-in-place / JSONC-refused for all six formats, Codex block round-tripped throughtomllib; guidance offered not written; CLI fast path never importsdashboard; routes; snapshot slice; live templates carry the new surfaces.test_ingest_applies_redaction_and_cap; making uninstall clobber the container fails 5/6test_install_merges_and_uninstall_removes_only_ours. Both restored to green.moat-testsjob inci.ymlandmake test-selfdiag.make lintpieces: daemon allowlist OK (147 entries), py39 unions OK, AC ratchet holding, ruff clean on the new files,node --checkon app.js.python -m clawmetry mcpservedtools/listwith the ten tools against the running daemon; the running 0.12.80x daemon was not answering its local query server in this session, and the tool reported "running but did not answer within 10s" rather than "not running".~/.clawmetry/nocloudwas absent after the test run and restored with the documented content.Not in this PR
selfReports(clawmetry-cloud); the Guard card and transcript panel show an honest "local store" state on cloud until it lands./api/signals(parallel work);get_signal_ratescallsquery_signal_ratesthrough the daemon and says so when it is absent.🤖 Generated with Claude Code
https://claude.ai/code/session_01ERBVJarDT4Vw1hJox5efBB