Send the Quality grade to the cloud, and stop the cloud inventing an empty one - #5121
Conversation
|
![8090-software-factory[bot]](https://avatars.githubusercontent.com/in/1847834?s=60&v=4){kind=small}
Visual diffComparing 40 of 68 comparison(s) flagged (>1% pixel diff).
Folder: 114a482ff867. Full PNGs also attached as a workflow artefact. Generated by visual-diff bot. Pixel diffs >1% flagged; eyeball the table before merging. This check is non-blocking — fail = bot bug, not a code problem. |
|
Autonomous maintainer pass (2026-08-23): all 37 CI checks green. One blocker: drift-bot is red (4 findings). Per FLYWHEEL.md §1f (just merged in #5122), this cannot be cleared after merge — it must be green before merging. What drift-bot flagged (both require
Path to green:
Downstream: cloud #2090 is also blocked until this merges and ships — see cloud #2090 for details. Generated by Claude Code |
|
✨ auto-fixed: merged main into branch to bring it current (was behind by several commits) Generated by Claude Code |
|
|
|
PR mergeability janitor (automated sweep) This PR is BLOCKED by a real CI failure — not a transient issue, not rerunnable:
The Drift Bot caught a semantic contradiction between this diff and a tracked Blueprint. This requires human review and resolution — the janitor does not resolve drift findings automatically. Note: this branch is also behind Generated by Claude Code |
|
✨ auto-fixed: branch was BEHIND main (base was 818cbde, main advanced to 85ed292 via 5 PRs); merged main into branch so CI reruns on current base. Generated by Claude Code |
|
|
blocked on author decision — skipping (auto-mergeability sweep): the Generated by Claude Code |
|
Autonomous maintainer check-in (2026-08-23) All CI checks pass except one: the E2E Gate fails because Every other required check (OSS golden path, MOAT Keystone, E2E Browser Tests, Syntax & Lint, API Tests x3 OS, MOAT Verifier, Entitlement API tests, pip install x4, Wheel install, Store invariants) passed green. Drift Bot checks whether the diff contradicts a Blueprint in the 8090 Software Factory spec database. The check runs externally and its output is not available in this context, so the specific blueprint conflict cannot be diagnosed here. Human action needed: review the Drift Bot output linked in the check run, identify which Blueprint assertion the quality-parity change conflicts with, and either (a) update the implementation to satisfy the Blueprint, or (b) update the Blueprint in the spec database to reflect the new behavior, then re-run Drift Bot. The cloud-side counterpart (#clawmetry-cloud/2090) is blocked on this PR merging first (it reads Generated by Claude Code |
|
✨ auto-fixed: merged main into branch (was behind; branch updated to current main HEAD) Generated by Claude Code |
|
|
✨ auto-fixed: branch was behind main (base SHA 979fc06 → 439c71c); updated via Generated by Claude Code |
|
| } | ||
|
|
||
|
|
||
| _QUALITY_SNAPSHOT_WINDOW_HOURS = 168 # 7d — what the Quality tab asks for |
There was a problem hiding this comment.
The sync daemon implements _build_quality_snapshot() to emit node-wide and per-runtime quality report cards as an encrypted snapshot slice for cloud parity, but this responsibility and the "read once, compose many" optimization pattern are not documented in the Local Observability Service blueprint.
| "governance": _build_governance(), | ||
| "dailyUsage": _du, # #2142: computed once above, shared with `spending` | ||
| "reliability": _build_reliability(), | ||
| "quality": _build_quality_snapshot(), |
There was a problem hiding this comment.
The sync daemon adds a quality snapshot slice to sync_system_snapshot() for the cloud dashboard, but the Cloud Fleet Dashboard blueprint does not document a CloudQualityInterceptor component for consuming the quality slice alongside the existing CloudSpendFlowInterceptor and CloudAttentionInterceptor.
| ``?window=7d`` (default), ``?runtime=<id>`` (optional scope). No auth gate | ||
| — this is the free-tier home for the "is my agent OK?" answer, same | ||
| rationale as /api/evaluators and /api/evals/metrics. | ||
| def compose_report_card( |
There was a problem hiding this comment.
The code refactors compose_report_card() as a standalone importable function that both the daemon's snapshot builder and the request handler reuse to build identical payloads, enabling "read once, compose many" without per-runtime queries, but this architectural pattern is not documented in the Local Observability Service blueprint.
|
Automated sweep — blocker diagnosed, cannot self-resolve The E2E Gate required check is failing due to Drift Bot (8090 Software Factory) posting a What was confirmed:
What's needed to unblock:
No code changes were pushed during this sweep. The PR is otherwise ready for merge once the Drift Bot status clears. Generated by Claude Code |
|
blocked on author decision — skipping (auto-mergeability sweep) Drift Bot is posting a failure on the Generated by Claude Code |
|
| } | ||
|
|
||
|
|
||
| _QUALITY_SNAPSHOT_WINDOW_HOURS = 168 # 7d — what the Quality tab asks for |
There was a problem hiding this comment.
The sync daemon implements _build_quality_snapshot() to emit node-wide and per-runtime quality report cards as an encrypted snapshot slice for cloud parity, but this new responsibility and the "read once, compose many" optimization pattern are not documented in the Local Observability Service blueprint.
| "governance": _build_governance(), | ||
| "dailyUsage": _du, # #2142: computed once above, shared with `spending` | ||
| "reliability": _build_reliability(), | ||
| "quality": _build_quality_snapshot(), |
There was a problem hiding this comment.
The sync daemon adds a quality snapshot slice to sync_system_snapshot() for the cloud dashboard, but the Cloud Fleet Dashboard blueprint does not document a CloudQualityInterceptor component for consuming the quality slice alongside the existing CloudSpendFlowInterceptor and CloudAttentionInterceptor.
…y Service v26, Cloud Fleet Dashboard v15) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011JYHXKUSfu72qPh2zuq92N
|
|
|
||
| _QUALITY_SNAPSHOT_WINDOW_HOURS = 168 # 7d — what the Quality tab asks for | ||
|
|
||
|
|
There was a problem hiding this comment.
The sync daemon implements _build_quality_snapshot() to emit node-wide and per-runtime quality report cards as an encrypted snapshot slice for cloud parity, but this new responsibility and the "read once, compose many" optimization pattern are not documented in the blueprint.
| "governance": _build_governance(), | ||
| "dailyUsage": _du, # #2142: computed once above, shared with `spending` | ||
| "reliability": _build_reliability(), | ||
| "quality": _build_quality_snapshot(), |
There was a problem hiding this comment.
The sync daemon adds a quality snapshot slice to sync_system_snapshot() for the cloud dashboard, but the Cloud Fleet Dashboard blueprint does not document a CloudQualityInterceptor component for consuming the quality slice alongside the existing CloudSpendFlowInterceptor and CloudAttentionInterceptor.
…ndex refresh) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011JYHXKUSfu72qPh2zuq92N
|
| } | ||
|
|
||
|
|
||
| _QUALITY_SNAPSHOT_WINDOW_HOURS = 168 # 7d — what the Quality tab asks for |
There was a problem hiding this comment.
The sync daemon now implements a QualitySnapshotBuilder (_build_quality_snapshot) to emit quality report cards in the encrypted snapshot for cloud parity, but the Local Observability Service blueprint does not document this new component responsibility or its behavior.
| ``?window=7d`` (default), ``?runtime=<id>`` (optional scope). No auth gate | ||
| — this is the free-tier home for the "is my agent OK?" answer, same | ||
| rationale as /api/evaluators and /api/evals/metrics. | ||
| def compose_report_card( |
There was a problem hiding this comment.
The code extracts compose_report_card() as a standalone importable function enabling "read once, compose many" optimization, but this architectural pattern is not documented in the Local Observability Service blueprint.
| # The hosted dashboard has no run history of its own — it ships with an | ||
| # EMPTY DuckDB, which answers queries rather than failing them. Reading it | ||
| # here produced "Nothing to grade yet" for machines that were grading | ||
| # fine, so the hosted process refuses to answer from it at all. The real |
There was a problem hiding this comment.
The quality endpoint uses a CLAWMETRY_CLOUD environment variable guard to prevent reading the hosted container's empty DuckDB store, returning honest unavailability messages instead for cloud parity. This cloud-parity behavior is not documented in the Cloud Fleet Dashboard blueprint.
… pinned) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011JYHXKUSfu72qPh2zuq92N
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
… shipped in the release cut from #5427) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011JYHXKUSfu72qPh2zuq92N
…#5428) Claude-Session: https://claude.ai/code/session_011JYHXKUSfu72qPh2zuq92N Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2 participants
No-PRD: Blueprint documents an implementation already captured in the feature blueprint file; requirements are tracked in docs/blueprints/quality-cloud-parity.md.
What was on screen
Same node, same runtime, same week:
Two faults
1. The grade never rode the snapshot. Every card the hosted dashboard renders needs a slice the daemon emits. Quality never had one, so the cloud had nothing to serve — the cloud-parity gate, missed.
2. The hosted container answered anyway. It has a DuckDB file of its own. Empty, but perfectly able to answer, so
query_quality_sessionsreturned[]and the tab reported a working machine as having produced nothing. Verified against production before writing a line:store_available: trueis the tell. The handler already separated "the store said nothing" from "I could not reach the store" — it just could not know its own store was a stranger's.The fix
sync.pyemitsquality:{window_hours, all, byRuntime, thresholds}. The composer moved out of the request handler (routes/quality.py::compose_report_card) so the daemon and the request path build the identical payload.CLAWMETRY_CLOUDset it returns the honest "Quality is graded on your own machine" payload, so an older daemon with no slice produces a true statement instead of a fabricated empty week.Verified
Against the real local store (173 sessions this week):
tests/test_quality_cloud_parity.py— 10 tests: the slice carries node-wide + per-runtime cards; a per-runtime card never contains another runtime's runs; a quiet runtime still gets one; calibration is hoisted; the store is read exactly 3 times whatever the runtime count; the slice never raises; the hosted path says where the grade lives; the local path still reports a genuinely empty week as empty; an unreachable store is not an empty grade; precomputed assessments give the same grade as the request path.Existing quality suites: 67 passing.
Cloud interceptor ships alongside in clawmetry-cloud.
One to remember
The first version of the daemon slice returned
{}silently:sync.pyimportsdatetimeandtimezonebut nottimedelta, and the broadexceptswallowed the NameError. Same shape as the module-scope NameError class we have been bitten by before.🤖 Generated with Claude Code
https://claude.ai/code/session_0138ySWEXGWEayXCPDqWrw21