Skip to content

fix(v7): repair end-to-end runtime and dashboard failures - #76

Merged
9thLevelSoftware merged 4 commits into
mainfrom
codex/v7-end-to-end-acceptance
Nov 12, 2026
Merged

9thLevelSoftware merged 4 commits into
mainfrom
codex/v7-end-to-end-acceptance

Conversation

@9thLevelSoftware

Copy link
Copy Markdown
Owner

Disposable end-to-end acceptance found four failures in production paths: command reminders read the wrong database generation, configured OTLP tracing exported nothing, Windows task restart could stall loading NumPy in a worker, and populated community/graph dashboards rendered empty because the shipped D3 global name did not match the renderers. This change fixes those paths and adds focused regressions, including real OTLP collectors and tests using the actual shipped D3 bundle.

The pass also adds a real core-gateway counsel/idempotency/restart regression and corrects client documentation to describe the installed remind-only hooks accurately. Detailed evidence and artifact hashes are in docs/release/v7/acceptance-2026-11-11.md.

Validation:

  • Full Python suite: 3,957 passed, 2,097 subtests passed; 76 explicit optional/platform skips.
  • Separate configured runs closed all 18 Redis cases, 18 offline E2B SDK cases, three OTLP cases and two public-ingestion cases; four platform-only safeguards also passed on Linux.
  • Ruff lint/format, v7 mypy and release inventory passed; independent correctness and Ponytail reviews found no outstanding issues.
  • Private wheel/sdist installs, Python 3.10/3.12 memory workflows on both transports, actual Redis/model/OTLP integrations, an installed Claude Code workflow, and all six browser dashboard shells passed. The two changed dashboards also passed from the rebuilt installed wheel.
  • Ten-minute concurrent workload: 1,604 recalls, ten writes and ten outcomes; all owned child resources were gone after shutdown.

This is functional acceptance, not final-release certification. OpenCode's provider OAuth refresh returns 401 before MCP and requires local reauthentication. Live E2B lacks credentials. Third-party Apps-host compatibility and the release-scale/eight-hour performance gates remain unverified.

Copilot AI balanced review requested due to automatic review settings November 12, 2026 01:59

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

default_db = storage_dir / "daem0nmcp.db"
return default_db if default_db.is_file() else None
try:
return resolve_active_database(storage_dir).path

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: The resolved v7 database is WAL-mode, and the connect below opens it with a bare ?mode=ro URI

resolve_active_database() now returns the active generation database (for a migrated store: migrations/v7/<run>/candidate.db), and every Daem0n database is WAL-mode (daem0nmcp/database.py:104). Opening a WAL-mode file read-only without immutable=1 lets SQLite create -wal/-shm sidecars inside the user's storage directory; when it cannot create them the read fails with SQLITE_CANTOPEN, which _load_active_rules swallows, so the hook enforces no rules at all (fail-open).

The repo already has the correct helper in the module this file imports: use sqlite3.connect(readonly_database_uri(db_file), uri=True, timeout=2.0) and add readonly_database_uri to the ..storage_activation import. It adds immutable=1 when no sidecars exist, which is the "offline inspection must leave no sidecars" policy in daem0nmcp/workspace.py:383-390.

_NATIVE_PROFILE_MODULES = {
"local": ("qdrant_client",),
"models-local": ("sentence_transformers", "onnx", "onnxruntime"),
"local": ("numpy", "qdrant_client"),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: numpy is not declared by the local extra or by PROFILES["local"], so readiness and the preload now disagree

PROFILES["local"].distributions is ("qdrant-client",) (daem0nmcp/capabilities.py:36) and the local extra installs only qdrant-client, yet this tuple now requires numpy to import. The capability registry therefore reports local: ready while the Windows preload additionally demands a working numpy, so the same install reports two different capability states depending on the platform.

Either declare numpy in the local extra and in PROFILES["local"].distributions so both sources agree, or drop numpy from the local tuple and rely on models-local/late-interaction, whose extras do declare it.

Comment thread daem0nmcp/tracing.py
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import (
OTLPSpanExporter,
)
resource = Resource.create({"service.name": "daem0nmcp"}).merge(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Reuse get_service_name() instead of the hardcoded service name

This module already resolves the service name in one place (get_service_name()), but the resource hardcodes "daem0nmcp" and then re-merges the environment to undo it. Resource.create() already applies OTEL_RESOURCE_ATTRIBUTES and OTEL_SERVICE_NAME, so resource = Resource.create({"service.name": get_service_name()}) yields the same resource (OTEL_SERVICE_NAME wins over OTEL_RESOURCE_ATTRIBUTES per the OTel spec) with one merge instead of two, and the two service-name sources can no longer drift.

if (data.communities.length && globalThis.D3) {
const root = globalThis.D3.hierarchy(hierarchy(data)).sum(value => value.value || 0);
globalThis.D3.treemap().size([960, 540]).padding(3)(root);
if (data.communities.length && globalThis.d3) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: A missing D3 bundle now renders "No communities found."

After this fix the else branch is reachable with populated data whenever the bundle fails to load, and it states that there are no communities — a false statement about the data rather than a load failure. Split the conditions: keep "No communities found." for an empty data.communities, and show a distinct message (e.g. "Community map is unavailable.") when globalThis.d3 is missing.

scope = {"workspace_id": workspace.workspace_id}
service = f"daem0n-observability-{transport}"
with _collector(protocol) as (endpoint, messages):
control = TracerProvider(resource=Resource.create({"service.name": "control"}))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Make the positive control independent of the ambient OTEL_* environment

server_environment() strips OTEL_* from the child process, but this control provider is built in the parent, where Resource.create() merges the ambient OTEL_SERVICE_NAME/OTEL_RESOURCE_ATTRIBUTES. A CI image that exports them can change the control's service name, and the assertion below then fails with "the collector received nothing" instead of a real diagnosis.

Clear them for the parent before building the control, as tests/test_tracing.py:45-53 already does.

@kilo-code-bot

kilo-code-bot Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Correctness / Safety Findings

No correctness or safety findings.

Ponytail Review

Ponytail: Lean already. Ship.

Ponytail net: 0 lines.

Suggested Minimal Patch

No patch needed.

Final Merge Guidance

Can merge as-is.

Files Reviewed (18 files)
  • README.md
  • daem0nmcp/api/v7/launcher.py
  • daem0nmcp/api/v7/production.py
  • daem0nmcp/claude_hooks/pre_bash.py
  • daem0nmcp/tracing.py
  • daem0nmcp/ui/static/renderers/community.js
  • daem0nmcp/ui/static/renderers/graph.js
  • docs/release/v7/acceptance-2026-11-11.md
  • docs/release/v7/progress.md
  • docs/release/v7/tool-process-coverage.md
  • docs/v7-operations.md
  • tests/api_v7/test_process_gateway.py
  • tests/api_v7/test_process_observability.py
  • tests/api_v7/test_production.py
  • tests/js/test_ui_renderers.mjs
  • tests/test_claude_hooks_pre_bash.py
  • tests/test_tracing.py
  • tests/test_v7_dreaming_runtime.py
Previous Review Summaries (2 snapshots, latest commit 3771ad5)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 3771ad5)

Status: 1 Issue Found | Recommendation: Merge

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 1
Issue Details (click to expand)

SUGGESTION

File Line Issue
daem0nmcp/ui/static/renderers/community.js 88 A populated payload with a missing D3 bundle renders "No communities found." (fallback text at line 124), which states a false fact about the data instead of the load failure. Optional wording fix.

Earlier Findings, Re-Verified Against Current HEAD

  • daem0nmcp/claude_hooks/pre_bash.py:68 — no action needed. readonly_database_uri() must not be used by this live hook: daem0nmcp/workspace.py:383-385 requires live readers to take part in SQLite locking because immutable/no-lock readers are unsafe across checkpoints, and readonly_database_uri() adds immutable=1 whenever the sidecars are absent. The acceptance evidence at docs/release/v7/acceptance-2026-11-11.md:167-174 matches that policy. The hook's ?mode=ro connect is the repository's live-reader convention; only the URI string is hand-rolled, and sqlite_file_uri(db_file, mode="ro") — already used by the sibling hook daem0nmcp/claude_hooks/session_start.py:73 — is the shared helper for it.
  • daem0nmcp/api/v7/production.py:245 — no action needed. qdrant-client declares numpy as a hard dependency (uv.lock:3788-3797), so PROFILES["local"].distributions = ("qdrant-client",) already covers the module the local profile preloads; the two sources cannot disagree.
  • tests/api_v7/test_process_observability.py:109 — fixed by 3771ad5. Clearing the ambient OTEL_* values through monkeypatch before the control provider is built removes the inherited OTEL_SDK_DISABLED and exporter settings, and the child process still receives only the explicit overrides because server_environment() allowlists the parent environment.
  • daem0nmcp/tracing.py:111 — no action needed. The repository's own acceptance test (tests/api_v7/test_process_observability.py:127-129) sets OTEL_SERVICE_NAME and service.name=ignored in OTEL_RESOURCE_ATTRIBUTES and filters the exported spans by the OTEL_SERVICE_NAME value, and the acceptance run reports all three collector cases passing. The trailing OTELResourceDetector().detect() merge re-applies the environment's attributes after the literal, so the exported service name already follows the environment. Substituting get_service_name() would only change the unrelated case where OTEL_SERVICE_NAME is unset.

Ponytail Review

tests/api_v7/test_process_observability.py:L92: reuse: the OTEL_* clearing loop repeats the identical loop in tests/test_tracing.py:49. Move it to one fixture in tests/conftest.py and call it from both tests (optional).

Ponytail net: -4 lines.

Suggested Minimal Patch

One file, one line in daem0nmcp/ui/static/renderers/community.js (line 124), which distinguishes an empty payload from a missing library:

      message.textContent = data.communities.length
        ? "Community map is unavailable."
        : "No communities found.";

Nothing else needs to change.

Fix these issues in Kilo Cloud

Files Reviewed (18 files)
  • README.md - 0 issues
  • daem0nmcp/api/v7/launcher.py - 0 issues
  • daem0nmcp/api/v7/production.py - 0 issues
  • daem0nmcp/claude_hooks/pre_bash.py - 0 issues
  • daem0nmcp/tracing.py - 0 issues
  • daem0nmcp/ui/static/renderers/community.js - 1 issue
  • daem0nmcp/ui/static/renderers/graph.js - 0 issues
  • docs/release/v7/acceptance-2026-11-11.md - 0 issues
  • docs/release/v7/progress.md - 0 issues
  • docs/release/v7/tool-process-coverage.md - 0 issues
  • docs/v7-operations.md - 0 issues
  • tests/api_v7/test_process_gateway.py - 0 issues
  • tests/api_v7/test_process_observability.py - 0 issues
  • tests/api_v7/test_production.py - 0 issues
  • tests/js/test_ui_renderers.mjs - 0 issues
  • tests/test_claude_hooks_pre_bash.py - 0 issues
  • tests/test_tracing.py - 0 issues
  • tests/test_v7_dreaming_runtime.py - 0 issues

Previous review (commit 3fc2618)

Status: 5 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 4
Issue Details (click to expand)

WARNING

File Line Issue
daem0nmcp/claude_hooks/pre_bash.py 68 The resolved active database is WAL-mode, but the connect opens it with a bare ?mode=ro URI: SQLite can create -wal/-shm sidecars in the user's storage directory, and when it cannot, the read fails and the hook silently enforces no rules. Use the existing readonly_database_uri() helper.

SUGGESTION

File Line Issue
daem0nmcp/api/v7/production.py 245 local now preloads numpy, but neither the local extra nor PROFILES["local"] declares it, so capability readiness and the preload disagree and the reported status becomes platform-dependent.
daem0nmcp/tracing.py 111 The resource hardcodes "daem0nmcp" and re-merges the environment to undo it; reuse the module's own get_service_name().
daem0nmcp/ui/static/renderers/community.js 88 A missing D3 bundle renders "No communities found." for a populated payload.
tests/api_v7/test_process_observability.py 104 The positive control is built in the parent process and inherits ambient OTEL_SERVICE_NAME/OTEL_RESOURCE_ATTRIBUTES.

Ponytail Review

daem0nmcp/tracing.py:L111: reuse: the hardcoded "daem0nmcp" duplicates get_service_name() in the same module, and the second env merge exists only to undo it. Replace with Resource.create({"service.name": get_service_name()}).

daem0nmcp/api/v7/production.py:L244: shrink: numpy is repeated in three profile tuples and imported once per profile. Model it as one shared prerequisite fanned out to its dependent profiles (optional).

daem0nmcp/api/v7/production.py:L278: shrink: the "auto-enabled profiles use lightweight spec checks" comment now contradicts the Windows branch right below it, and os.name == "nt" plus the "numpy" literal are re-evaluated for every module of every profile. Fold the two comments and hoist the platform test and module set into named constants (optional).

tests/api_v7/test_process_observability.py:L47: reuse: the loopback HTTP collector repeats the server boilerplate already in tests/api_v7/process_client.py:50. Extract one shared helper (optional).

tests/api_v7/test_process_observability.py:L163: test-shrink: assert all("private-observability-query" not in str(item) ...) scans the protobuf text repr and cannot distinguish "redacted" from "never collected". Assert on the decoded span attributes of the tools/call memory_recall span instead.

Ponytail net: -6 lines.

Suggested Minimal Patch

One file, one line: in daem0nmcp/claude_hooks/pre_bash.py, add readonly_database_uri to the ..storage_activation import and open the resolved database with sqlite3.connect(readonly_database_uri(db_file), uri=True, timeout=2.0). The four suggestions are independent single-line edits (production.py:245, tracing.py:111, community.js:88, test_process_observability.py:104); none needs a new dependency.

Fix these issues in Kilo Cloud

Files Reviewed (18 files)
  • README.md - 0 issues
  • daem0nmcp/api/v7/launcher.py - 0 issues
  • daem0nmcp/api/v7/production.py - 1 issue
  • daem0nmcp/claude_hooks/pre_bash.py - 1 issue
  • daem0nmcp/tracing.py - 1 issue
  • daem0nmcp/ui/static/renderers/community.js - 1 issue
  • daem0nmcp/ui/static/renderers/graph.js - 0 issues
  • docs/release/v7/acceptance-2026-11-11.md - 0 issues
  • docs/release/v7/progress.md - 0 issues
  • docs/release/v7/tool-process-coverage.md - 0 issues
  • docs/v7-operations.md - 0 issues
  • tests/api_v7/test_process_gateway.py - 0 issues
  • tests/api_v7/test_process_observability.py - 1 issue
  • tests/api_v7/test_production.py - 0 issues
  • tests/js/test_ui_renderers.mjs - 0 issues
  • tests/test_claude_hooks_pre_bash.py - 0 issues
  • tests/test_tracing.py - 0 issues
  • tests/test_v7_dreaming_runtime.py - 0 issues

Reviewed by minimax-m3 · Input: 46.7K · Output: 15.4K · Cached: 535.4K

Review guidance: REVIEW.md from base branch main

@9thLevelSoftware
9thLevelSoftware merged commit 6fccf2b into main Nov 12, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants