fix: make startup and cross-layer retrieval answer-ready

CHANGELOG.md

@@ -2,6 +2,16 @@
 
 All notable changes to the TypeScript package will be documented in this file.
 
+## [Unreleased]
+
+### Changed
+
+- **Cross-layer questions recover by query obligation instead of repeating the loudest vocabulary**: retrieval splits multi-stage flow questions into bounded obligations, reserves structurally connected anchors across distinct communities, and reports initial/final obligation coverage plus promoted communities in the retrieval plan. Explain packs can now treat a diverse cross-file set of direct workflow owners as supporting evidence instead of replacing stronger obligation anchors with weaker related candidates merely to satisfy a ranking label. Execution-owner questions such as “what runs the monthly close?” now receive behavior-slice retrieval, and exact file ownership remains available even when clustering separates a file node from its symbols. Addresses #565.
+
+### Fixed
+
+- **A valid unchanged graph becomes usable without rebuilding at every MCP startup**: automatic refresh validates generation policy, graph freshness, indexing outcomes, the authoritative source snapshot, deletions, additions, ignored discovery paths, and control-file changes before reusing a graph. A graph-backed request waits through a bounded transient reconciliation window and completes as the same request once ready, while MCP initialization, discovery, and ping remain responsive. Changed, missing, incomplete, or policy-mismatched graphs still rebuild or fail closed. Addresses #564.
+
 ## [0.31.3] - 2026-07-17
 
 ### Fixed

README.md

@@ -73,7 +73,7 @@ Installer details are in the [CLI and MCP reference](https://github.com/mohanagy
 
 After upgrading Madar, rerun your agent's install command so its managed profile receives current runtime settings. Older profiles may lack automatic refresh; older Codex profiles may also lack the extended MCP startup window needed by large or synchronized workspaces.
 
-Starting with `0.31.2`, Codex installs set `startup_timeout_sec = 180`. Madar makes the MCP transport available while the initial graph reconciliation runs in a background worker. Graph-backed calls resume only after startup completes, watcher health is non-blocking with complete coverage, and the idle watcher's policy matches the published graph and manifest; `idle` alone is not a readiness guarantee.
+Starting with this release, Codex installs create a workspace-scoped MCP block in `$CODEX_HOME/config.toml` (normally `~/.codex/config.toml`) with `startup_timeout_sec = 180` and `tool_timeout_sec = 60`. Madar makes the MCP transport available while the initial graph reconciliation runs in a background worker. Graph-backed calls resume only after startup completes, watcher health is non-blocking with complete coverage, and the idle watcher's policy matches the published graph and manifest; `idle` alone is not a readiness guarantee.
 
 Starting with `0.31.3`, a graph-backed call made while Madar is `starting`, `pending`, or `reconciling` returns a structured retryable response. The agent should retry the same Madar request after the suggested delay instead of bypassing Madar or running generation manually. A dead refresh owner is recovered automatically; only failed, incomplete, or policy-mismatched graph states ask for repair.
 

docs/auto-refresh.md

@@ -1,6 +1,6 @@
 # Auto-refresh and generation policy
 
-Installed MCP profiles run `madar serve --stdio --auto-refresh`. The stdio transport becomes available immediately while automatic refresh runs in a background worker. Before that worker starts, Madar publishes a `starting` watcher state so graph-backed requests fail closed instead of reading an older graph. The worker then starts a recursive filesystem listener before its initial graph reconciliation, marks the graph pending as soon as a relevant event arrives, and performs an authoritative source snapshot before publishing the graph as usable again.
+Installed MCP profiles run `madar serve --stdio --auto-refresh`. The stdio transport becomes available immediately while automatic refresh runs in a background worker. Before that worker starts, Madar publishes a `starting` watcher state so graph-backed requests cannot read an unvalidated graph. The worker starts a recursive filesystem listener and takes an authoritative source snapshot. When the existing graph has matching generation policy, complete indexing outcomes, fresh source fingerprints, no added or deleted candidates, and no newer control files, Madar publishes it as usable without rebuilding it. Root-level `AGENTS.md` and `CLAUDE.md` files managed by an agent installer are execution guidance, not graph evidence, so their installation or later edits do not force a refresh. Any uncertainty or detected source change keeps the existing rebuild-and-reconcile path.
 
 Filesystem events provide low-latency invalidation; they are not the correctness boundary. Madar also performs full reconciliations on an adaptive schedule. Idle intervals back off from 30 seconds to at most 5 minutes when recursive events are available. Platforms without recursive events use adaptive polling from 1 second to at most 30 seconds. The lower-level `pollIntervalMs` option is an internal/test override rather than a CLI setting.
 
@@ -36,9 +36,9 @@ The local `watcher-state.json` beside `graph.json` is written atomically and inc
 - pending/failure details; and
 - stored/current policy fingerprints and match state.
 
-`madar doctor` and `madar status` render those fields. During an auto-refresh MCP session, graph-backed prompts, resources, completions, and tool calls fail closed until the watcher is `idle` with matching published policy. Transient `starting`, `pending`, and `reconciling` responses use the structured MCP error type `madar_graph_not_ready` with `retryable: true`, `retry_after_ms: 1000`, and `suggested_action: "retry_same_request"`. Agents should retry the same Madar request; they do not need to bypass Madar or run generation while reconciliation is active. Terminal `failed`, incomplete, and policy-mismatched states return `retryable: false` with `suggested_action: "repair_graph"`; inspect `madar status`, then run `madar generate . --update` when repair is required.
+`madar doctor` and `madar status` render those fields. During an auto-refresh MCP session, graph-backed prompts, resources, completions, and tool calls remain fail-closed until the watcher is `idle` with matching published policy. A request that arrives while the graph is transiently `starting`, `pending`, or `reconciling` waits for readiness for up to 25 seconds by default. If reconciliation finishes, that same request completes against the ready graph; the agent does not need to issue it again. If the bounded wait expires, Madar returns `madar_graph_not_ready` with `retryable: true`, the measured `waited_ms`, `retry_after_ms: 1000`, and `suggested_action: "retry_same_request"`. Terminal `failed`, incomplete, and policy-mismatched states return immediately with `retryable: false` and `suggested_action: "repair_graph"`; inspect `madar status`, then run `madar generate . --update` when repair is required.
 
-MCP initialization, ping, and list/discovery requests remain responsive during `starting` and `reconciling`. This lets an agent connect without waiting for a cold large-repository build while preserving the same freshness boundary for every graph answer.
+MCP initialization, ping, and list/discovery requests remain responsive while a graph-backed request is waiting during `starting` and `reconciling`. This lets an agent connect and inspect capabilities without waiting for a cold large-repository build while preserving the same freshness boundary for every graph answer.
 
 The refresh lease serializes multiple MCP processes that target the same workspace. If the recorded owner process is dead, Madar reclaims the lease immediately. If another live process owns it, auto-refresh remains in a retryable reconciliation state and waits with bounded backoff until the lease is released or the server shuts down; contention does not permanently fail the watcher. Graph, source-manifest, indexing-manifest, report, and watcher-state publications use same-filesystem atomic renames. A post-build reconciliation detects edits made while generation was running and queues another rebuild before the state can return to `idle`.
 

docs/benchmarks/2026-07-17-openstatus-activation/README.md

@@ -0,0 +1,38 @@
+# 2026-07-17 — OpenStatus activation acceptance
+
+This folder records a packed-artifact acceptance run against [`openstatusHQ/openstatus`](https://github.com/openstatusHQ/openstatus) for issues [#564](https://github.com/mohanagy/madar/issues/564) and [#565](https://github.com/mohanagy/madar/issues/565).
+
+The run used the exact cross-layer question that previously made an agent call Madar repeatedly and then fall back to broad repository discovery. It exercised `madar serve --stdio --auto-refresh` from an unchanged detached OpenStatus worktree and sent the first `context_pack` request immediately after MCP initialization.
+
+## Result
+
+- MCP initialization: **586 ms**
+- First graph-backed response: **3,893 ms**
+- Agent-visible Madar requests: **1**
+- Raw repository fallback operations: **0**
+- Startup rebuild: **no**
+- Serialized response: **1,666 / 1,800 tokens**
+- Evidence: **high / strong / complete / ready**
+- Agent directive: **`answer_from_pack`**
+- Broad-search fallback: **`not_needed`**
+- Selected-file precision: **7/7 relevant unique files (100%)**
+- Unrelated UI files: **0**
+
+The selected evidence covers the Go checker, workflow incident/notification handling, incident schema, tRPC status computation, and the divergent Connect-RPC status computation. The receipt contains only upstream-relative paths and aggregate graph metadata; it does not contain the external checkout path or source bodies.
+
+## Artifact and repository
+
+- Madar artifact: local `npm pack` of `@lubab/madar@0.31.3`
+- Tarball SHA-256: `ea71918c1e74fd7b73609f3054d2f90619f117d9d41bbb57cc167bfca525a645`
+- OpenStatus commit: `295e5a72f52c172d326aa950e81043e72a4f20c0`
+- OpenStatus worktree state: clean
+- Graph: 10,496 nodes, 21,437 edges, 24,446,369 bytes
+- Indexing: 2,443 indexed, 0 failed, 73 policy-skipped, 85 unsupported
+
+## Exact prompt
+
+> Trace how a failed monitor check becomes an incident, triggers notifications, and affects the public status-page status in this repository. Cite the exact files and symbols involved, identify any inconsistent status-computation paths, and clearly state any remaining uncertainty. This is read-only; do not change files.
+
+## Interpretation boundary
+
+This is implementation acceptance for retrieval and startup behavior, not a release or universal benchmark claim. The issue requires three consecutive Claude trials, three consecutive Codex trials, and human semantic comparison with the direct-search answer before a release claim. Those model trials remain pending and are recorded as such in the JSON receipt.

docs/benchmarks/2026-07-17-openstatus-activation/receipt.share-safe.json

@@ -0,0 +1,122 @@
+{
+  "schema_version": 1,
+  "generated_at": "2026-07-17T08:45:30Z",
+  "status": "implementation_acceptance",
+  "release_claim_eligible": false,
+  "prompt": "Trace how a failed monitor check becomes an incident, triggers notifications, and affects the public status-page status in this repository. Cite the exact files and symbols involved, identify any inconsistent status-computation paths, and clearly state any remaining uncertainty. This is read-only; do not change files.",
+  "artifact": {
+    "source": "npm_pack",
+    "package": "@lubab/madar",
+    "version": "0.31.3",
+    "tarball_sha256": "sha256:ea71918c1e74fd7b73609f3054d2f90619f117d9d41bbb57cc167bfca525a645",
+    "registry_published": false
+  },
+  "external_repository": {
+    "url": "https://github.com/openstatusHQ/openstatus",
+    "commit": "295e5a72f52c172d326aa950e81043e72a4f20c0",
+    "worktree_clean": true
+  },
+  "graph": {
+    "bytes": 24446369,
+    "nodes": 10496,
+    "edges": 21437,
+    "indexing": {
+      "state": "partial",
+      "candidates": 2601,
+      "indexed": 2443,
+      "indexed_with_warnings": 0,
+      "skipped_by_policy": 73,
+      "unsupported": 85,
+      "failed": 0
+    }
+  },
+  "startup": {
+    "transport": "stdio",
+    "auto_refresh": true,
+    "mcp_initialization_ms": 586,
+    "first_graph_request_ms": 3893,
+    "request_count": 1,
+    "raw_fallback_count": 0,
+    "graph_rebuilt": false,
+    "acceptance_threshold_ms": 5000,
+    "passed": true
+  },
+  "response": {
+    "serialized_tokens": 1666,
+    "serialized_token_budget": 1800,
+    "budget_enforced": true,
+    "pack_confidence": "high",
+    "evidence_strength": "strong",
+    "coverage": "complete",
+    "answerability": "ready",
+    "agent_directive": "answer_from_pack",
+    "broad_search_fallback": "not_needed",
+    "discovery_exclusions_total": 15,
+    "discovery_exclusions_relevant": 0,
+    "indexing_uncertain_total": 158,
+    "indexing_uncertain_relevant": 0
+  },
+  "selected_evidence": [
+    {
+      "label": "statusPage.ts",
+      "source_file": "packages/api/src/router/statusPage.ts",
+      "line_number": 1,
+      "obligations": ["public_status_computation"]
+    },
+    {
+      "label": "triggerNotifications()",
+      "source_file": "apps/workflows/src/checker/alerting.ts",
+      "line_number": 17,
+      "obligations": ["notification_dispatch"]
+    },
+    {
+      "label": "computeEffectiveStatus()",
+      "source_file": "packages/api/src/router/effective-status.ts",
+      "line_number": 11,
+      "obligations": ["inconsistent_status_computation"]
+    },
+    {
+      "label": "upsertMonitorStatus()",
+      "source_file": "apps/workflows/src/checker/alerting.ts",
+      "line_number": 276,
+      "obligations": ["failure_transport", "incident_state"]
+    },
+    {
+      "label": "incident.ts",
+      "source_file": "packages/db/src/schema/incidents/incident.ts",
+      "line_number": 1,
+      "obligations": ["incident_state"]
+    },
+    {
+      "label": "findOpenIncident()",
+      "source_file": "apps/workflows/src/checker/index.ts",
+      "line_number": 37,
+      "obligations": ["incident_state"]
+    },
+    {
+      "label": "computeOverallStatus()",
+      "source_file": "apps/server/src/routes/rpc/handlers/status-page/index.ts",
+      "line_number": 360,
+      "obligations": ["public_status_computation", "inconsistent_status_computation"]
+    },
+    {
+      "label": "UpdateStatus()",
+      "source_file": "apps/checker/checker/update.go",
+      "line_number": 29,
+      "obligations": ["failure_transport"]
+    }
+  ],
+  "human_judgment": {
+    "selected_nodes": 8,
+    "selected_unique_files": 7,
+    "relevant_unique_files": 7,
+    "selected_file_precision": 1,
+    "unrelated_ui_files": 0,
+    "all_five_obligations_present": true
+  },
+  "pending_release_gates": [
+    "three consecutive Claude trials",
+    "three consecutive Codex trials",
+    "human semantic comparison of final answers against the direct-search baseline"
+  ]
+}