ci: run the graph canary after both deploys settle

[creatornader]

Both follow-ups deferred from #599, which fixed the graph-node OOM itself.

1. The canary ran in parallel with the deploy it was gating

deploy-log-node (which owned the canary step) and deploy-graph-node both depend only on changes, so they ran concurrently with no ordering. A commit touching a shared path — packages/mcp/*, package.json, pnpm-lock.yaml, pnpm-workspace.yaml, tsconfig.base.json, .dockerignore — sets both service flags, and the canary could then poll graph-node while its machine was restarting into the new release. graph-node replays its whole record archive on start, so that window serves 502s and the canary fails on a system that is fine.

To be clear about attribution: this did not cause the 2026-07-29 flap. Those runs were apps/dashboard/* changes, which set log_node only, so graph-node never deployed. It is a separate latent source of false failures in the same check.

The canary now lives in its own graph-canary job that needs both deploys.

It also now runs when only graph-node deployed. The canary exercises log submit through graph index, so a graph-node-only release is exactly the change it should gate. Previously it ran on log-node releases only — which is precisely why the trace regression reached production without the canary ever seeing it.

deploy-log-node loses its pnpm install and workspace build; they existed only for the canary, and check-log-smoke.mjs imports node:crypto alone.

The if: uses !cancelled() with explicit guards, since a skipped needs entry otherwise cascades:

log_node	graph_node	canary
true	false	runs (unchanged)
false	true	runs (new)
true	true	runs, after both (the fix)
false	false	skipped
either deploy failed/cancelled		skipped

No branch-protection impact: the 8 required checks are all from ci.yml and security-scan.yml; deploy-services contributes none.

2. Headroom is finite, and nothing was watching it

#599 removed the trigger and raised the ceiling. It did not change the shape that produced the outage.

The pointed detail: graph-node had a disk watchdog, warning at 80% and erroring at 95%. Disk was never the constraint — the archive was 68MB on a 1GB volume — while the heap sat at 91% and killed the process. The service was watching the wrong resource.

Adds the matching heap watchdog, mirroring the disk one: threshold-transition logging only, unref()'d interval, plus one unconditional line per boot so every restart leaves a datapoint. record_count is in the message on purpose — headroom divided by records/day is the runway.

Thresholds are 70% / 85%, below disk's 80/95, because heap has no graceful degradation: crossing the V8 limit aborts the process immediately, per-request allocation sits on top of the live set so the usable ceiling is under 100%, and recovery costs a full archive replay that serves errors throughout.

The threshold logic is its own module (src/heap-watchdog.ts) so it is unit-tested. An alerting path that never fires reads as an all-clear, which is worse than no alert.

Filed as P059 rather than fixed here, because the structural choice is a real decision: disk-backed store, bounded working set, sharding by context_id, or continuing to buy machine size. Measured at filing: 112,781 records, 439MB RSS, 780MB ceiling, ingest ~4k–7k records/day at ~3.4KB heap each — roughly 15–22MB/day, so weeks of runway, not quarters. persistence.ts already named ~10^5 records as the boundary; the log passed it in July 2026. The watchdog's warn threshold is P059's intended trigger.

P059 also records the two properties any successor must preserve: §3.2.4 edge derivation stays deterministic over the full record set, and the §1.9 revocation registry stays a global scan. A bounded working set is the option most in tension with both.

Verification

• services/graph-node: 124/124 tests pass (115 before, +9 heap-watchdog).
• Watchdog verified against a running server, not just asserted: atrib-graph: heap 29MB/168MB (17.1%) at 0 records.
• Workflow YAML parsed and the if: truth table walked case by case (table above).
• pnpm doc-sync: 14 checks pass. It caught two bare spec refs in the P059 text, now inline-linked.
• Cross-references are bidirectional: persistence.ts and fly.toml both point at P059 and the watchdog.