Skip to content

feat(daemon): supervisor-owned rlm spawn ledger as family authority - #1387

Merged
sethkarten merged 12 commits into
mainfrom
feat/rlm-spawn-ledger
Aug 14, 2026
Merged

feat(daemon): supervisor-owned rlm spawn ledger as family authority#1387
sethkarten merged 12 commits into
mainfrom
feat/rlm-spawn-ledger

Conversation

@snimu

@snimu snimu commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

What this is

A daemon-owned record of subagent spawns, so "which sessions form this agent's family" is answered by reading one file the daemon wrote itself, instead of re-deriving the topology from session headers and registry files every time.

The daemon admits every rlm spawn, performs every rename, and records every deletion. At those moments it knows the parent session file, the child session file, the child id, the depth, and the name — firsthand. This PR records that knowledge in an append-only ledger at <agentDir>/rlm-ledger/<hash-of-sessions-dir>.jsonl and serves family/sibling queries from it.

Why

The v0.8 catalog work (#1333/#1370) derives saved-session families by walking session headers and per-parent rlm-subagents.jsonl registries at read time, validating shapes as it goes. Three times in two weeks, real on-disk data violated an assumed shape (fork session headers, registry childIds, legacy depth fields) and the fail-closed validation broke family messaging for the whole profile. Each fix added more shape assumptions. The root problem is structural: topology gets reconstructed from files that many writers produce, so every writer-behavior assumption is a chance to be wrong about real data.

Recording spawn edges at the source removes the reconstruction step entirely. Fork headers, legacy header shapes, and registry field conventions are never consulted. There is no python helper, no body scanning, and it works on plain fs reads and stats (no dir_fd/O_NOFOLLOW, so nothing platform-specific).

What it does

  • rlm-ledger.ts: append-only JSONL ledger, one per sessions dir (0600 file, 0700 dir, outside the sessions dir so transcript-copy workflows never touch it). Records: spawn (parent, child, childId, depth, name), rename, delete (with a reason enum). Last-writer-wins per (childId, child). Bounded reads (32MiB/100k records) that fail loudly on interior corruption but tolerate a torn final line from an interrupted append. Unknown ops at v1 are skipped, not fatal.
  • Wiring: spawn recorded at admission (awaited, so an admitted spawn is durably on disk), delete at deletion (awaited, self-heals a crash-lost delete on retry), rename at all three rename paths (worker active-session, worker offline, supervisor offline).
  • Reader: family()/siblings() — ledger edges plus a bounded readdir of the sessions dir for roots, stat-and-drop reconciliation for sessions deleted behind the daemon's back, depth consistency checked only between ledger-known depths (a contradictory edge is dropped and logged, never the whole family). The supervisor's three catalog.siblings call sites (named create, name reservation) now use it.
  • Seeding: on first use with no ledger, edges are derived once from the existing registries using the same tolerant read the daemon already uses for passive hydration (accepts sub-* ids, absent/zero depths, ignores fork headers). Seed failure degrades to flat families — it never fails closed. Registries keep being written exactly as before; this PR adds an authority source, it does not remove anything.

Numbers (real profile: 785 family members — 179 roots / 523 depth-1 / 83 depth-2, 433MB of sessions)

this PR walk (#1370 head) walk (#1333 as merged)
one-time seed 7.2s
cold family() 1.34s (display fill; topology alone ~53ms) 2.4s throws
warm family() 53ms 1.8s throws
per-message helper processes 0 ~2 ~330

The seeded ledger for that profile is 216KB / 600 records; steady-state growth is ~340 bytes per spawn.

Testing

  • New rlm-ledger.test.ts (15 tests): ledger semantics (LWW, duplicate-path rejection, delete releasing paths, rename), bounded-read and malformed-line behavior (torn final line tolerated, interior corruption fatal, future-op skipped, v2 fatal), depth handling (contradiction drops the edge, nested-daemon roots), seeding from fixtures that mirror the exact registry shape the daemon writes (including legacy rlmDepth: 0 entries and running→completed pairs), seed-failure degradation, memoization, and end-to-end wiring through real daemon internals (spawn at admission, rename, delete reasons, crash-lost delete healing on retry).
  • Supervisor tests cover ledger-backed name reservation and offline rename.
  • Targeted daemon suites: 367 passed / 8 skipped. npm run check clean. The full vitest run has ~93 failures in auth/extensions suites that fail identically on the merge base (env-dependent, unrelated).

Notes and accepted limitations

  • Same-user tampering is out of scope: a local process can edit the ledger just as it can edit registries or headers today. This is mistake-hardening with one daemon-owned source of truth, not a security boundary — same as the walk's real guarantee.
  • Supervisor and workers both append to the same ledger file (small O_APPEND writes; duplicate-path checks are per-process advisory). A fresh profile raced by two processes can double-seed; the records are identical and last-writer-wins collapses them.
  • First spawn on a large pre-existing profile pays the one-time seed inside admission (7s on the 433MB profile above; header-only reads, bounded).
  • A failed spawn append logs rather than failing admission (a TODO marks this for revisit once the ledger is the messaging authority).
  • Sessions from before the ledger that seeding could not attribute appear as flat roots — degraded, not broken.

A follow-up PR (stacked on this one) consolidates the registries' topology role away entirely; this PR deliberately keeps registries byte-identical so it can be reviewed and shipped alone.


Note

Medium Risk
Changes how daemon resolves RLM sibling/name topology and adds fsync-backed ledger I/O on the subagent admission path; spawn-append failures are still logged rather than failing admission (documented TODO).

Overview
Adds a daemon-owned append-only spawn ledger (RlmSpawnLedger in rlm-ledger.ts) keyed per sessions dir under <agentDir>/rlm-ledger/, recording spawn, rename, and delete (with reasons) instead of inferring RLM family topology from session headers and registry walks at read time.

AgentDaemon appends spawns when admitting running subagents, ledger renames on active/offline renames, and deletes after registry tombstones (with self-heal when the registry is already deleted but the ledger edge is still live). Subagent admission now awaits rlmSpawnLedger().flush() so the spawn record is durable before the runtime is returned; cancelled releases pass revoked as the delete reason.

DaemonSupervisor switches name reservation and sibling discovery from catalog.siblings to rlmLedgerSiblings() (ledger-backed), and writes ledger renames for offline rename_saved_session when no worker is attached. Per-parent rlm-subagents.jsonl registries are still written unchanged for other consumers.

The ledger module handles lazy seeding from existing registries, bounded replay, torn-tail repair, and family/siblings queries with file-existence reconciliation; extensive tests cover ledger semantics and daemon/supervisor wiring.

Reviewed by Cursor Bugbot for commit 76c4bf3. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add supervisor-owned RLM spawn ledger to track session family topology in daemon mode

  • Introduces a new append-only ledger (rlm-ledger.ts) that records spawn, rename, and delete events for RLM subagent sessions, keyed per sessions directory.
  • AgentDaemon appends spawn records on subagent admission (with a durable flush before continuing), rename records on both active and saved-session renames, and delete records on subagent removal with a reason ('user' or 'revoked').
  • DaemonSupervisor replaces catalog.siblings() lookups with rlmLedgerSiblings() so sibling/parent topology for name reservation and admission is sourced from the ledger instead of the registry catalog.
  • The ledger supports bounded, tolerant replay with last-writer-wins semantics, torn-line recovery, lazy seeding from per-parent registries, and atomic seed publication.
  • Risk: subagent admission now awaits a ledger flush on the critical path, adding I/O latency to each spawn.

Macroscope summarized 76c4bf3.

snimu added 7 commits August 14, 2026 13:03
- enforce spawn invariants at append (never write what the reader rejects)
- verify depth monotonicity only between ledger-known depths; drop+log a
  contradictory edge instead of failing the whole family
- tolerate one torn final line (no trailing newline) on read; truncate it
  before the next append; interior malformed lines stay fail-closed
- skip v:1 records with unknown ops (forward compat); v!==1 fails loudly
- shared registry seed source (header-id read, tolerant LWW registry parse)
- canonicalSessionPath for all path keys; realpath the sessions dir
- flush() to await durable appends; document multi-writer O_APPEND reality
- supervisor holds its own ledger instance; the three catalog.siblings
  call sites (named create, saved rename reservation, saved-name check)
  now read ledger-backed siblings
- offline rename_saved_session at the supervisor appends a ledger rename
- worker spawn appends are awaited before admission returns (no self-heal
  exists for a lost spawn record)
- ledger delete is appended after the registry tombstone succeeds
- recordRlmSubagentDeletion awaits the ledger delete (tombstone-first
  ordering kept)
- a retried deletion over an existing tombstone finishes a ledger delete
  lost to a crash instead of leaving a permanent ghost live edge
- TODO at the spawn-append failure log: revisit failing admission once
  the ledger is the messaging authority
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts
Comment thread packages/coding-agent/src/modes/daemon/rlm-ledger.ts
Comment thread packages/coding-agent/src/modes/daemon/rlm-ledger.ts
Comment thread packages/coding-agent/src/modes/daemon/rlm-ledger.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts Outdated
- await the ledger rename at the active-session rename write point so a
  rename is durable before its name reservation is released
- sessionRow strips header-claimed parentSessionPath/rlmDepth: topology
  in ledger rows is exclusively ledger-sourced (fork headers no longer
  leak a parent onto root rows)
- seeding is atomic: collect all records, publish via temp file + rename;
  an interrupted seed leaves no ledger file and re-seeds next time; a
  racing live append wins and suppresses the seed
- siblings() falls back to a lone root-shaped row when the target's edge
  was reconciliation-dropped (parent file gone) but the child file exists
- drop the unused public rlmLedgerFamily/rlmLedgerSiblings wrappers on
  AgentDaemon (tests use the ledger via daemon internals)
Comment thread packages/coding-agent/src/modes/daemon/rlm-ledger.ts
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts
Comment thread packages/coding-agent/src/modes/daemon/rlm-ledger.ts
Comment thread packages/coding-agent/src/modes/daemon/rlm-ledger.ts
Comment thread packages/coding-agent/src/modes/daemon/rlm-ledger.ts
- truncateTornTailSync works on raw buffers with byte offsets: string
  indices diverge from byte offsets on multi-byte UTF-8 names, so the
  old truncate could cut into a preceding valid record and poison the
  ledger; also hardened cross-process with a byte-stable double read
  and same-fd fstat/ftruncate (residual race stays documented)
- seed publish uses linkSync (EEXIST => live append wins, seed dropped)
  instead of existsSync+renameSync, whose clobbering rename could lose
  a racing append with no self-heal
Comment thread packages/coding-agent/src/modes/daemon/rlm-ledger.ts

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 9bc0f82. Configure here.

Comment thread packages/coding-agent/src/modes/daemon/rlm-ledger.ts Outdated
snimu added 2 commits August 14, 2026 18:01
…ish fallback

- readAllSync and the torn-tail repair enforce RLM_LEDGER_MAX_BYTES
  before any file-sized allocation, throwing the same loud bounded-read
  error as replaySync (outside the swallowing repair try-block)
- seed publish falls back to check-then-rename when linkSync fails with
  anything but EEXIST (filesystems without hard links); EEXIST still
  means the racing live append wins; fallback path is logged
Comment thread packages/coding-agent/src/modes/daemon/rlm-ledger.ts Outdated
A seed past RLM_LEDGER_MAX_BYTES/RECORDS would publish a ledger every
replaySync refuses to read. Check the single serialized payload against
both bounds before publishing and skip seeding entirely (flat families,
the documented degradation mode) — logged, not thrown, so the guard
cannot recreate the seedAttempted-sticks failure shape.
@sethkarten
sethkarten self-requested a review August 14, 2026 18:42
@sethkarten
sethkarten merged commit 97b994c into main Aug 14, 2026
18 checks passed
@sethkarten
sethkarten deleted the feat/rlm-spawn-ledger branch August 14, 2026 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants