Skip to content

No writer bounds or warns on episode body size (em-store/em-revise/em-consolidate/em-promote/em-topic-tracks) #595

Description

@lantisprime

No substrate writer bounds or warns on episode body size. Verified 2026-07-25:

$ grep -n "length\|byteLength\|size\|MAX_\|limit\|bound" scripts/em-store.mjs
# every match is argv parsing or array-emptiness — no size check

$ grep -n "length\|byteLength\|size\|MAX_\|limit\|bound" scripts/em-revise.mjs
# same

Why this matters now

This asymmetry is the root cause of the read-truncation defect fixed in the EMR-1 branch. em-search --read carried a 49152 serialized-byte cap while the writers carried nothing, so the store accepted bodies it then refused to return, discarding the remainder silently.

Removing the read cap fixes the data loss. It does not make unbounded growth visible. The flagship multiagent playbook reached 65,450 bytes with nothing warning about it, and RFC-011:114 was written on the assumption it would stay "~24KB serialized and unaffected".

Current state: 10 active episodes in the local store exceed the old 49152 bound, largest 20260619-104210-…-142d at 103,660 bytes.

Proposed fix

A stderr warning, exit code unchanged, when a stored body exceeds a stated threshold. Never a rejection — ten episodes already exceed it and rejection would strand stored data.

Same-class enumeration (all five deferred together, none left silently uncovered)

Writer Can produce an oversized body? Note
em-store yes direct write
em-revise yes direct write
em-consolidate yes the actual culprit — folded 2 episodes into the 65,450-byte …-9152
em-promote yes derived global lesson write-back
em-topic-tracks yes derived global lesson write-back

5-field DEFER record (from docs/plans/em-search-read-paging.md §17)

  1. Run the scenario — confirmed above; neither writer has any size check.
  2. Spec check — no spec line requires a write-time warning. RFC-011:114 requires one only for a selected playbook's file size, already implemented at em-trigger-index.mjs:244. Deferral permitted.
  3. History check — the round-2 RFC-011 panel (GLM N2, kimi F16) found this same asymmetry and answered it at the read, never at the write. That choice is what shipped the bug.
  4. Same-class check — table above; all five deferred together.
  5. Residual risk — nil for correctness once the read cap is removed, since every episode becomes retrievable in full. The residual is that unbounded growth stays invisible at creation time.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions