Skip to content

Substrate has no continuation edge: a document larger than one episode cannot be modeled #596

Description

@lantisprime

The substrate has no way to express "this episode continues that one". A document larger than one comfortable episode has no modeling answer.

The two edges that exist both mean the opposite

supersedes means replaces. Using it to chain parts would hide part 1:

  • scripts/em-search.mjs:376-378if (!includeSuperseded) results = results.filter(e => e.status !== 'superseded'), so part 1 disappears from every default search.
  • RFC-011 R2 resolves a playbook id to "the terminal active revision", so a pointer at a two-part document silently renders only the tail as though it were the whole thing.

That is a worse failure than truncation, because nothing flags it.

consolidates merges in the opposite direction. It folds N episodes into one larger body. It is not a workaround, it is the cause: 20260722-114639-…-9152 carries "consolidates":["20260721-232108-…-aae5","20260722-014644-…-ba75"], i.e. em-consolidate produced that 65,450-byte body from two sources. The chain terminal …-99bf consolidates five.

So the substrate has a mechanism that creates oversized episodes and none that splits them.

Why this is the root cause

The RFC-011 round-2 panel found the underlying fact — RFC-011:114, "no episode-body size bound exists anywhere in the substrate today" — and responded by capping the read. That produced the truncation defect fixed in EMR-1. A continuation edge is the answer that addresses the modeling gap instead of coping with it.

Cost, stated honestly

This is the most expensive of the three options considered and touches the schema:

  • an index-row field (part_of / continues) and its schema version bump
  • chain-walk handling in em-search.mjs (currently :329,339 handle only supersedes, superseded_by, consolidates)
  • resolution rules in em-trigger-index.mjs so a pointer names part 1 rather than the terminal
  • a decision on how search, ranking, and em-graph treat parts

Per CAPABILITIES.md, a new way of relating episodes is an RFC conversation, not a constant change.

5-field DEFER record (from docs/plans/em-search-read-paging.md §17)

  1. Run the scenario — confirmed absent; the chain walk at em-search.mjs:329,339 handles only the three edges above.
  2. Spec check — no R-number requires it. RFC-011 R7 requires a single-episode read, not a multi-episode document model.
  3. History check — the round-2 panel found the size-bound gap and addressed it at the read; a split was never proposed.
  4. Same-class check — class is the five writers that can produce oversized bodies (see the write-time-warning issue); all deferred together.
  5. Residual risk — graceful. After the EMR-1 read fix every episode is retrievable in full, so nothing is lost. The residual is modeling debt: em-consolidate can keep folding N episodes into one unbounded body.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions