Skip to content

Y.Doc sharding strategy for multi-space #13

Description

@brylie

Compendium currently has one process-global Y.Doc and Awareness instance. That is correct for the local, single-workspace MVP, but multi-space (#6) cannot safely remain a global replication unit: opening one page should not load, synchronize, or broadcast every page in the workspace.

This issue decides and delivers the CRDT boundary for the multi-space architecture. #30 supplies the workspace-scoped registry/routing foundation; #31 establishes the measured sizing envelope before and after the change.

  • Define the shard model: a small workspace catalog Y.Doc for spaces, hierarchy, titles, permissions, and view metadata; independently subscribed content Y.Docs per Document (or a documented bounded grouping); Collection data in its own Y.Doc, partitionable if required
  • Specify the ownership, lifecycle, and routing rules for a workspace/shard: authenticated WebSocket upgrade, explicit workspace and shard context, lazy load, flush, and safe idle unload
  • Make UI and MCP access shard-aware. A page or Collection subscription must receive only its own CRDT updates; services must not reach a process-global getYDoc()
  • Scope Awareness/presence to the active content shard. Preserve cross-record agent holds through workspace-level coordination and project hold state into the relevant open shards without leaking it to unrelated spaces
  • Replace the single latest-snapshot model with workspace-and-shard-keyed persistence. Decide whether append-only update logging plus periodic snapshots is needed before deployment; document compaction, recovery, and migration behavior
  • Migrate an existing single-workspace Y.Doc to one default Space and the new shards without losing record IDs, document hierarchy, links, content, or audit attribution
  • Add end-to-end isolation coverage: a client or MCP token authorized for one space/shard cannot synchronize, observe awareness from, search, or mutate another
  • Document the horizontal-scaling boundary: multiple application instances must route a shard to a single owner or use a shared synchronization/persistence backplane; independent mutable in-memory replicas are not valid

Done when: #6 can create a second space whose clients only subscribe to that space catalog and explicitly opened content shards; an edit and presence update never reach an unrelated space; existing data migrates losslessly; and the measured resource envelope from #31 is recorded against the shipped design.

Tightly coupled to #6. Build on #30, and use #31 to validate the choice. See data-model.md §2, persistence.md, collaboration.md, and architecture.md.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    Status
    In progress

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions