Skip to content

CRDT capacity benchmark and resource observability #31

Description

@brylie

Establish a measured operating envelope for Compendium before and during #13s CRDT-sharding work. This complements #24, which is specifically production sync-latency observability; this issue covers state size, memory, initial sync, broadcast fan-out, and recovery behavior.

  • Build repeatable seeded-workspace scenarios spanning documents, blocks, Collection rows, edit churn, concurrent browser clients, and representative MCP agent write bursts
  • Measure per workspace/shard: encoded CRDT state size, cold-load and initial-sync bytes/latency, server heap, client heap, CPU/event-loop delay, update-apply latency, WebSocket fan-out bytes, and snapshot/restart time
  • Record an initial sizing envelope and SLO-oriented thresholds that trigger sharding, partitioning, compaction, or capacity work. Do not adopt arbitrary universal record-count limits without measurement
  • Compare the current global-document baseline with the Y.Doc sharding strategy for multi-space #13 shard design and publish the results in a durable engineering note
  • Add a bounded benchmark/regression check suitable for CI, with larger scenarios runnable on demand
  • Feed the live sync-latency dimension into Sync-latency observability #24 rather than creating a second competing latency metric

Done when: maintainers can reproduce a representative CRDT load profile, see where memory/network/recovery costs grow, and make #13 implementation decisions from recorded measurements rather than intuition.

Run before finalizing #13s shard boundary; keep the resulting ongoing latency signal aligned with #24.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    Status
    Done

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions