You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Implements the actual shard split from the approved #112 design, building on #113 Phase A's catalog (PR #119). Original scope below, refined by two
detailed scoping/implementation passes this session — see PR #124 (holds
eviction-wiring fix) and PR #125 (shard-aware service-layer resolution).
The record-locator gap for individual rows — PR Make every service function shard-aware, without cutting over shard assignment yet #125 adds reserveRecordLocator/releaseRecordLocator/resolveShardForParent/ resolveShardForRecord to the catalog, closing the gap where write_record/delete_record/hold_records/release_records only ever
receive a bare recordId with no parent hint.
Every service-layer function's resolveWorkspaceContext() call derives
a real selector — records.ts, collections.ts, holds.ts, search.ts,
and permissions.ts's requireAccessibleRecord all resolve their actual
target shard via the catalog now, instead of defaulting. Proven correct
for a genuinely separate shard via tests that manually construct one
(PR Make every service function shard-aware, without cutting over shard assignment yet #125), without cutting over shard assignment yet — see below.
The cross-shard holds design fork — hold_records/release_records
now group recordIds by resolved shard and operate against each shard's
own Awareness, merging results (a cross-document agent batch is a stated
acceptance criterion — collaboration.md).
src/lib/data/records.ts and src/lib/server/audit-observer.ts turned
out to need no changes for Collection shards specifically: a
per-collection shard doc with the same internal map shape (a collections
map with one entry, a records map with just that collection's rows)
works correctly unchanged, and topLevelMaps stays correct since the map
names don't change.
What's left
The actual shard-assignment cutover.createCollection still
deliberately assigns shardId: 'default' — PR Make every service function shard-aware, without cutting over shard assignment yet #125's resolution
mechanism is proven correct but not yet exercised in production. Flipping
this over requires, together, in one PR (a genuine flag-day cutover, no
safe partial version — the browser's live Table view would silently lose
sync, or the SSR page would show nothing, the moment a Collection's
content moves to a shard the client doesn't know to connect to):
createCollection assigns a real, distinct shardId per Collection.
src/lib/server/attach-ws.ts: parse the WS room-name path segment into
a real, catalog-validated shard selector instead of discarding it (the
known gap from issue Workspace-scoped CRDT registry and authenticated sync routing #30). In Phase 0 (no auth at all, by design — see
README.md's "Out of scope" section), the reasonable trust model is:
validate a client-supplied room name against the catalog's real shard
locator rather than trust it verbatim — equivalent trust to what
already exists everywhere else in Phase 0, just made explicit.
src/lib/client/yjs-client.ts (currently a bare module singleton, one
hardcoded 'workspace' room) + the 8+ Svelte components/routes that
call it directly (TableCollectionView.svelte, CalendarCollectionView.svelte, BoardCollectionView.svelte, FieldMenu.svelte, FieldManagerDialog.svelte, Sidebar.svelte, CollectionViewBlock.svelte, table/[id]/+page.svelte, doc/[id]/+page.svelte) need real per-shard room connections.
src/routes/table/[id]/+page.server.ts (bypasses the service layer
already, pre-existing) needs to resolve and hand the client the real
shard selector.
Document sharding. A separate, larger piece than Collections —
Documents have hierarchy and recursive delete, unlike Collections, so src/lib/data/records.ts's parentKindOf, deleteDocument's recursive
descendant delete, and repairEmbeddedViewsAfter*Removal (currently
whole-doc scans) need to become catalog-driven cross-shard operations.
src/lib/mcp/tokens.ts: add workspace/shard scoping to access_tokens'
allowlists (currently flat, workspace-unaware document/collection id
arrays) — needed once more than one workspace is real, not strictly
blocking the Collection-shard cutover above.
Not in scope: the SSE catalog feed (Phase C, #121), lazy load/unload
(Phase D, #122), and the capacity benchmark re-run (Phase E, #123).
Done when: no single global content Y.Doc path remains — every boundary
(routes, services, MCP, WebSocket, holds, audit) resolves and operates on
the correct per-Document/per-Collection shard.
Implements the actual shard split from the approved #112 design, building on
#113 Phase A's catalog (PR #119). Original scope below, refined by two
detailed scoping/implementation passes this session — see PR #124 (holds
eviction-wiring fix) and PR #125 (shard-aware service-layer resolution).
Done
holds.ts'sinitHoldEvictionprocess-global one-shot guard bug —fixed in PR fix: wire hold eviction per-Awareness-instance, not process-wide #124 with a per-
Awareness-instanceWeakSetguard.reserveRecordLocator/releaseRecordLocator/resolveShardForParent/resolveShardForRecordto the catalog, closing the gap wherewrite_record/delete_record/hold_records/release_recordsonly everreceive a bare
recordIdwith no parent hint.resolveWorkspaceContext()call derivesa real selector —
records.ts,collections.ts,holds.ts,search.ts,and
permissions.ts'srequireAccessibleRecordall resolve their actualtarget shard via the catalog now, instead of defaulting. Proven correct
for a genuinely separate shard via tests that manually construct one
(PR Make every service function shard-aware, without cutting over shard assignment yet #125), without cutting over shard assignment yet — see below.
hold_records/release_recordsnow group
recordIdsby resolved shard and operate against each shard'sown Awareness, merging results (a cross-document agent batch is a stated
acceptance criterion — collaboration.md).
src/lib/data/records.tsandsrc/lib/server/audit-observer.tsturnedout to need no changes for Collection shards specifically: a
per-collection shard doc with the same internal map shape (a
collectionsmap with one entry, a
recordsmap with just that collection's rows)works correctly unchanged, and
topLevelMapsstays correct since the mapnames don't change.
What's left
createCollectionstilldeliberately assigns
shardId: 'default'— PR Make every service function shard-aware, without cutting over shard assignment yet #125's resolutionmechanism is proven correct but not yet exercised in production. Flipping
this over requires, together, in one PR (a genuine flag-day cutover, no
safe partial version — the browser's live Table view would silently lose
sync, or the SSR page would show nothing, the moment a Collection's
content moves to a shard the client doesn't know to connect to):
createCollectionassigns a real, distinctshardIdper Collection.src/lib/server/attach-ws.ts: parse the WS room-name path segment intoa real, catalog-validated shard selector instead of discarding it (the
known gap from issue Workspace-scoped CRDT registry and authenticated sync routing #30). In Phase 0 (no auth at all, by design — see
README.md's "Out of scope" section), the reasonable trust model is:
validate a client-supplied room name against the catalog's real shard
locator rather than trust it verbatim — equivalent trust to what
already exists everywhere else in Phase 0, just made explicit.
src/lib/client/yjs-client.ts(currently a bare module singleton, onehardcoded
'workspace'room) + the 8+ Svelte components/routes thatcall it directly (
TableCollectionView.svelte,CalendarCollectionView.svelte,BoardCollectionView.svelte,FieldMenu.svelte,FieldManagerDialog.svelte,Sidebar.svelte,CollectionViewBlock.svelte,table/[id]/+page.svelte,doc/[id]/+page.svelte) need real per-shard room connections.src/routes/table/[id]/+page.server.ts(bypasses the service layeralready, pre-existing) needs to resolve and hand the client the real
shard selector.
Documents have hierarchy and recursive delete, unlike Collections, so
src/lib/data/records.ts'sparentKindOf,deleteDocument's recursivedescendant delete, and
repairEmbeddedViewsAfter*Removal(currentlywhole-doc scans) need to become catalog-driven cross-shard operations.
src/lib/mcp/tokens.ts: add workspace/shard scoping toaccess_tokens'allowlists (currently flat, workspace-unaware document/collection id
arrays) — needed once more than one workspace is real, not strictly
blocking the Collection-shard cutover above.
Not in scope: the SSE catalog feed (Phase C, #121), lazy load/unload
(Phase D, #122), and the capacity benchmark re-run (Phase E, #123).
Done when: no single global content Y.Doc path remains — every boundary
(routes, services, MCP, WebSocket, holds, audit) resolves and operates on
the correct per-Document/per-Collection shard.