Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/specifications/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Canonical specification for each implemented subsystem, translating [`prd.md`](../prd.md) into concrete engineering decisions. Split by feature rather than kept as one document, so a change to one subsystem doesn't require editing (or risk going stale in) an unrelated one.

- [`architecture.md`](./architecture.md) — process model (one Node process, `/ws` + `/mcp`) and SvelteKit app structure.
- [`workspace-sharding.md`](./workspace-sharding.md) — proposed Workspace catalog, content-shard, and committed-catalog-write/SSE design for #112; pending #31 measurement before approval.
- [`workspace-sharding.md`](./workspace-sharding.md) — approved Workspace catalog, content-shard, and committed-catalog-write/SSE design (#112); #113 must still prove it with real shard-aware transport measurements before it's treated as implemented.
- [`data-model.md`](./data-model.md) — the Record/Document/Collection types and their Yjs mapping.
- [`collaboration.md`](./collaboration.md) — presence and holds, built on Yjs Awareness.
- [`mcp-tools.md`](./mcp-tools.md) — the MCP tool surface and permission scoping.
Expand Down
7 changes: 4 additions & 3 deletions docs/specifications/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,10 @@ architecture boundary is therefore document- and Collection-level Yjs shards,
with workspace navigation/catalog updates kept outside those content shards.
That is a direction validated by the baseline, not a claim that shard-aware
routing is shipped: Phase 0 still resolves every live connection to the one
default shard. #112 defines the proposed catalog/shard design, and #113 must
prove it with real per-shard transport measurements before this architecture
can describe it as implemented.
default shard. [`workspace-sharding.md`](./workspace-sharding.md) (#112)
records the **approved** catalog/shard design, and #113 must prove it with
real per-shard transport measurements before this architecture can describe
it as implemented.

**Why HTTP MCP transport, not stdio:** Claude Desktop, Claude Code, and ChatGPT all support pointing at a remote MCP server via URL + bearer token in their own config (per the docs linked in the PRD). That matches the "simple local access token" decision better than stdio, which would mean each client spawns and manages its own subprocess — more moving parts for no benefit when everything's already running as one long-lived local service.

Expand Down
2 changes: 1 addition & 1 deletion docs/specifications/collaboration.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Built on Yjs Awareness, not a custom channel. Yjs ships an **Awareness** protoco

- Each connected client (browser tab or MCP session) publishes an Awareness state: `{ actor: ActorId, heldRecordIds: string[] }`.
- **Human cursor presence → implicit hold:** the browser UI updates its own Awareness state's `heldRecordIds` to `[currentBlockId]` whenever the cursor moves, debounced ~1.5s on move-away (per PRD).
- **Agent hold request:** the MCP server's `hold_records` tool handler checks the _aggregate_ Awareness state across all connected clients for each requested record ID. A record already present in another client's `heldRecordIds` is denied for this request; everything else is granted. This directly implements the PRD's per-record (not all-or-nothing) acceptance criterion.
- **Agent hold request:** the MCP server's `hold_records` tool handler checks the _aggregate_ Awareness state across all connected clients for each requested record ID. A record already present in another client's `heldRecordIds` is denied for this request; everything else is granted. This directly implements the PRD's per-record (not all-or-nothing) acceptance criterion. Accurate for Phase 0's single global `Y.Doc`/Awareness pair; [`workspace-sharding.md`](./workspace-sharding.md) (#112, approved) moves Awareness to be shard-local once #113 lands, with a separate workspace-scoped hold coordinator (§3.3) aggregating across shards for a cross-document agent batch — the per-record semantics above don't change, only where the aggregation happens.
- **TTL:** two distinct timeouts are involved, not one:
- y-protocols' own Awareness implementation clears any client's state after **30s** of no heartbeat (`outdatedTimeout`) — this is what catches an abruptly-dropped connection (e.g. a browser tab closing without a clean disconnect).
- Agent holds additionally carry a dedicated **100s** `AGENT_HOLD_TTL_MS` timer (`src/lib/server/holds.ts`), scheduled server-side whenever a hold is granted or renewed, sitting inside the PRD's 90–120s auto-release target. This is what auto-releases a hold an agent forgot to explicitly release even while its underlying connection/token is still otherwise alive — the 30s Awareness timeout alone wouldn't cover that case, since a stateless MCP agent has no persistent heartbeat to time out in the first place.
Expand Down
2 changes: 1 addition & 1 deletion docs/specifications/data-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ Blocks are deliberately a small, documentation-oriented set rather than one type

## 4. Yjs mapping

One `Y.Doc` for the whole workspace (single-tenant, single workspace — no need for multiple docs yet).
One `Y.Doc` for the whole workspace (single-tenant, single workspace — no need for multiple docs yet). This is accurate Phase-0 behavior, not a permanent architectural stance: [`workspace-sharding.md`](./workspace-sharding.md) (#112) has approved a future one-`Y.Doc`-per-Document/Collection direction, backed by a server-owned SQLite catalog for title/hierarchy/scope; #113 implements and measures it before this table is rewritten to match.

| Concept | Yjs type | Notes |
| ----------------- | ------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
Expand Down
2 changes: 1 addition & 1 deletion docs/specifications/mcp-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,4 @@ All tools operate on `WorkspaceRecord` uniformly — no separate "block tools" v
| `create_record` | `parentId, afterRecordId?, blockType? \| properties?, referencedRecordId?` | `{recordId}` | no hold needed — no prior content to protect; `referencedRecordId` sets a `page_link` block's target in the same call — only accepted when `blockType` is `page_link` and `parentId` names a Document; the target must be an existing, caller-accessible Document, else the call is rejected with one generic "not an accessible Document" error that never distinguishes "doesn't exist" from "exists but out of scope" (see [`internal-links.md`](./internal-links.md) §4, [`audit-coverage.md`](./audit-coverage.md) §3); `blockType: 'collection_view'` is accepted the same generic way every `BlockType` is, but unlike `page_link` its `referencedRecordId`/`viewConfig` aren't yet validated on this path (no accessible-Collection check, no `viewConfig` field at all in the current MCP schema) — see [`collection-views.md`](./collection-views.md) §6 |
| `delete_record` | `recordId` | `{success: boolean}` | no hold needed |

**Permission scoping:** each access token (see [`persistence.md`](./persistence.md) §1) carries an allowlist of Document/Collection IDs. Every tool call resolves the target record and checks its `parentId` against that allowlist before granting a hold or applying any write — a denied record's Awareness/`Y.Doc` state is only ever read (to resolve it and to run the check itself), never mutated on its behalf. This is what "per-agent permission scoping" means concretely, even single-tenant: a token scoped to one Document can't read or write anything else, which is directly testable per the PRD's permission-denied acceptance criterion. A denied call is itself audited (as `<action>_denied`, carrying only the target ID the caller already supplied) — see [`audit-coverage.md`](./audit-coverage.md) §3 — so a trust review can see repeated denied attempts, not just successful ones.
**Permission scoping:** each access token (see [`persistence.md`](./persistence.md) §1) carries an allowlist of Document/Collection IDs, checked with operation-specific rules rather than one uniform record-parent check: `get_document`, `query_collection`, and document mutations (`create_document`, `move_document`, `delete_document`) authorize the target Document/Collection ID itself; record operations (`hold_records`, `write_record`, `create_record`, `delete_record`, `release_records`) authorize the target record's `parentId`; `list_documents`, `list_collections`, and `search_workspace` filter their results down to allowed Document/Collection IDs rather than authorizing a single target; and `move_document` validates both the moved `documentId` and the destination `parentDocumentId`. [`workspace-sharding.md`](./workspace-sharding.md) (#112, approved) §5 extends this same scoping into a trusted-routing rule at the MCP boundary once #113 lands, resolving a token's Workspace/Space scope and target shard before any hold or write — a denied record's Awareness/`Y.Doc` state is only ever read (to resolve it and to run the check itself), never mutated on its behalf. This is what "per-agent permission scoping" means concretely, even single-tenant: a token scoped to one Document can't read or write anything else, which is directly testable per the PRD's permission-denied acceptance criterion. A denied call is itself audited (as `<action>_denied`, carrying only the target ID the caller already supplied) — see [`audit-coverage.md`](./audit-coverage.md) §3 — so a trust review can see repeated denied attempts, not just successful ones.
2 changes: 1 addition & 1 deletion docs/specifications/persistence.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

[Drizzle](https://orm.drizzle.team/) (Apache 2.0 — same license as this project's own goal, not just compatible with it) manages every SQLite table, including the read model in §2 — one schema/migration story (`drizzle-kit`) instead of hand-written SQL for some tables and a separate library for others. It's also the SvelteKit-idiomatic choice: the Svelte CLI ships a built-in `drizzle` setup command.

- `snapshots` — periodic binary snapshots of the `Y.Doc` state (via `Y.encodeStateAsUpdate`). Simpler than an update-log replay for now; revisit if snapshot size becomes a problem. Keyed by `(workspace_id, shard_id)` (`src/lib/server/workspace-store.ts`'s `DEFAULT_WORKSPACE_ID`/`DEFAULT_SHARD_ID` today — see `architecture.md` §1) — every read/write/retention-prune query in `store.ts`'s `getSnapshotStore()` scopes by that key, so a second workspace's snapshots, once #13/#6 land, can never collide with or prune the first's. A given key's snapshot is loaded lazily, the first time `resolveWorkspaceContext()` resolves that workspace/shard — not eagerly for every context on process start, since Phase 0 only ever resolves the one default key anyway. The [2026-08-30 capacity baseline](../benchmarks/crdt-capacity-baseline-2026-08-30.md) confirms this snapshot path for a small daily workspace, but also establishes a global-snapshot budget: at 2 MiB or more, or when the associated event-loop p99 reaches 100 ms, prioritize document/Collection sharding or compaction rather than extending the Phase 0 global-state envelope. Those are engineering escalation signals, not a claim that either future design is already implemented.
- `snapshots` — periodic binary snapshots of the `Y.Doc` state (via `Y.encodeStateAsUpdate`). Simpler than an update-log replay for now; revisit if snapshot size becomes a problem. Keyed by `(workspace_id, shard_id)` (`src/lib/server/workspace-store.ts`'s `DEFAULT_WORKSPACE_ID`/`DEFAULT_SHARD_ID` today — see `architecture.md` §1) — every read/write/retention-prune query in `store.ts`'s `getSnapshotStore()` scopes by that key, so a second workspace's snapshots, once #13/#6 land, can never collide with or prune the first's. A given key's snapshot is loaded lazily, the first time `resolveWorkspaceContext()` resolves that workspace/shard — not eagerly for every context on process start, since Phase 0 only ever resolves the one default key anyway. The [2026-08-30 capacity baseline](../benchmarks/crdt-capacity-baseline-2026-08-30.md) confirms this snapshot path for a small daily workspace, but also establishes a global-snapshot budget: at 2 MiB or more, or when the associated event-loop p99 reaches 100 ms, prioritize document/Collection sharding or compaction rather than extending the Phase 0 global-state envelope. Those are engineering escalation signals, not a claim that either future design is already implemented. [`workspace-sharding.md`](./workspace-sharding.md) (#112) is the approved sharding design these signals point toward, including its own snapshot/recovery contract (§6) that supersedes the single `(workspace_id, shard_id)` key above once #113 implements it.
- `audit_log` — append-only: `id, actor_json, action, target_record_id, timestamp, diff_json`. Populated on every write/delete tool call, denied MCP attempt, and UI-originated write — including UI writes that bypass the service layer entirely, via a generic `Y.Doc`-level observer — satisfying the PRD's audit-log requirement; see [`audit-coverage.md`](./audit-coverage.md) for exactly what's covered, what's deliberately excluded, and how attribution and volume are handled.
- `access_tokens` — `token_hash, client_label, allowed_document_ids, allowed_collection_ids, created_at`. `client_label` is a free-text field set when the token is created (e.g., "Claude Desktop") — this is what powers the "Brylie · via Claude Desktop" attribution tag, since there's no per-vendor OAuth to source it from otherwise.

Expand Down
63 changes: 56 additions & 7 deletions docs/specifications/workspace-sharding.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
# Workspace catalog and CRDT sharding

**Status:** Proposed — #31's capacity baseline is complete; design approval is
pending review of the contracts in this document.
**Status:** Approved (2026-08-30) — the catalog/content-shard boundary,
ownership model, trusted-routing rules, committed-catalog-write/SSE contract,
and migration design in this document are the approved direction for #113
(implementation) and #114 (migration and isolation). Approval is conditional
per §8/§10: #113 must reproduce the required measurements with real
shard-aware transport before this boundary is treated as load-bearing, and
its results may still adjust the Collection partition threshold, event-
retention window, or snapshot cadence.

**Depends on:** [`prd.md`](../prd.md), [`architecture.md`](./architecture.md),
[`persistence.md`](./persistence.md), [`collaboration.md`](./collaboration.md),
Expand Down Expand Up @@ -29,13 +35,13 @@ not as the universal transport for every reactive interface element.
- Yjs WebSocket and Awareness connections are scoped to the explicitly opened
Document or Collection shard.

This is intentionally a proposed decision, not an implementation commitment.
The completed [#31 capacity baseline](../benchmarks/crdt-capacity-baseline-2026-08-30.md)
This is the approved architecture direction, not yet an implementation
commitment. The completed [#31 capacity baseline](../benchmarks/crdt-capacity-baseline-2026-08-30.md)
supports a small daily Phase-0 workspace and establishes a global-state
escalation boundary. Its document-size projection supports this shard direction,
but does not substitute for real shard-aware transport measurements. The
approval record for #112 must state which assumptions survived that measurement
and which must be proved by #113.
but does not substitute for real shard-aware transport measurements. §10
records which assumptions survived that measurement and which #113 must still
prove.

## 2. Terms and boundaries

Expand Down Expand Up @@ -323,3 +329,46 @@ require further partitioning or compaction.
the same Compendium version concurrently; a later scale-out implementation
proves lease, outbox-claim, and event-backplane coordination within one
Deployment instance.

## 10. Approval decision (recorded 2026-08-30)

Approved for #113/#114 to build against, on the completed #31 baseline and
the contracts specified above:

- The catalog/content-shard split in §1–§3: a durable SQLite workspace
catalog owning Space, Document, and Collection identity/title/hierarchy/
scope, separate from one Y.Doc per Document and one Y.Doc per Collection.
- The trusted-routing rule in §5: every boundary (page load, WebSocket
upgrade, SSE, MCP, persistence) resolves a server-authorized Workspace/
Space/shard context; a client-supplied selector is never authority.
- The committed-catalog-write/durable-operation/SSE contract in §4, including
the `pending_content → content_durable → publishable → published` state
machine, the public-revision/outbox-cursor ordering guarantees, and the
`catalog-resync` behavior for scope changes and cursor gaps.
- The migration design in §7: versioned, idempotent, checksum-verified,
ID-preserving migration of the current single-space workspace into one
default Space, with the original snapshot retained read-only until
verification passes.
- The deployment-ownership and coordination rules in §2: one process per
Deployment instance today; a future scale-out deployment requires durable
shard-ownership leases, transactional outbox claims, and a shared event
backplane — independent mutable in-memory replicas are invalid.

Explicitly deferred, not settled by this approval:

- The exact Collection row-partition threshold, event-retention window, and
snapshot cadence — §8 requires #113 to supply these from real shard-aware
measurements, not from the #31 global-workspace projection.
- Whether the Document/Collection shard granularity itself holds at the
concurrency and payload sizes #113 measures; a materially different result
reopens this document rather than being silently absorbed into #113.
- Where a **saved, shareable view configuration** (PRD P1) is addressed once
it exists as its own catalog-navigable artifact rather than an embedded
`collection_view` block's `viewConfig` (data-model.md §4) — out of scope
here because the feature itself isn't built yet; #113 or a follow-up must
extend §3.1's catalog fields to cover it before that feature lands.
- Storage-engine changes to `data-model.md` §4 ("One `Y.Doc` for the whole
workspace") and `collaboration.md`'s aggregate-Awareness description — both
remain accurate descriptions of current Phase-0 code and are superseded
only once #113 actually implements per-shard `Y.Doc`s, not by this
approval alone.
Loading