Skip to content

Comments

[DO NOT MERGE]: HSM Sync Engine#69

Open
dtkav wants to merge 139 commits intomainfrom
merge-hsm
Open

[DO NOT MERGE]: HSM Sync Engine#69
dtkav wants to merge 139 commits intomainfrom
merge-hsm

Conversation

@dtkav
Copy link
Member

@dtkav dtkav commented Feb 19, 2026

No description provided.

dtkav and others added 30 commits February 16, 2026 16:00
Adds RelayMetrics class that integrates with the obsidian-metrics plugin
when available, with graceful no-op fallback when disabled.
- Banner now automatically detects mobile Obsidian ≥1.11.0 and renders
  as a header button (short text) vs traditional banner (long text)
- Added BannerText type supporting string | { short, long }
- Removed setLoginIcon/clearLoginButton from LoggedOutView
- Removed setMergeButton/clearMergeButton from LiveView
- Removed font-weight bold and text-shadow from desktop banner
- Use Obsidian's native modal title via setTitle()
- Remove duplicate modal-content class nesting
- Use SlimSettingItem for inline toggle and button layout
- Add visible background to toggle track for dark backgrounds
- Fix folder name overflow with ellipsis truncation
- Prevent folder icons from shrinking
Initial foundation for device sync and recommended settings.
Adds test infrastructure for the new hierarchical state machine (HSM)
that will manage document synchronization between disk, local CRDT,
and remote CRDT.

Test harness includes:
- Type definitions for events, effects, and state
- Event factory functions for creating serializable test events
- Stub HSM implementation for test development
- Custom assertions (expectEffect, expectState, etc.)
- 23 passing tests covering core state transitions

Designed for future recording/replay support - all events are
plain serializable objects with snapshot capability.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add node-diff3.d.ts type declarations for portable module resolution
- Add required vaultId to MergeHSMConfig (convention: ${appId}-relay-doc-${guid})
- Add getVaultId callback to MergeManagerConfig and ShadowManagerConfig
- Fix PersistUpdatesEffect to use dbName/update instead of guid/updates
- Fix TimeProvider fallbacks to use DefaultTimeProvider instead of partial object
- Fix SerializableSnapshot structure in recording bridge
- Wire up MergeManager in main.ts with proper vaultId generator
- Update MergeManager tests with required getVaultId

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
MergeHSM._path and _guid were private with no public accessors,
causing hsm.path to be undefined and falling back to the guid.

Closes BUG-009.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Document: Clean up existing HSM provider listeners before re-adding
  to prevent accumulation on repeated setup calls.
- HasProvider: Remove status listener once connected instead of relying
  on manual cleanup in destroy().
- LiveViews: Use releaseLock() instead of directly setting userLock.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Cherry-pick event subscription infrastructure from background_sync into
the YSweetProvider (CBOR-decoded messageEvent, subscribe/unsubscribe
protocol) and wire SharedFolder to forward document.updated events to
MergeManager.handleIdleRemoteUpdate() instead of the previous ydoc
update listener approach.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…sisted CRDT state

MergeManager was created without persistence callbacks, causing every
HSM to receive empty PERSISTENCE_LOADED events. This made every file
appear brand-new and triggered spurious merge conflicts on first edit.

Closes BUG-011.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
… directly

RelayCanvasView.release() set canvas.userLock = false without notifying
MergeManager, so the HSM never received RELEASE_LOCK and stayed stuck
in active mode for canvas files. Added Canvas.releaseLock() matching
Document's pattern and wired it into RelayCanvasView.release().

Closes BUG-012.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
On first ACQUIRE_LOCK, if localDoc is empty (no prior CRDT data in
IndexedDB), read current disk contents via getDiskState and call
initializeLocalDoc to populate the CRDT. Also sends DISK_CHANGED so the
HSM tracks disk metadata. Without this, localDoc stayed empty despite
the editor showing file content, causing spurious merge conflicts.

Closes BUG-013.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Corrections 1-7 from specs/persistence-corrections.md:
- Fix Y.Text field name 'content' → 'contents' to match existing IndexedDB data
- Attach IndexeddbPersistence to localDoc via injectable CreatePersistence factory
- Destroy persistence in cleanupYDocs() on RELEASE_LOCK
- Gate active.entering → active.tracking on YDOCS_READY (persistence 'synced')
- Remove loadUpdates callback from MergeManager (persistence loads internally)
- Skip Document.ts IndexeddbPersistence when HSM active mode is enabled
- Retarget uploadDoc() to insert into localDoc via MergeHSM when HSM enabled

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When files are created in shared folders via uploadDoc, the LCA (Last
Common Ancestor) is now initialized to establish the baseline sync point.
This prevents merge conflicts with an empty base when editing newly
created files.

Changes:
- Add initializeLCA() method to MergeHSM for setting up sync baseline
- Call initializeLCA in both HSM active mode and standard upload paths
- Persist PERSIST_STATE effects to IndexedDB via saveMergeState

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Track pending write operations and wait for them to complete in
destroy() before closing the database. This prevents data loss when
the persistence layer is torn down while writes are still in flight.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…k error

When clicking the merge conflict banner on a local-only shared folder
(no relayId), checkStale() would call backgroundSync.downloadItem()
which throws "Unable to decode S3RN" since there's no server to
download from.

Now checkStale() checks for relayId before attempting server download.
For local-only folders, it skips the download and just compares the
local CRDT with disk contents.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
dtkav and others added 30 commits February 16, 2026 16:03
Add seeded PRNG-driven random delays to test infrastructure to catch
timing-related bugs. Controlled via environment variables:
- TEST_ASYNC_DELAYS=1 enables random delays (0-10ms)
- TEST_SEED=<number> sets reproducible random sequence

Changes:
- Add random.ts with Mulberry32 PRNG fountain
- Add async delays to persistence sync, destroy, and diskLoader
- Add sendAcquireLock/sendAcquireLockToTracking helpers that properly
  await async state transitions
- Update loadToIdle to wait for persistence sync before returning
- Update loadToConflict to await state after acquireLock
- Convert ~30 tests to use async-aware helpers

All 89 tests pass with and without TEST_ASYNC_DELAYS=1.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
DRIFT_DETECTED events were using a non-standard format (type/timestamp/state)
which broke the visualizer. Now uses standard fields (ns/ts/path/event/from/to).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove enableMergeHSMRecording flag (always install bridge, lightweight without onEntry)
- Delete RecordingMergeHSM.ts and generateTest.ts (unused)
- Consolidate StreamingEntry into HSMLogEntry
- Simplify recording bridge, replay, and serialization modules
- Update tests for new recording API

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The relay-live-editor CSS class is added asynchronously after acquireLock().
Previously, edits arriving before initialization were lost.

- Don't destroy plugin early if CSS class not yet present
- Buffer CM6 changes that arrive before HSM is ready
- Replay buffered edits once initialization completes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Validates remote Yjs updates before applying them to prevent
corrupted updates from breaking document state.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Disk edits in idle mode now create a fork snapshot of localDoc before
ingesting changes. This enables three-way reconciliation when the
provider reconnects: fork.base serves as the common ancestor for
merging local (disk edit) and remote changes.

SyncGate controls CRDT op flow between localDoc and remoteDoc, blocking
remote-to-local merges while a fork is unreconciled.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Block syncLocalToRemote() while a fork is unreconciled. This prevents
local changes from propagating to remote before fork reconciliation
confirms they're safe.

When the fork clears (clearForkAndUpdateLCA), pending outbound changes
are flushed by recomputing the diff from localDoc to remoteDoc.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When a fork is created in idle mode (disk edit while provider not synced),
the HSM emits REQUEST_PROVIDER_SYNC effect. SharedFolder handles this by:
1. Downloading latest state via backgroundSync
2. Applying updates to remoteDoc
3. Sending CONNECTED + PROVIDER_SYNCED to HSM
4. HSM then runs fork reconciliation

This ensures disk edits are reconciled promptly without waiting for the
user to open the file.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When PROVIDER_SYNCED arrives in active.tracking with a fork present,
reconcileForkInActive runs three-way merge and dispatches granular
changes to the editor via computeDiffChanges (not full replace).

This ensures fork reconciliation happens promptly when the user has
the file open, rather than waiting for close/reopen.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fork reconciliation now queries provider.synced via callback instead of
storing duplicate state in _syncGate.providerSynced for idle mode.

Changes:
- Add isProviderSynced callback to MergeHSMConfig
- invokeForkReconcile uses callback to check provider state
- Add awaitingProvider guard to stay in idle.localAhead until synced
- Add hasFork guard to accumulate REMOTE_UPDATE during fork reconciliation
- Test harness tracks provider state and passes callback

This avoids state ownership issues where the HSM would store providerSynced
but not receive DISCONNECTED events in idle mode to clear it.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
destroyLocalDoc() now nulls out references synchronously and does async
IDB cleanup on captured refs, preventing races where wake() recreates
localDoc while cleanup is still running. wake() and processWakeQueue()
call ensureLocalDocForIdle() to recreate localDoc after hibernation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove moduleNameMapper mocks for node-diff3 and y-indexeddb so tests
run against real implementations. Add a transform rule for src/*.js
files so ts-jest handles the ESM syntax in y-indexeddb.js.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Track OBSIDIAN_FILE_OPENED/OBSIDIAN_FILE_UNLOADED events to maintain
an isObsidianFileOpen flag on each HSM. DiskIntegration checks this
flag before executing WRITE_DISK effects, blocking writes when the
editor has the file open to prevent content duplication.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Simplify invokeForkReconcile to always use diff3 via remoteDoc,
removing the "remote unchanged" shortcut that bypassed three-way merge.
Reset providerSynced on fork creation and RELEASE_LOCK so reconciliation
waits for a fresh sync. Emit REQUEST_PROVIDER_SYNC when releasing lock
with an active fork.

Add patchLCAHash() to async-compute LCA hashes after reconciliation
instead of blocking on the hash computation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously, all active documents skipped remote update injection.
Now the skip is gated on enableDirectRemoteUpdates, allowing the
enqueueDownload path (which fetches full server state) to work for
active documents when direct injection is disabled.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add ingestDiskToLocalDoc action that applies pending disk contents to
localDoc. Change idle.synced DISK_CHANGED transition to re-enter
idle.localAhead with ingestion instead of going straight to diverged.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The guard checked for sharedFolder.remote during state change
callbacks, which is no longer needed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ication

Adds the OpCapture module (reversible CRDT op capture) and integrates it
end-to-end: persistence in IDB via y-indexeddb, test harness support, and
consumption during fork reconciliation. When both disk and remote edit the
same content, redundant disk ops are reversed; unique disk ops are dropped
(kept in CRDT). Also fixes diff3 tokenization to use split(/(\n)/) so
adjacent-line changes are handled independently.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a file was deleted or a shared folder was removed from settings,
the per-document y-indexeddb databases (and folder-level database)
were left behind as orphans in IndexedDB. This adds deleteDatabase
calls in deleteFile() and SharedFolders.delete() after the in-memory
objects are destroyed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three fixes to the fork→diverged→idle-merge path:

1. clearForkKeepDiverged: clear pendingIdleUpdates (already evaluated
   by diff3) and restore pendingDiskContents from localDoc so the
   three-way merge sees the real disk content, not the LCA fallback.

2. invokeIdleThreeWayAutoMerge: read remoteDoc text directly instead
   of applying pendingIdleUpdates to localDoc via raw Y.applyUpdate.
   The raw CRDT merge causes interleaving corruption on conflicting
   edits. If remoteDoc isn't available yet, bail and wait for
   REMOTE_UPDATE to reenter idle.diverged.

3. clearForkAndUpdateLCA: clear pendingIdleUpdates for hygiene.

Also adds:
- MERGE_CONFLICT transition in active.tracking for fork conflicts
  detected during reconcileForkInActive
- OBSIDIAN_SAVE_FRONTMATTER / OBSIDIAN_METADATA_SYNC diagnostic
  events for correlating metadata editor hooks with drift events
- Regression test for the corruption scenario

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Path was stored at construction time and never updated on rename,
causing stale paths in logging and debug tools. Now MergeHSM takes
a getPath callback that computes the current path from the source
of truth (Document.path via SharedFolder's files map).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
BackgroundSync.syncDocumentWebsocket only checked userLock before
disconnecting, missing cases where the MergeHSM was actively managing
the document. Also defers awareness cursor updates in RemoteSelections
to avoid re-entrant EditorView.update errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…hout LCA

Two changes:
- Populate conflictData when fork-reconcile detects divergence, and add
  hasPreexistingConflict guard in active.entering.reconciling so the
  conflict banner shows when opening a diverged file.
- Allow invokeIdleRemoteAutoMerge to write to disk when there is no LCA
  and no file on disk (initial sync from a remote peer). Block writes
  when there is no LCA but a file exists (up-migration safety).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When Obsidian's auto-save flushes (DISK_CHANGED with dirty === false),
advance the LCA snapshot so it stays fresh during long editing sessions.
This improves conflict detection quality if the user goes offline.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Svelte skipped re-rendering the Layers icon because view.tracking was
a getter on the same object reference. Pass tracking as a primitive
boolean prop so Svelte detects the change.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When .staging-only marker file is present, npm run build prints a
warning and exits. Use npm run build:force to bypass, or use the
staging build for local development.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Deduplicate HSM state change logs in LiveView (only log when statePath
actually changes) and remove per-render debug logs from MetadataRenderer,
PreviewRenderer, and ViewHookPlugin.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace monkey-patching in E2ERecordingBridge with push-based callbacks
from MergeHSM. The bridge is now a passive sink that receives transition
info (from/to state, event, effects) via MergeManager, eliminating
ObservableMap subscriptions that caused high-frequency Postie deliveries
on every keystroke.

Extract plugin-level debug surface into RelayDebugAPI (window.__relayDebug),
separating the per-folder write path (E2ERecordingBridge) from the
plugin-level read path (RelayDebugAPI).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant