[Beautyl0ve]

feat(settings): sync user preferences across authenticated clients

Fixes #2595

Why

DeerFlow currently stores base UI/model preferences only in one browser's
localStorage. Clearing browser data or signing in on another device loses those
choices. Uploading the existing blob directly would also be unsafe: it is unscoped,
contains state that must remain local, and can be observed by multiple accounts or
tabs on the same origin.

What changed

• Adds authenticated, owner-scoped GET, first-writer-wins PUT, and nested-merge
  PATCH /api/user-preferences endpoints over a strict browser-safe allowlist.
• Adds nullable users.preferences JSON plus a revision counter in migration
  0014_user_preferences, branching directly from 0013_mcp_task_notifications.
• Synchronizes notification-enabled, token-usage display, and base model/context
  defaults. Thread/workspace state, credentials, and browser/OS notification
  permission never enter the contract.
• Sends leaf-only patches and persists per-leaf mutation/ack slots, so concurrent tabs
  cannot overwrite unrelated settings or acknowledge a newer unsent edit.
• Serializes each user's bootstrap handshake and writes with Web Locks. Browsers
  without that primitive keep writes local and pending instead of risking reordering.
• Protects hydration with version checks and activation snapshots, preventing a stale
  request or activation-gap edit from erasing newer local state.
• Uses bounded revision-CAS retries in the repository, including retrying the complete
  read/merge/CAS cycle for SQLite SQLITE_BUSY and SQLITE_BUSY_SNAPSHOT failures.
• Treats an existing server value as authoritative. A user without a server record can
  claim a valid legacy browser value once; caches and pending mutations are user-scoped.
• Sends an expected-user guard with every request. The backend still derives ownership
  only from authentication and rejects a stale tab after another tab changes the
  origin-wide cookie.
• Keeps auth-disabled/static deployments local-only and documents migration, fallback,
  account isolation, concurrency, and data-boundary behavior.

Surface area

• Frontend UI — page / component / setting / interaction under frontend/
• Backend API — endpoint / SSE event / request-response shape under backend/app
• Agents / LangGraph — agent node, graph wiring, langgraph.json, or prompt change
• Sandbox — docker/ or sandboxed execution
• Skills — change under skills/
• Dependencies — new/upgraded entry in backend/pyproject.toml or frontend/package.json (say what it buys us)
• Default behavior change — changes existing behavior without the user opting in (default model, default setting, data shape)
• Docs / tests / CI only — no runtime behavior change

Screenshots / Recording

There is no visual delta: existing settings controls keep their current appearance and
gain authenticated persistence behind the scenes. A real two-profile/browser recording
has not been attached.

Validation

Frontend full unit suite:              134 files / 1,056 tests passed
Targeted settings concurrency suite:   36 tests passed
Frontend TypeScript typecheck:          passed
Changed-file ESLint / Prettier checks: passed

Backend preference/bootstrap/migration suite: 49 tests passed
Upstream-adjacent migration/extension suite:  27 tests passed
Repository guidance tests:                    12 tests passed
Backend Ruff check / format (13 files):        passed

Alembic heads: 0014_user_preferences (single head)
git diff --check: passed

The SQLite concurrency coverage uses two WAL connections and exercises the actual
SQLITE_BUSY_SNAPSHOT code path while verifying that disjoint patches survive. The
migration coverage verifies both a fresh upgrade and downgrade/re-upgrade for existing
users.

Not run: live PostgreSQL transaction integration, a real two-browser/device smoke test,
or frontend build/E2E. PostgreSQL statements were reviewed and dialect-compiled.
Same-field concurrent writes remain last-successful-commit-wins.

Migration publication note: the latest merged main used by this branch ends at
0013_mcp_task_notifications; this PR therefore claims 0014_user_preferences
directly from it. Open PRs #4918
and #4843 currently contain
competing 0014 revisions. Whichever PR lands later must rebase and renumber/rechain
its migration before merge.

AI assistance

Tool(s) used: Codex

How you used it: Assisted with implementation, concurrency review, regression
tests, documentation, and local validation.

• I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.

[willem-bd]

Reviewed at head a252e92 (backend: 48 tests in test_user_preferences + persistence bootstrap/migration files pass locally via uv run pytest; frontend: the full new tests/unit/core/settings + layout-boundaries suite passes via rstest). The overall design is solid: the strict allowlist + extra="forbid"/strict=True schemas, owner derivation from the cookie (never the header), the conditional-update initialization, and the version/outbox race handling in UserSettingsSyncLifecycle all check out — I traced the activation-gap, stale-tab, account-switch, and Web-Lock-claim races and they are guarded. Findings below are one correctness gap in the cross-tab echo path plus two robustness notes. The deliberate 0015 revision gap over 0013 is already documented in the PR description and the migration docstring, so no comment there.

[willem-bd]

This cross-tab echo emits a full-state patch (emitBaseSettingsMutation() with no key falls through to toFullUserSettingsPatch, which sends explicit null for every unset context field), and the backend treats null as "delete the key". That can erase server-side values this browser never hydrated: with two tabs open, tab A's scoped edit (e.g. a notification toggle → {notification} patch, model untouched) writes the shared cache; tab B then receives this storage event and PATCHes the entire local state, whose context is empty, so model_name/mode/reasoning_effort are sent as null — clearing a default model that another device set on the server after this browser hydrated. Nobody touched the model on this device, yet the other device's selection is gone (and the DOM test "the same account's tab cache still produces a synchronized mutation" pins the full-patch-with-nulls behavior). Since local edits go through updateLocalSettings(key)/updateThreadSettings(key) and produce section-scoped patches, the storage-event path could do the same by diffing the previous baseSettings snapshot against the freshly read cache and emitting only the sections that changed — that confines the echo to what actually changed and stops the null-clearing clobber.

[willem-bd]

Suggestion (robustness/self-heal): _response re-validates the stored JSON with UserPreferences.model_validate, so a stored payload that stops validating — a future enum tightening (this repo already evolved tokenUsage shapes once), or a manual DB fixup — makes GET and PATCH return 500 for that user. Because UserSettingsSyncController.start() swallows every failure and never retries in-session, sync then silently dies, and the client can't recover via PUT (it only PUTs when GET returns settings: null, not when GET 500s). Degrading an invalid stored record to settings: None on GET (keeping revision) would let the normal first-writer-wins PUT re-initialize and self-heal, at the cost of one round-trip.

[willem-bd]

Nit: the CAS retry loop does 5 back-to-back attempts with no backoff, and when it exhausts, PATCH surfaces as 409 which the frontend turns into a sticky writeFailed (only cleared by the next user mutation or the next page-load handshake). For a settings blob where contention is rare but possible across Gateway workers, a tiny await asyncio.sleep(...) with jitter between attempts would make exhaustion effectively unreachable; alternatively the frontend could clear writeFailed after a later successful GET.

[WillemJiang]

@Beautyl0ve thanks for your contribution, please resolve the conflicts.

[Beautyl0ve]

@WillemJiang Thanks for the reminder. I have merged the latest main and resolved the conflicts. The preferences migration is now 0017_user_preferences, chained after 0016_subagent_batches. I re-ran the focused backend migration/preferences suite (52 tests), frontend settings suite (45 tests), frontend typecheck, guidance checks, Ruff, and Alembic head validation; all pass locally. Please take another look when you have a chance.

[willem-bd]

Two additional correctness issues found in the settings recovery paths.

[Beautyl0ve]

@WillemJiang Conflicts with the current main are resolved in fe90b283. The competing migration slot is now linearized as 0017_personal_access_tokens -> 0018_user_preferences, and both get_user_repository and the upstream get_pat_repo dependency are preserved.

Validation after the merge:

• Alembic: single head (0018_user_preferences)
• Backend migration/preferences/PAT regression: 118 passed
• Frontend settings sync: 51 passed
• TypeScript typecheck and Ruff: passed

The PR is mergeable again and ready for another review. Thank you!

[willem-bd]

Reviewed at head fe90b28. All three findings from the earlier rounds are addressed at this head: handleStorage now forwards only leaf-level diffs with persist=false (no more full-state echo), _response() degrades a schema-invalid stored record to settings: null (backed by the MalformedUserPreferences read-edge guard for undecodable SQLite JSON), and the CAS loops back off through _sleep_before_preference_retry. The migration collision is resolved correctly too — 0018_user_preferences chains after 0017_personal_access_tokens, which is main's current head, and the four touched bootstrap/migration tests agree. What remains: one correctness/livelock bug in the new cross-tab storage path, one bootstrap-failure gap that silences the outbox for a whole tab session, and a dead-code nit.

[willem-bd]

[P1] This branch rewrites the shared cache unconditionally, and a storage event fires in the other tab for every successful setItem even when the value is identical (the spec's setItem step broadcasts unconditionally — only clear() checks emptiness, which is why StorageEvent carries oldValue/newValue). With two tabs of the same account open this is an unbounded ping-pong: tab A writes V → tab B's handler reads V, folds nothing, writes V → tab A's handler writes V → … forever, with no user input. Each round is a JSON parse + stringify + localStorage write + emitChange() re-render in both tabs, so the tab pair spins CPU and hammers storage for as long as both stay open.

The write is load-bearing when applyPendingUserSettings actually folds this tab's pending leaves (the other tab cannot fold them), so the fix is to skip the no-op case rather than the write — serialize once and compare against event.newValue, or have applyPendingUserSettings report whether it changed anything, before calling saveUserSettingsCache. A regression test that dispatches a second storage event for an unchanged value and asserts exactly one write would pin it.

[willem-bd]

[P2] A failed handshake disables the write path for the tab's remaining lifetime, which is stronger than "the next authenticated page load tries again". bootstrapped is only ever set in hydrateResponse, scheduleWrites() bails on !this.bootstrapped, and the mutation subscription also gates on bootstrapped — so if the first GET (or the follow-up PUT) throws once, a Gateway restart, a brief 503, being offline at mount, or a 409 from the expected-user guard all leave durable outbox leaves from this and earlier sessions unflushed, and whenIdle() resolves immediately, even after connectivity returns.

Smaller note in the same path: a throwing GET inside the lock callback makes withWriteLock resolve false, so line 127 issues a second GET that normally fails identically. A bounded handshake retry (e.g. on online / visibilitychange) would close both, and stays safe because drainWrites only ever sends leaf patches the user already produced locally.

[willem-bd]

[nit] savePendingBaseSettingsPatch is exported but only tests call it, and the legacy single-key outbox format it clears (deerflow.user-settings-pending.<userId> plus the legacy-ack key, read back in readPendingPatchEntries) has never been written by any released build — this PR is the first to introduce those keys. That leaves a permanently-read-but-never-written compatibility branch in the settings hot path. Consider dropping the legacy branch and letting tests write leaf envelopes directly.

[Beautyl0ve]

@willem-bd Thanks for the careful follow-up. I confirmed both remaining correctness issues at fe90b283:

• the authenticated storage handler persists the merged cache even when no pending leaf changed it, so an unchanged cross-tab event can ping-pong indefinitely;
• a failed initial handshake leaves started=true and bootstrapped=false, so later mutations cannot restart or drain the durable outbox during that tab session.

I’ll address the first by skipping the no-op cache write while preserving the write when local pending leaves are folded, and the second with a bounded/event-driven handshake retry that avoids the duplicate fallback GET. I’ll add regressions for both and remove the never-shipped legacy outbox compatibility branch/helper in the same update.

[Beautyl0ve]

Closing to focus current contributions on agent-runtime correctness. This non-agent settings-sync branch is now large, conflicted, and still has unresolved correctness findings; a future attempt should restart from current main with a smaller scope.