Skip to content

perf(chat): instant channel re-entry — per-channel message cache + persistent WebSocket#273

Merged
haowei2000 merged 4 commits into
developfrom
claude/channel-initial-load-perf-92401d
Jul 20, 2026
Merged

perf(chat): instant channel re-entry — per-channel message cache + persistent WebSocket#273
haowei2000 merged 4 commits into
developfrom
claude/channel-initial-load-perf-92401d

Conversation

@haowei2000

Copy link
Copy Markdown
Collaborator

Problem

Opening a channel was slow, and every switch paid the full cost again:

  1. No client-side cachingChannelView wiped messages and cold-fetched 50 rows on every entry, even re-entering a channel viewed seconds earlier.
  2. A fresh WebSocket per channel switch — the realtime hook was keyed on channelId, so each switch tore down the socket and paid TCP/WS + auth + subscribe (2 RTT) again.
  3. Duplicate first page — if the subscribe ack won the race against the initial history load, the catch-up ran with since_seq=0 and re-fetched a full page the load was already fetching.
  4. Auto-scroll only fired on message-count growth, which a cache seed doesn't trigger.

The backend needed no changes — list_channel_messages is already keyset-paginated with batched hydration (3 queries/page) and the right indexes.

Changes (Discord/Telegram-style, frontend only)

  • chatCache.ts (new) — per-channel in-memory LRU cache (messages + members, 30 channels). Re-entry renders the cached window instantly, then a since_seq catch-up merges anything missed. Seeds trim to the newest 100 rows (hasMore flips back on) and clear interrupted streaming flags.
  • useChatRealtime.ts — the browser socket is now a module-level singleton shared across switches: switching sends unsubscribe/subscribe on the live connection. Late frames for a left channel are dropped by channel_id; an idle socket closes after 30s; logout closes it immediately. Public API and exports unchanged.
  • ChannelView.tsx — warm path seeds from cache and catch-ups incrementally; the subscribe-ack race is fixed (catch-up defers until the real seq cursor exists); every async response is guarded so a reply landing after a channel switch is discarded instead of merged into the wrong channel's state/cache.
  • MessageList.tsx — a channel switch jumps straight to the bottom of the new timeline.

Verification

Headless Chrome against the kind cluster (Vite → in-cluster gateway):

  • Warm re-entry: rows visible in 134–152ms, network shows only a since_seq delta — 0 full-page refetches.
  • 0 new WebSocket connections across A→B→A→B switches (the extra opens in dev logs are StrictMode double-mount noise + the by-design separate user-scope notification socket).
  • After 5 rounds of subscription churn, a REST-posted message still arrives live over the reused socket with no refetch.
  • vitest 83/83 (incl. new chatCache tests: LRU eviction, seed trimming, streaming-flag clearing); tsc clean except the known PdfViewer baseline.

🤖 Generated with Claude Code

haowei2000 and others added 2 commits July 19, 2026 20:24
…nt WS

First channel entry (and every switch) paid a full cold reload: messages
were wiped and re-fetched, a brand-new WebSocket was handshaken per
channel, and a subscribe-ack race could re-fetch the whole first page a
second time. Discord/Telegram-style fixes, all client-side (the gateway
query path is already keyset-paginated and batched):

- chatCache.ts: per-channel in-memory LRU cache (messages + members).
  Re-entering a channel renders the cached window instantly, then a
  since_seq catch-up merges anything missed. Seeds are trimmed to the
  newest 100 rows (hasMore flips back on) and streaming flags cleared.
- useChatRealtime: the browser socket is now a module-level singleton
  shared across channel switches — switching sends unsubscribe/subscribe
  on the live connection instead of paying TCP/WS + auth per switch.
  Late frames are dropped by channel_id; an idle socket closes after a
  grace period; logout closes it immediately.
- ChannelView: the subscribe ack no longer races the initial history
  load into a duplicate since_seq=0 page fetch — the catch-up defers
  until the real seq cursor exists. Async responses landing after a
  channel switch are discarded instead of merged into the wrong channel.
- MessageList: a channel switch jumps straight to the bottom (the old
  auto-scroll only fired on message-count growth, which a cache seed
  doesn't trigger).

Verified against the kind cluster with headless Chrome: warm re-entry
renders in ~140ms with zero full-page refetches and zero new WS
connections across A→B→A→B switches; live delivery on the reused socket
still works after subscription churn. vitest 83/83.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The iOS client already had the other two web fixes by construction (one
app-wide socket with subscribe/unsubscribe per channel; catchUp guarded
by loadedOnce/highestSeq), but ChatsHomeView rebuilt ChatView — and with
it a fresh ChatModel — on every switch via .id(channelId), so each entry
cold-reloaded 50 messages + members.

- ChatModelStore (new): LRU (12 channels) of ChatModel instances hung
  off AppModel, cleared on sign-out. The open channel is always the most
  recently touched entry, so it can't be evicted.
- ChatView now receives the cached model instead of constructing one;
  the view still remounts per channel, the model survives.
- ChatModel: warm loadInitial path renders the in-memory history
  immediately, scrolls to bottom, marks read, refreshes members in the
  background, and heals via the existing since_seq catchUp (in-flight
  guarded — the subscribe ack requests the same catch-up). detach()
  trims retained history to the newest 100 rows (hasMoreBefore flips
  back on) so re-entry render cost stays bounded. channel DTO refreshes
  on reuse so renames still show.

xcodebuild (Debug, iOS Simulator) succeeds.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@haowei2000

Copy link
Copy Markdown
Collaborator Author

Added the iOS counterpart in 520b1b2.

iOS already had two of the three web fixes by construction — a single app-wide socket with per-channel subscribe/unsubscribe, and a catchUp guarded against the since_seq=0 race — but ChatsHomeView rebuilt ChatModel on every switch (.id(channelId)), so each channel entry still cold-reloaded 50 messages + members.

Now AppModel.chatModels (new ChatModelStore, LRU of 12) retains one ChatModel per channel: re-entry renders the in-memory history instantly, scrolls to bottom, refreshes members in the background, and heals via the existing since_seq catch-up. Retained history is trimmed to the newest 100 rows on detach; the store is cleared on sign-out. Verified with xcodebuild (Debug, iOS Simulator) — build succeeds.

…eviction

Two platform-convention upgrades on top of the in-memory ChatModel cache:

- MessageStore (new, SwiftData): every visited channel's newest 200
  finalized messages persist across launches as full-DTO JSON blobs
  (schema-stable while the DTO evolves) keyed by unique msg_id, plus a
  per-channel hasMoreBefore flag so upward pagination works straight off
  a cold start. Cold loadInitial renders the persisted window first and
  the network refresh lands on top; if >1 page arrived while offline
  (fresh page not contiguous with the cached window) only the fresh page
  is kept so pagination stays gap-free. Writes are debounced (1 s) off
  upsert/loadOlder and flushed on detach; deleted messages are removed
  from disk so they can't resurrect; the store is wiped on sign-out.
  MessageDto becomes Codable (nested types already were).

  Gotcha found by a macOS SwiftData harness driving the real store: a
  trim fetch with fetchOffset run BEFORE saving saw every pending insert
  as overflow and deleted the batch it was meant to keep (SwiftData does
  not apply offset windows reliably to unsaved objects). save() now
  persists first, then trims by slicing in code.

- ChatModelStore responds to UIApplication.didReceiveMemoryWarning by
  shedding every cached model except the open channel's — safe to be
  aggressive now that evicted history reloads from disk, not network.

Verified: harness passes round-trip/ordering, unique upsert, 200-row
trim + hasMoreBefore, in-flight row skipping, delete, channel isolation,
removeAll, and cross-instance relaunch persistence. xcodebuild (Debug,
iOS Simulator) succeeds.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@haowei2000

Copy link
Copy Markdown
Collaborator Author

1b90be3 adds the two iOS platform-convention upgrades:

Offline-first persistence (SwiftData) — new MessageStore keeps each visited channel's newest 200 finalized messages on disk as full-DTO JSON (schema-stable), so a cold app launch renders history instantly and the network refresh lands on top. Handles the non-contiguous case (>1 page arrived while offline → keep only the fresh page so upward pagination stays gap-free), debounces writes, flushes on channel exit, deletes tombstoned messages from disk, and wipes on sign-out.

Memory-pressure evictionChatModelStore sheds every cached model except the open channel's on didReceiveMemoryWarning; safe now that evicted history reloads from disk instead of the network.

Notable bug caught by a macOS harness driving the real store: SwiftData does not apply fetchOffset windows reliably to pending (unsaved) objects — a pre-save trim fetch classified every fresh insert as overflow and deleted the batch it had just written. save() now persists first and trims by slicing in code. Harness covers round-trip/ordering, unique upsert, trim + hasMoreBefore, in-flight skipping, delete, channel isolation, removeAll, and cross-instance relaunch persistence; xcodebuild (Debug, iOS Simulator) succeeds.

Conflict in apps/ios/Sources/State/ChatModel.swift: #274 (iOS @-mention
picker) built its `mentionPool` inside the members-fetch block of
loadInitial, which this branch had extracted into refreshMembers().

Resolved by keeping this branch's loadInitial (offline-first seed +
contiguity check) and moving #274's mentionPool construction into
refreshMembers(). Strictly better than either side: refreshMembers()
also runs on warm re-entry, so a member who joined while the user was
away now appears in the mention picker without a cold reload — #274's
version only built the pool on a cold load.

DTOs.swift, ChatView.swift and ComposerView.swift auto-merged.
xcodebuild (Debug, iOS Simulator) succeeds; SwiftData harness and
frontend vitest (83/83) still pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@haowei2000
haowei2000 merged commit 6aad9d5 into develop Jul 20, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant