feat(workspace): event-driven file freshness, drop the 2s polling - #1090
Draft
ChrAlpha wants to merge 15 commits into
Draft
feat(workspace): event-driven file freshness, drop the 2s polling#1090ChrAlpha wants to merge 15 commits into
ChrAlpha wants to merge 15 commits into
Conversation
Replace the unconditional 2s polling in files-pane, file-viewer and panel-preview with an attention-gated freshness ticker: passive refresh runs only while the surface is visible and a turn is streaming for the current bot, with one catch-up tick on regained attention. fs-change signals now carry path-scoped refreshes into the tree, background refreshes are serialized per directory and stay silent on failure.
Each web WS connection subscribes to the fsevent hub for its bot;
debounced change batches arrive as {type: fs_changed, paths} where a
null paths means unknown scope. The web client routes them into the
existing fs-change beacon, so the files pane, file viewer, and preview
refresh from server-side signals regardless of which session, surface,
or member caused the change.
FSWrite/FSUpload/FSMkdir/FSDelete/FSRename/FSExtract publish the touched paths to the fsevent hub after success, so every connected viewer of the bot refreshes when another tab, member, or the desktop app mutates the workspace.
assembleTools wraps write/edit (path-scoped) and exec/apply_patch (wildcard) so every successful execution publishes to the fsevent hub with the turn's bot id — covering web, channel, schedule, background and subagent surfaces alike.
An fs-change sink wraps the prompt event stream: successful ToolCallEnd events whose mapper-normalized name is fs-mutating publish their paths to the fsevent hub with the prompt's bot id. Tool classification is shared with the native wrapper via fsevent.ToolChange.
New server-stream RPC on ContainerService: the in-container bridge watches one directory non-recursively via fsnotify, coalesces events into 100ms batches of container-visible paths, and ends the stream when the watched directory disappears. bridge.Client.WatchDir exposes it host-side with ErrWatchUnsupported for older bridges.
FSWatchService keeps one refcounted bridge WatchDir stream per (bot, dir), driven by each web WS connection's fs_watch message — the files pane reports its expanded directories (debounced, resent on reconnect via the reliable-request queue) and clears them when hidden. Batches flow into the fsevent hub; a dying stream publishes its directory as stale and retries once while still wanted; bridges without the RPC back off via ErrWatchUnsupported. Watch lifetime equals viewer attention, so idle workspaces carry no watches.
Hub deliveries now run on one goroutine per subscription draining a single merged pending batch, so a wedged WebSocket writer can neither block flush timers (goroutine pile-up) nor delay other subscribers; backlog coalesces instead of queueing. The ws writer also sets a write deadline so a peer that stops reading cannot wedge the writer goroutine that Close() waits on.
…restart after unsupported TTL An established watch stream dying now publishes a wildcard — the previous path-scoped signal only refreshed the parent listing, leaving the stale directory's own contents unrefreshed. Client-side, dirsFromChangedPaths includes each changed path itself alongside its parent so a directory-valued signal (deleted watched dir) re-lists that directory too. SetSubscription also (re)acquires every wanted key rather than only newly added ones, so a subscription whose watch was marked unsupported restarts after the TTL even when the client re-sends an identical directory set (workspace upgraded to a watch-capable bridge).
…cation The unsupported mark now schedules its own TTL-expiry timer that reacquires still-wanted keys — the client's fs_watch reporter suppresses identical sets, so nothing external retriggers acquisition after an in-place bridge upgrade. startWatchLocked also enforces per-bot (128) and process-wide (1024) watch budgets: every unique (bot, dir) costs a host goroutine, a bridge stream, and bridge watch state, and the per-connection 64-dir cap alone lets many connections amplify without bound. Over-budget keys stay subscribed but watchless; existing watches are never evicted.
Each inotify instance is a scarce kernel resource (fs.inotify.max_user_instances defaults to 128); one watcher per stream let a busy files pane starve every other inotify user in the container. Streams now share a single lazily-started watcher with per-directory refcounted Add/Remove and per-sink mailboxes; sink overflow degrades to reporting the watched directory itself as changed, and watcher failure ends all streams so hosts re-subscribe against a fresh instance.
…rees Budget-rejected keys stayed watchless forever once capacity opened up: releaseLocked only deleted the freed watch, and the client suppresses unchanged fs_watch reports, so nothing retriggered acquisition. Any release path (drop, set shrink) that actually frees a watch now runs one promotion pass over still-wanted watchless keys, bounded by the existing budget and unsupported gates.
runWatch's deletion paths (unsupported and stream failure) freed budget without promoting, so a cross-bot waiter could stay watchless indefinitely while unsupported-TTL timers only reacquired the failing bot's own keys. Both branches now run the promotion pass under the same lock; the just-died key is excluded so its recovery stays with the delayed retry instead of spinning through promotion on a persistent failure. Invariant: every s.watches deletion is followed by one promotion pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
解决 memohai/Memoh-Cloud#220(文件面板每 2 秒轮询
fs/list)。问题
Web UI 的文件树、文件查看器、预览面板各自持有一个无条件的 2 秒
setInterval,只要标签页开着就持续拉取。除请求量本身(每 tick = 根目录 + 每个展开目录各一条,叠加 JWT + DB 鉴权 + bridge RPC)外,在 Cloud 上后果更重:E2B 沙箱按数据面活动判定 AutoPause,且数据面调用会透明唤醒 PAUSED 沙箱——挂着文件面板的标签页 = 沙箱永远无法暂停。方案
对照 VS Code(watcher 推送、零轮询)、GitHub Codespaces(空闲策略在会话层而非文件层)、JupyterLab(无 watch 基建时的退避轮询基线)后取形:
internal/fsevent:进程内 per-bot hub,200ms 合并窗口,>16 路径坍缩为通配;每订阅者独立信箱 goroutine,慢消费者不拖累他人。assembleTools内,write/edit 带路径、exec/apply_patch 通配,覆盖 web/渠道/schedule/后台全部触发面);ACP prompt 事件 sink(mapper 已归一化为 native 工具词汇,分类器fsevent.ToolChange两侧共享);bridgeWatchDir流(见下)。fs_changed {paths|null},前端汇入现有 fs-beacon,树按变更路径的父目录+自身定向重列;客户端用fs_watch {dirs}上报展开目录集(借 reliable-request 队列断线自动重发)。WatchDir(新 gRPC,非递归):全部流复用单一 fsnotify watcher(每个 inotify instance 是稀缺内核资源,默认 128/user),按目录 refcount Add/Remove,sink 溢出退化为整目录重列;旧 bridge 返回 Unimplemented 自动退避,Cloud/E2B 天然走无 watch 降级。FSWatchService:watch 生命周期 = 观察者注意力(面板隐藏即清空);per-bot 128 / 全局 1024 预算;不变量:每处 watch 删除后跟一次 promotion pass(排除 just-died key 以保留重试退避);unsupported 标记自带 TTL 到期自动重获。空闲标签页净流量归零;活跃期刷新由事件驱动。
验证
go build ./...、go test ./...全绿;internal/fsevent、internal/workspace{,/bridge,/bridgesvc}、internal/handlers追加-race -count=3绿。useMediaGallery.test.ts、panel-preview.test.ts两个既有基线失败(在 origin/main 上同样失败,环境性:node 环境缺 localStorage / vue-i18n mock)。备注
mise run bridge:build(WatchDir 需新 bridge 二进制;旧的优雅降级为纯事件模式)。