feat(browser): switch screencast stream from poll-JPEG to CDP screencast frames - #505
Open
angri450 wants to merge 4 commits into
Open
feat(browser): switch screencast stream from poll-JPEG to CDP screencast frames#505angri450 wants to merge 4 commits into
angri450 wants to merge 4 commits into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Tab 切换导致 CDP client 更换时，旧 screencast handler/会话的清理与 ack 连接绑定存在问题，可能引发帧停更与 handler 泄漏。
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
这个 PR 将 dashboard 的浏览器画面推流从“每 250ms 轮询 Page.captureScreenshot”切换为基于 CDP 的事件驱动 Page.startScreencast 帧推送，并保留轮询作为不支持 screencast 的 CDP 端点的 fallback。整体目标是在 headless/小 VPS 场景下显著降低无变化页面的 CPU/带宽浪费、避免截图超时带来的卡顿，并让帧率随页面变化而变化。
Changes:
- 新增
Page.screencastFrame监听器：推送{"type":"frame"}并对每帧sessionId做Page.screencastFrameAck - stream loop 改为：screencast（主路径）+ 1s 一次 session/tabs snapshot + 轮询截图 fallback
- 支持 tab switch 时检测 CDP client 变化并尝试重新启动 screencast
File summaries
| File | Description |
|---|---|
src/octop/api/routers/browser/stream.py |
将浏览器 WS 推流从轮询 JPEG 改为基于 CDP screencast 的事件驱动帧推送，并在不支持时回退到轮询截图 |
Review details
Suppressed comments (1)
src/octop/api/routers/browser/stream.py:239
- On tab switch,
_stream_loopstarts a new screencast on the new CDP client but never detaches the oldPage.screencastFramehandler or stops the old screencast. This can leak handlers and leave the previous target streaming until Chrome stalls, and it also makes the_screencast_handlersmap grow over time. Detach/stop the previous client before switching to the new one.
# A tab switch replaces the CDP client, so restart screencast on
# the new client to keep frames flowing.
current_client = sess._internal.client # noqa: SLF001
if not listen_only and current_client is not last_client:
last_client = current_client
- Files reviewed: 1/1 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+64
to
+88
| def _make_screencast_handler(sess: Any, ws: WebSocket) -> Any: | ||
| """Construct a Page.screencastFrame listener: forward frame + ack. | ||
|
|
||
| The ack is sent through ``sess._internal.client`` (the *current* client) | ||
| rather than the one captured at registration, so frames keep being acked | ||
| after a tab switch replaces the CDP client. | ||
| """ | ||
| last_sent = 0.0 | ||
|
|
||
| async def _on_frame(params: dict[str, Any]) -> None: | ||
| nonlocal last_sent | ||
| data = params.get("data") | ||
| sid = params.get("sessionId") | ||
| now = time.monotonic() | ||
| if data and (now - last_sent) >= _SCREENCAST_MIN_INTERVAL_S: | ||
| last_sent = now | ||
| await _send_json(ws, {"type": "frame", "data": data}) | ||
| # Must ack regardless of forwarding, otherwise Chrome stops sending | ||
| # subsequent screencast frames. | ||
| if sid is not None: | ||
| with contextlib.suppress(Exception): | ||
| await sess._internal.client.send( # noqa: SLF001 | ||
| "Page.screencastFrameAck", {"sessionId": sid} | ||
| ) | ||
|
|
…ast frames
Replace the 4fps poll loop (Page.captureScreenshot + 1.5s timeout) in
browser-stream with event-driven Page.startScreencast frames:
- Frames are pushed only when the page content changes: static pages
cost zero frames, interactive pages follow the actual frame rate
instead of a hard 4fps ceiling.
- Frame size follows the viewport (no maxWidth/maxHeight), keeping
canvas coordinates 1:1 with CDP input coordinates - fixes pointer
drift that a shrunk screencast frame causes.
- ACK every screencastFrame via its sessionId, otherwise Chrome stops
sending frames.
- The screencast listener is re-registered when a tab switch replaces
the CDP client; the poll loop remains as a fallback for CDP endpoints
without screencast support.
- Wire protocol unchanged: dashboard still receives
{"type":"frame","data":...}, no frontend changes required.
Verified end-to-end against a live Octop instance: initial frame
arrives immediately, resize adapts frame size without restarting the
stream, and static pages push no frames.
- Input events (mouse, wheel, resize) and screencast acks now use CDPClient.send_no_wait: no round-trip wait per event, so rapid drags stop piling up latency. Serial awaits in the stream handler keep socket write order stable. - Screencast jpeg quality 80 -> 60 and throttle cap 30 -> 20fps: bounds bandwidth for remote clients (a full-page change frame at 80q/25fps needs ~47Mbps; now ~3.5Mbps at 20fps). 20fps is still smooth for dragging. Note: requires harness-browser >= 0.7.7 providing CDPClient.send_no_wait; falls back to send() when absent (attribute check).
Older harness-browser (<0.7.7) lacks send_no_wait; route through a helper that falls back to the awaiting send() so the stream keeps working without a hard dependency bump. Newer versions get the fire-and-forget fast path.
angri450
force-pushed
the
feature/browser-screencast-stream
branch
from
September 2, 2026 10:17
00baf06 to
2dcc8da
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The dashboard browser stream currently polls
Page.captureScreenshotevery 250ms (4fps ceiling, 1.5s per-capture timeout). On headless servers this is both laggy (fixed 4fps, captures can hang on slow pages) and wasteful (full-viewport JPEG every tick even when nothing changed).This PR switches the stream to event-driven
Page.startScreencastframes, the same mechanism DevTools itself uses:maxWidth/maxHeight), keeping canvas coordinates 1:1 with CDP input coordinates — fixes pointer drift that a shrunk frame causes.screencastFrameis acked via itssessionId(required, otherwise Chrome stops sending frames).Wire protocol is unchanged: the dashboard still receives
{"type":"frame","data":...}, so no frontend changes are required.Motivation
Reported in #485: the harness-browser Chromium stays resident forever; and the poll-based stream makes the workbench browser feel unusable on small VPSes (4fps + 1.5s timeout hangs). This PR addresses the streaming half. An idle-timeout for the resident browser is a separate follow-up.
Test plan
Verified end-to-end against a live Octop 0.9.28 instance (headless Chromium 151):
start(sub-second).resize(Emulation.setDeviceMetricsOverride) adapts frame size automatically — no stream restart needed.Unit tests for the stream module (
test_browser_stream_listen.py,test_browser_stream_mouse.py) should remain green — the listen-only loop is untouched, and the mouse dispatch path is unchanged.