Skip to content

fix(preview): adaptive streaming to fix blur, bandwidth, and latency#422

Merged
ArgaFairuz merged 1 commit into
mainfrom
fix/browser-preview-stream-quality
Jul 15, 2026
Merged

fix(preview): adaptive streaming to fix blur, bandwidth, and latency#422
ArgaFairuz merged 1 commit into
mainfrom
fix/browser-preview-stream-quality

Conversation

@ArgaFairuz

Copy link
Copy Markdown
Collaborator

Rework the browser preview pipeline into a Chrome-Remote-Desktop-style adaptive stream:

  • Capture at native viewport resolution; display fit-scale is CSS-only. Client renders frames 1:1 (canvas backing store follows frame size), removing the upscaling that made the preview permanently blurry.
  • Prefer VP9 in quantizer mode, negotiated with the client's decode support at stream start (VP8 bitrate-mode fallback): cheap per-frame quality during motion, raised further under congestion.
  • Static top-off: when the screencast goes quiet for 300ms, push one lossless PNG screenshot through the encoder near-losslessly so text sharpens right after motion stops; idle pages cost ~0 bandwidth. Deferred while the channel is congested and abandoned if motion resumes, so large refreshes never add latency.
  • Reliable ordered DataChannel (VP8/VP9 delta chains cannot tolerate loss/reorder); latency is bounded by source-side frame dropping with a small 64KB backpressure window instead — queued bytes are display lag, so the stream degrades to coarser-but-current, never stale.
  • Fragment frames over 64KB (packet type 2) to stay under SCTP message-size limits; packet header now carries a codec byte.
  • Keyframes on demand only: channel open, encoder reconfigure, or the new preview:browser-stream-keyframe request (PLI equivalent) that also works on still pages; no periodic keyframe bursts.
  • Rate-limit encoding to the configured framerate at the CDP feeder with a trailing edge — the last frame of a damage burst is always encoded, keeping input feedback immediate while capping CPU and bandwidth (screencast fires at compositor rate, up to 60fps).
  • Add a 5s grace period for transient ICE 'disconnected' states on flaky networks (mobile/tunnel) — fixes the endless loading/reconnect loop when the preview is accessed remotely.
  • Drop audio chunks during congestion; the client's playback scheduler handles gaps cleanly.

Rework the browser preview pipeline into a Chrome-Remote-Desktop-style
adaptive stream:

- Capture at native viewport resolution; display fit-scale is CSS-only.
  Client renders frames 1:1 (canvas backing store follows frame size),
  removing the upscaling that made the preview permanently blurry.
- Prefer VP9 in quantizer mode, negotiated with the client's decode
  support at stream start (VP8 bitrate-mode fallback): cheap per-frame
  quality during motion, raised further under congestion.
- Static top-off: when the screencast goes quiet for 300ms, push one
  lossless PNG screenshot through the encoder near-losslessly so text
  sharpens right after motion stops; idle pages cost ~0 bandwidth.
  Deferred while the channel is congested and abandoned if motion
  resumes, so large refreshes never add latency.
- Reliable ordered DataChannel (VP8/VP9 delta chains cannot tolerate
  loss/reorder); latency is bounded by source-side frame dropping with
  a small 64KB backpressure window instead — queued bytes are display
  lag, so the stream degrades to coarser-but-current, never stale.
- Fragment frames over 64KB (packet type 2) to stay under SCTP
  message-size limits; packet header now carries a codec byte.
- Keyframes on demand only: channel open, encoder reconfigure, or the
  new preview:browser-stream-keyframe request (PLI equivalent) that
  also works on still pages; no periodic keyframe bursts.
- Rate-limit encoding to the configured framerate at the CDP feeder
  with a trailing edge — the last frame of a damage burst is always
  encoded, keeping input feedback immediate while capping CPU and
  bandwidth (screencast fires at compositor rate, up to 60fps).
- Add a 5s grace period for transient ICE 'disconnected' states on
  flaky networks (mobile/tunnel) — fixes the endless loading/reconnect
  loop when the preview is accessed remotely.
- Drop audio chunks during congestion; the client's playback scheduler
  handles gaps cleanly.
@ArgaFairuz
ArgaFairuz merged commit 75ecb8c into main Jul 15, 2026
1 check passed
@ArgaFairuz
ArgaFairuz deleted the fix/browser-preview-stream-quality branch July 15, 2026 04:59
@ArgaFairuz ArgaFairuz mentioned this pull request Jul 16, 2026
ArgaFairuz added a commit that referenced this pull request Jul 17, 2026
main's opencode PR (#424) is the squash-merge of this branch's own 467ca9a,
so the five conflicting opencode files were resolved in favour of our version
(a strict superset). The new preview (#422) and subagent-trim (#425) work
merged cleanly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant