60fps HDMI pipeline: zero-copy VPU encode + RGA hardware CSC + kmssink double-vsync fix

[NickEngmann]

Problem

The HDMI passthrough pipeline capped at 27-30fps at every resolution (dipping to 23 during shows) despite the VPU being rated for 4K60, making media playback visibly janky.

Root causes and fixes

Four stacked bottlenecks, found and fixed in order:

Encoder side (companion branch: garagehq/ustreamer → fix/fps-early-buffer-release, commit 32ab470)

1. Deferred V4L2 buffer release starved capture to half rate. Encode workers held their capture buffer until the dispatcher next picked them (~100ms), permanently pinning 4 of 5 V4L2 buffers — captured_fps dropped 60→30 the moment a client attached. Buffers are now released inside the worker right after encode.
2. The CPU read every pixel of uncached V4L2 mmap memory per frame (~90ms per 1080p NV24 frame vs ~5ms of actual VPU encode). Replaced with hardware paths: NV12 sources import the V4L2 DMABUF directly into MPP (true zero-copy); NV24 (Roku 4:4:4) converts via two RGA passes using an RGBA plane-reinterpretation trick (~2.4ms); BGR24 (Google TV) via RGA imcvtcolor. Any hardware-path failure falls back to the legacy CPU path permanently with one log line. Pixel-modifying features (blocking composite, notification overlay) still route through the CPU path while active — all overlays verified working.

Display side (this branch)

3. videobalance omitted when color settings are neutral — CPU per-pixel color correction caps 4K at ~56fps and breaks the DMABuf zero-copy path into kmssink. The pipeline now rebuilds with/without the element when settings cross neutral.
4. kmssink double-vsync — with the TV attached, display pinned at exactly 30.00fps on a genuine 60Hz mode. Measured proof: the rockchip BSP's legacy drmModeSetPlane ioctl blocks until the flip latches at vblank (16.67ms/call), and kmssink then waits for a second vblank internally → 2 vblanks/frame. Fixed with skip-vsync=true (upstream property intended exactly for this); frame pacing remains vsync-locked by the blocking SetPlane, so no tearing.

Results (measured live)

Metric	Before	After
Stream served (1080p60 NV24 source)	29.4 fps	60.0 fps
Display output (TV attached, 4K60 mode)	30.00 fps	59.99 fps
Grab-to-expose latency	110 ms	17 ms
4K encode headroom (EXT_DMA, 4 workers, 10s soak)	—	66.7 fps aggregate

4K resilience validated with real 4K NV12 content in CMA dma-heap buffers (same memory class as V4L2 capture buffers); display decode benchmarks 120fps at 4K.

Preserved features (verified live)

Blocking overlay (header/vocab/preview/stats/OCR snippet), notification overlay, /snapshot + /snapshot/raw, OCR + VLM detection, autonomous mode — all confirmed working; stream returns to 60fps after a block ends.

Follow-up documented in CLAUDE.md

Color settings are currently neutral (fastest path). The VOP2 plane interprets the video as BT.601/Limited matching the data the RGA path preserves, so neutral should look correct; the old sat=1.4/bri=0.2 likely compensated for the pre-RGA CPU conversion. A COLOR_RANGE lead is documented if color needs revisiting.

🤖 Generated with Claude Code