Skip to content

LATENCY.md

Latest commit

 

History

History
41 lines (28 loc) · 1.9 KB

LATENCY.md

File metadata and controls

41 lines (28 loc) · 1.9 KB
Raw

Latency & Performance Breakdown

vcam is designed for ultra-low glass-to-screen latency (~50-100ms over USB). This doc breaks down where time is spent in the pipeline and how we keep it fast.

Latency Breakdown (USB Mode, 720p @ 30fps)

Pipeline Stage Latency Implementation Details
Camera Capture & Encoding ~20–40 ms Android MediaCodec H.264 hardware encoder reading directly from a CameraX Surface.
Transport (ADB USB) ~5–15 ms TCP framing over adb forward local socket.
Decode & Texture Upload ~10–25 ms OpenH264 C decoder linked directly into the Zig host process with zero-copy buffer binding.
Total Preview Latency ~50–100 ms Direct OpenGL/GUI preview window output.
Total Virtual Camera Latency ~80–150 ms PipeWire SPA node / v4l2loopback buffer write.

Key Performance Optimizations

  1. Direct Surface Hardware Encoding

    • CameraX streams straight into MediaCodec via EGL/Surface. No CPU pixel copying or bitmap conversion on Android.

  2. Ring-Buffer Frame Dropping

    • The desktop receiver uses a single-slot buffer. If the decoder or UI lags behind, old frames are dropped immediately so latency never accumulates over time.

  3. In-Process OpenH264 Decoder

    • OpenH264 is compiled directly into the Zig desktop binary. No external process IPC or socket serialization overhead on the desktop side.

  4. Shared Single-Stream Architecture

    • A single H.264 network stream drives both the desktop preview and virtual camera sinks (/dev/video* and PipeWire) concurrently.

Performance HUD Metrics

The desktop HUD measures pipeline health in real time:

  • FPS: Actual rendered frame rate.
  • Bitrate: Incoming stream bandwidth (Kbps / Mbps).
  • RTT: Control channel ping to the Android device.
  • Keyframe Age: Time elapsed since the last H.264 I-frame.