Hance Live: virtual camera for real-time capture

[RichardBray]

Summary

Add Hance Live: a virtual camera that applies the full effect chain to a live
capture device in real time and publishes the result as a macOS camera, so
Camtasia (and anything else with a camera picker) can record the graded feed
directly.

Desktop app only. The CLI is explicitly out of scope for this feature.

Target for v1: 1080p30 out of a capture card into Camtasia. Built-in FaceTime
camera, 4K, and 60fps are follow-ups, not v1.

Why this is tractable

The engine is already most of the way there:

• Every pass in packages/wgpu/src/renderer.rs:301 is stateless per frame.
  The only time-varying inputs are frame_count into the grain
  (renderer.rs:542) and camera-shake (renderer.rs:606) uniforms. Nothing
  reads a previous frame and nothing buffers, so the full chain adds zero
  frames of latency. All effects can run live.
• hance-gpu is already a streaming frame-in/frame-out loop
  (packages/wgpu/src/main.rs:53), not a batch tool.
• The WGSL in packages/core/shaders/ is the single source of truth, included
  by both the Rust renderer (renderer.rs:7) and the browser WebGPU preview
  (packages/ui/app/gpu/renderer.ts). No shader is written twice.

What does not work as-is

1. The frame path would cross four processes. Electrobun gives us a Bun
process and a WKWebView, no native code. Capture is AVFoundation and the camera
extension is a separate system-launched process, so a naive port routes every
frame capture -> Bun -> pipe -> hance-gpu -> pipe -> Bun -> extension. At
1080p that is ~500MB/s each way over stdio.

2. Readback is synchronous. render_frame blocks on map_async +
poll(Maintain::Wait) (renderer.rs:661), so CPU and GPU never overlap. Fine
for export throughput, wrong for a live preview.

3. Params are init-only. InitMessage (packages/wgpu/src/params.rs:5)
fixes params at process start; changing a slider today means restarting the
sidecar. Live needs a param-update channel on the running loop.

4. Electrobun cannot currently host a system extension. A CMIO extension
must be a nested signed bundle at Contents/Library/SystemExtensions/ with its
own identifier and provisioning profile, matching the host app's team ID. The
copy: block in packages/desktop/electrobun.config.ts is a plain file copy,
and entitlements.plist carries the loosest possible set
(disable-library-validation, allow-unsigned-executable-memory), the opposite
of what an extension host wants.

Architecture

Keep the Electrobun app out of the frame path entirely.

capture card --AVFoundation--> [ hance-live helper (Swift) ]
                                   |  CVPixelBuffer -> wgpu -> CVPixelBuffer
                                   |  (Rust renderer via C ABI, GPU-resident)
                                   |
                    +--------------+---------------+
                    |                              |
              IOSurface to                  downscaled preview
           CMIO camera extension            + control channel
                    |                              |
               Camtasia                    Electrobun webview

• Swift owns capture (AVFoundation), the extension (CMIO is Swift/ObjC
  only), and the IOSurface handoff.
• Rust keeps the effect chain. packages/wgpu splits into a lib.rs
  exposing init/render/set_params/destroy over a C ABI, plus a thin main.rs
  that keeps today's stdio loop. The file-export path is untouched.
• The webview becomes a control panel: it sends param updates and receives a
  scaled preview. Params are tiny and fit the existing local HTTP server in
  packages/desktop/src/bun/server.ts.

Preview resolution: output to Camtasia is full 1080p; the preview only needs
540p-720p. Note that grain reads weaker when downscaled, so grain values must be
confirmed against a real recording, not tuned by eye from the preview.

Phases

Phase 0 - Unblock signing

• Enrol in the Apple Developer Program. Local testing of a camera extension
  needs the com.apple.developer.system-extension.install entitlement,
  which needs a provisioning profile, which needs a paid account. There is
  no meaningful local-only path.

Phase 1 - Prove the delivery (do this before any Hance code)

• Minimal CMIO camera extension that publishes a hardcoded colour frame.
• Nest it in the Electrobun-built .app, sign it, install it, confirm it
  appears in Camtasia's source picker showing the test pattern.
• Extend electrobun.config.ts / the build script with the nested-bundle
  signing step; add a system-extension entitlements file separate from the
  permissive host entitlements.plist.

This is the highest-risk, lowest-visual-payoff phase and it is independent of
the effects. If a green rectangle reaches Camtasia, the rest is known work.

Phase 2 - Rust renderer as a library

• Split packages/wgpu into lib.rs (C ABI: hance_gpu_init,
  hance_gpu_render, hance_gpu_set_params, hance_gpu_destroy) and a
  thin main.rs retaining the stdio loop.
• Add a param-update entry point so params are no longer init-only.
• Build as a staticlib alongside the existing binary; existing export tests
  must keep passing unchanged.

Phase 3 - Capture and pipeline

• Swift helper: enumerate AVFoundation devices, open the capture card, pull
  CVPixelBuffers at 1080p30.
• Bridge frames into wgpu. Zero-copy via wgpu-hal's Metal backend wrapping
  a MTLTexture created from the IOSurface.
  Risk: wgpu exposes no public IOSurface support, so this is unsafe
  interop against an internal API and wgpu = "24" bumps can break it.
  Fallback: copy through CPU memory. Two memcpys of a 1080p frame is
  ~8MB per frame, unnoticeable at 30fps and entirely shippable for v1. Take
  the fallback without hesitation if the zero-copy path stalls.
• Handle capture-card colour formats correctly (10-bit / YUV variants land
  on the wrong values if converted naively).
• Publish finished frames to the extension.

Phase 4 - UI

• New entry point in the app: choose between opening a file (existing flow)
  and Hance Live.
• Device picker.
• Live preview at reduced resolution.
• Existing effect controls drive the live chain through the param-update
  channel, reusing the current panels.
• Menu entry in packages/desktop/src/bun/menu.ts.

Phase 5 - Ship

• Extension install/uninstall UX, including the macOS approval prompt (the
  single biggest install-funnel drop-off).
• Helper lifecycle: start, stop, crash recovery, device disconnect.
• End-to-end test on real hardware (Elgato, not the built-in camera).
• Docs.

Open questions

• Which capture card is the reference device for testing?
• Does Hance Live ship in the free app or gate behind the pro licence?
• Should the app structure change now that the desktop app, not the CLI, is the
  centre of gravity? The repo, build, and release flow are still shaped around
  shipping a single binary.

Out of scope for v1

CLI support, Windows/Linux, 4K, 60fps, built-in FaceTime camera, audio,
Zoom/Teams/Meet compatibility testing.