Skip to content

feat: add video support for intercom lines (WHIP/WHEP, H264) - #684

Open
bjowe wants to merge 4 commits into
Eyevinn:mainfrom
bjowe:feat/video-support
Open

bjowe wants to merge 4 commits into
Eyevinn:mainfrom
bjowe:feat/video-support

Conversation

@bjowe

@bjowe bjowe commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator

Adds in-call participant video: publish the local camera via WHIP and consume remote video via WHEP (H264/VP8 over SMB), with self-preview, per-tile video grid, source pinning, return-feed and fullscreen controls, plus the video-enabled toggle in production setup.

Closes #692

Companion PR (backend): Eyevinn/intercom-manager#315

Adds in-call participant video: publish the local camera via WHIP and
consume remote video via WHEP (H264 over SMB), with self-preview,
per-tile video grid, source pinning, return-feed and fullscreen
controls, plus the video-enabled toggle in production setup.
@bjowe
bjowe force-pushed the feat/video-support branch from e37fafa to e94e56f Compare September 15, 2026 11:03

@birme birme left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated code-reviewer verdict (daily-backlog-pr Phase 3): NEEDS CHANGES

Reviewed the full +4752/−261 diff across 69 files with the branch checked out. Feature-scoped work that mirrors the existing audio patterns well, with good coverage of the new pure logic. Two concrete media-lifecycle bugs should be fixed before shipping a camera feature:

High

  1. Camera capture is never stopped on call exit (privacy/hardware leak). production-line.tsx:149 destructures only const [inputVideoStream] = useVideoInput({...}), discarding the third tuple element — the reset() that stops the camera tracks. useVideoInput.reset (use-video-input.ts:67-72) is the only place videoInput.getTracks().forEach(t => t.stop()) runs, and it's never called. The effect cleanup (:62-64) only sets aborted = true; REMOVE_CALL (global-state-reducer.ts:123) doesn't stop mediaStreamVideoInput; and rtcPeerConnection.close() doesn't stop local getUserMedia tracks. Net: camera stays live + recording indicator on after leaving a video line, until the tab closes. Capture and call the video reset on exit/unmount.
  2. Frame-monitor setIntervals leak on unmount. attachShowWhenReady (video-element-factory.ts:470-490) starts a 500ms interval per remote video element, cleared only by stopFrameMonitor, which is called only from removeThisElement on track ended/removetrack. useVideoElements cleanup (use-video-elements.ts:11-24) nulls srcObject but doesn't call stopFrameMonitor, nor does the useRtcConnection teardown. On abrupt teardown/navigation the interval + rVFC loop keep running against detached elements. Call stopFrameMonitor for every element in cleanUpVideo.

Medium

  1. checkbox.tsx:76-87 — input made readOnly + tabIndex={-1} with toggle delegated to wrapper onClick, synthesizing a fake ChangeEvent via as unknown as. Removes keyboard focus/space-to-toggle from a shared primitive (a11y regression + scope creep). Prefer a native disabled prop.
  2. PC held in useState (never recreated) but stream refs are in the effect deps — a stream-ref change would close() the PC and the re-run no-ops behind signalingState === "closed" guards, dead-ending the call. Latent today (no device-switch re-acquire); add a guard/comment.
  3. No test asserting camera tracks stop / frame monitors clear on exit — exactly where the leaks live.

Nits

  • Raw console.warn/error in prod paths vs the project logger util; video-element-factory.ts builds ~540 lines of tile chrome via imperative createElement+inline styles vs Emotion styled() used elsewhere.

Notes: no user-input URL construction (all via backend API_URL), no new bundle secrets, reducer ERROR change well-contained, CI green. Both High items are small localized fixes.

…n#692)

Capture the useVideoInput reset() and invoke it on exit and on unmount so
camera tracks (getUserMedia) are stopped, releasing the hardware/indicator.
Call stopFrameMonitor for every element in useVideoElements cleanUpVideo so
the 500ms frame-monitor setInterval no longer leaks on teardown.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@birme

birme commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

This PR looks like part of a multi-repo/multi-PR dependency stack together with Eyevinn/intercom-manager#315 (backend video support). Holding off on automated review/merge here — this needs a human (or the relevant implementation agent) to assess the whole stack together, not a per-PR pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: add video support for intercom lines (WHIP/WHEP, H264)

3 participants