[dfattal]

Design only — no code, no header, no index.json note (those land together in R1; the catalog lint requires both).

Why

On 3D laptops/monitors the stereo camera is the eye tracker's and is held exclusively while tracking, so the web SDK's 3D call module (@displayxr/inline3d/call, RFC 0002) gets NotReadableError: Device in use; when the tracker is idle the device opens as a raw, uncalibrated 1280x480 grey pair. The vendor's own apps read frames from the vendor SDK's shared memory instead — vendor knowledge that belongs in the plug-in.

What

• docs/adr/ADR-043-stereo-camera-source.md — plug-in produces frames (appended xrt_plugin_iface slots, ADR-020, after ADR-042's lift factory), service owns capture/fan-out/vendor-neutral rectification by the active device's calibration, runtime enforces camera privacy. Alternatives: getUserMedia only, vendor-app bridge, vendor code in the browser (rejected: isolation), camera-provider plug-in type (deferred).
• docs/specs/extensions/XR_DXR_stereo_camera.md — vendor-neutral spec: enumerate + state events, calibration raw/rectified, start/stop, latest-wins acquire over a pinned 3-slot shared-memory ring with per-stream wake handles (optional D3D11/AHB transports), eye-tracking coexistence, privacy (OS consent, per-exe consent with browser delegation, foreground-only, indicator, kill switch, anti-fingerprinting), plug-in slots, sim_display fake, displayxr-cli camera probes.
• docs/roadmap/stereo-camera-source.md — DisplayXR Browser exposes each camera as an ordinary getUserMedia device + one hint (MediaTrackSettings.displayxrStereo), Windows and Android; Leia SR sourcing notes; phased plan R1-R3 / L1 / B1 / A1 / B2; risks; open questions.

Merge-order note

ADR index lines follow ADR-041; the lift PR (ADR-042, feat/lift-ext) adds its line at the same spot — trivial conflict. The planned iface slots append after create_dp_d3d11_lift, so R1 must land after the lift PR.

Open questions (maintainer)

See roadmap §G — name/scope, iface slots vs provider plug-in type, in-process apps, runtime consent UX, SR SDK asks (versioned header w/ sequence + timestamp, non-stealing wake, calibration API, serial = folder name), CNSDK frame sharing, browser capture-service process topology, hint shape, RAW exposure to pages.

🤖 Generated with Claude Code

[dfattal]

Field facts (2026-09-25/26). These feed the design, and #1746 folds them into the roadmap's §C Android section and into §F. They replace the "CNSDK gives SBS" assumption in §C.1.

Android tablet (NP02J / K68), measured

• Camera ids. Camera2 exposes 6 ids. 0 and 2 are back lenses, 1 and 3 front lenses. 4 (back) and 5 (front) are LOGICAL multi-cameras, i.e. the stereo pairs. They are hidden from getCameraIdList() and are opened by hard-coded id. Physical lens per eye: front L = 3, R = 1; back L = 2, R = 0.
• The DisplayXR Browser for Android cannot reach the pair. enumerateDevices() lists no video inputs, getUserMedia gets only the front single camera, and facingMode: 'environment' fails with NotFoundError. Filed as DisplayXR/displayxr-browser-pvt#175.
• The vendor app's stereo path, and what CNSDK actually does. The vendor app calls LeiaCameraBuilder(ctx).setStereoMode(true).setFront(b).setConvergenceMode(AUTO).setPreviewSurfaceView(surfaceTexture) and receives a ready 2560×720 SBS image. LeiaCameraBuilder is not part of CNSDK. It is a separate, private wrapper over plain Camera2: it opens the logical camera with one output per lens, and its shader does the SBS pack and the rectification. CNSDK supplies only the calibration (per-camera rotations Rx/Ry/Rz from its display-config service, config v4.0; the /sdcard/rect.js file the vendor app also reads is absent on the unit) and in-app face tracking.
• What the pair delivers. Two separate 1280×720 NV12/YUV streams, one per lens, at exactly 30.00 fps. The two eyes are hardware-synchronised (identical timestamps). Exposure to app takes about 97–100 ms. The frames are raw: not SBS, not rectified, not mirrored. The front eyes need mirroring to give viewer L/R. Vertical misalignment is about −2 px (median), roll about 0.09°.
• Exclusivity. Opening the front pair (5) evicts the head-tracking service's camera (it held camera 3 at 10 fps). The tracker then stays wedged ("Camera has been invalidated") and never reacquires it. The rear pair does not conflict. The vendor's answer is in-app tracking: the capturing process feeds a third 640×480 left-lens stream to CNSDK in-app mode. CNSDK's hand-off runs face detection on the caller's thread (~119 ms), which dragged the pair to 24 fps when it was called on the camera thread. It must run on a worker behind a latest-frame slot.

Windows

On two SR machines the eye tracker holds the stereo camera exclusively: getUserMedia → NotReadableError. This confirms the design.

Design consequence

• On Android the provider lives in the runtime APK (ADR-038), not in Chromium. The runtime service must be the one owner of the front pair, and it must run tracking in-process from the same capture: a third stream goes to the in-app CNSDK tracker on a worker thread. A page, another plug-in, or an app must never open the pair behind the tracker.
• Transport: AHardwareBuffer per eye (NV12). Rectification happens in the runtime (R2's rectifier, fed the CNSDK rotations) or in the consumer.
• A1 contract gap: R1's plug-in frame is one CPU SBS image. The tablet needs a per-eye AHB layout, which can be appended per ADR-020.

Vendor asks (added to §F)

1. The head-tracking service must reacquire its camera when another client releases it.
2. Can the tracker run from the pair's own frames, so no third stream is needed?
3. Expose the pair's calibration through a documented CNSDK call.

Maintainer defaults applied in R1 (recorded in roadmap §G, and implemented in #1746):

• stereo-only, with names and structs kept extensible;
• xrt_plugin_iface slots behind XRT_PLUGIN_IFACE_HAS_STEREO_CAMERA;
• service clients only (in-process instances enumerate zero);
• no raw frames to pages (RAW is refused to the browser's PRESENT_OWNER connection).

One pre-existing lint item on this PR: docs/adr/README.md was stale because of a hand-added "(Proposed)" suffix, which the generator strips. #1746 regenerates it. ADR-043's link to ADR-042 stays dead until the lift PR lands.