[dfattal]

Phase R1 of the stereo camera source (ADR-043, XR_DXR_stereo_camera, roadmap). No hardware is needed: the display's stereo camera becomes a plug-in-provided source that the service owns and clients consume, end to end against a sim_display fake.

Merge order (important)

Rebased onto post-lift main (2026-10-03). Lift (ADR-042, #1781) shipped in v2.22.0 and ADR-045 (get_platform_state) followed, so the placeholder scheme below is history. Order now: #1746 (R1) → #1747 (R2) → DisplayXR/displayxr-leia-plugin#273 (L1) → B1 (browser).

What the rebase moved (details in the rebase comment):

• xrt_plugin_iface: the reserved_adr042_create_dp_d3d11_lift placeholder is gone; the six camera slots are appended AFTER create_dp_d3d11_lift (member 19, offset 136) and get_platform_state (member 20, offset 144) — members 21–26, offsets 152–192, sizeof 200. tests_stereo_camera pins that order (lift right after the vk fingerprint, platform-state right after lift, camera block right after platform-state and ending the struct).
• Type values relocated 1004999290–300 → 1004999300–310: 290 had meanwhile been taken by XR_TYPE_WEAVE_SUBMIT_OVERLAY_UNCHANGED_DXR (weave spec v14). Safe: spec v1, nothing shipped chains them. Registry rows added; next free is 320+.
• displayxr-cli selftest: CLI_SELFTEST_BAD_STEREO_CAMERA is exit 12 (11 is lift's BAD_LIFT_CAPS).
• Append-at-end conflicts (xrt_openxr_includes.h, ipc_server.h, per-client teardown, oxr_*, cli, tests/CMakeLists.txt, env census count 100 → 102) resolved lift-first, camera-second. proto.json's stereo_camera_* calls already sat after instance_declare_peer, i.e. after lift's.
• No DP vtable slots are touched by this PR (the D3D11 lift slots BASE+25..29 are untouched); the camera lives on the iface only.

Original (pre-rebase) merge-order note, for the record

1. docs(stereo-camera): plug-in-provided stereo camera source (ADR-043, XR_DXR_stereo_camera design) #1743 (design docs) first. This branch is stacked on it (15ff2dfe2 is its commit).
2. The lift PR (feat/lift-ext, ADR-042) before this one. Lift claims the xrt_plugin_iface slot right after vk_bundle_fn_table_offset. This PR holds that offset with a same-size placeholder (reserved_adr042_create_dp_d3d11_lift), so the camera slots already sit at their final offsets and either merge order is ABI-safe. tests_stereo_camera asserts that the camera block starts right after the lift slot, or after its placeholder while lift is absent. The rebase onto lift is mechanical:
   • xrt_plugin.h: delete the placeholder; lift's create_dp_d3d11_lift takes its place.
   • openxr/README.md registry: drop the reserved 270–289 row in favour of lift's two rows.
   • Append-at-end conflicts only: xrt_openxr_includes.h, ipc_server.h (ipc_client_state tail), ipc_server_per_client_thread.c teardown, oxr_api_negotiate.c / oxr_api_funcs.h / oxr_objects.h / oxr_api_verify.h / oxr_defines.h, tests/CMakeLists.txt.
   • lint (dead link) is red until lift lands: ADR-043 from docs(stereo-camera): plug-in-provided stereo camera source (ADR-043, XR_DXR_stereo_camera design) #1743 links ADR-042-…md, which only exists on the lift branch.

What's in it

• Header openxr/XR_DXR_stereo_camera.h (spec v1, type values 1004999300–310) and its index.json catalog note. gen_extensions_index.py --check passes.
• Plug-in slots (ADR-020 append-only, XRT_PLUGIN_IFACE_HAS_STEREO_CAMERA): enumerate, get_calibration (OpenCV R,T of the active device), open (which doubles as the tracker keep-alive), wait_frame, release_frame, close.
• sim_display fake, turned on by SIM_DISPLAY_FAKE_STEREO_CAMERA=1 in the service environment. It renders a random-dot pair: background at 12 px disparity, a bar at 40 px (68° HFOV, 50 mm, 2.0 / 0.6 m), and a frame counter in both halves. Knobs: _SIZE, _FPS, _FORMAT, _SUSPEND_PERIOD_MS.
• Service camera manager (ipc/server/ipc_server_stereo_camera.c, platform-neutral):
  • One thread per open camera. The source opens on the first start and closes 2 s after the last stop.
  • Each stream gets a 3-slot latest-wins pinned ring in shared memory, format conversion (GRAY8 / NV12 / BGRA8), decimation, and a wake handle of its own.
  • Wake handles are a SYNCHRONIZE-only auto-reset event on Windows and a pipe on POSIX. On Windows the section is handed out as a read-only (FILE_MAP_READ) duplicate.
  • Camera states WAITING / AVAILABLE / SUSPENDED / UNAVAILABLE, with pins cleared on suspension.
  • The D3D11-texture and AHB / dma-buf transports are designed in that file's header comment.
• IPC (12 stereo_camera_* calls). Streams are owned by the connection. The client-side xrt_stereo_camera_client aspect is carried by the IPC xrt_instance.
• OpenXR (9 entry points, instance-level). An in-process instance enumerates zero.
• displayxr-cli camera list | calib | probe, and a stereo_camera_caps check in selftest.
• Privacy hooks, deny by default (the full R3 work is later):
  • Stream start and calibration reads are refused with XR_ERROR_PERMISSION_INSUFFICIENT unless the service has DXR_STEREO_CAMERA_DEV_ALLOW=1.
  • DXR_STEREO_CAMERA=0 is a kill switch.
  • RAW is refused to PRESENT_OWNER clients (the browser).
  • Foreground-rule and indicator hook points are marked in the code.
• Docs: the spec, roadmap and ADR describe R1 as built and record the maintainer decisions (stereo-only, iface slots, service clients only, no raw frames to pages). The roadmap's Android section is rewritten from the NP02J/K68 measurements. Also updated: the xrt_plugin_iface reference and the env-var census.

Test evidence (macOS, Debug, hybrid build)

• tests_stereo_camera: 9 cases, 1616 assertions pass. They cover:

  • the ring (pinned slot byte-stable over 1000 publishes; skip counting)
  • the decimator (30→15/20/10 exact, no burst after a 2 s pause)
  • conversion round trips
  • the persistent id
  • the fake's disparities measured as 12 ± 0.5 and 40 ± 0.5 px
  • iface adjacency

  tests_ipc_proto passes.

• camera probe against the fake:

  • 29.95 Hz measured over 150 frames (index 1..150, 0 gaps).
  • Block disparity 12 px (38/54 blocks), 40 px (16/54).
  • Service stats: source 30.49 Hz, mean plug-in→publish latency 12.4 ms.

• Three concurrent probes, fanned out from one open:

  • GRAY8 capped at 15 Hz → 15.02 Hz
  • BGRA8 at the source rate → 29.96 Hz
  • NV12 --raw capped at 10 Hz → 10.00 Hz

• Suspend square wave (1 s): 3 suspensions, 92 frames in 6 s, and the stream resumes after each one.

• Killing the service mid-stream: it exits cleanly and the source closes.

• Without DEV_ALLOW, calib and start return -39 (NOT_AUTHORIZED). With DXR_STEREO_CAMERA=0, zero cameras are listed.

• OpenXR level: a scratch app dlopens the runtime with XRT_FORCE_MODE=ipc and runs enumerate → calibration → create → start → transport → 60 × acquire at 30.00 Hz of captureTime. The consent refusal comes back as XR_ERROR_PERMISSION_INSUFFICIENT, and in-process gets 0 cameras.

• selftest: PASS: stereo_camera_caps — 1 camera(s); … flags 0x1f with the fake, and 0 cameras (OK) without it.

• MinGW-w64: build-mingw-check.sh aux_util drv_sim_display passes. A -fsyntax-only pass over the Windows branches of the server, client, CLI and oxr files is clean. That pass also caught _Static_assert on static const XrFlags64, which is fixed.

Deviations from the design

• wait_frame returns enum xrt_plugin_stereo_camera_wait instead of xrt_result_t values, so no new result codes were needed.
• The plug-in's extrinsics use OpenCV's x_R = R·x_L + T; the service converts them to the XR rightFromLeft pose.
• Rings are per stream, one pin each, rather than one ring shared across streams.
• The R1 fake is undistorted and reports NATIVELY_RECTIFIED. The distorted variant arrives with R2's rectifier. A calibrated-but-not-rectified source delivers RAW with the frame flagged as such.
• State-change events are defined but not delivered yet: that needs a server→client instance-event channel.
• persistentId is not keyed yet (R3).

Not tested

Windows MSVC, Linux and Android builds and runs (CI covers the builds). Any real hardware. The GPU transports (not implemented). The malformed-description path of stereo_camera_caps (the fake cannot produce one).

🤖 Generated with Claude Code

[dfattal]

Follow-ups from the first SR-hardware run (pushed; R2 rebased onto this):

• C5287 (oxr_stereo_camera.c): the XR-vs-xrt enum pins no longer compare two enum types; each side is pinned against the literal (2920da5a0). MSVC raised it even through (int) casts.
• Rate (24bcbdad3): the service measures every source over the first 60 frames after each open (u_stereo_camera_rate_meter) and the descriptor / decimator / stream info use that. Until then it uses the plug-in's value, and 0 now means unknown (xrt_plugin + XR header docs, spec). camera list prints "rate unknown (measured once a stream runs)". Selftest accepts 0. Sim knob: _ADVERTISED_FPS=0.
• Probe disparity window (8a8c30f1a): the window is now f·B/0.4 m from the camera's own baseline + HFOV, clamped to [64, 0.6·eye], instead of a fixed 64 px. Blocks that peak on the bound are counted and excluded, never reported as the bound. Sim knob: _BASELINE_MM (at 120 mm the probe reads 28 / 95 px, search 0..143).

Verified: macOS build, tests_stereo_camera, MinGW check + a standalone MinGW syntax pass over the oxr, ipc_server and cli files, and camera list/probe on the fake with unknown rate and 120 mm. Not run on MSVC here.

[dfattal]

Rebased onto post-lift main (2026-10-03)

Lift (ADR-042, #1781, v2.22.0) and ADR-045 (get_platform_state) are on main, so the placeholder scheme is retired. All three stereo-camera branches were force-pushed (--force-with-lease), every commit kept, one mechanical fixup commit at the end of R1. Order: #1746 (R1) → #1747 (R2) → DisplayXR/displayxr-leia-plugin#273 (L1) → B1.

branch	old head	new head	base
feat/stereo-camera-r1	8a8c30f1a	cd2927aae	origin/main @ 1f64f78bf (v2.25.3)
feat/stereo-camera-r2	f433d31fc	0191f5481	new R1 — f433d31fc (PRESENT_OWNER owner quota) survives as 0191f5481
leia feat/stereo-camera-l1	aa3253d	e9fe9f7	plug-in main @ 48afe3f (v2.9.0+2)

What moved where

xrt_plugin_iface (ADR-020 append-only). reserved_adr042_create_dp_d3d11_lift is deleted. Main had appended ADR-045's get_platform_state after lift's factory, so the camera block goes after both — create_dp_d3d11_lift is no longer the last pre-camera member:

member	index	offset
vk_bundle_fn_table_offset	18	132
create_dp_d3d11_lift (ADR-042)	19	136
get_platform_state (ADR-045)	20	144
stereo_camera_enumerate	21	152
stereo_camera_get_calibration	22	160
stereo_camera_open	23	168
stereo_camera_wait_frame	24	176
stereo_camera_release_frame	25	184
stereo_camera_close	26	192
sizeof(struct xrt_plugin_iface)		200

tests_stereo_camera's ABI case now pins the whole chain (lift right after the vk fingerprint → platform-state right after lift → camera block right after platform-state and ending the struct) with lift REAL, no #ifdef fallback. No DP vtable slot is touched (D3D11 lift's BASE+25..29 untouched).

Type values relocated 1004999290–300 → 1004999300–310. Not a lift collision — 290 was taken on main by XR_TYPE_WEAVE_SUBMIT_OVERLAY_UNCHANGED_DXR (weave spec v14). Safe: spec v1, draft PR, nothing shipped chains them. Registry (openxr/README.md) rows 300–309 / 310–319, next free 320+. gen_extensions_index.py --check and gen_adr_index.py --check pass.

Other resolutions. CLI_SELFTEST_BAD_STEREO_CAMERA = exit 12 (11 is lift's). Append-at-end conflicts in xrt_openxr_includes.h, ipc_server.h, per-client teardown, oxr_api_funcs.h / oxr_api_negotiate.c / oxr_api_verify.h / oxr_defines.h / oxr_extension_support.h / oxr_objects.h, cli CMake/main/common, tests/CMakeLists.txt, env census (100 → 102 names), sim_display_plugin.c iface init — all lift-first, camera-second. proto.json's stereo_camera_* keys already sat after instance_declare_peer (after lift's); the generated IPC files are build-time, nothing checked in.

L1 (leia). Rebased over leia_platform_state / lift-NeurD / compose-colour-test additions (keep-both in leia_plugin.c iface, CMakeLists, build-linux.{sh,yml}, CLAUDE.md). The camera slots stay #ifdef XRT_PLUGIN_IFACE_HAS_STEREO_CAMERA; the committed pins (v2.16.9 Win / v2.18.0 Linux) are untouched, so a stock build is byte-identical. Build it against the rebased R2 head via DXR_RUNTIME_SOURCE_DIR (PR #273 step 0).

Verified on macOS (R2 worktree, Debug hybrid build)

• tests_stereo_camera, tests_stereo_rectify, tests_stereo_vrefine, tests_ipc_proto, tests_lift_mailbox: all pass.
• displayxr-cli camera probe --rectified against the DISTORTED sim fake (fake + DXR_STEREO_CAMERA_DEV_ALLOW=1 in the service env, XRT_PLUGIN_SEARCH_PATH at the fresh dylib): 116 frames at 30.00 Hz, disparity modes 11.30 px → 2.005 m / 37.81 px → 0.599 m, signed-dy median +0.001 px, refinement ALIGNED, PASS — rows aligned. camera calib 1 --rectified prints P1/P2.
• displayxr-cli selftest in-process with the fake: PASS: stereo_camera_caps — 1 camera(s) … flags 0x17, exit 0.
• build-mingw-check.sh aux_util drv_sim_display: PASSED.
• L1: test_stereo_camera_parse passes natively (-std=c99 -pedantic); MinGW -fsyntax-only of leia_plugin.c against R2 headers (slots filled) and main headers (slots out) shows only the two pre-existing leiasr_geometry_* implicit-declaration errors that origin/main's file has too.

Windows box, please re-run

MSVC build of R2 (dev-setup.bat --leia with the plug-in's L1 checkout next to it), tests_stereo_camera (the ABI case with lift real), displayxr-cli selftest, then the #273 on-hardware recipe (camera probe --rectified on the SR tracker, exit-code check: a malformed camera is now 12). Also confirm the owner-quota fix still admits the browser's sibling processes.

[dfattal]

Landed on main via #1819, which carried the whole stack (R1 → R2 → R3, 37 commits) in order after one rebase onto current main — main is at 5d167c4bc, tree identical to the head that passed CI and the hardware gates. Closing rather than merging separately: a rebase-merge rewrites commit ids, so this PR cannot show as merged even though every commit of it is in.