feat(lift): app rig drives lift viewpoints + metric depth aux output

[dfattal]

Part of #1858, closes #1860, closes #1861.

Stacked on #1864 (base feat/lift-viewpoint-policy, rebased onto its amended head edf61ecdf). This covers runtime items R5 and R6 (phases 2 and 3) of the lift camera design:

• R5: the app's own view rig drives the lift viewpoints.
• R6: metric depth comes back as an auxiliary output, aligned to display space.

XR_DXR_lift goes to spec v3. It is a separate version, not folded into v2, because v2 (#1864) can ship on its own. Apps key these features on SPEC_VERSION >= 3.

Commits

1. a5af88f5d: API + DP contract. Pushed early so the vendor plug-in can build P2 against it.
2. Implementation. Lift thread, service, IPC, state tracker, sim_display fake, tests.
3. Docs. Spec v3, ADR-048 Addendum A, plug-in iface reference, extensions index.

API (XR_DXR_lift.h)

• App rig (R5). An XrDisplayRigDXR or XrCameraRigDXR (from XR_DXR_view_rig) can be chained on XrLiftOptionsDXR, or directly on XrWeaveRectLiftDXR.
  • The direct chain wins over the options chain, and a camera rig wins over a display rig.
  • The runtime derives the TRACKED viewpoints with the same displayxr-common rig core that xrLocateViews runs, using the lifted rect as the screen. No math is duplicated.
• EXPLICIT viewpoints on lifted weave rects are now accepted, with the same validation and rect rebase as on a submit.
• XrLiftDepthRequestDXR (283), chained on XrLiftStreamCreateInfoDXR.
  • Valid on SBS / NVIEW streams only. On a DEPTH or GAUSSIANS stream it returns XR_ERROR_VALIDATION_FAILURE.
  • preferredFormat is a DXGI format hint.
• XrLiftDepthResultDXR (284), chained on XrLiftResultDXR. It carries:
  • depthValid.
  • depthTexture and fence, which follow the views' export rule: handed out on the first acquire that carries depth, then on each reallocation. It is the same fence object as the views', with the same fenceValue.
  • extent, format, units, encoding with valueScale / valueOffset, and nearDepth / farDepth.
  • sameInference and frameId.
  • Intrinsics: focalLengthPx and principalPointPx.
  • convergenceDepth and viewpoint.
  • depthToDisplay, a column-major 4×4 matrix.
• XrLiftDepthPropertiesDXR (285), chained on XrLiftPropertiesDXR: auxDepthSupported and auxDepthSemantics.

DP contract

All changes are append-only (ADR-020). There is no ABI bump.

• xrt_dp_lift_params gains viewpoint_source (TRACKED / EXPLICIT / DISPLAY_RIG / CAMERA_RIG) and nominal_z_m. Feature macro: XRT_DP_LIFT_HAS_APP_RIG.
• xrt_dp_lift_caps gains aux_outputs and aux_depth_semantics.
• xrt_dp_lift_stream_info gains aux_outputs and aux_depth_format.
• New struct xrt_dp_lift_depth (feature macro XRT_DP_LIFT_HAS_AUX_DEPTH). It carries:
  • The DP-owned R32F / R16F texture, valid until the next convert.
  • Units and encoding, with scale / offset.
  • Source size, focal length and principal point.
  • Near / far, convergence_depth, same_inference and vendor_frame_id.
• New D3D11 DP slot 33, lift_get_depth (XRT_DP_D3D11_HAS_LIFT_DEPTH).
  • Called right after a successful lift_convert of a stream that asked for depth.
  • I chose a new slot over an out-struct on lift_convert because that signature has shipped and is frozen. With a separate slot, an older plug-in simply has no slot, which reads as "no depth".

The math

• Display rig. The viewpoints are dxr_display3d_compute_views(...).eye_display, with the rect as the screen and m2v = 1. This equals the located eye_display / m2v.
• Camera rig. The viewpoint is E = Z0·(l·invd + ẑ), with Z0 = rect_h / (2·tan(vfov/2)).
  • This is the physical viewer whose Kooima frustum onto the rect equals the camera's off-axis frustum. The proof is in spec §4.2.
  • With convergenceDiopters 0 (convergence at infinity), all views collapse into one, so the lifted content has no parallax.
• Rig and the feat(lift): window-relative viewpoints + lift viewpoint policy (gains, axis mode, ease-back) #1864 policy.
  • The rig replaces the policy's ipd / parallax factors.
  • The axis mask, ease-back and clamp still run on the viewer before the rig maps it.
  • To get viewpoints identical to xrLocateViews, chain XrLiftViewControlDXR with XYZ and recentering OFF.
• depthToDisplay (spec §4.3):
  • X = R + C + (P(un,vn) − C)·d/dc, where C is the viewpoint midpoint and P is the texel's position on the rect.
  • A texel at the convergence depth lands on the screen plane. As d → 0, it lands at the viewpoint.
  • The map is linear in (un·d, vn·d, d, 1), so it is a single matrix.
  • It is only reported for METRIC depth with a known convergence depth. RELATIVE depth reports neither intrinsics nor a transform.

Implementation notes

• New pure helpers. u_lift_rig.{h,c} maps the rig; aux_util now links displayxr::math_core. u_lift_depth.{h,c} builds the depth transform.
• Rig lifetime. The rig rides with the options, like EXPLICIT viewpoints. Options sent without a rig clear it; a submit with no options keeps the stream's last rig.
• Depth path.
  • The depth is copied into the same out ring slot as the views, as a second keyed-mutex texture.
  • From there it goes to a per-stream depth export texture, signalled by the views' fence.
  • IPC: a new lift_get_depth_output call. lift_stream_create carries the aux request, and ipc_lift_result carries xrt_lift_depth_info.
• Weave-rect EXPLICIT viewpoints over IPC.
  • 8 rects' worth would overflow IPC_BUF_SIZE (1024). So each rect's viewpoints are staged with a new lift_weave_rect_viewpoints call just before lift_weave_rects.
  • Static asserts pin the lift message sizes.
  • Client and service must ship as a matched pair, as for every IPC change.
• sim_display fake lift. It returns a synthetic metric ramp (1–2 m) through lift_get_depth with SIM_DISPLAY_FAKE_LIFT=1, so the whole path can run without hardware.

Logging

• WARN once per change: [lift] stream N app display rig: ipd=… parallax=… perspective=…, … app camera rig: …, and … app rig cleared.
• WARN once per stream:
  • [lift] stream N auxiliary depth: WxH fmt=F units=metric|relative encoding=… same_inference=… convergence=… intrinsics=yes|no transform=yes|no
  • [lift] stream N depth export texture WxH fmt=F ready
  • [lift] stream N asked for auxiliary depth but the module returned none[ (the plug-in has no lift_get_depth slot)]
• The stream-create log line now includes aux=.

Verified on the build box

• Build. scripts\build_windows.bat generate + build in the worktree is clean, with no new warnings.
• Lift tests. ctest -C Release -R lift passes: tests_lift_mailbox, tests_aux_lift_viewpoint, and the new tests_aux_lift_rig_depth (120 assertions). The new test checks:
  • Display rig: viewpoints equal the located eye_display / m2v after window rebase, with the same frusta.
  • Camera rig: frusta equal the camera3d core's.
  • Infinite convergence collapses the views.
  • The convergence depth lands on the rect, and d → 0 lands at the viewpoint.
  • Rays stay collinear; invalid inputs are rejected; decode is correct.
• Full ctest. 100/103 pass. The 3 failures are tests_oxr_view_space_mode_floor, _mode_pinned and _mode_floor_off.
  • They fail at first_unfillable_mode against this box's registered display plug-in.
  • This PR touches no view-space or mode code.
  • I did not re-run them on the base branch, so I have not confirmed they also fail there.
• Dev CLI. _package\bin\displayxr-cli.exe selftest PASSED, including lift_caps against the installed released vendor plug-in. So older plug-in struct sizes still work with the new contract.
• Doc checks. gen_adr_index.py --check, gen_extensions_index.py --check and check_doc_paths.py all pass.
• Not run. No service, app or panel tests (shared-box rules).

Owed hardware tests

These need the service restarted with this build. Test 3 also needs the vendor plug-in built against this contract (P2).

1. sim fake depth. Start the service with SIM_DISPLAY_FAKE_LIFT=1 and the sim plug-in. Create an SBS stream with XrLiftDepthRequestDXR, submit frames, and acquire with XrLiftDepthResultDXR chained.
   • Expected result fields: depthValid true, R32F at input size, METRIC / LINEAR, valueOffset 1, intrinsics valid, transformValid true, and the same fenceValue as the views.
   • Expected log: [lift] stream N auxiliary depth: WxH fmt=41 units=metric … transform=yes.
   • A texel mapped through depthToDisplay at d = convergenceDepth (1.5 m with AUTO) should land on the rect.
2. Rig on a lifted rect. Chain an XrDisplayRigDXR on a weave-rect lift, for example ipd 0.5 and perspective 1.5.
   • Expected log: [lift] stream N app display rig: ipd=0.50 … perspective=1.50.
   • The XrLiftResultViewpointsDXR echo should equal the app's xrLocateViews eye_display / m2v (chain XYZ + OFF to get an exact match).
   • Then swap in an XrCameraRigDXR and change convergenceDiopters live. The lifted parallax should change in step with the app's own 3D (eyeball on the panel with a mixed 3D + lifted-video app).
3. Vendor metric depth (after P2). With the metric model active, XrLiftDepthPropertiesDXR should report supported + METRIC. On a synthetic scene of known depth, the convergence plane should land at z = 0.
4. EXPLICIT viewpoints on a weave rect. They should be accepted, with no VALIDATION_FAILURE. The INFO line [lift] stream N viewpoints: … axis=3 should show the rebased explicit eyes.

🤖 Generated with Claude Code

[dfattal]

The three tests_oxr_view_space_mode_{floor,pinned,floor_off} failures from the full ctest run also fail on the base branch (#1864 head edf61ecdf, same box) at first_unfillable_mode. So this PR didn't cause them. They depend on whichever display plug-in this box has registered.