Skip to content

feat(lift): window-relative viewpoints + lift viewpoint policy (gains, axis mode, ease-back) - #1864

Merged
dfattal merged 4 commits into
mainfrom
feat/lift-viewpoint-policy
Oct 8, 2026
Merged

dfattal merged 4 commits into
mainfrom
feat/lift-viewpoint-policy

Conversation

@dfattal

@dfattal dfattal commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Part of #1858, closes #1859.

Phase 1 runtime items R1–R4 of the lift camera-control design: lifted 2D content gets window-relative viewpoints, and the runtime (not the vendor plug-in) owns the viewpoint policy (display-rig ipd/parallax factors, axis mode, offset clamp, ease-back). New decision record: ADR-048.

What changes

API (XR_DXR_lift spec v1 → 2) — first commit, pushed early so the vendor plug-in can build against it:

  • XrLiftViewControlDXR (XR_TYPE_LIFT_VIEW_CONTROL_DXR = 1004999281): ipdFactor, parallaxFactor, axisMode (X / XY / XYZ), maxOffsetMeters, recenterMode (OFF / EASE_BACK), recenterHoldSeconds, recenterTimeConstantSeconds. Chains on XrLiftOptionsDXR or directly on XrWeaveRectLiftDXR (direct wins); sticky per stream; TRACKED viewpoints only.
  • XrLiftResultViewpointsDXR (1004999282), chained on XrLiftResultDXR: echoes the viewpoints a result was synthesized for, plus the region's centre and size, so apps can render matching 3D.
  • Allocation registry rows added (281/282 in lift's second decade).

DP contract (xrt_dp_lift.h, append-only per ADR-020, no ABI bump, feature macro XRT_DP_LIFT_HAS_VIEWPOINT_POLICY): xrt_dp_lift_params gains rect_width_m, rect_height_m, baseline_m, axis_mode, max_offset_m, viewpoint_frame (DISPLAY = panel centre, the meaning of a short struct_size; RECT = lifted rect centre). The plug-in only translates units: normalise by baseline_m, honour every component it is sent.

R1 rebase. The lift thread rebases every viewpoint to the centre of the lifted region: the weave rect on the weave client's window (rect px → window fraction → the window's metres, via comp_d3d11_service_get_client_app_window_metrics), or for xrSubmitLiftFrameDXR the session's window; a connection with no window (displayxr-cli lift) keeps the panel centre. EXPLICIT viewpoints are rebased the same way and otherwise untouched.

R3 policy pipeline on the lift thread (resolve_viewpoints in d3d11_lift.cpp, before lift_convert): rebase → ipd factor (rig step 1a) → parallax factor toward the nominal viewer (step 1b, nominal z from the system info, default 0.5 m) → axis mask → per-stream recenter filter → x/y clamp. The math is a pure-C module, src/xrt/auxiliary/util/u_lift_viewpoint.{h,c}, with Catch2 tests (tests/tests_aux_lift_viewpoint.cpp): rebase, sanitize, rig factors, axis mask + clamp, hold-then-decay with tau, frame-rate independence (30 vs 120 Hz), sub-threshold offsets never start the hold, temporary look-around after a re-settle, reset, OFF.

  • EASE_BACK (David's words): the camera eases back to (0,0,0), the scene camera origin — if you move your head you see look-around temporarily, then after the hold the camera returns to centre with no look-around; any new head motion gives temporary look-around again. (An "ease to the rect axis" variant was dropped: it lost look-around until the viewer moved back.)
  • Recenter filter: frame-rate independent (1 - exp(-dt/tau), dt capped at 0.25 s); one state per stream; reset on stream create, when no eyes are known, and whenever the eye tracker reports no lock.
  • IPC: ipc_lift_params / ipc_lift_weave_rect carry the view control; ipc_lift_result carries the echo (all well inside IPC_BUF_SIZE; client and service ship as a matched pair, as for every IPC change).

R4 docs. XR_DXR_lift.md §4.1 (frame + policy + echo + what the DP is told), §11 contract row, §14 version history, header table → v2; ADR-048 "Lift viewpoint policy"; xrt_plugin_iface.md lift section; index.json summary; ADR index regenerated (gen_adr_index.py --check OK, check_doc_paths.py OK, gen_extensions_index.py --check OK).

Default behaviour changes (please read)

Existing lifted content — the browser's lifted inline video/images — changes without opting in:

  1. The viewpoint is centred on the lifted rect / window, not the panel. This is the fix.
  2. EASE_BACK is on by default (X axis, 1 s hold, 2 s time constant): moving the head gives temporary look-around; after holding for a second the camera glides back to the scene camera origin over ~2 s; a new head movement looks around again immediately.

For A/B, set DXR_LIFT_RECENTER=off in the service's environment (read once when the lift module is created; logs [lift] DXR_LIFT_RECENTER=off: viewpoint recentering forced OFF for every stream). The rebase has no switch (it is a correctness fix).

Note: the axis mask now pins z to the nominal viewing distance in X / XY mode (before, raw z passed through to the plug-in, which discarded it).

Logging

  • WARN once per stream per policy change: [lift] stream N viewpoint policy: ipd=… parallax=… axis=… max_offset=… recenter=ease-back|off hold=… tau=… (rect-relative viewpoints, ADR-048).
  • INFO, inside the existing 5 s per-stream stats throttle: [lift] stream N viewpoints: rect centre=(x, y) m size=WxH m mid=(x, y, z) baseline=B m axis=A recenter=ease-back|off.

Verified on the build box

  • scripts\build_windows.bat generate + build in the worktree: clean.
  • ctest -R tests_aux_lift_viewpoint and tests_lift_mailbox: pass.
  • No service / app runs (shared box rules).

Owed hardware tests (need the service restarted with this build + the matching plug-in)

  1. Browser lifted video in a corner window. Put a browser window with a lifted inline video in a panel corner, sit facing that window. Expected: the lifted picture is straight-on (no skew as if seen from the panel centre). Log: [lift] stream N viewpoints: rect centre=(±0.1x, ±0.0y) m size=… — the centre tracks the rect, mid x ≈ 0 when you face it. Expect recenter=ease-back.
  2. Ease-back. Lean ~10 cm sideways and hold. Expected: immediate look-around, then after ~1 s the view glides back to straight-on over ~2 s; moving again looks around immediately. INFO mid= x decays toward 0 while held.
  3. A/B. Relaunch the service with DXR_LIFT_RECENTER=off: the WARN above appears once at lift-module creation; holding off-axis keeps the skewed view.
  4. Tracking loss. Leave the camera's view and return: no jump to an old reference (filter resets).
  5. Echo. An app chaining XrLiftResultViewpointsDXR gets viewpointCountOutput 2 with x separated by baseline_m around the window-relative midpoint.

🤖 Generated with Claude Code

@dfattal
dfattal requested a review from a team as a code owner October 8, 2026 07:53
dfattal added a commit to DisplayXR/displayxr-leia-plugin that referenced this pull request Oct 8, 2026
…inal unit, y/z, per-stream knobs)

Plug-in half of lift Phase 1 (DisplayXR/displayxr-runtime#1858, #316 P1;
runtime side DisplayXR/displayxr-runtime#1864). The runtime now owns the
viewpoint policy (rebase to the lifted rect, ipd/parallax factors, axis mask,
clamp, recentering — runtime ADR-048); the plug-in only translates units.

- leia_lift_viewpoint.{h,c}: pure mapping, host-tested
  (tests/test_lift_viewpoint.c, Linux CI). Legacy contract unchanged (x only,
  63 mm, +-3). Policy: the unit stays the NOMINAL 63 mm (NeurD's +-0.5 pair)
  so the runtime's ipd factor shows through (a 32 mm pair -> +-0.254);
  honour y on axis XY/XYZ (YGain); map z on XYZ as the fractional distance
  change from the panel's nominal viewing distance (ZGain, default 0.5,
  clamped +-1); map the runtime's midpoint clamp to
  +-ViewGain*(max_offset + baseline/2)/0.063 so the full eye pair survives at
  the edge. baseline_m is used only for that margin and the log.
- D3D11 DP reads the appended xrt_dp_lift_params fields with
  LEIA_LIFT_COVERS behind XRT_DP_LIFT_HAS_VIEWPOINT_POLICY; a short
  struct_size or older runtime headers keep today's behaviour exactly.
- Per-convert knobs (ViewGain, YGain, ZGain, ConvGain, DepthGain, Dilate,
  Scale) are re-read and snapshotted per stream at stream create.
- docs/lift-neurd.md: mapping rewritten (stale y claim fixed), z assumption,
  knob table + lifetimes, Verifying modes=0x3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dfattal and others added 4 commits October 8, 2026 02:31
…relative DP viewpoint contract

API surface for the lift viewpoint policy (ADR-048, #1859), committed first so
the vendor plug-in can build against it:

- XR_DXR_lift SPEC_VERSION 2: XrLiftViewControlDXR (281; ipdFactor,
  parallaxFactor, axisMode X/XY/XYZ, maxOffsetMeters, recenterMode
  OFF/EASE_TO_CURRENT/EASE_TO_NEUTRAL, hold + time constant), chained on
  XrLiftOptionsDXR or XrWeaveRectLiftDXR; XrLiftResultViewpointsDXR (282)
  echoes the viewpoints a result was synthesized for.
- xrt_dp_lift_params: appended rect_width_m, rect_height_m, baseline_m,
  axis_mode, max_offset_m, viewpoint_frame (DISPLAY / RECT), announced by
  XRT_DP_LIFT_HAS_VIEWPOINT_POLICY (append-only per ADR-020, no ABI bump).
- Allocation registry rows for 281/282; lift_convert doc names the frame.

Part of #1858.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…on the lift thread

R1 + R3 of the lift viewpoint policy (ADR-048, #1859):

- R1: every lift viewpoint is rebased to the centre of the lifted region —
  the weave rect on the weave client's window, or for xrSubmitLiftFrameDXR
  the session's window (panel centre when there is none) — using the
  service's existing client window metrics. The DP gets viewpoint_frame
  RECT plus the region's size in metres.
- R3: TRACKED viewpoints run through the stream's policy before
  lift_convert: ipd factor (rig step 1a), parallax factor toward the
  nominal viewer (1b), axis mask (X default), per-stream recenter filter
  (EASE_TO_CURRENT default: 1 s hold, 2 s time constant; EASE_TO_NEUTRAL;
  OFF), x/y clamp. baseline_m / axis_mode / max_offset_m filled for the DP.
  Filter is frame-rate independent and resets on stream create and on
  tracking loss. DXR_LIFT_RECENTER=off forces recentering off (A/B).
- XrLiftViewControlDXR parsed from XrLiftOptionsDXR and XrWeaveRectLiftDXR,
  carried over IPC, sticky per stream; XrLiftResultViewpointsDXR echoes the
  viewpoints, rect centre and size on every acquire.
- Pure-C math in u_lift_viewpoint.{h,c} with Catch2 tests
  (tests_aux_lift_viewpoint).

Default behaviour changes for existing lifted content: rect-centred
viewpoints and the ease-to-current recentering.

Part of #1858.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
R4 of #1859: XR_DXR_lift.md §4.1 (viewpoint frame, XrLiftViewControlDXR,
recentering, echo, what the DP is told), contract row, version history and
header table (v2); ADR-048; plug-in interface reference for the appended
xrt_dp_lift_params fields; extensions index note; ADR index regenerated.

Part of #1858.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Design change (David, #1859): the camera eases back to the scene camera
origin — moving the head gives temporary look-around, then after the hold
the camera returns to centre with no look-around. That is the former
EASE_TO_CURRENT behaviour, now XR_LIFT_RECENTER_MODE_EASE_BACK_DXR (value 1,
still the default). EASE_TO_NEUTRAL (value 2) is removed outright: the
extension is unreleased and spec v2 is introduced by this same PR.

Internal enums, the u_lift_viewpoint filter (no gain path), unit tests,
log strings (recenter=ease-back/off), spec §4.1 and ADR-048 updated.
Defaults (EASE_BACK, 1 s hold, 2 s) and DXR_LIFT_RECENTER=off unchanged.

Part of #1858.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dfattal
dfattal force-pushed the feat/lift-viewpoint-policy branch from 461c989 to edf61ec Compare October 8, 2026 09:32
@dfattal
dfattal merged commit ba9be36 into main Oct 8, 2026
40 checks passed
@dfattal
dfattal deleted the feat/lift-viewpoint-policy branch October 8, 2026 22:39
dfattal added a commit to DisplayXR/displayxr-leia-plugin that referenced this pull request Oct 8, 2026
…inal unit, y/z, per-stream knobs)

Plug-in half of lift Phase 1 (DisplayXR/displayxr-runtime#1858, #316 P1;
runtime side DisplayXR/displayxr-runtime#1864). The runtime now owns the
viewpoint policy (rebase to the lifted rect, ipd/parallax factors, axis mask,
clamp, recentering — runtime ADR-048); the plug-in only translates units.

- leia_lift_viewpoint.{h,c}: pure mapping, host-tested
  (tests/test_lift_viewpoint.c, Linux CI). Legacy contract unchanged (x only,
  63 mm, +-3). Policy: the unit stays the NOMINAL 63 mm (NeurD's +-0.5 pair)
  so the runtime's ipd factor shows through (a 32 mm pair -> +-0.254);
  honour y on axis XY/XYZ (YGain); map z on XYZ as the fractional distance
  change from the panel's nominal viewing distance (ZGain, default 0.5,
  clamped +-1); map the runtime's midpoint clamp to
  +-ViewGain*(max_offset + baseline/2)/0.063 so the full eye pair survives at
  the edge. baseline_m is used only for that margin and the log.
- D3D11 DP reads the appended xrt_dp_lift_params fields with
  LEIA_LIFT_COVERS behind XRT_DP_LIFT_HAS_VIEWPOINT_POLICY; a short
  struct_size or older runtime headers keep today's behaviour exactly.
- Per-convert knobs (ViewGain, YGain, ZGain, ConvGain, DepthGain, Dilate,
  Scale) are re-read and snapshotted per stream at stream create.
- docs/lift-neurd.md: mapping rewritten (stale y claim fixed), z assumption,
  knob table + lifetimes, Verifying modes=0x3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dfattal added a commit to DisplayXR/displayxr-leia-plugin that referenced this pull request Oct 8, 2026
…inal unit, y/z, per-stream knobs)

Plug-in half of lift Phase 1 (DisplayXR/displayxr-runtime#1858, #316 P1;
runtime side DisplayXR/displayxr-runtime#1864). The runtime now owns the
viewpoint policy (rebase to the lifted rect, ipd/parallax factors, axis mask,
clamp, recentering — runtime ADR-048); the plug-in only translates units.

- leia_lift_viewpoint.{h,c}: pure mapping, host-tested
  (tests/test_lift_viewpoint.c, Linux CI). Legacy contract unchanged (x only,
  63 mm, +-3). Policy: the unit stays the NOMINAL 63 mm (NeurD's +-0.5 pair)
  so the runtime's ipd factor shows through (a 32 mm pair -> +-0.254);
  honour y on axis XY/XYZ (YGain); map z on XYZ as the fractional distance
  change from the panel's nominal viewing distance (ZGain, default 0.5,
  clamped +-1); map the runtime's midpoint clamp to
  +-ViewGain*(max_offset + baseline/2)/0.063 so the full eye pair survives at
  the edge. baseline_m is used only for that margin and the log.
- D3D11 DP reads the appended xrt_dp_lift_params fields with
  LEIA_LIFT_COVERS behind XRT_DP_LIFT_HAS_VIEWPOINT_POLICY; a short
  struct_size or older runtime headers keep today's behaviour exactly.
- Per-convert knobs (ViewGain, YGain, ZGain, ConvGain, DepthGain, Dilate,
  Scale) are re-read and snapshotted per stream at stream create.
- docs/lift-neurd.md: mapping rewritten (stale y claim fixed), z assumption,
  knob table + lifetimes, Verifying modes=0x3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dfattal added a commit to DisplayXR/displayxr-leia-plugin that referenced this pull request Oct 8, 2026
…eline, y/z, per-stream knobs) (#317)

* feat(lift): honour the runtime's viewpoint policy (rect-relative, nominal unit, y/z, per-stream knobs)

Plug-in half of lift Phase 1 (DisplayXR/displayxr-runtime#1858, #316 P1;
runtime side DisplayXR/displayxr-runtime#1864). The runtime now owns the
viewpoint policy (rebase to the lifted rect, ipd/parallax factors, axis mask,
clamp, recentering — runtime ADR-048); the plug-in only translates units.

- leia_lift_viewpoint.{h,c}: pure mapping, host-tested
  (tests/test_lift_viewpoint.c, Linux CI). Legacy contract unchanged (x only,
  63 mm, +-3). Policy: the unit stays the NOMINAL 63 mm (NeurD's +-0.5 pair)
  so the runtime's ipd factor shows through (a 32 mm pair -> +-0.254);
  honour y on axis XY/XYZ (YGain); map z on XYZ as the fractional distance
  change from the panel's nominal viewing distance (ZGain, default 0.5,
  clamped +-1); map the runtime's midpoint clamp to
  +-ViewGain*(max_offset + baseline/2)/0.063 so the full eye pair survives at
  the edge. baseline_m is used only for that margin and the log.
- D3D11 DP reads the appended xrt_dp_lift_params fields with
  LEIA_LIFT_COVERS behind XRT_DP_LIFT_HAS_VIEWPOINT_POLICY; a short
  struct_size or older runtime headers keep today's behaviour exactly.
- Per-convert knobs (ViewGain, YGain, ZGain, ConvGain, DepthGain, Dilate,
  Scale) are re-read and snapshotted per stream at stream create.
- docs/lift-neurd.md: mapping rewritten (stale y claim fixed), z assumption,
  knob table + lifetimes, Verifying modes=0x3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(lift): include <stddef.h> for NULL in leia_lift_viewpoint.c

glibc's <math.h> does not define NULL, so the host-run test_lift_viewpoint
target failed to compile on every Linux lane ('NULL' undeclared). MSVC's
headers happened to pull it in transitively.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
dfattal added a commit that referenced this pull request Oct 8, 2026
v2.31.0 shipped XRT_DP_LIFT_HAS_VIEWPOINT_POLICY (#1864) in xrt_dp_lift.h,
which no track listed, so the gate skipped the leia-plugin windows repin.
The plug-in's viewpoint-policy code (leia-plugin#317) then compiles out at
its v2.30.0 pin. Add the header to the default surface and the windows
track (lift is D3D11-only, so not linux), and widen the completeness test
to xrt_dp_*.h. Dry run: v2.30.0 -> v2.31.0 windows now => BUMP.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Lift: window-relative viewpoints + XrLiftViewControlDXR (gains, axis mode, clamp, ease-back) + ADR

1 participant