Repository navigation
feat(lift): honour the runtime's viewpoint policy (rect-relative, baseline, y/z, per-stream knobs) - #317
Merged
Conversation
dfattal
force-pushed
the
feat/lift-viewpoint-policy
branch
from
October 8, 2026 08:14
9c8f8b5 to
4e31e87
Compare
dfattal
force-pushed
the
feat/lift-viewpoint-policy
branch
from
October 8, 2026 22:41
4e31e87 to
1e259c6
Compare
…inal unit, y/z, per-stream knobs) Plug-in half of lift Phase 1 (DisplayXR/displayxr-runtime#1858, #316 P1; runtime side DisplayXR/displayxr-runtime#1864). The runtime now owns the viewpoint policy (rebase to the lifted rect, ipd/parallax factors, axis mask, clamp, recentering — runtime ADR-048); the plug-in only translates units. - leia_lift_viewpoint.{h,c}: pure mapping, host-tested (tests/test_lift_viewpoint.c, Linux CI). Legacy contract unchanged (x only, 63 mm, +-3). Policy: the unit stays the NOMINAL 63 mm (NeurD's +-0.5 pair) so the runtime's ipd factor shows through (a 32 mm pair -> +-0.254); honour y on axis XY/XYZ (YGain); map z on XYZ as the fractional distance change from the panel's nominal viewing distance (ZGain, default 0.5, clamped +-1); map the runtime's midpoint clamp to +-ViewGain*(max_offset + baseline/2)/0.063 so the full eye pair survives at the edge. baseline_m is used only for that margin and the log. - D3D11 DP reads the appended xrt_dp_lift_params fields with LEIA_LIFT_COVERS behind XRT_DP_LIFT_HAS_VIEWPOINT_POLICY; a short struct_size or older runtime headers keep today's behaviour exactly. - Per-convert knobs (ViewGain, YGain, ZGain, ConvGain, DepthGain, Dilate, Scale) are re-read and snapshotted per stream at stream create. - docs/lift-neurd.md: mapping rewritten (stale y claim fixed), z assumption, knob table + lifetimes, Verifying modes=0x3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
glibc's <math.h> does not define NULL, so the host-run test_lift_viewpoint
target failed to compile on every Linux lane ('NULL' undeclared). MSVC's
headers happened to pull it in transitively.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dfattal
force-pushed
the
feat/lift-viewpoint-policy
branch
from
October 8, 2026 22:51
1e259c6 to
911871b
Compare
dfattal
added a commit
that referenced
this pull request
Oct 8, 2026
glibc's <math.h> does not define NULL, so the host-run test_lift_depth target failed to compile on every Linux lane (same fix as #317's leia_lift_viewpoint.c). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dfattal
added a commit
that referenced
this pull request
Oct 9, 2026
glibc's <math.h> does not define NULL, so the host-run test_lift_depth target failed to compile on every Linux lane (same fix as #317's leia_lift_viewpoint.c). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dfattal
added a commit
that referenced
this pull request
Oct 9, 2026
glibc's <math.h> does not define NULL, so the host-run test_lift_depth target failed to compile on every Linux lane (same fix as #317's leia_lift_viewpoint.c). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dfattal
added a commit
that referenced
this pull request
Oct 12, 2026
glibc's <math.h> does not define NULL, so the host-run test_lift_depth target failed to compile on every Linux lane (same fix as #317's leia_lift_viewpoint.c). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dfattal
added a commit
that referenced
this pull request
Oct 12, 2026
…9) (#318) * feat(lift): metric depth export + NeurD off-axis camera (NeurD >= 0.4.9) Plug-in Phase 3 (P2) of the lift camera / metric depth design (DisplayXR/displayxr-runtime#1858, #316), plus adoption of NeurD's metric off-axis camera. Runtime contract: feat/lift-app-rig-and-depth (ADR-048 Addendum A). NeurD side: LeiaInc/media_sdk #559 (per-stream props + float depth) and #560 (convert_stream_dx_offaxis), both INTRODUCED_IN 0.4.9. - Per-stream props: with NeurD_set/get_stream_prop_*, inpaint, auto- convergence, gain, dilation and convergence are set per NeurD stream (CONVERGENCE always overridden so each stream keeps its own auto result); a refusal falls back to the global path once, with one WARN. - Depth export: caps report aux_outputs = AUX_DEPTH (METRIC under VideoModel=metric, else RELATIVE) when NeurD has get_stream_depth_dx on DirectML. Streams created with AUX_DEPTH set RETAIN_FLOAT_DEPTH; after each convert the R32F depth of the SAME inference is bridged to the caller's device like the SBS output (legacy-shared texture, same event-query drain) and reported by the new slot 33 lift_get_depth (XRT_DP_D3D11_HAS_LIFT_DEPTH): metric = metres, LINEAR scale 1 (#559's shader already divides S / raw); relative = raw 0..255, INVERSE scale 1/255. Intrinsics passed through (depth texels); convergence_depth = the depth at zero disparity: h0 = 0.5 - 5c (legacy renderer) or 1 - C/D (off-axis), metric S / (255 h0), relative 1/h0. - Off-axis camera: with convert_stream_dx_offaxis and the runtime's viewpoint policy (RECT frame, rect size), SBS views use NeurD's metric Kooima camera: screen = rect in metres, eyes = the runtime's rect-relative viewpoints (z = distance from the rect plane, as given; app-rig sources reproduced without gain), nominal eye = the runtime's nominal_z_m. Relief: relative model ReliefDepthM (0.08 m, matches the calibrated legacy look) at the requested / NeurD-auto convergence; metric model matches the metric depth gradient at the screen (D = N / h0), capped by MetricReliefMaxM (0.3 m). OffAxis=0 forces the legacy dimensionless path; a refused call falls back for that frame. - Both 0.4.9 features compile only when the fetched NeurD.h declares them (CMake detects each separately) and run only when the loaded NeurD's table has them, so NeurD 0.4.6-0.4.8 and the v0.4.6 header pin are unchanged. - leia_lift_depth.{h,c}: the pure math, host-tested on the Linux lane (tests/test_lift_depth.c). docs/lift-neurd.md: depth export, off-axis camera, knob table (OffAxis, ReliefDepthM, MetricReliefMaxM), gating. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(lift): include <stddef.h> for NULL in leia_lift_depth.c glibc's <math.h> does not define NULL, so the host-run test_lift_depth target failed to compile on every Linux lane (same fix as #317's leia_lift_viewpoint.c). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(lift): use NeurD_convert_stream_dx_ex (views + depth of one call) NeurD's 0.4.9 API was reworked in media_sdk #561 (supersedes #559/#560): the float depth now comes back from the same convert call as the views, and the metric off-axis camera is a camera option of that call. - Build gates: DXR_LEIA_NEURD_HAS_STREAM_PROPS (NeurD_set_stream_prop_1i) and DXR_LEIA_NEURD_HAS_CONVERT_EX (NeurD_convert_stream_dx_ex) replace _HAS_FLOAT_DEPTH / _HAS_OFFAXIS. Run time still gates on LEIA_NEURD_HAS. - A stream that needs depth (aux depth, or the metric off-axis scale) converts with want_depth: the OFFAXIS camera when it applies, else the dimensionless pattern (NeurD's default +/-0.5 pair when untracked). No RETAIN prop, no separate getter. - Off-axis failures still fall back to the legacy camera for that frame. A depth-only _ex failure falls back to the plain convert with no depth for that frame. - Builds against NeurD v0.4.6 headers compile all of this out, as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build(lift): pin NeurD headers to v0.4.9-DEV (NeurD_convert_stream_dx_ex) media_sdk#561 merged to dev as fd9590f and is tagged v0.4.9-DEV, the first tag whose NeurD.h declares per-stream props and NeurD_convert_stream_dx_ex, so this build compiles both features in. Move to v0.4.9 once that release tag exists; a version's -DEV / -QA / release tags carry the same headers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build(lift): pin NeurD headers to the v0.4.9 release; resolve rebase onto main NeurD v0.4.9 shipped to PROD on 2026-10-11 (media_sdk dev@20bf39e, the commit that passed QA as v0.4.9-QA). Headers are identical to v0.4.9-DEV. Also keep both sides of the leia_lift_neurd_caps conflict with #345 (module info + aux depth fields). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of DisplayXR/displayxr-runtime#1858, addresses #316 (P1).
Plug-in half of Phase 1 of the lift camera design (window-aware lookaround). The runtime now owns the viewpoint policy (rebase to the lifted rect, ipd/parallax factors, axis mask, clamp, recentering ease — runtime ADR-048); the plug-in only translates units.
Depends on DisplayXR/displayxr-runtime#1864 for the new fields (
XRT_DP_LIFT_HAS_VIEWPOINT_POLICY, branchfeat/lift-viewpoint-policy), but works without it: against runtime headers that lack the macro the new code compiles out, and against a runtime binary that passes a shortxrt_dp_lift_params::struct_sizethe behaviour is byte-for-byte today's (x only, 63 mm unit, ±3 clamp).What changes
src/drv_leia/leia_lift_viewpoint.{h,c}(platform-neutral C, no NeurD/SR/D3D):x_n = clamp(G·x/0.063, ±3),y_n = z_n = 0(old runtime, or eyes from the DP's own tracker).x_n = G·x/0.063;y_n = Gy·G·y/0.063whenaxis_mode ≥ XY;z_n = Gz·(z − z_ref)/z_ref(clamped ±1) whenaxis_mode == XYZ; x/y clamp±G·(max_offset_m + baseline_m/2)/0.063when the runtime clamped (baseline 0 → 63 mm), else ±3.baseline_mis used only for that clamp margin and the log.leia_display_processor_d3d11.cpp): reads the appended fields withLEIA_LIFT_COVERS(gated on the last field,viewpoint_frame),#ifdef XRT_DP_LIFT_HAS_VIEWPOINT_POLICY;z_ref= the panel's nominal viewing distance from the cached SR geometry (leiasr_geometry_get), i.e. the same value the runtime receives viaget_display_infoand pins z to; unknown → 0.5 m (the runtime's default). static_assert that the axis values match the runtime's.ViewGain, newYGain(default 1.0) andZGain(default 0.5, signed), plusConvGain/DepthGain/Dilate/Scaleare re-read (env > HKLM > default) and snapshotted at each stream create, so a registry change applies to the next stream without a service restart. Process-level knobs (enable, backend, versions, video model) unchanged. Stream-create WARN lists the snapshot.viewpoints = runtime policy (frame rect, axis X, baseline …)/legacy mapping (…)); one process-wide WARN the first time vertical look-around is honoured (vertical parallax used to cause jumps).docs/lift-neurd.md): viewpoint mapping rewritten (fixes the stale "y_n = clamp(G·y/0.063)" claim), z assumption documented, knob table gainsYGain/ZGain/DepthGain/Dilate/VideoModel, knob lifetimes, Verifyingmodes=0x3(was0x7; NVIEW is never offered).src/drv_leia/tests/test_lift_viewpoint.c, built + run by the Linux CI lane (same pattern astest_display_claims_win).Decisions worth a reviewer's eye
baseline_m: the runtime already applied ipdFactor to the eyes; dividing by the post-factor baseline would cancel it (every pair → ±0.5). A 32 mm pair maps to ±0.254·G.baseline_m/2off it, so the per-eye limit adds that half-pair (otherwise the pair collapses at the edge). A runtime clamp wider than ±3 is honoured.Verified
origin/main(policy compiled out).test_lift_viewpointpasses (MSVC/W4 /WXlocally; Linux CI runs it).Owed (hardware, panel + NeurD)
viewpoints = legacy mapping (… runtime predates the viewpoint policy)).viewpoints = runtime policy (frame rect, axis X, baseline ~6x mm …).honouring VERTICAL look-aroundlogged once; tuneYGainif needed.zsign/magnitude; tuneZGain.🤖 Generated with Claude Code