Skip to content

feat(XR_DXR_cursor_depth): v3 anchor modes - HYBRID default + 0.005 margin (ADR-046) - #1847

Draft
dfattal wants to merge 7 commits into
mainfrom
feat/cursor-depth-anchor
Draft

dfattal wants to merge 7 commits into
mainfrom
feat/cursor-depth-anchor

Conversation

@dfattal

@dfattal dfattal commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Why

Feedback from a tester on a real head-tracked panel, about the ADR-046 depth-aware cursor:

  1. The sprite sits on the cyclopean ray through the pointer's canvas point. Its image on the glass therefore ignores head motion: it has disparity but no motion parallax. Everything around it parallaxes, so the cursor reads as "at the glass". A world-fixed cursor reads correctly, but sits ~1–2 mm off the click point.
  2. The default margin, 0.03 eye baselines, floats the cursor ~1–2 cm off the content. 0.003–0.005 rests on it.

What

u_cursor_depth: u_cursor_depth_place_anchored() with three anchor modes. E is the cyclopean eye, S the canvas point under the pointer, f the display normal, e = dot(S−E, f) and t = 1/(1−d).

  • SCREEN: C = E + t·(S−E). This is the v1/v2 placement.
  • WORLD: C = S − f·(1−t)·e.
  • HYBRID (new default): while the pointer moves (|Δu| or |Δv| > 1e-4, first step, or a filter re-prime), use the SCREEN position and store that locate's line of sight {E₀, S₀, D₀ = e}. While it is still, C = E₀ + s·(S₀ − E₀) with s = 1 − (1−t)·e/D₀: the point on the anchor-time line at the current depth in front of the glass. Continuous on the stop frame (s = t); world-fixed under lateral head motion (parallax like content); a depth change slides C along the anchor-time line, so it stays on the click point. Same rule as web SDK inline3d 1.37.1.
  • Height and orientation are unchanged in every mode. Hit testing is unchanged.
  • u_cursor_depth_filter_will_prime() exposes the filter's snap condition, so the session re-anchors on exactly those steps.
  • The default margin goes from 0.03 to 0.005.

XR_DXR_cursor_depth spec v3: a new optional input struct, XrCursorDepthOptionsDXR (1004999323), chained on XrCursorDepthHintDXR::next. It carries anchorMode (HYBRID=0 / SCREEN=1 / WORLD=2) and margin (≤0 or non-finite means the default). No existing struct grows. The registry row in openxr_includes/openxr/README.md is updated (324–329 still held).

  • Without the struct, an app gets HYBRID and the 0.005 margin. This deliberately changes the default behaviour from v2, at the tester's request. Apps that want the old placement chain SCREEN with margin = 0.03. This is called out in the spec's version history and in ADR-046 §7.
  • The anchor state sits next to the filter in oxr_session and is dropped whenever the filter re-primes.
  • The zero-cost rule (§0) still holds: the options struct is looked up only behind the hint.

Docs: ADR-046 gets a new §7 (the conflict, the modes, and why HYBRID beats both pure modes, with evidence), a rewritten head-motion consequence bullet, and the new margin row. The spec gets §2.4, an updated runtime-behaviour section and usage snippet, and a v3 history row.

Reference app: cube_handle_metal_macos reads DISPLAYXR_CURSOR_DEPTH_ANCHOR=hybrid|screen|world. When the variable is unset, the app doesn't chain the struct, so the runtime default applies.

Evidence

Unit tests (tests_aux_cursor_depth): 23 cases and 388 assertions, all passing. The new tests pin:

  • HYBRID, off-axis viewer, content appears / changes depth under a still pointer (anchored at d = 0): at every depth C projects from E₀ onto S₀ (the click point) and has the requested disparity; WORLD does not stay on the click point; a later lateral head move keeps C fixed.
  • HYBRID, pointer still, head moving laterally: C is unchanged in locate space, so its image on the glass moves (SCREEN's doesn't); head toward the glass: C slides along the anchor line.
  • HYBRID, pointer moving: equals SCREEN, and the anchor stores the locate's line of sight.
  • Continuity on the stop frame, sub-epsilon UV noise counting as still, and a depth change under a still pointer equalling SCREEN for an unmoved head.
  • WORLD: C = S − f·(1−t)·e; disparity round-trips on- and off-axis; WORLD equals SCREEN on-axis.
  • Default margin 0.005; will_prime matches every case where the filter snaps.

The old "head motion does not move the cursor on the glass" test is now explicitly the SCREEN-mode property.

Mutation testing: 14 of 15 mutants caught: drop the D₀ ratio (s = t), E for E₀, current e for D₀, flipped lift sign in s, the earlier normal-lift (foot) rule, WORLD sign, HYBRID→SCREEN, WORLD→SCREEN, ignoring the re-prime, always "moved", lift = t·e, margin 0.03, exact-compare move test, will_prime ignoring a stale gap. S for S₀ survives: for a still pointer on a display-fixed canvas, S equals S₀, so it is an equivalent mutant.

Headless captures: macOS, sim_display SBS, cube_handle_metal_macos, DISPLAYXR_CURSOR_DEPTH_UV=0.5,0.5. Atlas 1512×1646, two tiles stacked. Disparity = cursor x in the right tile minus the left tile; the cube's nearest edge under the cursor is about −15 px. The scripted pointer never moves, so HYBRID anchors at d = 0 on frame 1 and the cursor then rises to the cube with the pointer still. sim_display's viewer is 0.10 m above the canvas centre.

anchor target d cursor disparity cursor y in tile (pointer at 411)
unset (HYBRID) −0.040 −16.6 px 411
hybrid −0.040 −16.6 px 411
screen −0.040 −16.6 px 411
world −0.040 −16.6 px 438
foot-anchored HYBRID (first push) −0.040 −16.6 px 438
before (0.03 margin, SCREEN) −0.066 −23.4 px —

HYBRID now stays on the click point like SCREEN; the first push's foot-anchored rule drifted 27 px, as WORLD does. In every capture the midpoint of the two cursor images is at u = 0.4995.

Not tested

  • Not tested on a head-tracked panel. sim_display has no tracking, so the parallax behaviour is pinned only by the unit tests.
  • No interactive mouse run, so HYBRID's re-anchoring on real pointer motion wasn't observed in an app.
  • Not tested over IPC/service or on non-Metal compositors. The placement code is shared state-tracker code, so it should behave the same there.
  • The depth-layer source (DISPLAYXR_CURSOR_DEPTH_SOURCE=layer) was not re-run with the new modes. It needs a -DXRT_FEATURE_OPENXR_LAYER_DEPTH=ON build.
  • The web SDK's parallel implementation was not cross-checked numerically against these tests.

🤖 Generated with Claude Code

dfattal and others added 5 commits October 7, 2026 02:11
…in (ADR-046)

A tester on a real head-tracked panel reported two problems with the v1/v2
placement:

1. The sprite sits on the cyclopean ray through the pointer's canvas point,
   so its image on the glass is head-motion invariant: disparity but no
   motion parallax. Everything else parallaxes on a head-tracked display, so
   the cursor reads as "at the glass". A world-fixed cursor reads correctly
   but drifts ~1-2 mm off the click point.
2. The 0.03 margin floats the cursor ~1-2 cm off the content; 0.003-0.005
   rests on it.

u_cursor_depth_place_anchored() adds three anchor modes:
- SCREEN: C = E + t(S - E) (the old behaviour, u_cursor_depth_place).
- WORLD:  C = S - f (1 - t) e.
- HYBRID: SCREEN while the pointer moves, storing the foot
  F = C + f (1 - t) e on the glass; while the pointer is still,
  C = F - f (1 - t) e with the current t and e. Re-anchors whenever the
  filter re-primes. Continuous at the stop frame.

u_cursor_depth_filter_will_prime() exposes the filter's snap condition so the
caller can re-anchor on exactly those steps. Default margin 0.03 -> 0.005.

Tests pin HYBRID world-fixed-while-still, HYBRID == SCREEN while moving,
stop-frame continuity, WORLD geometry + disparity round trip, the new margin,
and will_prime; the old head-motion test is now explicitly the SCREEN
property. 11/11 mutants (sign flips, mode swaps, dropped re-prime, move
epsilon, margin) are caught.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…default

New optional input struct XrCursorDepthOptionsDXR (1004999323), chained on
XrCursorDepthHintDXR::next: anchorMode (HYBRID = 0 default, SCREEN = 1,
WORLD = 2) and margin (<= 0 or non-finite = runtime default 0.005).

Absent struct = HYBRID + the default margin. This deliberately changes the
default behaviour vs v2 (tester-driven, ADR-046); apps wanting the v1/v2
placement chain SCREEN.

The session keeps the HYBRID anchor beside the disparity filter and drops it
whenever the filter re-primes. Zero-cost rule unchanged: the options struct is
only looked up behind the hint.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ADR-046 gains section 7 (the motion-parallax vs disparity conflict a tester
found on a real head-tracked panel, the three modes, and why HYBRID beats both
pure modes: exact aim while moving, correct parallax while still, drift only
while the pointer is still at |d| x head displacement). The head-motion
consequence bullet is rewritten accordingly; margin row -> 0.005.

Spec: section 2.4 XrCursorDepthOptionsDXR + mode table, runtime behaviour,
usage snippet, and a v3 history row calling out the deliberate default change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…creen|world

Chains XrCursorDepthOptionsDXR (spec v3) with the requested anchor mode on
the hint. Unset: the struct is not chained and the runtime default (HYBRID)
applies. Needs XR_DXR_cursor_depth v3; warns and ignores otherwise.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hange drift term

Captures per anchor mode (sim_display SBS, UV 0.5,0.5): cursor disparity
-15.4/-16.6/-16.6/-16.7 px (unset/hybrid/screen/world) vs -23.4 px with the
old 0.03 margin; the cube edge is ~-15 px.

The captures also show that HYBRID's still-pointer drift has a second term
beyond head motion: if the disparity changes under a still pointer, the
sprite rises along the normal, so its image moves by ~|d - d0| x the viewer's
lateral offset from the click point. A scripted pointer anchors on frame 1 at
d0 = 0, so headless HYBRID lands where WORLD does (27 px below the click point
with sim_display's 0.10 m-high nominal viewer). Documented in ADR section 7
and the spec mode table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dfattal and others added 2 commits October 7, 2026 02:19
The foot-anchored HYBRID lifted a still pointer's sprite along the display
normal, so a depth change under a still pointer moved it off the click point
by ~|d - d0| x the viewer's lateral offset (27 px headless with sim_display's
0.10 m-high viewer, anchored at d = 0 on frame 1).

Now a moving placement stores its line of sight {E0, S0, D0 = eyeToCanvas};
while still, C = E0 + s (S0 - E0) with s = 1 - (1 - t) eyeToCanvas / D0: the
point on that line at the current depth in front of the glass. Same geometry
gives s = t (exact continuity); a lateral head move at fixed t leaves C fixed
(parallax like content); a depth change slides C along the anchor-time line,
so it stays on the click point. WORLD and SCREEN unchanged. Matches web SDK
inline3d 1.37.1.

Tests: new off-axis "content appears under a still pointer" case (HYBRID
projects from E0 onto S0 at every depth, WORLD does not, and stays world-fixed
under a later lateral head move); depth-change and head-toward-glass checks
rewritten for the line rule; the anchor stores the locate's line of sight.
Mutants: 14/15 caught (drop D0 ratio, E for E0, current e for D0, lift sign,
normal-lift foot rule, plus the earlier set). S for S0 survives: for a still
pointer on a display-fixed canvas S == S0, so it is equivalent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… re-run evidence

ADR section 7 and spec section 2.4 describe the line-of-sight anchor and its
three properties (continuity, parallax, no drift from depth changes), record
why the foot-anchored draft was dropped, and replace the evidence table:
HYBRID (and the unset default) now sits on the click point like SCREEN
(y 411 in the tile), WORLD at 438; disparity -16.6 px in every mode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant