diff --git a/docs/adr/ADR-046-depth-aware-cursor.md b/docs/adr/ADR-046-depth-aware-cursor.md index 9dfc396da..4b1aec7b1 100644 --- a/docs/adr/ADR-046-depth-aware-cursor.md +++ b/docs/adr/ADR-046-depth-aware-cursor.md @@ -1,6 +1,6 @@ # ADR-046: Depth-aware cursor — opt-in only; the app knows the depth, the runtime places the cursor -**Status:** Proposed (2026-10-06) · Phase 1 implemented · Phase 3a implemented (Metal) · Amendment 1 (lifted content, 2026-10-10) accepted · Amendment 2 (one policy, runtime draws, app chooses the look, 2026-10-11) accepted · spec: +**Status:** Proposed (2026-10-06) · Phase 1 implemented · Phase 3a implemented (Metal) · spec v3 anchor modes (2026-10-07) · Amendment 1 (lifted content, 2026-10-10) accepted · Amendment 2 (one policy, runtime draws, app chooses the look, 2026-10-11) accepted · spec: [XR_DXR_cursor_depth.md](../specs/extensions/XR_DXR_cursor_depth.md) · sibling of [ADR-040](ADR-040-rear-depth-budget.md) (the same cue conflict, at the cursor instead of the desktop) @@ -94,8 +94,9 @@ everything from the views that call has just produced: - **E**, the cyclopean eye, is the midpoint of that pair. - A point's depth is **t**: its distance in front of E along the display normal, divided by S's distance. So t = 1 on the canvas, and t < 1 in front of it. -- The sprite goes at **C = E + t·(S − E)**, with height scaled by t. Because C stays on the - cyclopean ray, the cursor never slides sideways as it rises. +- The sprite's height is scaled by t, so its apparent size is constant. Where it goes + laterally is the anchor mode (§7). The default, HYBRID, puts it at **C = E + t·(S − E)** + while the pointer moves: on the cyclopean ray, so it never slides sideways as it rises. Because only located views are used, the result is correct for: @@ -120,7 +121,7 @@ The eye compares disparities, so the margin, clamp and slew rates are expressed | Parameter | Default | Why | |---|---|---| -| margin in front of content | 0.03 | about 2 mm crossed on screen at 65 mm IPD: reads as in front without looking detached | +| margin in front of content | 0.03 | about 2 mm crossed on screen at 65 mm IPD: reads as in front without looking detached. Spec v3 briefly lowered it to 0.005 after a head-tracked-panel test; Amendment 2 A.2 restores 0.03. Apps can override it (`XrCursorDepthOptionsDXR::margin`) | | clamp | [−0.6, +0.6] — **superseded by Amendment 2: [−0.6, 0], never behind the glass** | t ≥ 0.625: never more than about 3/8 of the way to the eye | | rise time constant | 30 ms | never lag behind content that comes forward, or the violation is visible | | sink time constant | 250 ms | no flicker when the footprint crosses an edge | @@ -154,7 +155,7 @@ checks, `DISPLAYXR_CURSOR_DEPTH_UV=u,v` scripts the cursor position. Atlas captu | Cursor UV | Over | Cursor disparity between views | Meaning | |---|---|---|---| -| 0.5, 0.5 | the cube | −23.4 px; the cube's nearest edge there is about −15 px | in front of the cube | +| 0.5, 0.5 | the cube | −23.4 px with the original 0.03 margin; the cube's nearest edge there is about −15 px (for spec v3's 0.005 margin, see §7) | in front of the cube | | 0.5, 0.65 | the floor grid, which the app doesn't report as content | 0 px | on the display plane | | 0.1, 0.1 | empty space | 0 px | on the display plane | @@ -260,6 +261,82 @@ slightly nearer cube edge. A run without the request, in hint mode and with `DISPLAYXR_CURSOR_DEPTH=0`, logs none of the Phase 3a one-time WARNs. A run with the request logs each of them exactly once. +### 7. Spec v3: anchor modes, and why HYBRID is the default + +**What the tester saw.** On a real head-tracked panel, the v1/v2 cursor read as "at the glass" +even when its disparity put it on the content. The cause is a cue conflict of its own. The sprite +sat on the cyclopean ray through the pointer's canvas point (§2), so each eye's image of it on the +glass was S ∓ (baseline/2)·d whatever the head did. That is disparity with **no motion parallax**. +On a head-tracked display everything else parallaxes as the head moves, so a cursor that doesn't +reads as lying on the one surface that also doesn't: the glass. A world-fixed cursor read +correctly, but sat a millimetre or two beside the click point. + +**The modes** (`XrCursorDepthOptionsDXR::anchorMode`, a new struct chained on the hint; no +existing struct grows). With f the display normal away from the viewer and e = dot(S − E, f): + +| Mode | Sprite centre C | Aim | Parallax | +|---|---|---|---| +| SCREEN (v1/v2) | E + t·(S − E) | exact, always | none: reads as at the glass | +| WORLD | S − f·(1 − t)·e | ~1–2 mm off for an off-axis viewer, always | correct | +| **HYBRID (default)** | moving: SCREEN, and store the line of sight (E₀, S₀, D₀ = e); still: E₀ + s·(S₀ − E₀), s = 1 − (1 − t)·e / D₀ | exact while moving, and on the click point while still unless the head moves | correct while still | + +"Moving" means `cursorUV` changed by more than 1e-4 since the last moving placement. HYBRID +re-anchors on the first hinted locate and whenever the time filter re-primes; the anchor lives +beside the filter in the session and is touched only by a hinted locate, so §0 holds. + +**The anchor is the line of sight at the last pointer move, not a point.** While the pointer is +still, C is the point on the anchor-time line E₀ → S₀ whose distance in front of the glass, +(1 − s)·D₀, equals the current depth (1 − t)·e. Three properties follow, all unit-tested: + +- **Continuity:** on the frame the pointer stops, the geometry is the anchor's, so s = t and C is + the last moving placement exactly. No jump. +- **Parallax:** a lateral head move at fixed t leaves e, and so s and C, unchanged. The sprite is + world-fixed and parallaxes like content at its depth. +- **No drift from depth changes:** when the content under a still pointer changes depth, C slides + along the anchor-time line, so from the eye that aimed it, it stays on the click point. + +An earlier draft of the rule stored the line's *foot* on the glass and lifted C from it along the +display normal. The headless captures below exposed the flaw: a depth change under a still +pointer then moved the sprite off the click point by about |Δd| × the viewer's lateral offset +(27 px for sim_display's viewer, 0.10 m above the canvas centre, when the scripted pointer +anchored at d = 0 on frame 1 and the cursor then rose to the cube). The web SDK takes the same +line-of-sight rule (inline3d 1.37.1). + +**Why HYBRID beats both pure modes.** While the pointer moves, the user is aiming, and the eye +tracks the cursor, not the head; there SCREEN's exact aim is what matters and a parallax deficit +over a few frames is invisible. While it is still, the user is looking at the scene, often moving +their head to see around it, and the cursor must parallax like the content it rests on; there +WORLD's behaviour is right. The only cost is that the drawn sprite drifts from the click point as +the **head** moves while the pointer is still: its image on the glass moves by about |d| × the +head's lateral displacement. For content near the glass (d ≈ −0.005 with the new margin), 20 cm of +head motion moves it about 1 mm. The next mouse movement re-aims it exactly. Hit testing is +unchanged: the click is always at the pointer's canvas point. + +**Evidence** (macOS, sim_display SBS, `cube_handle_metal_macos`, `DISPLAYXR_CURSOR_DEPTH_UV=0.5,0.5`, +atlas 1512×1646, two tiles of 1512×823 stacked; disparity = cursor x in the right-eye tile minus +the left-eye tile; the cube's nearest edge under the cursor is about −15 px). The scripted pointer +never moves, so HYBRID anchors on frame 1 at d = 0 and the cursor then rises to the cube with the +pointer still: the depth-change case. + +| `DISPLAYXR_CURSOR_DEPTH_ANCHOR` | Target d | Cursor disparity | Cursor y in tile (pointer at 411) | +|---|---|---|---| +| unset (options not chained → HYBRID) | −0.040 | −16.6 px | 411 | +| `hybrid` | −0.040 | −16.6 px | 411 | +| `screen` | −0.040 | −16.6 px | 411 | +| `world` | −0.040 | −16.6 px | 438 | +| *foot-anchored draft of HYBRID* | *−0.040* | *−16.6 px* | *438* | +| *before v3: 0.03 margin, SCREEN (§5)* | *−0.066* | *−23.4 px* | — | + +All sit just in front of the cube's edge, about 7 px nearer it than with the old margin. HYBRID +stays on the click point like SCREEN; WORLD sits 27 px below it for this off-axis viewer. In every +capture the midpoint of the two cursor images is at the requested u (0.4995). + +**A deliberate default change.** An app that doesn't chain the options struct gets HYBRID and +the 0.005 margin, which is different from v2. It is tester-driven, both changes go the same way +(the cursor reads as on the content), and an app that wants the old placement chains +`XR_CURSOR_DEPTH_ANCHOR_MODE_SCREEN_DXR` with `margin = 0.03`. The web SDK's depth cursor +implements the identical rule with the same numbers. + ## Consequences - An app that already raycasts or reads depth gets a correct, consistent cursor for one struct @@ -272,14 +349,20 @@ each of them exactly once. that to the edge itself. Phase 3a as built does not remove this lag. It reads frame N's submitted depth, but the result reaches the placement one or two frames later, because waiting on the GPU would stall the pipeline (§6). So it has the same lag, and the same mitigation. -- **Head motion and look-around.** The sprite sits on the cyclopean ray through the cursor's - canvas point. So each eye sees it on the glass at S ∓ (baseline/2)·d, which doesn't depend on - where the head is (unit-tested across three head poses). The cursor's image on the panel - therefore cannot swim with tracking motion or jitter. Only the content under the line of sight - changes as the user looks around, and the hit test is redone every frame. This is deliberate - SCREEN anchoring, not world anchoring. A mouse is a 2D screen-space device, so the cursor slides - over a surface under head motion instead of sticking to it. If it stuck to the surface, the - click target would drift on the glass as the head moved. +- **Head motion and look-around (revised in spec v3, §7).** v1/v2 anchored the sprite to the + screen: on the cyclopean ray, so each eye saw it on the glass at S ∓ (baseline/2)·d whatever + the head did (still unit-tested, now as the SCREEN-mode property). The reasoning was that a + mouse is a 2D screen-space device and its click target must not drift. Testing on a real + head-tracked panel showed the price: a sprite with disparity but no motion parallax conflicts + with everything around it that does parallax, and reads as at the glass. The default is now + HYBRID: screen-anchored while the pointer moves (exact aim), and anchored to that line of + sight while it is still (correct parallax, unit-tested: the sprite does not move in locate + space as the head moves laterally). + The click target never drifts, because hit testing stays at the pointer's canvas point. Only + the drawn sprite drifts, by |d| × head displacement (about 1 mm for 20 cm of head motion over + content near the glass), and only until the mouse next moves. A depth change under a still + pointer does not drift it: the anchor is the line of sight, not a point (§7). SCREEN and WORLD remain + available through `XrCursorDepthOptionsDXR`. - An app-drawn cursor shows the app's frame latency, not the hardware cursor's. Phase 2's late cursor read narrows that but cannot remove it. This is inherent to any cursor that has disparity. @@ -476,6 +559,20 @@ sprite and the depth, and the runtime owns the drawing. Apps get the same contra runtime as the sprite (B.1). Pages then style the depth cursor with plain CSS, and the SDK needs no cursor code for it. +#### §A as built in the runtime (#1847, `XR_DXR_cursor_depth` spec v4) + +- `u_cursor_depth_target()` is A.1 verbatim; the upper clamp is 0 whatever the tuning says + (`max_disparity` defaults to 0 and a value > 0 is treated as 0). Margin default 0.03 (A.2), + still overridable through `XrCursorDepthOptionsDXR::margin`. +- A.3 is `u_cursor_depth_should_draw()` (threshold −0.005). The state tracker reports it as + `XrCursorDepthPlacementDXR::isActive`, whose meaning was already "draw nothing, show the OS + cursor", so no struct changed. The filter and the HYBRID anchor step on every locate, drawn or + not, so the hand-back is smooth. `disparity` / `targetDisparity` stay filled as diagnostics. +- The 0.5 px floor is left to the drawing backend (Part B): the placement does not know the + panel's pixel pitch, and −0.005 baselines (~0.33 mm at 65 mm IPD) is above 0.5 px on every + current panel. +- A.5 (hybrid line-of-sight anchor) was already the v3 default and is unchanged. + ### Rollout 1. **Groundwork:** diff --git a/docs/specs/extensions/XR_DXR_cursor_depth.md b/docs/specs/extensions/XR_DXR_cursor_depth.md index 3d2c139c3..25af4d59b 100644 --- a/docs/specs/extensions/XR_DXR_cursor_depth.md +++ b/docs/specs/extensions/XR_DXR_cursor_depth.md @@ -3,8 +3,8 @@ | Property | Value | |----------|-------| | Extension Name | `XR_DXR_cursor_depth` | -| Spec Version | 2 | -| Type Values | `XR_TYPE_CURSOR_DEPTH_HINT_DXR` (1004999320) · `XR_TYPE_CURSOR_DEPTH_PLACEMENT_DXR` (1004999321) · `XR_TYPE_CURSOR_DEPTH_SOURCE_DXR` (1004999322, v2) | +| Spec Version | 4 | +| Type Values | `XR_TYPE_CURSOR_DEPTH_HINT_DXR` (1004999320) · `XR_TYPE_CURSOR_DEPTH_PLACEMENT_DXR` (1004999321) · `XR_TYPE_CURSOR_DEPTH_SOURCE_DXR` (1004999322, v2) · `XR_TYPE_CURSOR_DEPTH_OPTIONS_DXR` (1004999323, v3) | | Author | The DisplayXR Project | | Platform | All. Works on every graphics API, in-process and over IPC; solved in the state tracker. | | Decision record | [ADR-046](../../adr/ADR-046-depth-aware-cursor.md) | @@ -17,7 +17,8 @@ The OS cursor is drawn on the display plane, at zero disparity. Over content tha glass, that is a **depth violation**: occlusion says the cursor is in front, and disparity says it is behind. `XR_DXR_cursor_depth` lifts the cursor off the plane. The app reports the nearest content under the cursor. The runtime returns a pose and size for a cursor sprite just in front -of that content, smoothed over time. The app draws the sprite. +of that content, smoothed over time. The app draws the sprite. Content on or behind the plane +causes no conflict, so there the cursor stays the OS cursor (v4: never behind the glass). > The **app** owns the depth (it knows its scene). The **runtime** owns placement and policy > (where, how big, how it moves), so every app's cursor behaves the same. The **app** draws. @@ -62,11 +63,11 @@ sprite: the violation returns at every silhouette. | Field | Meaning | |---|---| -| `isActive` | `XR_FALSE` means draw nothing and show the OS cursor. This happens when no hint was chained, fewer than two views are active (2D), the cursor is off the canvas, or the view geometry is degenerate. | -| `pose` | The sprite centre and orientation in the locate space. The orientation is the display plane's: draw the quad in its local XY plane, facing +Z. The position lies on the line from the viewer through the cursor's point on the canvas, so the cursor never slides sideways as its depth changes. | +| `isActive` | `XR_FALSE` means draw nothing and show the OS cursor. This happens when no hint was chained, fewer than two views are active (2D), the cursor is off the canvas, or the view geometry is degenerate, and (spec v4) whenever the filtered disparity is above the draw threshold, −0.005: on the plane, the OS cursor is the cursor. `pose` is then identity and `height` 0. `disparity` and `targetDisparity` are still filled whenever the geometry was valid, as diagnostics. | +| `pose` | The sprite centre and orientation in the locate space. The orientation is the display plane's: draw the quad in its local XY plane, facing +Z. The position follows the anchor mode (§2.4). With the default (HYBRID) it lies on the line from the viewer through the cursor's point on the canvas while the pointer moves, and stays fixed in the locate space while the pointer is still. | | `height` | Sprite height in locate-space units, scaled with depth so the apparent size is constant. | -| `disparity` | The filtered disparity, in eye-baseline units: 0 is the display plane and < 0 is in front. Diagnostic only. | -| `targetDisparity` | The unfiltered target: the content disparity minus the margin, then clamped. Diagnostic only. | +| `disparity` | The filtered disparity, in eye-baseline units: 0 is the display plane and < 0 is in front. Never > 0 since v4. Diagnostic only. | +| `targetDisparity` | The unfiltered target: the content disparity minus the margin, clamped to [−0.6, 0]; 0 when the content is on or behind the plane. Diagnostic only. | ### 2.3 `XrCursorDepthSourceDXR` (input, spec v2) @@ -113,14 +114,52 @@ display plane, and the runtime logs a one-time warning. None is ever an error. - The far end of the mapping is the cleared background. It counts as no content. - The nearest texel in the footprint wins, across both outer views. +### 2.4 `XrCursorDepthOptionsDXR` (input, spec v3) + +Chained on `XrCursorDepthHintDXR::next`, in any order with `XrCursorDepthSourceDXR`. Optional: +**absent means HYBRID anchoring and the default margin.** It is read only behind the hint, so it +costs nothing on its own. + +| Field | Meaning | +|---|---| +| `anchorMode` | Where the sprite goes laterally as the viewer's head moves. `XR_CURSOR_DEPTH_ANCHOR_MODE_HYBRID_DXR` (0, the default), `_SCREEN_DXR` (1), `_WORLD_DXR` (2). An unknown value behaves as HYBRID. | +| `margin` | How far in front of the content the cursor floats, as a disparity in eye-baseline units. ≤ 0 or non-finite selects the runtime default, 0.03 (v3 used 0.005). | + +**Anchor modes.** E is the cyclopean eye, S the canvas point under the pointer, f the unit display +normal pointing away from the viewer, e = dot(S − E, f), d the filtered disparity and +t = 1/(1 − d). The sprite's centre C is: + +| Mode | C | On a head-tracked display | +|---|---|---| +| `SCREEN` | E + t·(S − E): on the cyclopean ray | The image on the glass ignores head motion. The cursor has disparity but **no motion parallax**, so while everything else parallaxes it reads as "at the glass". The v1/v2 behaviour. | +| `WORLD` | S − f·(1 − t)·e: straight in front of S along the normal | Parallaxes like content at its depth. Off-axis, it sits a millimetre or two beside the click point even while the pointer moves. | +| `HYBRID` | While the pointer **moves**: the SCREEN position, and the runtime stores that locate's line of sight: E₀ = E, S₀ = S, D₀ = e. While the pointer is **still**: C = E₀ + s·(S₀ − E₀) with s = 1 − (1 − t)·e / D₀, the point on the anchor-time line whose distance in front of the glass equals the current depth (1 − t)·e. | Exact aim while moving. While still: world-fixed under lateral head motion (correct parallax, s unchanged); a depth change slides C along the anchor-time line, so it stays on the click point for a viewer who hasn't moved; continuous on the frame the pointer stops (s = t). The only drift from the click point is from head motion while the pointer is still: about \|d\| × the head's lateral displacement, so 6 mm for 20 cm at d = −0.03 (content just in front of the glass). | + +"Moves" means `cursorUV` changed by more than 1e-4 in u or v since the last moving placement. +HYBRID also re-anchors (takes the SCREEN position) on the first hinted locate and whenever the +time filter re-primes (a gap > 0.5 s, or time going backwards). The height +(`cursorHeight` × canvas height × t) and the orientation are the same in every mode. Hit testing +is unchanged: the click stays at the pointer's canvas point; the anchor mode only changes where +the sprite is drawn. + +The same rule, with the same numbers, is implemented by the web SDK's depth cursor. + ## 3. Runtime behaviour The canvas point under the cursor is triangulated from the outermost pair of **located views**. Every policy number is applied in disparity d = 1 − 1/t, where t is depth relative to the canvas along the display normal measured from the cyclopean eye: -- margin: 0.03 baseline in front of the content; -- clamp: d ∈ [−0.6, 0.6]; +- **never behind the glass** (v4, ADR-046 Amendment 2 A.1): target = d < 0 ? max(d − margin, −0.6) : 0. + Content on or behind the display plane leaves the cursor on the plane; +- margin: 0.03 baseline in front of the content (overridable with `XrCursorDepthOptionsDXR::margin`); +- lateral anchor: HYBRID unless `XrCursorDepthOptionsDXR` selects another mode (§2.4); +- clamp: d ∈ [−0.6, 0]; +- draw threshold (v4, Amendment 2 A.3): `isActive` only while the filtered d ≤ −0.005 (about 0.3 mm + on screen at a 65 mm IPD). The filter keeps running while inactive, so the drawn cursor hands + back to the OS cursor smoothly and returns without a jump. The ADR's additional 0.5-panel-pixel + floor is a drawing-backend concern (the placement does not know the pixel pitch); −0.005 is + above 0.5 px on every current panel; - rising toward the viewer: 30 ms time constant; - sinking away: 250 ms time constant; - a gap longer than 0.5 s snaps the filter; @@ -145,6 +184,12 @@ locate.viewConfigurationType = viewConfig; locate.displayTime = frameState.predictedDisplayTime; locate.space = appSpace; +// Optional (v3): keep the v1/v2 screen-anchored placement instead of HYBRID. +// XrCursorDepthOptionsDXR opts = {XR_TYPE_CURSOR_DEPTH_OPTIONS_DXR}; +// opts.anchorMode = XR_CURSOR_DEPTH_ANCHOR_MODE_SCREEN_DXR; +// opts.margin = 0.0f; // runtime default +// hint.next = &opts; + XrCursorDepthPlacementDXR cursor = {XR_TYPE_CURSOR_DEPTH_PLACEMENT_DXR}; XrViewState viewState = {XR_TYPE_VIEW_STATE, &cursor}; xrLocateViews(session, &locate, &viewState, viewCount, &viewCount, views); @@ -178,3 +223,5 @@ For splats, use the renderer's alpha-weighted expected depth, not a raycast. |---|---| | 1 | Initial: app hint in, placement out, app draws (ADR-046 Phase 1). | | 2 | `XrCursorDepthSourceDXR` (1004999322): opt-in depth-layer source. The runtime reads the nearest content from the depth submitted with the projection layer (ADR-046 Phase 3a). Metal compositor first. | +| 3 | `XrCursorDepthOptionsDXR` (1004999323): anchor mode (HYBRID / SCREEN / WORLD) and margin. HYBRID anchors the line of sight at the last pointer move (§2.4). **Defaults change, deliberately:** the default anchor is now HYBRID (was effectively SCREEN), and the default margin is 0.005 (was 0.03). Both follow testing on a real head-tracked panel: the screen-anchored cursor had disparity but no motion parallax and read as "at the glass", and 0.03 floated it a visibly detached 1–2 cm off the content, where 0.003–0.005 rests on it. An app that wants the v2 placement chains the options struct with `XR_CURSOR_DEPTH_ANCHOR_MODE_SCREEN_DXR` and `margin = 0.03`. | +| 4 | No new structs; **behaviour changes** (ADR-046 Amendment 2 A, one cursor policy everywhere). **Never behind the glass:** target = d < 0 ? max(d − margin, −0.6) : 0, so content on or behind the display plane leaves the cursor on the plane (v1–v3 let it sink onto content behind the glass, up to +0.6). **Margin back to 0.03** (the reference app's value since Phase 1). **Draw threshold:** `isActive` is now also `XR_FALSE` while the filtered disparity is above −0.005, which is the existing meaning of `isActive = XR_FALSE` (draw nothing, show the OS cursor); `disparity` / `targetDisparity` stay filled as diagnostics. HYBRID stays the default anchor. | diff --git a/src/external/openxr_includes/openxr/README.md b/src/external/openxr_includes/openxr/README.md index 18cec30db..c6de545a3 100644 --- a/src/external/openxr_includes/openxr/README.md +++ b/src/external/openxr_includes/openxr/README.md @@ -51,7 +51,7 @@ Rules: | 1004999290–299 | `XR_DXR_weave` (v14+ additions) | 290 = `XR_TYPE_WEAVE_SUBMIT_OVERLAY_UNCHANGED_DXR` (the v4 overlay atlas is unchanged since the previous submit, so a DP may reuse its lens prefilter of the 2D layer, spec v14, ADR-027 Amendment); 291 = `XR_TYPE_WEAVE_SUBMIT_OVERLAY_FILTER_DXR` (the caller's lens-filter strength for the 2D layer, 0..1, omitted = the DP's default, spec v15); 292 = `XR_TYPE_WEAVE_OUTPUT_RECT_PARTS_DXR` (per-screen segments of the bound window with per-screen eyes, and each weave rect split into per-screen parts, spec v19, #1884). The third weave decade (240–249 is full) | | 1004999300–309 | `XR_DXR_stereo_camera` | 300 = `XR_TYPE_STEREO_CAMERA_PROPERTIES_DXR`, 301 = `XR_TYPE_STEREO_CAMERA_CALIBRATION_DXR`, 302 = `XR_TYPE_STEREO_CAMERA_STREAM_CREATE_INFO_DXR`, 303 = `XR_TYPE_STEREO_CAMERA_FRAME_DXR`, 304 = `XR_TYPE_STEREO_CAMERA_STREAM_INFO_DXR`, 305 = `XR_TYPE_STEREO_CAMERA_STREAM_TRANSPORT_DXR`, 306 = `XR_TYPE_STEREO_CAMERA_STREAM_STATS_DXR`, 307 = `XR_TYPE_EVENT_DATA_STEREO_CAMERA_STATE_CHANGED_DXR`, 308 = `XR_TYPE_EVENT_DATA_STEREO_CAMERAS_CHANGED_DXR` (spec v1, ADR-043). 309 = `XR_OBJECT_TYPE_STEREO_CAMERA_STREAM_DXR` — an `XrObjectType`, not an `XrStructureType`. **Relocated from 290–299** on the rebase onto post-lift `main`: 290 had meanwhile been taken by `XR_TYPE_WEAVE_SUBMIT_OVERLAY_UNCHANGED_DXR` (weave spec v14). Safe to move: spec v1, draft PR, no shipped app chains it | | 1004999310–319 | `XR_DXR_stereo_camera` (continued) | 310 = `XR_STEREO_CAMERA_FRAME_NOT_READY_DXR` — a **success**-class `XrResult` (positive). A second decade because 300–309 is full. Spec v2 (R3, ADR-043 §7): 311 = `XR_TYPE_STEREO_CAMERA_CLIENT_INFO_DXR` (camera-only client declaration on `XrInstanceCreateInfo`); 312 = `XR_ERROR_STEREO_CAMERA_CONSENT_REFUSED_DXR`, 313 = `XR_ERROR_STEREO_CAMERA_DISABLED_DXR`, 314 = `XR_ERROR_STEREO_CAMERA_BUSY_DXR`, 316 = `XR_ERROR_STEREO_CAMERA_STREAM_ENDED_DXR` — **error**-class `XrResult`s (negated in the header); 315 = `XR_TYPE_EVENT_DATA_STEREO_CAMERA_STREAM_ENDED_DXR` | -| 1004999320–329 | `XR_DXR_cursor_depth` | 320 = `XR_TYPE_CURSOR_DEPTH_HINT_DXR`, 321 = `XR_TYPE_CURSOR_DEPTH_PLACEMENT_DXR` (spec v1, ADR-046). 322 = `XR_TYPE_CURSOR_DEPTH_SOURCE_DXR` (spec v2, Phase 3a depth-layer source). 323–329 held for the Phase 2/3b draw + measure requests | +| 1004999320–329 | `XR_DXR_cursor_depth` | 320 = `XR_TYPE_CURSOR_DEPTH_HINT_DXR`, 321 = `XR_TYPE_CURSOR_DEPTH_PLACEMENT_DXR` (spec v1, ADR-046). 322 = `XR_TYPE_CURSOR_DEPTH_SOURCE_DXR` (spec v2, Phase 3a depth-layer source). 323 = `XR_TYPE_CURSOR_DEPTH_OPTIONS_DXR` (spec v3, anchor mode + margin). 324–329 held for the Phase 2/3b draw + measure requests | | 1004999330–339 | `XR_DXR_cocoa_window_binding` (v8+ additions) | 330 = `XR_TYPE_COCOA_WINDOW_PLACEMENT_INFO_DXR` (spec v8, ADR-050 — app opt-out of the runtime-owned macOS window drag / resize). A fresh decade: the extension's original value 1004999004 sits in the shared low block | | 1004999340+ | **next free** | | diff --git a/src/external/openxr_includes/openxr/XR_DXR_cursor_depth.h b/src/external/openxr_includes/openxr/XR_DXR_cursor_depth.h index 09c73b129..343cb9000 100644 --- a/src/external/openxr_includes/openxr/XR_DXR_cursor_depth.h +++ b/src/external/openxr_includes/openxr/XR_DXR_cursor_depth.h @@ -48,6 +48,24 @@ * sends no nearest point; the runtime reads a cursor-sized patch of that depth * itself (ADR-046 Phase 3a). * + * Spec v3 adds @ref XrCursorDepthOptionsDXR (anchor mode + margin, both + * optional) and CHANGES THE DEFAULTS, deliberately, after testing on a real + * head-tracked panel: the default anchor is now HYBRID (the sprite follows the + * pointer's line of sight while the pointer moves, and is world-fixed while it + * is still, so it motion-parallaxes like the content it rests on) and the + * default margin was 0.005 (v4 restores 0.03). An app that wants the v1/v2 + * placement chains the options struct with XR_CURSOR_DEPTH_ANCHOR_MODE_SCREEN_DXR. + * + * Spec v4 (ADR-046 Amendment 2 A, one cursor policy everywhere) changes + * behaviour, not structs: + * - NEVER BEHIND THE GLASS: target = d < 0 ? max(d - margin, -0.6) : 0. + * Content on or behind the display plane leaves the cursor on the plane. + * - default margin 0.03 again; + * - isActive is XR_TRUE only while the FILTERED disparity is at least 0.005 + * eye baselines in front of the plane. On the plane the OS cursor is the + * cursor: draw nothing and show it. The filter keeps running while + * inactive, so the drawn cursor hands back and returns smoothly. + * * One call, one space. The hint is chained on XrViewLocateInfo::next and the * result on XrViewState::next of the SAME xrLocateViews call, and both are in * XrViewLocateInfo::space. The hint normally comes from the previous frame's @@ -70,7 +88,7 @@ extern "C" { #endif #define XR_DXR_cursor_depth 1 -#define XR_DXR_cursor_depth_SPEC_VERSION 2 +#define XR_DXR_cursor_depth_SPEC_VERSION 4 #define XR_DXR_CURSOR_DEPTH_EXTENSION_NAME "XR_DXR_cursor_depth" // Reserved 1004999320-329 (next free decade after stereo_camera's 310-319). @@ -79,6 +97,8 @@ extern "C" { #define XR_TYPE_CURSOR_DEPTH_PLACEMENT_DXR ((XrStructureType)1004999321) //! Spec v2 (ADR-046 Phase 3a). #define XR_TYPE_CURSOR_DEPTH_SOURCE_DXR ((XrStructureType)1004999322) +//! Spec v3 (anchor mode + margin). +#define XR_TYPE_CURSOR_DEPTH_OPTIONS_DXR ((XrStructureType)1004999323) // ---- Input: app chains this on XrViewLocateInfo::next; runtime reads it. ---- @@ -167,6 +187,61 @@ typedef struct XrCursorDepthSourceDXR { XrCursorDepthSourceKindDXR source; } XrCursorDepthSourceDXR; +// ---- Spec v3: anchor mode + margin. ---- + +/*! + * @brief Where the sprite goes laterally as the viewer's head moves. + * + * E is the cyclopean eye, S the canvas point under the pointer, f the display + * normal (away from the viewer), t the sprite's depth as a fraction of E's + * distance to the canvas (t = 1 on the canvas, < 1 in front of it). + */ +typedef enum XrCursorDepthAnchorModeDXR { + /*! + * The default. While the pointer moves: SCREEN placement (exact aim). + * While it is still: on the line of sight of the last pointer move + * (anchor-time eye -> canvas point), at the current depth - so lateral + * head motion leaves it world-fixed and it parallaxes exactly like + * content at that depth, while a depth change slides it along that line + * and keeps it on the click point. Continuous where the pointer stops. + * Drift from the click point comes only from head motion while the + * pointer is still, and is ~mm for content near the glass. + */ + XR_CURSOR_DEPTH_ANCHOR_MODE_HYBRID_DXR = 0, + /*! + * Spec v1/v2 behaviour: on the cyclopean ray E -> S. Its image on the + * glass ignores head motion - disparity, but no motion parallax, which on + * a head-tracked display reads as "at the glass". + */ + XR_CURSOR_DEPTH_ANCHOR_MODE_SCREEN_DXR = 1, + /*! + * Straight in front of S along the display normal, at the same depth: + * always parallaxes correctly, but sits off the click point (~1-2 mm for + * an off-axis viewer) even while the pointer moves. + */ + XR_CURSOR_DEPTH_ANCHOR_MODE_WORLD_DXR = 2, + XR_CURSOR_DEPTH_ANCHOR_MODE_MAX_ENUM_DXR = 0x7FFFFFFF +} XrCursorDepthAnchorModeDXR; + +/*! + * @brief Optional, chained on XrCursorDepthHintDXR::next (spec v3). + * + * Absent = HYBRID anchoring and the runtime's default margin. Like the hint, + * it is read only on a locate that chains the hint. + */ +typedef struct XrCursorDepthOptionsDXR { + XrStructureType type; //!< Must be XR_TYPE_CURSOR_DEPTH_OPTIONS_DXR + const void* XR_MAY_ALIAS next; + //! Unknown values behave as XR_CURSOR_DEPTH_ANCHOR_MODE_HYBRID_DXR. + XrCursorDepthAnchorModeDXR anchorMode; + /*! + * How far in front of the content the cursor floats, as a disparity in + * eye-baseline units (> 0). <= 0 or non-finite selects the runtime + * default (0.03; spec v3 used 0.005). + */ + float margin; +} XrCursorDepthOptionsDXR; + // ---- Result: app chains this on XrViewState::next; runtime fills it. ---- /*! @@ -182,16 +257,22 @@ typedef struct XrCursorDepthPlacementDXR { void* XR_MAY_ALIAS next; /*! * XR_FALSE when there is nothing to draw: no hint chained on this locate, - * fewer than two active views (2D), a cursor off the canvas, or degenerate - * view geometry. Every other field is then zero / identity. + * fewer than two active views (2D), a cursor off the canvas, degenerate + * view geometry, or (spec v4) a filtered disparity above the draw + * threshold (-0.005: the cursor is on, or within 0.005 baselines of, the + * display plane - the OS cursor is the cursor). pose is then identity and + * height 0; disparity and targetDisparity are still filled whenever the + * geometry was valid (diagnostics). */ XrBool32 isActive; /*! * Sprite centre and orientation in XrViewLocateInfo::space. The * orientation is the display plane's: draw the sprite in its local XY - * plane, +X right, +Y up, facing +Z (toward the viewer). The position lies - * on the line from the viewer through the cursor's point on the canvas, - * so the cursor never appears to slide sideways as its depth changes. + * plane, +X right, +Y up, facing +Z (toward the viewer). The position + * follows the anchor mode (XrCursorDepthAnchorModeDXR): with SCREEN, and + * with HYBRID while the pointer moves, it lies on the line from the + * viewer through the cursor's point on the canvas, so the cursor never + * appears to slide sideways as its depth changes. */ XrPosef pose; /*! @@ -201,11 +282,12 @@ typedef struct XrCursorDepthPlacementDXR { float height; /*! * Filtered disparity, in units of the eye baseline: 0 = on the display - * plane, < 0 = in front of it (crossed), > 0 = behind. Diagnostic; the - * pose already encodes it. + * plane, < 0 = in front of it (crossed). Never > 0 since spec v4 (never + * behind the glass). Diagnostic; the pose already encodes it. */ float disparity; - //! Unfiltered target disparity this locate (content + margin, clamped). Diagnostic. + //! Unfiltered target disparity this locate: content - margin, clamped to + //! [-0.6, 0]; 0 when the content is on or behind the plane. Diagnostic. float targetDisparity; } XrCursorDepthPlacementDXR; diff --git a/src/xrt/auxiliary/util/u_cursor_depth.c b/src/xrt/auxiliary/util/u_cursor_depth.c index 1f3c3ef6d..8b604213e 100644 --- a/src/xrt/auxiliary/util/u_cursor_depth.c +++ b/src/xrt/auxiliary/util/u_cursor_depth.c @@ -66,12 +66,21 @@ view_ray(const struct u_cursor_depth_view *view, float u, float v) void u_cursor_depth_tuning_defaults(struct u_cursor_depth_tuning *t) { - // 3% of the baseline is ~2 mm of crossed on-screen disparity for a 65 mm - // IPD: enough to read as "in front", small enough not to look detached. - t->margin = 0.03f; + // 3% of the baseline: about 2 mm of crossed on-screen disparity at a 65 mm + // IPD - reads as in front without looking detached. Spec v3 briefly used + // 0.005; ADR-046 Amendment 2 A.2 restores 0.03, the value the reference + // app (cube_handle_metal_macos) has shown since Phase 1. + t->margin = U_CURSOR_DEPTH_DEFAULT_MARGIN; // d = -0.6 is t = 0.625: the cursor may come ~3/8 of the way to the eye. t->min_disparity = -0.6f; - t->max_disparity = 0.6f; + // Never behind the glass (Amendment 2 A.1): the cursor is on the canvas + // or in front of it. u_cursor_depth_target() also caps any configured + // value at 0, so this cannot be raised. + t->max_disparity = 0.0f; + // Amendment 2 A.3: nothing is drawn (the OS cursor is the cursor) until + // the filtered disparity is at least 0.5% of the baseline in front of + // the canvas (~0.3 mm on screen at a 65 mm IPD). + t->draw_threshold = U_CURSOR_DEPTH_DRAW_THRESHOLD; t->rise_tau_s = 0.03f; t->sink_tau_s = 0.25f; t->stale_s = 0.5f; @@ -155,23 +164,51 @@ u_cursor_depth_target(const struct u_cursor_depth_tuning *t, bool has_content, f if (!has_content || !isfinite(content_disparity)) { return 0.0f; } + // Never behind the glass (ADR-046 Amendment 2 A.1): content on or behind + // the canvas leaves the cursor ON the canvas; only content that comes out + // of the screen pushes it forward, margin in front of it. + if (!(content_disparity < 0.0f)) { + return 0.0f; + } float d = content_disparity - t->margin; if (d < t->min_disparity) { d = t->min_disparity; } - if (d > t->max_disparity) { - d = t->max_disparity; + // The upper clamp is [.., 0] whatever the tuning says: a max_disparity > 0 + // (or a margin <= 0 letting d reach above the content) can never put the + // cursor behind the glass. + const float max_d = (t->max_disparity < 0.0f) ? t->max_disparity : 0.0f; + if (d > max_d) { + d = max_d; } return d; } +bool +u_cursor_depth_should_draw(const struct u_cursor_depth_tuning *t, float filtered_disparity) +{ + // A threshold at or behind the canvas would draw a sprite ON the glass, + // where the OS cursor is the cursor (A.3): fall back to the default. + const float thr = (t->draw_threshold < 0.0f && isfinite(t->draw_threshold)) ? t->draw_threshold + : U_CURSOR_DEPTH_DRAW_THRESHOLD; + return isfinite(filtered_disparity) && filtered_disparity <= thr; +} + +bool +u_cursor_depth_filter_will_prime(const struct u_cursor_depth_filter *f, + const struct u_cursor_depth_tuning *t, + uint64_t now_ns) +{ + return !f->primed || now_ns < f->last_ns || (double)(now_ns - f->last_ns) * 1e-9 > (double)t->stale_s; +} + float u_cursor_depth_filter_step(struct u_cursor_depth_filter *f, const struct u_cursor_depth_tuning *t, float target, uint64_t now_ns) { - if (!f->primed || now_ns < f->last_ns || (double)(now_ns - f->last_ns) * 1e-9 > (double)t->stale_s) { + if (u_cursor_depth_filter_will_prime(f, t, now_ns)) { f->primed = true; f->disparity = target; f->last_ns = now_ns; @@ -205,6 +242,60 @@ u_cursor_depth_place(const struct u_cursor_depth_geometry *g, *out_height = height_fraction * g->canvas_height * t; } +void +u_cursor_depth_place_anchored(const struct u_cursor_depth_geometry *g, + float disparity, + float height_fraction, + enum u_cursor_depth_anchor_mode mode, + struct u_cursor_depth_anchor *anchor, + float u, + float v, + bool reprimed, + uint64_t now_ns, + struct xrt_vec3 *out_position, + float *out_height) +{ + // SCREEN placement and the height, shared by every mode. + u_cursor_depth_place(g, disparity, height_fraction, out_position, out_height); + if (mode == U_CURSOR_DEPTH_ANCHOR_SCREEN) { + return; + } + + const float t = 1.0f / (1.0f - disparity); + // How far C sits in front of the canvas plane, along the normal. + const float lift = (1.0f - t) * g->eye_to_canvas; + + if (mode == U_CURSOR_DEPTH_ANCHOR_WORLD || anchor == NULL) { + if (mode == U_CURSOR_DEPTH_ANCHOR_WORLD) { + *out_position = v3_add_scaled(g->canvas_point, g->forward, -lift); + } + return; // HYBRID without state degrades to SCREEN + } + + // HYBRID. + const bool moved = + !anchor->has_anchor || reprimed || fabsf(u - anchor->last_u) > U_CURSOR_DEPTH_ANCHOR_MOVE_EPS || + fabsf(v - anchor->last_v) > U_CURSOR_DEPTH_ANCHOR_MOVE_EPS || !(anchor->eye_to_canvas > 0.0f); + if (moved) { + // Screen placement (already in *out_position); remember this line of sight. + anchor->eye = g->eye; + anchor->canvas_point = g->canvas_point; + anchor->eye_to_canvas = g->eye_to_canvas; + anchor->last_u = u; + anchor->last_v = v; + anchor->last_ns = now_ns; + anchor->has_anchor = true; + return; + } + // Pointer still: the point on the anchor-time line of sight E0 -> S0 whose + // distance in front of the glass is the CURRENT depth, lift. Same geometry + // gives s = t (continuous at the stop); a lateral head move leaves s and so + // C unchanged (world-fixed, parallax like content); a depth change slides C + // along that line, so from E0 it stays on S0 (the click point). + const float s = 1.0f - lift / anchor->eye_to_canvas; + *out_position = v3_add_scaled(anchor->eye, v3_sub(anchor->canvas_point, anchor->eye), s); +} + /* * diff --git a/src/xrt/auxiliary/util/u_cursor_depth.h b/src/xrt/auxiliary/util/u_cursor_depth.h index 42a65aaed..909490858 100644 --- a/src/xrt/auxiliary/util/u_cursor_depth.h +++ b/src/xrt/auxiliary/util/u_cursor_depth.h @@ -21,9 +21,27 @@ * (on-screen disparity = baseline * d): 0 on the canvas, < 0 in front. * Disparity, not distance, is what the eye compares, so margins, clamps * and slew rates are expressed there; - * - the sprite goes at C = E + t (S - E) — on the cyclopean ray, so it never - * slides sideways as it rises — with height scaled by t so its apparent - * size is constant. + * - ONE POLICY (ADR-046 Amendment 2 A): the cursor is never behind the + * glass. target = d < 0 ? max(d - margin, d_min) : 0, margin 0.03, and + * nothing is drawn (the OS cursor is the cursor) while the filtered + * disparity is above the draw threshold, -0.005 + * (u_cursor_depth_should_draw). The filter keeps stepping while nothing + * is drawn, so the drawn cursor hands back smoothly; + * - the sprite's height is scaled by t so its apparent size is constant; + * - WHERE laterally it goes is the anchor mode (spec v3): + * SCREEN C = E + t (S - E): on the cyclopean ray. Its image on the glass + * ignores head motion - disparity but no motion parallax, which + * on a head-tracked display reads as "at the glass". + * WORLD C = S - f (1 - t) e: straight in front of the pointer's canvas + * point along the normal (e = eye-to-canvas). Parallaxes like + * content, but sits ~mm off the click point off-axis. + * HYBRID (default) SCREEN while the pointer moves - exact aim - storing + * that locate's line of sight (E0, S0, e0). While the pointer is + * still: C = E0 + s (S0 - E0), s = 1 - (1 - t) e / e0 - the point + * on the anchor-time line at the current depth in front of the + * glass. Continuous at the stop; a lateral head move keeps C + * world-fixed (parallax); a depth change slides C along the + * anchor-time line, so it stays on the click point. * * Pure C: time is a parameter, never read here. * @@ -45,6 +63,19 @@ extern "C" { //! Default sprite height, as a fraction of the canvas height. #define U_CURSOR_DEPTH_DEFAULT_HEIGHT 0.03f +//! Default margin in front of content, eye-baseline units (Amendment 2 A.2). +#define U_CURSOR_DEPTH_DEFAULT_MARGIN 0.03f + +/*! + * Default draw threshold, eye-baseline units (Amendment 2 A.3): a sprite is + * drawn only while the filtered disparity is at or in front of this. + * + * The ADR also floors it at 0.5 px on the panel. The placement does not know + * the panel's pixel pitch, so that floor is the drawing backend's to apply; + * at a 65 mm IPD, -0.005 is ~0.33 mm, above 0.5 px on every panel shipped. + */ +#define U_CURSOR_DEPTH_DRAW_THRESHOLD (-0.005f) + /*! * One located view: the pose and fov exactly as xrLocateViews returns them. */ @@ -76,8 +107,12 @@ struct u_cursor_depth_tuning float margin; //! Most crossed disparity the cursor may reach (comfort clamp, < 0). float min_disparity; - //! Most uncrossed disparity the cursor may reach (> 0). + //! Upper clamp, <= 0. Default 0 (the canvas): never behind the glass. + //! Values > 0 are treated as 0 (Amendment 2 A.1). float max_disparity; + //! Draw only while the filtered disparity is <= this (< 0; default + //! U_CURSOR_DEPTH_DRAW_THRESHOLD; >= 0 or non-finite = the default). + float draw_threshold; //! Time constant when rising toward the viewer, seconds (fast: never lag behind content). float rise_tau_s; //! Time constant when sinking away from the viewer, seconds (slow: no flicker at edges). @@ -96,6 +131,41 @@ struct u_cursor_depth_filter uint64_t last_ns; }; +/*! + * Where the sprite goes laterally (XR_DXR_cursor_depth spec v3). The values + * equal the XrCursorDepthAnchorModeDXR ones. + */ +enum u_cursor_depth_anchor_mode +{ + //! SCREEN while the pointer moves, world-fixed while it is still. Default. + U_CURSOR_DEPTH_ANCHOR_HYBRID = 0, + //! On the cyclopean ray through the pointer's canvas point (spec v1/v2). + U_CURSOR_DEPTH_ANCHOR_SCREEN = 1, + //! In front of the pointer's canvas point along the display normal. + U_CURSOR_DEPTH_ANCHOR_WORLD = 2, +}; + +/*! + * HYBRID anchor state. Zero-initialise; reset whenever the filter re-primes. + */ +struct u_cursor_depth_anchor +{ + bool has_anchor; + //! E0: the cyclopean eye of the last moving placement. + struct xrt_vec3 eye; + //! S0: the canvas point under the pointer at the last moving placement. + struct xrt_vec3 canvas_point; + //! e0: E0's distance to the canvas along the normal then (> 0). + float eye_to_canvas; + //! Pointer UV of that placement. + float last_u, last_v; + //! Time of that placement. + uint64_t last_ns; +}; + +//! Pointer UV change beyond which HYBRID treats the pointer as moving. +#define U_CURSOR_DEPTH_ANCHOR_MOVE_EPS 1e-4f + void u_cursor_depth_tuning_defaults(struct u_cursor_depth_tuning *t); @@ -122,12 +192,33 @@ bool u_cursor_depth_point_disparity(const struct u_cursor_depth_geometry *g, const struct xrt_vec3 *p, float *out_disparity); /*! - * The unfiltered target: the content's disparity minus the margin, clamped; - * 0 (the canvas) when there is no content under the cursor. + * The unfiltered target (ADR-046 Amendment 2 A.1): + * content_disparity < 0 ? max(content_disparity - margin, min_disparity) : 0, + * and never above 0 for any tuning. 0 (the canvas) when there is no content + * under the cursor, or the content is on or behind the canvas. */ float u_cursor_depth_target(const struct u_cursor_depth_tuning *t, bool has_content, float content_disparity); +/*! + * Amendment 2 A.3: should a sprite be drawn at this FILTERED disparity? + * True only at or in front of the draw threshold. While false the OS cursor + * is the cursor; keep stepping the filter regardless so the hand-back is + * smooth. + */ +bool +u_cursor_depth_should_draw(const struct u_cursor_depth_tuning *t, float filtered_disparity); + +/*! + * True if the next u_cursor_depth_filter_step() at @p now_ns will re-prime + * (snap) rather than smooth: first step, clock going backwards, or a gap + * longer than t->stale_s. HYBRID drops its anchor on exactly these steps. + */ +bool +u_cursor_depth_filter_will_prime(const struct u_cursor_depth_filter *f, + const struct u_cursor_depth_tuning *t, + uint64_t now_ns); + /*! * Advance the filter toward @p target and return the new disparity. Calls with * a time not after the previous one do not advance it (a second locate in the @@ -141,7 +232,7 @@ u_cursor_depth_filter_step(struct u_cursor_depth_filter *f, /*! * Where to draw a sprite at @p disparity, and how tall, for a requested height - * of @p height_fraction of the canvas. + * of @p height_fraction of the canvas. SCREEN anchoring (on the cyclopean ray). */ void u_cursor_depth_place(const struct u_cursor_depth_geometry *g, @@ -150,6 +241,29 @@ u_cursor_depth_place(const struct u_cursor_depth_geometry *g, struct xrt_vec3 *out_position, float *out_height); +/*! + * u_cursor_depth_place() with an anchor mode (spec v3). + * + * @param anchor HYBRID state; may be NULL for SCREEN and WORLD (untouched). + * @param u, v the pointer UV this placement is for. + * @param reprimed the filter re-primed on this step (HYBRID re-anchors). + * @param now_ns the step's time (recorded in the anchor). + * + * Height is the same in every mode. An unknown mode behaves as HYBRID. + */ +void +u_cursor_depth_place_anchored(const struct u_cursor_depth_geometry *g, + float disparity, + float height_fraction, + enum u_cursor_depth_anchor_mode mode, + struct u_cursor_depth_anchor *anchor, + float u, + float v, + bool reprimed, + uint64_t now_ns, + struct xrt_vec3 *out_position, + float *out_height); + /* * diff --git a/src/xrt/state_trackers/oxr/oxr_objects.h b/src/xrt/state_trackers/oxr/oxr_objects.h index 8c29e9698..7eb6c3f48 100644 --- a/src/xrt/state_trackers/oxr/oxr_objects.h +++ b/src/xrt/state_trackers/oxr/oxr_objects.h @@ -3103,6 +3103,10 @@ struct oxr_session //! only by a locate that chains an XrCursorDepthHintDXR - a session that //! never sends a hint never steps it. struct u_cursor_depth_filter cursor_depth_filter; + //! XR_DXR_cursor_depth v3: HYBRID anchor state, beside the filter and + //! under the same rule (only a hinted locate touches it). Dropped + //! whenever the filter re-primes. + struct u_cursor_depth_anchor cursor_depth_anchor; /*! * XR_DXR_cursor_depth v2, depth-layer source (ADR-046 Phase 3a). diff --git a/src/xrt/state_trackers/oxr/oxr_session.c b/src/xrt/state_trackers/oxr/oxr_session.c index b46ea1643..27f28f95d 100644 --- a/src/xrt/state_trackers/oxr/oxr_session.c +++ b/src/xrt/state_trackers/oxr/oxr_session.c @@ -2465,6 +2465,22 @@ oxr_session_fill_cursor_depth(struct oxr_session *sess, struct u_cursor_depth_tuning tuning; u_cursor_depth_tuning_defaults(&tuning); + // Spec v3: optional anchor mode + margin, read only behind the hint. + // Absent = HYBRID + the default margin. + enum u_cursor_depth_anchor_mode anchor_mode = U_CURSOR_DEPTH_ANCHOR_HYBRID; + const XrCursorDepthOptionsDXR *options = + OXR_GET_INPUT_FROM_CHAIN(hint, XR_TYPE_CURSOR_DEPTH_OPTIONS_DXR, XrCursorDepthOptionsDXR); + if (options != NULL) { + if (options->anchorMode == XR_CURSOR_DEPTH_ANCHOR_MODE_SCREEN_DXR) { + anchor_mode = U_CURSOR_DEPTH_ANCHOR_SCREEN; + } else if (options->anchorMode == XR_CURSOR_DEPTH_ANCHOR_MODE_WORLD_DXR) { + anchor_mode = U_CURSOR_DEPTH_ANCHOR_WORLD; + } + if (options->margin > 0.0f && isfinite(options->margin)) { + tuning.margin = options->margin; + } + } + float content = 0.0f; bool has_content; // Spec v2: the source struct is looked up ONLY here, behind the hint - @@ -2479,19 +2495,33 @@ oxr_session_fill_cursor_depth(struct oxr_session *sess, has_content = hint->hasContent && u_cursor_depth_point_disparity(&g, &nearest, &content); } const float target = u_cursor_depth_target(&tuning, has_content, content); - const float disparity = u_cursor_depth_filter_step(&sess->cursor_depth_filter, &tuning, target, - (uint64_t)viewLocateInfo->displayTime); + const uint64_t now_ns = (uint64_t)viewLocateInfo->displayTime; + const bool reprimed = u_cursor_depth_filter_will_prime(&sess->cursor_depth_filter, &tuning, now_ns); + if (reprimed) { + sess->cursor_depth_anchor.has_anchor = false; + } + const float disparity = u_cursor_depth_filter_step(&sess->cursor_depth_filter, &tuning, target, now_ns); struct xrt_vec3 position; float height; - u_cursor_depth_place(&g, disparity, hint->cursorHeight, &position, &height); + u_cursor_depth_place_anchored(&g, disparity, hint->cursorHeight, anchor_mode, &sess->cursor_depth_anchor, + hint->cursorUV.x, hint->cursorUV.y, reprimed, now_ns, &position, &height); + + // Diagnostics are filled whether or not a sprite is drawn (spec v4). + out->disparity = disparity; + out->targetDisparity = target; + // Amendment 2 A.3: on the plane it is the OS cursor. isActive (= "draw + // a sprite, hide the OS cursor") only while the FILTERED disparity is in + // front of the draw threshold. The filter and the anchor above stepped + // regardless, so the drawn cursor hands back and returns smoothly. + if (!u_cursor_depth_should_draw(&tuning, disparity)) { + return; + } out->isActive = XR_TRUE; out->pose.orientation = (XrQuaternionf){g.orientation.x, g.orientation.y, g.orientation.z, g.orientation.w}; out->pose.position = (XrVector3f){position.x, position.y, position.z}; out->height = height; - out->disparity = disparity; - out->targetDisparity = target; } #endif // OXR_HAVE_DXR_cursor_depth diff --git a/test_apps/handle/cube_handle_metal_macos/main.mm b/test_apps/handle/cube_handle_metal_macos/main.mm index ea647a3a1..cf6cdea78 100644 --- a/test_apps/handle/cube_handle_metal_macos/main.mm +++ b/test_apps/handle/cube_handle_metal_macos/main.mm @@ -897,7 +897,12 @@ static void EnsureDepthTexture(MetalRenderer &r, uint32_t w, uint32_t h) // FOOTPRINT (rays from the outer eyes through a ring of points around the // hotspot) and hands the nearest hit to xrLocateViews. The runtime returns // where to draw the cursor sprite; the app draws it last, depth test off, and -// hides the OS cursor while it does. Opt out with DISPLAYXR_CURSOR_DEPTH=0. +// hides the OS cursor while it does. Since spec v4 (ADR-046 Amendment 2) that +// is only while content pops out of the screen under the cursor: over the +// floor, empty space or anything on/behind the glass, isActive is false and the +// OS cursor is the cursor. Opt out with DISPLAYXR_CURSOR_DEPTH=0; +// pick the anchor mode with DISPLAYXR_CURSOR_DEPTH_ANCHOR=hybrid|screen|world +// (unset = the runtime default, HYBRID). // ============================================================================ static const float kCubeSize = 0.06f; @@ -1701,6 +1706,12 @@ static void UpdateCameraMovement(InputState& state, float deltaTime, float displ // submits depth (XR_KHR_composition_layer_depth) and lets the runtime // find the nearest content under the cursor, instead of the hit test. bool cursorDepthFromLayer = false; + // XR_DXR_cursor_depth v3: DISPLAYXR_CURSOR_DEPTH_ANCHOR=hybrid|screen|world + // chains XrCursorDepthOptionsDXR with that anchor mode. Unset: the struct + // is not chained, and the runtime defaults to HYBRID. + bool cursorDepthChainOptions = false; + XrCursorDepthAnchorModeDXR cursorDepthAnchor = + XR_CURSOR_DEPTH_ANCHOR_MODE_HYBRID_DXR; SwapchainInfo depthSwapchain = {}; XrViewConfigurationType viewConfigType; std::vector configViews; @@ -1857,6 +1868,40 @@ static bool InitializeOpenXR(AppXrSession &app) } LOG_INFO("XR_DXR_cursor_depth source: %s", app.cursorDepthFromLayer ? "SUBMITTED DEPTH (runtime reads it)" : "app hit test"); + + // Opt-in (spec v3): pick the anchor mode explicitly. + const char *anchor = getenv("DISPLAYXR_CURSOR_DEPTH_ANCHOR"); + if (anchor != nullptr && anchor[0] != '\0') { + bool known = true; + if (strcmp(anchor, "hybrid") == 0) { + app.cursorDepthAnchor = XR_CURSOR_DEPTH_ANCHOR_MODE_HYBRID_DXR; + } else if (strcmp(anchor, "screen") == 0) { + app.cursorDepthAnchor = XR_CURSOR_DEPTH_ANCHOR_MODE_SCREEN_DXR; + } else if (strcmp(anchor, "world") == 0) { + app.cursorDepthAnchor = XR_CURSOR_DEPTH_ANCHOR_MODE_WORLD_DXR; + } else { + known = false; + LOG_WARN("DISPLAYXR_CURSOR_DEPTH_ANCHOR=%s: expected " + "hybrid|screen|world - not chaining options", + anchor); + } + if (known && app.cursorDepthSpecVersion < 3) { + LOG_WARN("DISPLAYXR_CURSOR_DEPTH_ANCHOR needs " + "XR_DXR_cursor_depth v3 (have v%u) - ignored", + app.cursorDepthSpecVersion); + } else if (known) { + app.cursorDepthChainOptions = true; + } + } + LOG_INFO( + "XR_DXR_cursor_depth anchor: %s", + !app.cursorDepthChainOptions + ? "runtime default (options not chained)" + : app.cursorDepthAnchor == XR_CURSOR_DEPTH_ANCHOR_MODE_SCREEN_DXR + ? "SCREEN" + : app.cursorDepthAnchor == XR_CURSOR_DEPTH_ANCHOR_MODE_WORLD_DXR + ? "WORLD" + : "HYBRID"); } XrInstanceCreateInfo createInfo = {XR_TYPE_INSTANCE_CREATE_INFO}; @@ -3160,6 +3205,11 @@ int main(int argc, char **argv) // Spec v2 (ADR-046 3a): "find the nearest point in the depth I submit". XrCursorDepthSourceDXR cursorSource = {XR_TYPE_CURSOR_DEPTH_SOURCE_DXR}; cursorSource.source = XR_CURSOR_DEPTH_SOURCE_SUBMITTED_DEPTH_DXR; + // Spec v3: anchor mode (margin left at the runtime default). + XrCursorDepthOptionsDXR cursorOptions = { + XR_TYPE_CURSOR_DEPTH_OPTIONS_DXR}; + cursorOptions.anchorMode = app.cursorDepthAnchor; + cursorOptions.margin = 0.0f; bool cursorInCanvas = false; if (app.hasCursorDepthExt && g_window != nil && g_metalView != nil) { NSPoint pw = [g_window mouseLocationOutsideOfEventStream]; @@ -3199,13 +3249,18 @@ int main(int argc, char **argv) : XR_FALSE; cursorHint.nearestPoint = hit; } - // Chain: locateInfo -> hint [-> source] -> (rig struct). + // Chain: locateInfo -> hint [-> options] [-> source] -> + // (rig struct). + const void *hintTail = locateInfo.next; if (app.cursorDepthFromLayer) { - cursorSource.next = locateInfo.next; - cursorHint.next = &cursorSource; - } else { - cursorHint.next = locateInfo.next; + cursorSource.next = hintTail; + hintTail = &cursorSource; + } + if (app.cursorDepthChainOptions) { + cursorOptions.next = hintTail; + hintTail = &cursorOptions; } + cursorHint.next = hintTail; locateInfo.next = &cursorHint; } } diff --git a/tests/tests_aux_cursor_depth.cpp b/tests/tests_aux_cursor_depth.cpp index 09dc96461..103cefe93 100644 --- a/tests/tests_aux_cursor_depth.cpp +++ b/tests/tests_aux_cursor_depth.cpp @@ -81,6 +81,14 @@ require_near(xrt_vec3 a, xrt_vec3 b, float eps = 1e-4f) CHECK(a.z == Catch::Approx(b.z).margin(eps)); } +xrt_vec3 +on_glass(xrt_vec3 eye, xrt_vec3 p) +{ + // Where the line from eye through p crosses the canvas plane z = 0. + const float k = eye.z / (eye.z - p.z); + return xrt_vec3{eye.x + (p.x - eye.x) * k, eye.y + (p.y - eye.y) * k, 0.0f}; +} + } // namespace TEST_CASE("cursor_depth: the canvas point under the cursor is where the view rays meet") @@ -221,12 +229,133 @@ TEST_CASE("cursor_depth: target = content minus margin, clamped; nothing under i CHECK(u_cursor_depth_target(&t, false, -0.4f) == 0.0f); CHECK(u_cursor_depth_target(&t, true, NAN) == 0.0f); CHECK(u_cursor_depth_target(&t, true, -0.2f) == Catch::Approx(-0.2f - t.margin)); - CHECK(u_cursor_depth_target(&t, true, 0.0f) == Catch::Approx(-t.margin)); CHECK(u_cursor_depth_target(&t, true, -5.0f) == t.min_disparity); - CHECK(u_cursor_depth_target(&t, true, 0.95f) == t.max_disparity); CHECK(t.max_disparity < 1.0f); // place() divides by 1 - d } +TEST_CASE("cursor_depth: never behind the glass (Amendment 2 A.1)") +{ + const auto t = tuning(); + CHECK(t.max_disparity == 0.0f); + // Content ON the canvas: the cursor stays on it (it used to float margin in front). + CHECK(u_cursor_depth_target(&t, true, 0.0f) == 0.0f); + // Content behind it: the cursor stays on the canvas (it used to sink onto it). + CHECK(u_cursor_depth_target(&t, true, 0.3f) == 0.0f); + CHECK(u_cursor_depth_target(&t, true, 0.95f) == 0.0f); + CHECK(u_cursor_depth_target(&t, true, INFINITY) == 0.0f); + // Content just in front: margin in front of it. + CHECK(u_cursor_depth_target(&t, true, -0.001f) == Catch::Approx(-0.001f - t.margin)); + + // A tuning asking for room behind the glass cannot get it. + auto loose = t; + loose.max_disparity = 0.6f; + CHECK(u_cursor_depth_target(&loose, true, 0.3f) == 0.0f); + loose.margin = -0.2f; // a margin pushing the cursor BEHIND content in front + CHECK(u_cursor_depth_target(&loose, true, -0.1f) <= 0.0f); +} + +TEST_CASE("cursor_depth: property - the target is never > 0, for any content and geometry") +{ + // Random heads, cursor UVs and content points (in front of and behind the + // canvas, through the real geometry + disparity path), random tunings. + uint32_t seed = 0x1847u; + auto rnd = [&seed]() { + seed = seed * 1664525u + 1013904223u; + return (float)(seed >> 8) / (float)(1u << 24); + }; + int behind = 0, in_front = 0; + for (int i = 0; i < 4000; i++) { + const xrt_vec3 head{(rnd() - 0.5f) * 0.4f, (rnd() - 0.5f) * 0.2f, 0.35f + rnd() * 0.6f}; + const auto a = kooima({head.x - 0.032f, head.y, head.z}); + const auto b = kooima({head.x + 0.032f, head.y, head.z}); + const float u = rnd(), v = rnd(); + u_cursor_depth_geometry g{}; + REQUIRE(u_cursor_depth_geometry_solve(&a, &b, u, v, &g)); + // Content anywhere from 90% of the way to the eye to 3 m behind the glass. + const float z = -3.0f + rnd() * (3.0f + 0.9f * head.z); + const xrt_vec3 p{(rnd() - 0.5f) * 0.6f, (rnd() - 0.5f) * 0.4f, z}; + float d = 0.0f; + if (!u_cursor_depth_point_disparity(&g, &p, &d)) { + continue; + } + auto t = tuning(); + if (i % 3 == 1) { + t.max_disparity = rnd() * 0.8f; // misconfigured: room behind the glass + } + if (i % 5 == 2) { + t.margin = (rnd() - 0.5f) * 0.1f; // including <= 0 + } + const float target = u_cursor_depth_target(&t, true, d); + CHECK(target <= 0.0f); + if (d >= 0.0f) { + behind++; + CHECK(target == 0.0f); + } else { + in_front++; + if (t.margin > 0.0f && t.max_disparity >= 0.0f) { + // In front of the content, inside the comfort clamp. + CHECK(target == Catch::Approx(std::fmax(d - t.margin, t.min_disparity))); + if (d - t.margin > t.min_disparity) { + CHECK(target < d); + } + } + } + } + CHECK(behind > 500); + CHECK(in_front > 500); +} + +TEST_CASE("cursor_depth: draw threshold - on the plane it is the OS cursor (Amendment 2 A.3)") +{ + const auto t = tuning(); + CHECK(t.draw_threshold == Catch::Approx(-0.005f)); + CHECK_FALSE(u_cursor_depth_should_draw(&t, 0.0f)); + CHECK_FALSE(u_cursor_depth_should_draw(&t, -0.0049f)); + CHECK(u_cursor_depth_should_draw(&t, -0.0051f)); + CHECK(u_cursor_depth_should_draw(&t, -0.3f)); + CHECK_FALSE(u_cursor_depth_should_draw(&t, NAN)); + CHECK_FALSE(u_cursor_depth_should_draw(&t, 0.2f)); + // A threshold at or behind the glass falls back to the default. + auto bad = t; + bad.draw_threshold = 0.1f; + CHECK_FALSE(u_cursor_depth_should_draw(&bad, 0.0f)); + CHECK_FALSE(u_cursor_depth_should_draw(&bad, -0.004f)); + CHECK(u_cursor_depth_should_draw(&bad, -0.006f)); +} + +TEST_CASE("cursor_depth: the drawn cursor hands back smoothly - the filter steps while hidden") +{ + // Content pops out, then goes away: the cursor sinks back over ~250 ms, + // drawn until it crosses the threshold, then hidden; never a jump to 0. + const auto t = tuning(); + u_cursor_depth_filter f{}; + uint64_t now = 1000 * MS; + float d = u_cursor_depth_filter_step(&f, &t, u_cursor_depth_target(&t, true, -0.2f), now); + CHECK(u_cursor_depth_should_draw(&t, d)); + float prev = d; + bool hidden = false; + for (int i = 0; i < 200; i++) { + now += 16 * MS; + d = u_cursor_depth_filter_step(&f, &t, u_cursor_depth_target(&t, true, 0.4f), now); + CHECK(d >= prev); + CHECK(d - prev < 0.05f); + CHECK(d <= 0.0f); + if (!u_cursor_depth_should_draw(&t, d)) { + hidden = true; + } else { + CHECK_FALSE(hidden); // once handed back, stays handed back + } + prev = d; + } + CHECK(hidden); + // It comes back fast when content pops out again. + now += 16 * MS; + d = u_cursor_depth_filter_step(&f, &t, u_cursor_depth_target(&t, true, -0.2f), now); + now += 16 * MS; + d = u_cursor_depth_filter_step(&f, &t, u_cursor_depth_target(&t, true, -0.2f), now); + CHECK(u_cursor_depth_should_draw(&t, d)); +} + TEST_CASE("cursor_depth: the filter rises fast and sinks slowly") { const auto t = tuning(); @@ -266,19 +395,18 @@ TEST_CASE("cursor_depth: a long gap or a clock going backwards re-primes") CHECK(u_cursor_depth_filter_step(&f, &t, -0.2f, 1500 * MS) == -0.2f); } -TEST_CASE("cursor_depth: head motion does not move the cursor on the glass (screen-anchored, jitter-free)") +TEST_CASE("cursor_depth: SCREEN anchor - head motion does not move the cursor on the glass") { - // Project the placed sprite back onto the canvas (z = 0) from each eye. For a fixed cursor - // UV and disparity, those two screen points must not depend on where the head is: the - // sprite sits on the cyclopean ray, so each eye sees it at S -/+ (baseline/2)*d — tracking - // motion or jitter cannot make it swim, only the content under it can change its depth. - auto on_glass = [](xrt_vec3 eye, xrt_vec3 p) { - const float k = eye.z / (eye.z - p.z); - return xrt_vec3{eye.x + (p.x - eye.x) * k, eye.y + (p.y - eye.y) * k, 0.0f}; - }; + // The SCREEN-mode property (spec v1/v2 behaviour: u_cursor_depth_place and + // U_CURSOR_DEPTH_ANCHOR_SCREEN). Project the placed sprite back onto the canvas (z = 0) + // from each eye. For a fixed cursor UV and disparity, those two screen points do not + // depend on where the head is: the sprite sits on the cyclopean ray, so each eye sees it + // at S -/+ (baseline/2)*d. That is exactly why it has NO motion parallax - the conflict + // HYBRID (the default since spec v3) resolves while the pointer is still. const float d = -0.4f; xrt_vec3 ref_l{}, ref_r{}; const xrt_vec3 heads[] = {{0.0f, 0.0f, 0.6f}, {0.12f, -0.05f, 0.5f}, {-0.2f, 0.08f, 0.75f}}; + u_cursor_depth_anchor anchor{}; for (int i = 0; i < 3; i++) { const xrt_vec3 h = heads[i]; const xrt_vec3 el{h.x - 0.032f, h.y, h.z}, er{h.x + 0.032f, h.y, h.z}; @@ -286,9 +414,15 @@ TEST_CASE("cursor_depth: head motion does not move the cursor on the glass (scre const auto b = kooima(er); u_cursor_depth_geometry g{}; REQUIRE(u_cursor_depth_geometry_solve(&a, &b, 0.3f, 0.4f, &g)); - xrt_vec3 c{}; - float hgt = 0.0f; + xrt_vec3 c{}, c2{}; + float hgt = 0.0f, hgt2 = 0.0f; u_cursor_depth_place(&g, d, 0.03f, &c, &hgt); + // The anchored entry point in SCREEN mode is the same placement, and ignores the anchor. + u_cursor_depth_place_anchored(&g, d, 0.03f, U_CURSOR_DEPTH_ANCHOR_SCREEN, &anchor, 0.3f, 0.4f, false, + (uint64_t)(i + 1) * 16 * MS, &c2, &hgt2); + require_near(c2, c, 1e-6f); + CHECK(hgt2 == hgt); + CHECK_FALSE(anchor.has_anchor); const xrt_vec3 sl = on_glass(el, c), sr = on_glass(er, c); // On-screen disparity is baseline * d, whatever the head pose. CHECK(sr.x - sl.x == Catch::Approx(0.064f * d).margin(1e-5)); @@ -304,6 +438,266 @@ TEST_CASE("cursor_depth: head motion does not move the cursor on the glass (scre } +/* + * + * Spec v3: anchor modes. + * + */ + +namespace { + +struct HeadGeom +{ + xrt_vec3 el, er; + u_cursor_depth_geometry g; +}; + +HeadGeom +head(xrt_vec3 h, float u, float v) +{ + HeadGeom r{}; + r.el = {h.x - 0.032f, h.y, h.z}; + r.er = {h.x + 0.032f, h.y, h.z}; + const auto a = kooima(r.el); + const auto b = kooima(r.er); + REQUIRE(u_cursor_depth_geometry_solve(&a, &b, u, v, &r.g)); + return r; +} + +} // namespace + +TEST_CASE("cursor_depth: margin default is 0.03 (Amendment 2 A.2)") +{ + // Spec v3 briefly used 0.005; Amendment 2 restores the reference app's 0.03. + CHECK(tuning().margin == Catch::Approx(0.03f)); + CHECK(U_CURSOR_DEPTH_DEFAULT_MARGIN == Catch::Approx(0.03f)); +} + +TEST_CASE("cursor_depth: HYBRID - pointer moving = SCREEN placement (on the cyclopean ray)") +{ + const float d = -0.4f; + u_cursor_depth_anchor anchor{}; + const float uvs[][2] = {{0.3f, 0.4f}, {0.31f, 0.4f}, {0.31f, 0.45f}, {0.7f, 0.2f}}; + const xrt_vec3 heads[] = { + {0.0f, 0.0f, 0.6f}, {0.12f, -0.05f, 0.5f}, {-0.2f, 0.08f, 0.75f}, {0.05f, 0.0f, 0.6f}}; + for (int i = 0; i < 4; i++) { + const HeadGeom hg = head(heads[i], uvs[i][0], uvs[i][1]); + xrt_vec3 c{}, cs{}; + float h = 0.0f, hs = 0.0f; + u_cursor_depth_place_anchored(&hg.g, d, 0.03f, U_CURSOR_DEPTH_ANCHOR_HYBRID, &anchor, uvs[i][0], + uvs[i][1], false, (uint64_t)(i + 1) * 16 * MS, &c, &h); + u_cursor_depth_place(&hg.g, d, 0.03f, &cs, &hs); + require_near(c, cs, 1e-6f); + CHECK(h == hs); + CHECK(anchor.has_anchor); + // The stored anchor is this locate's line of sight. + require_near(anchor.eye, hg.g.eye, 1e-6f); + require_near(anchor.canvas_point, hg.g.canvas_point, 1e-6f); + CHECK(anchor.eye_to_canvas == hg.g.eye_to_canvas); + } +} + +TEST_CASE("cursor_depth: HYBRID - pointer still, head moves = world-fixed (parallaxes like content)") +{ + const float d = -0.4f; + const float u = 0.3f, v = 0.4f; + u_cursor_depth_anchor anchor{}; + + // Frame 1: the pointer arrives (moving) with the head off-axis. + const HeadGeom h0 = head({0.1f, 0.03f, 0.6f}, u, v); + xrt_vec3 c0{}; + float hgt = 0.0f; + u_cursor_depth_place_anchored(&h0.g, d, 0.03f, U_CURSOR_DEPTH_ANCHOR_HYBRID, &anchor, u, v, false, 16 * MS, &c0, + &hgt); + const xrt_vec3 img0 = on_glass(h0.el, c0); + + // Then the pointer is still and the head moves laterally (same distance): + // the sprite stays put in locate space. + const xrt_vec3 lateral[] = {{-0.1f, 0.0f, 0.6f}, {0.2f, -0.06f, 0.6f}, {0.0f, 0.0f, 0.6f}}; + uint64_t now = 16 * MS; + for (const auto &hp : lateral) { + now += 16 * MS; + const HeadGeom hg = head(hp, u, v); + xrt_vec3 c{}; + u_cursor_depth_place_anchored(&hg.g, d, 0.03f, U_CURSOR_DEPTH_ANCHOR_HYBRID, &anchor, u, v, false, now, + &c, &hgt); + require_near(c, c0, 1e-5f); + + // ... so its image on the glass moves with the head like content at that depth, + // while a SCREEN-anchored sprite's image would not move at all. + xrt_vec3 cs{}; + float hs = 0.0f; + u_cursor_depth_place(&hg.g, d, 0.03f, &cs, &hs); + CHECK(std::fabs(on_glass(hg.el, c).x - img0.x) > 1e-3f); + CHECK(on_glass(hg.el, cs).x == Catch::Approx(img0.x).margin(1e-5)); + } + + // Head moves toward the glass: the same disparity is a shallower depth. The + // sprite slides along the anchor-time line of sight to that depth. + now += 16 * MS; + const HeadGeom hz = head({0.0f, 0.0f, 0.45f}, u, v); + xrt_vec3 cz{}; + u_cursor_depth_place_anchored(&hz.g, d, 0.03f, U_CURSOR_DEPTH_ANCHOR_HYBRID, &anchor, u, v, false, now, &cz, + &hgt); + require_near(on_glass(h0.g.eye, cz), h0.g.canvas_point, 5e-5f); + float dz = 0.0f; + REQUIRE(u_cursor_depth_point_disparity(&hz.g, &cz, &dz)); + CHECK(dz == Catch::Approx(d).margin(2e-4)); + + // A re-prime (stale gap) re-anchors: back on the cyclopean ray. + const HeadGeom hr = head({-0.15f, 0.0f, 0.6f}, u, v); + xrt_vec3 cr{}, cs{}; + float hs = 0.0f; + u_cursor_depth_place_anchored(&hr.g, d, 0.03f, U_CURSOR_DEPTH_ANCHOR_HYBRID, &anchor, u, v, true, + now + 2000 * MS, &cr, &hgt); + u_cursor_depth_place(&hr.g, d, 0.03f, &cs, &hs); + require_near(cr, cs, 1e-6f); + CHECK(std::fabs(cr.x - c0.x) > 1e-3f); +} + +TEST_CASE("cursor_depth: HYBRID - continuous on the frame the pointer stops") +{ + u_cursor_depth_anchor anchor{}; + const HeadGeom prev = head({0.12f, -0.04f, 0.55f}, 0.6f, 0.33f); + const HeadGeom hg = head({0.12f, -0.04f, 0.55f}, 0.62f, 0.33f); + float hgt = 0.0f; + + // Moving: a frame at another UV, then the final moving frame. + xrt_vec3 c_prev{}, c_moving{}, c_still{}; + u_cursor_depth_place_anchored(&prev.g, -0.25f, 0.03f, U_CURSOR_DEPTH_ANCHOR_HYBRID, &anchor, 0.6f, 0.33f, false, + 16 * MS, &c_prev, &hgt); + u_cursor_depth_place_anchored(&hg.g, -0.3f, 0.03f, U_CURSOR_DEPTH_ANCHOR_HYBRID, &anchor, 0.62f, 0.33f, false, + 32 * MS, &c_moving, &hgt); + // Stop frame: same UV, same t, same head. + u_cursor_depth_place_anchored(&hg.g, -0.3f, 0.03f, U_CURSOR_DEPTH_ANCHOR_HYBRID, &anchor, 0.62f, 0.33f, false, + 48 * MS, &c_still, &hgt); + require_near(c_still, c_moving, 1e-6f); + + // A UV change below the move threshold is "still" too (no re-anchor on noise). + xrt_vec3 c_noise{}; + u_cursor_depth_place_anchored(&hg.g, -0.3f, 0.03f, U_CURSOR_DEPTH_ANCHOR_HYBRID, &anchor, 0.62f + 5e-5f, 0.33f, + false, 64 * MS, &c_noise, &hgt); + require_near(c_noise, c_moving, 1e-6f); + CHECK(anchor.last_ns == 32 * MS); + + // Still, but the content under it changes depth (same head): the sprite slides + // along the line of sight, i.e. it equals the SCREEN placement at the new depth. + xrt_vec3 c_deeper{}, c_deeper_screen{}; + float hs = 0.0f; + u_cursor_depth_place_anchored(&hg.g, -0.1f, 0.03f, U_CURSOR_DEPTH_ANCHOR_HYBRID, &anchor, 0.62f, 0.33f, false, + 80 * MS, &c_deeper, &hgt); + u_cursor_depth_place(&hg.g, -0.1f, 0.03f, &c_deeper_screen, &hs); + require_near(c_deeper, c_deeper_screen, 1e-5f); + float dd = 0.0f; + REQUIRE(u_cursor_depth_point_disparity(&hg.g, &c_deeper, &dd)); + CHECK(dd == Catch::Approx(-0.1f).margin(1e-5)); +} + +TEST_CASE("cursor_depth: HYBRID - content appears under a still pointer: stays on the click point, WORLD does not") +{ + // The case that motivated anchoring the LINE OF SIGHT rather than its foot: the + // anchor is stored with the cursor on the glass (d0 = 0, e.g. before the first + // hit test), then the content under the still pointer pops out. A viewer well + // off-axis (eye 0.10 m above and 0.08 m left of the pointer's canvas point). + const float u = 0.75f, v = 0.5f; + const HeadGeom hg = head({0.0f, 0.1f, 0.6f}, u, v); + u_cursor_depth_anchor anchor{}; + xrt_vec3 c{}; + float hgt = 0.0f; + u_cursor_depth_place_anchored(&hg.g, 0.0f, 0.03f, U_CURSOR_DEPTH_ANCHOR_HYBRID, &anchor, u, v, false, 16 * MS, + &c, &hgt); + require_near(c, hg.g.canvas_point, 1e-5f); + + uint64_t now = 16 * MS; + const float ds[] = {-0.02f, -0.1f, -0.3f, 0.2f}; + for (float d : ds) { + now += 16 * MS; + u_cursor_depth_place_anchored(&hg.g, d, 0.03f, U_CURSOR_DEPTH_ANCHOR_HYBRID, &anchor, u, v, false, now, + &c, &hgt); + // On the E0 -> S0 line: seen from E0 it projects onto S0, the click point... + require_near(on_glass(anchor.eye, c), anchor.canvas_point, 5e-5f); + // ... at the requested depth. + float back = 0.0f; + REQUIRE(u_cursor_depth_point_disparity(&hg.g, &c, &back)); + CHECK(back == Catch::Approx(d).margin(1e-4)); + + // WORLD rises along the normal instead: off the click point for this viewer. + xrt_vec3 cw{}; + u_cursor_depth_place_anchored(&hg.g, d, 0.03f, U_CURSOR_DEPTH_ANCHOR_WORLD, nullptr, u, v, false, now, + &cw, &hgt); + const xrt_vec3 pw = on_glass(hg.g.eye, cw); + CHECK(std::fabs(pw.y - hg.g.canvas_point.y) + std::fabs(pw.x - hg.g.canvas_point.x) > 1e-3f); + } + + // And after the head moves laterally, the sprite is still world-fixed at fixed t. + xrt_vec3 c_before = c; + const HeadGeom moved_head = head({0.12f, 0.1f, 0.6f}, u, v); + xrt_vec3 c_after{}; + u_cursor_depth_place_anchored(&moved_head.g, 0.2f, 0.03f, U_CURSOR_DEPTH_ANCHOR_HYBRID, &anchor, u, v, false, + now + 16 * MS, &c_after, &hgt); + require_near(c_after, c_before, 1e-5f); +} + +TEST_CASE("cursor_depth: WORLD - in front of the canvas point along the normal, at the requested disparity") +{ + const float d = -0.37f; + const float t = 1.0f / (1.0f - d); + + // On-axis: E centred on S. + { + const HeadGeom hg = head({0.0f, 0.0f, 0.6f}, 0.5f, 0.5f); + xrt_vec3 c{}, cs{}; + float h = 0.0f, hs = 0.0f; + u_cursor_depth_place_anchored(&hg.g, d, 0.03f, U_CURSOR_DEPTH_ANCHOR_WORLD, nullptr, 0.5f, 0.5f, false, + 16 * MS, &c, &h); + const float lift = (1.0f - t) * hg.g.eye_to_canvas; + require_near(c, + {hg.g.canvas_point.x - hg.g.forward.x * lift, hg.g.canvas_point.y - hg.g.forward.y * lift, + hg.g.canvas_point.z - hg.g.forward.z * lift}, + 1e-6f); + CHECK(c.z > 0.0f); // in front of the glass (toward the viewer) + float back = 0.0f; + REQUIRE(u_cursor_depth_point_disparity(&hg.g, &c, &back)); + CHECK(back == Catch::Approx(d).margin(1e-5)); + // On-axis, WORLD and SCREEN coincide; the height is the same in every mode. + u_cursor_depth_place(&hg.g, d, 0.03f, &cs, &hs); + require_near(c, cs, 1e-6f); + CHECK(h == hs); + } + + // Off-axis: straight in front of S (same x, y), not on the line of sight. + { + const HeadGeom hg = head({0.15f, 0.05f, 0.6f}, 0.2f, 0.7f); + xrt_vec3 c{}, cs{}; + float h = 0.0f, hs = 0.0f; + u_cursor_depth_anchor untouched{}; + u_cursor_depth_place_anchored(&hg.g, d, 0.03f, U_CURSOR_DEPTH_ANCHOR_WORLD, &untouched, 0.2f, 0.7f, + false, 16 * MS, &c, &h); + CHECK_FALSE(untouched.has_anchor); + CHECK(c.x == Catch::Approx(hg.g.canvas_point.x).margin(1e-6)); + CHECK(c.y == Catch::Approx(hg.g.canvas_point.y).margin(1e-6)); + CHECK(c.z == Catch::Approx((1.0f - t) * hg.g.eye_to_canvas).margin(1e-6)); + float back = 0.0f; + REQUIRE(u_cursor_depth_point_disparity(&hg.g, &c, &back)); + CHECK(back == Catch::Approx(d).margin(1e-5)); + u_cursor_depth_place(&hg.g, d, 0.03f, &cs, &hs); + CHECK(std::fabs(cs.x - c.x) > 1e-3f); + } +} + +TEST_CASE("cursor_depth: filter_will_prime predicts exactly the steps that snap") +{ + const auto t = tuning(); + u_cursor_depth_filter f{}; + CHECK(u_cursor_depth_filter_will_prime(&f, &t, 1000 * MS)); + u_cursor_depth_filter_step(&f, &t, -0.3f, 1000 * MS); + CHECK_FALSE(u_cursor_depth_filter_will_prime(&f, &t, 1016 * MS)); + CHECK_FALSE(u_cursor_depth_filter_will_prime(&f, &t, 1000 * MS)); // same frame + CHECK(u_cursor_depth_filter_will_prime(&f, &t, 999 * MS)); // clock backwards + CHECK(u_cursor_depth_filter_will_prime(&f, &t, 2000 * MS)); // stale gap +} + + /* * * Phase 3a: the depth-layer source (spec v2).