Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 63 additions & 3 deletions docs/adr/ADR-046-depth-aware-cursor.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ADR-046: Depth-aware cursor — opt-in only; the app knows the depth, the runtime places the cursor

**Status:** Proposed (2026-10-06) · Phase 1 implemented · Phase 3a implemented (Metal) · spec v3 anchor modes (2026-10-07) · Amendment 1 (lifted content, 2026-10-10) accepted · Amendment 2 (one policy, runtime draws, app chooses the look, 2026-10-11) accepted, §B built on Metal (spec v5) · spec:
**Status:** Proposed (2026-10-06) · Phase 1 implemented · Phase 3a implemented (Metal) · spec v3 anchor modes (2026-10-07) · Amendment 1 (lifted content, 2026-10-10) accepted · Amendment 2 (one policy, runtime draws, app chooses the look, 2026-10-11) accepted, §B built on Metal, Vulkan and GL (spec v5) · spec:
[XR_DXR_cursor_depth.md](../specs/extensions/XR_DXR_cursor_depth.md) · sibling of
[ADR-040](ADR-040-rear-depth-budget.md) (the same cue conflict, at the cursor instead of the desktop)

Expand Down Expand Up @@ -605,9 +605,69 @@ sprite and the depth, and the runtime owns the drawing. Apps get the same contra
metres per point (pixel pitch × backing scale). Off the main thread the last snapshot is
kept, else the classic arrow (12 × 19, white fill, dark outline, rasterised procedurally,
4 × 4 supersampled). `DXR_CURSOR_DEFAULT_LOOK=arrow` forces the arrow.
- **Not yet:** D3D11, D3D12, GL, vk_native and the service (`svc_draw_cursor_quads`) report
- **Not yet:** D3D11, D3D12 and the service (`svc_draw_cursor_quads`) report
`willDraw = XR_FALSE`; display zones; multi-screen routed frames draw per tile without the
segment routing; the 0.5 px floor of A.3 is not applied (−0.005 is above it on current panels).
segment routing on Metal (Vulkan and GL skip them); the 0.5 px floor of A.3 is not applied
(−0.005 is above it on current panels).

#### §B as built on Vulkan and GL (`XR_DXR_cursor_depth` spec v5, unchanged)

- **One default look for every backend.** The Metal compositor's default-look code moved to
`compositor/util/comp_cursor_look.{h,c}` (+ `comp_cursor_look_macos.m`), a small library the
Metal, vk_native and GL compositors all link: it picks the look (mirrored `[NSCursor
currentCursor]` on the main thread, else the last snapshot, else the procedural arrow from
`u_cursor_draw_arrow_rgba()`), rasterises it once per change into premultiplied, sRGB-encoded
RGBA8, and reports aspect, hotspot and size in points plus the main screen's backing scale.
Each compositor only uploads the pixels when the look's `generation` moves.
`DXR_CURSOR_DEFAULT_LOOK=arrow` forces the arrow on all three. **OS-cursor mirroring is
macOS-only:** Linux (`XFixesGetCursorImage`) and Windows (`GetCursorInfo` + `GetIconInfo`)
draw the arrow; mirroring there is future work.
- **State tracker.** Additive only: `oxr_session_cursor_draw_supported()` /
`_hand_request()` gained the `is_vk_native_compositor` and `is_gl_native_compositor` arms.
The placement, the per-view map and the zero-cost gate are the Metal ones.
- **vk_native.** The request is consumed by the next commit (one field test otherwise) and
resolved there into tile-normalised corners per view (`u_cursor_draw_corners`). A cursor frame
never zero-copies and always takes the renderer's **compose render pass** (the blit fast path
cannot blend): the cursor is the pass's last draw, one quad per view tile through the pass's
own pipelines (the QUAD vertex path with an affine MVP built from three corners; straight or
premultiplied blend), scissored to the tile, depth off. Because the pass renders into the
private `_SRGB` compose target and publishes to the atlas with a raw copy, the sprite blends
in linear light, an `_SRGB` sprite is decoded on sample (the same single source-view choice as
every layer, `compose_source_view()`), and the atlas stays ENCODED — never double-encoded. The
default look is an `R8G8B8A8_SRGB` image (UNORM under the legacy colour hatch) uploaded
through a staging buffer inside the same command buffer. Nothing is created before the first
request; the descriptor pool carries one extra set for the cursor. **Ordering differs from
Metal:** vk_native stamps window-space layers in a later, weave-time pass (several call sites,
some on the async weave hand-off), so the cursor is drawn over projection, quad and equirect2
layers but **under** window-space layers. Both the projection-only and the post-compose
`$TMPDIR/displayxr_atlas_trigger` captures show it.
- **GL.** Drawn after the window-space layers, before the DP, into whatever the atlas pass is
painting: the private `GL_SRGB8_ALPHA8` compose target with `GL_FRAMEBUFFER_SRGB` on, or the
ENCODED atlas directly. The source is bound through `gl_bind_layer_source()` (decode iff
composing), the same rule as every layer; a cursor frame counts as one more sub-rect layer in
`gl_frame_takes_fast_path()` (a scene-linear UNORM sprite forces the compose path when it
exists). A small program (corners as uniforms, `gl_VertexID` strip), per-tile viewport +
scissor, blend state restored. A GL sprite is sampled bottom-up (its top is its last row —
the Metal compositor's GL-source flip). A layered (`arraySize > 1`) GL sprite is not drawn
yet (one-time WARN). On Apple's GL 4.1 there is no `glCopyImageSubData`, so every macOS GL
frame composes in encoded space, cursor included; the linear compose-target path was not run
here.
- **Measured** (macOS, sim-display Cropped SBS, cube reference apps at `cursorUV` 0.5,0.5 over
the cube, `DXR_CURSOR_DEFAULT_LOOK=arrow`): GL −23.6 px in the 1512 × 1646 atlas (Metal −24.0
on the same app geometry), vk_native −22.0 px in its 1280 × 1440 atlas; nothing drawn at
0.1,0.1; the arrow's fill reads (255,255,255) and its outline (5,5,8), the custom ring
(0,220,255) — exactly the authored bytes, so no encode is lost or doubled.
- **Known issue (GL on macOS, not caused by the draw):** `cube_handle_gl_macos` crashes in
roughly 1 of 5 launches once the depth cursor is ACTIVE — in Apple's GL driver, on the #868
repaint thread (`gl_repaint_thread` → `glBlitFramebuffer`) or in the app's own draw. It
follows the app's `[NSCursor hide]`, not the runtime draw: app-draw mode crashes too; the
runtime draw with the OS cursor left visible (0/10), `DXR_WEAVE_REPAINT=0` (0/8) and the
extension off (0/20) do not. The GL cube never hid the OS cursor before this adoption, so this
is a latent repaint-thread / AppKit race in the GL compositor surfacing — runtime#1947. Until it
is fixed the GL cube's depth cursor is **opt-in** (`DISPLAYXR_CURSOR_DEPTH=1`); the Metal and
Vulkan cubes stay default-on.
- **Found in passing (Metal, #1941):** on the Vulkan-app-on-Metal fallback a custom sprite is
handed over as the client swapchain and cast to the native Metal type — runtime#1948.

### Rollout

Expand Down
2 changes: 1 addition & 1 deletion docs/roadmap/control-panel-performance-settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -350,7 +350,7 @@ library linked into both · **CLI** = `displayxr-cli.exe` (reporting only, contr
| `DXR_WEAVE_REPAINT_TRACE` | `auxiliary/util/u_repaint_gate.h` (`u_repaint_trace_*`, wired into the vk_native + d3d11 repaint loops) | `getenv`, cached | off | App | 4 | #1257 verification instrumentation: one WARN row per ~5 s per loop with real tick cadence, replay/pace durations, and per-gate bail counts. Says where missing repaints went; the witness only says how many landed |
| `DXR_APP_FRAME_DIVISOR` | `auxiliary/util/u_app_partition.h` (`u_app_partition_throttle` in each in-process wait_frame; engagement read by the repaint gate + loop tick) | `getenv`, cached | off (1) | App | 2 | #1257 slot partition: xrWaitFrame releases the app every Dth vblank (D=2..8) and the repaint loop fills the other slots at panel rate with a KNOWN N=D schedule — steady output cadence, no fire/commit collision by construction. **Supported tier: the #918 split (d3d11 bridge / hybrid) only** — verified steady 60 with eyeball sign-off; in-process vk_native/d3d12/gl tiers REFUSE cleanly (fill-loop tick starvation, 17-19 ms vs 1.8-2.3 ms on the bridge; follow-up). The natural Control Panel "smooth motion, lower app rate" control and the hook for face-tracking-driven rate later. Not wired into Metal or the IPC/service path |
| `DXR_APP_FRAME_DIVISOR_ANY_TIER` | `auxiliary/util/u_app_partition.h` (`u_app_partition_throttle` tier gate) | `getenv`, cached | off | App | 4 | Bring-up override: lets the partition throttle engage on an UNSUPPORTED tier (which collapses the panel there today). For the in-process-tier follow-up work only |
| `DXR_CURSOR_DEFAULT_LOOK` | `compositor/metal/comp_metal_compositor.m` (`metal_cursor_update_default`) | `getenv`, `static` cached | unset (mirror the OS cursor) | App | 4 | `arrow` forces the classic-arrow fallback for the runtime-drawn depth cursor's default look (XR_DXR_cursor_depth v5, ADR-046 Amendment 2 B) instead of mirroring `[NSCursor currentCursor]`. Verification only; Metal only today |
| `DXR_CURSOR_DEFAULT_LOOK` | `compositor/util/comp_cursor_look.c` (`force_arrow`, shared by the Metal, vk_native and GL compositors) | `getenv`, `static` cached | unset (mirror the OS cursor on macOS; the arrow elsewhere) | App | 4 | `arrow` forces the classic-arrow fallback for the runtime-drawn depth cursor's default look (XR_DXR_cursor_depth v5, ADR-046 Amendment 2 B) instead of mirroring `[NSCursor currentCursor]`. Verification only |
| `DXR_DP_FORWARD_HORIZON` | `compositor/util/comp_weave_latency_win.h` (`predict_weave_to_scanout_ns`, feeds the DP `set_predicted_scanout` slot from the split + d3d11 + d3d12 weave paths) | `getenv`, cached | **on** (kill switch) | App | 1 | #206: per-weave FORWARD horizon for the vendor eye predictor, computed from the vsync-locked vblank grid (DXGI frame statistics — measured period, not nominal). `0` disables → the DP falls back to its retrospective smoothed heuristic, the pre-#206 behavior |
| `DXR_DP_FORWARD_HORIZON_LOOP` | `compositor/util/comp_weave_latency_win.h` (`predict_weave_to_scanout_ns` probes it; `po_observe`, fed from `after_present`'s resolve pass, learns the value) | `getenv`, cached | **on** (kill switch) | App | 1 | #1435: learns the pipeline's whole-period offset (composition frame on DComp chains, queued frame at governor depth ≥ 2) from realised flips — mode of the last 32 resolved weaves, ≥ 24 agreeing — and adds it to the forward horizon. `0` pins the offset at 0, the pre-#1435 feed |
| `DXR_FILL_FENCE_PARK` | `compositor/vk_native/comp_vk_native_compositor.c` (fill path of `vk_dp_weave_and_present`) | `getenv`, cached | **on** (kill switch) | App | 1 | #1264 S1: a fill's GPU fence wait releases the compositor lock (re-validated on retake; superseded fills are dropped). `0` restores the lock-held synchronous wait |
Expand Down
10 changes: 7 additions & 3 deletions docs/specs/extensions/XR_DXR_cursor_depth.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ chain it, for diagnostics).

| Field | Meaning |
|---|---|
| `image` | An ordinary color swapchain this session created with `xrCreateSwapchain`, in the app's own graphics API, filled with any sprite and released at least once. The runtime samples its **last released image** (array layer 0, whole image) as **straight (non-premultiplied) alpha**, decoded per its format (an `_SRGB` format decodes). It must stay alive until the `xrEndFrame` of every frame that names it. `XR_NULL_HANDLE` = the runtime's **default look**: the current OS cursor's shape where the runtime can read it, else the classic arrow (12 × 19, white fill, dark outline, tip = hotspot). |
| `image` | An ordinary color swapchain this session created with `xrCreateSwapchain`, in the app's own graphics API, filled with any sprite and released at least once. The runtime samples its **last released image** (array layer 0, whole image) as **straight (non-premultiplied) alpha**, decoded per its format (an `_SRGB` format decodes). It must stay alive until the `xrEndFrame` of every frame that names it. On OpenGL the image's top is its **last** row (GL's bottom-left origin, which is how a GL app renders into it); upload a top-down bitmap bottom row first. `XR_NULL_HANDLE` = the runtime's **default look**: the current OS cursor's shape where the runtime can read it, else the classic arrow (12 × 19, white fill, dark outline, tip = hotspot). |
| `hotSpot` | The click point in the sprite, normalised, origin top-left, v down. Placed at the cursor. Ignored for the default look (it brings its own). |
| `sizeMeters` | The sprite's **height** on the panel at the display plane, metres; width follows the image aspect. Scaled with depth, so its apparent size never changes. ≤ 0 or non-finite = the OS cursor's size (the default look's own size; 19 OS points for a custom sprite). |
| `visible` | `XR_FALSE`: draw nothing. |
Expand All @@ -175,8 +175,12 @@ the runtime cannot draw (another backend, a space mismatch, a zones frame), it r
`XR_FALSE`, so the fallback is always the flat OS cursor, never no cursor. The runtime never
touches the OS cursor itself.

**Backends (v5).** Metal (in-process) draws. D3D11, D3D12, GL, vk_native and the IPC/service path
report `willDraw = XR_FALSE` with a one-time WARN (ADR-046 Amendment 2 rollout step 3).
**Backends (v5).** The in-process Metal, Vulkan (vk_native) and OpenGL compositors draw. D3D11,
D3D12 and the IPC/service path report `willDraw = XR_FALSE` with a one-time WARN (ADR-046
Amendment 2 rollout step 3). On Vulkan the cursor is drawn after the projection, quad and
equirect2 layers but **under** window-space layers, which that compositor stamps in a later pass
(ADR-046 Amendment 2, "§B as built on Vulkan and GL"). The default look mirrors the OS cursor on
macOS only; elsewhere it is the classic arrow.

### 2.7 `XrCursorDepthLiftRectDXR` / `XrCursorDepthLiftStateDXR` (spec v6) — lifted content

Expand Down
15 changes: 15 additions & 0 deletions src/xrt/compositor/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,21 @@ if(APPLE)
target_include_directories(comp_macos_window PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
endif()

###
# XR_DXR_cursor_depth v5 (ADR-046 Amendment 2 B): the runtime-drawn cursor's
# default look (the mirrored OS cursor, else the classic arrow), shared by the
# Metal, Vulkan and GL compositors so they cannot drift apart on it. Its own
# tiny library rather than part of comp_util: comp_gl does not link comp_util
# (which is Vulkan-gated), and none of this needs a graphics API.
#
add_library(comp_cursor_look STATIC util/comp_cursor_look.c util/comp_cursor_look.h)
target_link_libraries(comp_cursor_look PUBLIC xrt-interfaces PRIVATE aux_util)
target_include_directories(comp_cursor_look PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
if(APPLE)
target_sources(comp_cursor_look PRIVATE util/comp_cursor_look_macos.m)
target_link_libraries(comp_cursor_look PRIVATE "-framework AppKit")
endif()

###
# Shared desktop-Linux service window (#710, #967): the runtime-owned X11
# surface (Xorg or XWayland) the service's comp_multi shared spatial surface
Expand Down
2 changes: 2 additions & 0 deletions src/xrt/compositor/gl/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ target_link_libraries(
aux_math
aux_ogl
xrt-external-stb
# XR_DXR_cursor_depth v5: the runtime-drawn cursor's default look.
comp_cursor_look
)

target_include_directories(comp_gl PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/..)
Expand Down
Loading
Loading