Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 55 additions & 3 deletions docs/adr/ADR-046-depth-aware-cursor.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ADR-046: Depth-aware cursor — opt-in only; the app knows the depth, the runtime places the cursor

**Status:** Proposed (2026-10-06) · Phase 1 implemented · Phase 3a implemented (Metal) · spec v3 anchor modes (2026-10-07) · Amendment 1 (lifted content, 2026-10-10) accepted · Amendment 2 (one policy, runtime draws, app chooses the look, 2026-10-11) accepted, §B built on Metal, Vulkan and GL (spec v5) · spec:
**Status:** Proposed (2026-10-06) · Phase 1 implemented · Phase 3a implemented (Metal) · spec v3 anchor modes (2026-10-07) · Amendment 1 (lifted content, 2026-10-10) accepted · Amendment 2 (one policy, runtime draws, app chooses the look, 2026-10-11) accepted, §B built on Metal, Vulkan, GL, D3D11 and D3D12 (spec v5) · spec:
[XR_DXR_cursor_depth.md](../specs/extensions/XR_DXR_cursor_depth.md) · sibling of
[ADR-040](ADR-040-rear-depth-budget.md) (the same cue conflict, at the cursor instead of the desktop)

Expand Down Expand Up @@ -605,7 +605,7 @@ sprite and the depth, and the runtime owns the drawing. Apps get the same contra
metres per point (pixel pitch × backing scale). Off the main thread the last snapshot is
kept, else the classic arrow (12 × 19, white fill, dark outline, rasterised procedurally,
4 × 4 supersampled). `DXR_CURSOR_DEFAULT_LOOK=arrow` forces the arrow.
- **Not yet:** D3D11, D3D12 and the service (`svc_draw_cursor_quads`) report
- **Not yet:** the service (`svc_draw_cursor_quads`) reports
`willDraw = XR_FALSE`; display zones (since drawn, runtime#1949, below); multi-screen routed frames draw per tile without the
segment routing on Metal (Vulkan and GL skip them); the 0.5 px floor of A.3 is not applied
(−0.005 is above it on current panels).
Expand Down Expand Up @@ -690,11 +690,63 @@ sprite and the depth, and the runtime owns the drawing. Apps get the same contra
neighbouring 3D zone. `u_cursor_draw_canvas_px_h()` sizes the sprite against the zone's height
(the canvas its unit was built for). A non-zone request is untouched (clip = whole tile).
Zero cost unchanged: all of it runs only on a requested frame.
- **D3D11 / D3D12** (draft runtime#1944) adopt the same two helpers at their draw site.
- **D3D11 / D3D12** (runtime#1944) use the same two helpers in their resolve step (`u_cursor_draw_into_zone` with the window dims the renderer scales `zone_3d.rect` by; the scissor is tile box ∩ the returned clip), skipping only on a zones-frame / zone-request mismatch.
- **Reference app.** `cube_zones_{metal,vk}_macos` adopt `dxr::CursorDepth` via
`test_apps/common/cursor_depth_zones.h`: pick the topmost 3D zone under the pointer, hand the
helper the pointer in that zone's UV, chain the hint only on that zone's locate.

#### §B as built on D3D11 and D3D12 (in-process; `XR_DXR_cursor_depth` spec v5)

- **Same contract as Metal.** The state tracker hands `comp_d3d11/12_compositor_set_cursor_draw_request`
the same one-frame `u_cursor_draw_request`; the commit consumes it with one field test. A cursor
frame never zero-copies. The compositor resolves the request into per-tile corners
(`u_cursor_draw_corners`) — sprite height in metres ÷ the canvas height in metres (canvas px ×
the panel's pixel pitch from the window metrics) — and hands them to its renderer before the
projection pass. The renderer draws them **last in the window-space pass**: after the
projection, quad and window-space layers, before the compose publish and (D3D12) the atlas's
final barrier, so before the display processor. One alpha-blended triangle strip per tile
(straight for the app's sprite, premultiplied for the default look), depth off (no DSV), no
culling, scissored to the tile (∩ the zone in a display-zones frame — see the zones section
above). Multi-screen routed frames draw nothing (one-time WARN). `willDraw` is gated in the
state tracker (`u_cursor_draw_placement_drawable`, runtime#1951) for every backend alike.
- **Shaders.** One shared HLSL pair, `d3d_shared/comp_cursor_shaders.h` (corners in b0 — a
D3D11 constant buffer, 16 D3D12 root constants on the existing quad root signature), compiled
with `D3DCompile` on the first cursor frame only, like every other in-process shader here. A
`DXR_LAYERED` variant samples array layer 0 of a layered sprite. D3D12 builds six pipelines
(`[straight | straight layered | premultiplied][_SRGB target]`), since a D3D12 PSO bakes its
blend and RTV format.
- **Colour (ADR-044).** The cursor takes the frame's treatment: on a composing frame it is drawn
into the private `_SRGB` compose target through the source's format-honest view (decoding), on
a fast-path frame straight into the encoded atlas through the non-decoding view. A
non-`_SRGB` app sprite (scene-linear, owes the encode) makes the frame compose; the cursor is
not counted as a contributing layer, so a fast-path frame blends the sprite's anti-aliased
edges in encoded space, as on Metal. The default look is uploaded once per shape into a
TYPELESS texture with a UNORM and an `_SRGB` view, so the white arrow is white either way.
- **Default look (A.6) — the shared `comp_cursor_look`** (the same entry point Metal, Vulkan and
GL use), whose Windows half is `comp_cursor_look_win.c`: `GetCursorInfo()` → `HCURSOR`, drawn
with `DrawIconEx` into a 32-bit DIB **twice, over black and over white**, and recovered by
`u_cursor_draw_unblend` (alpha = 1 − (white − black); the render over black is the
premultiplied colour). That reads every cursor kind the same way — 32-bit alpha, colour + AND
mask, monochrome AND/XOR — and turns XOR (invert) pixels into opaque white. Drawn at the size
the OS draws it on this panel: `GetSystemMetricsForDpi(SM_CYCURSOR, dpi)` with the window's
`GetDpiForWindow` (passed as the look's backing scale, DPI / 96), or the image's own size when
larger (accessibility pointer sizes); its `height_pt` is that size in 96-DPI points, so the
physical size is `height_pt` × DPI / 96 × the panel's pixel pitch. Cached by `HCURSOR` + scale,
so it re-rasterises only on a shape change. `GetCursorInfo` is global, so the compositor thread
may call it. While the app has hidden its cursor with `SetCursor(NULL)` — which it does as soon
as a locate reports `willDraw`, i.e. on most drawn frames — the shape is the **class cursor of
the window under the pointer**, else the last shape, else `IDC_ARROW`. With
`DXR_CURSOR_DEFAULT_LOOK=arrow`, or if the cursor cannot be read: the shared classic arrow.
- **App sprite.** Straight alpha, last released image, hotspot as given; `sizeMeters` ≤ 0 = 19 px
at the window's DPI × pixel pitch (the classic arrow's size, as Metal's 19 pt). D3D12 dips the
image RENDER_TARGET → PIXEL_SHADER_RESOURCE → RENDER_TARGET around the draw, as for
window-space layers.
- **Reference apps.** `cube_handle_d3d11_win` / `cube_handle_d3d12_win` adopt v5 the way
`cube_handle_metal_macos` does (analytic cube hit test → hint; `DISPLAYXR_CURSOR_DEPTH_DRAW`,
`_SPRITE=custom`, `_UV`, `DISPLAYXR_CURSOR_DEPTH=0`), hiding the OS cursor over the client
area exactly while `willDraw` (`WM_SETCURSOR` → `SetCursor(NULL)`). Their `app` mode keeps the
OS cursor and logs the placement; it draws no in-app crosshair.

### Rollout

1. **Groundwork:**
Expand Down
2 changes: 1 addition & 1 deletion docs/roadmap/control-panel-performance-settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -350,7 +350,7 @@ library linked into both · **CLI** = `displayxr-cli.exe` (reporting only, contr
| `DXR_WEAVE_REPAINT_TRACE` | `auxiliary/util/u_repaint_gate.h` (`u_repaint_trace_*`, wired into the vk_native + d3d11 repaint loops) | `getenv`, cached | off | App | 4 | #1257 verification instrumentation: one WARN row per ~5 s per loop with real tick cadence, replay/pace durations, and per-gate bail counts. Says where missing repaints went; the witness only says how many landed |
| `DXR_APP_FRAME_DIVISOR` | `auxiliary/util/u_app_partition.h` (`u_app_partition_throttle` in each in-process wait_frame; engagement read by the repaint gate + loop tick) | `getenv`, cached | off (1) | App | 2 | #1257 slot partition: xrWaitFrame releases the app every Dth vblank (D=2..8) and the repaint loop fills the other slots at panel rate with a KNOWN N=D schedule — steady output cadence, no fire/commit collision by construction. **Supported tier: the #918 split (d3d11 bridge / hybrid) only** — verified steady 60 with eyeball sign-off; in-process vk_native/d3d12/gl tiers REFUSE cleanly (fill-loop tick starvation, 17-19 ms vs 1.8-2.3 ms on the bridge; follow-up). The natural Control Panel "smooth motion, lower app rate" control and the hook for face-tracking-driven rate later. Not wired into Metal or the IPC/service path |
| `DXR_APP_FRAME_DIVISOR_ANY_TIER` | `auxiliary/util/u_app_partition.h` (`u_app_partition_throttle` tier gate) | `getenv`, cached | off | App | 4 | Bring-up override: lets the partition throttle engage on an UNSUPPORTED tier (which collapses the panel there today). For the in-process-tier follow-up work only |
| `DXR_CURSOR_DEFAULT_LOOK` | `compositor/util/comp_cursor_look.c` (`force_arrow`, shared by the Metal, vk_native and GL compositors) | `getenv`, `static` cached | unset (mirror the OS cursor on macOS; the arrow elsewhere) | App | 4 | `arrow` forces the classic-arrow fallback for the runtime-drawn depth cursor's default look (XR_DXR_cursor_depth v5, ADR-046 Amendment 2 B) instead of mirroring `[NSCursor currentCursor]`. Verification only |
| `DXR_CURSOR_DEFAULT_LOOK` | `compositor/util/comp_cursor_look.c` (`force_arrow`, shared by the Metal, vk_native, GL, D3D11 and D3D12 compositors) | `getenv`, `static` cached | unset (mirror the OS cursor on macOS and Windows; the arrow elsewhere) | App | 4 | `arrow` forces the classic-arrow fallback for the runtime-drawn depth cursor's default look (XR_DXR_cursor_depth v5, ADR-046 Amendment 2 B) instead of mirroring the OS cursor (`[NSCursor currentCursor]` on macOS, `GetCursorInfo` on Windows). Verification only |
| `DXR_DP_FORWARD_HORIZON` | `compositor/util/comp_weave_latency_win.h` (`predict_weave_to_scanout_ns`, feeds the DP `set_predicted_scanout` slot from the split + d3d11 + d3d12 weave paths) | `getenv`, cached | **on** (kill switch) | App | 1 | #206: per-weave FORWARD horizon for the vendor eye predictor, computed from the vsync-locked vblank grid (DXGI frame statistics — measured period, not nominal). `0` disables → the DP falls back to its retrospective smoothed heuristic, the pre-#206 behavior |
| `DXR_DP_FORWARD_HORIZON_LOOP` | `compositor/util/comp_weave_latency_win.h` (`predict_weave_to_scanout_ns` probes it; `po_observe`, fed from `after_present`'s resolve pass, learns the value) | `getenv`, cached | **on** (kill switch) | App | 1 | #1435: learns the pipeline's whole-period offset (composition frame on DComp chains, queued frame at governor depth ≥ 2) from realised flips — mode of the last 32 resolved weaves, ≥ 24 agreeing — and adds it to the forward horizon. `0` pins the offset at 0, the pre-#1435 feed |
| `DXR_FILL_FENCE_PARK` | `compositor/vk_native/comp_vk_native_compositor.c` (fill path of `vk_dp_weave_and_present`) | `getenv`, cached | **on** (kill switch) | App | 1 | #1264 S1: a fill's GPU fence wait releases the compositor lock (re-validated on retake; superseded fills are dropped). `0` restores the lock-held synchronous wait |
Expand Down
8 changes: 5 additions & 3 deletions docs/specs/extensions/XR_DXR_cursor_depth.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,12 +192,14 @@ the runtime cannot draw (another backend, a space mismatch, a zones frame withou
`XR_FALSE`, so the fallback is always the flat OS cursor, never no cursor. The runtime never
touches the OS cursor itself.

**Backends (v5).** The in-process Metal, Vulkan (vk_native) and OpenGL compositors draw. D3D11,
D3D12 and the IPC/service path report `willDraw = XR_FALSE` with a one-time WARN (ADR-046
**Backends (v5).** The in-process Metal, Vulkan (vk_native), OpenGL, D3D11 and D3D12 compositors
draw. The IPC/service path reports `willDraw = XR_FALSE` with a one-time WARN (ADR-046
Amendment 2 rollout step 3). On Vulkan the cursor is drawn after the projection, quad and
equirect2 layers but **under** window-space layers, which that compositor stamps in a later pass
(ADR-046 Amendment 2, "§B as built on Vulkan and GL"). The default look mirrors the OS cursor on
macOS only; elsewhere it is the classic arrow.
macOS and Windows (on Windows `GetCursorInfo`, or - while the app has hidden it - the class
cursor of the window under the pointer, drawn at the size the OS draws it on the panel);
elsewhere it is the classic arrow.

### 2.7 `XrCursorDepthLiftRectDXR` / `XrCursorDepthLiftStateDXR` (spec v6) — lifted content

Expand Down
32 changes: 32 additions & 0 deletions src/xrt/auxiliary/util/u_cursor_depth.c
Original file line number Diff line number Diff line change
Expand Up @@ -750,3 +750,35 @@ u_cursor_draw_arrow_rgba(uint8_t *rgba, uint32_t w, uint32_t h)
}
}
}

void
u_cursor_draw_unblend(const uint8_t *on_black, const uint8_t *on_white, uint32_t pixel_count, uint8_t *out)
{
if (on_black == NULL || on_white == NULL || out == NULL) {
return;
}
for (uint32_t i = 0; i < pixel_count; i++) {
const uint8_t *b = on_black + (size_t)i * 4;
const uint8_t *w = on_white + (size_t)i * 4;
uint8_t *o = out + (size_t)i * 4;
// How much of the background shows through, summed over 3 channels.
// Negative where the cursor inverts (w < b): that is fully opaque.
int through = 0;
for (int c = 0; c < 3; c++) {
through += (int)w[c] - (int)b[c];
}
int a = 255 - (through + 1) / 3;
if (a > 255) {
a = 255;
}
if (a <= 0) {
o[0] = o[1] = o[2] = o[3] = 0;
continue;
}
// Over black, the render IS colour x coverage: premultiplied already.
for (int c = 0; c < 3; c++) {
o[c] = b[c] > a ? (uint8_t)a : b[c];
}
o[3] = (uint8_t)a;
}
}
17 changes: 17 additions & 0 deletions src/xrt/auxiliary/util/u_cursor_depth.h
Original file line number Diff line number Diff line change
Expand Up @@ -622,6 +622,23 @@ u_cursor_draw_canvas_px_h(const struct u_cursor_draw_request *req, uint32_t canv
void
u_cursor_draw_arrow_rgba(uint8_t *rgba, uint32_t w, uint32_t h);

/*!
* Recover a cursor image from two renders of it: one over opaque BLACK, one
* over opaque WHITE. This is how the Windows default look reads ANY HCURSOR -
* 32-bit alpha, colour + AND mask, or monochrome AND/XOR - without branching
* on its type: whatever the OS draws, the two backgrounds reveal it.
*
* Per pixel, with b and w the two renders (bytes 0..2 colour, byte 3 ignored):
* alpha = 1 - mean(w - b), and the render over black IS the premultiplied
* colour (clamped to alpha). A pixel the cursor INVERTS (white over black,
* black over white - the I-beam's XOR pixels) comes out opaque WHITE: a
* compositor cannot invert what is behind it before the weave, and white is
* the classic arrow's fill. Output: PREMULTIPLIED alpha (as the classic arrow
* and comp_cursor_look), channels in the input's order.
*/
void
u_cursor_draw_unblend(const uint8_t *on_black, const uint8_t *on_white, uint32_t pixel_count, uint8_t *out);

#ifdef __cplusplus
}
#endif
5 changes: 5 additions & 0 deletions src/xrt/compositor/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -268,6 +268,11 @@ target_include_directories(comp_cursor_look PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
if(APPLE)
target_sources(comp_cursor_look PRIVATE util/comp_cursor_look_macos.m)
target_link_libraries(comp_cursor_look PRIVATE "-framework AppKit")
elseif(WIN32)
# The Windows OS-cursor mirror (GetCursorInfo / DrawIconEx into a DIB),
# shared by the D3D11 and D3D12 compositors.
target_sources(comp_cursor_look PRIVATE util/comp_cursor_look_win.c)
target_link_libraries(comp_cursor_look PRIVATE user32 gdi32)
endif()

###
Expand Down
2 changes: 2 additions & 0 deletions src/xrt/compositor/d3d11/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,8 @@ if(WIN32 AND XRT_HAVE_D3D11)
aux_d3d
comp_xbridge
comp_d3d11_segs
# XR_DXR_cursor_depth v5: the runtime-drawn cursor's default look
comp_cursor_look
xrt-external-stb
d3d11
d3d12
Expand Down
Loading
Loading