diff --git a/docs/adr/ADR-046-depth-aware-cursor.md b/docs/adr/ADR-046-depth-aware-cursor.md index 530133115a..6862a0825a 100644 --- a/docs/adr/ADR-046-depth-aware-cursor.md +++ b/docs/adr/ADR-046-depth-aware-cursor.md @@ -1,6 +1,6 @@ # ADR-046: Depth-aware cursor — opt-in only; the app knows the depth, the runtime places the cursor -**Status:** Proposed (2026-10-06) · Phase 1 implemented · Phase 3a implemented (Metal) · spec v3 anchor modes (2026-10-07) · Amendment 1 (lifted content, 2026-10-10) accepted · Amendment 2 (one policy, runtime draws, app chooses the look, 2026-10-11) accepted, §B built on Metal (spec v5) · spec: +**Status:** Proposed (2026-10-06) · Phase 1 implemented · Phase 3a implemented (Metal) · spec v3 anchor modes (2026-10-07) · Amendment 1 (lifted content, 2026-10-10) accepted · Amendment 2 (one policy, runtime draws, app chooses the look, 2026-10-11) accepted, §B built on Metal, Vulkan and GL (spec v5) · spec: [XR_DXR_cursor_depth.md](../specs/extensions/XR_DXR_cursor_depth.md) · sibling of [ADR-040](ADR-040-rear-depth-budget.md) (the same cue conflict, at the cursor instead of the desktop) @@ -605,9 +605,69 @@ sprite and the depth, and the runtime owns the drawing. Apps get the same contra metres per point (pixel pitch × backing scale). Off the main thread the last snapshot is kept, else the classic arrow (12 × 19, white fill, dark outline, rasterised procedurally, 4 × 4 supersampled). `DXR_CURSOR_DEFAULT_LOOK=arrow` forces the arrow. -- **Not yet:** D3D11, D3D12, GL, vk_native and the service (`svc_draw_cursor_quads`) report +- **Not yet:** D3D11, D3D12 and the service (`svc_draw_cursor_quads`) report `willDraw = XR_FALSE`; display zones; multi-screen routed frames draw per tile without the - segment routing; the 0.5 px floor of A.3 is not applied (−0.005 is above it on current panels). + segment routing on Metal (Vulkan and GL skip them); the 0.5 px floor of A.3 is not applied + (−0.005 is above it on current panels). + +#### §B as built on Vulkan and GL (`XR_DXR_cursor_depth` spec v5, unchanged) + +- **One default look for every backend.** The Metal compositor's default-look code moved to + `compositor/util/comp_cursor_look.{h,c}` (+ `comp_cursor_look_macos.m`), a small library the + Metal, vk_native and GL compositors all link: it picks the look (mirrored `[NSCursor + currentCursor]` on the main thread, else the last snapshot, else the procedural arrow from + `u_cursor_draw_arrow_rgba()`), rasterises it once per change into premultiplied, sRGB-encoded + RGBA8, and reports aspect, hotspot and size in points plus the main screen's backing scale. + Each compositor only uploads the pixels when the look's `generation` moves. + `DXR_CURSOR_DEFAULT_LOOK=arrow` forces the arrow on all three. **OS-cursor mirroring is + macOS-only:** Linux (`XFixesGetCursorImage`) and Windows (`GetCursorInfo` + `GetIconInfo`) + draw the arrow; mirroring there is future work. +- **State tracker.** Additive only: `oxr_session_cursor_draw_supported()` / + `_hand_request()` gained the `is_vk_native_compositor` and `is_gl_native_compositor` arms. + The placement, the per-view map and the zero-cost gate are the Metal ones. +- **vk_native.** The request is consumed by the next commit (one field test otherwise) and + resolved there into tile-normalised corners per view (`u_cursor_draw_corners`). A cursor frame + never zero-copies and always takes the renderer's **compose render pass** (the blit fast path + cannot blend): the cursor is the pass's last draw, one quad per view tile through the pass's + own pipelines (the QUAD vertex path with an affine MVP built from three corners; straight or + premultiplied blend), scissored to the tile, depth off. Because the pass renders into the + private `_SRGB` compose target and publishes to the atlas with a raw copy, the sprite blends + in linear light, an `_SRGB` sprite is decoded on sample (the same single source-view choice as + every layer, `compose_source_view()`), and the atlas stays ENCODED — never double-encoded. The + default look is an `R8G8B8A8_SRGB` image (UNORM under the legacy colour hatch) uploaded + through a staging buffer inside the same command buffer. Nothing is created before the first + request; the descriptor pool carries one extra set for the cursor. **Ordering differs from + Metal:** vk_native stamps window-space layers in a later, weave-time pass (several call sites, + some on the async weave hand-off), so the cursor is drawn over projection, quad and equirect2 + layers but **under** window-space layers. Both the projection-only and the post-compose + `$TMPDIR/displayxr_atlas_trigger` captures show it. +- **GL.** Drawn after the window-space layers, before the DP, into whatever the atlas pass is + painting: the private `GL_SRGB8_ALPHA8` compose target with `GL_FRAMEBUFFER_SRGB` on, or the + ENCODED atlas directly. The source is bound through `gl_bind_layer_source()` (decode iff + composing), the same rule as every layer; a cursor frame counts as one more sub-rect layer in + `gl_frame_takes_fast_path()` (a scene-linear UNORM sprite forces the compose path when it + exists). A small program (corners as uniforms, `gl_VertexID` strip), per-tile viewport + + scissor, blend state restored. A GL sprite is sampled bottom-up (its top is its last row — + the Metal compositor's GL-source flip). A layered (`arraySize > 1`) GL sprite is not drawn + yet (one-time WARN). On Apple's GL 4.1 there is no `glCopyImageSubData`, so every macOS GL + frame composes in encoded space, cursor included; the linear compose-target path was not run + here. +- **Measured** (macOS, sim-display Cropped SBS, cube reference apps at `cursorUV` 0.5,0.5 over + the cube, `DXR_CURSOR_DEFAULT_LOOK=arrow`): GL −23.6 px in the 1512 × 1646 atlas (Metal −24.0 + on the same app geometry), vk_native −22.0 px in its 1280 × 1440 atlas; nothing drawn at + 0.1,0.1; the arrow's fill reads (255,255,255) and its outline (5,5,8), the custom ring + (0,220,255) — exactly the authored bytes, so no encode is lost or doubled. +- **Known issue (GL on macOS, not caused by the draw):** `cube_handle_gl_macos` crashes in + roughly 1 of 5 launches once the depth cursor is ACTIVE — in Apple's GL driver, on the #868 + repaint thread (`gl_repaint_thread` → `glBlitFramebuffer`) or in the app's own draw. It + follows the app's `[NSCursor hide]`, not the runtime draw: app-draw mode crashes too; the + runtime draw with the OS cursor left visible (0/10), `DXR_WEAVE_REPAINT=0` (0/8) and the + extension off (0/20) do not. The GL cube never hid the OS cursor before this adoption, so this + is a latent repaint-thread / AppKit race in the GL compositor surfacing — runtime#1947. Until it + is fixed the GL cube's depth cursor is **opt-in** (`DISPLAYXR_CURSOR_DEPTH=1`); the Metal and + Vulkan cubes stay default-on. +- **Found in passing (Metal, #1941):** on the Vulkan-app-on-Metal fallback a custom sprite is + handed over as the client swapchain and cast to the native Metal type — runtime#1948. ### Rollout diff --git a/docs/roadmap/control-panel-performance-settings.md b/docs/roadmap/control-panel-performance-settings.md index 793038b7b3..b0e6f88b8a 100644 --- a/docs/roadmap/control-panel-performance-settings.md +++ b/docs/roadmap/control-panel-performance-settings.md @@ -350,7 +350,7 @@ library linked into both · **CLI** = `displayxr-cli.exe` (reporting only, contr | `DXR_WEAVE_REPAINT_TRACE` | `auxiliary/util/u_repaint_gate.h` (`u_repaint_trace_*`, wired into the vk_native + d3d11 repaint loops) | `getenv`, cached | off | App | 4 | #1257 verification instrumentation: one WARN row per ~5 s per loop with real tick cadence, replay/pace durations, and per-gate bail counts. Says where missing repaints went; the witness only says how many landed | | `DXR_APP_FRAME_DIVISOR` | `auxiliary/util/u_app_partition.h` (`u_app_partition_throttle` in each in-process wait_frame; engagement read by the repaint gate + loop tick) | `getenv`, cached | off (1) | App | 2 | #1257 slot partition: xrWaitFrame releases the app every Dth vblank (D=2..8) and the repaint loop fills the other slots at panel rate with a KNOWN N=D schedule — steady output cadence, no fire/commit collision by construction. **Supported tier: the #918 split (d3d11 bridge / hybrid) only** — verified steady 60 with eyeball sign-off; in-process vk_native/d3d12/gl tiers REFUSE cleanly (fill-loop tick starvation, 17-19 ms vs 1.8-2.3 ms on the bridge; follow-up). The natural Control Panel "smooth motion, lower app rate" control and the hook for face-tracking-driven rate later. Not wired into Metal or the IPC/service path | | `DXR_APP_FRAME_DIVISOR_ANY_TIER` | `auxiliary/util/u_app_partition.h` (`u_app_partition_throttle` tier gate) | `getenv`, cached | off | App | 4 | Bring-up override: lets the partition throttle engage on an UNSUPPORTED tier (which collapses the panel there today). For the in-process-tier follow-up work only | -| `DXR_CURSOR_DEFAULT_LOOK` | `compositor/metal/comp_metal_compositor.m` (`metal_cursor_update_default`) | `getenv`, `static` cached | unset (mirror the OS cursor) | App | 4 | `arrow` forces the classic-arrow fallback for the runtime-drawn depth cursor's default look (XR_DXR_cursor_depth v5, ADR-046 Amendment 2 B) instead of mirroring `[NSCursor currentCursor]`. Verification only; Metal only today | +| `DXR_CURSOR_DEFAULT_LOOK` | `compositor/util/comp_cursor_look.c` (`force_arrow`, shared by the Metal, vk_native and GL compositors) | `getenv`, `static` cached | unset (mirror the OS cursor on macOS; the arrow elsewhere) | App | 4 | `arrow` forces the classic-arrow fallback for the runtime-drawn depth cursor's default look (XR_DXR_cursor_depth v5, ADR-046 Amendment 2 B) instead of mirroring `[NSCursor currentCursor]`. Verification only | | `DXR_DP_FORWARD_HORIZON` | `compositor/util/comp_weave_latency_win.h` (`predict_weave_to_scanout_ns`, feeds the DP `set_predicted_scanout` slot from the split + d3d11 + d3d12 weave paths) | `getenv`, cached | **on** (kill switch) | App | 1 | #206: per-weave FORWARD horizon for the vendor eye predictor, computed from the vsync-locked vblank grid (DXGI frame statistics — measured period, not nominal). `0` disables → the DP falls back to its retrospective smoothed heuristic, the pre-#206 behavior | | `DXR_DP_FORWARD_HORIZON_LOOP` | `compositor/util/comp_weave_latency_win.h` (`predict_weave_to_scanout_ns` probes it; `po_observe`, fed from `after_present`'s resolve pass, learns the value) | `getenv`, cached | **on** (kill switch) | App | 1 | #1435: learns the pipeline's whole-period offset (composition frame on DComp chains, queued frame at governor depth ≥ 2) from realised flips — mode of the last 32 resolved weaves, ≥ 24 agreeing — and adds it to the forward horizon. `0` pins the offset at 0, the pre-#1435 feed | | `DXR_FILL_FENCE_PARK` | `compositor/vk_native/comp_vk_native_compositor.c` (fill path of `vk_dp_weave_and_present`) | `getenv`, cached | **on** (kill switch) | App | 1 | #1264 S1: a fill's GPU fence wait releases the compositor lock (re-validated on retake; superseded fills are dropped). `0` restores the lock-held synchronous wait | diff --git a/docs/specs/extensions/XR_DXR_cursor_depth.md b/docs/specs/extensions/XR_DXR_cursor_depth.md index 09cca94632..51fccee69e 100644 --- a/docs/specs/extensions/XR_DXR_cursor_depth.md +++ b/docs/specs/extensions/XR_DXR_cursor_depth.md @@ -153,7 +153,7 @@ chain it, for diagnostics). | Field | Meaning | |---|---| -| `image` | An ordinary color swapchain this session created with `xrCreateSwapchain`, in the app's own graphics API, filled with any sprite and released at least once. The runtime samples its **last released image** (array layer 0, whole image) as **straight (non-premultiplied) alpha**, decoded per its format (an `_SRGB` format decodes). It must stay alive until the `xrEndFrame` of every frame that names it. `XR_NULL_HANDLE` = the runtime's **default look**: the current OS cursor's shape where the runtime can read it, else the classic arrow (12 × 19, white fill, dark outline, tip = hotspot). | +| `image` | An ordinary color swapchain this session created with `xrCreateSwapchain`, in the app's own graphics API, filled with any sprite and released at least once. The runtime samples its **last released image** (array layer 0, whole image) as **straight (non-premultiplied) alpha**, decoded per its format (an `_SRGB` format decodes). It must stay alive until the `xrEndFrame` of every frame that names it. On OpenGL the image's top is its **last** row (GL's bottom-left origin, which is how a GL app renders into it); upload a top-down bitmap bottom row first. `XR_NULL_HANDLE` = the runtime's **default look**: the current OS cursor's shape where the runtime can read it, else the classic arrow (12 × 19, white fill, dark outline, tip = hotspot). | | `hotSpot` | The click point in the sprite, normalised, origin top-left, v down. Placed at the cursor. Ignored for the default look (it brings its own). | | `sizeMeters` | The sprite's **height** on the panel at the display plane, metres; width follows the image aspect. Scaled with depth, so its apparent size never changes. ≤ 0 or non-finite = the OS cursor's size (the default look's own size; 19 OS points for a custom sprite). | | `visible` | `XR_FALSE`: draw nothing. | @@ -175,8 +175,12 @@ the runtime cannot draw (another backend, a space mismatch, a zones frame), it r `XR_FALSE`, so the fallback is always the flat OS cursor, never no cursor. The runtime never touches the OS cursor itself. -**Backends (v5).** Metal (in-process) draws. D3D11, D3D12, GL, vk_native and the IPC/service path -report `willDraw = XR_FALSE` with a one-time WARN (ADR-046 Amendment 2 rollout step 3). +**Backends (v5).** The in-process Metal, Vulkan (vk_native) and OpenGL compositors draw. D3D11, +D3D12 and the IPC/service path report `willDraw = XR_FALSE` with a one-time WARN (ADR-046 +Amendment 2 rollout step 3). On Vulkan the cursor is drawn after the projection, quad and +equirect2 layers but **under** window-space layers, which that compositor stamps in a later pass +(ADR-046 Amendment 2, "§B as built on Vulkan and GL"). The default look mirrors the OS cursor on +macOS only; elsewhere it is the classic arrow. ### 2.7 `XrCursorDepthLiftRectDXR` / `XrCursorDepthLiftStateDXR` (spec v6) — lifted content diff --git a/src/xrt/compositor/CMakeLists.txt b/src/xrt/compositor/CMakeLists.txt index a595e01b88..5407030165 100644 --- a/src/xrt/compositor/CMakeLists.txt +++ b/src/xrt/compositor/CMakeLists.txt @@ -255,6 +255,21 @@ if(APPLE) target_include_directories(comp_macos_window PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}) endif() +### +# XR_DXR_cursor_depth v5 (ADR-046 Amendment 2 B): the runtime-drawn cursor's +# default look (the mirrored OS cursor, else the classic arrow), shared by the +# Metal, Vulkan and GL compositors so they cannot drift apart on it. Its own +# tiny library rather than part of comp_util: comp_gl does not link comp_util +# (which is Vulkan-gated), and none of this needs a graphics API. +# +add_library(comp_cursor_look STATIC util/comp_cursor_look.c util/comp_cursor_look.h) +target_link_libraries(comp_cursor_look PUBLIC xrt-interfaces PRIVATE aux_util) +target_include_directories(comp_cursor_look PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}) +if(APPLE) + target_sources(comp_cursor_look PRIVATE util/comp_cursor_look_macos.m) + target_link_libraries(comp_cursor_look PRIVATE "-framework AppKit") +endif() + ### # Shared desktop-Linux service window (#710, #967): the runtime-owned X11 # surface (Xorg or XWayland) the service's comp_multi shared spatial surface diff --git a/src/xrt/compositor/gl/CMakeLists.txt b/src/xrt/compositor/gl/CMakeLists.txt index b0ab38ff72..9c91cb4d94 100644 --- a/src/xrt/compositor/gl/CMakeLists.txt +++ b/src/xrt/compositor/gl/CMakeLists.txt @@ -26,6 +26,8 @@ target_link_libraries( aux_math aux_ogl xrt-external-stb + # XR_DXR_cursor_depth v5: the runtime-drawn cursor's default look. + comp_cursor_look ) target_include_directories(comp_gl PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/..) diff --git a/src/xrt/compositor/gl/comp_gl_compositor.cpp b/src/xrt/compositor/gl/comp_gl_compositor.cpp index 432b620555..450a99af51 100644 --- a/src/xrt/compositor/gl/comp_gl_compositor.cpp +++ b/src/xrt/compositor/gl/comp_gl_compositor.cpp @@ -16,6 +16,7 @@ #include "d3d11/comp_d3d11_window.h" #endif +#include "util/comp_cursor_look.h" #include "util/comp_layer_accum.h" // Multi-screen #1883: the painted-partition type the repaint snapshot carries // (declarations only; the segment path itself is Windows-only). @@ -64,6 +65,7 @@ #include "util/u_tiling.h" #include "util/u_canvas.h" #include "util/u_capture_intent.h" +#include "util/u_cursor_depth.h" #include "util/u_repaint_gate.h" #include "util/u_fill_thread_win.h" #include "util/u_image_capture.h" @@ -723,6 +725,38 @@ struct comp_gl_compositor GLuint program_blit; //!< Shader for blitting eye to atlas texture GLuint program_blit_array; //!< Blit shader for LAYERED (array) swapchains (sampler2DArray) GLuint program_window_space; //!< Window-space layer (positioned quad) + + /*! + * XR_DXR_cursor_depth v5, the runtime draws the cursor (ADR-046 + * Amendment 2 B). + * + * ZERO COST UNLESS REQUESTED: all zero for a session that never chained + * XrCursorDepthDrawInfoDXR. layer_commit tests `req.requested` and does + * nothing else; the program and the default-look texture are built on + * the first real request only. + */ + struct + { + //! One-frame request from the state tracker, consumed by the next commit. + struct u_cursor_draw_request req; + //! This commit carries a cursor (read by gl_frame_takes_fast_path). + bool pending; + //! ...and its source is display-referred (an sRGB sprite or the + //! default look), so a fast-path frame may draw it undecoded. + bool pending_source_srgb; + GLuint program; + bool program_failed; + GLint loc_corners; + GLint loc_tex; + //! The default look's sprite (premultiplied, sRGB-encoded bytes). + GLuint default_tex; + uint64_t default_gen; + bool default_uploaded; + //! The shared default look (mirrored OS cursor, else the arrow). + struct comp_cursor_look look; + bool announced; + bool warned_array; + } cursor_draw; //! #1581 XrCompositionLayerQuad — world-placed quad through the per-view //! camera. Straight vs premultiplied alpha is blend STATE in GL, so one //! program covers both; the `_array` twin only exists for arraySize>1 @@ -5899,6 +5933,16 @@ gl_frame_takes_fast_path(struct comp_gl_compositor *c) } } + // XR_DXR_cursor_depth v5: the runtime-drawn cursor is one more sub-rect + // layer blended over the frame, so it is a compose case like a quad - + // and a scene-linear (UNORM) sprite owes the encode. + if (c->cursor_draw.pending) { + contributing++; + if (!c->cursor_draw.pending_source_srgb) { + all_sources_srgb = false; + } + } + if (c->legacy_color) { return true; } @@ -5906,6 +5950,254 @@ gl_frame_takes_fast_path(struct comp_gl_compositor *c) gl_has_srgb_decode_ext(); } +/* + * + * XR_DXR_cursor_depth v5 - the runtime draws the cursor (ADR-046 Amendment 2 B) + * + * After the app's layers are composited into the atlas (projection, quad, + * window-space) and before the display processor, one sprite per view: a + * textured, alpha-blended triangle strip, depth test off, scissored to the + * tile, at the corners the state tracker's per-view map gives. It draws into + * whatever the atlas pass is painting - the private GL_SRGB8_ALPHA8 compose + * target with GL_FRAMEBUFFER_SRGB on (linear blend, sources decoded), or the + * ENCODED atlas directly on the fast path (sources read undecoded) - and + * binds its source through gl_bind_layer_source(), the same rule as every + * layer, so the atlas is never double-encoded. Built on the first request. + * + */ + +static const char *VS_CURSOR = + "#version 330 core\n" + "// corners[k]: xy = NDC, zw = texture uv; a 4-vertex triangle strip.\n" + "uniform vec4 u_corners[4];\n" + "out vec2 v_uv;\n" + "void main() {\n" + " vec4 c = u_corners[gl_VertexID];\n" + " v_uv = c.zw;\n" + " gl_Position = vec4(c.xy, 0.0, 1.0);\n" + "}\n"; + +static const char *FS_CURSOR = + "#version 330 core\n" + "in vec2 v_uv;\n" + "out vec4 fragColor;\n" + "uniform sampler2D u_texture;\n" + "void main() {\n" + " fragColor = texture(u_texture, v_uv);\n" + "}\n"; + +static bool +gl_cursor_ensure_program(struct comp_gl_compositor *c) +{ + if (c->cursor_draw.program != 0) { + return true; + } + if (c->cursor_draw.program_failed) { + return false; + } + c->cursor_draw.program = create_program(VS_CURSOR, FS_CURSOR); + if (c->cursor_draw.program == 0) { + c->cursor_draw.program_failed = true; // never retried per frame + U_LOG_E("cursor_draw: GL program build failed - cursor not drawn"); + return false; + } + c->cursor_draw.loc_corners = glGetUniformLocation(c->cursor_draw.program, "u_corners"); + c->cursor_draw.loc_tex = glGetUniformLocation(c->cursor_draw.program, "u_texture"); + return true; +} + +/*! + * The default look (Amendment 2 A.6), shared with the Metal and Vulkan + * compositors (comp_cursor_look). Uploaded only when its image changes, as + * GL_SRGB8_ALPHA8 so gl_bind_layer_source() can read it decoded (compose + * target) or undecoded (fast path); GL_RGBA8 holding the same encoded bytes + * on a stack without GL_EXT_texture_sRGB_decode, where only the latter is + * expressible. Rows go in top row first, so texture t = 0 is the TOP. + */ +static bool +gl_cursor_update_default(struct comp_gl_compositor *c, float backing_scale) +{ + struct comp_cursor_look *look = &c->cursor_draw.look; + if (!comp_cursor_look_update(look, backing_scale)) { + return c->cursor_draw.default_tex != 0; + } + if (c->cursor_draw.default_uploaded && c->cursor_draw.default_gen == look->generation) { + return true; + } + if (c->cursor_draw.default_tex == 0) { + glGenTextures(1, &c->cursor_draw.default_tex); + } + GLint prev_tex = 0; + glGetIntegerv(GL_TEXTURE_BINDING_2D, &prev_tex); + GLint prev_align = 4; + glGetIntegerv(GL_UNPACK_ALIGNMENT, &prev_align); + glBindTexture(GL_TEXTURE_2D, c->cursor_draw.default_tex); + glPixelStorei(GL_UNPACK_ALIGNMENT, 4); + glTexImage2D(GL_TEXTURE_2D, 0, gl_has_srgb_decode_ext() ? GL_SRGB8_ALPHA8 : GL_RGBA8, (GLsizei)look->w, + (GLsizei)look->h, 0, GL_RGBA, GL_UNSIGNED_BYTE, look->rgba); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); + glPixelStorei(GL_UNPACK_ALIGNMENT, prev_align); + glBindTexture(GL_TEXTURE_2D, (GLuint)prev_tex); + c->cursor_draw.default_gen = look->generation; + c->cursor_draw.default_uploaded = true; + return true; +} + +/*! + * Draw the requested cursor into every tile of the bound atlas framebuffer. + * + * @return true if anything was drawn (the caller republishes the compose + * target). + */ +static bool +gl_cursor_draw(struct comp_gl_compositor *c, const struct u_cursor_draw_request *req) +{ + if (!gl_cursor_ensure_program(c)) { + return false; + } + // Panel pixel pitch (metres per backing pixel) and the OS point. + if (!c->sys_info_set || c->sys_info.display_pixel_height == 0 || !(c->sys_info.display_height_m > 0.0f)) { + return false; + } + const float m_per_px = c->sys_info.display_height_m / (float)c->sys_info.display_pixel_height; + const float backing_scale = comp_cursor_look_backing_scale(); + const float m_per_pt = m_per_px * backing_scale; + // The canvas the Kooima views were built for: the window's client area. + uint32_t canvas_px_h = 0; + struct xrt_window_metrics wm; + if (comp_gl_compositor_get_window_metrics(&c->base.base, &wm) && wm.valid) { + canvas_px_h = wm.window_pixel_height; + } + if (canvas_px_h == 0) { + canvas_px_h = c->eff_rows * c->eff_tile_h; + } + if (canvas_px_h == 0) { + return false; + } + const float canvas_h_m = (float)canvas_px_h * m_per_px; + + GLuint tex = 0; + float aspect, hot_u, hot_v, size_m; + bool premul; + bool flip_v; + if (req->image != NULL) { + struct comp_gl_swapchain *gsc = gl_swapchain(req->image); + if (req->image_index >= gsc->image_count || gsc->info.height == 0) { + return false; + } + if (gsc->target != GL_TEXTURE_2D) { + if (!c->cursor_draw.warned_array) { + c->cursor_draw.warned_array = true; + U_LOG_W( + "cursor_draw: a layered (arraySize > 1) GL sprite swapchain is not drawn yet " + "(one-time warning)"); + } + return false; + } + tex = gsc->textures[req->image_index]; + aspect = (float)gsc->info.width / (float)gsc->info.height; + hot_u = req->hot_u; + hot_v = req->hot_v; + size_m = req->size_m > 0.0f ? req->size_m : U_CURSOR_DRAW_DEFAULT_POINTS * m_per_pt; + premul = false; // spec: the app's sprite is STRAIGHT alpha + // A GL app's image is bottom-up (its last row is its top), as GL + // renders it - same rule as the Metal compositor's GL-source flip. + flip_v = true; + } else { + if (!gl_cursor_update_default(c, backing_scale)) { + return false; + } + tex = c->cursor_draw.default_tex; + aspect = c->cursor_draw.look.aspect; + hot_u = c->cursor_draw.look.hot_u; + hot_v = c->cursor_draw.look.hot_v; + size_m = req->size_m > 0.0f ? req->size_m : c->cursor_draw.look.height_pt * m_per_pt; + premul = true; // the look's pixels are premultiplied + flip_v = false; // uploaded top row first + } + const float height = size_m / canvas_h_m; // sprite height, in units + + // Mono content fills the whole content region; otherwise view i owns + // tile i of the effective grid (#542), placed bottom-left-origin (#1625). + const uint32_t cols = c->eff_cols > 0 ? c->eff_cols : 1; + const uint32_t rows = c->eff_rows > 0 ? c->eff_rows : 1; + const bool mono = c->eff_views <= 1; + const uint32_t tiles = mono ? 1 : cols * rows; + const uint32_t n = req->view_count < tiles ? req->view_count : tiles; + if (n == 0 || c->eff_tile_w == 0 || c->eff_tile_h == 0) { + return false; + } + + GLint prev_viewport[4]; + glGetIntegerv(GL_VIEWPORT, prev_viewport); + GLint prev_scissor[4]; + glGetIntegerv(GL_SCISSOR_BOX, prev_scissor); + + glUseProgram(c->cursor_draw.program); + glBindVertexArray(c->vao_empty); + glActiveTexture(GL_TEXTURE0); + gl_bind_layer_source(GL_TEXTURE_2D, tex, c->compose_active); + glUniform1i(c->cursor_draw.loc_tex, 0); + glEnable(GL_BLEND); + glBlendFuncSeparate(premul ? GL_ONE : GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, GL_ONE, GL_ONE_MINUS_SRC_ALPHA); + // The scissor is mandatory: a cursor at a tile edge must not spill into + // the neighbour view (the DP would weave it into the wrong eye). + glEnable(GL_SCISSOR_TEST); + + static const float uv[4][2] = {{0, 0}, {0, 1}, {1, 0}, {1, 1}}; + for (uint32_t i = 0; i < n; i++) { + uint32_t bx = 0, by = 0, bw = cols * c->eff_tile_w, bh = rows * c->eff_tile_h; + if (!mono) { + u_tiling_view_origin_gl(i, cols, rows, c->eff_tile_w, c->eff_tile_h, &bx, &by); + bw = c->eff_tile_w; + bh = c->eff_tile_h; + } + glViewport((GLint)bx, (GLint)by, (GLsizei)bw, (GLsizei)bh); + glScissor((GLint)bx, (GLint)by, (GLsizei)bw, (GLsizei)bh); + + float xy[4][2]; + u_cursor_draw_corners(&req->view[i], height, aspect, hot_u, hot_v, xy); + float corners[4][4]; + for (int k = 0; k < 4; k++) { + // Tile-normalised, y DOWN -> GL NDC, y UP. + corners[k][0] = xy[k][0] * 2.0f - 1.0f; + corners[k][1] = 1.0f - xy[k][1] * 2.0f; + corners[k][2] = uv[k][0]; + corners[k][3] = flip_v ? 1.0f - uv[k][1] : uv[k][1]; + } + glUniform4fv(c->cursor_draw.loc_corners, 4, &corners[0][0]); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + } + + glDisable(GL_SCISSOR_TEST); + glDisable(GL_BLEND); + glBlendFunc(GL_ONE, GL_ZERO); + glScissor(prev_scissor[0], prev_scissor[1], prev_scissor[2], prev_scissor[3]); + glViewport(prev_viewport[0], prev_viewport[1], prev_viewport[2], prev_viewport[3]); + glBindTexture(GL_TEXTURE_2D, 0); + glUseProgram(0); + + if (!c->cursor_draw.announced) { + c->cursor_draw.announced = true; + U_LOG_W( + "cursor_draw: GL compositor drawing the cursor (%s, %.1f mm tall, %u view(s), %s; ADR-046 " + "Amendment 2 B, one-time notice)", + req->image != NULL ? "app sprite" : "default look", size_m * 1000.0f, n, + c->compose_active ? "linear compose target" : "encoded atlas"); + } + return true; +} + +void +comp_gl_compositor_set_cursor_draw_request(struct xrt_compositor *xc, const struct u_cursor_draw_request *req) +{ + struct comp_gl_compositor *c = gl_comp(xc); + c->cursor_draw.req = *req; +} + /*! * The frame path proper. Runs with c->mutex HELD — see the locking wrapper * below. Keeps its early returns, which is why the lock lives in the wrapper. @@ -5943,6 +6235,24 @@ gl_compositor_layer_commit_locked(struct xrt_compositor *xc, xrt_graphics_sync_h } } + /* + * XR_DXR_cursor_depth v5 (ADR-046 Amendment 2 B). ZERO COST UNLESS + * REQUESTED: one field test, false for every session that never chained + * XrCursorDepthDrawInfoDXR. The request covers this commit only. Zones + * frames are refused by the state tracker already; belt and braces. + */ + struct u_cursor_draw_request cursor_req = {}; + bool cursor_pending = false; + c->cursor_draw.pending = false; + if (c->cursor_draw.req.requested) { + cursor_req = c->cursor_draw.req; + c->cursor_draw.req.requested = false; + cursor_pending = !c->zones_frame && u_cursor_draw_should_draw(&cursor_req); + c->cursor_draw.pending = cursor_pending; + c->cursor_draw.pending_source_srgb = + cursor_req.image == NULL || gl_swapchain_is_srgb(gl_swapchain(cursor_req.image)); + } + // Save previous GL context and switch to compositor's #ifdef XRT_OS_WINDOWS HDC prev_hdc = wglGetCurrentDC(); @@ -6141,7 +6451,10 @@ gl_compositor_layer_commit_locked(struct xrt_compositor *xc, xrt_graphics_sync_h if (idx < c->xdev->rendering_mode_count) mode = &c->xdev->rendering_modes[idx]; } - if (mode != NULL && c->layer_accum.layer_count == 1 && !seg_routed_frame) { + // XR_DXR_cursor_depth v5: a frame carrying a runtime-drawn cursor + // composes into the atlas - the cursor is drawn there, never into the + // app's own image. + if (mode != NULL && c->layer_accum.layer_count == 1 && !seg_routed_frame && !cursor_pending) { struct comp_layer *layer = &c->layer_accum.layers[0]; if (layer->data.type == XRT_LAYER_PROJECTION || layer->data.type == XRT_LAYER_PROJECTION_DEPTH) { @@ -6763,6 +7076,20 @@ gl_compositor_layer_commit_locked(struct xrt_compositor *xc, xrt_graphics_sync_h glDisable(GL_BLEND); } + // XR_DXR_cursor_depth v5 (ADR-046 Amendment 2 B): the runtime-drawn + // cursor, LAST - over the projection, quad and window-space layers, never + // occluded - and before the display processor. Into the same target the + // pass painted, so the publish below (and the /tmp/dxr_atlas_trigger dump + // after it) carries it. Not on a multi-screen routed frame (a tile is a + // per-segment mosaic there). + bool cursor_routed = false; +#ifdef XRT_OS_WINDOWS + cursor_routed = seg_routed_frame; +#endif + if (cursor_pending && !cursor_routed && gl_cursor_draw(c, &cursor_req)) { + window_space_drew = true; // republish the compose target + } + // #1610: republish only if the window-space pass actually painted, then // hand the pass back with the encode OFF — everything downstream (the // Local2D flatten, the weave, the present blit) writes GL_RGBA8 targets @@ -7169,6 +7496,12 @@ gl_compositor_destroy(struct xrt_compositor *xc) if (c->program_blit) glDeleteProgram(c->program_blit); if (c->program_blit_array) glDeleteProgram(c->program_blit_array); if (c->program_window_space) glDeleteProgram(c->program_window_space); + // XR_DXR_cursor_depth v5 draw resources (zero unless ever requested). + if (c->cursor_draw.program) + glDeleteProgram(c->cursor_draw.program); + if (c->cursor_draw.default_tex) + glDeleteTextures(1, &c->cursor_draw.default_tex); + comp_cursor_look_fini(&c->cursor_draw.look); if (c->program_masked_composite) glDeleteProgram(c->program_masked_composite); if (c->program_quad) glDeleteProgram(c->program_quad); if (c->program_quad_array) glDeleteProgram(c->program_quad_array); diff --git a/src/xrt/compositor/gl/comp_gl_compositor.h b/src/xrt/compositor/gl/comp_gl_compositor.h index bcb4cc2d80..6ec63c63f3 100644 --- a/src/xrt/compositor/gl/comp_gl_compositor.h +++ b/src/xrt/compositor/gl/comp_gl_compositor.h @@ -173,6 +173,21 @@ comp_gl_compositor_zones_set_frame_wish(struct xrt_compositor *xc, void *mask); bool comp_gl_compositor_get_recommended_view_size(struct xrt_compositor *xc, uint32_t *out_w, uint32_t *out_h); +struct u_cursor_draw_request; + +/*! + * XR_DXR_cursor_depth v5 (ADR-046 Amendment 2 B): draw the cursor sprite into + * every view tile at the NEXT layer_commit only, after the app's layers + * (projection, quad, window-space) and before the display processor. Called by + * the state tracker only on a frame whose locate chained + * XrCursorDepthDrawInfoDXR and reported willDraw; a compositor never handed a + * request does no cursor work at all. + * + * @ingroup comp_gl + */ +void +comp_gl_compositor_set_cursor_draw_request(struct xrt_compositor *xc, const struct u_cursor_draw_request *req); + struct xrt_screen_list; struct xrt_segment_metrics; struct xrt_segment_view_routing; diff --git a/src/xrt/compositor/metal/CMakeLists.txt b/src/xrt/compositor/metal/CMakeLists.txt index 62f0a7bc60..209d754d05 100644 --- a/src/xrt/compositor/metal/CMakeLists.txt +++ b/src/xrt/compositor/metal/CMakeLists.txt @@ -40,6 +40,8 @@ if(APPLE AND XRT_HAVE_METAL) # itself Vulkan-gated, so this adds no new configuration. aux_vk xrt-external-stb + # XR_DXR_cursor_depth v5: the runtime-drawn cursor's default look. + comp_cursor_look "-framework Metal" "-framework Cocoa" "-framework QuartzCore" diff --git a/src/xrt/compositor/metal/comp_metal_compositor.m b/src/xrt/compositor/metal/comp_metal_compositor.m index d97ba6dab4..75a8fe3b51 100644 --- a/src/xrt/compositor/metal/comp_metal_compositor.m +++ b/src/xrt/compositor/metal/comp_metal_compositor.m @@ -23,6 +23,7 @@ #include "comp_metal_segments.h" // multi-screen on macOS #include "comp_metal_placement.h" // runtime-owned window placement (ADR-050) +#include "util/comp_cursor_look.h" #include "util/comp_layer_accum.h" // #1581 quad layers: the shared per-view camera (#1580) and the N-view // LEFT/RIGHT eye-visibility rule. @@ -509,14 +510,12 @@ //! Built lazily: [premultiplied?][_sRGB target?]. id pipe[2][2]; bool pipes_failed; - //! The default look's sprite (premultiplied, _sRGB, PixelFormatView). + //! The default look's sprite (premultiplied, _sRGB, PixelFormatView), + //! uploaded from @ref look whenever its generation moves. id default_tex; - //! The OS cursor it mirrors (retained), nil for the arrow fallback. - NSCursor *default_os_cursor; - float default_aspect; //!< width / height - float default_hot_u; //!< hotspot, normalised (top-left, v down) - float default_hot_v; - float default_height_pt; //!< height in OS points + uint64_t default_gen; + //! The shared default look (mirrored OS cursor, else the arrow). + struct comp_cursor_look look; bool announced; bool warned_zones; } cursor_draw; @@ -4101,111 +4100,25 @@ - (BOOL)wantsUpdateLayer return true; } -//! The classic arrow fallback (Amendment 2 A.6): 12 x 19, tip = hotspot. -static bool -metal_cursor_use_arrow(struct comp_metal_compositor *c) -{ - if (c->cursor_draw.default_tex != nil && c->cursor_draw.default_os_cursor == nil) { - return true; // already the arrow - } - const uint32_t h = 76; // 4 px per arrow px: crisp up to 4x scale - const uint32_t w = (uint32_t)ceilf((float)h * U_CURSOR_ARROW_WIDTH_PX / U_CURSOR_ARROW_HEIGHT_PX); - uint8_t *rgba = calloc((size_t)w * h, 4); - if (rgba == NULL) { - return false; - } - u_cursor_draw_arrow_rgba(rgba, w, h); - const bool ok = metal_cursor_set_default_tex(c, rgba, w, h); - free(rgba); - if (!ok) { - return false; - } - [c->cursor_draw.default_os_cursor release]; - c->cursor_draw.default_os_cursor = nil; - c->cursor_draw.default_aspect = (float)w / (float)h; - c->cursor_draw.default_hot_u = 0.0f; - c->cursor_draw.default_hot_v = 0.0f; - c->cursor_draw.default_height_pt = U_CURSOR_DRAW_DEFAULT_POINTS; - U_LOG_W("cursor_draw: default look = the classic arrow (12x19, tip = hotspot)"); - return true; -} - /*! - * The default look (Amendment 2 A.6): mirror the CURRENT OS cursor shape. - * - * AppKit is main-thread-only, so the shape is read only when this commit runs - * on the main thread (an app that pumps its own events there, as the - * cube_*_macos apps do). Off the main thread the last snapshot is kept, or the - * classic arrow if there never was one. DXR_CURSOR_DEFAULT_LOOK=arrow forces - * the arrow. The image is re-rasterised only when the cursor object changes. + * The default look (Amendment 2 A.6), shared with the Vulkan and GL + * compositors (comp_cursor_look): the mirrored OS cursor when the commit runs + * on the main thread, else the last snapshot, else the classic arrow. The + * texture is re-uploaded only when the look's image changes. */ static bool metal_cursor_update_default(struct comp_metal_compositor *c, float backing_scale) { - static int force_arrow = -1; - if (force_arrow < 0) { - const char *e = getenv("DXR_CURSOR_DEFAULT_LOOK"); - force_arrow = (e != NULL && strcmp(e, "arrow") == 0) ? 1 : 0; - } - if (force_arrow || ![NSThread isMainThread]) { - return c->cursor_draw.default_tex != nil || metal_cursor_use_arrow(c); - } - - NSCursor *cur = [NSCursor currentCursor]; - if (cur == nil || cur.image == nil) { - return c->cursor_draw.default_tex != nil || metal_cursor_use_arrow(c); + if (!comp_cursor_look_update(&c->cursor_draw.look, backing_scale)) { + return c->cursor_draw.default_tex != nil; } - if (cur == c->cursor_draw.default_os_cursor && c->cursor_draw.default_tex != nil) { - return true; + if (c->cursor_draw.default_tex == nil || c->cursor_draw.default_gen != c->cursor_draw.look.generation) { + if (!metal_cursor_set_default_tex(c, c->cursor_draw.look.rgba, c->cursor_draw.look.w, + c->cursor_draw.look.h)) { + return c->cursor_draw.default_tex != nil; + } + c->cursor_draw.default_gen = c->cursor_draw.look.generation; } - - NSImage *img = cur.image; - const NSSize sz = img.size; // points - const NSPoint hs = cur.hotSpot; // points, origin top-left - const CGFloat scale = backing_scale > 0.0f ? backing_scale : 2.0; - const uint32_t w = (uint32_t)ceil(sz.width * scale); - const uint32_t h = (uint32_t)ceil(sz.height * scale); - if (sz.width <= 0.0 || sz.height <= 0.0 || w == 0 || h == 0 || w > 512 || h > 512) { - return c->cursor_draw.default_tex != nil || metal_cursor_use_arrow(c); - } - uint8_t *rgba = calloc((size_t)w * h, 4); - CGColorSpaceRef cs = CGColorSpaceCreateWithName(kCGColorSpaceSRGB); - CGContextRef ctx = (rgba != NULL && cs != NULL) - ? CGBitmapContextCreate(rgba, w, h, 8, (size_t)w * 4, cs, - kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big) - : NULL; - bool ok = false; - if (ctx != NULL) { - // Bitmap-context memory holds the TOP row first, so the upload below - // lands top-left-origin, as Metal samples it. - NSGraphicsContext *gc = [NSGraphicsContext graphicsContextWithCGContext:ctx flipped:NO]; - [NSGraphicsContext saveGraphicsState]; - [NSGraphicsContext setCurrentContext:gc]; - [img drawInRect:NSMakeRect(0, 0, w, h) - fromRect:NSZeroRect - operation:NSCompositingOperationCopy - fraction:1.0]; - [NSGraphicsContext restoreGraphicsState]; - CGContextFlush(ctx); - ok = metal_cursor_set_default_tex(c, rgba, w, h); - CGContextRelease(ctx); - } - if (cs != NULL) { - CGColorSpaceRelease(cs); - } - free(rgba); - if (!ok) { - return c->cursor_draw.default_tex != nil || metal_cursor_use_arrow(c); - } - [c->cursor_draw.default_os_cursor release]; - c->cursor_draw.default_os_cursor = [cur retain]; - c->cursor_draw.default_aspect = (float)(sz.width / sz.height); - c->cursor_draw.default_hot_u = (float)(hs.x / sz.width); - c->cursor_draw.default_hot_v = (float)(hs.y / sz.height); - c->cursor_draw.default_height_pt = (float)sz.height; - // Not per frame: only when the app's cursor object changes. - U_LOG_W("cursor_draw: default look = the current OS cursor (%.0fx%.0f pt, hotspot %.1f,%.1f%s)", sz.width, - sz.height, hs.x, hs.y, cur == [NSCursor arrowCursor] ? ", arrowCursor" : ""); return true; } @@ -4263,10 +4176,10 @@ - (BOOL)wantsUpdateLayer return; } src = metal_layer_source_view(c->cursor_draw.default_tex, 0, compose_linear); - aspect = c->cursor_draw.default_aspect; - hot_u = c->cursor_draw.default_hot_u; - hot_v = c->cursor_draw.default_hot_v; - size_m = req->size_m > 0.0f ? req->size_m : c->cursor_draw.default_height_pt * m_per_pt; + aspect = c->cursor_draw.look.aspect; + hot_u = c->cursor_draw.look.hot_u; + hot_v = c->cursor_draw.look.hot_v; + size_m = req->size_m > 0.0f ? req->size_m : c->cursor_draw.look.height_pt * m_per_pt; premul = true; // CGBitmapContext output } const float height = size_m / canvas_h_m; // sprite height, in units @@ -6005,8 +5918,7 @@ - (BOOL)wantsUpdateLayer } [c->cursor_draw.default_tex release]; c->cursor_draw.default_tex = nil; - [c->cursor_draw.default_os_cursor release]; - c->cursor_draw.default_os_cursor = nil; + comp_cursor_look_fini(&c->cursor_draw.look); [c->depth_texture release]; c->depth_texture = nil; [c->projection_pipeline release]; diff --git a/src/xrt/compositor/util/comp_cursor_look.c b/src/xrt/compositor/util/comp_cursor_look.c new file mode 100644 index 0000000000..287718bf82 --- /dev/null +++ b/src/xrt/compositor/util/comp_cursor_look.c @@ -0,0 +1,113 @@ +// Copyright 2026, The DisplayXR Project and its contributors +// SPDX-License-Identifier: BSL-1.0 +/*! + * @file + * @brief The runtime-drawn cursor's default look (platform-neutral half). + * @author David Fattal + * @ingroup comp_util + */ + +#include "comp_cursor_look.h" + +#include "util/u_cursor_depth.h" +#include "util/u_logging.h" + +#include +#include +#include + +//! DXR_CURSOR_DEFAULT_LOOK=arrow forces the classic arrow. +static bool +force_arrow(void) +{ + static int cached = -1; + if (cached < 0) { + const char *e = getenv("DXR_CURSOR_DEFAULT_LOOK"); + cached = (e != NULL && strcmp(e, "arrow") == 0) ? 1 : 0; + } + return cached == 1; +} + +//! The classic arrow (Amendment 2 A.6): 12 x 19, tip = hotspot. +static bool +use_arrow(struct comp_cursor_look *look) +{ + if (look->rgba != NULL && look->os_cursor == NULL) { + return true; // already the arrow + } + const uint32_t h = 76; // 4 px per arrow px: crisp up to 4x scale + const uint32_t w = (uint32_t)ceilf((float)h * U_CURSOR_ARROW_WIDTH_PX / U_CURSOR_ARROW_HEIGHT_PX); + uint8_t *rgba = calloc((size_t)w * h, 4); + if (rgba == NULL) { + return false; + } + u_cursor_draw_arrow_rgba(rgba, w, h); + free(look->rgba); + look->rgba = rgba; + look->w = w; + look->h = h; + if (look->os_cursor != NULL) { + comp_cursor_look_platform_release(look->os_cursor); + look->os_cursor = NULL; + } + look->aspect = (float)w / (float)h; + look->hot_u = 0.0f; + look->hot_v = 0.0f; + look->height_pt = U_CURSOR_DRAW_DEFAULT_POINTS; + look->generation++; + U_LOG_W("cursor_draw: default look = the classic arrow (12x19, tip = hotspot)"); + return true; +} + +bool +comp_cursor_look_update(struct comp_cursor_look *look, float backing_scale) +{ + if (!force_arrow() && comp_cursor_look_platform_mirror(look, backing_scale)) { + return true; + } + // Could not read the OS cursor here: keep the last image, else the arrow. + return look->rgba != NULL || use_arrow(look); +} + +float +comp_cursor_look_backing_scale(void) +{ + return comp_cursor_look_platform_backing_scale(); +} + +void +comp_cursor_look_fini(struct comp_cursor_look *look) +{ + free(look->rgba); + look->rgba = NULL; + if (look->os_cursor != NULL) { + comp_cursor_look_platform_release(look->os_cursor); + look->os_cursor = NULL; + } +} + +#ifndef __APPLE__ +/* + * No OS-cursor mirror on this platform yet (Linux: XFixesGetCursorImage is + * the follow-up; Windows: GetCursorInfo + GetIconInfo). The arrow it is. + */ +bool +comp_cursor_look_platform_mirror(struct comp_cursor_look *look, float backing_scale) +{ + (void)look; + (void)backing_scale; + return false; +} + +void +comp_cursor_look_platform_release(void *os_cursor) +{ + (void)os_cursor; +} + +float +comp_cursor_look_platform_backing_scale(void) +{ + return 1.0f; +} +#endif diff --git a/src/xrt/compositor/util/comp_cursor_look.h b/src/xrt/compositor/util/comp_cursor_look.h new file mode 100644 index 0000000000..d473b7e61b --- /dev/null +++ b/src/xrt/compositor/util/comp_cursor_look.h @@ -0,0 +1,111 @@ +// Copyright 2026, The DisplayXR Project and its contributors +// SPDX-License-Identifier: BSL-1.0 +/*! + * @file + * @brief The runtime-drawn cursor's default look, shared by every compositor. + * @author David Fattal + * @ingroup comp_util + * + * XR_DXR_cursor_depth v5 (ADR-046 Amendment 2 A.6 / B): when the app hands + * the runtime no sprite, the runtime draws its DEFAULT LOOK - the current OS + * cursor's shape where the runtime can read it, else the classic arrow + * (u_cursor_draw_arrow_rgba: 12 x 19, white fill, dark outline, tip = + * hotspot). + * + * This module owns the CPU side of that choice, so the Metal, Vulkan and GL + * compositors cannot drift apart on it: which look, its pixels (premultiplied, + * sRGB-encoded RGBA8, top row first), its aspect, hotspot and size in OS + * points. A compositor only uploads the pixels whenever @ref + * comp_cursor_look::generation changes and samples them. + * + * OS-cursor mirroring is macOS-only today ([NSCursor currentCursor], read + * only when the commit runs on the main thread - AppKit is main-thread-only). + * Elsewhere, and whenever the shape cannot be read, the look is the arrow. + * `DXR_CURSOR_DEFAULT_LOOK=arrow` forces the arrow everywhere. + * + * ZERO COST UNLESS REQUESTED (ADR-046 section 0): nothing here runs, and + * nothing is allocated, until a compositor is handed a draw request with no + * app sprite. + */ + +#pragma once + +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +/*! + * The default look's current image. Zero-initialise; release with + * comp_cursor_look_fini(). + */ +struct comp_cursor_look +{ + //! Premultiplied, sRGB-encoded RGBA8, top row first (w * h * 4 bytes). + uint8_t *rgba; + uint32_t w, h; + //! Width / height. + float aspect; + //! Hotspot, normalised (origin top-left, v down). + float hot_u, hot_v; + //! Height in OS points (the arrow: U_CURSOR_DRAW_DEFAULT_POINTS). + float height_pt; + //! Bumped every time the pixels change: re-upload when it moves. + uint64_t generation; + //! The mirrored OS cursor (platform object, retained), NULL for the arrow. + void *os_cursor; +}; + +/*! + * Refresh the default look for this commit: re-read the OS cursor when the + * platform allows it here, else keep the last image, else the arrow. + * + * @param backing_scale The output's backing pixels per OS point (raster + * resolution of a mirrored OS cursor); <= 0 picks the platform's. + * @return false only if no image could be produced at all. + */ +bool +comp_cursor_look_update(struct comp_cursor_look *look, float backing_scale); + +/*! + * Backing pixels per OS point of the main screen (macOS), 1 elsewhere. Read on + * the main thread and cached; off the main thread before the first read, 2 on + * macOS. The panel's metres per OS point is its pixel pitch times this. + */ +float +comp_cursor_look_backing_scale(void); + +//! Free the image and release a mirrored OS cursor. +void +comp_cursor_look_fini(struct comp_cursor_look *look); + + +/* + * + * Internal - the platform half (comp_cursor_look_macos.m on Apple). + * + */ + +/*! + * Mirror the current OS cursor into @p look if the platform can read it now + * and it is not already the mirrored one. + * + * @return true if @p look now holds the OS cursor (changed or not), false if + * it could not be read here (the caller falls back). + */ +bool +comp_cursor_look_platform_mirror(struct comp_cursor_look *look, float backing_scale); + +//! Release a mirrored platform cursor object. +void +comp_cursor_look_platform_release(void *os_cursor); + +//! @copydoc comp_cursor_look_backing_scale +float +comp_cursor_look_platform_backing_scale(void); + +#ifdef __cplusplus +} +#endif diff --git a/src/xrt/compositor/util/comp_cursor_look_macos.m b/src/xrt/compositor/util/comp_cursor_look_macos.m new file mode 100644 index 0000000000..597a2044ec --- /dev/null +++ b/src/xrt/compositor/util/comp_cursor_look_macos.m @@ -0,0 +1,120 @@ +// Copyright 2026, The DisplayXR Project and its contributors +// SPDX-License-Identifier: BSL-1.0 +/*! + * @file + * @brief The runtime-drawn cursor's default look: macOS OS-cursor mirror. + * @author David Fattal + * @ingroup comp_util + * + * Compiled without ARC (the NSCursor is retained by hand, as the Metal + * compositor this was lifted from did). + */ + +#include "comp_cursor_look.h" + +#include "util/u_logging.h" + +#import + +#include +#include + +static float g_backing_scale = 0.0f; + +float +comp_cursor_look_platform_backing_scale(void) +{ + if ([NSThread isMainThread]) { + NSScreen *s = [NSScreen mainScreen]; + if (s != nil && s.backingScaleFactor > 0.0) { + g_backing_scale = (float)s.backingScaleFactor; + } + } + return g_backing_scale > 0.0f ? g_backing_scale : 2.0f; +} + +void +comp_cursor_look_platform_release(void *os_cursor) +{ + [(NSCursor *)os_cursor release]; +} + +/*! + * Mirror the CURRENT OS cursor shape. + * + * AppKit is main-thread-only, so the shape is read only when this commit runs + * on the main thread (an app that pumps its own events there, as the + * cube_*_macos apps do). The image is re-rasterised only when the cursor + * object changes. + */ +bool +comp_cursor_look_platform_mirror(struct comp_cursor_look *look, float backing_scale) +{ + if (![NSThread isMainThread]) { + return false; + } + NSCursor *cur = [NSCursor currentCursor]; + if (cur == nil || cur.image == nil) { + return false; + } + if ((void *)cur == look->os_cursor && look->rgba != NULL) { + return true; + } + + NSImage *img = cur.image; + const NSSize sz = img.size; // points + const NSPoint hs = cur.hotSpot; // points, origin top-left + const CGFloat scale = backing_scale > 0.0f ? backing_scale : comp_cursor_look_platform_backing_scale(); + const uint32_t w = (uint32_t)ceil(sz.width * scale); + const uint32_t h = (uint32_t)ceil(sz.height * scale); + if (sz.width <= 0.0 || sz.height <= 0.0 || w == 0 || h == 0 || w > 512 || h > 512) { + return false; + } + uint8_t *rgba = calloc((size_t)w * h, 4); + CGColorSpaceRef cs = CGColorSpaceCreateWithName(kCGColorSpaceSRGB); + CGContextRef ctx = (rgba != NULL && cs != NULL) + ? CGBitmapContextCreate(rgba, w, h, 8, (size_t)w * 4, cs, + kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big) + : NULL; + bool ok = false; + if (ctx != NULL) { + // Bitmap-context memory holds the TOP row first, so the image is + // top-left-origin, as the shared look promises. + NSGraphicsContext *gc = [NSGraphicsContext graphicsContextWithCGContext:ctx flipped:NO]; + [NSGraphicsContext saveGraphicsState]; + [NSGraphicsContext setCurrentContext:gc]; + [img drawInRect:NSMakeRect(0, 0, w, h) + fromRect:NSZeroRect + operation:NSCompositingOperationCopy + fraction:1.0]; + [NSGraphicsContext restoreGraphicsState]; + CGContextFlush(ctx); + CGContextRelease(ctx); + ok = true; + } + if (cs != NULL) { + CGColorSpaceRelease(cs); + } + if (!ok) { + free(rgba); + return false; + } + + free(look->rgba); + look->rgba = rgba; + look->w = w; + look->h = h; + if (look->os_cursor != NULL) { + [(NSCursor *)look->os_cursor release]; + } + look->os_cursor = [cur retain]; + look->aspect = (float)(sz.width / sz.height); + look->hot_u = (float)(hs.x / sz.width); + look->hot_v = (float)(hs.y / sz.height); + look->height_pt = (float)sz.height; + look->generation++; + // Not per frame: only when the app's cursor object changes. + U_LOG_W("cursor_draw: default look = the current OS cursor (%.0fx%.0f pt, hotspot %.1f,%.1f%s)", sz.width, + sz.height, hs.x, hs.y, cur == [NSCursor arrowCursor] ? ", arrowCursor" : ""); + return true; +} diff --git a/src/xrt/compositor/vk_native/CMakeLists.txt b/src/xrt/compositor/vk_native/CMakeLists.txt index 2350bc5921..bb896e241f 100644 --- a/src/xrt/compositor/vk_native/CMakeLists.txt +++ b/src/xrt/compositor/vk_native/CMakeLists.txt @@ -83,6 +83,8 @@ if((WIN32 OR APPLE OR ANDROID OR (XRT_HAVE_LINUX AND XRT_HAVE_XCB)) AND XRT_HAVE aux_os aux_math xrt-external-stb + # XR_DXR_cursor_depth v5: the runtime-drawn cursor's default look. + comp_cursor_look ) target_include_directories(comp_vk_native PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/..) diff --git a/src/xrt/compositor/vk_native/comp_vk_native_compositor.c b/src/xrt/compositor/vk_native/comp_vk_native_compositor.c index 6f3f3a18b4..60aee1a127 100644 --- a/src/xrt/compositor/vk_native/comp_vk_native_compositor.c +++ b/src/xrt/compositor/vk_native/comp_vk_native_compositor.c @@ -21,6 +21,7 @@ #include "comp_vk_native_segments.h" #include "util/comp_swapchain_ring.h" +#include "util/comp_cursor_look.h" #include "util/comp_zone_tier1.h" #include "util/comp_layer_accum.h" #include "util/comp_bg2d.h" @@ -65,6 +66,7 @@ #include "util/u_tiling.h" #include "util/u_canvas.h" #include "util/u_capture_intent.h" +#include "util/u_cursor_depth.h" #include "util/u_repaint_gate.h" #include "util/u_fill_thread_win.h" #include "util/u_image_capture.h" @@ -1523,6 +1525,27 @@ struct comp_vk_native_compositor //! one pool; resetting it twice while a shared cmd buffer still references //! the earlier sets is a use-after-reset. Cleared at the top of layer_commit. bool local2d_pool_reset_this_frame; + + /*! + * XR_DXR_cursor_depth v5, the runtime draws the cursor (ADR-046 + * Amendment 2 B). + * + * ZERO COST UNLESS REQUESTED: all zero for a session that never chained + * XrCursorDepthDrawInfoDXR. layer_commit tests `req.requested` and does + * nothing else; the default look and the renderer's sprite image are + * built on the first real request only. + */ + struct + { + //! One-frame request from the state tracker, consumed by the next commit. + struct u_cursor_draw_request req; + //! The shared default look, uploaded to the renderer on change. + struct comp_cursor_look look; + uint64_t default_gen; + bool default_uploaded; + bool warned_zones; + bool warned_routed; + } cursor_draw; }; /* @@ -9421,6 +9444,98 @@ vk_shared_submit_and_wait(struct comp_vk_native_compositor *c, const VkSubmitInf return VK_SUCCESS; } +/*! + * XR_DXR_cursor_depth v5 (ADR-046 Amendment 2 B): turn the state tracker's + * per-view placement into this frame's renderer cursor - which image, its + * size in canvas units, and each view's corners in its tile. + * + * @param canvas_px_h The canvas (window) height in backing pixels: the size + * the Kooima views were built for, so one unit = canvas height. + * @return false if nothing can be drawn this frame (the frame is untouched). + */ +static bool +vk_cursor_resolve(struct comp_vk_native_compositor *c, + const struct u_cursor_draw_request *req, + uint32_t canvas_px_h, + struct comp_vk_native_cursor_draw *out) +{ + if (c->zones_frame) { + // The state tracker already refuses zones frames; belt and braces. + if (!c->cursor_draw.warned_zones) { + c->cursor_draw.warned_zones = true; + U_LOG_W("cursor_draw: zones frame - cursor not drawn (one-time warning)"); + } + return false; + } + if (c->eff_layout.route_count > 0) { + // Multi-screen M3: a tile is a per-segment mosaic, not one view. + if (!c->cursor_draw.warned_routed) { + c->cursor_draw.warned_routed = true; + U_LOG_W("cursor_draw: multi-screen routed frame - cursor not drawn (one-time warning)"); + } + return false; + } + // Panel pixel pitch (metres per backing pixel) and the OS point. + if (!c->sys_info_set || c->sys_info.display_pixel_height == 0 || !(c->sys_info.display_height_m > 0.0f) || + canvas_px_h == 0) { + return false; + } + const float m_per_px = c->sys_info.display_height_m / (float)c->sys_info.display_pixel_height; + const float backing_scale = comp_cursor_look_backing_scale(); + const float m_per_pt = m_per_px * backing_scale; + const float canvas_h_m = (float)canvas_px_h * m_per_px; + + float aspect, hot_u, hot_v, size_m; + memset(out, 0, sizeof(*out)); + if (req->image != NULL) { + uint32_t w = 0, h = 0; + comp_vk_native_swapchain_get_dimensions(req->image, &w, &h); + if (w == 0 || h == 0) { + return false; + } + aspect = (float)w / (float)h; + hot_u = req->hot_u; + hot_v = req->hot_v; + size_m = req->size_m > 0.0f ? req->size_m : U_CURSOR_DRAW_DEFAULT_POINTS * m_per_pt; + out->xsc = req->image; + out->image_index = req->image_index; + out->premultiplied = false; // spec: the app's sprite is STRAIGHT alpha + } else { + struct comp_cursor_look *look = &c->cursor_draw.look; + if (!comp_cursor_look_update(look, backing_scale)) { + return false; + } + if (!c->cursor_draw.default_uploaded || c->cursor_draw.default_gen != look->generation) { + if (!comp_vk_native_renderer_set_cursor_default_image(c->renderer, look->rgba, look->w, + look->h)) { + return false; + } + c->cursor_draw.default_gen = look->generation; + c->cursor_draw.default_uploaded = true; + } + aspect = look->aspect; + hot_u = look->hot_u; + hot_v = look->hot_v; + size_m = req->size_m > 0.0f ? req->size_m : look->height_pt * m_per_pt; + out->premultiplied = true; // the look's pixels are premultiplied + } + const float height = size_m / canvas_h_m; // sprite height, in units + + out->view_count = req->view_count < XRT_MAX_VIEWS ? req->view_count : XRT_MAX_VIEWS; + for (uint32_t i = 0; i < out->view_count; i++) { + u_cursor_draw_corners(&req->view[i], height, aspect, hot_u, hot_v, out->corners[i]); + } + out->active = true; + return true; +} + +void +comp_vk_native_compositor_set_cursor_draw_request(struct xrt_compositor *xc, const struct u_cursor_draw_request *req) +{ + struct comp_vk_native_compositor *c = vk_comp(xc); + c->cursor_draw.req = *req; +} + static xrt_result_t vk_compositor_layer_commit_locked(struct xrt_compositor *xc, xrt_graphics_sync_handle_t sync_handle, @@ -9710,6 +9825,20 @@ vk_compositor_layer_commit_locked(struct xrt_compositor *xc, // capture path — they must all agree on the frame's geometry. vk_compute_effective_layout(c); + /* + * XR_DXR_cursor_depth v5 (ADR-046 Amendment 2 B). ZERO COST UNLESS + * REQUESTED: one field test, false for every session that never chained + * XrCursorDepthDrawInfoDXR. The request covers this commit only. + */ + struct comp_vk_native_cursor_draw cursor_draw = {0}; + if (c->cursor_draw.req.requested) { + struct u_cursor_draw_request cursor_req = c->cursor_draw.req; + c->cursor_draw.req.requested = false; + if (u_cursor_draw_should_draw(&cursor_req)) { + (void)vk_cursor_resolve(c, &cursor_req, tgt_height, &cursor_draw); + } + } + // Zero-copy check: can we pass the app's swapchain directly to the DP? bool zero_copy = false; uint64_t zc_image_u64 = 0; @@ -9876,6 +10005,16 @@ vk_compositor_layer_commit_locked(struct xrt_compositor *xc, } #endif + /* + * XR_DXR_cursor_depth v5: a frame carrying a runtime-drawn cursor + * composes into the atlas - the cursor is drawn there, never into the + * app's own image. Same shape as the guards above: a fact applied to the + * RESULT of `u_tiling_can_zero_copy()` (ADR-030), not a second gate. + */ + if (cursor_draw.active) { + zero_copy = false; + } + // Record the frame's effective capture source = exactly what the DP will // receive (renderer atlas, or the zero-copy app swapchain). The atlas // capture reads this so it shows the DP's input regardless of render mode @@ -9895,6 +10034,11 @@ vk_compositor_layer_commit_locked(struct xrt_compositor *xc, // Render projection layers to atlas texture (skip if zero-copy) xrt_result_t xret = XRT_SUCCESS; if (!zero_copy) { + if (cursor_draw.active) { + // Consumed by the draw below (drawn last in its compose pass, + // so the projection-only capture point after it shows it). + comp_vk_native_renderer_set_cursor(c->renderer, &cursor_draw); + } xret = comp_vk_native_renderer_draw( c->renderer, &c->layer_accum, &left_eye, &right_eye, tgt_width, tgt_height, &c->eff_layout); if (xret != XRT_SUCCESS) { @@ -10872,6 +11016,7 @@ vk_compositor_destroy(struct xrt_compositor *xc) if (c->renderer != NULL) { comp_vk_native_renderer_destroy(&c->renderer); } + comp_cursor_look_fini(&c->cursor_draw.look); // XR_DXR_cursor_depth v5 (empty unless requested) comp_vk_alpha_probe_destroy(&c->alpha_probe); diff --git a/src/xrt/compositor/vk_native/comp_vk_native_compositor.h b/src/xrt/compositor/vk_native/comp_vk_native_compositor.h index f5ae80220e..ab24396f48 100644 --- a/src/xrt/compositor/vk_native/comp_vk_native_compositor.h +++ b/src/xrt/compositor/vk_native/comp_vk_native_compositor.h @@ -491,6 +491,20 @@ comp_vk_native_compositor_zones_set_frame_wish(struct xrt_compositor *xc, void * bool comp_vk_native_compositor_get_recommended_view_size(struct xrt_compositor *xc, uint32_t *out_w, uint32_t *out_h); +struct u_cursor_draw_request; + +/*! + * XR_DXR_cursor_depth v5 (ADR-046 Amendment 2 B): draw the cursor sprite into + * every view tile at the NEXT layer_commit only, after the layers the atlas + * pass composes and before the display processor. Called by the state + * tracker only on a frame whose locate chained XrCursorDepthDrawInfoDXR and + * reported willDraw; a compositor never handed a request does no cursor work. + * + * @ingroup comp_vk_native + */ +void +comp_vk_native_compositor_set_cursor_draw_request(struct xrt_compositor *xc, const struct u_cursor_draw_request *req); + #ifdef __cplusplus } #endif diff --git a/src/xrt/compositor/vk_native/comp_vk_native_renderer.c b/src/xrt/compositor/vk_native/comp_vk_native_renderer.c index b66517d42c..9d62d3301a 100644 --- a/src/xrt/compositor/vk_native/comp_vk_native_renderer.c +++ b/src/xrt/compositor/vk_native/comp_vk_native_renderer.c @@ -312,6 +312,38 @@ struct comp_vk_native_renderer * reach. NULL (the default) restores the owned-atlas path verbatim. */ struct comp_vk_deposit *deposit; + + /*! + * XR_DXR_cursor_depth v5, the runtime draws the cursor (ADR-046 + * Amendment 2 B). + * + * ZERO COST UNLESS REQUESTED: all zero for a session that never chained + * XrCursorDepthDrawInfoDXR. The draw tests `req.active` once; the + * default-look image and its staging buffer are created on the first + * request that needs them. The pipelines are the compose pass's own. + */ + struct + { + //! One-frame request from the compositor, consumed by the next draw. + struct comp_vk_native_cursor_draw req; + //! The default look's sprite: R8G8B8A8 `_SRGB` (UNORM under the + //! legacy colour hatch, which samples every source undecoded), + //! steady-state SHADER_READ_ONLY_OPTIMAL once uploaded. + VkImage default_image; + VkDeviceMemory default_memory; + VkImageView default_view; + uint32_t default_w, default_h; + //! Host-visible staging for the next upload. + VkBuffer staging; + VkDeviceMemory staging_memory; + VkDeviceSize staging_size; + //! The staged pixels still have to be copied into default_image. + bool upload_pending; + //! default_image has been uploaded at least once. + bool default_ready; + bool warned_no_pass; + bool announced; + } cursor; }; /*! @@ -936,6 +968,9 @@ comp_vk_native_renderer_create(struct comp_vk_native_compositor *c, return XRT_SUCCESS; } +static void +cursor_destroy(struct comp_vk_native_renderer *r); + void comp_vk_native_renderer_destroy(struct comp_vk_native_renderer **renderer_ptr) { @@ -950,6 +985,7 @@ comp_vk_native_renderer_destroy(struct comp_vk_native_renderer **renderer_ptr) destroy_atlas_resources(r); zone_draw_destroy(r); + cursor_destroy(r); // XR_DXR_cursor_depth v5 (all null unless ever requested) // After destroy_atlas_resources — that call drops the borrowed handles // into the deposit's ring, this one frees the ring itself. comp_vk_deposit_destroy(&r->deposit); @@ -1317,14 +1353,16 @@ zone_draw_ensure(struct comp_vk_native_renderer *r) return false; } + // + 1: the runtime-drawn cursor's set (XR_DXR_cursor_depth v5), so a + // frame that exhausts the layer budget still has one for the cursor. VkDescriptorPoolSize pool_size = { .type = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER, - .descriptorCount = VK_ZONE_MAX_DRAWS, + .descriptorCount = VK_ZONE_MAX_DRAWS + 1, }; VkDescriptorPoolCreateInfo dp_ci = { .sType = VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO, - .maxSets = VK_ZONE_MAX_DRAWS, + .maxSets = VK_ZONE_MAX_DRAWS + 1, .poolSizeCount = 1, .pPoolSizes = &pool_size, }; @@ -1771,6 +1809,333 @@ renderer_submit_draw_and_wait(struct comp_vk_native_renderer *r, VkSubmitInfo *s return VK_SUCCESS; } +/*! + * The view the compose pass SAMPLES a swapchain image through - the ONE + * source-view choice (#1589), for every layer and the cursor sprite alike. + * + * The view in the format the app asked for, so the GPU DECODES an `_SRGB` + * source to linear and passes a UNORM one through as the linear values it + * holds; the pass composites in linear light and its attachment re-encodes on + * write, so this is the read half of a matched pair. Under the legacy hatch + * the pass blends in encoded space, so the non-decoding view. + */ +static uint64_t +compose_source_view(struct xrt_swapchain *xsc, uint32_t image_index) +{ + return u_color_legacy_unorm_encoded() ? comp_vk_native_swapchain_get_image_view(xsc, image_index) + : comp_vk_native_swapchain_get_true_image_view(xsc, image_index); +} + +/* + * + * XR_DXR_cursor_depth v5 - the runtime draws the cursor (ADR-046 Amendment 2 B) + * + * The cursor rides the compose pass: one more alpha-blended quad per view tile + * after every layer the pass draws, through the pass's own pipelines (the QUAD + * vertex path with an affine MVP), into the private `_SRGB` compose target - + * so it blends in linear light, its source is sampled as its declared format, + * and the atlas receives ENCODED bytes by the pass's raw copy, exactly like + * every other layer. Never double-encoded, never a second colour rule. + * + */ + +static void +cursor_destroy(struct comp_vk_native_renderer *r) +{ + struct vk_bundle *vk = r->vk; + if (r->cursor.default_view != VK_NULL_HANDLE) { + vk->vkDestroyImageView(vk->device, r->cursor.default_view, NULL); + r->cursor.default_view = VK_NULL_HANDLE; + } + if (r->cursor.default_image != VK_NULL_HANDLE) { + vk->vkDestroyImage(vk->device, r->cursor.default_image, NULL); + r->cursor.default_image = VK_NULL_HANDLE; + } + if (r->cursor.default_memory != VK_NULL_HANDLE) { + vk->vkFreeMemory(vk->device, r->cursor.default_memory, NULL); + r->cursor.default_memory = VK_NULL_HANDLE; + } + if (r->cursor.staging != VK_NULL_HANDLE) { + vk->vkDestroyBuffer(vk->device, r->cursor.staging, NULL); + r->cursor.staging = VK_NULL_HANDLE; + } + if (r->cursor.staging_memory != VK_NULL_HANDLE) { + vk->vkFreeMemory(vk->device, r->cursor.staging_memory, NULL); + r->cursor.staging_memory = VK_NULL_HANDLE; + } + r->cursor.staging_size = 0; + r->cursor.default_w = 0; + r->cursor.default_h = 0; + r->cursor.upload_pending = false; + r->cursor.default_ready = false; +} + +void +comp_vk_native_renderer_set_cursor(struct comp_vk_native_renderer *r, const struct comp_vk_native_cursor_draw *cursor) +{ + if (r == NULL || cursor == NULL) { + return; + } + r->cursor.req = *cursor; +} + +bool +comp_vk_native_renderer_set_cursor_default_image(struct comp_vk_native_renderer *r, + const uint8_t *rgba, + uint32_t width, + uint32_t height) +{ + if (r == NULL || rgba == NULL || width == 0 || height == 0) { + return false; + } + struct vk_bundle *vk = r->vk; + VkResult res; + + // A new size needs a new image. Only ever called between draws, and every + // draw waits its own fence, so nothing in flight references the old one. + if (r->cursor.default_image != VK_NULL_HANDLE && + (r->cursor.default_w != width || r->cursor.default_h != height)) { + vk->vkDestroyImageView(vk->device, r->cursor.default_view, NULL); + vk->vkDestroyImage(vk->device, r->cursor.default_image, NULL); + vk->vkFreeMemory(vk->device, r->cursor.default_memory, NULL); + r->cursor.default_view = VK_NULL_HANDLE; + r->cursor.default_image = VK_NULL_HANDLE; + r->cursor.default_memory = VK_NULL_HANDLE; + r->cursor.default_ready = false; + } + if (r->cursor.default_image == VK_NULL_HANDLE) { + // The bytes are sRGB-ENCODED. Sampled decoded (the `_SRGB` format) + // into the linear-blend pass; under the legacy hatch the pass blends + // in encoded space and every source is read undecoded, so UNORM. + const VkFormat fmt = + u_color_legacy_unorm_encoded() ? VK_FORMAT_R8G8B8A8_UNORM : VK_FORMAT_R8G8B8A8_SRGB; + res = vk_create_image_simple(vk, (VkExtent2D){width, height}, fmt, + VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_TRANSFER_DST_BIT, + &r->cursor.default_memory, &r->cursor.default_image); + if (res != VK_SUCCESS) { + U_LOG_E("cursor_draw: default-look image %ux%u failed: %d", width, height, res); + r->cursor.default_image = VK_NULL_HANDLE; + r->cursor.default_memory = VK_NULL_HANDLE; + return false; + } + const VkImageSubresourceRange range = {VK_IMAGE_ASPECT_COLOR_BIT, 0, 1, 0, 1}; + res = vk_create_view(vk, r->cursor.default_image, VK_IMAGE_VIEW_TYPE_2D, fmt, range, + &r->cursor.default_view); + if (res != VK_SUCCESS) { + U_LOG_E("cursor_draw: default-look view failed: %d", res); + cursor_destroy(r); + return false; + } + r->cursor.default_w = width; + r->cursor.default_h = height; + } + + const VkDeviceSize size = (VkDeviceSize)width * height * 4; + if (r->cursor.staging_size < size) { + if (r->cursor.staging != VK_NULL_HANDLE) { + vk->vkDestroyBuffer(vk->device, r->cursor.staging, NULL); + vk->vkFreeMemory(vk->device, r->cursor.staging_memory, NULL); + r->cursor.staging = VK_NULL_HANDLE; + r->cursor.staging_memory = VK_NULL_HANDLE; + r->cursor.staging_size = 0; + } + if (!vk_buffer_init(vk, size, VK_BUFFER_USAGE_TRANSFER_SRC_BIT, + VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT, + &r->cursor.staging, &r->cursor.staging_memory)) { + U_LOG_E("cursor_draw: default-look staging buffer failed"); + r->cursor.staging = VK_NULL_HANDLE; + r->cursor.staging_memory = VK_NULL_HANDLE; + return false; + } + r->cursor.staging_size = size; + } + void *mapped = NULL; + res = vk->vkMapMemory(vk->device, r->cursor.staging_memory, 0, size, 0, &mapped); + if (res != VK_SUCCESS || mapped == NULL) { + return false; + } + memcpy(mapped, rgba, (size_t)size); + vk->vkUnmapMemory(vk->device, r->cursor.staging_memory); + r->cursor.upload_pending = true; + return true; +} + +/*! + * Before the pass: copy staged default-look pixels into its image (only when + * they changed), and resolve the view + sampler kind the cursor samples. + * + * @param[out] out_app_image The app sprite's VkImage, to transition for + * sampling with the layers' sources; VK_NULL_HANDLE otherwise. + * @return false if there is nothing to sample this frame. + */ +static bool +cursor_prepare(struct comp_vk_native_renderer *r, + VkCommandBuffer cmd, + const struct comp_vk_native_cursor_draw *cur, + VkImageView *out_view, + enum vk_compose_sampler *out_sampler, + VkImage *out_app_image) +{ + struct vk_bundle *vk = r->vk; + *out_app_image = VK_NULL_HANDLE; + *out_sampler = VK_COMPOSE_SAMPLER_2D; + + if (cur->xsc != NULL) { + VkImage img = (VkImage)(uintptr_t)comp_vk_native_swapchain_get_image(cur->xsc, cur->image_index); + // Same rule as every layer source (#1589): an `_SRGB` sprite + // decodes into the linear blend. + const uint64_t v = compose_source_view(cur->xsc, cur->image_index); + if (img == VK_NULL_HANDLE || v == 0) { + return false; + } + *out_view = (VkImageView)(uintptr_t)v; + *out_sampler = comp_vk_native_swapchain_get_array_size(cur->xsc) > 1 ? VK_COMPOSE_SAMPLER_2D_ARRAY + : VK_COMPOSE_SAMPLER_2D; + *out_app_image = img; + return true; + } + + if (r->cursor.default_image == VK_NULL_HANDLE) { + return false; + } + if (r->cursor.upload_pending) { + cmd_image_barrier( + vk, cmd, r->cursor.default_image, + r->cursor.default_ready ? VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL : VK_IMAGE_LAYOUT_UNDEFINED, + VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, VK_ACCESS_SHADER_READ_BIT, VK_ACCESS_TRANSFER_WRITE_BIT, + VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT); + const VkBufferImageCopy region = { + .imageSubresource = {VK_IMAGE_ASPECT_COLOR_BIT, 0, 0, 1}, + .imageExtent = {r->cursor.default_w, r->cursor.default_h, 1}, + }; + vk->vkCmdCopyBufferToImage(cmd, r->cursor.staging, r->cursor.default_image, + VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 1, ®ion); + cmd_image_barrier(vk, cmd, r->cursor.default_image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, + VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL, VK_ACCESS_TRANSFER_WRITE_BIT, + VK_ACCESS_SHADER_READ_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, + VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT); + r->cursor.upload_pending = false; + r->cursor.default_ready = true; + } + if (!r->cursor.default_ready) { + return false; + } + *out_view = r->cursor.default_view; + return true; +} + +/*! + * Inside the pass, after every layer: one quad per view tile. + * + * The compose vertex shader's QUAD path maps the unit square's + * `pos = (u - 0.5, 0.5 - v)` through `mvp`; an affine mvp built from three of + * the sprite's tile-normalised corners puts texel (u, v) exactly where + * u_cursor_draw_corners() said, for any per-view map (Vulkan NDC +y runs down + * the viewport, the same direction as the tile's y). + */ +static void +cursor_record_draws(struct comp_vk_native_renderer *r, + VkCommandBuffer cmd, + const struct comp_vk_native_cursor_draw *cur, + VkImageView src_view, + enum vk_compose_sampler sampler_kind, + const struct comp_vk_native_eff_layout *layout) +{ + struct vk_bundle *vk = r->vk; + + VkDescriptorSetAllocateInfo ds_ai = { + .sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_ALLOCATE_INFO, + .descriptorPool = r->zone.descriptor_pool, + .descriptorSetCount = 1, + .pSetLayouts = &r->zone.set_layout, + }; + VkDescriptorSet set = VK_NULL_HANDLE; + if (vk->vkAllocateDescriptorSets(vk->device, &ds_ai, &set) != VK_SUCCESS) { + return; + } + VkDescriptorImageInfo image_info = { + .sampler = r->zone.sampler, + .imageView = src_view, + .imageLayout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL, + }; + VkWriteDescriptorSet write = { + .sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET, + .dstSet = set, + .dstBinding = 0, + .descriptorCount = 1, + .descriptorType = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER, + .pImageInfo = &image_info, + }; + vk->vkUpdateDescriptorSets(vk->device, 1, &write, 0, NULL); + + const enum vk_compose_blend blend = cur->premultiplied ? VK_COMPOSE_BLEND_PREMULT : VK_COMPOSE_BLEND_UNPREMULT; + vk->vkCmdBindPipeline(cmd, VK_PIPELINE_BIND_POINT_GRAPHICS, r->zone.pipelines[blend][sampler_kind]); + vk->vkCmdBindDescriptorSets(cmd, VK_PIPELINE_BIND_POINT_GRAPHICS, r->zone.pipeline_layout, 0, 1, &set, 0, NULL); + + // Mono content fills the whole content region (one tile box); otherwise + // view i owns tile i of the effective grid (#542). + const bool mono = layout->views <= 1; + const uint32_t tiles = mono ? 1 : layout->cols * layout->rows; + const uint32_t n = cur->view_count < tiles ? cur->view_count : tiles; + for (uint32_t i = 0; i < n; i++) { + float bx, by, bw, bh; + if (mono) { + bx = 0.0f; + by = 0.0f; + bw = (float)(layout->cols * layout->tile_w); + bh = (float)(layout->rows * layout->tile_h); + } else { + bx = (float)((i % layout->cols) * layout->tile_w); + by = (float)((i / layout->cols) * layout->tile_h); + bw = (float)layout->tile_w; + bh = (float)layout->tile_h; + } + if (bw <= 0.0f || bh <= 0.0f || bx >= (float)r->atlas_alloc_width || + by >= (float)r->atlas_alloc_height) { + continue; + } + VkViewport vp = {.x = bx, .y = by, .width = bw, .height = bh, .minDepth = 0.0f, .maxDepth = 1.0f}; + // The scissor is mandatory: a cursor at a tile edge must not spill + // into the neighbour view (the DP would weave it into the wrong eye). + const float sx1 = bx + bw > (float)r->atlas_alloc_width ? (float)r->atlas_alloc_width : bx + bw; + const float sy1 = by + bh > (float)r->atlas_alloc_height ? (float)r->atlas_alloc_height : by + bh; + VkRect2D scissor = { + .offset = {(int32_t)bx, (int32_t)by}, + .extent = {(uint32_t)(sx1 - bx), (uint32_t)(sy1 - by)}, + }; + + const float(*c)[2] = cur->corners[i]; + const float ex[2] = {c[2][0] - c[0][0], c[2][1] - c[0][1]}; // u: (0,0) -> (1,0) + const float ey[2] = {c[1][0] - c[0][0], c[1][1] - c[0][1]}; // v: (0,0) -> (0,1) + const float mid[2] = {c[0][0] + 0.5f * (ex[0] + ey[0]), c[0][1] + 0.5f * (ex[1] + ey[1])}; + // Column-major: NDC = 2 * (mid + pos.x * ex - pos.y * ey) - 1. + struct vk_compose_push push = { + .mvp = {2.0f * ex[0], 2.0f * ex[1], 0.0f, 0.0f, // + -2.0f * ey[0], -2.0f * ey[1], 0.0f, 0.0f, // + 0.0f, 0.0f, 1.0f, 0.0f, // + 2.0f * mid[0] - 1.0f, 2.0f * mid[1] - 1.0f, 0.0f, 1.0f}, + .src_rect = {0.0f, 0.0f, 1.0f, 1.0f}, + .params = {0.0f, 1.0f, 0.0f, 0.0f}, // array slice 0, QUAD path + .color_scale = {1.0f, 1.0f, 1.0f, 1.0f}, + .color_bias = {0.0f, 0.0f, 0.0f, 0.0f}, + }; + vk->vkCmdSetViewport(cmd, 0, 1, &vp); + vk->vkCmdSetScissor(cmd, 0, 1, &scissor); + vk->vkCmdPushConstants(cmd, r->zone.pipeline_layout, + VK_SHADER_STAGE_VERTEX_BIT | VK_SHADER_STAGE_FRAGMENT_BIT, 0, sizeof(push), + &push); + vk->vkCmdDraw(cmd, 6, 1, 0, 0); + } + + if (!r->cursor.announced) { + r->cursor.announced = true; + U_LOG_W( + "cursor_draw: Vulkan compositor drawing the cursor (%s, %u view(s); ADR-046 Amendment 2 B, " + "one-time notice)", + cur->xsc != NULL ? "app sprite" : "default look", n); + } +} + //! Draw-based projection pass for zones frames: clear via loadOp, draw each //! zone's view tiles alpha-over in layer-list order, finish in //! SHADER_READ_ONLY. Mirrors comp_d3d11_renderer's zone branch. @@ -1779,7 +2144,8 @@ draw_zones_pass(struct comp_vk_native_renderer *r, struct comp_layer_accum *layers, uint32_t target_width, uint32_t target_height, - const struct comp_vk_native_eff_layout *layout) + const struct comp_vk_native_eff_layout *layout, + const struct comp_vk_native_cursor_draw *cursor) { struct vk_bundle *vk = r->vk; @@ -1795,7 +2161,7 @@ draw_zones_pass(struct comp_vk_native_renderer *r, // Transition each unique source image for sampling. A zone's view tiles // share one swapchain image, so dedupe — re-transitioning would declare // a wrong oldLayout. - VkImage transitioned[VK_ZONE_MAX_DRAWS]; + VkImage transitioned[VK_ZONE_MAX_DRAWS + 1]; uint32_t transitioned_count = 0; for (uint32_t i = 0; i < layers->layer_count; i++) { struct comp_layer *layer = &layers->layers[i]; @@ -1851,6 +2217,34 @@ draw_zones_pass(struct comp_vk_native_renderer *r, } } + /* + * XR_DXR_cursor_depth v5: the cursor's source, resolved (and the default + * look uploaded) outside the pass. An app sprite is transitioned with the + * layers' sources - and handed back with them below - unless a layer + * already samples the same image. + */ + VkImageView cursor_view = VK_NULL_HANDLE; + enum vk_compose_sampler cursor_sampler = VK_COMPOSE_SAMPLER_2D; + bool cursor_ok = false; + if (cursor != NULL && cursor->active) { + VkImage cursor_img = VK_NULL_HANDLE; + cursor_ok = cursor_prepare(r, cmd, cursor, &cursor_view, &cursor_sampler, &cursor_img); + if (cursor_ok && cursor_img != VK_NULL_HANDLE) { + bool seen = false; + for (uint32_t t = 0; t < transitioned_count; t++) { + seen = seen || transitioned[t] == cursor_img; + } + if (!seen) { + transitioned[transitioned_count++] = cursor_img; + cmd_image_barrier(vk, cmd, cursor_img, VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL, + VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL, + VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT, VK_ACCESS_SHADER_READ_BIT, + VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT, + VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT); + } + } + } + /* * Clear alpha must follow the SAME rule as the blit path, not a * hardcoded 0. @@ -2058,10 +2452,7 @@ draw_zones_pass(struct comp_vk_native_renderer *r, * and #1572's UNORM scratch; neither is wired in here, * and wiring them in would double-encode. */ - const uint64_t src_view_u64 = - u_color_legacy_unorm_encoded() - ? comp_vk_native_swapchain_get_image_view(xsc, sub->image_index) - : comp_vk_native_swapchain_get_true_image_view(xsc, sub->image_index); + const uint64_t src_view_u64 = compose_source_view(xsc, sub->image_index); VkImageView src_view = (VkImageView)(uintptr_t)src_view_u64; if (src_view == VK_NULL_HANDLE) { continue; @@ -2286,6 +2677,14 @@ draw_zones_pass(struct comp_vk_native_renderer *r, } } + // XR_DXR_cursor_depth v5 (ADR-046 Amendment 2 B): the runtime-drawn + // cursor, LAST in the pass - over every layer it composed, never + // occluded - and before the atlas is published to the DP. Zones frames + // are refused upstream (the state tracker) and here. + if (cursor_ok && !zones_frame) { + cursor_record_draws(r, cmd, cursor, cursor_view, cursor_sampler, layout); + } + vk->vkCmdEndRenderPass(cmd); /* @@ -2471,11 +2870,27 @@ comp_vk_native_renderer_draw(struct comp_vk_native_renderer *r, const bool structure_allows_raw = !zones_frame && (drawable == 0 || (drawable == 1 && compose_frame_base_is_projection(layers))); - const bool want_pass = - !(colour_allows_raw && structure_allows_raw) || debug_get_bool_option_vk_force_compose_pass(); + /* + * XR_DXR_cursor_depth v5 (ADR-046 Amendment 2 B): consumed by THIS draw + * only. A frame carrying the runtime-drawn cursor composes through the + * pass - the blit cannot blend - so the cursor is drawn after the layers + * it composes. ZERO COST UNLESS REQUESTED: one field test. + */ + struct comp_vk_native_cursor_draw cursor = {0}; + if (r->cursor.req.active) { + cursor = r->cursor.req; + r->cursor.req.active = false; + } + + const bool want_pass = !(colour_allows_raw && structure_allows_raw) || + debug_get_bool_option_vk_force_compose_pass() || cursor.active; if (want_pass && zone_pass_usable(r, layers, layout)) { - return draw_zones_pass(r, layers, target_width, target_height, layout); + return draw_zones_pass(r, layers, target_width, target_height, layout, cursor.active ? &cursor : NULL); + } + if (cursor.active && !r->cursor.warned_no_pass) { + r->cursor.warned_no_pass = true; + U_LOG_W("cursor_draw: the Vulkan compose pass is unavailable - cursor not drawn (one-time warning)"); } if (want_pass) { static bool fallback_warned = false; diff --git a/src/xrt/compositor/vk_native/comp_vk_native_renderer.h b/src/xrt/compositor/vk_native/comp_vk_native_renderer.h index bb735fd952..435fa98584 100644 --- a/src/xrt/compositor/vk_native/comp_vk_native_renderer.h +++ b/src/xrt/compositor/vk_native/comp_vk_native_renderer.h @@ -10,6 +10,7 @@ #pragma once #include "xrt/xrt_defines.h" +#include "xrt/xrt_limits.h" #include "xrt/xrt_display_metrics.h" #include "xrt/xrt_results.h" @@ -21,6 +22,7 @@ struct comp_vk_native_renderer; struct comp_vk_native_compositor; struct comp_layer_accum; struct comp_vk_deposit; +struct xrt_swapchain; #ifdef __cplusplus extern "C" { @@ -336,6 +338,63 @@ void comp_vk_native_renderer_set_transparent(struct comp_vk_native_renderer *renderer, bool transparent_background); +/*! + * XR_DXR_cursor_depth v5 (ADR-046 Amendment 2 B): one frame's runtime-drawn + * cursor, resolved by the compositor - where the sprite's corners land in + * each view's tile and which image to sample. Zero-initialised = no cursor. + */ +struct comp_vk_native_cursor_draw +{ + //! Draw at the next comp_vk_native_renderer_draw only. + bool active; + //! Views mapped below; view i is drawn into atlas tile i. + uint32_t view_count; + /*! + * Per view, the sprite's corners in TILE-NORMALISED coordinates (origin + * top-left, y down), in texture order (0,0), (0,1), (1,0), (1,1) - the + * u_cursor_draw_corners() output. + */ + float corners[XRT_MAX_VIEWS][4][2]; + //! The app's sprite swapchain (sampled at @ref image_index, array layer + //! 0) or NULL for the renderer's default-look image. + struct xrt_swapchain *xsc; + uint32_t image_index; + //! The source holds premultiplied alpha (the default look) rather than + //! straight (the app's sprite, per the spec). + bool premultiplied; +}; + +/*! + * Hand the renderer the cursor to draw at its NEXT draw (consumed there). + * + * The draw forces the compose render pass for that frame (the blit fast path + * cannot blend) and draws one alpha-blended quad per view tile AFTER every + * layer the pass composes (projection, quad, equirect2) and before the atlas + * is published - blending in linear light through the pass's private `_SRGB` + * target, sampling the sprite as its declared format, so the atlas stays + * ENCODED and is never double-encoded. Nothing is built unless this is called. + * + * @ingroup comp_vk_native + */ +void +comp_vk_native_renderer_set_cursor(struct comp_vk_native_renderer *renderer, + const struct comp_vk_native_cursor_draw *cursor); + +/*! + * Upload the default look's pixels (premultiplied, sRGB-ENCODED RGBA8, top + * row first): staged now, copied into the renderer's own sprite image by the + * next draw. Call only when the look's image changed. + * + * @return false if the image or its staging could not be created. + * + * @ingroup comp_vk_native + */ +bool +comp_vk_native_renderer_set_cursor_default_image(struct comp_vk_native_renderer *renderer, + const uint8_t *rgba, + uint32_t width, + uint32_t height); + #ifdef __cplusplus } #endif diff --git a/src/xrt/state_trackers/oxr/oxr_session.c b/src/xrt/state_trackers/oxr/oxr_session.c index 80709076bf..371a23461e 100644 --- a/src/xrt/state_trackers/oxr/oxr_session.c +++ b/src/xrt/state_trackers/oxr/oxr_session.c @@ -2319,9 +2319,9 @@ oxr_session_cursor_depth_hand_request(struct oxr_session *sess, const struct u_c /* * XR_DXR_cursor_depth v5 - the runtime draws the cursor (ADR-046 Amendment 2 - * B). Metal (in-process) first; the other native compositors and the - * service report "cannot draw", so locates report willDraw = false and the - * app keeps its flat OS cursor - never no cursor. + * B). The in-process Metal, vk_native and GL compositors draw; the others and + * the service report "cannot draw", so locates report willDraw = false and + * the app keeps its flat OS cursor - never no cursor. */ bool oxr_session_cursor_draw_supported(struct oxr_session *sess) @@ -2334,8 +2334,18 @@ oxr_session_cursor_draw_supported(struct oxr_session *sess) return true; } #endif - // TODO(ADR-046 Amendment 2 B, rollout step 3): D3D11, D3D12, GL, - // vk_native and the IPC/service path (svc_draw_cursor_quads). +#ifdef XRT_HAVE_VK_NATIVE_COMPOSITOR + if (sess->is_vk_native_compositor) { + return true; + } +#endif +#ifdef XRT_HAVE_GL_NATIVE_COMPOSITOR + if (sess->is_gl_native_compositor) { + return true; + } +#endif + // TODO(ADR-046 Amendment 2 B, rollout step 3): D3D11, D3D12 and the + // IPC/service path (svc_draw_cursor_quads). return false; } @@ -2350,6 +2360,18 @@ oxr_session_cursor_draw_hand_request(struct oxr_session *sess, const struct u_cu comp_metal_compositor_set_cursor_draw_request(&sess->xcn->base, req); return true; } +#endif +#ifdef XRT_HAVE_VK_NATIVE_COMPOSITOR + if (sess->is_vk_native_compositor) { + comp_vk_native_compositor_set_cursor_draw_request(&sess->xcn->base, req); + return true; + } +#endif +#ifdef XRT_HAVE_GL_NATIVE_COMPOSITOR + if (sess->is_gl_native_compositor) { + comp_gl_compositor_set_cursor_draw_request(&sess->xcn->base, req); + return true; + } #endif (void)req; return false; diff --git a/test_apps/handle/cube_handle_gl_macos/CMakeLists.txt b/test_apps/handle/cube_handle_gl_macos/CMakeLists.txt index a55b10c461..50463c6211 100644 --- a/test_apps/handle/cube_handle_gl_macos/CMakeLists.txt +++ b/test_apps/handle/cube_handle_gl_macos/CMakeLists.txt @@ -60,7 +60,9 @@ set(DISPLAYXR_EXTENSIONS_INCLUDE_DIR "${_dxr_ext_includes}" CACHE PATH FetchContent_Declare( displayxr_common GIT_REPOSITORY https://github.com/DisplayXR/displayxr-common.git - GIT_TAG v2.0.0 + # v2.29.0: dxr::CursorDepth (common/cursor_depth.h), the XR_DXR_cursor_depth + # app helper (ADR-046 Amendment 2) - same pin as the aggregate test_apps build. + GIT_TAG v2.29.0 GIT_SHALLOW TRUE ) FetchContent_MakeAvailable(displayxr_common) diff --git a/test_apps/handle/cube_handle_gl_macos/main.mm b/test_apps/handle/cube_handle_gl_macos/main.mm index a0f12cae7d..b36286e4d5 100644 --- a/test_apps/handle/cube_handle_gl_macos/main.mm +++ b/test_apps/handle/cube_handle_gl_macos/main.mm @@ -32,6 +32,7 @@ #include #include #include +#include #include #include @@ -48,6 +49,8 @@ #include #include #include +#include +#include "cursor_depth.h" // dxr::CursorDepth — XR_DXR_cursor_depth bookkeeping (ADR-046) #include "dxr_view_config.h" // #1486 PRIMARY_MULTIVIEW_DXR opt-in // ============================================================================ @@ -432,6 +435,10 @@ static GLuint LinkProgram(GLuint vs, GLuint fs) GLuint cubeVAO, cubeVBO, cubeEBO; GLuint gridVAO, gridVBO; int gridVertexCount; + // XR_DXR_cursor_depth app-drawn sprite (DISPLAYXR_CURSOR_DEPTH_DRAW=app): + // crosshair lines, drawn with the grid program. + GLuint cursorVAO = 0, cursorVBO = 0; + int cursorVertexCount = 0; // Textures (GL_TEXTURE_RECTANGLE for loaded images) GLuint textures[3]; // basecolor, normal, AO @@ -571,6 +578,27 @@ static bool InitRenderer(GLRenderer &r) glBindVertexArray(0); + // XR_DXR_cursor_depth app-drawn sprite: crosshair + small square in its + // XY plane, unit = the sprite height the runtime returns (same lines as + // cube_handle_vk_macos). + { + static const GridVertex cursorLines[] = { + {{-0.5f, 0.0f, 0.0f}}, {{-0.15f, 0.0f, 0.0f}}, {{0.15f, 0.0f, 0.0f}}, {{0.5f, 0.0f, 0.0f}}, + {{0.0f, -0.5f, 0.0f}}, {{0.0f, -0.15f, 0.0f}}, {{0.0f, 0.15f, 0.0f}}, {{0.0f, 0.5f, 0.0f}}, + {{-0.15f, -0.15f, 0.0f}}, {{0.15f, -0.15f, 0.0f}}, {{0.15f, -0.15f, 0.0f}}, {{0.15f, 0.15f, 0.0f}}, + {{0.15f, 0.15f, 0.0f}}, {{-0.15f, 0.15f, 0.0f}}, {{-0.15f, 0.15f, 0.0f}}, {{-0.15f, -0.15f, 0.0f}}, + }; + r.cursorVertexCount = (int)(sizeof(cursorLines) / sizeof(cursorLines[0])); + glGenVertexArrays(1, &r.cursorVAO); + glBindVertexArray(r.cursorVAO); + glGenBuffers(1, &r.cursorVBO); + glBindBuffer(GL_ARRAY_BUFFER, r.cursorVBO); + glBufferData(GL_ARRAY_BUFFER, sizeof(cursorLines), cursorLines, GL_STATIC_DRAW); + glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, sizeof(GridVertex), (void *)0); + glEnableVertexAttribArray(0); + glBindVertexArray(0); + } + // Load textures std::string texDir = GetTextureDir(); r.textures[0] = LoadTextureRect((texDir + "Wood_Crate_001_basecolor.jpg").c_str(), @@ -613,6 +641,83 @@ static void EnsureFBO(GLRenderer &r, uint32_t w, uint32_t h) // Render scene into swapchain texture (GL_TEXTURE_RECTANGLE) // ============================================================================ +// ============================================================================ +// XR_DXR_cursor_depth (ADR-046) — same adoption as cube_handle_metal_macos. +// +// The app owns the depth: it answers "ray -> first cube hit" (RayHitCube) and +// displayxr-common's dxr::CursorDepth does the rest — the pointer as a canvas +// UV, the footprint rays from the outer eyes of the previous locate, the +// nearest hit, the struct chain on xrLocateViews, and hiding the OS cursor +// exactly while the depth cursor replaces it. Spec v5 (ADR-046 +// Amendment 2 B), the default: the runtime DRAWS the cursor +// (XrCursorDepthDrawInfoDXR) - its default look, or our own sprite with +// DISPLAYXR_CURSOR_DEPTH_SPRITE=custom - and the app hides the OS cursor +// exactly while XrCursorDepthDrawStateDXR::willDraw. +// DISPLAYXR_CURSOR_DEPTH_DRAW=app draws a crosshair from the placement +// instead. OPT-IN here: DISPLAYXR_CURSOR_DEPTH=1 (runtime#1947); DISPLAYXR_CURSOR_DEPTH_UV=u,v +// scripts the cursor for headless capture. +// ============================================================================ + +static const float kCubeSize = 0.06f; +static const float kCursorHeight = 0.04f; // fraction of canvas height + +struct CursorDepthDraw { + bool active = false; + XrPosef pose = {{0, 0, 0, 1}, {0, 0, 0}}; + float height = 0.0f; +}; +static CursorDepthDraw g_cursorDraw; + +// Slab test against the cube as RenderScene places it: centred at +// (0, size/2, 0), edge `size`, rotated `rot` about +Y (mat4_rotation_y). +static bool RayHitCube(const XrVector3f &o, const XrVector3f &d, float rot, float &tOut) +{ + const float c = cosf(rot), s = sinf(rot), h = kCubeSize * 0.5f; + // local = R^T (world - centre); R's columns are (c,0,-s), (0,1,0), (s,0,c). + const float ox = o.x, oy = o.y - h, oz = o.z; + const float lo[3] = {c * ox - s * oz, oy, s * ox + c * oz}; + const float ld[3] = {c * d.x - s * d.z, d.y, s * d.x + c * d.z}; + float t0 = 0.0f, t1 = 1e30f; + for (int i = 0; i < 3; i++) { + if (fabsf(ld[i]) < 1e-9f) { + if (lo[i] < -h || lo[i] > h) return false; + continue; + } + float ta = (-h - lo[i]) / ld[i], tb = (h - lo[i]) / ld[i]; + if (ta > tb) std::swap(ta, tb); + if (ta > t0) t0 = ta; + if (tb < t1) t1 = tb; + if (t0 > t1) return false; + } + tOut = t0; + return true; +} + +// Model matrix of the app-drawn cursor sprite: pose (rotation then +// translation), scaled by the returned height. Column-major, like the other +// mat4_* helpers. +static void CursorSpriteModel(float *out) +{ + const XrQuaternionf q = g_cursorDraw.pose.orientation; + float rot[16]; + mat4_identity(rot); + rot[0] = 1 - 2 * (q.y * q.y + q.z * q.z); + rot[1] = 2 * (q.x * q.y + q.z * q.w); + rot[2] = 2 * (q.x * q.z - q.y * q.w); + rot[4] = 2 * (q.x * q.y - q.z * q.w); + rot[5] = 1 - 2 * (q.x * q.x + q.z * q.z); + rot[6] = 2 * (q.y * q.z + q.x * q.w); + rot[8] = 2 * (q.x * q.z + q.y * q.w); + rot[9] = 2 * (q.y * q.z - q.x * q.w); + rot[10] = 1 - 2 * (q.x * q.x + q.y * q.y); + float trans[16], scl[16], tmp[16]; + mat4_translation(trans, g_cursorDraw.pose.position.x, g_cursorDraw.pose.position.y, + g_cursorDraw.pose.position.z); + mat4_scaling(scl, g_cursorDraw.height); + mat4_multiply(tmp, rot, scl); + mat4_multiply(out, trans, tmp); +} + static void RenderScene(GLRenderer &r, GLuint targetTex, uint32_t targetW, uint32_t targetH, const EyeRenderParams *eyes, int eyeCount) { @@ -706,6 +811,21 @@ static void RenderScene(GLRenderer &r, GLuint targetTex, uint32_t targetW, uint3 glBindVertexArray(r.gridVAO); glDrawArrays(GL_LINES, 0, r.gridVertexCount); } + + // --- Depth-aware cursor, app-drawn (DISPLAYXR_CURSOR_DEPTH_DRAW=app): + // last, never occluded. In runtime mode the runtime draws it. --- + if (g_cursorDraw.active && r.cursorVAO != 0) { + float model[16], mvp[16]; + CursorSpriteModel(model); + mat4_multiply(mvp, vp_mat, model); + glDisable(GL_DEPTH_TEST); + glUseProgram(r.gridProgram); + glUniformMatrix4fv(r.gridLocMVP, 1, GL_FALSE, mvp); + glUniform4f(r.gridLocColor, 1.0f, 0.85f, 0.1f, 1.0f); // display-referred amber + glBindVertexArray(r.cursorVAO); + glDrawArrays(GL_LINES, 0, r.cursorVertexCount); + glEnable(GL_DEPTH_TEST); + } } glDisable(GL_SCISSOR_TEST); @@ -1186,6 +1306,18 @@ static void UpdateCameraMovement(InputState& state, float deltaTime, float displ // XR_DXR_atlas_capture (W6 of #396): runtime-owned 'I'-key atlas capture. bool hasAtlasCaptureExt = false; bool hasViewRigExt = false; // XR_DXR_view_rig (#396 W7) + bool hasCursorDepthExt = false; // XR_DXR_cursor_depth (ADR-046) + uint32_t cursorDepthSpecVersion = 0; + // XR_DXR_cursor_depth v5 (ADR-046 Amendment 2 B): + // DISPLAYXR_CURSOR_DEPTH_DRAW=runtime (default) | app. Runtime: chain + // XrCursorDepthDrawInfoDXR and let the runtime draw the cursor; hide the + // OS cursor exactly while XrCursorDepthDrawStateDXR::willDraw. App: draw + // the crosshair ourselves from the placement. + bool cursorDepthRuntimeDraw = false; + // DISPLAYXR_CURSOR_DEPTH_SPRITE=custom: hand the runtime our own sprite + // (a cyan ring with a dot) instead of its default look. + bool cursorDepthCustomSprite = false; + XrSwapchain cursorSpriteSwapchain = XR_NULL_HANDLE; PFN_xrCaptureAtlasDXR pfnCaptureAtlasEXT = nullptr; // Enumerated rendering mode info. currentModeIndex is initialized to mode 1 // as a fallback for runtimes that don't expose isActive; v13+ runtimes @@ -1240,6 +1372,10 @@ static bool InitializeOpenXR(AppXrSession &app) app.hasAtlasCaptureExt = true; if (strcmp(e.extensionName, XR_DXR_VIEW_RIG_EXTENSION_NAME) == 0) app.hasViewRigExt = true; + if (strcmp(e.extensionName, XR_DXR_CURSOR_DEPTH_EXTENSION_NAME) == 0) { + app.hasCursorDepthExt = true; + app.cursorDepthSpecVersion = e.extensionVersion; + } } if (!app.hasMacosGlBinding) { @@ -1270,6 +1406,38 @@ static bool InitializeOpenXR(AppXrSession &app) if (app.hasViewRigExt) { enabledExts.push_back(XR_DXR_VIEW_RIG_EXTENSION_NAME); } + // XR_DXR_cursor_depth (ADR-046): OPT-IN on this app only - + // DISPLAYXR_CURSOR_DEPTH=1 enables it (the runtime otherwise does no + // cursor work at all). Unlike the Metal and Vulkan cubes (default on), the + // GL cube crashes ~1 in 5 launches in Apple's GL driver while the OS cursor + // is hidden - a GL compositor repaint-thread vs AppKit race, runtime#1947. + // Flip back to default-on once that is fixed. + if (app.hasCursorDepthExt) { + const char *opt = getenv("DISPLAYXR_CURSOR_DEPTH"); + if (opt != nullptr && strcmp(opt, "1") == 0) { + enabledExts.push_back(XR_DXR_CURSOR_DEPTH_EXTENSION_NAME); + } else { + app.hasCursorDepthExt = false; + } + } + LOG_INFO("XR_DXR_cursor_depth: %s", app.hasCursorDepthExt ? "ENABLED" : "off"); + if (app.hasCursorDepthExt) { + // Spec v5: who draws the cursor. Default: the runtime. + const char *drawWho = getenv("DISPLAYXR_CURSOR_DEPTH_DRAW"); + const bool wantApp = drawWho != nullptr && strcmp(drawWho, "app") == 0; + if (!wantApp && app.cursorDepthSpecVersion >= 5) { + app.cursorDepthRuntimeDraw = true; + const char *sprite = getenv("DISPLAYXR_CURSOR_DEPTH_SPRITE"); + app.cursorDepthCustomSprite = sprite != nullptr && strcmp(sprite, "custom") == 0; + } else if (!wantApp) { + LOG_WARN("XR_DXR_cursor_depth v%u has no runtime-drawn cursor (needs v5) - drawing it ourselves", + app.cursorDepthSpecVersion); + } + LOG_INFO("XR_DXR_cursor_depth draw: %s", + !app.cursorDepthRuntimeDraw ? "APP (crosshair from the placement)" + : app.cursorDepthCustomSprite ? "RUNTIME, custom sprite (cyan ring)" + : "RUNTIME, default look"); + } XrInstanceCreateInfo createInfo = {XR_TYPE_INSTANCE_CREATE_INFO}; strncpy(createInfo.applicationInfo.applicationName, "GLCubeExtOpenXR", @@ -1599,6 +1767,90 @@ static bool CreateSwapchain(AppXrSession &app) // (An app that genuinely hands over a top-down image says so with // XR_COMPOSITION_LAYER_IMAGE_LAYOUT_VERTICAL_FLIP_BIT_FB -> xrt_layer_data::flip_y, // which the compositor honours in post_transform.) +// ============================================================================ +// XR_DXR_cursor_depth v5 — a custom cursor sprite for the runtime to draw +// ============================================================================ + +// DISPLAYXR_CURSOR_DEPTH_SPRITE=custom: a 64x64 cyan ring with a centre dot +// (hotspot = the centre), dark-edged so it reads over the white cube. Filled +// once (acquire / fill / release); the runtime samples the last released +// image every frame, as STRAIGHT alpha. GL_SRGB8_ALPHA8 when offered: the +// bytes are display-referred. Uploaded BOTTOM row first - a GL image's top is +// its last row, which is how the runtime reads a GL sprite. +static bool CreateCursorSprite(AppXrSession &app) +{ + const uint32_t S = 64; + uint32_t formatCount = 0; + xrEnumerateSwapchainFormats(app.session, 0, &formatCount, nullptr); + std::vector formats(formatCount); + xrEnumerateSwapchainFormats(app.session, formatCount, &formatCount, formats.data()); + int64_t fmt = GL_RGBA8; + for (auto f : formats) { + if (f == GL_SRGB8_ALPHA8) fmt = GL_SRGB8_ALPHA8; + } + + XrSwapchainCreateInfo sci = {XR_TYPE_SWAPCHAIN_CREATE_INFO}; + sci.usageFlags = XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT | XR_SWAPCHAIN_USAGE_SAMPLED_BIT; + sci.format = fmt; + sci.sampleCount = 1; + sci.width = S; + sci.height = S; + sci.faceCount = 1; + sci.arraySize = 1; + sci.mipCount = 1; + if (XR_FAILED(xrCreateSwapchain(app.session, &sci, &app.cursorSpriteSwapchain))) { + return false; + } + uint32_t n = 0; + xrEnumerateSwapchainImages(app.cursorSpriteSwapchain, 0, &n, nullptr); + std::vector imgs(n, {XR_TYPE_SWAPCHAIN_IMAGE_OPENGL_KHR}); + if (n == 0 || XR_FAILED(xrEnumerateSwapchainImages(app.cursorSpriteSwapchain, n, &n, + (XrSwapchainImageBaseHeader *)imgs.data()))) { + return false; + } + XrSwapchainImageAcquireInfo ai = {XR_TYPE_SWAPCHAIN_IMAGE_ACQUIRE_INFO}; + uint32_t idx = 0; + if (XR_FAILED(xrAcquireSwapchainImage(app.cursorSpriteSwapchain, &ai, &idx))) { + return false; + } + XrSwapchainImageWaitInfo wi = {XR_TYPE_SWAPCHAIN_IMAGE_WAIT_INFO}; + wi.timeout = XR_INFINITE_DURATION; + xrWaitSwapchainImage(app.cursorSpriteSwapchain, &wi); + + std::vector buf((size_t)S * S * 4); + auto cover = [](float d, float r0, float r1) { // 1 inside [r0, r1], 1-px soft edges + const float a = std::min(d - r0 + 0.5f, r1 - d + 0.5f); + return a <= 0.0f ? 0.0f : (a >= 1.0f ? 1.0f : a); + }; + for (uint32_t y = 0; y < S; y++) { + for (uint32_t x = 0; x < S; x++) { + const float d = hypotf((float)x + 0.5f - S * 0.5f, (float)y + 0.5f - S * 0.5f); + const float ring = std::max(cover(d, 21.0f, 27.0f), cover(d, 0.0f, 5.0f)); // cyan + const float edge = std::max(cover(d, 19.5f, 28.5f), cover(d, 0.0f, 6.5f)); // dark rim + // Row y counted from the TOP of the sprite -> GL row S-1-y. + uint8_t *px = &buf[((size_t)(S - 1 - y) * S + x) * 4]; + // RGBA, straight alpha: cyan where the ring is, near-black rim. + px[0] = 0; + px[1] = (uint8_t)(220.0f * ring); + px[2] = (uint8_t)(255.0f * ring); + px[3] = (uint8_t)(255.0f * std::max(ring, edge)); + } + } + GLint prev = 0; + glGetIntegerv(GL_TEXTURE_BINDING_2D, &prev); + glBindTexture(GL_TEXTURE_2D, imgs[idx].image); + glPixelStorei(GL_UNPACK_ALIGNMENT, 4); + glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, (GLsizei)S, (GLsizei)S, GL_RGBA, GL_UNSIGNED_BYTE, buf.data()); + glBindTexture(GL_TEXTURE_2D, (GLuint)prev); + glFlush(); + + XrSwapchainImageReleaseInfo ri = {XR_TYPE_SWAPCHAIN_IMAGE_RELEASE_INFO}; + xrReleaseSwapchainImage(app.cursorSpriteSwapchain, &ri); + LOG_INFO("Cursor sprite swapchain created: %ux%u, format 0x%llx (cyan ring, hotspot = centre)", S, S, + (long long)fmt); + return true; +} + static bool CreateAndFillQuadTexture(AppXrSession &app, uint32_t size) { XrSwapchainCreateInfo sci = {XR_TYPE_SWAPCHAIN_CREATE_INFO}; @@ -1848,6 +2100,32 @@ int main(int argc, char **argv) LOG_ERROR("Failed to create swapchain"); return 1; } + if (app.cursorDepthRuntimeDraw && app.cursorDepthCustomSprite && !CreateCursorSprite(app)) { + LOG_WARN("Cursor sprite swapchain creation failed - using the runtime's default look"); + app.cursorDepthCustomSprite = false; + } + + // XR_DXR_cursor_depth (ADR-046): the displayxr-common helper. enable(0) + // (extension off / opted out) leaves it inert: no structs, no OS cursor. + dxr::CursorDepth cursor; + cursor.enable(app.hasCursorDepthExt ? app.cursorDepthSpecVersion : 0); + cursor.setWindowCocoa((__bridge void *)g_glView); + cursor.setCursorHeight(kCursorHeight); + cursor.setWantPlacement(true); // the diagnostics below + if (!app.cursorDepthRuntimeDraw) { + cursor.setDrawMode(dxr::CursorDepth::DrawMode::App); + cursor.setFootprintBodyOffset(dxr::kCursorCenteredBodyOffset); // our crosshair is centred + } else if (app.cursorDepthCustomSprite) { + cursor.setSprite(app.cursorSpriteSwapchain, {0.5f, 0.5f}, 0.01f); // ring: hotspot = centre, 1 cm + } + cursor.setRayCast([&renderer](const dxr::CursorRay &r) -> std::optional { + float t; + return RayHitCube(r.origin, r.dir, renderer.cubeRotation, t) ? std::optional(t) : std::nullopt; + }); + if (const char *fx = getenv("DISPLAYXR_CURSOR_DEPTH_UV")) { // scripted cursor for headless verification + float fu, fv; + if (sscanf(fx, "%f,%f", &fu, &fv) == 2) cursor.setPointerUV(fu, fv); + } // HUD window-space layer swapchain (XR_EXT_window_space_layer). // GL on macOS routes through the Metal native compositor with IOSurface- @@ -2048,9 +2326,50 @@ int main(int argc, char **argv) viewState.next = &viewRigRaw; } + // XR_DXR_cursor_depth (ADR-046): pointer UV, the footprint hit test + // against the previous locate's views (a frame late, which the + // runtime's filter absorbs), and the hint / draw-request / state + // structs on both chains. Inert when the extension is off. + cursor.chain(locateInfo, viewState); + uint32_t viewCount = 0; xrLocateViews(app.session, &locateInfo, &viewState, (uint32_t)views.size(), &viewCount, views.data()); + if (cursor.enabled()) { + const bool d3 = (app.currentModeIndex < app.renderingModeCount) + ? app.renderingModeDisplay3D[app.currentModeIndex] : true; + const uint32_t mvc = (app.currentModeIndex < app.renderingModeCount) + ? app.renderingModeViewCounts[app.currentModeIndex] : 2; + // Hides / shows the OS cursor; remembers the views for next frame. + cursor.afterLocate(views.data(), d3 ? (mvc < viewCount ? mvc : viewCount) : 1); + + // App mode: we draw the crosshair from the placement. + const XrCursorDepthPlacementDXR &placement = cursor.placement(); + g_cursorDraw.active = cursor.appShouldDraw(); + g_cursorDraw.pose = placement.pose; + g_cursorDraw.height = placement.height; + + static int willDrawLogged = 0; + if (cursor.runtimeDraws() && willDrawLogged < 3 && cursor.pointerInCanvas()) { + static int lastWillDraw = -1; + if ((int)cursor.drawState().willDraw != lastWillDraw) { + lastWillDraw = (int)cursor.drawState().willDraw; + willDrawLogged++; + LOG_INFO("cursor_depth: runtime willDraw=%s (uv=%.3f,%.3f disparity=%.3f)", + cursor.drawState().willDraw ? "TRUE" : "FALSE", cursor.hint().cursorUV.x, + cursor.hint().cursorUV.y, placement.disparity); + } + } + static int cursorLogged = 0; + if (placement.isActive && cursor.hint().hasContent && cursorLogged < 3) { + cursorLogged++; + LOG_INFO("cursor_depth: uv=(%.3f,%.3f) target=%.3f disparity=%.3f h=%.4f pos=(%.3f,%.3f,%.3f)", + cursor.hint().cursorUV.x, cursor.hint().cursorUV.y, placement.targetDisparity, + placement.disparity, placement.height, placement.pose.position.x, + placement.pose.position.y, placement.pose.position.z); + } + } + // Capture the runtime's resolved CANVAS size (the window client area in // meters) — the physical_height_m the Kooima/rig math runs on, which the // C-toggle converter must match (windowed → smaller than the display). @@ -2561,6 +2880,8 @@ int main(int argc, char **argv) if (hudSwapchain.swapchain != XR_NULL_HANDLE) { xrDestroySwapchain(hudSwapchain.swapchain); } + if (app.cursorSpriteSwapchain != XR_NULL_HANDLE) + xrDestroySwapchain(app.cursorSpriteSwapchain); if (app.swapchain.swapchain) xrDestroySwapchain(app.swapchain.swapchain); if (app.localSpace) diff --git a/test_apps/handle/cube_handle_metal_macos/CMakeLists.txt b/test_apps/handle/cube_handle_metal_macos/CMakeLists.txt index 1c82ae6190..056fb9599a 100644 --- a/test_apps/handle/cube_handle_metal_macos/CMakeLists.txt +++ b/test_apps/handle/cube_handle_metal_macos/CMakeLists.txt @@ -64,7 +64,9 @@ FetchContent_Declare( # R8G8B8A8 (ADR-044 §7, displayxr-common#72) — the Metal compositor is # format-honest, so a UNORM HUD of CPU-rasterized (encoded) bytes would be # encoded twice. Same pin as the VK macOS cube apps. - GIT_TAG v2.27.0 + # v2.29.0: dxr::CursorDepth (common/cursor_depth.h), the XR_DXR_cursor_depth + # app helper (ADR-046 Amendment 2) - same pin as the aggregate test_apps build. + GIT_TAG v2.29.0 GIT_SHALLOW TRUE ) FetchContent_MakeAvailable(displayxr_common) diff --git a/test_apps/handle/cube_handle_vk_macos/CMakeLists.txt b/test_apps/handle/cube_handle_vk_macos/CMakeLists.txt index adc8590d20..b73484186a 100644 --- a/test_apps/handle/cube_handle_vk_macos/CMakeLists.txt +++ b/test_apps/handle/cube_handle_vk_macos/CMakeLists.txt @@ -70,7 +70,9 @@ FetchContent_Declare( # v2.27.0: window-space / HUD swapchains prefer the _SRGB sibling of # R8G8B8A8 (dxr::ChooseWindowSpaceSwapchainFormat; ADR-044 §7, # displayxr-common#72) — a UNORM HUD of encoded bytes was encoded twice. - GIT_TAG v2.27.0 + # v2.29.0: dxr::CursorDepth (common/cursor_depth.h), the XR_DXR_cursor_depth + # app helper (ADR-046 Amendment 2) - same pin as the aggregate test_apps build. + GIT_TAG v2.29.0 GIT_SHALLOW TRUE ) FetchContent_MakeAvailable(displayxr_common) diff --git a/test_apps/handle/cube_handle_vk_macos/main.mm b/test_apps/handle/cube_handle_vk_macos/main.mm index 09671eead9..414e338080 100644 --- a/test_apps/handle/cube_handle_vk_macos/main.mm +++ b/test_apps/handle/cube_handle_vk_macos/main.mm @@ -31,9 +31,11 @@ #include #include #include +#include "cursor_depth.h" // dxr::CursorDepth — XR_DXR_cursor_depth bookkeeping (ADR-046) #include #include "dxr_view_config.h" // #1486 PRIMARY_MULTIVIEW_DXR opt-in +#include #include #include #include @@ -1732,12 +1734,18 @@ static bool InitializeVkRenderer(VkRenderer& renderer, VkDevice device, VkPhysic // ============================================================================ // XR_DXR_cursor_depth (ADR-046) — same adoption as cube_handle_metal_macos. // -// The app owns the depth: each frame it hit-tests the cube under the cursor -// FOOTPRINT (rays from the outer eyes through a ring of points around the -// hotspot) and hands the nearest hit to xrLocateViews. The runtime returns -// where to draw the cursor sprite; the app draws it last, depth test off, and -// hides the OS cursor while it does. Opt out with DISPLAYXR_CURSOR_DEPTH=0; -// DISPLAYXR_CURSOR_DEPTH_UV=u,v scripts the cursor for headless capture. +// The app owns the depth: it answers "ray -> first cube hit" (RayHitCube) and +// displayxr-common's dxr::CursorDepth does the rest — the pointer as a canvas +// UV, the footprint rays from the outer eyes of the previous locate, the +// nearest hit, the struct chain on xrLocateViews, and hiding the OS cursor +// exactly while the depth cursor replaces it. Spec v5 (ADR-046 Amendment 2 B), the +// default: the runtime DRAWS it (XrCursorDepthDrawInfoDXR) - its default look, +// or our own sprite with DISPLAYXR_CURSOR_DEPTH_SPRITE=custom - and the app +// hides the OS cursor exactly while XrCursorDepthDrawStateDXR::willDraw. +// DISPLAYXR_CURSOR_DEPTH_DRAW=app: the app draws its crosshair last, depth +// test off, and hides the OS cursor while it does. Opt out with +// DISPLAYXR_CURSOR_DEPTH=0; DISPLAYXR_CURSOR_DEPTH_UV=u,v scripts the cursor +// for headless capture. // ============================================================================ static const float kCubeSize = 0.06f; @@ -1775,53 +1783,6 @@ static bool RayHitCube(const XrVector3f &o, const XrVector3f &d, float rot, floa return true; } -// Ray from a located view through canvas point (u, v): Kooima frusta all -// frame the canvas, so (u, v) maps linearly onto the fov tangents. -static XrVector3f ViewRayDir(const XrView &view, float u, float v) -{ - const float tl = tanf(view.fov.angleLeft), tr = tanf(view.fov.angleRight); - const float tu = tanf(view.fov.angleUp), td = tanf(view.fov.angleDown); - float dx, dy, dz; - quat_rotate_vec3(view.pose.orientation, tl + u * (tr - tl), tu - v * (tu - td), -1.0f, &dx, &dy, &dz); - return {dx, dy, dz}; -} - -// Nearest cube point (to the viewer) under the cursor footprint, using the -// outer views of the previous locate. Footprint = the sprite plus 50%. -static bool CursorFootprintHit(const std::vector &views, uint32_t active, float u, float v, - float canvasAspect, float rot, XrVector3f &nearest) -{ - if (active < 2 || views.size() < active) return false; - const XrView *outer[2] = {&views[0], &views[active - 1]}; - float fx, fy, fz; - quat_rotate_vec3(views[0].pose.orientation, 0.0f, 0.0f, -1.0f, &fx, &fy, &fz); - const float r = 0.75f * kCursorHeight; // footprint radius, in canvas heights - bool found = false; - float best = 1e30f; - for (int k = -1; k < 8; k++) { - float su = u, sv = v; - if (k >= 0) { - const float a = (float)k * 0.78539816f; - su += r * cosf(a) / canvasAspect; - sv += r * sinf(a); - } - for (const XrView *view : outer) { - const XrVector3f d = ViewRayDir(*view, su, sv); - float t; - if (!RayHitCube(view->pose.position, d, rot, t)) continue; - const XrVector3f p = {view->pose.position.x + d.x * t, view->pose.position.y + d.y * t, - view->pose.position.z + d.z * t}; - const float depth = p.x * fx + p.y * fy + p.z * fz; // smaller = nearer the viewer - if (depth < best) { - best = depth; - nearest = p; - found = true; - } - } - } - return found; -} - // Model matrix of the cursor sprite: pose (rotation then translation), scaled // by the returned height. Column-major, like the other mat4_* helpers. static void CursorSpriteModel(float *out) @@ -2536,6 +2497,17 @@ static void UpdateCameraMovement(InputState& state, float deltaTime, float displ bool hasAtlasCaptureExt = false; bool hasViewRigExt = false; // XR_DXR_view_rig (#396 W7) bool hasCursorDepthExt = false; // XR_DXR_cursor_depth (ADR-046) + uint32_t cursorDepthSpecVersion = 0; + // XR_DXR_cursor_depth v5 (ADR-046 Amendment 2 B): + // DISPLAYXR_CURSOR_DEPTH_DRAW=runtime (default) | app. Runtime: chain + // XrCursorDepthDrawInfoDXR and let the runtime draw the cursor; hide the + // OS cursor exactly while XrCursorDepthDrawStateDXR::willDraw. App: draw + // the crosshair ourselves from the placement (v1-v4 behaviour). + bool cursorDepthRuntimeDraw = false; + // DISPLAYXR_CURSOR_DEPTH_SPRITE=custom: hand the runtime our own sprite + // (a cyan ring with a dot) instead of its default look. + bool cursorDepthCustomSprite = false; + XrSwapchain cursorSpriteSwapchain = XR_NULL_HANDLE; PFN_xrCaptureAtlasDXR pfnCaptureAtlasEXT = nullptr; // XR_DXR_spatial_workspace (#48): when running OOP via the service, the @@ -2599,6 +2571,7 @@ static bool InitializeOpenXR(AppXrSession& xr) { } if (strcmp(ext.extensionName, XR_DXR_CURSOR_DEPTH_EXTENSION_NAME) == 0) { xr.hasCursorDepthExt = true; + xr.cursorDepthSpecVersion = ext.extensionVersion; } if (strcmp(ext.extensionName, XR_DXR_SPATIAL_WORKSPACE_EXTENSION_NAME) == 0) { xr.hasSpatialWorkspaceExt = true; @@ -2639,6 +2612,23 @@ static bool InitializeOpenXR(AppXrSession& xr) { } } LOG_INFO("XR_DXR_cursor_depth: %s", xr.hasCursorDepthExt ? "ENABLED" : "off"); + if (xr.hasCursorDepthExt) { + // Spec v5: who draws the cursor. Default: the runtime. + const char *drawWho = getenv("DISPLAYXR_CURSOR_DEPTH_DRAW"); + const bool wantApp = drawWho != nullptr && strcmp(drawWho, "app") == 0; + if (!wantApp && xr.cursorDepthSpecVersion >= 5) { + xr.cursorDepthRuntimeDraw = true; + const char *sprite = getenv("DISPLAYXR_CURSOR_DEPTH_SPRITE"); + xr.cursorDepthCustomSprite = sprite != nullptr && strcmp(sprite, "custom") == 0; + } else if (!wantApp) { + LOG_WARN("XR_DXR_cursor_depth v%u has no runtime-drawn cursor (needs v5) - drawing it ourselves", + xr.cursorDepthSpecVersion); + } + LOG_INFO("XR_DXR_cursor_depth draw: %s", + !xr.cursorDepthRuntimeDraw ? "APP (crosshair from the placement)" + : xr.cursorDepthCustomSprite ? "RUNTIME, custom sprite (cyan ring)" + : "RUNTIME, default look"); + } if (xr.hasSpatialWorkspaceExt) { enabledExtensions.push_back(XR_DXR_SPATIAL_WORKSPACE_EXTENSION_NAME); } @@ -3369,6 +3359,97 @@ static bool UploadHudPixels(HudVkUploader& up, VkImage hudImage, // Rows are authored TOP-DOWN (row 0 = the image's top) and uploaded with // vkCmdCopyBufferToImage, so uv.y = 0 is the top — the same mapping the GL // probe gets from glTexSubImage2D and the Metal one from replaceRegion. +// ============================================================================ +// XR_DXR_cursor_depth v5 — a custom cursor sprite for the runtime to draw +// ============================================================================ + +// DISPLAYXR_CURSOR_DEPTH_SPRITE=custom: a 64x64 cyan ring with a centre dot +// (hotspot = the centre), dark-edged so it reads over the white cube. Filled +// once (acquire / fill / release) through the HUD uploader; the runtime +// samples the last released image every frame, as STRAIGHT alpha. `_SRGB` +// when offered: the bytes are display-referred. Row 0 = the top (Vulkan). +static bool CreateCursorSprite(AppXrSession& xr, VkDevice device, VkPhysicalDevice phys, VkQueue queue, + VkCommandPool pool) +{ + const uint32_t S = 64; + uint32_t formatCount = 0; + xrEnumerateSwapchainFormats(xr.session, 0, &formatCount, nullptr); + std::vector formats(formatCount); + xrEnumerateSwapchainFormats(xr.session, formatCount, &formatCount, formats.data()); + int64_t fmt = 0; + const int64_t preferred[] = {(int64_t)VK_FORMAT_R8G8B8A8_SRGB, (int64_t)VK_FORMAT_B8G8R8A8_SRGB, + (int64_t)VK_FORMAT_R8G8B8A8_UNORM, (int64_t)VK_FORMAT_B8G8R8A8_UNORM}; + for (int64_t pref : preferred) { + for (int64_t f : formats) { + if (f == pref && fmt == 0) fmt = pref; + } + } + if (fmt == 0) return false; + const bool bgra = fmt == (int64_t)VK_FORMAT_B8G8R8A8_SRGB || fmt == (int64_t)VK_FORMAT_B8G8R8A8_UNORM; + + XrSwapchainCreateInfo sci = {XR_TYPE_SWAPCHAIN_CREATE_INFO}; + sci.usageFlags = XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT | XR_SWAPCHAIN_USAGE_SAMPLED_BIT | + XR_SWAPCHAIN_USAGE_TRANSFER_DST_BIT; + sci.format = fmt; + sci.sampleCount = 1; + sci.width = S; + sci.height = S; + sci.faceCount = 1; + sci.arraySize = 1; + sci.mipCount = 1; + if (XR_FAILED(xrCreateSwapchain(xr.session, &sci, &xr.cursorSpriteSwapchain))) { + return false; + } + uint32_t n = 0; + xrEnumerateSwapchainImages(xr.cursorSpriteSwapchain, 0, &n, nullptr); + std::vector imgs(n, {XR_TYPE_SWAPCHAIN_IMAGE_VULKAN_KHR}); + if (n == 0 || XR_FAILED(xrEnumerateSwapchainImages(xr.cursorSpriteSwapchain, n, &n, + (XrSwapchainImageBaseHeader*)imgs.data()))) { + return false; + } + XrSwapchainImageAcquireInfo ai = {XR_TYPE_SWAPCHAIN_IMAGE_ACQUIRE_INFO}; + uint32_t idx = 0; + if (XR_FAILED(xrAcquireSwapchainImage(xr.cursorSpriteSwapchain, &ai, &idx))) { + return false; + } + XrSwapchainImageWaitInfo wi = {XR_TYPE_SWAPCHAIN_IMAGE_WAIT_INFO}; + wi.timeout = XR_INFINITE_DURATION; + xrWaitSwapchainImage(xr.cursorSpriteSwapchain, &wi); + + std::vector buf((size_t)S * S * 4); + auto cover = [](float d, float r0, float r1) { // 1 inside [r0, r1], 1-px soft edges + const float a = std::min(d - r0 + 0.5f, r1 - d + 0.5f); + return a <= 0.0f ? 0.0f : (a >= 1.0f ? 1.0f : a); + }; + for (uint32_t y = 0; y < S; y++) { + for (uint32_t x = 0; x < S; x++) { + const float d = hypotf((float)x + 0.5f - S * 0.5f, (float)y + 0.5f - S * 0.5f); + const float ring = std::max(cover(d, 21.0f, 27.0f), cover(d, 0.0f, 5.0f)); // cyan + const float edge = std::max(cover(d, 19.5f, 28.5f), cover(d, 0.0f, 6.5f)); // dark rim + uint8_t *px = &buf[((size_t)y * S + x) * 4]; + // Straight alpha: cyan where the ring is, near-black rim. + const uint8_t r = 0, g = (uint8_t)(220.0f * ring), b = (uint8_t)(255.0f * ring); + px[0] = bgra ? b : r; + px[1] = g; + px[2] = bgra ? r : b; + px[3] = (uint8_t)(255.0f * std::max(ring, edge)); + } + } + + HudVkUploader up = {}; + bool ok = InitializeHudVkUploader(up, device, phys, queue, pool, S, S) && + UploadHudPixels(up, imgs[idx].image, buf.data(), S * 4); + CleanupHudVkUploader(up); + + XrSwapchainImageReleaseInfo ri = {XR_TYPE_SWAPCHAIN_IMAGE_RELEASE_INFO}; + xrReleaseSwapchainImage(xr.cursorSpriteSwapchain, &ri); + if (ok) { + LOG_INFO("Cursor sprite swapchain created: %ux%u, format %lld (cyan ring, hotspot = centre)", S, S, + (long long)fmt); + } + return ok; +} + static bool CreateAndFillQuadTexture(AppXrSession& xr, uint32_t size, VkDevice device, VkPhysicalDevice phys, VkQueue queue, VkCommandPool pool) @@ -3710,6 +3791,9 @@ static bool EndFrameWithQuads(AppXrSession& xr, XrTime displayTime, } static void CleanupOpenXR(AppXrSession& xr) { + if (xr.cursorSpriteSwapchain != XR_NULL_HANDLE) { + xrDestroySwapchain(xr.cursorSpriteSwapchain); + } if (xr.swapchain.swapchain != XR_NULL_HANDLE) { xrDestroySwapchain(xr.swapchain.swapchain); } @@ -3933,6 +4017,33 @@ int main() { vkDestroyInstance(vkInstance, nullptr); return 1; } + if (xr.cursorDepthRuntimeDraw && xr.cursorDepthCustomSprite && + !CreateCursorSprite(xr, vkDevice, physDevice, graphicsQueue, vkRenderer.commandPool)) { + LOG_WARN("Cursor sprite swapchain creation failed - using the runtime's default look"); + xr.cursorDepthCustomSprite = false; + } + + // XR_DXR_cursor_depth (ADR-046): the displayxr-common helper. enable(0) + // (extension off / opted out) leaves it inert: no structs, no OS cursor. + dxr::CursorDepth cursor; + cursor.enable(xr.hasCursorDepthExt ? xr.cursorDepthSpecVersion : 0); + cursor.setWindowCocoa((__bridge void *)g_metalView); + cursor.setCursorHeight(kCursorHeight); + cursor.setWantPlacement(true); // the diagnostics below + if (!xr.cursorDepthRuntimeDraw) { + cursor.setDrawMode(dxr::CursorDepth::DrawMode::App); + cursor.setFootprintBodyOffset(dxr::kCursorCenteredBodyOffset); // our crosshair is centred + } else if (xr.cursorDepthCustomSprite) { + cursor.setSprite(xr.cursorSpriteSwapchain, {0.5f, 0.5f}, 0.01f); // ring: hotspot = centre, 1 cm + } + cursor.setRayCast([&vkRenderer](const dxr::CursorRay &r) -> std::optional { + float t; + return RayHitCube(r.origin, r.dir, vkRenderer.cubeRotation, t) ? std::optional(t) : std::nullopt; + }); + if (const char *fx = getenv("DISPLAYXR_CURSOR_DEPTH_UV")) { // scripted cursor for headless verification + float fu, fv; + if (sscanf(fx, "%f,%f", &fu, &fv) == 2) cursor.setPointerUV(fu, fv); + } { uint32_t count = xr.swapchain.imageCount; @@ -4345,53 +4456,12 @@ int main() { viewActivity.next = &viewBindings; viewState.next = &viewActivity; - // XR_DXR_cursor_depth (ADR-046): hint from the previous - // locate's views (the hit test needs rays, the rays need - // views - one frame late, which the runtime's filter absorbs). - static std::vector prevViews; - static uint32_t prevActive = 0; - XrCursorDepthHintDXR cursorHint = {XR_TYPE_CURSOR_DEPTH_HINT_DXR}; - XrCursorDepthPlacementDXR cursorPlacement = {XR_TYPE_CURSOR_DEPTH_PLACEMENT_DXR}; - if (xr.hasCursorDepthExt && g_window != nil && g_metalView != nil) { - NSPoint pw = [g_window mouseLocationOutsideOfEventStream]; - NSPoint pv = [g_metalView convertPoint:pw fromView:nil]; - NSRect b = [g_metalView bounds]; - if (b.size.width > 0 && b.size.height > 0) { - float u = (float)(pv.x / b.size.width); - float v = [g_metalView isFlipped] ? (float)(pv.y / b.size.height) - : (float)(1.0 - pv.y / b.size.height); - // Scripted cursor for headless verification: DISPLAYXR_CURSOR_DEPTH_UV=u,v - static float fixedU = -1.0f, fixedV = -1.0f; - static bool fixedParsed = false; - if (!fixedParsed) { - fixedParsed = true; - const char *fxy = getenv("DISPLAYXR_CURSOR_DEPTH_UV"); - if (fxy == nullptr || sscanf(fxy, "%f,%f", &fixedU, &fixedV) != 2) { - fixedU = fixedV = -1.0f; - } - } - if (fixedU >= 0.0f) { - u = fixedU; - v = fixedV; - } - if (u >= 0.0f && u <= 1.0f && v >= 0.0f && v <= 1.0f) { - cursorHint.cursorUV = {u, v}; - cursorHint.cursorHeight = kCursorHeight; - XrVector3f hit = {0.0f, 0.0f, 0.0f}; - cursorHint.hasContent = - CursorFootprintHit(prevViews, prevActive, u, v, - (float)(b.size.width / b.size.height), - vkRenderer.cubeRotation, hit) - ? XR_TRUE - : XR_FALSE; - cursorHint.nearestPoint = hit; - cursorHint.next = locateInfo.next; - locateInfo.next = &cursorHint; - } - } - cursorPlacement.next = viewState.next; - viewState.next = &cursorPlacement; - } + // XR_DXR_cursor_depth (ADR-046): pointer UV, the footprint + // hit test against the previous locate's views (a frame + // late, which the runtime's filter absorbs), and the hint / + // draw-request / state structs on both chains. Inert when + // the extension is off. + cursor.chain(locateInfo, viewState); uint32_t viewCount = (uint32_t)xr.configViews.size(); std::vector views(viewCount, {XR_TYPE_VIEW}); @@ -4399,30 +4469,36 @@ int main() { XrResult locResult = xrLocateViews(xr.session, &locateInfo, &viewState, viewCount, &viewCount, views.data()); - if (xr.hasCursorDepthExt && XR_SUCCEEDED(locResult)) { - prevViews.assign(views.begin(), views.begin() + viewCount); + if (cursor.enabled() && XR_SUCCEEDED(locResult)) { const bool d3 = (xr.currentModeIndex < xr.renderingModeCount) ? xr.renderingModeDisplay3D[xr.currentModeIndex] : true; - prevActive = d3 ? (modeViewCount < viewCount ? modeViewCount : viewCount) : 1; - - g_cursorDraw.active = cursorPlacement.isActive == XR_TRUE; - g_cursorDraw.pose = cursorPlacement.pose; - g_cursorDraw.height = cursorPlacement.height; - // The app owns its window's OS cursor: hidden exactly - // while the depth cursor replaces it. - static bool osCursorHidden = false; - if (g_cursorDraw.active != osCursorHidden) { - if (g_cursorDraw.active) [NSCursor hide]; else [NSCursor unhide]; - osCursorHidden = g_cursorDraw.active; + // Hides / shows the OS cursor; remembers the views for next frame. + cursor.afterLocate(views.data(), d3 ? (modeViewCount < viewCount ? modeViewCount : viewCount) : 1); + + // App mode: we draw the crosshair from the placement. + const XrCursorDepthPlacementDXR &placement = cursor.placement(); + g_cursorDraw.active = cursor.appShouldDraw(); + g_cursorDraw.pose = placement.pose; + g_cursorDraw.height = placement.height; + + static int willDrawLogged = 0; + if (cursor.runtimeDraws() && willDrawLogged < 3 && cursor.pointerInCanvas()) { + static int lastWillDraw = -1; + if ((int)cursor.drawState().willDraw != lastWillDraw) { + lastWillDraw = (int)cursor.drawState().willDraw; + willDrawLogged++; + LOG_INFO("cursor_depth: runtime willDraw=%s (uv=%.3f,%.3f disparity=%.3f)", + cursor.drawState().willDraw ? "TRUE" : "FALSE", cursor.hint().cursorUV.x, + cursor.hint().cursorUV.y, placement.disparity); + } } static int cursorLogged = 0; - if (g_cursorDraw.active && cursorHint.hasContent && cursorLogged < 3) { + if (placement.isActive && cursor.hint().hasContent && cursorLogged < 3) { cursorLogged++; LOG_INFO("cursor_depth: uv=(%.3f,%.3f) target=%.3f disparity=%.3f h=%.4f pos=(%.3f,%.3f,%.3f)", - cursorHint.cursorUV.x, cursorHint.cursorUV.y, cursorPlacement.targetDisparity, - cursorPlacement.disparity, cursorPlacement.height, - cursorPlacement.pose.position.x, cursorPlacement.pose.position.y, - cursorPlacement.pose.position.z); + cursor.hint().cursorUV.x, cursor.hint().cursorUV.y, placement.targetDisparity, + placement.disparity, placement.height, placement.pose.position.x, + placement.pose.position.y, placement.pose.position.z); } }