Skip to content

Metal compositor: custom cursor sprite cast to the wrong swapchain type on the VK-via-Metal fallback (#1941) #1948

Closed

Description

@dfattal

Summary

On the Vulkan-app-on-Metal-compositor fallback (oxr_session_populate_vk_with_metal_native, used on macOS when vk_native is unavailable), a custom runtime-drawn cursor sprite (XrCursorDepthDrawInfoDXR::image != XR_NULL_HANDLE, XR_DXR_cursor_depth v5, #1941) is handed to the Metal compositor as the client Vulkan swapchain, and the Metal compositor casts it to its native swapchain type. This was found by reading the code while reviewing #1946 and has not been reproduced. The default look (image = XR_NULL_HANDLE) is unaffected.

Evidence

  • src/xrt/state_trackers/oxr/oxr_session_gfx_vk_native.c:166: on this route sess->compositor = &xcvk->base (a comp_vk_client wrapper), while sess->xcn is the comp_metal compositor (:187 is_vk_via_metal = true, and is_metal_native_compositor = true). The app's swapchains are therefore struct client_vk_swapchain (src/xrt/compositor/client/comp_vk_client.h:46), which wrap the native one in ->xscn.
  • src/xrt/state_trackers/oxr/oxr_session_frame_end.c:2052 (cursor_draw_submit): req.image = sc->swapchain; hands over that client swapchain unchanged.
  • src/xrt/compositor/metal/comp_metal_compositor.m:4160 (metal_cursor_draw): struct comp_metal_swapchain *msc = metal_swapchain(req->image);. metal_swapchain() (:579) is a plain cast, so msc->image_count / msc->images[] are read from a client_vk_swapchain.

The expected result is a garbage image count or texture: the cursor is not drawn, or an invalid MTLTexture is sampled and the app may crash.

Likely fix

In cursor_draw_submit (or in the Metal hand-off), resolve the native swapchain for wrapped sessions: on is_vk_via_metal take ((struct client_vk_swapchain *)sc->swapchain)->xscn. The macOS GL-on-Metal interop route may need the same check for its client swapchains. Alternatively, report willDraw = XR_FALSE for a custom sprite on wrapped sessions until the hand-off resolves it.

Not done

There is no repro yet: on macOS the reference Vulkan cube takes vk_native, not this fallback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions