Skip to content

feat(display_info): per-screen enumeration, display spaces, session binding (multi-screen M1) - #1852

Merged
dfattal merged 10 commits into
mainfrom
feat/multi-screen-m1-display-api
Oct 7, 2026
Merged

dfattal merged 10 commits into
mainfrom
feat/multi-screen-m1-display-api

Conversation

@dfattal

@dfattal dfattal commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Part of #69 (M1). Refs #1849 (plan + ADR-047), #1851 (M0, this PR's base). M0 (#1851) is merged; this PR is rebased onto main.

What

Multi-screen milestone M1: an app can now see each display, not only the one XrDisplayInfoDXR describes.

  • XR_DXR_display_info v22 (append-only; no existing struct, value or behaviour changes)
    • xrEnumerateDisplaysDXR (two-call) → XrDisplayDXR { displayId, XrDisplayInfoDXR info (by value), desktopRect, nativePixelWidth/Height, desktopScale, XrEyeTrackingModeCapabilitiesDXR eyeTracking, flags (PRIMARY / TRACKED / SYSTEM_DEFAULT), vendorPluginId[64], deviceName[64] }. The SYSTEM_DEFAULT entry comes first and reports exactly what XrSystemProperties does (active-mode view scale included).
    • XrDisplaySpaceCreateInfoDXR + xrCreateDisplaySpaceDXR: a DISPLAY space per display. v22 limit: every display space resolves to the session's single display plane (head pose for runtime-window sessions, head tracking origin for app-window / bridge sessions, the pose XrViewDisplayRawDXR::displayPlanePose reports). Per-display poses come in M3; no shared room frame.
    • XrSessionDisplayBindingDXR on XrSessionCreateInfo. v22 effect: binding to a non-default display makes that display's physical size + nominal viewer the session's display-scoped Kooima inputs. Which DP weaves is unchanged (M2/M3). 0 = runtime decides. Unknown id → XR_ERROR_VALIDATION_FAILURE.
    • Type values 1004999214–216; 217 reserved for XrViewDisplayBindingsDXR (M3). Registry README updated.
  • Runtime backing. xrt_screen / xrt_screen_info / xrt_screen_list (xrt/xrt_screen.h); target_screens_build turns each per-monitor registry entry (M0) into a screen. Each screen's info comes from, in order: (1) the system info, if it is the system-default screen; (2) the owning plug-in's new get_display_info_for_monitor slot; (3) EDID-derived defaults: mm → metres, connector mode → pixels, view scale 1, no eye tracking, nominal viewer at the distance that keeps the system panel's vertical FOV. The pure rules live in util/u_screen_info.h.
  • Plug-in ABI (no version bump). xrt_plugin_iface::get_display_info_for_monitor(inst, const xrt_display_descriptor*, const xrt_display_physical*, xrt_plugin_display_info*) is appended after stereo_camera_close (ADR-020 append-only, struct_size-gated). XRT_PLUGIN_API_VERSION_CURRENT stays 5. New feature macro: XRT_PLUGIN_IFACE_HAS_DISPLAY_INFO_FOR_MONITOR. sim_display implements it. The Leia plug-in is untouched; its other monitors get the EDID-derived defaults.
    • EDID mm and the device mode go in a new struct passed by pointer, xrt_display_physical, and not in xrt_display_descriptor. probe_displays receives descriptors as an array, so appending a field would change the stride under every shipped plug-in: a silent layout break, even though the old comment said "append with no bump". The descriptor doc now warns about this.
  • IPC. A new system_enumerate_displays message carries struct xrt_screen_list and is answered through the new optional xrt_instance::enumerate_displays slot. The native instance and the IPC client instance both implement it. The by-value xrt_system_compositor_info is unchanged. No new message is needed for display spaces: the existing space-overseer IPC covers them.
  • Visibility. displayxr-cli info has a :: Displays block (and a displays JSON array) built by the same target_screens_build. cube_handle_vk_linux logs one line per display at startup.
  • Docs.
    • XR_DXR_display_info.md: v22 section, revision row, OPEN 1 rewritten as partially resolved.
    • view-configuration-model.md: what a session binding does and does not change.
    • xrt_plugin_iface.md: the new slot.

displayxr-cli info on ds1-linux (eDP-1 + Acer DS1 on HDMI-1, installed leia-sr active)

 :: Displays (XR_DXR_display_info v22 — what xrEnumerateDisplaysDXR reports)
    [0] 0xcb5af163f67dd93c 'HDMI-1' plug-in='leia-sr' SYSTEM_DEFAULT TRACKED
        desktop 3840x2160 @ (3456,0)  native 3840x2160  scale 2.00  edid 344x193 mm
        info[system]: 0.3440 x 0.1930 m  3840x2160 px  nominal=(0.000, 0.000, 0.600) m  view scale 0.500 x 0.500  eye tracking MANAGED (default MANAGED)
    [1] 0x50cab5190604ffb9 'eDP-1' plug-in='sim-display' PRIMARY
        desktop 3456x2160 @ (0,0)  native 2880x1800  scale 1.67  edid 300x190 mm
        info[derived]: 0.3000 x 0.1900 m  2880x1800 px  nominal=(0.000, 0.000, 0.590) m  view scale 1.000 x 1.000  eye tracking none

With DXR_PLUGIN_EXCLUSIVE=sim-display and the dev sim-display, HDMI-1 reports info[plugin-monitor] (the new sim slot: EDID 344x193 mm, 3840x2160).

Headless end-to-end, in-process: a probe that only calls xrCreateInstance and xrGetSystem (no session, no window), then xrEnumerateDisplaysDXR. It returned the same two displays:

  • the HDMI-1 flags are SYSTEM_DEFAULT|TRACKED, and its info matches XrSystemProperties;
  • capacity=1 returns XR_ERROR_SIZE_INSUFFICIENT;
  • xrCreateDisplaySpaceDXR resolves through xrGetInstanceProcAddr.

Tests

  • New tests_target_screens (6 cases: default pick, fallback, slot vs derived, empty registry, id stability).
  • New tests_aux_screen_info: 8 cases covering the three source rules, a slot that declines, a plug-in whose struct_size predates the slot (never called), unknown mm, no system info, and xrt_screen_list_find.
  • tests_stereo_camera: the slot-order pin now expects the new slot to end the iface.
  • ./scripts/build_linux.sh (headless) + selftest: PASSED. --hybrid --apps + selftest: PASSED.
  • ctest: everything passes except tests_rig_composer and tests_oxr_view_space* (4 variants). Those already fail on this box on main.
  • scripts/check_displayxr_app.py test_apps/cube_handle_vk_linux is clean. gen_extensions_index.py --check, check_doc_paths.py and gen_adr_index.py --check pass.

Review fixes (follow-up commits)

  • 1d42570f3: the registry is published under g_display_info_mutex. The refresh callback now holds that mutex across the display-info refresh, the registry rebuild and the desktop-rect fill, and the registry is resolved into a local first. Lock order stays g_display_info_mutex → g_refresh_mutex.
  • defcf2ff5: the default screen is the monitor whose rect contains the resolved panel origin, so a screen can no longer carry one monitor's id with another monitor's rect.
  • 2e0631580: new tests_target_screens covers target_screens_build and the default-screen pick.
  • f1c474f03: where the connector is known, displayId hashes EDID identity + connector and no longer the desktop position, so it survives a monitor rearrangement. Windows ids are unchanged (still position-based) and the docs say so. Also adds the spec's missing v21 row.

The ids in the CLI output above predate the rebase onto main. M0's review fixes made the connector key card-prefixed (card1-HDMI-A-1), and the id hash now includes that prefix, so the ids changed again. eDP-1 is now 0x9e6f2f2a48d37339. HDMI-1's new id could not be read: the DS1 was disconnected at the time of the rebase.

Not verified

  • IPC at runtime. The message, handler and client compile in the hybrid build. I did not start a dev service on the shared DS1 box.
  • The DISPLAY space pose and the session binding's Kooima effect. Both need a session, which means a windowed app, and that is not allowed on this box.
  • Windows. A MinGW syntax check of target_screens.c is clean when given a Windows.h shim. This box's mingw toolchain fails on pre-existing headers (Windows.h case, timespec_get), so it does not cover the rest; Windows CI does.

🤖 Generated with Claude Code

@dfattal
dfattal requested a review from a team as a code owner October 7, 2026 18:02
@dfattal
dfattal force-pushed the feat/multi-screen-m0-screen-registry branch from 010da94 to be5490f Compare October 7, 2026 20:59
Base automatically changed from feat/multi-screen-m0-screen-registry to main October 7, 2026 21:38
dfattal and others added 10 commits October 7, 2026 14:38
…ion (multi-screen M1)

Appends xrt_plugin_iface::get_display_info_for_monitor (ADR-020: append-only,
struct_size-gated, no XRT_PLUGIN_API_VERSION_CURRENT bump), announced by
XRT_PLUGIN_IFACE_HAS_DISPLAY_INFO_FOR_MONITOR. EDID mm + the connector's device
mode travel in a new by-pointer struct xrt_display_physical instead of growing
xrt_display_descriptor: probe_displays receives descriptors as an ARRAY, so
growing that struct would change the stride under every existing plug-in.

xrt_screen.h carries the per-screen record (xrt_screen / xrt_screen_info /
xrt_screen_list); util/u_screen_info.h resolves one monitor's info: the
system-default screen reports the system info verbatim, else the owning
plug-in's slot, else EDID-derived defaults at the system panel's vertical FOV.
sim_display implements the slot. Unit-tested in tests_aux_screen_info.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ver IPC (multi-screen M1)

xrt_instance::enumerate_displays (optional slot) returns the system's screens:
target_screens_build turns each per-monitor registry entry into an xrt_screen
(desktop rect, device mode, compositor scale, EDID mm, output name, winning
plug-in, display info), system-default first. The native instance answers from
the system it created; the IPC client asks the service through the new
system_enumerate_displays message, so xrt_system_compositor_info keeps its
by-value layout. The loader exposes its monitor side table
(target_plugin_get_monitor_record) for this.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…sion display binding

XR_DXR_display_info SPEC_VERSION 22 (append-only): XrDisplayDXR +
xrEnumerateDisplaysDXR (two-call), XrDisplaySpaceCreateInfoDXR +
xrCreateDisplaySpaceDXR, XrSessionDisplayBindingDXR on XrSessionCreateInfo.
Type values 1004999214-216; 217 reserved for XrViewDisplayBindingsDXR (M3).

oxr fills XrDisplayDXR from the instance's screen list (the SYSTEM_DEFAULT
entry reports exactly what XrSystemProperties does, active-mode view scale
included). A DISPLAY space resolves to the session's single display plane
(head pose for runtime-window sessions, head tracking origin for
external-window / bridge sessions). A session bound to a non-default display
uses that display's physical size + nominal viewer for its display-scoped
Kooima; the weaving DP is unchanged until segmentation (M2/M3).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
displayxr-cli info gains a ':: Displays' block (and a 'displays' JSON array)
built by the same target_screens_build the runtime uses, so M1 is verifiable
headless. cube_handle_vk_linux logs xrEnumerateDisplaysDXR at startup, one
line per display.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ion binding; plug-in slot reference

XR_DXR_display_info spec: v22 section, revision row, OPEN 1 rewritten as
partially resolved (what M2/M3/M5 still owe). view-configuration-model: what a
session display binding does and does not change. xrt_plugin_iface reference:
get_display_info_for_monitor, its source order and why xrt_display_physical is
a separate by-pointer struct.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…itor

The ADR-020 slot-order pin asserted the camera block ends xrt_plugin_iface;
multi-screen M1 appends one slot after it. Pin that slot's position too.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…lay-info mutex

t_instance_enumerate_displays read the registry under g_display_info_mutex, but
refresh_display_processors_cb rebuilt it (build_dp_registry) and the desktop
rect (fill_display_desktop_info) after refresh_display_info_from_plugin had
released that mutex, and target_plugin_resolve_displays zeroes its output
before taking the loader's lock. Under the service, a session created with an
XrSessionDisplayBindingDXR while another client connects could see an empty or
half-built registry and reject a valid displayId.

The registry is now resolved into a local and published in one copy; the
callback holds g_display_info_mutex across the display-info refresh, the
registry rebuild and the desktop-rect fill; the system is published to the
enumerate path only once all three exist. Lock order stays
g_display_info_mutex -> g_refresh_mutex.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…lver picked

target_screens_build took xrt_dp_registry_primary_entry() as the system-default
screen — entries[0] of the active plug-in's monitors — and then overwrote its
rect, device name and primary flag with the #1301-resolved panel rect, even
when the resolver had placed the panel on a different monitor: one monitor's
id with another's rect.

target_screens_pick_default now picks the entry whose rect contains the
resolved origin (preferring one the active plug-in won), falls back to
primary_entry only when none does, and the system's placement is copied onto
the default screen only in the first case.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…en pick

tests_target_screens drives the real builder against a hand-made registry, with
the loader's monitor side table filled through target_plugin_build_descriptors
from a fake EDID list: the active plug-in owning two monitors with the panel
resolved onto the second (the id/rect mix-up the previous commit fixes), the
primary_entry fallback that must keep its own rect, the active-plug-in
preference, slot vs EDID-derived info for non-default screens, and the
empty-registry synthesis.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…nector is known

The monitor id (XR_DXR_display_info v22 displayId) hashed the desktop
position, and the service rebuilds its registry on every client connect, so an
id an app held changed when the user rearranged monitors, while the header
and spec called it stable and EDID/connector-derived.

Where the platform names the connector (desktop Linux) the id is now EDID
manufacturer + product + serial + connector, no position. The connector keeps
its DRM card prefix ("card1-HDMI-A-1", as M0 records it), so two GPUs exposing
the same connector name do not collide; a kernel card renumbering therefore
changes the id, which the docs state. The key is unique per machine:
stable across rearrangements and processes, new only when another panel is
plugged into the port. Without a connector name (Windows) the old
mfr + product + position hash is kept, so Windows ids are unchanged; the
header, spec and plugin-discovery doc now say exactly that and tell apps to
re-enumerate after a topology change. Pinned in tests_target_screens.

Also adds the spec's missing v21 revision row (XrViewActivityStateDXR,
ADR-041).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@dfattal
dfattal force-pushed the feat/multi-screen-m1-display-api branch from f1c474f to 9b454aa Compare October 7, 2026 21:40
@dfattal
dfattal merged commit 1e01196 into main Oct 7, 2026
40 checks passed
@dfattal
dfattal deleted the feat/multi-screen-m1-display-api branch October 7, 2026 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant