Skip to content

Multi-display compositing on single machine #69

Open
Open
Multi-display compositing on single machine#69
Milestone
M6: Spatial Shell

Description

@dfattal
dfattal
opened on Mar 9, 2026

Spec: docs/architecture/multi-display-single-machine.md — this issue tracks implementation.

Original design document (archived)

Summary

Extend the multi-compositor to route layers to multiple physical displays connected to the same machine. Each display has a known pose in a shared room-scale coordinate system.

Architecture

                                    ┌→ native compositor → Display A (pose A)
Apps → IPC → multi-compositor ──────┤→ native compositor → Display B (pose B)
                                    └→ native compositor → Display C (pose C)

One displayxr-service process discovers and manages all local displays. The multi-compositor treats each as a separate render target with its own pose.

Tasks

  • Enumerate multiple physical displays from a single service instance (multiple Leia/sim_display devices)
  • Define per-display pose configuration (JSON config or calibration)
  • Route compositor layers to the correct display(s) based on window pose vs display frustum
  • Handle windows spanning multiple displays (split compositing)
  • Frame synchronization across local displays

Depends on

Context

This is step 2a of the multi-display architecture. All displays are local to one machine, one process, shared memory — no network involved.

Activity

  1. dfattal
    added this to the M6: Spatial Shell milestone on Mar 9, 2026
  2. dfattal
    added 14 commits that reference this issue on Apr 2, 2026

    EDID-based display identification for vendor-neutral 3D display detec…

    a4aa840

    EDID probe: upgrade diagnostic logs to WARN level for visibility (#69)

    1361a87

    EDID probe: add detailed diagnostics for SetupAPI/GDI enumeration deb…

    7188bd9

    EDID probe: add entry-point trace logs to diagnose missing output (#69)

    b05ea26

    EDID probe: switch to GUID_DEVCLASS_MONITOR + hardware ID correlation (…

    e5f9d02

    EDID probe: remove verbose trace logs now that probe works (#69)

    be77b9a

    ADR-015/roadmap: HWND ownership model for phase snapping + DWM drag c…

    ee57c56

    EDID-based display identification for vendor-neutral 3D display detec…

    a4be4cb

    EDID probe: upgrade diagnostic logs to WARN level for visibility (#69)

    4be1a37

    EDID probe: add detailed diagnostics for SetupAPI/GDI enumeration deb…

    2a7d3ea

    EDID probe: add entry-point trace logs to diagnose missing output (#69)

    d357da9

    EDID probe: switch to GUID_DEVCLASS_MONITOR + hardware ID correlation (…

    c00c196

    EDID probe: remove verbose trace logs now that probe works (#69)

    942c55b

    ADR-015/roadmap: HWND ownership model for phase snapping + DWM drag c…

    0855bdb

  3. 7 remaining items

  4. dfattal commented on May 31, 2026

    @dfattal
    dfattal
    on May 31, 2026
    CollaboratorAuthor

    Refreshing this in light of two things now in tree:

    1. The vendor-neutral enumerator exists and is wired up read-only. src/xrt/auxiliary/os/os_display_edid.* (os_display_edid_enumerate() → mfr/product/dims/pos/refresh/primary) is now surfaced via displayxr-cli displays + a Control Panel "Displays" tab (displayxr-cli + Control Panel: vendor-neutral connected-display enumeration (EDID) #380). It's the substrate for the descriptor array below.
    2. A vendor validator already exists. The Leia plug-in's leia_edid_probe_display() (EDID table + SR-SDK + service checks) already produces per-monitor identity; today it only feeds the binary probe(). probe_displays() just re-shapes that into per-monitor claims.

    Sketched the probe_displays() ABI addition here: docs/roadmap/probe-displays-iface-sketch.md. Summary:

    • Append an optional, NULL-safe probe_displays(inst, descriptors[], n, out_claims[], max) to xrt_plugin_iface (struct_size-guarded, no ABI-version bump per ADR-020).
    • Runtime enumerates monitors via os_display_edid, assigns stable monitor_ids, asks each plug-in for claims {monitor_id, confidence(FALLBACK/EDID/VERIFIED), supported_apis, serial}.
    • Resolve per monitor (highest confidence; ties by ProbeOrder; per-display user override) → build the ADR-015 DP factory registry (monitor_id → factories), single-entry = today's behavior.
    • Back-compat: probe_displays == NULL → fall back to binary probe() as a single primary-monitor claim. sim_display claims all at FALLBACK.
    • Per-display override generalizes DisplayXR Control Panel: GUI for runtime/plug-in/display info + DP switching (CLI-backed) #378's global PreferredPlugin to PreferredPlugin\<monitor-key>.
    • Leia migration is a thin wrapper over the existing leia_edid_probe_display().

    (#380 is the read-only enumeration surface; this issue stays the owner of the runtime routing.)

  5. dfattal
    added 4 commits that reference this issue on May 31, 2026

    feat(cli,panel): vendor-neutral connected-display enumeration via EDID (

    f679daf

    docs(roadmap): sketch probe_displays() per-display vendor-claim iface (…

    4156792

    feat(runtime): probe_displays() per-display vendor DP claims (Phase 1, …

    8c04ba2

    docs(roadmap): Phase 3 design decisions for multi-display (#69)

    be57f2a

  6. dfattal commented on Jun 1, 2026

    @dfattal
    dfattal
    on Jun 1, 2026
    CollaboratorAuthor

    Phases 1–2 shipped; Phase 3 design captured

    Shipped: xrt_plugin_iface::probe_displays() + xrt_dp_factory_registry + per-monitor resolution (runtime v1.9.0, PR #382) and Leia's probe_displays() (leia v1.2.0, PR #25). displayxr-cli displays --claims shows resolved per-monitor claims.

    Phase 3 (compositors consuming the registry) — design decisions now in docs/roadmap/multi-display-single-machine.md → Phase 3 design decisions:

    • Window-ownership split — hosted: N runtime-owned per-display windows (real HWND each) → no Vendor SR-SDK requirements for multi-display routing (R1-R4) displayxr-leia-plugin#43. handle/texture: the app owns one window, so split-weave inside one compositor with multiple DPs; HWND/phase owner = majority display, secondary DPs window_handle = NULL → #111 required (and only here). Not the IPC/comp_multi path — that's multi-app.
    • Gating spike: can the SR weaver interlace an arbitrary sub-rect with a supplied phase and no window? Blocks the handle/texture variant; confirm with Leia.
    • Atlas tile sizing = window(handle)/canvas(texture) size × scaleXY (not display-native) → the only per-monitor variable is scaleXY → take max over engaged displays (oversampling is free, undersampling is visible; majority is for the HWND owner, not atlas sizing). Engaged-set change rides the existing recommended-dims/mode-change event; static-max-over-all-displays is the no-realloc fallback.
    • Sequencing: 3a = no-regression registry-accessor refactor that degenerates to today's scalar path on a single display (verifiable now); 3b = split-weave (needs a second 3D display).

    Remaining follow-ups: FPC-serial read (multi-Leia camera pairing), per-display PreferredPlugin\<monitor-key> override.

  7. dfattal
    added 3 commits that reference this issue on Jun 1, 2026

    docs(roadmap): SR-weaver three-tier vendor ask; correct hosted/#111 (#69

    e8544d3

    docs(roadmap): current multi-display state + sharpen 3a recommendation (

    69dac3d

    feat(compositor): route DP factory selection through the registry (#387…

    864313b
  8. dfattal
    added a commit that references this issue on Sep 19, 2026

    feat(plugin-loader): DXR_PLUGIN_EXCLUSIVE + DXR_INPUT_PROVIDERS=0 for…

    efbe943

  9. dfattal commented on Oct 7, 2026

    @dfattal
    dfattal
    on Oct 7, 2026
    CollaboratorAuthor

    Re-scoped 2026-10-07 — this is now the multi-screen epic. Plan: docs/roadmap/multi-screen.md + ADR-047 (PR #1849), written jointly with the LeiaSR side.

    What changed vs. the archived design above: a spanning window is split into per-screen segments (own DP, own present origin, own eyes, own Kooima frustum), not one atlas sliced at the display boundary (ADR-015 §3–§4 superseded); per-screen views ride PRIMARY_MULTIVIEW_DXR via a XR_DXR_display_info revision (display enumeration, DISPLAY space per display, per-view display binding). No shared room frame is required (#46 stays deferred).

    Milestones (each demoable on ds1-linux):

    LeiaSR side: phases A (external weaver routing), B (display enumeration + refresh getter), C (display binding) merged on the Linux line at 876620d62 on 2026-10-07; q3 twins LeiaInc/LeiaSR#433/#435/#437 in review. Control Panel view: #793.

    2026-10-07 — M0–M4 merged (rebase, review fixes included): runtime #1851 (M0, 959be0b), #1852 (M1, 1e01196), #1853 (M2, 4a521aa), #1854 (M3 + mixed-vendor + DXR_SCREEN_PLUGIN, 785396f); plug-in displayxr-leia-plugin#311 (fdeb5348e) and #313 (cc249a457, supersedes #312). M4 human-verified on ds1-linux by David (anaglyph laptop half, woven tracked DS1 half). Remaining: M5 (two Leia panels — LeiaSR phase D in progress on the Windows rig), M6 (service/hosted/Windows), native-Wayland segments, hotplug re-enumeration, Control Panel #793.

  10. dfattal
    added 4 commits that reference this issue on Oct 7, 2026

    docs: multi-screen plan + ADR-046 (segments and per-screen views; sup…

    2bc3183

    docs: multi-screen plan + ADR-046 (segments and per-screen views; sup…

    6d3d3e9

    feat(linux): enumerate monitors with EDID identity (RandR joined to D…

    cbf5b26

    feat(linux): enumerate monitors with EDID identity (RandR joined to D…

    22983ba
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions