Skip to content

feat(weave): multi-screen M6 for present-owners — weave rects per screen, per-screen eyes out (XR_DXR_weave v19) - #1913

Merged
dfattal merged 3 commits into
mainfrom
feat/1884-weave-rects-per-screen
Oct 10, 2026
Merged

dfattal merged 3 commits into
mainfrom
feat/1884-weave-rects-per-screen

Conversation

@dfattal

@dfattal dfattal commented Oct 10, 2026

Copy link
Copy Markdown
Collaborator

Part of #1884

What

The runtime half of multi-screen M6 for the browser. A present-owner (XR_DXR_weave) whose bound window straddles two 3D screens is now woven per screen by comp_d3d11_service_weave_submit, and the caller gets one eye set per screen plus every weave rect cut at the seam.

  • Service (comp_d3d11_service.cpp): batch (v3) and N-view (v6) submits run the bound window's client area through comp_segments_compute (canvas = the whole window). The session's weave DP weaves the primary screen's segment. Each other screen's windowless DP (create_dp_d3d11_for_screen, set_present_origin, viewport + scissor = the segment) weaves its own, all into the same woven handback. Where nothing can weave, the segment is flat 2D. The v4 overlay is composited after that by the runtime's post-weave blit, because no segment DP covers the whole window. This reuses comp_d3d11_segments from feat(multi-screen): segments + per-segment views on the service / IPC path (stacked on #1878) #1881. The present-owner gets a manager of its own (weave_segs), made on sys->device, which is where the handback and the Service crash-loops serving a browser (present-owner) client on v2.8.0-112 — access violation in igd10iumd64 via SimulatedRealityDirectX during weave #1172 ingest DP live. It is built from the same opt-in screen list (compositor_segments_enable) and released with the compositor.
  • Unchanged paths: a window on one screen, a client that never opted in (every single-screen box), and a legacy single-rect submit all take the existing single-DP path. weave_seg_update returns false before anything else for these.
  • Drag snap: xrWeaveSnapWindowRectDXR asks comp_d3d11_segments_snap_dp. That picks the DP of the screen holding the majority of the window at the proposed position, using comp_segments_owner_* with Amendment 2's 20 % / 0.5 s hysteresis. It is a separate owner from the window-handle one, because nothing is rebuilt. If that screen has no DP yet or its DP cannot snap, the call falls back to the panel DP.
  • Segments manager: a client_presents field was appended to comp_d3d11_segments_frame (zero-init = old behaviour) so segment DPs are told what the session DP is told, plus the snap-owner query.
  • IPC: weave_get_segments is appended to proto.json. It returns the last submit's xrt_segment_metrics (the per-segment-views table) with each screen's eyes. The primary's eyes are the submit's own base eyes; the others are predicted from their segment DPs. weave_submit's reply is unchanged.
  • OpenXR (oxr_weave.c): when the caller chains the new struct, the state tracker fetches the table and cuts the submitted rects at the seams with u_weave_rect_parts (new, header-only, auxiliary/util/). The rects cross the wire once.

Why

The browser is a present-owner, so it never reaches the in-process sites or the direct-pipeline segment path from #1881. Its inline-3D element across two panels was woven with the primary screen's lattice and viewer on both halves. The issue calls this the "wrong lattice / looks around from the wrong camera" symptom.

Extension revision: XR_DXR_weave 18 → 19

New, out-only, chained on XrWeaveOutputDXR::next. The struct type is XR_TYPE_WEAVE_OUTPUT_RECT_PARTS_DXR = 1004999292, registered in openxr_includes/openxr/README.md.

XrWeaveOutputRectPartsDXR { type, next,
  uint32_t segmentCount;                          // 0 = one DP: use XrWeaveOutputDXR::eyes
  XrWeaveScreenSegmentDXR segments[2];            // displayId, windowRect, screenRect, screenSize,
                                                  // screenSizeMeters, isPrimary, woven,
                                                  // eyeCount/eyes[8]/eyesValid/eyesTracking (THAT screen's display space)
  uint32_t partCount;
  XrWeaveRectPartDXR parts[64];                   // rectIndex, segmentIndex, windowRect, rectRelative, screenRect
  uint64_t generation; }                          // moves on a geometry change only

Old callers see no change. A caller that does not chain the struct sends exactly the pre-v19 messages. A v19 caller on an older runtime gets the struct skipped, so it stays zero, which reads as "use the base eyes". The header, the spec section §5g plus the version-history row, and the index.json summary are all updated. python3 scripts/gen_extensions_index.py --check passes.

Consumer contract (browser, displayxr-browser-pvt)

  1. Byte-sync the vendored XR_DXR_weave.h (spec 19). Gate on extensionVersion >= 19 only to expect parts; chaining is always safe.
  2. On every xrWeaveSubmitDXR (batch or v6), chain a zero-initialised XrWeaveOutputRectPartsDXR (type = XR_TYPE_WEAVE_OUTPUT_RECT_PARTS_DXR) on XrWeaveOutputDXR::next. It is about 4 KB, so keep it off the stack in hot code.
  3. If segmentCount == 0, render every rect from XrWeaveOutputDXR::eyes exactly as today.
  4. If segmentCount >= 1, for each parts[i]:
    • Render the pixels parts[i].rectRelative of rect parts[i].rectIndex from segments[parts[i].segmentIndex].eyes. Use them for the next frame, the same one-frame lag as the base eyes.
    • Those eyes are in that screen's display space: metres, origin at that screen's centre.
    • Frame the part's off-axis (Kooima) projection against that screen's physical rectangle. Use screenSizeMeters and screenSize (px), with the part placed on the screen at parts[i].screenRect. This is the same split the per-segment views make for OpenXR apps (oxr_segment_views).
    • A rect wholly on one screen yields exactly one part, the whole rect. A rect off every screen yields none.
    • On the batch layout, rectRelative is in rect pixels. Each SBS half holds the whole rect squeezed, so a part occupies rectRelative.x/w ÷ 2 inside each half. On v6, place it at the same window-relative position scaled by the mode's view scale in every tile.
    • If eyesValid == XR_FALSE for a segment (typically woven == XR_FALSE, shown flat as the left view), render that part from the base eyes.
  5. Use generation to re-plan per-part render targets only when it changes. The eyes change every frame and do not bump it.
  6. Parts are per submit, in that submit's rect order. A frame split over several submits (more than 32 rects) gets each submit's parts separately.
  7. Drag snap needs nothing new: keep calling xrWeaveSnapWindowRectDXR. It now snaps to the majority screen's lattice.
  8. Windows only for now (D3D11 service). macOS, Linux and Android always report segmentCount == 0.

Tests

  • scripts\build_windows.bat build: clean. The one C4090 warning in oxr_session.c is pre-existing; that file is untouched.
  • ctest -C Release: 111 of 114 pass. The 3 failures are the known-environmental tests_oxr_view_space_mode_floor, _mode_pinned and _mode_floor_off.
  • New tests_weave_segments (7 cases, 129 assertions):
    • a rect straddling the seam gives two parts that tile it exactly, with the right window, in-rect and on-screen rects;
    • a rect on one screen of a split window gives one part equal to the rect;
    • a window on the primary only gives count 0 and no parts (the single-DP path);
    • a window wholly on the second screen;
    • a screen at negative desktop x;
    • empty and off-screen rects, and the part cap;
    • the weave_get_segments reply fits IPC_BUF_SIZE, and two screens' different eyes survive the wire and land on the right parts.
  • tests_ipc_proto.py: 14 tests OK. A new case pins weave_get_segments as the last call and weave_submit's reply as unchanged; the earlier "last N calls" assertions were shifted by one.
  • scripts\build_windows.bat test-apps: weave_rpc_probe_d3d11_win builds. cube_handle_gl_win and cube_zones_d3d12_win failed only on a parallel texture-copy race in the post-build step, which is unrelated.
  • Not run on panels (another session holds the service and the rig).

Rig recipe (two 3D panels side by side, the primary on the left)

Use the existing present-owner probe, test_apps/probes/weave_rpc_probe_d3d11_win. This PR adds a v19 log to it.

  1. Push the build with scripts/push-runtime-pf.sh and register the dev Leia plug-in if needed. Start displayxr-service non-elevated (explorer.exe "…\displayxr-service.exe").
  2. Launch the probe forced-IPC from a non-elevated prompt: set XRT_FORCE_MODE=ipc && test_apps\build\bin\weave_rpc_probe_d3d11_win.exe --rect=full --size=1600x900. --rect=full makes the weave rect the whole client area, so it straddles with the window.
  3. Service log, once per manager: segments: present-owner … woven per screen by weave_submit; segment DPs on the render device …. Also segments: created a segment DP for screen 0x… via create_dp_d3d11_for_screen once the window touches the second panel.
  4. Drag the probe window across the seam:
    • The probe log goes from v19 per-screen: segments=0 … (one display processor: base eyes apply) to segments=2 parts=2.
    • The two segment lines should show different display ids and each panel's own eyes; the second panel's tracker eyes will differ from the primary's.
    • The two part lines should tile the rect at the seam.
    • Drag back and it returns to segments=0.
  5. By eye: both halves are woven, each with its own panel's lattice, and there is no crosstalk band on the second panel. The probe renders one eye pair only, so look-around per half is the browser's job.
  6. Drag snap: with the majority on the second panel for more than 0.5 s, the service log shows segments: drag snap follows screen 0x…'s lattice (majority, #1884) at INFO. The window should land phase-correct on the second panel.
  7. Capture: touch %TEMP%\dxr_weave_dump_trigger dumps the woven handback (dxr73_weave_output.png).

Open questions

  • Device. The issue says "DPs created on the output device". This PR makes the weave's segment DPs on sys->device (the render device) instead. That is where the woven handback lives and where the Service crash-loops serving a browser (present-owner) client on v2.8.0-112 — access violation in igd10iumd64 via SimulatedRealityDirectX during weave #1172 ingest DP already is, so it follows Amendment 1's rule that a segment DP lives where its target is. A DP on the output device could not write the handback under the Epic: weave/repaint next to scanout — output-device split for hybrid GPUs #918 split.
  • Hardware wish. The v8 per-region hardware wish is still published to the panel DP only, so only the primary screen's lens follows the content. Segment DPs get no wish.
  • More than two screens. A window across more than XRT_MAX_SEGMENTS (2) screens is woven per screen but reported with segmentCount 0 (one eye set).
  • Legacy single-rect submits stay single-DP. Segmenting them would need a per-element canvas cut, and the browser uses batch or v6.
  • Snap support on windowless DPs. The snap on a windowless segment DP depends on the plug-in implementing snap_window_rect without a window. It falls back to the panel DP if not, which is the old behaviour. This needs checking on the rig with the Leia plug-in.

🤖 Generated with Claude Code

dfattal and others added 3 commits October 10, 2026 11:22
…ve v19)

A present-owner's bound window that straddles two 3D screens is now woven
per screen by comp_d3d11_service_weave_submit, each screen's part by that
screen's display processor, and the caller gets one eye set per screen plus
every weave rect cut at the seams (#1884, ADR-047 Amendment 4).

Service: batch (v3) and v6 submits from a client that sent
compositor_segments_enable run the bound window through a segment manager of
the present-owner's own (comp_d3d11_segments on sys->device, where the
handback and the #1172 ingest DP live): comp_segments_compute over the client
area, the session's weave DP on the primary screen, windowless per-screen DPs
elsewhere (scissor = the segment, present origin), flat 2D where nothing can
weave, the v4 overlay composited by the post-weave blit. Single-screen windows,
legacy single-rect submits and clients that never opted in take the existing
path unchanged. The drag snap asks the majority screen's segment DP
(comp_d3d11_segments_snap_dp, Amendment 2's hysteresis) and falls back to the
panel DP.

Extension: XR_DXR_weave spec 19 adds XrWeaveOutputRectPartsDXR (chained on
XrWeaveOutputDXR) with XrWeaveScreenSegmentDXR / XrWeaveRectPartDXR; type
1004999292. IPC: weave_get_segments appended to proto.json (the
xrt_segment_metrics table with each screen's eyes), fetched only when the
caller chains the struct; the client cuts its rects with u_weave_rect_parts.

Tests: tests_weave_segments (straddling / single-screen / off-primary /
negative-desktop / cap cases, and the weave_get_segments reply round trip),
tests_ipc_proto pins the append.

Part of #1884

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
weave_rpc_probe_d3d11_win chains XrWeaveOutputRectPartsDXR when the runtime
reports XR_DXR_weave spec >= 19 and logs the per-screen table and the weave
rect's parts on every geometry change, so a two-panel rig can watch a window
dragged across the seam go from one display processor to two and back.

Part of #1884

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ract

Present-owner weave rects are segmented per screen with per-screen eyes out:
ADR-047 Amendment 4, comp-segments.md "Present-owners on the weave service",
XR_DXR_weave.md section 5g + version history + consumers, the index.json
summary, and the multi-screen roadmap's M6 row.

Part of #1884

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dfattal
dfattal requested a review from a team as a code owner October 10, 2026 18:23
@dfattal

dfattal commented Oct 10, 2026

Copy link
Copy Markdown
Collaborator Author

Rig (2026-10-10, two panels, dev service from this branch, dev Leia plug-in): the forced-IPC present-owner probe with --rect=full dragged across the seam → service logged segments: present-owner … a bound window spanning monitors is woven per screen by weave_submit and created a segment DP for screen 0xb72e… via create_dp_d3d11_for_screen. Not verified: the probe's v19 per-screen log (the probe binary predates the probe change) and the browser half (browser-pvt). Merging on David's direction to prioritise VK/GL + shell/browser.

@dfattal
dfattal merged commit 5261d6b into main Oct 10, 2026
41 checks passed
@dfattal
dfattal deleted the feat/1884-weave-rects-per-screen branch October 10, 2026 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant