Skip to content

feat(multi-screen): segments + per-segment views on the service / IPC path (stacked on #1878) - #1881

Merged
dfattal merged 8 commits into
mainfrom
feat/win-segments-service-path
Oct 9, 2026
Merged

dfattal merged 8 commits into
mainfrom
feat/win-segments-service-path

Conversation

@dfattal

@dfattal dfattal commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #1878 (feat/win-segments-under-split) — merge that first; this branch is rebased on it and its diff against main includes #1878's commits until then.

Multi-screen segments (ADR-047 D2/D3) on the service / IPC path: an _ipc client composited by displayxr-service.exe whose window straddles two monitors is woven per screen (one DP per screen, each with its own canvas rect, present origin and, with per-segment views, its own viewer), exactly as the in-process D3D11 compositor does since #1857/#1863/#1878.

Design

  1. Window geometry. The app's real HWND (the XR_DXR_win32_window_binding handle the service already presents into as the APP_HWND presenter, fc->render.hwnd). The render thread reads its client area with GetClientRect + ClientToScreen in the per-monitor-DPI-aware service, which gives device px, the space the registry's monitor rects use. Both calls work cross-process and send no message. Same source as comp_d3d11_service_get_client_app_window_metrics.
  2. Screens registry. It already lives in the service: the server instance's enumerate_displays (target_screens / u_screen_info, plug-in get_display_info_for_monitor), which xrEnumerateDisplaysDXR reaches over IPC today. The client sends only its session's XrSessionDisplayBindingDXR pin (compositor_segments_enable). The IPC handler enumerates the service's own screens and hands the client compositor a heap copy, plus a copy of the system info (the DP registry), as in-process set_screens does.
  3. DP creation. One comp_d3d11_segments manager per client. It is the in-process manager, split into a new comp_d3d11_segs static lib (with the output composite unit it blits flat 2D through) so the service links it without the whole in-process compositor, following the comp_xbridge precedent. It is built lazily on the render thread on svc_out_device (the OUTPUT device under the Epic: weave/repaint next to scanout — output-device split for hybrid GPUs #918 split) with the same create_dp_d3d11_for_screen factory slot. The panel DP, bound to the app's HWND, is the primary and keeps drag phase-snap. Only one client's segment DPs exist at a time: a focus change releases the outgoing presenter's set. Teardown runs under render_mutex.
  4. Where it weaves. Only the direct single-client pipeline (pipeline_default_policy_render) with an APP_HWND presenter. pipeline_segments_weave replaces that frame's single process_atlas. A hosted client presents into the service window on the panel. A present-owner or CLIENT_TEXTURE client weaves on its own thread. A zones frame keeps the single DP. None of these is segmented.
  5. IPC metrics + routing. Two messages are appended to proto.json: compositor_get_segment_metrics (xrt_segment_metrics by value) and compositor_set_view_routing (xrt_segment_view_routing by value). Both are static_asserted to fit IPC_BUF_SIZE, and the protocol stays append-only (a test pins the three new calls at the end). The service fills each segment's eyes at query time: the primary's from the panel DP, the others' from their segment DPs. A table older than 250 ms is not handed out, so a client that lost the panel locates one view set. The client-side get_segment_metrics / set_view_routing are IPC proxies. oxr_session_locate_views then runs unchanged, with two IPC-only differences: a segment locate keeps its client-side Kooima FOVs over the device's single-set answer, and it never takes the server locate_views_rig call, which knows one view set. Routing is sent change-only from xrEndFrame. The service lays a routed commit into the client atlas as the mosaic (comp_segments_route_place, the same rect mapping the crop reads). A routed frame never zero-copies.
  6. View-set capacity. On Windows a service session now counts the service's registry, from the by-value copy of xrt_system_compositor_info (the factory pointers are service addresses and are only NULL-tested). On a box with two D3D11-DP screens PRIMARY_MULTIVIEW_DXR doubles, as it does in-process.
  7. Compose / shell mode: scoped out, documented as a no-op. multi_compositor_render keeps one presenter, the service window on the panel, so a workspace window is never segmented and its table stays empty. Per-screen workspace presenters are a separate decision, recorded in ADR-047 Amendment 3.
  8. Byte-identical: single-screen IPC clients (capacity 1, so no enable, no per-locate fetch, no routing sends, no service-side work), any client that does not opt in, compose mode, Android and Linux (the Linux capacity branch still excludes service mode, and the new client calls are gated on a Windows-only enable).

Not yet on the service path (follow-ups)

  • In a routed frame, quad, equirect and cylinder layers are drawn once per tile with that tile's camera. In-process draws them once per segment.
  • A second projection layer in a routed frame is routed only where the per-tile pass visits it (segment 0).
  • Hosted windows and the workspace get no per-screen presenters (see 7).

Files

  • src/xrt/compositor/d3d11/CMakeLists.txt: new comp_d3d11_segs lib (no source edits to comp_d3d11_compositor.cpp / comp_d3d11_segments.*).
  • src/xrt/compositor/util/comp_segments_route.h (new): routed placement and change-only publish.
  • src/xrt/compositor/d3d11_service/{comp_d3d11_service.cpp,.h,CMakeLists.txt}: segment path, mosaic routing, per-client state, IPC-facing setters and getters.
  • src/xrt/ipc/{shared/proto.json,server/ipc_server_handler.c,client/ipc_client_compositor.c,client/ipc_client.h}.
  • src/xrt/state_trackers/oxr/{oxr_system.c,oxr_session.c,oxr_session_frame_end.c,oxr_objects.h,oxr_segment_views.h}.
  • tests/tests_segments_ipc.cpp (new), tests/tests_ipc_proto.py, tests/CMakeLists.txt.
  • Docs: comp-segments.md § Service / IPC path, service-architecture.md §4.3, roadmap/multi-screen.md M6, ADR-047 Amendment 3.

Testing (win box, laptop panel only — no rig run)

  • scripts\build_windows.bat build: exit 0 (no new warnings).
  • ctest -C Release: 101/104 pass. The 3 failures are tests_oxr_view_space_mode_{floor,pinned,floor_off}, which fail on this box independently of this change. The new tests_segments_ipc (7 cases, 81 assertions) and the extended tests_ipc_proto pass, as do tests_comp_segments and tests_oxr_segment_views.

Rig check (forced IPC, two panels: laptop at (0,0), DS1 at (3840,0))

  1. Build the branch: scripts\build_windows.bat build test-apps.
  2. Stop the installed service (tray → Quit) so the dev one can own the IPC pipe, then start the dev service at Medium integrity with a bat run through Explorer:
    rem run_dev_service.bat
    cd /d C:\Users\SPARKS~1\Documents\GitHub\displayxr-runtime-wt-segsvc\_package\bin
    start "" displayxr-service.exe
    explorer.exe C:\path\to\run_dev_service.bat. The service reads its own env: put any DXR_* knob, e.g. DXR_SEGMENTS=0 for an A/B, in that bat.
  3. Client, from a non-elevated terminal:
    set XR_RUNTIME_JSON=C:\Users\SPARKS~1\Documents\GitHub\displayxr-runtime-wt-segsvc\_package\DisplayXR_win64.json
    set XRT_FORCE_MODE=ipc
    test_apps\handle\cube_handle_d3d11_win\build\cube_handle_d3d11_win.exe
    Drag the window across the seam.
  4. Service log (%LOCALAPPDATA%\DisplayXR\DisplayXR_displayxr-service.*.log):
    • segments: IPC client 0x… (pid N) enabled segments — 2 screen(s) in the registry, pinned display 0x0000000000000000
    • segments: 2 screens, plug-in '…' — a window spanning monitors is woven per monitor (multi-screen M6, D3D11)
    • segments: IPC client 0x… (pid N) — a window spanning monitors is woven per screen by the service's direct pipeline; segment DPs on the OUTPUT device 0x… (multi-screen M6, service path)
    • segments: created a segment DP for screen 0x… ('…', plug-in '…') via create_dp_d3d11_for_screen — …
  5. Client log (DisplayXR_cube_handle_d3d11_win.*.log): per-segment views: segments=2 capacity=2 active=2 reported=4 -> total=4; [0] … [1] … (WARN, once per change).
  6. Eyeball: each half woven on its own panel. The DS1 half renders from its own viewer (the pinned nominal viewer on the untracked panel). Moving the window fully onto one panel returns to the single DP within ~0.5 s (hysteresis).

🤖 Generated with Claude Code

@dfattal
dfattal requested a review from a team as a code owner October 9, 2026 07:48
@dfattal
dfattal force-pushed the feat/win-segments-service-path branch from 1c53c8c to 57d8f9e Compare October 9, 2026 09:44
dfattal and others added 8 commits October 9, 2026 02:58
comp_d3d11_segments and the output composite unit it blits flat 2D through
move into their own static library, linked by comp_d3d11 as before, so the
D3D11 service compositor can segment IPC clients' windows without linking
the whole in-process compositor (the comp_xbridge precedent).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
util/comp_segments_route.h (header-only, backend-agnostic): where one
submitted view of a routed projection layer lands in the atlas mosaic
(comp_segments_route_place, all-or-nothing validation, same rect mapping
as the per-segment crop) and the generation-bumping publish of a segment
table. oxr_segment_views_route_differs: the client's change-only send rule
for an out-of-process compositor. Multi-screen M3 on the service path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The direct single-client pipeline (pipeline_default_policy_render) weaves an
APP_HWND presenter whose window spans monitors through the client's own
comp_d3d11_segments manager, made on svc_out_device (the output device under
the #918 split): the panel DP on the primary screen's segment, windowless
per-screen DPs (create_dp_d3d11_for_screen) on the others, flat 2D where no
DP can weave. Opt-in per client (comp_d3d11_service_set_client_segment_screens,
fed the service's own screen registry); a client that never opts in is woven
by the single process_atlas exactly as before.

Per-segment views: the last direct weave's table is published per client
(fresh for 250 ms; eyes predicted at query time) and the client's routing
lays a routed commit into the atlas as the mosaic (service_proj_view_blit_resolve,
never zero-copy). One client's segment DPs at a time: a focus change releases
the outgoing presenter's. Compose / workspace mode is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three calls appended to proto.json: compositor_segments_enable (the session's
display pin; the service enumerates its own screens), compositor_get_segment_metrics
(xrt_segment_metrics by value) and compositor_set_view_routing
(xrt_segment_view_routing by value), with server handlers that are inert off
the D3D11 service and client proxies. Both payloads are asserted to fit
IPC_BUF_SIZE.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Windows service session advertises the per-segment view-set capacity from
the service's registry (the by-value copy; factory pointers only NULL-tested),
enables segments at session create when that capacity is above 1, fetches the
service's table in xrLocateViews and sends the routing change-only from
xrEndFrame. A segment locate keeps its client-side Kooima FOVs and never takes
the single-set server rig call. Single-screen boxes send nothing new.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests_segments_ipc: routed mosaic placement (incl. a 2D commit and an
inconsistent routing), the change-only publish and send rules, and that the
two wire messages fit one IPC buffer and carry their payload verbatim.
tests_ipc_proto: the three calls are appended at the end of proto.json.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
comp-segments: Service / IPC path section; service-architecture: the direct
pipeline's segment path; roadmap M6 status; ADR-047 Amendment 2 (the service
owns the screens, only the direct pipeline segments, compose mode stays
single-screen, table + routing by value).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Amendment 2 is the window handle following the majority segment (#1880),
which lands first; the service / IPC path becomes Amendment 3 everywhere
this branch names it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@dfattal
dfattal force-pushed the feat/win-segments-service-path branch from 57d8f9e to 3eba6fc Compare October 9, 2026 09:59
@dfattal
dfattal merged commit 520d510 into main Oct 9, 2026
40 checks passed
@dfattal
dfattal deleted the feat/win-segments-service-path branch October 9, 2026 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant