Skip to content

design/feat(lift): in-process lift for engine apps (D3D12) - #1867

Merged
dfattal merged 1 commit into
mainfrom
feat/lift-inprocess-d3d12
Oct 8, 2026
Merged

dfattal merged 1 commit into
mainfrom
feat/lift-inprocess-d3d12

Conversation

@dfattal

@dfattal dfattal commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator

Part of #1858, addresses #1862.

What

XR_DXR_lift was IPC-only: every in-process session (every engine app, including the Unity display provider, which renders in-process on Unity's own D3D12 device) got XR_ERROR_FEATURE_UNSUPPORTED. This PR adds ADR-049 and implements its recommendation:

An in-process D3D11 / D3D12 session on Windows reaches the service's conversion module over a lift-only, sessionless IPC connection it owns. Rendering stays in-process; conversion stays in the one service. No plug-in, ABI, wire or header change.

Why C (ADR-049 has the full evaluation)

(A) lift module in the app process (B) native D3D12 lift slot (C) lift over a lift-only IPC connection
Vendor conversion runtime per machine one per lifting process one per lifting process one
VRAM / warm-up with a browser running doubled doubled shared
Cross-app scheduling (ADR-042 §4) per process per process machine-wide
Plug-in ABI / vendor work none / none appended slots / a D3D12 path each none / none
Per-call latency lowest lowest +1 IPC round trip (~0.1 ms; conversion dominates)
Hybrid adapters app's adapter app's adapter service ingest adapter (app must match)

The first vendor conversion runtime keeps its properties process-global and runs on a single worker, so a second process loading it doubles VRAM and warm-up and competes for the GPU with no shared scheduler. One module per machine outweighs a sub-millisecond hop.

The existing export is already D3D12-consumable: an NT-handle texture without a keyed mutex and an ID3D11Fence created D3D11_FENCE_FLAG_SHARED, both opened by ID3D12Device::OpenSharedHandle.

Implementation

  • ipc_client_lift_link.{h,c} (new, ipc_client): one sessionless APP connection per in-process session, named <exe> (in-process lift). It connects on a background thread on first use, so the caller never waits; it never launches the service (it dials only when the service's single-instance mutex exists), retries a failed connect at most every 5 s, and has a refcounted DEAD state plus generation counter so a service restart drops the streams but never the session.
  • oxr_lift.c: each call picks a route, either the IPC session's own connection or the in-process session's link, and then runs the same ipc_client_lift_* code. While connecting, properties read ACTIVATING; with no service they read UNAVAILABLE (supportedModes = 0), so the ADR-042 consumer contract works unmodified. Create-stream waits up to 6 s for an in-flight connect. On an in-process session, a dead lift connection returns XR_ERROR_RUNTIME_FAILURE instead of losing the session, and its streams then return XR_ERROR_FEATURE_UNSUPPORTED (recreate them). GL / VK / Metal in-process sessions and non-Windows platforms are unchanged (FEATURE_UNSUPPORTED).
  • ipc_client_compositor.c: the eight one-line lift bridges become one connection accessor. The IPC-session path is functionally unchanged, and the weave-rects bridge is kept.
  • oxr_session.c / oxr_objects.h: the session owns the link and frees it after its lift streams.
  • Docs: ADR-049 (+ index), XR_DXR_lift.md §2 availability, new §5.3 "D3D12 callers" (open/wait the result; D3D12 inputs without a keyed mutex must be completed and double-buffered), §9, §10, §13, version-history note (no SPEC_VERSION bump: detection is unchanged, since an older runtime answers FEATURE_UNSUPPORTED), the Unity app-path doc, CLAUDE.md extension line.

Verified

  • Windows build (scripts\build_windows.bat build) — see the comment below for the result.
  • No unit test: the new logic is a connect/refcount state machine around a real pipe. A meaningful test would dial the live service, which is off-limits on the shared box (rules).

Owed (not runnable here: shared box, no service tests)

  1. Minimal D3D12 test app (cube_lift_handle_d3d12_win or a --lift mode in cube_handle_d3d12_win): in-process D3D12 session, enable XR_DXR_lift, poll xrGetLiftPropertiesDXR until READY, create an SBS stream, render a test image into a D3D12_HEAP_FLAG_SHARED texture (double-buffered), signal + CPU-wait, xrSubmitLiftFrameDXR, xrAcquireLiftResultDXR → OpenSharedHandle texture + fence → Wait → draw the SBS result into the two eye tiles. Expected app-log WARNs: [lift] in-process: lift-only service connection up (generation 1); service log: [lift] stream N export texture WxH fmt=… + fence ready. displayxr-cli clients lists <exe> (in-process lift) as APP without a session.
  2. Refused-connection path: run the same app on a dev runtime whose git tag differs from the installed service (the version-tag gate refuses the connection) → properties UNAVAILABLE, one WARN ... no service connection for 2D->3D conversion (connect refused or failed) ..., no session loss.
  3. Service restart mid-stream (supervising session): one WARN ... connection (generation 1) is gone ..., the next submit returns XR_ERROR_FEATURE_UNSUPPORTED, properties go ACTIVATING → READY, and a new stream works (generation 2). The app keeps rendering throughout.
  4. Hybrid box: an app whose D3D12 device is on a different adapter than the service ingest adapter sees OpenSharedHandle fail; it must fall back (follow-up: the lift adapter LUID in properties).

Follow-ups (ADR-049)

  • XrLiftAdapterPropertiesDXR (lift adapter LUID) and XrLiftSubmitSyncDXR (submit-side fence for D3D12 inputs). Both are spec bumps, to be coordinated with ADR-048's bump (feat(lift): window-relative viewpoints + lift viewpoint policy (gains, axis mode, ease-back) #1864).
  • displayxr-unity provider work (separate PR): gate on properties READY, manage the stream lifecycle (recreate on FEATURE_UNSUPPORTED after a reconnect), double-buffer the D3D12 input + fence wait, OpenSharedHandle export + queue Wait, check the adapter, and add a C# surface (e.g. a lift component for camera / video textures).

🤖 Generated with Claude Code

@dfattal
dfattal requested a review from a team as a code owner October 8, 2026 10:11
@dfattal
dfattal force-pushed the feat/lift-inprocess-d3d12 branch from c86a452 to d2fea8a Compare October 8, 2026 22:59
…over a lift-only connection (ADR-049)

XR_DXR_lift was IPC-only, so engine apps, which render in-process (the Unity
display provider on its own D3D12 device), got XR_ERROR_FEATURE_UNSUPPORTED.
Lift streams belong to a connection, not a session, so an in-process D3D11/D3D12
session on Windows now opens a sessionless APP connection on its first lift call
(ipc_client_lift_link: background connect, never launches the service, 5 s retry,
refcounted DEAD state + generation) and runs the ordinary lift calls on it.
Results are the existing NT-handle texture (no keyed mutex) + shared fence, which
ID3D12Device::OpenSharedHandle opens directly.

One vendor conversion runtime per machine (shared VRAM, one warm-up, one
scheduler) is the deciding reason over an in-process module or a D3D12 DP slot;
ADR-049 evaluates the three options. No plug-in, ABI, wire or header change.
The IPC-session path is functionally unchanged (its one-line bridges collapse to
a connection accessor). A lost lift connection never loses an in-process session.

Part of #1858, addresses #1862.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dfattal
dfattal force-pushed the feat/lift-inprocess-d3d12 branch from d2fea8a to cd6ae9e Compare October 8, 2026 22:59
@dfattal

dfattal commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto main after #1864 + #1868 (win/lift merge train):

  • docs-lint (run 37761775365) was a stale ADR index: docs/README.md lacked the ADR-049 row. Regenerated with scripts/gen_adr_index.py; the index now lists ADR-048 and ADR-049.
  • oxr_lift.c: kept this PR's route model (IPC connection or in-process lift link) and moved feat(lift): app rig drives lift viewpoints + metric depth aux output #1868's v3 additions onto it: XrLiftDepthRequestDXR validation runs before the route opens; ipc_client_lift_stream_create takes aux_outputs/aux_depth_format; submit passes view_control and rig; the depth export (lift_fill_depth_result) now uses ipc_client_lift_get_depth_output/get_fence on the result's route, and the route closes after the depth fill, carrying the failing xrt result. The get_depth_output bridge feat(lift): app rig drives lift viewpoints + metric depth aux output #1868 added in ipc_client_compositor.c goes away with the other bridges.
  • oxr_objects.h: the stream keeps both depth_requested/depth_exported (v3) and link_gen (ADR-049).
  • XR_DXR_lift.md: Spec Version stays 3 (from feat(lift): app rig drives lift viewpoints + metric depth aux output #1868), with this PR's in-process wording in the type row, ADR-049 added to the decision row, and the in-process FEATURE_UNSUPPORTED row. The v3 validation row stays: EXPLICIT on a weave rect is legal from v3.

Verified locally: scripts\build_windows.bat build is clean, and ctest -C Release -R lift passes 3/3 (lift_mailbox, aux_lift_viewpoint, aux_lift_rig_depth). The end-to-end in-process path is still untested on hardware.

🤖 Generated with Claude Code

@dfattal
dfattal merged commit 580aef9 into main Oct 8, 2026
40 checks passed
@dfattal
dfattal deleted the feat/lift-inprocess-d3d12 branch October 8, 2026 23:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant