feat(lift): XR_DXR_lift — vendor 2D→3D conversion module (ADR-042) - #1781
Merged
Merged
Conversation
…-279, catalog note (ADR-042) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…t ring state machine, with unit tests Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ug-in factory, sim_display FAKE lift Five append-only slots on xrt_display_processor_d3d11 (lift_get_caps, lift_stream_create, lift_stream_destroy, lift_convert, lift_convert_blob) per ADR-020, plus the optional xrt_plugin_iface::create_dp_d3d11_lift factory so a plug-in can build a DP that serves lift only (no weaver, no tracker). sim_display fills the slots only under SIM_DISPLAY_FAKE_LIFT=1: shifted SBS/N-view, gradient depth, a two-layer 3DGS PLY. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ift entry points
- d3d11_lift.{h,cpp}: one lift thread on a dedicated device (ID3D11Multithread
protected) with the plug-in's lift-only DP; per-stream latest-wins mailbox and
N=2 output ring as keyed-mutex shared textures crossing service<->lift device;
client export texture + fence (weave output pattern); blob latch; tracked eyes
always passed as explicit viewpoints; DXR_LIFT=0 kill switch.
- comp_d3d11_service: system-level lift API; lift-flagged weave rects woven at
the rect's current position from the latest result (batch v3 + v6 N-view),
flat until the first result.
- IPC: lift_* calls in proto.json (connection-owned streams, varlen blob),
server handlers, ipc_client_lift.{h,c}, xc bridges.
- oxr_lift.c: xrGetLiftPropertiesDXR / xrCreate|DestroyLiftStreamDXR /
xrSubmitLiftFrameDXR / xrAcquireLiftResultDXR / xrAcquireLiftBlobDXR.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…tats, focal_px - oxr_weave.c: XrWeaveSubmitLiftRectsDXR — validated, forwarded as lift_weave_rects right before the submit it belongs to. - XrLiftPriorityDXR + xrSetLiftStreamPriorityDXR / xrGetLiftStreamStatsDXR: each lift-thread round converts every HIGH stream with a new frame, one NORMAL round-robin, LOW every 4th round, PAUSED never (u_lift_sched, unit-tested); effective rate from publish intervals. - xrt_dp_lift_params.focal_px / XrLiftOptionsDXR::focalPx (appended) for photo -> Gaussians modules. - Spin fix: pending frames only wake the lift thread once the module is READY. - Tests: scheduler, rate, and the sim_display fake PLY's format. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…check - lift caps [--json]: connects over IPC as DIAG, polls through ACTIVATING. - lift probe <image|frames_dir> (Windows): shared keyed-mutex input textures, sequential or --pipelined submits, reads results back through the exported texture + fence, writes lift_out_<i>.png (depth normalised) / .ply, prints per-frame submit->acquire and service latency, stream stats. - selftest: lift_caps (WARP + lift-only factory); absence never fails, malformed caps -> CLI_SELFTEST_BAD_LIFT_CAPS (11). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…g in XR_DXR_lift.h Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…rvice threads/locks, CLAUDE.md - docs/specs/extensions/XR_DXR_lift.md: lifecycle, states, submit mailbox, texture + blob acquire, weave-rect lift, priority scheduling + stats, timestamps, service threading, error table, vendor contract, probe, consumers, version history. - ADR-042: a READY vendor module supersedes the browser/SDK open default (vendor Gaussian modules supersede the open MoGe + generator lift the same way); SBS-not-woven; async one frame behind; geometry from the weave rect; browser + web SDK consumer contract. - xrt_plugin_iface.md: the five lift slots + create_dp_d3d11_lift. - service-architecture.md: the lift thread, its lock, lock order. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…k; add --no-write / --write-every N (the readback+encode dominated the reported latency and capped the pipelined rate)
…on, non-elevated CLI, NeurD version gate, --no-write)
…alpha instead of forcing 1.0 — present-owner weaves showed the page as black around the 3D tiles (#1742)
…R_LIFT_MAX_INPUT_EDGE, default 1920) A lift-flagged weave rect was snapshotted at DEVICE pixels. On a real Leia 8K panel at 300% DPI with the NeurD DirectML module, a fullscreen YouTube player is a 7680x4319 rect, so the module synthesized a 15360x4319 SBS per frame: 1080p input converts in 48 ms, 4K in 61 ms, 8K in 132 ms (~6.5 Hz). The vendor's own autoscaling (inputScale / INPUT_AUTOSCALING) shrinks only inference, not view synthesis, so the cap has to happen before the DP. The spec already stretches the newest result into the rect's CURRENT position, so a smaller snapshot is legal and costs little on the panel. ADR-042: the runtime owns this policy. - d3d11_lift: weave-rect (SRV) snapshots are scaled so the long edge is at most DXR_LIFT_MAX_INPUT_EDGE (read once at lift-module create in the service; default 1920, 0 = off, 1-255 clamp to 256). The long edge becomes the cap rounded down to even, the short edge scales by the same factor to the nearest even value (min 2). The in-slot is allocated at the scaled size, and the mailbox commits the SCALED dims so the module sees its real input size. The capped blit uses a new 4-tap bilinear box-filter PS (exact up to 4x, taps clamped inside the sub-rect); the uncapped blit keeps the 1:1 point path. A focal_px hint is scaled with the input. - xrSubmitLiftFrameDXR frames (d3d11_lift_submit_handle) are never capped. - One WARN per stream when the cap engages or its capped size changes. - u_lift_cap_dims / u_lift_max_input_edge_parse are pure helpers in u_lift_mailbox, pinned by tests_lift_mailbox. - Docs: XR_DXR_lift.md step 1; census row in control-panel-performance-settings.md (a "Convert-to-3D quality" tier). Consumers already sample the result into the rect (lift_weave_rect_batch, lift_weave_rects_nview: blit_to_atlas_texture with pin.width/height as the source size and the rect as the destination, linear sampler). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s (and subtitles in them) flat A lift-flagged weave rect often holds a film with black bars, sometimes with subtitles in them. Converting the bars wastes module time, the hard black edge confuses depth, and subtitles get lifted with the picture. Service side, before the DXR_LIFT_MAX_INPUT_EDGE cap (ADR-042: the runtime owns the policy): - measure: a small GPU pass writes each row/column bucket's non-black fraction (512x2 R32F), read back through a 3-deep staging ring with DO_NOT_WAIT, so the weave never stalls (letterbox_step); - decide: u_lift_letterbox_update (pure, ctested) — a bar is the run from an edge below 25% non-black; a shorter bar is extended to the opposite one when the extra rows are only sparsely lit (<60%), so dense subtitles stay in the bar; bars grow after a 45-frame settle, shrink at once, black frames change nothing, a resize resets; - crop: submit_locked snapshots only the active area (cap applies to it), recording the normalised active rect on the input slot -> output slot -> pin, so each result is placed with the crop it was made with; - recompose: lift_weave_rect_batch stretches the result into the active sub-rect and writes the bar strips from the 2D input identically into both views (zero disparity); the v6 N-view path keeps the caller's flat tiles in the bars. DXR_LIFT_LETTERBOX (service env, default on, 0 = off); never applied to an app's own xrSubmitLiftFrameDXR frame. One WARN per stream per crop change. Verified on the 8K Leia panel (NeurD 0.4.4 DirectML, DXR browser #6): a 3386x1904 rect holding a 2.39:1 still with a subtitle in the bottom bar -> "letterbox crop 3386x1904 -> active 3386x1422 (bars top 240 bottom 242)", module input 1920x806 instead of 1920x1080; weave-dump SBS: top bar and subtitle bar |L-R| = 0.00, active area 2.75. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ne pillarbox Panel feedback on 73a87ce (8K Leia, NeurD 0.4.4, YouTube in DXR browser #6): - a sliver of the black bar was still woven: the profile resolves a bar edge to one bucket (~h/512 rows) and NeurD leaves a seam at the crop edge. Recompose now draws the flat bar strips AFTER the result, each reaching ceil(size/512)+4 px into the active area; - subtitles made the bar weave intermittently: the log showed the bottom bar flipping 186 -> 76 -> 74 -> 0 while the top held 184 (one caption frame shrank the crop at once, regrowing took 45 frames). Shrinking now needs picture (>= 25%) to intrude for 6 consecutive frames and by more than the jitter tolerance; the symmetry rule also accepts a band separated from the picture by a black gap, however dense its text; - a spurious pillarbox (left 514 right 516) committed during a dark stretch: bars now GROW only into truly black buckets (< 3% lit). u_lift_letterbox keeps two readings of each profile (grow = black, keep = no picture). Tests: +2 cases (dark edges never grow a bar; 300 frames of flickering dense captions keep the bottom bar), un-crop case updated to the 6-frame debounce; 23/23 [lift] pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Panel (YouTube, build #7): bars settled at 250, then 254, then 256 — three re-crops, each re-sizing the module input, within seconds. Once a crop is in effect, growth now commits only when an edge grows beyond a deadband of max(8 px, size/256); the recompose's flat bars already reach ~7 px into the active area. First engagement is unchanged. +1 ctest (4 px thicker = no re-crop; 40 px thicker = re-crops); 24/24 [lift]. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…icture David on the panel: the flat bars' ~7 px overlap into the active area (4d3ba8c, there to hide one profile bucket of slack + the module's edge seam) makes the woven picture and the flat picture rows disagree at the border. The profile is now one bucket per row/column (BINS_MAX 8192, one 64 KB async readback), bar sizes are no longer rounded down to even, and the recompose tiles the rect exactly: the result fills precisely the cropped rect it was converted from, the flat bars exactly the rest. Also: every crop-change WARN carries the deciding row profile (64 bands, digit = max non-black fraction x10), so an unexpected un-crop in the field says what was in the bars (captions, player UI). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
XR_DXR_lift — vendor 2D→3D conversion exposed by the runtime (ADR-042)
Lands the lift series that has been panel-tested on the
lift/on-*branches since 2026-09-25. 16 commits, rebased ontomain; each is a self-contained step, so this is a rebase-merge.What it adds
XR_DXR_liftextension (header, spec, catalog note; type values 1004999270–280) and its OpenXR entry points.u_lift_mailbox— latest-wins input mailbox + pinned output ring, with unit tests.XRT_DP_D3D11_HAS_LIFT, appended at BASE+25..29) and a lift-only plug-in factory (XRT_PLUGIN_IFACE_HAS_D3D11_LIFT_FACTORY).sim_displayships a FAKE lift for hardware-free testing.DXR_LIFT_MAX_INPUT_EDGE, letterbox crop).displayxr-cli lift caps/lift probe, and alift_capsselftest check.Compatibility
struct_size-gated: a released plug-in without lift loads unchanged and reports lift UNAVAILABLE; a lift plug-in on an older runtime is never read past slot 24.XR_DXR_liftand the active plug-in fills the slots.Verification
displayxr-cli selftestPASS incl.lift_capswith the NeurD-backed Leia plug-in (neurd-directml, NeurD 0.4.6);lift capsREADY.weave_submit, holdsimmediate_ctx_mutexat both call sites, and touches none of the retyped textures.Follow-ups (not in this PR)
weave_submitcreates the crop RTV in the caller's format; an_SRGB-typed weave input would double-encode lifted pixels. No known caller sends one.🤖 Generated with Claude Code