DisplayXR — Unreal Plugin

Unreal Engine plugin for rendering on eye-tracked 3D light field displays via the DisplayXR OpenXR runtime. Works with any OpenXR-compatible 3D display.

Table of Contents

• Overview
• Requirements
• Installing the Plugin
• Scene Setup
  • Camera-Centric Mode
  • Display-Centric Mode
• Building Your App
• Documentation
• Related Repositories

New to the plugin? Start with the Display Rig Setup guide — it covers the pawn and camera configuration needed for stereo 3D, plus the differences between the two rig modes. For outstanding work and planned parity with the Unity sibling plugin, see TODO.

---

Overview

The plugin hooks into Unreal's OpenXR pipeline to provide:

• Eye-tracked stereo rendering — Kooima asymmetric frustum projection driven by real-time eye positions from the DisplayXR runtime
• Two stereo rig modes — camera-centric (the camera moves through the scene) or display-centric (the camera is the display plane and the viewer moves around it)
• Atlas handoff — UE's stereo atlas is handed to the OpenXR swapchain each frame, with the window's 2D UI composited into every eye (see AtlasHandoff, UICompositing)
• Editor preview — a standalone OpenXR session in the editor so you can see stereo output without running PIE (see EditorPreview)

One unified session loads the DisplayXR OpenXR runtime directly on every platform; UE's OpenXR plugin is not a dependency. Platform differences (Windows D3D12, macOS Metal, Android Vulkan) live inside the session and compositor, not in a product-level compile flag. See Architecture.md for the full picture.

Requirements

• Unreal Engine 5.7 (the plugin declares "EngineVersion": "5.7.0"; older engines will refuse to load it without a rebuild)
• DisplayXR OpenXR runtime installed (see displayxr-runtime)
• Windows, macOS, or Android target
• Visual Studio (Windows) or Xcode (macOS) matching your UE version

Installing the Plugin

1. Create a UE project (or open an existing one).
2. Make a Plugins folder next to the .uproject if one doesn't exist.
3. Clone this repository to Plugins/DisplayXR.
4. Regenerate project files and build the project from your IDE.

Scene Setup

Camera-Centric Mode

Attach a UDisplayXRCamera component under your pawn's camera. Set IpdFactor, ParallaxFactor, and InvConvergenceDistance as desired. The rig follows the camera.

Display-Centric Mode

Attach a UDisplayXRDisplay component under the camera. The camera transform is the display plane — the viewer moves around it — so there is no separate display actor to place. With several cameras on one pawn, attach one rig under each camera; only the rig on the camera being rendered from is active.

See DisplayRigSetup for the full walkthrough.

Building Your App

Use the provided script to package the plugin against a target UE version:

Scripts\PackagePlugin.bat 5.6

Output lands in Packages/DisplayXR_<version>/. Drop the packaged plugin into your shipping project's Plugins/ folder.

For per-project builds, use UAT directly (see Scripts/PackageApp.py for a reference invocation).

Documentation

In-depth docs live in Docs/DisplayXR/:

• QuickStart — install runtime → install plugin → press Play
• Architecture — one-page class hierarchy, ownership, per-frame flow
• AtlasHandoff — UE → OpenXR swapchain pipeline (zero-copy flow and the swapchain contract)
• DisplayRigSetup — pawn/camera rig configuration, input, rig modes
• UICompositing — how 2D UI (UMG/Slate) reaches both eyes at the screen plane
• InstancedStereo — vr.InstancedStereo support, the UE 5.7 sky-light capture crash and the plugin's workaround
• EditorPreview — current SceneCapture2D-based preview
• EditorPreviewNative — in-flight plan for native FDisplayXRDevice → PIE preview
• EyeTracking — xrLocateViews → Kooima → per-view projection pipeline
• MacSetup — macOS-specific setup quirks
• CompositorIntegration — prior compositor design (historical)
• ADRs — decision records for load-bearing architectural choices
• TODO — outstanding work, parity items, in-flight branches

Related Repositories

• DisplayXR/displayxr-runtime — DisplayXR OpenXR runtime. Extensions: XR_DXR_display_info, XR_DXR_win32_window_binding, XR_DXR_cocoa_window_binding.
• DisplayXR/displayxr-unity — Unity sibling plugin. Reference implementation for rig patterns and editor preview; both plugins consume XR_DXR_view_rig and vendor no view math.