No test suite: nothing in CI or locally ever opens a scene or starts the provider #317

State today

This repo has no tests.

• No Tests/ directory
• No test assembly definition — the only four .asmdefs are com.displayxr.unity{,.editor} and the two URP ones
• No file anywhere matching *test*
• CI is build-native.yml, lint.yml, no-vendored-math.yml — a native compile, a linter, and a drift guard

Nothing in CI, and nothing runnable locally, ever opens a scene, starts the provider, or asserts on what it produces. Every regression in this plugin is currently found by a human pressing Play.

Why that has bitten us specifically

The recently-closed issue set is dominated by failures that a scene-level harness catches and a compile check cannot — samples that ship no .unity scene and build their content in Start() (#303, #308, #312), a post-build processor clobbering machine state (#306), overlay focus/keyboard state machines (#304), transparent-app visibility that regressed twice and shipped both times because a cloaked window passes every "did it start" check (#295, #296), requests silently dropped before IsRunning (#293).

That last class is the important one. A weaving plugin has failure modes where the process is healthy, the logs are clean, and the only evidence is what is or isn't on screen. Those are exactly the regressions that reach a release, and exactly the ones no unit test framing will catch.

Proposal

A behaviour-test suite: a set of fixture scenes, each exercising one rig configuration or provider path, driven through Play Mode with assertions on observable state.

Candidate first coverage, chosen to match where the bugs actually are:

• Both rig modes reach a running session and produce two distinct per-eye projection matrices
• BiRP / URP / HDRP each render through their declared path (MultiPass vs SPI) rather than silently falling back
• Multi-camera scenes: DisplayXRRigManager hands off to the expected active rig on switch
• Scene load / unload / reload — including the case that has broken repeatedly, transitioning to and from a scene with no DisplayXR rig
• A transparent-overlay app is actually visible after startup (the visibility matrix, not just "the process is alive")
• Samples open and are non-empty on load

Then wire it into CI on a Windows runner so it gates PRs alongside build-native.

Prior art

The legacy Leia Unity plugin ran precisely this and it earned its keep — a large share of that project's bug backlog is named after the test that caught the regression (2DSceneWithLeiaDisplay, BacklightOFFScene, MultiCamerasCompositing, MultipleCamerasSwitch, face-tracking animations). The harness design, the fixture-scene conventions, and the list of what turned out to be worth asserting are all transferable, and are a better starting point than a blank Tests/ folder.

Open question

How much can run headless on a CI runner versus needing a real display and a real weaver. Probably a split: a headless tier gating every PR, and a hardware tier run on a box with a panel. Worth deciding before writing fixtures, since it shapes what the assertions can look at.