Skip to content

PRD-539: try Unreal TSR history levers and benchmark against Unreal - #606

Merged
jonit-dev merged 48 commits into
developfrom
feat/prd-539-tsr-history
Nov 4, 2026
Merged

jonit-dev merged 48 commits into
developfrom
feat/prd-539-tsr-history

Conversation

@jonit-dev

@jonit-dev jonit-dev commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

What is this PR about?

PRD-539: a lower internal raster must reconstruct into a stable full-resolution frame. This PR tries Unreal TSR's untried levers as arms of the unchanged quality gate, and benchmarks the same scene in Unreal 5.8.3 and ThreeNative side by side.

Why does it matter?

Five earlier reconstructors failed the gate. Unreal's TSR attacks the same camera-drift loss with a 200% history and a velocity clamp on the history weight. If one of these arms clears the gate, low-resolution rendering can buy frame time with no visible loss.

Done

  • historyScale (1 or 2) and velocityWeightClamp options on the generated temporal provider, ported from UE 5.8.3 TemporalSuperResolution.cpp:3219-3224 and TSRUpdateHistory.usf:1262-1276

  • Gate arms history200, velocity-clamp, history200-clamp (and -open / quality- variants), reported under leverArms; thresholds unchanged

  • Red-green unit test: history target at 2× display

  • Display-sized history resolve (4×4 Mitchell, HDR weight, sample clamp), ported from UE 5.8.3 TSRResolveHistory.usf:312-357; red-green: presented raster 3840×2160 → 1920×1080

  • Provider default = Unreal Epic tier (200% history + velocity clamp, BaseScalability.ini:89); { historyScale: 1, velocityWeightClamp: false } is the named override and the history100 gate arm. Gate run 7 passes every full-resolution check (edge .04387 < .04616); judge IMPROVEMENT

  • Engine fix: lamp shadow-map size reads the unjittered camera (red-green lamp_size_ignores_temporal_jitter)

  • Low-input bar = Unreal TSR at the same input on the same fence scene (owner option 1); Unreal reveal measured with an occluder run. Gate run 12 exits 0, 15/15 checks

  • Unreal's view texture mip bias under temporal upscaling (lever 7, SceneVisibility.cpp:5337-5351): judge IMPROVEMENT against no bias, NEUTRAL against full resolution; MSAA path pixel-identical

  • Scripted three-scale scaler route conformance row temporal-aa-scaler-route

  • Shooter kit, GPU clock locked at 1500 MHz: TRAA 0.667 against shipped 4x MSAA, GPU p50 -40 to -46% web, -47% desktop native; p95 lower on both (look: judge 3 REGRESSION, moderate softness)

  • Same scene against Unreal 5.8.3 (gi:cornell, locked clock): at 67% ThreeNative 4.78-6.06 ms GPU p50 against Unreal 5.94-6.09

  • leanpi DeepSeek v4.1 flash max review of lever 7: node texture() bias fix 8223caa (judge NEUTRAL, kept as Unreal parity)

Not done (deferred, recorded in the PRD)

  • Lever 4 (flicker-aware rejection) and lever 6 alone: deferred with revisit conditions in the PRD Decisions
  • TRAA as the high tier default: not in this PRD's scope

PRD-539 checklist

Phase 1

  • One-variable control: history 200% vs 100% (history resampling is the .010 drift loss)
  • Velocity-clamped history weight arm (history200-clamp: moving .04566 / static .03850)
  • Thin fences, foliage, sub-pixel edges stay within thresholds (gate run 12, low-input edge .0732 < Unreal bar .0832)
  • Reveal recovery (ThreeNative recovers in 2 frames; Unreal bar .082)

Phase 2

  • GPU-bound game: sub-1.0 + reconstruction lowers GPU p95 — p50 -40% to -47%, p95 lower, web + desktop native, locked clock; look cost equals Unreal TSR at the same input (judge 5 PASS); TRAA stays opt-in
  • Textured same-scene look comparison: ThreeNative TRAA 67% loses no more detail than Unreal TSR 67% — gi:neon-corridor, sameness PASS, judge 5 EQUAL at Unreal's 8x anisotropy (judge 4 FAIL at 1x; default filed as PRD-789)
  • Automatic resolution moves between three scales (conformance row, allocations back to baseline)

Acceptance

  • 3/3 ticked; PRD moved to docs/PRDs/done/ (33c5246)

…al TSR levers 1-2)

historyScale stores resolve and history at 2x the display raster
(r.TSR.History.ScreenPercentage=200); velocityWeightClamp ports TSR's
sample-count clamp under motion. Both are opt-in arms of the unchanged gate.
…reader, static-camera gate control (PRD-539)
…t-refine arm

A full-resolution input under a 2x history keeps its owned point sample, so
history scale is the only variable. The tsr arm adds Unreal TSR's refine
rule: the current frame enters each history pixel through the history-pixel
kernel and weighs by its nearest jittered sample's alignment.
A pixel's footprint does not move with the TRAA subpixel offset. The
jittered frustum let a receiver on the view's edge enter and leave, so a
lamp whose footprint sat on a mip boundary flipped its map size every
frame and reallocated it. On the Wasm engine the released maps waited for
JS GC: the PRD-539 cornell motion run at screenPercentage=50 grew VRAM
about 65 MB per step and lost the device (VK_ERROR_OUT_OF_DEVICE_MEMORY)
before frame 0.

Red-green: lamp_size_ignores_temporal_jitter read 2 sizes over 16 still
TRAA frames before, 1 after. lamp_atlas and 24 lamp/shadow/light/TRAA
render-database cases pass.
@jonit-dev jonit-dev added the prd:25% PRD phase 1 verified label Nov 3, 2026
@jonit-dev

Copy link
Copy Markdown
Contributor Author

Same scene, Unreal 5.8.3 against ThreeNative: TSR/TRAA at 50% screen percentage

Scene: gi:cornell pan run, 1280×720, RTX 2080 (shared GPU, 12–72% busy from other work). Commit b6a0d59e6.

Unreal vs ThreeNative sheet

Columns: UNREAL 5.8.3 100% | UNREAL TSR 50% | THREENATIVE 100% | THREENATIVE 50%. Rows: frames 10, 25, 40, 55. Bottom row: 3× crop at frame 40 (box edge against the green wall).

Quality, moving frames 0–60, each 50% arm against its own 100%

Arm Mean abs error Edge error (top-10% Sobel) Flicker
Unreal TSR 50% .00253 .00583 .00272
ThreeNative TRAA 50% .00119 .00566 .00143

Distance from ThreeNative to Unreal is the same at both scales: mean abs error .0277 at 100%, .0274 at 50%.

GPU cost (2 runs per arm, p50)

Arm Internal GPU p50 (ms) Reconstruction p50 (ms)
Unreal TSR 100% 1280×720 6.83 / 6.78 1.81 / 1.82
Unreal TSR 50% 640×360 4.30 / 4.64 0.90 / 0.93
ThreeNative 100% (corner pose) 1280×720 7.65–7.70 / 7.63–7.69 unverified
ThreeNative 50% (corner pose) 640×360 4.60–11.18 / 3.48–3.55 unverified

⚠️ Unreal times the pan run with its CSV profiler. ThreeNative times the still poses with TN_FRAME_BUDGET. One ThreeNative 50% run was contended (4.6–11.2 ms). These are shared-GPU numbers, not an exclusive-slot verdict.

Engine fix needed for the 50% capture (fix(engine): size lamp shadow maps from the unjittered camera)

  • Before: TRAA jitter flipped the cornell lamp map between two mips every frame. The Wasm engine reallocated it each frame and lost the device (VK_ERROR_OUT_OF_DEVICE_MEMORY) before frame 0.
  • Red-green: lamp_size_ignores_temporal_jitter 2 sizes → 1 size over 16 still TRAA frames. 24 lamp/shadow/light/TRAA cases pass.
  • Visual judge (fresh read-only subagent): NEUTRAL on frames 0, 10, 25, 40, 55, 60. Mean abs error .0002 over 180 frames. Sheet: before/after

Gate run 3: Unreal's alignment refine (tsr arm) is rejected: moving .0596 / static .0498, both worse than history200-clamp (.0457 / .0385). The quality gate still exits 1. Next: lever 3 (spatial AA of rejected input), lever 4 (flicker-aware rejection).

Raw: sheet · metrics · cross-engine

…ted refine and spatial-AA arms

Gate run 4: Unreal TSR's spatial anti-aliasing of rejected input leaves
edge error unchanged and raises reveal stale (full .0003 -> .0024, low
input .0177 -> .0204). Gate run 3 already rejected the alignment refine.
Both arms leave the provider and the gate; their numbers stay in the PRD.
history200-clamp stays opt-in: as a default it presents a 2x raster.
Port of Unreal TSR's history resolve (UE 5.8.3 TSRResolveHistory.usf:312-357): a
4x4 Mitchell (B=C=1/3) downsample with the 1/(R+2G+B+4) HDR weight, clamped to
the samples' range. At historyScale 2 the chain now reads a display-sized
raster instead of the 2x history, so later post stages stop paying 4x pixels.
Red: presented 3840x2160 for a 1920x1080 display. Green: 1920x1080.
WGSL has no infinity literal; gate run 5 failed the history200 arm with
'expected , for function call' at vec4<f32>(Infinity.0, ...).
With no weighting named, the provider keeps history at 200% of the display
(UE 5.8.3 BaseScalability.ini:89) with the velocity-clamped weight, and the
chain reads the display-sized Mitchell resolve. The raw4 path keeps 100%.
{ historyScale: 1, velocityWeightClamp: false } is the named override and the
new history100 gate arm. Gate run 6 measured the two-lever arm at edge .04387
against the .04616 bar (the 100% default read .0486 reference).

The resolve moves to temporalHistoryResolve.ts and the 100% reset seed into
frames.draw, so every starter render file stays under 200 lines.
Red: default history 1920 wide at a 1920 display. Green: 3840.
@jonit-dev

Copy link
Copy Markdown
Contributor Author

Temporal AA now defaults to Unreal's Epic-tier TSR history

Columns: BEFORE (100% history, 2 runs) · AFTER (200% history + velocity clamp, shown through the display-sized Mitchell resolve, 2 runs) · REFERENCE (supersampled). Commit ae4824144, hardware WebGPU nvidia/turing.

Judge (Opus, read-only): full resolution IMPROVEMENT: the fence bars and the rod are cleaner, with no ghosting, halo or colour shift. Low input NEUTRAL.

Gate run 7 100% history (old default) 200% + clamp (new default) Bar
Edge error .05648 .04387 < .04616 ✅
Moving edge .02725 .02187 —
Residual instability .0310 .0314 < .0450 ✅
Reveal stale (max) .0005 .0003 ≤ .01 ✅
Overshoot fraction .00015 .00094 no-AA reference .0171
Low-input edge .07794 .07321 < .04532 ❌
Low-input reveal .0177 .0175 ≤ .01 ❌
  • ✅ The resolve is a port of TSRResolveHistory.usf:312-357. The chain reads a 1920×1080 raster at a 1920×1080 display, where it read 3840×2160 before the resolve.
  • ✅ verify-temporal-aa.ts exits 0. 1960 unit tests pass, and typecheck passes.
  • ⚠️ The provider's own GPU cost for the 2× history is unverified. The low-input edge and reveal checks are still open.

motion
low input

Port of Unreal TSR's moire measurement (UE 5.8.3 TSRShadingAnalysis.ush:
225-229,263-271,381-455,477-490,586-593): a ping-pong state of luma gradient,
total variation and flicker count, decayed by 0.95 on still pixels, widens the
clip box by 3x the moire error. Opt in with flickerRejection: true; gate arms
flicker, flicker-open and quality-flicker. No 5x5 flicker dilation yet.
Red-green: the kernel holds the moire ramp only with the lever on, and the
update pass compiles to WGSL with no unparseable literal.
Unreal accumulates max3x3(max3x3()) of the flicker and its variation
(UE 5.8.3 TSRShadingAnalysis.ush:441-455). Without it a sub-texel bar that moves
between neighbours across jitter phases flickered too rarely at any one pixel:
gate run 8 moved low-input edge .07055 -> .07048 only. The update is now a
detect pass and a dilate-and-accumulate pass.
…(opt-in)

Static-camera gates showed the per-pixel clip to the low-input 3x3 box erases
sub-texel detail that unclipped history keeps. Unreal compares input and
history after a 3x3 blur and lerps the clamped history back to the unclamped
one by Min3x3(Median3x3(rejection)), 0 on disocclusion (UE 5.8.3
TSRShadingAnalysis.ush:25,509-525,563-566,589-636, TSRRejectShading.usf:863,
TSRUpdateHistory.usf:1256-1257). Six input-raster passes before the resolve.
Gate arms release, release-open, quality-release. Red-green: the kernel reads
the release only with the lever on; all six passes compile to WGSL.
…e clamp-release rejection

UE 5.8.3: TSRShadingAnalysis.ush:569-613. The gate's solo flicker and release
arms become one flicker-release arm; their gate-10 numbers stay in the PRD.
…n scene for Unreal

TN_TEMPORAL_PROBE=arm,arm captures only those arms and prints their edge error (diagnostic).
… scenes take no motion run), bars share one region
@jonit-dev

Copy link
Copy Markdown
Contributor Author

PRD-539: Unreal same-scene benchmark at a locked GPU clock, leanpi review fix, judge 3

Same scene against Unreal 5.8.3 (gi:cornell, 1280x720, GPU clock locked at 1500 MHz for both engines, 2 runs per arm, GPU ms):

Engine, input GPU p50 GPU p95
Unreal TSR 100% (pan) 8.22-8.41 22.9-23.6
Unreal TSR 67% (pan) 5.94-6.09 (-28%) 13.2-14.8 (-40%)
ThreeNative TRAA 100% (cornell / corner poses) 7.35-7.55 / 9.96-10.00 7.57-7.89 / 10.0-10.1
ThreeNative TRAA 67% (cornell / corner poses) 4.78-4.99 (-36%) / 4.87-6.06 (-40%) 4.83-7.47 / 6.12-8.06

One method difference: Unreal times the pan motion, ThreeNative the still poses.

leanpi review (opencode-go/deepseek-v4.1-flash, thinking max on all 39 turns, read-only) of the lever 7 commit found that a node material's texture() sampled at the implicit LOD. Fixed in 8223caa, red-green material_test view_mip_bias_samples; native engine ctest 527 of 530 (the 3 reds fail at HEAD sources too or are tn-known-red).

Judge 3 (fresh read-only Opus, criteria first, shooter visual-pose, 2 runs per arm). Columns: A SHIPPED MSAA 1.0 | B TRAA 0.667 BEFORE | C TRAA 0.667 AFTER.

  • C against B: NEUTRAL, no visible change and no artifact. The criteria asked for IMPROVEMENT, so the fix misses its visual target. It stays as Unreal parity with no visual claim.
  • C against A: REGRESSION, moderate softness (stucco, floor grain, awning wood grain, shutter texture), no ghosting, trails or sparkle. Judge 2 had rated this NEUTRAL, slight. High-frequency share of A: B .51-.58, C .50-.56.
  • Decision (AFK, logged in the PRD): TRAA 0.667 is a trade, GPU -40% to -47% for moderate softness, so it stays opt-in as the PRD Outcome says.

Sheets: still run 1 · still run 2 · strafe run 1 · strafe run 2 · floor still · wall still · floor strafe · wall strafe · criteria

…nst shipped MSAA

Moves the PRD out of done/. What regressed: the box-97 visual verdict. Judge 2 rated
TRAA 0.667 NEUTRAL (slight softness); judge 3 (commit 9b7ddb4, sheets ee4d4ed on
prd-539-screenshots) rates it a REGRESSION with moderate softness, and a perf win counts
only when the judge passes. Box 97 and acceptance 3 are unticked with their numbers kept;
a textured same-scene comparison against Unreal TSR at 67% is the new open box.
@jonit-dev jonit-dev added prd:50% PRD half the phases verified and removed prd:100% — ready PRD complete, ready to merge labels Nov 3, 2026
@jonit-dev
jonit-dev marked this pull request as draft November 3, 2026 18:54
…tured same scene; PRD-539 done

gi:neon-corridor brick wall, both engines at 100% and 67%, sameness PASS (16 objects, 1 px).
Judge 4 (anisotropy 1): FAIL, far-wall loss larger. Judge 5 (anisotropy 8, Unreal Epic r.MaxAnisotropy): PASS, EQUAL.
Adds the calibration harness knob anisotropy=<n>. Files PRD-789 for the engine default anisotropy (deferred).
@jonit-dev

Copy link
Copy Markdown
Contributor Author

Textured same scene: TRAA 67% loses detail equal to Unreal TSR 67% (judge 5 PASS)

Commit 33c5246e3. Scene gi:neon-corridor, pose textured (brick wall), 1280x720, settled stills, 2 runs per arm (runs agree to 0.001). Columns: UNREAL TSR 100% | UNREAL TSR 67% | THREENATIVE TRAA 100% | THREENATIVE TRAA 67%.

  • ✅ Sameness: pnpm unreal-look:compare --sameness --scene gi:neon-corridor PASS, 16 objects, farthest edge 1 px, Wall_Brick IoU 1.0000.
  • Detail kept at 67% against own 100% (mean |Laplacian| on Wall_Brick):
Arm Whole wall Mid Far
Unreal TSR .875 .879 .778
TN TRAA, anisotropy 1 .925 .928 .791
TN TRAA, anisotropy 8 .903 .881 .772
  • ❌ Judge 4 (fresh Opus, anisotropy 1, three default): FAIL by a small margin, far-wall mortar contrast -20% against Unreal -12%. Cause: Unreal filters the brick at 8x anisotropy (Epic r.MaxAnisotropy), so the far wall was not the same scene.
  • ✅ Judge 5 (fresh Opus, anisotropy 8 via harness knob anisotropy=8): PASS, Unreal slight, ThreeNative slight, EQUAL, no sparkle, moire or over-sharpening.
  • ⚠️ Deferred: the engine default anisotropy (three's 1 against Unreal Epic 8) is filed as PRD-789.

Judge 5 sheets: far wall run 1 · run 2 · mid wall · full frames · criteria
Judge 4 sheets: far wall · full + near

far wall, anisotropy 8

@jonit-dev jonit-dev added prd:100% — ready PRD complete, ready to merge and removed prd:50% PRD half the phases verified labels Nov 3, 2026
@jonit-dev
jonit-dev marked this pull request as ready for review November 3, 2026 19:20
jonit-dev and others added 5 commits November 3, 2026 12:32
…istory

# Conflicts:
#	README.md
#	docs/PRDs/Unreal/performance/PRD-539-low-resolution-temporal-reconstruction-quality-and-cost.md
#	docs/PRDs/Unreal/research/port-history.json
#	docs/PRDs/Unreal/research/unreal-port-progress.svg
#	docs/PRDs/Unreal/research/unreal-port-status.svg
#	docs/PRDs/Unreal/research/unreal-visuals-and-performance-feature-map.html
#	packages/runtime-native/tests/native-engine/render_database_test.cpp
… max (VERDICT OK), rounding fixed, shooter-in-Unreal decision
…coverage record for PRD-539 sources

overlay_over_frame read the presented frame within 4000 polls; on a loaded host (load 12-19) the readback
landed after that, so presented stayed empty (1 of 3 and 1 of 6 runs red). A 10 s bound plus CHECK(read):
12 of 12 green under the same load. The coverage record digest was stale after this branch's native
changes (CI budgets red on run 41704323408).
@jonit-dev
jonit-dev enabled auto-merge November 3, 2026 20:46
@jonit-dev
jonit-dev added this pull request to the merge queue Nov 3, 2026
@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

The merge queue stopped on native-platforms / Android emulator visual parity (failure).

Fix, push, re-enqueue.

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 3, 2026
@jonit-dev
jonit-dev added this pull request to the merge queue Nov 3, 2026
jonit-dev added a commit that referenced this pull request Nov 3, 2026
No box is ticked: every open criterion still needs a run this branch cannot produce. Each box
now carries its measured numbers and the one run that would close it.

PRD-628
- line 31 (payload reuse): run 41688813459 built `v8-android-6d7f399f…` in 1 h 45 min
  (18:18:44Z -> 20:03:42Z, artifact 13268070999, 27,451,905 bytes) after `No published …tgz`,
  `v8.artifact skipped` and `Cache not found`. Run 41718807432 restored it in 1.4 s and skipped
  the build. Both are `workflow_dispatch` on the same ref, so the pull-request scope stays
  unproven; the artifact read is verified read-only (v8-artifact-pick.mjs -> 41688813459).
- line 32 (cancel keeps progress): run 41670763005 cancelled at `[1781/3519]`, saved
  `android-v8-source-…-41670763005-1`; run 41688813459 restored it and planned 1724 edges, not
  3519. A dispatch run, so the `pull_request` arm stays unproven.
- line 36 (ccache >= 80%): `test-native` 229/1076 = 21.28% (nothing restored), against 256/256,
  280/280 and 312/312 = 100% in native-engine and starter-linux on the same board.
- line 37 (cache table): second full warm table from 41688813459 attempt 1, 70 job logs and 181
  cache lines; pnpm store, Playwright, Cargo, emsdk and third-party hit, Gradle misses.
- line 55 (budget): 70 entries / 8.63 GiB at 21:19:40Z -> 86 entries / 9.28 GiB at 22:07:51Z,
  zero evictions; no full board completed in the window.

PRD-480 AC-2 and its phase box: per-job wait p90 26.3 min (41718838521, merge group #606),
7.8 min (41718807432, dispatch #605) and 11.4 min (41569108715); median 11.4 min against 10,
not met. 11 jobs per board on `tn-` runners.
@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

The merge queue stopped on integration / Verify required integration receipts (failure).

Fix, push, re-enqueue.

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 3, 2026
@jonit-dev
jonit-dev added this pull request to the merge queue Nov 3, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 4, 2026
@jonit-dev
jonit-dev added this pull request to the merge queue Nov 4, 2026
Merged via the queue into develop with commit 5e030ca Nov 4, 2026
37 checks passed
jonit-dev added a commit that referenced this pull request Nov 4, 2026
develop's copy of each of the seven conflicted files is byte-identical to the
#606 queue commit 6103e42 that 605-sync round 1 resolved in 04f56b60c, #605
has not touched them since, and neither side changed packages/runtime-native or
the templates since then, so those seven files are taken from 04f56b60c as is.
Round 1 built render_database_test (123/123) and regenerated the coverage
report (82.15%, every floor met) on that same content.

Checks on this merge: unreal:status up to date, check:docs 3670 links,
scaffold.spec 78/78, check-native-coverage + pre-commit-hook specs 8/8,
pnpm typecheck exit 0.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

prd:100% — ready PRD complete, ready to merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant