Implementation tracker

ReScaleFrame is a monorepo. All first-party components share this history and release version. Separate runtime/plugin DLLs do not imply separate repositories.

Repository foundation

Native CMake targets for bootstrap, orchestrator, AC7 plugin, and launcher.
Minimal C ABI for plugin metadata and executable recognition.
Rust workspace with an egui status surface.
Architecture, UE4.18 source leads, and presentation research retained in the repository.
Local Release build, C/C++ SDK compatibility, plugin contract checks, Rust formatting, and Clippy verification.
GitHub Actions workflow for native and Rust verification; individual run results are tracked in Actions.

First working target

Decrypted module capture and import map. Cross-built, Wine-tested, and run in AC7 on 6 September; that run was not MSVC-verified. Evidence.
AC7 frame/resource analysis and live view-buffer mapping. Captures identify sparse velocity and a separate half-size mask, plus a temporal-filter candidate. Evidence.
Temporal jitter enabled through the engine AA gate and measured in perspective views. Game-tested; the recorded patch build was not MSVC-verified.
Console-variable access and r.ScreenPercentage changes verified in game; the recorded build was not MSVC-verified.
Rust input model: quality levels, UE motion encoding, and all-blocker viability reporting. Unit-tested; no GPU calls. Backend guide.
Rust jitter/motion unit conversions and scale-dependent jitter sequence length. Unit-tested against recorded values; live C/C++ sign/scale agreement remains open below.
Jitter sample count follows render scale; a timer restores scale after mission loading. Game-tested through a full mission.
Streamline DLSS adapter: loading, device handover, support/render-size queries, tags, constants, and evaluation. Runs in AC7.
View layout mapped through matrix identities, including camera basis, projection, and ClipToPrevClip. Offline check: 11 captured perspective views, no failures.
D3D11 motion decode with bias/scale parameters, unwritten sentinel, and compute-state handling. Cross-built and tested under Wine/DXVK with source-encoded values and an axis flip. Game-buffer dumps matched the reference range and unwritten fraction.
AC7 view reader with matrix/size checks, main-view classification, pixel jitter, and unjittered projection. Recorded dataset: 50 recognized buffers, ten perspective views, secondary views marked. Review identifies remaining validation defects.
TemporalAAJitter located at 0x720 by comparing pre/post-patch captures, then checked against projection entries and live pixel offsets.
Orchestrator frame assembly converts plugin camera/resource data into a DLSS frame and rejects unusable combinations. Unit-tested without GPU work.
Live DLSS evaluation. Initial run: 2,176 recognized passes, 2,175 evaluations, no refusals. Mission run on 7 September: 7,917 evaluations, no refusals, 1024×576 input and 2048×1152 output. Recorded images show recovered detail and a complete scene; flight showed no obvious smearing. F7 is a debug display; reinsertion, grading, HUD, and controlled motion validation remain pending. Evidence.
Conditional composed-colour selection for the DLSS bridge. Persistent input watch and AC7 recombine rule are cross-built and synthetic-tested under Wine. Each frame falls back to the identified colour unless a matching composition is observed; no new per-frame allocations. Briefing relief presence was reported game-tested on 7 September. Reconstruction quality, hangar/flight regression, resource overwrite timing, and F6 remain open. Decision and checks.
Conditional translucent depth for the research DLSS bridge. IA/VS shadowing and immediate depth-only replay use startup allocations and restore disturbed state. Same-frame composed layer/source-depth identity gates backend selection; absent layers keep original depth. Synthetic pixel readback covers accumulation, opaque occlusion, frame reset and state restoration under Wine. Game-tested in the briefing on 7 September: 733,861 of 744,726 candidate draws replayed. Flight regression remains unverified. Decision and checks.
Separate translucency at its own resolution. 4.18 halves the layer and doubles it back on composite, so at a 50% render scale the briefing relief reached any reconstruction as a quarter-resolution image. Patching the halving out fixed the relief and the cannon tracers at once. The scale is now a four-byte immediate inside the patched instruction, aligned so it can be rewritten while the game runs, and derived from the render scale in effect so it follows a quality level instead of being fixed. Going above the scene's resolution needs the engine to size the layer's depth to match, which four one-byte patches enable by narrowing Scale < 1.f to Scale == 1.f; they are no-ops for every scale the engine produces on its own. Game-tested on 7 September: the briefing relief draws at 2048×1152 inside a 1024×576 scene with a matching depth, and reaches the reconstruction. Flight, the post-mission replay and other heavy screens are unverified. Evidence.

Reinsert the result. A debug view exists behind F7 and is game-tested: a full screen draw over the back buffer from inside the Present hook, with a rough tonemap so linear scene colour is viewable. It is what showed the reconstruction moving, which is the only way ghosting and a motion vector's sign can be judged. It is not the real path, which reinserts the reconstructed scene before the game's own composite so the grade and the interface survive. That is the remaining structural piece and the reason the picture is ungraded and has no HUD.

The whole path is built behind F6 and has been run against the game. It took three runs to get
there and each failed differently, which is worth keeping because the failures were all the
same mistake about bindings.

First run: nothing happened at all. The tail walk had taken a 2048×32 strip as the composite,
a UI bar the final draw also reads, and promoted that. Zero gates opened, so F6 did precisely
nothing. Fixed by rejecting any input less than half the height of the target it is drawn into.

Second run: the panel said no tail. The rule required the final draw to have exactly one input,
and it has seven, because D3D11 leaves shader resource slots bound until something replaces
them. `frame_tap.h` warns about this in as many words and the tail code ignored it, so the tail
had never been identified in any run. This is the same failure as the composite selection, the
interface format and the layer identity: a running game binds more than it reads.

Third run: the tail is found and reinsertion runs, and it blew up the interface. Later runs
the same day fixed the blow-up (the plan went stale on screen changes and is now restaked)
and left the picture cleaner but the interface soft, with the briefing's 3D environment
missing under F6.

Superseded, 7 September. The interface was never a target to promote. AC7 rasterizes it at a
fixed 1920x1080 and, on the briefing and hangar, draws it as world-space widget quads into its
own render-resolution `R8G8B8A8` layer, depth-tested against the scene, before compositing it
itself. Promotion cannot sharpen that; extraction can. The mechanism below is replaced by the
[representation plan](representation-plan.md): a mod-owned premultiplied UI layer, a HUD-less
scene, and a composite at present. The account of the three runs stays because the failures
were all the same mistake about bindings.

Finding the tail: the frame tap shadows the output merger and describes the draws into a
render target it is asked to watch, with extent, viewport, ordinal within the pass, and every
pixel shader input with its slot number. The loader points it at the swap chain's back buffer,
takes the composite among what that draw reads (eight-bit colour, at least half the target's
height), then watches that. An "interface target" used to be picked out of the composite's
inputs by its format; that rule is retired, since the surface it picked is a render-resolution
target of unknown role and converter targets are 1920x1080. None of this is in a capture here,
because the exported action list records render target bindings and not shader resource
bindings, which [ac7-frame-capture.md](research/ac7-frame-capture.md) states as a limitation
twice.

Doing the substitution: `runtime/graphics/scene_reinsert` promotes the composite and the
interface target to output resolution, points scene colour at the reconstruction, and hands
the tap a plan. The tap then swaps those bindings before forwarding them and scales viewports
and scissor rectangles while a promoted target is bound, so the game tonemaps and grades the
reconstruction with its own shaders, draws its own interface over it at output resolution, and
its final upscale into the back buffer becomes a copy.

Two decisions in that, both load bearing. Scene colour is substituted only after the composite
has been bound in the frame, because the scene passes read scene colour while they are still
writing it and an ungated substitution is a feedback loop rather than an upscale. And the
reconstruction now runs at that same moment rather than at Present, because it is the one
point where the scene is finished and nothing downstream has read it; evaluating at Present
would leave the scene a frame behind the interface drawn over it. The price is a mid-frame
evaluate, so the whole pipeline is saved and restored around it through
`runtime/graphics/d3d11_state`.

Cross-built with mingw-w64 and tested under Wine on DXVK, against a real device: the watch
across seven states, the substitution checked by asking the context what actually got bound,
the state save checked stage by stage after being deliberately disturbed, and the plan checked
for what it gates and what it leaves alone. What no test here can reach is whether the
substitution produces a correct picture, which needs the game's own shaders reading the game's
own constants.

Known and not fixed: bloom is still computed from the render resolution scene, so the glow
composited over the reconstruction is low resolution. Post process shaders that address texels
rather than sampling normalised will address the wrong ones, because their constants still
describe the buffer the engine believes it has. Both are visible only in a rendered result.

In-game overlay. The three pieces are now joined by a caller: loader/proxy/overlay_host loads the egui DLL, builds the renderer against the game's device, subclasses the window the swap chain presents to, and lays out and draws one frame per Present. F5 opens it. It starts as soon as the game has a device rather than at F8, so the panel can be opened to see that the backend is not running and why, and it fills its refusal line in the order the pipeline actually fails.

The bridge applies start, scale, debug view, reinsertion, dump and capture intents after
drawing on the Present thread. Quality and the master enable toggle remain unwired.

F5 game runs reached layout but failed in back-buffer view creation. The observer could
select a helper device before the presenting device existed. The host now derives its
device/context from the presenting chain, checks back-buffer ownership, and releases its
target references each frame. The worker-thread target experiment was discarded.

Cross-built with mingw-w64. A regression test creates the wrong observer device deliberately
and checks actual rendered pixels, target restoration and ResizeBuffers. The real egui DLL
also passes that test under Wine with DXVK and RenderDoc, without DLSS. The old host
reproduces the view-creation access violation in that two-device setup. AC7 validation of
the fix remains pending. See [the investigation](research/ac7-overlay-device.md).

Representation

The representation plan covers everything after the SR input chain: UI extraction into a mod-owned premultiplied layer, the DX11 to DX12 presentation bridge, and a vendor-neutral contract that DLSS, FSR and XeSS implement for super resolution and frame generation. Each milestone ends deployed to the game and is judged by the log lines named in the plan. Status classes are the tracker's: built, synthetic-tested, game-tested, device-tested.

M2, extraction. Built, game-tested, and the test changed the design: the mechanism works and the insertion point is wrong. Five runs on 7 September 2026.

fullscreen_pass: one triangle, four modes, the premultiplied composite all three frame generation SDKs specify. Verified by breaking it (e1572f4).
ui_layer: R8G8B8A8_UNORM at back-buffer extent, cleared to zero, double buffered, shareable. Its test measures under DXVK that Unreal's translucent blend leaves coverage at zero and that the patched blend accumulates, which is what the divert rests on (061f55b).
The divert primitive: retarget through the originals, viewport scaled by the fraction covered, blend alpha patched from a cache, everything restored, refusals counted with a reason (a45ce13).
Wired end to end on F3 (fd67216).
The runs. The interface reaches the screen at native resolution, 59584 draws diverted with no refusals and every written frame composited, which is the thing promotion could not do. Three faults were found and two of them were mine: the composite read the tail walk's back buffer field, which that walk releases after thirty-two frames, so the interface was diverted and never put back; and the colour transform belonged at the composite rather than at the layer.
Measured against the game rather than assumed: AC7's interface blend is already ONE / INV_SRC_ALPHA on colour and alpha with write mask 0xf, so coverage was never the problem. The quads read the scene and its blur and glow chain as inputs, at render resolution, which makes them composites rather than overlays.
The route changes. Compositing at present skips AC7's own UI composite, its glow and its grade, and on the title screen the diverted widget texture is the whole picture, so the frame comes out flat and discoloured. Promotion of AC7's interface target is the correction: the game composites it, so the colour and the glow are the game's, and the promoted target is itself the premultiplied layer frame generation wants. The argument recorded here against promotion was wrong on its premise and is marked superseded rather than deleted.
scene_promote replacing scene_reinsert. Deliberately not done before the run: interface promotion is the fallback if extraction fails, and deleting it first would remove the only thing that has produced a result.
egui drawn into the layer, so it rides generated frames later.

M4, the presentation bridge, has its riskiest piece answered as far as this machine can answer it.

shared_surface: D3D11 textures and a fence created shareable and opened on D3D12, with the export, open and signal round trip covered (510281f).
Measured, and the answer is yes. Under a prefix built by eng/wine-test-prefix.sh from an installed Proton's DXVK and vkd3d-proton, every stage passes: a D3D11 texture opens on a D3D12 device on the same adapter, and a fence signalled on one side is seen on the other. Risk 7 is retired. Under the default prefix WineD3D returns E_NOTIMPL and the fixture skips, which is why the script exists: a fixture that had only run there would have reported the bridge impossible and been wrong about the only environment that matters.
texture_dump fails under DXVK with the subprocess killed, reproducibly, while passing under WineD3D. Unexamined.
The swap chain facade, the ring, and the intercept.

M5, the vendor-neutral contract, is built and tested without hardware.

backend.h: reconstruction and generation providers, capabilities, resources with their D3D12 states and lifetimes, and the swap chain ownership every vendor takes differently.
negotiate: a pure function over what each vendor said, so the combinations nobody has the hardware to try are decided and tested here. The Streamline single-session conflict, the no-bridge refusal, the multiplier cap, and the two different causes that produce the same D3D12 route (fc54132).
Wrapping today's Streamline path behind it, the orchestrator session, and SDK ABI 2.

M6 and M8 have their vendor structure, with no implementation behind it.

rsf_backend_fsr and rsf_backend_xess: providers that exist whether or not the SDK headers do, answering NOT_COMPILED rather than failing to link, with capabilities filled from what each SDK is rather than from what this machine has. Both branches compile (fc54132, 7fa92f0).
Everything that does the work.

M3 has its foundation only.

Retained from the earlier list and folded into those milestones: the loader and orchestrator handshake (M5), plugin lifecycle (M5), the synthetic bridge (M4), XeLL and MFG on the Claw (M8, M10), frame identity and scene/HUD boundaries (M1 to M3), reinsertion into post-processing (M2), combined SR and MFG (M6 to M10). Still outside the plan:

Standalone profiles, native egui rendering/input, and handheld controls.
WSGM launch/profile integration, limiter coordination, and rendered-frame AutoTDP inputs.
Matched-condition visual, frame-time, latency, and power measurements.

AC7 motion and velocity improvements

The engine's camera transform is already available through ClipToPrevClip; camera motion is not being estimated from images. The current DLSS path decodes the sparse object-velocity buffer and asks Streamline to resolve unwritten pixels using depth and that transform. The tasks below are investigations and proposed improvements, not confirmed defects or game-tested changes.

Engine integration improvements

Use the existing plugin/runtime ownership split. These tasks extend the first working target and the motion investigations above; none is a claim of completed implementation.

Capture view data on the CPU when UE4 constructs or uploads it, with the matching render-frame and view identity. Measure the current constant-buffer staging allocation, copy, and immediate Map cost before replacing it. Reusing staging allocations is an interim improvement only; do not introduce stale camera data to avoid a synchronous readback.
Promote discovered AC7 passes into explicit, verified engine-function or shader identities, with resource checks as confirmation. Keep broad format/binding discovery as a diagnostic mode. Select resources for a known pass, view, and frame rather than the first plausible set.
Establish a frame record with render-frame ID, view ID, valid rectangles, camera/exposure state, and resource-use boundaries. Distinguish rendered frames, Present calls, and generated frames. Define immediate-consumption versus copy-before-reuse obligations; retaining a texture's allocation does not preserve its contents.
Complete reinsertion at the chosen scene reconstruction boundary before expanding into FG. Return an output-resolution result to downstream passes and preserve grading, effects, and an output-resolution HUD. Bypass only the original filtering/upscaling that SR replaces; investigate cloud-specific temporal accumulation before disabling any temporal pass. The substitution and the output-resolution HUD path are built behind F6 and described in the first working target above; what remains here is running it against the game, the bloom and texel-addressing consequences recorded there, and the cloud temporal question.
Verify the engine's exposure and pre-exposure convention, frame alignment, and backend conversion. Identifying a 1x1 texture alone is insufficient. Compare against auto-exposure during bright/dark transitions and record the source and effective values in diagnostics.
Verify texture mip selection at reduced render resolution. Apply any required bias through appropriate engine/material paths, restore it when SR is disabled, and compare fine-detail recovery against shimmer. Avoid indiscriminate global bias or sharpening as a substitute.
Sequence the integration work around CPU-side view capture, explicit pass/frame identification, correct reinsertion, and exposure/mip tuning before additional backends and FG. Build the egui inspection and capture tools alongside these steps so they can validate each change.

egui development and validation workflow

Use the prepared egui overlay as the main in-game development and validation interface as well as the eventual player settings UI. Prioritize effective settings, buffer inspection, frame-time plots, and capture of the next N frames; prove the capture format before building offline sequence replay.

Repository review follow-up

Open defects from the 7 September review. These are not fixed by the documentation rewrite.

The validation plan defines acceptance. Keep build, synthetic, game, and target-device results separate.

Navigation Menu

Uh oh!

Implementation tracker

Repository foundation

First working target

Representation

AC7 motion and velocity improvements

Engine integration improvements

egui development and validation workflow

Repository review follow-up

Uh oh!

FilesExpand file tree

implementation.md

Latest commit

History

implementation.md

File metadata and controls

Implementation tracker

Repository foundation

First working target

Representation

AC7 motion and velocity improvements

Engine integration improvements

egui development and validation workflow

Repository review follow-up