Skip to content

DLSS-NR: multipass, up to five chained model passes - #23

Open
y4my4my4m wants to merge 3 commits into
Dagherbou:dlss-neural-renderingfrom
y4my4my4m:nr-multipass-for-upstream
Open

DLSS-NR: multipass, up to five chained model passes#23
y4my4my4m wants to merge 3 commits into
Dagherbou:dlss-neural-renderingfrom
y4my4my4m:nr-multipass-for-upstream

Conversation

@y4my4my4m

Copy link
Copy Markdown

Runs the model more than once over the same frame, so the effect can be pushed past what a single evaluate gives. Up to five passes. Off by default: Passes=1 takes the same path the code takes today.

This branch is based on 21274132, before the last 15 commits on dlss-neural-rendering. Supersampling and the reversible proxy rewrote much of the same function, so this will need rebasing. I have deliberately not attempted that — see the note at the end.

Why it was removed, and what changed

0f79b9dd pulled multi-pass out because "per-pass features created and evaluated on the same command list in the same frame, which the code's own comment identifies as the crash that 'died on a creation frame'", and left a prescription: "a feature per pass built a frame ahead the way the main feature is, and a residual carry rather than a full re-feed".

That is what this does.

Anchored chain

work[2] = { output, passScratch }. Pass p writes work[p & 1] and reads what p-1 wrote, so no pass reads and writes one surface and no copy sits between them. modelInput is never written, so the resolve's edit = model - proxy is the whole chain's rather than the last pass's — the anchor the removal commit asked for instead of a full re-feed.

Every pass is handed the frame's own guides and the frame's own motion scale. Each feature sees one frame per frame; telling a later pass nothing moved would be a lie about a full frame of camera travel.

Built a frame ahead

Each pass feature is created on its own frame and first evaluated on the next, exactly as the main feature is. A build frame is a full no-op for the pass — no encode, no evaluate, no resolve, target restored — so nothing is created and evaluated on one list.

Builds are spaced by kSettleFrames (30), so 1 to 5 ramps over roughly 120 frames with four dropped frames scattered through it rather than four consecutive. The chain recomputes its length from the live pointers each frame, so a partially built set runs only the passes it holds and the effect strengthens in steps.

"A frame ahead" is checked against presents, not against g_frames: that counter counts Dispatch calls, and EvaluateAfterUpscale is called per upscaler evaluate with no per-present gate, so a title with two non-FG evaluates on one open list would have rebuilt the exact hazard 0f79b9dd reverted. It uses State::frameCount where the swapchain is wrapped and falls back to a command-list pointer test where it is not.

Failure and cost

A refusal past pass 0 costs that pass, not the session: the feature is parked, passCeiling drops so it is not attempted again, and the resolve runs against the last good surface. Pass 0 stays fatal as today.

Before each build the free video memory is checked against what the previous create actually cost, measured rather than guessed. Not enough room defers the build rather than lowering the ceiling. passScratch allocation failure latches through passCeiling so a starved system does not retry a committed-resource allocation every frame for the rest of the session.

DlssNrUseProxy forces the count to 1 where it is computed, so the proxy path allocates no scratch, builds no extra features and runs no chain.

Default path

At Passes=1: no scratch allocated, the build ramp is gated on it, passes is forced to 1, one evaluate with identical arguments, and every state transition collapses to the same Barrier calls in the same order (Barrier skips from == to).

Also here

6c6a9d10 is a prerequisite — multipass does not apply without it, and it stands on its own:

  • The creation frame ran outside ScopedNrStateEnvelope. It returns before the envelope is constructed, so NGX's descriptor heaps, root signature and pipeline were left bound on the game's list on the one frame the comment above that return calls "the dice-roll that hung the GPU".
  • ReadableGuide transitioned depth and motion away from NON_PIXEL_SHADER_RESOURCE unconditionally — the same assumption eb86c532 removed for the output. A typed guide took the other branch and received no barrier at all before being handed to the model. Both now read their arrival state from DepthResourceBarrier / MVResourceBarrier, and the exposure texture from ExposureResourceBarrier, the keys every upscaler here already honours. All no-ops when unset.

5a34e8d1 documents that the highlight guard is applied once to the finished composition while Passes compounds the ratio it bounds, so a value near the pass count keeps the headroom per pass roughly constant. Found empirically — 1 pass at 1.0, 2 at 2.0, 3 at 3.0 — and it matches where the clamp sits in the shader.

What I did not do

Rebasing onto supersampling. The conflicts are additive rather than contradictory, but the open questions are design ones: whether the down-leg runs once after the chain or per pass, and whether passScratch should be allocated at the super-native size. Guessing wrong there gives a picture that looks plausible and is subtly wrong, and you know that architecture. Happy to do the rebase if you say which way those should go.

Tested on an RTX 4090 under Proton with Cyberpunk 2077, up to 3 passes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant