Skip to content

perf(amf): minimize native output latency - #461

Merged
Nonary merged 20 commits into
Nonary:masterfrom
RamazanKara:codex/amf-low-latency-fast-path
Oct 12, 2026
Merged

Nonary merged 20 commits into
Nonary:masterfrom
RamazanKara:codex/amf-low-latency-fast-path

Conversation

@RamazanKara

@RamazanKara RamazanKara commented Aug 25, 2026 •

Copy link
Copy Markdown
Collaborator

Native AMF can delay an already-ready packet behind another capture/conversion iteration and allow application backlog to grow. This PR reduces those delays and makes native AMD the preferred automatic implementation on Windows, with one compatible FFmpeg AMF rollback after an encoder-attributable initialization failure.

Accepted AMD changes

  • Serialized direct output queries for the no-PreAnalysis path; retained output pump for contention, lookahead, retries, and draining.
  • Deliver ready native packets before/after capture waits, including bootstrap/IDR iterations, without changing capture threading or GPU handoffs.
  • Bound application backlog separately from driver queue capacity; publish accepted-input metadata before output and recycle surfaces only after AMF releases ownership.
  • Default AMD quality to speed across backend, both settings UIs, documentation, and translated defaults. Preserve explicit quality settings. This can trade compression efficiency for latency at a fixed bitrate.
  • Request the optional split-frame hint after configuration and log readbacks without claiming verified multi-engine execution. Opt-in amd_av1_tiles = 1/2/4; default 0 preserves negotiated/preset behavior.
  • Keep FFmpeg's Windows BCrypt dependency with FFmpeg so focused build consumers link correctly.

Default and initialization rollback

Automatic selection prefers amdvce_experimental (the historical native adapter name); amdvce_ffmpeg remains the explicit legacy implementation. Existing amdvce/amdvce_legacy aliases and explicit implementation choices remain strict.

For an automatic native stream, only a confirmed AMF library/component creation or encoder initialization failure can trigger rollback, after the native worker has destroyed its resources and released the runtime fence. The legacy adapter uses the same negotiated codec, depth, chroma, HDR colorspace, and resolved metadata; its capabilities are probed separately. Explicit native-only options, intra refresh, and unverified decoder reference limits block incompatible rollback. Unsupported requested formats/settings, D3D/capture/conversion errors, cancellation, fence contention, network failures, and unsafe watchdog timeouts do not trigger rollback. A failed rollback ends that stream. The stream keeps its successful legacy choice through reinitialization; later streams retry native. Existing safety quarantine for an abandoned vendor call remains intact.

Excluded from the final diff

WGC bookkeeping/mutex changes, DXGI cursor-copy/handoff changes, capture GPU priority/policy changes, host/helper power-throttling changes, and the driver-version packaging change. Current master installer/signing policy is preserved. The author had already reverted absolute capture GPU priority; the remaining excluded behavior is removed before merge.

Validation and measurement limit

All 39 local native AMF lifecycle/output/configuration/rollback cases pass with -Wall -Wextra -Werror, AddressSanitizer, and UndefinedBehaviorSanitizer. All 23 settings parity tests pass. New regression cases cover native success, encoder create/init failure, explicit native/legacy selection, incompatible and unsupported requirements, unrelated failures, terminal rollback failure, cleanup, and next-stream native retry. Additional regression checks protect producer-side cleanup after late/cancelled ready handoffs, and retain the latest converted image for static/IDR repeats when ready output skips its submission. Both regression checks fail with the earlier behavior and pass with the fixes.

Exact head 3f44775209efb3f5918d74401ee8296c1cdbb67c passed PR CI, hosted Windows build/full tests, and hosted Linux validation (168 CTest checks). No live host install, driver change, or manual release publication is part of this PR. AMD hardware stream measurements cannot be run on the review Mac; no measured latency improvement or multi-engine execution is claimed for this revision. Original AMD work and commit attribution by Ramazan Kara are retained.

Preserve explicit quality choices while aligning backend, settings UI, documentation and translated defaults. Copy fresh cursor frames directly to the shared encoder image before saving the cursor-free desktop cache. Release WGC capture ownership before diagnostics, move encoder border setup outside the capture mutex, and reject abandoned or failed GPU handoffs. Add behavioral ordering and default-consistency regression tests.
@000MDK

000MDK commented Sep 7, 2026

Copy link
Copy Markdown

ported this to 1.19.0-beta.3 codebase, (testbuild, more infos)
but i used the state available then (8b43ce8) - which is outdated already (but seems to work np) - still WIP? If you could give me some headsup when you feel done, I'd really appreciate it! Anyways, big thanks for your work on AMF!

@RamazanKara

Copy link
Copy Markdown
Collaborator Author

@000MDK runs stable for me. Under 4 - 5 ms. Couldnt find any further optimization potential or regression so from here on I would consider this running stable. I also build it against the latest 20 beta release and it runs fine.

@Nonary

Nonary commented Sep 16, 2026

Copy link
Copy Markdown
Owner

Planning to accept this later this week when my weekly usage limit reset and or double checking with GPT-6-Sol (rumored this week) which apparently will be more intelligent than Astra.... which is just insanity at this point.

Note it is in the plans to rewrite everything from scratch, the future streaming host will be named Amaterasu; however Vibeshine and Vibepollo will be maintained and developed until that project reaches 1:1 feature parity, which then both Vibeshine and Vibepollo will upgrade to Amaterasu

@WiiMoritz

Copy link
Copy Markdown

@RamazanKara Tested this beta build on an RX 6600. It feels noticeably smoother with lower latency, and the FPS looks as smooth as on the real display. No issues so far. Thanks!

@000MDK

000MDK commented Sep 19, 2026

Copy link
Copy Markdown

here is a 2.0.0-beta.3 testbuild with this PR integrated (also comes with the pre-RDNA fallback for Polaris-cards)

Can confirm, works great!

@RamazanKara

Copy link
Copy Markdown
Collaborator Author

@Nonary thank you. Great News. Looking forward to the rewrite

@000MDK Thanks for the feedback and test build

@RamazanKara

RamazanKara commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator Author

Here is a demo evidence by one of the test users. The host encoding times are on 9070xt. Pretty impressive and the lowest of all streaming servers including foundation Sunshine and funk tested. Split Frame Encoding on supported hardware is a pretty standalone feature RDT_20260924_1112564659319164066723495.jpg

@WiiMoritz

Copy link
Copy Markdown

here is a 2.0.0-beta.3 testbuild with this PR integrated (also comes with the pre-RDNA fallback for Polaris-cards)

Can confirm, works great!

I tried this version and it's not smooth as the 1.19.0-beta.3 version, I tried lots of settings and without success, I reverted back to 1.19.0-beta.3, it's perfect here.

Windows 11 can give windowless processes EcoQoS scheduling and ignore their timer-resolution requests. Both apply to the host while streaming and to the WGC helper. Request full QoS and honored timer resolution for the host during a stream, restoring system control when streaming stops, and for the helper for its lifetime. Windows 10 falls back to the execution-speed policy alone.
When the realtime GPU scheduling class is granted, request the same absolute GPU thread priority the encoder device already uses, so the capture copy that conversion waits on does not queue behind game work. The NVIDIA+HAGS high-class downgrade keeps relative priority 7.
@RamazanKara

Copy link
Copy Markdown
Collaborator Author

Update, 28 Sep: two small follow-ups on this branch, plus a separate PR for the 2.0 line.

New commits here

  • perf(windows): opt streaming processes out of power throttling: the host (while streaming) and the WGC helper request full CPU QoS and honored timer resolution. Windows 11 can otherwise treat these windowless processes as background work: EcoQoS scheduling on efficiency cores at lower clocks, and ignored timer-resolution requests that stretch 1 ms waits. System control is restored when streaming stops. Windows 10 falls back to the execution-speed policy alone.
  • perf(capture): raise capture GPU thread priority with the realtime class: when the realtime GPU scheduling class is granted, the capture device uses the same absolute GPU thread priority as the encoder device, so the copy that conversion waits on no longer queues behind game work at a lower priority. The NVIDIA + HAGS fallback to the high class keeps relative priority 7.
  • PR Windows CI passed: https://github.com/Nonary/Vibepollo/actions/runs/36473285742

Separate PR for 2.0 (vibe-test): #528

The once-per-second WGC display re-validation added in 2.0 runs on the capture thread right before the pacing sleep. #528 moves it to a background thread. It is not AMF-specific, and master doesn't have that check, so I kept it out of this PR. It is the most likely 2.0-specific cause of the smoothness difference compared with 1.19 that was reported here, but that is not confirmed yet.

Test build

2.0.0-beta.3, this PR, both follow-ups and #528: https://github.com/RamazanKara/Vibepollo/releases/tag/native-amf-beta3-amf2-20260928 (unsigned prerelease; to keep an existing controller driver, use VibepolloSetup.exe /qn INSTALL_VIRTUAL_GAMEPAD_DRIVER=0).

@WiiMoritz, if you can, please compare this build against 1.19.0-beta.3 for smoothness. It would help to know your capture method (WGC or DDX) and whether any hitch recurs about once per second.

No new latency number is claimed for these changes yet; a stream measurement is still pending.

@RamazanKara

Copy link
Copy Markdown
Collaborator Author

Follow-up, 29 Sep: I traced and fixed the remaining host latency spikes. Both fixes are in the WGC capture path, not native AMF, so they're in a separate PR against vibe-test: #530.

  • A per-stage trace showed every spike in the encode stage. The encoder was waiting behind GPU copies: the WGC helper copied each frame twice, and the host copied it a third time on a separate device before handing it to the encoder.
  • Now the encoder reads the helper's frame directly, and the helper copies once when nothing is in the way.
  • Live phone stream (AV1 HDR 1968x2184 at 120 fps, RX 7900 XT), host processing latency per 10 s window: median 3.55-3.66 → 3.21-3.28 ms, p99 4.46-4.72 → 3.37-3.74 ms, max up to 5.4 → up to 4.6 ms. Frames at least 1 ms above the median went from 2-29 → 0-2 per window.

Test build with this PR, #528 and #530: https://github.com/RamazanKara/Vibepollo/releases/tag/native-amf-beta3-amf4-20260929

@WiiMoritz

Copy link
Copy Markdown

Follow-up, 29 Sep: I traced and fixed the remaining host latency spikes. Both fixes are in the WGC capture path, not native AMF, so they're in a separate PR against vibe-test: #530.

  • A per-stage trace showed every spike in the encode stage. The encoder was waiting behind GPU copies: the WGC helper copied each frame twice, and the host copied it a third time on a separate device before handing it to the encoder.
  • Now the encoder reads the helper's frame directly, and the helper copies once when nothing is in the way.
  • Live phone stream (AV1 HDR 1968x2184 at 120 fps, RX 7900 XT), host processing latency per 10 s window: median 3.55-3.66 → 3.21-3.28 ms, p99 4.46-4.72 → 3.37-3.74 ms, max up to 5.4 → up to 4.6 ms. Frames at least 1 ms above the median went from 2-29 → 0-2 per window.

Test build with this PR, #528 and #530: https://github.com/RamazanKara/Vibepollo/releases/tag/native-amf-beta3-amf4-20260929

I tested this and it's perfect! Played for about 7 hours, not a single issue!

@Nonary

Nonary commented Oct 3, 2026

Copy link
Copy Markdown
Owner

I have concerns about the capture GPU priority change to absolute priority 30. This is hard realtime and does not yield to lower-priority GPU work, so it could cause FPS drops or worse frame times in games when the GPU is heavily loaded, without offering much benefit to capture if capture is already keeping up.

I don't currently observe capture starvation, and there isn't a measured benefit for this change yet. Please revert the capture GPU priority change and retain the existing relative +7 behavior. The other changes look fine to me.

Absolute GPU thread priority 30 is hard realtime and does not yield to
lower-priority work, so with the GPU fully loaded it could cost the game
frame rate or frame-time consistency, and no capture starvation has
been measured that it would fix. The capture device goes back to
relative priority 7 in every scheduling class.

This reverts commit 0d3cb24.

Signed-off-by: Ramazan Kara <github@r-kara.de>
…ency-fast-path

Conflicts were additive: master's HEVC GDR GOP setup now runs before the
split-frame hint, the WGC snapshot computes master's composition
timestamp after the early keyed-mutex release, and both sides' includes
and parity tests are kept.

Signed-off-by: Ramazan Kara <github@r-kara.de>
@RamazanKara

Copy link
Copy Markdown
Collaborator Author

Reverted the capture GPU priority change: the capture device is back to relative priority 7 in every scheduling class (636a321).

I also merged current master to clear the conflicts. They were additive; master's HEVC GDR GOP setup now runs before the split-frame hint. Builds with -Werror, and the native AMF policy tests and the v2 parity tests pass locally.

The Arch Linux job failed in "Initialize containers" before any build step ran, so that one looks like a runner issue and may just need a re-run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants