Repository navigation
perf(wgc): encode the helper frame directly and publish it with one copy - #347
Merged
Merged
Conversation
With WGC capture the helper already publishes every frame into a shared D3D11 texture, but the host copied it into a pooled image before the encoder converted it, costing a full-frame GPU copy per frame. Add wgc_direct_encoder_input (default on, latched per capture session). When it is active, each pool image aliases the helper's shared texture through its own duplicated NT handle, opened once per image on the encoder device. The capture thread only claims the newest frame id; the encoder takes the shared keyed mutex on its own device, reads the frame record under it, and converts the texture in place. Switching the option off restores the copy path. The encoder pairs the actual converted pixels with their frame id and timestamp: the helper may publish a newer frame between the claim and the lock, so the conversion reports the id and timestamp it really read through converted_frame_t, and the encode loops use those instead of the claimed ones. A per-encoder-device cursor skips a frame that was already converted, so the same pixels are never converted or encoded twice; the AMF device cancels its reserved input surface when it skips. A restarted helper bumps the frame record generation, which restarts the ids. Sources that need a stable image of their own keep the copy path: constant frame rate capture, which re-sends its cached image on a timeout that the cursor would drop as a repeat, and PyroWave, which pins and reads the capture image's own texture and keyed mutex. Constant frame rate is latched with the input mode, so changing the capture option cannot leave a direct pool with nothing to re-send. PyroWave runs on the shared capture, so any live PyroWave consumer is part of the shared capture's configuration: a PyroWave client joining a capture that already latched direct mode restarts it onto the copy path through the existing demand check (a PyroWave client leaving does not restart it). Software encoding and DXGI capture are unaffected. A direct image whose frame record cannot be read is skipped like a repeat, without touching the frame cursor, and logged at most once a second. That only happens if the helper died mid-publish, and the IPC session's broken-pipe handling restarts capture in that case. The timestamp choice and the skip of an already-converted frame are one policy shared by both encode loops, and only frames actually converted are counted as such. Signed-off-by: Ramazan Kara <github@r-kara.de>
…ng up The helper copied every WGC frame into a scratch texture and then, on its delivery thread, from the scratch texture into the shared texture the host reads, so each frame cost two GPU copies even when nothing was queued. When no older frame is queued or mid-delivery, the capture callback now copies the WGC frame straight into the shared texture, as long as the helper's D3D context is free and the keyed mutex can be acquired without waiting. Otherwise the frame takes the scratch handoff as before, so a newer frame is never published ahead of an older one. The helper counts direct publications in its periodic diagnostics. The scratch fallback no longer holds the D3D context while it waits for the host. The delivery thread used to hold the context lock for up to 200 ms waiting on the shared keyed mutex, and the capture callback needs that lock for its scratch copy, so encoder backpressure could stall FrameArrived. It now waits for the keyed mutex alone and then takes the context only if it is free; if the callback is mid-copy, it hands the texture back and waits for the context instead. Neither side holds one lock while waiting on the other, and the frame stays in the delivering state throughout, which keeps the callback from publishing a newer frame ahead of it. Signed-off-by: Ramazan Kara <github@r-kara.de>
RamazanKara
added a commit
to RamazanKara/Vibepollo
that referenced
this pull request
Oct 23, 2026
With WGC capture the helper already publishes every frame into a shared D3D11 texture, but the host copied it into a pooled image before the encoder converted it, costing a full-frame GPU copy per frame. Add wgc_direct_encoder_input (default on, latched per capture session). When it is active, each pool image aliases the helper's shared texture through its own duplicated NT handle, opened once per image on the encoder device. The capture thread only claims the newest frame id; the encoder takes the shared keyed mutex on its own device, reads the frame record under it, and converts the texture in place. Switching the option off restores the copy path. The encoder pairs the actual converted pixels with their frame id and timestamp: the helper may publish a newer frame between the claim and the lock, so the conversion reports the id and timestamp it really read through converted_frame_t, and the encode loops use those instead of the claimed ones. A per-encoder-device cursor skips a frame that was already converted, so the same pixels are never converted or encoded twice; the AMF device cancels its reserved input surface when it skips. A restarted helper bumps the frame record generation, which restarts the ids. Sources that need a stable image of their own keep the copy path: constant frame rate capture, which re-sends its cached image on a timeout that the cursor would drop as a repeat, and PyroWave, which pins and reads the capture image's own texture and keyed mutex. Constant frame rate is latched with the input mode, so changing the capture option cannot leave a direct pool with nothing to re-send. PyroWave runs on the shared capture, so any live PyroWave consumer is part of the shared capture's configuration: a PyroWave client joining a capture that already latched direct mode restarts it onto the copy path through the existing demand check (a PyroWave client leaving does not restart it). Software encoding and DXGI capture are unaffected. A direct image whose frame record cannot be read is skipped like a repeat, without touching the frame cursor, and logged at most once a second. That only happens if the helper died mid-publish, and the IPC session's broken-pipe handling restarts capture in that case. The timestamp choice and the skip of an already-converted frame are one policy shared by both encode loops, and only frames actually converted are counted as such. Vibepollo stamps frames before converting them, so both encode loops now replace capture_timestamp and host_processing_timestamp with the converted frame's stamps before the pacing stamp is derived from them (the async loop's frame_variation smoothing then starts from the converted frame). capture_pacing_timestamp stays the claim time, since it anchors the capture loop's pacing rather than the frame. The demand check keeps Vibepollo's pacing context argument. A repeated direct frame is skipped, which must not delay a requested keyframe until the next capture. The async loop carries the request into the next iteration (retry_idr), which encodes the last input as the IDR without waiting; the sync loop encodes the last input as the IDR in the same pass, unstamped like the no-new-frame path. The decision is video::policy::retry_idr_after_repeat. This is the Vibepollo port of Nonary/vibeshine#347 and the follow-up fix. Signed-off-by: Ramazan Kara <github@r-kara.de>
RamazanKara
added a commit
to RamazanKara/Vibepollo
that referenced
this pull request
Oct 23, 2026
…ng up The helper copied every WGC frame into a scratch texture and then, on its delivery thread, from the scratch texture into the shared texture the host reads, so each frame cost two GPU copies even when nothing was queued. When no older frame is queued or mid-delivery, the capture callback now copies the WGC frame straight into the shared texture, as long as the helper's D3D context is free and the keyed mutex can be acquired without waiting. Otherwise the frame takes the scratch handoff as before, so a newer frame is never published ahead of an older one. The helper counts direct publications in its periodic diagnostics. The scratch fallback no longer holds the D3D context while it waits for the host. The delivery thread used to hold the context lock for up to 200 ms waiting on the shared keyed mutex, and the capture callback needs that lock for its scratch copy, so encoder backpressure could stall FrameArrived. It now waits for the keyed mutex alone and then takes the context only if it is free; if the callback is mid-copy, it hands the texture back and waits for the context instead. Neither side holds one lock while waiting on the other, and the frame stays in the delivering state throughout, which keeps the callback from publishing a newer frame ahead of it. This is the Vibepollo port of Nonary/vibeshine#347. Signed-off-by: Ramazan Kara <github@r-kara.de>
This was referenced Oct 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
co-written by Claude Opus 5.5
This brings Vibepollo#530 to vibeshine canary, rebased onto current canary and reworked after your review there. Vibepollo gets the same change afterwards.
What it does
wgc_direct_encoder_input, default on, fixed for each capture session). Each pool image aliases the helper's shared texture through its own duplicated NT handle. The capture thread only claims the newest frame id, and the encoder takes the keyed mutex on its own device, so the host's snapshot copy and one keyed-mutex hand-off per frame are gone. Turning the option off restores the copy path.AcquireSync(0, 0)succeeds, the WGC frame goes straight into the shared texture. Otherwise it takes the scratch handoff as before. The helper diagnostics countdirect_published.Your two review points
_d3d_context_mutex, then takes the context only if it's free; if the callback is mid-copy it hands the texture back and waits for the context instead. The frame stays in the delivering state throughout, so publication order holds.New since the Vibepollo PR
wgcc) and PyroWave keep the copy path.wgccre-sends its cached image on a timeout, which the cursor would drop. PyroWave needs the capture image's own texture and keyed mutex. Both are latched with the input mode.Tested
RX 7900 XT, Windows 11, native AMF, loopback Moonlight receiver with a barcode strip, 1920x1080 at 120 fps, physical display, direct input on vs off, alternating runs:
-DBUILD_WERROR=ON; each commit compiles on its own. WGC 29/29, video capture source 14/14, video policy 20/20, web unit 91/91. The remaining ctest failures are identical on plain canary. Linux, macOS and FreeBSD builds are left to CI; the Windows-only code stays behind the existing guards.