Skip to content

fix(windows): preserve cross-packet resampling continuity - #24

Closed
My-Denia wants to merge 1 commit into
base/os920-upstream-2896c1f7from
os920-upstream-candidate
Closed

My-Denia wants to merge 1 commit into
base/os920-upstream-2896c1f7from
os920-upstream-candidate

Conversation

@My-Denia

@My-Denia My-Denia commented Oct 11, 2026 •

Copy link
Copy Markdown
Owner

fix(windows): preserve non-integer resampling continuity across packets

Fixes the Windows capture helper's audio path for issue getopenscreen#920, item 24 (study issue; this PR is the continuity slice only). Upstream-candidate staging PR: single clean commit on the latest upstream/main, carrying everything the earlier fork review PR (#23) verified — its 6-commit history was review scaffolding, not the submission shape.

The defect

convertAudioWithGain's non-integer-rate branch kept no cross-packet state: every packet recomputed its output-frame count with llround(packetFrames × rateRatio) and restarted interpolation phase at zero. The same signal produced different output depending only on packet boundaries — deterministic counterexample at the patch's base commit (mono float32→PCM16, one AudioDecimatorState per stream, as AudioMixer owns them):

Case Geometry single-shot packetized
A 48k→44.1k, 100-frame packets 14700 frames 14720 (+0.125 frames/packet drift)
D 44.1k→48k, 100-frame packets 16000 frames 16023 (≈+0.1565/packet)
B/C exact-multiple packets (160/480) — byte-identical (controls)

Upside: exact-multiple packet geometry — including the standard 10 ms/480-frame WASAPI packet — was already correct; the defect bites on non-aligned packet geometry (non-10 ms endpoint periods, partial first packets, odd rates). The integer-factor decimation path (96/192k→48k) already carried cross-packet state upstream and is untouched.

The change (4 files)

electron/native/wgc-capture/src/audio_sample_utils.{h,cpp} + audio_sample_utils_test.cpp, plus a real-machine results row in technical-documentation/testing/manual-e2e-checklist.md:

  • Cumulative integer-rational accounting K(N)=max(1,round½↑(N·T/S)), C(N)=min(K,⌈(N−1)T/S⌉) — output counts never depend on the partition; checked counters fail safe.
  • Absolute-position interpolation with the legacy double arithmetic preserved verbatim (same expression order, byte-identical for unchanged geometry), deferred right-endpoint windows (bounded by ⌈T/S+½⌉; ≤2 frames at 48k↔44.1k), bounded retained history (⌈S/2T⌉+2 frames, geometry not stream length).
  • finalizeAudioConversion: append-only tail emission so the pre-existing stateless one-shot overload keeps its exact whole-stream contract through ONE conversion path (it is the stateless wrapper's internal step; the streaming/production path never finalizes — its documented bounded tail is dropped at reset/stop, and that behavior is pinned by tests).
  • T==S immediate emission (exact at frac≡0): the 48 kHz float→PCM16 microphone path — the shape real production actually traverses — keeps per-packet emission counts identical to the old code.
  • Zero mixer call-site changes; narrow test-only friend struct AudioMixerTestAccess.
  • 54 new assertions (51 continuity + 3 starvation-resume with integer/same-rate controls); all 58 original assertions and tolerances unchanged; a --resample-continuity-only focused mode and an OS920_PRISTINE_REGRESSION build switch that compiles the new tests against pristine sources to produce assertion-based old-FAIL.

Evidence

  • Old-FAIL (assertion-based): expanded test file on pristine base — build exit 0, run exit 1, 9 named failures (A 14720/14700, D 16023/16000, ragged both directions, rounding totals, formats/gain, mono→stereo); B/C controls pass on the old code.
  • New-PASS: focused 54/54; full native suite 176/176, 0 FAIL, 0 SKIP (re-verified on this PR's exact base, upstream/main 2896c1f7; Windows-local MSVC builds — GitHub CI has no native C++ job and does not build or run the wgc-capture suite); fc /b byte controls — one-shot outputs and B/C packetized outputs identical old-vs-fixed; A/D packetized differ exactly by the fix.
  • Real Windows 11 machine (both endpoints 48 kHz, measured: single-valued 480-frame/10 ms packets, 0 non-aligned; AAC-compatible encoder target 48 kHz logged): 6×60 s matched-rate recordings (3 old / 3 new, interleaved) + pause/resume takes — all valid, marker spacing 3.0000–3.001 s, old/new within each other's repeat envelope, zero clipping; captured real callback streams replayed through old/new converters with real boundaries: 6/6 byte-identical. No regression observed; a follow-up real mismatched-rate pass (owner-enabled device-format change) then ran the non-integer path live: 44.1 kHz loopback source (JM6PRO, 3496/3496 real packets, all exactly 441 frames — perfectly 10 ms-aligned) against a forced 48 kHz AAC target via the production OPENSCREEN_WGC_FORCE_AAC_SOURCE_RATE surface; old vs fixed decoded output is time-aligned at zero offset, 97.4 % byte-identical with residual maxdiff 170 (AAC block-boundary quantization), each build repeating byte-identically. This hardware cannot produce the non-aligned packet geometry the defect class needs, so the old-FAIL/new-PASS defect proof remains the deterministic suites; the real pass evidences end-to-end no-regression on the non-integer path. Aliasing on non-integer downsampling (case E: 23 kHz→21.1 kHz alias survives) is a known limitation, deliberately out of scope. Starvation-resume state continuity is fixed and behavior-tested (deterministic mixer-path tests with integer/same-rate controls); on real hardware only the startup empty-queue padding (~70 ms) was observed — a full starve-with-history-then-resume event was not reproduced outside the deterministic tests.
  • Minimality independently reviewed (keep-as-is verdict with removal-consequence analysis): deleting the finalizer either resurrects the two-arithmetic divergence class that caused this defect or silently changes the stateless overload's frame-count contract; per-packet rounding, packet-endpoint clamping, and hot-path frame-holding are each the defect or a regression by construction.

Base branch note

This PR's base is the auxiliary branch base/os920-upstream-2896c1f7 pinned at upstream/main 2896c1f7b9ad46cf53419e6edd360da2d2d80ac3 (the latest tip at delivery time), so the diff shows exactly the four intended files with no drift (fork main is 400+ commits behind upstream — deliberately untouched). The three audio files' blobs are identical between this base and the earlier validated base a997ce76 (upstream did not touch them in between); the checklist row is applied on top of upstream's own newer checklist edits. One commit, no fork-only CI scaffolding.

Not in this PR

Anti-alias filtering for non-integer downsampling; mixer clock/cushion changes; anything from getopenscreen#1076. Cross-platform (macOS/Linux) audio paths don't share this code.

Copilot AI balanced review requested due to automatic review settings October 11, 2026 07:26

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0a08024d-68c0-42c6-ab68-a88495955590

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: bfd004e2-4639-4b90-bdca-ea189ae5c4e1


📥 Commits

Reviewing files that changed from the base of the PR and between 2896c1f and 5764aa2.



📒 Files selected for processing (4)
  • electron/native/wgc-capture/src/audio_sample_utils.cpp
  • electron/native/wgc-capture/src/audio_sample_utils.h
  • electron/native/wgc-capture/src/audio_sample_utils_test.cpp
  • technical-documentation/testing/manual-e2e-checklist.md


Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.




📝 Summary
  • **目的:**修复 Windows WGC 音频非整数采样率转换依赖数据包边界的问题。转换现在按累计输入帧计算输出位置,并跨数据包保留插值状态。
  • **行为:**非整数转换会延迟依赖最新输入帧的输出,直到插值窗口完整或转换终结。单次无状态转换会自动终结并补齐尾帧;实时流不会终结,剩余尾帧会在重置或停止时丢弃。同采样率转换仍立即输出;整数降采样路径保持原行为。
  • API:AudioDecimatorState 增加插值状态管理接口,并新增 finalizeAudioConversion。未报告配置或数据格式变更。
  • **兼容性与风险:**变更仅涉及 Windows WGC 路径。饥饿源恢复时,混音器会丢弃插值状态,以避免跨音频间隙沿用历史;整数降采样状态保持原行为。非整数降采样混叠仍不在本次范围内。
  • **验证:**作者报告 MSVC 本机聚焦测试 54/54、完整原生测试 176/176 通过。GitHub CI 不运行该原生测试套件。设备录制的 6/6 条回放与捕获流逐字节一致;真实设备测试使用的非整数转换数据包均为 441 帧,未覆盖非对齐数据包缺陷。
  • **剩余限制:**未能在真实硬件上复现带历史状态的饥饿恢复场景;该场景由确定性混音器测试覆盖。没有证据表明已部署或发布。
📝 Summary

Walkthrough

Windows WGC 音频路径新增跨数据包的非整数采样率转换状态,按累计帧位置输出,并在流终结时补齐尾帧。混音器来源从饥饿状态恢复时会放弃该来源的插值历史。新增连续性、状态生命周期和混音器行为测试。

Changes

Windows WGC 音频重采样连续性

Layer / File(s) Summary
插值状态与帧配额
electron/native/wgc-capture/src/audio_sample_utils.h, electron/native/wgc-capture/src/audio_sample_utils.cpp
AudioDecimatorState 新增跨数据包插值状态、累计输入与输出计数、全局帧配额及历史保留逻辑。模式切换、溢出、终结和重置会按对应路径更新状态。
分包转换与尾部终结
electron/native/wgc-capture/src/audio_sample_utils.cpp, electron/native/wgc-capture/src/audio_sample_utils.h, electron/native/wgc-capture/src/audio_sample_utils_test.cpp
非整数转换按绝对输出位置处理,并延后依赖最新输入帧的输出。finalizeAudioConversion 补齐配额内的尾帧。新增测试比较不同分包方式的输出,并覆盖配额、格式、增益和终结行为。
混音器恢复与连续性验证
electron/native/wgc-capture/src/audio_sample_utils.cpp, electron/native/wgc-capture/src/audio_sample_utils.h, electron/native/wgc-capture/src/audio_sample_utils_test.cpp, technical-documentation/testing/manual-e2e-checklist.md
来源从饥饿状态恢复时,混音器放弃该来源的插值状态。测试覆盖队列独立性、暂停与恢复行为,以及恢复后的插值冷启动。手动验证记录了设备测试范围与限制。

Suggested reviewers: etiennelescot



Priority: ⬇️ Low

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 5764a

This change makes Windows non-integer audio resampling consistent across packet boundaries. It also resets interpolation history when a starved source resumes. No concrete defect remains open. The main residual caveat is that the native tests are not exercised by CI.

Pre-merge checks | Passed 3 | Failed 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check Warning 描述包含详细的缺陷说明、实现范围、测试结果和限制,但没有按模板提供 Type of change、Release impact、Desktop impact 等必需章节。描述还使用了“Fixes #920”,同时说明本 PR 只覆盖连续性子问题,未明确表示完整解决该问题。 按模板补充 Summary、Related issue、Type of change、Release impact、Desktop impact、Screenshots / video 和 Testing 章节。将“Fixes #920”改为“Refs #920”或“Related to #920”,除非本 PR 完整解决 issue #920。
✅ Passed checks (3 passed)
Check name Status Explanation
Title check Passed 标题准确描述了主要改动:修复 Windows 非整数重采样的跨数据包连续性。标题没有使用模糊表述,也没有声称已经发布或完全修复。
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.



✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR



  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-11T13:31:31.824698Z 2ae4dcc New commits
🔒 Security Review ✅ Completed 2026-10-11T07:28:11.287334Z c5a5a0f PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c5a5a0f613

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread electron/native/wgc-capture/src/audio_sample_utils.cpp
@My-Denia
My-Denia force-pushed the os920-upstream-candidate branch from c5a5a0f to 5764aa2 Compare October 11, 2026 08:35
@My-Denia

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@My-Denia
My-Denia force-pushed the os920-upstream-candidate branch from 5764aa2 to 2ae4dcc Compare October 11, 2026 13:29
@My-Denia My-Denia closed this Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants