fix(*): elminate flickering in interpolated output and improve color consistency with input#1471
Open
exwm wants to merge 1 commit intok4yt3x:masterfrom
Open
fix(*): elminate flickering in interpolated output and improve color consistency with input#1471exwm wants to merge 1 commit intok4yt3x:masterfrom
exwm wants to merge 1 commit intok4yt3x:masterfrom
Conversation
… consistency with input Add a virtual `Interpolator::normalize()` method that runs pass-through frames through the same pixel-format round-trip as interpolated frames, eliminating temporal colour inconsistencies and flickering in alternating frames (assuming 2x interpolation). Refactor `in_time_base_`, `out_time_base_`, and `out_pix_fmt_` from `InterpolatorRIFE` private members into `Interpolator` protected members to avoid duplication in future derived classes. Capture authoritative color properties (color_space, color_range, color_primaries, color_trc) from the decoder context once during `init()` and store them in the `Processor` base class. Introduce `make_color_hint()`, which creates a metadata-only `AVFrame` carrying these properties and forwards it to `avframe_to_ncnn_mat()` and `ncnn_mat_to_avframe()`, ensuring `sws_scale` always uses a consistent colour matrix instead of falling back to BT.601 defaults whenever a frame carries `AVCOL_SPC_UNSPECIFIED`. Additional changes in `src/conversions.cpp`: - Add `avcol_spc_to_sws_cs()` to map `AVColorSpace` to SWS colorspace constants with a BT.601 fallback and a thread-safe one-shot warning via `std::once_flag`. - Replace hardcoded `SWS_BILINEAR` with `static constexpr int SWS_FILTER = SWS_BICUBIC`. - Extract `apply_sws_colorspace()` helper to deduplicate the two `sws_setColorspaceDetails` call sites; failures emit a one-time `warn` on the first occurrence followed by `debug` logs with a running count thereafter. - Validate the return value of `sws_scale()` against `dst_frame->height` before returning. Register `src/interpolator.cpp` in `CMakeLists.txt`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issues
Change Details
Add a virtual
Interpolator::normalize()method that runs pass-through framesthrough the same pixel-format round-trip as interpolated frames, eliminating
temporal colour inconsistencies and flickering in alternating frames (assuming 2x interpolation).
Refactor
in_time_base_,out_time_base_, andout_pix_fmt_fromInterpolatorRIFEprivate members intoInterpolatorprotected members to avoid duplication in future derived classes.Capture authoritative color properties (color_space, color_range, color_primaries, color_trc) from the decoder context once during
init()and store them in theProcessorbase class. Introducemake_color_hint(), which creates a metadata-onlyAVFramecarrying these properties and forwards it toavframe_to_ncnn_mat()andncnn_mat_to_avframe(), ensuringsws_scalealways uses a consistent colour matrix instead of falling back to BT.601 defaults whenever a frame carriesAVCOL_SPC_UNSPECIFIED.Additional changes in
src/conversions.cpp:avcol_spc_to_sws_cs()to mapAVColorSpaceto SWS colorspace constants with a BT.601 fallback and a thread-safe one-shot warning viastd::once_flag.SWS_BILINEARwithstatic constexpr int SWS_FILTER = SWS_BICUBIC.apply_sws_colorspace()helper to deduplicate the twosws_setColorspaceDetailscall sites; failures emit a one-timewarnon the first occurrence followed bydebuglogs with a running count thereafter.sws_scale()againstdst_frame->heightbefore returning.Register
src/interpolator.cppinCMakeLists.txt.Testing
--frame-rate-mul 2 --processor rife --rife-model rife-v4.26 --pix-fmt yuv420p -c libx264before and after this change -> observed flickering in video due to alternating artifacting/noise patterns eliminated--processor realesrgan --scaling-factor 2 --realesrgan-model realesr-animevideov3 --pix-fmt yuv420p -c libx264and output before and after the change seems the same, given that the input video used bt709 and yuv420p which may not exercise some of the new defensive code around color space handling.