Skip to content

fix(*): elminate flickering in interpolated output and improve color consistency with input#1471

Open
exwm wants to merge 1 commit intok4yt3x:masterfrom
exwm:master
Open

fix(*): elminate flickering in interpolated output and improve color consistency with input#1471
exwm wants to merge 1 commit intok4yt3x:masterfrom
exwm:master

Conversation

@exwm
Copy link
Copy Markdown

@exwm exwm commented Mar 12, 2026

Related Issues

Change Details

Add a virtual Interpolator::normalize() method that runs pass-through frames
through the same pixel-format round-trip as interpolated frames, eliminating
temporal colour inconsistencies and flickering in alternating frames (assuming 2x interpolation).

Refactor in_time_base_, out_time_base_, and out_pix_fmt_ from InterpolatorRIFE private members into Interpolator protected members to avoid duplication in future derived classes.

Capture authoritative color properties (color_space, color_range, color_primaries, color_trc) from the decoder context once during init() and store them in the Processor base class. Introduce make_color_hint(), which creates a metadata-only AVFrame carrying these properties and forwards it to avframe_to_ncnn_mat() and ncnn_mat_to_avframe(), ensuring sws_scale always uses a consistent colour matrix instead of falling back to BT.601 defaults whenever a frame carries AVCOL_SPC_UNSPECIFIED.

Additional changes in src/conversions.cpp:

  • Add avcol_spc_to_sws_cs() to map AVColorSpace to SWS colorspace constants with a BT.601 fallback and a thread-safe one-shot warning via std::once_flag.
  • Replace hardcoded SWS_BILINEAR with static constexpr int SWS_FILTER = SWS_BICUBIC.
  • Extract apply_sws_colorspace() helper to deduplicate the two sws_setColorspaceDetails call sites; failures emit a one-time warn on the first occurrence followed by debug logs with a running count thereafter.
  • Validate the return value of sws_scale() against dst_frame->height before returning.

Register src/interpolator.cpp in CMakeLists.txt.

Testing

  • Interpolated test video with options --frame-rate-mul 2 --processor rife --rife-model rife-v4.26 --pix-fmt yuv420p -c libx264 before and after this change -> observed flickering in video due to alternating artifacting/noise patterns eliminated
  • Upscaled video with options --processor realesrgan --scaling-factor 2 --realesrgan-model realesr-animevideov3 --pix-fmt yuv420p -c libx264 and output before and after the change seems the same, given that the input video used bt709 and yuv420p which may not exercise some of the new defensive code around color space handling.

@exwm exwm changed the title fix(*): elminate flickering in interpolated output and improve color consistency with input consistency with input fix(*): elminate flickering in interpolated output and improve color consistency with input Mar 12, 2026
… consistency with input

Add a virtual `Interpolator::normalize()` method that runs pass-through frames
through the same pixel-format round-trip as interpolated frames, eliminating
temporal colour inconsistencies and flickering in alternating frames (assuming 2x interpolation).

Refactor `in_time_base_`, `out_time_base_`, and `out_pix_fmt_` from
`InterpolatorRIFE` private members into `Interpolator` protected members to avoid
duplication in future derived classes.

Capture authoritative color properties (color_space, color_range, color_primaries,
color_trc) from the decoder context once during `init()` and store them in the
`Processor` base class. Introduce `make_color_hint()`, which creates a metadata-only
`AVFrame` carrying these properties and forwards it to `avframe_to_ncnn_mat()` and
`ncnn_mat_to_avframe()`, ensuring `sws_scale` always uses a consistent colour matrix
instead of falling back to BT.601 defaults whenever a frame carries
`AVCOL_SPC_UNSPECIFIED`.

Additional changes in `src/conversions.cpp`:
- Add `avcol_spc_to_sws_cs()` to map `AVColorSpace` to SWS colorspace constants
  with a BT.601 fallback and a thread-safe one-shot warning via `std::once_flag`.
- Replace hardcoded `SWS_BILINEAR` with `static constexpr int SWS_FILTER = SWS_BICUBIC`.
- Extract `apply_sws_colorspace()` helper to deduplicate the two
  `sws_setColorspaceDetails` call sites; failures emit a one-time `warn` on the
  first occurrence followed by `debug` logs with a running count thereafter.
- Validate the return value of `sws_scale()` against `dst_frame->height` before
  returning.

Register `src/interpolator.cpp` in `CMakeLists.txt`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

hqdn3d ffmpeg filter (bootleg solution to eliminate flicker from RIFE) Flickering problem on the video screen after using RIFE frame interpolation

1 participant