Skip to content

Measuring phase against frame 0 wraps once the surface drifts; use frame-to-frame differences plus a cumulative sum #17

Closed

Description

@joeljose

Severity: Medium. It limits recording length and robustness on real rigs.

Problem

Every frame's phase is measured against frame 0: np.angle(coeffs * conj(ref)). The result is wrapped to [−π, π]. Once the surface or camera drifts more than about half a wavelength of a band away from where it was in frame 0 (a few pixels at level 1, less at finer scales), the phase difference wraps. The signal then jumps by 2π and the amplitude weighting can't hide it. The longer the clip (e.g. 22,859 or 38,083 frames for the MIT samples), the more likely this is.

The README "Future Work" section notes the related loss of amplitude for very small vibrations.

Evidence

Synthetic 0.05 px tones with a 3 px linear drift over 0.5 s, with -fl 100 applied so the drift itself is filtered out: correlation with the truth drops from 0.998 (0.3 px drift) to 0.952 (3 px drift), and in-band RMS from 0.50 to 0.43. What remains is wrap artefacts that the high-pass can't remove.

Suggested fix

Use the approach that works for the sibling motion-magnification projects:

  • Compute frame-to-frame phase differences, angle(c_t · conj(c_{t−1})). These are always small at audio frame rates, so they never wrap.
  • Cumulatively sum them per coefficient to get unwrapped phase. Then high-pass per band before amplitude-weighted spatial summation.
  • This uses the same memory as now: keep one previous frame of coefficients instead of the frame-0 reference.
  • Alternatively, refresh the reference every K frames and stitch the segments together, removing the offset at each boundary.
  • Keep the current reference mode behind a flag for comparison.

Acceptance criteria

  • The 3 px-drift synthetic case gives correlation ≥ 0.99 with -fl 100.
  • No regression on the no-drift cases.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions