Skip to content

Explain the method from first principles (docs/theory.md) - #75

Merged
joeljose merged 1 commit into
mainfrom
docs/theory-tutorial
Sep 29, 2026
Merged

joeljose merged 1 commit into
mainfrom
docs/theory-tutorial

Conversation

@joeljose

Copy link
Copy Markdown
Owner

Closes #74

New: docs/theory.md (≈3,500 words)

A tutorial from first principles, linked from a new README section, "How to learn this":

  1. What magnifying motion means.
  2. Eulerian vs phase-based, including Wu et al.'s (1+α)δ < λ/8 bound.
  3. Local phase: the derivation that a shift δ changes phase by ωδ, why k·Δφ moves an edge kδ, and the wrap-around limit.
  4. The DTCWT: why not the DWT, the two trees and Hilbert pair, level-1 vs Q-shift filters (every --biort/--qshift option with its tap counts), six orientations, levels/sizes/4:1 redundancy, what --nlevels means spatially.
  5. Following phase over time: normalise, conjugate product, cumulative sum, and why zero coefficients give no motion.
  6. Temporal filtering: width mode as G(f) = H2(f)·[1 + (k−1)(1 − Hlow(f))], zero-phase windows, the 0.2327 factor, band mode, frequency resolution.
  7. Reconstruction.
  8. The unmagnified lowpass residual.
  9. Validation summary.
  10. Colour (yiq).
  11. Choosing parameters.
  12. Comparison table: Eulerian, steerable, Riesz and DTCWT.
  13. Glossary.
  14. Nine references.

New: scripts/make_theory_figures.py

Builds the five figures in docs/images/theory/ (536 KB total) from the actual pipeline functions. Needs matplotlib, which the script's docstring says; it isn't a project dependency.

  • DTCWT sub-bands of a face crop (3 levels × 6 orientations).
  • Phase change against sub-pixel image shift per level. Measured slopes give effective wavelengths of 4.1 / 6.0 / 11.2 / 19.6 / 43.3 px for levels 1–5. Half a turn is reached at 2.1 / 3.0 / 5.6 / 9.8 / 21.7 px, which explains the ~3 px limit measured in Reduce artefacts at high k: amplitude-weighted phase smoothing, limits on large phase shifts, per-level magnification #39.
  • Shift invariance: a blob moved in 1/8 px steps. DTCWT level energy stays within 1%; Haar DWT swings 54–62%.
  • Width-mode filter responses and the resulting gain against band mode.
  • One face.mp4 coefficient through φ, φ0, detail and the ×10 detail.

README corrections

  • The pipeline diagram used "complex division" (it has been a conjugate multiply since CPU path: exact-zero wavelet coefficients produce NaN, which shows up as black regions in the output #22). It now also shows band mode, and the text mentions --color-space yiq.
  • Unsourced claims replaced:
    • "10–100x amplification" → Wadhwa et al.'s reported ~4× over linear Eulerian, plus our measured ~3 px limit.
    • "~21x overcomplete" → configuration-dependent.
    • "~5x faster than steerable pyramids" → removed; the table now shows 4:1 redundancy.
    • GPU "~5x faster" → the measured face.mp4 times (24 s vs 60 s).
  • Project structure and CHANGELOG updated.

I left out statements I couldn't verify, e.g. whether Wadhwa et al. magnify luma only. The example coefficient's oscillation is described as measured: about 0.3 Hz, slow head motion rather than the pulse.

Add a tutorial that builds the method up from what magnifying motion
means: local phase and why scaling it moves an edge, the DTCWT (two
trees, level-1 and Q-shift filters, six orientations, levels and
redundancy), following phase over time, the two temporal filtering
modes, reconstruction and the lowpass residual, colour, parameter
choice, a comparison with Eulerian and steerable/Riesz methods, a
glossary and further reading.

Its figures come from scripts/make_theory_figures.py, which runs the
pipeline code. It also measures the per-level phase slope, which shows
where the ~3 px displacement limit comes from: the two finest levels
reach a half turn at 2-3 px.

README: a "How to learn this" path, a pipeline diagram that matches the
code (conjugate multiply, band mode, luma mode), and the unsourced
"10-100x", "~21x overcomplete" and "~5x faster" claims replaced with
sourced or measured statements.

Closes #74
@joeljose
joeljose merged commit 0c9e84b into main Sep 29, 2026
2 checks passed
@joeljose
joeljose deleted the docs/theory-tutorial branch September 29, 2026 05:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Docs: explain the algorithm from first principles (theory tutorial, figures, README fixes)

1 participant