diff --git a/CLAUDE.md b/CLAUDE.md index c508f5a2..a7f7ec0e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -786,17 +786,39 @@ seconds of every track converging — through the intro, which is the least repr — and pays that cost again on every device, on every play. Same reasoning as the loudness: the audio does not change, so neither does the answer. Measure once, keep it in `track_analysis`, serve it. -- The TEMPO is **Deezer's own** where Deezer has one: its public API publishes a `bpm` per track, - exact and free. Anything else (a local upload, a track with no figure) is measured from a - low-passed envelope — ffmpeg decimates to 1 kHz u8 and the per-frame peak is a `max`/`min` over a - ten-byte slice, so no audio sample is ever touched from Python — **and only when something in +- The TEMPO is **beat-tracked from the file** (`deezer/beats.py`, `ANALYSIS_VERSION` 5): a numpy port + of the analyser Mixxx ships as its default — qm-dsp's complex-spectral-difference onsets (44.1 kHz, + 1024/512) and TempoTrackV2 (comb filter bank + Viterbi for the period, Ellis-style dynamic + programming for the beats), then Mixxx's BeatUtils (constant regions within 25 ms, the longest + joined to those agreeing in tempo and phase, rounded to the simplest value its error bars allow). + GPL-2.0-or-later code, credited in the file. The envelope estimator it replaced read 154 for a 150 + BPM record and 133 for 132 (2.7%: a beat lost every nine seconds); on the 28 arranged records + (`tools/tempo_eval.py`) it was 17/26 within 3.5%, the tracker is **23/26 exact** and 26/26 with + Deezer's figure as witness, ~1 s for a five-minute file. And Deezer has NO figure for nearly all + of a hard-dance library (2 of 3 462 files here carried a non-zero one), so the estimator was what + everybody got. Three rules sit on top, each measured: + - **Hardcore is doubled back**: the Rayleigh weighting centres on 120 BPM and tracks a 200 BPM + kick on every other beat (Mixxx's own failure). When the half beats strike as hard as the + beats — both in the kick band (43-129 Hz) and across the spectrum, between 0.85 and 1.3 of the + beats — the tempo is doubled (frenchcore, Krach, speedcore read 0.93-1.06; nothing tracked right + falls in the window; hard house's offbeat donk took the phase and read 13.7, hence the cap). + - **No beat, no tempo**: the old attack gate (`_attack_rate`, 40 rises a minute in the bass) and + the beats' salience against the quarter-beat positions ≥ 1.12 (real masters 1.16-3.3, pads + 1.01-1.07 — a thin margin, stated; the player's locked grid re-checks presence anyway). + - **Deezer's figure only witnesses the OCTAVE** (`_reconcile_tracked`): Deezer at twice the + tracked tempo moves it up (dnb at 87/174, double-kick metal); at half it is Deezer's hardcore + listing and ignored, unless alternate beats are unequal in the bass (< 0.3: a one-drop tracked + at double time) — then it halves; any other disagreement follows the tracker's confidence. + Without numpy the envelope estimator below still answers. A version bump re-measures the library + on boot (`webui/analysis.reanalyse_if_stale`) instead of waiting for the button. +- The fallback envelope estimator measures a low-passed envelope — ffmpeg decimates to 1 kHz u8 and + the per-frame peak is a `max`/`min` over a ten-byte slice — **and only when something in that band STRIKES** (`_attack_rate`, `ANALYSIS_VERSION` 3). The block peak of a steady note ripples with its phase, and the normalised autocorrelation turned that ripple into a confident beat: a held 82 Hz note read 240 BPM at 0.79, a held chord 231 at 0.50, and the player anchored on it. The same bytes, squared and smoothed, now have to rise 1 dB within 30 ms at least 40 times a minute (held tones and chords: 0; the sparsest record with a beat: 99.7). The rise is small on - purpose — a limitered hard kick's tail sits 2 dB under its attack, and 3 dB lost those. Every - record with a beat measures exactly what it did before; `tools/tempo_eval.py` is the check. + purpose — a limitered hard kick's tail sits 2 dB under its attack, and 3 dB lost those. - The DESCRIPTORS come from ffmpeg's `aspectralstats` and `ebur128`: centroid, spread, flatness, entropy, rolloff and flux per frame, plus integrated loudness and **loudness range**. LRA is the single best "how squashed is this" axis there is — a limitered hardcore master sits near 3 LU, a diff --git a/supysonic/deezer/analysis.py b/supysonic/deezer/analysis.py index b4b1e94f..f9e442fd 100644 --- a/supysonic/deezer/analysis.py +++ b/supysonic/deezer/analysis.py @@ -71,7 +71,10 @@ # a grid share of 0.06 and trance was doubled to 280), the hard families # are written on the scales these measures have, and a verdict's # confidence is backed by how much its rules actually found. -ANALYSIS_VERSION = 4 +# 5 the tempo is BEAT-TRACKED (beats.py, Mixxx's analyser) and Deezer's +# figure only witnesses its octave; the envelope estimator read 154 for +# a 150 BPM record and the wrong octave on half the non-club records. +ANALYSIS_VERSION = 5 # Never analyse more than this much of a file. Ten minutes is far more than any # verdict needs, and it is what stops a two-hour DJ set costing two hours of @@ -623,15 +626,48 @@ def _grid_share(odf, period): return num / den if den > 1e-9 else 0.0 +#: The beat tracker's salience (beats.py `_salience`: on-beat onsets against +#: those a quarter of a beat away) under which there is no beat to serve. +#: Measured: twenty real masters from this library read 1.16 (a dense hard +#: techno) to 3.3, the eval's records 1.33 (dubstep) to 29; the ambient pads +#: 1.07 and a detuned-saw pad 1.01. The margin is thin on both sides, and +#: stated: a phantom tempo here is caught again by the player, whose locked +#: grid only counts as present where the music folds onto it. +MIN_BEAT_SALIENCE = 1.12 + + def _measure_tempo(path): """(bpm, confidence, grid_share) measured from the file itself. ``(None, 0.0, 0.0)`` when nothing in the low band strikes often enough to carry a beat at all (``_attack_rate``): no tempo is better than a phantom. + + The tempo is the BEAT TRACKER's (beats.py: Mixxx's analyser, ported) when + numpy is there — exact to the hundredth on a record made to a click, where + the envelope estimator below read 154 for 150. Without numpy the envelope + estimator still answers, as it always did. """ odf, attacks = _low_envelope(path) if attacks < MIN_ATTACKS_PER_MINUTE: return None, 0.0, 0.0 + from . import beats + + if beats.available(): + r = beats.analyse(path) + if r is None or r["salience"] < MIN_BEAT_SALIENCE: + return None, 0.0, 0.0 + bpm = r["bpm"] + if not (MIN_BPM <= bpm <= MAX_BPM): + return None, 0.0, 0.0 + smooth = _smooth(odf) + grid = _grid_share(smooth, 60 * ENV_HZ / bpm) if len(smooth) >= 8 * ENV_HZ else 0.0 + _remember_witness(path, r["alternation"]) + return round(bpm, 3), round(r["confidence"], 3), round(grid, 3) + return _measure_tempo_envelope(odf) + + +def _measure_tempo_envelope(odf): + """The envelope estimator: the fallback when numpy is not installed.""" odf = _smooth(odf) n = len(odf) win = 8 * ENV_HZ @@ -1248,6 +1284,73 @@ def _reconcile_octave(published, measured, measured_conf, published_conf, source return published, published_conf, source +# The tracker's alternation reading per file, for the reconciliation that +# follows the measurement in the same analysis (kept apart so _measure_tempo's +# (bpm, confidence, grid) contract is unchanged). Bounded; taken once. +_witness_lock = threading.Lock() +_witness = OrderedDict() + + +def _remember_witness(path, alternation): + with _witness_lock: + _witness[path] = alternation + while len(_witness) > 64: + _witness.popitem(last=False) + + +def _take_witness(path): + with _witness_lock: + return _witness.pop(path, None) + + +def _tracker() -> bool: + from . import beats + + return beats.available() + + +#: Alternate beats this unequal in the bass mean the track's beat is every +#: OTHER tracked one (see beats.py `alternation`): measured, a one-drop +#: tracked at double time 0.21; every hard-dance record 0.93-1.00; the +#: kick-on-one-and-three records it must never halve (boom bap, pop, piano +#: rag) 0.33-0.41 — and those agree with Deezer anyway. +HALF_ALTERNATION = 0.3 +#: How sure the tracker must be (the share of the track its constant tempo +#: was taken from) to be served over a published figure it disagrees with. +TRACKED_CONF = 0.5 + + +def _reconcile_tracked(published, measured, conf, alternation=None): + """The beat tracker's tempo, with Deezer's figure as a witness of OCTAVE + only. (bpm, confidence, source). + + The tracker is the authority on the tempo — it is exact where Deezer's + figure is rounded or simply wrong — and it can be wrong only one way the + signal cannot settle: the octave of a genre whose half-time and full-time + readings fold alike (dnb at 87 or 174, double-kick metal at 90 or 180). + So: Deezer at TWICE the tracked tempo moves it up an octave; Deezer at + HALF is Deezer's own well-known listing of hardcore and is ignored — + unless alternate tracked beats are clearly unequal in the bass, which is a + half-time groove tracked at double time (a one-drop), and both witnesses + then agree; any other disagreement goes to the tracker when it is sure, + to Deezer when it is not. + """ + if not published: + return measured, conf, "measured" + r = published / measured + if abs(r - 1) < _SAME: + return measured, max(conf, 0.9), "measured" + if abs(r / 2 - 1) < _SAME and measured * 2 <= MAX_BPM: + return round(measured * 2, 3), max(conf, 0.85), "measured+octave" + if abs(r * 2 - 1) < _SAME: + if alternation is not None and alternation < HALF_ALTERNATION and measured / 2 >= MIN_BPM: + return round(measured / 2, 3), max(conf, 0.85), "measured+octave" + return measured, conf, "measured" + if conf >= TRACKED_CONF: + return measured, conf, "measured" + return published, 0.8, "deezer" + + # --- is Deezer's figure right? --------------------------------------------------- #: Below this the file's own measurement is not a witness either way. AUDIT_MIN_CONF = 0.3 @@ -1395,7 +1498,11 @@ def analyze_track_verbose(track: Track, provider=None, force: bool = False): pulse = 0.0 try: measured, conf, pulse = _measure_tempo(track.path) - if not bpm and measured: + if measured and _tracker(): + bpm, bpm_conf, bpm_source = _reconcile_tracked( + bpm, measured, conf, _take_witness(track.path) + ) + elif not bpm and measured: bpm, bpm_conf, bpm_source = measured, conf, "measured" elif bpm and measured: bpm, bpm_conf, bpm_source = _reconcile_octave( diff --git a/supysonic/deezer/beats.py b/supysonic/deezer/beats.py new file mode 100644 index 00000000..03b6a0a1 --- /dev/null +++ b/supysonic/deezer/beats.py @@ -0,0 +1,577 @@ +# This file is part of Supysonic. +# Supysonic is a Python implementation of the Subsonic server API. +# +# Distributed under terms of the GNU AGPLv3 license. +# +# The beat tracker is a port of the one Mixxx ships as its default analyser: +# qm-dsp's DetectionFunction (DF_COMPLEXSD) and TempoTrackV2 — QM DSP Library, +# Centre for Digital Music, Queen Mary University of London, copyright +# 2008-2009 Matthew Davies and QMUL, GPL-2.0-or-later — and Mixxx's +# BeatUtils (constant regions, the rounded constant tempo), GPL-2.0-or-later. +# Both licences allow this combination under the AGPLv3. + +"""The whole-file BEAT TRACKER: the tempo the server serves. + +The old estimator (analysis._measure_tempo) autocorrelated a 1 kHz envelope of +the bass and bucketed the answer to a whole BPM: 154 for a 150 BPM record, 133 +for 132 — 2.7% off, which walks a beat away from the music every nine seconds +— and on half the non-club records the wrong octave. Rather than tune it, this +is the analyser DJs already trust: Mixxx's default, which is the Queen Mary +tracker (Davies & Plumbley, "Context-dependent beat tracking of musical +audio", 2007) followed by Mixxx's own constant-tempo pass. + + 1. ONSETS. ffmpeg decodes to 44.1 kHz mono; a 1024-sample Hann window every + 512 samples (86 Hz) gives a complex spectral difference: how far each bin + lands from where its magnitude and phase PREDICTED it would be. A steady + note continues its phase and costs nothing; an attack does not. + 2. PERIOD. Over ~6 s windows every 1.5 s: the autocorrelation of the onsets, + a comb filter bank (four harmonics of each period), weighted by a + Rayleigh curve around 120 BPM; then a Viterbi path through the windows + with a transition that only lets the period drift slowly. + 3. BEATS. Dynamic programming along the onset function (Ellis 2007): every + frame's best predecessor about one period back, scored by how strong the + onset is and how close to the period the gap is; backtracked from the + strongest point of the last period. + 4. CONSTANT TEMPO (Mixxx BeatUtils). The beat list is cut into regions that + fit a constant beat length within 25 ms; the longest one, joined to the + earliest and latest regions it agrees with in tempo AND phase, gives the + tempo, rounded to the simplest value its own error bars allow (a whole + BPM, then halves, thirds…). A track recorded to a click comes out exact. + +Everything is numpy, in chunks, so a two-hour set costs memory for a chunk, +not for the set. Without numpy the module reports itself unavailable and the +caller keeps the old estimator. +""" + +from __future__ import annotations + +import logging +import math +import subprocess +import threading + +logger = logging.getLogger(__name__) + +try: # pragma: no cover - the image has it; a bare install may not + import numpy as np +except Exception: # noqa: BLE001 + np = None + +#: The tracker's own rate: 512-sample steps at 44.1 kHz is what TempoTrackV2's +#: constants (the Rayleigh centre, the period range) are written against. +SR = 44100 +STEP = 512 +WINDOW = 1024 +DF_HZ = SR / STEP +#: Read at most this much of a file. Mixxx reads the whole track; a DJ set +#: twelve times as long tells nothing more about its tempo. +MAX_SECONDS = 20 * 60 +#: Samples per chunk of decoded audio (a multiple of STEP): 512 frames, about +#: 20 MB of working arrays — the analysis runs on several workers at once. +CHUNK = STEP * 512 +#: A decode that has not finished by then is killed: one wedged ffmpeg must +#: not hold a library job's worker for ever. +DECODE_TIMEOUT = 300 + +EPS = 0.0000008 +#: The bins of the low-band onset function: 43-129 Hz at 43 Hz a bin, the +#: kick's body. Up to 215 Hz it heard hard house's offbeat 300 Hz "donk" (a +#: Hann window's skirt) as a kick on every half beat and doubled 128 to 256. +LOW_BINS = slice(1, 4) +#: ...and the wider bass band (43-215 Hz) the alternation witness reads, where +#: the kick and the bass line together say which beats carry the groove. +BASS_BINS = slice(1, 6) + +# BeatUtils +MAX_PHASE_ERROR = 0.025 # s +MAX_PHASE_ERROR_SUM = 0.1 # s +MAX_OUTLIERS = 1 +MIN_REGION_BEATS = 16 + + +def available() -> bool: + return np is not None + + +# --- 1. the onset detection function --------------------------------------------- + + +def _decode(path, seconds=MAX_SECONDS): + """ffmpeg's 44.1 kHz mono float32 PCM, as a stream of arrays.""" + cmd = [ + "ffmpeg", "-v", "error", "-nostdin", "-t", str(seconds), "-i", path, + "-vn", "-ac", "1", "-ar", str(SR), "-f", "f32le", "-", + ] + proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + timer = threading.Timer(DECODE_TIMEOUT, proc.kill) + timer.daemon = True + timer.start() + try: + while True: + buf = proc.stdout.read(CHUNK * 4) + if not buf: + break + yield np.frombuffer(buf[: len(buf) // 4 * 4], dtype="= 2 else ph1 + ph1 = ph[-1] + mag1 = mag[-1] + out.append(vals) + tail = buf[n * STEP :] + if not out: + return np.zeros(0), np.zeros(0), np.zeros(0) + return np.concatenate(out), np.concatenate(lows), np.concatenate(basses) + + +# --- 2. the beat period -------------------------------------------------------------- + + +def _adaptive_threshold(x): + """MathUtilities::adaptiveThreshold: subtract a local mean (8 before, 7 + after), floor at zero.""" + n = len(x) + if n == 0: + return x + c = np.concatenate([[0.0], np.cumsum(x)]) + i = np.arange(n) + first = np.maximum(0, i - 8) + last = np.minimum(n - 1, i + 7) + mean = (c[last + 1] - c[first]) / (last - first + 1) + return np.maximum(x - mean, 0.0) + + +WV_LEN = 128 + + +def _rayleigh(inputtempo=120.0): + rayparam = (60 * 44100 / 512.0) / inputtempo + i = np.arange(WV_LEN, dtype=np.float64) + return (i / rayparam**2) * np.exp(-(i**2) / (2 * rayparam**2)) + + +def _comb_index(): + """The (rcf row, acf lag, weight) triples of TempoTrackV2::get_rcf's comb: + for each period i, four harmonics a, each spread over 2a-1 lags.""" + rows, lags, w = [], [], [] + for i in range(2, WV_LEN): + for a in range(1, 5): + for b in range(1 - a, a): + rows.append(i - 1) + lags.append(a * i + b - 1) + w.append(1.0 / (2 * a - 1)) + return np.array(rows), np.array(lags), np.array(w) + + +_COMB = None + + +def _rcf(frame, wv): + global _COMB + if _COMB is None: + _COMB = _comb_index() + rows, lags, w = _COMB + x = _adaptive_threshold(frame) + n = len(x) + # The biased-by-overlap autocorrelation, by FFT. + f = np.fft.rfft(x, 2 * n) + acf = np.fft.irfft(f * np.conj(f))[:n] / (n - np.arange(n)) + rcf = np.zeros(WV_LEN) + np.add.at(rcf, rows, acf[lags] * wv[rows] * w) + rcf = _adaptive_threshold(rcf) + EPS + return rcf / (rcf.sum() + EPS) + + +def beat_period(df, inputtempo=120.0): + """TempoTrackV2::calculateBeatPeriod: the beat period (in DF frames) for + every 128 frames of the onset function.""" + wv = _rayleigh(inputtempo) + winlen, hop = 512, 128 + n = len(df) + cols = [] + for i in range(-winlen // 2, n - winlen // 2, hop): + frame = np.zeros(winlen) + k = max(0, -i) + l = min(winlen, n - i) + frame[k:l] = df[i + k : i + l] + cols.append(_rcf(frame, wv)) + out = np.zeros(n // 128 + 1, dtype=np.int64) + if len(cols) < 2: + return out + rcf = np.array(cols) # T x Q + T, Q = rcf.shape + sigma = 8.0 + tmat = np.zeros((Q, Q)) + idx = np.arange(20, Q - 20) + tmat[np.ix_(idx, idx)] = np.exp(-((idx[None, :] - idx[:, None]) ** 2) / (2 * sigma**2)) + delta = np.zeros((T, Q)) + psi = np.zeros((T, Q), dtype=np.int64) + delta[0] = wv * rcf[0] + delta[0] /= delta[0].sum() + EPS + for t in range(1, T): + cand = delta[t - 1][None, :] * tmat # [j, i] + psi[t] = np.argmax(cand, axis=1) + best = cand[np.arange(Q), psi[t]] + # qm-dsp's get_max_ind returns 0 when nothing beats zero. + psi[t][best <= 0] = 0 + delta[t] = best * rcf[t] + delta[t] /= delta[t].sum() + EPS + path = np.zeros(T, dtype=np.int64) + path[T - 1] = int(np.argmax(delta[T - 1])) if delta[T - 1].max() > 0 else 0 + for t in range(T - 2, 0, -1): + path[t] = psi[t + 1][path[t + 1]] + path[0] = psi[1][path[1]] + m = min(len(out), T) + out[:m] = path[:m] + return out + + +# --- 3. the beats ---------------------------------------------------------------------- + + +def track_beats(df, periods, alpha=0.9, tightness=4.0): + """TempoTrackV2::calculateBeats: beat positions, in DF frames.""" + n = len(df) + if n == 0 or len(periods) == 0: + return [] + cumscore = np.zeros(n) + backlink = np.full(n, -1, dtype=np.int64) + old = None + txwt = None + for i in range(n): + period = int(periods[min(i // 128, len(periods) - 1)]) + prange_min = -2 * period + if period != old: + old = period + prange_max = int(period / -2) if period >= 0 else 0 + ln = prange_max - prange_min + 1 + j = np.arange(max(ln, 0), dtype=np.float64) + mu = float(period) if period else 1.0 + with np.errstate(divide="ignore", invalid="ignore"): + txwt = np.exp(-0.5 * (tightness * np.log((round(2 * mu) - j) / mu)) ** 2) + txwt = np.nan_to_num(txwt, nan=0.0) + lo = i + prange_min + start = max(0, -lo) + vv, xx = 0.0, 0 + if start < len(txwt): + seg = cumscore[lo + start : lo + len(txwt)] + if len(seg): + sc = txwt[start : start + len(seg)] * seg + k = int(np.argmax(sc)) + if sc[k] > 0: + vv, xx = float(sc[k]), lo + start + k + cumscore[i] = alpha * vv + (1 - alpha) * df[i] + backlink[i] = xx + last = int(periods[-1]) or 1 + tail0 = max(0, n - last) + start = int(np.argmax(cumscore[tail0:])) + tail0 if cumscore[tail0:].max() > 0 else tail0 + start = min(start, n - 1) + beats = [start] + while backlink[beats[-1]] > 0: + b = beats[-1] + if backlink[b] == b: + break + beats.append(int(backlink[b])) + beats.reverse() + return beats + + +# --- 4. the constant tempo (Mixxx BeatUtils) -------------------------------------------- + + +def const_regions(beats, sr): + """BeatUtils::retrieveConstRegions over beat positions in samples.""" + if len(beats) < 2: + return [] + max_err = MAX_PHASE_ERROR * sr + max_err_sum = MAX_PHASE_ERROR_SUM * sr + left, right = 0, len(beats) - 1 + regions = [] + while left < len(beats) - 1: + mean = (beats[right] - beats[left]) / (right - left) + outliers = 0 + ironed = beats[left] + err_sum = 0.0 + i = left + 1 + while i <= right: + ironed += mean + err = ironed - beats[i] + err_sum += err + if abs(err) > max_err: + outliers += 1 + if outliers > MAX_OUTLIERS or i == left + 1: + break + if abs(err_sum) > max_err_sum: + break + i += 1 + if i > right: + border = 0.0 + if right > left + 2: + first = beats[left + 1] - beats[left] + last = beats[right] - beats[right - 1] + border = abs(first + last - 2 * mean) + if border < max_err / 2: + regions.append((beats[left], mean)) + left = right + right = len(beats) - 1 + continue + right -= 1 + regions.append((beats[-1], 0.0)) + return regions + + +def _try_snap(lo, center, hi, fraction): + snap = round(center * fraction) / fraction + return snap if lo < snap < hi else None + + +def round_bpm(lo, center, hi): + """BeatUtils::roundBpmWithinRange.""" + for frac, ok in ((1.0, True), (2.0, center < 85), (2.0 / 3.0, center > 127), (3.0, True), (12.0, True)): + if ok: + s = _try_snap(lo, center, hi, frac) + if s is not None: + return s + return center + + +def const_bpm(regions, sr): + """BeatUtils::makeConstBpm: (bpm, covered) where `covered` is how much of + the beat span the regions the tempo was taken from cover, or (None, 0).""" + if len(regions) < 2: + return None, 0.0 + err = MAX_PHASE_ERROR * sr + mid, longest, beat_len = 0, 0.0, 0.0 + for i in range(len(regions) - 1): + length = regions[i + 1][0] - regions[i][0] + if length > longest: + longest, beat_len, mid = length, regions[i][1], i + if longest == 0 or beat_len <= 0: + return None, 0.0 + nbeats = int(longest / beat_len + 0.5) + bl_min = beat_len - err / nbeats + bl_max = beat_len + err / nbeats + start_i, end_i = mid, mid + + def agrees(i): + length = regions[i + 1][0] - regions[i][0] + nb = int(length / regions[i][1] + 0.5) if regions[i][1] > 0 else 0 + if nb < MIN_REGION_BEATS: + return None + lo = regions[i][1] - err / nb + hi = regions[i][1] + err / nb + if not (lo < beat_len < hi): + return None + return lo, hi + + for i in range(mid): + r = agrees(i) + if r is None: + continue + span = regions[mid + 1][0] - regions[i][0] + lo, hi = max(bl_min, r[0]), min(bl_max, r[1]) + if round(span / lo) != round(span / hi): + continue + nb = round(span / hi) + nbl = span / nb + if bl_min < nbl < bl_max: + longest, beat_len, nbeats = span, nbl, nb + bl_min = beat_len - err / nbeats + bl_max = beat_len + err / nbeats + start_i = i + break + for i in range(len(regions) - 2, mid, -1): + r = agrees(i) + if r is None: + continue + span = regions[i + 1][0] - regions[start_i][0] + lo, hi = max(bl_min, r[0]), min(bl_max, r[1]) + if round(span / lo) != round(span / hi): + continue + nb = round(span / hi) + nbl = span / nb + if bl_min < nbl < bl_max: + longest, beat_len, nbeats = span, nbl, nb + end_i = i + break + bl_min = beat_len - err / nbeats + bl_max = beat_len + err / nbeats + bpm = round_bpm(60.0 * sr / bl_max, 60.0 * sr / beat_len, 60.0 * sr / bl_min) + total = regions[-1][0] - regions[0][0] + covered = (regions[end_i + 1][0] - regions[start_i][0]) / total if total > 0 else 0.0 + return bpm, covered + + +# --- the whole pass ------------------------------------------------------------------------ + + +def analyse(path, seconds=MAX_SECONDS): + """{bpm, confidence, beats (seconds), first_beat, df_mean} for a file, or + None when numpy is missing or the file yields too little to track. + + ``confidence`` is how much of the beat span the constant tempo was taken + from (Mixxx's regions that agree in tempo and phase): 1.0 is a track on a + click from end to end, a few tenths is a beat found here and there. + """ + if np is None: + return None + df, low, bass = detection_function(_decode(path, seconds)) + # Mixxx: drop trailing zeros and the first two values (the noise the + # first frames' empty history reads as an onset). + nz = len(df) + while nz > 0 and df[nz - 1] <= 0: + nz -= 1 + df = df[2:nz] + low = low[2:nz] + bass = bass[2:nz] + if len(df) < DF_HZ * 6: + return None + work = df.copy() + periods = beat_period(work) + # calculateBeats is run on the raw df in Mixxx (the low-pass filter_df is + # applied by the VAMP plugin, not by the analyser): kept as Mixxx has it. + ibeats = track_beats(df, periods) + if len(ibeats) < 4: + return None + sr = SR + beats = [b * STEP + STEP / 2 for b in ibeats] + if len(beats) < MIN_REGION_BEATS: + bpm = 60.0 * (len(beats) - 1) * sr / (beats[-1] - beats[0]) + covered = 0.0 + else: + bpm, covered = const_bpm(const_regions(beats, sr), sr) + if not bpm or not math.isfinite(bpm): + return None + ib = np.asarray(ibeats, dtype=np.float64) + salience = _salience(df, ib) + # THE OCTAVE, ONE WAY ONLY. The Rayleigh weighting centres the period on + # 120 BPM, so a 200 BPM frenchcore kick is tracked on every other beat — + # Mixxx's own well-known hardcore failure. When every HALF beat strikes as + # hard as the beat, in the bass AND across the band, the tempo is twice + # what was tracked. Measured on the eval's arranged records: frenchcore, + # Krach and speedcore (tracked at half) read 0.93-1.06 on both, every + # correctly tracked record stays out of the window (dubstep, the nearest: + # 0.87 on the kick, 0.75 overall; techno's offbeat hats 0.66 overall but + # 0.23 on the kick; metal's double kick 1.82 on the kick, 0.60 overall). Halving is never decided here: a kick on one and three looks like + # it and is right. + # How unequal alternate beats are in the bass (weaker / stronger median): + # near 1 when the kick lands on every beat, low when only every other + # beat carries it (a one-drop tracked at double time reads 0.21). It is + # a witness for the caller, never acted on here. + alternation = 1.0 + if len(ib) >= 8: + on = _peaks(bass, ib) + a, b = float(np.median(on[0::2])), float(np.median(on[1::2])) + alternation = min(a, b) / max(a, b) if max(a, b) > 0 else 1.0 + doubled = False + if len(ib) >= 8 and bpm * 2 <= MAX_BPM: + mids = (ib[1:] + ib[:-1]) / 2 + broad = _ratio(df, ib, mids) + kick = _ratio(low, ib, mids) + if DOUBLE_RATIO <= broad <= DOUBLE_MAX and DOUBLE_RATIO <= kick <= DOUBLE_MAX: + bpm *= 2 + doubled = True + return { + "bpm": round(float(bpm), 3), + "confidence": round(float(covered), 3), + "salience": round(float(salience), 3), + "doubled": doubled, + "alternation": round(alternation, 3), + "beats": [round(b / sr, 4) for b in beats], + } + + +#: The highest tempo the server will serve (analysis.MAX_BPM). +MAX_BPM = 300 +#: Half-beat onsets against on-beat onsets, both bands, past which the tempo +#: is doubled (see `analyse`). +DOUBLE_RATIO = 0.85 +#: ...and no more than this: half beats far STRONGER than the beats mean the +#: tracker's phase sits on an offbeat (hard house's donk took it: the kicks +#: then read 13.7 times the "beats" in the bass), not that the tempo is double. +DOUBLE_MAX = 1.3 + + +def _peaks(x, pos, w=2): + """The onset function's maximum within ±w frames of each position.""" + n = len(x) + out = np.empty(len(pos)) + for k, p in enumerate(pos): + c = int(round(p)) + a, b = max(0, c - w), min(n, c + w + 1) + out[k] = x[a:b].max() if a < b else 0.0 + return out + + +def _ratio(x, on, off): + return float(np.median(_peaks(x, off)) / (np.median(_peaks(x, on)) + 1e-12)) + + +def _salience(df, ib): + """How much the onsets ON the beats stand out from those a quarter of a + beat either side of them (medians). Against the onset function's own + median it read 1.23-1.37 on real limitered masters with an obvious beat, + too close to a pad's 1.15; against the quarter positions a real master + reads 1.16 at the very least, an ambient pad 1.07 and a detuned-saw pad + 1.01 — a beat is where the onsets ARE, not merely where they are loud.""" + if len(ib) < 4: + return 0.0 + p = float(np.median(np.diff(ib))) + off = np.concatenate([ib + p / 4, ib + 3 * p / 4]) + off = off[off < len(df) - 1] + return float(np.median(_peaks(df, ib)) / (np.median(_peaks(df, off)) + 1e-12)) diff --git a/supysonic/web.py b/supysonic/web.py index 07f03c8f..472f5e31 100644 --- a/supysonic/web.py +++ b/supysonic/web.py @@ -74,6 +74,7 @@ def setup_deezer(app): from .webui import genre as _wg _wa.resume_if_interrupted(app) + _wa.reanalyse_if_stale(app) _wg.resume_if_interrupted(app) _wg.relabel_if_stale(app) return app.deezer diff --git a/supysonic/webui/analysis.py b/supysonic/webui/analysis.py index 766ed4a0..5b05e6b6 100644 --- a/supysonic/webui/analysis.py +++ b/supysonic/webui/analysis.py @@ -229,6 +229,63 @@ def start(): threading.Thread(target=start, name="analysis-resume", daemon=True).start() +def reanalyse_if_stale(app): + """Re-measure the library on boot when its verdicts are older than this + release's analysis (``ANALYSIS_VERSION``). + + A version bump means a stored verdict would come out differently today — + version 5 replaced the tempo estimator with a beat tracker, and a library + left on the old figures keeps serving 154 for a 150 BPM track until + somebody happens to press the button. The ordinary (not forced) backfill + re-measures exactly the stale rows and skips the current ones, so this is + the button pressed once, by the release that needs it. Nothing starts when + every row is current, when a run is already going (the interrupted-run + resume above takes precedence), or without ffmpeg. + """ + from ..deezer.analysis import ANALYSIS_VERSION, ffmpeg_available + + def start(): + import time + + time.sleep(RESUME_DELAY + 5) + if not ffmpeg_available(): + return + with app.app_context(): + from ..db import close_connection, open_connection + + try: + open_connection(reuse=True) + stale = ( + TrackAnalysis.select(TrackAnalysis.track) + .where(TrackAnalysis.version < ANALYSIS_VERSION) + .limit(1) + .exists() + ) + total = Track.select().where(Track.last_modification > 0).count() if stale else 0 + except Exception: + logger.debug("Could not look for stale analyses", exc_info=True) + return + finally: + try: + close_connection() + except Exception: + pass + if not stale: + return + with _analysis_lock: + if _analysis_job["running"]: + return + _analysis_job.update( + running=True, started=now().isoformat(), finished=None, force=False, + workers=_workers(), total=total, scanned=0, done=0, skipped=0, + failed=0, error=None, failures=[], failure_reasons={}, + ) + logger.info("Analyses older than version %s: re-measuring the library", ANALYSIS_VERSION) + _run_analysis(app, False, _workers(), None) + + threading.Thread(target=start, name="analysis-upgrade", daemon=True).start() + + @webapi.route("/analysis/backfill", methods=["POST"]) @login_required @admin_required diff --git a/tests/test_beats.py b/tests/test_beats.py new file mode 100644 index 00000000..e44d2137 --- /dev/null +++ b/tests/test_beats.py @@ -0,0 +1,213 @@ +# This file is part of Supysonic. +# Supysonic is a Python implementation of the Subsonic server API. +# +# Distributed under terms of the GNU AGPLv3 license. + +"""The server's beat tracker (supysonic/deezer/beats.py, Mixxx's analyser +ported) and what analysis.py serves from it. + +The signal tests run the real ffmpeg on lavfi-generated material whose answer +is physics — a kick every 60/128 s is 128 BPM — and hold the tracker to the +hundredth, which the envelope estimator it replaced never was (it read 154 +for 150). The rest pins the pure parts (Mixxx's constant-tempo pass, the +reconciliation with Deezer's figure) on numbers worked out by hand. Every +measured value is written next to its assertion. + +Needs numpy and ffmpeg; the CI image job has both, and NS_REQUIRE_FFMPEG=1 +turns a missing ffmpeg into a failure there. +""" + +import os +import shutil +import subprocess +import tempfile +import unittest + +try: + import numpy # noqa: F401 + + HAVE_NUMPY = True +except Exception: # pragma: no cover + HAVE_NUMPY = False + +HAVE_FFMPEG = shutil.which("ffmpeg") is not None +if not HAVE_FFMPEG and os.environ.get("NS_REQUIRE_FFMPEG") == "1": # pragma: no cover + raise RuntimeError("NS_REQUIRE_FFMPEG=1 but ffmpeg is not on PATH") + + +def _kick_expr(bpm, amp=0.8, tail=0.0): + p = 60.0 / bpm + body = f"exp(-9*mod(t\\,{p}))" if not tail else f"({tail}+{1 - tail:.2f}*exp(-14*mod(t\\,{p})))" + return f"{amp}*sin(2*PI*(45+160*exp(-35*mod(t\\,{p})))*mod(t\\,{p}))*{body}" + + +@unittest.skipUnless(HAVE_NUMPY and HAVE_FFMPEG, "needs numpy and ffmpeg") +class TrackerTestCase(unittest.TestCase): + def setUp(self): + self.dir = tempfile.mkdtemp() + + def tearDown(self): + shutil.rmtree(self.dir, ignore_errors=True) + + def file(self, name, expr, seconds=40): + path = os.path.join(self.dir, name + ".flac") + subprocess.run( + ["ffmpeg", "-v", "error", "-nostdin", "-y", "-f", "lavfi", + "-i", f"aevalsrc='{expr}':s=44100:d={seconds}", "-c:a", "flac", path], + check=True, timeout=60, + ) + return path + + def test_a_kick_reads_at_its_exact_tempo(self): + from supysonic.deezer import beats + + # Measured 90.0 / 128.0 / 150.0 / 174.0 exactly, coverage 0.90-0.98. + # The envelope estimator read these to the nearest whole BPM at best. + for bpm in (90, 128, 150, 174): + with self.subTest(bpm=bpm): + r = beats.analyse(self.file(f"k{bpm}", _kick_expr(bpm))) + self.assertAlmostEqual(r["bpm"], bpm, delta=0.05) + self.assertGreater(r["confidence"], 0.8) + + def test_a_tempo_between_whole_numbers_is_kept(self): + # 140.5 is not a whole BPM: Mixxx's rounding only snaps to a simpler + # value that the measurement's own error bars allow. Measured 140.5. + from supysonic.deezer import beats + + r = beats.analyse(self.file("k1405", _kick_expr(140.5), seconds=60)) + self.assertAlmostEqual(r["bpm"], 140.5, delta=0.05) + + def test_hardcore_is_not_read_at_half_tempo(self): + """The Rayleigh weighting centres on 120 BPM, so a 200 BPM kick is + tracked on every other beat (Mixxx's own hardcore failure); the half + beats striking as hard as the beats doubles it back. Measured 200.0, + doubled; a limitered tail (-2.5 dB) changes nothing.""" + from supysonic.deezer import beats + + for name, expr in (("fc", _kick_expr(200)), ("fcl", _kick_expr(200, 0.9, 0.75))): + with self.subTest(name): + r = beats.analyse(self.file(name, expr)) + self.assertAlmostEqual(r["bpm"], 200, delta=0.05) + + def test_an_offbeat_stab_does_not_double_the_tempo(self): + """Hard house's donk: a 300 Hz stab on every offbeat, as loud as the + kick. The tracker's phase may sit on the donk, the kicks then read 13.7 + times the 'beats' in the bass — that is not a doubled tempo (DOUBLE_MAX). + Measured 128.0.""" + from supysonic.deezer import beats + + p = 60.0 / 128 + donk = f"0.5*sin(2*PI*300*t)*exp(-25*mod(t+{p / 2}\\,{p}))" + r = beats.analyse(self.file("donk", _kick_expr(128, 0.5) + "+" + donk)) + self.assertAlmostEqual(r["bpm"], 128, delta=0.05) + self.assertFalse(r["doubled"]) + + def test_the_server_serves_no_tempo_for_a_pad(self): + """Three detuned saws beating against each other: the bass band rises + 84 times a minute (the attack gate lets it through) but no onset lands + where a beat would. Salience measured 1.01 against a floor of 1.12; a + kick reads 3+.""" + from supysonic.deezer import analysis as ana + + pad = "0.3*(2*mod(110*t\\,1)-1)+0.3*(2*mod(110.7*t\\,1)-1)+0.3*(2*mod(109.4*t\\,1)-1)" + self.assertEqual(ana._measure_tempo(self.file("pad", pad, 30)), (None, 0.0, 0.0)) + bpm, conf, _grid = ana._measure_tempo(self.file("k", _kick_expr(128))) + self.assertEqual(bpm, 128.0) + + def test_without_numpy_the_envelope_estimator_still_answers(self): + from supysonic.deezer import analysis as ana + from supysonic.deezer import beats + + path = self.file("k", _kick_expr(128)) + saved = beats.np + beats.np = None + try: + self.assertFalse(beats.available()) + bpm, _conf, _grid = ana._measure_tempo(path) + finally: + beats.np = saved + # The old estimator: within a BPM, as it always was. + self.assertAlmostEqual(bpm, 128, delta=1.0) + + +@unittest.skipUnless(HAVE_NUMPY, "needs numpy") +class ConstantTempoTestCase(unittest.TestCase): + """Mixxx's BeatUtils on beat lists built by hand.""" + + def test_a_jittered_click_comes_out_exact(self): + # 120 BPM at 44.1 kHz with the QM tracker's own ±256-sample quantisation. + from supysonic.deezer import beats + + sr = 44100 + period = 60 * sr / 120 + jitter = [0, 256, -256, 128, -128, 0, 256, -256] + b = [i * period + jitter[i % len(jitter)] for i in range(200)] + bpm, covered = beats.const_bpm(beats.const_regions(b, sr), sr) + self.assertEqual(bpm, 120.0) + self.assertGreater(covered, 0.99) + + def test_a_beatless_bridge_does_not_move_the_tempo(self): + # 64 beats, a stretch where the beats float off the grid, 64 more on the + # SAME grid: the tempo is the grid's, and the regions either side + # are joined across the bridge. + from supysonic.deezer import beats + + sr = 44100 + period = 60 * sr / 128 + b = [i * period for i in range(64)] + b += [64 * period + k * period * 1.13 for k in range(1, 12)] + b += [(96 + i) * period for i in range(64)] + bpm, _covered = beats.const_bpm(beats.const_regions(b, sr), sr) + self.assertEqual(bpm, 128.0) + + def test_rounding_takes_the_simplest_value_the_error_allows(self): + from supysonic.deezer import beats + + self.assertEqual(beats.round_bpm(127.9, 128.04, 128.2), 128.0) + # No whole BPM inside: halves are only tried under 85 BPM... + self.assertEqual(beats.round_bpm(80.4, 80.52, 80.6), 80.5) + # ...and above it a third, then a twelfth — even when a half would also + # fit, as here (140.5 and 140.33 are both inside). + self.assertAlmostEqual(beats.round_bpm(140.30, 140.33, 140.36), 140 + 1 / 3) + self.assertAlmostEqual(beats.round_bpm(140.30, 140.42, 140.55), 140 + 1 / 3) + # Nothing simple inside the range: the measurement itself. + self.assertEqual(beats.round_bpm(140.01, 140.02, 140.03), 140.02) + + +class ReconcileTestCase(unittest.TestCase): + """The tracker's tempo against Deezer's figure (analysis._reconcile_tracked).""" + + def r(self, published, measured, conf=0.9, alternation=None): + from supysonic.deezer import analysis as ana + + return ana._reconcile_tracked(published, measured, conf, alternation) + + def test_the_tracker_is_the_tempo(self): + self.assertEqual(self.r(None, 150.0)[::2], (150.0, "measured")) + # Deezer rounded: the tracker's figure is served. + self.assertEqual(self.r(150.4, 150.0)[0], 150.0) + + def test_deezer_at_twice_moves_the_octave_up(self): + # dnb tracked at 87, Deezer 174: the signal cannot tell, Deezer can. + bpm, _conf, src = self.r(174.0, 87.0) + self.assertEqual((bpm, src), (174.0, "measured+octave")) + + def test_deezer_at_half_is_its_hardcore_listing(self): + # Frenchcore at 200 that Deezer lists at 100, every beat a kick. + self.assertEqual(self.r(100.0, 200.0, alternation=0.95)[0], 200.0) + + def test_deezer_at_half_and_a_half_time_groove_halve(self): + # A one-drop tracked at double time: alternate beats unequal (0.21 + # measured) AND Deezer at half — both witnesses agree. + self.assertEqual(self.r(76.0, 152.0, alternation=0.21)[0], 76.0) + # One witness is not enough. + self.assertEqual(self.r(76.0, 152.0, alternation=None)[0], 152.0) + + def test_any_other_disagreement_follows_confidence(self): + self.assertEqual(self.r(117.0, 175.0, conf=0.9)[0], 175.0) + bpm, conf, src = self.r(117.0, 175.0, conf=0.2) + self.assertEqual((bpm, src), (117.0, "deezer")) + + +if __name__ == "__main__": + unittest.main() diff --git a/tools/tempo_eval.py b/tools/tempo_eval.py index 3e8161e4..f0b4083c 100755 --- a/tools/tempo_eval.py +++ b/tools/tempo_eval.py @@ -10,8 +10,9 @@ detuned-saw pad — which --render also makes with ffmpeg. Any other audio file dropped in DIR/real is measured and listed without a verdict. -A change to supysonic/deezer/analysis.py's tempo pass is an ANALYSIS_VERSION -bump that re-measures every library; run this before and after, and compare. +It scores analysis._measure_tempo, which is the beat tracker (deezer/beats.py) +when numpy is installed and the envelope estimator otherwise. A change to +either is an ANALYSIS_VERSION bump that re-measures every library; run this before and after, and compare. """ import argparse