Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 4 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Requires React Native **0.75 or later** with the New Architecture enabled.

## How it works

Audio is captured through platform-native APIs (AVAudioEngine on iOS, Oboe on Android) and fed into a lock-free SPSC ring buffer. A C++ worker thread drains the buffer in fixed-size frames, runs the signal through a preprocessing pipeline, then through an ensemble of three pitch detectors (YIN, PYIN, cepstrum). The ensemble votes for the best estimate and fires an `onPitch` event on the JS thread.
Audio is captured through platform-native APIs (AVAudioEngine on iOS, Oboe on Android) and fed into a lock-free SPSC ring buffer. A C++ worker thread drains the buffer in fixed-size frames, runs the signal through a preprocessing pipeline, then through two complementary pitch detectors: probabilistic YIN (pYIN) as the precise primary estimator and a cepstrum detector as an independent corroborator. Their fused result fires an `onPitch` event on the JS thread.

The audio callback allocates nothing at runtime — all working buffers are pre-allocated during initialization.

Expand Down Expand Up @@ -196,10 +196,9 @@ The shared C++ core (`cpp/`) compiles as a static library on both platforms.
|---|---|---|
| High-pass filter | `BiquadHpf` | Direct-Form II Transposed, 70 Hz cutoff (configurable), Q 0.707 |
| Windowing | `Window` | Hann window, precomputed coefficients |
| Pitch detection | `EnsembleSelector` | Runs YIN, PYIN, and cepstrum; votes by agreement within 1 semitone |
| — detector 1 | `YinPitchDetector` | YIN with parabolic interpolation |
| — detector 2 | `PyinPitchDetector` | Probabilistic YIN; prunes harmonic aliases |
| — detector 3 | `CepstrumPitchDetector` | Real cepstrum via radix-2 FFT; SNR-based confidence |
| Pitch detection | `DetectorFusion` | Fuses a precise primary detector with a coarse corroborator (agree/clash/solo rules within 1 semitone) |
| — primary | `PyinPitchDetector` | Probabilistic YIN (Beta-distributed threshold prior, per-candidate probability mass) |
| — corroborator | `CepstrumPitchDetector` | Real cepstrum via radix-2 FFT; peak-vs-rival prominence confidence |
| Note mapping | `NoteMapper` | Hz → MIDI, note name, octave, cents deviation |
| SNR estimation | `SnrEstimator` | Signal RMS vs. noise-floor EMA |
| Post-processing | `PostProcessor` | Median-5 filter, EMA smoothing (configurable), note-transition hysteresis (configurable) |
Expand Down
3 changes: 1 addition & 2 deletions cpp/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ set(CMAKE_CXX_STANDARD_REQUIRED ON)

add_library(tuner_engine_core STATIC
src/NoteMapper.cpp
src/YinPitchDetector.cpp
src/Window.cpp
src/BiquadHpf.cpp
src/SnrEstimator.cpp
Expand All @@ -18,7 +17,7 @@ add_library(tuner_engine_core STATIC
src/AudioFrameDispatcher.cpp
src/CepstrumPitchDetector.cpp
src/PyinPitchDetector.cpp
src/EnsembleSelector.cpp
src/DetectorFusion.cpp
src/StringMatcher.cpp
)

Expand Down
8 changes: 4 additions & 4 deletions cpp/include/CepstrumPitchDetector.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,15 @@ class CepstrumPitchDetector : public IPitchDetector {

void reset() override {}
void setFrequencyRange(float minHz, float maxHz) override;
void setThreshold(float threshold) override;
void setProminenceThreshold(float threshold);

private:
float sampleRate_;
int frameSize_;

float minHz_ = 60.0f;
float maxHz_ = 1200.0f;
float threshold_ = 0.10f; // minimum peak prominence to be considered voiced
float minHz_ = 60.0f;
float maxHz_ = 1200.0f;
float prominenceThreshold_ = 0.10f; // minimum peak prominence to be considered voiced

std::vector<float> hann_;
std::vector<std::complex<float>> fftBuf_;
Expand Down
32 changes: 32 additions & 0 deletions cpp/include/DetectorFusion.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#pragma once

#include "IPitchDetector.hpp"
#include <memory>

// Fuses two complementary pitch detectors into a single estimate:
// - primary (pYIN): precise time-domain frequency, the pitch we report
// - corroborator (cepstrum): coarser spectral estimate, independent evidence
//
// Their frequencies are never averaged — the two estimates have very different
// resolutions, so mixing them would only blur the precise one. The corroborator
// exists to confirm or contest the primary:
// - agree → primary's frequency, confidences combined as independent evidence
// - clash → still the primary's frequency, but its confidence dampened in
// proportion to the corroborator's conviction
// - solo → the only voiced result, mildly dampened (less if it's the primary)
class DetectorFusion : public IPitchDetector {
public:
DetectorFusion(std::unique_ptr<IPitchDetector> primary,
std::unique_ptr<IPitchDetector> corroborator);

DetectorResult detect(const float* frame, int frameLength, float sampleRate) override;

void reset() override;
void setFrequencyRange(float minHz, float maxHz) override;

private:
std::unique_ptr<IPitchDetector> primary_;
std::unique_ptr<IPitchDetector> corroborator_;

static float semitoneDistance(float frequencyA, float frequencyB);
};
28 changes: 0 additions & 28 deletions cpp/include/EnsembleSelector.hpp

This file was deleted.

5 changes: 2 additions & 3 deletions cpp/include/IPitchDetector.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,11 @@ class IPitchDetector {
public:
virtual ~IPitchDetector() = default;

// Detect pitch from a pre-filtered, windowed frame of length n at the given sample rate.
// Detect pitch from a pre-filtered frame of length n at the given sample rate.
virtual DetectorResult detect(const float* frame, int n, float sampleRate) = 0;

// Reset any inter-frame state (Viterbi, smoothing, etc.).
// Reset any inter-frame state (pitch tracking, smoothing, etc.).
virtual void reset() {}

virtual void setFrequencyRange(float minHz, float maxHz) = 0;
virtual void setThreshold(float threshold) = 0;
};
45 changes: 30 additions & 15 deletions cpp/include/PyinPitchDetector.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -3,33 +3,48 @@
#include "IPitchDetector.hpp"
#include <vector>

// Probabilistic YIN (PYIN) — unlike plain YIN, it collects every CMND local minimum
// below threshold and picks the most probable one rather than the first.
// This eliminates the octave errors that occur when the first minimum corresponds
// to a harmonic period rather than the fundamental.
// Probabilistic YIN (pYIN, Mauch & Dixon 2014) — frame-wise stage.
//
// Plain YIN picks the first CMND minimum below one fixed threshold, so a single
// unlucky threshold choice produces octave errors. pYIN instead treats the
// threshold as a random variable with a Beta(2, 18) prior: every CMND local
// minimum accumulates the prior mass of all thresholds at which YIN would have
// picked it. The candidate with the most mass wins.
//
// The paper's HMM/Viterbi tracking stage is replaced by two lighter mechanisms
// suited to real-time tuning: a small selection bonus for candidates near the
// previous frame's pitch (here), and the pipeline's PostProcessor
// (median + EMA + hysteresis) downstream.
class PyinPitchDetector : public IPitchDetector {
public:
PyinPitchDetector(float sampleRate, int frameSize);

DetectorResult detect(const float* frame, int n, float sampleRate) override;
DetectorResult detect(const float* frame, int frameLength, float sampleRate) override;

void reset() override {}
void reset() override;
void setFrequencyRange(float minHz, float maxHz) override;
void setThreshold(float threshold) override;

private:
struct PitchCandidate {
int lag; // integer sample lag of the CMND local minimum
float cmndDepth; // CMND value at the minimum (lower = more periodic)
float probability; // pYIN mass accumulated across all thresholds
};

float sampleRate_;
int frameSize_;

float minHz_ = 60.0f;
float maxHz_ = 1200.0f;
float threshold_ = 0.25f; // wider than YIN — captures all plausible candidates
float minFrequencyHz_ = 60.0f;
float maxFrequencyHz_ = 1200.0f;

std::vector<float> squaredDifference_; // YIN step 2, indexed by lag
std::vector<float> normalizedDifference_; // CMND, YIN step 3, indexed by lag
std::vector<PitchCandidate> candidates_; // pre-allocated, reused each frame

std::vector<float> diff_;
std::vector<float> cmnd_;
std::vector<float> thresholdLevels_; // equally spaced YIN thresholds in (0, 1]
std::vector<float> thresholdPriors_; // Beta(2, 18) weight per level, sums to 1

struct Candidate { int tau; float prob; };
std::vector<Candidate> candidates_; // pre-allocated, reused each frame
float previousPitchHz_ = 0.0f; // continuity-bonus target; 0 = no history

float parabolicInterpolation(int tau) const;
float refineLagByParabola(int lag) const;
};
37 changes: 0 additions & 37 deletions cpp/include/YinPitchDetector.hpp

This file was deleted.

56 changes: 34 additions & 22 deletions cpp/src/CepstrumPitchDetector.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ void CepstrumPitchDetector::setFrequencyRange(float minHz, float maxHz) {
maxHz_ = maxHz;
}

void CepstrumPitchDetector::setThreshold(float threshold) {
threshold_ = threshold;
void CepstrumPitchDetector::setProminenceThreshold(float threshold) {
prominenceThreshold_ = threshold;
}

DetectorResult CepstrumPitchDetector::detect(const float* frame, int n, float sampleRate) {
Expand Down Expand Up @@ -63,17 +63,23 @@ DetectorResult CepstrumPitchDetector::detect(const float* frame, int n, float sa
const int tauMax = std::min(frameSize_ / 2 - 1, static_cast<int>(sr / minHz_));
if (tauMin >= tauMax) return DetectorResult{};

// Find the quefrency peak
float maxVal = -1e30f;
// Find the tallest strict local maximum in the quefrency range. The global
// maximum often sits on the range boundary where the low-quefrency envelope
// tail is still decaying — that is not a periodicity peak, and accepting it
// produced confident junk detections pinned to the frequency-range edge.
float maxVal = -1e30f;
int bestTau = -1;
for (int q = tauMin; q <= tauMax; ++q) {
for (int q = tauMin + 1; q < tauMax; ++q) {
const float v = fftBuf_[q].real();
if (v > maxVal) { maxVal = v; bestTau = q; }
if (v > fftBuf_[q - 1].real() && v >= fftBuf_[q + 1].real() && v > maxVal) {
maxVal = v;
bestTau = q;
}
}
if (bestTau < 0) return DetectorResult{};

const float prominence = peakProminence(tauMin, tauMax, bestTau);
if (prominence < threshold_) return DetectorResult{};
if (prominence < prominenceThreshold_) return DetectorResult{};

// Sub-sample refinement via parabolic interpolation
float peakTau = static_cast<float>(bestTau);
Expand All @@ -94,23 +100,29 @@ DetectorResult CepstrumPitchDetector::detect(const float* frame, int n, float sa
float CepstrumPitchDetector::peakProminence(int tauMin, int tauMax, int peakTau) const {
const float peak = fftBuf_[peakTau].real();

// RMS and mean of the quefrency range (excluding the peak itself)
// Mean of the quefrency range the baseline the peaks rise above.
float sum = 0.0f;
float sumSq = 0.0f;
const int count = tauMax - tauMin + 1;
for (int q = tauMin; q <= tauMax; ++q) {
const float v = fftBuf_[q].real();
sum += v;
sumSq += v * v;
sum += fftBuf_[q].real();
}
const float mean = sum / static_cast<float>(count);
const float rms = std::sqrt(sumSq / static_cast<float>(count));

// Confidence: how much the peak exceeds the RMS level.
// A cepstrum with no clear periodicity (pure sine) has peak ≈ RMS → conf ≈ 0.
// A strong harmonic signal has peak >> RMS → conf → 1.
if (rms < kEps) return 0.0f;
const float snr = (peak - mean) / rms;
// Map snr range [0, 5] → [0, 1]; cap both sides
return std::max(0.0f, std::min(1.0f, snr / 5.0f));
const float mean = sum / static_cast<float>(count);

const float peakHeight = peak - mean;
if (peakHeight < kEps) return 0.0f;

// Highest rival: the tallest value outside the peak's own ±10% neighbourhood.
// A truly harmonic signal has one dominant quefrency peak; a pure sine or
// noise produces several comparable peaks, so the rival nearly matches the
// peak and the prominence collapses. Scale-invariant by construction —
// no fixed SNR scale to saturate.
const int exclusionRadius = std::max(1, peakTau / 10);
float rival = -1e30f;
for (int q = tauMin; q <= tauMax; ++q) {
if (std::abs(q - peakTau) <= exclusionRadius) continue;
rival = std::max(rival, fftBuf_[q].real());
}
const float rivalHeight = std::max(0.0f, rival - mean);

return std::max(0.0f, std::min(1.0f, 1.0f - rivalHeight / peakHeight));
}
Loading
Loading