[chazmaniandinkle]

Slice 1 of RFC-0002 (#158): the shared ear library and the inbound gate. Stacked on rfc/0002-shared-ear-music-modality, so the diff against main also shows the RFC commit; the slice itself is one commit.

What this adds

• ear/ package. ear.read(samples, sr) -> EarReading for a chunk, ear.read_clip(...) for a whole file or utterance (long files are sampled in 2 s windows, Silero still scans the whole clip).
  • kind in {speech, music, noise, silence}, a confidence for each (they sum to 1), plus rms_db, peak_db, voiced_fraction, f0_hz, median spectral flatness, band_energy (sub/low/mid/high), 12-bin chroma, chroma_entropy, and Silero's speech fraction/segments.
  • Speech evidence is Silero via vad.detect_speech. Music vs noise is spectral flatness plus pitch-class concentration, plain numpy FFTs (no librosa dependency, no per-frame Python loops).
  • If Silero cannot load, the reading says so (vad_available=False, note vad_unavailable) and nothing is called speech.
• ear.EarGate (a modality.Gate). VoiceGate now subclasses it, so every caller that goes through bus.perceive is gated: only kind=speech reaches STT. If the ear itself errors, VoiceGate falls back to the old plain-Silero check; a bare EarGate fails closed.
• Non-speech never produces a transcript. ModalityBus.perceive_outcome() returns (event, non_speech); non_speech is a CognitiveEvent with empty content, metadata = {non_speech: True, kind, confidences, reading, decoder: None}, and a modality.non_speech bus event is emitted. perceive() keeps its old contract (returns None).
• HTTP: /v1/transcribe runs the gate before Whisper and returns {"transcript": "", "non_speech": true, "kind": ..., "confidences": ...} for music/noise/silence. /v1/bus/perceive returns status: "non_speech" with the kind.

Tests

tests/test_ear.py (60 tests), generated fixtures only:

• speech: 3 short TTS clips (Kokoro, 16 kHz, ~170 KB each) in tests/fixtures/ear/, regenerable with scripts/make_ear_speech_fixtures.py
• music: numpy square/triangle chord progressions with kick/snare/noise-hat drums, no vocals: two 8-bar instrumentals at 174 and 180 BPM, a slow triangle pad progression, a square-wave melody
• noise: white, pink. silence: digital zero, -70 dBFS room tone

Checks: every fixture classified correctly as a whole clip, in 2 s chunks, and in 0.5 s chunks (see caveat below); music and noise never read as speech at four gain levels; speech under a noise bed is still speech; a pure 440 Hz tone gives the right level, pitch class (A) and f0; int16 / stereo / 48 kHz / empty / NaN inputs; the gate never calls the (spy) STT decoder on any music/noise/silence fixture, including the 174 BPM instrumental delivered as a WAV file; /v1/transcribe and /v1/bus/perceive behaviour; ear-error fallbacks; no-Silero degradation; speed floor of 5x real time.

Suite before/after (same venv, run on the RFC base and on this branch):

	passed	failed	skipped
base (rfc/0002-...)	1118	2	1
this branch	1178	2	1

The 2 failures are the same on both: tests/test_websocket_transport.py::test_websocket_library_installed and ::test_real_websocket_upgrade_through_uvicorn. 60 new tests, 0 new failures. (Both runs also end in a Bus error: 10 at interpreter exit after the pytest summary; same on base.) tests/test_transcribe_endpoint.py got a one-line bypass of the new gate in its two fixtures, because those tests post a silent WAV to check request handling and the ear correctly stops silence before STT.

Speed

python scripts/bench_ear.py, warm, median of 5 passes, Apple Silicon, machine lightly loaded (load average ~6), Silero on CPU through torch:

input	chunk	ms per chunk	x real time
speech (TTS)	0.5 s	1.50	334
speech (TTS)	2.0 s	5.54	361
music (174 BPM)	0.5 s	1.52	330
music (174 BPM)	2.0 s	5.53	362
noise (pink)	0.5 s	1.50	334
noise (pink)	2.0 s	5.45	367

A 69 s instrumental through read_clip takes about 170 ms. The same benchmark taken while the full test suite was running on the same machine (load average ~70) gave 3.5-3.9 ms per 0.5 s chunk (~130x), so the margin over the 5x target holds under heavy load.

Things to know

• RFC §4 numbers. The prototype (RMS, pYIN, chroma, flatness) measured 53 ms per 0.5 s chunk; this is about 1.5 ms. I did not profile the prototype, so I can't say how much of the gap is pYIN. Here pitch is a normalised-autocorrelation voicing/f0 estimate on the FFT already computed, not pYIN, and Silero adds about 1 ms. Slice 5 (hum to notes) will want a real pitch tracker.
• Short chunks. At 0.5 s a pause inside speech reads as silence (a quiet sentence-final syllable read as music until the silence centre was raised to -40 dBFS). The test holds the line at: never noise, >= 70% speech. Callers that need a verdict on a stream should use utterance-length audio or smooth over a few chunks.
• Silence floor is a level threshold (-40 dBFS centre), so quiet music reads as silence: in a gain sweep on the 174 BPM fixture (0.5 s chunks), music at about -38 dBFS RMS and louder is read as music; at -42 dBFS and quieter every chunk reads as silence. Speech from the TTS fixtures sits around -30 dBFS or louder per 2 s window.
• Not implemented (later slices): true_peak_db, key, tempo, onsets, end-of-turn probability, anomalies, hallucination-risk flag, the outbound ear.
• Not gated: channels.py's browser-channel _transcribe calls mlx_whisper directly and bypasses the bus (the browser runs Silero client-side). Music through that path still reaches Whisper.
• Pre-existing bug found, fixed here as a side effect: the old VoiceGate.check defaulted to sample_width=2 (int16), but inbound.py calls bus.perceive with float32 bytes and no sample_width. The old gate therefore reinterpreted float32 as int16 (garbage at twice the length). EarGate defaults to float32 and also parses RIFF/WAV payloads.
• vendor/pipecat_vad/data/silero_vad.onnx is committed as a 0-byte file on this base, so the vendored ONNX path (voice_confidence) cannot load from a clean checkout. The ear uses the torch-hub Silero path in vad.py, which is what detect_speech already did. Not touched here.
• Behaviour change: /v1/vad and the MCP vad_check now report has_speech: false for music/noise, because they use the same gate.