From 4a0061089c3e8bc26a840cc771e0487ff066114d Mon Sep 17 00:00:00 2001 From: Alexander Abramov Date: Wed, 17 Jun 2026 15:33:40 +0200 Subject: [PATCH] fix(diarization): upgrade bootstrap to pyannote.audio>=4 (fixes #3) whisply 0.14.1 hard-pins pyannote.audio==3.4.0 / whisperx==3.7.8, but the diarization path loads pyannote/speaker-diarization-community-1 with `token=`, which require pyannote.audio>=4.0. Fresh installs therefore failed with `unexpected keyword argument 'token'` (and `'plda'` after patching to use_auth_token). The author's venv was manually on the 4.0.4/3.8.6 baseline, so it never surfaced locally. - transcribe.sh: bootstrap now upgrades to whisperx>=3.8.6 + pyannote.audio>=4.0 and self-heals pre-fix venvs (idempotent major-version probe). - transcribe_whisperx.py: lazy `import whisperx` (module now importable without the ML stack) + preflight version guard that emits an actionable message and skips diarization gracefully on pyannote<4 instead of a cryptic TypeError. - tests: unit-test the version guard (runs in the pytest+numpy-only CI). - setup.md / CHANGELOG: document the baseline + troubleshooting. - docs/adr-faster-whisper.md: faster-whisper is already the whisperX engine; a direct integration is rejected. --- CHANGELOG.md | 8 ++++++ docs/adr-faster-whisper.md | 47 +++++++++++++++++++++++++++++++ references/setup.md | 16 ++++++++++- scripts/transcribe.sh | 17 +++++++++++ scripts/transcribe_whisperx.py | 45 ++++++++++++++++++++++++++++- tests/test_transcribe_whisperx.py | 33 ++++++++++++++++++++++ 6 files changed, 164 insertions(+), 2 deletions(-) create mode 100644 docs/adr-faster-whisper.md create mode 100644 tests/test_transcribe_whisperx.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 99cf84a..3c9f018 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,14 @@ versioning follows [Semantic Versioning](https://semver.org/). ## [Unreleased] +### Fixed +- **Diarization failed on fresh installs** with `TypeError: ... unexpected keyword argument 'token'` + (or `'plda'`) — whisply pins `pyannote.audio==3.4.0`, but the `community-1` pipeline + `token=` + need pyannote.audio ≥ 4.0. The bootstrap now upgrades to the tested `whisperx>=3.8.6` / + `pyannote.audio>=4.0` baseline and self-heals venvs created before this fix; a preflight guard + emits an actionable message (instead of a cryptic crash) if an older pyannote is still present. + ([#3](https://github.com/AlexanderAbramovPav/scriba/issues/3)) + ## [0.1.0] — 2026-06-06 First public release. diff --git a/docs/adr-faster-whisper.md b/docs/adr-faster-whisper.md new file mode 100644 index 0000000..b4a7e11 --- /dev/null +++ b/docs/adr-faster-whisper.md @@ -0,0 +1,47 @@ +# ADR: Adopting faster-whisper as a direct ASR backend + +- **Status:** Rejected (no action) — 2026-06-17 +- **Context:** SYSTRAN/faster-whisper (CTranslate2 reimplementation of Whisper) was proposed as a + way to speed up / improve scriba's transcription. + +## Decision + +**Do not add faster-whisper as a direct backend. scriba already uses it — transitively, through +whisperX — and that is the right level of integration.** + +## Why + +1. **It's already the engine.** whisperX's ASR path *is* faster-whisper. The skill venv ships + `faster-whisper 1.2.1` + `ctranslate2 4.7.2` as whisperX dependencies, and `whisperx.load_model()` + instantiates faster-whisper's CT2 `WhisperModel`. `scripts/transcribe_whisperx.py` already knows + this (it passes `hotwords` only when faster-whisper's signature supports it). So scriba's default + accuracy path runs on faster-whisper today. + +2. **A direct integration buys nothing on quality/speed.** Same CT2 engine and weights → identical + decode speed and WER. There is no faster path hiding underneath. + +3. **It would *cost* accuracy.** scriba's diarization, word↔speaker reconciliation, per-word + confidence and enrollment (C1/C2/C3) all depend on whisperX's **wav2vec2 forced alignment** for + word timestamps, which is more accurate than faster-whisper's native DTW timestamps. Going + "direct" means re-adding `whisperx.align` anyway, so the whisperX dependency stays — for less. + +4. **faster-whisper's headline edge doesn't apply here.** Its `BatchedInferencePipeline` (v1.1+) + speeds up batched VAD-chunked decoding, but scriba intentionally runs `batch_size=1` so each + segment streams to the live progress ticker as it's decoded. Batching would trade that UX away. + +5. **More surface to maintain.** scriba already has three ASR routes (whisperX-CPU, whisply/MLX for + `--fast`, GigaAM-RU opt-in). A fourth, redundant one is net negative. + +## The one good adjacent idea + +The genuinely useful thought behind "adopt faster-whisper directly" is **decoupling ASR from the +pyannote-pinned diarization stack** — the coupling that caused [issue #3](https://github.com/AlexanderAbramovPav/scriba/issues/3), +where whisply's `pyannote.audio==3.4.0` pin broke the community-1 pipeline. That is solved by +**pinning pyannote independently** (the bootstrap now upgrades to `pyannote.audio>=4.0`), not by +swapping the ASR engine. + +## Revisit if + +- whisperX becomes unmaintained or stops tracking new Whisper / pyannote releases, **or** +- we decide to drop whisperX entirely — in which case faster-whisper (ASR) + a standalone wav2vec2 + aligner + independently-pinned pyannote (diarization) would be the clean decomposition. diff --git a/references/setup.md b/references/setup.md index e6c5f8f..0d388e0 100644 --- a/references/setup.md +++ b/references/setup.md @@ -22,7 +22,16 @@ The wrapper bootstraps an isolated venv on first use: bash skills/scriba/scripts/transcribe.sh --bootstrap ``` This creates `skills/scriba/.venv` (Python 3.12 via `uv`) and installs -`whisply[mlx]` (pulls torch, torchaudio, pyannote, whisperX, and MLX). +`whisply[mlx]` (pulls torch, torchaudio, pyannote, whisperX, and MLX), then **upgrades the +diarization stack to the tested baseline** `whisperx>=3.8.6` + `pyannote.audio>=4.0`. + +> **Why the upgrade step.** whisply 0.14.1 hard-pins `pyannote.audio==3.4.0` / `whisperx==3.7.8`, +> but the diarization path loads `pyannote/speaker-diarization-community-1` with `token=` — both +> require pyannote.audio ≥ 4.0 (3.x uses `use_auth_token` and lacks community-1's PLDA clustering). +> Without the upgrade, diarization fails with a `token` / `plda` `TypeError` (issue #3). The +> bootstrap is **idempotent and self-healing**: re-running `--bootstrap` on a venv created before +> this fix detects the stale pyannote and upgrades it in place (transcription is unaffected; you +> just lose speaker labels until the upgrade runs). ## Engine / modes @@ -76,6 +85,11 @@ so either spike outcome works. No code change needed either way. ## Troubleshooting +- **Diarization fails with `unexpected keyword argument 'token'` or `'plda'`** (issue #3): your + venv is on **pyannote.audio 3.x** (whisply's pin), but the community-1 pipeline needs ≥ 4.0. + Re-run `bash skills/scriba/scripts/transcribe.sh --bootstrap` to self-heal, or upgrade manually: + `uv pip install --python skills/scriba/.venv/bin/python -U "whisperx>=3.8.6" "pyannote.audio>=4.0"`. + Transcription itself is unaffected — only speaker labels are skipped until the upgrade runs. - **OpenMP / `libomp` crash** (`OMP: Error #15` or a hard abort): the wrapper already exports `KMP_DUPLICATE_LIB_OK=TRUE`. If running whisply manually, prefix the same env var. - **ctranslate2 / torch version conflict** (per whisply docs): ensure `torch==2.8.0` and diff --git a/scripts/transcribe.sh b/scripts/transcribe.sh index dcc5a63..b41394d 100755 --- a/scripts/transcribe.sh +++ b/scripts/transcribe.sh @@ -9,9 +9,25 @@ HF_TOKEN_FILE="${HF_TOKEN_FILE:-$HOME/.config/scriba/hf_token}" CALIB_FILE="${CALIB_FILE:-$HOME/.config/scriba/calibration.json}" ETA_HELPER="$SKILL_DIR/scripts/eta_helper.py" +# Force the diarization path onto the tested baseline (pyannote.audio 4.x + whisperX 3.8.6). +# whisply 0.14.1 hard-pins `pyannote.audio==3.4.0` and `whisperx==3.7.8`, but our wrapper loads +# `pyannote/speaker-diarization-community-1` with `token=` — both require pyannote.audio>=4.0 +# (3.x uses `use_auth_token` and lacks community-1's PLDA clustering → the `token`/`plda` +# TypeErrors in issue #3). Upgrading whisperx pulls pyannote 4.x transitively; we name both for +# clarity. Idempotent: a no-op once the venv is already on 4.x. See references/setup.md. +ensure_diar_baseline() { + # Fast major-version probe via importlib.metadata (no torch import). Exit 0 = already >=4. + if "$PY" -c "import importlib.metadata as m,sys; sys.exit(0 if int(m.version('pyannote.audio').split('.')[0])>=4 else 1)" 2>/dev/null; then + return 0 + fi + echo "Upgrading diarization stack to whisperx>=3.8.6 + pyannote.audio>=4.0 (overrides whisply's stale pins) ..." >&2 + uv pip install --python "$PY" --upgrade "whisperx>=3.8.6" "pyannote.audio>=4.0" +} + bootstrap() { if [[ -x "$WHISPLY" ]]; then echo "whisply already installed at $WHISPLY" >&2 + ensure_diar_baseline # self-heal venvs bootstrapped before this fix (issue #3) return 0 fi command -v uv >/dev/null || { echo "ERROR: uv not found on PATH" >&2; exit 1; } @@ -19,6 +35,7 @@ bootstrap() { uv venv --python 3.12 "$VENV" echo "Installing whisply[mlx] (pulls torch/pyannote/whisperX; takes a few minutes) ..." >&2 uv pip install --python "$PY" "whisply[mlx]" + ensure_diar_baseline echo "Bootstrap done. If you hit an OpenMP crash, KMP_DUPLICATE_LIB_OK=TRUE is already set by this script." >&2 } diff --git a/scripts/transcribe_whisperx.py b/scripts/transcribe_whisperx.py index c2e633b..2189ffc 100644 --- a/scripts/transcribe_whisperx.py +++ b/scripts/transcribe_whisperx.py @@ -49,7 +49,10 @@ logging.getLogger("lightning.pytorch").setLevel(logging.ERROR) logging.getLogger("lightning_fabric").setLevel(logging.ERROR) -import whisperx # noqa: E402 +# whisperx is imported lazily inside main() (not at module top) so this module can be imported +# for unit tests without the heavy ML stack installed — CI installs only pytest+numpy and never +# exercises the ML path. The warning filters above still run at import time, before main() +# triggers `import whisperx`, preserving the silence-before-load ordering. class TextProgressHook: @@ -127,7 +130,32 @@ def __exit__(self, *exc) -> None: self._thread.join(timeout=1.0) +def diarization_backend_ok(version_str: str) -> tuple[bool, str]: + """Is the installed pyannote.audio new enough for the community-1 diarization pipeline? + + scriba loads `pyannote/speaker-diarization-community-1` with `token=` — both require + pyannote.audio >= 4.0. On 3.x, `from_pretrained` takes `use_auth_token` (not `token`) and + `SpeakerDiarization` has no `plda` clustering, so the call dies with a cryptic TypeError + (issue #3). whisply pins pyannote 3.4.0, so a stale venv can reach here; return an actionable + message instead of letting the doomed call throw. Pure (no imports) so it's unit-testable. + """ + try: + major = int(str(version_str).split(".")[0]) + except (ValueError, AttributeError, IndexError): + # Unparseable version → don't block; let the call proceed and the outer except handle it. + return True, "" + if major >= 4: + return True, "" + return ( + False, + f"pyannote.audio {version_str} is too old for the community-1 diarization model " + "(needs >=4.0); re-run `bash scripts/transcribe.sh --bootstrap` to self-heal, or " + "`uv pip install -U 'whisperx>=3.8.6' 'pyannote.audio>=4.0'`", + ) + + def main() -> int: + import whisperx # heavy; deferred so this module imports cleanly for unit tests (see top note) ap = argparse.ArgumentParser() ap.add_argument("--audio", required=True, help="path to 16kHz mono WAV") ap.add_argument("--output", required=True, help="path to write result JSON") @@ -244,7 +272,20 @@ def main() -> int: # enrichment below is well-defined even when diarization is skipped/no token. overlap_turns: list = [] + # Preflight: community-1 + token= need pyannote.audio>=4.0. whisply pins 3.4.0, so a stale + # install would otherwise die deep inside from_pretrained() with the cryptic token/plda + # TypeError from issue #3. Read the version cheaply (importlib.metadata, no torch import) and + # surface an actionable message. The bootstrap self-heals; this guard is the belt to that braces. + diar_ready, diar_msg = True, "" if args.annotate and args.hf_token: + import importlib.metadata as _ilm + try: + _pa_ver = _ilm.version("pyannote.audio") + except _ilm.PackageNotFoundError: + _pa_ver = "0" + diar_ready, diar_msg = diarization_backend_ok(_pa_ver) + + if args.annotate and args.hf_token and diar_ready: log("diarizing (pyannote/speaker-diarization-community-1)") try: # We bypass whisperX's `DiarizationPipeline.__call__` because it doesn't pass @@ -337,6 +378,8 @@ def main() -> int: log(f"WARN: enrollment skipped ({type(e).__name__}: {e})") except Exception as e: log(f"WARN: diarization failed ({type(e).__name__}: {e}); continuing without speaker labels") + elif args.annotate and args.hf_token and not diar_ready: + log(f"WARN: {diar_msg}; continuing without speaker labels") elif args.annotate and not args.hf_token: log("WARN: --annotate requested but no --hf-token; skipping diarization") diff --git a/tests/test_transcribe_whisperx.py b/tests/test_transcribe_whisperx.py new file mode 100644 index 0000000..c547de5 --- /dev/null +++ b/tests/test_transcribe_whisperx.py @@ -0,0 +1,33 @@ +import pathlib, sys +HERE = pathlib.Path(__file__).resolve().parent +sys.path.insert(0, str(HERE.parent / "scripts")) +# Imports cleanly without the ML stack: `import whisperx` is deferred into main(), and CI installs +# only pytest+numpy. This guards the regression in issue #3 (diarization died on pyannote 3.x). +import transcribe_whisperx as tw + + +def test_pyannote_4x_is_ok(): + for v in ("4.0.0", "4.0.4", "4.1.0", "5.0.0"): + ok, msg = tw.diarization_backend_ok(v) + assert ok is True + assert msg == "" + + +def test_pyannote_3x_is_rejected_with_actionable_message(): + # whisply pins pyannote.audio==3.4.0 → the version that broke for the issue reporter. + for v in ("3.4.0", "3.1.1", "3.0.0"): + ok, msg = tw.diarization_backend_ok(v) + assert ok is False + # Message must name the package and the required floor so the user can self-serve. + assert "pyannote.audio" in msg + assert ">=4.0" in msg + assert v in msg + + +def test_unparseable_version_does_not_block(): + # Defensive: a weird/missing version string must not hard-fail the run — let the real call + # (and its surrounding except) decide. Returns ok=True so we don't skip diarization spuriously. + for v in ("", "unknown", None): + ok, msg = tw.diarization_backend_ok(v) + assert ok is True + assert msg == ""