Repository navigation
Conversation
wassgha
force-pushed
the
fix-alignment-forced
branch
from
August 2, 2026 21:09
bd9f064 to
69c3d3e
Compare
# Conflicts: # workers/transcription.worker.ts
The CTC aligner loaded itself with bare from_pretrained calls, so its 86 MB download happened behind a static "Aligning words…" with no bar — indistinguishable from a hang on a cold cache. Register it in the existing ModelManager instead, so it shares the download progress, cache labeling and unload-on-GPU-loss path with the ASR models. It needs the processor, model and tokenizer separately rather than a pipeline(), so it is a plain ModelDefinition wired to transformersProgress rather than transformersModel(). The aligner warms in the background while Whisper runs, so its bytes must not fight the "Transcribing…" line: speech models always own the progress line, and the aligner only claims it once transcription is done and we are actually waiting on it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
# Conflicts: # workers/transcription.worker.ts
wav2vec2 is not in the same family as the ASR models — it is an acoustic aligner, not a transcriber, and not something the user picks. But the deciding factor is lifecycle, not taxonomy. The ASR registry exists under a WebGPU→WASM fallback policy whose recovery step is unloadAll(). The aligner never asks for a device, so transformers.js runs it on WASM (DEFAULT_DEVICE), where a lost GPU cannot touch it. Sharing the registry meant a GPU loss threw away 86 MB of perfectly good weights and forced a needless session rebuild. Splitting also removes the id special-casing the shared registry forced into the progress subscription. The ASR handler goes back to "whatever is loading", and the aligner's own subscription is scoped to the await in forceAlign — the subscription's lifetime is the gate, so there is no module-level flag left to fall out of step if that path is ever re-entered. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
subscribe() only fires on the next change, so a load that had started but not yet received its first byte left the UI on the previous message. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
cursor
Bot
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Whisper’s word timestamps come from DTW over decoder attention — often late, drifting, weak on fricatives, and sometimes collapsing a word to a sliver. This PR adds a WhisperX-style post-pass: keep Whisper for the transcript, then measure word boundaries with a CTC acoustic model.
Pipeline
lib/align.ts) — loudness envelope (incl. high band for fricatives), onset refinement, pauseless lag estimate, collapsed-word repair. Also the fallback when CTC is unavailable.lib/forcedAlign.ts) — Viterbi over wav2vec2 emissions (Xenova/wav2vec2-base-960h, q4 ~86 MB)."..."recovery).Worker integration
ModelManagerso ASR WebGPU→WASMunloadAll()does not throw away WASM aligner weights.Tests
tests/align-test.ts(envelope, fricatives, collapsed words).tests/forced-align-test.ts(Viterbi, batching, unspellable words).Follow-up
Multilingual model map + Parakeet sharing the same post-pass: #65.