Repository navigation
Conversation
# Conflicts: # workers/transcription.worker.ts
The CTC aligner loaded itself with bare from_pretrained calls, so its 86 MB download happened behind a static "Aligning words…" with no bar — indistinguishable from a hang on a cold cache. Register it in the existing ModelManager instead, so it shares the download progress, cache labeling and unload-on-GPU-loss path with the ASR models. It needs the processor, model and tokenizer separately rather than a pipeline(), so it is a plain ModelDefinition wired to transformersProgress rather than transformersModel(). The aligner warms in the background while Whisper runs, so its bytes must not fight the "Transcribing…" line: speech models always own the progress line, and the aligner only claims it once transcription is done and we are actually waiting on it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
# Conflicts: # workers/transcription.worker.ts
wav2vec2 is not in the same family as the ASR models — it is an acoustic aligner, not a transcriber, and not something the user picks. But the deciding factor is lifecycle, not taxonomy. The ASR registry exists under a WebGPU→WASM fallback policy whose recovery step is unloadAll(). The aligner never asks for a device, so transformers.js runs it on WASM (DEFAULT_DEVICE), where a lost GPU cannot touch it. Sharing the registry meant a GPU loss threw away 86 MB of perfectly good weights and forced a needless session rebuild. Splitting also removes the id special-casing the shared registry forced into the progress subscription. The ASR handler goes back to "whatever is loading", and the aligner's own subscription is scoped to the await in forceAlign — the subscription's lifetime is the gate, so there is no module-level flag left to fall out of step if that path is ever re-entered. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
subscribe() only fires on the next change, so a load that had started but not yet received its first byte left the UI on the previous message. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Share the VAD → CTC → expand post-pass between Whisper and Parakeet, and pick a language-specific aligner (English wav2vec2, MMS for es/fr/de, XLS-R for zh) so non-English runs no longer warm an English-only model. Co-authored-by: Wassim Gharbi <wassgha@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extends the forced-alignment work from #61 with two follow-ups:
VAD/envelope → CTC force-align → expand → disfluencies).Align model map
enXenova/wav2vec2-base-960h(~86 MB q4)es/fr/deonnx-community/mms-300m-1130-forced-aligner-ONNX(~240 MB q4)zhonnx-community/wav2vec2-large-xlsr-53-chinese-zh-cn-ONNX(~240 MB q4)Languages without a model (none currently in the supported set) keep the heuristic-only path. Aligners only warm/download for the active transcript language.
Also
refineWordTimestamps()used by Whisper and ParakeetnormalizeForCtcsupportslatin-upper/latin-lower/cjk|when present; MMS has none)Built on top of
fix-alignment-forced(#61).