diff --git a/authors/franklin-wilster.md b/authors/franklin-wilster.md new file mode 100644 index 00000000..df506598 --- /dev/null +++ b/authors/franklin-wilster.md @@ -0,0 +1,8 @@ +Author: Franklin Wilster Title: Technical Contributor Description: Franklin +Wilster is a technical contributor from Brazil focused on automation and +open-source software. His technical contributions center on practical +automation and improvements to open-source projects. Author Image: +![Franklin Wilster](https://avatars.githubusercontent.com/franklincg) Author +GitHub: [franklincg](https://github.com/franklincg) Company Name: Independent +Company Description: Independent technical contributor focused on automation +and open-source software. diff --git a/definitions/20260906_definition_local_asr_inference.md b/definitions/20260906_definition_local_asr_inference.md new file mode 100644 index 00000000..5293464a --- /dev/null +++ b/definitions/20260906_definition_local_asr_inference.md @@ -0,0 +1,43 @@ +--- +title: 'Local ASR Inference' +description: 'Running automatic speech recognition within a controlled local compute environment instead of a hosted transcription API.' +date: 2026-09-06 +author: 'Franklin Wilster' +--- + +# Local ASR Inference + +## Definition + +Local automatic speech recognition (ASR) inference is the process of converting +recorded speech into text by executing an ASR model inside a user-controlled +computer, container, or development workspace. Unlike a hosted transcription +API, the inference path does not need to upload source audio to a third-party +service. + +“Local” describes where inference runs, not necessarily how every dependency +arrives. Python packages and pretrained model assets may require network access +during setup or first use. A workflow becomes offline-capable only after all +required code and model files are available in a persistent local cache. + +## Context and Usage + +Developers choose local ASR inference when they need greater control over audio +handling, model selection, compute devices, costs, or repeatability. A typical +workflow fixes the application version, model identifier, cache directory, +device, input checksum, and dependency versions. These records allow another +run to reproduce the same conditions and help distinguish environmental changes +from changes in transcription accuracy. + +Local inference can improve privacy by keeping transcription audio within the +controlled runtime after setup. It does not remove every privacy concern: +download hosts, workspace access, logs, backups, mounted storage, and retention +policies still require review. CPU execution is often the most portable +baseline, while GPU execution depends on compatible hardware, drivers, runtime, +and machine-learning packages. + +Sapat's SpeechBrain provider is one example. It loads an ASR model through +SpeechBrain's Python inference API and transcribes a locally prepared WAV file +without an API key. See the guide to +[running SpeechBrain with Sapat in Daytona](../guides/20260906_run_speechbrain_transcription_with_sapat_in_daytona.md) +for a reproducible setup. diff --git a/guides/20260906_run_speechbrain_transcription_with_sapat_in_daytona.md b/guides/20260906_run_speechbrain_transcription_with_sapat_in_daytona.md new file mode 100644 index 00000000..5bbd21d8 --- /dev/null +++ b/guides/20260906_run_speechbrain_transcription_with_sapat_in_daytona.md @@ -0,0 +1,318 @@ +--- +title: 'Run Local SpeechBrain ASR with Sapat in Daytona' +description: + 'Build a reproducible Daytona workspace for private, Python-native SpeechBrain transcription with controlled model, cache, and device settings.' +date: 2026-09-06 +author: 'Franklin Wilster' +tags: ['daytona', 'speechbrain', 'speech-to-text'] +--- + +# Run Local SpeechBrain ASR with Sapat in Daytona + +Speech-to-text experiments are difficult to compare when the runtime, model, +and cache change between runs. A Daytona workspace gives the experiment a +repeatable environment, while Sapat provides one command-line interface for +audio preparation and transcription. With Sapat's new SpeechBrain provider, +the inference step runs in Python in the workspace instead of sending audio to +a hosted transcription API. + +This guide builds that workflow around [local ASR inference](../definitions/20260906_definition_local_asr_inference.md). +It pins the companion implementation, makes model storage and device selection +visible, and validates both the environment and the resulting transcript. The +result is useful for private English audio and repeatable ASR evaluation, with +one important qualification: the first run may contact a model host to download +assets. The workflow is offline-capable only after those assets are cached. + +## TL;DR + +- Install Sapat's SpeechBrain extra from the exact implementation commit behind + [Sapat PR #77](https://github.com/nibzard/sapat/pull/77). +- Use `SPEECHBRAIN_MODEL`, `SPEECHBRAIN_SAVEDIR`, and `SPEECHBRAIN_DEVICE` to + make the model source, cache location, and execution device explicit. +- The default `speechbrain/asr-crdnn-rnnlm-librispeech` model is an English + LibriSpeech ASR model. `--language` does not turn it into a multilingual model. +- No API key is required. Network access can still be required on the first run + while SpeechBrain obtains model assets. +- Sapat converts an input file to 16 kHz mono WAV, calls SpeechBrain locally, + writes a same-stem `.txt` file, and removes the temporary WAV. + +![SpeechBrain transcription workflow](assets/20260906_run_speechbrain_transcription_with_sapat_in_daytona_workflow.svg) + +## Prerequisites + +You need an active Daytona workspace with a Linux shell, network access for +initial package and model downloads, and enough disk space for Python packages, +the selected model, and input media. The workspace should have Python 3.8 or +newer, Git, and `ffmpeg`. Sapat shells out to `ffmpeg` before inference, even +though SpeechBrain itself is a Python library. + +Use a short, clearly spoken English MP3 or MP4 file that you are authorized to +process. The default model was trained for English speech in the LibriSpeech +domain, so accents, noise, overlapping speakers, specialist vocabulary, and +audio unlike audiobooks may reduce accuracy. A GPU is optional; begin with CPU +to establish a portable baseline. + +This walkthrough uses the terminal inside an existing Daytona workspace. +Daytona products and interfaces evolve, so create or open the workspace using +the current [Daytona documentation](https://www.daytona.io/docs/) rather than +relying on an unverified CLI sequence. + +## Prepare the Daytona workspace + +Create a project directory in the workspace, install the operating-system +tools, and isolate Python dependencies in a virtual environment: + +```bash +mkdir -p speechbrain-sapat-demo/samples +cd speechbrain-sapat-demo + +sudo apt-get update +sudo apt-get install -y ffmpeg git python3-venv + +python3 -m venv .venv +. .venv/bin/activate +python -m pip install --upgrade pip +``` + +Confirm which executables the workspace will use. Recording these versions is +more useful than assuming two workspaces are identical: + +```bash +python --version +python -m pip --version +ffmpeg -version | head -n 1 +git --version +``` + +Keep the sample outside any directory that contains unrelated MP4 files. When +Sapat receives a directory, its current CLI processes every top-level `*.mp4` +file in that directory; it does not recursively discover MP3, WAV, or nested +files. + +## Install the SpeechBrain provider branch + +At the time of writing, the provider is in the companion +[Sapat implementation PR](https://github.com/nibzard/sapat/pull/77), not the +upstream default branch. Install the contributor branch at commit +`4eef5131cd1aefd1d6aa064d4b8962c3ce720018` so that rebuilding the workspace +does not silently follow later branch changes: + +```bash +python -m pip install \ + "sapat[speechbrain] @ git+https://github.com/franklincg/sapat.git@4eef5131cd1aefd1d6aa064d4b8962c3ce720018" +``` + +The `speechbrain` extra installs SpeechBrain alongside Sapat. It does not add an +API credential. Verify the installed Sapat source revision and capture resolved +package versions for later comparison: + +```bash +python -m pip show sapat speechbrain torch +python -m pip freeze > environment.freeze.txt +sapat --version +``` + +Pinning the Sapat commit is stronger than installing a moving branch, but this +single command does not make the entire environment bit-for-bit immutable. +Transitive package resolution and the files served for a model identifier may +change. For a controlled experiment, retain `environment.freeze.txt`, the +workspace image information, and checksums described in the validation section. + +## Configure model, cache, and device + +Sapat loads a `.env` file from the working environment. Create one in the demo +directory with the three provider settings: + +```dotenv +SPEECHBRAIN_MODEL=speechbrain/asr-crdnn-rnnlm-librispeech +SPEECHBRAIN_SAVEDIR=.cache/speechbrain/asr-crdnn-rnnlm-librispeech +SPEECHBRAIN_DEVICE=cpu +``` + +`SPEECHBRAIN_MODEL` selects the source passed to SpeechBrain. The default shown +above is `speechbrain/asr-crdnn-rnnlm-librispeech`, an English LibriSpeech ASR +model. An explicit non-default `--model` value on the Sapat command line takes +precedence over `SPEECHBRAIN_MODEL`; the environment variable replaces the +model only when Sapat is otherwise using its default. + +`SPEECHBRAIN_SAVEDIR` supplies a predictable directory to SpeechBrain's model +loader. Keeping it under the project makes cache inspection and workspace +snapshotting straightforward. Do not commit downloaded weights casually: they +are large artifacts and may have licensing or redistribution conditions. Cache +persistence also depends on the lifecycle and storage policy of your Daytona +workspace. + +`SPEECHBRAIN_DEVICE` becomes SpeechBrain's PyTorch device. `cpu` is the provider +default and the most reproducible starting point. No secret belongs in this +file for SpeechBrain, because this provider has no API key requirement. + +## Understand the first run and cache + +The provider imports the maintained +`speechbrain.inference.ASR.EncoderDecoderASR` class, constructs it with +`from_hparams`, and sends the prepared WAV file to `transcribe_file`. During the +first `from_hparams` call, SpeechBrain may download configuration, tokenizer, +and model assets for the selected source. Allow network access and expect the +first transcription to take longer. + +After a successful run, inspect the configured cache without changing it: + +```bash +find .cache/speechbrain -maxdepth 3 -type f -printf '%P\n' | sort +du -sh .cache/speechbrain +``` + +Only call the setup offline after testing it without network access and +confirming that every required asset resolves from the cache. A cached model +does not cover missing Python wheels, a recreated virtual environment, or an +ephemeral workspace volume. + +## Choose CPU or an available GPU + +For the baseline, leave `SPEECHBRAIN_DEVICE=cpu`. This avoids assuming CUDA is +exposed to the workspace and makes comparisons between ordinary Linux +workspaces simpler. CPU inference may be slower, especially for long recordings. + +If the Daytona workspace exposes a compatible accelerator and the installed +PyTorch build recognizes it, check PyTorch before changing the configuration: + +```bash +python -c "import torch; print(torch.cuda.is_available())" +python -c "import torch; print(torch.cuda.get_device_name(0))" +``` + +Only when both the workspace and PyTorch are correctly configured should you +set, for example: + +```dotenv +SPEECHBRAIN_DEVICE=cuda:0 +``` + +This selects a device; it does not guarantee that every SpeechBrain model, +PyTorch build, CUDA runtime, and workspace GPU combination is compatible. If a +CUDA run fails, restore `cpu` before diagnosing model or audio problems. + +## Transcribe an English recording + +Copy an authorized recording to `samples/english-speech.mp3`, then record its +checksum so repeated trials use identical bytes: + +```bash +sha256sum samples/english-speech.mp3 +ffprobe -v error -show_entries format=duration \ + -of default=noprint_wrappers=1:nokey=1 \ + samples/english-speech.mp3 +``` + +Run Sapat with the provider named explicitly: + +```bash +sapat samples/english-speech.mp3 \ + --provider speechbrain \ + --model speechbrain/asr-crdnn-rnnlm-librispeech \ + --language en \ + --quality L +``` + +The CLI displays the selected provider, model, and language. It converts the +MP3 to `samples/english-speech.wav` as 16 kHz, mono, signed 16-bit audio; +SpeechBrain transcribes that WAV locally; Sapat writes +`samples/english-speech.txt`; and the temporary WAV is removed. + +Avoid passing a WAV input directly in this implementation, and ensure there is +no valuable same-stem WAV beside an MP3 or MP4 input. Sapat treats the +provider-preferred WAV path as temporary and removes it after processing. Also +note that the SpeechBrain provider currently ignores `language`, transcription +prompt, and temperature values. `--language en` documents intent but does not +alter this model. `--correct` is unsupported for this provider and produces a +warning rather than an LLM-corrected transcript. + +## Validate the result + +First confirm that provider discovery sees SpeechBrain: + +```bash +python -c \ + "from sapat.providers import get_available_providers as g; print('speechbrain' in g())" +``` + +The output should be `True`. After transcription, check that the text exists, +is non-empty, and can be reviewed without printing private content to shared +logs: + +```bash +test -s samples/english-speech.txt +wc -w samples/english-speech.txt +sha256sum samples/english-speech.txt +``` + +For meaningful accuracy validation, prepare a trusted reference transcript and +compare normalized output using word error rate or a documented manual rubric. +Run the same audio twice with the same frozen environment, model cache, device, +and command, then compare transcript hashes. Matching hashes demonstrate repeat +output for that setup; they do not prove accuracy. + +The companion change was also validated at the code level. Its targeted +provider tests passed 18/18. Those tests mock `EncoderDecoderASR`, +`from_hparams`, and `transcribe_file`, so they verify provider configuration, +argument flow, output handling, and errors without downloading model weights. +The full local suite passed 193 tests and had one unrelated, pre-existing +Windows WhisperX path-parsing failure. That result is useful regression context, +not evidence that a real model download or every hardware path works. + +## Privacy and reproducibility boundaries + +Once dependencies and model assets are present, audio inference occurs in the +Daytona workspace through SpeechBrain rather than a remote transcription API. +That can reduce disclosure of source audio and removes API-key handling from the +transcription step. It does not automatically make the whole system private. +Initial downloads contact external package and model hosts, and workspace +administrators, storage snapshots, logs, backups, or mounted directories may +still expose data. Apply your organization's retention and access rules. + +For each evaluated run, retain the Sapat commit, `pip freeze` output, model +identifier, cache inventory or approved checksums, device, input checksum, +command, and transcript checksum. Keep CPU and GPU results as separate baselines. +This small manifest turns “same workspace” into evidence that another reviewer +can inspect. + +## Troubleshooting + +**`Provider 'speechbrain' is not available`:** Activate `.venv`, then run +`python -m pip show sapat speechbrain`. Reinstall the pinned extra if SpeechBrain +is absent. Provider registration requires the `speechbrain` package to import. + +**`ffmpeg` is missing or conversion fails:** Confirm `ffmpeg -version` works in +the workspace and that the input is readable. Sapat suppresses the converter's +normal output, so use `ffprobe` directly to diagnose malformed media. + +**The first run stalls or reports a model-loading error:** Check network access, +free disk space, and permissions on `SPEECHBRAIN_SAVEDIR`. A partially populated +cache is not proof that all assets downloaded successfully. + +**CUDA fails:** Set `SPEECHBRAIN_DEVICE=cpu` and rerun. If CPU works, inspect the +workspace's GPU exposure and the PyTorch/CUDA combination separately. Device +selection alone cannot supply drivers or a CUDA-enabled PyTorch build. + +**The transcript is poor or the language is wrong:** Verify the source audio +and remember that the default model is English-only. Changing `--language` does +not make it multilingual. Select a compatible SpeechBrain ASR source deliberately +and validate it; do not assume all model repositories share the same inference +interface or accuracy profile. + +## Conclusion + +You now have a Daytona-hosted Sapat workflow in which SpeechBrain performs +Python-native local inference with an explicit model, cache, and device. The +pinned provider commit and captured environment improve repeatability, while +checksums and reference-text comparison separate reproducibility from accuracy. +After the first successful download, the cache can support offline-capable +transcription, subject to workspace storage and dependency availability. + +## References + +- [Sapat SpeechBrain implementation PR](https://github.com/nibzard/sapat/pull/77) +- [SpeechBrain EncoderDecoderASR API](https://speechbrain.readthedocs.io/en/latest/API/speechbrain.inference.ASR.html) +- [Default SpeechBrain model card](https://huggingface.co/speechbrain/asr-crdnn-rnnlm-librispeech) +- [Sapat repository](https://github.com/nibzard/sapat) +- [Daytona documentation](https://www.daytona.io/docs/) diff --git a/guides/assets/20260906_run_speechbrain_transcription_with_sapat_in_daytona_workflow.svg b/guides/assets/20260906_run_speechbrain_transcription_with_sapat_in_daytona_workflow.svg new file mode 100644 index 00000000..c5dcd30b --- /dev/null +++ b/guides/assets/20260906_run_speechbrain_transcription_with_sapat_in_daytona_workflow.svg @@ -0,0 +1,39 @@ + + + Local SpeechBrain transcription workflow in Daytona + A five-step workflow from a Daytona workspace through Sapat and the SpeechBrain provider to a cached model and local transcript. + + + + + + + Private, repeatable transcription path + + + + + + + + + + + + + + + + Daytona + workspace + Sapat + SpeechBrain + provider + Cached + model + Local + transcript + + Audio stays on the inference path after required packages and model assets are available. + +