GPU support for transcription

[kerta1n]

Currently transcription (for subtitle generation) seems to be whisper on CPU only, which takes a VERY long time for clips that are 10/20m long.

Is it possible to add support whisper on GPU, as well as other engines like WhisperX, faster-whisper (CTranslate2), parakeet (NVIDIA), Sherpa-ONNX, etc. on the same machine?

There is also no per-word transcription - I only get clusters of words transcribed per one caption block. An engine that does per-word transcription would make agentic editing even faster (more accurate at tasks like aligning clips on different tracks based on their transcript).

Also, is it possible to add support for remote whisper transcription (e.g. Groq API which has a VERY generous free tier, OpenAI's API themselves, or even selfhosted (this would mean following verbose_json spec, Speaches project is a good example, it follows OpenAI audio API & verbose_json standard)

[jub0t]

Currently transcription (for subtitle generation) seems to be whisper on CPU only, which takes a VERY long time for clips that are 10/20m long.

Is it possible to add support whisper on GPU, as well as other engines like WhisperX, faster-whisper (CTranslate2), parakeet (NVIDIA), Sherpa-ONNX, etc. on the same machine?

There is also no per-word transcription - I only get clusters of words transcribed per one caption block. An engine that does per-word transcription would make agentic editing even faster (more accurate at tasks like aligning clips on different tracks based on their transcript).

10-20 minutes is really poor performance, This is definitely an area we can improve in. I'll look into more transcription engines and how we can improve the Transcription tool and add more features/options like selecting between per word transcription and other alternatives.

[kerta1n]

It is actually longer than 10 minutes, it took about 25m for a 10m clip (on an 12th gen Intel core). In my opinion the first priority is offloading whisper to the GPU for all platforms (then add different ASR models like WhisperX, faster, parakeet, as well as other runtimes like Sherpa-ONNX can be added and tested last, then after, add in API transcription with per-word support)

[jub0t]

Also, is it possible to add support for remote whisper transcription (e.g. Groq API which has a VERY generous free tier, OpenAI's API themselves, or even selfhosted (this would mean following verbose_json spec, Speaches project is a good example, it follows OpenAI audio API & verbose_json standard)

Concat does have an MCP (Settings > Remote) that you can connect to. I had Claude generate some quick docs for those who are interested, though I haven't had the time to evaluate and test the docs myself.

[kerta1n]

I think this was mistakenly posted in response to someone else?

Regardless, great to see MCP support is now materializing though

[kerta1n]

Another thing I remembered is that Parakeet on CPU itself is EXTREMELY fast compared to Whisper (on CPU). So even just adding parakeet support to the model list to the next release would speed things up without the effort needed to test GPU (mappings, binaries and whatever else).