A Telegram bot that transcribes, translates, and summarizes voice messages. Everything runs locally: whisper.cpp for speech-to-text and llama.cpp with Nemotron 3.5 Bolt E4B for translation and summaries. No audio or text is sent to a third-party service.
Note on Serverless History: The last commit of the bot being serverless is 1ccb016ccabeb320b0c7637d3c15fc9bdedb2a48.
- Send a voice, audio, or video note to the bot
- The bot decodes it with ffmpeg, measures its real length from the decoded audio, and transcribes it with a local whisper.cpp server
- For
/translate,/summarizeand/caveman, the transcript goes to a local llama.cpp server, which translates it into English or summarizes it straight from the original language - Results are sent back and cached for future use
The bot can be added to groups to automatically transcribe voice messages. In groups, transcripts and translations are sent as an expandable quote, so long transcripts stay a few lines tall until someone taps them. You can manually transcribe videos and audio files by replying to them with the bot commands. In DMs, the bot will automatically transcribe most media sent to it, and replies in plain text.
/start: Initializes the bot and provides a welcome message/help: Provides information on how to use the bot and its features/transcribe: Transcribes the voice, audio, or video note in the reply message/phonon: Transcribes the voice, audio, or video note in the reply message with Phonon-2.5 instead of Whisper/translate(aliases:english,en): Translates (into English) the voice, audio, or video note in the reply message/summarize: Summarizes the voice, audio, or video note in the reply message/caveman: Summarizes the voice, audio, or video note in a "caveman" style/privacy: Shows the privacy policy/limits(aliases:/ratelimit,/ratelimits): Shows current rate limit information/donate: Shows cryptocurrency donation addresses to support the project
- Language: Rust with async/await
- Telegram API: Built using the
teloxidecrate - Transcription: whisper.cpp server (
large-v3-turbo) over HTTP withreqwestand rustls /phonon: Phonon-2.5, a 156 MB low-bit build of NVIDIA's Parakeet TDT 0.6B v3 for 25 languages, served byfermion serveon the CPU and kept in RAM (about 1.2 GB). It gets the same decoded 16 kHz mono WAV as whisper.cpp. Its transcripts are cached separately from whisper.cpp's- Translation & summaries: llama.cpp server (
NVIDIA-Nemotron-3.5-Bolt-E4BUD-Q4_K_XLGGUF from Unsloth) through its OpenAI-compatible chat completions endpoint, with reasoning turned off - TLS: Uses rustls everywhere (no OpenSSL dependencies)
- Logging: Dual output to stdout and
bot.logfile using fern - Caching:
- Transcriptions and translations cached for 7 days;
/translateand the summaries reuse a cached transcript instead of running whisper.cpp again - Summaries (default and caveman) cached for 1 hour
- In-memory cache uses per-result expiration timestamps with automatic cleanup
- Transcriptions and translations cached for 7 days;
- Rate Limiting:
- Per-user tracking: 25 messages per minute, 150 messages per hour
- Temporary per-user rate-limit records are cleaned up after 2 hours
- Reacts with 🙊 emoji when per-user limit is exceeded
- Applies to all audio operations (transcribe, translate, summarize, caveman)
- LLM Limits (
/translate,/summarize,/caveman): llama.cpp is much slower than whisper.cpp and shares its GPU, so these commands have their own limits on top of the ones above- Per user: 15 requests per hour and 50 per day; reacts with 🙊 when exceeded
- Whole bot: 100 requests per hour; reacts with 😴 when reached
/translateaccepts recordings up to 10 minutes (translating takes about 15 seconds per minute of audio); summaries accept the full 30 minutes- Lengths are measured from the decoded audio, never taken from Telegram, because the sender's app sets that number and it can be 0 or wrong
- Only actual llama.cpp requests count: cached results and recordings without speech are free
- Failures: Reacts with 💔 emoji instead of replying when whisper.cpp or llama.cpp can't produce a result
TELEGRAM_BOT_TOKEN: The token for your Telegram botWHISPER_LOCAL_URL: whisper.cpp inference endpoint (default:http://host.docker.internal:8080/inference)LLM_URL: llama.cpp chat completions endpoint (default:http://host.docker.internal:8081/v1/chat/completions)PHONON_URL: Phonon transcription endpoint for/phonon(default:http://host.docker.internal:8082/v1/audio/transcriptions)
The bot needs three servers running on the host: whisper.cpp, llama.cpp and Phonon. whisper.cpp and llama.cpp share one GPU, so start whisper.cpp first. llama-server then fits as many model layers into the VRAM that is left as it can and keeps the rest in RAM. Phonon runs on the CPU only.
scripts/start-whisper-tmux.sh # whisper.cpp on :8080
scripts/start-llama-tmux.sh # llama.cpp on :8081, waits for whisper.cpp
scripts/start-phonon-tmux.sh # Phonon-2.5 on :8082, for /phononstart-llama-tmux.sh expects a CUDA build of llama.cpp in ~/llama.cpp
(override with LLAMA_CPP_DIR) and downloads the model GGUF on first start.
start-phonon-tmux.sh installs Phonon into ~/phonon/venv with
uv on first start (override with PHONON_DIR);
the model downloads on the server's first start.
To start them after a reboot, add them to the crontab in this order:
@reboot /path/to/duck_transcriber/scripts/start-whisper-tmux.sh
@reboot /path/to/duck_transcriber/scripts/start-llama-tmux.sh
@reboot /path/to/duck_transcriber/scripts/start-phonon-tmux.sh
- Install Rust: https://rustup.rs/
- Clone the repository:
git clone https://github.com/DuckyBlender/duck_transcriber.git - Create
.envfile:The bot also needsTELEGRAM_BOT_TOKEN=your_bot_token WHISPER_LOCAL_URL=http://localhost:8080/inference LLM_URL=http://localhost:8081/v1/chat/completions PHONON_URL=http://localhost:8082/v1/audio/transcriptions
ffmpegon thePATH: every recording is decoded with it before it is measured and transcribed. - Run the bot:
cargo run --release
Build and run the bot in Docker with an optimized multi-stage build. whisper.cpp llama.cpp and Phonon are not part of Docker or compose; start them on the host first (see above), then start duck_transcriber:
docker compose up -dThe compose file maps host.docker.internal to the host gateway so the
container can call the host servers.
The Dockerfile uses cargo-chef for efficient dependency caching, resulting in faster rebuilds.
- Robust Error Handling: All errors are properly handled and logged
- Quiet Failures: Uses 🙊 for per-user limits, 😴 when the whole bot is at its LLM limit, and reacts with 💔 (no error message) when transcription, translation or summarization fails, so a server outage doesn't flood groups with error replies
If you find this bot useful and would like to help cover hosting costs, donations are greatly appreciated! You can donate using various cryptocurrencies:
- Bitcoin:
bc1qqvahwa3we2eymeku5489kpdk29pvddnn8yruzy - Bitcoin Lightning:
duckyblender@strike.me - Ethereum (or any ERC20 token):
0xd399774D0a969FC2ddF7D0D0f4f165638541408c - Nano:
nano_3ociqkh6taqqu7q7h99oiyuasnkugm7bss87r1r4eph7dym3tmp3cebtosc5 - Monero:
49afHkLvAEAXmdCqhWDCYvZiBD942ouwrhxbwto2HXzoGW877umX2aEFcVf8qzpawuFTBo28TK5Kg8sSzSQSXh7D7YoKEeK
You can also use the /donate command in the bot to view these addresses directly.
This project is licensed under the GNU General Public License v3.0. See the LICENSE file for details.
Contributions are welcome! If you'd like to help improve this bot, please open a pull request with your changes.