What
An examples/auto-captions script: run speech-to-text on a clip's audio, turn the word timestamps into FableCut text clips, write them into project.json.
This is the most-asked question about the caption feature ("is it auto-caption?"). The editor deliberately has no built-in ASR — captions are kind:"text" clips and the intelligence is supposed to live outside. This example makes that concrete.
Sketch
- Extract audio:
ffmpeg -i media/talk.mp4 -ar 16000 -ac 1 audio.wav
- Transcribe with word timestamps (whisper.cpp, faster-whisper, or any STT — the example should be engine-agnostic with whisper as the default)
- Group words into caption lines (~4 words / ~1.8s per line)
- Emit one text clip per line on V3:
start/duration from the timestamps, textAnim: "karaoke" with wordRate matched to the actual speech pace so the highlight lands on the spoken word
PUT /api/project (or fablecut_patch_project via MCP) — the open editor shows the captions immediately
~50 lines of Python or Node. No changes to the editor itself.
Done when
examples/auto-captions/README.md + script exist; running it against a talking-head clip produces synced karaoke captions in the open editor.
What
An
examples/auto-captionsscript: run speech-to-text on a clip's audio, turn the word timestamps into FableCut text clips, write them intoproject.json.This is the most-asked question about the caption feature ("is it auto-caption?"). The editor deliberately has no built-in ASR — captions are
kind:"text"clips and the intelligence is supposed to live outside. This example makes that concrete.Sketch
ffmpeg -i media/talk.mp4 -ar 16000 -ac 1 audio.wavstart/durationfrom the timestamps,textAnim: "karaoke"withwordRatematched to the actual speech pace so the highlight lands on the spoken wordPUT /api/project(orfablecut_patch_projectvia MCP) — the open editor shows the captions immediately~50 lines of Python or Node. No changes to the editor itself.
Done when
examples/auto-captions/README.md+ script exist; running it against a talking-head clip produces synced karaoke captions in the open editor.