You write the script, out comes the edited video.
I built this so I wouldn't have to edit every video for a faceless YouTube channel by hand: narrated explainer videos on psychology and behavior, with stick-figure illustrations and burned-in subtitles. I write the script and the pipeline does the rest: voice, images, subtitles and final assembly.
Garabato is Spanish for "doodle".
Each sentence in the script is preceded by a line of fields describing its
image. The fields are a closed vocabulary, so the same line always produces
the same frame. Those two shot types are drawn by the repo itself with PIL, no
GPU and no model, when CLIP_SOURCE is comfyui.
The idea was to fix the real bottleneck. Recording the voice and finding or drawing an image for every sentence is what makes an eight-minute video cost a whole day. Everything else (script, judgment, review) is still mine, because that's where the value is.
A plain-text script goes in through input/script.txt and a
final_video.mp4 comes out:
input/script.txt
↓ parse_script.py segments.json
↓ generate_voice.py narration.mp3 + beat_timings.json
↓ generate_image_prompts.py image_prompts.json
↓ generate_clips.py clips/seg_NNN.mp4
↓ generate_subtitles.py subtitles.srt + subtitles.ass
↓ assemble_video.py final_video.mp4
↓ check_audio.py validation report
python run_pipeline.py [slug]Every step is idempotent: if you run it again it skips what's already done, so you can stop halfway and resume, or redo just one stretch.
That list is the execution order, but it leaves out the two decisions that actually hold the pipeline together:
The clock. generate_voice.py concatenates the audio with no gaps and
writes the real start and end of each beat to beat_timings.json. Images and
subtitles don't keep time on their own: both read that file. That's why they
never drift from the audio, and why a video generated before that file
existed can't be synced without redoing it.
The fork. Where each image comes from isn't decided when generating the
clips but earlier, when building the prompts
(generate_image_prompts.py:750). With gemini or manual, a
natural-language prompt is sent for every beat, and the model also
illustrates the title cards and diagrams. With comfyui, those two types get
an empty prompt and PIL draws them, and only scene shots go through the closed
catalog. Same script, two different paths: that's why title cards look
different depending on the backend.
The script isn't loose prose. Each narration sentence is preceded by a line of fields in parentheses that describes its image:
TITLE: Why Habits Change Everything
NARRATION:
[SEGMENT 1]
(shot: medium | char: 1, worried | pose: arms-crossed | prop: red-X right | bg: plain-white)
Every time you try to change, your brain treats it like a threat.
(shot: text-frame | text: 'STATUS QUO BIAS')
Scientists call this the status quo bias.
[SEGMENT 2]
(shot: diagram | left: question-mark | right: lightbulb | arrow: right yellow | bg: plain-white | desc: a question mark on the left turning into a glowing lightbulb on the right)
A simple question turns into a real discovery.
The fields are a closed vocabulary, not free text. scripts/_catalog.py is
the source of truth and prompts/beat_format.md the reference for writing.
This started out as free prose, and that was the project's biggest mistake. A fast diffusion model pays attention to three or four elements per image and that's it: give it a paragraph and it picks which ones. With closed fields the prompt is built with a dictionary lookup, comes out the same twice in a row and can be validated before spending a single generation.
The desc: field is the deliberate exception: a literal description of
what's in the frame, a single frozen moment. It's what makes the image
illustrate the sentence instead of settling for a generic pose. Without
desc:, the beat falls back to the catalog, so older scripts keep working.
Before generating anything:
python scripts/lint_script.py [slug]The linter warns but doesn't block: it checks that poses are in the catalog,
that text: fields don't exceed four words and that the mix of shot types
resembles the one that works.
| Piece | Note |
|---|---|
| Python | 3.10 or later |
| FFmpeg | with h264_nvenc. The binary I use ships without libx264 |
| Voice | Chatterbox, local, on GPU |
| Images | Gemini 2.5 Flash Image via API, or ComfyUI with Flux Schnell locally |
| Subtitles | stable-ts, which downloads the Whisper base model the first time |
| Fonts | None needed. It uses whatever you have; see Fonts |
pip install -r requirements.txtAll configuration lives in config.py: voice, style, paths, image backend.
There are no settings scattered across the scripts.
Never in a versioned file. Either the GEMINI_API_KEY environment variable,
or a secrets_local.py in the root (it's in .gitignore):
GEMINI_API_KEY = "..."config.py checks the environment first and then the file.
Nothing to install. config.py searches by file name in the Linux, macOS and
Windows font directories, and ends with a generic bold (DejaVu Sans,
Liberation Sans, Noto Sans, Arial) that exists on any machine. If it finds
none, the renderers fall back to PIL's default font instead of crashing.
To use your own, either:
- Drop the
.ttfintoassets/fonts/. Whatever is there wins over everything else, with no install and no code changes. That folder is in.gitignore. - Put the file name, without extension, at the start of
FONT_PATHS_BOLDorFONT_PATHS_REGULARinconfig.py.
The handwritten fonts at the top of the list (Caveat, Architects Daughter, Patrick Hand) are optional: if you ever install them, they're picked up automatically and the text fits the stick figure better.
Subtitles are separate, because the one painting them when burning them in is
libass, which resolves by family name, not by path: that's
SUBTITLE_STYLE["font"].
CLIP_SOURCE in config.py picks one of three paths:
"manual":scripts/export_prompts.pywrites you the list of prompts with the exact file name for each one. You generate them by hand on the web, save them asclips/seg_NNN_src.pngand the pipeline builds the clips from whatever it finds. It's free and can be done in batches."gemini": the same, but automated via API. It needs billing enabled; the image API has no free tier. It comes to about three euros per video."comfyui": ComfyUI with Flux Schnell locally. It's the original backend and I keep it as a fallback.
To keep the stick figure the same in every shot, assets/character_ref.png is
passed as a reference in every image with a character. That model sheet is
generated once with the prompt in assets/character_ref_prompt.txt.
config.py All configuration
run_pipeline.py Orchestrator for the seven steps
│
├── scripts/
│ ├── _catalog.py Closed vocabulary: field → prompt fragment
│ ├── parse_script.py script.txt → segments.json
│ ├── generate_voice.py Chatterbox → narration.mp3
│ ├── generate_image_prompts.py Audio timing + prompt assembly
│ ├── generate_clips.py Dispatches to the right backend
│ ├── _gemini.py Gemini backend with retries and throttling
│ ├── _comfyui.py Batched ComfyUI backend + fades
│ ├── _text_frame.py PIL rendering of title cards
│ ├── _diagram_frame.py PIL rendering of diagrams
│ ├── _sprites.py Icon compositing over the image
│ ├── generate_subtitles.py Forced alignment → .srt + .ass
│ ├── assemble_video.py Concatenation, audio, music and burn-in
│ ├── check_audio.py Validates LUFS, duration and words
│ └── lint_script.py Script validator, non-blocking
│
├── prompts/
│ ├── script_template.md Instructions for writing a full script
│ ├── beat_format.md Field and catalog specification
│ └── generate.md Script generation prompt
│
├── tools/
│ └── make_readme_figures.py Regenerates docs/img/ with the repo's renderers
│
├── docs/
│ ├── decisions.md Why each thing is the way it is
│ └── img/ README figures
│
├── assets/
│ ├── sprites/ PNG icons with alpha
│ ├── fonts/ Your font, if you want one (not versioned)
│ └── character_ref.png The stick figure's model sheet
│
└── output/<slug>/ Everything the pipeline generates
The figures in this README aren't hand-made screenshots: they come from
python tools/make_readme_figures.py. The frames figure calls
_text_frame.py and _diagram_frame.py, the same modules the pipeline uses,
so if you change the style you regenerate it and the README stops lying. The
diagram is drawn by the same script, in two versions for GitHub's light and
dark themes.
They're in docs/decisions.md: why the negative prompt
is inert and writing one is pointless, why the seed is computed per segment
and not per global index, why I removed the Ken Burns effect, why the
character description is injected automatically instead of written in the
script.
The encoder. My FFmpeg is built without libx264, so everything goes
through h264_nvenc. It's already set in _comfyui.py and
assemble_video.py, but if someone takes this to another machine it's the
first thing to check.
The GPU crashes with ComfyUI. Generating images locally at full power took my card down. Two things fixed it almost entirely: lowering the power limit to 80-85%, which costs 10% speed and drops 10 to 15 degrees, and closing the browser before running the pipeline, because hardware acceleration competes for VRAM. If it still crashes you need a full reboot; restarting the driver doesn't bring it back. This doesn't happen with the Gemini backend.
Encoding on Windows. You need to set PYTHONIOENCODING=utf-8 and
PYTHONUTF8=1 or the console blows up on accented characters. And if you
write input/script.txt from PowerShell with Set-Content -Encoding utf8 it
adds a BOM that parse_script.py can't handle.
Videos from before the audio fix. Videos generated before the voice
concatenation became gapless have longer narration and no
beat_timings.json, so images and subtitles drift from the audio. To fix one:
delete narration.mp3, narration_processed.mp3, image_prompts.json and
the subtitles.* files, and run it again. Per-beat audio is kept and reused.
The pipeline works end to end. What's solved:
- Structured beat format, closed catalogs and a linter.
- Gapless voice between beats, with
beat_timings.jsonas the single timeline for images and subtitles. I measured 0.05 seconds of drift over 83 beats. - Character-aware prompts: the stick figure only appears when the beat asks for it, and object shots carry an explicit instruction that nobody should appear.
- Karaoke-style subtitles, split at beat boundaries so the text never overlaps an image change, with the bottom fifth of the frame kept clear so nothing sits behind them.
- Audio post-processing with EQ and normalization to -14 LUFS.
What's missing or half-done:
- The voice is Chatterbox's stock one. I still need to record or clone my own
and point
CHATTERBOX_REF_AUDIOat it. - Background music (
BACKGROUND_MUSIC_PATH) isn't set. - The LoRA training in
tools/lora_training/became obsolete when I switched to generating images with a character reference. I'm keeping it because the dataset prep code and the experiments are useful, but it's no longer part of the pipeline.
TTS_SPEED = 1.0. Time-stretching below 0.9x sounds robotic.h264_nvencfor all encoding.- No web UI and no database. Files and the command line.
- With ComfyUI, Flux Schnell runs at
cfg=1.0with theeulersampler. Atcfg=1.0the negative prompt does nothing, so quality is controlled only by what's asked for in the positive prompt.
MIT, in LICENSE. The models each backend uses come with their own licenses.
