Ramble at your screen. Get a finished YouTube video. Automatically.
An agent-driven pipeline that turns daily screen recordings + messy narration into a
polished, narrated, branded video — and schedules it to YouTube.
See how it works »
·
Design Decisions
·
Roadmap (Stages 9–12)
🟢 Status: Core engine ready for real videos. Stages 1–8 are implemented for the current workflow: local Tella export in, polished narrated YouTube-ready video out. The pipeline turns a Tella screen recording +
.srtinto a finished, re-voiced 1080p video in your cloned voice, with footage re-timed to the narration (via themake-videoskill, mirrored in this repo for Claude Code and Codex). Built:.srt→ clean chunked script guided byhouse-style.md→ ElevenLabs voice with TTS-safe brand pronunciation → footage re-sync → FFmpeg assembly (H.264crf 18+ 160k AAC) → optional real-face intro + manual per-video outro wrap → logo overlay → music bed under intro/chapter cards/outro → 3.5s animated chapter cards →adjustments.jsonreview summary, plus declarative per-chunk auto-zoom via the Tella MCP (plan-zooms→ steadymanualZoom, applied on the original recording before re-timing). Long-form renders do not burn captions by default; use--captionsonly for short-form or explicit captioned variants. Publishing supports private upload plus optionalpublishAt, andbun run retro <slug>keeps recurring lessons inhouse-style.md. Future stages (9–12) live inplan.md.
The current reliable input path is local files, not automatic "today's Tella clips" ingest. Export/download the main Tella screen recording and subtitle file, then put them in:
videos/<slug>/recording.mp4
videos/<slug>/<anything>.srt
Optional clips are filename-based, so ordering is explicit:
videos/<slug>/intro.mp4 # prepended before the screen body
videos/<slug>/recording.mp4 # main narrated screen body
videos/<slug>/outro.mp4 # appended after the body
videos/<slug>/metadata.json # YouTube title/description/tags/publishAt
Tella MCP is used for visual edits on the original Tella project — layout, zoom, highlight, blur, export — once the agent has the project IDs in videos/<slug>/tella.json. It does not yet auto-discover the day's recordings and decide which clip is intro/body/outro.
There are three "make-video" layers:
- Canonical skill:
aimhco/skills/make-videois the reusable agent playbook. - Repo mirrors: .agents/skills/make-video/SKILL.md is Codex-discoverable, and .claude/skills/make-video/SKILL.md is kept for Claude Code. In either Codex or Claude Code, ask for
make video <slug>and the agent should run the workflow, checkpoints, fixes, QA, publish, and retro. - Bun command:
bun run make-video <slug>is the lower-level renderer. It assumesscript.json,recording.mp4, and optionalintro.mp4/outro.mp4already exist.
Table of Contents
Making a single tutorial video by hand took hours — retakes, editing, fighting a timeline. The kind of work that makes you ship one video every few months, or none at all.
aimh-video-engine removes the part that hurts. You share your screen, ramble through what you're doing (mistakes and all), and the engine produces a finished video: a clean voiceover in your own cloned voice, tight pacing, auto-zooms, optional captions, chapter cards, music, your logo — scheduled to YouTube. The only on-camera moment is a short, freely-spoken intro showing your real face.
This was sparked by Nate Herk's video, "Claude Fable 5 Made This Entire Video By Itself" — a single /goal prompt where Claude wrote a script, cloned a voice via ElevenLabs, rendered a HeyGen avatar, generated motion graphics as code, and stitched everything with FFmpeg, verifying frames as it went.
The key realization: that video was 100% synthetic — no screen recording at all. A talking-head avatar is the whole frame. That format doesn't fit a developer build-log.
aimh-video-engine is a hybrid: the screen recording is the content, and AI narration is the connective tissue. There is no avatar — for a screen-first video, an AI face in a corner is the worst-value spend, and a real webcam intro is more authentic. The result is a real tutorial, not a synthetic explainer.
Why automate at zero revenue? This isn't about money. It (1) eliminates the work that otherwise stops videos from being made, and (2) is itself an open-source artifact + portfolio piece demonstrating agentic engineering. Both pay off regardless of channel growth.
| Tool | Role |
|---|---|
| Claude Code / Codex | Orchestrator + scriptwriter + QA (the brain) |
| Tella (+ MCP) | Screen recording, visual editing, the automation surface |
| ElevenLabs | Your cloned voice (Multilingual v2 default) |
| FFmpeg | Voiceover mux, music bed, logo overlay, optional burned captions |
| YouTube Data API | Scheduled publishing (private + publishAt) |
| CleanShotX | Optional quick screen grabs |
Eight stages. Human review checkpoints stay explicit where judgment matters; the agent handles the mechanical commands.
INPUTS: Tella recordings (screen + rough scratch narration) · house-style.md
· curated music library · logo.png · per-video intro/outro assets
1. INGEST Pull today's clips + timed transcripts from Tella (MCP).
│
2. SCRIPT Agent rewrites the messy transcript → clean, chunked
│ narration, grounded strictly in the transcript. ✋ you edit
3. VOICE ElevenLabs → one VO clip per chunk in your cloned voice
│ (durations drive the visual timing).
4. VISUAL EDIT Tella MCP: auto-zoom (declarative manualZoom cues) ·
(Tella MCP) fullscreen screen-only body layout · blur secrets ·
│ highlights.
5. ASSEMBLE+MUX Tella export (4K, muted) → FFmpeg: lay VO · music bed ·
(Tella+ffmpeg) logo overlay · optional captions · 3.5s chapter cards ·
│ prepend 🎬 real-face intro · append manual outro.
6. QA Agent checks: durations align, captions
│ match if enabled, secrets blurred, audio levels sane
│ → auto-fix. ✋ you watch
7. PUBLISH YouTube Data API: title + description (auto chapters),
│ privacyStatus=private + publishAt → scheduled, auto-public.
8. RETRO After upload/schedule, agent drafts durable lessons in
(self-improve) videos/<slug>/retro.json; approved rules merge into
│ house-style.md. Next video starts smarter.
The core insight — voice is the spine. Because there is no face in the body, footage is sized to the voice (not the reverse). Content is sacred; only idle/dead time is trimmed or sped up. If footage is all-essential, the agent lengthens the narration instead of cutting. Action timing comes from your live narration timestamps ("now I click Deploy") in the timed transcript, plus Tella's cursor/click data. Tolerance is loose — no lip-sync — so "the click is on screen while you talk about it" is enough.
Every major fork, with the reasoning. This is the project's decision record.
| # | Decision | Why |
|---|---|---|
| 1 | No HeyGen avatar | For a screen-first / PiP format, a corner avatar is the worst-value spend. Tella records a real webcam face for free, which is more authentic for a small technical channel. Saved ~$29/mo + ~$18/video. |
| 2 | Faceless body | You cannot have a real talking face and a rewritten voice — lips won't match. So the body shows the screen only; the cleaned voiceover plays over it. |
| 3 | Voiceover from a rewritten script ("Y-path") | The whole point is to ramble freely while recording, then let an agent rewrite and a cloned voice deliver it. Cleaning your raw audio ("X-path") was rejected — for this creator the hard part is performing, not editing. |
| 4 | Real-face intro, manual outro | A fresh ~15s intro (real face, real voice, today's clothes = internally consistent) gives a human moment with natural lip-sync. Current convention: intro is face-only inside the blue grid frame. For now, each video can supply a manual videos/<slug>/outro.mp4; the engine normalizes it and adds deterministic outro music underneath. A generated/reusable spoken outro is deferred to Stages 9–12. |
| 5 | Tella as the engine (not DIY CleanShotX + FFmpeg) | Tella's MCP is the automation surface and the auto-polish (zoom, layouts, blur/highlights). Dropping it to save $26 would forfeit both. CleanShotX kept for quick grabs only. |
| 6 | The coding agent orchestrates and writes the script (not ChatGPT) | Claude Code and Codex can both run the harness with the local tools; routing the script to ChatGPT adds a key + cost + a copy-paste step for no gain. Transcription is free from Tella. |
| 7 | ElevenLabs for voice (Multilingual v2) | Replaces HeyGen's voice role — not an added subscription. v2 for consistent narration; Eleven v3 optional for more expression. |
| 8 | FFmpeg for audio + branding | Tella's MCP can't replace audio or add music, so FFmpeg muxes the VO, normalizes intro/outro audio, mixes a music bed under the spoken intro and transition cards, overlays the logo, and can burn captions when --captions is explicitly requested. |
| 9 | Animated chapter cards generated as code | Tella has no native title-card creation. The agent renders branded 3.5s animated cards → MP4 → inserts them before chapter starts. Replaces the manual Google Slides → PNG step; timestamps also auto-fill YouTube description chapters. |
| 10 | YouTube Data API for publishing (not Tella upload) | FFmpeg holds the final cut, so we upload that. The Data API is also the only route to scheduled publishing (private + publishAt) — giving YouTube hours to index. |
| 11 | Self-improving loop | house-style.md is read every run and updated by a post-video retro, so corrections become permanent rules. This is what makes it an engine, not a one-off. |
| 12 | Secret-leak QA gate: balanced | Flags obvious keys/.env/tokens before publish without paranoid-blocking anything key-shaped. |
- HeyGen avatar (any usage) — uncanny and low-value in a screen-first format; cost scales with avatar minutes.
- X-path (clean real recorded audio) — doesn't reduce the creator's actual pain (performing), only engineering risk. Knowingly traded engineering risk for the freedom to ramble.
- CleanShotX + FFmpeg only — free, but throws away the MCP automation surface and the auto-polish; false economy.
- ChatGPT for transcribe/script — Tella transcribes for free; the coding agent scripts natively. ChatGPT only as an optional manual paste step.
- fal.ai / OpenRouter / kie.ai for voice (now) — kept as future cost/flexibility options behind the pluggable TTS interface; none beats a professional clone of your voice today.
| Item | Role | Cost |
|---|---|---|
| Tella Pro | Record + edit engine + MCP | $26/mo |
| ElevenLabs Creator | Cloned voice | $22/mo (or Starter $5 to validate) |
| Claude Code / Codex | Orchestrator + scriptwriter | already owned |
| FFmpeg / CleanShotX | Audio/branding / capture | free |
| YouTube Data API | Publishing | free |
| HeyGen | — cut — | $0 |
| Fixed total | ~$48/mo | |
| Per ~4.5-min video | within plan limits | ~$0 (~20+ videos/mo before overage) |
For reference, the original avatar-based concept was ~$70/mo + ~$18/video.
| Stage | Owner | Input → Output |
|---|---|---|
| 1. Ingest | make-video skill (Tella MCP) |
Tella clips → transcript.json |
| 2. Script | Agent ✋ | transcript + house-style.md → script.json |
| 3. Voice | src/elevenlabs.ts |
script.json → vo/*.mp3 |
| 4. Visual edit | scripts/plan-zooms.ts + Agent (Tella MCP) |
script.json zoom cues → zoom-plan.json → zoomed recording.mp4; body layout convention is fullscreen screen-only; blur/highlights are applied through Tella MCP when available |
| 5. Mux | scripts/make-video.ts + src/finish.ts (FFmpeg) |
visuals + VO + music + logo + optional captions + 3.5s chapter cards + intro + outro → final.mp4 + adjustments.json |
| 6. QA | Agent ✋ | final.mp4 + adjustments.json → checks pass / fixes |
| 7. Publish | scripts/publish.ts (YouTube API) |
final.mp4 + metadata.json → scheduled video |
| 8. Retro | Agent + scripts/retro.ts |
approved corrections → house-style.md |
Guiding principle: agents do judgment, scripts do determinism. The LLM handles taste (script, zoom placement, music choice, QA). Small bun-run TypeScript scripts handle the mechanical work (API calls, FFmpeg, upload) so they're reliable, testable, and re-runnable without spending tokens. Every stage reads/writes files in videos/<slug>/, so any stage re-runs independently.
Every stage validates its own outputs before the next starts, and every stage is idempotent + resumable — a failed run resumes from the broken stage and never re-spends ElevenLabs/YouTube quota.
Hard gates (blocking):
- 🔒 Cost — script must pass your ✋ review before any ElevenLabs spend.
- 🔒 Secret-leak (balanced) — Stage 6 scans rendered frames for visible keys/
.env/tokens; if found and unblurred, loops back toadd_blurbefore publish. - 🔒 Safe publish — always upload
private+publishAt; verify success; auto-goes-public on schedule.
Graceful degradation: ElevenLabs down → pause; Tella export fails → FFmpeg from raw clips; card render fails → simple text-card fallback. A --dry-run flag runs Stages 1–6 and stops before publish.
To de-risk the two unproven assumptions (rambling → a good script; footage↔voice re-sync feeling natural), build in this order:
- Thin end-to-end Y slice — one short recording through every stage with minimal polish. Validates the core magic before investing in polish.
- Polish — animated chapter cards, music, fullscreen body layout, blur, highlights.
- Self-improving loop —
house-style.md+ the retro stage. - Future stages 9–12 — see
plan.md.
aimh-video-engine/
├── AGENTS.md # Codex project guidance
├── .agents/skills/make-video/SKILL.md # Codex-discoverable make-video mirror
├── .claude/skills/make-video/SKILL.md # the make-video orchestrator skill
├── src/
│ ├── types.ts # shared types
│ ├── transcript.ts # transcript loader/validator
│ ├── align.ts # planSegments() — sizes footage to voice (the re-sync core)
│ ├── ffprobe.ts # ffprobeDuration()
│ ├── elevenlabs.ts # synthesizeChunk() — cloned-voice VO (cached)
│ ├── adjustments.ts # adjustments.json + review summary rows
│ └── finish.ts # assembleVideo() — FFmpeg cut/speed/pad/concat/mux
├── scripts/make-video.ts # CLI: script.json + recording.mp4 -> final.mp4 + adjustments.json
├── tests/ # bun tests (align, transcript, finish)
├── videos/<slug>/ # per-video working dir (gitignored): recording.mp4 · *.srt · script.json · vo/ · final.mp4 · adjustments.json
├── docs/plans/ # implementation plans
├── plan.md # future stages 9–12
└── README.md
The core pipeline is implemented and runs today, plus intro/outro wrap, per-chunk auto-zoom (Stage 4), music, chapter cards, logo overlay, optional captions, TTS-safe
aimh.copronunciation, optional ElevenLabs pronunciation dictionary locators, fullscreen body-layout conventions, deterministic music under manual outros,adjustments.jsonreview summaries, scheduled YouTube publishing, and the post-video retro loop.
- Bun (JavaScript/TypeScript runtime)
- FFmpeg with libass (needed only when using
--captions).brew install ffmpegusually includes it — check withffmpeg -version | grep libass. If yours doesn't (e.g. a minimal build), install the fullerffmpeg-fullformula and pointFFMPEG/FFPROBEat it (see Environment Variables). - Claude Code or Codex for the agent-driven workflow
- A Tella account (Pro) with the MCP connected
- An ElevenLabs account (Creator) with a cloned voice
- A Google Cloud project with the YouTube Data API enabled (OAuth)
| Variable | Description |
|---|---|
ELEVENLABS_API_KEY |
ElevenLabs API key |
ELEVENLABS_VOICE_ID |
Your cloned voice id |
ELEVENLABS_PRONUNCIATION_DICTIONARY_ID / ELEVENLABS_PRONUNCIATION_DICTIONARY_VERSION_ID |
Optional. When both are set, every ElevenLabs request includes pronunciation_dictionary_locators. |
YOUTUBE_CLIENT_ID / YOUTUBE_CLIENT_SECRET |
YouTube Data API OAuth |
YOUTUBE_REFRESH_TOKEN |
Long-lived YouTube auth token |
FFMPEG / FFPROBE |
Optional. Paths to the ffmpeg/ffprobe binaries; default to those on PATH. Set these to a libass-enabled build (e.g. /usr/local/opt/ffmpeg-full/bin/ffmpeg) if your default ffmpeg lacks libass. |
Working Today
The intended operator flow is agent-driven. You should be able to tell Claude Code CLI or Codex CLI to “make video <slug>,” then review the checkpoints. The agent can run lower-level commands internally, but you should not have to remember the whole command set.
- Record your screen in Tella while rambling through what you're doing (and add zoom/blur in Tella if you like).
- Export the recording (
.mp4) and its subtitles (.srt) intovideos/<slug>/(asrecording.mp4+ the.srt). - In Claude Code or Codex, invoke the
make-videoskill — the agent readshouse-style.mdplus the.srt, writes a clean chunkedscript.json, and (after your ✋ approval) runsbun run make-video <slug>. - The engine synthesizes your cloned voice, re-times the footage to it, optionally inserts 3.5s chapter cards, wraps a real-face
intro.mp4plus manual per-videooutro.mp4, and writesvideos/<slug>/final.mp4(1080p H.264crf 18) plusvideos/<slug>/adjustments.json. It also overlays yourassets/logo.pngas a top-right watermark. The agent runs QA, surfaces the adjustments table (VO duration, captions, chapter timestamps, zoom/highlight/blur status, music, logo, intro/outro, final duration/resolution), you review the result (✋), and you tell the agent what needs correction. - If something is wrong, describe the issue in plain language. Examples: ElevenLabs mispronounced a term, Tella exported the wrong visual segment, a zoom/highlight/blur is misplaced, the intro/outro audio balance is off, or a layout/background needs changing. The agent should make the targeted fix, re-render, re-run QA, and ask you to review again.
- When approved, the agent uploads or schedules the video using the YouTube publishing step.
- At the end of the session, tell the agent:
run retro for <slug>. The agent writes durable lessons intovideos/<slug>/retro.json; you review the rules; then the agent runsbun run retro <slug> --applyto updatehouse-style.md.
- Body: fullscreen screen recording only. In Tella, use
screen-only/fullscreenfor the base layout and preserve the full screen withscreenFit: "letterbox"when Tella accepts it. No camera bubble, side-by-side, or presenter layout in the body. - Intro: face-only, real voice, real lip-sync. The current sample style is the webcam shot inside the blue grid frame; the pipeline preserves the intro's own audio and adds quiet music underneath when a body music track is selected.
- Outro: intended to be a fullscreen, faceless branded end-card. The code does not enforce that composition; it normalizes and appends the supplied
outro.mp4. - Captions: off by default for long-form. Use
--captionsonly for Shorts or explicit captioned variants.
The ElevenLabs call prepares TTS-only text before sending it to eleven_multilingual_v2, so written script/captions can keep aimh.co while synthesized speech receives A-I-M-H dot co. After changing script text or TTS pronunciation logic, delete the affected cached videos/<slug>/vo/<chunk>.mp3 files before re-running so the audio regenerates.
ElevenLabs also supports pronunciation dictionaries through the Text-to-Speech API. Set ELEVENLABS_PRONUNCIATION_DICTIONARY_ID and ELEVENLABS_PRONUNCIATION_DICTIONARY_VERSION_ID to pass pronunciation_dictionary_locators on each synthesis request.
The retro loop is intentionally review-gated. Use it after a video is uploaded or scheduled, when the session has enough context to know which corrections were durable.
- Finish/render/QA the video.
- Upload or schedule it to YouTube.
- At the end of the session, tell the agent:
run retro for <slug>. - The agent writes durable lessons into
videos/<slug>/retro.json. - You review the rules.
- The agent runs
bun run retro <slug> --apply.
For direct CLI use, create or review the retro file with:
bun run retro <slug>Replace the template with durable lessons that should apply to future videos, then apply them. The command refuses to apply the untouched placeholder template:
bun run retro <slug> --applyThe command merges approved rules into house-style.md and skips duplicates, so reruns are safe.
Deferred to Stages 9–12: generated spoken-outro wiring and word-level captions after the new-mic re-record.
The current publishing slice uploads videos/<slug>/final.mp4 as a private YouTube video by default, and optionally schedules it with publishAt.
- Create a Desktop OAuth client in Google Cloud with the YouTube Data API enabled.
- Set
YOUTUBE_CLIENT_IDandYOUTUBE_CLIENT_SECRETin.env. - Run
bun run youtube-login, open the printed URL, and add the printedYOUTUBE_REFRESH_TOKENto.env. - Add
videos/<slug>/metadata.json:
{
"title": "Video title",
"description": "Video description",
"tags": ["aimh", "video"],
"privacyStatus": "private",
"publishAt": "2026-06-25T14:00:00Z",
"selfDeclaredMadeForKids": false
}- Omit
publishAtfor a plain private upload. Include it for a scheduled release. The timestamp must be a valid RFC 3339 datetime. - Check the request without uploading:
bun run publish <slug> --dry-run. - Upload only when ready:
bun run publish <slug> --yes.
Core engine: Stages 1–8 (above). Deferred future stages live in plan.md:
- Stage 9 — Shorts: auto-generate vertical clips from the final video.
- Stage 10 — SEO pack: optimized titles, descriptions, tags, chapters.
- Stage 11 — Distribute: cross-post + schedule across platforms.
- Stage 12 — Thumbnails: automated, on-brand thumbnail generation.
See the open issues for a full list of proposed features and known issues.
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Commit your Changes (
git commit -m 'Add some AmazingFeature') - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Distributed under the MIT License. See LICENSE for more information.
Denny — denny@aimh.co
Website: https://www.aimh.co
Project Link: https://github.com/aimhco/aimh-video-engine
- Nate Herk — the inspiration
- Tella — screen recording + MCP
- ElevenLabs — voice cloning
- FFmpeg — the swiss-army knife
- Best-README-Template — README structure
- Shields.io — README badges