Audio Transcription to Video Generator

A Python application that automatically transcribes audio files and generates vertical videos (1080x1920) with synchronized captions - perfect for social media shorts!

Features

• 🎤 Automatic Transcription: Uses OpenAI's Whisper for accurate speech-to-text
• 🎬 Video Generation: Creates vertical videos (1080x1920) with synchronized captions
• 🎨 Customizable Styling: Configure fonts, colors, and text positioning
• 🖼️ Image Watermarks: Add logo/branding with adjustable opacity, position, and size
• ⚙️ Flexible Configuration: JSON config file + command-line overrides
• 📱 Social Media Ready: Output format optimized for TikTok, Instagram Reels, YouTube Shorts

Installation

Prerequisites

• Python 3.8 or higher
• ffmpeg (required by MoviePy)

Install ffmpeg

On Ubuntu/Debian:

sudo apt update
sudo apt install ffmpeg

On macOS:

brew install ffmpeg

On Windows: Download from ffmpeg.org and add to PATH.

Install Python Dependencies

pip install -r requirements.txt

Note: First installation will download PyTorch and Whisper models (~2GB). This may take a few minutes.

Quick Start

Basic usage with default settings:

python main.py input.mp3 output.mp4

This will:

1. Transcribe input.mp3 using Whisper
2. Generate output.mp4 with white text on black background
3. Use default font (Arial) and text styling

Configuration

Using config.json

Edit config.json to customize default settings:

{
  "video": {
    "width": 1080,
    "height": 1920,
    "fps": 30,
    "background_color": "#000000"
  },
  "text": {
    "font": "Arial",
    "font_size": 60,
    "color": "#FFFFFF",
    "position": "center",
    "stroke_color": "#000000",
    "stroke_width": 2,
    "max_words_per_line": 3,
    "interline": 20
  },
  "transcription": {
    "model": "base",
    "language": null
  },
  "watermark": {
    "enabled": false,
    "image_path": null,
    "opacity": 0.7,
    "position": "bottom-right",
    "size": 0.15
  }
}

Configuration Options

Video Settings

• width, height: Video dimensions (default: 1080x1920 for vertical)
• fps: Frames per second (default: 30)
• background_color: Hex color code (e.g., "#000000" for black)
• preset: Encoding speed/quality balance - "ultrafast", "superfast", "veryfast", "faster", "fast", "medium", "slow" (default: "fast")
• threads: Number of CPU threads to use for encoding (default: 8)

Text Settings

• font: Font family name (e.g., "Arial", "Impact", "Courier")
• font_size: Text size in pixels
• color: Text color in hex (e.g., "#FFFFFF" for white)
• position: Text vertical position ("center", "top", "bottom")
• stroke_color: Outline color for better readability
• stroke_width: Outline thickness in pixels
• max_words_per_line: Words per caption (default: 3)

Transcription Settings

• model: Whisper model size - tiny, base, small, medium, large
  • tiny: Fastest, less accurate (~75MB)
  • base: Good balance (~150MB) - recommended
  • small: Better accuracy (~500MB)
  • medium, large: Best accuracy, slower (~1.5GB+)
• language: Language code (e.g., "en", "es", "fr") or null for auto-detect

Watermark Settings

• enabled: Set to true to enable watermark overlay
• image_path: Path to watermark image file (PNG with transparency recommended)
• opacity: Transparency level from 0.0 (invisible) to 1.0 (fully opaque)
• position: Watermark placement:
  • Corners: "top-left", "top-right", "bottom-left", "bottom-right"
  • Edges: "top-center", "bottom-center", "left-center", "right-center"
  • Center: "center"
• size: Watermark size as ratio of video width (e.g., 0.15 = 15% of video width)

Command-Line Overrides

Override config settings for individual runs:

# Custom colors
python main.py input.mp3 output.mp4 \
  --text-color "#FFFF00" \
  --background-color "#0000FF"

# Custom font and size
python main.py input.mp3 output.mp4 \
  --font "Impact" \
  --font-size 80

# Better transcription model
python main.py input.mp3 output.mp4 --model small

# Combine multiple options
python main.py input.mp3 output.mp4 \
  --font "Arial" \
  --font-size 70 \
  --text-color "#FFD700" \
  --background-color "#1a1a1a" \
  --model small

Usage Examples

Example 1: TikTok Style (Yellow text on black)

python main.py podcast.mp3 tiktok_short.mp4 \
  --font "Impact" \
  --font-size 80 \
  --text-color "#FFFF00" \
  --background-color "#000000"

Example 2: Instagram Reels (White text on gradient)

python main.py interview.mp3 reel.mp4 \
  --font "Arial" \
  --text-color "#FFFFFF" \
  --background-color "#1a1a2e"

Example 3: YouTube Shorts (Bold red on white)

python main.py tutorial.m4a short.mp4 \
  --font "Arial-Bold" \
  --text-color "#FF0000" \
  --background-color "#FFFFFF" \
  --model small

Supported Audio Formats

• MP3 (.mp3)
• WAV (.wav)
• M4A (.m4a)
• FLAC (.flac)
• OGG (.ogg)
• MP4 (.mp4 - audio will be extracted)

Output

• Format: MP4 (H.264 video + AAC audio)
• Resolution: 1080x1920 (vertical/portrait)
• Compatibility: Works on all major platforms (TikTok, Instagram, YouTube, etc.)

Troubleshooting

"ffmpeg not found"

Install ffmpeg using instructions above.

"Font not found" error

Use a system font. To list available fonts:

• Linux: fc-list : family | sort | uniq
• macOS: Available in Font Book app
• Windows: Check C:\Windows\Fonts

Safe fonts that work everywhere: Arial, Courier, Times-Roman

Video rendering is slow

• Try a smaller Whisper model (--model tiny or --model base)
• Reduce FPS in config ("fps": 24)
• Use shorter audio clips for testing

Text is cut off

• Reduce font_size in config
• Reduce max_words_per_line for longer words

Advanced Tips

Testing with a Short Clip

For quick testing, use a short audio clip (10-30 seconds) to avoid long rendering times.

Custom Config Files

Use different configs for different styles:

python main.py input.mp3 output.mp4 --config tiktok_style.json

Batch Processing

Process multiple files with a simple script:

for file in *.mp3; do
  python main.py "$file" "${file%.mp3}.mp4"
done

Project Structure

shorts-generator/
├── main.py              # Main entry point
├── transcriber.py       # Audio transcription module
├── video_generator.py   # Video generation module
├── config.json          # Default configuration
├── requirements.txt     # Python dependencies
└── README.md           # This file

License

This project is open source and available for personal and commercial use.

Contributing

Contributions are welcome! Feel free to submit issues or pull requests.

Credits

Built with:

• OpenAI Whisper - Speech recognition
• MoviePy - Video editing
• Pillow - Image processing