Audiobook Generator

Generate audiobooks with automatic character detection, dialogue attribution, text-to-speech, and video generation.

Features

• Character Detection - Automatically detect characters and attribute dialogue using BookNLP and regex-based fallback
• TTS Integration - Generate audio with ElevenLabs, assigning unique voices to each character
• Document Parsing - Extract text and images from PDF and EPUB files
• Video Generation - Create YouTube-ready videos with Ken Burns effects or AI animation

Installation

# Clone the repository
git clone https://github.com/keepkalm/audiobook_generator.git
cd audiobook_generator

# Install the package
pip install -e .

# Download spaCy model for BookNLP
python -m spacy download en_core_web_sm

# Install FFmpeg (required for video generation)
brew install ffmpeg  # macOS

Quick Start

Analyze a document for characters and dialogue

audiobook analyze book.pdf -o analysis.json --verbose
audiobook analyze novel.epub -o analysis.json
audiobook analyze story.txt -o analysis.json

Generate an audiobook

# Set your ElevenLabs API key
export ELEVEN_API_KEY="your-api-key"

# Generate audiobook from document
audiobook generate book.pdf -o audiobook.mp3

# Generate from previous analysis
audiobook generate analysis.json -o audiobook.mp3 --narrator "voice-id"

Create a video

# Generate video with Ken Burns effects
audiobook video book.pdf --audio audiobook.mp3 -o video.mp4

# Preview mode (fast, lower quality)
audiobook video book.pdf --audio audiobook.mp3 -o preview.mp4 --preview

# 4K output
audiobook video book.pdf --audio audiobook.mp3 -o video.mp4 --resolution 4k

CLI Commands

Command	Description
analyze	Analyze document for characters and dialogue
parse	Parse PDF/EPUB and extract text
generate	Generate audiobook with TTS
video	Generate video from document images and audio
voices	List available ElevenLabs voices
assign-voices	Preview voice assignments for characters
characters	List detected characters
segments	Display dialogue/narration segments
version	Show version and dependency status

Configuration

Environment Variables

Variable	Description
ELEVEN_API_KEY	ElevenLabs API key for TTS
REPLICATE_API_TOKEN	Replicate API token for AI animation

Video Options

Option	Values	Description
--resolution	480p, 720p, 1080p, 4k	Output resolution
--animation	ken-burns, static, ai	Animation style
--crossfade	seconds	Transition duration between images
--preview	flag	Fast preview mode

Project Structure

audiobook_generator/
├── src/audiobook_generator/
│   ├── analysis/          # Character detection & dialogue extraction
│   │   ├── booknlp_wrapper.py
│   │   ├── character_detector.py
│   │   ├── dialogue_extractor.py
│   │   └── models.py
│   ├── parsers/           # PDF/EPUB document parsing
│   │   ├── pdf_parser.py
│   │   ├── epub_parser.py
│   │   └── factory.py
│   ├── tts/               # Text-to-speech integration
│   │   ├── elevenlabs_tts.py
│   │   ├── voice_assigner.py
│   │   └── audio_generator.py
│   ├── video/             # Video generation
│   │   ├── ken_burns.py
│   │   ├── replicate_animator.py
│   │   ├── compositor.py
│   │   └── generator.py
│   └── cli.py             # Command-line interface
└── tests/

Dependencies

Python Packages

• typer - CLI framework
• rich - Terminal formatting
• booknlp - Literary text analysis
• spacy - NLP
• elevenlabs - TTS API
• pydub - Audio processing
• pymupdf - PDF parsing
• ebooklib - EPUB parsing
• beautifulsoup4 - HTML parsing
• replicate - AI animation API

System Dependencies

• FFmpeg - Video encoding (brew install ffmpeg)

How It Works

Character Detection

1. BookNLP Analysis - Processes text for character identification, coreference resolution, and quotation detection
2. Dialogue Extraction - Regex patterns identify quoted speech and dialogue tags ("said", "asked", etc.)
3. Speaker Attribution - Combines BookNLP output with conversation flow analysis to attribute dialogue

Voice Assignment

• Characters are assigned voices based on detected gender
• Male/female/neutral voice pools from ElevenLabs
• Narrator gets a distinct voice
• Fully customizable via JSON configuration

Video Generation

1. Image Extraction - Pull images from PDF/EPUB documents
2. Ken Burns Effects - Apply pan/zoom animations using FFmpeg
3. AI Animation (optional) - Use Stable Video Diffusion via Replicate
4. Composition - Combine clips with crossfades and sync to audio

Examples

Analyze and generate with custom voices

# Analyze the document
audiobook analyze novel.epub -o analysis.json

# Preview voice assignments
audiobook assign-voices analysis.json -o voices.json

# Edit voices.json to customize, then generate
audiobook generate analysis.json -o audiobook.mp3

Full pipeline: document to YouTube video

# 1. Parse and analyze
audiobook analyze book.pdf -o analysis.json --verbose

# 2. Generate audio
audiobook generate analysis.json -o audiobook.mp3

# 3. Create video
audiobook video book.pdf --audio audiobook.mp3 -o final.mp4 --resolution 1080p

License

MIT