Skip to content

SETUP.md

Latest commit

 

History

History
230 lines (168 loc) · 4.18 KB

SETUP.md

File metadata and controls

230 lines (168 loc) · 4.18 KB
Raw

Setup Guide

Requirements

  • Python 3.10+
  • CUDA 12.x (for GPU) or CPU-only mode
  • ffmpeg (for audio format conversion)
  • 4-8GB VRAM (GPU) or 8-16GB RAM (CPU)

Installation Methods

Docker (Recommended)

GPU Setup:

# Clone repository
git clone https://github.com/your-repo/Qwen3-TTS-FastAPI.git
cd Qwen3-TTS-FastAPI

# Copy environment file
cp .env.example .env

# Edit .env to configure model and settings
nano .env

# Build and run
docker-compose up -d

CPU Setup:

docker-compose -f docker-compose.cpu.yml up -d

Local Python Installation

# Create virtual environment
python -m venv venv
source venv/bin/activate  # Linux/Mac
# or: venv\Scripts\activate  # Windows

# Install dependencies
pip install -r requirements.txt

# Install ffmpeg
# Ubuntu/Debian:
sudo apt install ffmpeg
# macOS:
brew install ffmpeg
# Windows: Download from https://ffmpeg.org/download.html

# Copy and configure environment
cp .env.example .env
nano .env

# Run server
cd app
python server.py

Configuration

Environment Variables

Edit .env file:

# Model Selection
MODEL_NAME=0.6B                    # Options: 0.6B, 1.7B, or full HuggingFace path
# Full paths:
# - Qwen/Qwen3-TTS-12Hz-0.6B-Base
# - Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice
# - Qwen/Qwen3-TTS-12Hz-1.7B-Base
# - Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice
# - Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign

# Device
DEVICE=cuda:0                      # cuda:0, cuda:1, or cpu

# Server
HOST=0.0.0.0
PORT=8880

# Defaults
DEFAULT_VOICE=aiden
DEFAULT_FORMAT=mp3
DEFAULT_LANGUAGE=English

# Paths (Docker uses /app paths)
VOICES_DIR=/app/voices
CACHE_DIR=/app/cache

Model Selection Guide

Model VRAM Features
0.6B (0.6B-CustomVoice) ~2GB Preset speakers, voice cloning
1.7B (1.7B-CustomVoice) ~4GB Preset speakers, voice cloning, style instruction
Qwen/Qwen3-TTS-12Hz-0.6B-Base ~2GB Voice cloning only (no presets)
Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign ~4GB All features + voice design from text

Port Configuration

Default internal port is 8880. Docker maps this to external ports:

# docker-compose.yml
ports:
  - "8005:8880"  # Access at localhost:8005

Verification

Check Health

curl http://localhost:8880/health

Expected response:

{
  "status": "healthy",
  "model": "Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice",
  "device": "cuda:0",
  "has_preset_speakers": true,
  "supports_voice_design": false,
  "supports_voice_cloning": true,
  "supports_streaming": true,
  "supports_style_instruction": false
}

Test Generation

curl -X POST http://localhost:8880/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"input": "Hello, this is a test.", "voice": "aiden"}' \
  --output test.mp3

# Play the audio
ffplay test.mp3

Access Web UI

Open browser: http://localhost:8880/web/

Access API Docs

Open browser: http://localhost:8880/docs

Troubleshooting

CUDA Out of Memory

  • Use a smaller model (MODEL_NAME=0.6B)
  • Use CPU mode (DEVICE=cpu)
  • Close other GPU applications

Flash Attention Warning

Warning: flash-attn is not installed

This is normal for CPU mode. Flash Attention is GPU-only and not required.

ffmpeg Not Found

Install ffmpeg:

# Ubuntu/Debian
sudo apt install ffmpeg

# macOS
brew install ffmpeg

# Docker images include ffmpeg

Model Download Slow

Models are downloaded from HuggingFace on first run. Set cache directory:

export HF_HOME=/path/to/cache

Or in Docker, mount a volume:

volumes:
  - ./hf_cache:/root/.cache/huggingface

Permission Denied on voices/

chmod 755 voices/
# Or in Docker, check volume permissions

Gradio UI (Optional)

A separate Gradio interface is available:

cd ui
pip install gradio requests
python app.py

Or with Docker:

docker-compose --profile ui up -d

Access at: http://localhost:7860

Integration with Open WebUI

Configure TTS in Open WebUI settings:

  • TTS Engine: OpenAI
  • API Base URL: http://localhost:8880/v1
  • API Key: (leave empty or any value)
  • Model: tts-1
  • Voice: aiden (or any available voice)