✍️ HandWrite

AI-powered text-to-handwriting synthesis engine.
Convert typed text into realistic handwritten documents with customizable imperfections, layout, and style.

---

📖 Overview

HandWrite is a full-stack application that transforms typed text into realistic handwritten documents. It uses a Graves RNN neural network to generate handwriting strokes as SVG paths, then composites them onto paper backgrounds with configurable imperfections — pressure variation, ink smudging, wobble, and more — producing output that closely mimics genuine handwriting.

Key Features

• AI Handwriting Generation — Graves RNN model generates natural stroke patterns from arbitrary text input
• Multi-page Document Support — Automatically paginates long text across multiple pages with consistent sizing
• Custom Paper Backgrounds — Upload your own ruled/blank paper scans as backgrounds
• Ink Color Control — Choose any ink color for the generated handwriting
• Layout Customization — Configurable margins, line spacing (0.1×–3.0×), word spacing, and page rotation
• Natural Variation System — Randomize line start positions, word spacing, line spacing, line inclination, and per-word inclination for realistic imperfection
• Imperfections Engine — Stroke wobble, pen pressure variation, slant inconsistency, random strikethroughs, ink smudging, and bleed-through effects
• Mixed Character Rendering — Unsupported characters automatically rendered with a handwriting-style font fallback (Caveat)
• Export Options — Download as individual PNGs, ZIP archive, or multi-page PDF
• Real-time Progress — WebSocket-based live generation progress updates
• Session Management — Isolated sessions with automatic cleanup (24-hour TTL for inactive sessions; completed sessions persist until deleted)

---

🏗️ Architecture

HandWrite/
├── backend/                    # FastAPI Python backend
│   ├── app/
│   │   ├── main.py             # FastAPI app entry point & CORS setup
│   │   ├── config.py           # Pydantic settings (env-based config)
│   │   ├── api/                # REST & WebSocket endpoints
│   │   │   ├── session.py      # Session creation
│   │   │   ├── upload.py       # Paper & style image upload
│   │   │   ├── generation.py   # Page generation orchestration
│   │   │   ├── download.py     # PNG/ZIP/PDF export
│   │   │   ├── pageset.py      # Page set save/load (ZIP + manifest)
│   │   │   ├── queue.py        # Background generation queue API
│   │   │   └── websocket.py    # Real-time progress broadcast
│   │   ├── models/             # AI model implementations
│   │   │   ├── base.py         # Abstract HandwritingModel base class
│   │   │   ├── graves_rnn.py   # Graves RNN (primary, fully implemented)
│   │   │   ├── hwt.py          # HWT GAN (stub)
│   │   │   ├── diffusion_pen.py# DiffusionPen (stub)
│   │   │   ├── one_dm.py       # One-DM (stub)
│   │   │   └── registry.py     # Lazy-loading model registry
│   │   ├── models_src/         # Third-party model source code
│   │   │   └── handwriting-synthesis-master/  # Graves RNN implementation
│   │   ├── compositor/         # Image composition pipeline
│   │   │   ├── layout.py       # Line positioning, spacing, inclination
│   │   │   ├── renderer.py     # Page rendering & word inclination
│   │   │   ├── imperfections.py# Wobble, pressure, slant, smudge, bleed-through
│   │   │   └── baseline.py     # Baseline utilities
│   │   ├── schemas/
│   │   │   └── models.py       # Pydantic request/response models
│   │   └── services/
│   │       ├── session_manager.py  # Session lifecycle & cleanup
│   │       ├── queue_worker.py     # Background queue processing
│   │       ├── image_processor.py  # Image processing utilities
│   │       └── strikethrough.py    # Strikethrough effect generation
│   ├── scripts/
│   │   └── test_graves_rnn.py  # Model integration test
│   ├── requirements.txt        # Python dependencies
│   └── Dockerfile              # Backend container definition
│
├── frontend/                   # React + Vite frontend
│   ├── src/
│   │   ├── App.jsx             # Main application shell
│   │   ├── main.jsx            # React entry point
│   │   ├── index.css           # Global styles & design tokens
│   │   ├── api/
│   │   │   └── client.js       # Axios API client
│   │   ├── store/
│   │   │   └── appStore.js     # Zustand global state
│   │   ├── hooks/
│   │   │   └── useWebSocket.js # WebSocket hook for live progress
│   │   └── components/
│   │       ├── Layout/         # Header, Sidebar (page previews)
│   │       ├── Controls/       # LayoutControls, ImperfectionsPanel
│   │       ├── Upload/         # PaperUpload, StyleUpload, TextAreaSelector
│   │       ├── Generation/     # GenerationProgress overlay
│   │       ├── Session/        # SessionNameModal, SessionsPanel
│   │       └── common/         # Slider, Button, Card
│   ├── package.json
│   ├── vite.config.js
│   └── index.html
│
├── docker-compose.yml          # Full-stack Docker orchestration
└── README.md

---

🖥️ Local Deployment

Prerequisites

Requirement	Version	Notes
Python	3.10+	3.10 recommended (TensorFlow compatibility)
Node.js	18+	LTS recommended
npm	9+	Comes with Node.js
Git	Any	To clone the repository
Cairo	System	Required by cairosvg for SVG→PNG conversion

System Dependencies

Windows

# Install Cairo via GTK (required for cairosvg)
# Download GTK3 runtime from: https://github.com/niconiahi/gtkwin32/releases
# Or install via Chocolatey:
choco install gtk-runtime

# Alternatively, install MSYS2 and use:
pacman -S mingw-w64-x86_64-cairo

Ubuntu / Debian

sudo apt-get update && sudo apt-get install -y \
    build-essential libcairo2-dev libglib2.0-0 \
    fonts-dejavu wget

# Optional: Install Caveat handwriting font for better fallback rendering
mkdir -p ~/.local/share/fonts
wget -O ~/.local/share/fonts/Caveat-Regular.ttf \
    "https://github.com/google/fonts/raw/main/ofl/caveat/Caveat%5Bwght%5D.ttf"
fc-cache -f -v

macOS

brew install cairo pango gdk-pixbuf libffi

---

Option A: Manual Setup (Recommended for Development)

1. Clone the Repository

git clone https://github.com/yourusername/HandWrite.git
cd HandWrite

2. Backend Setup

# Create and activate a virtual environment
cd backend
python -m venv venv

# Windows
.\venv\Scripts\activate

# Linux/macOS
source venv/bin/activate

# Install Python dependencies
pip install -r requirements.txt

⚠️ TensorFlow Note: The project ships with GPU-capable TensorFlow (tensorflow[and-cuda]) — it automatically runs on CPU when no GPU is present. The Graves RNN model depends on tf-keras for backward compatibility with TF1-style code, so TF_USE_LEGACY_KERAS must be set before running:

$env:TF_USE_LEGACY_KERAS = "1"

On Linux/macOS:

export TF_USE_LEGACY_KERAS=1

On machines without a GPU you can substitute tensorflow-cpu in backend/requirements.txt to skip the CUDA libraries.

3. Start the Backend

# From the backend/ directory, with venv activated
cd backend
uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload

The API will be available at http://localhost:8000. Verify with:

curl http://localhost:8000/health
# Expected: {"status":"ok"}

The first request that triggers generation will take longer as the Graves RNN model loads its weights (~40MB) into memory. Subsequent requests will be fast.

4. Frontend Setup

Open a new terminal window:

cd frontend
npm install

5. Start the Frontend

npm run dev

The UI will be available at http://localhost:3000.

💡 API URL: The frontend defaults to http://localhost:8000 for the backend. If your backend runs on a different host/port, set the environment variable before starting:

# Linux/macOS
VITE_API_URL=http://your-backend:8000 npm run dev

# Windows PowerShell
$env:VITE_API_URL = "http://your-backend:8000"; npm run dev

---

Option B: Docker Compose

# From the project root
docker-compose up --build

Service	URL
Frontend	http://localhost:3000
Backend	http://localhost:8000

Docker volumes:

• model_weights — Persists downloaded model weights (MODEL_WEIGHTS_DIR) between container restarts
• session_data — Persists session data (TEMP_DIR) between container restarts
• ./backend/app — Mounted for live code reload during development
• ./frontend/src — Mounted for hot module replacement

To run in the background:

docker-compose up -d --build

To stop:

docker-compose down

Enabling GPU Acceleration (Docker)

The backend is configured to use your NVIDIA GPU if available, which dramatically speeds up generation. To enable this, you must prepare your host machine:

For Windows (Docker Desktop)

1. Install NVIDIA Drivers: Ensure the latest NVIDIA drivers are installed on your Windows host.
2. Enable WSL 2: Ensure WSL 2 is installed and enabled on your system.
3. Configure Docker Desktop: Open Docker Desktop settings, navigate to General, and ensure Use the WSL 2 based engine is checked. Docker Desktop automatically handles the NVIDIA Container Toolkit integration.

For Linux

1. Install NVIDIA Drivers: Install the proprietary NVIDIA drivers for your distribution.
2. Install NVIDIA Container Toolkit:

   curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
     && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
     sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
     sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
   
   sudo apt-get update
   sudo apt-get install -y nvidia-container-toolkit
   sudo nvidia-ctk runtime configure --runtime=docker
   sudo systemctl restart docker

3. Once configured, running docker-compose up --build will automatically utilize the GPU.

---

🚀 Usage Guide

Basic Workflow

1. Open the app at http://localhost:3000
2. Upload paper (optional) — Upload a scanned paper image as the background. If skipped, a plain white A4 page is used.
3. Select model — Choose from the dropdown in the header. Currently, Graves RNN is the only fully implemented model.
4. Type your text in the central editor panel
5. Adjust layout settings in the left sidebar:
   • Margins, ink color
   • Line & word spacing
   • Natural variation controls
6. Configure imperfections for realism:
   • Stroke wobble, pressure variation, slant
   • Random strikethroughs
   • Ink smudge & bleed-through
7. Click "Generate" — The system will:
   • Split text into lines
   • Generate handwriting strokes via the AI model
   • Apply imperfections
   • Composite onto the paper background
   • Automatically paginate if text overflows
8. Preview & download pages from the right sidebar. Export as PNG, ZIP, or PDF.

Layout Controls

Control	Range	Description
Line Spacing	0.1–3.0	Vertical space multiplier between lines
Word Spacing	5–50 px	Horizontal gap between words
Line Start Randomness	0–50 px	Random horizontal offset for each line start
Word Spacing Randomness	0–100%	Random jitter on word spacing
Line Spacing Randomness	0–100%	Random variation in vertical spacing between lines
Line Inclination Randomness	0–3°	Random tilt applied to entire lines
Word Inclination Randomness	0–5°	Random tilt applied to individual words
Page Rotation	±15°	Global rotation of all handwriting on the page

Imperfection Controls

Control	Range	Description
Stroke Wobble	0.0–1.0	Per-scanline displacement simulating hand tremor
Pressure Variation	0.0–1.0	Alpha modulation simulating pen pressure changes
Slant Inconsistency	0.0–1.0	Affine shear to simulate natural handwriting slant
Strikethrough Probability	0.0–0.3	Chance of adding a strikethrough to any word
Ink Smudge	0.0–1.0	Localized Gaussian blur to simulate ink smudging
Bleed-through	0.0–1.0	Simulated ink bleed from reverse side

---

🔌 API Reference

Base URL: http://localhost:8000

Health Check

GET /health
→ {"status": "ok"}

Session Management

POST /session/new
→ {"session_id": "uuid-string"}

File Upload

POST /session/{session_id}/upload-paper
Body: multipart/form-data (files[])
→ {"message": "...", "files": ["/path/to/paper_....png"]}

POST /session/{session_id}/upload-style
Body: multipart/form-data (files[])
→ {"message": "...", "files": ["/path/to/style_....png"]}

Generation

POST /session/{session_id}/estimate-layout
Body: {"text": "...", "layout_config": {...}}
→ {"estimated_pages": N, "chars_per_page": N}

POST /session/{session_id}/generate-page
Body: {
  "page_index": 0,
  "text_slice": "Your text here",
  "layout": { ... LayoutConfig ... },
  "imperfections": { ... ImperfectionConfig ... },
  "model_name": "graves_rnn"
}
→ {"message": "...", "page_url": "...", "lines_placed": N, "total_lines": N, "chars_consumed": N}

Download

GET /session/{session_id}/page/{page_index}
→ PNG image

GET /session/{session_id}/download/page/{page_index}
→ PNG file download

GET /session/{session_id}/download/all?format=zip
→ ZIP archive of all pages

GET /session/{session_id}/download/all?format=pdf
→ Multi-page PDF document

WebSocket (Live Progress)

WS /session/{session_id}/ws

Messages received (single-page generation):
  {"type": "generation_start", "page_index": N}
  {"type": "model_loading", "page_index": N}
  {"type": "model_loaded", "page_index": N}
  {"type": "line_progress", "line_index": N, "total_lines": N, "page_index": N}
  {"type": "post_processing", "current": N, "total": N, "page_index": N}
  {"type": "page_complete", "page_index": N}

Messages received (queue-based generation):
  {"type": "queue_update", "sessions": [...]}
  {"type": "queue_progress", "session_id": ..., "phase": ..., "line_index": N, "total_lines": N, ...}
  {"type": "queue_page_complete", "session_id": ..., "page_index": N, "pages_generated": N}
  {"type": "queue_session_complete", "session_id": ..., "pages_generated": N}

---

🧠 AI Models

Graves RNN (Primary — Fully Implemented)

Based on Alex Graves' 2013 paper "Generating Sequences With Recurrent Neural Networks". This model generates handwriting as a sequence of pen strokes using a mixture density network conditioned on character input.

• Input: Text string
• Output: SVG stroke paths → rasterized to PNG
• No style reference images required (uses 13 built-in handwriting styles)
• Supported characters: A-Z (except Q, X, Z → mapped to alternatives), a-z, 0-9, common punctuation
• Unsupported characters: Rendered with Caveat handwriting font fallback

Other Models (Stubs — Not Yet Implemented)

Model	Type	Style Refs	Status
HWT	GAN	15 images	Stub
DiffusionPen	Diffusion	5 images	Stub
One-DM	Diffusion	1 image	Stub

---

⚙️ Environment Variables

Variable	Default	Description
TF_USE_LEGACY_KERAS	1	Required. Use legacy Keras API
PYTHONPATH	/app (Docker)	Python module resolution path
MODEL_WEIGHTS_DIR	weights	Directory for model weight files (Graves RNN checkpoints)
TEMP_DIR	temp/handwrite_sessions	Session data storage directory
SESSION_TTL_HOURS	24	Inactive sessions are auto-cleaned after this many hours
VITE_API_URL	http://localhost:8000	Backend API URL for the frontend

---

🐛 Troubleshooting

cairosvg import error / DLL not found (Windows)

Cairo native libraries must be installed system-wide. Install GTK3 runtime or use MSYS2. See System Dependencies.

TensorFlow / Keras version conflict

Make sure TF_USE_LEGACY_KERAS=1 is set. The Graves RNN model depends on tf-keras for backward compatibility with TF1-style code.

# Windows PowerShell
$env:TF_USE_LEGACY_KERAS = "1"

Model loads slowly on first request

The Graves RNN model weights (~40MB) are loaded lazily on first generation request. This is normal — subsequent requests reuse the loaded model.

Frontend shows "Session not found" errors

The backend auto-cleans inactive sessions after 24 hours (SESSION_TTL_HOURS). If you leave the tab idle, the session may expire. Refresh the page to create a new session.

Port conflicts

• Backend default: 8000 — change with uvicorn app.main:app --port XXXX
• Frontend default: 3000 — change in frontend/vite.config.js (server.port)

---

📄 License

This project is for personal / educational use. The Graves RNN model source is from sjvasquez/handwriting-synthesis.