Real-time American Sign Language recognition • 97.2% accuracy • Privacy-first
Local ASL alphabet recognition (A-Z) from webcam. No cloud, no tracking, no data collection.
Performance: 97.2% test accuracy • <5ms latency • <5MB model size
# Clone & install
git clone https://github.com/borisgraudt/asl-ai.git
cd asl-ai
python3 -m venv env && source env/bin/activate
pip install -r requirements.txt
# Run
python main.pyPress q to quit.
# Prepare data
python scripts/prepare.py
# Train
python scripts/train.pyModels saved to models/, plots to plots/.
Full dataset available on Kaggle: 10,508 samples • 26 classes • Balanced
# Option 1: Via Kaggle CLI (recommended)
kaggle datasets download -d borisgraudt/asl-alphabet-hand-landmarks
unzip asl-alphabet-hand-landmarks.zip
# Kaggle dataset has 'landmarks/' folder, move contents to data/raw_gestures/
mkdir -p data/raw_gestures
cp -r landmarks/* data/raw_gestures/
rm -rf landmarks asl-alphabet-hand-landmarks.zip
# Option 2: Manual download
# 1. Download from https://kaggle.com/datasets/borisgraudt/asl-alphabet-hand-landmarks
# 2. Extract the zip file
# 3. Copy contents of landmarks/ folder to data/raw_gestures/Note: This repo includes sample data in data/sample_raw_gestures/ for quick testing. For full training, download the Kaggle dataset.
Webcam → MediaPipe → Preprocessing → Neural Net → Text
(256→128→64)
Input: 63D hand landmarks (21 points × xyz) Hidden: Dense layers with Dropout + BatchNorm Output: 26-class softmax (A-Z)
TensorFlow • MediaPipe • OpenCV • NumPy • scikit-learn
| Metric | Value |
|---|---|
| Test Accuracy | 97.2% |
| Inference | <5ms |
| Throughput | 21k samples/sec |
| Model Size | <5MB |
- Technical Report — Methodology & experiments
- Model Card — Model details & limitations
- Contributing — Development workflow
- Architecture — System design
Privacy: All processing runs locally. No data transmission.
Limitations:
- Static alphabet signs only (no sentences/grammar)
- Lighting & camera quality dependent
- Not a replacement for professional interpretation
Built with accessibility in mind. Feedback from deaf/HoH community welcome.
- Sentence-level translation
- Dynamic gestures
- Mobile deployment (iOS/Android)
- Multi-language sign languages
@software{asl_ai_2024,
title = {ASL\&AI: Real-Time American Sign Language Recognition},
author = {Graudt, Boris},
year = {2024},
url = {https://github.com/borisgraudt/asl-ai},
license = {MIT}
}MIT © 2024 Boris Graudt