Gesture Vision is a computer vision-based sign language interpreter that detects hand gestures in real-time using MediaPipe and classifies them into sign language gestures. The system can recognize gestures, alphabets, and convert recognized text into speech.
✔ Real-time hand tracking using MediaPipe ✔ Gesture classification using rule-based landmark detection
✔ ASL alphabet recognition ✔ Sign language phrase detection ✔ Dataset loading for ML training
✔ Text-to-speech generation using gTTS
flowchart LR
A[Webcam Input] --> B[Hand Tracking - MediaPipe]
B --> C[Extract Hand Landmarks]
C --> D[Gesture Classifier]
D --> E[ASL Sign Recognition]
E --> F[Display Gesture Text]
F --> G[Text To Speech]
G --> H[Audio Output]
flowchart TD
A[Start Webcam] --> B[Detect Hand Landmarks]
B --> C[Extract Landmark Coordinates]
C --> D[Apply Rule-Based Classification]
D --> E{Gesture Identified}
E -->|Alphabet| F[ASL Alphabet Recognition]
E -->|Phrase| G[Sign Language Recognition]
F --> H[Display Letter]
G --> I[Display Gesture Text]
flowchart LR
A[hand_tracking.py] --> B[MediaPipe Hands]
B --> C[gesture_classifier.py]
C --> D[asl_sign_recognition.py]
D --> E[asl_alphabet_recognition.py]
E --> F[text_to_speech.py]
Gesture-Vision/
├── hand_tracking.py # Detects hand landmarks using MediaPipe
├── gesture_classifier.py # Classifies gestures using landmark rules
├── asl_sign_recognition.py # Recognizes ASL gestures and phrases
├── asl_alphabet_recognition.py # Detects ASL alphabet letters
├── dataset_loader.py # Loads dataset images for training
└── text_to_speech.py # Converts recognized text to speech
| Technology | Purpose |
|---|---|
| Python | Primary programming language |
| OpenCV | Video capture and image processing |
| MediaPipe | Hand tracking and landmark detection |
| NumPy | Numerical operations and array handling |
| TensorFlow | Dataset preparation and ML utilities |
| gTTS | Google Text-to-Speech generation |
1. Clone the repository
git clone https://github.com/akankshacore/Gesture-Vision.git
cd Gesture-Vision
2. Install dependencies
pip install opencv-python mediapipe numpy gTTS tensorflow
To run specific components of the system, execute the following commands:
- Hand Tracking:
python hand_tracking.py - Gesture Classification:
python gesture_classifier.py - ASL Sign Recognition:
python asl_sign_recognition.py - Alphabet Recognition:
python asl_alphabet_recognition.py - Speech Synthesis:
python text_to_speech.py
The system converts recognized text into audio using Google Text-to-Speech (gTTS).
Example output:
"Hello, how are you?"
- Train Deep Learning models (CNN/LSTM) for higher accuracy.
- Implement real-time sentence formation logic.
- Deploy as a mobile application using Flutter or React Native.
- Integrate Speech-to-Sign (reverse translation).
Akanksha