handGestV2 — Real-Time Hand Posture Matching Demo

A lightweight demo for real-time hand posture recognition using MediaPipe Hands and OpenCV. The system extracts a compact angle-based feature vector from hand landmarks and matches it against user-registered posture templates with tolerance gating — no trained model or dataset required.

---

Demo Scope

This repository is a lightweight interactive demo for static hand-posture recognition. It is not a benchmarked classifier, production gesture-recognition system, or research evaluation package. The matching logic uses template comparison with a tolerance threshold — not a learned model.

---

How It Works

1. Capture — read a frame from the webcam.
2. Detect — MediaPipe Hands locates 21 hand landmarks.
3. Extract features — compute the angle between a reference vector (wrist → middle-finger MCP) and each of the five fingertip vectors (wrist → tip), producing a 5-element angle vector in radians.
4. Match — compare the current angle vector against every registered posture template. A posture matches only if all five per-finger relative errors fall below the tolerance threshold (TOLERANCE, default 0.50). Among passing templates, the one with the smallest Euclidean distance wins.
5. Display — overlay the best match label and distance on the frame.

Angle Extraction (brief)

Each finger angle is the arc-cosine of the normalised dot product between the reference vector and the fingertip vector — a compact geometric representation that is less sensitive to absolute hand scale than raw pixel coordinates, though not formally invariant to arbitrary rotation or viewpoint.

---

Requirements

• Python 3.10–3.12
• Webcam accessible via DirectShow (Windows), V4L2 (Linux), or AVFoundation (macOS)

Dependencies

Package	Version	Role
mediapipe	0.10.21	Hand landmark detection (CPU)
opencv-python	4.10.0.84	Video capture and frame rendering
numpy	1.26.4	Angle computation

pywin32, SpeechRecognition, and pyaudio appear in requirements.txt but are not used by the core demo. They are left-over from earlier experimentation and can be safely removed.

---

Quick Start

# 1. Create & activate a virtual environment
py -3.11 -m venv venv            # Windows
.\venv\Scripts\Activate.ps1      # Windows PowerShell
# or
python3 -m venv venv && source venv/bin/activate   # macOS / Linux

# 2. Install dependencies
pip install -r requirements.txt

# 3. (Optional) Verify camera access
python quick_cam_test.py

# 4. Launch
python app.py

---

Usage

Runtime Controls

Key	Action
S	Register the current hand pose as a named posture template
ESC	Quit — releases camera and closes the window

Registering a Posture

With a right hand visible in the frame, press S. The terminal prompts for a label:

Detected angles (radians): [0.622 0.078 0.069 0.121 0.218]
Enter name for this posture: open_palm
✅ Registered posture 'open_palm' and saved.

The template is appended to registered_postures.json immediately — no restart required.

Live Matching

Once at least one posture is registered, every frame is matched automatically. The best match (if within tolerance) is rendered as an overlay:

open_palm (0.037)

where 0.037 is the L2 distance between the current and template angle vectors.

---

Project Structure

handGestV2/
├── app.py                      # Main loop — capture → detect → match → render
├── quick_cam_test.py           # Minimal camera sanity check
├── registered_postures.json    # Saved posture templates (auto-generated)
├── requirements.txt            # Dependencies
└── README.md

---

Limitations

• Static postures only — recognises fixed hand shapes, not dynamic gesture sequences or motion trajectories.
• Template matching, not a learned model — accuracy depends on how representative the registered template is; there is no training, generalisation, or formal evaluation.
• Right hand only — the current implementation filters for MediaPipe's "Right" handedness label.
• Viewpoint and occlusion sensitive — landmark quality degrades with extreme angles, self-occlusion, or poor lighting.
• No formal benchmark — no test set, accuracy metrics, or cross-user evaluation is included.
• Single tolerance threshold — one global TOLERANCE value governs all postures; per-posture tuning is not supported.

---

Configuration

Parameter	Location	Default	Effect
TOLERANCE	app.py	0.50	Maximum allowed per-finger relative error. Lower = stricter matching.
max_num_hands	app.py	2	MediaPipe detects up to 2 hands; only the right hand is used.
min_detection_confidence	app.py	0.7	Landmark detector confidence threshold.
min_tracking_confidence	app.py	0.5	Landmark tracker confidence threshold.

---

Extending

• Left-hand support — remove the handedness == 'Right' filter in the main loop.
• Stricter / looser matching — adjust TOLERANCE in app.py.
• Edit templates — registered_postures.json is a plain {name: [θ₀..θ₄]} dict; edit directly.

---

Troubleshooting

Symptom	Likely Cause	Fix
"No working camera found"	No camera on indices 0–4	Check connection; close Zoom / Teams / OBS
Black / frozen frame	Camera opened but not streaming	Try cv2.VideoCapture(0, cv2.CAP_DSHOW)
No hand detected	Poor lighting or hand too close/far	Even lighting; keep hand 30–80 cm from lens
High latency	CPU-bound inference	Close background processes; reduce resolution
Import errors	Dependency mismatch	Re-run pip install -r requirements.txt in venv

---

License

See repository root for license details.