GazePoint — Iris-Based Cursor Control

Control your computer cursor using only your eyes. GazePoint uses real-time iris tracking via MediaPipe to move the mouse, click, scroll, and type — hands-free.

---

Features

Feature	Description
Iris Cursor Control	Move the cursor by moving your eyes — no mouse needed
Dual-Eye Tracking	Averages both eyes for stability and noise reduction
Head Motion Compensation	Nose-tip tracking subtracts head drift so cursor stays locked to gaze, not head position
Adaptive Smoothing	Light smoothing during fast saccades, heavier smoothing at rest — cursor keeps up without jittering
Full-Screen Calibration	4-point calibration overlay maps your personal eye range to the actual screen corners
Dead Zone	Ignores micro eye movements to keep the cursor stable when fixating
Double-Blink → Left Click	Blink twice quickly (within 0.8 s) to left-click
Triple-Blink → Double Click	Blink three times quickly to double-click
Long Blink → Right Click	Hold eyes closed for 1.5 s to right-click
Left Wink → Scroll Up	Close only your left eye to scroll up
Right Wink → Scroll Down	Close only your right eye to scroll down
Voice Typing	Speak to type text using Google Speech Recognition
Configurable	All parameters (sensitivity, smoothing, dead zone, blink timing) adjustable in config.py

---

Quick Start

Prerequisites

• Python 3.11 (required for MediaPipe compatibility)
• Webcam
• Microphone (for voice typing only)

1. Clone the repo

git clone https://github.com/parth-garg01/GazePoint.git
cd GazePoint

2. Install dependencies

pip install opencv-python mediapipe pyautogui SpeechRecognition pillow screeninfo

3. Download the model

Download face_landmarker.task and place it in the project root directory.

4. Run

python main.py

On Windows with multiple Python versions use py -3.11 main.py

---

How to Use

Step 1 — Calibrate (recommended before first use)

1. Click the "Calibrate" button
2. A full-screen overlay appears with a red dot at each corner
3. Keep your head still — move only your eyes to look at each dot
4. Hold your gaze on each dot for 4 seconds (Top-Left → Top-Right → Bottom-Left → Bottom-Right)
5. The first 0.6 s per point is a settle period — your eyes just need to reach the dot, data collection starts after
6. Calibration bounds are logged in the app for verification

Step 2 — Start Tracking

1. Click "Start Eye Tracking"
2. Move your eyes to control the cursor

Gestures

Gesture	Action
Double-blink (2 blinks within 0.8 s)	Left click
Triple-blink (3 blinks within 1.2 s)	Double click
Hold eyes closed for 1.5 s	Right click
Close left eye only	Scroll up
Close right eye only	Scroll down

Step 3 — Voice Typing (optional)

1. Click "Start Voice Typing"
2. Speak clearly — recognized text is typed at the cursor position

---

Configuration

All tuning parameters are in config.py:

# Cursor reach — >1.0 lets you reach screen edges more easily
GAIN = 1.2

# Smoothing when gaze is still (higher = smoother but laggier)
SMOOTHING_ALPHA = 0.78
# Smoothing during fast eye movement (lower = more responsive)
SMOOTHING_ALPHA_FAST = 0.55

# Ignore movements smaller than this (normalized units)
DEAD_ZONE_NORM = 0.005

# Head compensation strength (0 = off)
HEAD_COMPENSATION_X = 0.20
HEAD_COMPENSATION_Y = 0.18

# Blink detection
BLINK_THRESHOLD   = 0.25   # EAR below this = eye closed
CLICK_COOLDOWN    = 0.8    # Min seconds between clicks
DOUBLE_BLINK_TIME = 0.8    # Window for second blink → left click
TRIPLE_BLINK_TIME = 1.2    # Window for third blink → double click
LONG_BLINK_TIME   = 1.5    # Hold closed this long → right click

# Wink / scroll
WINK_OPEN_THRESHOLD = 0.30   # EAR above this = eye is open
SCROLL_AMOUNT       = 3      # Lines per wink
WINK_COOLDOWN       = 0.6    # Min seconds between scrolls

# Calibration
CALIB_SECONDS_PER_POINT = 4    # Seconds to stare at each corner
CALIB_SETTLE_TIME       = 0.6  # Seconds to skip at start of each point

Tuning tips:

• Cursor too sensitive / shaky? → Increase SMOOTHING_ALPHA (try 0.85) or increase DEAD_ZONE_NORM (try 0.010)
• Cursor too slow to respond? → Decrease SMOOTHING_ALPHA (try 0.70) or lower SMOOTHING_ALPHA_FAST (try 0.45)
• Can't reach screen edges? → Recalibrate with eyes at real corners, or increase GAIN (try 1.4)
• Cursor drifts when head moves? → Increase HEAD_COMPENSATION_X/Y (try 0.30)
• Accidental clicks? → Raise BLINK_THRESHOLD (try 0.20) or increase CLICK_COOLDOWN

---

Project Structure

GazePoint/
├── main.py              # GUI application & calibration state machine
├── eye_tracker.py       # Iris tracking, head compensation, smoothing & gestures
├── voice_handler.py     # Speech-to-text via Google Speech Recognition
├── utils.py             # Math helpers (EAR, distance, clamping, screen size)
├── config.py            # All tunable parameters
├── test_imports.py      # Sanity check — verifies model, imports, screen size
├── requirements.txt     # Python dependencies
├── face_landmarker.task # MediaPipe model file (download separately)
└── .gitignore

---

Tech Stack

• MediaPipe — Face Landmarker with 478-point mesh including iris landmarks
• OpenCV — Webcam capture and frame processing
• PyAutoGUI — System cursor control, clicking, and scrolling
• SpeechRecognition — Voice-to-text via Google API
• Tkinter — GUI and calibration overlay

---

How It Works

1. Capture — Webcam frame is horizontally flipped for a natural mirror view
2. Detect — MediaPipe FaceLandmarker identifies 478 facial landmarks, including 4 iris points per eye and eyelid points
3. Normalize — Iris X is normalized by horizontal eye width; iris Y is normalized by vertical eyelid span (eye_h), not eye width — this gives equal sensitivity in both axes
4. Average — Both eyes are averaged to reduce noise
5. Head Compensation — Nose-tip position is smoothed and compared to its calibration reference; any drift is subtracted from the iris signal to cancel head movement
6. Dead Zone — Micro-movements below threshold are discarded before smoothing
7. Adaptive Smooth — Fast exponential smoothing (SMOOTHING_ALPHA_FAST) during saccades, heavier smoothing (SMOOTHING_ALPHA) when still
8. Scale — Calibrated eye range is mapped to full screen coordinates with a gain factor
9. Move — PyAutoGUI moves the system cursor to the computed position
10. Gesture Detection — Eye Aspect Ratio (EAR) detects blinks and winks; two quick blinks → left click, three quick blinks → double click, long hold → right click, single-eye wink → scroll

---

Troubleshooting

Problem	Solution
Cursor drifts when head moves	Recalibrate — or increase HEAD_COMPENSATION_X/Y
Vertical tracking weak / cursor won't go up or down	Recalibrate; ensure lighting is even so eyelids are visible
Model not found error	Place face_landmarker.task in the project root directory
Can't reach screen edges	Run calibration and look at the true screen corners
Accidental clicks	Increase BLINK_THRESHOLD (try 0.20) or raise CLICK_COOLDOWN
Voice typing not working	Check microphone permissions and internet connection
Cursor jitters at rest	Increase DEAD_ZONE_NORM (try 0.010)

---

License

This project is open source under the MIT License.

---

Acknowledgements

• MediaPipe by Google for face landmark detection
• PyAutoGUI for cross-platform cursor control