Real-Time Gesture Recognition with Arduino and Machine Learning

⚠️ Language Notice

All source code and comments in this repository are written in Italian, as the project was developed for a technical school assignment in Italy.

---

Project Overview

This project implements a real-time gesture recognition system using an ultrasonic distance sensor (HC-SR04), an Arduino Uno, and a machine learning classifier developed in Python.

Three gestures are recognized based on movement patterns detected by the sensor:

• Carezza (Stroke)
• Incoraggiamento (Encouragement gesture)
• Schiaffo (Slap)

Once detected, the result is displayed on a 16x2 LCD display connected to the Arduino board.

---

Objectives

• Integrate sensor data acquisition, machine learning, and embedded systems in a single functional pipeline.
• Classify human gestures in real-time with visual output on an LCD display.
• Provide a concrete example of how Artificial Intelligence can interact with the physical world.

---

Hardware Components

Component	Purpose
Arduino Uno	Controls the sensor and LCD display
HC-SR04 Sensor	Captures distance and movement information
LCD 16x2	Displays the recognized gesture and confidence
USB Connection	Serial communication between Arduino and PC

---

Software & Libraries

Python

• scikit-learn – Machine learning model (Random Forest Classifier)
• numpy & pandas – Data management
• joblib – Model serialization
• pyserial – Serial communication with Arduino

Arduino

• LiquidCrystal – Control of the LCD display
• Serial API – Communication with Python over USB

---

Workflow

1. Data Collection

   • The ultrasonic sensor measures distance variations during gestures.
   • Distance data is collected via Arduino and sent to Python over serial.

2. Model Training

   • Data is labeled and saved as .csv files.
   • A Random Forest Classifier is trained using scikit-learn.
   • The trained model and label encoder are saved for future use.

3. Real-Time Recognition

   • The sensor collects data for 2 seconds per gesture.
   • Python processes the input and predicts the gesture using the trained model.
   • The prediction is sent back to Arduino in the format:
     gesture;confidence

4. Output Display

   • Arduino receives the result and displays the gesture name and confidence percentage on the LCD.

---

Example Output

Python Console

Campioni raccolti: 85
Gesto rilevato: INCORAGGIAMENTO (67.1%)
Invio ad Arduino: incoraggiamento;67.1

LCD Display

incoraggiamento
Conf: 67.1%

---

Issues Encountered and Solutions

Issue	Solution
Serial port conflict	Cannot use Arduino Serial Monitor and Python at the same time. Use only Python for serial communication during testing.
LCD display did not update	Improved serial parsing and buffer management on Arduino.
Sensor not returning data	Increased the initial delay and added robust error handling in Python.
Gesture confusion	Improved data quality and collected more samples to reduce misclassification between similar gestures.

---

References

• scikit-learn: RandomForestClassifier
• Joblib – Model persistence
• Arduino Serial Communication
• HC-SR04 Datasheet

---

Author

Project developed as part of a technical school assignment in Informatics and Embedded Systems.