Wearable Gesture-Controlled IoT for Two-Wheeled Delivery — a compact, low-power system (Arduino Nano / ESP32 + flex sensors + HC-05) that lets delivery riders accept/reject orders via natural hand gestures. Audio notifications play through a Bluetooth helmet; a companion Flutter app listens for gestures and maps them to actions in a mock delivery app — minimizing phone interaction and improving on-road safety.
GestureRide — Wearable Gesture-Controlled IoT for Delivery Riders
Tagline: Accept or reject delivery orders with a hand gesture — no phone-touch required.
Overview: This project implements a wearable IoT system enabling two-wheeled delivery riders to control essential delivery app actions (e.g., accept/reject orders) using simple hand gestures detected by flex sensors. The hardware sends gesture events over Bluetooth (HC-05 / ESP32 BLE), the companion Flutter app receives these events (even while running in background via an Android foreground service), plays an audio notification through a Bluetooth helmet, and maps gestures to mock delivery-app actions.
Goals: Reduce on-road smartphone interactions. Provide intuitive gesture mapping (swipe up = accept, swipe down = reject) Maintain low power, ergonomic form factor. Provide a mock delivery app to test integrations and UX.
Key features: Gesture detection using flex sensors (voltage divider + analog read). Bluetooth communication to companion smartphone app (HC-05 SPP or BLE). Audio notification played via helmet Bluetooth speaker. Gesture-to-action mapping (accept/reject) in a mock delivery app. Background gesture listener on Android (foreground service pattern). Minimal latency and ergonomic placement for quick gestures.
Hardware components: Arduino Nano or ESP32 / NodeMCU (ESP32 recommended for BLE & low-power) Flex sensors (2–3, placed on glove/finger) HC-05 Bluetooth module (if using Arduino Nano + classic SPP) Battery (3.7V Li-ion with boost to 5V if using Arduino Nano) — or 3.3V battery if using ESP32 (remember voltage compatibility) Charger / TP4056 module (if using Li-ion) Jumper wires, breadboard / small perfboard, enclosure such as a wrist band or glove patch CASON Helmet Bluetooth Device (or any helmet BT speaker) — used for audio output. (Example product link provided in mentor’s note.)
System architecture:
-
Sensor node (wearable) MCU (Arduino Nano or ESP32) reads flex sensors, runs a simple gesture recognition algorithm (thresholds/time windows or simple classifier), and sends compact gesture codes over Bluetooth. Power: battery + regulator/booster.
-
Smartphone companion app (Flutter) Bluetooth client: connects to HC-05 (SPP) or ESP32 BLE service. Background service / foreground notification to keep Bluetooth listening active. Plays audio notification via helmet speaker and updates the mock delivery app UI or triggers accept/reject actions.
-
Mock delivery app Receives orders (generated internally for demo) and exposes accept/reject endpoints. The companion app listens for gestures and performs the action in mock app programmatically (or via shared state).
Wiring & hardware notes (summary)
Important: verify voltage compatibility — HC-05 and Arduino Nano use 5V, ESP32 uses 3.3V. TX/RX level-shifting may be required.
Typical Arduino Nano + HC-05 + flex sensor wiring Flex sensor: form a voltage divider 5V → Flex sensor → Analog pin A0 → 10kΩ → GND (Analog read at A0; values vary with bend)
HC-05: HC-05 VCC → 5V (or 3.3V if module supports it) HC-05 GND → GND HC-05 TX → Arduino RX (D0) (use voltage divider if HC-05 TX > MCU Rx voltage requirements) HC-05 RX → Arduino TX (D1) (if Arduino TX is 5V and HC-05 RX requires 3.3V, use level shifter)
Common GND between battery, MCU, HC-05, sensors. Battery: If using Li-ion (~3.7V) and Arduino Nano (5V) route through boost converter; if using ESP32, ensure stable 3.3V regulator. ESP32 approach (recommended) Flex sensor voltage divider to ADC pin (3.3V reference) ESP32 BLE or SPP (Serial over USB or classic BT using modules). ESP32 can be powered from 3.7V via regulator recommended for stable 3.3V.
Gesture recognition strategy (simple / robust) Simple threshold method: sample analog values over a short window and detect BEND_UP / BEND_DOWN patterns. Debounce & smoothing: moving average filter and minimum time between gestures (e.g., 300 ms) to avoid false positives.
Mapping:
Swipe up (finger straightening quickly) → ACCEPT Swipe down (finger bending quickly) → REJECT Optional: Use a lightweight classifier (tiny decision tree) if multiple sensors and gestures required.
Example microcontroller sketch (Arduino Nano + HC-05)
Purpose: read flex sensor, detect simple gestures, send simple ASCII commands over Serial to HC-05. Tune thresholds after real-world testing. For ESP32, replace SoftwareSerial with hardware UART or BLE service.
Flutter companion app (high-level)
Main responsibilities: Connect to the wearable via Bluetooth (classic SPP for HC-05 or BLE for ESP32). Play audio notifications through connected helmet Bluetooth speaker when a new order arrives. Run a background/foreground service to keep Bluetooth open and respond to gestures when the device is locked or app is backgrounded (Android: use foreground service; iOS: limited background BT privileges). Expose a mock delivery UI (order incoming modal) and programmatically accept/reject based on gestures.
Recommended packages (Flutter): flutter_blue_plus or flutter_blue (BLE) OR flutter_bluetooth_serial (classic SPP, Android-focused) permission_handler (manage required permissions) audio_service + just_audio (for background audio playback / directing audio to BT helmet) flutter_foreground_task or workmanager / android_alarm_manager (for persistent background tasks; Android only real foreground service is needed for continuous BT) provider / bloc (state management)
High-level app logic
- Pair/connect to helmet and wearable. Ensure connection to wearable established.
- Mock delivery engine: a local module that periodically generates “incoming order” events (title, pickup, distance).
- When an incoming order is generated:
Play TTS or pre-recorded audio via just_audio/audio_service so the rider hears it through the helmet.
Start a short timer (e.g., 8–12s) waiting for a gesture input.
- Bluetooth listener: when a gesture command (ACCEPT/REJECT) is received, forward the action to the mock delivery UI and update state.
- Background behavior: run a foreground service on Android to continue listening while locked or app in background. Display a persistent notification while active (required for background BT).
Permissions to declare (Android):
BLUETOOTH, BLUETOOTH_ADMIN, BLUETOOTH_CONNECT, BLUETOOTH_SCAN (Android 12+), ACCESS_FINE_LOCATION (if required for BLE scanning on older Android), FOREGROUND_SERVICE, RECORD_AUDIO (if using TTS or audio capture), and WAKE_LOCK (optional).
Note on iOS: Background BLE is restricted; full background behavior may not be possible on iOS without special entitlements. Focus on Android for full background listening.
Mock delivery app
A lightweight Flutter app page that simulates incoming orders and displays order details. Accept/reject actions triggered either by on-screen buttons (for manual testing) or programmatically by the companion app’s Bluetooth listener. Use simple REST-like local events or shared state to simulate actions.
Setup & Installation
- Firmware (Arduino/ESP32)
Clone/checkout the firmware/ folder. Open the Arduino sketch in Arduino IDE or PlatformIO. Adjust pin numbers & thresholds in the code to match your wiring. Upload to your MCU (if using HC-05, ensure pairing settings 9600 baud are compatible).
- Flutter companion app
Install Flutter SDK (stable). git clone cd flutter_app flutter pub get Configure Android AndroidManifest.xml with required BT & foreground service permissions. Connect Android device and run: flutter run --release -d Pair the HC-05 with the phone manually (Settings → Bluetooth) before using classic SPP.
- Mock delivery app
cd mock_app flutter pub get flutter run to test locally.
Testing & debugging tips: Calibrate flex sensors: log raw ADC values over Serial to find comfortable thresholds. Use Serial Monitor / logs: print the gesture events on MCU and in Flutter logs to verify end-to-end flow. Test audio path: pair helmet and play a short audio from the phone (outside the app) to confirm the helmet audio works. Simulate orders: mock app produces orders; ensure companion app receives them and audio plays. Check background behavior on Android: when locking screen, ensure the foreground notification persists and Bluetooth events are still processed.
Security & privacy considerations
Bluetooth pairing: HC-05 typically uses PIN — avoid insecure pairing in production. Background services: require user consent and clear notification; inform users about continuous listening. Permissions: request only necessary permissions, explain purpose to users. No automatic remote control: mock app demonstrates concept only; do not integrate with real third-party delivery platforms without their APIs and security approval.
Limitations & future work: iOS background BLE and classic SPP backgrounding have significant restrictions — focus on Android for production-ready background behavior. Current gesture detection is threshold-based; future work can use a small ML model (TinyML) for higher accuracy and richer gesture vocabularies. Integrate with real delivery platforms via their official APIs (requires partner access & security clearance). Add haptic confirmation (vibration in helmet) for safe feedback without audio.
References
Research on haptic feedback & rider safety (e.g., Petermeijer et al., Prasad et al.) Libraries and Flutter packages used (listed inside each project pubspec.yaml). Datasheets for flex sensors, HC-05, Arduino/ESP32 used in docs/ folder.
Suggested repo structure
gesture-ride/ ├─ firmware/ │ ├─ arduino_nano/ │ │ └─ gesture_sender.ino │ └─ esp32/ │ └─ gesture_sender_esp32.ino ├─ flutter_companion/ │ ├─ lib/ │ ├─ android/ │ └─ pubspec.yaml ├─ mock_delivery_app/ │ ├─ lib/ │ └─ pubspec.yaml ├─ docs/ │ ├─ wiring_diagram.png │ └─ gesture_calibration_notes.md ├─ README.md └─ LICENSE
Quick start (developer friendly)
- Upload firmware/arduino_nano/gesture_sender.ino to your Arduino + HC-05.
- Pair HC-05 with Android phone (Settings → Bluetooth → pair with PIN 1234/0000).
- Run flutter_companion on Android device; connect to the wearable via Bluetooth from the app.
- Launch mock_delivery_app (on same device or separate) and simulate an incoming order.
- Make a gesture — companion app should receive ACCEPT / REJECT and update mock app.
Closing notes
This repository aims to be a working prototype demonstrating how a simple, low-cost wearable can reduce smartphone distraction for delivery riders. The combination of flex sensors + lightweight MCU + Bluetooth + a companion mobile app provides a feasible path toward safer last-mile delivery workflows. Calibration, user testing, and careful background/permission handling are essential before any real-world deployments.
License
MIT License — see LICENSE file.