AI-Powered Smart Vision Assistant for the Visually Impaired
Netra is a native Android application designed to act as a "digital eye" for blind and visually impaired users. It uses Deep Learning (YOLOv8) and Sensor Fusion to detect objects in real-time, estimate distance, and ensure user safety through fall detection and automated lighting.
- Real-Time Object Detection: Uses the YOLOv8 Nano model (running on-device via TensorFlow Lite) to identify 80+ types of objects (Person, Chair, Car, Obstacles, etc.) instantly.
- Offline Capability: All AI processing happens locally on the phone. No internet connection is required.
- Smart Text-to-Speech (TTS): Announces detected objects clearly.
- Anti-Spam Logic: Includes a "throttle" system to prevent the app from repeating the same object name continuously (waits 3 seconds before re-announcing).
- Proximity Alert: The phone vibrates when an object gets too close (filling >60% of the screen), warning the user of potential collisions.
- Fall Detection: Monitors the accelerometer for sudden G-force spikes (> 3G). If a fall is detected, it triggers a loud SOS Siren and voice alert.
- Smart Flashlight: Automatically turns on the camera flash if the environment is dark (< 10 lux) and objects are detected, ensuring the AI can still "see" in the dark.
- Language: Kotlin
- Platform: Android (Min SDK 24 / Android 7.0+)
- Camera: CameraX (ImageAnalysis use case)
- Machine Learning: TensorFlow Lite (Interpreter API)
- Model: YOLOv8 Nano (
int8/float32) - Sensors: Accelerometer (Fall Detection), Light Sensor (Auto-Flash)
| Object Detection | Night Mode | Fall Alert |
|---|---|---|
| (Place screenshot here) | (Place screenshot here) | (Place screenshot here) |
Blue bounding boxes indicate detected objects with confidence scores.
- Android Studio (Koala/Ladybug or newer)
- Android Device (Android 7.0 or higher)
- Clone the Repository
git clone [https://github.com/yourusername/netra.git](https://github.com/yourusername/netra.git)
- Add the Model File (Crucial)
- Download the
yolov8n.tflitemodel (approx 6MB). - Place it in:
app/src/main/assets/yolov8n.tflite. - Ensure
labels.txtis also in the same folder.
- Download the
- Build the Project
- Open in Android Studio.
- Sync Gradle files.
- Run on a physical device (Emulators may lag with CameraX).
- Input: CameraX captures frames at 30fps.
- Processing:
- Frames are converted to Bitmaps and resized to 640x640.
- YoloDetector runs the TFLite interpreter.
- Raw output (8400 boxes) is filtered using Non-Maximum Suppression (NMS) to find the best matches.
- Logic Controller:
- If object confidence > 60% -> Draw Box.
- If box width > 60% of screen -> Vibrate.
- If new object -> Speak (TTS).
- If light < 10 lux -> Toggle Flash.
- Output: Overlay View updates with blue boxes; TTS speaks the result.
- SMS Integration: Send GPS location to emergency contacts upon fall detection.
- "Find My" Mode: Voice command to search for specific items (e.g., "Find my keys").
- Distance Estimation: Announce distance in meters/feet.
This project is open-source and available under the MIT License.