AI-Powered Skin Lesion Screening for Clinical Decision Support
This repository contains the prototype developed in 24 hours during the TinyHack Turin 2025 hackathon, awarded 1st place.
It is an experimental research prototype for skin lesion image classification.
Not a medical device. Not for diagnostic use. Use only for educational and research purposes.
MelaNoMore is a computer vision-based decision support tool designed to assist dermatologists in the early detection and classification of skin lesions. By combining edge AI on the Arduino Nicla Vision with a powerful server-side deep learning model, MelaNoMore provides a two-tier screening system that balances speed, accuracy, and accessibility.
- ⚡ Fast on-device screening - Binary classification (suspicious vs. non-suspicious) directly on the Nicla Vision
- 🧠 Advanced server-side analysis - Multi-class classification across 7 lesion types using Vision Transformer
- 📡 Seamless WiFi integration - Automatic image upload for suspicious cases
- 🎯 Distance guidance - Time-of-Flight sensor ensures correct positioning
- 💡 Intuitive LED feedback - Color-coded status for real-time user guidance
- 🌐 Web dashboard - Accessible interface for reviewing detailed predictions
MelaNoMore serves as a "second set of eyes" for dermatologists during clinical examinations:
- Pre-screening happens instantly on the handheld device, even offline
- Suspicious cases are automatically escalated to a more powerful model running on a server
- Detailed classification helps dermatologists make informed decisions about diagnosis and treatment
This workflow helps:
- ⏱️ Save time during patient consultations
- 🔍 Reduce diagnostic errors
- 🩺 Improve early detection of dangerous skin cancers like melanoma
MelaNoMore employs a two-tier classification system:
- Model:
fai-cls-n-coco(Focoos AI) - Task: Binary classification (suspicious vs. non-suspicious)
- Input: 96x96 RGB images
- Inference: Real-time, on-device
- Output: Classification + LED feedback
- Model: Vision Transformer Large (ViT-L/16)
- Task: Multi-class classification across 7 lesion types
- Parameters: ~305M
- Inference: Triggered only for suspicious cases
- Output: Detailed classification with confidence scores
[Nicla Vision] -> Capture Image -> On-Device Inference
|
Suspicious?
|
[YES] -> WiFi Upload -> [Flask Server] -> ViT-L/16 Inference
| |
[NO] Web Dashboard
|
Green LED
HAM10000 (Human Against Machine with 10,000 training images) Source: ISIC Archive - Collection 212
- Actinic Keratoses
- Basal Cell Carcinoma
- Benign Keratosis-like lesions
- Dermatofibroma
- Melanocytic Nevi
- Melanoma
- Vascular lesions
- Architecture:
fai-cls-n-coco(Focoos AI) - Pretraining: COCO dataset
- Fine-tuning: Binary classification on HAM10000 (suspicious vs. non-suspicious)
- Input Size: 96x96
- Export Format: ONNX with quantization
- Deployment: Flashed via Zant
- Architecture: Vision Transformer Large (ViT-L/16)
- Parameters: ~305M
- Pretraining: ImageNet-21k
- Fine-tuning: HAM10000 with Focal Loss (to handle class imbalance)
- Framework: PyTorch
|
Edge Model (Nicla Vision) Binary Classification
|
Server Model (ViT-L/16) Multi-Class Classification
|
- Arduino Nicla Vision - Edge AI processing and image capture
- PC/Server - Flask web server + Vision Transformer inference
The Nicla Vision is integrated into a handheld, dermatoscope-like form factor for ease of use in clinical settings.
- Arduino IDE
- Focoos AI platform (model training & export)
- ONNX runtime
- Zant (firmware flashing)
- Python 3.8+
- Flask (web server)
- PyTorch
- torchvision
- scikit-learn
- NumPy, Pandas
- Assemble the MelaNoMore handheld device with Nicla Vision
- Ensure ToF sensor and camera are properly connected
- Verify LED indicators are functional
# Flash the ONNX model to Nicla Vision using Zant
zant flash --model model.onnx --device nicla-vision# Open sketch.ino in Arduino IDE and upload to Nicla Vision
arduino-cli compile --fqbn arduino:mbed_nicla:nicla_vision sketch.ino
arduino-cli upload -p /dev/ttyACM0 --fqbn arduino:mbed_nicla:nicla_vision sketch.ino# Navigate to server directory
cd server/
# Create venv
python -m venv venv
# Activate venv
source venv/bin/activate
# Install dependencies
pip install -r requirements.txt
# Start Flask server
python app.py --host 0.0.0.0 --port 8000Update WiFi credentials in sketch.ino:
const char* ssid = "YOUR_WIFI_SSID";
const char* password = "YOUR_WIFI_PASSWORD";
const char* serverURL = "http://YOUR_SERVER_IP:8000/ingest";-
Position Device
- Hold the MelaNoMore device over the patient's skin lesion
- Yellow LED indicates positioning mode
- ToF sensor ensures correct distance (3-5 cm)
-
Capture Image
- Once positioned correctly, Blue LED activates
- Image is automatically captured
-
On-Device Classification
- Nicla Vision runs binary classification
- Green LED = Non-suspicious (Done)
- Red LED = Suspicious (Image uploaded to server)
-
Server Analysis (for suspicious cases only)
- Image automatically transmitted via WiFi
- Vision Transformer performs detailed 7-class classification
- Results appear on web dashboard
-
Review Results
- Dermatologist reviews detailed classification on PC dashboard
- Prediction includes lesion type and confidence scores
- Clinical decision made based on AI recommendation + medical expertise
| LED Color | Meaning |
|---|---|
| 🟡 Yellow | Positioning - adjust distance |
| 🔵 Blue | Capturing image |
| 🟢 Green | Non-suspicious lesion detected |
| 🔴 Red | Suspicious lesion - uploading to server |
SkinCancerDetection_TinyHack/
├── README.md # Project overview and documentation
├── 1st_tier/
│ └── sketch.ino # Arduino Nicla Vision firmware (tier-1 classifier)
├── 2nd_tier/
│ ├── dataset.py # Dataset utilities for HAM10000
│ ├── evaluate.py # Evaluation pipeline and metrics
│ ├── metrics.py # Metric computation helpers
│ ├── model.py # Vision Transformer model definition
│ ├── train.py # Training entrypoint
│ ├── visualize.py # Visual analysis scripts
│ └── requirements.txt # Python dependencies for training
├── server/
│ ├── app.py # Flask server entrypoint
│ ├── classifier.py # Tier-2 inference orchestration
│ ├── requirements.txt # Runtime dependencies for the server
│ ├── static/
│ │ ├── dashboard.js # Dashboard interactivity
│ │ └── styles.css # Dashboard styling
│ ├── templates/
│ │ ├── base.html # Base HTML layout
│ │ └── dashboard.html # Dashboard view
│ ├── models/
│ │ ├── best_model.pth # Fine-tuned ViT weights
│ │ ├── inference_api.py # Model loading and inference helpers
│ │ └── INTEGRATION_GUIDE.md # Deployment/integration notes
│ └── incoming/ # Uploaded lesion images from the device
└── venv/ # (Optional) Local Python virtual environment
- Quantization: ONNX model optimized for embedded inference
- Resolution: 96x96 input size balances accuracy and speed
- Offline capability: On-device inference works without internet
- Focal Loss: Addresses class imbalance in HAM10000 dataset
- Vision Transformer: State-of-the-art architecture for medical imaging
- Scalability: Server can handle multiple concurrent requests
- ToF + Camera: Ensures consistent image quality
- WiFi + LED: Provides seamless user experience
- Mobile app integration for patient record management
- Database integration for longitudinal lesion tracking
- Multi-language support for global deployment
- Real-time analytics dashboard for dermatology clinics
- HIPAA-compliant encryption for patient data
- Dataset: HAM10000 - ISIC Archive
- Focoos AI: Model training and export platform
- Arduino Nicla Vision: Official Documentation
- Z-Ant: Beer model timing
- Vision Transformer: An Image is Worth 16x16 Words
The fine‑tuned Vision Transformer weights (best_model.pth, ~3.4 GB) are not included in this repository to keep the codebase small and fast to clone. When the file is absent the application automatically falls back to a mock classifier so you can still run the full ingestion + dashboard flow.
This project was developed as part of a 24-hour hackathon challenge.
Special thanks to the organizers of the hackathon and the creators of the Focoos, Zant, and Arduino Nicla Vision platforms for making edge AI accessible to developers.
MelaNoMore - Empowering dermatologists with AI-driven decision support 🩺
Built with ❤️ by Arslan, Vittorio, and Alessandro