Pentavision - AI Stock Analysis System

An AI-powered retail shelf stock level analysis system that uses computer vision and natural language processing to detect and monitor inventory levels in real-time.

What is Pentavision?

Pentavision analyzes shelf images to automatically detect stock levels for produce items (apples, bananas, oranges, broccoli, carrots) using a hybrid AI approach combining:

• Qwen2.5-VL-7B-Instruct: Vision-language model for stock level estimation
• YOLOv11x: Object detection model for product identification
• FastAPI Backend: REST API for image processing
• React Frontend: Web dashboard for visualization and management

How It's Made

Architecture

Frontend (React) → Backend (FastAPI) → AI Models (Qwen + YOLO)

Technology Stack

Backend:

• Python 3.9+, FastAPI, Uvicorn
• PyTorch, Transformers (Qwen), Ultralytics (YOLO)
• OpenCV, Pillow for image processing

Frontend:

• React 18, TypeScript, Vite
• Tailwind CSS, Recharts
• React Router, Firebase

Infrastructure:

• Node.js (camera server)
• Nginx (reverse proxy)
• Systemd (service management)

Project Structure

pentavision/
├── backend-new/          # FastAPI backend
│   ├── api/             # API modules (api.py, hybrid.py, qwen_stock.py, yolo_visualize.py)
│   ├── camera-server/   # RTSP camera streaming server
│   ├── models/          # AI models (qwen_vl/, yolov11x.pt)
│   ├── media/           # Uploaded and processed images
│   ├── requirements.txt
│   ├── run.sh           # Development startup
│   └── start_production.sh  # Production startup
│
├── frontend/             # React frontend
│   ├── src/
│   │   ├── components/  # React components
│   │   ├── pages/       # Page components
│   │   └── config/      # Configuration
│   └── package.json
│
├── start_full_system.sh  # Start everything
└── stop_all.sh          # Stop everything

Prerequisites

• Python 3.9+
• Node.js 18+
• NVIDIA GPU with CUDA (recommended)
• 16GB+ RAM (32GB recommended)
• Models:
  • Qwen2.5-VL-7B-Instruct at /data/shared/nobackup/qwen_vl (or symlink)
  • YOLOv11x weights at backend-new/models/yolov11x.pt

Installation

Backend Setup

cd pentavision/backend-new

# Create virtual environment
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Set up model symlink (if needed)
ln -s /data/shared/nobackup/qwen_vl models/qwen_vl

Frontend Setup

cd pentavision/frontend
npm install

Camera Server Setup (Optional)

cd pentavision/backend-new/camera-server
npm install

How to Start

Start Full System (Backend + Frontend)

cd pentavision
./start_full_system.sh

This starts:

• Backend API at http://localhost:8080
• Frontend at http://localhost:3000

Start Backend Only

cd pentavision/backend-new
./run.sh

Backend will be available at:

• API: http://localhost:8080
• Docs: http://localhost:8080/docs
• Health: http://localhost:8080/health

Start Frontend Only

cd pentavision/frontend
npm run dev

Frontend will be available at http://localhost:3000

Start Production Backend

cd pentavision/backend-new
./start_production.sh

Start with Systemd Service

# Copy and configure service file
sudo cp pentavision/backend-new/pentavision-backend.service /etc/systemd/system/
sudo nano /etc/systemd/system/pentavision-backend.service  # Update paths

# Enable and start
sudo systemctl daemon-reload
sudo systemctl enable pentavision-backend
sudo systemctl start pentavision-backend

# Check status
sudo systemctl status pentavision-backend

How to Stop

Stop All Services

cd pentavision
./stop_all.sh

Stop Backend Only

Press Ctrl+C if running in foreground, or:

# If using systemd
sudo systemctl stop pentavision-backend

# If running in background, find and kill process
pkill -f "uvicorn api.api:app"

Stop Frontend Only

Press Ctrl+C in the terminal running the frontend.

API Endpoints

Main Endpoints

• POST /analyze - Upload image for stock analysis
• GET /health - System health check
• GET /status - Detailed system status
• GET /api/latest-annotated-image - Get latest analyzed image
• GET /api/historical-stock-data?days=7 - Get historical data
• POST /cleanup - Clean up temporary files

API Documentation

Once backend is running:

• Swagger UI: http://localhost:8080/docs
• ReDoc: http://localhost:8080/redoc

Configuration

Backend Environment Variables

Set in start_production.sh or systemd service:

CUDA_VISIBLE_DEVICES=0
TOKENIZERS_PARALLELISM=false
QWEN_MODEL_PATH=/data/shared/nobackup/qwen_vl
YOLO_MODEL_PATH=/path/to/yolov11x.pt
MEDIA_DIR=/path/to/media
LOG_DIR=/var/log/pentavision

Frontend API Configuration

Update frontend/src/config/api.ts:

export const API_BASE_URL = 'http://localhost:8080';

Troubleshooting

Models Not Loading

• Verify model paths are correct
• Check GPU memory: nvidia-smi
• Ensure CUDA is installed
• Check file permissions

Out of Memory

• Use single GPU: CUDA_VISIBLE_DEVICES=0
• Clear GPU cache in code
• Close other GPU applications

Connection Issues

• Verify backend is running on port 8080
• Check CORS configuration
• Verify API_BASE_URL in frontend config
• Check firewall rules

Check Logs

# Systemd logs
sudo journalctl -u pentavision-backend -f

# Application logs
tail -f /var/log/pentavision/backend.log

Monitoring

System Monitoring

cd pentavision/backend-new

# Interactive dashboard
python monitor_dashboard.py --continuous

# Command-line monitoring
python monitor_system.py --duration 10

Monitors: GPU usage, CPU, memory, disk, and process information.

License

[Specify license]

---

Version: 1.0.0