MATE-ROV-2026-AI

Overview

This repository contains the computer vision pipeline used for crab detection and counting in the MATE ROV context. The runtime path in main.py uses a YOLO-based detector, receives camera frames from a ROS web video stream, and publishes both crab count and annotated detection frames through ROS Bridge.

Detection Approaches Implemented

1) YOLO-Based Detection (Primary Runtime Path)

Files:

• main.py
• YOLO_Detection.py
• video_stream_web.py
• ros_publisher.py

Pipeline summary:

1. VideoStream opens the HTTP stream from a ROS web endpoint.
2. CrabDetector loads the trained YOLO model from best.pt.
3. Inference runs per frame with confidence threshold 0.6.
4. Bounding boxes are drawn using Ultralytics plotting utilities.
5. Only detections with class label European Green Crab are counted.
6. Annotated frame and crab count are published to ROS Bridge.

Design intent:

• Use a trained detector to improve robustness over illumination and background variation.
• Keep the count logic class-specific to avoid counting non-target detections.

2) Color-Based Detection (Experimental / Alternative)

Files:

• color_based_detection/crab_detection.py
• color_based_detection/crab_detection_with_video.py

Pipeline summary:

1. Convert frame from BGR to HSV.
2. Segment crab-like pixels using fixed HSV thresholds.
3. Clean masks with morphological opening and closing.
4. Extract contours and filter small regions by area.
5. Track objects across frames with centroid-distance tracking.
6. Estimate crab appearance type (Light/Dark) from region intensity.

Design intent:

• Provide a lightweight fallback or baseline without model inference.
• Demonstrate classical vision + tracking methods for controlled scenes.

ROS Communication Design

Topics used by runtime path

Published:

• /crab_count as std_msgs/Int32
• /AI_Detection as sensor_msgs/Image

Consumed:

• HTTP stream endpoint on port 8080 using topic query /cam_front/image_raw

Publishing strategy currently implemented

The image frame is published as standard sensor_msgs/Image payload where data contains a raw uint8[] array.

This aligns with standard ROS image semantics and improves interoperability with typical ROS consumers.

Requirements

Install dependencies in your active Python environment:

pip install ultralytics opencv-python roslibpy numpy matplotlib

Notes:

• matplotlib is used by experimental scripts in color_based_detection.
• Ensure ROS Bridge is reachable from the machine running this code.

Exact Command to Run the Main File

Run from the project root:

python main.py

Runtime Behavior

• The script connects to:
  • Video stream: http://192.168.1.100:8080/stream?topic=/cam_front/image_raw
  • ROS Bridge: 192.168.1.100:9090
• A live window named AI Detection is shown.
• Press q to stop.

Model File

best.pt must exist in the project root directory for the current runtime configuration.