SORT Multi-Object Tracking

A Python implementation of the SORT (Simple Online and Realtime Tracking) approach combined with YOLOv8 object detection and OpenCV for multi-object tracking in video.

The tracker uses Kalman filtering for motion prediction and the Hungarian algorithm with Intersection over Union (IoU) for frame-to-frame data association.

Overview

The system processes each video frame through the following pipeline:

Video Frame
    ↓
YOLOv8 Object Detection
    ↓
Bounding-Box Detections
    ↓
IoU Calculation
    ↓
Hungarian Algorithm
    ↓
Kalman Filter Update
    ↓
Track Management
    ↓
Confirmed Tracks + IDs
    ↓
OpenCV Visualization

The SORT tracking logic is implemented in this repository rather than using an external SORT tracking package.

Architecture

flowchart TD
    A[Video Frame] --> B[YOLOv8 Detector]
    B --> C[Bounding-Box Detections]
    C --> D[IoU Calculation]
    D --> E[Hungarian Algorithm]
    E --> F[Matched / Unmatched Tracks]
    F --> G[Kalman Filter Update]
    F --> H[New Track Creation]
    G --> I[Track Management]
    H --> I
    I --> J[Confirmed Tracks + IDs]
    J --> K[OpenCV Visualization]

Loading

The system separates object detection from tracking. YOLOv8 provides frame-level detections, while the custom SORT implementation handles motion prediction, data association, and track lifecycle management.

How It Works

Each tracked object is represented using a 7-dimensional Kalman Filter state:

[u, v, s, r, u', v', s']

where:

• u, v — bounding-box center coordinates
• s — bounding-box area
• r — bounding-box aspect ratio
• u', v', s' — estimated velocities

The detector provides measurements of [u, v, s, r], while the Kalman Filter estimates the object's motion between observations.

1. Prediction

At the beginning of each frame, every existing track is advanced using the Kalman Filter prediction step.

This allows the tracker to estimate where an object should appear even when a detection is temporarily missing.

2. Data Association

Predicted bounding boxes are compared with the current detections using Intersection over Union (IoU).

The IoU matrix is converted into a cost matrix:

cost = 1 - IoU

The Hungarian algorithm is then used to find an optimal assignment between predicted tracks and detections.

Assignments below the configured IoU threshold are rejected.

3. Track Updates

Matched detections are passed to their corresponding tracks, where the Kalman Filter is updated with the new measurement.

Unmatched detections create new tracks.

Existing tracks are allowed to survive temporary missed detections. Tracks that remain unmatched for more than the configured maximum age are deleted.

4. Track Confirmation and Persistence

New tracks are created from unmatched detections but are not immediately included in the output.

A track is output once it has accumulated at least min_hits successful detections.

After confirmation, a track can remain alive during temporary missed detections. During these frames, its bounding box is provided by the Kalman Filter prediction. A track is deleted once time_since_update exceeds max_age.

Project Structure

object_tracker/
├── sort.py
├── tracker.py
├── test_sort.py
├── requirements.txt
├── .gitignore
└── README.md

sort.py

Contains the core tracking implementation:

• IoU calculation
• Kalman Filter based tracking
• Bounding-box state conversion
• Hungarian-algorithm data association
• Track creation
• Track updates
• Track deletion
• Track confirmation

tracker.py

Contains the complete detection and tracking pipeline:

• YOLOv8 model loading
• Video input using OpenCV
• Object detection
• SORT tracker integration
• Bounding-box and ID visualization
• Video and window cleanup

test_sort.py

Contains automated tests for the core tracking behavior, including:

• IoU calculations
• Single-object tracking
• Multi-object tracking
• Temporary missed detections
• Track deletion
• New track creation
• Low-IoU association rejection
• Empty detection handling

Detection Configuration

The YOLOv8 detector currently uses the following parameters:

Parameter	Value
Confidence threshold	0.37
NMS IoU threshold	0.60

These values are defined in tracker.py.

Tracking Configuration

The SORT tracker currently uses:

Parameter	Value
Maximum missed frames	15
Minimum hits for output	10
Tracker IoU threshold	0.30

These parameters can be changed in tracker.py:

MAX_AGE = 15
MIN_HITS = 10
TRACKER_IOU_THRESHOLD = 0.3

Requirements

The project requires:

• Python 3
• NumPy
• SciPy
• FilterPy
• OpenCV
• Ultralytics

Install the dependencies with:

pip install -r requirements.txt

Running the Tracker

The default configuration expects:

test_video.mp4
yolov8n.pt

in the project directory.

Run the tracker with:

python tracker.py

The program will:

1. Open the input video.
2. Run YOLOv8 object detection on each frame.
3. Pass the detections to the SORT tracker.
4. Draw tracked bounding boxes and IDs.
5. Display the result in an OpenCV window.

Press q to stop the program.

The tracking function can also be called directly:

from tracker import run_tracking

run_tracking(
    video_path="test_video.mp4",
    model_path="yolov8n.pt",
)

Testing

The core tracker is covered by an automated test suite using pytest.

Run the tests with:

python -m pytest -q

The current test suite contains 11 tests covering the main tracking behaviors.

Key Technologies

• Python
• NumPy
• SciPy
• FilterPy
• OpenCV
• Ultralytics YOLOv8
• Kalman Filtering
• Hungarian Algorithm
• Intersection over Union (IoU)

Reference

This project is based on the tracking approach described in:

Bewley et al., "Simple Online and Realtime Tracking" (SORT)

SORT combines Kalman filtering and frame-by-frame data association to maintain object identities across video frames.

Implementation Focus

The project focuses on implementing and integrating the main components of the tracking pipeline:

• Bounding-box representation
• IoU calculation
• Kalman Filter prediction and update
• IoU-based cost matrix construction
• Hungarian-algorithm data association
• Track creation and deletion
• Track confirmation
• YOLOv8 + SORT integration
• Automated behavioral testing