Deep Learning for Soccer Pass Receiver Prediction in Broadcast Images

We propose a multi-stage system for Football/Soccer Pass Receiver Prediction. The system consists of 4 major stages:

1. Player and Ball Detection using YOLO
2. Team Identification using Clustering on Images
3. Perspective Transformation to aerial view
4. Construction of Player Graph and Pass Receiver Prediction using a Graph Attention Network (GATv2)

Dataset:

We develop a new dataset nicknamed 'SoccerPass' to train/evaluate our system. The dataset is loosely based on data collected from the SoccerNetv2 database. The SoccerPass dataset is constructed by hand-picking passing frames from over 30 top European broadcast matches. The matches cover a wide range of teams and competitions such as English Premier League, Bundesliga, French Ligue 1 and UEFA Champions League. We selected $\sim1.2$k frames where a pass was about to be performed, and annotate each image with the desired attributes.

Dataset Link: Google Drive.

Organization:

Detection.ipynb - Contains results of object detection models for player/ball localization.
Clustering.ipynb - Contains results of clustering for Team Identification.
PerspectiveTransform.ipynb - Contains results of Perspective Transformation to obtain ground coordinates.
Receiver Prediction.ipynb - Contains results of Pass Receiver Prediction using GNNs.

E2E.ipynb - Contains results of end-to-end system evaluated on our dataset.
E2E_Video.ipynb - Contains results of end-to-end system evaluate on a video clip. Note that we only use short clips as this is not a real-time solution.

/configs - Contains config files for models
/gnn - Contains build + training code for GNN models
/PerspectiveTransform - Contains code for perspective transform based on this original implementation
/yolomodels - Contains build + training code for YOLOv7,v8 models based on original implementations

How to Run

To run the main script, execute:

python main.py

How to Test

To run the tests, execute:

pytest test_main.py

Ownership:

Sanjeev Narasimhan | sn3007@columbia.edu

Pranav Deevi | pid2104@columbia.edu

Vishal Bhardwaj | vb2573@columbia.edu