English | 简体中文

Vision-based Real-time Structural Displacement Measurement Using Deep Learning

Official code release for:

Haoyou Zhang, Xiaowei Cheng, Yi Li, Hong Guan, "Vision-based Real-time Measurement of Structural Translational Motion Using Deep Learning and Object Tracking Methods," Applied Mathematical Modelling, 2026. doi.org/10.1016/j.apm.2026.117187

This project measures structural translational displacement in real time by detecting and tracking a chessboard target with an improved YOLOv5 detector and Deep SORT. It contributes:

• Black-White Attention (BWA) module — a CEB + CAB attention block tailored to chessboard-target detection, reaching 93.33% AP (vs. SE/ECA/CBAM/CA baselines).
• Weighted Non-Maximum Suppression (WNMS) — mitigates bounding-box jitter across frames.
• Deep SORT tracking — keeps the target identity through short-term occlusion.
• Sub-pixel-accurate validation on a dynamic shaking-table test (RMSE 2.05 mm) and a static cyclic test (RMSE 0.11 mm), plus out-of-plane (depth) displacement estimation, all at ~35 FPS.

Demo video

Real shaking-table test: the Y-BWA-W + Deep SORT framework tracking the chessboard target in real time, with the live displacement time history plotted alongside. (Full-resolution video)

Quick start

conda create -n yolo python=3.8
conda activate yolo
pip install -r requirements.txt

python predict.py        # run inference with the included trained weights

Trained weights (model_data/best_epoch_weights.pth, model_data/ckpt_target_epoch50.t7) are already included, so no training is required to try the model.

For a full description of the repository layout, environment setup, dataset details, and how to run/reproduce every script (including the paper's validation experiments), see USAGE.md.

Repository at a glance

Path	What it is
predict.py, main.py, yolo.py	Detection / tracking inference entry points
predict_video.py	Track one user-selected target through any recorded video (see USAGE.md § 5.8)
validation.py, depth_measurement.py	Scripts reproducing the paper's in-plane and out-of-plane validation experiments
nets/, utils/	Model architectures (BWA-YOLO, Deep SORT) and supporting utilities (WNMS, dataloaders, perspective correction)
model_data/	Trained weights, class/anchor files, Deep SORT config, and dataset annotation index (see USAGE.md § Dataset)
experimental_data/, logs/	Recorded test videos/sensor data and script outputs — not tracked in this repo (see Data availability below), kept locally / regenerated by running the scripts

Data availability

The raw experimental recordings and the full 2,132-image labelled dataset (too large for a normal Git push) are openly published on Zenodo under CC-BY 4.0:

Two archives are provided — VOCdevkit.zip (24.2 GB, the labelled train/validation image set) and experimental_data.zip (19.6 GB, raw camera recordings + sensor logs). See USAGE.md § Dataset for exactly what's in each archive and where to extract them.

Built on

This implementation extends bubbliiiing/yolov5-pytorch (YOLOv5 detector) and ZQPei/deep_sort_pytorch (Deep SORT tracker) with the proposed BWA attention module and WNMS. See USAGE.md § References for the full list of underlying methods and papers.

Citation

@article{zhang2026vision,
  title   = {Vision-based Real-time Measurement of Structural Translational Motion Using Deep Learning and Object Tracking Methods},
  author  = {Zhang, Haoyou and Cheng, Xiaowei and Li, Yi and Guan, Hong},
  journal = {Applied Mathematical Modelling},
  year    = {2026},
  doi     = {10.1016/j.apm.2026.117187}
}

If you use the dataset itself, please cite it separately — see USAGE.md § Citation for the dataset's Zenodo DOI and BibTeX entry.

License

MIT — see LICENSE.

Contact

Haoyou Zhang — haoyou.zhang@marquette.edu

Questions, issues, or ideas for extending this to other targets/applications are welcome — feel free to reach out, I enjoy discussing this work.