Cross-Sample Prototype Matching and Multiscale Spatial Correlation Consistency for Defect Segmentation Under Limited Annotations
Defect segmentation is critical a task in industrial quality control and manufacturing systems, yet acquiring pixel-level annotations for training deep learning models is costly and time-consuming. Existing semi-supervised methods often struggle with challenges in industrial images, including low contrast, pixel-level class ambiguity, and extreme defect variability, which lead to prediction noise and biases when labeled samples are limited. To address these issues, we propose ProMSC🔍 (Cross-Sample Prototype Matching with Multiscale Spatial Correlation Consistency), a novel semi-supervised framework that leverages a small set of annotated samples to guide learning from unlabeled data.
ProMSC integrates cross-sample prototype matching between labeled and unlabeled samples at the feature map level to improve the representation of unlabeled samples, multiscale spatial correlation and positional alignment to enforce structural consistency between weakly and strongly augmented views, and pseudo-label refinement via correlation matrix to enhance prediction confidence. This combination effectively mitigates the challenges of low contrast, class ambiguity, and extreme defect variability, enabling robust learning from limited labeled data along with large pool of unlabeled data. Extensive experiments on five benchmark defect segmentation datasets demonstrate that ProMSC outperforms several state-of-the-art semi-supervised segmentation approaches.
⭐ Details and specific analysis is found at : (https://www.sciencedirect.com/science/article/pii/S095741742602213X)
⭐ If you find this work useful, please star this repository!⭐
🔥 Please cite ourwork as follows:
@article{SIME2026133304,
title = {Cross-sample prototype matching and multiscale spatial correlation consistency for defect segmentation under limited annotations},
journal = {Expert Systems with Applications},
volume = {331},
pages = {133304},
year = {2026},
issn = {0957-4174},
doi = {https://doi.org/10.1016/j.eswa.2026.133304},
url = {https://www.sciencedirect.com/science/article/pii/S095741742602213X},
author = {Dejene M. Sime and Nan Ouyang and Kai Sheng and Adnan A. Qaseem and Yiting Liu and Xiaojiang Ren},
keywords = {Defect segmentation, Semi-supervised learning, Multi-scale spatial correlation, Cross-sample prototypes, Pseudo-labeling, Consistency regularization}}
This repository provides scripts for training, evaluation, and segmentation using the ProMSC framework.
Follow the steps below to reproduce the results.
1️⃣ Train ProMSC: run the 'train_promsc.sh' in the training_scripts directory
2️⃣ Evaluate ProMSC: run the 'testing.py' files in the testing_scripts directory
3️⃣ Dataloader and testing might be slightly different for different datasets, check accordingly
4️⃣ Scripts for TSNE plotting and complexity analysis are also provided
Defect segmentation is essential for industrial quality control, yet it faces major challenges due to the high cost and time required for pixel-level annotations. Industrial images often suffer from low contrast, pixel-level class ambiguity, extreme defect variability (in size, shape, and appearance), class imbalance, and subtle textures. These issues cause noisy pseudo-labels, confirmation bias, and error propagation in traditional semi-supervised learning (SSL) methods, which struggle with limited labeled data while relying on abundant unlabeled samples.
💡This paper introduces ProMSC, a novel semi-supervised framework designed to address these limitations by tightly coupling pseudo-labeling with enhanced consistency regularization:
- ⚡Cross-Sample Prototype Matching🧬: Builds class-wise prototypes from labeled samples (using ground-truth masks) and unlabeled samples (using pseudo-labels) at the feature-map level. This anchors unlabeled representations to reliable semantic centers from labeled data, promotes compact intra-class clustering, and pushes decision boundaries toward low-density regions to improve pseudo-label quality and reduce confirmation bias.
- ⚡Multiscale Spatial Correlation Consistency🌊: Computes spatial correlation matrices at multiple encoder scales to capture higher-order structural relationships (beyond local pixels). These are enforced to remain consistent between weakly and strongly augmented views, preserving geometric and structural properties of defects across scales.
- ⚡Positional Alignment & Pseudo-Label Refinement🔄: Introduces positional feature consistency for local stability and uses the learned correlation matrix to reweight decoder logits, enhancing the confidence and spatial coherence of pseudo-labels.
Figure: ProMSC overall architecture
Figure: ProMSC details
- 🏆ProMSC achieves state-of-the-art performance on five diverse benchmark datasets (NEU-Seg, DAGM, MTD, Crack500, TUT), demonstrating strong robustness even with very limited annotations. The approach effectively mitigates low-contrast and variability issues common in real-world industrial settings.
Figure: Visualization of Feature Maps
Figure: Results tables
Figure: Visualization of segmentation results
For questions, collaborations, or further discussion regarding this work, please feel free to reach out. ❤️💬🐛We welcome feedback, suggestions, and collaboration opportunities⭐⭐⭐.
- 📧 Email: djene.mengistu@gmail.com
- 🌐 GitHub: https://github.com/djene-mengistu