COMPUTER VISION · STRUCTURAL IMAGERY · PIXEL SEGMENTATION
A multidataset deep-learning study for detecting and segmenting wall cracks at pixel level — with controlled model comparison, diagnostic evaluation, and threshold-aware operating-point selection.
Cracks in concrete and wall surfaces are often thin, irregular, low-area structures surrounded by large amounts of background.
That makes the problem more difficult than simply deciding whether an image contains a crack.
The goal of this project is to study pixel-level crack segmentation and answer several practical questions:
- Which segmentation architecture performs best on the same validation data?
- How well does each model localize thin crack structures?
- How should no-crack images be handled during evaluation?
- How much does the prediction threshold affect segmentation quality?
- Can false positives be reduced without sacrificing too much crack IoU?
- How do model failures differ across difficult surface textures and crack geometries?
Rather than relying on a single accuracy value, the project evaluates the models through IoU, Dice, precision-recall behavior, no-crack false positives, qualitative overlays, error galleries, and threshold sweeps.
flowchart LR
A["Multidataset Images"] --> B["EDA & Data Preparation"]
B --> C["Frozen Train / Validation Splits"]
C --> D1["U-Net + EfficientNet-B3"]
C --> D2["DeepLabV3+ + EfficientNet-B3"]
C --> D3["U-Net++ + EfficientNet-B3"]
D1 --> E["Standardized Evaluation"]
D2 --> E
D3 --> E
E --> F["IoU / Dice"]
E --> G["Pixel PR Analysis"]
E --> H["No-Crack FP Analysis"]
E --> I["Diagnostic Galleries"]
F --> J["Threshold Sweep"]
G --> J
H --> J
J --> K["Risk-Controlled Operating Point"]
K --> L["CrackSAFE-MRC Demo"]
U-Net provides the first full segmentation baseline in the research pipeline.
The model uses an EfficientNet-B3 encoder and is evaluated through:
- Training and validation curves
- Pixel-level segmentation metrics
- Crack-only IoU
- Crack-only Dice
- No-crack false-positive behavior
- IoU distributions
- Precision-recall analysis
- Bounding-box / oriented-box diagnostics
- Worst-performing sample galleries
- Resolution-parity evaluation
- Threshold tuning
DeepLabV3+ introduces a different segmentation strategy focused on stronger multi-scale contextual representation.
The experiment records:
- Training history
- Validation metrics
- Crack-area distributions
- Prediction-confidence distributions
- Pixel precision-recall behavior
- Bounding / oriented bounding-box overlays
- Crack-only IoU distributions
- Worst-case segmentation samples
- Model configuration
- Threshold analysis
U-Net++ extends the encoder-decoder concept through nested skip pathways and becomes the strongest model in the final threshold sweep for crack-only overlap.
The experiment includes:
- Best and last checkpoints
- Training history
- Validation metrics
- Pixel PR analysis
- Crack-area diagnostics
- Confidence distributions
- IoU distributions
- Failure galleries
- Bounding-box diagnostics
- Threshold tuning
The standardized evaluation protocol freezes the training and validation splits so every model is evaluated against the same samples.
| Split | Total | Crack Present | No Crack |
|---|---|---|---|
| Training | 9,603 | 8,367 | 1,236 |
| Validation | 1,695 | 1,477 | 218 |
Split hashes are recorded in the Phase-6 protocol to make accidental split changes detectable.
A crack can occupy only a very small fraction of an image.
For that reason, pixel accuracy alone can be misleading.
A model could correctly classify most background pixels while still failing to capture the crack itself.
This project therefore uses several complementary measurements.
For ground-truth crack pixels (A) and predicted crack pixels (B):
IoU = |A ∩ B| / |A ∪ B|
Higher IoU means stronger overlap between the predicted crack mask and the true crack region.
Dice = 2|A ∩ B| / (|A| + |B|)
Dice provides another measure of segmentation overlap and is especially useful when foreground pixels are sparse.
Metrics are also reported specifically on validation images that actually contain cracks.
This prevents the large number of easy background pixels from hiding segmentation weaknesses.
No-crack samples are handled explicitly:
Ground truth empty + prediction empty → score = 1
Ground truth empty + predicted crack → score = 0
Ground truth contains crack → standard overlap metric
No-crack validation images provide a direct test of unwanted detections.
FP Rate = no-crack images with predicted crack
------------------------------------
total no-crack images
This matters because a practical crack-detection system should not mark clean surfaces as damaged.
Pixel-level precision-recall behavior is evaluated to study the foreground/background tradeoff under strong class imbalance.
The final operating-point study evaluates thresholds from the segmentation probability maps rather than assuming that 0.50 is automatically optimal.
| Model | Threshold | Crack-Only IoU | Crack-Only Dice |
|---|---|---|---|
| U-Net + EfficientNet-B3 | 0.90 | 0.5686 | 0.7105 |
| DeepLabV3+ + EfficientNet-B3 | 0.40 | 0.5730 | 0.7143 |
| U-Net++ + EfficientNet-B3 | 0.40 | 0.5807 | 0.7216 |
Maximizing crack IoU is not the only objective.
The Phase-8 policy also searches for an operating threshold that minimizes false positives on clean images while keeping crack-only IoU within 98% of the model's best crack-only IoU.
| Model | Selected Threshold | Crack IoU | No-Crack FP Rate |
|---|---|---|---|
| U-Net | 0.90 | 0.5686 | 1.83% |
| DeepLabV3+ | 0.50 | 0.5728 | 0.46% |
| U-Net++ | 0.50 | 0.5807 | 2.75% |
U-Net++ preserves the strongest crack segmentation overlap.
DeepLabV3+ provides the lowest false-positive rate on no-crack images at the selected risk-controlled operating point.
This illustrates an important deployment lesson:
The model with the highest segmentation score is not automatically the best model for every operating requirement.
A major goal of the later project phases is to make model comparison fairer and easier to audit.
The standardized protocol requires models to use:
Same frozen train split
Same frozen validation split
Recorded split hashes
Comparable evaluation outputs
Crack-only reporting
Overall reporting
No-crack false-positive reporting
Pixel precision–recall analysis
Consistent artifact naming
Explicit threshold policy
This reduces the risk of declaring one model better simply because it was evaluated under a different data split or metric definition.
The repository does not stop at headline scores.
It stores visual and statistical artifacts that help answer why a model succeeds or fails.
- Loss curves
- IoU curves
- Pixel-accuracy curves
- Precision-recall curves
- Crack-only IoU histograms
- Overall IoU histograms
- Ground-truth crack-area distributions
- Predicted crack-area distributions
- Prediction-confidence distributions
- Segmentation overlays
- Bounding-box overlays
- Oriented bounding-box overlays
- Worst-performing sample galleries
- Crack-only failure galleries
The repository preserves the development process rather than only keeping a final metric.
Exploratory analysis
Data splitting
U-Net + EfficientNet-B3
Training history
Validation evaluation
Failure analysis
DeepLabV3+ experiment
EfficientNet-B3 encoder
Validation metrics
PR analysis
Confidence analysis
Detection overlays
U-Net++ experiment
EfficientNet-B3 encoder
Validation artifacts
Checkpoint comparison
Diagnostic outputs
Cross-model comparison
Comparison tables
Aggregated metrics
Model ranking artifacts
Frozen splits
Split hashes
Metric definitions
Reproducibility rules
Output naming convention
Threshold policy preparation
U-Net evaluation parity
256 × 256 evaluation
448 × 448 evaluation
Comparable diagnostic outputs
Threshold sweep
IoU / FP tradeoff
Best-IoU threshold
Best-Dice threshold
Risk-controlled operating point
wallCrackWith2datasets/
│
├── Phase-1,2,3(U-Net + EfficientNet-B3 encoder)/
│ ├── EDA/
│ ├── training history
│ ├── model checkpoints
│ ├── validation metrics
│ ├── IoU / loss / PR plots
│ └── prediction diagnostics
│
├── wallcrack-phase-4-deeplabv3/
│ ├── DeepLabV3+ artifacts
│ ├── validation metrics
│ ├── confidence diagnostics
│ ├── PR analysis
│ └── prediction galleries
│
├── wallcrack-phase-4B-unetpp-efficientnetb3-proto1-44/
│ ├── U-Net++ checkpoints
│ ├── training history
│ ├── validation metrics
│ ├── PR analysis
│ └── diagnostic figures
│
├── Phase-5 comapre/
│ └── model comparison artifacts
│
├── Phase-6/
│ ├── evaluation protocol
│ ├── reproducibility configuration
│ ├── run manifest
│ └── split summary
│
├── Phase7_Model1_Phase3R_Diagnostics_Parity/
│ ├── unet_efficientnetb3_eval256/
│ └── unet_efficientnetb3_eval448/
│
├── Phase8_Threshold_Tuning_Operating_Point/
│ ├── threshold sweep
│ ├── threshold curves
│ ├── tradeoff plot
│ └── threshold policy
│
└── README.md
The research is also represented through a Hugging Face Space.
This repository is primarily a research artifact archive.
It currently emphasizes:
- Trained model checkpoints
- Evaluation reports
- Frozen split information
- Diagnostic figures
- Model-comparison outputs
- Threshold-tuning artifacts
- Reproducibility metadata
It is not currently organized as a single installable Python package or one-command training application.
That distinction is intentional in this README so that the repository is represented accurately.
Several challenges remain important when interpreting the results.
Cracks may occupy only a very small percentage of the image, creating severe foreground/background imbalance.
Textures, joints, stains, shadows, edges, and surface damage can visually resemble cracks.
Performance on the evaluation datasets does not guarantee the same performance on unseen buildings, materials, cameras, lighting conditions, or geographic environments.
Binary segmentation quality changes with the probability threshold, which is why threshold selection is explicitly studied rather than hidden.
A model that detects cracks well but repeatedly marks clean walls as damaged may not be appropriate for real inspection workflows.
Possible next steps include:
- Add more construction materials
- Increase lighting diversity
- Add more no-crack surfaces
- Test additional external datasets
- Transformer-based segmentation
- SegFormer
- DeepLab variants
- Attention-enhanced U-Net architectures
- Ensemble experiments
- Cross-dataset generalization
- Confidence calibration
- Uncertainty estimation
- Precision/recall operating points
- Boundary-sensitive crack metrics
- Upload-and-segment interface
- Crack measurement
- Severity estimation
- Crack-length estimation
- Crack-width analysis
- Inspection-history tracking
- Consolidated inference pipeline
- Model export
- Optimized inference
- API serving
- Mobile-friendly inspection interface
The experiments show why a crack-segmentation system should not be judged by a single number.
A strong evaluation also needs to ask:
How accurately is the crack localized?
How does the model behave on clean surfaces?
Which failures are being hidden by aggregate metrics?
How sensitive is the model to its probability threshold?
What operating point makes sense for the intended application?
That is why this repository includes not only model checkpoints, but also diagnostic galleries, crack-only metrics, no-crack analysis, standardized evaluation, and threshold policies.
