Skip to content

Add results.box.image_metrics and errors/ + correct/ visualize folders - #902

Merged
EHxuban11 merged 2 commits into
devfrom
887-image-metrics
Sep 25, 2026
Merged

EHxuban11 merged 2 commits into
devfrom
887-image-metrics

Conversation

@EHxuban11

@EHxuban11 EHxuban11 commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

What: val() results carry box.image_metrics (per-image precision, recall, F1, TP, FP, FN), and visualize=True sorts its images into visualize/errors/ and visualize/correct/.
Why: #887 asks to see only the samples the model got wrong. #890 added the ecosystem's visualize but it draws every image; the ecosystem's way to find the wrong ones is results.box.image_metrics (docs.ultralytics.com/modes/val, docs only).

  • image_metrics: dict, image filename -> {"precision", "recall", "f1", "tp", "fp", "fn"}, same keys as the ecosystem. Always computed for detect and segment (the ecosystem computes it on every run too), with the visualize matching: confidence 0.25 or the run's conf if higher, IoU 0.5, class-aware. Precision/recall are 0.0 when their denominator is 0. An image whose filename was already seen (same name in another folder) is keyed by its full path, so no entry is overwritten. Ground truth outside classes= is dropped, matching the prediction filter.
  • val() still returns the metrics dict: ValidationMetrics is a dict subclass that adds .box. metrics["..."], == and json.dumps are unchanged. Wrapped in model.val() and exported-backend val() only; the trainer keeps the plain dict. Other tasks return a plain dict as before.
  • visualize images go to visualize/errors/ (any FP or FN; wrong top-1 for classification) or visualize/correct/, so the mistakes are one folder. Unreleased layout from Add val(visualize=True) error-analysis images #890, so nothing shipped moves. A rerun clears old images from both subfolders.
  • Scoring never breaks the mAP update: skipped when there are no targets, failures logged once per run.

Deviations from the ecosystem, on purpose:

  • Segmentation image_metrics count boxes and live on .box; there is no mask-based .seg.image_metrics. Pose and OBB have none yet.
  • The ecosystem does not document the confidence threshold it counts at; this uses the visualize one so the numbers match the drawn images.
  • No image-count cap (Feature Request: error analysis visualization #887 asked for one; the ecosystem has none). errors/ and filtering image_metrics cover it.

Check: _score_images in validation/detection_validator.py; with_image_metrics in validation/base.py.
Verified (macOS, CPU): unit suite 7692 passed, 3 failed (the test_detr_cpu_export_matrix ONNX parity cases that fail on clean dev on this machine and pass in CI). LibreYOLO9t on coco8: mAP identical with and without visualize, 1 image in correct/ and 3 in errors/, matching image_metrics, and each image's TP/FP/FN header matches its entry.
Not verified: CUDA, DDP (each rank keeps its own images' entries; not gathered), exported-backend val() live.
Opened by an agent.

Closes #887

Code provenance

Original code written for this PR against LibreYOLO's own first-party code; no third-party code ported, adapted, or introduced; no GPL/AGPL/LGPL/non-commercial/unknown-license material involved. Names follow the ecosystem's public documentation only.

RetriggerConfidence Score: 5/5

The PR appears safe to merge based on the changes since the previous review.

Summary

The PR adds per-image box metrics to detection and segmentation validation results and sorts validation visualizations into errors and correct folders. Since the previous review, it changes duplicate-filename handling so a later image uses its full path as a metrics key.

Reviews (2) · Last reviewed commit: "Key a repeated image filename by its ful..."

Comment thread libreyolo/validation/detection_validator.py Outdated
@EHxuban11
EHxuban11 merged commit cf57f10 into dev Sep 25, 2026
14 of 15 checks passed
@EHxuban11
EHxuban11 deleted the 887-image-metrics branch September 26, 2026 15:35
EHxuban11 added a commit that referenced this pull request Sep 26, 2026
Add results.box.image_metrics and errors/ + correct/ visualize folders
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant