VIRTUOUS_CYCLE

Active Learning under Expert-Budget Constraints: A Human-in-the-Loop Pipeline for Diabetic Retinopathy Lesion Detection

A pool-based active learning framework for fine-tuning a YOLO detector with as few labeled images as possible. It implements and compares four query strategies for picking which unlabeled images to send to an oracle (human annotator) each round.

Strategies

	Strategy	Score
A	Random	uniform random sampling
B	Least Confidence	1 - mean(detection confidence) over an image
C	Hybrid Diversity	bottom-ρ·k by mean confidence (most uncertain) ∪ random (1-ρ)·k from the rest
D	MC Dropout	λ·(1 - confidence) + (1-λ)·u, where u is the per-class variance of confidence over T stochastic forward passes (dropout injected into the backbone's C2f layers)

See active_learning/strategies.py for the exact implementation of each strategy.

Installation

pip install -r requirements.txt

Requires a YOLO checkpoint compatible with Ultralytics (e.g. yolov8x.pt, auto-downloaded by name on first use) and a CUDA GPU for reasonable training time.

Data format

Point DATA_ROOT in your config at a YOLO-format detection dataset laid out as:

<DATA_ROOT>/all/images/*.jpg
<DATA_ROOT>/all/labels/*.txt   # one row per object: "class x_center y_center w h" (normalized)

An image with no objects should have either no label file or an empty one. Class names/count are set via CLASS_NAMES in the config — the dataset itself is not bundled with this repo and must be supplied separately.

Usage

1. Edit configs/experiment.yaml: set DATA_ROOT, CLASS_NAMES, BASE_MODEL, and the AL protocol / training hyperparameters.
2. Run from the repo root:

python -m active_learning.experiment --cfg configs/experiment.yaml

Each round trains YOLO on the currently labeled set, evaluates on a fixed held-out test split, then queries the next batch of images from the unlabeled pool using the configured strategy.

Output

Written under OUTPUT_DIR:

• results_seed{seed}_{strategy}.csv — per-round metrics (mAP@50, mAP@50-95, precision, recall)
• results_all.json — combined raw results across all strategies/seeds
• learning_curves.png — seed-averaged learning curves (one panel per metric)

To regenerate the plot from a saved results_all.json without rerunning the experiment:

python -m active_learning.plotting --results results/results_all.json --output results

Repository structure

active_learning/
├── strategies.py   # query strategies A–D
├── experiment.py   # AL loop: train → evaluate → query, per round/seed
└── plotting.py     # seed-averaged learning-curve figure
configs/
└── experiment.yaml # data paths, AL protocol, training hyperparameters, strategy params