PlantAIResearch

Evaluating the robustness of modern deep learning models to image quality degradation in plant disease diagnosis from leaf photographs.

This repository contains the full, reproducible experimental pipeline behind the paper "Evaluating the Robustness of Modern Deep Learning Models to Image Quality Degradation in Plant Disease Diagnosis from Leaf Photographs." It fine-tunes four modern architectures on the PlantVillage dataset, evaluates them under five types of synthetic image corruption at five severity levels each, and produces all tables, figures, and statistical tests reported in the paper — with no manually entered or assumed numbers.

Key finding

Clean-test accuracy does not predict robustness. All four models exceed 99% accuracy on clean PlantVillage images, but under synthetic corruption (Gaussian noise, blur, brightness reduction, JPEG compression, resolution loss) their behavior diverges sharply: ConvNeXt-Tiny and ViT-B/16 are significantly more robust than ResNet-50 and EfficientNet-B0 (Friedman test, χ² = 34.40, p < 10⁻⁶; Nemenyi post-hoc, α = 0.05). EfficientNet-B0 — despite having the second-highest clean accuracy of the four — is the least robust model overall and collapses to a single dominant prediction under severe Gaussian noise (6.1% accuracy, near the 2.6% random-guessing floor for 38 classes). Full results, figures, and statistical detail are in the paper (paper/article_draft_en.md), included in this repository together with its figures (paper/graphs/).

Repository structure

PlantAIResearch/
├── data/
│   ├── raw/            # PlantVillage in ImageFolder layout: raw/{train,val,test}/<class>/*.jpg
│   │                    # (populated by prepare_data.py — not committed to git, see .gitignore)
│   └── degraded/        # corrupted copies of the test set, generated by degrade.py
├── models/
│   └── checkpoints/     # trained model checkpoints (*.pt) + a config.json per model
├── results/
│   ├── metrics_<model>_<condition>.json   # full metrics + confusion matrix for every run
│   ├── summary.csv                        # summary table across all runs
│   └── mce_summary.csv                    # mean Corruption Error per model
├── graphs/               # accuracy-vs-severity plots (PNG), one per corruption type
├── paper/
│   ├── article_draft_en.md   # the paper itself, with tables and embedded figures
│   └── graphs/                # copies of the figures referenced by the paper
├── utils.py              # seeding, dataloaders, metrics, experiment config
├── prepare_data.py       # downloads PlantVillage and lays it out as ImageFolder
├── degrade.py             # generates the five types of corrupted test images
├── train.py                # fine-tunes each of the four architectures
├── evaluate.py               # evaluates a checkpoint on clean/corrupted data
├── analyze.py                 # statistical tests (Friedman/Nemenyi), mCE, plots
├── requirements.txt
├── LICENSE
└── CITATION.cff

What each script does

Script	Responsibility	Typical invocation
utils.py	Shared library code: seeding for reproducibility, ImageFolder-based DataLoader construction, metric computation (Accuracy/Precision/Recall/F1, confusion matrix), experiment config save/load. Not run directly.	—
prepare_data.py	Downloads the official PlantVillage split files and image archive from Hugging Face Hub (mohanty/PlantVillage) and materializes data/raw/{train,val,test}/<class>/*.jpg. The official leaf-grouped train/test split is preserved; val is carved out of train via a stratified split. Does not require a GPU.	python prepare_data.py
degrade.py	Applies five synthetic corruption types (Gaussian blur, Gaussian noise, brightness reduction, JPEG compression, downscale–upscale) at five severity levels each to a clean image folder (normally data/raw/test), writing corrupted copies to data/degraded/<corruption>/severity_<n>/<class>/.	python degrade.py --input data/raw/test --output data/degraded
train.py	Fine-tunes one architecture (resnet50, efficientnet_b0, convnext_tiny, or vit_b_16) on data/raw/train, validating on data/raw/val. Two-phase schedule: linear probing (frozen backbone) for the first --freeze_backbone_epochs epochs, then full fine-tuning at a reduced learning rate. Saves the best checkpoint (by validation macro-F1) to models/checkpoints/. Requires a GPU for practical runtimes.	python train.py --model resnet50
evaluate.py	Loads a trained checkpoint and evaluates it on a clean directory (--data_dir) and/or on every corrupted condition under data/degraded (--sweep_all), appending a row per condition to results/summary.csv and writing a full-detail JSON per run.	python evaluate.py --checkpoint models/checkpoints/resnet50_best.pt --data_dir data/raw/test --condition clean --sweep_all
analyze.py	Reads results/summary.csv, plots accuracy-vs-severity per corruption type into graphs/, computes mean Corruption Error (mCE, normalized against ResNet-50) into results/mce_summary.csv, and runs the Friedman test (plus Nemenyi post-hoc, if scikit-posthocs is installed) for statistical comparison of the four architectures.	python analyze.py

Requirements

• Python 3.10–3.12
• A CUDA-capable GPU is strongly recommended for train.py (fine-tuning four architectures on ~54k images on CPU is possible but takes far longer). prepare_data.py and degrade.py do not require a GPU.
• ~3 GB of free disk space (the PlantVillage archive is ~2.2 GB; corrupted copies and checkpoints add a few GB more).

All Python dependencies are listed in requirements.txt:

torch>=2.2
torchvision>=0.17
timm>=0.9
huggingface_hub>=0.23
opencv-python-headless>=4.9
numpy>=1.26
pandas>=2.2
matplotlib>=3.8
scikit-learn>=1.4
scipy>=1.12
scikit-posthocs>=0.9
Pillow>=10.2
tqdm>=4.66

timm is included for convenience/future extension but is not currently imported by any script — all four architectures are built directly from torchvision.models.

Installation

git clone https://github.com/gorasatryanGH/PlantAIResearch.git
cd PlantAIResearch
python -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate

Install PyTorch with CUDA support first, matching your GPU driver (see pytorch.org/get-started/locally for the exact command for your platform), for example:

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu124

Then install the remaining dependencies:

pip install -r requirements.txt

Verify that PyTorch sees your GPU:

python -c "import torch; print(torch.__version__, torch.cuda.is_available())"

Running the full pipeline

Run each command from the repository root, in order, waiting for each to finish before starting the next:

# 1. Download PlantVillage and lay it out as ImageFolder (no GPU needed, ~2.2 GB download)
python prepare_data.py

# 2. Fine-tune all four architectures (GPU strongly recommended)
python train.py --model resnet50
python train.py --model efficientnet_b0
python train.py --model convnext_tiny
python train.py --model vit_b_16

# 3. Generate corrupted versions of the test set
python degrade.py --input data/raw/test --output data/degraded \
    --corruptions gaussian_blur gaussian_noise brightness_down jpeg_compression downscale_upscale \
    --severities 1 2 3 4 5

# 4. Evaluate each model on the clean test set and on every corrupted condition
python evaluate.py --checkpoint models/checkpoints/resnet50_best.pt        --data_dir data/raw/test --condition clean --sweep_all
python evaluate.py --checkpoint models/checkpoints/efficientnet_b0_best.pt --data_dir data/raw/test --condition clean --sweep_all
python evaluate.py --checkpoint models/checkpoints/convnext_tiny_best.pt   --data_dir data/raw/test --condition clean --sweep_all
python evaluate.py --checkpoint models/checkpoints/vit_b_16_best.pt        --data_dir data/raw/test --condition clean --sweep_all

# 5. Statistical analysis and plots
python analyze.py

Outputs land in results/ (tables, per-run JSON metrics) and graphs/ (PNG plots) at the repository root.

If train.py raises a CUDA out-of-memory error, reduce the batch size, e.g. python train.py --model resnet50 --batch_size 16.

Methodological invariants

These are enforced by the code and are worth restating for anyone auditing or extending this repository:

1. The training set is never corrupted. degrade.py is only ever applied to evaluation data (typically data/raw/test). Training on corrupted images and then evaluating robustness against the same corruption types would conflate data augmentation with genuine out-of-the-box robustness.
2. The same test set is reused across severity levels. Only the corruption strength changes between severity 1 and 5 — the underlying set of images stays identical, so accuracy differences are attributable to corruption strength alone.
3. All stochastic steps use a fixed seed (42). Weight initialization order effects, the train/val split, and data ordering are reproducible run-to-run, modulo the well-known limits of exact CUDA-kernel-level determinism (see the docstring of utils.set_seed).
4. Class order is checked at evaluation time. evaluate.py compares ImageFolder.classes between the training-time class list (stored in the checkpoint) and the evaluation directory, and warns if they differ — a common, silent source of label-mismatch bugs in this kind of pipeline.
5. The official leaf-grouped PlantVillage split is preserved. prepare_data.py uses the dataset authors' splits/color_train.txt / splits/color_test.txt, in which photographs of the same physical leaf never appear in both partitions, avoiding a common source of inflated accuracy in prior work on this dataset.

Known limitations

• PlantVillage images are captured under controlled laboratory conditions (uniform background, standardized lighting). This repository measures sensitivity to technical image-quality degradation in isolation; it does not measure robustness to the full combination of factors present in field deployment (background clutter, viewpoint, multiple leaves per frame), which is a separate, already well-studied problem (see the PlantDoc dataset).
• The five corruption functions in degrade.py are synthetic approximations of real capture artifacts (implemented via OpenCV/PIL), not calibrated models of any specific smartphone sensor.
• Robustness is evaluated only for models fine-tuned on clean data; this is a deliberate choice (see utils.build_transforms docstring) to avoid conflating training-time data augmentation with architectural robustness.

Citation

If you use this code, please cite the accompanying paper (see CITATION.cff) and the original PlantVillage dataset papers:

• Hughes, D. P., & Salathé, M. (2015). An Open Access Repository of Images on Plant Health to Enable the Development of Mobile Disease Diagnostics. arXiv:1511.08060.
• Mohanty, S. P., Hughes, D. P., & Salathé, M. (2016). Using Deep Learning for Image-Based Plant Disease Detection. Frontiers in Plant Science, 7:1419.

License

Code in this repository is released under the MIT License (see LICENSE). The PlantVillage dataset itself is distributed by its original authors under its own terms — see the mohanty/PlantVillage dataset card on Hugging Face for details before redistributing any data.