Reproducing "Understanding Deep Learning Requires Rethinking Generalization"

A minimal reproduction of the core experiments from Zhang et al. (2017) for ECE 57000 — AI Course Project, Track 1: TinyReproductions.

Paper: C. Zhang, S. Bengio, M. Hardt, B. Recht, O. Vinyals. "Understanding Deep Learning Requires Rethinking Generalization." ICLR 2017.

---

Project Structure

├── README.md
├── requirements.txt
├── src/
│   ├── dataset.py             # CorruptedCIFAR10, ShuffledPixelCIFAR10, GaussianNoiseCIFAR10
│   ├── model.py               # SimpleCNN architecture
│   ├── train.py               # train_one_epoch(), evaluate(), run_experiment()
│   └── utils.py               # set_seed(), get_device(), save/load history, all plotting
└── scripts/
    ├── run_core.py            # Experiment 1: true labels vs fully random labels
    ├── run_corruption_sweep.py # Experiment 2: label corruption sweep 0–100%
    ├── run_regularization.py  # Experiment 3: regularization ablation (6 conditions)
    └── run_noise.py           # Experiment 4: shuffled pixels and Gaussian noise inputs

Results (JSON histories, PNG figures) are saved to results/. The results/ directory must exist before running. It currently includes my pre-computed outputs. The data/ directory is created automatically on the first run to cache the CIFAR-10 download.

---

All code in this repo is mine. The paper (Zhang et al. 2017) is the source of the research questions and experiment design, but it does not include code. I used the PyTorch, torchvision, NumPy, and matplotlib documentation for understanding and usage. No external repositories were copied or adapted.

---

Setup

Requires Python 3.8+ and a CUDA-capable GPU (experiments also run on CPU, just slower).

python -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate
pip install -r requirements.txt

CIFAR-10 (~170 MB) is downloaded automatically on first run.

---

Running Experiments

Each script is self-contained with hardcoded hyperparameters. Run from the project root:

# Experiment 1: true labels vs random labels (~25 min on a laptop GPU)
python scripts/run_core.py

# Experiment 2: corruption sweep 0%, 20%, 40%, 60%, 80%, 100% (~90 min total)
python scripts/run_corruption_sweep.py

# Experiment 3: regularization ablation — 6 conditions (~60 min total)
python scripts/run_regularization.py

# Experiment 4: shuffled pixels and Gaussian noise inputs (~40 min)
python scripts/run_noise.py

Each script prints per-epoch progress and writes to results/<experiment>/:

• history_*.json — training history (loss, accuracy)
• fig_*.png — plots similar to the paper's figures

---

Experiment Summary

Script	Reproduces	What it tests
run_core.py	Fig. 1a	True labels vs. fully random labels
run_corruption_sweep.py	Fig. 1b, 1c	Generalization degrading from 0→100% label noise
run_regularization.py	Table 1	Whether weight decay / augmentation prevents memorization
run_noise.py	Fig. 1a (extended)	Memorization with shuffled-pixel and Gaussian noise inputs

---

Model

SimpleCNN — 3 conv blocks (64→128→256 channels, BatchNorm, ReLU, MaxPool) + 2 FC layers. Total parameters: 2,474,506 (~2.5M). Defined in src/model.py.

Hyperparameters

Parameter	Value
Optimizer	SGD, momentum 0.9
Learning rate	0.01, ExponentialLR γ=0.95/epoch
Batch size	128
Weight decay	0.0 (Exp. 1/2/4); 1e-3 (Exp. 3)
Epochs	100 (true labels), 150 (partial corruption), 200 (random labels)
Seed	42