Reproducibility package for the article:
C. E. Ramirez Ovalle, A. Alvarez Bustos, A. J. Castrillón. Graph Energy and Nuclear-Norm Budgeting in Fully Connected Neural Networks. Submitted to Journal of Computational Mathematics and Data Science (Elsevier), 2025.
This repository contains the complete source code to reproduce all numerical experiments reported in the paper. Everything is consolidated in a single Jupyter notebook (EXPARTFINAL.ipynb) — there are no external scripts or separate files required.
The main contributions verified here are:
- Hölder stability bound (Theorems 2–3): energy fluctuations around a minimiser are controlled by √(loss gap).
- ADMM convergence (Proposition 1, Corollary 4): the constrained training algorithm achieves zero constraint violation.
- Four-method comparison (Experiment 3): STD, WD, NUC, ADMM across 3 depths × 3 seeds.
- Joint vs per-layer projection (Experiment 4): per-layer budget allocation collapses at depth ≥ 6.
- Spectral analysis (Experiment 5): effective rank reduction under energy constraints.
- Five-method comparison with Spectral Normalization (Experiment 6): STD, WD, NUC, SN, ADMM across 3 depths × 5 seeds.
.
├── EXPARTFINAL.ipynb # ALL experiments (1–6) in a single self-contained notebook
├── README.md # This file
└── results/ # Pre-computed outputs (figures + tables)
├── fig1_holder_mc.{pdf,png} # Exp 1: Hölder MC verification
├── fig2_admm_diagnostics.{pdf,png} # Exp 2: ADMM convergence diagnostics
├── fig3_dynamics.{pdf,png} # Exp 3: Training dynamics (loss, acc, energy)
├── fig4_acc_depth.{pdf,png} # Exp 3: Accuracy by depth (bar chart)
├── fig5_projection.{pdf,png} # Exp 4: Joint vs per-layer projection
├── fig6_spectral.{pdf,png} # Exp 5: Singular value profiles
├── fig7_rank.{pdf,png} # Exp 5: Effective rank by depth
├── fig8_sn_comparison.{pdf,png}# Exp 6: Five-method accuracy bars
├── fig9_sn_dynamics.{pdf,png} # Exp 6: Training dynamics (5 methods)
├── fig10_sn_spectral.{pdf,png} # Exp 6: Singular value profiles (5 methods)
├── fig11_sn_energy_smax.{pdf,png} # Exp 6: Energy + σ_max summary
├── table1_holder_mc.csv # Exp 1: MC statistics
├── table2_admm.csv # Exp 2: ADMM residuals
├── table3_comparison.csv # Exp 3: Four-method comparison
├── table4_projection.csv # Exp 4: Projection ablation
├── table5_spectral.csv # Exp 5: Effective rank ratios
├── table6_sn_comparison.csv # Exp 6: SN comparison (accuracy + energy)
├── table7_sn_spectral.csv # Exp 6: SN spectral analysis
└── table_summary.csv # Aggregated verification summary
Note:
data/(MNIST) is auto-downloaded on first run and git-ignored.
pip install torch torchvision numpy pandas matplotlib scipyjupyter notebook EXPARTFINAL.ipynb
# Run All Cells (Cell → Run All)That's it — one notebook, one command. All 6 experiments run sequentially, producing every table and figure saved to results/.
Verifies Theorems 2–3: the adjustment ratio ‖ΔΩ‖ / √(ΔL) stays below 1 across 200 trials on both synthetic regression and MNIST binary classification.
Outputs: fig1_holder_mc.{pdf,png}, table1_holder_mc.csv
Tracks primal residual, dual residual, energy, and constraint violation across 25 epochs for 3L, 6L, 9L architectures. Verifies Proposition 1 and Corollary 4.
Outputs: fig2_admm_diagnostics.{pdf,png}, table2_admm.csv
Compares STD (no regularisation), WD (L₂ decay), NUC (soft nuclear penalty), SN (spectral normalisation), and ADMM (hard energy constraint) on MNIST with 10,000 training samples.
| Depth | STD | WD | NUC | SN | ADMM | ADMM Ω≤τ? |
|---|---|---|---|---|---|---|
| 3L | 96.2% | 96.7% | 96.5% | 95.3% | 96.2% | 5/5 ✓ |
| 6L | 96.2% | 96.5% | 96.1% | 93.6% | 95.4% | 5/5 ✓ |
| 9L | 94.8% | 94.2% | 93.4% | 78.3±5.8% | 88.2±5.4% | 5/5 ✓ |
Outputs: fig3_dynamics.{pdf,png}, fig4_acc_depth.{pdf,png}, table3_comparison.csv
| Depth | Per-layer (%) | Joint (%) | WD (%) |
|---|---|---|---|
| 3L | 94.10 | 95.90 | 96.00 |
| 6L | 10.05 | 95.00 | 96.60 |
| 9L | 11.65 | 79.30 | 94.70 |
Per-layer projection collapses to chance level (≈10%) at depth ≥ 6.
Outputs: fig5_projection.{pdf,png}, table4_projection.csv
ADMM reduces effective rank by 2.4% (3L), 5.8% (6L), 1.5% (9L) relative to unconstrained training. NUC achieves stronger compression (4.5–10.6%) but cannot guarantee the energy budget.
Outputs: fig6_spectral.{pdf,png}, fig7_energy_smax.{pdf,png}, table5_spectral.csv
Compares all five methods (STD, WD, NUC, SN, ADMM) across 3 depths × 5 seeds = 75 runs. SN constrains σ_max per layer but fails to control total graph energy. At 9L, SN collapses while ADMM maintains certified constraint satisfaction.
Outputs: fig8–fig11, table6_sn_comparison.csv, table7_sn_spectral.csv
All experiments use the same configuration (defined in Cell 4 of the notebook):
| Parameter | Value | Description |
|---|---|---|
N_TRAIN |
10,000 | Training subset size |
N_VAL |
2,000 | Validation subset size |
BATCH_SIZE |
64 | Mini-batch size |
EP |
25 | Training epochs |
LR |
0.01 | Learning rate (SGD) |
WD_LAMBDA |
1e-3 | Weight decay coefficient |
BUDGET_FRAC |
0.80 | Energy budget τ = 80% of Ω(θ_STD) |
SEEDS |
42, 123, 456, 789, 1024 | Random seeds |
| Component | Version / Spec |
|---|---|
| CPU | Apple M3 Pro (11 cores) |
| RAM | 18 GB unified memory |
| OS | macOS |
| Python | 3.11.11 |
| PyTorch | 2.8.0 (CPU only) |
| Runtime | ≈ 15 min total (all 6 experiments) |
All experiments fix random seeds at three levels:
random.seed(s),numpy.random.seed(s),torch.manual_seed(s)torch.backends.cudnn.deterministic = True- MNIST indices are sampled with fixed seeds before each experiment group
Results may vary slightly across hardware due to floating-point non-associativity, but all qualitative conclusions (constraint satisfaction, SN collapse, per-layer failure) are robust.
If you use this code, please cite:
@article{RamirezOvalle2025GraphEnergy,
author = {Ramirez Ovalle, Carlos . and Alvarez, Abel. and Castrillón Andrés Julian.},
title = {Graph Energy as Spectral Regularizer for Deep Neural Networks},
journal = {Journal of Computational Mathematics and Data Science},
year = {2025},
note = {Submitted}
}This project is licensed under the MIT License — see LICENSE for details.