AutoGAM: Automated General Additive Models

A research engineering artifact for fast, interpretable regression with automated pairwise interaction discovery.

AutoGAM explores a practical middle ground between linear models and high-capacity black-box learners. It extends Generalized Additive Models (GAMs) with a fixed spline-basis workflow and automated interaction screening, targeting the common ML engineering tradeoff between interpretability, accuracy, reproducibility, and runtime.

This repository is a companion to the MSc thesis "AutoGAM: Automated General Additive Models" by Sebastien Toscano and Venketa Koganti. It organizes the paper, methodology, extracted figures, benchmark summary, and citation material for the AutoGAM research work.

Status

Area	Current state
Research paper	Available at paper/AutoGAM.pdf
Methodology	Summarized in docs/methodology.md
Research summary	Available in docs/research_summary.md
Visual assets	Extracted thesis figures and selected research figures under assets/figures/

This repository is a documented ML research artifact.

Why This Project Matters

Many production ML systems do not need the most complex model available. They need models that are accurate enough, fast to train, stable at inference, easy to inspect, and credible under review.

AutoGAM is designed around that reality:

Need	AutoGAM response
Nonlinear relationships	Uses spline terms instead of forcing linear effects
Interaction discovery	Screens pairwise feature interactions automatically
Reproducibility	Freezes spline basis construction after training
Interpretability	Keeps model terms explicit and decomposable
Benchmarking	Compares against OLS, Random Forest, XGBoost/GBT, and MLPs
Runtime discipline	Measures training, inference, and tuning cost

Core Idea

Traditional GAMs provide interpretable nonlinear modeling, but they often require manual feature engineering and interaction specification. AutoGAM adds an automated layer around the GAM workflow:

Raw tabular data
    -> split and scale
    -> construct fixed spline basis
    -> screen pairwise interactions with OLS
    -> keep statistically supported interactions
    -> fit interpretable spline model
    -> benchmark accuracy and runtime

The result is a model family positioned between simple statistical baselines and heavier ensemble or neural methods.

Key Innovations

Fixed Spline Basis

AutoGAM constructs spline basis functions during training and reuses the same basis at validation and inference time. This prevents silent basis reconstruction, improves reproducibility, and makes prediction behavior easier to reason about.

Automated Interaction Detection

Candidate pairwise interactions are screened using an OLS-based statistical selection step. Significant interactions are ranked and capped before being added as explicit spline terms.

Interpretable Nonlinearity

Each retained term remains inspectable. Instead of treating predictive performance as the only goal, AutoGAM preserves a readable additive structure that supports model debugging, review, and stakeholder explanation.

Runtime-Aware Benchmarking

The thesis benchmarks not only MAE, RMSE, and R2, but also training time, inference time, and hyperparameter optimization cost. This makes the project closer to applied ML systems evaluation than a purely academic model comparison.

Benchmark Summary

Experiments used a synthetic benchmark and five real-world regression datasets:

Dataset	Purpose
Synthetic regression	Ground-truth recovery and interaction validation
California Housing	Real-world tabular regression with socioeconomic features
Electrical Grid Stability	Nonlinear dynamics in simulated grid behavior
MAGIC Gamma Telescope	Larger scientific benchmark with noisy nonlinear structure
Diabetes	Small medical regression benchmark
Wine Quality Red	Physicochemical feature interactions

Baselines included:

• Ordinary Least Squares
• Random Forest
• Gradient Boosted Trees / XGBoost-style boosting
• Multi-Layer Perceptron
• Main-effects GAM variant

High-level findings:

• Interaction terms improved main-effects GAM performance in several settings.
• AutoGAM trained in seconds across the evaluated datasets.
• The strongest black-box models, especially tree ensembles, often achieved the best raw accuracy.
• AutoGAM's value is strongest where interpretability, repeatability, and fast experimentation matter alongside predictive quality.

Visual Evidence

The full thesis includes the detailed plots and benchmark figures. A few representative visuals have been extracted into the repository for fast review.

Figure	File
Interaction screening heatmap	assets/figures/interaction_screening.png
Synthetic benchmark summary	assets/figures/synthetic_benchmark_summary.png
Synthetic test evaluation	assets/figures/synthetic_test_evaluation.png
Full extracted figure set	assets/figures/thesis-extracted/

Architecture

                    +----------------------+
                    |  Benchmark Datasets  |
                    +----------+-----------+
                               |
                               v
                    +----------------------+
                    | Split / Scale / Clip |
                    +----------+-----------+
                               |
                               v
                    +----------------------+
                    | Fixed Spline Basis   |
                    +----------+-----------+
                               |
              +----------------+----------------+
              |                                 |
              v                                 v
   +----------------------+          +----------------------+
   | Main Effects Model   |          | Interaction Screen   |
   | spline terms only    |          | pairwise OLS tests   |
   +----------+-----------+          +----------+-----------+
              |                                 |
              +----------------+----------------+
                               |
                               v
                    +----------------------+
                    | AutoGAM Fit          |
                    | selected terms       |
                    +----------+-----------+
                               |
                               v
                    +----------------------+
                    | Evaluation           |
                    | accuracy + runtime   |
                    +----------------------+

Repository Structure

.
|-- README.md
|-- paper/
|   |-- AutoGAM.pdf
|   `-- README.md
|-- docs/
|   |-- project_overview.md
|   |-- methodology.md
|   `-- research_summary.md
|-- assets/
|   |-- figures/
|   `-- tables/

Content organization:

• docs/*.md for narrative technical documentation.
• paper/* for the source thesis PDF and paper metadata.
• assets/figures/* for extracted research figures and curated visuals.
• assets/tables/* for benchmark summaries.

Limitations

AutoGAM is not positioned as a universal replacement for tree ensembles or neural networks. Its current scope is narrower and intentionally interpretable:

• Focuses on regression with continuous outcomes.
• Pairwise interaction screening scales quadratically with feature count.
• Continuous variables are the most natural fit; categorical handling requires preprocessing.
• Fixed spline bases improve reproducibility but limit extrapolation outside the training range.
• Adaptive smoothing and penalization are outside the evaluated scope.

Thesis Citation

If you reference this work, cite:

@mastersthesis{toscano_koganti_autogam,
  title  = {AutoGAM: Automated General Additive Models},
  author = {Toscano, Sebastien and Koganti, Venketa},
  school = {SKEMA Business School},
  year   = {2026},
  type   = {MSc Thesis}
}