ft_linear_regression

Linear regression from scratch in C++17 — gradient descent with feature normalization, three separate programs for training, prediction, and visualization.

---

Overview

This project implements univariate linear regression without any machine learning library. It learns the relationship between a car's mileage and its price using the gradient descent algorithm, then lets you query predictions interactively.

The model is split into three independent programs:

Program	Purpose
train	Read data.csv, run gradient descent, save θ
predict	Load saved θ, accept mileage input, return price
plot	Visualize raw data, regression line, or both

---

How it works

The model

The regression line is defined as:

price = θ₀ + θ₁ × mileage

θ₀ is the intercept and θ₁ is the slope. Both start at 0.0 and are iteratively updated by gradient descent.

Gradient descent

At each iteration the algorithm computes how far off each prediction is (the error), then nudges θ₀ and θ₁ in the direction that reduces the total error:

tmpθ₀ = learningRate × (1/m) × Σ (estimatePrice(xᵢ) − yᵢ)
tmpθ₁ = learningRate × (1/m) × Σ (estimatePrice(xᵢ) − yᵢ) × xᵢ
θ₀ -= tmpθ₀
θ₁ -= tmpθ₁

Both thetas are updated simultaneously using temporary values so neither update influences the other within the same iteration.

Feature normalization

Raw mileage values (e.g. 240000) cause very small, slow gradient steps. The training program normalizes both mileage and price to zero mean and unit variance (Z-score normalization) before running gradient descent:

x_norm = (x − mean) / std

After training converges, the thetas are denormalized back to the original scale so that predict and plot can use raw km and dollar values directly.

Metrics

After training you can evaluate the model with:

• MSE (Mean Squared Error) — average squared prediction error
• R² (coefficient of determination) — proportion of variance explained by the model. A value of 1.0 is a perfect fit; 0.0 means the model does no better than predicting the mean.

---

Prerequisites

Compiler

clang++ --version   # requires C++17 support
# or: g++ --version

gnuplot (required only for plot)

Ubuntu / Debian

sudo apt-get update && sudo apt-get install -y gnuplot

Fedora / RHEL

sudo dnf install gnuplot

Arch Linux

sudo pacman -S gnuplot

Verify the install:

gnuplot --version

Boost libraries

Boost is required for the plotting helper because gnuplot-iostream.h uses Boost stream and tuple support under the hood.

---

Build

# build all three programs
make

# build individually
make train
make predict
make plot

# rebuild from scratch
make re

# remove object files
make clean

# remove object files and executables
make fclean

The Makefile checks for gnuplot before building plot and will print an error if it is not found.

---

Usage

1 — Train the model

./train

Reads data.csv from the current directory, runs gradient descent, and writes the learned parameters to thetas.txt.

╔════════════════════════════════════════╗
║  LINEAR REGRESSION - TRAINING MODE    ║
╚════════════════════════════════════════╝

> Loading dataset..................      [OK]
> Sorting data by mileage..........      [OK]
> Initializing gradient descent....      [OK]
> Saving model parameters...........     [OK]

✓ Model saved to thetas.txt
✓ You can now use the predict program!

2 — Predict a price

./predict

Loads thetas.txt and enters an interactive loop. Enter a mileage in km and get an estimated price. Type q to quit.

The Vehicle Mileage (or 'q' to quit): 100000

┌──────────────────────────────────────┐
│  PREDICTION RESULTS                  │
├──────────────────────────────────────┤
│  Mileage:             100000 km      │
│  Estimated Price:       5800 $       │
└──────────────────────────────────────┘

3 — Visualize

./plot

Opens an interactive menu (requires thetas.txt to be present):

╔════════════════════════════════════════╗
║  Linear Regression Visualization      ║
╠════════════════════════════════════════╣
║  0 │ Raw Data Plot                    ║
║  1 │ Regression Line Plot             ║
║  2 │ Complete Analysis Plot           ║
║  3 │ Exit                             ║
╚════════════════════════════════════════╝

---

Configuration

Training hyperparameters can be overridden at compile time:

make train CXXFLAGS="-Wall -Wextra -Werror -std=c++17 -DITERATIONS=500 -DLEARNING_RATE=0.8"

Constant	Default	Description
ITERATIONS	100	Number of gradient descent iterations
LEARNING_RATE	1.5	Step size for each gradient descent step

A learning rate that is too large will cause the loss to diverge; too small will converge slowly. The default of 1.5 is aggressive but works well because the data is normalized before training.

---

Data format

data.csv must be a comma-separated file with a header row and two columns:

km,price
240000,3650
139800,3800
...

Values are automatically scaled by ÷ 1000 internally (km → ×10³ km, price → ×10³ $) to keep numbers in a convenient range for display.

---

Repository layout

├── Makefile
├── README.md
├── data.csv                  ← training data (km, price)
├── thetas.txt                ← generated after ./train
├── gnuplot-iostream.h        ← header-only gnuplot C++ interface (Boost)
└── srcs/
    ├── linearRegression.h    ← class definition, constants, systemBoot declaration
    ├── linearRegression.cpp  ← training, normalization, metrics, I/O
    ├── trainModel.cpp        ← main() for ./train
    ├── predictPrice.cpp      ← main() for ./predict
    └── plotUtils.cpp         ← gnuplot-iostream visualization

---

Notes

• thetas.txt is deleted when you quit predict with q. Re-run ./train to regenerate it.
• train and predict have no external dependencies beyond a C++17 compiler.
• plot requires both gnuplot and Boost. The Makefile will exit early with an error if gnuplot is not in your $PATH.
• On headless servers (no display), gnuplot may fail to open a window. Install Xvfb or switch to gnuplot's png terminal.