ML for Materials Science

A hands-on exploration of core machine learning techniques applied to crystal-structure identification using Steinhardt bond-orientational order parameters.

Each module is a self-contained Python script that can be run independently.

---

Project Structure

ml-materials/
├── data/
│   ├── structure_data.csv        # Crystal structure dataset (FCC / BCC / HCP)
│   └── synthetic_pca_data.csv    # Synthetic 2-D dataset for PCA
│
├── 01_tiny_language_model/
│   └── tiny_lm.py                # Transformer-style LM built with NumPy only
│
├── 02_pca/
│   └── pca.py                    # PCA from scratch + scikit-learn verification
│
├── 03_supervised_classification/
│   └── classification.py         # Logistic Regression + Decision Tree
│
├── 04_neural_network/
│   └── neural_network.py         # MLP classifier with Keras / TensorFlow
│
├── 05_unsupervised_clustering/
│   └── kmeans_clustering.py      # K-Means clustering + evaluation
│
├── results/
│   ├── 01_tiny_lm_results.txt
│   ├── 02_pca_results.txt
│   ├── 03_classification_results.txt
│   ├── 04_neural_network_results.txt
│   ├── 05_kmeans_results.txt
│   └── plots/                    # All generated figures (PNG)
│
├── requirements.txt
└── README.md

---

Modules

01 — Tiny Language Model (01_tiny_language_model/)

A minimal transformer-style language model built entirely from NumPy — no deep learning frameworks.

Covers:

• Tokenisation and vocabulary construction
• Word embeddings
• Scaled dot-product self-attention (Q, K, V matrices)
• Feed-forward network (ReLU activation)
• Cross-entropy loss with manual SGD backpropagation

Task: given the context ["the", "battery", "is"], predict the next word.

---

02 — PCA (02_pca/)

Principal Component Analysis implemented from scratch, then verified with scikit-learn.

Covers:

• Mean-centring
• Manual covariance matrix computation
• Eigendecomposition
• Projection onto top-k principal components
• sklearn PCA + StandardScaler pipeline

Dataset: synthetic 2-D data with correlated features.

---

03 — Supervised Classification (03_supervised_classification/)

Two classifiers trained to distinguish FCC / BCC / HCP crystal structures.

Model	Decision Boundary	Interpretability
Logistic Regression	Linear	Coefficients per feature
Decision Tree	Non-linear	IF-THEN rules

Features: Steinhardt bond-orientational order parameters (q2–q8).
Evaluation: accuracy, cross-validation, confusion matrix.

---

04 — Neural Network (04_neural_network/)

A feedforward MLP (Keras / TensorFlow) for crystal structure classification.

Architecture:

Input → Dense(64, ReLU) → Dense(32, ReLU) → Dense(3, Softmax)

Covers:

• Label encoding + one-hot encoding
• Training/validation accuracy and loss curves
• Adam optimiser, categorical cross-entropy
• Confusion matrix

Also includes a theoretical overview of Active Learning — a strategy for reducing labelling cost by querying only the most uncertain samples.

---

05 — K-Means Clustering (05_unsupervised_clustering/)

Unsupervised clustering applied to crystal structure data — no labels used during training.

Covers:

• K-Means with k-means++ initialisation
• StandardScaler preprocessing
• Majority-vote label remapping for evaluation
• Adjusted Rand Index (ARI) and Normalised Mutual Information (NMI)

Results are compared against ground-truth labels to measure how well the geometry of the feature space reflects the true crystal structure categories.

---

Dataset

structure_data.csv

Crystal structure data described by Steinhardt bond-orientational order parameters.

Steinhardt parameters are rotationally and translationally invariant descriptors based on spherical harmonics:

$$q_l(i) = \left( \frac{4\pi}{2l+1} \sum_{m=-l}^{l} |q_{lm}(i)|^2 \right)^{1/2}$$

Column	Description
q2, q4, q5, q6, q8	Steinhardt order parameters
label	Crystal structure: fcc, bcc, or hcp

synthetic_pca_data.csv

Two-dimensional synthetic dataset with correlated features, used for PCA demonstration.

---

Setup

# Clone the repository
git clone https://github.com/<your-username>/ml-materials.git
cd ml-materials

# Install dependencies
pip install -r requirements.txt

Place structure_data.csv and synthetic_pca_data.csv in the data/ directory.

---

Running the Modules

Each module is a standalone script. Run from its own directory:

# Module 1 — Tiny LM (no external data needed)
cd 01_tiny_language_model
python tiny_lm.py

# Module 2 — PCA
cd 02_pca
python pca.py

# Module 3 — Supervised classification
cd 03_supervised_classification
python classification.py

# Module 4 — Neural network (requires TensorFlow)
cd 04_neural_network
python neural_network.py

# Module 5 — K-Means clustering
cd 05_unsupervised_clustering
python kmeans_clustering.py

Each script writes a detailed results file to results/ and saves all plots to results/plots/.

---

Dependencies

Package	Purpose
numpy	Numerical computation, linear algebra
pandas	Data loading and manipulation
scikit-learn	ML models, preprocessing, evaluation
matplotlib	Plotting
seaborn	Statistical visualisation
tensorflow	Neural network (Module 4 only)