Building a complete LLM from scratch. No libraries. No shortcuts. Just NumPy, math, and first principles.

LLM Which Was Trained On A Consumer GPU

DeepSeek-style MoE + MLA from scratch in PyTorch, with a router-specialization probe (mutual information) and a dense/+MoE/+MLA ablation. Educational, nano-scale, measured.

A large language model built from scratch using NumPy. Currently, it is in the initial stage of the development process using numpy.🔧

From-scratch PyTorch: frontier LLM techniques as of 2026-Q1 — the Muon optimizer and Multi-Token Prediction, plus a base BPE tokenizer. Self-contained, self-checking modules.

From-scratch PyTorch GPT: the GPT-2 skeleton plus the components of current open LLMs — RMSNorm, RoPE, SwiGLU, GQA, no-bias + tied embeddings. Each isolated and self-checking.

Some attempts, based on MicroGPT.

Building a Large Language Model from the ground up.