- [x] MLP
- [x] Token Embedding

- [x] Batch Normalization
- [x] Layer Normalization

- [x] Logistic Regression - Theory
- [x] Gradient Descent/Autograd - Theory

- [x] Byte Pair Encoding

- [x] Rotary Positional Encoding

- [x] Quanitization
- [-] LoRA Finetuning
- [-] QLoRA

- [x] Attention
- [x] Decoder Transformer (GPT)
- [x] Encoder Transformer (BERT - NER/Sentiment Analysis)
- [x] Full Transformer / Cross Attention
- [x] T5

- [x] Convolution / CNN
- [x] ResNet
- [x] YOLO

- [x] RNN
- [x] GRU
- [x] LSTM

- [x] VAE
- [ ] GAN
- [ ] Diffusion Model
- [x] Autoencoders

- [x] KV Cache
- [x] Parallelization

- [x] Vision Transformer
- [x] UNet
- [ ] VLA

- [x] Scaling laws - Theory

- [x] Distributed Data Processing
- [x] Profiling
- [x] Flash attention
- [ ] MOE

- [x] karparthy/llama.c
- [x] Flux
- [ ] VGGT