Rebuilding GPT’s foundation — no shortcuts, no HuggingFace.
This repo implements the Transformer architecture (the backbone of GPT and modern LLMs) from first principles in PyTorch. It’s not about speed. It’s about understanding the math → code → model.
- Scaled Dot-Product Attention
- Multi-Head Attention
- Positional Encoding
- Encoder & Decoder Layers
- End-to-End Transformer (+ toy training loop)
Clone locally:
git clone https://github.com/umer-ateeq/Coding-Transformers-using-pytorch.git
cd Coding-Transformers-using-pytorch💡 Star ⭐ if you find it useful — feedback from researchers & practitioners is welcome!