This project is a character-level Transformer language model built from scratch with PyTorch. It learns patterns in a text dataset and uses them to generate new text one character at a time.
The goal is to understand how Transformers work by implementing their core components step by step, including self-attention, Transformer blocks, training, and text generation.
Train the model, then generate text:
uv run python train.py
uv run python generate.py