Releases: Ashx098/Mini-LLM
Releases · Ashx098/Mini-LLM
Mini LLM V 1.0 Full Implementation of SOTA LLM architecture
This release introduces Mini-LLM v1, an 80M parameter decoder-only transformer trained entirely from scratch, built using a fully custom architecture, custom tokenizer, and a bespoke training loop.
This project demonstrates how modern LLMs are constructed end-to-end including tokenization, architecture design, optimization, and inference while remaining readable, hackable, and research-friendly.
Mini-LLM is NOT instruction-tuned; it is a pure pretrained base model, suitable for learning, further training, and experimentation.