Skip to content

Releases: Ashx098/Mini-LLM

Mini LLM V 1.0 Full Implementation of SOTA LLM architecture

08 Dec 06:14

Choose a tag to compare

This release introduces Mini-LLM v1, an 80M parameter decoder-only transformer trained entirely from scratch, built using a fully custom architecture, custom tokenizer, and a bespoke training loop.

This project demonstrates how modern LLMs are constructed end-to-end including tokenization, architecture design, optimization, and inference while remaining readable, hackable, and research-friendly.

Mini-LLM is NOT instruction-tuned; it is a pure pretrained base model, suitable for learning, further training, and experimentation.