A tiny GPT-style language model for training on CPU, Apple Silicon, or NVIDIA GPU.
- 🚀 Multi-device support: MPS (Apple Silicon) | CUDA (NVIDIA) | CPU
- 📦 30M parameters: Small enough for CPU training
- 🔄 Scalable configs: micro → large (30M - 250M params)
- 💬 Chat fine-tuning: Convert base model to chat assistant
- ⚡ Auto-optimization: Batch size and settings tuned per device
# Setup
python3 -m venv venv
source venv/bin/activate
pip install torch numpy tiktoken datasets
# Check available devices
python3 device.py
# Train base model
python3 train_quick.py
# Chat with model
python3 chat.py "Hello!"# Install PyTorch with MPS support
pip install torch
# Verify MPS is available
python3 -c "import torch; print(torch.backends.mps.is_available())"
# Should print: True
# Train (will auto-detect and use MPS)
python3 train_quick.py# Install PyTorch with CUDA
pip install torch --index-url https://download.pytorch.org/whl/cu121
# Verify CUDA is available
python3 -c "import torch; print(torch.cuda.is_available())"
# Should print: True
# Train (will auto-detect and use CUDA)
python3 train_quick.pyWorks automatically if no GPU is detected.
| Device | Training Speed | Inference Speed | Batch Size |
|---|---|---|---|
| CPU (6 cores) | ~5 steps/sec | ~20 tok/s | 4 |
| MPS (M1) | ~20-30 steps/sec | ~100 tok/s | 16 |
| MPS (M2/M3) | ~40-60 steps/sec | ~200 tok/s | 16 |
| CUDA (RTX 3060) | ~100 steps/sec | ~500 tok/s | 8 |
| CUDA (RTX 3080) | ~200 steps/sec | ~800 tok/s | 32 |
| CUDA (RTX 4090) | ~500 steps/sec | ~2000 tok/s | 64 |
nano-llm/
├── config.py # Model configs (micro → large)
├── model.py # GPT architecture
├── device.py # Device detection (MPS/CUDA/CPU)
├── train_quick.py # Quick base training
├── train_chat.py # Chat fine-tuning
├── chat.py # Interactive chat
├── generate.py # Text generation
└── out/
├── model.pt # Base model checkpoint
└── chat_model.pt # Chat model checkpoint
python3 train_quick.py- 500 steps, batch size auto-tuned for your device
- Outputs:
out/model.pt
python3 train_chat.py- 300 steps, fine-tunes base model
- Outputs:
out/chat_model.pt
python3 chat.pypython3 chat.py "Tell me a joke"
python3 generate.py out/model.pt "Once upon a time"| Name | Params | Layers | Heads | Embed | Use Case |
|---|---|---|---|---|---|
| micro | ~30M | 6 | 6 | 384 | Testing, CPU |
| tiny | ~50M | 8 | 8 | 512 | Quick experiments |
| small | ~80M | 10 | 10 | 640 | Decent quality |
| medium | ~125M | 12 | 12 | 768 | Cloud training |
| large | ~250M | 16 | 16 | 1024 | Production |
For larger models, use free cloud GPUs:
!git clone <your-repo>
!pip install torch tiktoken datasets
!python device.py # Should show CUDA available
!python train_quick.py- 30 hours/week free GPU
- Enable GPU in Notebook settings
The device.py module automatically:
- Checks for Apple Silicon MPS
- Falls back to NVIDIA CUDA
- Falls back to CPU
from device import get_device, get_device_info, optimize_for_device
# Print all device info
print(get_device_info())
# Get best device
device = get_device()
# Get optimal settings
opts = optimize_for_device(device)
print(f"Batch size: {opts['batch_size']}")torch>=2.0
numpy
tiktoken
datasetsMIT