Teaching a 1.3B-parameter LLM (MiniCPM5-1B-SFT) to play chess on a single consumer GPU (RTX 4090) — and rigorously measuring what actually works. The project covers supervised fine-tuning, RLVR/GRPO reasoning training, self-play conversion curricula, verifier heads, and a hand-built (no-Stockfish) alpha-beta engine, all with an honest, calibrated evaluation harness.
Read RESULTS.md for the full findings. TIMELINE.md is the original roadmap, kept as history.
| What | Result |
|---|---|
| Fixing SFT (completion-only loss) | top-1 move-match 10% → 21%, ACPL 261 → 188 |
| RLVR (think → reward only the final move) | ACPL 197 → 150; the one lever that beat the SFT plateau |
| Deep human-style reasoning + RLVR | best accuracy ever (top-1 27.6%, top-3 50.4%) — yet lower game-Elo: the wall is conversion, not accuracy |
| Conversion wall (KQvK/KRvK vs random) | RL failed 3× (0–2%); one SFT pass on 16.9k oracle demos: 2% → 59% — procedures must be demonstrated, not reward-shaped |
| Hand-built pure-Python engine (no Stockfish) | ~1538 calibrated Elo at depth 4; ablation shows the LLM's guidance is decorative |
python -m venv venv
venv\Scripts\activate
pip install -r requirements.txt
python src/download_assets.py # fetches Stockfish into bin/ (used as data oracle & eval anchor)venv\Scripts\python.exe src\play_human.py # you (White) vs the LLM
venv\Scripts\python.exe src\play_human.py --opponent engine --depth 4 # vs the ~1538 hand-built engine
venv\Scripts\python.exe src\chat.py --chess # chat with the fine-tune directlyMoves in UCI (e2e4) or SAN (Nf3, O-O); commands moves, board, undo, fen, quit.
# 1. Generate positions + SFT data (Stockfish as oracle)
venv\Scripts\python.exe src\gen_chess_data.py
venv\Scripts\python.exe src\build_reasoning_data.py # grounded <think> cold-start data
# 2. Cold-start SFT (completion-only loss; --grad-checkpoint needed on 24GB VRAM)
venv\Scripts\python.exe src\train_v2.py --mode reasoning --output-dir models\minicpm_chess_reasoning_cold --grad-checkpoint
# 3. RLVR: GRPO where the reward is the final move's Stockfish centipawn loss
venv\Scripts\python.exe src\grpo_reasoning.py --cold models\minicpm_chess_reasoning_cold --output-dir models\minicpm_chess_rlvr
# 4. Conversion curriculum (engine-free): oracle demos -> SFT, or confinement-reward RL
venv\Scripts\python.exe src\gen_convert_demos.py --suite KQK,KRK --games 1500
venv\Scripts\python.exe src\selfplay_rl.py --reward convert --cold models\minicpm_chess_reasoning_cold_v2
# 5. Evaluate: fast move-quality + calibrated game-play Elo
venv\Scripts\python.exe src\eval_movematch.py --lora-dir models\minicpm_chess_rlvr --greedy
venv\Scripts\python.exe src\evaluate_v2.py --lora-dir models\minicpm_chess_policy_v2 --agent policy --ladder pipelines\ladder_calibrated.json
venv\Scripts\python.exe src\evaluate_v2.py --eval-base --agent puresearch --search-depth 4 --ladder pipelines\ladder_calibrated.json # engine-only ablationLong unattended runs use src\auto_pipeline.py --spec pipelines\<spec>.json, which executes a staged JSON pipeline detached from any terminal and logs progress to reports/orchestrate_status.json.
LLM Chess Player/
├── src/
│ ├── gen_chess_data.py # position/data generator (phase quotas, endgame coverage)
│ ├── build_reasoning_data.py # grounded <think> cold-start SFT data
│ ├── train_v2.py # SFT trainer: completion-only loss, LoRA, adapter chaining
│ ├── grpo_reasoning.py # RLVR: GRPO rewarding only the final move's quality
│ ├── selfplay_rl.py # conversion curriculum RL (Stockfish-free rewards)
│ ├── gen_convert_demos.py # engine-free oracle demos for won-endgame technique
│ ├── convert_reward.py # confinement reward + conversion oracle
│ ├── eval_movematch.py # top-1/top-3/ACPL vs Stockfish (fast signal)
│ ├── evaluate_v2.py # game-play Elo vs calibrated opponent ladder (95% CI)
│ ├── chess_eval.py # hand-built evaluation function (no Stockfish)
│ ├── search_engine.py # alpha-beta + quiescence + TT + null-move (~1538 Elo @ d4)
│ ├── play_human.py # play the model or engine in your terminal
│ ├── chat.py # chat with the base model or the chess fine-tune
│ └── ... # verifiers, STaR/DPO experiments, annotators, v1 scripts
├── pipelines/ # ladder configs + overnight pipeline specs
├── reports/ # eval history (CSV), probe results, sample outputs
├── data/ # (gitignored) generated datasets
├── models/ # (gitignored) LoRA adapters & checkpoints
├── bin/ # (gitignored) Stockfish binary
├── RESULTS.md # full findings & honest measurement story
└── TIMELINE.md # original roadmap (superseded; kept as history)
Note: train.py, evaluate_elo.py, expert_iteration.py, and dataset_generator.py are the superseded v1 pipeline, kept for the record — RESULTS.md explains what was wrong with them.
Everything runs locally on one RTX 4090 (24 GB), bfloat16 + LoRA. Training scripts cap VRAM (--mem-fraction) to avoid Windows WDDM shared-memory spill, auto-resume from checkpoints, and hold a keep-awake lock during long runs.