Lead AI Developer — production AI infrastructure by day, neural networks rebuilt from scratch by night, so I actually understand what I ship.
- 🤖 Lead a private, multi-tenant AI platform — composable agents + RAG over a customer's own documents, in production.
- 🔬 I rebuild modern architectures from first principles — no
nn.Transformer, no wrapper I can't explain. - 🎮 Also: a Godot game shipped on Steam, and a degree in hardware / electronics.
- 📚 Permanent student, pushing from LLMs toward CUDA and quantum.
|
|
The modern dense transformer. The GPT-2 skeleton upgraded to the 2026 open-weight stack — RMSNorm · RoPE · SwiGLU · GQA · tied weights — built one self-checking component at a time. Trained on a single RTX 3090 (val ≈ 1.48). |
|
|
The sparse template: MoE + MLA in one model. Built from scratch with the instruments to study it — a labeled multi-domain corpus, a router-specialization probe (mutual information), and a seed-averaged stack ablation that confirms the balancing ↔ specialization tradeoff. |
|
|
Frontier techniques, from scratch. The Muon optimizer (used to train Kimi K2) and Multi-Token Prediction (DeepSeek-V3, Gemma 4, GLM, Qwen), plus a base BPE tokenizer — each self-contained and self-checking, with an AdamW-vs-Muon benchmark. |
At work: I lead a private, multi-tenant enterprise AI platform — composable agents and retrieval over a customer's own documents, shipped to production. (Architecture kept private.)
A tool to prototype game assets in my own pixel-art style: the model gives the vibe, a from-scratch numpy pipeline enforces the rules — real grid · exact palette · dithering — and I finish in Aseprite. Split by engine so the stronger one wins.
|
|
The from-scratch core. Turns any image into real pixel art by enforcing the hard rules no model respects — real grid · exact N-color palette · dithering — plus a human-like saliency / value-first downscale and REF_LOCK (lock a reference's native grid + palette). Pure numpy, no GPU, engine-agnostic. |
|
|
The autoregressive engine. Janus-Pro (image as tokens, like a text transformer) fine-tuned with a LoRA of my style + constrained decoding. The lane closest to my from-scratch transformer work — and the best-fidelity results so far. |
|
|
The diffusion engine. SDXL + a pixel-art LoRA + IP-Adapter — the mature control stack (LoRA / IP-Adapter / ControlNet), the competing engine and fallback. |
🔒 Private while I polish them — opening soon.
I didn't invent these — I learned them by rebuilding and verifying each one in code, following Karpathy's Zero to Hero and the open-model papers.
| Built | What it is | Learned from |
|---|---|---|
| micrograd | a tiny autograd engine — backprop by hand | Karpathy · Zero to Hero |
| makemore | MLP, BatchNorm, manual backprop, WaveNet | Karpathy · Zero to Hero |
| self-attention | the attention mechanism, from the math up | Attention Is All You Need |
| modern-nanoGPT | the modern dense transformer (RMSNorm · RoPE · GQA · SwiGLU) | nanoGPT + Llama / Mistral / Qwen |
| nano-moe-mla | sparse MoE + MLA in one model, with a routing probe + ablation | DeepSeek-V2/V3 papers |
| Muon optimizer | orthogonalized-momentum (Newton-Schulz) + a benchmark | Keller Jordan · Moonshot (Kimi K2) |
| Multi-Token Prediction | a 2nd head predicting t+2 — a denser training signal | DeepSeek-V3 |
| BPE tokenizer | byte-pair encoding, exact round-trip | Karpathy · minBPE |
Method: from-scratch first; a wrapper is only allowed once I can name what it wraps and the trade-off.
|
✅ Done
|
🔄 In progress
|
|
⏭️ Next
|
🧪 Later
|
AI / ML
Backend
Frontend
Infra & also