llama.cpp fork with TurboQuant WHT-rotated KV cache & weight compression + Gemma 4 MTP and Qwen 3.6 NextN speculative decoding (+30-50% throughput).

DMax: Aggressive Parallel Decoding for dLLMs

Fused TBQ4 Flash Attention + MTP + Shared Tensors for llama.cpp — 82+ tok/s with lossless 4.25 bpv KV cache at 200K context on RTX 4090

Run Hermes Agent + Claude Code locally on llama.cpp — zero API costs. A 4h / 7M-token session that would have cost $94 on Claude Opus 4.7

Curated collection of research on the limitations of next-token prediction and methods that go beyond it.

Optimized vLLM setup for Qwen3.6-27B-FP8 on dual RTX PRO 6000 Blackwell (192 GB GDDR7, no NVLink) ; config, benchmark sweep results, and custom chat template with thinking mode off by default.

Deploy Qwen3.6-35B-A3B (Q4_K_XL) + MTP speculative decoding on a single NVIDIA L4 24GB — GCP g2-standard-8 — via the official llama.cpp Docker image. Decode-optimized to ~91–99 tok/s (min ~91 chat, max ~99 math), lossless (full GPU residency + ECC-off).

Optimized vLLM setup for Gemma 4 31B NVFP4 with MTP on dual RTX PRO 6000 Blackwell using vllm and docker: native FP4 Tensor Cores, Multi-Token Prediction (96.5% acceptance rate), and prefix caching. Includes benchmark results and replication scripts.

PyTorch implementations of modern open-source LLM architectures (Llama, Qwen, DeepSeek, Gemma, GPT-OSS, Kimi, and more) — written from scratch for readability and learning, based on Sebastian Raschka's LLM Architecture Gallery.

From-scratch PyTorch: frontier LLM techniques as of 2026-Q1 — the Muon optimizer and Multi-Token Prediction, plus a base BPE tokenizer. Self-contained, self-checking modules.

Faithful from-scratch reimplementation of DeepSeek-V3 (MLA + MoE + MTP), scaled for Chinchilla-optimal 422M training on a single A100 80GB

六爻深度调度多令牌预测引擎 - 太极矩阵 M2 模块 | Six-Yao Depth Scheduled MTP Engine

Multi-Token Prediction benchmarks for Gemma 4 on Apple Silicon — LiteRT-LM, transformers, and llama.cpp at batch=1 on a MacBook M4 Pro. ~2× speedup reproducible in one specific runtime.

Benchmarks Multi-Token Prediction in llama.cpp against standard autoregressive baselines. Measures tokens per second, perplexity delta, and speculative acceptance rate across model sizes.

ChemMiniQ3-SAbRLo is a lightweight experimental generative model for chemistry, built on mini Qwen2-like arch, designed for rapid prototyping of HuggingFace AutoModel and AutoTokenizer compatibility, and fast iteration of Multi-Token Prediction (MTP) and RL fine-tuning algorithms/rewards.

A lightweight experimental generative model for chemistry, with mini Qwen2-like architecture and horizon loss and biologically-aware RL fine-tuning on SELFIES molecular representations.

Multi-Ensemble Memory-Elastic Token Prediction — a numpy implementation of DeepSeek's DSpark speculative decoding + the MEMTP elastic-ensemble extension, on AMD Strix Halo (gfx1151).

Research code for ProbeRoute, a probe-initialized sparse routing method for frozen-backbone multi-token prediction

Reverse-engineering how DeepSeek achieved frontier LLM performance at a fraction of the cost — through hands-on PyTorch implementations of MLA, MoE, MTP, RoPE, and quantization.

Arch Linux PKGBUILD for llama.cpp with ROCm/HIP + Gemma 4 MTP (PR #23398)