escha-port — a QTIP-style 2-bit format in llama.cpp

A working port of the ESCHAM weight format (Escha Labs) into llama.cpp: GGUF conversion, a CPU reference decoder, CUDA and HIP kernels, and the evaluation harness used to prove the result.

Reference model: Qwen3.8-27B, 2.469 bits per weight.

The finding worth the write-up: the tensor cores did not win on arithmetic. Swapping FFMA for mma in isolation measures 1.05×. The ×3 on prefill came from register pressure — the smaller f16 accumulator freed the registers that made band blocking possible, and band blocking is what stopped the kernel re-reading x. The instruction was the enabler, not the cause. A second finding cost more to learn: the decode lookup table, the obvious optimisation, was generating 13× more L2 traffic than the weights it was meant to save.

The honest bottom line is below — against a real Q4_K_XL this format wins on size and loses on everything else.

Results

Q4_K_XL is the reference point, not a strawman: it is what this machine actually runs in production.

	escha	Q4_K_XL
weights on disk	9.66 GiB	16.35 GiB
perplexity (wikitext-2-raw, 580 chunks)	7.3679 ± 0.047	6.9744 ± 0.045
generation	34.19 t/s	41.40 t/s
prompt processing (pp512)	610.20 t/s	1382.86 t/s

Every row is apples-to-apples, but the two blocks come from different cards. The perplexity row is a five-arm campaign on the gfx1151 APU — one binary, one card, the same 580 chunks for every arm. The speed rows are llama-bench on one RTX 3090, same boot, same binary. The CUDA build's own quality gate returns 7.3695 ± 0.047 for escha, three decimals from the APU figure, which is why reading the two blocks as one picture is safe.

Read this honestly: escha wins on size, by 1.7×. It loses on quality and on both speeds. The format is interesting when memory is the binding constraint — it leaves roughly twice the room for KV cache on a 24 GB card — and not otherwise.

Against the format's own reference implementation, this port is more accurate. On an identical protocol, Escha Labs' SGLang runtime scores 7.5106 ± 0.122 where the native GGUF here scores 7.3679 — 1.9 % better, not worse. (A separate q8_0 "oracle" build, used only as a bench to validate the reconstruction chain, scores 7.3693; it is 26.65 GiB and is not a deliverable.) Reconstruction fidelity is the thing most likely to go wrong silently in a port like this, so that number is the one that matters most.

Kernel work

The prefill kernel went through eight stages, from a naive one-tile-per-warp version to a tensor-core kernel with band blocking:

stage	pp512	tg
one tile per warp	14.69	3.56
inline codec, lookup table removed	72.90	26.39
half2 decode	110.86	34.75
batched prefill, shared x	200.29	34.78
tensor cores + band blocking	610.20	34.19

×33 on prefill overall, with generation quality unchanged — the same binary on the same boot scores 7.3695 on both code paths.

Two findings worth more than the throughput number:

• The lookup table was the real trap. It generated 13× more L2 traffic than the weights themselves. Deleting it and computing the codec inline was the single largest win.
• Tensor cores did not win on arithmetic. Swapping FFMA for mma in isolation measures 1.05×. They win because the smaller accumulator frees the registers that make band blocking possible, which is what removes the redundant re-reads of x. The two effects do not separate.

Correctness

• Logits vs a q8_0 oracle: Pearson 0.999873, argmax identical, top-10 10/10.
• crosscheck.py: 855 comparable tensors against the q8_0 oracle, tolerance 0.02, 0 mismatches (the model holds 1251 tensors; the rest are escha-coded and have no direct counterpart).
• test-escha-nj: the error against the reference path jumps exactly at the dispatch boundary and stays flat above it — the signature of an f16 rounding difference, not a layout bug.

Layout

harness/   conversion, oracle comparison, perplexity gate
docs/      WRITEUP.md — full technical write-up (English)
           appendix/ — raw measurement logs and perplexity protocol (French)
kernel/    standalone tensor-core micro-benchmark

Paths are configured by environment variable: ESCHA_CKPT, ESCHA_GGUF, ESCHA_ORACLE, ESCHA_PPL_TEXT, GGUF_PY.

The code

The kernels and the loader live on a llama.cpp branch, 28 commits against upstream master:

YanissAmz/llama.cpp · feature/escha-format

That branch carries ggml/src/ggml-cuda/escham.cu (the CUDA and HIP kernels), the GGUF loader changes, and the test-escha-* suite. This repository holds the write-up and the evaluation harness.

Status and limits

• The GGML_TYPE_ESCHAM_2 / ESCHAM_3 ids (43, 44) are not reserved. They are the next free slots after upstream's GGML_TYPE_COUNT = 43. If upstream claims 43, files collide — but both directions fail loudly (missing tensor, or an out-of-range type), never silently.
• The tensor-core path is CUDA only. gfx1151 (RDNA3.5) keeps the previous kernel; its WMMA fragment layout differs and was not ported.
• The 8-column dispatch threshold is tuned on a 3090. GGML_ESCHA_MMA_MIN re-measures it without recompiling; no other card was tested.
• No fast CPU path. The CPU decoder exists for tests, not for serving.
• A batch of 33 tokens drops to 336 t/s against 487 at 32: the block width is fixed at 32, so 33 columns pad to 64. Known, not fixed.

The detailed write-up in docs/WRITEUP.md records the measurements that failed and the reasoning that turned out wrong, alongside the ones that worked.

License

MIT, matching upstream llama.cpp. harness/logit_dump.cpp, kernel/mma_spike.cu and harness/gguf_escha.py derive from llama.cpp / gguf-py; the LICENSE file is upstream's, kept verbatim.