lu-zero · Oct 21, 2026 · Oct 21, 2026 · Oct 21, 2026 · Oct 21, 2026 · Oct 21, 2026
diff --git a/...TEXT-kolibri-1-kolibri1-for-causal-lm/ISSUE-LOCAL-01M5E7ZJW0YXR2QZXQESJEP0MX.md b/...TEXT-kolibri-1-kolibri1-for-causal-lm/ISSUE-LOCAL-01M5E7ZJW0YXR2QZXQESJEP0MX.md
@@ -0,0 +1,19 @@
+ID: ISSUE-LOCAL-01M5E7ZJW0YXR2QZXQESJEP0MX
+Title: Record the measured Kolibri-1 CPU performance and re-baseline the public benchmark index
+Row: MODEL-TEXT-kolibri-1-kolibri1-for-causal-lm
+State: CLOSED
+Kind: gap
+GitHub: -
+Mirror: PENDING
+Availability: FULL
+Created: 2026-10-21
+Updated: 2026-10-21
+Closed: 2026-10-21
+
+## Problem
+
+The row's public performance record does not exist: docs/BENCHMARKS.md has no Kolibri-1 CPU entry, docs/benchmarks/ has no Kolibri-1 detail page, and the dequant-cache default flip (PR #3488) leaves its decode and prefill benchmark rows owed to a node-3 window. The 12 tok/s campaign target has no published disposition, and the row spec has no Outcome section, so the tree carries no quotable number for the shipped Kolibri-1 CPU configuration.
+
+## Resolution
+
+2026-10-21: the row's public performance record now exists. docs/benchmarks/kolibri1-cpu-dequant-cache-default.md (new public page, with the weights record: Aleph-Alpha/Kolibri-1 @ 5bb7075ba22db733e3834b1c71d2245537cc14ed, 32 shards, 74 GiB; the bf16 sibling named as refused) plus its index row in docs/BENCHMARKS.md and a Kolibri-1, CPU section carrying the decode, prefill, thread-scaling and spec-decode rows; the dated evidence in docs/bench-evidence/kolibri1-cpu-dequant-cache-default-20261021/report.md; the spec's ## Outcome section. The campaign's 12 tok/s target is recorded as MET by the shipped default (spec decode 12.74-13.14 tok/s against 6.27-6.30 with the dequant cache off), with the host-specificity, the 4% noise floor and the 16 GiB-leg trade-off recorded beside it.
diff --git a/.agents/specs/kolibri-1-cpu.md b/.agents/specs/kolibri-1-cpu.md
@@ -648,6 +648,96 @@ live-gated on the real model dir) into the R7 encode test:
   STOP: that would mean the scanner semantics differ, not just the
   recognition — report as NEEDS_DECISION rather than widening anything.
 
+## Outcome
+
+MEASURED (2026-10-20/21; node 3, aarch64, 128 cores, 255 GB RAM, governor
+`performance` at 3.4 GHz, 32 threads, one NUMA node, the file mutex held,
+the box quiet; the bench and the spec probe rebuilt from the committed
+head). The noise floor came from the repeated legs: two runs of the same
+cell (same binary, weights, configuration) differed by 3.5% in the t32
+cache-off leg (4.02 against 4.16) and by 4.7% in the decode table's
+cache-off leg (3.62 against 3.79), so no cell below is quoted more finely
+than that, and two legs that differ by less than 4% are a tie.
+
+| leg | decode tok/s (the in-repo bench: 128 tokens in, 64 greedy out) |
+|---|---|
+| cache OFF — the configuration `main` shipped before the default landed | 3.55 / 3.62 / 3.79 (median **3.62**) |
+| default, first implementation (8.25 GiB budget, 33% hit rate, 16068 evictions) | 3.84 (one run) |
+| **default, repaired** (26.5 GiB budget from a 212 GiB process-start reading; hits 72200, misses 17746, evictions 7967, 80% hit rate) | **7.39 / 7.45 / 7.47** |
+| explicit `VT_KOLIBRI1_DEQUANT_CACHE_MB=16384` | 6.99 / 7.05 / 7.06 |
+
+Prefill p512 with the cache off: t8 12.27-12.30, t16 23.78-23.89, t32
+43.02-43.33 tok/s. Decode thread scaling with the cache off, empty prompt:
+t8 1.38, t16 2.59, t32 4.02 tok/s, and the ratios that row derives are
+1.88x, then 1.55x, and 2.91x over the whole range. The n-gram spec-decode
+probe — the campaign's serving number — reads, cache OFF, plain 3.90-3.92
+and spec 6.27-6.30; with the **default** (repaired), plain 8.63-9.36 and
+spec **12.74-13.14**; with an explicit 16 GiB budget, plain 8.33-8.73 and
+spec 10.42-10.55.
+
+Every leg above is ONE measurement set. The evidence report names the file
+and the date of each one, and it lists the other sets the session took for
+the same cells instead of mixing them into the tables.
+
+WHAT THE DEFAULT NOW DELIVERS. The configuration that ships with the FP8
+dequant cache ON decodes at **7.45 tok/s** on the in-repo production shape
+(128 tokens in, 64 greedy out), **2.06x** the cache-off 3.62 and
+1.97x-2.10x the cache-off leg's own band (7.45 against 3.55-3.79), and its
+n-gram spec-decode leg reads **12.74-13.14 tok/s**, so the campaign's
+**12 tok/s target is MET**. The target is not met with the cache off
+(6.27-6.30), and the default's spec leg is **2.03x** the cache-off leg's
+(12.74 against 6.27): the number belongs to the default and to the default's
+host, not to the model in the abstract. The engine-level reading clears it
+as well: 4.53 -> 13.78 tok/s (3.04x, 2026-10-12, t32 with a 32 GiB cache,
+in `## Now` above). PR #3488 has **LANDED**: `d9e669582` (the flip),
+`e451be705` (the sampling-moment repair and the knee floor) and `3ce14f35c`
+(the knee refusal) are on `main`, so `main` now ships the **cache-ON**
+default and the row's numbers are the shipped configuration's numbers.
+
+REJECTED, each with the measurement that rejected it. The **sub-knee
+budget** (the default's first implementation: 8.25 GiB, 33% hit rate, 16068
+evictions) read 3.84 tok/s, which is +6.1% over the cache-off median
+(3.84/3.62) and +1.3% over the cache-off leg's best run (3.84/3.79), so
+under the 4% rule it is **not a tie** to the cache-off median. It is not a
+win either: the leg pays 8.25 GiB of memory for a 33% hit rate and leaves
+the 2x of the repaired default on the table, which is why the policy
+refuses a sub-knee budget outright and why the sampling moment is part of
+the policy rather than an implementation detail of it. The **explicit 16
+GiB budget** as the recommended setting read 6.99-7.06 tok/s in the
+published set against 7.39-7.47 for the derived default, a margin of 5.7%
+(7.45/7.05) that is above the noise floor and small for 10.5 GiB more cache,
+and 1.95x the cache-off median (7.05 against 3.62), so it is **not** claimed
+as a reason to prefer one over the other, and pinning 16 GiB by hand takes
+the host's own memory report out of a decision the report makes better.
+**Spin tuning** (`VT_CPU_SPIN_ROUNDS`) moved the same cell from 4.019 to
+4.193 tok/s at 32 threads (+4.3%) and from 2.587 to 2.581 at 16 (-0.2%), so
+the two paired readings do not agree in sign, the pair does not separate the
+arms, and no spin setting is recommended. One paired reading on one host is
+not a settled lever question.
+
+THE CAVEATS TRAVEL WITH EVERY NUMBER ABOVE. They are **host-specific**: 32
+threads on one NUMA node at 3.4 GHz, governor `performance`, aarch64, node 3
+of a shared box. They are **not** a vLLM comparison and not a parity claim:
+no vLLM or plugin denominator exists for this model, and the author's
+plugin is still unverified as a runnable oracle. They sit on a **~4% noise
+floor**, so the 12.74 against a target of 12 is a **+6.2% margin, about 1.5
+noise floors**: it clears the target as measured, and the margin is not
+wide. And the **16 GiB leg carries its own trade-off**: it is the shape the
+PR #3488 body quotes, and a reader who pins 16 GiB explicitly gets a number
+5.7% below the derived default for 10.5 GiB less cache. The derived default
+(26.5 GiB from a 212 GiB process-start reading) is the configuration this
+row's numbers were taken at, and the policy refuses a host that cannot
+afford the knee.
+
+THE RECORDS. The public page is
+`docs/benchmarks/kolibri1-cpu-dequant-cache-default.md`, with the index row
+in `docs/BENCHMARKS.md` and the dated evidence in
+`docs/bench-evidence/kolibri1-cpu-dequant-cache-default-20261021/report.md`.
+The public recipe still pins the short revision `e52eb462` for the
+checkpoint; the full revision of the artifact these numbers were measured
+on is recorded on the public page, and the bf16 sibling artifact is named
+there as **refused**, not shipped.
+
 ## Git integration
 
 ONE pull request (developer's standing choice for this row).

diff --git a/docs/BENCHMARKS.md b/docs/BENCHMARKS.md
@@ -12,6 +12,7 @@ to the file stem.
 | `vllm-online-serving` | The binding comparison. vLLM's production graphed config, never `--enforce-eager` | Results | [Detail](benchmarks/vllm-online-serving.md) |
 | `memory` | Peak memory across the serving window. Each row names its own model, host and window | Results | [Detail](benchmarks/memory.md) |
 | `llama-cpp-cpu` | llama.cpp on CPU, including the RPi5/A76 gating arm that does not transfer | Results | [Detail](benchmarks/llama-cpp-cpu.md) |
+| `kolibri1-cpu-dequant-cache-default` | Kolibri-1 on CPU with the dequant cache on by default: decode, prefill and the n-gram spec-decode serving number | Results | [Detail](benchmarks/kolibri1-cpu-dequant-cache-default.md) |
 | `mlx-lm-apple-m4` | MLX-LM on Apple M4, warm, batch 1, 6 interleaved runs | Results | [Detail](benchmarks/mlx-lm-apple-m4.md) |
 | `dwarfstar-gguf` | DwarfStar vs ds4 on DeepSeek-V4-Flash GGUF, GB10, byte-exact | Results | [Detail](benchmarks/dwarfstar-gguf.md) |
 | `speculative-decoding` | MTP / DFlash / n-gram / DSpark rows against vLLM | Results | [Detail](benchmarks/speculative-decoding.md) |
@@ -293,6 +294,63 @@ it goes our way. Output tokens are **byte-identical** to llama.cpp's greedy
 decode and to our own CPU reference path. Single-stream only: we have not
 measured concurrent serving against llama.cpp's server.
 
+## Kolibri-1, CPU
+
+Aleph Alpha's mixture-of-experts text model on the CPU path, FP8
+block-quantized checkpoint. Node 3 of the shared aarch64 box (128 cores,
+255 GB RAM, governor `performance` at 3.4 GHz), 32 threads, one NUMA node,
+the file mutex held, the box quiet. Detail:
+[Kolibri-1 on CPU](benchmarks/kolibri1-cpu-dequant-cache-default.md).
+
+| Leg | Decode tok/s (the in-repo bench: 128 tokens in, 64 greedy out) | Prefill tok/s (p512, cache off) |
+|---|---|---|
+| cache OFF — the configuration `main` shipped before the default landed | 3.55 / 3.62 / 3.79 (median **3.62**) | 12.27-12.30 / 23.78-23.89 / **43.02-43.33** tok/s at 8 / 16 / 32 threads |
+| default, first implementation (8.25 GiB budget, 33% hit rate) | 3.84 (one run) | — |
+| **default, repaired (26.5 GiB budget, 80% hit rate)** | **7.39 / 7.45 / 7.47 (median 7.45)** | — |
+| explicit 16 GiB budget | 6.99 / 7.05 / 7.06 (median 7.05) | — |
+
+The default is the FP8 dequant cache ON (PR #3488 — landed on `main` as
+`d9e669582`, `e451be705` and `3ce14f35c`; at the time of writing it was
+open, and `main` was at `891c4f27d`). **The ~2x of the default over the
+cache-off configuration (7.45 against 3.62, 2.06x) is what the published set
+shows;** the explicit 16 GiB leg is 1.95x the same baseline (7.05 against
+3.62), and the default is 1.97x-2.10x the cache-off leg's own band (7.45
+against 3.55-3.79). **Its margin over an explicit 16 GiB budget is 5.7% in
+the published set, a small margin for 10.5 GiB more cache, and is NOT claimed
+as a reason to prefer one over the other.** A sub-knee budget buys nothing:
+the default's first implementation (8.25 GiB, 33% hit rate) read 3.84 tok/s,
+which is +6.1% over the cache-off median and +1.3% over that leg's best run,
+so under the ~4% noise floor rule it is not a tie (the floor: 3.5% in the t32
+cache-off leg, 4.02 against 4.16, and 4.7% in the decode table's cache-off
+leg, 3.62 against 3.79), and it pays 8.25 GiB of memory for a 33% hit rate.
+The pool's spin budget is not a lever here
+(`VT_CPU_SPIN_ROUNDS=0` moved the same cell from 4.019 to 4.193 tok/s at 32
+threads, +4.3%, and from 2.587 to 2.581 at 16, -0.2% — the two paired
+readings do not agree in sign, so the pair does not separate the arms).
+Decode scales 1.38 / 2.59 / 4.02 tok/s at 8 / 16 / 32 threads, 1.88x, then
+1.55x, and 2.91x over the whole range, so the knee is inside the 32-thread
+node.
+
+The n-gram spec-decode probe, plain against spec decode, tok/s, three runs
+per leg:
+
+| Leg | Plain | Spec decode |
+|---|---|---|
+| cache OFF | 3.90-3.92 | 6.27-6.30 |
+| **default, repaired** | 8.63-9.36 | **12.74-13.14** |
+| explicit 16 GiB budget | 8.33-8.73 | 10.42-10.55 |
+
+**The campaign's 12 tok/s target is MET by the shipped default.** The
+n-gram spec-decode probe reads **12.74-13.14 tok/s** with the default
+(plain 8.63-9.36) against **6.27-6.30** with the cache off, and the
+default's spec leg is **2.03x** the cache-off leg's (12.74 against 6.27), so
+the number needs the dequant cache ON, a 32-thread node-3 host at 3.4 GHz,
+and it clears the target by 6.2%, about 1.5 times the ~4% noise floor. The
+engine-level reading clears it too: 4.53 -> 13.78 tok/s (3.04x, 2026-10-12,
+t32 with a 32 GiB cache). No vLLM or plugin denominator exists for this model, and no bf16 arm
+can be measured: the loader refuses a bf16 Kolibri-1 artifact by name
+(`src/vllm/model_executor/models/kolibri1_weights.cpp:72`).
+
 ## MLX-LM, Apple M4
 
 Qwen3-0.6B, warm, batch 1, 6 interleaved runs.
@@ -366,6 +424,7 @@ in the tree, default-OFF, for reproducibility; detail in the benchmark record.
 | MTP | Qwen3.6-27B NVFP4 | token-identical to vLLM MTP, **~4% faster at c1**; on-par at c2-c8 | `DONE` |
 | DFlash | Qwen3.6-27B NVFP4 | **2.9x over spec-off** (10.16 → 29.32 tok/s), at/above vLLM DFlash-on (**1.003x**, non-overlapping bands) | `DONE` |
 | n-gram | Qwen3.6-27B NVFP4 | draft-free (`SPEC-NGRAM`); 27B 5/5 STRICT our-ngram-ON == vLLM-ngram-ON, 180/180 drafts accepted (correctness only, no speed row yet) | `DONE` |
+| n-gram | Kolibri-1 (CPU) | draft-free (`SPEC-NGRAM`); the campaign's **12 tok/s target is MET** by the shipped default: spec decode **12.74-13.14 tok/s** (plain 8.63-9.36) against **6.27-6.30** with the dequant cache off, a **2.03x** ratio (12.74 against 6.27). Host-specific (node 3, 32 threads, 3.4 GHz), and the margin over the target is 6.2%, about 1.5 times the ~4% noise floor ([detail](benchmarks/kolibri1-cpu-dequant-cache-default.md)) | `DONE` |
 | DSpark | Qwen3.6-35B-A3B NVFP4 + RedHatAI dspark draft, k=8 | Token-identical to spec-off. Warm c1 **1.15x** at **20.8% acceptance vs upstream 7.7%**. nsys: verify runs EAGER (0 graph launches vs 1/step) and the MoE costs 1.7x more GPU per token at T=9 | `ACTIVE` |
 | Breadth (EAGLE1/3, suffix, ngram-gpu, dynamic-k, ...) | n/a | enumerated from vLLM source + `INVENTORIED` 2026-08-06 (`.agents/specs/spec-decode-inventory.md`), unmeasured | `INVENTORIED` |