[lu-zero]

record(MODEL-TEXT-kolibri-1-kolibri1-for-causal-lm): the measured Kolibri-1 CPU performance and the 12 tok/s verdict

The row had no public performance record. docs/BENCHMARKS.md carried no
Kolibri-1 CPU entry, docs/benchmarks/ had no detail page for it, the
dequant-cache default flip (PR #3488) left its decode and prefill rows owed
to a node-3 window, and the campaign's 12 tok/s target had no published
disposition. This change records the measurement taken on 2026-10-20/21 on
node 3 (aarch64, 128 cores, 255 GB RAM, governor performance at 3.4 GHz),
32 threads, one NUMA node, the file mutex held, the box quiet, and
re-baselines the public index against it. No product code and no test
changes.

PR #3488 has since landed on main as d9e669582 (the flip), e451be705
(the sampling-moment repair and the knee floor) and 3ce14f35c (the knee
refusal), so main now ships the cache-ON default and the numbers below are
the shipped configuration's numbers.

The decode bench is the in-repo production-shape bench, 128 tokens in and 64
greedy out, three runs per leg, tok/s:

leg	runs	median
cache OFF, what main shipped before the default landed	3.55 / 3.62 / 3.79	3.62
default, first implementation (8.25 GiB, 33% hit rate)	3.84 (one run)	3.84
default, repaired (26.5 GiB, 80% hit rate)	7.39 / 7.45 / 7.47	7.45
explicit 16 GiB budget	6.99 / 7.05 / 7.06	7.05

Every cell is one measurement set, named with its file and its date in the
evidence report: the session's bench_repair.log (2026-10-21 02:03-02:09),
the binary rebuilt from the committed head d47173c51 of PR #3488 - the
head the measurement was taken on - plus the single run the sub-knee leg's
own policy commit quotes. That PR has since landed on main as
d9e669582, e451be705 and 3ce14f35c, so the default this table
measures is the configuration main now ships.

The n-gram spec-decode probe, plain against spec decode, tok/s, three runs
per leg:

leg	plain	spec decode
cache OFF	3.90-3.92	6.27-6.30
default, repaired	8.63-9.36	12.74-13.14
explicit 16 GiB budget	8.33-8.73	10.42-10.55

Prefill p512 with the cache off: 12.27-12.30 / 23.78-23.89 / 43.02-43.33
tok/s at 8 / 16 / 32 threads. Decode thread scaling with the cache off, empty
prompt: 1.38 / 2.59 / 4.02 tok/s at 8 / 16 / 32 threads, and the ratios
that row derives are 1.88x, then 1.55x, and 2.91x over the whole range.

THE 12 tok/s TARGET IS MET by the shipped default. The n-gram spec-decode
probe reads 12.74-13.14 tok/s with the dequant cache on (plain 8.63-9.36)
against 6.27-6.30 with the cache off, and the default's spec leg is 2.03x
the cache-off leg's (12.74 against 6.27), so the number needs the cache ON.
The engine-level reading clears it as well: 4.53 -> 13.78 tok/s, 3.04x,
2026-10-12, t32 with a 32 GiB cache.

The caveats travel with the number. The box's noise floor is ~4% (two runs
of the same cell differed by 3.5% and by 4.7%), so 12.74 against a target of
12 is a +6.2% margin, about 1.5 noise floors. The numbers are host-specific:
32 threads on one NUMA node at 3.4 GHz. The default's margin over an
explicit 16 GiB budget is 5.7% in the published set, a small margin for
10.5 GiB more cache, and is NOT claimed as a reason to prefer one over the
other; what that set shows is the ~2x of the default over the cache-off
configuration (7.45 against 3.62, 2.06x, and 1.97x-2.10x the cache-off leg's
own band of 3.55-3.79), with the explicit 16 GiB leg at 1.95x the same
baseline (7.05 against 3.62). The rejected alternatives carry
their own measurements: the sub-knee budget (8.25 GiB, 33% hit rate) read
3.84 tok/s, +6.1% over the cache-off median and +1.3% over its best run, so
it is not a tie under the 4% rule and it still buys nothing; and the spin
budget is a wash (t32 4.019 against 4.193, +4.3%, and t16 2.587 against
2.581, -0.2%, with VT_CPU_SPIN_ROUNDS=0).

The weights are recorded on the public page: Aleph-Alpha/Kolibri-1 at
revision 5bb7075ba22db733e3834b1c71d2245537cc14ed, 32 shards,
78,827,029,120 bytes of tensors (74 GiB on disk), FP8 block-quantized. No
shard SHA-256 is recorded in this tree and the page says so. The sibling
Aleph-Alpha/Kolibri-1-BF16 at revision
cc6d729e6a024c1e59ad48887fdef95ff37b534d (149 GiB) is named there as
REFUSED, not shipped: the loader rejects a checkpoint without the FP8
quantization_config with "a bf16 Kolibri-1 artifact is not an implemented
arm" (src/vllm/model_executor/models/kolibri1_weights.cpp:72), so no bf16
measurement exists and none is claimed.

The records: the new public page
docs/benchmarks/kolibri1-cpu-dequant-cache-default.md with its index row in
docs/BENCHMARKS.md, the Kolibri-1, CPU section and the speculative-decoding
ledger row there, a reproduce.md entry, the dated evidence
docs/bench-evidence/kolibri1-cpu-dequant-cache-default-20261021/report.md,
and the row spec's ## Outcome section.

A FRESH REVIEW OF THAT RECORD FOUND THREE DEFECTS, AND THEY ARE REPAIRED
HERE. The sub-knee leg's 3.84 tok/s was published as "a 1.3% difference from
the cache-off median" and as "a tie": 3.84/3.62 = 1.061 is +6.1% over the
median, and the 1.3% is the distance to the best cache-off run, 3.84/3.79 =
1.013. Under the page's own rule a leg under 4% is a tie, so 1.06x is not
one, and the label is gone. The thread-scaling ratios were published as
1.89x and 1.36x where the row derives 1.88x and 1.55x. And the record mixed
two measurement windows into single legs and then claimed that the repair
commit "agrees cell by cell"; it does not, because the explicit 16 GiB leg
reads 6.99-7.06 in the published set against 7.05777 / 7.05438 / 6.98659 in
the repair commit's own set. Every leg now names one set with its file and
its date, the evidence report lists the other sets instead of mixing them,
and it lists every derived number with its arithmetic.

The audit behind that repair found five more of the same kind. The decode
table was labelled "empty prompt, 32 generated tokens" while its cells come
from the in-repo production shape. The explicit 16 GiB leg published 6.38 /
6.95 / 6.99, and no set produced that triple. The spin pair at 16 threads
was published with the wrong sign. The prefill row published the best of two
runs per thread count. The default's margin over the explicit 16 GiB budget
was published as "inside that leg's own spread", which held only for the
mixed set. The noise floor's example pair was in no retained log, so the
floor is now the record's own repeated legs. None of the repairs touches the
12 tok/s verdict.

A FRESH RE-REVIEW FOUND TWO CROSS-DOCUMENT GAPS, AND THEY ARE REPAIRED
HERE. The re-review re-derived every published number from the source logs
and confirmed all of them, then found that the 8 to 32 thread-scaling ratio
(2.91x) was in the benchmark detail page and in the row spec's ## Outcome but
not in the public index, and that the spec-decode ratio (2.03x, 12.74 against
the cache-off 6.27) was in the dated evidence report only. Neither gap
contradicted a published cell, but a reader comparing the index with the
detail page saw two different derived sets. The four records now carry one
derived set: the index holds the whole thread-scaling triple (1.88x, then
1.55x, and 2.91x), the spec-decode ratio, the explicit 16 GiB leg's 1.95x,
the 1.97x-2.10x band, the noise floor's two examples, the prefill row's
whole thread range, the spin pair's cells and the spec-decode legs in a
table of their own; the detail page holds the spec-decode ratio and the
engine-level reading; the row spec's ## Outcome holds the spec-decode
ratio, the 1.95x and the band. No measured cell and no derived value
changed, and every number in the four records still re-derives from the
cells the records publish.

THIS BRANCH WAS REBASED ONTO THE MAIN THAT CARRIES THE LANDED DEFAULT
(3ce14f3), and the four records were updated to the state they now
describe: the spec's ## Outcome, the public page, the index row and the
dated evidence report all say that PR #3488 has landed, name its landing
revisions d9e6695, e451be7 and 3ce14f3, and present the numbers as the
shipped configuration's numbers. No measured cell and no derived value
changed: the update is text only, and every published number still
re-derives from the cells the records publish.

Two things this record does not say, on purpose. The default is PR #3488,
which was OPEN when this was written and has since LANDED on main as
d9e669582, e451be705 and 3ce14f35c, so the pages now say that the
cache-ON configuration is what main ships and that the cache-off leg is
the baseline main shipped before the default landed. And no vLLM or plugin
denominator exists for this model, so nothing here is a parity claim. The public recipe still pins the short revision e52eb462
for the checkpoint; the full revision of the measured artifact is on the
benchmark page, and the difference is reported rather than papered over.

References: ISSUE-LOCAL-01M5E7ZJW0YXR2QZXQESJEP0MX

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:zai/glm-5.3-flash [maki]