[lu-zero]

The Kolibri-1 CPU forward re-dequantises every used FP8 weight on every
forward unless the decoded weights stay resident, so the shipped
configuration paid that cost: the cache was OFF by default. The cache is
worth more than 2x on CPU decode (4.02 tok/s with it off against 9.07
tok/s with a 16 GiB budget, 2026-10-20, node 3, aarch64, 3.4 GHz, 32
threads, empty-prompt 32-token decode; prefill p512 43.33 tok/s with it
off), so the default is the cached path. The budget that default derives
from the host is the part this body corrects: the first form of it was
planned at the wrong moment and could land below the measured knee, so
the default shipped the slow configuration by accident.

The default budget of the Kolibri-1 CPU dequant cache was planned on the
first use of the process-wide cache, and the first use happens during the
74 GB checkpoint load: the page cache is full by then, so MemAvailable
reads 66 GiB where the host's idle reading is ~212 GiB. The divisor
yielded 8.25 GiB, and a budget that low buys nothing. Measured on node 3
(aarch64, 3.4 GHz, 32 threads, empty-prompt 32-token decode, 2026-10-21):
the default leg showed budget_mib=8454 from budget_source_mib=67638,
hits=9097, misses=18349, evictions=16068, a 33% hit rate and 3.84 tok/s,
against 6.38 tok/s for an explicit 16384 MiB budget and 3.62 tok/s with
the cache off.

The policy now reads the host at PROCESS START, before our own load
depresses the reading, and floors the budget at the measured knee:

honest_bytes = max(start_bytes, plan_bytes)
honest_bytes == 0              -> budget = 16 GiB   (UNKNOWN)
honest_bytes < 16 GiB + 8 GiB  -> budget = 0        (DISABLED)
otherwise                      -> budget = clamp(honest_bytes / 8,
                                                    16 GiB, 32 GiB)

The process-start reading is a namespace-scope initialiser, so it runs
before main and before any model load. The KNEE (16 GiB) is the FLOOR for
every host that can afford it: a budget under the knee thrashes, it costs
the memory and it buys nothing, so the policy never serves one. The
RESERVE (8 GiB) is the room the host must keep free beside the knee: a
host that reports less than the knee plus the reserve cannot afford the
knee at all, so the cache is DISABLED (budget 0) and the forward runs the
UNCACHED path, byte for byte. That is the explicit rule: a sub-knee
cache is WORSE than useless, so the cache either runs at or above the knee
or it does not run. The 32 GiB cap bounds the footprint, and an unreadable
report uses the KNEE: an unknown budget is not a reason to refuse (the
house rule) and not a reason to hand out a sub-knee budget; a host that
cannot hold the knee degrades to the uncached path through the existing
allocation fallback.

The env contract is unchanged: 0 disables, N sets exactly N MiB, and an
unparseable value falls back to the default policy rather than to
disabled. The uncached fallback is byte-for-byte unchanged: a cache that
cannot get its bytes degrades to the original threaded pool dequant.

Measured after the repair (2026-10-21, node 3, aarch64, 3.4 GHz, 32
threads, VT_KOLIBRI1_PROFILE=1, the file mutex held for the window). The
first window, empty-prompt 32-token decode: the DEFAULT leg planned
budget_mib=27157 from budget_start_mib=217262 and budget_plan_mib=143920,
ended at hits=72190 misses=17756 evictions=7993 (an 80% hit rate) and
delivered 7.44833, 7.39141 and 7.46827 tok/s over three runs. The
explicit 16384 MiB leg, same window and same binary, delivered 7.05777,
7.05438 and 6.98659 tok/s; the cache-off leg delivered 3.62428, 3.78799
and 3.54865 tok/s. The 72 GiB gap between the plan-time reading and the
process-start reading IS the checkpoint load, and the honest reading
closes it.

The second window, the production-shape gate
(/tmp/build-dq/tests/test_kolibri1_decode_bench, 128-token prompt, 64
greedy tokens, numactl --cpunodebind=3 --interleave=all,
VLLM_CPP_CPU_THREADS=32, the file mutex held):

leg budget_MiB hits misses evictions hit rate tok/s
default r1 23143 71717 18229 10072 79.7% 7.58
default r2 27107 72184 17762 8019 80.3% 7.36
default r3 26971 72169 17777 8089 80.2% 7.50
16384 MiB r1-3 16384 71097 18849 13396 79.0% 7.15 / 7.23 / 7.14
0 (off) r1-3 0 0 0 0 0.0% 3.49 / 3.58 / 2.45

The default leg is enabled, its budget is 22.6-26.5 GiB (at or above the
16 GiB knee), its hit rate is ~80% against the defect's 33%, and its
throughput (7.36-7.58 tok/s) is above the 6.38 tok/s the finding measured
for an explicit 16 GiB budget and above the same-binary explicit 16 GiB
legs (7.14-7.23 tok/s). The off leg confirms the uncached fallback:
enabled=0, no cache traffic, and the SAME token chain (11622...) as the
cached legs, so the fallback is byte-for-byte. The last off leg ran under
a load average of 53.9 (another build on the box) and is the low outlier.

Red-first: against the pre-repair policy the new cases fail at runtime,
not at compile time. The witness keeps the knee a literal, so it
compiles against the old policy too: DefaultBudgetBytesFor(66 GiB) was
8.25 GiB and the assertion failed (8.25 GiB against 16 GiB), and it
passes now. A probe compiled against the base commit's header (84d98a9)
reports budget_bytes=8858370048 (8.25 GiB) for the load-depressed 66 GiB
reading and exits 1; the repaired policy reports budget_bytes=17179869184
(16.00 GiB) and exits 0. The policy case pins the knee floor, the cap,
the divisor, the honest maximum of the two readings, the clean DISABLE of
a host that cannot afford the knee (a budget of 0, the uncached path), the
UNKNOWN rule (the knee, never a refusal and never a sub-knee budget), and
the band for every pair of the two moments. The process-level case pins
that the plan's source reading is that maximum, that the budget is the
policy applied to it, and that the process-start reading was really taken.

Mutations, each run and then restored byte for byte (the header was
checked against its sha256 after each one): an honest reading that takes
the MINIMUM of the two moments fails 4 assertions; a policy without the
knee floor fails 5 assertions across 3 cases, the helper included; a
process-start capture that never happens fails the capture assertion. On
the final policy: removing the KNEE floor fails 993 assertions across 2
cases; removing the clean-DISABLE rule fails 3 assertions; making the
UNKNOWN branch refuse instead of serving the knee fails 1 assertion. The
header's sha256 is the same before and after the three runs.

Gates: test_kolibri1_dequant_cache 13/13 (66150 assertions on the
final policy; 132200 on the sampling-repair commit), test_kolibri1 27/27,
test_kolibri1_w2 7/7, test_kolibri1_numa 3/3, test_kolibri1_dequant 3/3,
test_kolibri1_moe_glue 9/9, test_kolibri1_bfdot_gemv 5/5,
test_kolibri1_decode_bench SUCCESS. The W3 token gate and the n-gram spec
gate are red on pristine origin/main and reproduce there with the SAME
numbers as on this tree: W3 reads 134/138 compared positions, 4 near-tie
flips, 0 hard flips, worst topk diff 3.40201, worst sum diff 87462; the
n-gram spec gate keeps its stale anchor (11622 against the recorded
109726) and the same acceptance counts (162/297 accepted, 146/270 in the
probe harness, 150/297 on the torch chains).

WHAT THE REPAIRED DEFAULT DELIVERS (in-repo bench, node 3, aarch64, 3.4 GHz, 32 threads, quiet box, rebuilt binary; 3 runs per leg):

leg budget plain decode spec-decode
default (repaired) 26.6 GiB 8.63-9.36 12.74-13.14
explicit VT_..._CACHE_MB=16384 16 GiB 8.33-8.73 10.42-10.55
cache off (the shipped one) 0 3.90-3.92 6.27-6.30

The default leg's profile: budget_mib=27199 from a 212 GiB start-of-process reading, hits=72200, misses=17746, evictions=7967 - an 80 percent hit rate, against 33 percent and 16068 evictions for the pre-repair 8.25 GiB budget that made the default worthless. The repaired default is 2.2x the shipped configuration, and with speculative decode it passes the campaign's 12 tok/s target (12.74-13.14 against 6.27-6.30 shipped). The explicit 16 GiB leg is the same policy at a smaller budget: more budget, more hits, more throughput.

Docs: the row spec, docs/ENVIRONMENT.md and docs/FEATURES.md carry the
corrected policy, the measured knee, and the honest statement that a
sub-knee budget is WORSE than useless (it costs the memory and buys
nothing), that the sampling moment is part of the policy, and that a host
which cannot afford the knee gets the cache disabled rather than a
thrashing sub-knee cache.

References: ISSUE-LOCAL-01M5DVY9ZWF1XXAM2HSHH5BDRK

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:zai/glm-5.3-flash [maki]