Repository navigation
kolibri1: cross-check the CPU gates against llama.cpp 62318cd57 - #3485
Merged
lu-zero merged 1 commit intoOct 20, 2026
Merged
Conversation
Add upstream llama.cpp's freshly-merged Kolibri-1 support (#30673, merge 62318cd57, 2026-10-17) as a third independent reference for the row's correctness record, and record the three-way agreement measurement. No product code changes: this is the oracle pin, its gateability evidence, and one local issue. The pin advances from 11fe0215 to 62318cd57 because that revision is the first upstream object that loads the released Aleph-Alpha/Kolibri-1 checkpoint at all. The gateability run built CPU-only from a clean tree at the exact commit (GGML_NATIVE=ON, aarch64 FEAT_BF16, the same hardware class as our production bf16-dot default), converted the fp8 checkpoint through the upstream converter to a bf16 GGUF (903 tensors, 149069.2 MiB, sha256 76ddb05e458cce1a023d42cd4f13dc8e8152fbb6501c1f680918fc7d2a0fcefd), and generated coherently end-to-end. The measurement: over the primary oracle's 202 aligned teacher-forced positions, llama.cpp top-1 is 198/202 = 98.0% — identical to our committed production bf16-dot number. Chain decode vs the golden chains is 126/256 = 49.2% with 3 of 8 prompts token-exact, the near-tie re-rank shape our own chains show; the aligned numbers are the numerics statistic, the chain number is not a quality verdict. Nothing was tuned on either side. Closes .agents/issues/MODEL-TEXT-kolibri-1-kolibri1-for-causal-lm/ ISSUE-LOCAL-01M5CRZ3J4CFX1KEJD6ZRE4A8R. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:zai/glm-5.3-flash [maki]
lu-zero
force-pushed
the
row/KOLIBRI-LLAMACPP-CROSSCHECK
branch
from
October 20, 2026 14:30
74ead29 to
6fee3fc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
kolibri1: cross-check the CPU gates against llama.cpp 62318cd57
Add upstream llama.cpp's freshly-merged Kolibri-1 support (ggml-org/llama.cpp
#30673, merge
62318cd57, 2026-10-17) as a third independent reference forthe MODEL-TEXT-kolibri-1 correctness record, and measure three-way agreement.
No product code changes: the oracle pin, its gateability evidence, and one
local issue.
Pin.
.agents/oracles/llama-cpp.mdadvances from11fe0215to62318cd57— the first upstream object that loads the releasedAleph-Alpha/Kolibri-1 checkpoint at all. Gates recorded against
11fe0215stay attributed to it; non-kolibri denominators are not re-based.
Gateability. Built CPU-only from a clean tree at the exact commit
(
GGML_NATIVE=ON, aarch64 FEAT_BF16 — the same hardware class as ourproduction bf16-dot default), converted the fp8 checkpoint through the
upstream converter (
convert_hf_to_gguf.py, defaultauto): 903 tensors,general.architecture = kolibri1,MOSTLY_BF16, 149,069.2 MiB, sha25676ddb05e458cce1a023d42cd4f13dc8e8152fbb6501c1f680918fc7d2a0fcefd. Servedand generated coherently end-to-end. Executed path:
llama-server -t 16,all other defaults.
Three-way agreement (primary = aleph-alpha-inference top-100 rows, 202
aligned teacher-forced positions over the 8-prompt battery):
62318cd57bf16 GGUFChain decode llama.cpp vs the golden chains: 126/256 = 49.2%, 3 of 8 prompts
token-exact through 32 steps, first divergences at steps 0/1/0/5/7 — the
near-tie re-rank shape our own chains show. The aligned numbers are the
numerics statistic; nothing was tuned on either side.
Gates.
check-agent-recordok,agent-preflight.sh --stagedok,commit style ok, trailer contract ok.
agent-ready.pyreports onepre-existing failure on this host:
check-windows-portability.pyfailsidentically on a clean
origin/maincheckout (the ps1 fake-tool contractcannot find
dotnet--NoProfile), which this change does not touch.Evidence:
docs/bench-evidence/kolibri1-llamacpp-crosscheck-20261020.md.docs/USAGE.mdowes no line: no command, config key, or workflow changed.This change was produced under the AGENTS.md protocol; a fresh reviewer
still owes the mutation pass on the head.
CAVEAT (added post-review): the 98.0% figures were measured against the primary top-100 rows as they exist on main, which carry a confirmed off-by-one (writer stored each row one position behind the generated chain; repair in flight on row/PRIMARY-ROWS-OFFBYONE). Both sides used the same lagged alignment, so the equality of the two percentages — two independent implementations agreeing exactly — is the meaningful statistic; the absolute 98.0% label is provisional until the repaired rows land.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:zai/glm-5.3-flash [maki]