Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -170,3 +170,36 @@ On the P150 (host thalia, /tmp/umdtrial-install, /tmp/tt-metal-umdtrial), the ko
~214 s vs ~181 s. A generation-counter scheme could cut the destruction
count ~30x — follow-up if the decode regression matters.


- BAND-GATE RATIFICATION + RESCORE (row/TT-B2II-BAND-GATE, 2026-10-19):
the developer RATIFIED the band gate as the P150 correctness standard
for this device: a sequence is ON iff it has ZERO hard flips —
divergences inside the established 2.5-nat near-tie band are not gate
failures; the exact-token ON/OFF counting is no longer the bar; the
primary oracle rows (the W3 141/145 CPU baseline) remain the
denominator. The scorer now prints the band-gate verdict line
(`BAND GATE (ratified 2026-10-19): X/Y sequences ON`, with the worst
in-band divergence against the band edge) beside the unchanged
exact-token reporting — a reporting change, not a tolerance change;
the classifier (the per-position nat gap vs the golden chain,
`test_kolibri1_tt_b2ii.cpp:1222` `-gap <= kNatBand`, band 2.5) is
unchanged.
THREE-RUN VERDICT on origin/main (61d333e52, P150 non-fleet, the
device recipe; byte-identical flip sets across all three runs per
arm, no classifier defect found — no flip beyond the band is recorded
as a near-tie on either arm):
- HOST-FREE ON (VT_TT_HOST_FREE_DECODE default): BAND GATE 3/8
sequences ON, worst in-band divergence -1.747 nats (margin 0.753
nats to the 2.5-nat edge); exact-token unchanged at 3/11 (3
near-ties, 5 hard: Australia, Wissen, x1=, Ein gutes, counting).
- HOST-FREE OFF (VT_TT_HOST_FREE_DECODE=0): BAND GATE 6/8 sequences
ON, worst in-band divergence -2.157 nats (margin 0.343 nats);
exact-token unchanged at 12/20 (6 near-ties, 2 hard: der Mond step
0, counting step 0).
Under the ratified standard the gate verdict is STABLE and REPORTABLE
(3/8 ON, 6/8 ON): it still fails its "all sequences ON" ideal, so the
gate remains open on the hard-flip sequences, but the ON/OFF state is
now a measured, dated, stable quantity instead of an exact-token
coincidence count. The three pre-existing assertion failures remain,
documented: `:1263` (argmax_matches >= 141), `:1264`
(hard_flips == 0), `:1419` (instr_hard_flips == 0).
15 changes: 15 additions & 0 deletions .agents/specs/kolibri-tt.md
Original file line number Diff line number Diff line change
Expand Up @@ -506,6 +506,21 @@ ReshapeAndCache / device residual norm), the `kGdnDecode` wide-range
drift and the `kMatmulBTQuantGrouped` Q4_K decode envelope miss, the
141/145 pass, and the bench anchor 109726.

BAND-GATE STANDARD RATIFIED + B2b-ii TOKEN GATE RESCORED (2026-10-19,
`row/TT-B2II-BAND-GATE`): the developer ratified the band gate as the
P150 correctness standard for the B2b-ii device token gate — a sequence
is ON iff it has ZERO hard flips (divergences beyond the 2.5-nat
near-tie band); exact-token ON/OFF counting is no longer the bar, and
the primary oracle rows stay the denominator. The scorer prints the
band-gate verdict beside the exact-token counts (reporting only; the
classifier is unchanged). Three runs on 61d333e52, byte-identical flip
sets per arm: host-free ON — BAND GATE 3/8 sequences ON (worst in-band
-1.747 nats, margin 0.753), exact-token 3/11; host-free OFF — 6/8 ON
(worst in-band -2.157 nats, margin 0.343), exact-token 12/20. Evidence:
issue ISSUE-LOCAL-01M4P21TZ1Q9CA7P66XBH0BQTX (2026-10-19 entry). The
141/145 pass and the hard-flip sequences stay owed; the pre-existing
`:1263`/`:1264`/`:1419` assertion failures remain, documented.


## Git integration

Expand Down
15 changes: 15 additions & 0 deletions tests/vllm/models/test_kolibri1_tt_b2ii.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1153,6 +1153,12 @@ TEST_CASE("kolibri1 TT B2b-ii: device token gate — the golden argmax chains "

int64_t argmax_matches = 0, argmax_total = 0, near_ties = 0, hard_flips = 0;
std::vector<std::string> flip_records;
// The RATIFIED BAND GATE (developer ratification, 2026-10-19): a sequence
// is ON iff it has ZERO hard flips — divergences INSIDE the kNatBand band
// are not gate failures. Reported alongside the exact-token count; the
// exact-token CHECKs below stay as the documented pre-existing red.
int64_t band_on = 0, band_total = 0;
double worst_inband_gap = 0.0;

for (const GateGolden& gp : LoadGateGoldens()) {
// Fresh KV per prompt (zeroed: no stale context can leak across
Expand All @@ -1176,6 +1182,7 @@ TEST_CASE("kolibri1 TT B2b-ii: device token gate — the golden argmax chains "
std::vector<float> logits;
int32_t prev_got = -1;
bool diverged = false;
bool seq_hard_flip = false;
for (int32_t step = 0; step < kGateSteps && !diverged; ++step) {
// The prompt walk first: one token per step, no comparison.
if (static_cast<size_t>(step) < gp.input_ids.size()) {
Expand Down Expand Up @@ -1221,8 +1228,10 @@ TEST_CASE("kolibri1 TT B2b-ii: device token gate — the golden argmax chains "
std::to_string(gap));
if (-gap <= kNatBand) {
++near_ties;
worst_inband_gap = std::min(worst_inband_gap, gap);
} else {
++hard_flips;
seq_hard_flip = true;
MESSAGE("HARD flip at '" << gp.prompt << "' step " << d
<< ": got " << got << " want " << want
<< " (gap " << gap << " beyond band "
Expand All @@ -1232,8 +1241,14 @@ TEST_CASE("kolibri1 TT B2b-ii: device token gate — the golden argmax chains "
break;
}
}
++band_total;
if (!seq_hard_flip) ++band_on;
}

MESSAGE("BAND GATE (ratified 2026-10-19): " << band_on << "/" << band_total
<< " sequences ON (zero hard flips); worst in-band divergence "
<< worst_inband_gap << " nats against the " << kNatBand
<< "-nat band edge");
MESSAGE("ARGMAX CHAIN: " << argmax_matches << "/" << argmax_total
<< " compared positions match the golden greedy "
"decode; flips "
Expand Down
Loading