security: reject malformed model tensors at the untrusted-mirror boundary (C half of #368) - #413
Merged
Conversation
…dary Colibri loads model directories and safetensors from mirrors it does not control, so the file's declared shapes and byte spans are attacker-influenced input. Three memory-safety holes on that boundary, independently confirmed (incl. a from-scratch adversarial audit that re-derived the same two) and present in the shipped v1.0.0: - st.h st_read_f32: numel came from the shape, nbytes from the offsets, with no cross-check. A crafted tensor whose shape inflates numel past nbytes made the BF16/F16 loop read past the malloc'd raw buffer and the F32 memcpy write past the caller's config-sized destination (heap OOB read + write). Now enforce numel*esz == nbytes before any copy. - st.h header parse: the shape product could overflow int64 to a small/negative numel that would then pass the cross-check. Guard each multiply. - glm.c qt_resolve_fmt (new, replaces the three duplicated "?1:?2:3" fmt sites in qt_from_disk and both expert_load arms): the old inference SILENTLY fell to int2 for any unrecognized weight byte count, so a too-short weight became a valid int2 whose matmul read O*I nibbles past the buffer; and an oversized scale array overflowed the per-row t->s. Now the weight bytes must match a known int8/int4/int2 layout and the scale array must match the expected per-row (O) or grouped (O*ng) cardinality, else refuse. - glm.c config/generation_config slurp: unbounded ftell -> malloc(n+1) gave a hostile file a load-time OOM, and on malloc failure b[got]=0 was a NULL deref. Cap at 256 MB and NULL-check. Verified: TF token-exactness unchanged on every quant format (full-precision 32/32, int4 11/32, int2 1/32, mix 5/32 -- byte-identical to the pre-change binary); fmt=4 grouped path preserved (the scale check is by construction the same condition detect_group_size already imposed); a hand-crafted hostile safetensors is refused cleanly; ASan+UBSan clean on legit and hostile loads (only the pre-existing intentional startup leaks remain). These are the C trust-boundary items of #368, landed as a minimal standalone fix; the server-side and build items of that PR follow via its rebase. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 19, 2026
fabio-rovai
added a commit
to fabio-rovai/colibri
that referenced
this pull request
Jul 20, 2026
…rter, tests New weight format: int3 with ONE f32 scale per 64-input group (3.5 bits/weight effective). Per group: 16B low plane (2 bits/val, int2 layout) + 8B high plane (1 bit/val), values [-4,3] stored v+4. Same quantization math as tools/quant_ablation.py _quant_last_dim(bits=3, group=64) from JustVugg#132, whose OLMoE ablation measured int3-g64 BEATING the shipped per-row int4 on quality (-7.5 vs -9.3pp) at ~14% fewer bits. Engine (placed per the JustVugg#391 split): matmul_i3 + pack_int3_g64 + I3_* layout helpers in quant.h next to their kernel family; fmt=5 branches in colibri.c's qt_bytes/qt_alloc/qt_fill/matmul_qt/embed_row/qt_addrow/qt_matvec_rows. Format detection now goes through JustVugg#413's qt_resolve_fmt: fmt=5 registers its distinct weight-byte layout O*ceil(I/64)*24 and its scale cardinality O*ceil(I/64) there, validated against [O,I] like every other format. int3-g64 and grouped-int4-at-gs=64 carry the SAME scale count, so the weight bytes are the int3 tag; row formats keep precedence for the small-I shapes where byte counts coincide. The io_uring expert path still used the raw ?1:?2:3 byte inference (it missed fmt=4 grouping entirely and never set gs) — converted to qt_resolve_fmt like the other two expert paths. Backends: qt_cuda_upload returns 0 for fmt=5 (tensor stays CPU-side, the documented fallback), the dense CUDA matmul gate excludes fmt=5, and Metal's existing fmt gates (gemm fmt<=3, moe fmt 1/2) already reject it. Converter: quant_int3_g64 in convert_fp8_to_int4.py; --ebits 3/--xbits 3 now emit it (previously bits=3 silently produced int4). Tests: tests/test_int3.c (bit-exact pack/unpack vs reference, matmul_i3 vs dequant-matmul incl. short tail groups and the real GLM I=7168, QT plumbing, qt_resolve_fmt disambiguation incl. the same-scale-count fmt=4/fmt=5 pair, outlier-rows RMS: int3-g64 3.3x lower error than per-row int4), tests/test_int3_load.c (hand-rolled .safetensors fixture through st_init + qt_from_disk: fmt=5 detected and loaded next to an int4 control tensor), tests/test_int3_convert.py (NumPy pack round-trip vs independent decoder). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fabio-rovai
added a commit
to fabio-rovai/colibri
that referenced
this pull request
Jul 20, 2026
…rter, tests New weight format: int3 with ONE f32 scale per 64-input group (3.5 bits/weight effective). Per group: 16B low plane (2 bits/val, int2 layout) + 8B high plane (1 bit/val), values [-4,3] stored v+4. Same quantization math as tools/quant_ablation.py _quant_last_dim(bits=3, group=64) from JustVugg#132, whose OLMoE ablation measured int3-g64 BEATING the shipped per-row int4 on quality (-7.5 vs -9.3pp) at ~14% fewer bits. Engine (placed per the JustVugg#391 split): matmul_i3 + pack_int3_g64 + I3_* layout helpers in quant.h next to their kernel family; fmt=5 branches in colibri.c's qt_bytes/qt_alloc/qt_fill/matmul_qt/embed_row/qt_addrow/qt_matvec_rows. Format detection now goes through JustVugg#413's qt_resolve_fmt: fmt=5 registers its distinct weight-byte layout O*ceil(I/64)*24 and its scale cardinality O*ceil(I/64) there, validated against [O,I] like every other format. int3-g64 and grouped-int4-at-gs=64 carry the SAME scale count, so the weight bytes are the int3 tag; row formats keep precedence for the small-I shapes where byte counts coincide. The io_uring expert path still used the raw ?1:?2:3 byte inference (it missed fmt=4 grouping entirely and never set gs) — converted to qt_resolve_fmt like the other two expert paths. Backends: qt_cuda_upload returns 0 for fmt=5 (tensor stays CPU-side, the documented fallback), the dense CUDA matmul gate excludes fmt=5, and Metal's existing fmt gates (gemm fmt<=3, moe fmt 1/2) already reject it. Converter: quant_int3_g64 in convert_fp8_to_int4.py; --ebits 3/--xbits 3 now emit it (previously bits=3 silently produced int4). Tests: tests/test_int3.c (bit-exact pack/unpack vs reference, matmul_i3 vs dequant-matmul incl. short tail groups and the real GLM I=7168, QT plumbing, qt_resolve_fmt disambiguation incl. the same-scale-count fmt=4/fmt=5 pair, outlier-rows RMS: int3-g64 3.3x lower error than per-row int4), tests/test_int3_load.c (hand-rolled .safetensors fixture through st_init + qt_from_disk: fmt=5 detected and loaded next to an int4 control tensor), tests/test_int3_convert.py (NumPy pack round-trip vs independent decoder). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
JustVugg
added a commit
that referenced
this pull request
Jul 20, 2026
int3-g64 (fmt=5) container: 3-bit weights with per-group (gs=64) scales. Measured 3.34x lower outlier-row RMS than int4-row at ~25% fewer bytes (#132 ablation point). Rebased by @fabio-rovai onto post-#426 dev; fmt=5 registered in qt_resolve_fmt (#413 security gate) with fmt-4/fmt-5 disambiguation tested at real GLM I=7168; io_uring expert path converted to qt_resolve_fmt (fixes latent fmt-4 mis-tag); CUDA graceful CPU-fallback. Verified locally: no regression on existing formats, test_int3 + CI green.
JustVugg
added a commit
to KingIcyCreamProjects/colibri
that referenced
this pull request
Jul 21, 2026
… conflicts + fix fmt=5 scale cap Conflict resolution (13 hunks, 6 files): - colibri.c / st.h: dev already carries an equal-or-stronger guard for the same threats (qt_resolve_fmt resolves AND validates the quant layout, refusing unknown byte counts instead of falling through to int2; the shape-product overflow guard is present). Took dev's side — no check is lost. - olmoe.c: the load_expert_w hardening targeted a function the olmoe refactor replaced; dev now validates the equivalent path in load_expert_merged. - web/*: i18n churn only, took dev's strings. This PR's unique value is preserved and is the point of the merge: the server hardening in openai_server.py (+121), plus json.h, tok.h, Makefile and the downloader/requirements pinning. Bug found and fixed while validating: st_read_f32_cap's call site bounded the scale read with the PER-ROW cardinality (O), which is correct for int8/int4/int2 but rejects grouped formats — int3-g64 (fmt=5) keeps O*i3_groups(I) scales, so a legitimate container was refused (tests/test_int3_load failed). The cap now matches the cardinality qt_resolve_fmt already validates and falloc reserved. Verified: clean build, token-exact unchanged (fp 32/32, int4 21/32), tests/test_int3_load ok, full make check OK. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Minimal, standalone fix for the C model-file trust boundary — the memory-safety half of #368. Colibri loads model dirs + safetensors from mirrors it doesn't control, so the file's declared shapes and byte spans are attacker-influenced input.
Findings fixed (all present in v1.0.0, independently verified against the tree)
st_read_f32(st.h) — the serious one.numelcomes from the tensor's shape,nbytesfrom its offsets; no cross-check. A crafted tensor with an inflated shape made the BF16/F16 loop read past themalloc'd raw buffer and the F32memcpywrite past the caller's config-sized destination. Nownumel*esz == nbytesis enforced before any copy.st.h) —numel *= shape[k]could wrap int64 to a small/negative value that then passes the cross-check. Each multiply is now guarded.glm.c) — the?1:?2:3fmt inference (duplicated at three sites) fell to int2 for any unrecognized weight byte count, so a too-short weight became a valid int2 whose matmul readO*Inibbles past the buffer, and an oversized scale array overflowed the per-rowt->s. Replaced by oneqt_resolve_fmt()that validates weight and scale byte counts against[O,I]or refuses.glm.c) —ftell → malloc(n+1)gave a hostileconfig.jsona load-time OOM, andb[got]=0was a NULL-deref on malloc failure. Capped at 256 MB + NULL-checked.Why not just wait for #368
#368 is the right full pass (server + build too) but needs a rebase and a split decision. This lands the memory-corruption items now, as a v1.0.1 candidate, with the smallest possible surface. The server-side items (fail-closed bind, Host allowlist, telemetry auth) still come via #368's rebase — no overlap.
Verification
detect_group_sizealready imposes.Refs #368
🤖 Generated with Claude Code