refactor: split glm.c → colibri.c + 4 header modules (−18%) by ZacharyZcR · Pull Request #391 · JustVugg/colibri

ZacharyZcR · 2026-07-18T19:57:28Z

Summary

Rename glm.c → colibri.c (the project is called colibrì, not glm)
Extract four self-contained modules into header-only files, following the existing st.h/tier.h/grammar.h pattern:

file	lines	content
`quant.h`	672	SIMD matmul kernels (AVX2/AVX-512/VNNI/NEON/i8mm/VSX), quantization — pure compute, zero Model dependency
`sample.h`	143	RNG (xorshift64*), top-p distribution, stop-set management
`kv_persist.h`	121	`.coli_kv` on-disk KV cache persistence
`telemetry.h`	189	Dashboard protocol lines (TIERS/EMAP/HITS), stats dump, usage save/load, hardware probe

Main engine: 6588 → 5396 lines (−18%)
make colibri is the new primary target; make glm kept as a phony alias
CI, setup.sh, coli CLI, and all 10 test files updated
make check passes: C + Python, 73 tests, zero warnings

Motivation

The 6.6k-line single file was the #1 barrier to review and community contribution — every PR touching the engine conflicted with every other. This is the first split pass, targeting the four cleanest modules (lowest coupling). The remaining ~5.4k lines (MoE dispatch, attention, expert I/O, serve protocol, main) are candidates for a follow-up once this lands.

Test plan

make colibri — zero errors, zero warnings (Linux, CPU)
make check — 73 tests pass (C unit + Python)
make glm — backward-compat alias works
CI (Linux / Windows / macOS) — automated on push

JustVugg · 2026-07-19T10:43:15Z

Sequencing note after v1.0.0 shipped: this refactor is wanted, but it's a 78-file rename+split that conflicts with everything in flight — merging it now would invalidate the rebase we just requested on #399 (the KV-quant series, which predates today's dev movement in the exact code this PR relocates). Proposed order: #399 lands first (it's semantically hard, this PR is mechanically hard — rebasing mechanics over semantics is the cheaper direction), then this gets a fresh rebase in a quiet window and goes in as the only open change touching the core. We'll ping you when the runway is clear — apologies for the wait, and thanks for the patience.

Rename glm.c → colibri.c and extract four self-contained modules into header-only files (same pattern as st.h/tier.h/grammar.h): quant.h (672 lines) — SIMD matmul kernels, quantization sample.h (143 lines) — RNG, top-p sampling, stop-set kv_persist.h (121 lines) — .coli_kv disk persistence telemetry.h (189 lines) — dashboard protocol, stats, usage Main engine file shrinks from 6588 to 5396 lines (−18%). Build system: primary target is now colibri$(EXE); `make glm` remains as a phony alias for backward compat. CI, setup.sh, coli CLI, and all 10 test files that include the engine are updated. make check passes (C + Python, 73 tests, zero warnings).

New files: README.zh-CN.md — simplified Chinese (大陆用词) README.it.md — Italian (the project's "mother tongue") All four READMEs now link to each other in a consistent nav bar. Updated zh-TW to reflect glm.c → colibri.c rename and new headers.

Lightweight i18n without react-i18next: a LocaleProvider context + useLocale() hook with {{var}} interpolation, auto-detect from navigator.language, persisted to localStorage. 98 translation keys across 4 locale files (en/zh-CN/zh-TW/it). All user-visible strings in App/Brain/Profiling/ErrorBoundary are now t() calls. Language switcher added to the sidebar footer. Build clean (tsc + vite), 18 tests pass.

@monotophic

…d onto #391 split) Re-derivation of e7/disk-class-instr @ de6dd6d (base caa49f7) onto origin/dev @ 61004dc, after PR #391 split c/glm.c into c/colibri.c plus quant.h/sample.h/kv_persist.h/telemetry.h/grammar.h. Semantic equivalence, not a copy: same events, same accounting, re-sited onto the new tree. Placement: colibri.c, unchanged from before the split. telemetry.h (#391) is the dashboard/stats module (HWINFO/TIERS/EMAP/HITS protocol lines + usage persistence) — prof_report(), expert_load_impl(), moe(), g_prof_io and g_edisk_ns all stayed in colibri.c, so DISK-CLASS's aggregation and printing follow them there. No relocation needed, no DEVIATION. Read path: #362 (prefetcher-v3, 22509fc) turned out to be testbed-only scope per its own merge message ("glm.c untouched") — confirmed zero c/glm.c changes in that merge's diff. The seven expert_load() call sites this patch touches (pipe_worker, expert_host_ensure, moe()'s OMP miss loop, pilot_realload, repin_pass_limit, pin_load x2) are structurally identical to the pre-split tree; the demand flag re-attaches at the same sites with the same semantics (1 only at moe()'s own PIPE/OMP miss path, 0 everywhere else), so DISK-CLASS still counts demand loads only. One real drift, unrelated to #362: #417 (cfcc742) fixed the exact "Metal pre-routed FASE A never bumps the real elast/eaccess_clock" defect this feature's comments described as a documented, deliberately-unfixed upstream issue — the real clock now ticks in FASE A too. The private elast_dc/eaccess_clock_dc clock is kept anyway: its job was never only to route around that freeze, it also snapshots pre-bump state so a call's own routing bump can't contaminate its own classification, and keeping DISK-CLASS's bookkeeping fully separate from stock elast state is what makes "byte-identical with PROF=0" provable by construction instead of by argument. Code comments referencing the old defect are updated to reflect the fix (historical note + #417/cfcc742 pointer) rather than describing a bug that no longer exists. Gates: make glm METAL=0 and METAL=1 both clean, zero warnings (matches stock 61004dc, also built clean with zero warnings for comparison). make test-c: 0 failures. make test-python: 77 tests, OK. Authored by Fable 5 in Claude Code, analysis in partnership with @monotophic. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…k broken on dev since JustVugg#270 landed after the JustVugg#391 split Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…391 The #270 rebase resolved the TEST_BINS list but missed the test_pipe_block rule prerequisite and its #include, both still pointing at glm.c (which #391 renamed to colibri.c) — 'No rule to make target glm.c' broke the Linux C test suite on dev. Point both at colibri.c. Build-verified locally. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Update the rebased documentation and comments to name colibri.c after the #391 coordinator split, and correct the E5 comparison table to reflect its instrumentation and OOM-unwind touches. Co-authored-by: Christopher Brand <brand.christopher.c@gmail.com>

…k broken on dev since JustVugg#270 landed after the JustVugg#391 split Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…rter, tests New weight format: int3 with ONE f32 scale per 64-input group (3.5 bits/weight effective). Per group: 16B low plane (2 bits/val, int2 layout) + 8B high plane (1 bit/val), values [-4,3] stored v+4. Same quantization math as tools/quant_ablation.py _quant_last_dim(bits=3, group=64) from JustVugg#132, whose OLMoE ablation measured int3-g64 BEATING the shipped per-row int4 on quality (-7.5 vs -9.3pp) at ~14% fewer bits. Engine (placed per the JustVugg#391 split): matmul_i3 + pack_int3_g64 + I3_* layout helpers in quant.h next to their kernel family; fmt=5 branches in colibri.c's qt_bytes/qt_alloc/qt_fill/matmul_qt/embed_row/qt_addrow/qt_matvec_rows. Format detection now goes through JustVugg#413's qt_resolve_fmt: fmt=5 registers its distinct weight-byte layout O*ceil(I/64)*24 and its scale cardinality O*ceil(I/64) there, validated against [O,I] like every other format. int3-g64 and grouped-int4-at-gs=64 carry the SAME scale count, so the weight bytes are the int3 tag; row formats keep precedence for the small-I shapes where byte counts coincide. The io_uring expert path still used the raw ?1:?2:3 byte inference (it missed fmt=4 grouping entirely and never set gs) — converted to qt_resolve_fmt like the other two expert paths. Backends: qt_cuda_upload returns 0 for fmt=5 (tensor stays CPU-side, the documented fallback), the dense CUDA matmul gate excludes fmt=5, and Metal's existing fmt gates (gemm fmt<=3, moe fmt 1/2) already reject it. Converter: quant_int3_g64 in convert_fp8_to_int4.py; --ebits 3/--xbits 3 now emit it (previously bits=3 silently produced int4). Tests: tests/test_int3.c (bit-exact pack/unpack vs reference, matmul_i3 vs dequant-matmul incl. short tail groups and the real GLM I=7168, QT plumbing, qt_resolve_fmt disambiguation incl. the same-scale-count fmt=4/fmt=5 pair, outlier-rows RMS: int3-g64 3.3x lower error than per-row int4), tests/test_int3_load.c (hand-rolled .safetensors fixture through st_init + qt_from_disk: fmt=5 detected and loaded next to an int4 control tensor), tests/test_int3_convert.py (NumPy pack round-trip vs independent decoder). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@woolcoxm

…orrectness) Fixes a live correctness bug on dev: with CUDA_DENSE=1 on a g64 (fmt=4) container, the dense matmul and attention-absorb kernels applied scales per-row (scales[o] / wscale[row]) while the uploaded scale array is per-group [O × ceil(I/gs)] — wrong scale for nearly every row, garbage output ('odesk odesk…'). quant_matmul now applies group scales inline (matching CPU matmul_i4_grouped exactly), absorb_scale handles fmt=4 in the attention kernels (the kv_b crash), w4a16/TC fast paths stay correctly gated to fmt=2, and a fused CPU gate+up pair (matmul_i4_grouped_pair, AVX2) lands as a bonus. Also adds the fmt=4→CPU fallback log requested in review. Credits: @woolcoxm (author, rebase onto post-#391 dev), @mohamedmastouri2000-boop (root-cause isolation + hardware verification on RTX 5080/sm_120: garbage→coherent, 952 dense tensors + 109 experts fully VRAM-resident, zero fallbacks). Verified locally: clean build, token-exact tiny models unchanged; CI 8/8 green.

…— rebased onto colibri.c Rebases the JustVugg#399 series onto current dev (colibri.c + extracted headers, post-JustVugg#391), which also picks up JustVugg#445's CUDA_RELEASE_HOST pin-budget fix for free. The KV-quant work is additive — dev had no KV8/TQ code — so the split just relocated scaffolding: * colibri.c — KV8/TQ globals + KVState fp8/packed byte caches, kv_alloc, the attention consumers (CPU rotated-int4 orthogonality path; Metal/CUDA native fused kernels dispatch), env parsing, the pin_load KV8-shadow VRAM projection (merged with JustVugg#445's additive prefix budget). * kv_fp8.h / kv_tq.h — the fp8 e4m3 and rotated-int4/PolarQuant codecs (new headers). * kv_persist.h — .coli_kv disk format v2 (fp8) / v3 (TQ): magic-tagged, on-load format detection, v1->v2/v3 in-RAM upgrade + self-heal, fsync durability. * backend_metal.mm — two-library split (fixes the f32 fast-math regression: fast-math for f32/consumer kernels, safe-math only for the RNE encoders) + native codec-1 TQ4 fused attention (Hadamard-orthogonality: rotate the query once, dot packed nibbles, unrotate the context; no f32 restage). * backend_cuda.cu/.h — native codec-1 TQ4 kernel (attention_absorb_kernel_tq) + host entry. * openai_server.py — dispatcher skips unrecognized engine telemetry frames instead of tearing down (unified single/multi-slot interface, version-skew tolerant). Note: the KV-tier flag definitions (g_kv8/g_tq/g_tq_bits/g_tq_codec/g_kv_shadow) moved above the kv_persist.h include so the disk format can see them. Validated on this rebase: `make colibri` clean; `make test-c` green (incl. kv_fp8, kv_tq identities + MLA-consumer equivalence, and kv_disk v1/v2/v3 round-trip + upgrade + reject + self-heal); `make metal-test` green (f32 byte-exact, native TQ4 1.3-1.5x faster than staging, layer decode ok). CUDA compiles clean on the fleet box (sm_89+sm_120, cuda-test incl. attention_absorb_tq) and ran coherent end-to-end on GLM-5.2 744B. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Retarget PR JustVugg#377 onto current dev (origin/dev @ 4aca059) after JustVugg#391 split glm.c into colibri.c + quant.h/sample.h/kv_persist.h/telemetry.h. Reconciles four collisions: - JustVugg#391 split: all glm.c hunks resited -- prof_*/g_prof_io/ProfBase live in colibri.c (NOT telemetry.h); tiers_emit/emap_emit -> telemetry.h (with a rammap_slot forward-decl); st_fd_is_tmpfs/st_fd_fs_magic -> st.h; .coli_kv state-dir -> serve_ctx_init/run_serve/run_serve_mux + main. - DISK-CLASS (1f00142): prof_physical_read_bytes/ProfPhysicalWire merged ON TOP of dev's dc_* fields; PROF protocol line extended 9->17 fields additively; expert_load_impl 6-arg `demand` signature preserved; g_prof_io routed through prof_ssd_tensor_bytes (tmpfs-excluded). - DUAL-SSD mirror (JustVugg#298/JustVugg#469): ESlot.backing hand-merged with dev's aslab/afslab; map_of_fd exact-length + MADV_HUGEPAGE composes with rep_bfd. - int3 fmt=5 (JustVugg#168): rammap_bind_one reuses dev's qt_resolve_fmt/detect_group_size (inline fmt-detection dropped; duplicate detect_group_size not re-added). Verified: make colibri 0 warnings; make check 187 tests pass (test_uring skips in sandboxes via the PR's helpers; test_rammap builds against colibri.c and passes). Default path byte-identical -- oracle-safe by construction. Co-Authored-By: Claude <noreply@anthropic.com>

… conflicts + fix fmt=5 scale cap Conflict resolution (13 hunks, 6 files): - colibri.c / st.h: dev already carries an equal-or-stronger guard for the same threats (qt_resolve_fmt resolves AND validates the quant layout, refusing unknown byte counts instead of falling through to int2; the shape-product overflow guard is present). Took dev's side — no check is lost. - olmoe.c: the load_expert_w hardening targeted a function the olmoe refactor replaced; dev now validates the equivalent path in load_expert_merged. - web/*: i18n churn only, took dev's strings. This PR's unique value is preserved and is the point of the merge: the server hardening in openai_server.py (+121), plus json.h, tok.h, Makefile and the downloader/requirements pinning. Bug found and fixed while validating: st_read_f32_cap's call site bounded the scale read with the PER-ROW cardinality (O), which is correct for int8/int4/int2 but rejects grouped formats — int3-g64 (fmt=5) keeps O*i3_groups(I) scales, so a legitimate container was refused (tests/test_int3_load failed). The cap now matches the cardinality qt_resolve_fmt already validates and falloc reserved. Verified: clean build, token-exact unchanged (fp 32/32, int4 21/32), tests/test_int3_load ok, full make check OK. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The JustVugg#391 rename left release.yml building 'make glm' (an alias that now produces 'colibri') and then asserting/copying 'c/glm', which no longer exists. CI never catches this: only a tag push runs this workflow, so the v1.1.0 tag build failed on macOS at 'ls -lh glm' with the other jobs cancelled and the release skipped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@RDouglasSharp

python3 openai_server.py --model <dir> looked for a binary named 'glm' next to itself. Since JustVugg#391 the build produces 'colibri', so direct invocation was broken on any clean checkout -- it only appeared to work in trees that still had a stale glm from an older build. Spotted by @RDouglasSharp while working on JustVugg#488. Resolve the engine by probing colibri / colibri.exe first and falling back to glm / glm.exe, the same order the coli launcher uses, so old trees keep starting. Verified by removing the stale glm and running the gateway: it resolves to colibri and the engine launches. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Retarget PR JustVugg#377 onto current dev (origin/dev @ 4aca059) after JustVugg#391 split glm.c into colibri.c + quant.h/sample.h/kv_persist.h/telemetry.h. Reconciles four collisions: - JustVugg#391 split: all glm.c hunks resited -- prof_*/g_prof_io/ProfBase live in colibri.c (NOT telemetry.h); tiers_emit/emap_emit -> telemetry.h (with a rammap_slot forward-decl); st_fd_is_tmpfs/st_fd_fs_magic -> st.h; .coli_kv state-dir -> serve_ctx_init/run_serve/run_serve_mux + main. - DISK-CLASS (1f00142): prof_physical_read_bytes/ProfPhysicalWire merged ON TOP of dev's dc_* fields; PROF protocol line extended 9->17 fields additively; expert_load_impl 6-arg `demand` signature preserved; g_prof_io routed through prof_ssd_tensor_bytes (tmpfs-excluded). - DUAL-SSD mirror (JustVugg#298/JustVugg#469): ESlot.backing hand-merged with dev's aslab/afslab; map_of_fd exact-length + MADV_HUGEPAGE composes with rep_bfd. - int3 fmt=5 (JustVugg#168): rammap_bind_one reuses dev's qt_resolve_fmt/detect_group_size (inline fmt-detection dropped; duplicate detect_group_size not re-added). Verified: make colibri 0 warnings; make check 187 tests pass (test_uring skips in sandboxes via the PR's helpers; test_rammap builds against colibri.c and passes). Default path byte-identical -- oracle-safe by construction. Co-Authored-By: Claude <noreply@anthropic.com>

…— rebased onto colibri.c Rebases the JustVugg#399 series onto current dev (colibri.c + extracted headers, post-JustVugg#391), which also picks up JustVugg#445's CUDA_RELEASE_HOST pin-budget fix for free. The KV-quant work is additive — dev had no KV8/TQ code — so the split just relocated scaffolding: * colibri.c — KV8/TQ globals + KVState fp8/packed byte caches, kv_alloc, the attention consumers (CPU rotated-int4 orthogonality path; Metal/CUDA native fused kernels dispatch), env parsing, the pin_load KV8-shadow VRAM projection (merged with JustVugg#445's additive prefix budget). * kv_fp8.h / kv_tq.h — the fp8 e4m3 and rotated-int4/PolarQuant codecs (new headers). * kv_persist.h — .coli_kv disk format v2 (fp8) / v3 (TQ): magic-tagged, on-load format detection, v1->v2/v3 in-RAM upgrade + self-heal, fsync durability. * backend_metal.mm — two-library split (fixes the f32 fast-math regression: fast-math for f32/consumer kernels, safe-math only for the RNE encoders) + native codec-1 TQ4 fused attention (Hadamard-orthogonality: rotate the query once, dot packed nibbles, unrotate the context; no f32 restage). * backend_cuda.cu/.h — native codec-1 TQ4 kernel (attention_absorb_kernel_tq) + host entry. * openai_server.py — dispatcher skips unrecognized engine telemetry frames instead of tearing down (unified single/multi-slot interface, version-skew tolerant). Note: the KV-tier flag definitions (g_kv8/g_tq/g_tq_bits/g_tq_codec/g_kv_shadow) moved above the kv_persist.h include so the disk format can see them. Validated on this rebase: `make colibri` clean; `make test-c` green (incl. kv_fp8, kv_tq identities + MLA-consumer equivalence, and kv_disk v1/v2/v3 round-trip + upgrade + reject + self-heal); `make metal-test` green (f32 byte-exact, native TQ4 1.3-1.5x faster than staging, layer decode ok). CUDA compiles clean on the fleet box (sm_89+sm_120, cuda-test incl. attention_absorb_tq) and ran coherent end-to-end on GLM-5.2 744B. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

ZacharyZcR changed the base branch from dev to main July 18, 2026 20:05

JustVugg mentioned this pull request Jul 19, 2026

Release v1.0.0 #400

Merged

ZacharyZcR added 4 commits July 19, 2026 21:12

ci: fix Windows CUDA DLL check — glm.exe → colibri.exe

93b4a8e

ZacharyZcR force-pushed the refactor/split-colibri branch from a1badb0 to 93b4a8e Compare July 19, 2026 13:12

ZacharyZcR changed the base branch from main to dev July 19, 2026 13:12

ZacharyZcR added 2 commits July 19, 2026 21:15

fix: remove duplicate argmax_v — use NaN-safe version from sample.h

bc69a9a

fix: update test_logit_nan to include colibri.c instead of glm.c

083fda5

JustVugg merged commit 61004dc into JustVugg:dev Jul 19, 2026
8 checks passed

rgbkrk mentioned this pull request Jul 19, 2026

Metal: rebase MTLResidencySet expert residency onto split runtime #426

Merged

monotophic mentioned this pull request Jul 19, 2026

PROF: DISK-CLASS — per-load cold/warm disk instrumentation #438

Merged

cdhdt mentioned this pull request Jul 19, 2026

[Performance]: AVX-VNNI int8/int4 dot kernels use a single accumulator — x86 parity with the measured NEON 4-accumulator path #443

Closed

JustVugg mentioned this pull request Jul 20, 2026

fmt=4 CUDA attention crash — FIXED (#298); benchmark tracker + follow-ups (PROFILE accounting, cache warmth) #334

Closed

3 tasks

JustVugg mentioned this pull request Jul 20, 2026

Add DeepSeek V4 Flash CPU inference with NVMe expert streaming #165

Open

9 tasks

RDouglasSharp mentioned this pull request Jul 21, 2026

Metal: fix grid-size truncation in coli_metal_gemm at large prefill (long-context corruption) #488

Merged

JustVugg mentioned this pull request Jul 22, 2026

Inkling (Thinking Machines 975B MoE): new engine, o200k tokenizer, bf16->int4 converter #312

Merged

JustVugg mentioned this pull request Jul 25, 2026

[Bug]: Nix flake — packages.default fails to build (installPhase copies c/glm, but make glm produces c/colibri); apps.glm points to a non-existent path #595

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: split glm.c → colibri.c + 4 header modules (−18%)#391

refactor: split glm.c → colibri.c + 4 header modules (−18%)#391
JustVugg merged 6 commits into
JustVugg:devfrom
ZacharyZcR:refactor/split-colibri

ZacharyZcR commented Jul 18, 2026 •

edited

Loading

Uh oh!

JustVugg commented Jul 19, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ZacharyZcR commented Jul 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Motivation

Test plan

Uh oh!

JustVugg commented Jul 19, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ZacharyZcR commented Jul 18, 2026 •

edited

Loading