GGUF: support all quant types, add qwen35moe - #131

Open

vcruz305 wants to merge 36 commits into

FlashML-org:mainfrom

vcruz305:feat/generic-gguf

Commits on Aug 24, 2026

feat(gguf): generalize quant-type support beyond Q4_0/Q8_0/Q6_K

vcruz305
and
claude
committed
c5e2b43
feat(gguf): qwen35moe config + tensor-name adapter (not yet registered)

vcruz305
and
claude
committed
6f40e50
test(gguf): resolve IQ3S_N_SCALE when parsing block_iq3_s from ggml-common.h

vcruz305
and
claude
committed
a9614af
feat(gguf): serve qwen35moe GGUF checkpoints (Ornith-1.5-35B-A3B)

vcruz305
and
claude
committed
8f17e8d
fix(gguf): stop _LAYER_MAP naming attributes the qwen35moe module lacks

vcruz305
and
claude
committed
fcdbd0c
test(gguf): assert merged-projection parts map to None, not invented names

vcruz305
and
claude
committed
1a035bc
fix(gguf): shared-expert weights were silently dropped; conv1d/gate dtype+shape

vcruz305
and
claude
committed
e6c8248
fix(gguf): lm_head was never loaded (output.weight dropped by the blk. filter)

vcruz305
and
claude
committed
af612b9
fix(gguf): read every quant type from the file; bank specs match alloc_layer_banks

vcruz305
and
claude
committed
e6b4684
fix(gguf): tokenizer could not be built for qwen35moe (KeyError in transformers)

vcruz305
and
claude
committed
0231506
fix(gguf): the GGUF op swap was never invoked, so loading died on embed_tokens.weight

vcruz305
and
claude
committed
0a9e940
fix(gguf): stray gguf_expert_types kwarg on resolve_moe_cache_auto()

vcruz305
and
claude
committed
4e70902
fix(gguf): expert gate_up fusion interleaved experts, producing fluent garbage

vcruz305
and
claude
committed
6d0297f
fix(gguf): missing (1+w) shift on Qwen3.5 RMSNorm weights

vcruz305
and
claude
committed
aec2db3
fix(gguf): ssm_a holds A not A_log; revert the (1+w) norm shift

vcruz305
and
claude
committed
853507b
fix(gguf): un-tile V heads -- llama.cpp reorders them, FreeToken expects HF order

vcruz305
and
claude
committed
711c34b
fix(gguf): ssm_out needs the CUDA dequantizer (pure-torch path is Q4_0/Q6_K only)

vcruz305
and
claude
committed
811a2b3
docs(gguf): mixed-bank error now names the cause and the --pure workaround

vcruz305
and
claude
committed
4163a93
feat(gguf): serve the dense qwen35 variant (Qwen3.8-27B)

vcruz305
and
claude
committed
439085b
docs: document the GGUF architectures and quant types in models.md
vcruz305
committed
39a8c6e
fix(gguf): the no-routed-experts assert fired on dense checkpoints
vcruz305
committed
08f702e
fix(gguf): tokenizer arch map was missing the dense qwen35 entry
vcruz305
committed
bd49652
feat(gguf): multi-shard GGUF reading + qwen3moe architecture

vcruz305
and
claude
committed
834ed4d
fix(gguf): shard glob typo made every shard set look incomplete
vcruz305
committed
ac12e72
test(gguf): rewrite the shard tests to emit GGUF bytes directly
vcruz305
committed
89dff4b
docs: multi-shard and qwen3moe, with the measured results
vcruz305
committed
0f26f7e
feat(cpu-moe): Q4_K and Q6_K expert kernels for the cpu/hybrid backends
vcruz305
committed
6f6c864

Commits on Aug 25, 2026

Commits on Aug 26, 2026