feat: add Qwen3-30B-A3B engine

[opxyc]

Summary

Adds qwen3moe.c, a new engine for Qwen/Qwen3-30B-A3B (30B total / ~3B active params), forked from olmoe.c since its plain softmax-router/no-shared-expert shape is the closer starting point than GLM's MLA+DSA+shared-expert design. Two real architectural gaps needed fixing, not just config plumbing:

• GQA attention: 32 query heads share 4 KV heads, and head_dim=128 is explicit in config.json — not derivable as hidden/n_heads the way olmoe.c assumed (32×128=4096 ≠ hidden_size 2048). Rewrote attention() with separate Dq/Dkv widths, a query-head-to-KV-group mapping, and a KV cache sized by n_kv_heads.
• Per-head QK-RMSNorm: Qwen3's q_norm/k_norm apply per attention head (weight length = head_dim) before RoPE — olmoe.c's existing norm wiring is the right mechanism but the wrong shape/granularity.

Everything else (st.h tensor loading, int8 quantize/dequant, the per-layer expert LRU cache, PILOT prefetch, the router shape itself) ports over unchanged. Also adds tools/make_qwen3moe_bench_model.py (tiny synthetic Qwen3MoE fixture for a token-exact correctness gate, deliberately keeping Dq/Dkv/hidden distinct so it actually exercises the GQA reshape) and tools/convert_qwen3moe.py (same disk-safe streaming/resumable design as this fork's OLMoE converter PR).

Scope: correctness-validated engine + converter only. No chat/serve mode, OpenAI-server integration, or coli CLI wiring yet — planned as follow-up once this lands, mirroring the split already done for the OLMoE side of this work.

Validation

• make -C c check — same disk-quota caveat as the sibling PRs (pre-existing, unrelated test); portable build and C unit tests pass.
• CUDA changes were tested with make -C c cuda-test (if applicable) — N/A.
• Performance claims include hardware, commands, and repeatable measurements — no performance claim; this PR is about correctness, not speed.

Token-exact validation:

python tools/make_qwen3moe_bench_model.py --output qwen3moe_bench
python tools/convert_qwen3moe.py --model qwen3moe_bench --out qwen3moe_bench_i8 --bits 8
SNAP=qwen3moe_bench_i8 ./qwen3moe 16 8 qwen3moe_bench/ref_qwen3moe.json

8/8 greedy tokens match the transformers reference exactly — under plain decode, PILOT=1 prefetch, and a forced-small expert cache (cap=4, evicting every access). Also converted and ran the real Qwen/Qwen3-30B-A3B checkpoint end-to-end (int8, all 6144 experts present) — coherent, correctly-formatted, deterministic output (byte-identical across two independent runs of the same prompt).

Compatibility

• The default CPU build remains dependency-free
• No model files, generated binaries, or benchmark artifacts are included

[JustVugg]

Thank you for this — a Qwen3-30B-A3B engine is real work, and the separate-engine structure is exactly right.

I'm leaving this open, but being honest about timing: we have a lot of model engines in flight right now that need to actually work end-to-end (GLM-5.2, plus Inkling #312 and DeepSeek V4 #165 mid-integration), and the focus at the moment is getting those running well rather than adding more families to test. So this isn't rejected — it just isn't where the attention is going this week.

The structure is sound and it stays on the table. When the current engines are solid and there's room to bring in another, this is a strong starting point. Thanks for the clean work, and for your patience while we get the in-flight ones over the line.

[opxyc]

I did this so that I have something that fits my hardware. Thanks for looking into it. @JustVugg

[rofl0r]

out of curiosity: is there a specific reason you target the less capable qwen3 over qwen3.6?

[techfury90]

I was wondering the same thing- I feel like Qwen 3.6 35B A3B is the meta target that everyone's overlooking. I am extremely tempted to try to hack up something for that because my intuition leads me to think this is the model target that unlocks practical local agentic coding workflows on 12-16GB consumer GPUs when combined with expert streaming.

IMO, that is far more interesting than trying to get any of the larger models working here. Unlock Qwen 3.6 and you massively alter the fundamental economics of local agentic coding workflows.

[rofl0r]

Qwen 3.6 35B A3B [snip] ... the model target that unlocks practical local agentic coding workflows on 12-16GB consumer GPUs when combined with expert streaming.

i'm already running the model with ik_llama.cpp on a 6-core cpu (no GPU at all) at 15-10 tok/s (15 with MTP, but gets slower as context window increases). that's already quite useable if you use a harness with a slim system prompt. if you have >24 GB RAM you can already do that too. colibri would unlock using it on machines with even less RAM (and maybe make it even faster).

[techfury90]

I think it can be made much faster than that with careful GPU expert streaming and "cheapish" GPUs.

Let's do some quick back of the envelope math here.

I've got a box with a 12GB RTX5070 and 32 GB of RAM right here. On Qwen 3.5 9B, I get around 80 tok/s with 4 bit quantization and ~200k context (8 bit KV cache quantization). That's 12.5ms per token.

Now, I also have a M3 Pro MBP with 36 GB of unified memory. It's a decent bit slower at 3.6 35B A3B than the RTX5070 on 3.5 9B. However, I find that the tok/s for 35B A3B is ~40 versus ~15-20 for 9B. Naively extrapolating that to the 5070 suggests a possible burst tok/s rate on the order of 160-240 tok/s. If the hot set of experts is on the order of 20% of the model's total parameters... then it will almost certainly run faster than the 80 tok/s on 9B. Even if it takes on the order of 10ms to DMA a few experts in... the high burst rate could make this extremely compelling.