[ghisguth]

Base: paperniuk/splash apple7-m1-1.4 (branch apple7-m1-pq20-zero-as-min, commit 4ee562b).

---

This one takes #15's ZeroAsMin view to PQ2_0, Prism ML's ternary format (2-bit codes, zero point 1, one half
scale d per 128 inputs). PQ2_0 is almost all of prism-ml/Ternary-Bonsai-2-27B-gguf (the 27B target in 6.7 GB, 10 GB with its
draft), so it fits 16 GB M1/M2 Macs. Now that your fix for #22 (6fb9807) lets Bonsai run on Apple7/8, Bonsai decodes
3% faster at one lane and 13% at four on my M2 Max, with the same tokens. As before, I used Claude Code to author it
and tested it by hand.

Before, PQ2_0 ran on the F32 register tile as a seeded zero-point format: each 16-input half closed its chain against
the table's seed, with a seed load and an epilogue per half. Through ZeroAsMin the zero point is an exact min
(m = -d), so chains close once per 32 inputs against the table's sums with b = -129 d, exact. The coefficient unit
stays four spans, so the host's splits don't change. I did the same for Apple9 upstream in incoai#569 (3-7% on
projections there). Q2_0 shares FmtTwoBit and could take the same line, but I have no Q2_0 model to test it, so I
left it.

Update: the second commit (037f451, by @abssoul) adds the same line for Q2_0, which shares FmtTwoBit with PQ2_0, so
the view is exact for it too. On his M1 Max with ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF:Q2_0 the Q2_0 register
projections take 9-15% less time at 1-4 lanes, and teacher-forced logits stay within rounding (details in his commit
message). Q2_0 is about 1% of that model's decode cycle, so there is no end-to-end number for it. With both commits,
gguf-projection and linear-plan pass on my M2 Max with and without shader validation.

What changed

• In runtime/metal/kernels/decode/linear_gguf_sgmatrix.metal: specialized F32Format<FmtPQ20> to ZeroAsMin<FmtPQ20>, and updated the doc comment above ZeroAsMin.
• The instance macro GGUF_SG_FORMAT instantiates gguf_decode_sgf_pq20_* using F32Format<FmtPQ20>::type.
• Shape<ZeroAsMin<FmtPQ20>> evaluates to HasMin = 1, Seeded = 0, CG = 1, UnitSpans = 4 (unchanged from seeded PQ2_0's 4 spans).
• Accumulation order changes from two 16-input chains seeded from the table to one 32-input chain closed against the table's sum; outputs move within gguf-projection's fp64 bound.

Results (M2 Max, 38-core GPU)

1. Fused decode cycle (decode-profile, 2048 prompt tokens, bf16 KV, ABBA)

Lane count	Before ms (A)	After ms (B)	Δ (%)
B1 (1 lane)	90.00	87.32	−3.0%
B2 (2 lanes)	158.40	149.17	−5.8%
B3 (3 lanes)	216.34	201.06	−7.1%
B4 (4 lanes)	294.67	263.57	−10.6% (−31.1 ms)

Repeat runs agree within 0.24%.

2. End-to-end decode throughput (bench-all, ANE share 0.35)

Model	1 lane tok/s	2 lanes tok/s	4 lanes tok/s
Ternary-Bonsai-2-27B (Before)	71.6	82.2	88.3
Ternary-Bonsai-2-27B (After)	73.6	87.4	99.7
Gain (%)	+2.8%	+6.3%	+12.9%

MLX baseline drift check before and after ran within 0.1%. Prefill latency is unchanged (77.88 s 14K cold, 23.78 s hit+4K, 10.64 s 2K).

In backend-benchmark (--scenario decode --samples 3), output token hashes and draft acceptance rate (0.831) are bit-identical across all lanes between before and after.

3. Isolated projection kernels (kernel-ab.sh, ABBA, from the first round of this change)

Projection	Lane	Config	Before ms (A)	After ms (B)	Δ (%)
pq20 17408×5120 gate/up	L1	register S4	0.4730	0.4577	−3.2%
pq20 17408×5120 gate/up	L2	register S4	0.8500	0.7930	−6.7%
pq20 17408×5120 gate/up	L3	register S4	1.1952	1.1028	−7.7%
pq20 17408×5120 gate/up	L4	register S4	1.6329	1.4294	−12.5%
pq20 5120×17408 down	L1	register S8	0.2503	0.2419	−3.4%
pq20 5120×17408 down	L2	register S8	0.4557	0.4192	−8.0%
pq20 5120×17408 down	L3	register S8	0.6270	0.5778	−7.8%
pq20 5120×17408 down	L4	register S8	0.8580	0.7354	−14.3%
pq20 5120×6144 residual	L1	register S4	0.0958	0.0934	−2.5%
pq20 5120×6144 residual	L2	register S4	0.1673	0.1564	−6.5%
pq20 5120×6144 residual	L3	register S4	0.2587	0.2381	−8.0%
pq20 5120×6144 residual	L4	register S4	0.3554	0.3026	−14.9%
pq20 fused (10240+6144+256)	L1	register S4	0.2658	0.2582	−2.9%
pq20 fused (10240+6144+256)	L2	register S4	0.4855	0.4463	−8.1%
pq20 fused (10240+6144+256)	L3	register S4	0.6513	0.6007	−7.8%
pq20 fused (10240+6144+256)	L4	register S4	0.8770	0.7667	−12.6%
pq20 248320×5120 vocab	L1	register S1	3.0269	2.9096	−3.9%
pq20 248320×5120 vocab	L2	register S1	5.5959	5.1914	−7.2%
pq20 248320×5120 vocab	L3	register S1	7.6279	7.0320	−7.8%
pq20 248320×5120 vocab	L4	register S1	10.1152	9.2018	−9.0%
q40 17408×5120 (control)	L1	register S4	0.4185	0.4166	−0.5%
q40 17408×5120 (control)	L4	register S4	1.4362	1.4235	−0.9%
q4k 17408×5120 (control)	L1	register S4	0.4256	0.4267	+0.2%
q4k 17408×5120 (control)	L4	register S4	1.4106	1.4324	+1.5%

Controls remain flat within run-to-run noise.

Testing

• linear-plan: PASS without validation, PASS with MTL_SHADER_VALIDATION=1.
• gguf-projection (every register-tile format, PQ2_0 among them, against an fp64 reference at 1-4 lanes): PASS
  without validation, PASS with MTL_SHADER_VALIDATION=1.
• make check: CPU tests, the Python suite (1320 tests) and ruff pass. In the Metal tests, everything up to
  gguf-moe passes under validation (this includes gguf-rotation and gguf-projection). gguf-moe times out
  under validation on my M2 Max on apple7-m1-1.4 itself too (the 120 s command limit, same as perf: column tile for the MMA paths one- and two-row expert tiles #29 reports on an
  M1 Max), and that stops the rest of test-engine-metal, so the tests after it didn't run in this make check.

[abssoul]

You left the Q2_0 line untested for the lack of a model — I have one: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF:Q2_0, whose dense decode projections run gguf_decode_sgf_q20_* through this kernel (its routed experts are MoE on moe_gguf_mma, untouched). I added the one specialization on top of your branch and ran your battery on an M1 Max (quiet machine):

Change: apple7-m1-q20-zero-as-min in my fork (commit 037f4513, based on your 4ee562b5): F32Format<FmtQ20> → ZeroAsMin<FmtQ20>, one line plus the comment — the formats share FmtTwoBit's algebra (value = d · (q − 1), zero point 1), so the same m = −d, b = −129d view is exact.

Testing (mirrors yours):

• gguf-projection PASS under MTL_SHADER_VALIDATION=1 — the 43 register formats against the fp64 reference, 1–4 lanes, S 1–16, fused and one-pass bitwise checks.
• Isolated projections, gguf-projection-benchmark, median of 100 (register tile, before → after):

projection	lane	before ms	after ms	Δ
q20 17408×5120 gate/up, reg_gu S1	L1	0.757	0.692	−8.6%
	L2	1.468	1.324	−9.8%
	L3	2.090	1.829	−12.5%
	L4	2.621	2.357	−10.1%
q20 5120×6144 residual, reg S4	L1	0.143	0.130	−9.1%
	L4	0.485	0.412	−15.0%

The staged tiles are flat, as expected — same shape and magnitude as your PQ2_0 table.

• Correctness on the real pack: teacher-logits window 2048:256 (257 teacher-forced rows of the vocabulary; the stock release metallib against this change): top-1 agreement 251/257 (97.7%), mean KL 0.0016, max |Δlogit| 0.9 — the fp-rounding class gguf-projection's fp64 bound allows.

Honest caveat: in this pack the Q2_0 projections are about 1% of the decode cycle, so there is no end-to-end gain to claim from it; a model whose dense projections are Q2_0 should take the same range your PQ2_0 line measures. If you want the line, the branch is one commit on top of yours — happy to rebase it wherever it fits.

[abssoul]

Environment for the numbers above, which I should have spelled out there:

• MacBook Pro 14" (2021), Apple M1 Max: 32-core GPU, 64 GB unified memory; macOS 27.0.1 (26A434); quiet machine (no background GPU/CPU load).
• splash 1.4.0-m1 installed release. Stock side = the release splash.metallib (a build from the 1.4.0-m1 tag compares byte-identical to it); after side = the branch built with the same toolchain (xcrun metal -std=metal4.0 -O3), only linear_gguf_sgmatrix.metal differing.
• Model: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF:Q2_0 (66.4 GB GSQ-RCO pack, auto-selected DFlash2 draft).
• gguf-projection-benchmark medians over 100 rounds; teacher-logits with --expert-memory 40G --ple-memory 2G (the server-like split), window 2048:256.

[ghisguth]

@abssoul great, thank you for testing it on a real Q2_0 model. I had a look at your commit, it is the same line as for PQ2_0 and the algebra is the same, so it is exact for Q2_0 too. Your numbers look like my PQ2_0 ones.

I would keep it as your PR, it is your work and your measurement on a model I don't have. Maybe open it on top of this one (or after it is merged), it is a one-line follow-up. @paperniuk unless you prefer it in this PR, then I can cherry-pick it here with abssoul as author.

[abssoul]

Please cherry-pick it into this PR — one PR for the whole zero-as-min family is easier for everyone. No need to keep my name on it either: the branch is public (abssoul:apple7-m1-q20-zero-as-min, single commit 037f4513 on top of your 4ee562b5) if a plain cherry-pick is easiest, and you are just as welcome to squash it into your commit or relabel it however you like. The attribution does not matter to me — the line landing is the only part that matters. @paperniuk whichever form is the least work for you.

[ghisguth]

@abssoul ok, thank you! I'm running gguf-projection and linear-plan with your commit on my M2 Max now (with and without shader validation). If they pass I will add it to this PR as your commit this morning, unless @paperniuk prefers otherwise.

[ghisguth]

@abssoul thank you! gguf-projection and linear-plan pass with your commit on my M2 Max, with and without shader validation, so I added it to this PR as your commit.

[ghisguth]

@paperniuk heads up for when you merge upstream. I sent the Apple9 version of these zero-point views upstream (#569). jianc99 merged them in #602 and then replaced them with one rule for all families (870f07d, in 1.5.0), which was faster than my views on M4 and M5 Max. 1.4.1 doesn’t conflict in this file, but 1.5.0 will conflict with the F32 tile.

If you want, I can port his rule to the F32 instances and measure it on my M2 Max. If it’s as fast, the fork can drop its own views and the merge gets simpler. This PR can go in as is either way.

[paperniuk]

Thanks, merged. Ternary Bonsai PQ2_0 on an M1 Max goes from 35.0 to 36.1 tok/s at one lane and 38.6 to 41.0 at two. It touched the same F32Format lines as #35, I kept both.