[ghisguth]

Base: paperniuk/splash apple7-m1-1.3.2 (branch apple7-m1-gdn-verify-1.3.2, fd01a92). The same commit is on
apple7-m1-1.4 as apple7-m1-gdn-verify; it cherry-picks without conflicts.

---

This one speeds up the fused GDN verify decode (verify_gdn_fused_table16f), which was 5.2% of a one-lane MLX 4-bit
decode cycle on my M2 Max. With it, the kernel takes 35% less time at one lane, and decode is 1.2–2.5% faster on
all four models I tried (MLX 4-bit, Q4_0, UD-Q4_K_M, UD-Q5_K_M). The bits are unchanged. As before, I used Claude Code to author it and tested it by hand.

What changed

Only the Table16F instances of kernels/decode/gdn.metal change, which only Apple7/8 run (two template flags set
from is_same_v<Table, Table16F>). The plain and Table16 kernels and the host code are untouched.

• Conv rows per tap. Timing ablations showed the prologue costing about 30 µs per layer at one lane, nearly all of
  it in GdnConvHistory::at: every load of the four-tap convolution chose between the carried rows, the pending rows'
  tape and the packed rows. The row a tap reads is the same for all of a simdgroup's channels, so the prologue now
  chooses each tap's row once (GdnConvHistory::row) and loads the channels from it. The arithmetic is unchanged.
• Outputs kept per lane. The scan stored every token's output from lane 0, a divergent branch per token and
  batch. Lane t now keeps token t's outputs with a select and stores them once per batch.
• No test changes. gdn-decode's replay already compares the Table16F decode with the plain one bit for bit over
  changing batches, for all three shapes (27B, vh32 and Flash-Next's), so it compares the new code with the old.

Results (M2 Max, 38-core GPU)

The kernel alone, the 27B's 48 GDN layers in one command with DRAM-cold states, µs per layer at 1 / 2 / 4 lanes,
ABBA order:

	1 lane	2 lanes	4 lanes
Before	87.3	129.0	250.7
Conv rows per tap	60.0	101.5	170.4
Both changes	56.5	96.7	161.6
Δ (%)	−35	−25	−36

decode-profile, MLX 4-bit, 2048-token prompt, ms per cycle at 1 / 2 / 3 / 4 lanes, ABBA, on apple7-m1-1.3.2
(on apple7-m1-1.4 the cycle gains are −2.1 / −1.7 / −1.4 / −1.4%):

	GDN verify	Cycle
Before	4.25 / 6.15 / 8.14 / 11.99	80.8 / 140.5 / 192.4 / 257.5
After	2.71 / 4.46 / 5.33 / 7.57	79.4 / 138.6 / 189.6 / 253.4
Δ (%)	−36 / −27 / −35 / −37	−1.7 / −1.4 / −1.5 / −1.6

make test-performance-real (ANE share 0.35, MLX first and last), decode in tok/s at
1 / 2 / 4 lanes, on apple7-m1-1.3.2. Prefill doesn't change (14K cold, hit+4K and 2K within 0.2%), and both MLX runs
of each build agree within 0.2%.

Model	Before	After	Δ (%)
MLX 4-bit	89.2 / 104.2 / 113.2	91.2 / 105.5 / 115.1	+2.2 / +1.2 / +1.7
Q4_0	84.0 / 97.5 / 108.9	85.3 / 98.9 / 111.2	+1.5 / +1.4 / +2.1
UD-Q4_K_M	79.7 / 95.4 / 108.4	81.1 / 96.5 / 111.1	+1.8 / +1.2 / +2.5
UD-Q5_K_M	73.2 / 89.6 / 103.7	74.2 / 90.8 / 105.3	+1.4 / +1.3 / +1.5

On apple7-m1-1.4: MLX 4-bit +2.4 / +1.6 / +1.8%, UD-Q4_K_M +2.0 / +1.9 / +2.0%.

Together with the decode norm PR (they touch different files and merge cleanly), decode-profile on
apple7-m1-1.3.2 gives a cycle of 81.2 / 140.3 / 192.6 / 257.7 → 77.4 / 136.9 / 187.7 / 252.1 ms
(−4.6 / −2.4 / −2.5 / −2.2%), about the sum of the two. I didn't run the full benchmark on both together.

What didn't help

More threadgroups per head (each scanning part of the value rows, prologue duplicated) and 16 scan simdgroups per
threadgroup were both slower, even at one lane: after the prologue fix the scan is bound by each core's throughput,
not by latency. Two and eight rows in flight stay worse than four.

Note on maintenance

To keep Apple9+ untouched, the new code sits behind two flags next to the old code in gdn_decode_prologue and
gdn_decode_scan, so if upstream changes those functions it may conflict there. Both changes are bit exact on every
family, so if you prefer a smaller diff, I have an ungated version too (just replaces the code). Your call.

The same two changes would apply to Apple9+'s Table16 and plain kernels, but I can't measure those here.

Testing

• gdn-decode passes on the M2 Max (the Table16F replay against the plain decode, bit for bit), with and without
  MTL_SHADER_VALIDATION. gguf-projection passes too.
• make check passes on the M2 Max on apple7-m1-1.3.2. (On apple7-m1-1.4, the 1.4 port's own test failures,
  q8-chunked-prefill not building, then gguf-moe, qwen4-kernels, paged-attention-plan and metal-backend,
  fail the same way with and without this change.)
• make test-performance-real with MLX 4-bit, Q4_0, UD-Q4_K_M and UD-Q5_K_M before and after on apple7-m1-1.3.2,
  and with MLX 4-bit and UD-Q4_K_M on apple7-m1-1.4.

[ghisguth]

@paperniuk BTW, I also pushed the same change on top of apple7-m1-1.4: https://github.com/ghisguth/splash/tree/apple7-m1-gdn-verify. It is one commit and cherry-picks without conflicts, so you can take it from there if it's easier. If you prefer a separate PR to apple7-m1-1.4, let me know and I will open it.

[paperniuk]

Thanks! Confirmed on M1 Max (32-core GPU). gdn-decode passes.

backend-benchmark --scenario decode on mlx-community/Qwen3.8-27B-4bit, ANE share 0.35, ABBA with #26 in the same run, 3 samples each, on apple7-m1-1.3.2. Decode in tok/s at 1 / 2 / 3 / 4 lanes; output tokens identical.

	Before	After	Δ (%)
MLX 4-bit	73.9 / 85.7 / 92.0 / 92.9	75.5 / 86.7 / 94.1 / 94.7	+2.2 / +1.2 / +2.2 / +1.9

Merged.