[ghisguth]

Base: paperniuk/splash apple7-m1-1.3.2 (branch apple7-m1-norm-loads-1.3.2: the change, then the test).
The same two commits are on apple7-m1-1.4 as apple7-m1-norm-loads; they cherry-pick without conflicts.
Independent of the GDN verify PR.

---

This one is a small change with a large effect: the decode RMS norm that writes the register tile's Table16F
(norm_rms_table16f_decode, 141 per 27B cycle) takes 60% less time on my M2 Max, which is about 2% of a one-lane decode
cycle, and decode is 1–2% faster. The bits are unchanged. As before, I used Claude Code to author it and tested it by hand.

What changed

Only the Table16F instances of norm_rms_table (kernels/shared/normalization.metal) change, which only Apple7/8
run. The plain and Table16 norms are untouched.

• The wide-row path loaded its first chunk and the table's span pairs with column < width ? row[column] : 0. On
  Apple7/8 these conditional loads were most of the norm's time. With Clamped (set from
  is_same_v<Table, Table16F>), every load is issued at a column clamped into the row, and the zero is selected
  after the load. The values, their order and the arithmetic are unchanged.
• A new test, norm-table (second commit): these norms had no test of their own. It runs the Table16 and Table16F
  decode norms with bf16 and F32 weights at widths 1024 to 17408 (the streaming and the chunked path), one to four
  lanes and inputs of very different magnitudes, and requires their rows to match the plain norm's bit for bit and
  their table and sums to match the register tile's own preparation byte for byte. It passes before and after the
  change.
• I also compared the changed kernel with the unchanged one byte for byte with a throwaway harness (outputs, table
  and sums; widths 2048 to 17408, 8 to 32 rows, both weight types, three input scales).

I found this while working on the GDN verify kernel, where a per-load pointer choice cost 30% of the kernel.

Results (M2 Max, 38-core GPU)

The 27B's 141 eight-row norms back to back, each with its own weights: 15.3 → 6.1 µs each at 8 rows (−60%), 17.8 →
5.7 µs at 32 rows.

decode-profile, MLX 4-bit, 2048-token prompt, ms per cycle at 1 / 2 / 3 / 4 lanes, ABBA, on apple7-m1-1.3.2
(the norm's line is from separate dispatches):

	Norm (Table16F)	Cycle
Before	4.50 / 4.43 / 4.43 / 4.46	81.1 / 139.8 / 192.3 / 258.0
After	2.42 / 2.46 / 2.45 / 2.48	79.4 / 138.6 / 190.5 / 256.3
Δ (%)	−46 / −44 / −45 / −44	−2.1 / −0.8 / −1.0 / −0.7

On apple7-m1-1.4 the cycle gains are −2.3 / −1.7 / −0.8 / −0.8%.

make test-performance-real (ANE share 0.35, MLX first and last), decode in tok/s at
1 / 2 / 4 lanes, on apple7-m1-1.3.2. Prefill doesn't change (within 0.2%), and both MLX runs of each build agree
within 0.2%.

Model	Before	After	Δ (%)
MLX 4-bit	89.2 / 104.2 / 113.2	91.2 / 105.4 / 113.9	+2.2 / +1.2 / +0.6
Q4_0	84.0 / 97.5 / 108.9	85.3 / 98.5 / 110.0	+1.5 / +1.0 / +1.0
UD-Q4_K_M	79.7 / 95.4 / 108.4	81.1 / 96.8 / 109.8	+1.8 / +1.5 / +1.3
UD-Q5_K_M	73.2 / 89.6 / 103.7	74.0 / 90.7 / 102.8	+1.1 / +1.2 / −0.9

The UD-Q5_K_M 4-lane number is one run. The norm is the same kernel for every quant and decode-profile shows the
4-lane cycle 0.7% faster, so I think it's noise, but I didn't rerun it. On apple7-m1-1.4: MLX 4-bit
+2.8 / +1.6 / +0.8%, UD-Q4_K_M +2.0 / +1.5 / +0.4%.

Together with the GDN verify PR (they touch different files and merge cleanly), decode-profile on
apple7-m1-1.3.2 gives a cycle of 81.2 / 140.3 / 192.6 / 257.7 → 77.4 / 136.9 / 187.7 / 252.1 ms
(−4.6 / −2.4 / −2.5 / −2.2%), about the sum of the two. I didn't run the full benchmark on both together.

The new norm-table test is a separate commit and adds a file to dev/native.mk. If it's too much to carry on
upstream merges, feel free to drop that commit. The change is checked the same way by the harness I mention above.

Testing

• norm-table and the byte-for-byte harness, each with and without MTL_SHADER_VALIDATION.
• gguf-projection passes on the M2 Max with and without shader validation.
• make check passes on the M2 Max on apple7-m1-1.3.2.
• make test-performance-real with MLX 4-bit, Q4_0, UD-Q4_K_M and UD-Q5_K_M before and after on apple7-m1-1.3.2,
  and with MLX 4-bit and UD-Q4_K_M on apple7-m1-1.4.

[ghisguth]

@paperniuk Same as #25, I also pushed this change on top of apple7-m1-1.4: https://github.com/ghisguth/splash/tree/apple7-m1-norm-loads. It is two commits (the change, then the test) and they cherry-pick without conflicts. If you prefer a separate PR to apple7-m1-1.4, let me know and I will open it.

BTW, #25 and #26 touch different files and merge cleanly together. With both, decode cycle on my M2 Max is −4.6% at one lane.

[paperniuk]

Thanks, and thanks for the test! Confirmed on M1 Max (32-core GPU). norm-table passes.

backend-benchmark --scenario decode on mlx-community/Qwen3.8-27B-4bit, ANE share 0.35, ABBA with #25 in the same run, 3 samples each, on apple7-m1-1.3.2. Decode in tok/s at 1 / 2 / 3 / 4 lanes; output tokens identical.

	Before	After	Δ (%)
MLX 4-bit	73.9 / 85.7 / 92.0 / 92.9	75.9 / 86.6 / 93.0 / 93.4	+2.7 / +1.1 / +1.0 / +0.6

Merged.