turbo3 CPU dequant (dequantize_row_turbo3_0) skips the inverse WHT, so the to_float round-trip is broken (cosine ~0). Patch + repro inside.

[hexbinoct]

Hi, this is hexbinoct. Full transparency up front: this is a fix from Opus 4.8, all of it, 100%. Since the policy of your project is to use AI for assistance only, I did not go with a PR. Instead, here is an issue with the fix in it, so please take from it what you may. Either way, the fix is there. Thanks!

Summary

In the consumer fork johndpope/llama-cpp-turboquant @ feature/planarquant-kv-cache, the CPU dequantizer for turbo3 (dequantize_row_turbo3_0 in ggml/src/ggml-turbo-quant.c) is a stub that never applies the inverse Walsh Hadamard transform. The encoder rotates with turbo_cpu_fwht before quantizing, but the dequantizer returns centroid * norm directly, so reconstructed values stay in the rotated Hadamard domain. The result is a round-trip that is essentially orthogonal to the input.

This is the function registered as the turbo3 to_float type trait, so it is the path used by the CPU backend, llama-quantize, and any tool that dequantizes turbo3 on CPU. The CUDA and Metal inference paths use their own kernels and are out of scope for this report.

Reproduction

The repo already ships a round-trip test at tests/test-turbo-quant.c. It prints results but does not assert, so the breakage was easy to miss. Building and running it (Qwen-irrelevant, pure quant math):

Before (current stub dequant):

Test 1 (turbo3, e0):     Cosine = -0.088   (should be ~1.0)
Test 2 (turbo3, sin*10): Cosine =  0.074   MSE = 91.88
Test 3 (turbo4, control):Cosine =  0.987   (turbo4 path is fine)

A second independent check, encoding with ggml_quantize_chunk(GGML_TYPE_TURBO3_0, ...) and decoding via the registered to_float, over 1024 random values: rel_MSE = 2.105 (good would be around 0.03 for 3-bit).

Root cause

turbo_cpu_fwht is an orthogonal map y = D2 . (1/sqrt(N)) H . D1 . x (sign flip, Hadamard butterflies, scale and sign flip). Its inverse is its transpose: apply the second sign array, the same symmetric self inverse butterflies, then the scale and the first sign array. The stub skipped all of that.

Fix

Add an inverse WHT and make the dequantizer group aware (reconstruct the centroids for the whole WHT group, inverse rotate, then scale by the corrected norm), mirroring the encoder. Patch against ggml/src/ggml-turbo-quant.c:

@@ turbo_cpu_fwht ... @@
     for (int i = 0; i < group_size; i++) x[i] *= inv_sqrt * s2[i];
 }
 
+/* CPU inverse WHT (transpose of turbo_cpu_fwht): apply s2, the symmetric
+ * self-inverse Hadamard butterflies, then inv_sqrt and s1. */
+static void turbo_cpu_iwht(float * x, int group_size) {
+    const float * s1 = turbo_cpu_s1;
+    const float * s2 = turbo_cpu_s2;
+    const float inv_sqrt = (group_size == 128) ? 0.08838834764831845f : 0.125f;
+    for (int i = 0; i < group_size; i++) x[i] *= s2[i];
+    for (int h = 1; h < group_size; h *= 2)
+        for (int i = 0; i < group_size; i += h * 2)
+            for (int j = i; j < i + h; j++) {
+                float a = x[j], b = x[j + h];
+                x[j] = a + b; x[j + h] = a - b;
+            }
+    for (int i = 0; i < group_size; i++) x[i] *= inv_sqrt * s1[i];
+}
+
@@ dequantize_row_turbo3_0 (replace stub body) @@
-    // Stub — Metal shader handles dequant on GPU.
-    ... centroid * norm with no inverse WHT ...
+    // group size mirrors quantize_row_turbo3_0_ref
+    extern int turbo3_cpu_wht_group_size;
+    int group_size = turbo3_cpu_wht_group_size;
+    if (group_size != 64 && group_size != 128) group_size = (k % 128 == 0) ? 128 : 64;
+    if (k % group_size != 0) group_size = (group_size == 128) ? 64 : 128;
+    const int n_groups = k / group_size;
+    const int blocks_per_group = group_size / QK_TURBO3;
+    for (int g = 0; g < n_groups; g++) {
+        const block_turbo3_0 * grp = x + g * blocks_per_group;
+        float buf[128];
+        for (int b = 0; b < blocks_per_group; b++)
+            for (int j = 0; j < QK_TURBO3; j++) {
+                uint8_t low2 = (grp[b].qs[j/4] >> ((j%4)*2)) & 0x3;
+                uint8_t hi1  = (grp[b].signs[j/8] >> (j%8)) & 0x1;
+                buf[b*QK_TURBO3 + j] = CENTROIDS_3BIT[low2 | (hi1 << 2)];
+            }
+        turbo_cpu_iwht(buf, group_size);
+        float norm = GGML_FP16_TO_FP32(grp[0].norm);
+        for (int j = 0; j < group_size; j++) y[g*group_size + j] = buf[j] * norm;
+    }

After the fix, the same tests/test-turbo-quant.c:

Test 1 (turbo3, e0):     Cosine = 1.000000
Test 2 (turbo3, sin*10): Cosine = 0.986
Test 3 (turbo4, control):Cosine = 0.987   (unchanged, fix is turbo3 only)

and the quantize_chunk to_float round-trip rel_MSE drops from 2.105 to 0.031, in line with iso3 (0.034) and planar3 (0.041).

Two small follow ups (optional)

1. tests/test-turbo-quant.c only prints; adding an assert on cosine close to 1 would catch regressions like this in CI.
2. The dequant reads the WHT group size from the global turbo3_cpu_wht_group_size. If to_float is ever called without the encoder having set it for that row, the fallback must match the size used at encode time. Worth a comment or a less stateful design.

I verified only turbo3 here (broken) and turbo4 (fine, via Test 3). turbo2 was not checked.

Tested on RTX 3070 (sm_86), CUDA 12.4, Windows, building the fork from feature/planarquant-kv-cache.