Make KVarN usable on ROCm RDNA3 (toward #122: D256 race, portable prefill, f32 WMMA prototype)

[raufaser]

Overview

Toward #122 (KVarN on ROCm is unusable: wrong results plus prefill far behind). This branch makes KVarN work on RDNA3 step by step, validated on gfx1100 (RX 7900 XTX, ROCm 7.14) with Qwen3.6-27B-Q5_K_S and kvarn6:

• D256 k00-combine race fix in fattn-mma-f16.cuh (tile_Q reused across k00 iterations, barrier only fired for np > 1), plus inter-tile sync. Enables the RDNA D256 route.
• HIP prompt-prefill (nq > 16) routes through portable-native direct-record attention, since RDNA WMMA VKQ accumulates in fp16 (~3e-4/call) and compounds over depth into a KLD collapse. Decode stays on WMMA. Opt out with GGML_KVARN_AMD_PROMPT_PORTABLE=0.
• Portable prefill optimizations (warp-shuffle reduction, fp32 rescale, QB=8 queries per block, hoisted token resolve): pp4096 107 to 291 t/s at identical accuracy.
• Accuracy ladder harness (GGML_KVARN_TEST_NKV_LADDER_ONLY): GPU-native vs CPU-materialized error over D128/256/512 and n_kv 256 to 8192.
• f32 WMMA accumulator prototype for DV=128/256 prefill (16-wide A, single wmma_f32 per K step, persistent f32 acc, mirroring DV=80/112), plus the VKQ_C entry-count fix it required. Still gated behind PORTABLE=0, default serving path unchanged.
• ub>64 fix: the three whole-tile dst_final_meta stores in flash_attn_ext_f16_process_tile were gated !is_kvarn_kv, so KVarN whole-tile blocks never published their (max, rowsum). Split tiles got meta from the stream-k fixup, but that skips boundary-aligned tiles, so those rows kept zero meta and the tail merge silently discarded their correct body values. Symptom: KVarN WMMA prompt path clean at ub<=64, garbage at ub>=96. Removing the gate makes whole-tile blocks publish their meta like dense FA. Validated: ub512 KLD on 4B / 27B / 35B-MoE at portable parity (4B bit-exact); 32k KLD 2.118 -> 0.029.

Results: ladder RMSE ~1e-5; 32k KLD 0.022 vs portable 0.023 (was 2.14 on the fp16 path), same-top 97.0%; prefill pp4096 286 (portable) vs 497 (f32-WMMA); decode ~33.5 both; reference q8_0 at 804.

Base: v0.4.7

Retargeted to v0.4.7 per maintainer request. The branch merges v0.4.7 (D=64 KV-head support, etc.) with this work; the only merge conflict was fattn-kvarn-portable.cuh, resolved by porting v0.4.7's D=64 support (RECORD_DIM, the v_original_domain original->rotated V Hadamard, D=64 dispatch) onto the portable QB-batching + warp-shuffle kernel.

• D=64 is CUDA-only. v0.4.7 fail-closes the KVarN D=64 route on HIP/ROCm (fattn-kvarn-dispatch.cu: D64 stripped from capabilities on non-CUDA backends "until each backend has independent runtime qualification"). So on this gfx1100 build D64 is never dispatched; the D=64 kernel here is compile-validated and mirrors v0.4.7's, but runtime qualification on HIP is out of scope. D128/256/512 are the validated ROCm paths.

Additional information

• Progress toward Bug Report: kvarn6 KV-quantization causes massive performance regression on ROCm (RX 7900 XTX) #122 (update posted there during the work).
• The f32 approach follows https://github.com/stew675/llama-cpp-rdna-boosts (question thread; their answer pointed at the DV=80/112 fp32-PV tiles as reference).
• Validate with: ctest KVarN subset, ladder env above, llama-perplexity KLD flow and llama-bench -p 4096 from AGENTS.md.
• Known, kept out of scope: the f32-WMMA default flip (still behind PORTABLE=0); two flaky AVs under long KLD runs share one fault offset with dumps saved, zero occurrences on portable, PORTABLE=1 is the fallback. (The ub>=96 serving divergence previously listed here is now fixed by the ub>64 commit above.)

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES. An AI coding assistant (Muse Spark supervised by the author) wrote the tile-spec and count changes plus test-harness knobs, ran the builds, ladder/KLD/bench measurements on the author's RX 7900 XTX, and drafted this description. All numbers above are measured, not estimated; the author reviewed the diff before submitting.

Update 2026-09-09: F32-WMMA is now the default + CUDA build fix (2 new commits)

This supersedes the "f32-WMMA default flip (still behind PORTABLE=0)" note above.

• bad6f86c1 ÔÇö F32-WMMA is now the default for HIP KVarN prompt-prefill
  (was: portable-native by default, WMMA opt-in). The ub-geometry fix plus the
  DV=128/256 fp32 accumulators make WMMA both fast and exact, validated at the
  serving default (-ub 512, gfx1100, kvarn6, ngl99): 4B KLD 0.006822 (bit-exact
  vs fp16 base), 27B 0.004814, 35B-MoE 0.004093; 32k KLD 0.029 (was 2.118);
  ladder D128/256/512 ~1e-5. Portable-native remains for unsupported shapes and
  as an explicit opt-in (GGML_KVARN_AMD_PROMPT_PORTABLE=1). Decode (nq<=16)
  unchanged. HIP-gated; CUDA/Vulkan untouched.
• edc1d2f64 ÔÇö CUDA build fix (pre-existing, found by the thermis RTX 4090
  build): the GGML_KVARN_PORTABLE_ATTRS debug print used HIP-only
  func-attribute APIs, breaking nvcc. Now HIP API on HIP / CUDA API on CUDA
  (debug-only, no behavior change). This PR as-merged did not build on CUDA;
  this fix is required.
• CUDA gate (thermis RTX 4090, GGML_CUDA=ON): full build clean; KVarN/FA ctest
  (kvarn|fattn-route|cuda-fattn) 14/14 pass, incl. test-kvarn. The flip is
  compiled out on CUDA (HIP-gated) ÔÇö expected no-op, confirmed.

Community validation needed: gfx1151 (Strix Halo) / gfx1201 (RDNA4)

We only have gfx1100 (RX 7900 XTX) plus CUDA/4090. If you run gfx1151 or
gfx1201, please validate this PR on your hardware and report back:

• Build clean (GGML_HIP=ON, GPU_TARGETS set for your arch at initial configure).
• The ub512 KLD trio (see "Validate with" above) ÔÇö especially whether F32-WMMA
  as-default is accurate at your serving -ub (state the -ub value).
• The accuracy ladder (GGML_KVARN_TEST_NKV_LADDER_ONLY=1).
• ctest -R "kvarn|fattn-route".
• RDNA4 note: D64 KVarN is CUDA-only by design (fail-closed on HIP); D128/256/512
  are the expected ROCm paths. Thank you!

Update 2026-09-09 (pm): merged v0.4.7 "complete optimized D64", conflict resolved

• origin/v0.4.7 advanced (e1f6d6fe6, complete optimized D64) after this PR's
  earlier v0.4.7 merge, conflicting in portable.cuh. Resolved by merging
  (cc566c899) and taking upstream's complete D64 rewrite.
• Our interim D64 hand-merge + portable QB-batching/warp-shuffle are superseded
  (upstream covers D64 natively; portable is fallback-only post-flip, so peak
  fallback perf is acceptable; re-propose batching as a follow-up if it matters).
  Our CUDA func-attributes fix proved unnecessary (upstream removed the HIP-only
  debug block).
• Preserved through the merge: the F32-WMMA default flip and the ub-geometry fix
  (both auto-merged). One 3-line follow-up (37ce9e337) drops the now-dead
  batched-portable call (fallback uses standard portable — correct, unbatched).
• Validation on the merged tree (gfx1100, default WMMA): ub512 trio identical to
  pre-merge (4B 0.006822 bit-exact, 27B 0.004814, 35B 0.004093); key-target build
  clean. PR is MERGEABLE/CLEAN.

[Anbeeld]

Please change the target branch to v0.4.7, and ensure changes are compatible with it.