tools: add mmq-tune, an MMQ tile-width autotuning harness

[roberteg16]

Summary

Adds tools/mmq-tune, a harness that measures the MMQ kernel at a forced mmq_x over a catalog of shapes taken from real models, so tile-width decisions can be made from measurement rather than from an occupancy model alone.

The core mirrors both branches of ggml_cuda_mul_mat_q and then calls launch_mul_mat_q<type, mmq_x> directly. For MoE it builds the ids tensor itself and drives ggml_cuda_launch_mm_ids_helper, so expert_bounds and the compacted grid are produced exactly as in production.

How the measurement is set up:

• Routing is drawn from a Zipf distribution, which is what measured MoE routing follows; a uniform distribution would be the one case the tile heuristic already assumes.
• Expert weights are generated distinctly per expert rather than cycling a few slices, which would shrink the weight working set far below the real one.
• Correctness is decided by majority vote over a position-weighted checksum of the output. Diffing against one chosen reference cannot work, because a wrong reference makes every other config look broken; the position weight is what makes a misplaced row visible.
• Timing interleaves one iteration per config per pass, so clock drift spreads across all configs instead of penalising whichever runs late.

The catalog holds the 99 distinct mul_mat_q shapes that 20 models execute (Qwen2.5/3/3.5/3.6, Llama-3.1, gemma-2/3/4, SmolLM2, GLM-4.7; 0.5B to 35B, dense and MoE), each tagged with its invocation count so aggregates reflect real time. The token count is a sweep parameter rather than part of the catalog, and it matters: the dense q8_0 mmq_x=64 cliff only appears around 64 tokens while the MoE tile-width gap needs 2048.

mmq_y and the warp count are device-side constexpr, so sweeping them needs a rebuild. They become GGML_CUDA_MMQ_Y_RDNA3_5 / GGML_CUDA_MMQ_NWARPS_RDNA3_5, wired to the GGML_HIP_MMQ_Y / GGML_HIP_MMQ_NWARPS cache variables, with sweep.py driving the rebuild loop. Defaults are unchanged, and only three pairs compile at all since the kernel asserts nwarps*tile_C::I == mmq_y; 64/4 measured best of the three.

All of it is behind GGML_MMQ_TUNE (default OFF), which keeps it out of the build entirely.

Usage

cmake -S . -B build -DGGML_HIP=ON -DGGML_MMQ_TUNE=ON
cmake --build build --target mmq-tune
./build/bin/mmq-tune --list
./build/bin/mmq-tune -m 64,2048 --output csv
python tools/mmq-tune/sweep.py --mmq-y 32,64,128 --mmq-nwarps 2,4,8

Test plan

• Default build (GGML_MMQ_TUNE off) does not reference the harness at all
• -DGGML_MMQ_TUNE=ON builds; mmq-tune --list shows the 99 shapes
• Sweep runs over the full catalog and reproduces the kernel timings behind HIP: fix MMQ tile-barrier race and tune tile width for RDNA3.5 #75

Notes for review

• Base is gfx11, not master.
• Independent of #; either can land first. Running the harness without that fix will show the mmq_x 16 and 32 widths returning non-deterministic results, which is the bug that PR fixes.