perf(gfx1100): share Q8 KV scans across verifier rows - #741
Merged
Kaden-Schutt merged 1 commit intoSep 10, 2026
Merged
Conversation
Base: beta b8092f7. Add wave32 R4/R8 Q8 flash-attention entries for head dimensions 128 and 256. The production route is fail-closed to exact gfx1100, sequential non-tree batches of 4..32 rows, Q8 KV, logical context >4096, and graph capture off. Admission/launcher drift falls back to the established batched Attend step; projection GEMMs remain batched. Canonical XT fresh-process A/B on RX 7900 XTX, HIP 7.2.53211-9999: target SHA256 9f91556f7e0431a077d03756a7102d0154108757289e6e5fe9a2d204c0c9eeb7, draft SHA256 d0a74a232a0e2166d889f823e91e0fbf778d21dd9668d7de055cdecb065401bc, prompt MD5 b4d0b63cddcac872648ddf3cdd92cac2, daemon MD5 512fccca7c7189559048a7aba17cb6c1. Batched samples 33.4/32.6/33.4 tok/s (median 33.4); multi-row 46.4/42.9/46.3 (median 46.3, +38.6%). All six runs: 200 tokens, tau 1.88, 69 cycles, output MD5 b501ab0e0102889bd63537f2006d4f61. Kernel oracle at hd256: R4/R8 speedups 0.98x/0.72x at 2048, 1.45x/2.01x at 4096, 1.99x/2.40x at 20676, and 1.94x/2.28x at 32768. Worst relative error 4.222e-7. ISA: R4 106 VGPR/41 SGPR; R8 186/58; no spills or private scratch. Validation: test_kernels 16/16; hipfire-arch-qwen35 193 passed; rdna-compute 242 passed; canonical XT serve battery 5/5 coherent; maps/env-doc/rustfmt/fmt-bomb/diff checks pass. The Redline PM4 arm remains blocked identically on base and candidate by the pre-existing gfx1100 private=32 scratch refusal in gemv_mq4g256v2_residual; this route is excluded during capture. Full fixture-bound evidence is recorded in docs/perf-checkpoints/2026-09-10-gfx1100-qwen38-multirow-verifier.md.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to the R4/R8 discussion in #695 and the qwen3.8 long-context report in #693. This is the multi-row verifier rebased and reviewed as one commit directly on current
beta(b8092f7c7).One wave now owns four or eight Q8 attention query rows and shares each K/V scan across them. The partials layout and reducer are unchanged, and projection GEMMs remain batched.
The route is fail-closed to:
gfx1100;head_dim128 or 256;>4096;The launcher repeats the arch/shape checks. If admission and launcher support ever drift, execution falls through to the established batched
Step::Attend; it does not fan out into per-row launches.HIPFIRE_FA_PERTOKEN_MIN_CTX=0disables the route.Canonical XT product evidence
RX 7900 XTX,
gfx1100, wave32, HIP7.2.53211-9999.qwen3.8-27b.mq4-xt: 14,980,361,216 bytes, SHA-2569f91556f7e0431a077d03756a7102d0154108757289e6e5fe9a2d204c0c9eeb7, MD5e45d15bfe0c9a87132697101d17cbed6;d0a74a232a0e2166d889f823e91e0fbf778d21dd9668d7de055cdecb065401bc, MD5013395583cd04206c8aa68f4d061983d;benchmarks/prompts/qwen38_issue693_longcode_20676.txt, 75,251 bytes, MD5b4d0b63cddcac872648ddf3cdd92cac2, 21,550 actual model tokens;512fccca7c7189559048a7aba17cb6c1.Six fresh native-daemon processes, order
off,on,on,off,off,on, Q8 KV, DFlash, greedy,HIPFIRE_VERIFY_GRAPH=0, 10-second DPM warmup, 200 output tokens. Same candidate binary and JIT cache; the sole arm delta was route threshold0versus4096. A separate unrecorded probe warmed the cache before the series.All six runs produced 200 tokens in 69 cycles with
tau=1.88, no errors, and identical output MD5b501ab0e0102889bd63537f2006d4f61. A separately built and warmed clean-beta daemon produced 33.5 tok/s with the same output/tau/cycles, matching the disabled candidate arm. The first baseline invocation was cold-JIT and was discarded rather than normalized away.The committed fixture-bound checkpoint records the complete protocol and identities; lifecycle is
historical, not an admission claim.Kernel evidence
Dedicated oracle, Qwen3.8-27B shape (24 Q heads / 4 KV heads / hd256), 100 timed iterations per cell:
Worst relative error was
4.222e-7(limit1e-3). The hd128 screen at ctx8192 measured 1.33x/2.03x, worst relative error3.419e-7.Radiowave metadata: R4 hd256 = 106 VGPR / 41 SGPR; R8 hd256 = 186 / 58; both wave32 with zero VGPR/SGPR spills and zero private scratch.
Validation
test_kernels: 16/16 on gfx1100;cargo test --release -p rdna-compute -p hipfire-arch-qwen35 --lib: 242/242 and 193/193 (4 ignored);serve_harness.py battery: 5/5 coherent, recall checks passed, no empty/runaway/attractor output;The canonical kernel-bucket Redline PM4 arm does not pass on either lane. Clean
betaand this candidate both stop at the same pre-existing limitation:Before that refusal both lanes had the same prefill/decode launch counts and hashes. The new multi-row route is excluded during graph/retained capture, so the failure is not in the new kernel; I am recording it explicitly rather than presenting the battery alone as a full kernel-bucket pass.
This improves verifier cost only. It does not improve draft/target agreement:
tauremains 1.88 on this fixture, so sufficiently low-tau workloads can still favor AR.