Skip to content

perf(gfx1100): share Q8 KV scans across verifier rows - #741

Merged
Kaden-Schutt merged 1 commit into
warpfront:betafrom
alpineQ:perf/gfx1100-multi-row-verifier-beta
Sep 10, 2026
Merged

perf(gfx1100): share Q8 KV scans across verifier rows#741
Kaden-Schutt merged 1 commit into
warpfront:betafrom
alpineQ:perf/gfx1100-multi-row-verifier-beta

Conversation

@alpineQ

@alpineQ alpineQ commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Follow-up to the R4/R8 discussion in #695 and the qwen3.8 long-context report in #693. This is the multi-row verifier rebased and reviewed as one commit directly on current beta (b8092f7c7).

One wave now owns four or eight Q8 attention query rows and shares each K/V scan across them. The partials layout and reducer are unchanged, and projection GEMMs remain batched.

The route is fail-closed to:

  • exact gfx1100;
  • Q8 KV;
  • head_dim 128 or 256;
  • sequential, non-tree batches of 4–32 rows;
  • logical context >4096;
  • graph capture off.

The launcher repeats the arch/shape checks. If admission and launcher support ever drift, execution falls through to the established batched Step::Attend; it does not fan out into per-row launches. HIPFIRE_FA_PERTOKEN_MIN_CTX=0 disables the route.

Canonical XT product evidence

RX 7900 XTX, gfx1100, wave32, HIP 7.2.53211-9999.

  • target qwen3.8-27b.mq4-xt: 14,980,361,216 bytes, SHA-256 9f91556f7e0431a077d03756a7102d0154108757289e6e5fe9a2d204c0c9eeb7, MD5 e45d15bfe0c9a87132697101d17cbed6;
  • draft: SHA-256 d0a74a232a0e2166d889f823e91e0fbf778d21dd9668d7de055cdecb065401bc, MD5 013395583cd04206c8aa68f4d061983d;
  • committed prompt: benchmarks/prompts/qwen38_issue693_longcode_20676.txt, 75,251 bytes, MD5 b4d0b63cddcac872648ddf3cdd92cac2, 21,550 actual model tokens;
  • tested daemon MD5 512fccca7c7189559048a7aba17cb6c1.

Six fresh native-daemon processes, order off,on,on,off,off,on, Q8 KV, DFlash, greedy, HIPFIRE_VERIFY_GRAPH=0, 10-second DPM warmup, 200 output tokens. Same candidate binary and JIT cache; the sole arm delta was route threshold 0 versus 4096. A separate unrecorded probe warmed the cache before the series.

route decode samples median delta
existing batched 33.4, 32.6, 33.4 tok/s 33.4
R4/R8 46.4, 42.9, 46.3 tok/s 46.3 +38.6%

All six runs produced 200 tokens in 69 cycles with tau=1.88, no errors, and identical output MD5 b501ab0e0102889bd63537f2006d4f61. A separately built and warmed clean-beta daemon produced 33.5 tok/s with the same output/tau/cycles, matching the disabled candidate arm. The first baseline invocation was cold-JIT and was discarded rather than normalized away.

The committed fixture-bound checkpoint records the complete protocol and identities; lifecycle is historical, not an admission claim.

Kernel evidence

Dedicated oracle, Qwen3.8-27B shape (24 Q heads / 4 KV heads / hd256), 100 timed iterations per cell:

context R4 vs batched R8 vs batched
2,048 0.98x 0.72x
4,096 1.45x 2.01x
20,676 1.99x 2.40x
32,768 1.94x 2.28x

Worst relative error was 4.222e-7 (limit 1e-3). The hd128 screen at ctx8192 measured 1.33x/2.03x, worst relative error 3.419e-7.

Radiowave metadata: R4 hd256 = 106 VGPR / 41 SGPR; R8 hd256 = 186 / 58; both wave32 with zero VGPR/SGPR spills and zero private scratch.

Validation

  • test_kernels: 16/16 on gfx1100;
  • cargo test --release -p rdna-compute -p hipfire-arch-qwen35 --lib: 242/242 and 193/193 (4 ignored);
  • canonical-XT serve_harness.py battery: 5/5 coherent, recall checks passed, no empty/runaway/attractor output;
  • release CLI/daemon and both hardware examples built;
  • crate maps, env-doc scan, changed-file rustfmt, fmt-bomb and diff checks passed.

The canonical kernel-bucket Redline PM4 arm does not pass on either lane. Clean beta and this candidate both stop at the same pre-existing limitation:

gemv_mq4g256v2_residual: GFX10/GFX11 PM4 dispatch does not yet support scratch
(private=32, dynamic_callstack=false)

Before that refusal both lanes had the same prefill/decode launch counts and hashes. The new multi-row route is excluded during graph/retained capture, so the failure is not in the new kernel; I am recording it explicitly rather than presenting the battery alone as a full kernel-bucket pass.

This improves verifier cost only. It does not improve draft/target agreement: tau remains 1.88 on this fixture, so sufficiently low-tau workloads can still favor AR.

Base: beta b8092f7.

Add wave32 R4/R8 Q8 flash-attention entries for head dimensions 128 and 256. The production route is fail-closed to exact gfx1100, sequential non-tree batches of 4..32 rows, Q8 KV, logical context >4096, and graph capture off. Admission/launcher drift falls back to the established batched Attend step; projection GEMMs remain batched.

Canonical XT fresh-process A/B on RX 7900 XTX, HIP 7.2.53211-9999: target SHA256 9f91556f7e0431a077d03756a7102d0154108757289e6e5fe9a2d204c0c9eeb7, draft SHA256 d0a74a232a0e2166d889f823e91e0fbf778d21dd9668d7de055cdecb065401bc, prompt MD5 b4d0b63cddcac872648ddf3cdd92cac2, daemon MD5 512fccca7c7189559048a7aba17cb6c1. Batched samples 33.4/32.6/33.4 tok/s (median 33.4); multi-row 46.4/42.9/46.3 (median 46.3, +38.6%). All six runs: 200 tokens, tau 1.88, 69 cycles, output MD5 b501ab0e0102889bd63537f2006d4f61.

Kernel oracle at hd256: R4/R8 speedups 0.98x/0.72x at 2048, 1.45x/2.01x at 4096, 1.99x/2.40x at 20676, and 1.94x/2.28x at 32768. Worst relative error 4.222e-7. ISA: R4 106 VGPR/41 SGPR; R8 186/58; no spills or private scratch.

Validation: test_kernels 16/16; hipfire-arch-qwen35 193 passed; rdna-compute 242 passed; canonical XT serve battery 5/5 coherent; maps/env-doc/rustfmt/fmt-bomb/diff checks pass. The Redline PM4 arm remains blocked identically on base and candidate by the pre-existing gfx1100 private=32 scratch refusal in gemv_mq4g256v2_residual; this route is excluded during capture. Full fixture-bound evidence is recorded in docs/perf-checkpoints/2026-09-10-gfx1100-qwen38-multirow-verifier.md.
@Kaden-Schutt
Kaden-Schutt merged commit f2a9ac6 into warpfront:beta Sep 10, 2026
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants