gfx1100: MQ4V2 (qt=44) decode at 0.1-0.3 tok/s - V2-ladder SKUs unusable on Windows/RDNA3

[SergioSV96]

Registry V2-ladder SKUs for Qwen3.8-27B decode at 0.1–0.3 tok/s on gfx1100 — ~200× slower than classic-MQ4G256 artifacts of the same architecture on identical hardware.

Repro

Windows 11, RX 7900 XTX (gfx1100), ROCm 7.2 HIP SDK. Master tip 363e93d7 (+ #631/#622 cherry-picks for dual-GPU binding), --locked build.

PS> hipfire pull qwen3.8:27b-mq4-xt
PS> hipfire bench qwen3.8-27b.mq4-xt --runs 5 --warmups 3 --max-tokens 128 `
      --backend noslots --workload stateless --spec off --json

Load succeeds (61 kernels JIT'd), but generation crawls:

"decode_tok_s":  { "median": 0.2,  "mean": 0.2, "min": 0.1, "max": 0.3 },
"prefill_tok_s": { "median": 0.4,  "mean": 11.0, "min": 0.2, "max": 27.3 },
"ttft_ms":       [ 69235, 60193, 96596, 901, 878 ]

For contrast, a classic-MQ4G256 artifact (converted from GGUF via --format mq4) of the same architecture decodes at tens of tok/s on this exact box, and its prefill ran ~500 tok/s-class. So the base HFQ4/MQ4G256 lanes are fine; the slowdown appears specific to the MQ4V2 (qt=44) execution path on gfx1100.

Impact

Every qwen3.8:27b-* V2-ladder SKU (mq3/mq4/mq5/mq6 × base/pro/xt, plus trunk) routes through this path — they convert and load cleanly but are not usable for inference on gfx1100 at current master. Presumably related to why registry text carries no live baselines for these SKUs and why #616 (packed-MQ4 backend) / #611 are still open.

Request

If MQ4V2-on-gfx11 is simply "not admitted yet", a registry/CLI warning at pull time would save users a 15 GB download + 86-minute bench discovery. If it's supposed to work, happy to run any diagnostic head — full bench JSON retained.

[HUSRCF]

@SergioSV96 — thank you for the sharp report. I downloaded and verified the exact registry artifact, then ran same-artifact controls on a Linux gfx1100 reference.

Artifact identity:

file:   qwen3.8-27b.mq4-xt
size:   14980361216 bytes
sha256: 9f91556f7e0431a077d03756a7102d0154108757289e6e5fe9a2d204c0c9eeb7
header: 497 tensors at quant_type 44

Hardware/runtime: Radeon Pro W7900 (gfx1100, 48 GB), Linux, ROCm/HIP 7.14, physical GPU1 exposed as daemon device 0. DFlash off, Q8 DeltaNet state, asym3 KV, noslots/stateless/spec-off.

On current master 5a19844f, using the issue's full --runs 5 --warmups 3 --max-tokens 128 protocol:

decode:  median 36.5 tok/s  [36.4, 36.6]
prefill: median 286.9 tok/s [284.5, 287.1]
TTFT:    median 83.6 ms     [83.6, 84.3]

I also built and ran your exact reported head 363e93d7 against the same artifact. A bounded 8-token discriminator remained healthy:

decode 37.5 tok/s
prefill 285.4 tok/s
TTFT 84.1 ms

Therefore this is not a general qt=44/gfx1100 execution regression, and it was not merely repaired by a later master commit. The remaining discriminator is the affected Windows ROCm 7.2 / dual-GPU / code-object environment.

Could you please run this first minimal A/B on the affected machine, using the dGPU-selection setup that produced the original report?

git fetch origin master
git switch master
git pull --ff-only origin master
cargo build --release --locked -p hipfire-cli -p hipfire-daemon

Remove-Item -Recurse -Force .hipfire_kernels -ErrorAction SilentlyContinue

target\release\hipfire.exe bench qwen3.8-27b.mq4-xt `
  --runs 1 --warmups 1 --max-tokens 8 `
  --backend noslots --workload stateless --spec off --json

Please include:

• the complete GPU dev ... and pre-compiled kernels ... lines;
• the resulting 8-token JSON;
• whether the clean .hipfire_kernels rebuild changes the result;
• Get-ChildItem Env:HIP_VISIBLE_DEVICES,Env:ROCR_VISIBLE_DEVICES,Env:HIPFIRE_TARGET_ARCH -ErrorAction SilentlyContinue;
• the two-GPU enumeration order from the same shell.

If the clean rebuild or dGPU-only visibility restores normal speed, the leading cause is wrong-architecture/stale code-object selection. If it remains near 0.2 tok/s with the correct gfx1100 device line, the next useful instrument is per-kernel timing on Windows to identify the specific qt=44 dispatch family carrying the ~200x cost.

[sruggier]

I also observed this on gfx1103 with Qwen 3.8 (on Linux 7.1.8), with speeds roughly 1/4 of llama.cpp (using the Vulkan or ROCm backends). I tried 3.6 to try to establish whether this issue was applicable, and the performance was much closer to llama.cpp, so it may be the same problem, or a related one.

[harrytyp]

Additional Windows/gfx1100 data point (RX 7900 XTX, ROCm 7.2 HIP SDK, hipfire 0.3.0 master, MQ4V2/qt=44 registry artifact):

Registry qwen3.8:27b produces non-language output on Windows. Math prompt with thinking off returns e.g. !!!!!!!!!!!!!!; Count to five. with thinking on fails validation (HTTP 500). Load and kernel JIT are clean, decode ~37-39 tok/s is plausible, but output is corrupt. Same box runs lemonade's ROCm llama.cpp build with correct output, so GPU/driver are not the cause.

Two related observations:

1. VMM KV backend crashes on large prefill (hipMemSetAccess: invalid argument, 4 MiB prefix / 2 MiB mappings - same as Windows: VMM KV backend fails at first decode (hipMemSetAccess invalid argument) - kv_backend=contiguous works #635). Cherry-picking PR fix(vmm): retry access per physical mapping on Windows #643 locally fixes it: 40k prefill now completes at 397 tok/s.
2. VMM + DFlash at ~40k ctx collapses to ~15 tok/s (vs ~37 AR), consistent with DFlash decode collapses below AR past ~10k ctx: verify-block target attention costs B×L per cycle #693's B x L verify cost. contiguous KV + DFlash at short ctx reaches 245 tok/s.

Happy to run diagnostics or a fixed head on this box.