perf(dflash, gfx1100): dedicated MQ4-V2 verify kernels, split-K residual, and launch fusion — merge-sort xt 209 → 285 tok/s - #702

Closed

Kaden-Schutt wants to merge 50 commits into

feat/dflash-dedicated-kernels-pr

Commits on Sep 4, 2026

bench: dflash verify-shape roofline microbench (M0)
Kaden-Schutt
committed
2474039
bench: handle qt13 file reality (same v2 byte stride, sizes asserted)
Kaden-Schutt
committed
7a13ad6
bench: fix shared-X K asserts (out_proj has own X)
Kaden-Schutt
committed
cb4ded9
bench: restore d() closure dropped in edit
Kaden-Schutt
committed
6d23e89
bench: BENCH_DEVICE + ROOFLINE_GBS env overrides for verify-shape microbench
Kaden-Schutt
committed
ef9e373
bench: route residual arm through arch-aware entry (gfx12 fix)
Kaden-Schutt
committed
dff595c
residual: gfx1100 split-K LDS tier (ks2/ks4/ks8) for DFlash verify N<=16
Kaden-Schutt
committed
9b09691
residual ksplit: parity gate 5e-5 + f64 association-floor reference
Kaden-Schutt
committed
7af36ff
residual ksplit: capture-safe tier + kill switch
Kaden-Schutt
committed
96f05da
residual ksplit: rustfmt, crate map, gate parity example behind lab
Kaden-Schutt
committed
a33ca76
residual ldsstage: port gfx12 LDS-staged GEMM to gfx1100 for DFlash verify tier
Kaden-Schutt
committed
f25c4d1
residual ldsstage: default ks4, ldsstage opt-in behind HIPFIRE_RESIDUAL_LDSSTAGE
Kaden-Schutt
committed
11534b1
residual ldsstage gfx1100: roll the fragment loop (unroll 1) to cut VGPRs
Kaden-Schutt
committed
561b780
residual ldsstage: rustfmt + crate map refresh
Kaden-Schutt
committed
e250d2a
dflash launch-fusion prescaffold: pure-extraction hooks + fusion ctx + sidecars + kill switches
Kaden-Schutt
committed
803d17c
S6-fa-prep-q8-pair: batch FA prep and pair Q8 KV writes on gfx1100
Kaden-Schutt
committed
96bef6d
S2-hidden-ring-scatter: exact commit5/scatter5 gfx1100 kernels for hidden-ring copies
Kaden-Schutt
committed
0b2b88f
S6-fa-prep-q8-pair: encode old rope FMA formation explicitly
Kaden-Schutt
committed
3002291
S3-f16-projection-inputs: exact FP16 from RMSNorm+FWHT producers for target projections
Kaden-Schutt
committed
ab8092e
S6-fa-prep-q8-pair: restore norm block dropped by bad range edit
Kaden-Schutt
committed
156223a
S6-fa-prep-q8-pair: unique slots in kv test; restore norm block
Kaden-Schutt
committed
316364a
S2-hidden-ring-scatter: advance head in direct-launcher arm after raw commit
Kaden-Schutt
committed
ea9c6b2
S1-snapshot-bulk-copy: descriptor-driven gfx1100 DeltaNet snapshot save/restore
Kaden-Schutt
committed
b7f7959
S7-draft-launch-collapse: batch draft embeddings and remove draft convert/fill/residual glue
Kaden-Schutt
committed
59939db
S5-gdn-pre-tape-fusion: fuse sigmoid, tape writes, conv, QK norm/interleave around GDN
Kaden-Schutt
committed
395d960
S2-hidden-ring-scatter: bound scatter dst check by loop max row n_rows
Kaden-Schutt
committed
b6ce1d7
S7-draft-launch-collapse: collapse the MQ4V2 arm (fixture draft dtype)
Kaden-Schutt
committed
2a22257
S4-f16-residual-inputs: fuse post-attention/down transforms through FP16 residual WMMA inputs
Kaden-Schutt
committed
77083a5
S6-fa-prep-q8-pair: register qwen35_fa_batch module (unbreaks dispatch build)
Kaden-Schutt
committed
9409d9a
S4-f16-residual-inputs: require LA sidecar window K to equal wo.k
Kaden-Schutt
committed
27b37f6
S8-topk-direct-lmhead: fused MQ4V2 lm_head + greedy DDTree top-K kernel
Kaden-Schutt
committed
e9f0282
S5-gdn-pre-tape-fusion: pin conv accumulation to old FMA tree with explicit fmaf
Kaden-Schutt
committed
05ad8a6
S5-gdn-pre-tape-fusion: replay V region must admit all V blocks
Kaden-Schutt
committed
85806df
S8-topk-direct-lmhead: clamp dead-lane X row (illegal access fix)
Kaden-Schutt
committed
46a7a93
S6-fa-prep-q8-pair: admit 24Q/4K geometry (Qwen3.8-27B FA)
Kaden-Schutt
committed
2933786
S8-topk-direct-lmhead: per-lane partials + lane-parallel phase 2
Kaden-Schutt
committed
f28486f
S8-topk-direct-lmhead: 14 resident blocks/CU (1344-wg grid)
Kaden-Schutt
committed
6746600
S8-topk-direct-lmhead: HIPFIRE_TDK_WGS grid cap + timing sweep
Kaden-Schutt
committed
cf6807d
S8-topk-direct-lmhead: inline RNE convert + two-level barrier
Kaden-Schutt
committed
2c40474
S8-topk-direct-lmhead: symmetric sense-reversing barrier
Kaden-Schutt
committed
81cac27
Revert "S8-topk-direct-lmhead: symmetric sense-reversing barrier"
Kaden-Schutt
committed
0207395
Revert "S8-topk-direct-lmhead: inline RNE convert + two-level barrier"
Kaden-Schutt
committed
346ae5b
Revert "S8-topk-direct-lmhead: HIPFIRE_TDK_WGS grid cap + timing sweep"
Kaden-Schutt
committed
98a32c4
Revert "S8-topk-direct-lmhead: 14 resident blocks/CU (1344-wg grid)"
Kaden-Schutt
committed
3c35869
Revert "S8-topk-direct-lmhead: per-lane partials + lane-parallel phase 2"
Kaden-Schutt
committed
e8e99ff
Revert "S8-topk-direct-lmhead: clamp dead-lane X row (illegal access fix)"
Kaden-Schutt
committed
90a29d7
Revert "S8-topk-direct-lmhead: fused MQ4V2 lm_head + greedy DDTree top-K kernel"
Kaden-Schutt
committed
84a889d
S5-gdn-pre-tape-fusion: two-phase Q/K with widened store scatter
Kaden-Schutt
committed
e385768
launch fusion: rustfmt, crate maps, gate examples, bank bypass debt
Kaden-Schutt
committed
87233ea

Commits on Sep 5, 2026

fix: enforce residual kill precedence and scratch rollback
Kaden-Schutt
committed
b119800