Repository navigation
qwen3.8-mtp: vendor mmul proj-engine xclbins (decode 1.77 -> 2.84 tok/s) - #909
Conversation
deltanet_layer.xclbin and attention_layer.xclbin rebuilt with the q4_K block kernel on aie::mmul (IRON PR AMD-ROCm-Internal/FastFlowLM_IRON#135, PR Check run 41264160514). Files are the CI artifacts, not local builds: 5881abeed388f52ecb7c4be5f1b71c6580876bb81d02dae5906f8a52fda8d162 deltanet_layer.xclbin 1caee645f17fc942911e629ba75d10209fed2a479f86f62c2a102aacbce576d3 attention_layer.xclbin Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
IRON#135 merged (5014861). Re-verified: the merge commit has no file differences from the PR head cf425f4 that run 41264160514 built, so these xclbins are the merged kernels. sha256 of the vendored files still match that run's artifacts (5881abee... deltanet, 1caee645... attention). Marking ready. |
…e xclbins Decode tok/s at 1k/2k/4k/8k context: 1.06/1.86/1.50/1.32 -> 2.04/2.23/2.70/2.31 (IRON PR Check run 41264160514, XRT perf-bench, the bench the page's numbers come from). Prefill and TTFT are unchanged by this kernel and left as is. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Added the docs benchmark update: docs/docs/benchmarks/qwen3.8_results.md decode row 1.06/1.86/1.50/1.32 -> 2.04/2.23/2.70/2.31 tok/s (1k/2k/4k/8k) and a regenerated qwen38_decoding.png in the same style. Numbers are IRON PR Check run 41264160514 XRT perf-bench for this PR's kernels (the same bench the page's previous numbers match: its baseline gave 1.50/1.32 at 4k/8k and prefill within 0.2%). Prefill and TTFT are unchanged by this kernel and untouched. |
What
Replaces the two Qwen3.8-27B-MTP layer xclbins with builds of the q4_K projection engine on
aie::mmul(internal IRON PR #135). Decode goes 1.77 -> 2.84 tok/s on a local NPU at unchanged MTP acceptance. The engine library is unchanged: no.so/.dllchange, only the two kernels.src/xclbins/Qwen3.8-27B-NPU2/deltanet_layer.xclbin5881abeed388f52ecb7c4be5f1b71c6580876bb81d02dae5906f8a52fda8d162xclbins-Qwen3_8_gate_delta_net_verifysrc/xclbins/Qwen3.8-27B-NPU2/attention_layer.xclbin1caee645f17fc942911e629ba75d10209fed2a479f86f62c2a102aacbce576d3xclbins-Qwen3_8_gated_attention_verify(layer.xclbin, renamed to the name the engine registers)IRON PR Check run: 41264160514 (the vendored files are exactly that run's artifacts; no local builds committed).
Verification
Output text diverges from stock after a few hundred characters (greedy decode with ~1-2% numeric noise in the layer), stays coherent.
Order
The IRON PR is still a draft. Per the usual order this should land after IRON merges; the kernel sources are not touched by the merge, but I will re-verify the merged run's artifacts against the sha256 above before this leaves draft.
🤖 Generated with Claude Code