Skip to content

fix(hcu): avoid double router weighting in DeepEP GEMM - #52

Open
alexanderbin123 wants to merge 2 commits into
v0.25.1from
fix/deepep-deepgemm-router-weight
Open

alexanderbin123 wants to merge 2 commits into
v0.25.1from
fix/deepep-deepgemm-router-weight

Conversation

@alexanderbin123

@alexanderbin123 alexanderbin123 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • align HCU DeepGEMM and DeepEP HT reduction with the official vLLM apply_router_weight_on_input contract
  • apply router weights exactly once: at input or during unpermute/reduce
  • share the reduction-weight helper across native DeepGEMM, Channel-FP8/INT8 DeepEP HT, and SlimQuant W4A8 HT
  • keep DeepEP LL masked/N32, pure-TP AITER, and other quantization ownership boundaries unchanged

Reference

This follows the official vLLM modular MoE design:

  • prepare/dispatch may apply router weights to the input
  • expert execution produces unweighted expert results
  • finalize/unpermute/reduce must use uniform weights when input weighting already occurred

Reference repository: https://github.com/vllm-project/vllm

Validation

Passed before opening this MR:

  • tests/runtime_patch/test_deep_gemm_utils.py: 13 passed
  • tests/runtime_patch/test_quant_gemm_aiter.py: 195 passed
  • tests/runtime_patch/test_moe_deepep.py: 84 passed
  • Python AST/bytecode compilation for modified modules
  • git diff --check
  • Qwen3.5-9B and Qwen3.5-35B-A3B integration test collection executed successfully

Additional real-model validation on 2026-09-03 using 8 gfx936 devices:

  • Qwen3.5-35B-A3B-W8A8, TP8/EP8 smoke: passed with allgather_reducescatter
  • Qwen3.5-35B-A3B-W8A8, DP8/EP8 with explicit DeepEP high-throughput: passed; selected DeepEPHTPrepareAndFinalize
  • Qwen3.5-35B-A3B BF16, DP8/EP8 with explicit DeepEP high-throughput: passed; selected DeepEPHTPrepareAndFinalize
  • Qwen3.5-35B-A3B BF16, DP8/EP8 with explicit DeepEP low-latency: passed; selected DeepEPLLPrepareAndFinalize

Environment-limited or unsupported configurations:

  • Qwen3.5-9B smoke: skipped because the HCU model runtime resource was unavailable
  • Qwen3.5-9B graph parity: skipped because the HCU model runtime resource was unavailable
  • Qwen3.5-35B-A3B TP+EP matrix: 6 skipped because the test requires gfx938 and the available devices are gfx936
  • Qwen3.5-35B-A3B-W8A8 with deepep_auto: rejected by the existing HCU guard because this Channel-INT8 auto path is validated only for DeepSeek-V4; explicit DeepEP HT is the supported Qwen3.5 route
  • Qwen3.5-35B-A3B-W8A8 with explicit DeepEP LL: rejected because no Int8 MoE backend supports this deployment configuration
  • Qwen3.5-35B-A3B-W8A8: no dedicated upstream local integration test was present in this v0.25.1 test tree

Known unrelated test prerequisite:

  • the full test_moe_deepep.py module has one existing environment failure in test_moe_runner_and_shared_experts_cold_replacement_contract because /models/vllm_0251 is not available; the remaining 84 tests passed.

alexanderbin123 and others added 2 commits September 3, 2026 18:09
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
@alexanderbin123
alexanderbin123 force-pushed the fix/deepep-deepgemm-router-weight branch from a839ebd to 02d3c81 Compare September 3, 2026 10:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant