Repository navigation
Conversation
Er1ckPang
requested review from
kisnwangg,
liuchongming74,
suteng,
yangzhenzhang02 and
yaoyifanx
as code owners
November 17, 2026 09:22
Er1ckPang
force-pushed
the
refactor/moe_ep_1115
branch
from
November 18, 2026 02:44
b485fd1 to
ea0d121
Compare
Er1ckPang
force-pushed
the
refactor/moe_ep_1115
branch
2 times, most recently
from
November 18, 2026 03:34
e3b3624 to
5f0a3b6
Compare
Er1ckPang
commented
Nov 18, 2026
Er1ckPang
force-pushed
the
refactor/moe_ep_1115
branch
3 times, most recently
from
November 18, 2026 07:32
71dd06d to
630b854
Compare
Er1ckPang
commented
Nov 18, 2026
Er1ckPang
commented
Nov 18, 2026
Er1ckPang
force-pushed
the
refactor/moe_ep_1115
branch
9 times, most recently
from
November 19, 2026 08:49
30eb4fd to
7bd526a
Compare
Contributor
Author
|
/check-pr |
Er1ckPang
force-pushed
the
refactor/moe_ep_1115
branch
3 times, most recently
from
November 19, 2026 14:11
66f09ea to
1683a4e
Compare
Er1ckPang
force-pushed
the
refactor/moe_ep_1115
branch
4 times, most recently
from
November 19, 2026 16:06
9d1d0df to
9306412
Compare
Er1ckPang
commented
Nov 19, 2026
Comment on lines
+196
to
+198
| if self.gate_up_proj.is_meta: | ||
| # HF's initializer no longer recognizes the replaced expert class after materialization. | ||
| self._hp_reset_after_materialization = True |
Contributor
Author
There was a problem hiding this comment.
确认下这块是在meta初始化适配,还是在这里适配
Comment on lines
+199
to
+200
| for name, child in module.named_children(): | ||
| self.add_module(name, child) |
| """Validate named input and output layout declarations against the module interface.""" | ||
| injection_fields = ("local_compute_fn", "inner_wrapper", "inner_target", "region_dispatch") | ||
| # A local execution policy does not inject a replacement compute function. | ||
| injection_fields = ("local_compute_fn", "inner_wrapper", "inner_target") |
Contributor
Author
There was a problem hiding this comment.
为了让父mlp模块可以设置region_dispatch=False,从而继续使用原来模型的mlp.forward
Comment on lines
+113
to
+117
| else: | ||
| order = local_ids.argsort(stable=True) | ||
| hidden = hidden.index_select(0, order) | ||
| rows = torch.arange(order.numel(), dtype=torch.int32, device=order.device) | ||
| recv_inverse = torch.empty_like(rows).scatter_(0, order, rows) |
Contributor
Author
There was a problem hiding this comment.
保留回退场景实现,是不是在ops目录下实现这个比较好
Er1ckPang
force-pushed
the
refactor/moe_ep_1115
branch
from
November 19, 2026 16:27
9306412 to
dbd500e
Compare
Er1ckPang
force-pushed
the
refactor/moe_ep_1115
branch
from
November 19, 2026 17:13
dbd500e to
bfdbbd5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Paired: GitHub #1594 ↔ GitCode !1968
What type of PR is this?
/kind refactor
What does this PR do / why do we need it:
基于 #1523 的 ModuleFamily 机制,把 routed 专家数学与 EP slot 运输分开。模型保留 router、aux-loss、共享专家、gate/mask 和返回结构;统一专家入口消费模型给出的
hidden_states/topk_ids/topk_weights。components/modules/moe/grouped_experts/。公开GroupedExperts和GroupedExpertsConfig,内部默认算法类不作为包级接口。forward直接执行validate_and_flatten_inputs -> permute -> dispatch -> compute_experts -> combine -> unpermute,阶段间直接传 Tensor、split 列表和逆排列索引。gate_up_proj [E,2I,H]、down_proj [E,H,I]。routing_weight_mode仅保留before_fc2和output。直接构造默认前者,HF 来源默认后者;概率只应用一次。init_method_std=0.02,FC2 std 为init_method_std / sqrt(2 * num_layers)。模型替换自动从来源/祖先 config 读取总层数和标准差;已有 Parameter 直接复用。Shard(0),保持完整 H/I,支持 dense TP 的 TP-extend-EP(ETP=1)。FSDP/HSDP 继续负责 unshard、预取和 reshard。moe_mlp_routed模板。父层保留原生 forward,不重复执行专家 EP,也不把完整输出标成 TP Partial。module_binding.py只借用作业已有 EP mesh 并注入 EPContext;没有额外参数保护层或state_contract()。模型接入:
transformers>=5.11.0,固定核对版本为 5.13.1。保留旧
components/modules/grouped_experts.py、公共 module-family 实现及受保护示例;删除未使用的 routing 辅助函数,routing.py最终与基线一致。Which issue(s) this PR fixes:
Fixes #1593
Test Plan and Test result:What scenarios were tested, and what were the verification results(Function, performance, reliability, etc.):
功能与门禁:
12 passed。tests/ut:BLOCKED。默认环境是 Transformers 4.57.1,缺少 DeepSeek-V4、hyper_multicore、OR-Tools、pyarrow 等,收集阶段 74 个依赖错误。GITCODE_TOKEN和JENKINS_URL,无法查询或触发。数值:
1.91e-6,最后一步 loss、logits 和 pre-clip 全局梯度范数一致,无累积漂移。1.07e-4(相对约2.2e-5),最大梯度范数差4.89e-4(相对约2.3e-4),来自 BF16 grouped-GEMM 累加顺序,未发现生产语义错误。性能:
完整逐步数据、关键日志与限制记录在本地报告
output/_codex-docs/moe_ep_refactor_validation_20261120.md(output 为忽略目录)。完整模型使用结构保持的裁剪配置与合成 token,未下载发布 checkpoint 或真实数据集。Self-checklist: