[atom-sgl-benchmark] Add manual OOB selection and Minimax-M3 for SGLang MI355 benchmark - #1671
Open
junyyang-amd wants to merge 4 commits into
Open
[atom-sgl-benchmark] Add manual OOB selection and Minimax-M3 for SGLang MI355 benchmark #1671junyyang-amd wants to merge 4 commits into
junyyang-amd wants to merge 4 commits into
Conversation
Contributor
🏷️ CI GuideRuns automatically on every eligible PR before approval:
Heavy model tests:
|
junyyang-amd
force-pushed
the
yjy/modify_options_for_mi355
branch
2 times, most recently
from
July 22, 2026 12:55
a27a019 to
c2f6aba
Compare
zhangxinyuanliuhengyu
added a commit
that referenced
this pull request
Jul 22, 2026
zhangxinyuanliuhengyu
added a commit
that referenced
this pull request
Jul 22, 2026
junyyang-amd
force-pushed
the
yjy/modify_options_for_mi355
branch
from
July 22, 2026 15:21
c2f6aba to
90e33b3
Compare
zhuyuhua-v
reviewed
Jul 23, 2026
| "prefix": "minimax-m3-mxfp4-tp4", | ||
| "extra_args": [ | ||
| "trust_remote_code", | ||
| "--tensor-parallel-size 4 --attention-backend aiter --page-size 128 --mem-fraction-static 0.8 --context-length 32768 --max-running-requests 128 --disable-radix-cache --disable-cuda-graph" |
Contributor
Author
There was a problem hiding this comment.
I have already made modifications based on the PR. Please review again.
Collaborator
There was a problem hiding this comment.
update m3 recipe here: https://github.com/ROCm/ATOM/blob/main/recipes/atom_sglang/MiniMax-M3.md could you please help also add nightly test cases?
junyyang-amd
force-pushed
the
yjy/modify_options_for_mi355
branch
from
July 23, 2026 03:03
90e33b3 to
9f340bd
Compare
valarLip
pushed a commit
that referenced
this pull request
Jul 27, 2026
* Fix SGLang DSV4 compatibility on ATOM Keep SGLang 0.5.14 DSV4 paths working with ATOM while gating the temporary AITER custom all-reduce IPC workaround to DSV4 run configurations. Co-authored-by: Cursor <cursoragent@cursor.com> * Update SGLang 0.5.15 validation defaults Keep Qwen SGLang runs on the page-size layout validated with 0.5.15 and carry the TP8 all-reduce workaround for large FP8 runs. Co-authored-by: Cursor <cursoragent@cursor.com> * Update SGLang default ref to 0.5.15 * Apply Black formatting fixes * Fix SGLang 0.5.15 manual validation blockers Add scoped compatibility fixes so the SGLang 0.5.15 ATOM path can start on MI355 validation images without failing on optional model imports, missing AITER shuffle helpers, or the Cohere2Moe strict decorator. Co-authored-by: Cursor <cursoragent@cursor.com> * Restore scoped SGLang validation changes Remove the provisional compatibility shims from the previous validation attempt so the PR keeps the validated ATOM and SGLang source paths unchanged. Co-authored-by: Cursor <cursoragent@cursor.com> * Restore custom collective defaults for SGLang validation Remove the temporary custom all-reduce disablement from SGLang validation configs and the matching ATOM gating now that the MI355 path is being validated without that workaround. Co-authored-by: Cursor <cursoragent@cursor.com> * Update SGLang validation release settings Add the DeepSeek-R1 predownload benchmark entry and keep the release workflow conditions aligned with the current SGLang validation branch state. Co-authored-by: Cursor <cursoragent@cursor.com> * Remove unrelated DeepSeek-R1 benchmark entry Drop the predownload benchmark entry that is unrelated to the SGLang 0.5.15 validation update. Co-authored-by: Cursor <cursoragent@cursor.com> * Restore docker release push gates Keep the existing release workflow gate and re-login logic intact because it is unrelated to the SGLang 0.5.15 validation changes. Co-authored-by: Cursor <cursoragent@cursor.com> * Keep SGLang upgrade scoped to plugin paths Remove unrelated ATOM core and native benchmark changes from the SGLang 0.5.15 PR, while carrying the MI308 Qwen3.5 page-size setting validated for SGLang. * Fix MiniMax-M3 SGLang processor compatibility Return an empty multimodal payload for text-only MiniMax-M3 requests while rejecting unsupported multimodal inputs explicitly. Co-authored-by: Cursor <cursoragent@cursor.com> * Refine SGLang validation config scope Keep Qwen validation on default page sizing and route MiniMax-M3 through SGLang's generic multimodal processor compatibility path. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix MiniMax-M3 processor registration Use SGLang's generic multimodal processor path for MiniMax-M3 while preserving compatibility with older modality enums. Co-authored-by: Cursor <cursoragent@cursor.com> * Restore SGLang validation configs Keep the SGLang 0.5.15 PR scoped to plugin changes by reverting benchmark and accuracy config files to main. Co-authored-by: Cursor <cursoragent@cursor.com> * Revert validation config changes Restore benchmark and accuracy validation configs to match main so this PR only carries plugin changes. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix SGLang capture mode import path Use the SGLang 0.5.15 runner import path for get_is_capture_mode while keeping the older cuda_graph_runner fallback. Co-authored-by: Cursor <cursoragent@cursor.com> * Support SGLang DSA graph compatibility * Apply Black formatting to SGLang compatibility patch Co-authored-by: Cursor <cursoragent@cursor.com> * Align SGLang image Transformers dependency Co-authored-by: Cursor <cursoragent@cursor.com> * Fix SGLang CI server bind on Docker runners with IPv6 disabled. Default SGLANG_HOST to 127.0.0.1 so launch_server does not attempt to bind ::1 when containers keep disable_ipv6=1. * Fix SGLang accuracy validation metadata Add split graph metadata hooks required by SGLang 0.5.15 and align the DeepSeek-V4-Flash accuracy recipe with the measured CI gate. Co-authored-by: Cursor <cursoragent@cursor.com> * Prefer CI workspace for SGLang validation Ensure validation scripts import the mounted PR checkout before the ATOM copy baked into manual images. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix GLM-5.2 SGLang DSA import compatibility Co-authored-by: Cursor <cursoragent@cursor.com> * Fix SGLang ForwardMode draft extend compatibility Co-authored-by: Cursor <cursoragent@cursor.com> * Apply Black formatting to ForwardMode compatibility Co-authored-by: Cursor <cursoragent@cursor.com> * Avoid root-owned Python cache files in SGLang CI. Co-authored-by: Cursor <cursoragent@cursor.com> * Recover SGLang accuracy workspace ownership before checkout. Co-authored-by: Cursor <cursoragent@cursor.com> * Tune SGLang accuracy stress settings. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix SGLang MI308 accuracy startup failures. Co-authored-by: Cursor <cursoragent@cursor.com> * Apply Black formatting to SGLang bridge fix. Co-authored-by: Cursor <cursoragent@cursor.com> * Trim SGLang 0.5.15 compatibility shims. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix DSV4 fp8 cache dtype detection. * Fix DSV4 decode page metadata. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix DSV4 SGLang fp8 SWA cache injection Route the SGLang draft-extend SWA injection through the native fp8 two-buffer cache arguments so fp8 runs update both NoPE and RoPE SWA pools instead of the bf16-only buffer. Co-authored-by: Cursor <cursoragent@cursor.com> * Limit Qwen3.5 SGLang loader threading Use the per-expert loader for the non-FP8 Qwen3.5-35B-A3B SGLang accuracy case to avoid under-filled batched MoE load groups without changing runtime inference behavior. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix DSV4 fp8 packed KV views. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix SGLang ForwardBatch backend fallback. Avoid assuming newer SGLang ForwardBatch objects expose attn_backend directly so DSA CUDA graph capture can fall back through the active forward context. Co-authored-by: Cursor <cursoragent@cursor.com> * Limit DeepSeek R1 TP4 loader threading Co-authored-by: Cursor <cursoragent@cursor.com> * Fallback DSV4 fp8 cache on unsupported GPUs. Co-authored-by: Cursor <cursoragent@cursor.com> * Use SGLang forward context for radix dtype. Co-authored-by: Cursor <cursoragent@cursor.com> * Trim SGLang fallback test scaffolding. Co-authored-by: Cursor <cursoragent@cursor.com> * Restore DSV4 native fp8 dtype check. Co-authored-by: Cursor <cursoragent@cursor.com> * Harden SGLang shard validation retries. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix GLM-5.2 SGLang accuracy launch Patch AITER runtime gaps required by GLM-5.2 PTPC MoE and route SGLang DSA metadata through ATOM so CI can complete real accuracy validation. Co-authored-by: Cursor <cursoragent@cursor.com> * Update SGLang default ref to 0.5.15.post1 Co-authored-by: Cursor <cursoragent@cursor.com> * Clean SGLang ROCm topk patch whitespace. Co-authored-by: Cursor <cursoragent@cursor.com> * Share SGLang plugin compatibility helpers Centralize draft-extend and DeepSeek-V4 fp8 support checks so SGLang plugin paths do not drift across backend files. * Fix DSV4 fp8 KV fallback scope Keep the requested SGLang fp8 KV cache dtype intact while conservatively disabling only the unsupported DSV4 native 2-buffer path when gfx detection fails. Co-authored-by: Cursor <cursoragent@cursor.com> * Revert DSV4 fp8 KV fallback scope Co-authored-by: Cursor <cursoragent@cursor.com> * Fix DSV4 SGLang fp8 KV fallback Co-authored-by: Cursor <cursoragent@cursor.com> * Fix GLM5.2 SGLang image compatibility * Format SGLang plugin test Co-authored-by: Cursor <cursoragent@cursor.com> * Add manual OOB tier selection for SGLang MI355 benchmarks * Add minimax-m3 for atom-sglang benchmark * Move glm-5.1 from p0 to p2 * Revert "Merge PR #1671 for SGLang benchmark P0 selection" This reverts commit ac2a67c, reversing changes made to b025be7. * Apply SGLang patches in release image build Co-authored-by: Cursor <cursoragent@cursor.com> * Clean up SGLang upgrade branch extras Co-authored-by: Cursor <cursoragent@cursor.com> * Fix SGLang DeepSeek MLA KV pool lookup Co-authored-by: Cursor <cursoragent@cursor.com> * Drop unrelated DSV4 config fallback changes Co-authored-by: Cursor <cursoragent@cursor.com> * Disable DSV4 Pro MTP3 draft-extend graph in CI Avoid the unstable draft-extend CUDA graph capture path for the DSV4 Pro MTP3 accuracy job while preserving speculative validation coverage. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix forward context ruff precheck Co-authored-by: Cursor <cursoragent@cursor.com> * Fix SGLang plugin Ruff precheckin lint Co-authored-by: Cursor <cursoragent@cursor.com> * Fix remaining SGLang plugin Ruff findings Co-authored-by: Cursor <cursoragent@cursor.com> * Remove SGLang KV pool compatibility helper Co-authored-by: Cursor <cursoragent@cursor.com> * fix(sglang): align DeepSeek-V4 MTP with SGLang 0.5.15 * fix(sglang): handle latest AITER V4 MLA signature * fix(sglang): clamp DSV4 draft-extend accept lens * fix(sglang): format DSV4 draft accept-lens guard * fix(sglang): place ruff noqa on exception guard * docs(sglang): document Cohere2 MoE backport lifecycle Record the upstream fix and removal condition so the temporary Docker patch can be retired with a future SGLang release. --------- Co-authored-by: root <root@hjbog-srdc-15.amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: xytpai <xytpai@foxmail.com> Co-authored-by: xinyuazh <xinyuazh@users.noreply.github.com> Co-authored-by: xinyuazh <xinyuazh@hjbog-srdc-15.amd.com> Co-authored-by: whx-sjtu <xiaowang990929@gmail.com> Co-authored-by: zhangxinyuanliuhengyu <zhangxinyuanliuhengyu@users.noreply.github.com> Co-authored-by: qichu <qichu@amd.com> Co-authored-by: junyyang <junyyang@amd.com> Co-authored-by: zhangxinyuanliuhengyu <190174499+zhangxinyuanliuhengyu@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation