Enable DeepSeek-V3 Wide-EP disaggregated inference + fix DP=32 correctness

[raviguptaamd]

Enable DeepSeek-V3 Wide-EP disaggregated inference + fix DP=32 correctness

Related: #176 (complementary; this PR is separate and surgical — #176 is untouched).

Summary

Enables DeepSeek-V3 (671B FP8) on the vllm_disagg MoRI-EP wide-EP path and fixes a
silent MoE token-routing corruption at 32-way expert parallelism (DP=32). Inspired by
and complementary to #176; kept as a separate, surgical PR (7 files, +164/−32) so #176 is
untouched.

Validated end-to-end with NIAH long-context accuracy on MI300X (gfx942) across 1P/1D, 2P/2D,
4P/4D. At DP=32 the fix restores accuracy to the DP≤16 baseline (~8/10, zero empty answers).

What's in here

Area	Change	Why
DP=32 correctness (headline)	Default prefill MoRI backend to mori_low_latency when PREFILL_DP_SIZE ≥ 32 (connectors/moriio.sh)	The mori_high_throughput (InterNodeV1) prefill dispatch silently mis-routes MoE tokens at ≥32-way EP → corrupt prefill KV → empty/wrong decode. low_latency (which decode already uses) is clean at DP=32. DP≤16 unchanged; any explicit override still wins.
Boot / OOM	Bound the attention workspace via --max-model-len (connectors/moriio.sh + models.yaml MAX_MODEL_LEN)	Without a cap, vLLM sizes the attention workspace for the model's full 163840 context → ~128 GiB alloc that OOMs alongside EP-sharded weights.
Image	Dockerfile builds from the shikpate ROCm/vLLM/MoRI-1.2.3 base + a GLM-stripped vLLM branch (docker/vllm_disagg_inference.ubuntu.amd.Dockerfile)	The validated Wide-EP MoRI-IO stack for DeepSeek-V3 (standard MLA; GLM sparse-DSA commits removed as unused).
Recipe env	3-tier models.yaml env precedence (image-baked < yaml < submit -e) + submit-time protect-list (vllm_disagg.sh, run_xPyD_models.slurm)	Lets a model's models.yaml env: recipe override image-baked ENV while a genuine submit-time -e still wins.
Accuracy harness	niah benchmark selector + generalized NIAH harness (multi-seed, warmup/readiness, timeout-vs-wrong, model-neutral) (run_xPyD_models.slurm, benchmark_niah.{py,sh})	Reproducible long-context accuracy gate used to validate the above.

Backend / image refs

• BASE_IMAGE: rocmshared/pytorch-private:vllm-rocm_07_22_2026_shikpate_mori1.2.3 (shikpate MoRI-1.2.3 base)
• VLLM_REPO/REF: https://github.com/raviguptaamd/vllm.git @ dsv3-wideep-clean
  (Wide-EP MoRI-IO base + a generic crash-safe rmsnorm probe fix; GLM sparse-DSA commits stripped —
  DeepSeek-V3 uses standard MLA. Delta vs base = only vllm/_aiter_ops.py, 2 lines.)
  This fork/branch is public.
• MoRI v1.2.1, AITER 0.1.16.post3, router ravgupta/discovery-dp-rank-roundrobin.

Validation (NIAH mean /10, 2k/8k/20k/35k words, 3 seeds)

Topology	Prefill DP	Backend	Result
1P/1D	8	HT	9.3 / 8.7 / 7.7 / 7.7 ✅
2P/2D	16	HT	9.3 / 8.3 / 8.3 / 8.0 ✅
4P/4D (before)	32	HT	0.3 / 5.3 / 7.3 / 2.0 ❌
4P/4D (this PR)	32	LL	7.7 / 7.3 / 8.0 / 7.7 ✅

Notes / follow-ups

• The DP=32 fix is a config-only default (backend selection), not a vLLM code change. No rebuild
  needed to toggle it; explicit PREFILL_MORI_BACKEND / VLLM_ALL2ALL_BACKEND still override.
• Root cause is a pre-existing MoRI HT (InterNodeV1) internode-dispatch bug at EP≥32 (reproduces on
  stock develop). The low_latency prefill path is the low-risk production fix here. The full HT-kernel
  fix is characterized for the MoRI team separately; open cost is the LL-vs-HT prefill throughput trade,
  quantifiable with a perf sweep.

Test plan

• Build the image from the Dockerfile (WITH_NIXL=0 lean is sufficient for the MoRI-EP path).
• BENCHMARK_SCRIPT=niah, MODEL_NAME=DeepSeek-V3, CONNECTOR=moriio WIDE_EP=1 EP_BACKEND=mori.
• Run 1P/1D, 2P/2D, 4P/4D; confirm NIAH means ≥ baseline and no found=0/10.
• Confirm 4P/4D log shows PREFILL_DP_SIZE=32 >= 32: defaulting prefill backend to mori_low_latency.

🤖 Generated with Claude Code