feat(rollout): migrate vLLM-Omni to stable 0.28.0 - #413
Merged
Merged
Conversation
… 5.12.1 vllm-omni 0.28.0rc1 requires transformers >=5.10.1,<5.15, above the repo's 5.6.x baseline. The base pin widens to admit both engine forks rather than moving to the new floor, because sglang hard-pins 5.6.0 and raising the base leaves that fork unsolvable; the vllm extra instead narrows itself to an exact 5.12.1. That version is deliberate — sglang 0.5.15+ pins it too, so unifying the two forks later is a bump rather than a second zoo-wide migration. The exact pin also matters because the repo carries no uv.lock, so a range here would follow the next patch release silently. torch is unchanged: vllm 0.28.0 declares the same 2.13.0 as 0.27.0, so the CUDA-13 stack carries over. Core vLLM 0.28 re-slots the reusable MoE workspace by lane within ubatch (ubatch_id * num_lanes + lane) rather than by ubatch alone, so the workspace pool patch now mirrors that arithmetic instead of indexing _current_workspaces directly. Reading num_lanes off the manager and the lane off the ContextVar keeps the old flat layout when either is absent. Nothing exercises lanes today — the diffusion rollout never enables DSpark speculative decoding — but both failure modes here are silent: a stale index writes None over the wrong slot and orphans a tensor that still has live views.
CjhHa1
marked this pull request as ready for review
September 14, 2026 09:09
CjhHa1
requested review from
KemingWu,
haonan3 and
leviking98z-rgb
as code owners
September 14, 2026 18:17
This was referenced Sep 16, 2026
Jayce-Ping
approved these changes
Sep 18, 2026
Jayce-Ping
left a comment
Collaborator
There was a problem hiding this comment.
Most changes are config updates and variable migrations with no blocking risk. GPU test results look good, and there are no conflicts with main. Suggest merging with high priority.
# Conflicts: # INSTALL.md # pyproject.toml
2 tasks
4 tasks
- Describe the split AR/diffusion sleep-wake path in engine docstrings. - Drop the stale sglang-unification rationale from the transformers pin. - Access vLLM 0.28 workspace-lane state directly; negative lanes are rejected by use_workspace_lane itself. - Drop a redundant str() on stage metadata and read stage_configs once. - Rename stage YAML/config references in examples to deploy config. - Hoist distributed_executor_backend to the top level in HI3 deploy configs; resolved stage configs are unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Migrates UniRL from
vllm-omni==0.27.0rc1to stable0.28.0(vllm==0.28.0,transformers==5.12.1,torch==2.13.0+cu130). This is a breaking runtime migration, not only a version pin.stage_configs_path/stage_argsassets with stabledeploy_configYAMLs and registeredPipelineConfigtopologies.OmniRequestOutput.outputscontract and routes AR sleep/wake through EngineCore while retaining diffusion worker task ACKs.mainfrom its explicit vLLM 0.27 runtime contract to 0.28; native IPC, sleep/wake, collective RPC, and cache-reset APIs remain signature-compatible.Dependency overrides remain coherent across mutually exclusive engines: the global uv overrides pin
diffusers==0.40.0(exactly what vllm-omni 0.28.0 requires) andkernels>=0.14.1,<0.15(satisfying vllm-omni'skernels==0.14.1and sglang 0.5.19's range), so both engine stacks share one diffusers/kernels version; installation docs no longer enable prereleases globally.Risk closure
StageConfigFactory; no production reference tostage_configs_path,stage_yaml, orstage_argsremains.engine_output_type=text; terminal think/recaption useslatent, preserving the legacy generation sampler while still returning text.hunyuan_image3_t2t.yaml,hunyuan_image3_i2t.yaml) had neitherenable_loranorworker_extension_cls, soar_lora_passthroughLoRA pushes could not land. The replacement deploy configs (hunyuan_image3_ar_rl.yaml,hunyuan_image3_i2t_rl.yaml) enable LoRA and installHI3ARWeightSyncExtension. This path is not covered by the GPU runs below.Test Plan
Exact stable environment on one 8×H20 node:
Static validation:
Real GPU validation:
Exact-stack 0.27.0rc1 vs 0.28.0 A/B on the same H20 node:
512x512x3: not byte-identical when 0.27 uses its requireddiffusers==0.38.0and 0.28 uses0.40.0; MAE0.4738/255, max channel delta28, 305115/786432 values changed. The earlier MAE-0 result had accidentally put both arms on diffusers 0.40 and is superseded.Qwen3OmniThinkerInputAdapterprompt-token hash across 0.27 sync, 0.28 async, and 0.28 with async execution disabled; all three produced identical 64 tokens, decoded text, and finish reason.prefill_finishedtransfer stops AR after 2 tokens, while the fixed 0.28 topology decodes the requested 128 tokens before DiT handoff.Performance follow-up (one warmup + five measured single-request runs):
0.4057s -> 0.4071s(+0.35%).0.4727s -> 0.4748s(+0.45%).1.3088s -> 1.3148s(+0.46%).0.4012s; 0.28 async0.2406s(-40.0%); 0.28 withasync_scheduling=false0.3307s(-17.6%). Within 0.28, async execution is 27.2% faster and does not change output.1.0466s -> 8.7852s) is not a regression metric because output work changes from 2 to 128 AR tokens.Full public-engine weight/lifecycle E2E:
0.389s, sleep15.603s, wake plus automatic LoRA restore0.612s, checksum equality, post-wake generation0.188s, and full GPU cleanup all pass.Residual risk
handle_sleep_task/handle_wake_task, whose exceptionsStagePoolreturns as{"supported": False, "error": ...}, leaving the engine alive forVLLMOmniEngine's partial-transition retry. This PR routes AR stages through EngineCoresleep/wake_up, whichStagePooltreats as control methods and re-raises, so a failure terminates the orchestrator and reaches the caller as a fatal RPC error (not a hang). The retry therefore cannot recover an AR-stage failure and only keeps it loud; diffusion stages still use worker task ACKs, where the retry remains effective. Upstream semantics; no UniRL change planned here.