Skip to content

chore: drop runtime guards that can no longer fire - #568

Merged
CjhHa1 merged 2 commits into
Tencent-Hunyuan:mainfrom
CjhHa1:chore/drop-stale-runtime-guards
Oct 12, 2026
Merged

CjhHa1 merged 2 commits into
Tencent-Hunyuan:mainfrom
CjhHa1:chore/drop-stale-runtime-guards

Conversation

@CjhHa1

@CjhHa1 CjhHa1 commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Follow-up to #560: remove runtime guards that can no longer fire under the current code and dependency pins. Pure deletion (1 line changed, 105 removed).

  • vLLM direct rollout pipe — protocol_version. engine.py and runtime.py each hard-coded _PROTOCOL_VERSION = 1 and checked the other side's value on every request/response. The runtime is a spawn child started from the same checkout and is never respawned once the connection breaks, so the two copies always agree; the check can only ever fail when someone bumps one copy and forgets the other — the same shape as the vLLM version guard fixed in fix(vllm): align direct rollout manifest guard with runtime pin #560. Removed the constant, the field, both checks, and its entry in the driver's reserved-payload set. request_id monotonicity and command validation stay; they catch stale or mismatched responses.
  • Qwen3-Omni — transformers 5.6.0 s_aux shim. _transformers_compat.py only activated when transformers is exactly 5.6.0, whose flash_attention_forward called s_aux.to(...) unconditionally. The repo now pins transformers>=5.12,<5.13 (build(sglang): migrate runtime to 0.5.19 #471), and 5.12.1 already guards s_aux.to(...) with if s_aux is not None. Deleted the module and its call in bundle.py.
  • Qwen3.5 — duplicate transformers floor. bundle.py::_model_class_for repeated the transformers>=5.0 check that validation.py::_MIN_TRANSFORMERS owns; both are below the 5.12 pin. Dropped the bundle copy.

Found while auditing for #560-style drift. The one guard that needs hardware to decide — Cosmos3's H20 cuBLAS workaround, gated to torch 2.10/cu128 while the repo pins torch 2.13/cu130 — is filed as #567 rather than changed here.

Related Issue

N/A (follow-up to #560; related: #567)

Test Plan

  • PYTHONPATH=. python3 /tmp/verify_vllm_pipe_protocol.py (kept outside the repo): sends a driver-shaped request through a multiprocessing.Pipe into runtime._recv_request, replies with runtime._send_response, and reads it with VLLMRolloutEngine._recv — round-trip succeeds without protocol_version; non-monotonic request_id, empty command, and a stale response are still rejected.
  • Compared transformers wheels: 5.6.0 integrations/flash_attention.py:84 calls s_aux.to(query.dtype) unconditionally; 5.12.1 :85-87 guards it with if s_aux is not None.
  • python3 -c "import unirl.models.qwen3_omni.bundle, unirl.models.qwen3_5.bundle" — ok.
  • SKIP=no-commit-to-branch pre-commit run --from-ref upstream/main --to-ref HEAD — pass; git diff --check upstream/main — pass.
  • GPU checks on H20 (gz), exact PR head 7003606, image qwen38next-runtime-base:ubuntu24-cu130-torch213-v2 (torch 2.13.0+cu130, transformers 5.12.1):
    • Qwen3.5-2B via Qwen3_5Bundle.from_config, eager and meta-init: both resolve Qwen3_5ForConditionalGeneration; greedy reply to "12*7" is 84.
    • Qwen3-Omni-30B-A3B-Instruct thinker via Qwen3OmniBundle.from_config with flash_attention_2: loads with 0 missing keys (only talker.*/code2wav.* unexpected, as for a thinker-only load); forward makes 48 flash-attention calls, all with s_aux=None, logits finite; greedy reply starts with 84.
    • examples/ar/qwen3_moe_grpo_30b_a3b_gsm8k_fsdp_vllm_tp4 on Qwen3-30B-A3B-Base, vLLM 0.28.0 TP4 + FSDP, 2 rollouts with one native IPC weight sync between them: exit 0, reward 0.56 → 0.69. The sync woke the weights, reloaded them layer-wise on all 4 TP workers, and committed without error before rollout 2 generated; every driver/runtime command (startup, generate, sleep/wake, weight update, shutdown) went over the pipe without protocol_version.

Compatibility / Risk

No config, checkpoint, or public API change. The driver/runtime pipe is internal to one process tree, so dropping protocol_version has no cross-version compatibility surface. Running Qwen3-Omni on transformers 5.6.0 (outside the declared pin) would hit the upstream s_aux=None bug again.

Reviewer Notes

AI-assisted change.

Checklist

  • I reviewed the changed code and removed unrelated/generated artifacts.
  • I updated tests, docs, and configs where needed, or explained why not.

- vLLM direct rollout: remove protocol_version from the driver/runtime pipe. The runtime is spawned from the same checkout, so the two hard-coded copies always agree and the check only adds a second place to forget on a bump (the same shape as the vLLM version guard fixed in Tencent-Hunyuan#560). request_id and command validation stay.
- Qwen3-Omni: remove the transformers 5.6.0 s_aux shim. transformers is pinned to >=5.12,<5.13, where flash_attention_forward already skips s_aux.to() for None.
- Qwen3.5: drop the bundle's duplicate transformers>=5.0 check; validation.py owns the version floor and the pin is >=5.12.
@github-actions github-actions Bot added the need review Ready and waiting for review label Oct 5, 2026
@CjhHa1
CjhHa1 merged commit 6624b28 into Tencent-Hunyuan:main Oct 12, 2026
5 checks passed
@CjhHa1
CjhHa1 deleted the chore/drop-stale-runtime-guards branch October 12, 2026 06:11
@github-actions github-actions Bot removed the need review Ready and waiting for review label Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant