trtllm diffusion: dual guidance (guidance_scale_2 / boundary_ratio) for Wan 2.2#25
Draft
Johan-de-R wants to merge 3 commits into
Draft
trtllm diffusion: dual guidance (guidance_scale_2 / boundary_ratio) for Wan 2.2#25Johan-de-R wants to merge 3 commits into
Johan-de-R wants to merge 3 commits into
Conversation
…onfig Wan 2.2 A14B is a two-expert MoE that needs dual guidance (e.g. 4.0/3.0 with a 0.875 timestep boundary). The diffusion worker only exposed a single --default-guidance-scale; guidance_scale_2 / boundary_ratio (Wan extra_param_specs) defaulted to None => single guidance. Add: - DiffusionConfig.default_guidance_scale_2 / default_boundary_ratio + the CLI flags --default-guidance-scale-2 / --default-boundary-ratio (backend_args). - DiffusionEngine.generate() injects them into req.params.extra_params BEFORE the spec-default merge so they take effect. - video_handler forwards the config defaults (config-only; the /v1/videos nvext doesn't carry them). Both default to None, so existing models (Flux, Wan single-guidance) are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The video handler defaulted response_format to "url", which uploads the clip to media_output_fs_url and returns a file reference. deepinfra's deepapi reads the video from data[0].b64_json and never sends response_format, so it received no video (ERR_MODEL "No video data"). Default to b64_json so the clip is returned inline; callers can still pass response_format="url" explicitly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Johan-de-R
had a problem deploying
to
external_collaborator
July 24, 2026 21:46 — with 
GitHub Actions
Failure
GitHub Actions
Failure
…ver guidance forwarding The b64_json default change flipped VideoGenerationHandler's default from "url" to "b64_json", which broke test_default_response_format_is_url (it asserted upload_to_fs was called and data[0].url was set). Rewrote it to test_default_response_format_is_b64_json (no upload; data[0].b64_json set) so the new default is pinned. Also added test_guidance_defaults_forwarded_to_engine: the handler must forward config.default_guidance_scale_2 / default_boundary_ratio to engine.generate() (dual guidance isn't carried in the /v1/videos nvext) -- the core of this PR had no test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Johan-de-R
had a problem deploying
to
external_collaborator
July 25, 2026 03:57 — with
GitHub Actions
Failure
GitHub Actions
Failure
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two changes to the TRT-LLM diffusion (VisualGen) backend so deepinfra can serve
Wan-AI/Wan2.2-T2V-A14B:--default-guidance-scale;guidance_scale_2/boundary_ratio(Wanextra_param_specs) defaulted toNone→ single guidance.b64_json— the video handler defaultedresponse_formatto"url"(upload tomedia_output_fs_url, return a file reference). deepinfra's deepapi reads the clip fromdata[0].b64_jsonand never sendsresponse_format, so it got no video (ERR_MODEL "No video data"). Now defaults tob64_json; callers can still passresponse_format="url"explicitly.Changes
configs/diffusion_config.py+backend_args.py:default_guidance_scale_2/default_boundary_ratiofields +--default-guidance-scale-2/--default-boundary-ratioflags (both defaultNone).engines/diffusion_engine.py:generate()injects them intoreq.params.extra_paramsbefore the spec-default merge, so they override theNonespec default.request_handlers/diffusion/video_handler.py: forwards the guidance config defaults (config-only — the/v1/videosnvext doesn't carry them); and defaultsresponse_formattob64_json.Both guidance defaults are
None, so Flux and single-guidance Wan are unaffected. Theb64_jsondefault only changes behavior when a caller omitsresponse_format(deepinfra's case).Why
deepinfra is serving
Wan-AI/Wan2.2-T2V-A14Bon this backend; goff (4.0/3.0) tested more coherent than single guidance. Paired backend PRs: deepinfra/backend#3759 (infra plumbing) + ai-dynamo#3717 (deepapi cutover).Validation
py_compileclean. Correction to an earlier claim: the b64 default did breaktest_default_response_format_is_url(it assertedupload_to_fswas called anddata[0].urlwas set) — rewrote it totest_default_response_format_is_b64_json, and addedtest_guidance_defaults_forwarded_to_engine(the guidance forwarding had no coverage). The dynamo suite runs in CI (pytest 8.0); it isn't runnable in the backend's di-main env, so these are py_compile-verified + correct-by-construction (mirror the existing handler tests). End-to-end serving smoke to be re-run after the worker image is rebuilt with these changes.🤖 Generated with Claude Code