Fix Diffusers CPU offload lifecycle - #6
Closed
iamwavecut wants to merge 1 commit into
Closed
iamwavecut wants to merge 1 commit into
iamwavecut wants to merge 1 commit into
Conversation
iamwavecut
force-pushed
the
agent/fix-vae-offload-root
branch
from
July 9, 2026 21:20
de29175 to
a999b14
Compare
iamwavecut
marked this pull request as ready for review
July 9, 2026 21:22
Author
|
Superseded by #14, replayed on the current upstream base with the new FSDP/refiner lifecycle integrated and focused regression tests. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This keeps the fix limited to the three runtime files involved in Diffusers CPU offload:
--cpu_offload {none,model,sequential}for single-process Diffusers inference;text_encoder->transformer->vae;No documentation, dependency, CI, benchmark, or test files are included.
Root causes
meta; their current parameter device is not the VAE execution device.Qwen3VLVisionModelreadspos_embed.weight.devicebefore invoking the childEmbedding. Sequential offload therefore needs Accelerate's intendedpreload_module_classespath for that visual parent instead of excluding the full text encoder from offload.GPU validation
Validated at commit
a999b14on one RTX PRO 4000 Blackwell 24 GB pod, PyTorch 2.8.0/CUDA 12.8, Diffusers 0.39.0, Accelerate 1.14.0, androbbyant/lingbot-video-dense-1.3b. All runs used the same prompts, seeds, dimensions, steps, dtypes, and Torch SDPA backend; only offload mode changed.TI2V, 320x192, 49 frames, 40 steps:
Decoded RGB output from both offload modes is identical to
none: 49 frames, 9,031,680 values compared, maximum absolute difference0.Synthetic dense-refiner smoke, 320x192, 49 frames, 4+4 steps:
Base and refined decoded RGB outputs are identical between model and sequential offload. They are also identical to the pre-fix
de29175model-offload baseline, showing that the lifecycle fix does not change generated pixels.The synthetic refiner
nonecontrol does not fit this 24 GB GPU because it preloads two complete dense pipelines; it reaches 23.41/23.42 GiB before generation. This is a fixture capacity limit, not an offload failure.Additional checks: focused hook/device tests passed externally (10/10), both supported Diffusers/Accelerate API pairs expose the required preload API,
py_compilepasses, andgit diff --checkis clean.