fix(ltx): bake the 2.3/22b distill LoRA, not the 19b one - #2
Merged
Merged
Conversation
The image shipped ltx-2-19b-distilled-lora-384 alongside the ltx-2.3-22b-dev transformer. Comparing safetensors headers, 1364 of that LoRA's 1371 modules bind to the 22b model, but the two that fail are the adaLN modulation layers: adaln_single.linear lora [24576, 4096] model [36864, 4096] audio_adaln_single.linear lora [12288, 2048] model [18432, 2048] ComfyUI logs a shape error for those (12288*2048 = 25165824, the size in the reported error) and applies the remaining modules anyway, leaving a half-distilled model: blocks adapted for few-step sampling driven by unpatched modulation. That degrades the 8+3-step LCM video output. Lightricks/LTX-2.3 ships the matching build; all 1660 of its modules bind to this transformer with no missing keys and no shape mismatches. The camera-control LoRAs are unaffected — all 480 of their modules already bind cleanly to the dev transformer and contain no adaLN or audio tensors. Pairs with the worker node-7 change; deploy this first, since the worker only names the file and the image must actually carry it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Mwf3U49A8BXV8Rp2zQGNB
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The image shipped
ltx-2-19b-distilled-lora-384alongside theltx-2.3-22b-devtransformer. That is a 19B LoRA on a 22B model, and it is 99% key-compatible — which is the trap, because it loads almost silently.Reading the safetensors headers directly and matching every key against the deployed transformer:
The two that fail are the adaLN modulation layers — the global timestep to shift/scale/gate path that drives every block. ComfyUI logs a shape error for them and applies the other 1364 modules anyway, leaving a half-distilled model: blocks adapted for few-step sampling, driven by unpatched modulation. At 8+3 LCM steps / cfg 1 that degrades the video.
This is the error seen on the running endpoint (12288 × 2048 = 25165824):
Note it reports the audio tensor, but
adaln_single.linear(video) fails the same way — it just was not the line that surfaced. The visible symptom is visual, not audio.Fix
Lightricks/LTX-2.3ships the matching build. Same comparison:The camera LoRAs were never the problem
Worth recording, since this was the standing assumption. All 480 camera-control modules already bind cleanly to the dev transformer:
They contain zero adaLN and zero audio tensors, so they cannot produce the error above. The June switch to the dev transformer was correct; only the distill LoRA that came with it was wrong.
Deploy order
Ship this first — the worker only names the file, the image has to carry it. Container → GHCR → manual RunPod New Release → then worker.
Pairs with worker: e-dream-ai/worker#59
Verification
After release, the shape error should be gone from the ComfyUI log. Then re-test camera motion per direction. This fixes a confirmed breakage, but it is not proven to be the only cause of weak camera motion — if motion is still absent with a clean log, that is a separate bug.
🤖 Generated with Claude Code
https://claude.ai/code/session_015Mwf3U49A8BXV8Rp2zQGNB