Skip to content

fix(ltx): bake the 2.3/22b distill LoRA, not the 19b one - #2

Merged
mystixxx merged 1 commit into
mainfrom
fix/ltx-distill-lora-2.3
Sep 6, 2026
Merged

mystixxx merged 1 commit into
mainfrom
fix/ltx-distill-lora-2.3

Conversation

@scottdraves

@scottdraves scottdraves commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

The image shipped ltx-2-19b-distilled-lora-384 alongside the ltx-2.3-22b-dev transformer. That is a 19B LoRA on a 22B model, and it is 99% key-compatible — which is the trap, because it loads almost silently.

Reading the safetensors headers directly and matching every key against the deployed transformer:

19B distill LoRA (currently deployed)
  modules: 1371 | bind OK: 1364 (99%) | MISSING: 5 | SHAPE mismatch: 2
    x adaln_single.linear        lora [24576, 4096]  model [36864, 4096]
    x audio_adaln_single.linear  lora [12288, 2048]  model [18432, 2048]

The two that fail are the adaLN modulation layers — the global timestep to shift/scale/gate path that drives every block. ComfyUI logs a shape error for them and applies the other 1364 modules anyway, leaving a half-distilled model: blocks adapted for few-step sampling, driven by unpatched modulation. At 8+3 LCM steps / cfg 1 that degrades the video.

This is the error seen on the running endpoint (12288 × 2048 = 25165824):

ERROR lora diffusion_model.audio_adaln_single.linear.weight
shape [18432, 2048] is invalid for input of size 25165824

Note it reports the audio tensor, but adaln_single.linear (video) fails the same way — it just was not the line that surfaced. The visible symptom is visual, not audio.

Fix

Lightricks/LTX-2.3 ships the matching build. Same comparison:

2.3 22B distill LoRA
  modules: 1660 | bind OK: 1660 (100%) | MISSING: 0 | MISMATCH: 0

The camera LoRAs were never the problem

Worth recording, since this was the standing assumption. All 480 camera-control modules already bind cleanly to the dev transformer:

camera LoRA transformer_blocks: 48 (0..47)
dev 22b   transformer_blocks: 48 (0..47)
modules: 480 | bind OK: 480 | MISSING: 0 | MISMATCH: 0

They contain zero adaLN and zero audio tensors, so they cannot produce the error above. The June switch to the dev transformer was correct; only the distill LoRA that came with it was wrong.

Deploy order

Ship this first — the worker only names the file, the image has to carry it. Container → GHCR → manual RunPod New Release → then worker.

Pairs with worker: e-dream-ai/worker#59

Verification

After release, the shape error should be gone from the ComfyUI log. Then re-test camera motion per direction. This fixes a confirmed breakage, but it is not proven to be the only cause of weak camera motion — if motion is still absent with a clean log, that is a separate bug.

🤖 Generated with Claude Code

https://claude.ai/code/session_015Mwf3U49A8BXV8Rp2zQGNB

The image shipped ltx-2-19b-distilled-lora-384 alongside the
ltx-2.3-22b-dev transformer. Comparing safetensors headers, 1364 of that
LoRA's 1371 modules bind to the 22b model, but the two that fail are the
adaLN modulation layers:

  adaln_single.linear        lora [24576, 4096]  model [36864, 4096]
  audio_adaln_single.linear  lora [12288, 2048]  model [18432, 2048]

ComfyUI logs a shape error for those (12288*2048 = 25165824, the size in
the reported error) and applies the remaining modules anyway, leaving a
half-distilled model: blocks adapted for few-step sampling driven by
unpatched modulation. That degrades the 8+3-step LCM video output.

Lightricks/LTX-2.3 ships the matching build; all 1660 of its modules bind
to this transformer with no missing keys and no shape mismatches.

The camera-control LoRAs are unaffected — all 480 of their modules already
bind cleanly to the dev transformer and contain no adaLN or audio tensors.

Pairs with the worker node-7 change; deploy this first, since the worker
only names the file and the image must actually carry it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Mwf3U49A8BXV8Rp2zQGNB
@mystixxx
mystixxx merged commit ea5f7ab into main Sep 6, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants