Skip to content

LTX: stop generating audio in createLtxI2VWorkflow #60

Description

@scottdraves

LTX 2.3 jobs currently generate an audio track we never want, use, or expose. It should be removed from the render path.

Where it is

All of it is in createLtxI2VWorkflow (src/workers/job-handlers.ts). The worker sends the whole ComfyUI graph in the job payload — the container's rp_handler.py requires job_input["workflow"] and hard-errors without it, and the Dockerfile never ADDs workflows/ — so this is a worker-only change for the graph, deployable without a container rebuild.

Audio nodes in the graph:

Node class_type Role
4 VAELoaderKJ loads LTX23_audio_vae_bf16.safetensors
31 LTXVEmptyLatentAudio empty audio latent
33 LTXVConcatAVLatent joins video+audio before pass 1
45 LTXVSeparateAVLatent splits pass 1 output
52 LTXVConcatAVLatent re-joins audio before pass 2
65 LTXVSeparateAVLatent splits pass 2 output
71 LTXVAudioVAEDecode decodes audio
80 VHS_VideoCombine consumes it via audio: ["71", 0]

The open question

LTX-2.3 is natively audio-video joint — both sampler passes run on a concatenated AV latent (3344, 5264). So this is not just deleting nodes; we need to know which of these is true:

  1. ComfyUI-LTXVideo supports a video-only latent path — the sampler accepts a bare video latent and the concat/separate pairs simply drop out. Cleanest outcome.
  2. The transformer requires both streams. Then the AV plumbing stays and we only drop the tail: remove node 71 and the audio input on node 80, so nothing is decoded or muxed. Saves the audio VAE decode and produces a clean silent MP4, but still pays the audio cost during sampling.

Worth checking the Lightricks/ComfyUI-LTXVideo node signatures for an audio-optional mode before assuming (2).

Payoff

  • No unwanted audio track on output
  • Option (1) would cut sampling cost meaningfully — the 22b model's audio branch is not small (4904 of its 7708 tensors are audio-side)
  • Option (2) still saves the audio VAE decode

Follow-up in the container

If (1) works, gpu-container-ltx can also drop the audio VAE download (Dockerfile:132-134, ~365MB) and the VAELoaderKJ dependency. Requires a rebuild + RunPod release, so it should land after the worker change is verified.

Note

Unrelated to, but currently entangled with, the distill-LoRA bug in worker#59 / gpu-container-ltx#2 — that one surfaces as an audio_adaln_single shape error, but it is a model mismatch, not audio generation, and the visible symptom there is video quality. Fix that first so the two are not confused.

🤖 Generated with Claude Code

https://claude.ai/code/session_015Mwf3U49A8BXV8Rp2zQGNB

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    • Status
      Web

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions