From b96ed36591142ffe8022369e19ff74ac8eb60ffb Mon Sep 17 00:00:00 2001 From: Brian Gebel Date: Wed, 27 May 2026 12:31:13 -0700 Subject: [PATCH] fix(core): use --index-url for torch family + parameterise via TORCH_INDEX arg MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ComfyUI's requirements.txt has bare `torch / torchaudio / torchvision` lines with no version pins or wheel URLs. The core Dockerfile used \`--extra-index-url https://download.pytorch.org/whl/cu128\` when installing those, but \`--extra-index-url\` is **additive**: pip picks the highest version found across PyPI + the extra index. When PyPI ships a newer torch than the cu128 index (which happens regularly — e.g. torch 2.12.0 landed on PyPI built for cu130 while the cu128 index still topped out at 2.11.0+cu128), pip picks the PyPI build for torch and the cu128 build for torchaudio (no newer torchaudio on PyPI). Result: torch compiled against cu130 + torchaudio compiled against cu128 → runtime crash on import: RuntimeError: Detected that PyTorch and TorchAudio were compiled with different CUDA versions. PyTorch has CUDA version 13.0 whereas TorchAudio has CUDA version 12.8. The current production image (built before torch 2.12.0 hit PyPI) works by luck; the next clean rebuild — with or without recent extra-requirements changes — would have failed regardless. ## Change Two-stage install in the core layer: 1. \`pip install --index-url https://download.pytorch.org/whl/${TORCH_INDEX} torch torchaudio torchvision\` — replaces PyPI for this command so all three are pulled from the same wheel index with a consistent CUDA ABI. 2. \`pip install -r /app/ComfyUI/requirements.txt\` then \`manager_requirements.txt\` — runs normally; torch family is already satisfied and pip skips re-resolution. Also adds \`ARG TORCH_INDEX=cu128\` parameterised via docker-bake.hcl: - \`core-cuda\` → \`TORCH_INDEX=cu128\` - \`core-cpu\` → \`TORCH_INDEX=cpu\` Adding a future cu130 variant becomes a single bake-target addition with \`args = { TORCH_INDEX = "cu130" }\`, no Dockerfile changes needed. ## Related Supersedes the closed #46 (which targeted the wrong layer with --extra-index-url cu130 in \`complete\` — would have masked the symptom while leaving the index-priority bug unfixed and also unnecessarily locking the image to cu130). Verified by local \`docker buildx bake complete-cuda\` rebuild: produces matching cu128 wheels for all torch-family packages; ComfyUI imports torch/torchaudio without the CUDA-mismatch error. Co-Authored-By: Claude Opus 4.7 (1M context) --- docker-bake.hcl | 2 ++ services/comfy/core/dockerfile.comfy.core | 22 ++++++++++++++++++++-- 2 files changed, 22 insertions(+), 2 deletions(-) diff --git a/docker-bake.hcl b/docker-bake.hcl index 64849a2..79ea180 100644 --- a/docker-bake.hcl +++ b/docker-bake.hcl @@ -67,6 +67,7 @@ target "core-cuda" { cache-to = ["type=inline"] args = { RUNTIME = "cuda" + TORCH_INDEX = "cu128" } depends_on = ["runtime-cuda"] } @@ -90,6 +91,7 @@ target "core-cpu" { cache-to = ["type=inline"] args = { RUNTIME = "cpu" + TORCH_INDEX = "cpu" } depends_on = ["runtime-cpu"] } diff --git a/services/comfy/core/dockerfile.comfy.core b/services/comfy/core/dockerfile.comfy.core index 7a12d10..b053174 100644 --- a/services/comfy/core/dockerfile.comfy.core +++ b/services/comfy/core/dockerfile.comfy.core @@ -21,11 +21,29 @@ RUN umask 022 && \ # We clone into a temporary directory and move to /app/ComfyUI to prevent # permission issues if we were to mount /app entirely in dev modes ARG RUNTIME=cuda + +# CUDA wheel index for the torch family (torch / torchaudio / torchvision). +# Defaults to cu128 for the cuda target; the cpu target overrides this to +# "cpu" via docker-bake.hcl. Future cuda variants (cu130, etc.) can be +# added by introducing a new bake target with `args = { TORCH_INDEX = "cu130" }`. +ARG TORCH_INDEX=cu128 + RUN --mount=type=cache,target=/app/.cache/pip \ umask 022 && \ git clone https://github.com/comfyanonymous/ComfyUI.git /app/ComfyUI && \ - pip install --extra-index-url https://download.pytorch.org/whl/cu128 \ - -r /app/ComfyUI/requirements.txt && \ + # Install torch family from the wheel index ONLY (not --extra-index-url). + # --extra-index-url is additive, so pip picks the highest version across + # PyPI + the index. When PyPI ships a newer torch than the cu* index + # (which happens regularly), pip picks the PyPI build, leaving torch and + # torchaudio compiled against different CUDA versions. Using --index-url + # replaces PyPI for this command, so resolution is constrained to the + # specified index and the CUDA ABI stays consistent across all three. + pip install --index-url https://download.pytorch.org/whl/${TORCH_INDEX} \ + torch torchaudio torchvision && \ + # Install the rest of ComfyUI's requirements normally. torch, torchaudio, + # and torchvision are already installed; pip will see them satisfied and + # skip re-resolution. + pip install -r /app/ComfyUI/requirements.txt && \ pip install -r /app/ComfyUI/manager_requirements.txt # ==================================================================