Add split-device text encoder routing

README.md

@@ -154,6 +154,95 @@ under `src/ideogram4/` take effect without reinstalling:
 pip install -e .
 ```
 
+### Experimental split-device inference
+
+`Ideogram4Pipeline.from_pretrained` accepts an optional `text_device`. This
+keeps Qwen3-VL on that device while the conditional and unconditional diffusion
+transformers, VAE, sampling state, and decode state remain on `device`. If
+`text_device` is omitted, all components use `device` exactly as before.
+
+```python
+import torch
+
+pipeline = Ideogram4Pipeline.from_pretrained(
+  device="cuda:0",
+  text_device="cuda:1",
+  dtype=torch.bfloat16,
+)
+```
+
+On split devices, Qwen runs once over the left-padded text block only. The
+resulting float32 text features are copied once to the diffusion device before
+sampling, and image-position zeros are allocated there. There are no planned
+cross-device transfers in the denoising loop. A direct GPU transfer is tried
+first; runtimes without peer transfer support fall back to CPU staging with a
+warning.
+
+AMD ROCm multi-GPU support is experimental. PyTorch ROCm still uses the
+`cuda:N` device spelling. Use explicit indexes for split placement and verify
+the runtime mapping rather than assuming device order:
+
+```bash
+python scripts/smoke_split_devices.py \
+  --caption-file /path/to/structured-caption.json \
+  --output /tmp/ideogram4-smoke.png \
+  --diffusion-device cuda:0 \
+  --text-device cuda:1 \
+  --height 256 --width 256 --num-steps 2
+```
+
+The smoke script prints visible devices, peer-access status, and peak allocated
+and reserved memory. It does not call a hosted Magic Prompt API.
+
+After the small smoke passes, run the acceptance configuration with the
+official 20-step sampler schedule:
+
+```bash
+python scripts/smoke_split_devices.py \
+  --caption-file /path/to/structured-caption.json \
+  --output /tmp/ideogram4-1024.png \
+  --diffusion-device cuda:0 \
+  --text-device cuda:1 \
+  --height 1024 --width 1024 \
+  --sampler-preset V4_DEFAULT_20
+```
+
+For the target Fedora/ROCm machine, the validation wrapper performs the full
+preflight, installs this checkout editable without replacing the existing ROCm
+PyTorch build, records `rocm-smi` telemetry, runs both core tests above, and can
+then launch ComfyUI:
+
+From the ComfyUI directory, this one command safely clones or updates both
+feature branches, creates a local structured test caption, runs the complete
+validator, and launches ComfyUI:
+
+```bash
+bash <(curl -fsSL https://raw.githubusercontent.com/Dillflix/ideogram4/feature/split-text-diffusion-devices/scripts/fedora_one_step.sh)
+```
+
+If ComfyUI is elsewhere, identify it on the same command:
+
+```bash
+COMFYUI_DIR=/absolute/path/to/ComfyUI bash <(curl -fsSL https://raw.githubusercontent.com/Dillflix/ideogram4/feature/split-text-diffusion-devices/scripts/fedora_one_step.sh)
+```
+
+The lower-level equivalent, useful when the repositories are already prepared,
+is:
+
+```bash
+chmod +x scripts/validate_rocm_split_devices.sh
+scripts/validate_rocm_split_devices.sh \
+  --caption /path/to/structured-caption.json \
+  --comfyui-dir /absolute/path/to/ComfyUI \
+  --launch-comfyui
+```
+
+The script fails before loading weights if the resolved device names do not
+contain `7900 XT` for the diffusion role and `8060S` for the text role. Use its
+explicit device/name options if the runtime mapping differs. Logs, peak-memory
+reports, generated images, and GPU telemetry are written to a timestamped
+validation directory.
+
 ### Model access
 
 The model weights are **gated** on Hugging Face, so you must accept the gate and

pyproject.toml

@@ -56,6 +56,10 @@ packages = ["src/ideogram4"]
 include = [
   "src/ideogram4",
   "src/ideogram4/magic_prompt_system_prompts/*.txt",
+  "scripts/fedora_one_step.sh",
+  "scripts/smoke_split_devices.py",
+  "scripts/validate_rocm_split_devices.sh",
+  "tests",
   "run_inference.py",
   "README.md",
   "pyproject.toml",

scripts/fedora_one_step.sh

@@ -0,0 +1,167 @@
+#!/usr/bin/env bash
+
+set -Eeuo pipefail
+
+CORE_REPO="https://github.com/Dillflix/ideogram4.git"
+CORE_BRANCH="feature/split-text-diffusion-devices"
+WRAPPER_REPO="https://github.com/Dillflix/ComfyUI-Ideogram4.git"
+WRAPPER_BRANCH="feature/multigpu-device-routing"
+COMFYUI_REPO="https://github.com/comfyanonymous/ComfyUI.git"
+REMOTE_NAME="dillflix-validation"
+STATE_ROOT="${XDG_DATA_HOME:-$HOME/.local/share}/ideogram4-split-validation"
+CORE_DIR="$STATE_ROOT/ideogram4"
+
+fail() {
+  echo "ERROR: $*" >&2
+  exit 1
+}
+
+is_comfyui_root() {
+  [[ -f "$1/main.py" && -f "$1/nodes.py" && -d "$1/comfy" ]]
+}
+
+find_comfyui() {
+  local candidate venv_parent
+  local -a candidates=()
+  if [[ -n "${COMFYUI_DIR:-}" ]]; then
+    candidate="$COMFYUI_DIR"
+    is_comfyui_root "$candidate" || fail "COMFYUI_DIR is not a ComfyUI checkout: $candidate"
+    readlink -f -- "$candidate"
+    return
+  fi
+
+  if [[ -n "${VIRTUAL_ENV:-}" ]]; then
+    venv_parent="$(dirname -- "$VIRTUAL_ENV")"
+    candidates+=("$venv_parent" "$(dirname -- "$venv_parent")")
+  fi
+  candidates+=("$PWD" "$HOME/ComfyUI" "$HOME/comfyui")
+
+  for candidate in "${candidates[@]}"; do
+    if is_comfyui_root "$candidate"; then
+      readlink -f -- "$candidate"
+      return
+    fi
+  done
+
+  while IFS= read -r candidate; do
+    candidate="$(dirname -- "$candidate")"
+    if is_comfyui_root "$candidate"; then
+      echo "$candidate"
+      return
+    fi
+  done < <(
+    find "$HOME" -maxdepth 8 -type f -name main.py -ipath '*comfyui*' \
+      -print 2>/dev/null | sort
+  )
+
+  candidate="$STATE_ROOT/ComfyUI"
+  echo "Existing ComfyUI checkout not found; cloning it into $candidate" >&2
+  if [[ ! -d "$candidate/.git" ]]; then
+    mkdir -p -- "$(dirname -- "$candidate")"
+    git clone "$COMFYUI_REPO" "$candidate" >&2
+  fi
+  readlink -f -- "$candidate"
+}
+
+checkout_branch() {
+  local repo_url="$1"
+  local branch="$2"
+  local destination="$3"
+
+  if [[ ! -d "$destination/.git" ]]; then
+    if [[ -e "$destination" && -n "$(find "$destination" -mindepth 1 -maxdepth 1 -print -quit 2>/dev/null)" ]]; then
+      fail "cannot clone into non-empty directory: $destination"
+    fi
+    mkdir -p -- "$(dirname -- "$destination")"
+    git clone --branch "$branch" --single-branch "$repo_url" "$destination"
+    return
+  fi
+
+  if [[ -n "$(git -C "$destination" status --porcelain)" ]]; then
+    fail "existing checkout has local changes; refusing to switch it: $destination"
+  fi
+
+  if git -C "$destination" remote get-url "$REMOTE_NAME" >/dev/null 2>&1; then
+    git -C "$destination" remote set-url "$REMOTE_NAME" "$repo_url"
+  else
+    git -C "$destination" remote add "$REMOTE_NAME" "$repo_url"
+  fi
+  git -C "$destination" fetch "$REMOTE_NAME" "$branch"
+
+  if git -C "$destination" show-ref --verify --quiet "refs/heads/$branch"; then
+    git -C "$destination" switch "$branch"
+  else
+    git -C "$destination" switch --create "$branch" --track "$REMOTE_NAME/$branch"
+  fi
+  git -C "$destination" merge --ff-only "$REMOTE_NAME/$branch"
+}
+
+command -v git >/dev/null 2>&1 || fail "git is required"
+[[ "$(uname -s)" == "Linux" ]] || fail "this launcher must run on Linux"
+
+COMFYUI_DIR="$(find_comfyui)"
+WRAPPER_DIR="$COMFYUI_DIR/custom_nodes/ComfyUI-Ideogram4"
+mkdir -p -- "$STATE_ROOT"
+
+echo "ComfyUI: $COMFYUI_DIR"
+echo "Validation workspace: $STATE_ROOT"
+echo "Preparing modified core..."
+checkout_branch "$CORE_REPO" "$CORE_BRANCH" "$CORE_DIR"
+echo "Preparing modified ComfyUI wrapper..."
+checkout_branch "$WRAPPER_REPO" "$WRAPPER_BRANCH" "$WRAPPER_DIR"
+
+CAPTION_FILE="$STATE_ROOT/validation-caption.json"
+cat >"$CAPTION_FILE" <<'JSON'
+{
+  "high_level_description": "A clean technical poster celebrating a dual-GPU image generation workstation.",
+  "style_description": {
+    "aesthetics": "precise, modern, polished, high contrast",
+    "lighting": "soft studio lighting with subtle cyan and amber rim light",
+    "medium": "graphic_design",
+    "art_style": "minimal technical poster, crisp geometric forms, premium product visualization",
+    "color_palette": ["#101820", "#00AEEF", "#FFB000", "#F5F7FA"]
+  },
+  "compositional_deconstruction": {
+    "background": "A deep charcoal studio backdrop with a faint grid and restrained cyan highlights.",
+    "elements": [
+      {
+        "type": "obj",
+        "bbox": [180, 120, 820, 880],
+        "desc": "Two elegant abstract GPU modules connected by a single luminous data arc, arranged symmetrically as a premium technical product composition."
+      },
+      {
+        "type": "text",
+        "bbox": [70, 180, 180, 820],
+        "text": "SPLIT COMPUTE",
+        "desc": "Large crisp uppercase geometric sans-serif title in white."
+      }
+    ]
+  }
+}
+JSON
+
+echo "Starting automated ROCm validation."
+echo "This downloads gated NF4 weights if they are not already cached."
+echo "Core and ComfyUI API tests run automatically; the validated server then remains running."
+
+VALIDATOR_ARGS=(
+  --caption "$CAPTION_FILE"
+  --comfyui-dir "$COMFYUI_DIR"
+  --launch-comfyui
+)
+
+if [[ -n "${VIRTUAL_ENV:-}" && -x "$VIRTUAL_ENV/bin/python" ]]; then
+  VALIDATOR_ARGS+=(--python "$VIRTUAL_ENV/bin/python")
+fi
+
+if [[ -n "${IDEOGRAM4_VALIDATION_OUTPUT:-}" ]]; then
+  VALIDATOR_ARGS+=(--output-dir "$IDEOGRAM4_VALIDATION_OUTPUT")
+fi
+if [[ -n "${IDEOGRAM4_DIFFUSION_DEVICE:-}" ]]; then
+  VALIDATOR_ARGS+=(--diffusion-device "$IDEOGRAM4_DIFFUSION_DEVICE")
+fi
+if [[ -n "${IDEOGRAM4_TEXT_DEVICE:-}" ]]; then
+  VALIDATOR_ARGS+=(--text-device "$IDEOGRAM4_TEXT_DEVICE")
+fi
+
+exec "$CORE_DIR/scripts/validate_rocm_split_devices.sh" "${VALIDATOR_ARGS[@]}"