Skip to content

Remote render node: route ltx-comfy + h3-local to cupcake (GX10) via remoteComfyUrl #86

Description

@taskmasterpeace

Summary

Add first-class remote ComfyUI render node support so DD's local video engines (ltx-comfy, h3-local) can render on cupcake (MKL's GX10 server: GB10, 131GB unified, ARM64, 24/7) instead of the local 4090 — freeing the workstation and unlocking the fp8 quality quant, resident-everything memory policy, and overnight batch.

Proof it already works (2026-08-06/07)

Both clients were driven unmodified against cupcake from the dev box:

  • ComfyLTXClientImpl(comfy_dir=<stub>, base_url="http://192.168.1.249:8188") — a stub dir containing a matching dd_comfy_profile.txt short-circuits _ensure_comfy's process management. Six renders completed through the real client (5s/10s/15s benchmarks + two 15s shots).
  • VideoAssemblerImpl stitched two cupcake shots into a 29.4s sequence (1.2s assembly).

Measured on cupcake (832×480, LTX-2.3 fp8 — the quant the 4090 can't run):

Clip Wall Note
2s 45s cold / 30s warm 4090: 127s cold / 34s warm on nvfp4
5s 52s
10s 93s 4090 nvfp4: ~90–100s — even, on the better quant
15s 153s linear ~10× realtime, no memory cliff

Full numbers + scripts live on cupcake: ~/machine-king-ai-os/benchmarks/video/ and scripts/dd-bench-driver.py. H3 models (int8 DiTs ×2, VAEs, Heretic encoders int8+nvfp4, turbo LoRA) are installed on cupcake as of 2026-08-07.

Design (small, behind the existing Protocols)

  1. Settings: new remoteComfyUrl (string, optional). Empty = current local behavior. Routed into both ComfyLTXClientImpl and ComfyH3LocalClientImpl constructors (both already take base_url).
  2. Process management: when the URL is non-localhost, skip _ensure_comfy launch/kill/profile logic entirely (just health-check /system_stats). The profile-mismatch relaunch dance exists only for 24GB — cupcake holds H3 + LTX resident simultaneously, so model switching costs nothing there.
  3. Input staging: replace _stage_input file-copy with POST /upload/image when remote (reference images, keyframes).
  4. Output retrieval: replace _output_path local-path return with a download via GET /view?filename=&subfolder=&type=output into outputs_dir when remote.
  5. LoRA staging: _ensure_lora_staged file-copy doesn't reach a remote node. Phase 1: document that cupcake keeps its own loras dir (distilled LoRA already there). Phase 2: upload via API or a sync script.
  6. UI: Settings → General field + a warmth-pill variant ("cupcake · remote") so it's honest about where the render runs.
  7. Cupcake-specific config: launch flags are the node's business, not DD's — cupcake's systemd service runs default smart memory (no --disable-smart-memory, no --vram-headroom). Do not send launch flags remotely.

Notes / gotchas already paid for

  • Cupcake ComfyUI is core-only latest master (v0.30+) — all LTX + MiniMax H3 nodes are core; no custom packs needed for base renders. (Multishot pack = separate follow-up.)
  • Cupcake needed python3-dev for Triton JIT (fp4 Gemma) — done.
  • Outputs on cupcake land in /srv/ai/outputs/comfy (policy path, never auto-deleted), not the ComfyUI default dir — /view retrieval is unaffected (type=output follows the configured output dir).
  • Endpoints: LAN http://192.168.1.249:8188, Wi-Fi fallback http://192.168.1.254:8188, tailnet http://100.103.225.77:8188 (works away from home; consider defaulting to the tailnet URL for laptop use).
  • Reference docs on cupcake: ~/machine-king-ai-os/docs/directors-desktop-integration.md. DD project memory: cupcake-render-node.

Acceptance

  • remoteComfyUrl set → an ltx-comfy queue job renders on cupcake and the mp4 appears in DD's Gallery like a local render
  • Same for h3-local incl. a reference-image (omni-ref) job
  • Field empty → behavior identical to today (4090 local path untouched)
  • Cancel works remotely (/interrupt)
  • Suites green: pytest / vitest / pyright 0 / tsc ×2

Follow-ups (separate issues when ready)

  • H3 multishot pack on cupcake (identity-through-seams long takes — the continuity answer)
  • Parallel gpu-slot jobs against cupcake (it has the memory; queue-level change)
  • Directors Palette (cloud) dispatching render jobs to cupcake — needs a Cloudflare Tunnel + service auth in front of the node; NOT LAN-exposure

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions