ComfyUI-ONNX (ONNX Image Filter)

Apply arbitrary ONNX image-processing models to images inside ComfyUI.

Models under ComfyUI/models/onnx/ (including nested folders) appear in a dropdown — no manual path editing.

---

Features

• Recursive discovery of .onnx files under models/onnx/
• Automatic ONNX graph inspection (input/output names, shapes, dtypes, NCHW/NHWC)
• RGB / RGBA / grayscale handling with automatic conversion and alpha restore
• uint8 / float16 / float32 IO with normalize-only-when-needed
• Dynamic spatial / batch shapes
• Image sequences / batches (B,H,W,C) processed frame-wise or in chunks
• Optional tiling with overlap blending
• CUDA-first provider selection with automatic CPU fallback
• Cached InferenceSession reuse
• Favorites, live metadata preview, and optional benchmark node

---

Installation

1. Copy or clone this repository into ComfyUI's custom nodes folder:

ComfyUI/custom_nodes/ComfyUI-ONNX/

2. Install Python dependencies into the same environment ComfyUI uses:

cd ComfyUI/custom_nodes/ComfyUI-ONNX
pip install -r requirements.txt

3. (Optional, GPU) Install the CUDA build of ONNX Runtime instead of / in addition to the CPU wheel:

pip uninstall -y onnxruntime
pip install onnxruntime-gpu>=1.18

4. Create the models folder (created automatically on first load) and add models:

ComfyUI/models/onnx/
├── Anime/anime_v1.onnx
├── Upscale/realesrgan.onnx
└── Stylization/monet.onnx

5. Restart ComfyUI.

---

Dependencies

Package	Required	Notes
onnxruntime	Yes (CPU)	>= 1.18
onnxruntime-gpu	Optional	CUDA 12.x builds recommended
onnx	Yes	Graph / opset / parameter inspection
numpy	Yes
torch	Yes	Already provided by ComfyUI
Pillow	Yes	Already provided by ComfyUI

Supported platforms: Windows, Linux, Python 3.10+, ComfyUI latest.

---

Nodes

ONNX Image Filter (Image/ONNX)

Input	Description
IMAGE	Input image
MODEL	Dropdown of discovered ONNX models
DEVICE	Auto / CUDA / CPU
Provider	Auto / CUDAExecutionProvider / CPUExecutionProvider
Refresh Models	Rescan models/onnx/ (then refresh node definitions / page to update the dropdown)
Tile Size	0 = disabled; otherwise split into tiles
Overlap	Tile overlap (default 32)
FP16	Prefer half precision when supported
Batch Size	Frames per ONNX run for sequences (default 1 = one frame at a time)
Favorite	Mark the current model as a favorite
Run Benchmark	Append benchmark stats to metadata

Output	Description
IMAGE	Processed image
metadata	Model summary (shapes, provider, opset, timings, …)

ONNX Model Favorite

Add / remove favorites. Favorites appear at the top of the dropdown with a ★ prefix.

ONNX Model Benchmark

Runs a synthetic benchmark and reports average time, FPS, provider, and CPU vs GPU speed when possible.

---

Example Workflow

Load Image
    │
    ▼
ONNX Image Filter
    │
    ▼
Save Image

1. Add Load Image
2. Add ONNX Image Filter (Image/ONNX)
3. Select a model from the dropdown
4. Connect to Save Image / Preview Image

Connect the metadata output to a Show Text / Preview Any node if you want the inspection summary visible on the canvas.

---

Screenshots

Place screenshots in docs/ if you publish this package:

• docs/node.png — the ONNX Image Filter node
• docs/workflow.png — Load Image → ONNX Image Filter → Save Image
• docs/models_folder.png — example models/onnx/ layout

---

Provider / CUDA Notes

Preferred order when Device and Provider are Auto:

1. CUDAExecutionProvider
2. CPUExecutionProvider

After session creation the node verifies that CUDA actually activated. ORT can silently ignore a broken CUDA install; when that happens the node logs a warning and continues on CPU.

If CUDA is unavailable, the node falls back to CPU and continues. It will not crash solely because GPU ORT is missing.

CUDA sessions use ComfyUI/PyTorch's current GPU index (device_id) and standard ORT options (arena_extend_strategy, cudnn_conv_use_max_workspace, do_copy_in_default_stream).

CUDA installation (short)

1. Install a CUDA 12.x-compatible GPU driver.
2. Install onnxruntime-gpu>=1.18 into the ComfyUI Python environment (do not keep the CPU-only onnxruntime wheel alongside it).
3. Confirm providers in a Python shell:

import onnxruntime as ort
print(ort.get_available_providers())

You should see CUDAExecutionProvider listed when the GPU build is working.

CPU fallback

• Set Device to CPU, or Provider to CPUExecutionProvider
• Or simply leave Auto and let the node fall back when CUDA is missing

FP16

The FP16 checkbox uses float16 feeds only when the ONNX model input is already float16. It does not convert float32 models. For half-precision IO, export or download a float16 ONNX graph.

---

How model inspection works

The node does not assume fixed input names, layouts, or dtypes. On load it inspects the ONNX graph / ORT session and detects:

• Input / output tensor names
• Datatypes (uint8, float16, float32, …)
• Shape + dynamic axes
• Channel count and layout (NCHW / NHWC)
• Whether values are likely 0–255 vs 0–1

ComfyUI images (B,H,W,C float [0,1]) are converted automatically and converted back after inference.

---

Tiling

Set Tile Size > 0 to process large images in overlapping tiles. Overlaps are feather-blended to reduce seams. Useful for high-resolution inputs or memory-constrained GPUs.

---

Logging

Each run prints a console summary similar to:

[ComfyUI-ONNX] Loaded model: .../models/onnx/Anime/anime_v1.onnx
[ComfyUI-ONNX] Provider: CUDAExecutionProvider (device 0)
[ComfyUI-ONNX] Active providers: ['CUDAExecutionProvider', 'CPUExecutionProvider']
[ComfyUI-ONNX] Input: input (1, 3, 512, 512) (1 frame(s))
[ComfyUI-ONNX] Output: output (1, 3, ?, ?)
[ComfyUI-ONNX] Datatype: float32 -> float32
[ComfyUI-ONNX] Resolution: 1x3x512x512
[ComfyUI-ONNX] Inference time: 0.0421s

Enable full Python tracebacks with:

set COMFYUI_ONNX_DEBUG=1

---

Troubleshooting

Symptom	Fix
(no onnx models found)	Put .onnx files under ComfyUI/models/onnx/, toggle Refresh Models, then refresh the browser / node definitions
CUDA provider unavailable	Install onnxruntime-gpu, or switch Device/Provider to CPU
Model has multiple inputs	Only single-image-input models are supported
Unsupported tensor shape	Model input/output is not rank 3/4 image-like
Input channels mismatch	Model expects a channel count that cannot be derived from RGB/RGBA/GRAY
Invalid ONNX graph	Re-export / validate the model with the onnx package
Dropdown not updating	Refresh Models clears the scan cache; reload the UI so INPUT_TYPES is re-evaluated
Slow CPU runs	Expected without onnxruntime-gpu; try tiling to reduce peak memory on GPU

---

FAQ

Q: Where do I put models?
ComfyUI/models/onnx/ — nested folders are fine. Dropdown labels use relative paths such as Upscale/realesrgan.onnx.

Q: Do I need to edit paths in the node?
No. Discovery is automatic.

Q: Can I search hundreds of models?
Use ComfyUI's built-in combo filter (type to filter). Favorites float matching models to the top.

Q: Are sessions reloaded every run?
No. Sessions are cached by model path + provider + FP16 preference and reused until ComfyUI exits.

Q: Does FP16 always help?
Only when the ONNX model itself uses float16 IO. Float32 models stay float32; enable CUDA via onnxruntime-gpu for GPU float32 acceleration.

Q: Multi-input / multi-output models?
Multiple image inputs are rejected with a clear error. Multiple outputs: the first image-like output is used.

---

Stretch goals included

• Automatic metadata display (metadata output + console logs)
• Live preview panel when the model changes (web extension)
• Favorites (★ prefix + Favorite toggle / helper node)
• Combo search via ComfyUI's native filterable dropdowns
• Benchmark button / dedicated benchmark node (avg time, FPS, CPU vs GPU)

---

License

MIT — see LICENSE.