From images on disk to reproducible experiments, production inference, and reusable clinical applications.
Quickstart · See it on real data · Large images · Bring PyTorch or MONAI · Ship an App · Automate with MCP · KonfAI Studio
KonfAI is a declarative medical-imaging execution engine. It turns a reproducible research workflow into patch-native training, complete medical-image inference, and a reusable application—without giving up the PyTorch and MONAI components you already trust.
One configuration model connects storage, transforms, model graphs, losses, training, prediction, evaluation, and output geometry. The resolved YAML is the experiment record: inspectable, diffable, and runnable by a researcher or an agent.
Trainer:
Model:
classpath: UNet.yml # a model, referenced by name
Dataset:
groups_src: { CT: {...}, SEG: {...} } # channel-first, lazy, patch-based
epochs: 100konfai TRAIN -c Config.yml --gpu 0 # then PREDICTION, then EVALUATION
KonfAI has powered top-ranking MICCAI-challenge results across segmentation, registration, and synthesis: SynthRAD2025 T1 · SynthRAD2025 T2 · CURVAS PDACVI · TrackRAD2025 · Panther · CURVAS
📄 Paper: KonfAI: A Modular and Fully Configurable Framework for Deep Learning in Medical Imaging (Boussot & Dillenseger, 2025)
🤖 Agent-operable. KonfAI ships an MCP server so an LLM agent can drive the entire experiment loop — inspect a dataset, author & validate YAML, launch train / predict / evaluate, monitor jobs, and compare runs — always grounded in the same reproducible configs a human would run. → Agents & MCP
- Evaluating KonfAI? → the App benchmark table below, then the docs landing.
- Running a published model? → the one-command App install (Real workloads).
- Adapting an experiment? → the Quickstart (train → predict → evaluate).
- Building an App? →
konfai-apps. - Contributing? →
AGENTS.md.
- Scale. Dataset patches can be read regionally from ITK, HDF5, DICOM, and OME-Zarr when preprocessing is stream-compatible; a bounded buffer preserves correctness when it is not. Prediction owns batching, TTA, ensembles, reductions, overlap reconstruction, geometry, and output writing.
- Reproduce. Training, prediction, and evaluation share named datasets and inspectable model graphs. Defaults are materialised into the YAML and configs travel with run artifacts.
- Ship and automate. Package the stable workflow as a local, Hugging Face, or remote App, use it from 3D Slicer, or let an MCP client operate the same validated builders and workspaces.
Already use another stack? Keep it. KonfAI can instantiate regular PyTorch and MONAI components, and its catalog includes documented compatibility paths for selected MONAI, nnU-Net, torchvision, and segmentation-models-pytorch models. See when to use KonfAI—or another tool.
| Layer | What KonfAI makes executable |
|---|---|
| Storage and data | Cases, modality groups, geometry, regional reads, transforms, cache/buffer policy, dataset patches |
| Learning | Named model graph, intermediate supervision, losses, metrics, optimizer, schedules, dataset- and model-level patching |
| Inference | Checkpoints, patch batches, TTA, ensembles, reductions, overlap blending, inverse transforms, medical-image outputs |
| Evidence | Resolved configs, checkpoints, statistics, predictions, per-case and aggregate evaluation JSON |
| Delivery | Local/Hugging Face Apps, HTTP jobs, external 3D Slicer client, uncertainty, evaluation, fine-tuning |
| Automation | Dataset inspection, config validation, smoke tests, jobs, metrics, run comparison through MCP |
That vertical integration is the product. YAML is its durable, inspectable interface—not the value proposition by itself.
The same App interface already ships full segmentation, synthesis and registration systems—not reduced demonstration networks:
| App | Workload | Published RTX PRO 5000 benchmark |
|---|---|---|
| TotalSegmentator-KonfAI | CT: 117 labels / 5 models · MRI: 50 labels / 2 models | CT total: ≈42 s / ≈20 GB VRAM / ≈19 GB RAM — 1.5–3.6× faster, 2.7–4.1× less host RAM than the original |
| MRSegmentator-KonfAI | MRI: 40 labels, 5-fold ensemble | ≈27 s / ≈22 GB VRAM — 1.6–2.6× faster, up to ~6× less host RAM than the original |
| ImpactSynth | four MR/CBCT→sCT variants, 2.5D UNet++, 5 models each | ≈24 s / ≈16 GB VRAM for the benchmark inference; ≈82 s full ensemble; ≈2 GB RAM |
| ImpactSeg | one model segments 11 structures from CT, MRI, or CBCT | ≈7 s / ≈10 GB VRAM / ≈1.6 GB RAM |
| IMPACT-Reg | 13 multimodal presets across elastix+IMPACT, ConvexAdam, and FireANTs | ConvexAdam_Composite: ≈5.1 s / ≈2.1 GB VRAM |
These figures retain each bundle's stated case, ensemble and hardware
conditions; they are evidence of executable scale, not a cross-task
leaderboard. The per-app time and RAM ratios come from each bundle's own
small/medium/large benchmark table (see the bundle READMEs under apps/). The bundles share the same App contract across local directories,
Hugging Face and HTTP, with SlicerKonfAI for general Apps and SlicerImpactReg
for dedicated registration.
Consuming a published workflow does not require authoring YAML. Install its
task-specific CLI and run one command, or address the same bundle through
konfai-apps; the complete configuration remains available when you need to
inspect, evaluate, fine-tune, or automate it.
pip install impact_synth_konfai
impact-synth-konfai synthesize MR -i input_mr.nii.gz -o output/
# The same packaged workflow through the generic App runtime
konfai-apps infer VBoussot/ImpactSynth:MR -i input_mr.nii.gz -o output/pip install "konfai[imaging]" # core + all imaging backends (recommended)
pip install konfai # core only (bring your own data reader)[imaging] pulls SimpleITK / h5py / pydicom / zarr — needed to read .mha,
.nii.gz, DICOM, and OME-Zarr. For the full extras matrix (ssim, fid,
lpips, export, cluster, …) and a reproducible Pixi setup, see the
installation guide.
KonfAI is command-driven; each CLI state maps to one YAML file:
| Command | Config | Does |
|---|---|---|
konfai TRAIN / RESUME |
Config.yml (Trainer:) |
fit a model |
konfai PREDICTION |
Prediction.yml (Predictor:) |
patch/TTA/ensemble inference → datasets |
konfai EVALUATION |
Evaluation.yml (Evaluator:) |
metrics on saved predictions |
Full CLI reference (flags, konfai-cluster, konfai-apps):
docs/reference/cli.
git clone https://github.com/fideus-labs/KonfAI.git && cd KonfAI
pip install -e ".[imaging]"
cd examples/Segmentation
# download the small public demo dataset
pip install -U "huggingface_hub[cli]"
hf download VBoussot/konfai-demo --repo-type dataset --include "Segmentation/**" --local-dir Dataset
mv Dataset/Segmentation/* Dataset/ && rmdir Dataset/Segmentation && rm -rf Dataset/.cache
# For a short smoke run, first set `epochs: 1` in Config.yml.
konfai TRAIN -y --gpu 0 --config Config.yml # use --cpu 1 if you have no GPU💡 After a run,
Config.ymlwill contain the resolved defaults KonfAI materialised — that's expected, and it's what makes runs reproducible.
Runtime depends on dataset size, hardware, and whether you keep the shipped 100-epoch setting. The one-epoch edit checks the complete path quickly; it is not intended to produce a useful checkpoint.
The full walkthrough (predict, evaluate, what to inspect, common first issues, notebook entry points) lives in the Quickstart.
Volumes are read as patches. Whether the volume is also held in RAM depends on
the workflow's loading regime (training caches, prediction and evaluation
stream; memory_budget makes it adaptive) and on whether your preprocessing
chain can be streamed — KonfAI derives streamability from the transforms you
declared:
| Regime | When | Memory held |
|---|---|---|
| Cache | training default | every case, resident for the whole run |
| Stream | predict/eval default or budget exceeded; chain streamable | one patch |
| Buffer | same triggers; chain not streamable | a FIFO of max(batch_size + 1, shuffle_window) cases |
A chain streams when every step declares the region it needs: the exact patch
(OneHot), a halo (Dilate), a remap (Flip), a resample (ResampleToShape),
or a whole-volume statistic read once from disk (Normalize). On the stream
path, a 16 GiB uncompressed .mha trains at patch 64³ under an 8 GiB memory cap
with a peak resident set of 0.46 GiB.
→ Patch streaming — what streams, what does not, and why.
Everything below is referenceable by name in YAML. See the built-in component catalogue for classpaths and constructor arguments.
| Kind | Examples | Catalogue |
|---|---|---|
| Models | UNet, NestedUNet, ResNet, VAE, VoxelMorph, GAN/diffusion families |
models |
| Losses & metrics | Dice, MAE, PSNR, SSIM, LPIPS, FID, CrossEntropyLoss, TRE, IMPACTReg, IMPACTSynth |
losses-metrics |
| Transforms | Standardize, Normalize, Clip, Resample*, OneHot, Crop (~40) |
transforms |
| Augmentations | Flip, Rotate, Elastix, Noise, CutOUT (~15) |
augmentations |
| Schedulers | weight (Constant, CosineAnnealing) + LR (PolyLRScheduler, Warmup, any torch) |
schedulers |
| Storage backends | ITK, HDF5, DICOM series, OME-Zarr | storage-backends |
Not limited to these: any importable class (monai.losses:DiceLoss,
torch:nn:L1Loss, or a local Model:MyNet) works via the module:Class form.
KonfAI is built to serve as a deterministic backend for LLM-driven experimentation. Through the KonfAI-MCP server, an agent can:
- 🔎 inspect datasets and infer their structure
- 📝 generate and validate YAML configurations
- 🚀 launch training / prediction / evaluation runs
- 📈 read live metrics, compare runs, and iterate
Every execution stays reproducible, structured, and grounded in the same YAML workflows a human would run — bridging LLM reasoning and real experimental execution. See the ecosystem map for the current status.
KonfAI Studio is
a single chatbot web UI over the MCP server. Point it at your own dataset and,
from the conversation alone, inspect the data, author or reuse a model, train,
predict, evaluate, compare runs, and view the volumes in a built-in NiiVue
viewer — every step a konfai-mcp tool call, the compute staying on your
machine. It is a product surface over konfai-mcp, not a new engine, and it
ships no API key: you bring your own LLM — your Claude Code subscription by
default, the Claude API with your key, or a fully local OpenAI-compatible server
such as Ollama or vLLM.
pip install konfai-studio
konfai-studio # -> http://127.0.0.1:8730| Package | What it is |
|---|---|
konfai |
the core framework (this repo) |
konfai-apps |
package a workflow as an app — CLI, HTTP server, Python API |
App bundles (apps/) |
ready-to-run: impact-synth, impact-seg, mrsegmentator, totalsegmentator, impact-reg |
| SlicerKonfAI | run segmentation, synthesis, evaluation, and uncertainty Apps from 3D Slicer |
| SlicerImpactReg | run IMPACT-Reg presets and inspect registration results in 3D Slicer |
| KonfAI-MCP | expose KonfAI to LLM agents — inspect data, author configs, launch and monitor runs |
| KonfAI Studio | a chat web UI over konfai-mcp — inspect data, train, predict, evaluate, and compare from one conversation |
See the ecosystem map for what is shipped vs. in-progress.
📚 Full docs: https://konfai.readthedocs.io/en/latest/
- Quickstart — first end-to-end run
- Core concepts — how YAML becomes Python objects
- Large images — regional reads, fallback, and tuning
- Adopt from PyTorch/MONAI — reuse and tool choice
- Component catalogue — everything you can configure
- Examples — runnable Segmentation & Synthesis workflows
🐳 Docker: vboussot/konfai —
guide.
git clone https://github.com/fideus-labs/KonfAI.git && cd KonfAI
pixi install
pixi run test # run the test suite
pixi run check # lint + format-check + test (run before pushing)Contributions are welcome — improve examples, clarify docs, add tests, or extend models / transforms / apps. See the developer guide.
AI coding agents: start with AGENTS.md — the canonical
reference for conventions, commands, and repository rules.
@article{boussot2025konfai,
title = {KonfAI: A Modular and Fully Configurable Framework for Deep Learning in Medical Imaging},
author = {Boussot, Valentin and Dillenseger, Jean-Louis},
journal = {arXiv preprint arXiv:2508.09823},
year = {2025}
}Licensed under Apache-2.0.