This use-case demo illustrates DLMO (Deep Learning Model Observer)-based assessment on a discrimination task as a surrogate for MRI resolution, as detailed in our DLMO paper. The implementation simulates multi-coil, sensitivity-encoded parallel MR imaging at varying acceleration factors, using fully sampled root of sum of squares (rSOS) reconstruction as the reference and U-Net post-processing of undersampled reconstructions as the test method.
The signal content can be adapted to a signal-present versus signal-absent configuration for detection tasks, as described in the CNN-IO paper. The same DLMO framework can also be applied to other acquisition settings and reconstruction methods.
To run the DLMO codes, begin by installing it from the following Github repository:
git clone https://github.com/DIDSR/DLMO.git
The trained models uploaded to this repository total approximately 2.17 GB. In case of bandwidth issues or if you only need to use the code, you can enable GitHub’s smudging option to download the models as pointers instead.
GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/DIDSR/DLMO.git #to avoid LFS quota issue and download this repo without the trained models
git lfs pull #in case you decide that you will fully download all the models in this repo
Create the required environments:
conda env create --name ddpm --file requirements/ddpm.yml #to generate MR images using trained DDPM. DDPM training implemented using https://github.com/openai/improved-diffusion
conda env create --name dlmo --file requirements/dlmo.yml #to run the dlmo codes
The R scripts in demo6 have been tested with versions 4.3.1 and 4.4.1.
Example codes below demonstrate how to run the DLMO assessment methodology for a discrimination task, serving as a surrogate for image resolution across reconstruction algorithms as a function of MR acceleration.
-
Generate a large batch of MR image samples using a trained DDPM1 model. The generated
.npzfile is the object input for demos 2 and 3. -
Insert singlet and doublet signals into DDPM-generated objects and save the resulting objects in HDF5 format.
-
MR acquisition and reconstruction
Perform forward projection and rSOS reconstruction on DDPM-generated objects to create test datasets in HDF5 format.
-
Train the deep learning-based model observer. This training demo is separated from the small end-to-end example and saves models for later testing.
-
A simple example of the DLMO framework
Run a compact workflow with bundled example for: 1) object generation, 2) image acquisition and reconstruction, 2) AI reconstruction, and 3) DLMO testing.
-
Run the MRMC2-based statistical analysis to ensure DLMO operates at the same level as human experts. This part includes sample size estimation using a pilot study and pivotal-study similarity testing to demonstrate that DLMO and human performance are within a predefined margin of 0.1. It uses source code from the iMRMC package3. We have included relevant R files from the package; therefore, pre-installation of the iMRMC library is not recommended.
DLMO is distributed under CC0 1.0 Liscense. See LICENSE for more information.
If you use the DLMO data or code in your project, please cite its arXiv paper:
@article{yu2026evaluating,
title={Evaluating the resolution of AI-based accelerated MR reconstruction using a deep learning-based model observer},
author={Yu, Zitong and Zeng, Rongping and Samuelson, Frank and Kc, Prabhat},
journal={arXiv preprint arXiv:2602.22535},
year={2026}
}
Footnotes
-
J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems, vol. 33, pp. 6840–6851, 2020. ↩
-
N. A. Obuchowski, “Multireader receiver operating characteristic studies: a comparison of study designs,” Acad. Radiol., vol. 2, no. 8, pp.709–716, 1995. ↩
-
FDA/CDRH, “iMRMC: Software for the Statistical Analysis of multi-reader multi-case studies,” RST Catalog, 2022, https://doi.org/10.5281/ZENODO.6628838. ↩