🧠 IHFNet: Incomplete Multimodal Hierarchical Feature Fusion Network for Mild Cognitive Impairment Conversion Prediction

This repository implements IHFNet and multiple baselines for pMCI vs sMCI classification on ADNI using T1 sMRI + FDG-PET + clinical features.

---

📌 1. Overview

🧭 1.1 Pipeline

🎯 1.2 What this repo provides

• Training with Stratified 5-fold cross-validation (default) via main_rebuild.py
• Multiple model options configured in model_object.py (IHFNet and baselines)
• Inference example on provided .nii sample files via Inference.py
• Cross-dataset evaluation ADNI1 ↔ ADNI2 via cross_dataset_test.py (loads 5 fold checkpoints and reports per-fold + mean metrics)

---

🗂️ 2. Repository Structure

.
├── assets/
├── example_data/
│   ├── test_mri.nii
│   ├── test_pet.nii
├── Net/
│   ├── IHFNet.py
│   ├── TriLightNet.py
│   ├── ResnetEncoder.py
│   ├── poolformer.py
│   ├── metaformer3D.py
│   ├── defineViT.py
│   ├── ComparisonNet.py
│   ├── basic.py
│   ├── kan.py
├── utils/
│   ├── api.py
│   ├── basic.py
│   ├── observer.py
├── Config.py
├── Dataset.py
├── loss_function.py
├── model_object.py
├── model_params.py
├── main_rebuild.py
├── Inference.py
├── cross_dataset_test.py
├── requirements.txt
└── README.md

---

🧬 3. Dataset

🏥 3.1 ADNI

The dataset is obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI), specifically the ADNI-1 and ADNI-2 cohorts.

ADNI link: https://adni.loni.usc.edu/

📁 3.2 Expected file organization

This codebase expects NIfTI volumes named by a shared identifier (e.g., subject/image id). The dataset loader will look for:

• MRI file: <mri_dir>/<ID>.nii
• PET file: <pet_dir>/<ID>.nii

Example layout:

<DATA_ROOT>/
├── ADNI1/
│   ├── MRI/
│   │   ├── <ID>.nii
│   │   └── ...
│   ├── PET/
│   │   ├── <ID>.nii
│   │   └── ...
│   ├── labels.csv
│   └── clinical.csv
└── ADNI2/
		├── MRI/
		├── PET/
		├── labels.csv
		└── clinical.csv

🧾 3.3 CSV formats

🏷️ 3.3.1 Label CSV (--csv_file)

Dataset.py reads a CSV where:

• Column 0: image id (without extension)
• Column 1: label string (must contain the task labels)

For the default task (pMCI vs sMCI), labels should be exactly pMCI or sMCI.

Minimal example:

PTID,label
002_S_0413,sMCI
002_S_0619,pMCI

Note: Internally the dataset maps sMCI -> 0, pMCI -> 1.

🩺 3.3.2 Clinical CSV (--cli_dir)

The clinical file is not included in this repo due to privacy. The default clinical feature extraction (get_clinical in Dataset.py) expects the following columns:

• PTID (must match the image id)
• PTGENDER, AGE, PTEDUCAT, FDG_bl, TAU_bl, PTAU_bl, APOE4

Missing values will be handled by the code (filled with zeros in the current implementation).

---

🧰 4. Environment Setup

📦 4.1 Installation

Create a clean Python environment (recommended) and install dependencies:

pip install -r requirements.txt

🧩 Notes on PyTorch wheels

requirements.txt pins torch==2.6.0+cu118 and torchvision==0.21.0+cu118. If pip install -r requirements.txt fails on your machine, install PyTorch first following the official instructions, then install the remaining packages.

🖥️ 4.2 Hardware requirement

The training entry in main_rebuild.py asserts CUDA availability. Run training on a CUDA-capable GPU environment.

---

🏋️ 5. Training (5-fold CV)

🚀 5.1 Quick start

The main training entry is:

python main_rebuild.py \
	--model IHFNet_With_MLP \
	--mri_dir <ADNI1/MRI> \
	--pet_dir <ADNI1/PET> \
	--cli_dir <ADNI1/clinical.csv> \
	--csv_file <ADNI1/labels.csv> \
	--device cuda:0 \
	--n_splits 5 \
	--batch_size 8 \
	--seed 42

Windows/PowerShell users: you can run the same command in one line, or use backticks ` for line continuation.

🧠 5.2 Model selection

Model indices are defined in model_object.py (dictionary models). Common choices:

• IHFNet / IHFNet_With_MLP / IHFNetWithoutCMIM
• IMF, HFBSurv, TriLightNet, etc.

Use --model <MODEL_INDEX> to switch.

💾 5.3 Outputs and checkpoints

During training, a timestamped output directory is created:

• ./<checkpoints_dir>_<YYYY-mm-dd_HH-MM>/

For each fold, the best checkpoint is saved as:

• <ExperimentName>_best_model_fold{1..n_splits}.pth

For IHFNet variants, <ExperimentName> is IHFNet (see model_object.py).

TensorBoard logs are written under:

• <output_dir>/summery/

---

🔍 6. Inference (example)

We provide sample MRI and PET images in example_data/.

Run:

python Inference.py

Before running, edit Inference.py to set:

• model_path: path to your .pth checkpoint
• cli_features: a list of 9 clinical features used by the script

Note: This script is a minimal example and does not load clinical CSV automatically.

---

🔁 7. Cross-Dataset Test (ADNI1 ↔ ADNI2)

The cross-dataset test script loads the 5 fold checkpoints trained on one dataset and evaluates on the other dataset, printing per-fold metrics and the mean.

Run:

python cross_dataset_test.py \
	--direction both \
	--models all \
	--n_splits 5 \
	--batch_size 8 \
	--device cuda:0 \
	--adni1_mri_dir <ADNI1/MRI> --adni1_pet_dir <ADNI1/PET> --adni1_cli_dir <ADNI1/clinical.csv> --adni1_csv_file <ADNI1/labels.csv> --adni1_weights_dir <ADNI1_checkpoints_dir> \
	--adni2_mri_dir <ADNI2/MRI> --adni2_pet_dir <ADNI2/PET> --adni2_cli_dir <ADNI2/clinical.csv> --adni2_csv_file <ADNI2/labels.csv> --adni2_weights_dir <ADNI2_checkpoints_dir>

Outputs:

• A JSON summary file will be written to ./cross_dataset_test_outputs/.

---

📊 8. Experimental Results

In the Modality column, M, P, and C denote MRI, PET, and clinical data.

🧪 8.1 ADNI-1 (mean ± std)

The following results are reported as mean ± std over 5 folds (as shown in the paper figure).

Method	Modality	ACC ↑	PRE ↑	BACC ↑	AUC ↑	F1 ↑
HOPE	M	0.611 ± 0.038	0.599 ± 0.042	0.699 ± 0.051	0.648 ± 0.062	0.593 ± 0.098
ResNet	M,P	0.725 ± 0.041	0.671 ± 0.057	0.693 ± 0.048	0.653 ± 0.069	0.606 ± 0.087
JSRL	M,P	0.582 ± 0.047	0.580 ± 0.054	0.566 ± 0.059	0.571 ± 0.073	0.580 ± 0.105
VAPL	M,C	0.630 ± 0.039	0.648 ± 0.048	0.628 ± 0.054	0.635 ± 0.067	0.651 ± 0.092
Diamond	M,P	0.736 ± 0.044	0.719 ± 0.056	0.693 ± 0.047	0.638 ± 0.076	0.591 ± 0.094
HyperFusionNet	M,P	0.688 ± 0.020	0.704 ± 0.180	0.585 ± 0.034	0.613 ± 0.075	0.336 ± 0.113
nnMamba	M,P	0.694 ± 0.070	0.629 ± 0.150	0.686 ± 0.062	0.684 ± 0.085	0.609 ± 0.074
MDLNet	M,P	0.738 ± 0.058	0.820 ± 0.165	0.650 ± 0.051	0.676 ± 0.053	0.477 ± 0.104
HFBSurv	M,P,C	0.740 ± 0.036	0.701 ± 0.052	0.714 ± 0.049	0.711 ± 0.074	0.630 ± 0.091
IMF	M,P,C	0.756 ± 0.040	0.740 ± 0.058	0.710 ± 0.050	0.720 ± 0.077	0.605 ± 0.089
MultiModalADNet	M,P,C	0.700 ± 0.083	0.641 ± 0.183	0.693 ± 0.147	0.650 ± 0.076	0.637 ± 0.061
IHFNet (Ours)	M,P,C	0.737 ± 0.043	0.708 ± 0.086	0.723 ± 0.061	0.738 ± 0.078	0.666 ± 0.130

🧪 8.2 ADNI-2 (mean ± std)

Method	Modality	ACC ↑	PRE ↑	BACC ↑	AUC ↑	F1 ↑
HOPE	M	0.701 ± 0.045	0.706 ± 0.053	0.645 ± 0.049	0.624 ± 0.078	0.505 ± 0.102
ResNet	M,P	0.809 ± 0.039	0.721 ± 0.061	0.683 ± 0.052	0.709 ± 0.083	0.510 ± 0.095
JSRL	M,P	0.650 ± 0.052	0.600 ± 0.067	0.655 ± 0.054	0.694 ± 0.090	0.519 ± 0.112
VAPL	M,C	0.712 ± 0.043	0.750 ± 0.071	0.672 ± 0.058	0.623 ± 0.086	0.561 ± 0.089
Diamond	M,P	0.818 ± 0.041	0.739 ± 0.065	0.682 ± 0.053	0.645 ± 0.079	0.509 ± 0.108
HyperFusionNet	M,P	0.791 ± 0.008	0.537 ± 0.011	0.750 ± 0.041	0.790 ± 0.044	0.595 ± 0.043
nnMamba	M,P	0.750 ± 0.049	0.481 ± 0.077	0.719 ± 0.050	0.782 ± 0.069	0.554 ± 0.075
MDLNet	M,P	0.769 ± 0.080	0.714 ± 0.214	0.649 ± 0.033	0.679 ± 0.098	0.446 ± 0.063
HFBSurv	M,P,C	0.813 ± 0.042	0.701 ± 0.072	0.688 ± 0.051	0.672 ± 0.084	0.539 ± 0.098
IMF	M,P,C	0.838 ± 0.044	0.737 ± 0.068	0.713 ± 0.055	0.757 ± 0.092	0.564 ± 0.101
MultiModalADNet	M,P,C	0.750 ± 0.066	0.531 ± 0.105	0.754 ± 0.027	0.801 ± 0.011	0.621 ± 0.033
IHFNet (Ours)	M,P,C	0.856 ± 0.056	0.777 ± 0.073	0.735 ± 0.047	0.812 ± 0.103	0.584 ± 0.119