MAT3D

Multi-Aperture Transformers for 3D (MAT3D) Segmentation of Clinical and Microscopic Images!

Paper Link: Access the Paper

Video Presentation

🔗 YouTube Link: https://youtu.be/JOZAs1t7yYw

Overview

MAT3D uses Multi-Aperture Transformers for accurate 3D segmentation in clinical and microscopic imaging

---

Framework

---

Environment Setup

Set up the environment for MAT3D as follows:

python3.10 -m venv MAT3D_env 
source MAT3D_env/bin/activate 
pip install -r requirements.txt

---

Dataset

Download Links:

• Synapse Multi-Organ Dataset
• ACDC Challenge Dataset

Folder Structure:

Organize your data as follows:

data/  
├── imagesTr/  
│   ├── img1.nii.gz  
│   ├── img2.nii.gz  
├── labelsTr/  
│   ├── label1.nii.gz  
│   ├── label2.nii.gz  
├── dataset.json  

---

Running the Code

This repository is built upon the foundational work provided in Synapse.

Training

Before training, configure the hyperparameters in the config.py file:

Hyperparameter Configuration:

• data_dir: Path to the dataset.
• saved_model_dir: Directory to save trained models and checkpoints.
• num_samples: Number of samples for training.
• num_classes: Number of target classes + background.
• input_size: Dimensions of input images/data.
• input_channels: Number of input channels (e.g., grayscale=1, RGB=3).
• feature_size: Size of feature vectors extracted by the model.
• use_checkpoint: Enable/disable model checkpointing.
• learning_rate: Initial learning rate.
• weight_decay: L2 penalty rate for regularization.
• max_iterations: Maximum number of training iterations.
• eval_num: Frequency of evaluations during training.

Start Training:

python3.10 main.py  

---

Results

Quantitative Results (Microscopic data):

Clinical data Visualization:

Microscopic data Visualization:

Citation

M. Sohaib, S. Shabani, S. A. Mohammed, G. Winkelmaier and B. Parvin, "Multi-Aperture Transformers for 3D (MAT3D) Segmentation of Clinical and Microscopic Images," 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Tucson, AZ, USA, 2025, pp. 4352-4361, doi: 10.1109/WACV61041.2025.00427.

---