Authors: Weerasinghe M.S.S , Jayasinghe C.H , Jayasooriya J.M.D.C
Supervisors: Dr. Charith Chitraranjan (internal), Dr. Isuru Wijesinghe (external)
Magnetic Resonance Imaging (MRI) is essential for breast cancer diagnosis, but high-resolution scans require long acquisition times that increase patient discomfort and motion artifact risk. K-space undersampling reduces scan duration but introduces aliasing artifacts and loss of diagnostic detail. Convolutional Neural Networks (CNNs), while effective in image-domain reconstruction, are poorly suited to k-space data because they rely on local receptive fields and translation invariance, whereas MRI artifacts are global frequency-domain phenomena. This work implements a K-Space Transformer that treats undersampled MRI reconstruction as a coordinate-query problem in frequency space, using Implicit Neural Representation (INR) with sinusoidal positional encoding, a hierarchical Low-Resolution (LR) to High-Resolution (HR) decoder, and an image-domain CNN refinement module with k-space data consistency. The model is trained on T1-weighted fat-saturated breast MRI from the Duke Breast MRI dataset (50 patients, 320×320 k-space) under retrospective undersampling at acceleration factors ×3, x5, x7,×10. A three-stage training schedule (LR → HR → refinement) with dual-domain deep supervision converges stably despite loss spikes at stage transitions. Quantitative evaluation against OUCR and SwinMR baselines shows that the hybrid model with image-domain refinement achieves the best PSNR and SSIM at all tested acceleration factors (e.g., PSNR 32.43 dB and SSIM 0.9594 at ×10), outperforming OUCR on SSIM across all factors and on PSNR at moderate accelerations (×3–×7). Qualitative results confirm effective artifact removal and restoration of fibroglandular tissue detail. These findings demonstrate that hybrid k-space and image-domain modeling is a strong direction for clinically useful accelerated breast MRI reconstruction.
- 1. Introduction
- 2. Related Work
- 3. Methodology
- 4. Implementation
- 5. Experiments and Results
- 6. Discussion and Conclusion
- 7. Code vs Research Configuration
- 8. Reproducibility and Usage
- 9. References
Breast cancer remains the most prevalent cancer among women globally, and Magnetic Resonance Imaging (MRI) serves as a cornerstone for diagnosis in high-risk patients. MRI provides high-resolution, radiation-free images with excellent soft-tissue contrast, but the lengthy scan times required for detailed breast imaging often exceed one hour, leading to patient discomfort and increased susceptibility to motion artifacts. Accelerated acquisition via k-space undersampling can reduce scan times to under 20 minutes, but it transforms image reconstruction into an ill-posed inverse problem. Incomplete k-space coverage causes aliasing artifacts and loss of fine anatomical detail critical for distinguishing benign from malignant lesions.
The fundamental relationship between k-space and the image domain is expressed as
Traditional reconstruction methods such as Parallel Imaging (SENSE, GRAPPA) and Compressed Sensing (CS) are limited by fixed mathematical priors and struggle under high acceleration factors. Deep Learning (DL) has emerged as a powerful alternative, but the majority of existing models particularly CNN based U-Nets operate in the image domain. CNNs rely on local receptive fields and the assumption of translation invariance, which are suboptimal for k-space data where spatial information is distributed across frequency components and accurate reconstruction requires capturing long range dependencies between distant frequency bins.
This research addresses this gap by implementing a K-Space Transformer framework tailored for breast MRI reconstruction. The model adopts an Implicit Neural Representation (INR), treating the k-space spectrogram as a continuous function where spatial coordinates are queried to reconstruct missing frequency data. Global self attention mechanisms capture non local dependencies essential for aliasing artifact removal. To manage computational complexity, a hierarchical decoder structure processes data through LR and HR stages, recovering both anatomical structure and fine lesion morphology. An image-domain refinement module with k-space data consistency further restores local spatial detail that pure frequency-domain decoding cannot fully capture.
The scope of this work is limited to T1-weighted fat-saturated breast MRI central slices from the Duke Breast MRI dataset, evaluated at acceleration factors ×3, ×5, ×7, and ×10 using Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) metrics against OUCR and SwinMR baselines.
Deep learning for MRI reconstruction has progressed through several architectural paradigms, each with distinct tradeoffs for breast imaging applications.
Convolutional approaches. CNN-based methods, particularly U-Net architectures with encoder-decoder structures and skip connections, form the foundation of modern DL MRI reconstruction. Residual learning, attention mechanisms, and dual-domain networks that alternate between k-space and image processing have extended CNN capabilities. However, CNNs remain fundamentally constrained by local receptive fields when applied directly to k-space data.
Transformer approaches. Vision Transformers (ViTs) and specialized MRI reconstruction transformers such as ReconFormer enable superior modeling of long-range dependencies through self-attention. The K-Space Transformer treats k-space coordinates as continuous query points with sinusoidal positional encoding, moving beyond discrete grid constraints. SwinMR applies shifted-window attention for efficient high-resolution MRI reconstruction.
Hybrid k-space and image-domain methods. Purely k-space or latent-domain transformers often produce residual artifacts and loss of fine anatomical detail. Hybrid strategies that combine frequency-domain processing with image-domain refinement, exemplified by Deep Cascade CNNs (DC-CNN), Variational Networks and SwinMR variants demonstrate that alternating k-space consistency enforcement with spatial-domain correction improves reconstruction quality under aggressive undersampling.
Breast MRI-specific gap. Despite progress in general MRI reconstruction, breast MRI remains underrepresented in the DL landscape. The unique characteristics of breast tissue dense fibroglandular structures, variable breast density patterns, and spatial heterogeneity, introduce domain-specific challenges that generic models inadequately address. Data scarcity (the fastMRI dataset contains only 300 3D breast scans) and limited investigation of hybrid architectures for 2D k-space breast MRI protocols further constrain specialized model development.
This work contributes a breast-specific K-Space Transformer with hierarchical decoding and image-domain refinement, evaluated comprehensively against established baselines across multiple acceleration factors.
Data source. The Duke Breast MRI dataset comprises complex, 3D dynamic contrast-enhanced volumes from 50 patients. Central slices from T1-weighted fat-saturated sequences were extracted to ensure representative balance of fibroglandular tissue and fatty background while minimizing coil sensitivity profile effects at volume edges.
Preprocessing pipeline. Raw MRI data in H5 format undergoes four stages before training:
-
K-space extraction and normalization. K-space data is center-cropped to 320×320 pixels, converted from complex values to 2-channel
[real, imaginary]representation, transformed to image domain via centered IFFT, per-slice normalized to zero mean and unit variance$\text{normalized} = (\text{image} - \mu) / \sigma$ , transformed back to k-space via FFT, and filtered by variance (lowest 20% variance slices removed). Output:k_data.npywith shape[N, 320, 320, 2]. -
Low-resolution generation. HR k-space is converted to image domain (IFFT), downsampled via 2× average pooling (stride 2), and converted back to k-space (FFT). Output:
LR_k_data.npywith shape[N, 160, 160, 2]. -
Undersampling mask generation. Six clinically relevant mask types are generated with stochastic variation: Cartesian equispaced, Cartesian random, variable-density Cartesian, uniform radial, variable-density radial, and spiral. All masks fully sample central k-space (~8% of lines). Output:
combined_masks.npywith shape[60, 320, 320]. -
Dataset partitioning. Data is split 70% training, 15% validation, and 15% test at the patient level with identical HR/LR correspondence across splits.
In-repo tooling covers stages 2 and 4 via kst-preprocess and kst-split. Raw H5 extraction and mask generation are performed externally.
| Array | Shape | Description |
|---|---|---|
| HR k-space | [N, 320, 320, 2] |
Fully sampled ground truth |
| LR k-space | [N, 160, 160, 2] |
Coarse supervision target |
| Mask bank | [60, 320, 320] |
Retrospective undersampling patterns |
Split outputs: train_k.npy, valid_k.npy, test_k.npy, corresponding *_lr_k.npy files, and split_indices.npz.
Unlike CNNs that ingest fixed grids, the K-Space Transformer processes a sequence of available k-space points. For each training sample, a random undersampling mask is applied to HR k-space. Sampled points and their 2D spatial frequency coordinates form encoder input tokens; unsampled coordinates serve as HR decoder queries.
Each token is defined as a tuple magnify=250 is applied to coordinates before summation with the MLP embedding of k-space values. This coordinate based tokenization supports non-Cartesian and arbitrary sampling patterns dynamically.
Variable-length token sequences are truncated to max_seq_len=8000 and padded by KSpaceCollator for batch processing. Masks are randomly reassigned during training (default: every epoch) for data augmentation. Implementation: data/tokenize.py, data/datasets.py.
The KSpaceTransformer model (model/transformer.py) learns a continuous function mapping spatial coordinates to k-space values, conditioned on observed samples.
Encoder (4 layers,
LR Decoder (4 layers). Queries are generated from normalized positional coordinates of a 64×64 downsampled grid (lr_size=64). Each layer applies multi-head cross-attention (MHCA) to encoder memory followed by MHSA. Per-layer complex value predictions are reshaped and transformed via IFFT to image-domain outputs with deep supervision at each layer.
HR Decoder (6 layers). Upsampled LR decoder output serves as context (key/value). To minimize memory consumption, HR layers retain only cross-attention and FFN (no self-attention). Unsampled k-space coordinates are queried to predict complex values, inserted into masked k-space via fill_in_k(), and transformed to image domain via IFFT at each layer.
Image-Domain Refinement Module (RM stage). Active only during the refinement training stage, this module receives HR decoder output, applies differentiable IFFT, processes through a CNN stack (LeakyReLU activations, 64 mid-channels, 3×3 kernels), and transforms back to k-space via FFT. Data consistency is enforced: conv_weight-scaled embedding. Implementation: model/blocks.py.
| Parameter | Default |
|---|---|
d_model |
256 |
n_head |
4 |
| Encoder / LR / HR layers | 4 / 4 / 6 |
dim_feedforward |
1024 |
lr_size |
64 |
batch_size |
4 |
max_seq_len |
8000 |
lr (AdamW) |
5e-4 |
| Optimizer schedule | Cosine annealing |
Defaults defined in config/schema.py.
A three-stage progressive training schedule (training/stage.py) targets coarse-to-fine reconstruction:
| Stage | Research Epochs | Code Defaults | Active Modules | Objective |
|---|---|---|---|---|
| LR | 1–50 | 1–50 | Encoder + LR decoder | Coarse 160×160 structure |
| HR | 51–150 | 51–100 | + HR decoder | Full 320×320 k-space |
| RM | 151–310 | 101–200 | + CNN refinement | Spatial artifact removal |
Stage 1 (LR). Encoder and LR decoder are supervised with 160×160 targets. HR decoder outputs are zeroed and remain untrained.
Stage 2 (HR). HR decoder is supervised with 320×320 targets. Encoder and LR decoder continue training to maintain coarse representation integrity.
Stage 3 (RM). Refinement module is supervised with high resolution image-domain targets. All components remain active so global frequency recovery and local spatial correction stay aligned.
Dual-domain deep supervision. The total loss aggregates weighted Mean Squared Error (MSE) across all active decoder layers in both k-space and image domains simultaneously, providing complementary supervision for frequency accuracy and spatial coherence.
The repository (kspace-transformer v0.1.0) implements the research pipeline as modular, testable research software. Each subsystem data processing, model design, training, inference, validation, and CLI orchestration is separated into dedicated modules.
.
├── cli/ # Executable workflows: train/test/preprocess/split/parity
├── config/ # Typed runtime schema, defaults, CLI argument binding
├── data/ # Tokenization, masks, grids, datasets, LR generation, split pipeline
├── inference/ # Inference runner and stage-aware evaluation path
├── model/ # Transformer, attention, decoders, refinement blocks
├── training/ # Trainer engine, stage scheduler, losses, metrics, checkpoints
├── utils/ # FFT utilities, device helpers, seed control, runtime tracker
├── validation/ # Parity comparison and acceptance gate logic
├── tests/ # Unit/integration/smoke coverage (17 test modules)
├── Results/ # Report, figures, comparison tables, reconstruction images
├── pyproject.toml
└── README.md
Key design patterns:
- Strict data contracts with validation:
TokenizedSample,ForwardOutputs,LossBreakdown,BatchTensors - Stage-aware forward pass: model behavior changes by training stage (HR outputs zeroed in LR stage; CNN skipped in HR stage)
- Adaptive mask reassignment during training for augmentation
- Sequence length control for tokenized sampled/unsampled streams
- AdamW optimization with cosine learning rate schedule
- Checkpoint lifecycle (
last.pth,best_valid_psnr.pth) - TensorBoard metric logging with collision-safe keys
- Runtime and peak-memory telemetry in workflow summaries
- Parity regression gate (
kst-parity) comparing baseline vs candidate runs
Install the package with pip install -e . and run the test suite with python -m pytest -q.
Baselines. OUCR (Over/Under complete Convolutional RNN) serves as the primary baseline. SwinMR provides a transformer-based reference benchmark.
Metrics. Peak Signal-to-Noise Ratio (PSNR, dB) measures signal fidelity; Structural Similarity Index (SSIM) evaluates preservation of anatomical structure, edges, and textures.
Acceleration factors. Models are evaluated at ×3, ×5, ×7, and ×10 undersampling using 60 retrospective masks spanning six sampling pattern types.
Hardware. Training was conducted on an NVIDIA RTX PRO 6000 GPU (RunPod). Total project compute cost was approximately ~$300 during the training time (310 epochs at 20–30 minutes per epoch).
Best model inference. Reported results for the full hybrid model use the refinement stage RM (refinement module active).
Baselines and Metrics. SwinMR and OUCR (Over/Under complete Convolutional RNN) serve as the primary baselines. The performance of the K-Space Transformer is evaluated using Peak Signal-to-Noise Ratio (PSNR, dB), Structural Similarity Index (SSIM), and Normalized Mean Squared Error (NMSE).
Analysis. The NMSE comparison (Figure 9) indicates that SwinMR exhibits significantly higher error across all acceleration factors compared to the transformer-based variants, while the baseline OUCR and the hybrid K-Space Transformer (with refinement) perform the best. Image-domain refinement provides increasing PSNR gains over the k-space-only model as acceleration increases: +1.79 dB (×3), +2.92 dB (×5), +3.22 dB (×7), and +4.07 dB (×10). SSIM gains follow the same trend (+0.0014 to +0.0102). The hybrid model with refinement beats OUCR on SSIM at all acceleration factors and on PSNR at ×3 (+0.34 dB), ×5 (+0.71 dB), and ×7 (+0.91 dB); at ×10, PSNR is 0.82 dB below OUCR while SSIM remains superior (0.9594 vs 0.9442). Both transformer variants substantially outperform SwinMR across all metrics and acceleration factors.
Peak Signal-to-Noise Ratio (PSNR):
| Method | ×3 | ×5 | ×7 | ×10 |
|---|---|---|---|---|
| SwinMR | 31.51 | 30.36 | 29.21 | 27.48 |
| OUCR (baseline) | 38.47 | 36.78 | 34.25 | 31.61 |
| K-Space Transformer (no refinement) | 37.02 | 34.57 | 31.94 | 28.36 |
| K-Space Transformer (with refinement) | 38.81 | 37.49 | 35.16 | 32.43 |
Structural Similarity Index (SSIM):
| Method | ×3 | ×5 | ×7 | ×10 |
|---|---|---|---|---|
| SwinMR | 0.9520 | 0.9341 | 0.9162 | 0.8893 |
| OUCR (baseline) | 0.9915 | 0.9797 | 0.9625 | 0.9442 |
| K-Space Transformer (no refinement) | 0.9923 | 0.9801 | 0.9670 | 0.9492 |
| K-Space Transformer (with refinement) | 0.9937 | 0.9820 | 0.9731 | 0.9594 |
During the LR stage (epochs 1–50), MSE loss declines steadily for both training and validation sets, reaching a local plateau near train 0.03 / val 0.035 by epoch 50. At the LR→HR transition (epoch 51), a significant loss spike occurs as the loss function expands from 8 to 20 terms; the model must simultaneously predict unsampled k-space points and optimize six additional HR decoder layers. The model adapts rapidly (epochs 51–80) by building on established LR representations, then converges smoothly to final HR MSE values of approximately 0.018 (train) and 0.023 (validation).
During the refinement stage, the model is trained exclusively using a composite loss function (L1 + Perceptual + SSIM). Initial loss values for this stage begin at approximately 0.8 for training and 0.85 for validation. Over the course of the 160 epochs, both metrics exhibit a steady decline, ultimately converging to approximately 0.156 (train) and 0.204 (validation) by epoch 310. This optimization trajectory demonstrates successful learning of spatial artifact suppression while effectively preserving k-space consistency.
Findings. This work demonstrates that a hybrid K-Space Transformer with image-domain refinement effectively reconstructs undersampled breast MRI across acceleration factors ×3 to ×10. The coordinate-query formulation with INR-style positional encoding captures global frequency dependencies that CNN-only approaches miss. The hierarchical LR→HR decoder reduces computational complexity while enabling coarse-to-fine learning. Three-stage training converges stably despite expected loss spikes at stage transitions, validating the progressive resolution strategy. Image-domain refinement preserves the global frequency structure learned during HR training while restoring local spatial detail, with the largest quantitative gains at the highest acceleration factors where residual artifacts are most severe.
Limitations. Evaluation is limited to a single dataset (50 patients, 2D central slices), T1-weighted sequences only, and retrospective undersampling simulation. No clinical reader study or regulatory validation was performed. The current codebase defaults differ from the research training configuration that produced the reported results (see Section 7). Generalization to other institutions, sequences (T2, DWI), and 3D volumetric data remains untested.
Conclusion. Hybrid k-space and image-domain modeling is a strong direction for clinically useful accelerated breast MRI reconstruction. The complete pipeline—from preprocessing through three-stage training to quantitative and qualitative evaluation—provides a reproducible foundation for further optimization and downstream clinical integration.
Full project report: Results/Project_Final_Report_Group32_draft.pdf
The reported results in Section 5 were produced with a research training configuration that differs from the codebase defaults in several aspects. Users reproducing or extending this work should be aware of these differences.
| Aspect | Research (reported results) | Codebase default |
|---|---|---|
| Total epochs | 310 | 200 |
| Stage boundaries | LR 50 / HR 100 / RM 160 | LR 50 / K 50 / RM 100 |
| CNN refinement depth | 5 conv layers | 4 conv + 1×1 output |
| Mask generation | 60 masks, 6 types | Expects combined_masks.npy |
- Python
>=3.10,<3.14 - NumPy
>=2.1,<3.0 - PyTorch
>=2.2,<3.0 - h5py
>=3.10,<4.0 - scikit-image
>=0.22,<1.0 - TensorBoard
>=2.15,<3.0 - tqdm
>=4.66,<5.0
Optional development tools: pytest, pytest-cov, ruff, mypy
python -m venv .venv
source .venv/bin/activateWindows PowerShell:
python -m venv .venv
.\.venv\Scripts\Activate.ps1Runtime install:
python -m pip install -e .Development install:
python -m pip install -e ".[dev]"Entrypoints: kst-preprocess, kst-split, kst-train, kst-test, kst-evaluate, kst-parity
Generate LR k-space from HR k-space:
kst-preprocess \
--input_hr_kspace_path ./data/processed/train_k.npy \
--output_lr_kspace_path ./data/processed/train_lr_k.npy \
--scale 2 \
--batch_size 50 \
--save_summary_path ./runs/preprocess_summary.jsonDeterministic train/validation/test split:
kst-split \
--hr_kspace_path ./data/processed/processed_data.npy \
--lr_kspace_path ./data/processed/LR_k_data.npy \
--split_output_dir ./data/processed/splits \
--train_ratio 0.7 \
--valid_ratio 0.15 \
--test_ratio 0.15 \
--shuffle true \
--random_seed 42 \
--save_summary_path ./runs/split_summary.jsonTrain (code defaults):
kst-train \
--output_dir ./runs/exp01 \
--train_hr_data_path ./data/processed/splits/train_k.npy \
--train_lr_data_path ./data/processed/splits/train_lr_k.npy \
--train_mask_path ./data/masks/combined_masks.npy \
--valid_hr_data_path ./data/processed/splits/valid_k.npy \
--valid_lr_data_path ./data/processed/splits/valid_lr_k.npy \
--valid_mask_path ./data/masks/combined_masks.npy \
--epoch_num 200 \
--pure_lr_training_epoch 50 \
--pure_k_training_epoch 100Train (research-matching schedule):
kst-train \
--output_dir ./runs/research_repro \
--train_hr_data_path ./data/processed/splits/train_k.npy \
--train_lr_data_path ./data/processed/splits/train_lr_k.npy \
--train_mask_path ./data/masks/combined_masks.npy \
--valid_hr_data_path ./data/processed/splits/valid_k.npy \
--valid_lr_data_path ./data/processed/splits/valid_lr_k.npy \
--valid_mask_path ./data/masks/combined_masks.npy \
--epoch_num 310 \
--pure_lr_training_epoch 50 \
--pure_k_training_epoch 150Primary artifacts under --output_dir:
tensorboard/checkpoints/last.pthcheckpoints/best_valid_psnr.pthtraining_summary.json
Inference/evaluation:
kst-test \
--output_dir ./runs/exp01 \
--checkpoint ./runs/exp01/checkpoints/best_valid_psnr.pth \
--test_hr_data_path ./data/processed/splits/test_k.npy \
--test_mask_path ./data/masks/combined_masks.npy \
--inference_stage RM \
--save_summary_path ./runs/exp01/inference_summary.jsonComprehensive evaluation by acceleration factor:
kst-evaluate \
--output_dir ./runs/exp01 \
--checkpoint ./runs/exp01/checkpoints/best_valid_psnr.pth \
--test_hr_data_path ./data/processed/splits/test_k.npy \
--test_mask_path ./data/masks/combined_masks.npy \
--mask_manifest ./data/masks/mask_manifest.json \
--acceleration_factors 3 5 7 10 \
--evaluation_stages K RMThis writes per-sample PSNR, SSIM, and NMSE; grouped statistics by acceleration;
K-to-RM improvement tables; metric plots; and consistently normalized qualitative comparisons
under output_dir/evaluation/.
Parity gate (baseline vs candidate):
kst-parity \
--baseline_train_summary ./runs/baseline/training_summary.json \
--baseline_inference_summary ./runs/baseline/inference_summary.json \
--candidate_train_summary ./runs/candidate/training_summary.json \
--candidate_inference_summary ./runs/candidate/inference_summary.json \
--psnr_drift_db 0.05 \
--ssim_drift 0.001 \
--runtime_drift_ratio 0.05 \
--memory_drift_ratio 0.10 \
--output_report ./runs/candidate/parity_report.jsonExit status: 0 = pass, 1 = fail.
- Deterministic seed configuration is enabled through runtime options.
- Metric reporting includes PSNR and SSIM for training and evaluation.
- Runtime summaries provide elapsed time and peak memory signals.
- Test suite covers contracts, data modules, model forward behavior, engine integration, CLI smoke coverage, and parity validation.
Run tests:
python -m pytest -qNotes for breast MRI experiments:
- Ensure mask banks are aligned with HR spatial resolution (320×320).
- Keep acquisition-specific preprocessing steps consistent across train/validation/test splits.
- Use parity reports when introducing architectural or hyperparameter changes to preserve clinical quality reconstruction behavior.













