Efficient Transformer for High-Resolution Image Motion Deblurring

Overview

This project builds upon the Restormer architecture, enhancing its efficiency and performance for the task of high-resolution image motion deblurring. The improvements include architectural modifications, advanced training techniques, and extended evaluations on diverse datasets to create a robust and efficient model for real-world deblurring challenges.

Key Features of This Work

• Reduced Model Complexity: The model complexity is reduced by 18.4%, improving inference speed and reducing memory requirements.
• Enhanced Training Pipeline: Incorporation of transformations such as color jitter, Gaussian blur, perspective transforms, and a new frequency-domain loss function to improve robustness and accuracy.
• Extensive Evaluation: Experiments performed on RealBlur-R, RealBlur-J, and the Ultra-High-Definition Motion Blurred (UHDM) datasets.
• Ablation Studies: Detailed analyses to quantify the impact of architectural and training modifications.

This project retains the core innovations of Restormer, including its multi-Dconv head transposed attention mechanism and gated-Dconv feed-forward network, while introducing custom enhancements tailored to motion deblurring tasks.

Architectural Modifications

Key Changes

1. Reduction in Parameters: Number of layers and transformer blocks reduced to lower computational overhead.
2. Increased Attention Heads: Doubling attention heads per stage to enhance feature extraction while balancing computational costs.
3. Custom Loss Function: Integration of a frequency-domain loss alongside L1 pixel-wise loss for better preservation of fine details.

These modifications resulted in faster convergence, improved stability, and better performance across a range of datasets and challenging scenarios.

Training Enhancements

Added Transformations

• Color Jitter: Simulates real-world variations in lighting conditions.
• Gaussian Blur: Adds robustness against noise and blurring artifacts.
• Perspective Transform: Models geometric distortions for diverse scenarios.

Frequency-Domain Loss

Incorporates Fourier transform analysis to emphasize high-frequency details, crucial for sharp edges and textures.

The combined effect of these augmentations improves the model’s ability to generalize across diverse real-world conditions.

Datasets

This project leverages a variety of datasets for training and evaluation:

1. GoPro Dataset: Synthetic motion blur images (1280x720 resolution).
   • Dataset: Papers With Code
   • 
2. RealBlur Dataset: Real-world motion blur images with ground truth references.
   • Dataset: RealBlur Dataset
   • Variants: RealBlur-R (RAW) and RealBlur-J (JPEG).
   • 
3. Ultra-High-Definition Motion Blurred (UHDM) Dataset: High-resolution images (4K-6K) with complex blur patterns.
   • Dataset: UHDM Dataset
   • 

Evaluation Metrics

Performance is measured using:

• PSNR (Peak Signal-to-Noise Ratio): Quantifies image restoration quality.
• SSIM (Structural Similarity Index): Evaluates perceptual and structural fidelity.
• DeltaE (Color Difference): Measures color accuracy using the DeltaE2000 metric.
• LPIPS (Learned Perceptual Image Patch Similarity): Assesses perceptual similarity between restored and ground truth images.

Results

• Achieved good performance on RealBlur-R and RealBlur-J datasets.
• Demonstrated strong generalization to the UHDM dataset, despite its challenging high-resolution scenarios.
• Significant improvements in robustness, as shown by hard positive and negative case analysis.

Some Examples:

Installation

Please follow the steps below:

# Clone the repository
git clone https://github.com/hamzafer/image-deblurring
cd image-deblurring

Refer to the original Restormer repository for detailed setup instructions and dependencies.

Usage

The model weights are available upon request.

Model weights can be found here: Model Weights

Running Inference

To test the improved model on your own images:

python demo.py --task Motion_Deblurring --input_dir /path/to/images --result_dir /path/to/save_results

Training

Follow the instructions in the train directory to train the model on your dataset.

Fine-Tuning

Fine-tuning scripts for RealBlur and UHDM datasets are available in the fine_tune directory.

Acknowledgments

This work builds upon the Restormer architecture by Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. We acknowledge their contributions and innovative work in developing an efficient transformer model for high-resolution image restoration.

Citation

If you use this work or the Restormer architecture, please cite:

@article{akmaral2025efficient,
    title={Efficient Transformer for High Resolution Image Motion Deblurring},
    author={Akmaral, Amanturdieva and Zafar, Muhammad Hamza},
    journal={arXiv preprint arXiv:2501.18403},
    year={2025}
}

@inproceedings{Zamir2021Restormer,
    title={Restormer: Efficient Transformer for High-Resolution Image Restoration},
    author={Syed Waqas Zamir and Aditya Arora and Salman Khan and Munawar Hayat 
            and Fahad Shahbaz Khan and Ming-Hsuan Yang},
    booktitle={CVPR},
    year={2022}
}