TriCo: Enhancing Images with Coupled Low-Resolution and Ultra-Dark Degradations

Official implementation of the ACM Multimedia 2024 paper:

Enhancing Images with Coupled Low-Resolution and Ultra-Dark Degradations: A Tri-level Learning Framework
Jiaxin Gao and Yaohua Liu
Proceedings of the 32nd ACM International Conference on Multimedia (ACM MM 2024), pp. 8642-8651

[Paper (ACM DL)] [Pre-trained models] [Baidu Netdisk, code: cjzk]

Overview

TriCo jointly brightens and super-resolves images affected by coupled low-resolution and ultra-dark degradations. Instead of treating low-light enhancement and super-resolution as isolated or simply cascaded tasks, the paper formulates their bidirectional cooperation as an Upper-Middle-Lower (U-M-L) tri-level learning framework.

The method contains two named components:

• Phased Gradient-Response (PGR): the training strategy that coordinates the three groups of variables and propagates inter-variable gradient feedback in phases.
• Integrated Hybrid Expert Modulator (IHEM): the feature modulation mechanism that combines intrinsic illumination priors with MobileSAM semantic features to guide high-frequency detail recovery.

Pipeline

Repository layout

TriCo/
|-- Figures/                         # Paper pipeline and qualitative results
|-- Super_Resolution/
|   |-- Datasets/                    # Dataset notes
|   `-- Options/                     # Training and testing YAML files
|-- basicsr/
|   |-- data/                        # Paired-image data pipeline
|   |-- metrics/                     # PSNR, SSIM, NIQE, and FID utilities
|   |-- models/
|   |   |-- archs/                   # TriCo, MobileSAM, and supporting modules
|   |   `-- image_restoration_model.py
|   |-- train_search.py              # Tri-level/PGR training entry point
|   `-- test.py                      # Evaluation entry point
|-- train_blo.sh                     # Legacy wrapper for tri-level training
`-- train.sh                         # Legacy single-stage wrapper

Reproduction prerequisites

This repository is a cleaned research-code snapshot, not a containerized release. Before training or testing, prepare the following items.

1. Environment

The code was developed with Python 3.8-era PyTorch/BasicSR APIs and requires an NVIDIA CUDA environment. Core imports include:

torch, torchvision, numpy, opencv-python, scipy, scikit-image,
PyYAML, tqdm, einops, lmdb, tensorboard, wandb, ipdb,
safetensors, icecream

Install a CUDA-compatible PyTorch and TorchVision pair first, then install the remaining packages in the same environment. The release does not include a fully pinned environment lockfile, so exact package versions should be recorded when reporting reproduced results.

2. MobileSAM checkpoint

IHEM uses a frozen MobileSAM image encoder. Download the official mobile_sam.pt checkpoint from the MobileSAM repository and place it at the path expected by the released implementation:

basicsr/models/archs/MobileSAM_master/weights/mobile_sam.pt

The model cannot be constructed without this file.

3. Dataset

The released training configuration targets the paired RELLISUR layout:

RELLISUR-Dataset/
|-- Train/
|   |-- LLLR/                       # low-light, low-resolution inputs
|   `-- NLHR-Duplicates/
|       `-- X2/                     # normal-light, high-resolution targets
`-- Test_crop/
    |-- LLLR/
    `-- NLHR-Duplicates/
        `-- X2/

Edit only the dataset paths in:

Super_Resolution/Options/TriCo_searchlossv1_lr3e-4_scale2_train.yml

Set dataroot_lq to LLLR and dataroot_gt to the corresponding NLHR-Duplicates/X2 directory for both the training and validation sections.

4. Pre-trained TriCo checkpoint

For evaluation, download a TriCo checkpoint from Google Drive or Baidu Netdisk, then set path.pretrain_network_g in the evaluation YAML to the downloaded .pth file.

Training

TriCo's paper-aligned training entry point is basicsr/train_search.py, which invokes the two PGR phases according to datasets.train.s1_iteration.

From the repository root, run:

CUDA_VISIBLE_DEVICES=0 python basicsr/train_search.py \
  -opt Super_Resolution/Options/TriCo_searchlossv1_lr3e-4_scale2_train.yml

The legacy wrapper performs the same type of launch but has a machine-specific GPU index:

bash train_blo.sh Super_Resolution/Options/TriCo_searchlossv1_lr3e-4_scale2_train.yml

Important configuration fields:

• network_g.type: BrZoSAMv5 is the released internal name for the TriCo generator.
• model_type: BrZoSearchLossV1DartsStage2 is the released internal name for the tri-level learning model.
• GN: true enables the gradient-response path used by PGR.
• use_split: true creates the training/validation split used for hyper-variable learning.
• s1_iteration controls the transition between the two PGR phases.

Testing

After setting the test dataset paths and path.pretrain_network_g in:

Super_Resolution/Options/TriCo_searchlossv1_lr3e-4_scale2_test.yml

run:

CUDA_VISIBLE_DEVICES=0 python basicsr/test.py \
  -opt Super_Resolution/Options/TriCo_searchlossv1_lr3e-4_scale2_test.yml

Outputs are written under results/<experiment-name>/visualization/.

Reproducibility note: the released *_test.yml is a legacy research configuration and still mirrors several training fields. Before evaluation, keep only the intended test dataset entries, point them to accessible data, and provide both the MobileSAM and TriCo checkpoints. This documentation does not claim an end-to-end run without those external artifacts.

Results

The paper evaluates TriCo on four real and synthetic benchmarks using fidelity, perceptual, and no-reference image-quality metrics. Please refer to the published paper for the complete experimental protocol, baseline settings, and quantitative tables.

Citation

@inproceedings{gao2024trico,
  author    = {Jiaxin Gao and Yaohua Liu},
  title     = {Enhancing Images with Coupled Low-Resolution and Ultra-Dark
               Degradations: A Tri-level Learning Framework},
  booktitle = {Proceedings of the 32nd ACM International Conference on Multimedia},
  pages     = {8642--8651},
  year      = {2024},
  doi       = {10.1145/3664647.3681682}
}

Acknowledgements

The released implementation is built on the BasicSR-style restoration framework and adapts components or code structure from Restormer, MIRNet, MobileSAM, and Segment Anything. Please also follow the licenses and citation requirements of these projects.

For questions about the paper or release, contact jiaxinn.gao@outlook.com.