Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"

- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
22 changes: 14 additions & 8 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,12 @@ on:
# 1) manual run from the Actions tab
workflow_dispatch:

# 2) tagged releases (v*)
# 3) any branch push that changes code/tests/dependency manifests
# 2) weekly run to catch new dependency releases against supported Python versions
schedule:
- cron: "17 3 * * 1"

# 3) tagged releases (v*)
# 4) any branch push that changes code/tests/dependency manifests
push:
branches:
- '**' # allow all branches (needed so branches still trigger)
Expand All @@ -30,7 +34,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.12"]
python-version: ["3.12", "3.13", "3.14"]

steps:
- uses: actions/checkout@v4
Expand All @@ -42,11 +46,13 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -e .
pip install pytest pytest-cov
pip install --index-url https://download.pytorch.org/whl/cpu torch
pip install -r requirements.txt
pip install numpy
python -m pip install --index-url https://download.pytorch.org/whl/cpu torch torchvision
python -m pip install -e ".[tests]"

- name: Show dependency versions
run: |
python -m pip list
python -m pip check

- name: Run tests with coverage
run: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:

- uses: actions/setup-python@v5
with:
python-version: "3.11"
python-version: "3.12"

- name: Install doc dependencies
run: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.10"
python-version: "3.12"
- name: Build
run: |
python -m pip install --upgrade build
Expand Down
24 changes: 15 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

| **PyPI** | **Versions** | **Docs & License** | **Tests** | **Reference** |
|:---------:|:-------------:|:------------------:|:----------:|:--------------:|
| [![PyPI](https://img.shields.io/pypi/v/opensr-srgan)](https://pypi.org/project/opensr-srgan/) | ![PythonVersion](https://img.shields.io/badge/Python-v3.10%20v3.12-blue.svg)<br>![PLVersion](https://img.shields.io/badge/PytorchLightning-v2%2B-blue.svg) | [![Docs](https://img.shields.io/badge/docs-mkdocs%20material-brightgreen)](https://srgan.opensr.eu)<br>![License: Apache](https://img.shields.io/badge/license-Apache%20License%202.0-blue) | [![CI](https://github.com/ESAOpenSR/SRGAN/actions/workflows/ci.yml/badge.svg)](https://github.com/ESAOpenSR/SRGAN/actions/workflows/ci.yml)<br>[![codecov](https://codecov.io/github/ESAOpenSR/SRGAN/graph/badge.svg?token=LQ69MIMLVE)](https://codecov.io/github/ESAOpenSR/SRGAN) | [![arXiv](https://img.shields.io/badge/arXiv-2511.10461-b31b1b.svg)](https://arxiv.org/abs/2511.10461) <br> [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.20088994.svg)](https://doi.org/10.5281/zenodo.20088994)
| [![PyPI](https://img.shields.io/pypi/v/opensr-srgan)](https://pypi.org/project/opensr-srgan/) | ![PythonVersion](https://img.shields.io/badge/Python-v3.12--v3.14-blue.svg)<br>![PLVersion](https://img.shields.io/badge/PyTorchLightning-v2%2B-blue.svg) | [![Docs](https://img.shields.io/badge/docs-mkdocs%20material-brightgreen)](https://srgan.opensr.eu)<br>![License: Apache](https://img.shields.io/badge/license-Apache%20License%202.0-blue) | [![CI](https://github.com/ESAOpenSR/SRGAN/actions/workflows/ci.yml/badge.svg)](https://github.com/ESAOpenSR/SRGAN/actions/workflows/ci.yml)<br>[![codecov](https://codecov.io/github/ESAOpenSR/SRGAN/graph/badge.svg?token=LQ69MIMLVE)](https://codecov.io/github/ESAOpenSR/SRGAN) | [![arXiv](https://img.shields.io/badge/arXiv-2511.10461-b31b1b.svg)](https://arxiv.org/abs/2511.10461) <br> [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.20088994.svg)](https://doi.org/10.5281/zenodo.20088994)



Expand All @@ -26,19 +26,25 @@ Full docs live at **[srgan.opensr.eu](https://srgan.opensr.eu/)**. They cover us
* **Remote-sensing aware losses:** combine spectral, perceptual, and adversarial objectives with tunable weights.
* **Stable training loop:** generator pretraining, adversarial ramp-ups, EMA, and multi-GPU Lightning support out of the box.
* **PyPI distribution:** `pip install opensr-srgan` for ready-to-use presets or custom configs.
* **Extensive Logging:** Logging all important information automatically to `WandB` for optimal insights.
* **Experiment tracking:** Log metrics and validation imagery to Weights & Biases, or use local CSV logs for lightweight runs.

---

## 🏗️ Configuration Examples

All key knobs are exposed via YAML in the `opensr_srgan/configs` folder:
All key knobs are exposed via YAML in the `opensr_srgan/configs` folder. You can keep using the legacy single-file workflow, or use the Hydra entry point for composable experiment presets and command-line overrides:

* **Model**: `in_channels`, `n_channels`, `n_blocks`, `scale`, ESRGAN knobs (`growth_channels`, `res_scale`, `out_channels`), `block_type ∈ {SRResNet, res, rcab, rrdb, lka}`
```bash
python -m opensr_srgan.train --config opensr_srgan/configs/config_training_example.yaml
python -m opensr_srgan.train_hydra experiment=10m Training.max_epochs=5 Logging.wandb.enabled=false
srgan-train experiment=20m Training.gpus=[0,1]
```

* **Model**: `Model.in_bands`, `Generator.n_channels`, `Generator.n_blocks`, `Generator.scaling_factor`, ESRGAN knobs (`growth_channels`, `res_scale`, `out_channels`, `use_icnr`), and `Generator.block_type ∈ {standard, res, rcab, rrdb, lka}`
* **Losses**: `l1_weight`, `sam_weight`, `perceptual_weight`, `tv_weight`, `adv_loss_beta`
* **Training**: `pretrain_g_only`, `g_pretrain_steps` (`-1` keeps generator-only pretraining active indefinitely), `adv_loss_ramp_steps`, `label_smoothing`, generator LR warmup (`Schedulers.g_warmup_steps`, `Schedulers.g_warmup_type`), discriminator cadence controls
* **Training**: `pretrain_g_only`, `g_pretrain_steps` (`-1` keeps generator-only pretraining active indefinitely), `adv_loss_ramp_steps`, `label_smoothing`, generator LR warmup (`Schedulers.g_warmup_steps`, `Schedulers.g_warmup_type`), EMA, and manual gradient clipping
* **Adversarial mode**: `Training.Losses.adv_loss_type` (`bce`/`wasserstein`) and optional `Training.Losses.relativistic_average_d` for BCE-based relativistic-average GAN updates
* **Data**: band order, normalization stats, crop sizes, augmentations
* **Data**: `Data.dataset_type`, dataloader sizes/workers, dataset paths/manifests, and normalization policy

---

Expand All @@ -60,7 +66,7 @@ The schedule and ramp make training **easier, safer, and more reproducible**.
| Component | Options | Config keys |
|-----------|---------|-------------|
| **Generators** | `SRResNet`, `res`, `rcab`, `rrdb`, `lka`, `esrgan`, `stochastic_gan` | `Generator.model_type`, depth via `Generator.n_blocks`, width via `Generator.n_channels`, kernels/scale plus ESRGAN-specific `growth_channels`, `res_scale`, `out_channels`. |
| **Discriminators** | `standard` `SRGAN`, `CNN`, `patchgan`, `esrgan` | `Discriminator.model_type`, granularity with `Discriminator.n_blocks`, spectral norm toggle via `Discriminator.use_spectral_norm`, ESRGAN-specific `base_channels`, `linear_size`. |
| **Discriminators** | `standard` SRGAN, `patchgan`, `esrgan` | `Discriminator.model_type`, granularity with `Discriminator.n_blocks`, spectral norm toggle via `Discriminator.use_spectral_norm`, ESRGAN-specific `base_channels`, `linear_size`. |
| **Content losses** | L1, Spectral Angle Mapper, VGG19/LPIPS perceptual metrics, Total Variation | Weighted by `Training.Losses.*` (e.g. `l1_weight`, `sam_weight`, `perceptual_weight`, `perceptual_metric`, `tv_weight`). |
| **Adversarial loss** | BCE‑with‑logits or Wasserstein critic | Controlled by `Training.Losses.adv_loss_type`, warmup via `Training.pretrain_g_only`, ramped by `adv_loss_ramp_steps`, capped at `adv_loss_beta`, optional label smoothing. For BCE, enable `Training.Losses.relativistic_average_d` for RaGAN-style relative logits. |

Expand All @@ -77,8 +83,8 @@ Follow the [installation instructions](https://srgan.opensr.eu/getting-started/)

* To test the package immediately, launch the Google Colab right now and follow along the introduction! [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/16W0FWr6py1J8P4po7JbNDMaepHUM97yL?usp=sharing)
* **Datasets:** Grab the bundled example dataset or learn how to register your own sources in the [data guide](https://srgan.opensr.eu/data/).
* **Training:** Launch training with `python -m opensr_srgan.train --config opensr_srgan/configs/config_10m.yaml` or import `train` from the package as described in the [training walkthrough](https://srgan.opensr.eu/training/).
* **Inference:** Ready-made presets and large-scene pipelines are described in the [inference section](https://srgan.opensr.eu/getting-started/inference/).
* **Training:** Launch a smoke test with `python -m opensr_srgan.train --config opensr_srgan/configs/config_training_example.yaml`, or use Hydra with `python -m opensr_srgan.train_hydra experiment=example`. Use the `10m` and `20m` presets once the matching datasets are configured.
* **Inference:** Ready-made presets and large-scene pipelines are described in the [inference section](https://srgan.opensr.eu/inference/).

---

Expand Down
14 changes: 14 additions & 0 deletions docs/api-reference/training.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,20 @@ OmegaConf-based experiment files.
- "!^_"
members_order: source

## Hydra entry point

::: opensr_srgan.train_hydra
handler: python
options:
show_root_heading: false
show_root_full_path: false
show_source: true
members:
- main
filters:
- "!^_"
members_order: source

## Trainer configuration helpers

::: opensr_srgan.utils.build_trainer_kwargs
Expand Down
8 changes: 4 additions & 4 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Because every generator variant (residual, RCAB, RRDB, large-kernel attention, E
`Generator.model_type`/`block_type` and `Discriminator.model_type`. Unsupported combinations fail fast with clear error
messages.
* **Loss construction.** `GeneratorContentLoss` (from `opensr_srgan.model.loss`) provides L1, spectral angle mapper (SAM), perceptual, and
total-variation terms. Adversarial supervision uses `torch.nn.BCEWithLogitsLoss` with optional label smoothing.
total-variation terms. Adversarial supervision uses BCE-with-logits or Wasserstein objectives, with optional label smoothing, relativistic-average BCE, and R1 penalty depending on the config.
* **Optimiser scheduling.** `configure_optimizers()` returns paired Adam optimisers (generator + discriminator) with
`ReduceLROnPlateau` schedulers that monitor a configurable validation metric.
* **Training orchestration.** `setup_lightning()` binds `training_step_PL2()` and enables manual optimisation
Expand Down Expand Up @@ -56,7 +56,7 @@ The generator zoo lives under `opensr_srgan/model/generators/` and can be select
* **Stochastic GAN generator (`cgan_generator.py`).** Extends the flexible generator with conditioning inputs and latent noise,
enabling experiments where auxiliary metadata influences the super-resolution output.
* **ESRGAN generator (`esrgan.py`).** Implements the RRDBNet trunk introduced with ESRGAN, exposing `n_blocks`, `growth_channels`,
and `res_scale` so you can dial in deeper receptive fields and sharper textures. The implementation supports original features like Relativistic Average GAN (RaGAN), and the codebase allows a two-step training phase (content-oriented pretraining of the generator followed by adversarial training with the discriminator), as originally proposed by the ESRGAN authors.
and `res_scale` so you can dial in deeper receptive fields and sharper textures. The shared training loop supports optional relativistic-average BCE and a two-step training phase (content-oriented pretraining of the generator followed by adversarial training with the discriminator).
* **Advanced variants (`SRGAN_advanced.py`).** Provides additional block implementations and compatibility aliases exposed in
`__init__.py` for backwards compatibility.

Expand Down Expand Up @@ -89,9 +89,9 @@ The same module exposes `return_metrics()` so validation can log PSNR/SSIM-style
## Data flow and normalisation

The Lightning module expects batches of `(lr_imgs, hr_imgs)` tensors supplied by the `LightningDataModule` returned from
`opensr_srgan/data/dataset_selector.py`. `predict_step()` and the validation hooks rely on two utilities from `opensr_srgan.utils.spectral_helpers`:
`opensr_srgan/data/dataset_selector.py`. `predict_step()` uses the configured `opensr_srgan.data.utils.Normalizer` plus histogram matching from `opensr_srgan.utils.radiometrics`:

* `normalise_10k`: Converts Sentinel-2 style reflectance values between `[0, 10000]` and `[0, 1]`.
* `Normalizer`: Applies the configured `Data.normalization` strategy, such as `normalise_10k`, `zero_one`, or a custom callable.
* `histogram`: Matches the SR histogram to the LR reference to minimise domain gaps during inference.

These helpers allow the generator to operate in a normalised space while still reporting outputs in physical units when needed.
Expand Down
27 changes: 21 additions & 6 deletions docs/configuration.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Configuration

ESA OpenSR relies on YAML files to control every aspect of the training pipeline. This page documents the available keys and how they influence the underlying code. Use `opensr_srgan/configs/config_20m.yaml` and `opensr_srgan/configs/config_10m.yaml` as starting points.
ESA OpenSR relies on YAML files to control every aspect of the training pipeline. This page documents the available keys and how they influence the underlying code. Use `opensr_srgan/configs/config_training_example.yaml` for a smoke test, `config_10m.yaml` for SEN2NAIP-style runs, or the Hydra presets described below for composable experiments.

## File structure

Expand All @@ -23,9 +23,9 @@ Each section maps directly to parameters consumed inside `opensr_srgan/model/SRG

| Key | Default | Description |
| --- | --- | --- |
| `train_batch_size` | 12 | Mini-batch size for the training dataloader. Falls back to `batch_size` if set. |
| `train_batch_size` | 8 | Mini-batch size for the training dataloader. Falls back to `batch_size` if set. |
| `val_batch_size` | 8 | Batch size for validation. |
| `num_workers` | 6 | Number of worker processes for both dataloaders. |
| `num_workers` | 4 | Number of worker processes for both dataloaders. |
| `prefetch_factor` | 2 | Additional batches prefetched by each worker. Ignored when `num_workers == 0`. |
| `dataset_type` | `ExampleDataset` | Dataset selector consumed by `opensr_srgan.data.dataset_selector.select_dataset`. |
| `normalization` | `'sen2_stretch'` | Normalisation strategy applied to input tensors. Accepts a string alias or a mapping (see below). |
Expand Down Expand Up @@ -211,7 +211,7 @@ Both optimisers share the same configuration keys because they use `torch.optim.

| Key | Default | Description |
| --- | --- | --- |
| `metric` | `val_metrics/l1` | Validation metric monitored for plateau detection. |
| `metric` | `val_g_loss` / `val_d_loss` fallback | Shared fallback monitor when `metric_g` or `metric_d` is omitted. Shipped configs set explicit monitor keys. |
| `metric_g` | — | Optional override for the generator scheduler monitor. |
| `metric_d` | — | Optional override for the discriminator scheduler monitor. |
| `patience_g` | `100` | Epochs with no improvement before reducing the generator LR. |
Expand All @@ -220,7 +220,6 @@ Both optimisers share the same configuration keys because they use `torch.optim.
| `factor_d` | `0.5` | Multiplicative factor applied to the discriminator LR upon plateau. |
| `cooldown` | `0` | Number of epochs to wait after an LR drop before resuming plateau checks. |
| `min_lr` | `1e-7` | Minimum learning rate allowed for both schedulers. |
| `verbose` | `True` | Enables scheduler logging messages. |
| `g_warmup_steps` | `2000` | Number of optimiser steps used for generator LR warmup. Set to `0` to disable. |
| `g_warmup_type` | `cosine` | Warmup curve for the generator LR (`cosine` or `linear`). |

Expand All @@ -235,7 +234,23 @@ different validation metrics.

| Key | Default | Description |
| --- | --- | --- |
| `num_val_images` | `5` | Number of validation batches visualised and logged to Weights & Biases each epoch. |
| `num_val_images` | `5` | Number of validation batches visualised and logged to Weights & Biases when W&B is enabled. |
| `wandb.enabled` | `False` in example configs | Enables `WandbLogger`; when false, training uses `CSVLogger`. |
| `wandb.entity` | `opensr` | W&B entity/team name. |
| `wandb.project` | `SRGAN_10m` | Project name used by W&B and by the default checkpoint directory. |
| `output_dir` | unset | Optional explicit run directory. Hydra sets this to its runtime output directory; legacy YAML runs fall back to `logs/<project>/<timestamp>`. |

## Hydra experiment configs

The legacy single-file configs remain supported, but new experiments can also be launched through Hydra:

```bash
python -m opensr_srgan.train_hydra experiment=example
python -m opensr_srgan.train_hydra experiment=10m Training.max_epochs=5 Logging.wandb.enabled=false
srgan-train experiment=20m Training.gpus=[0,1]
```

Hydra configs live under `opensr_srgan/configs/hydra`. The root `train.yaml` selects simple defaults, while `experiment/example.yaml`, `experiment/10m.yaml`, and `experiment/20m.yaml` reproduce the shipped flat YAML presets through grouped `data`, `model`, `training`, and `logging` configs. Choose `experiment=...` for a full known setup; use direct overrides for a temporary change without editing files. See [Hydra Experiments](hydra.md) for a complete walkthrough.

## Tips for managing configurations

Expand Down
14 changes: 13 additions & 1 deletion docs/data.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ from opensr_srgan.data.example_data.download_example_dataset import get_example_
get_example_dataset()
```

The helper pulls `example_dataset.zip` from the [`simon-donike/SR-GAN`](https://huggingface.co/simon-donike/SR-GAN) repository, extracts it, and removes the temporary archive once the copy completes.
The helper pulls `example_dataset.zip` from the [`simon-donike/SR-GAN`](https://huggingface.co/simon-donike/SR-GAN) repository through the Hugging Face cache and extracts it into the requested output directory.

### Directory layout

Expand All @@ -35,6 +35,18 @@ Data:
The training loop automatically instantiates `opensr_srgan.data.example_data.example_dataset.ExampleDataset` for both the
training and validation dataloaders.

## Built-in dataset selectors

`Data.dataset_type` is resolved by `opensr_srgan.data.dataset_selector.select_dataset`. The currently supported selectors are:

| `Data.dataset_type` | Required fields | Notes |
| --- | --- | --- |
| `ExampleDataset` | none beyond the downloaded `example_dataset/` folder | Quick smoke-test dataset generated from HR `.npz` chips. |
| `SEN2NAIP` / `sen2naip` | `Data.sen2naip_taco_file`; optional `Data.sen2naip_val_fraction` | Requires `tacoreader` and a Taco manifest. Used by the 10 m preset once the manifest path is configured. |
| `LRHRFolderDataset` | `Data.root_dir` or `Data.dataset_root` | Expects paired LR/HR folders; see the dataset class tests for the accepted layout. |

Older labels such as `S2_6b`, `SPOT6`, or `cv` are not registered selector keys in the current code. If you start from an older config that uses one of those names, update `Data.dataset_type` or add a matching branch to the selector.

## Adding new datasets

When you are ready to move beyond the bundled sample data you can register a custom dataset. The repository uses a single factory function `opensr_srgan.data.dataset_selector.select_dataset` to keep the training script agnostic of individual
Expand Down
Loading
Loading