master-thesis

Repository for code related to the Masters Thesis "Neural Panorama Stitching" (June 2025).

Author: Thomas Jaron-Strugala

The code base and framework ideas were inspired by Bundle-Adjusting Neural Radiance Fields by Chen-Hsuan Lin et al.

@inproceedings{lin2021barf,
  title={BARF: Bundle-Adjusting Neural Radiance Fields},
  author={Lin, Chen-Hsuan and Ma, Wei-Chiu and Torralba, Antonio and Lucey, Simon},
  booktitle={IEEE International Conference on Computer Vision ({ICCV})},
  year={2021}
}

Install

Install the dependencies by using make.

LINUX:

make venv-python
make install-deps
make install-cuda-deps

WINDOWS:

make venv-python-windows
make install-deps-windows
make install-cuda-deps-windows

The installation was tested using Windows 11 and Python 3.12.6

Run the model

Navigate to the folder containing the repository, then use

python src/model.py

which will execute the model using the dataset within data_example. The resulting image files can be found in the folder data_example/output/vis.

About the code

The main application code can be found in the src/ directory.

The code is structured as follows:

• src/models.py: Main entrypoint for the model, starts all necessary processes

• src/datasets.py: Code for manipulation and preparation of datasets. Also is able to load SIDAR datasets for evaluation purposes.

• src/networks.py: Code for the networks - contains the homography network and the neural image representation network

• src/estimators.py: Code used to allow for various feature extractors and matchers to be added to the pipeline. Currently only uses LightGlue and DISK.

• src/utils.py: Utility functions

• src/config.py: Loads the options.yaml within the used dataset or applies default values.

How to prepare datasets

The structure of the datasets is as follows:

data/
    rgb/ -- mandatory
    homography/ -- optional
    options.yaml -- mandatory

where the options.yaml file can contain the following settings:

H: 360 -- input image dimensions
W: 480
barf_c2f: -- BARF coarse to fine parameters, if c2f is desired. Values between 0 and 1.
- 0.0
- 0.0
blending_depth: 4 -- number of blending steps between images
blending_resize_factor: 0.9 -- resize factor for blending
dataset_images: -- list of image names (strings!) to use from the dataset.
- '2'
- '5'
- '15'
- '17'
- '48'
debug: true -- whether to use debug mode for more verbose logging
device: cuda -- device to run the model on ('cuda', 'cpu')
-- model parameters and options - changes here change the behaviour of the model
homography_estimation:
  feature_extractor: disk-depth
  matcher: lightglue
  min_inliers: 50
  num_features: 2048
optim:
  algo: Adam
  lr: 0.001
  lr_warp: 0.001
max_iter: 10000
posenc: true
posenc_depth: 8
warp:
  dof: 8
  fix_first: true
  type: homography
-- debugging options
log_image: 1000
log_scalar: 20
set_estimated_homs: true
use_cropped_images: false
use_sidar: false
vis_hom: false
tb: true
-- image input/output options
output_H: 270
output_W: 360
rescale_factor: 0.2

put the images that you want to align into the rgb/ folder. The homography/ folder is optional and can contain precomputed homographies in .mat format. Currently the model assumes that the homographies are generated by SIDAR.

The first image defined in dataset_images will be fixed and will be treated as the target image for all other images.