Unifying Appearance Codes and Bilateral Grids for Driving Scene Gaussian Splatting

This repository contains the official implementation of our NeurIPS 2025 paper: Unifying Appearance Codes and Bilateral Grids for Driving Scene Gaussian Splatting.

Authors: Nan Wang, Yuantao Chen, Lixing Xiao, Weiqing Xiao, Bohan Li, Zhaoxi Chen, Chongjie Ye, Shaocong Xu, Saining Zhang, Ziyang Yan, Pierre Merriaux, Lei Lei, Tianfan Xue and Hao Zhao^†

✨ News

• September 18, 2025: 🎉 Paper accepted to NeurIPS 2025!

• June 6, 2025: Release paper.

• June 5, 2025: Release preprocessed data and checkpoints.

• June 4, 2025: Release code and project page.

📊 Overview

We introduce Multi-Scale Bilateral Grids that unify appearance codes and bilateral grids, significantly improving geometric accuracy in dynamic, decoupled autonomous driving scene reconstruction.

---

📋 Table of Contents

• ✨ News
• 📊 Overview
• 🚀 Quick Start
  • Environment Setup
  • Dataset Preparation
• 🔧 Usage
  • Training
  • Evaluation
  • Rendering
• 📦 Preprocessed Data
• 🤝 Citation
• 🙏 Acknowledgments

🚀 Quick Start

Environment Setup

System Requirements

• Python: 3.9
• PyTorch: 2.2.0
• CUDA: 12.1
• GPU: NVIDIA GPU with at least 8GB VRAM (recommended: 16GB+)

Installation Steps

# Clone the repository
git clone --recursive https://github.com/BigCiLeng/bilateral-driving.git
cd bilateral-driving

# Create conda environment
conda create -n bilateraldriving python=3.9 -y
conda activate bilateraldriving

# Install PyTorch with CUDA support
conda install pytorch==2.2.0 torchvision==0.17.0 torchaudio==2.2.0 pytorch-cuda=12.1 -c pytorch -c nvidia

# Install requirements
pip install -r requirements.txt
pip install git+https://github.com/nerfstudio-project/gsplat.git@v1.3.0
pip install git+https://github.com/facebookresearch/pytorch3d.git
pip install git+https://github.com/NVlabs/nvdiffrast

# Install third-party dependencies
cd project/third_party/smplx/
pip install -e .

Dataset Preparation

You can download our preprocessed nuScenes data for quick start.

Download the dataset and arrange it as the following directory structure:

bilateral-driving/
├── data/
│   ├── nuscenes/
│   │   └── processed_10Hz/
│   │       └── ...
│   ├── argoverse/
│   │   └── ...
│   ├── pandaset/
│   │   └── ...
│   └── waymo/
│       └── ...
├── docs/
├── project/
└── ...

or following the same data preprocessing pipeline from drivestudio.

Quick Start Example

For a quick start, you can download the preprocessed nuScenes data and pre-trained models:

# Download preprocessed data (example for nuScenes)
# Place the downloaded data in the data/nuscenes/ directory

# Download pre-trained models
# Place the downloaded checkpoints in the ckpts/nuscenes_pretrained_checkpoints/ directory

# Run evaluation on a single scene
cd project
export PYTHONPATH=$(pwd)
python tools/eval_metrics.py \
    --resume_from ckpts/nuscenes_pretrained_checkpoints/checkpoint_final.pth

Click to expand data process instruction

• Waymo: Data Process Instruction
• NuScenes: Data Process Instruction
• ArgoVerse: Data Process Instruction
• PandaSet: Data Process Instruction

🔧 Usage

Training

Training Script

bash scripts/train.sh

Train a Single Scene

cd project
export PYTHONPATH=$(pwd)

python tools/train.py \
    --config_file configs/omnire_ms_bilateral.yaml \
    --output_root $output_root \
    --project $project \
    --run_name $expname \
    dataset=$dataset \
    data.scene_idx=$scene_idx \
    data.start_timestep=$start_timestep \
    data.end_timestep=$end_timestep \
    data.pixel_source.test_image_stride=$test_image_stride \
    data.pixel_source.load_smpl=$load_smpl

Parameters:

• $output_root: Path to save training outputs
• $project: Project name for logging
• $expname: Experiment name
• $dataset: Dataset name (e.g., nuscenes, waymo, pandaset, argoverse)
• $scene_idx: Scene index to train
• $start_timestep: Starting timestep
• $end_timestep: Ending timestep
• $test_image_stride: Test image stride
• $load_smpl: Whether to load SMPL data

Note: For first-time users, we recommend starting with the nuScenes dataset using the provided preprocessed data and pre-trained models.

Evaluation

Download the pre-trained model checkpoints and arrange them as the following directory structure:

bilateral-driving/
├── ckpts/
│   ├── nuscenes_pretrained_checkpoints/
│   ├── pandaset_pretrained_checkpoints/
│   ├── waymo_pretrained_checkpoints/
│   └── argoverse_pretrained_checkpoints/
├── data/
├── docs/
├── project/
└── ...

Evaluation Script

bash scripts/eval.sh

Evaluate a Single Scene

cd project
export PYTHONPATH=$(pwd)

python tools/eval_metrics.py \
    --resume_from $output_root/checkpoint_final.pth

Rendering

cd project
export PYTHONPATH=$(pwd)

python tools/render.py \
    --resume_from $output_root/checkpoint_final.pth

📦 Preprocessed Data

Dataset	Resources	Download Link	Resources	Download Link
nuScenes	preprocessed data	Onedrive	pretrained models (3cams)	Onedrive
			pretrained models (6cams)	Onedrive
Waymo	preprocessed data	Onedrive	pretrained models (3cams)	Onedrive
			pretrained models (5cams)	Onedrive
Pandaset	preprocessed data	Onedrive	pretrained models (3cams)	Onedrive
			pretrained models (6cams)	Onedrive
ArgoVerse	preprocessed data	Onedrive	pretrained models (3cams)	Onedrive
			pretrained models (7cams)	Onedrive

🤝 Citation

If you find this repository helpful, please consider citing our paper:

@article{wang2025unifying,
  title={Unifying Appearance Codes and Bilateral Grids for Driving Scene Gaussian Splatting},
  author={Wang, Nan and Chen, Yuantao and Xiao, Lixing and Xiao, Weiqing and Li, Bohan and Chen, Zhaoxi and Ye, Chongjie and Xu, Shaocong and Zhang, Saining and Yan, Ziyang and others},
  journal={arXiv preprint arXiv:2506.05280},
  year={2025}
}

🙏 Acknowledgments

Thanks to these excellent open-source works and models: DriveStudio, Bilarf, Street Gaussians.