[English] | 简体中文
This project converts fisheye camera images into 360° cylindrical panoramas using camera calibration and geometric projection. It can unwrap heavily distorted fisheye frames into more readable panoramic views in real time, suitable for robotics, surveillance, and computer vision tasks.
- Real-time processing: works with live streams (about 1 ms in the author's test environment).
- ROS + OpenCV integration: easy to deploy in robotic systems.
- Configurable parameters: customizable panorama resolution and mapping parameters.
- Lightweight and efficient: suitable for embedded or resource-limited platforms.
Input fisheye image:
Output 360° cylindrical panorama:
Video demo:
- Autonomous navigation: provides omnidirectional environmental perception for robots, drones, and autonomous vehicles.
- Surveillance: one fisheye camera can cover a wide area and reduce blind spots.
- Obstacle detection and mapping: can be integrated with vision algorithms for real-time perception.
- Omnidirectional vision: supports SLAM, visual odometry, and multi-camera fusion.
- VR / immersive video: converts fisheye footage into usable panoramic content.
Clone this repository into the src folder of your ROS workspace:
cd ~/catkin_ws/src
git clone https://github.com/canyueduxuan/fisheye2panorama.git
cd ..
catkin_make
rosrun fisheye2panorama fisheye2panorama_nodecd ~/catkin_ws/src/fisheye2panorama/dataset
rosbag play data.bagUse kalibr to calibrate your fisheye camera with eucm-none parameters, then write them into config/config.yaml. Also ensure your ROS topic names and message types match your actual device setup.
Since earlier experiment records were lost, here is a simple demo of four virtual pinhole cameras after undistortion (similar to vins-fisheye):
You can see that with eucm-none (suitable for >180°), boundary ghosting artifacts are removed. In practice, omni-radtan (<180°) may still produce edge ghosting.
If calibration conditions are limited or camera quality is poor, increasing cylinder_cy in config.yaml to shift the cylindrical camera optical axis upward can improve results without reducing vertical FOV.
Inspired by vins-fisheye.
3D Object Detection from a Single Fisheye Image Without a Single Fisheye Training Image shows that cylindrical images have radial translation invariance (fisheye images do not), making them more suitable for CNN-based 3D object detection.
