RESULT:

This work is based on this repo.

ORB-SLAM-3 Setup

You will have the best chance of ORB SLAM 3 working if you follow these directions exactly, including using a fresh Ubuntu 20.04 install on a newly created virtual machine.

Ensure your machine has at least 16GB of RAM and 4 cores. 32GB of RAM is recommended.

Install the VMware Workstation Player virtual machine. I did this on my Windows 11 desktop PC. During setup, customize the hardware to use the max amount of recommended RAM, and 4 processor cores. Whenever you're running the VM, kill everything else running on the PC to allow as much RAM to be used by ORB_SLAM3 as possible.

Download the Ubuntu 20.04 iso.

After setting up the OS, you may want to connect a USB device, such as a drive you have data stored on. You can do this via the top-left tab, but by default all options are grayed out. To fix this, power down the VM, navigate to your VM (mine is in Documents/Virtual Machines), and open the .vmx file with a text editor. Delete the following line

usb.restrictions.defaultAllow = “FALSE”

Now reopen the VM through the VMware launcher, and USB devices should be selectable.

Install dependencies

sudo add-apt-repository "deb http://security.ubuntu.com/ubuntu xenial-security main"
sudo apt update
sudo apt-get install build-essential
sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libdc1394-22-dev libjasper-dev
sudo apt-get install libglew-dev libboost-all-dev libssl-dev
sudo apt install libeigen3-dev
sudo apt-get install libcanberra-gtk-module

Install OpenCV

You will need two versions of OpenCV installed, 4.2.0 and 3.2.0. First install 4.2.0:

cd ~
mkdir Dev && cd Dev
git clone https://github.com/opencv/opencv.git
cd opencv
git checkout 4.2.0

Run nano ./modules/videoio/src/cap_ffmpeg_impl.hpp or your favorite text editor and add the following three lines at the top of the file:

#define AV_CODEC_FLAG_GLOBAL_HEADER (1 << 22)
#define CODEC_FLAG_GLOBAL_HEADER AV_CODEC_FLAG_GLOBAL_HEADER
#define AVFMT_RAWPICTURE 0x0020

Now build it.

mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=Release -D WITH_CUDA=OFF -D CMAKE_INSTALL_PREFIX=/usr/local ..
make -j 3
sudo make install
cd ~/Dev
mv opencv opencv4

Now install the older version:

cd ~
mkdir Dev2 && cd Dev2
git clone https://github.com/opencv/opencv.git
cd opencv
git checkout 3.2.0

Similarly, run nano ./modules/videoio/src/cap_ffmpeg_impl.hpp or your favorite text editor and add the following three lines at the top of the file:

#define AV_CODEC_FLAG_GLOBAL_HEADER (1 << 22)
#define CODEC_FLAG_GLOBAL_HEADER AV_CODEC_FLAG_GLOBAL_HEADER
#define AVFMT_RAWPICTURE 0x0020

Now build it.

mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=Release -D WITH_CUDA=OFF -D CMAKE_INSTALL_PREFIX=/usr/local ..
make -j 3
sudo make install

Install Pangolin

We will use an older commit version of Pangolin for compatibility.

cd ~/Dev2
git clone https://github.com/stevenlovegrove/Pangolin.git
cd Pangolin 
git checkout 86eb4975fc4fc8b5d92148c2e370045ae9bf9f5d
mkdir build 
cd build 
cmake .. -D CMAKE_BUILD_TYPE=Release 
make -j 3 
sudo make install

Install ORB_SLAM3

We will use an older version as well.

cd ~/Dev2
git clone https://github.com/UZ-SLAMLab/ORB_SLAM3.git 
cd ORB_SLAM3
git checkout ef9784101fbd28506b52f233315541ef8ba7af57

This version has some compilation errors that we need to fix. Run nano ./include/LoopClosing.h or your favorite text editor and change line 51 from

Eigen::aligned_allocator<std::pair<const KeyFrame*, g2o::Sim3> > > KeyFrameAndPose;

to

Eigen::aligned_allocator<std::pair<KeyFrame *const, g2o::Sim3> > > KeyFrameAndPose;

In CMakeLists.txt, change the OpenCV version line to

find_package(OpenCV 3.2)

We also need to make changes to System.cc. Open it with nano ./src/System.cc or your favorite text editor, and change both lines 584 and 701 from

Map* pBiggerMap;

to

Map* pBiggerMap = nullptr;

This latter error allowed the examples to run fine, but they caused a segmentation fault at the very end when trying to save the map.

Now all compilation errors should be fixed, and we can build ORB_SLAM3. Run their provided shell script build.sh, which will build all of the third party programs as well as ORB_SLAM3 itself.

./build.sh

It will likely show errors, but just run it a few times without changing anything, and it should succeed after a few attempts.

Download Example Data

We will use the EuRoC MH_01 easy dataset.

cd ~
mkdir -p Datasets/EuRoc
cd Datasets/EuRoc/
wget -c http://robotics.ethz.ch/~asl-datasets/ijrr_euroc_mav_dataset/machine_hall/MH_01_easy/MH_01_easy.zip
mkdir MH01
unzip MH_01_easy.zip -d MH01/

Run Simulation with Examples

cd ~/Dev/ORB_SLAM3

Then, choose one of the following to run. A map viewer as well as an image viewer should appear after it finishes setup.

# Mono
./Examples/Monocular/mono_euroc ./Vocabulary/ORBvoc.txt ./Examples/Monocular/EuRoC.yaml ~/Datasets/EuRoc/MH01 ./Examples/Monocular/EuRoC_TimeStamps/MH01.txt dataset-MH01_mono

# Mono + Inertial
./Examples/Monocular-Inertial/mono_inertial_euroc ./Vocabulary/ORBvoc.txt ./Examples/Monocular-Inertial/EuRoC.yaml ~/Datasets/EuRoc/MH01 ./Examples/Monocular-Inertial/EuRoC_TimeStamps/MH01.txt dataset-MH01_monoi

# Stereo
./Examples/Stereo/stereo_euroc ./Vocabulary/ORBvoc.txt ./Examples/Stereo/EuRoC.yaml ~/Datasets/EuRoc/MH01 ./Examples/Stereo/EuRoC_TimeStamps/MH01.txt dataset-MH01_stereo

# Stereo + Inertial
./Examples/Stereo-Inertial/stereo_inertial_euroc ./Vocabulary/ORBvoc.txt ./Examples/Stereo-Inertial/EuRoC.yaml ~/Datasets/EuRoc/MH01 ./Examples/Stereo-Inertial/EuRoC_TimeStamps/MH01.txt dataset-MH01_stereoi

If this works well, the first part of the set-up is complete! 😊

---

Using Our Own Data

I've done all of this section on a separate computer running Ubuntu 20.04 on a Virtual Machine (if you can you may create another different Machine on the same computer). This is needed because otherwise there may be some incompatibilities in different version of packages. We will use ROS Noetic.

# 1. Update your system
sudo apt update && sudo apt upgrade -y

# 2. Add the ROS package source
echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/ros-latest.list

# 3. Add the ROS key
sudo apt-key adv --keyserver 'hkp://keyserver.ubuntu.com:80' --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654

# 4. Update again after adding ROS source
sudo apt update

# 5. Install ROS Noetic (Desktop Full)
sudo apt install ros-noetic-desktop-full -y

# 6. Add ROS environment to bashrc
echo "source /opt/ros/noetic/setup.bash" >> ~/.bashrc
source ~/.bashrc

# 7. Install dependency tools
sudo apt install -y python3-rosdep python3-rosinstall python3-rosinstall-generator python3-wstool build-essential

# 8. Initialize rosdep
sudo rosdep init
rosdep update

We now have ros noetic and can build the dataset that will be used on the first computer with ORB_SLAM installed. We will need some scripts of the original repo.

# clone the repo
git clone https://github.com/kevin-robb/orb_slam_implementation

📷 Recording a ROS Bag from a USB Camera

To build a dataset from a USB camera using ROS, follow these steps. Each command should be run in a separate terminal.

---

1. Start ROS Core

roscore

---

2. Launch the USB Camera Node

rosrun usb_cam usb_cam_node _pixel_format:=yuyv

💡 You can adjust _pixel_format, resolution, or device path as needed. Example: _video_device:=/dev/video1 _image_width:=640 _image_height:=480

---

3. (Optional) Preview the Camera Feed

rosrun rqt_image_view rqt_image_view

---

4. Record the Camera Stream to a Bag File

rosbag record -O camera.bag /usb_cam/image_raw

🎥 This will save the video stream into camera.bag. Press Ctrl+C to stop recording when you're finished.

---

Converting from Rosbag to Image Set

Since we have not setup the ROS-compatible part of ORB_SLAM3, the only input we can use is a folder of images, with a corresponding .txt file of timestamps. To convert a rosbag of image data into this format, use the convert_images.py script from the original repo. It will work for both monocular and stereo cameras. It's setup with the topics for a RealSense D435 monocular camera and stereo images from the NUance autonomous car, but additional modes can be easily added to the cam_types dictionary in main() of this script. In fact, for the USB camera we do have to add a mode. On the file convert_images.py change the line

cam_options = {
"realsense": ("/camera/color/image_raw", "/NONE"),
"nuance": ("/camera_array/cam0/image_raw", "/camera_array/cam1/image_raw")
}

to

cam_options = {
"realsense": ("/camera/color/image_raw", "/NONE"),
"nuance": ("/camera_array/cam0/image_raw", "/camera_array/cam1/image_raw"),
"usb": ("/usb_cam/image_raw", "/NONE")
}

Now, we can run eveything by doing each of the following commands in a separate terminal.

roscore
python3 scripts/convert_images.py DATASET_NAME CAM_TYPE (I used python3 scripts/convert_images.py tests_usb_camera usb)
rosbag play camera.bag

Here DATASET_NAME will be the name of the resulting folder, which will be created automatically in your ~ directory.

Config and Stereo Rectification Parameters

We also need to provide ORB_SLAM3 a .yaml file containing calibration parameters for the cameras. For that, run the file calibration.py, where your image folder has pictures from multiple angles of the target chess board. In my case, the chessboard was 7x7, you may need to change that in the code.

Running ORB_SLAM3 with Our Data

We now have everything we need to run ORB_SLAM3 with this dataset. If you performed this section on a different machine, now copy the relevant files onto an external drive to use in the virtual machine.

Let's assume your external drive is called DRIVE, and it has a folder in its main directory called DATA. The directory structure should then look like:

DRIVE/
  DATA/
    DATASET_NAME/
      CONFIG.yaml
      timestamps.txt
        images/mav0/
          cam0/data/
            * all the left images
          cam1/data/
            * all the right images

So on the virtual machine, with the drive connected, you're almost ready to run ORB_SLAM3. Copy the bash script in scripts/run_orbslam.sh from this repository onto your virtual machine, and ensure it is executable with the command chmod +x run_orbslam.sh. Open it with your favorite text editor, and comment/uncomment the specified sections to set all the parameters accordingly. Then execute it:

cd ~
./run_orbslam.sh

SUGGESTED - Run the following commands:

cd ~/Dev/ORB_SLAM3

./Examples/Stereo/stereo_euroc ./Vocabulary/ORBvoc.txt ~/../../media/USERNAME/DRIVE/DATA/DATASET_NAME/CONFIG.yaml ~/../../media/USERNAME/DRIVE/DATA/DATASET_NAME/images ~/../../media/USERNAME/DRIVE/DATA/DATASET_NAME/timestamps.txt your_desired_output_dataset_name

where you should fill in USERNAME, DRIVE, DATASET_NAME, and CONFIG for your case. To run monocular instead of stereo, replace the first argument with ./Examples/Monocular/mono_euroc (I USED THIS ONE). The final command line argument will be used as part of the filename for the frame and keyframe trajectory files produced as output by ORB_SLAM3.

---