Draft: Release patch03

README.md

@@ -71,7 +71,9 @@ Clone HLOC:
 cd $HOME/dev/openvps
 git clone https://github.com/cvg/Hierarchical-Localization.git
 cd Hierarchical-Localization
-git checkout abb252080282e31147db6291206ca102c43353f7
+git checkout abb2520 # 2024-11-03, compatible with pycolmap<=3.11
+git checkout 2e2a551 # 2025-07-22, compatible with pycolmap>=3.12,<4.0
+git checkout c13273b # 2025-12-10, compatible with pycolmap==3.13,<4.0
 git submodule update --init --recursive
 ```
 
@@ -160,6 +162,11 @@ Stop services:
 docker compose --env-file docker.env --progress=plain down
 ```
 
+Run a single container for testing:
+```sh
+docker run --rm -it --gpus all openvps-backend:latest /bin/bash
+```
+
 ## Usage
 
 ### Recording
@@ -183,7 +190,6 @@ You can find example clients in Open AR Cloud repositores based on a WebXR clien
 The camera intrinsics are also required in the queries. If you send queries via spARcl, it submits the camera intrinsics automatically. If you use other clients, you need to find out somehow the intrinsics. The stock Android Photo app writes them into the EXIF metadata of the photos, you can read those out. You can also use images from StrayScanner recordings as queries (same sequence as for mapping or other recording, does not matter), because StrayScanner also saves the intrinsics. The Cesium client reads the intrinsics from the EXIF. The C++ and Python clients take the intrinsics from a JSON file that the user needs to write before the query.
 
 
-
 ## Troubleshooting
 
 ### ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm)
@@ -201,12 +207,24 @@ The identity provider cannot be reached due to either a connection issue or the
 ### FusionAuth login error
 We found that the FusionAuth container sometimes dies and this results in a misconfiguration error message on the login screen. In this occurs, restart the FusionAuth container.
 
-### CUDA/pytorch incompatibility error: 
+### Docker image and GPU driver incompatibility
+nvidia CUDA Docker images: https://gitlab.com/nvidia/container-images/cuda/-/blob/master/doc/supported-tags.md?ref_type=heads
+
+### Pytorch and GPU driver version incompatibility
 Minimum and Maximum CUDA capability supported by Pytorch depends on the version. If your GPU is older, you may need to downgrade the `torch` package that HLOC installs automatically. There is a commented section in the MapBuilder and MapLocalizer Dockerfiles for that.
 
+If you see PyTorch UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version xxxx)...
+Replace PyTorch with an earlier version if your GPU is not supported by the newest version by adding these two lines to the Dockerfile of mapbuilder-backend and maplocalizer:
+```
+RUN pip uninstall torch -y
+RUN pip install torch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1
+```
+
+When updating the host GPU drivers, there might be need to update the nvidia/cuda base images too. This command ensures to download the latest image and check the GPU compatibility: `docker run --rm --gpus all nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04 nvidia-smi` (or whichever base container is used in the subproject Dockerfiles). If the base image gets updated, you also need to rebuild the OpenVPS images.
+
 
 ## Known issues and feature ideas:
- - map height is not altitude but above ground
- - only one map can be active at a time (enough for educational purposes)
+ - The map height is not real altitude but above ground
+ - Only one map can be active at a time (sufficient for educational purposes)
  - MapBuilder should have datasets and maps separated from each other, because from one dataset we could create multiple maps by using different HLOC methods and/or parameters.
  - Support for selecting various HLOC configs on the frontend

mapbuilder/.gitignore

@@ -24,3 +24,5 @@ dist-ssr
 *.sw?
 
 /Hierarchical-Localization
+*__pycache__/
+*.pyc

mapbuilder/Backend/Dockerfile

@@ -76,8 +76,13 @@ COPY --from=hloc_context . /app/hloc
 RUN --mount=type=cache,target=/root/.cache/pip python3 -m pip install --no-build-isolation --verbose -e /app/hloc
 
 # Replace PyTorch with an earlier version if your GPU is not supported by the newest version
-#RUN pip uninstall torch -y
-#RUN pip install torch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1
+# If you see PyTorch UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version xxxx). 
+# Please update your GPU driver ... Alternatively, ... install a PyTorch version that has been compiled with your version of the CUDA driver.
+RUN pip uninstall torch -y
+RUN pip install torch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1
+
+# Replace pycolmap with an earlier version (see main README.md) that is compatible with the HLOC commit version
+RUN pip install pycolmap==3.13
 
 # Install scripts
 COPY --from=scripts_context . /app/scripts

mapbuilder/scripts_mapping/hloc_build_map.py

@@ -37,11 +37,15 @@
 import yaml # conda install -c conda-forge pyyaml
 import argparse
 from pathlib import Path
+import datetime
 
 from hloc import extract_features, match_features
 from hloc import pairs_from_exhaustive, pairs_from_retrieval, pairs_from_poses
 from hloc import reconstruction
 from pycolmap import CameraMode
+from test_gpu import getGpuInfo
+
+import torch # only for printing the version and features
 
 def read_yaml(file_path):
     with open(file_path, "r") as f:
@@ -57,6 +61,25 @@ def program_includes(step):
 
 def hloc_build_map(config):
     try:
+        print(f"# {datetime.datetime.now()} Building new HLoc map")
+        print("Checking GPU availability...")
+        print(getGpuInfo())
+        print(f"Pytorch version: {torch.__version__}")
+        device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+        print(f"Using device: {device}")
+        if device.type == 'cuda':
+            print(f"CUDA supported architectures: {torch.cuda.get_arch_list()}")
+            print(f"Device count: {torch.cuda.device_count()}")
+            print(f"Current device index: {torch.cuda.current_device()}")
+            print(f"Current device name: {torch.cuda.get_device_name(torch.cuda.current_device())}")
+            props = torch.cuda.get_device_properties(torch.cuda.current_device())
+            print(f"Devide properties: {props}")
+            print(f"  Compute Capability: {props.major}.{props.minor}")
+            print(f"  Total Memory: {props.total_memory / 1024**3:.2f} GB")
+            print(f"Memory Usage:")
+            print(f"Allocated: {round(torch.cuda.memory_allocated(0)/1024**3,1)} GB")
+            print(f"Cached:    {round(torch.cuda.memory_reserved(0)/1024**3,1)} GB")
+
         outputs = Path(config['hloc_reconstruction']['reconstruction_path'])
         features_path = outputs / 'features.h5'
         global_features_path = outputs / 'global_features.h5'

mapbuilder/scripts_mapping/stray_to_colmap.py

@@ -22,16 +22,35 @@
 
 def read_stray_data(scene):
     intrinsics = np.loadtxt(os.path.join(scene, 'camera_matrix.csv'), delimiter=',')
-    odometry = np.loadtxt(os.path.join(scene, 'odometry.csv'), delimiter=',', skiprows=1)
     imu = np.loadtxt(os.path.join(scene, 'imu.csv'), delimiter=',', skiprows=1)
     poses = []
-    for line in odometry:
-        # timestamp, frame, x, y, z, qx, qy, qz, qw
 
+    odometry = []
+    # Older format before StrayScanner v1.4:
+    # timestamp, frame, x, y, z, qx, qy, qz, qw
+    # New format since StrayScanner v1.4:
+    # timestamp, frame, x, y, z, qx, qy, qz, qw, fx, fy, cx, cy, distortion_center_x, distortion_center_y
+    # See https://github.com/strayrobots/scanner/releases/tag/v1.4 (2026-04-04)
+    # Read the first line of the odometry file to detect the format and parse accordingly.
+    with open(os.path.join(scene, 'odometry.csv'), 'r') as f:
+        first_line = f.readline()
+    if (first_line.count(',') == 8):
+        print("Detected older odometry format (timestamp, frame, x, y, z, qx, qy, qz, qw)")
+        odometry = np.loadtxt(os.path.join(scene, 'odometry.csv'), delimiter=',', skiprows=1)
+    elif (first_line.count(',') == 14):
+        print("Detected newer odometry format (timestamp, frame, x, y, z, qx, qy, qz, qw, fx, fy, cx, cy, distortion_center_x, distortion_center_y)")
+        # We noticed the some values (distortion_center_x, distortion_center_y) are missing from the odometry file,
+        # which leads to ValueError: could not convert string ' ' to float64 at row 0, column 14.
+        # To fix this, we can use the np.genfromtxt function instead of np.loadtxt, which can handle missing values.
+        # We will fill the missing values with NaN.
+        odometry = np.genfromtxt(os.path.join(scene, 'odometry.csv'), delimiter=',', skip_header=1, filling_values=np.nan)
+
+    # Get the poses of the frames from the odometry
+    for line in odometry:
         #timestamp = line[0]
         #frame_id = int(line[1])
-        position = line[2:5]
-        quaternion = line[5:]
+        position = line[2:5] # in the order of x, y, z
+        quaternion = line[5:9] # in the order of qx, qy, qz, qw
         T_WC = np.eye(4)
         T_WC[:3, :3] = Rotation.from_quat(quaternion).as_matrix()
         T_WC[:3, 3] = position

mapbuilder/scripts_mapping/test_gpu.py

@@ -0,0 +1,28 @@
+import sys
+import torch
+import json
+
+def getGpuInfo():
+    report = {}
+    pythonVersion = sys.version_info
+    report["Python version"] = str(pythonVersion)
+
+    pytorchVersion = torch.__version__
+    report["Pytorch version"] =  str(pytorchVersion)
+
+    cudaIsAvailable = torch.cuda.is_available()
+    report["CUDA is available"] = str(cudaIsAvailable)
+
+    if not cudaIsAvailable:
+        return report
+    numDevices = torch.cuda.device_count()
+    report["number of devices"] = str(numDevices)
+    curDeviceId = torch.cuda.current_device()
+    curDeviceName = torch.cuda.get_device_name(curDeviceId)
+    report["current device"] = str(curDeviceId) + " (" + str(curDeviceName) + ")"
+
+    return json.dumps(report)
+
+if __name__ == '__main__':
+    print("Checking GPU availability...")
+    print(getGpuInfo())

maplocalizer/Dockerfile

@@ -76,8 +76,13 @@ COPY --from=hloc_context . /app/hloc
 RUN --mount=type=cache,target=/root/.cache/pip python3 -m pip install --no-build-isolation --verbose -e /app/hloc
 
 # Replace PyTorch with an earlier version if your GPU is not supported by the newest version
-#RUN pip uninstall torch -y
-#RUN pip install torch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1
+# If you see PyTorch UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version xxxx). 
+# Please update your GPU driver ... Alternatively, ... install a PyTorch version that has been compiled with your version of the CUDA driver.
+RUN pip uninstall torch -y
+RUN pip install torch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1
+
+# Replace pycolmap with an earlier version (see main README.md) that is compatible with the HLOC commit version
+RUN pip install pycolmap==3.13
 
 # Install FastAPI
 RUN --mount=type=cache,target=/root/.cache/pip python3 -m pip install "fastapi[standard]" pydantic-settings

maplocalizer/server/hloc_localizer.py

@@ -117,8 +117,22 @@ def load_map_config(configPath:str|Path, rewriteRootDirFrom:str|None=None, rewri
     def load_map(self, config):
         self.config = config
 
+        print(f"Pytorch version: {torch.__version__}")
         self.device = 'cuda' if torch.cuda.is_available() else 'cpu'
-        print(f"Device: {self.device}")
+        print(f"Using device: {self.device}")
+        if self.device == 'cuda':
+            print(f"CUDA supported architectures: {torch.cuda.get_arch_list()}")
+            print(f"Device count: {torch.cuda.device_count()}")
+            print(f"Current device index: {torch.cuda.current_device()}")
+            print(f"Current device name: {torch.cuda.get_device_name(torch.cuda.current_device())}")
+            props = torch.cuda.get_device_properties(torch.cuda.current_device())
+            print(f"Devide properties: {props}")
+            print(f"  Compute Capability: {props.major}.{props.minor}")
+            print(f"  Total Memory: {props.total_memory / 1024**3:.2f} GB")
+            print(f"Memory Usage:")
+            print(f"Allocated: {round(torch.cuda.memory_allocated(0)/1024**3,1)} GB")
+            print(f"Cached:    {round(torch.cuda.memory_reserved(0)/1024**3,1)} GB")
+
 
         self.feature_conf = extract_features.confs[config['feature_conf']]
         print(f"Feature conf: {self.feature_conf}")

maplocalizer/server/main.py

@@ -10,6 +10,7 @@
 from fastapi.middleware.cors import CORSMiddleware
 
 import time
+import datetime
 from oscp.geoposeprotocol import GeoPoseRequest, GeoPoseResponse, verify_version_header
 import base64
 
@@ -20,6 +21,8 @@
 from hloc_localizer import HlocLocalizer
 from dummy_localizer import DummyLocalizer
 
+from test_gpu import getGpuInfo
+
 import env
 from functools import lru_cache
 @lru_cache
@@ -51,16 +54,23 @@ def get_settings():
 # print the env file
 print(get_settings())
 
+# print the GPU details
+print("Checking GPU availability...")
+print(getGpuInfo())
+
 
 @app.get("/")
 def read_root():
     return {"STATUS":"OpenVPS MapLocalizer is running. Use the /localize/geopose endpoint"}
 
+@app.get("/gpu_info")
+def gpu_info():
+    return getGpuInfo()
 
 # TODO: change to POST. We have it as GET for now so that it can be triggered simply from a browser
 @app.get('/load_map/{id}')
 async def load_map(id:str, response: Response):
-    print("Loading map: " + str(id))
+    print(f"# {datetime.datetime.now()} Loading map: {str(id)}")
     # NOTE: in the future, we can check whether this ID belongs to an HLoc map or other type of map, and load accordingly
 
     # check whether map with this id exists
@@ -142,19 +152,19 @@ async def localize():
 
 
 @app.get("/root_path")
-def read_main(request: Request):
+def root_path(request: Request):
     return {"root_path": request.scope.get("root_path")}
 
 
 @app.get("/current_map_id")
-def read_main():
+def current_map_id():
     return {"id": currentMapId}
 
 
 @app.post('/localize/geopose')
 async def localize(request: Request, response: Response):
     try:
-
+        print(f"# {datetime.datetime.now()} localize")
         # First verify the protocol version from the Accept header
         success, versionMajor, versionMinor = verify_version_header(request.headers)
         if not success:

maplocalizer/server/test_gpu.py

@@ -0,0 +1,28 @@
+import sys
+import torch
+import json
+
+def getGpuInfo():
+    report = {}
+    pythonVersion = sys.version_info
+    report["Python version"] = str(pythonVersion)
+
+    pytorchVersion = torch.__version__
+    report["Pytorch version"] =  str(pytorchVersion)
+
+    cudaIsAvailable = torch.cuda.is_available()
+    report["CUDA is available"] = str(cudaIsAvailable)
+
+    if not cudaIsAvailable:
+        return report
+    numDevices = torch.cuda.device_count()
+    report["number of devices"] = str(numDevices)
+    curDeviceId = torch.cuda.current_device()
+    curDeviceName = torch.cuda.get_device_name(curDeviceId)
+    report["current device"] = str(curDeviceId) + " (" + str(curDeviceName) + ")"
+
+    return json.dumps(report)
+
+if __name__ == '__main__':
+    print("Checking GPU availability...")
+    print(getGpuInfo())