From 4d3e1088c156419c018a7c8a78b8603cec7e3b47 Mon Sep 17 00:00:00 2001 From: Glenn Wagner Date: Tue, 14 Jul 2026 16:12:12 +1000 Subject: [PATCH] Implement the mapping API Adds commands for -getting hovermap status -setting scan name prefix -starting/stopping scans -fetching the name of all completed scans -downloading specified scan --- README.md | 44 +++++ docker/Dockerfile | 5 +- docker/run_docker | 31 ++- src/hovermap_api/CMakeLists.txt | 3 + src/hovermap_api/launch/api.launch | 4 + src/hovermap_api/package.xml | 1 + src/hovermap_api/scripts/configure_perception | 0 src/hovermap_api/scripts/http_interface_node | 187 ++++++++++++++++++ src/hovermap_api_msgs/CMakeLists.txt | 24 +++ src/hovermap_api_msgs/msg/HovermapStatus.msg | 4 + src/hovermap_api_msgs/msg/ScanInformation.msg | 2 + .../msg/ScanInformationList.msg | 1 + src/hovermap_api_msgs/package.xml | 19 ++ 13 files changed, 323 insertions(+), 2 deletions(-) mode change 100644 => 100755 src/hovermap_api/scripts/configure_perception create mode 100755 src/hovermap_api/scripts/http_interface_node create mode 100644 src/hovermap_api_msgs/CMakeLists.txt create mode 100644 src/hovermap_api_msgs/msg/HovermapStatus.msg create mode 100644 src/hovermap_api_msgs/msg/ScanInformation.msg create mode 100644 src/hovermap_api_msgs/msg/ScanInformationList.msg create mode 100644 src/hovermap_api_msgs/package.xml diff --git a/README.md b/README.md index 1d67f97..1082af3 100644 --- a/README.md +++ b/README.md @@ -92,11 +92,19 @@ used for your connection. On a different shell check the `rostopic list` output is: ``` bash + /cortex/download_scan + /cortex/hovermap_status /cortex/lidar/corrected /cortex/mule_bridge/status /cortex/occupancy_grid_map/configuration /cortex/occupancy_grid_map/data /cortex/odometry + /cortex/scan_download_successful + /cortex/scan_names_request + /cortex/scan_names_response + /cortex/set_scan_prefix + /cortex/start_scan + /cortex/stop_scan /cortex/tf /cortex/tf_static /rosout @@ -120,12 +128,20 @@ used for your connection. | cortex/odometry | nav_msgs/Odometry | SLAM corrected odometry | 100 | | | cortex/tf | tf_msgs/TFMessage | Non-static transform | Variable | | | cortex/tf_static | tf_msgs/TFMessage | Static transforms | Once (latched) | | +| cortex/hovermap_status | hovermap_api_msgs/HovermapStatus | Hovermap and current scan status | 1 | | +| cortex/scan_names_response | hovermap_api_msgs/ScanInformationList | List of scans stored on the Hovermap newest to oldest | On request | Reply to `cortex/scan_names_request` | +| cortex/scan_download_successful | std_msgs/Bool | Outcome of a scan download: `true` on success, `false` on failure | On download completion | Published after each `cortex/download_scan` request finishes | ### Sent from client | Topic name | Type | Description | |-------------------------------------------|------------------------|------------------------------------| | cortex/occupancy_grid_map/configuration | std_msgs/String | Occupancy grid config YAML endpoint| +| cortex/scan_names_request | std_msgs/Empty | Request the list of stored scans | +| cortex/set_scan_prefix | std_msgs/String | Set the prefix used to name new scans | +| cortex/start_scan | std_msgs/Empty | Start a mapping scan | +| cortex/stop_scan | std_msgs/Empty | Stop the current mapping scan | +| cortex/download_scan | std_msgs/String | Download a stored scan by name | ### Topic details @@ -169,6 +185,34 @@ used for your connection. - The configuration is sent to the Hovermap as a string representing a YAML file. A node is provided to do this. - The custom configuration is only used when using external_api, and is persistent across runs +6. Hovermap status (`cortex/hovermap_status`) + - Reports the current status of the Hovermap and any scan in progress. The `HovermapStatus` message contains: + - `scan_prefix`: the prefix currently applied when naming new scans (see `cortex/set_scan_prefix`) + - `current_scan_name`: the name of the current or most recent scan + - `free_space`: the storage available on the Hovermap, in bytes + - `scan_running`: `true` while a scan is in progress (including while it is starting up), `false` otherwise + +7. Setting the scan prefix (`cortex/set_scan_prefix`) + - Publish a `std_msgs/String` to `cortex/set_scan_prefix` to set the prefix used when naming new scans. New scans are named `_NN` where NN is a number starting at 01 and incremented after each scan is started. The prefix is persistent across boots. + +8. Starting and stopping a scan (`cortex/start_scan`, `cortex/stop_scan`) + - Publish a `std_msgs/Empty` to `cortex/start_scan` to start a mapping scan, or to `cortex/stop_scan` to stop the current scan. + - Starting and stopping can take a few seconds + +9. Listing stored scans (`cortex/scan_names_request` → `cortex/scan_names_response`) + - Publish a `std_msgs/Empty` to `cortex/scan_names_request` to request the list of scans stored on the Hovermap + - The Hovermap replies on `cortex/scan_names_response` with a `ScanInformationList`, ordered newest first. `ScanInformationList` and its constituent `ScanInformation` messages are custom messages available in `hovermap_api_msgs`. Each `ScanInformation` entry contains: + - `name`: the scan name, as used by `cortex/download_scan` + - `size`: the size of the scan on the Hovermap, in bytes (the downloaded archive is compressed and will be smaller) + +10. Downloading a scan (`cortex/download_scan`) + - Publish a `std_msgs/String` containing a scan `name` (from `cortex/scan_names_response`) to download that scan + - Scans with duration greater than 1.5 hours may cause the bridge to timeout while waiting for hovermap to prepare the scan for download. These scans can still be downloaded manually using a USB key or the webui. + - The scan is saved as `.zip` in `/data/downloads`. Bind-mount a host directory to this location by passing it to `run_docker` (e.g. `./docker/run_docker ~/hovermap_scans`) to retrieve downloads on the client machine. + - A scan cannot be downloaded while a scan is running, or while another download is already in progress + - Download progress is reported in rosout + - When the download finishes, a `std_msgs/Bool` is published on `cortex/scan_download_successful` reporting the outcome: `true` if the scan was downloaded successfully, `false` if it failed (e.g. a scan is running, another download is in progress, or a network error occurred) + ## Time Synchronisation The Hovermap is configured to act as an NTP server to allow API users synchronise the client's clock to the Hovermap's by: diff --git a/docker/Dockerfile b/docker/Dockerfile index d2ae900..52f49c5 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -16,9 +16,12 @@ RUN apt update && apt install -y \ python3-catkin-tools \ python3-pip +# Install pip dependencies (only rebuilds if requirements.txt changes) +RUN --mount=type=bind,src=src/mule_bridge/mule_bridge/requirements.txt,dst=/tmp/requirements.txt \ + python3 -m pip install -r /tmp/requirements.txt + # Build the repo into the context RUN --mount=type=bind,src=src/,dst=/ros_ws/src \ - python3 -m pip install -r src/mule_bridge/mule_bridge/requirements.txt && \ source /opt/ros/noetic/setup.bash && \ catkin init && \ catkin config --install && \ diff --git a/docker/run_docker b/docker/run_docker index 9d46f36..a5dcd8d 100755 --- a/docker/run_docker +++ b/docker/run_docker @@ -2,9 +2,38 @@ # Script to run the Docker container for the hovermap_ros_api. # ensure using the host network to prevent connection issues. +# Usage: run_docker [download_directory] set -euo pipefail +usage() { + echo "Usage: $(basename "$0") [download_directory]" + echo "" + echo "Run the hovermap_ros_api Docker container." + echo "" + echo "Arguments:" + echo " download_directory Optional host directory to bind-mount for scan downloads" + echo "" + echo "Options:" + echo " -h, --help Show this help message and exit" +} + +if [[ ${1:-} == "-h" || ${1:-} == "--help" ]]; then + usage + exit 0 +fi + IMAGE_NAME="hovermap_ros_api_image" REPO_ROOT="$(dirname "$(realpath "$0")")/.." -docker run -it --rm --network=host --mount type=bind,src="$REPO_ROOT/src",dst=/ros_ws/src "${IMAGE_NAME}" + +EXTRA_MOUNTS="" +if [[ $# -ge 1 ]]; then + DOWNLOAD_DIR="$(realpath "$1")" + mkdir -p "$DOWNLOAD_DIR" + EXTRA_MOUNTS="--mount type=bind,src=$DOWNLOAD_DIR,dst=/data/downloads" +fi + +docker run -it --rm --network=host \ + --mount type=bind,src="$REPO_ROOT/src",dst=/ros_ws/src \ + $EXTRA_MOUNTS \ + "${IMAGE_NAME}" diff --git a/src/hovermap_api/CMakeLists.txt b/src/hovermap_api/CMakeLists.txt index ee3cc1b..8330dae 100644 --- a/src/hovermap_api/CMakeLists.txt +++ b/src/hovermap_api/CMakeLists.txt @@ -2,12 +2,14 @@ cmake_minimum_required(VERSION 3.16.3) project(hovermap_api) find_package(catkin REQUIRED COMPONENTS + hovermap_api_msgs nav_msgs sensor_msgs tf2_ros ) catkin_package(CATKIN_DEPENDS + hovermap_api_msgs nav_msgs sensor_msgs tf2_ros @@ -16,6 +18,7 @@ catkin_package(CATKIN_DEPENDS catkin_install_python( PROGRAMS scripts/configure_perception + scripts/http_interface_node DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}) diff --git a/src/hovermap_api/launch/api.launch b/src/hovermap_api/launch/api.launch index 559400c..c2b1a48 100644 --- a/src/hovermap_api/launch/api.launch +++ b/src/hovermap_api/launch/api.launch @@ -3,4 +3,8 @@ + + + + diff --git a/src/hovermap_api/package.xml b/src/hovermap_api/package.xml index 9e1993c..ef44718 100644 --- a/src/hovermap_api/package.xml +++ b/src/hovermap_api/package.xml @@ -11,6 +11,7 @@ catkin mule_bridge + hovermap_api_msgs nav_msgs sensor_msgs tf2_ros diff --git a/src/hovermap_api/scripts/configure_perception b/src/hovermap_api/scripts/configure_perception old mode 100644 new mode 100755 diff --git a/src/hovermap_api/scripts/http_interface_node b/src/hovermap_api/scripts/http_interface_node new file mode 100755 index 0000000..8e29454 --- /dev/null +++ b/src/hovermap_api/scripts/http_interface_node @@ -0,0 +1,187 @@ +#!/usr/bin/python3 +import json +import os +import threading +import rospy +import urllib.request +import urllib.error +from std_msgs.msg import Bool, Empty, String +from hovermap_api_msgs.msg import ScanInformation, ScanInformationList, HovermapStatus + +# Maps ip_prefix from mule.yaml to the actual Hovermap address +IP_PREFIX_TO_ADDRESS = { + "192.168.2.0": "192.168.2.115", + "192.168.3.0": "192.168.3.115", + "10.9.0.0": "10.9.0.1", +} + +DOWNLOAD_DIR = "/data/downloads" + +STATUS_POLL_INTERVAL = 1 # seconds +SCAN_DOWNLOAD_PREP_TIMEOUT = 450.0 # seconds to wait for zip file to be generated + +# Payload states (from the /status "state" field) that do NOT correspond to a +# running scan. Every other state is a scan actively running or transitioning +# through the start/stop sequence, so scan_running is True for them. +NON_RUNNING_STATES = frozenset( + { + "Stopped", + "Disabled", + "Booting", + "USB Error", + "USB Mounted", + "Transferring Data", + "Processes Stopped", + } +) + + +class HttpInterfaceNode: + def __init__(self): + rospy.init_node("http_interface_node") + + ip_prefix = rospy.get_param("~ip_prefix") + assert isinstance(ip_prefix, str) # To make pyright happy + if ip_prefix not in IP_PREFIX_TO_ADDRESS: + rospy.logfatal(f"Unknown ip_prefix: {ip_prefix}") + raise RuntimeError(f"Unknown ip_prefix: {ip_prefix}") + + self._base_url = f"http://{IP_PREFIX_TO_ADDRESS[ip_prefix]}" + + os.makedirs(DOWNLOAD_DIR, exist_ok=True) + + rospy.Subscriber("start_scan", Empty, self._on_start_scan) + rospy.Subscriber("stop_scan", Empty, self._on_stop_scan) + rospy.Subscriber("scan_names_request", Empty, self._on_scan_names_request) + rospy.Subscriber("download_scan", String, self._on_download_scan) + rospy.Subscriber("set_scan_prefix", String, self._on_set_scan_prefix) + + self._scan_names_pub = rospy.Publisher( + "scan_names_response", ScanInformationList, queue_size=5 + ) + self._status_pub = rospy.Publisher( + "hovermap_status", HovermapStatus, queue_size=5 + ) + self._scan_download_successful_pub = rospy.Publisher( + "scan_download_successful", Bool, queue_size=5 + ) + + rospy.Timer(rospy.Duration(STATUS_POLL_INTERVAL), self._poll_status) + + rospy.loginfo(f"http_interface_node ready, target: {self._base_url}") + + def _get(self, path, log=True): + url = f"{self._base_url}{path}" + try: + req = urllib.request.Request(url) + with urllib.request.urlopen(req, timeout=5.0) as response: + body = response.read().decode() + if log: + rospy.loginfo(f"GET {url} -> {response.status}: {body}") + return body + except urllib.error.HTTPError as e: + rospy.logerr(f"GET {url} failed: HTTP {e.code} {e.reason}") + return None + except urllib.error.URLError as e: + rospy.logerr(f"GET {url} failed: {e.reason}") + return None + + def _on_start_scan(self, msg): + self._get("/startsystem?mission_type=1") + + def _on_stop_scan(self, msg): + self._get("/stopsystem") + + def _on_scan_names_request(self, msg): + body = self._get("/files") + if body is None: + return + try: + data = json.loads(body) + scans = sorted(data.get("scans", []), key=lambda s: s["time"], reverse=True) + scan_list = ScanInformationList() + for scan in scans: + info = ScanInformation() + info.name = scan["scan"] + info.size = scan["size"] + scan_list.scans.append(info) # pyright: ignore[reportOptionalMemberAccess] + self._scan_names_pub.publish(scan_list) + except (json.JSONDecodeError, KeyError) as e: + rospy.logerr(f"Failed to parse /files response: {e}") + + def _on_download_scan(self, msg): + thread = threading.Thread( + target=self._download_scan, args=(msg.data,), daemon=True + ) + thread.start() + + def _download_scan(self, scan_name): + """Downloads the specified scans from the Hovermap. + + Hovermap will abort returning a HTTP 404 error if a scan is running, or + if you are already downloading a scan, so we don't need guards + """ + url = f"{self._base_url}/downloadscan?scanname={scan_name}" + output_path = os.path.join(DOWNLOAD_DIR, f"{scan_name}.zip") + rospy.loginfo(f"Downloading scan '{scan_name}' to {output_path}") + result_msg = Bool() + try: + req = urllib.request.Request(url) + with urllib.request.urlopen( + req, timeout=SCAN_DOWNLOAD_PREP_TIMEOUT + ) as response: + rospy.loginfo("Zip file generation completed, starting download...") + total_size = int(response.headers.get("Content-Length", 0)) + downloaded = 0 + last_logged_pct = 0 + with open(output_path, "wb") as f: + while True: + chunk = response.read(1048576) + if not chunk: + break + f.write(chunk) + + # Progress messages + downloaded += len(chunk) + if total_size > 0: + pct = (downloaded * 100) // total_size + if pct >= last_logged_pct + 10: + last_logged_pct = pct - (pct % 10) + rospy.loginfo( + f"Download '{scan_name}': {last_logged_pct}%" + ) + rospy.loginfo(f"Download complete: {output_path}") + result_msg.data = True + self._scan_download_successful_pub.publish(result_msg) + except urllib.error.HTTPError as e: + rospy.logerr(f"Download failed for '{scan_name}': HTTP {e.code} {e.reason}") + self._scan_download_successful_pub.publish(result_msg) + except urllib.error.URLError as e: + rospy.logerr(f"Download failed for '{scan_name}': {e.reason}") + self._scan_download_successful_pub.publish(result_msg) + + def _on_set_scan_prefix(self, msg): + self._get(f"/setprefix?prefix={msg.data}") + + def _poll_status(self, event): + body = self._get("/status", log=False) + if body is None: + return + try: + data = json.loads(body) + status = HovermapStatus() + status.scan_prefix = data["scan_name"] + status.current_scan_name = data["scan_dir"] + status.free_space = data["freeSpace"] + status.scan_running = data["state"] not in NON_RUNNING_STATES + self._status_pub.publish(status) + except (json.JSONDecodeError, KeyError) as e: + rospy.logerr(f"Failed to parse /status response: {e}") + + def spin(self): + rospy.spin() + + +if __name__ == "__main__": + node = HttpInterfaceNode() + node.spin() diff --git a/src/hovermap_api_msgs/CMakeLists.txt b/src/hovermap_api_msgs/CMakeLists.txt new file mode 100644 index 0000000..f6b85b2 --- /dev/null +++ b/src/hovermap_api_msgs/CMakeLists.txt @@ -0,0 +1,24 @@ +cmake_minimum_required(VERSION 3.16) +project(hovermap_api_msgs) + +find_package(catkin REQUIRED COMPONENTS + message_generation + std_msgs +) + +add_message_files(FILES + ScanInformation.msg + ScanInformationList.msg + HovermapStatus.msg +) + +generate_messages( + DEPENDENCIES + std_msgs +) + +catkin_package( + CATKIN_DEPENDS + message_runtime + std_msgs +) diff --git a/src/hovermap_api_msgs/msg/HovermapStatus.msg b/src/hovermap_api_msgs/msg/HovermapStatus.msg new file mode 100644 index 0000000..f5b6a11 --- /dev/null +++ b/src/hovermap_api_msgs/msg/HovermapStatus.msg @@ -0,0 +1,4 @@ +string scan_prefix +string current_scan_name +string free_space +bool scan_running diff --git a/src/hovermap_api_msgs/msg/ScanInformation.msg b/src/hovermap_api_msgs/msg/ScanInformation.msg new file mode 100644 index 0000000..c44b5cd --- /dev/null +++ b/src/hovermap_api_msgs/msg/ScanInformation.msg @@ -0,0 +1,2 @@ +string name +int64 size # in bytes diff --git a/src/hovermap_api_msgs/msg/ScanInformationList.msg b/src/hovermap_api_msgs/msg/ScanInformationList.msg new file mode 100644 index 0000000..c53b0c5 --- /dev/null +++ b/src/hovermap_api_msgs/msg/ScanInformationList.msg @@ -0,0 +1 @@ +ScanInformation[] scans diff --git a/src/hovermap_api_msgs/package.xml b/src/hovermap_api_msgs/package.xml new file mode 100644 index 0000000..bc24cb9 --- /dev/null +++ b/src/hovermap_api_msgs/package.xml @@ -0,0 +1,19 @@ + + + hovermap_api_msgs + 0.1.0 + ROS messages for the Hovermap ROS API + + Glenn + + Proprietary + + Glenn + + catkin + + std_msgs + + message_generation + message_runtime +