Working notes for installing xr_teleoperate (operator side), unitree_sim_isaaclab (teleop simulator side), and unitree_rl_lab (Stage 1 reinforcement-learning trainer for the demo) on a fresh DGX Spark, so we can run the G1 (29 degrees of freedom) with Inspire RH56DFTP hands in simulation, on the real robot, and through massively-parallel motion-tracking training.
This file documents the install state, every gap found in the GitHub READMEs we followed, every workaround applied, every known issue, and where we are right now. It is the single source of truth for "what does the project look like on this machine."
CLAUDE.md (a sibling file) holds the do/don't rules for Claude. This README holds project state.
GitHub: https://github.com/raiseboxlab/unitree-g1-u6-spark (private).
The nine cloned-repo dependencies (xr_teleoperate, unitree_sdk2_python, unitree_sim_isaaclab, IsaacLab, cyclonedds, inspire_hand_ws, unitree_lab_isaac_lab, unitree_rl_lab, unitree_ros) are tracked as git submodules pinned to specific commits — the exact SHAs documented in "Versions and commits checked out" below. Each submodule's url in .gitmodules points at its public upstream GitHub repo (e.g. unitreerobotics/, isaac-sim/, eclipse-cyclonedds/, NaCl-1374/); none point at any fork under raiseboxlab. So this repo is fully self-contained and has no shared dependency on raiseboxlab/unitree-g1-u6 (the older macOS project).
The last three submodules (unitree_lab_isaac_lab, unitree_rl_lab, unitree_ros) only matter if you intend to run the demo's Stage 1 RL training — see Phase 7 of the install cookbook and Scenario D under "How to launch". For pure teleop (Scenarios A/B/C) you can ignore them.
To clone everything in one shot, with all submodules at the right SHAs:
git clone --recurse-submodules https://github.com/raiseboxlab/unitree-g1-u6-spark.git ~/Projects/unitree-g1-u6
cd ~/Projects/unitree-g1-u6If you forgot --recurse-submodules, you can fetch them after the fact:
git submodule update --init --recursiveLocal edits to files inside the submodules (currently three: auto_setup_env.sh and fetch_assets.sh in unitree_sim_isaaclab, plus robot_control/robot_arm.py in xr_teleoperate) live as patch files under /patches/. The install cookbook below shows how to apply them.
| Hardware | DGX Spark, NVIDIA GB10 (Grace Blackwell) |
| Architecture | aarch64 (linux-arm64) |
| OS | Ubuntu 24.04.4 LTS (noble), kernel 6.17 |
| GPU driver | NVIDIA 580.142, CUDA 13.0 reported by nvidia-smi |
| System Python | 3.12.3 (Ubuntu 24.04 prevents pip install into it via PEP 668 — Ubuntu's rule that blocks pip from changing the system Python's libraries) |
- Operator side:
xr_teleoperatefromunitreerobotics/xr_teleoperatereads the Quest 3's hand tracking and publishes commands on DDS topics (DDS = Data Distribution Service, the message bus used by all Unitree-stack pieces). - Simulator side:
unitree_sim_isaaclabfromunitreerobotics/unitree_sim_isaaclabruns Isaac Sim 5.1 with the G1 (29 DoF) + Inspire FTP hands scene. It subscribes to the same DDS topics that the real robot's PC2 would, so the operator stack doesn't know whether it's talking to a sim or to hardware. - Bridge between them: CycloneDDS on UDP, DDS domain 1 (sim mode), all on
127.0.0.1. - Goal flag:
--ee=inspire_ftponxr_teleoperate(FTP = the U6's actual hand model, not the Dex3 the README's example uses).
Three separate conda environments by design — each of the three upstream READMEs we followed asks for its own:
| Env | Python | What lives in it |
|---|---|---|
tv |
3.10 | xr_teleoperate, televuer, teleimager, unitree_sdk2py, dex_retargeting, inspire_sdkpy — operator side |
unitree_sim_env |
3.11 | Isaac Sim 5.1, Isaac Lab (develop branch HEAD of 2026-05-06), PyTorch, unitree_sdk2py, teleimager — teleop simulator side |
env_isaaclab |
3.11 | Isaac Sim 5.1, Isaac Lab v2.3.0, PyTorch (cu130 aarch64), unitree_rl_lab, rsl-rl-lib 3.3.0 + four other RL frameworks — Stage 1 RL trainer |
They never need to be active in the same shell. tv ↔ unitree_sim_env communicate over the DDS wire on 127.0.0.1 domain 1. env_isaaclab is standalone — it doesn't talk DDS to the other envs. It also keeps its own separate Isaac Lab checkout (unitree_lab_isaac_lab/ pinned to v2.3.0) so the teleop sim env stays on its working develop-branch HEAD; see Issue 31 for why that split exists.
/home/i/Projects/unitree-g1-u6/
├── README.md # this file
├── CLAUDE.md # rules Claude must follow on this project (behaviour only)
├── run_sim.sh # launches Isaac Sim in unitree_sim_env
├── run_teleop_dfx.sh # launches xr_teleoperate against the simulator (DFX wire format)
├── run_teleop_ftp.sh # launches xr_teleoperate against the real U6 robot (FTP wire format)
├── run_teleop_arms_only.sh # launches xr_teleoperate against the real U6 with arms only — for first tests when Inspire hands are not installed; auto-spawns the wire logger
├── preflight_real_robot.sh # one-shot pre-flight checks for the real-robot tests; --with-hands switches to the FTP scenario (auto-starts the bridge)
├── run_stage1_train.sh # launches or resumes Stage 1 PPO training in env_isaaclab; see doc/run_stage1_train.md
│
├── xr_teleoperate/ # cloned from github.com/unitreerobotics/xr_teleoperate
│ # currently checked out at commit 4c0afdb — the v1.5 release tag (see "Issue 15" below)
├── unitree_sdk2_python/ # cloned from github.com/unitreerobotics/unitree_sdk2_python
├── unitree_sim_isaaclab/ # cloned from github.com/unitreerobotics/unitree_sim_isaaclab
│ └── assets/ # 1.7 GB Isaac Sim USD files fetched via fetch_assets.sh
├── IsaacLab/ # cloned from github.com/isaac-sim/IsaacLab — develop-branch HEAD, used by unitree_sim_env
├── cyclonedds/ # cloned from github.com/eclipse-cyclonedds/cyclonedds (releases/0.10.x)
│ └── install/ # CycloneDDS C library built from source (used by unitree_sim_env)
├── inspire_hand_ws/ # cloned from github.com/NaCl-1374/inspire_hand_ws
│
├── unitree_lab_isaac_lab/ # cloned from github.com/isaac-sim/IsaacLab — pinned to tag v2.3.0, used by env_isaaclab (Issue 31)
├── unitree_rl_lab/ # cloned from github.com/unitreerobotics/unitree_rl_lab — Stage 1 RL training framework
├── unitree_ros/ # cloned from github.com/unitreerobotics/unitree_ros — URDF robot descriptions for unitree_rl_lab (Method 2)
│
├── pc2/ # files we deploy onto the G1's onboard Jetson (PC2)
│ ├── pc2_stop_video_services.sh # disable Unitree's videohub_pc4 supervisor before teleimager (Issue 25)
│ ├── pc2_restore_video_services.sh # re-enable the supervisor afterwards
│ └── cam_config_server.yaml # template for our single-RealSense G1; serial number is OUR camera's
│
├── tools/ # diagnostic + capture utilities run from the Spark
│ └── arm_sdk_wire_logger.py # subscribes to rt/arm_sdk on a chosen DDS domain and writes a CSV of the weight slot + 14 arm joint q targets, every published message; auto-spawned by run_teleop_*.sh
│
├── logs/ # captured wire traces from real-robot runs
│ └── baselines/ # known-good CSVs from the first successful real-robot tests; reference for regression diffs
│
├── .cdds_prefix/ # private folder of symlinks pointing at Ubuntu's CycloneDDS C library
│ # — needed by the Python wheel build for the tv env (see "Issue 2")
└── cert.pem, key.pem # the SSL cert/key the WebRTC server on PC2 needs (we generate them on the Spark and copy them to PC2)
| Where | What |
|---|---|
xr_teleoperate/ |
commit 4c0afdb (v1.5 release tag — has the Inspire FTP fix from Issue 15 AND the --sim → DDS domain 1 wiring needed to talk to the simulator) |
xr_teleoperate/teleop/teleimager/ (Git submodule, a Git feature that lets one repo include another) |
89d4613 (the SHA recorded by 4c0afdb) |
xr_teleoperate/teleop/televuer/ (Git submodule) |
948f65f6 |
xr_teleoperate/teleop/robot_control/dex-retargeting/ (Git submodule) |
d7753d38 |
unitree_sdk2_python/ |
commit d9467c5 |
unitree_sim_isaaclab/ |
commit e30c25b |
IsaacLab/ |
clone HEAD as of 2026-05-06 (develop branch — used by unitree_sim_env) |
cyclonedds/ |
branch releases/0.10.x |
inspire_hand_ws/ |
commit fc754900 |
unitree_lab_isaac_lab/ |
tag v2.3.0 (commit 3c6e67bb5, 2025-10-28 — the IsaacLab version Unitree RL Lab's README badge declares; see Issue 31) |
unitree_rl_lab/ |
commit 4960b84 (main HEAD as of 2025-11-19) |
unitree_ros/ |
commit 07eefbc (main HEAD as of 2026-04-30 — URDF source for env_isaaclab) |
When you conda activate tv or conda activate unitree_sim_env, a small shell script in each environment's etc/conda/activate.d/spark.sh runs automatically and sets the runtime environment variables that the libraries inside that environment need (see "Issues 10, 12, 13" below). When you conda deactivate, a matching etc/conda/deactivate.d/spark.sh undoes them so the parent shell is left clean.
What the hooks set:
| Env | Variable | Why |
|---|---|---|
tv |
LD_PRELOAD=$CONDA_PREFIX/lib/libcrypto.so.3 |
Forces conda's OpenSSL 3.6 to load before any C library pulls in Ubuntu's older OpenSSL 3.0.13. Without this, vuer.server import fails and televuer swallows the real error behind a misleading message. |
unitree_sim_env |
LD_PRELOAD=/lib/aarch64-linux-gnu/libgomp.so.1 |
Isaac Sim's installer asked for this. |
unitree_sim_env |
OMNI_KIT_ACCEPT_EULA=YES |
Makes import isaacsim skip the interactive EULA prompt that would otherwise block any non-interactive script. |
env_isaaclab |
LD_PRELOAD=/lib/aarch64-linux-gnu/libgomp.so.1 |
Same fix as unitree_sim_env — NVIDIA's pip_installation.rst explicitly prescribes loading the system libgomp first on aarch64. |
env_isaaclab |
OMNI_KIT_ACCEPT_EULA=YES |
Same fix as unitree_sim_env — skips Isaac Sim's first-import EULA prompt for unitree_rl_lab.sh -l, run_stage1_train.sh, and other non-interactive scripts. |
Run these phases top to bottom on a clean Spark and you should end up at the same working state we have now. Each step is the corrected command after our findings — for the original step in the upstream README and why we changed it, see the Issue letter at the end.
Install miniforge3 if you don't already have conda. Neither upstream README tells you to do this:
curl -fsSL -o /tmp/Miniforge3.sh https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-aarch64.sh
bash /tmp/Miniforge3.sh -b -p ~/miniforge3
~/miniforge3/bin/conda init bash
exec bash # reload so `conda` is on PATHClone this repo with submodules — that pulls every cloned-repo dependency at the exact SHA we tested with:
git clone --recurse-submodules https://github.com/raiseboxlab/unitree-g1-u6-spark.git ~/Projects/unitree-g1-u6
cd ~/Projects/unitree-g1-u6Apply our local edits to unitree_sim_isaaclab (Issues 4.1, 4.2, 5):
git -C unitree_sim_isaaclab apply ../patches/unitree_sim_isaaclab/auto_setup_env.sh.patch
git -C unitree_sim_isaaclab apply ../patches/unitree_sim_isaaclab/fetch_assets.sh.patchApply our local edit to xr_teleoperate (Issue 27 — --motion-mode entry weight ramp):
git -C xr_teleoperate apply ../patches/xr_teleoperate/robot_arm.py.entry_weight_ramp.patch(Optional but speeds things up) configure passwordless sudo for apt only — see the /etc/sudoers.d/i-installs example in the conversation history.
Note about the
git clonesteps in Phases 2 and 3 below. Those steps were originally written for a fresh-from-scratch install where each repo is cloned separately. With the recursive clone above, the submodules are already in their folders at the right SHAs — you can skip everygit clone … .gitandgit checkout <sha>line in the rest of the cookbook. Justcdinto the submodule folder when the cookbook says to and run the install commands directly.
sudo apt update
sudo apt install -y cyclonedds-dev cyclonedds-tools
PFX=$(pwd)/.cdds_prefix
mkdir -p "$PFX/lib" "$PFX/bin"
ln -sfn /usr/include "$PFX/include"
ln -sfn /usr/lib/aarch64-linux-gnu/libddsc.so "$PFX/lib/libddsc.so"
ln -sfn /usr/lib/aarch64-linux-gnu/libddsc.so.0.10.4 "$PFX/lib/libddsc.so.0.10.4"
ln -sfn /usr/lib/aarch64-linux-gnu/libcycloneddsidl.so "$PFX/lib/libcycloneddsidl.so"
ln -sfn /usr/lib/aarch64-linux-gnu/libcycloneddsidl.so.0.10.4 "$PFX/lib/libcycloneddsidl.so.0.10.4"
ln -sfn /usr/bin/idlc "$PFX/bin/idlc"Create the env, with pip in the spec (Issue 1):
conda create -n tv python=3.10 pip pinocchio=3.1.0 numpy=1.26.4 -c conda-forge -y
conda activate tvClone xr_teleoperate and check out the v1.5 release tag (Issue 15):
git clone https://github.com/unitreerobotics/xr_teleoperate.git
cd xr_teleoperate
git checkout 4c0afdb
git submodule update --init --depth 1
cd ..Pin logging_mp to the snake_case API (Issue 17):
pip install 'logging_mp==0.1.6'Install the two xr_teleoperate submodules (per upstream README §1.1):
cd xr_teleoperate
pip install -e teleop/teleimager --no-deps
pip install -e teleop/televuer
cd ..Pin params_proto<3.0 (Issue 10):
pip install 'params_proto<3.0'Install xr_teleoperate's requirements.txt — the upstream README §1.1 forgets this step (Issue 20):
pip install -r xr_teleoperate/requirements.txt
pip install pyzmq # needed by teleimager but not transitive (Issue 20 addendum)Generate the Quest 3 self-signed cert and copy to ~/.config/xr_teleoperate/ (Issue 7 is about the prompts):
cd xr_teleoperate/teleop/televuer
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
-subj "/CN=localhost" \
-keyout key.pem -out cert.pem
sudo ufw allow 8012
mkdir -p ~/.config/xr_teleoperate/
cp cert.pem key.pem ~/.config/xr_teleoperate/
cd ../../../Set the activate.d / deactivate.d hooks for the OpenSSL fix (Issues 10, 12):
mkdir -p ~/miniforge3/envs/tv/etc/conda/activate.d ~/miniforge3/envs/tv/etc/conda/deactivate.d
cat > ~/miniforge3/envs/tv/etc/conda/activate.d/spark.sh <<'EOF'
export _SPARK_OLD_LD_PRELOAD="${LD_PRELOAD:-}"
export LD_PRELOAD="${CONDA_PREFIX}/lib/libcrypto.so.3${LD_PRELOAD:+:$LD_PRELOAD}"
EOF
cat > ~/miniforge3/envs/tv/etc/conda/deactivate.d/spark.sh <<'EOF'
if [ -n "${_SPARK_OLD_LD_PRELOAD+x}" ]; then
export LD_PRELOAD="$_SPARK_OLD_LD_PRELOAD"
unset _SPARK_OLD_LD_PRELOAD
[ -z "$LD_PRELOAD" ] && unset LD_PRELOAD
fi
EOF
conda deactivate; conda activate tvClone unitree_sdk2_python and install with the shadow prefix as CYCLONEDDS_HOME (Issue 2):
git clone https://github.com/unitreerobotics/unitree_sdk2_python.git
cd unitree_sdk2_python
CYCLONEDDS_HOME=$HOME/Projects/unitree-g1-u6/.cdds_prefix pip install -e .
cd ..Install dex_retargeting (Issue 11 — upstream README never installs it):
conda install -y -c conda-forge nlopt
pip install pyyaml torch==2.3.0 \
--index-url https://download.pytorch.org/whl/cpu \
--extra-index-url https://pypi.org/simple
cd xr_teleoperate/teleop/robot_control/dex-retargeting
pip install -e . --no-deps
pip install pytransform3d trimesh anytree lxml
cd ../../../../Install inspire_sdkpy (Issue 21 — upstream READMEs never say where this lives):
git clone https://github.com/NaCl-1374/inspire_hand_ws.git
conda install -y -n tv -c conda-forge pyqt pymodbus pyserial
pip install -e inspire_hand_ws/inspire_hand_sdkVerify the tv env imports cleanly:
python -c "
import cyclonedds, unitree_sdk2py, teleimager
from cyclonedds.domain import DomainParticipant; DomainParticipant()
from televuer import TeleVuerWrapper
from vuer import Vuer
import dex_retargeting; from dex_retargeting.retargeting_config import RetargetingConfig
import inspire_sdkpy
import pinocchio
from pinocchio import casadi as cpin
print('TV ENV ALL OK')
"git clone https://github.com/unitreerobotics/unitree_sim_isaaclab.git
cd unitree_sim_isaaclabEdit auto_setup_env.sh (this script lives inside the cloned folder, so read it before editing — we're patching it because of three known issues):
- Issue 4.1 — bump torch version in the 5.1 branch:
- TORCH_PKG="torch==2.7.0 torchvision==0.22.0 torchaudio==2.7.0" + TORCH_PKG="torch==2.10.0 torchvision torchaudio"
- Issue 4.2 — add
pipto the conda create line:- conda create -y -n "$ENV_NAME" python="$PYTHON_VER" + conda create -y -n "$ENV_NAME" python="$PYTHON_VER" pip
Edit fetch_assets.sh to skip if assets already exist (Issue 5): add an early-exit at the top.
Run the auto-setup with PATH fix (Issue 3):
chmod +x auto_setup_env.sh fetch_assets.sh
PATH="$HOME/miniforge3/bin:$PATH" bash auto_setup_env.sh 5.1 unitree_sim_envIf the script exits non-zero in Phase 3 of its own run (Issue 9 — known transient), run the remaining steps manually:
conda activate unitree_sim_env
export CYCLONEDDS_HOME=$HOME/Projects/unitree-g1-u6/cyclonedds/install
cd $HOME/Projects/unitree-g1-u6/unitree_sdk2_python && pip install -e .
cd $HOME/Projects/unitree-g1-u6/unitree_sim_isaaclab && pip install -r requirements.txt
cd $HOME/Projects/unitree-g1-u6/unitree_sim_isaaclab/teleimager && pip install -e .
conda install -y -n unitree_sim_env -c conda-forge libstdcxx-ngUninstall torchaudio (Issue 14 — the cu126 build is ABI-incompatible with torch 2.9.0+cu130):
pip uninstall -y torchaudioSymlink CycloneDDS libraries into the env's lib folder so the wheel's RPATH finds them (Issue 22):
CDDS=$HOME/Projects/unitree-g1-u6/cyclonedds/install/lib
ENV_LIB=$HOME/miniforge3/envs/unitree_sim_env/lib
for f in libddsc.so libddsc.so.0 libddsc.so.0.10.5 \
libcycloneddsidl.so libcycloneddsidl.so.0 libcycloneddsidl.so.0.10.5; do
[ -e "$CDDS/$f" ] && ln -sfn "$CDDS/$f" "$ENV_LIB/$f"
doneSet the activate.d / deactivate.d hooks for unitree_sim_env (Issues 8, 13):
mkdir -p ~/miniforge3/envs/unitree_sim_env/etc/conda/activate.d \
~/miniforge3/envs/unitree_sim_env/etc/conda/deactivate.d
cat > ~/miniforge3/envs/unitree_sim_env/etc/conda/activate.d/spark.sh <<'EOF'
export _SPARK_OLD_LD_PRELOAD="${LD_PRELOAD:-}"
export LD_PRELOAD="/lib/aarch64-linux-gnu/libgomp.so.1${LD_PRELOAD:+:$LD_PRELOAD}"
export OMNI_KIT_ACCEPT_EULA=YES
EOF
cat > ~/miniforge3/envs/unitree_sim_env/etc/conda/deactivate.d/spark.sh <<'EOF'
if [ -n "${_SPARK_OLD_LD_PRELOAD+x}" ]; then
export LD_PRELOAD="$_SPARK_OLD_LD_PRELOAD"
unset _SPARK_OLD_LD_PRELOAD
[ -z "$LD_PRELOAD" ] && unset LD_PRELOAD
fi
unset OMNI_KIT_ACCEPT_EULA
EOF
conda deactivate; conda activate unitree_sim_envVerify the unitree_sim_env env imports cleanly:
python -c "
import torch, torchvision
print('torch', torch.__version__, 'CUDA:', torch.cuda.is_available())
import cyclonedds; from cyclonedds.domain import DomainParticipant; DomainParticipant()
import unitree_sdk2py, teleimager, isaacsim, isaaclab
print('UNITREE_SIM_ENV ALL OK')
"Only needed for Scenario B and Scenario C (real robot). Skip if you only run the simulator.
PC2 is the small NVIDIA Jetson Orin NX inside the robot's torso, running Ubuntu 20.04 (JetPack 5, GLIBC 2.31). It carries the chest-mounted Intel RealSense D435I camera. We need a separate Python environment on PC2 that runs silencht/teleimager — the image server that publishes the camera over ZMQ + WebRTC for xr_teleoperate on the Spark to consume. Default ssh alias from the Spark: ssh g1 (user unitree, IP 192.168.123.164). Passwordless sudo is required on PC2.
# 0. From the Spark, sanity-check ssh works.
ssh g1 'whoami; uname -a'
# 1. Install miniconda on PC2 (one-time; conda 26 will prompt to accept Anaconda TOS).
ssh g1 'curl -fsSL https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh -o /tmp/miniconda.sh && bash /tmp/miniconda.sh -b -p ~/miniconda3 && ~/miniconda3/bin/conda init bash'
# 2. Accept Anaconda's TOS for the default channels (conda 26 prompts on
# every channel access; -y on `conda create` does NOT bypass it).
ssh g1 'source ~/miniconda3/etc/profile.d/conda.sh && conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/main && conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/r'
# 3. Create the teleimager env (Python 3.10 — the conda-forge pyrealsense2 binding we need only ships py310/311/312/313).
ssh g1 'source ~/miniconda3/etc/profile.d/conda.sh && conda create -y -n teleimager python=3.10'
# 4. apt prerequisites (libuvc + libturbojpeg).
ssh g1 'sudo apt-get install -y libusb-1.0-0-dev libturbojpeg-dev'
# -> If apt complains about broken dependencies, do NOT blindly auto-fix.
# On our PC2 chrony and systemd-timesyncd both claimed the `time-daemon`
# virtual package, blocking install. We did:
# ssh g1 'sudo apt --fix-broken install -s' # simulation, prints proposed actions
# and inspected the proposal (it would remove chrony and upgrade
# systemd-timesyncd) BEFORE running it for real:
# ssh g1 'sudo apt --fix-broken install -y'
# Then re-run the install in step 4. A different PC2 may have a
# different proposal; review it before accepting.
# 5. Make sure PC2's clock is set. Out-of-the-box JetPack 5 sometimes leaves
# the RTC at Jan 1970, which makes the WebRTC TLS cert appear "not yet valid".
ssh g1 'sudo systemctl unmask systemd-timesyncd && sudo systemctl enable --now systemd-timesyncd && date'
# 6. Clone silencht/teleimager onto PC2.
ssh g1 'git clone https://github.com/silencht/teleimager.git ~/teleimager'
# 7. Install teleimager into the env (server extras include aiohttp + aiortc).
ssh g1 'source ~/miniconda3/etc/profile.d/conda.sh && conda activate teleimager && cd ~/teleimager && pip install -e ".[server]"'
# 8. Pin logging_mp 0.1.6 — silencht/teleimager uses the snake-case
# `get_logger` API; pip pulls 0.2.x which renamed it (Issue 24).
ssh g1 'source ~/miniconda3/etc/profile.d/conda.sh && conda activate teleimager && pip install "logging_mp==0.1.6"'
# 9. Install pyrealsense2 from conda-forge — the PyPI wheels need
# GLIBC 2.32+ which JetPack 5 doesn't have (Issue 23).
ssh g1 'source ~/miniconda3/etc/profile.d/conda.sh && conda activate teleimager && conda install -y -c conda-forge "pyrealsense2=2.56.5=py310h4e94696_100_cpu"'
# 10. UVC permissions: writes the udev rule, adds `unitree` to the video group.
# `setup_uvc.sh` is provided by silencht/teleimager. After it runs, verify
# group membership; if it isn't visible yet, log out of PC2 and back in
# (a `newgrp video` subshell would work too but doesn't compose with ssh).
ssh g1 'cd ~/teleimager && bash setup_uvc.sh'
ssh g1 'id | grep -q "(video)" && echo "video group OK" || echo "log out of PC2 and back in to pick up the video group"'
# 11. Copy the WebRTC SSL cert+key from the Spark to PC2's user config dir
# (teleimager's image_server.py looks here first, then env vars, then repo root).
ssh g1 'mkdir -p ~/.config/xr_teleoperate'
scp /home/i/Projects/unitree-g1-u6/cert.pem g1:~/.config/xr_teleoperate/cert.pem
scp /home/i/Projects/unitree-g1-u6/key.pem g1:~/.config/xr_teleoperate/key.pem
# 12. Discover the connected camera's serial number (--rs flag enables RealSense path).
ssh g1 'source ~/miniconda3/etc/profile.d/conda.sh && conda activate teleimager && cd ~/teleimager && python -m teleimager.image_server --cf --rs'
# -> Look for "RealSense serial numbers: ['XXXXXXXXXXXX']" in the output.
# -> Substitute that 12-digit serial into pc2/cam_config_server.yaml's head_camera.serial_number.
# -> Our Spark's pc2/cam_config_server.yaml is already filled with this G1's serial; for
# a fresh G1 + fresh camera, edit it before scp'ing.
# 13. Deploy our pc2/ files (config + helper scripts) to PC2.
ssh g1 'mkdir -p ~/bin'
scp /home/i/Projects/unitree-g1-u6/pc2/cam_config_server.yaml g1:~/teleimager/cam_config_server.yaml
scp /home/i/Projects/unitree-g1-u6/pc2/pc2_stop_video_services.sh g1:~/bin/pc2_stop_video_services.sh
scp /home/i/Projects/unitree-g1-u6/pc2/pc2_restore_video_services.sh g1:~/bin/pc2_restore_video_services.sh
ssh g1 'chmod +x ~/bin/pc2_stop_video_services.sh ~/bin/pc2_restore_video_services.sh'
# 14. Smoke-test: stop Unitree's video supervisor, start teleimager, confirm it streams.
ssh g1 '~/bin/pc2_stop_video_services.sh'
ssh g1 'source ~/miniconda3/etc/profile.d/conda.sh && conda activate teleimager && cd ~/teleimager && setsid nohup python -m teleimager.image_server --rs > /tmp/teleimager.log 2>&1 < /dev/null & disown; echo started'
sleep 12 && ssh g1 'tail -20 /tmp/teleimager.log; echo; ss -tlnp 2>/dev/null | grep -E "55555|60000|60001"'
# -> Expect "[Image Server] head_camera is ready." and ports 55555/60000/60001 listening.A no-window FPS check from the Spark (proves Spark↔PC2 ZMQ end-to-end at 30 FPS without any GUI):
source ~/miniforge3/etc/profile.d/conda.sh && conda activate tv
python - <<'PY'
import sys, time
sys.path.insert(0, "/home/i/Projects/unitree-g1-u6/xr_teleoperate/teleop/teleimager/src")
from teleimager.image_client import ImageClient
client = ImageClient(host="192.168.123.164")
start, n = time.monotonic(), 0
while time.monotonic() - start < 5.0:
img, fps = client.get_head_frame()
n += img is not None
time.sleep(0.005)
print(f"received {n} frames in 5s — fps_reported={fps:.2f}")
client.close()
PYFor a visual check from the Spark, use Chromium (not Firefox — Issue 26): browse https://192.168.123.164:60001/, accept the self-signed cert, click Start.
When you're done with a session, restore Unitree's stock video services:
ssh g1 '~/bin/pc2_restore_video_services.sh'Only needed for Scenario B (real robot with FTP hands installed). Skip if your hands aren't installed yet, or you only run Scenario C (arms-only) / Scenario A (sim).
The bridge — inspire_hand_sdk/example/Headless_driver_double.py from the inspire_hand_ws submodule — is a DDS↔Modbus-TCP shim. It runs on PC2, opens Modbus TCP connections to each hand at 192.168.123.210 (left) and 192.168.123.211 (right), subscribes to rt/inspire_hand/ctrl/{l,r} (where xr_teleoperate publishes finger commands), and publishes back rt/inspire_hand/state/{l,r} and rt/inspire_hand/touch/{l,r} (state + tactile from the hands' Modbus registers). Reuses the same teleimager conda env we set up in Phase 4.
# 1. Verify the hands are physically reachable on the LAN. Both must respond.
ssh g1 'ping -c 3 -W 2 192.168.123.210 && ping -c 3 -W 2 192.168.123.211'
ssh g1 'nc -zv -w 5 192.168.123.210 6000 && nc -zv -w 5 192.168.123.211 6000'
# 2. Deploy the SDK source tree to PC2 (so we can pip install -e it; same SHA as the submodule on the Spark).
scp -r /home/i/Projects/unitree-g1-u6/inspire_hand_ws/inspire_hand_sdk g1:/home/unitree/
# 3. Also deploy unitree_sdk2_python to PC2 (the bridge depends on it for DDS).
scp -r /home/i/Projects/unitree-g1-u6/unitree_sdk2_python g1:/home/unitree/
# 4. Install pymodbus into the teleimager env (cheap, pure-Python wheel).
ssh g1 'source ~/miniconda3/etc/profile.d/conda.sh && conda activate teleimager && pip install pymodbus==3.6.9'
# 5. Build the cyclonedds Python wheel against PC2's existing 0.10.2 C SDK
# install. PC2 has it at /home/unitree/cyclonedds_ws/install/cyclonedds — confirm
# headers + libs are there before running the build (Issue 28).
ssh g1 'ls /home/unitree/cyclonedds_ws/install/cyclonedds/include/idl/retcode.h /home/unitree/cyclonedds_ws/install/cyclonedds/lib/libddsc.so'
ssh g1 '
source ~/miniconda3/etc/profile.d/conda.sh
conda activate teleimager
CYCLONEDDS_HOME=/home/unitree/cyclonedds_ws/install/cyclonedds pip install cyclonedds==0.10.2 --no-cache-dir
'
# 6. Symlink the cyclonedds C libs into the conda env so the cyclonedds Python wheel finds
# libddsc.so at runtime without LD_LIBRARY_PATH (Issue-22 pattern: the wheel's RPATH points at
# the env's lib dir; we make those names resolve).
ssh g1 '
CDDS=/home/unitree/cyclonedds_ws/install/cyclonedds/lib
ENV_LIB=/home/unitree/miniconda3/envs/teleimager/lib
for f in libddsc.so libddsc.so.0 libddsc.so.0.10.2 libcycloneddsidl.so libcycloneddsidl.so.0 libcycloneddsidl.so.0.10.2; do
[ -e "$CDDS/$f" ] && ln -sfn "$CDDS/$f" "$ENV_LIB/$f"
done
'
# 7. Install unitree_sdk2_python into the teleimager env.
ssh g1 'source ~/miniconda3/etc/profile.d/conda.sh && conda activate teleimager && cd /home/unitree/unitree_sdk2_python && pip install -e .'
# 8. Install the Inspire Python SDK — but with --no-deps because its install_requires pulls in PyQt5 source build (~tens of minutes on aarch64).
ssh g1 'source ~/miniconda3/etc/profile.d/conda.sh && conda activate teleimager && cd /home/unitree/inspire_hand_sdk && pip install -e . --no-deps --no-build-isolation'
# 9. Add a Qt binding from conda-forge (mandatory because inspire_sdkpy/__init__.py imports
# qt_tabs unconditionally — see Issue 29). Conda-forge has prebuilt aarch64 packages; pip
# would build PyQt5 from source over many minutes.
ssh g1 'source ~/miniconda3/etc/profile.d/conda.sh && conda activate teleimager && conda install -y -c conda-forge pyqt pyqtgraph colorcet pyserial'
# 10. Smoke-test imports.
ssh g1 '
source ~/miniconda3/etc/profile.d/conda.sh
conda activate teleimager
python -c "
import pymodbus, numpy, cyclonedds, unitree_sdk2py
from inspire_sdkpy import inspire_sdk
print(\"all imports OK; ModbusDataHandler =\", inspire_sdk.ModbusDataHandler)
"
'
# -> Should print all OK and the ModbusDataHandler class.
# 11. Smoke-test the bridge itself by letting the preflight script start it.
./preflight_real_robot.sh --with-hands
# -> ALL GREEN means: bridge running on PC2, both hands publishing on rt/inspire_hand/state/{r,l},
# teleimager up, rt/lowstate streaming. Ready for Scenario B.The four scripts at the project root (run_sim.sh, run_teleop_dfx.sh, run_teleop_ftp.sh, run_teleop_arms_only.sh) wrap the conda-activate + Python invocation for each scenario. They use --headless (Issue 16) for sim and --ee=inspire_dfx for sim teleop (Issue 18) by default. The two real-robot scripts (run_teleop_arms_only.sh, run_teleop_ftp.sh) also auto-spawn the wire logger from tools/arm_sdk_wire_logger.py in the background and write the CSV to /tmp/run_teleop_*_wire_<timestamp>.csv for post-test analysis. Make them executable:
chmod +x run_sim.sh run_teleop_dfx.sh run_teleop_ftp.sh run_teleop_arms_only.sh preflight_real_robot.shThen proceed to "How to launch" below.
Only needed for the demo's Stage 1 — massively-parallel motion-tracking training of the G1 in Isaac Sim. Skip if you only run teleop scenarios.
What lives in this env: Isaac Sim 5.1.0 + Isaac Lab 2.3.0 + Unitree RL Lab (the trainer) + URDF descriptions for the G1 29-DoF robot. Total disk footprint: ~17 GB after install. Install procedure follows NVIDIA's pip_installation.rst and then Unitree RL Lab's README — verbatim where possible, with five known gotchas (Issues 30–34) folded in.
Create the env with Python 3.11 — required by Isaac Sim 5.1 (Issue 30) — and with pip explicitly in the spec (Issues 1, 4.2 mirror):
conda create -n env_isaaclab python=3.11 pip -y
conda activate env_isaaclab
pip install --upgrade pipInstall Isaac Sim 5.1.0 from NVIDIA's pypi (~5 GB download, ~13 GB on disk):
pip install "isaacsim[all,extscache]==5.1.0" --extra-index-url https://pypi.nvidia.comInstall the CUDA-13-enabled PyTorch wheel for aarch64 — NVIDIA's docs prescribe this specifically for Spark, overriding Isaac Sim's pinned torch 2.7.0 (Issue 8 redux):
pip install -U torch==2.9.0 torchvision==0.24.0 --index-url https://download.pytorch.org/whl/cu130Verify Isaac Sim launches and accept the EULA (one-time, interactive — must run in a real terminal at the desk):
unset LD_PRELOAD
export LD_PRELOAD="$LD_PRELOAD:/lib/aarch64-linux-gnu/libgomp.so.1"
isaacsim # answer Yes to EULA; close the window once the viewport rendersCheck out the Unitree-RL-Lab-compatible IsaacLab version (Issue 31) in our pre-cloned submodule and install Isaac Lab's source-tree extensions plus all five RL frameworks:
git -C unitree_lab_isaac_lab checkout v2.3.0
sudo apt install -y cmake build-essential
cd unitree_lab_isaac_lab
./isaaclab.sh --install
cd ..The script exits 1 in non-interactive shells because of a cosmetic tabs 4 call near the top — the pip installs themselves succeed. Verify Isaac Lab loads at the desk:
cd unitree_lab_isaac_lab && ./isaaclab.sh -p scripts/tutorials/00_sim/create_empty.py && cd ..Expect a black viewport (per NVIDIA's src_verify_isaaclab.rst). Close with Ctrl+C.
Pin rsl-rl-lib to 3.3.0 to match Unitree RL Lab's calling-code expectations (Issue 32):
pip install 'rsl-rl-lib==3.3.0'Install Unitree RL Lab editable:
cd unitree_rl_lab && ./unitree_rl_lab.sh -i && cd ..Apply our local edits to unitree_rl_lab/source/unitree_rl_lab/unitree_rl_lab/assets/robots/unitree.py (Issue 33). Three line-level changes:
-
Line 21 — set
UNITREE_ROS_DIR:- UNITREE_ROS_DIR = "path/to/unitree_ros" + UNITREE_ROS_DIR = "/home/i/Projects/unitree-g1-u6/unitree_ros"
-
Lines 398–403 (
UNITREE_G1_29DOF_CFG) — swap from USD spawn to URDF spawn (comment out the USD block, uncomment the URDF block). -
Lines 532–537 (
UNITREE_G1_29DOF_MIMIC_CFG) — same swap. The Mimic config is what our Stage 1 task uses.
These three edits should live as patches/unitree_rl_lab/unitree.py.patch for full reproducibility (TODO — patch file not yet created; for now the edits are documented inline and applied by hand).
Generate the NPZ motion file for the Stage 1 Mimic task (Issue 34):
cd unitree_rl_lab
python scripts/mimic/csv_to_npz.py \
-f source/unitree_rl_lab/unitree_rl_lab/tasks/mimic/robots/g1_29dof/gangnanm_style/G1_gangnam_style_V01.bvh_60hz.csv \
--input_fps 60 --headless &
NPZ=source/unitree_rl_lab/unitree_rl_lab/tasks/mimic/robots/g1_29dof/gangnanm_style/G1_gangnam_style_V01.bvh_60hz.npz
until [ -f "$NPZ" ]; do sleep 2; done # wait for the NPZ to land (~30 seconds)
pkill -KILL -f csv_to_npz.py # the script hangs in Isaac Sim teardown after writing the NPZ
cd ..Set the activate.d / deactivate.d hooks (Issues 8, 13 mirror — same LD_PRELOAD and EULA env-var as unitree_sim_env):
mkdir -p ~/miniforge3/envs/env_isaaclab/etc/conda/activate.d \
~/miniforge3/envs/env_isaaclab/etc/conda/deactivate.d
cat > ~/miniforge3/envs/env_isaaclab/etc/conda/activate.d/spark.sh <<'EOF'
export _SPARK_OLD_LD_PRELOAD="${LD_PRELOAD:-}"
export LD_PRELOAD="/lib/aarch64-linux-gnu/libgomp.so.1${LD_PRELOAD:+:$LD_PRELOAD}"
export OMNI_KIT_ACCEPT_EULA=YES
EOF
cat > ~/miniforge3/envs/env_isaaclab/etc/conda/deactivate.d/spark.sh <<'EOF'
if [ -n "${_SPARK_OLD_LD_PRELOAD+x}" ]; then
export LD_PRELOAD="$_SPARK_OLD_LD_PRELOAD"
unset _SPARK_OLD_LD_PRELOAD
[ -z "$LD_PRELOAD" ] && unset LD_PRELOAD
fi
unset OMNI_KIT_ACCEPT_EULA
EOF
conda deactivate; conda activate env_isaaclabVerify the env imports cleanly and Unitree RL Lab sees its tasks:
python -c "
import torch
print('torch', torch.__version__)
from importlib.metadata import version
print('isaacsim package version:', version('isaacsim'))
import isaacsim, isaaclab, unitree_rl_lab
print('isaaclab', isaaclab.__version__)
print('ENV_ISAACLAB ALL OK')
"
cd unitree_rl_lab && ./unitree_rl_lab.sh -l && cd ..The task-list output should include Unitree-G1-29dof-Mimic-Gangnanm-Style (the upstream typo Gangnanm is the actual registered task name — leave it alone).
Pick the scenario you're running and use the matching script. You can only run one teleop session at a time — both vuer servers want TCP port 8012 and both sshkeyboard listeners want exclusive access to the terminal stdin. The two run_teleop_*.sh scripts are for separate scenarios (sim vs real), not concurrent.
# Terminal 1 — Isaac Sim
cd /home/i/Projects/unitree-g1-u6
./run_sim.sh # opens the Isaac Sim viewer window
./run_sim.sh --headless # runs without a window (no aarch64 WebRTC in --no_render mode — see Issue 16)
SIM_TASK=Isaac-Stack-RgyBlock-G129-Inspire-Joint ./run_sim.sh # different task
# Terminal 2 — xr_teleoperate (run after the sim is ready)
cd /home/i/Projects/unitree-g1-u6
./run_teleop_dfx.sh
./run_teleop_dfx.sh --record # also records the episode (press s to start/stop)This uses the Inspire DFX wire format, which is what unitree_sim_isaaclab implements (see Issue 18).
cd /home/i/Projects/unitree-g1-u6
./preflight_real_robot.sh --with-hands # auto-starts the FTP bridge on PC2; ALL GREEN means ready
./run_teleop_ftp.sh --motion # loco controller stays alive — legs balance, can walk
./run_teleop_ftp.sh --motion --record # also records the episoderun_teleop_ftp.sh auto-spawns the wire logger from tools/arm_sdk_wire_logger.py and writes a CSV of every rt/arm_sdk message to /tmp/run_teleop_ftp_wire_<timestamp>.csv for post-test analysis (the path is printed when the script exits). Compare against the FTP baseline in logs/baselines/ for regression diffs.
This uses the Inspire FTP wire format, which is what the FTP bridge (Headless_driver_double.py from inspire_hand_sdk, installed on PC2 in Phase 5) consumes and bridges to Modbus TCP for the physical RH56DFTP hands at 192.168.123.210 (left) and 192.168.123.211 (right).
Safety prerequisites for Scenario B — see comments at top of run_teleop_ftp.sh. Robot must be on a stand or harness if running without --motion; with --motion it stands on the floor under loco-controller balance. Read doc/real_robot_arms_only_first_test.md for the launch-only-first methodology that's proven safe; the same applies here, just with hands attached. Hand commands at startup are "fully open" at the hand's hardware-default speed (we tested via run_hands_test.sh).
cd /home/i/Projects/unitree-g1-u6
./preflight_real_robot.sh # arms-only mode; auto-stops the FTP bridge if running; ALL GREEN means ready
./run_teleop_arms_only.sh --motionrun_teleop_arms_only.sh passes no --ee flag, which routes the launcher's end-effector dispatch through else: pass (line 204 of teleop_hand_and_arm.py) — no hand controller is created, no hand-state DDS subscriber waits 5 seconds for nothing, no hand commands are published into a void. It also auto-spawns the wire logger and writes the CSV to /tmp/run_teleop_arms_only_wire_<timestamp>.csv.
Read doc/real_robot_arms_only_first_test.md before the first real-robot test. It documents the actual launcher behaviour (with the Issue 27 entry-weight-ramp patch applied), the q=0 home pose (an "L" shape — upper arms down, forearms forward at ~90° elbows, hands at chest/upper-belly height — verified empirically with pinocchio.forwardKinematics on the URDF), the operator alignment ritual to avoid arm-swing on r, and the graceful disengagement behaviour on q. The procedure has been validated end-to-end on real hardware (2026-05-08); known-good wire traces are in logs/baselines/.
Browse to https://<spark-ip>:8012/?ws=wss://<spark-ip>:8012, click through the SSL warning, click "Virtual Reality", align your wrists with the robot's initial pose, and press r in the launcher's terminal to engage tracking. Press q to disengage and exit.
cd /home/i/Projects/unitree-g1-u6
# Fresh start, or auto-resume from the latest checkpoint of the latest run.
./run_stage1_train.sh
# Smaller grid for nicer TV framing or lighter rendering on the GPU.
./run_stage1_train.sh --num_envs 1024
./run_stage1_train.sh --num_envs 256
# Cap at 1000 iterations (~50 min rendered, ~10 min headless).
./run_stage1_train.sh --max_iterations 1000
# Headless training (no viewport, ~3× faster) for overnight runs.
./run_stage1_train.sh --headlessThe script defaults to the Unitree-G1-29dof-Mimic-Gangnanm-Style task (Mimic motion tracking against the Gangnam-Style reference motion — the typo Gangnanm is upstream). It auto-detects whether to start fresh or resume: if unitree_rl_lab/logs/rsl-rl/<task>/ already contains any model_*.pt checkpoint, the script adds --resume --load_run -1 --checkpoint -1 and continues from the latest. Otherwise a fresh run folder is created. Stop with Ctrl+C at any time; checkpoints save every 500 iterations.
Full operator documentation in doc/run_stage1_train.md — usage, expected runtime, where files land, and a troubleshooting table for the gotchas covered by Issues 30–34.
This scenario is independent of teleop. You can run it concurrently with Scenario A/B/C from a different terminal — env_isaaclab doesn't share DDS topics with the teleop side, and it uses its own Isaac Lab checkout (unitree_lab_isaac_lab/ at v2.3.0) so it can't break the IsaacLab/ install that unitree_sim_env depends on.
tvenv smoke-tested: all imports clean from a fresh shell.unitree_sim_envsmoke-tested: all imports clean from a fresh shell, includingimport isaacsimandimport isaaclab.- Sim launches successfully with
--headlessand publishesrt/lowstateon DDS domain 1; verified with a Python subscriber receiving frames at expected rate. tick counter ran past 289000 with no crashes during the long test session. - Sim launches with viewer too — the empty-stage screenshot we took confirmed Isaac Sim 5.1.0 boots and shows its UI.
xr_teleoperateat4c0afdbreaches the "Press [r] to start" ready banner and the Inspire controller is publishing hand commands onrt/inspire/cmdat high rate — verified in the log.- Full Quest 3 pipeline verified end-to-end on the simulator (
run_teleop_dfx.shagainstrun_sim.sh): arms AND fingers track the operator's hands; round-trip Quest 3 → televuer → xr_teleoperate → DDS domain 1 → unitree_sim_isaaclab → Isaac Sim physics → state back over DDS all working. - Scenario C (arms-only on real robot) verified end-to-end on 2026-05-08. The Issue 27 entry weight ramp (3 s) gave a clean 40 °/s slew from "thighs" to L-shape at launch; the existing 2 s exit ramp brought authority back to the loco controller cleanly on
q. Wire trace baseline at logs/baselines/2026-05-08_175607_real_robot_quest_teleop.csv. - Scenario B (real robot with FTP hands installed) verified end-to-end on 2026-05-08. PC2 FTP bridge installed (Phase 5 cookbook),
--with-handspreflight green, full Quest-driven run lasted ~130 s with ~80–100° travel on the major arm joints.Inspire_Controller_FTPreviewed end-to-end and confirmed safe at launch (initial command is "all fingers fully-open" at the hand's hardware-default speed; not a snap). Wire trace baseline at logs/baselines/2026-05-08_193946_real_robot_ftp_quest_teleop.csv. - Scenario D (Stage 1 RL training) install complete and training pipeline alive on 2026-05-11.
env_isaaclabbuilds clean,./unitree_rl_lab.sh -lreturns all 5 expected tasks includingUnitree-G1-29dof-Mimic-Gangnanm-Style, and./run_stage1_train.sh --headlessreached iteration ~60 in 3 minutes at ~3 s/iteration withMean rewardclimbing andMean episode lengthgrowing — the expected early-training pattern. Not yet run to convergence or filmed with a viewport; the centerpiece TV visual is pending. The 5 install gotchas we hit are written up as Issues 30–34 below.
What's still open:
- Modbus simulation harness — would let
run_teleop_ftp.shbe exercised end-to-end on the Spark with a Modbus mock instead of real PC2 + real hands. Currently the Spark setup is DDS-only (the operator side never speaks Modbus on the wire); a Modbus mock server plus theHeadless_driver_double.pybridge running locally would close that gap. Scoped in doc/modbus_harness_level1_scope.md — half a day of work, zero edits to cloned code. Deferred — not a blocker now that real-hardware tests pass. - Simulator missing FTP support — see Issue 18. Either file a feature request to add it to
unitree_sim_isaaclab, or write a local FTP↔DFX adapter for the Spark. Without it, sim testing necessarily uses the DFX wire format which doesn't exactly match what the real robot exposes. - Hand commands not in the wire logger schema yet.
tools/arm_sdk_wire_logger.pycapturesrt/arm_sdk(arm + weight slot) but notrt/inspire_hand/ctrl/{l,r}. Worth extending if we hit a hand-side regression to debug.
pip install succeeded at every step but the dependency resolver flagged conflicts that will be louder if and when they bite at runtime:
| Library reports | Cause | Likely impact |
|---|---|---|
isaacsim-core 5.1.0.0 wants torch==2.7.0, got 2.9.0+cu130 |
Isaac Lab's ensure_cuda_torch (called inside ./isaaclab.sh --install) re-installs torch with the cu13 build for the Spark, overriding Isaac Sim's pin |
Probably harmless; if it bites, we'll see import errors from C extensions |
isaacsim-core 5.1.0.0 wants torchvision==0.22.0, got 0.24.0 |
Same reason as above | Stable API across these versions; expect harmless |
isaacsim-core 5.1.0.0 wants torchaudio==2.7.0, NOT INSTALLED |
Isaac Lab's ensure_cuda_torch skipped torchaudio entirely |
Tested: neither isaacsim/ nor isaaclab/ actually import torchaudio. Installing the cu126 torchaudio breaks the env (Issue 14), so we leave it out |
rl-games 1.6.1 wants psutil<6.0.0,>=5.9.0, got 7.2.2 |
rl-games' upper bound is conservative | Only matters if you use rl-games as the RL framework — we do not |
isaacsim-kernel 5.1.0.0 wants psutil==5.9.8, got 7.2.2 (env_isaaclab only) |
robomimic (one of the 5 frameworks Isaac Lab installs by default) pulled psutil up to 7.2.2 |
Isaac Sim's kernel doesn't actually enforce the pin at runtime — import isaacsim works fine. No observed impact. |
fastapi 0.115.7 wants starlette<0.46.0, got 0.49.1 |
Resolved during requirements.txt step | Only matters if a code path actually uses fastapi-on-starlette interop |
dex-retargeting 0.4.7 wants pin>=2.7.0, NOT INSTALLED |
We installed pinocchio via conda-forge instead of the pip package called pin |
Code does import pinocchio, which works |
dex-retargeting 0.4.7 wants nlopt<2.8.0, got 2.10.1 |
We installed nlopt via conda-forge (only available aarch64 build) | API stable; expect harmless |
These are documented bug-by-bug. Each entry says: what the README expected, what actually happened on this Spark, why, and what we did about it. None of them involve editing files inside any of the cloned GitHub folders (other than the one place where checking out a different commit was the fix — Issue 15).
The README writes:
conda create -n tv python=3.10 pinocchio=3.1.0 numpy=1.26.4 -c conda-forgeWhat happens on this Spark: the conda-forge python package no longer pulls pip in as a dependency by default. The tv env ends up with python but no pip binary. After conda activate tv, the pip in PATH falls through to /usr/bin/pip (system Python 3.12), which Ubuntu 24.04 blocks via PEP 668 with error: externally-managed-environment. The next README line, pip install -e . --no-deps, cannot run.
What we did: added pip to the conda spec.
conda create -n tv python=3.10 pip pinocchio=3.1.0 numpy=1.26.4 -c conda-forgeWhy this happened on this machine and not the README author's machine: the README probably worked on Ubuntu 22.04 + an older conda-forge release where python still pulled pip. It is stale for Ubuntu 24.04 (which enforces PEP 668) plus a current conda-forge.
What the README expects: pip install -e . for unitree_sdk2_python will install all dependencies, including cyclonedds==0.10.2 (the Python bindings for CycloneDDS).
What actually happens on aarch64: there is no aarch64 wheel on PyPI for cyclonedds==0.10.2, so pip falls through to building from source. The source build needs:
- The Eclipse CycloneDDS C library + the
dds.hheader. - A specific folder layout:
lib/libddsc.sodirectly under whateverCYCLONEDDS_HOMEpoints to.
Two further surprises on Ubuntu 24.04:
- conda-forge has no
cycloneddspackage for linux-aarch64. - Ubuntu 24.04's
cyclonedds-devpackage puts the library at/usr/lib/aarch64-linux-gnu/libddsc.so(Debian's "multi-arch" location). The Python build helper only looks forlib/libddsc.sodirectly underCYCLONEDDS_HOMEand does not understand multi-arch paths. SoCYCLONEDDS_HOME=/usrfails even though the library is installed.
What we did:
sudo apt install -y cyclonedds-dev cyclonedds-tools # gets 0.10.4
# build a private folder of symlinks ("shadow prefix") in the layout the build helper expects
PFX=/home/i/Projects/unitree-g1-u6/.cdds_prefix
mkdir -p "$PFX/lib" "$PFX/bin"
ln -sfn /usr/include "$PFX/include"
ln -sfn /usr/lib/aarch64-linux-gnu/libddsc.so "$PFX/lib/libddsc.so"
ln -sfn /usr/lib/aarch64-linux-gnu/libddsc.so.0.10.4 "$PFX/lib/libddsc.so.0.10.4"
ln -sfn /usr/lib/aarch64-linux-gnu/libcycloneddsidl.so "$PFX/lib/libcycloneddsidl.so"
ln -sfn /usr/lib/aarch64-linux-gnu/libcycloneddsidl.so.0.10.4 "$PFX/lib/libcycloneddsidl.so.0.10.4"
ln -sfn /usr/bin/idlc "$PFX/bin/idlc"
CYCLONEDDS_HOME="$PFX" pip install -e ./unitree_sdk2_pythonThe unitree_sim_env later builds CycloneDDS from source (cyclonedds/install/) — that one already has a clean folder layout, so no shadow needed there.
The whole CycloneDDS prerequisite is not mentioned at all in xr_teleoperate's README — it inherits the assumption that PyPI ships a wheel.
The script does:
CONDA_BASE=$(conda info --base)
source "$CONDA_BASE/etc/profile.d/conda.sh"If you launch the script from a non-login shell where conda init's PATH addition has not run (for example, our background invocation), the first line dies with conda: command not found and the script aborts.
What we did: prepend conda's bin folder to PATH explicitly when invoking the script.
PATH="$HOME/miniforge3/bin:$PATH" bash auto_setup_env.sh 5.1 unitree_sim_envThe script could fix itself trivially with [ -z "$(command -v conda)" ] && source "$HOME/miniforge3/etc/profile.d/conda.sh".
The script pins (5.0/5.1 branches):
TORCH_PKG="torch==2.7.0 torchvision==0.22.0 torchaudio==2.7.0"
DEFAULT_CUDA="cu126"The PyTorch wheel index for cu126 ships only x86_64 and win_amd64 for torch 2.7.0. The smallest aarch64 + cu126 build available is torch 2.10.0. On the Spark this is the only option.
What we did: edited our local copy of auto_setup_env.sh (this is one of our own helper script — not inside any cloned GitHub folder) to bump the pin:
- TORCH_PKG="torch==2.7.0 torchvision==0.22.0 torchaudio==2.7.0"
+ TORCH_PKG="torch==2.10.0 torchvision torchaudio"Letting torchvision/torchaudio float so pip resolves compatible aarch64 wheels.
Risk noted: NVIDIA tested Isaac Sim 5.1 against torch 2.7; 2.10 is 3 minor versions newer. May or may not introduce Isaac Lab API friction — we will see.
The script does:
conda create -y -n "$ENV_NAME" python="$PYTHON_VER" # no pip
conda activate "$ENV_NAME"
eval "$PYTORCH_CMD" # pip falls through to base env's pipThis was the actual cause of Run 2's failure: the new env had python=3.11 but no pip, so conda activate put the env's bin folder on PATH but pip resolved to ~/miniforge3/bin/pip (base env, Python 3.13). The whole pip install torch … ran into base. Then pip install isaacsim==5.1.0 failed because the only aarch64 wheel requires Python 3.11 and pip was using 3.13.
What we did: added pip to the conda create line in our local copy.
- conda create -y -n "$ENV_NAME" python="$PYTHON_VER"
+ conda create -y -n "$ENV_NAME" python="$PYTHON_VER" pipThe script always deletes the existing assets/ folder and re-clones the 1.2 GB Hugging Face dataset, even if assets are already present. Re-running the wrapper script wastes 5–15 minutes on a guaranteed re-download.
What we did: added an early-exit at the top of our local copy: if assets/ already exists and is non-empty, exit 0.
The README says:
"Currently, our project has only been tested on RTX 3080, RTX 3090, and RTX 4090 GPUs. For the RTX 50 series GPUs, please use Isaac Sim version 5.0.0"
On aarch64 (Spark / GB10) this is wrong. Isaac Sim 5.0.0 has no aarch64 wheel on pypi.nvidia.com — only x86_64 + win_amd64. The aarch64 builds available are:
| Version | aarch64 wheel? |
|---|---|
| 4.5.0 | no |
| 5.0.0 | no |
| 5.1.0 | yes (cp311) |
| 6.0.0 | yes (cp312) |
So on the Spark we must use 5.1, not 5.0, despite the GB10 being a Blackwell GPU.
The Quest 3 path does:
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout key.pem -out cert.pem…which has no -subj, so it prompts for 7 fields. Cannot run non-interactively without piping newlines or adding -subj. The same line appears inside auto_setup_env.sh with the explicit instruction "Just keep pressing the Enter key" — fine for a human, awkward for a script.
What we did: fed printf '\n\n…' to the script's stdin when running it.
IsaacLab/isaaclab.sh --install calls ensure_cuda_torch twice. That function detects the host's CPU architecture and CUDA version and installs the "right" torch build. On the Spark (aarch64 + CUDA 13), it installs torch 2.9.0+cu130 + torchvision 0.24.0 from pypi.nvidia.com's aarch64 cu13 wheels. This:
- Overrides isaacsim 5.1.0's
torch==2.7.0/torchvision==0.22.0pins. Pip warns but does not roll back. - Skips
torchaudioentirely. No installer step adds an aarch64 torchaudio wheel that matches torch 2.9.0+cu130.
Net effect: Isaac Sim install reports success but is in a state pip considers inconsistent. We tested by import: isaacsim and isaaclab both load fine. Neither codebase actually imports torchaudio. So we leave it unsatisfied (see also Issue 14).
In one of our runs, ./isaaclab.sh --install finished successfully (printed [INFO] Setting up vscode settings... plus the LD_PRELOAD warning), then returned a non-zero exit code. The wrapper script's set -e killed it before the next 4 steps could run. Standalone re-run of ./isaaclab.sh --install afterwards exited 0; setup_vscode.py exited 0 standalone. The cause did not reproduce.
What we did: ran the remaining 4 steps manually:
conda activate unitree_sim_env
export CYCLONEDDS_HOME=/home/i/Projects/unitree-g1-u6/cyclonedds/install
cd /home/i/Projects/unitree-g1-u6/unitree_sdk2_python && pip install -e .
cd /home/i/Projects/unitree-g1-u6/unitree_sim_isaaclab && pip install -r requirements.txt
cd /home/i/Projects/unitree-g1-u6/unitree_sim_isaaclab/teleimager && pip install -e .
conda install -y -n unitree_sim_env -c conda-forge libstdcxx-ngAfter importing any C library that links against the system's libcrypto.so.3 (cyclonedds, teleimager, etc.), Ubuntu 24.04's older OpenSSL 3.0.13 wins the dynamic-loader race. Python's _ssl.cpython-310-aarch64-linux-gnu.so (built against conda-forge's OpenSSL 3.6) then fails:
ImportError: /lib/aarch64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.3.0' not found
(required by .../envs/tv/lib/python3.10/lib-dynload/_ssl.cpython-310-aarch64-linux-gnu.so)
Vuer's __init__.py does try: from vuer.server import Vuer, VuerSession; except ImportError: print(misleading_message) — so the real ImportError is swallowed and replaced with a "install with vuer[all]" message that has nothing to do with the actual problem. To televuer/__init__.py it looks like ImportError: cannot import name 'Vuer' from 'vuer'.
What we did: force-load conda's OpenSSL via LD_PRELOAD=$CONDA_PREFIX/lib/libcrypto.so.3. Made permanent via the tv env's activate.d hook (Issue 12).
git submodule update --init --depth 1 clones teleop/robot_control/dex-retargeting, but no README step does pip install on it. Yet teleop/robot_control/robot_arm_ik.py (imported during teleop_hand_and_arm.py startup) does from dex_retargeting.... Without the install, launch crashes at first import.
What we did, on aarch64 / Linux:
conda activate tv
conda install -y -c conda-forge nlopt # gets 2.10.1; PyPI nlopt 2.6.2 fails to build (StopIteration / PEP 479 bug); 2.8.0 has no aarch64 wheel
pip install pyyaml torch==2.3.0 pyzmq --index-url https://download.pytorch.org/whl/cpu --extra-index-url https://pypi.org/simple
cd /home/i/Projects/unitree-g1-u6/xr_teleoperate/teleop/robot_control/dex-retargeting
pip install -e . --no-deps # --no-deps because dex-retargeting's pyproject pins nlopt<2.8.0; conda-forge gave us 2.10.1
pip install pytransform3d trimesh anytree lxml # remaining deps (not auto-resolved with --no-deps)After this from dex_retargeting.retargeting_config import RetargetingConfig works.
The fix from Issue 10 is per-shell — losing it on next conda activate would re-break vuer. Solution: a conda activate.d hook that sets LD_PRELOAD on every activation, and a deactivate.d hook that restores the previous value.
~/miniforge3/envs/tv/etc/conda/activate.d/spark.sh:
export _SPARK_OLD_LD_PRELOAD="${LD_PRELOAD:-}"
export LD_PRELOAD="${CONDA_PREFIX}/lib/libcrypto.so.3${LD_PRELOAD:+:$LD_PRELOAD}"~/miniforge3/envs/tv/etc/conda/deactivate.d/spark.sh:
if [ -n "${_SPARK_OLD_LD_PRELOAD+x}" ]; then
export LD_PRELOAD="$_SPARK_OLD_LD_PRELOAD"
unset _SPARK_OLD_LD_PRELOAD
[ -z "$LD_PRELOAD" ] && unset LD_PRELOAD
fiimport isaacsim (or anything that triggers Omniverse Kit) prints:
By installing or using Omniverse Kit, I agree to the terms of NVIDIA OMNIVERSE LICENSE AGREEMENT (EULA)…
Do you accept the EULA? (Yes/No):
…and reads from stdin. In any script with no terminal, it hits EOF when reading a line and dies.
What we did: set OMNI_KIT_ACCEPT_EULA=YES before any import isaacsim. Made permanent via the unitree_sim_env activate.d hook.
~/miniforge3/envs/unitree_sim_env/etc/conda/activate.d/spark.sh:
export _SPARK_OLD_LD_PRELOAD="${LD_PRELOAD:-}"
export LD_PRELOAD="/lib/aarch64-linux-gnu/libgomp.so.1${LD_PRELOAD:+:$LD_PRELOAD}"
export OMNI_KIT_ACCEPT_EULA=YES14 — torchaudio cu126 ABI-incompatible with torch 2.9.0+cu130 (and isaacsim doesn't actually use it)
isaacsim 5.1.0's metadata pins torchaudio==2.7.0. There is no aarch64 wheel for torch 2.7.0+cu13x and therefore no matching torchaudio wheel either. Installing torchaudio==2.11.0+cu126 (the only available aarch64 build that matches the torch 2.x line) loads against torch 2.9.0+cu130 and crashes:
OSError: Could not load this library: …/torchaudio/lib/_torchaudio.abi3.so
…because the audio extension was compiled with cu126 ABI, not cu130.
We checked — neither isaacsim/ nor isaaclab/ actually do import torchaudio at runtime. So we uninstall torchaudio and accept the metadata-only pip warning. Anything that genuinely needed torchaudio would crash here anyway given the version gap.
Symptom. With xr_teleoperate checked out at the latest tagged release (6bef474, v1.5), ./run_teleop.sh reaches the Inspire FTP controller stage, waits 5 seconds for hand-state DDS topics, times out, then crashes:
File "/home/i/Projects/unitree-g1-u6/xr_teleoperate/teleop/robot_control/robot_hand_inspire.py", line 245, in _send_hand_command
left_cmd_msg = inspire_hand_default.get_inspire_hand_ctrl()
NameError: name 'inspire_hand_default' is not defined
Why it crashes (full code path read end to end). In the file at commit 6bef474:
- Lines 165–166, inside
Inspire_Controller_FTP.__init__:These are local tofrom inspire_sdkpy import inspire_dds # lazy import import inspire_sdkpy.inspire_hand_defaut as inspire_hand_default
__init__(a function-scope binding). They go out of scope the moment__init__returns. They are never stored onselfand never imported at the top of the file. - Line 209,
__init__spawns aProcess(target=self.control_process, …). - Line 264,
control_processruns in the spawned process and callsself._send_hand_command(...). - Lines 245 and 251,
_send_hand_commandreferencesinspire_hand_default.get_inspire_hand_ctrl()directly. Python looks forinspire_hand_defaultin: local scope of_send_hand_command(no), enclosing function (no — methods do not have one), module top level (no — not imported there), built-ins (no). Crashes withNameError.
This is a pure Python scoping bug. It would fail on every operating system, every Python version, regardless of fork vs spawn. It is not specific to our setup.
Public evidence in the repo's own commit history of this exact file:
14341b0 fix: move Inspire SDK import to module top-level (PEP8 fix & runtime stability)— a maintainer fixed the bug by moving both imports to the top of the file. The commit message explicitly says "runtime stability" — a real bug, not just style.04e35a3 [recover] lazy import— a later commit reverted the fix, putting the lazy (function-local) import back.
So the fix existed, then was rolled back, and our HEAD (6bef474) is after the rollback.
What we did (no edit to any cloned file). Checked out xr_teleoperate at commit 4c0afdb — the v1.5 release tag — where the fix is still in place AND simulation-mode wiring (--sim → DDS domain 1) was added. Note: we first tried 14341b0 because that's the explicit "fix" commit, but it predates the v1.5 simulation support, so --sim did not switch the DDS domain there and the operator side ended up subscribing on domain 0 while the simulator published on domain 1. Moved to 4c0afdb to get both fixes:
git -C xr_teleoperate checkout 4c0afdb
git -C xr_teleoperate submodule update --init --depth 1Submodule SHAs at this commit: teleimager 89d4613, televuer 948f65f6, dex-retargeting d7753d38. After checkout, reading xr_teleoperate/teleop/robot_control/robot_hand_inspire.py lines 1–11 confirms both inspire imports at module top level. Reading xr_teleoperate/teleop/teleop_hand_and_arm.py lines 100–105 confirms if args.sim: ChannelFactoryInitialize(1, …) — domain 1 when --sim is set.
Trade-off. Going to 4c0afdb loses any improvements made between that commit and HEAD (6bef474) — most notably the changes that come with 478248d [upgrade] logging_mp, teleimager. For the install-and-test phase that's acceptable. Revisit before any production deployment.
When we passed --no_render to sim_main.py, the script set the LIVESTREAM environment variable to a non-zero value, which caused Isaac Lab's app launcher to pick a livestream-enabled experience file. That experience pulled in omni.services.livestream.nvcf → omni.kit.livestream.webrtc, and omni.kit.livestream.webrtc has no aarch64 build on pypi.nvidia.com (only lx64 and wx64). The extension dependency solver aborted Isaac Sim startup.
What we did: use Isaac Lab's own --headless flag instead of --no_render. --headless does not turn on livestream and works on aarch64. Documented in run_sim.sh's usage notes.
When we first checked out 14341b0, the launcher crashed at line 6 of teleop_hand_and_arm.py:
AttributeError: module 'logging_mp' has no attribute 'basic_config'. Did you mean: 'basicConfig'?
The logging_mp library renamed basic_config → basicConfig between version 0.1.6 and 0.2.0. The version we had installed in tv (pulled in transitively when we installed televuer) was 0.2.1 — too new for this older xr_teleoperate commit.
What we did (in tv env):
pip install 'logging_mp==0.1.6'After that, import logging_mp; hasattr(logging_mp, 'basic_config') returns True, and the launcher boots cleanly.
Note: if anyone goes back to a newer xr_teleoperate commit (e.g. HEAD 6bef474, which sits after 478248d [upgrade] logging_mp, teleimager), that commit will want logging_mp 0.2.x (camelCase). The xr_teleoperate × logging_mp matrix has to line up; this is one of the things checking out an older xr_teleoperate commit costs us.
Symptom. With xr_teleoperate running --ee=inspire_ftp against unitree_sim_isaaclab, arms moved but fingers did not. Inspire controller waited 5 seconds for rt/inspire_hand/state/{l,r} and timed out with (L: False, R: False). Logs confirmed the simulator's own log line [inspire] Inspire Hand state publisher initialized — but a Python subscriber in our unitree_sim_env never received any message on rt/inspire_hand/state/l or rt/inspire_hand/state/r.
Root cause (read end-to-end of unitree_sim_isaaclab/dds/inspire_dds.py). The simulator's Inspire DDS bridge implements only the DFX (legacy / single-topic) wire format. Specifically:
- It publishes state on
rt/inspire/state(line 56) with typeMotorStates_fromunitree_sdk2py.idl.unitree_go.msg.dds_. - It subscribes to commands on
rt/inspire/cmd(line 68) with typeMotorCmds_from the same module.
Compare to what xr_teleoperate's Inspire_Controller_FTP does (read of xr_teleoperate/teleop/robot_control/robot_hand_inspire.py):
- Subscribes to
rt/inspire_hand/state/landrt/inspire_hand/state/r(split per hand) with typeinspire_hand_statefrominspire_sdkpy.inspire_dds. - Publishes commands to
rt/inspire_hand/ctrl/landrt/inspire_hand/ctrl/rwith typeinspire_hand_ctrlfrom the same module.
Different topic names, different IDL message types — no overlap. The simulator simply does not speak the FTP wire format. There is no inspire_ftp_dds.py in unitree_sim_isaaclab/dds/, no --enable_inspire_ftp_dds flag in sim_main.py. The simulator's --enable_inspire_dds only activates the DFX bridge.
Why we picked --ee=inspire_ftp originally. The real Unitree G1 U6 uses Inspire RH56DFTP hands wired through Modbus TCP on PC2 (the robot's onboard board). On the real robot, Headless_driver_double.py from inspire_sdkpy runs on PC2 and bridges FTP DDS topics to Modbus calls. So --ee=inspire_ftp is the correct flag for real hardware. We were trying to be hardware-faithful in the simulator too.
Where we deviated from the prescribed setup. The xr_teleoperate README's only worked example for sim deployment uses --ee=dex3 (line 274 of the README we cloned). The README never gives a worked example for Inspire FTP in sim. We extrapolated from the --ee parameter list and from the prior macOS project's notes; that extrapolation didn't account for the simulator missing FTP support.
Fix (no edits to any cloned code). Made two scripts at the project root: run_teleop_dfx.sh (uses --ee=inspire_dfx, talks to the simulator) and run_teleop_ftp.sh (uses --ee=inspire_ftp, talks to the real robot). xr_teleoperate's Inspire_Controller_DFX class uses rt/inspire/cmd and rt/inspire/state with MotorCmds_/MotorStates_ — byte-identical to what the simulator publishes/subscribes. After the change:
- Subscription succeeds immediately (
Subscribe dds ok.), no 5-second timeout. - Hand commands flow operator → sim, hand state flows sim → operator.
For real-robot deployment, use run_teleop_ftp.sh. The Modbus bridge on PC2 (independent of the simulator) consumes FTP DDS and drives the physical hands.
Open follow-up: the simulator's missing FTP support is a real feature gap from the perspective of "test on the Spark before going to the real robot." Two ways to close it later: (1) file a feature request on unitreerobotics/unitree_sim_isaaclab to add an FTP DDS bridge that mirrors dds/inspire_dds.py, (2) write a local FTP↔DFX adapter we run on the Spark alongside both processes (the Modbus simulation harness, scoped in doc/modbus_harness_level1_scope.md).
When ./run_teleop_dfx.sh (or the FTP variant) is launched from a script or background task whose stdin is not a real terminal, sshkeyboard (which xr_teleoperate uses to read r/q/s key presses) tries to put stdin into raw mode and dies with:
termios.error: (25, 'Inappropriate ioctl for device')
Effect: non-fatal. Only the keyboard listener thread dies; the rest of the launcher continues to run and to publish DDS commands. But the operator cannot press r to engage tracking from a backgrounded launch.
Workaround: for any run where the operator needs to press keys, launch from a real terminal (./run_teleop_dfx.sh or ./run_teleop_ftp.sh directly, not via background task or nohup). For full end-to-end use with the Quest headset, this is the operator's normal path anyway.
The xr_teleoperate README's §1.1 walks through installing the two submodules (teleimager, televuer) and generating SSL certs, but never tells you to run pip install -r requirements.txt for the operator's tv env. The requirements.txt lists four packages:
matplotlib==3.7.5
rerun-sdk==0.20.1
meshcat==0.3.2
sshkeyboard==2.3.1
Without these, python teleop_hand_and_arm.py … crashes early with ModuleNotFoundError: No module named 'meshcat' (imported by robot_arm_ik.py, line 2). Sshkeyboard is needed for the r/q/s keys; rerun-sdk for the recording visualisation.
What we did:
conda activate tv
pip install -r ~/Projects/unitree-g1-u6/xr_teleoperate/requirements.txtSide effect: matplotlib was downgraded from 3.10.x (whatever was already in the env) to 3.7.5 because of the pin. The macOS-era project notes flagged this as causing numpy conflicts; on this Spark we already had numpy 1.26.4 (compatible with both matplotlib versions), so no fallout.
The xr_teleoperate launcher with --ee=inspire_ftp imports inspire_sdkpy (the Python bindings + IDL types for the Inspire FTP wire format). Neither the xr_teleoperate README nor the unitree_sim_isaaclab README mentions where this library comes from. It is not on PyPI, and not a Git submodule of any of the cloned repos. There is no pip install inspire_sdkpy anywhere in any README.
The library lives at https://github.com/NaCl-1374/inspire_hand_ws (in the inspire_hand_sdk/ subfolder of that repo). On the real robot's PC2 it is probably installed manually by the integrator. For our Spark setup we had to clone and install it ourselves.
The library has three halves and our DDS-only setup needs only the first:
| Half | What it does | When we use it |
|---|---|---|
inspire_sdkpy.inspire_dds |
DDS IDL types (inspire_hand_ctrl, inspire_hand_state, inspire_hand_touch) |
Always — used by xr_teleoperate's FTP controller for message types |
inspire_sdkpy.inspire_sdk.ModbusDataHandler |
DDS↔Modbus bridge — subscribes to FTP DDS, writes Modbus registers, reads Modbus registers, publishes FTP DDS state | Real robot (runs on PC2 as Headless_driver_double.py); also our planned Modbus simulation harness |
inspire_sdkpy.qt_tabs |
PyQt5 GUI configurator | Never on the operator side; could be useful for hardware bring-up |
The library's setup.py declares hard dependencies on pymodbus, pyserial, and PyQt5. PyQt5 has no aarch64 wheel that builds on Python 3.10 — installing via plain pip fails. Solution: install the three native dependencies via conda-forge (which has aarch64 builds), then pip install -e the library with all dependencies satisfied.
What we did:
conda activate tv
git clone https://github.com/NaCl-1374/inspire_hand_ws.git ~/Projects/unitree-g1-u6/inspire_hand_ws
conda install -y -n tv -c conda-forge pyqt pymodbus pyserial
pip install -e ~/Projects/unitree-g1-u6/inspire_hand_ws/inspire_hand_sdkAfter this, from inspire_sdkpy import inspire_dds works, and so do the other two halves (ModbusDataHandler and qt_tabs) — though for sim work only inspire_dds is exercised.
We did not edit anything inside the cloned inspire_hand_ws folder. An earlier attempt that made the optional-import dance more permissive was reverted via a fresh git clone, on the user's instruction (cloned-repo files must remain byte-identical to GitHub).
22 — cyclonedds Python wheel built for unitree_sim_env has wrong RPATH; needs library symlinks into the env
When we let auto_setup_env.sh build cyclonedds==0.10.2 from source for unitree_sim_env (against the source build at cyclonedds/install/), the resulting C extension _clayer.cpython-311-aarch64-linux-gnu.so had its RPATH baked in to ~/miniforge3/envs/unitree_sim_env/lib — the env's lib folder. But the actual CycloneDDS shared libraries (libddsc.so.0 and friends) lived at ~/Projects/unitree-g1-u6/cyclonedds/install/lib/. Running import cyclonedds from this env therefore failed:
ImportError: libddsc.so.0: cannot open shared object file: No such file or directory
Verified with readelf -d _clayer*.so — the RPATH entry pointed at the env's lib but the linker also needed libddsc.so.0 (NEEDED). Since the linker found nothing at the RPATH location, it fell back to the standard search path, which doesn't include the source-build folder either, so it failed.
The tv env did not hit this because we built its cyclonedds==0.10.2 against the symlink shadow prefix at .cdds_prefix/lib/ — those symlinks already point at the system libraries that ARE on the standard search path.
What we did (unitree_sim_env): symlinked the relevant libraries from the source build into the env's lib folder, matching what the wheel's RPATH expects:
CDDS=$HOME/Projects/unitree-g1-u6/cyclonedds/install/lib
ENV_LIB=$HOME/miniforge3/envs/unitree_sim_env/lib
for f in libddsc.so libddsc.so.0 libddsc.so.0.10.5 libcycloneddsidl.so libcycloneddsidl.so.0 libcycloneddsidl.so.0.10.5; do
[ -e "$CDDS/$f" ] && ln -sfn "$CDDS/$f" "$ENV_LIB/$f"
doneAfter that from cyclonedds.domain import DomainParticipant; DomainParticipant() works in the env.
Why this is a Spark-specific surprise: on x86_64 Linux, cyclonedds==0.10.2 ships a binary wheel that bundles its own copy of the libraries. Pip never builds from source, so the RPATH issue never surfaces. On aarch64 there is no binary wheel — the build-from-source path is taken, and the wheel author's build script picks RPATH based on assumptions that happen to be wrong for our setup.
PC2 (the G1's onboard Jetson Orin NX) runs JetPack 5 / Ubuntu 20.04 with GLIBC 2.31. teleimager's image_server.py needs the Python bindings for Intel RealSense (pyrealsense2). The obvious pip install pyrealsense2 succeeded — but every import failed with:
ImportError: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found
(required by .../site-packages/pyrealsense2/pyrealsense2.cpython-310-aarch64-linux-gnu.so)
All four aarch64 wheels currently on PyPI (2.55.1, 2.56.4, 2.56.5, 2.57.7) are built against manylinux2014_aarch64, which assumes GLIBC ≥ 2.32. JetPack 5 doesn't have it. teleimager's own error message recommends building librealsense v2.50.0 from source — a 30+ minute build on the Orin.
What we did: installed the conda-forge build instead, which is built against an older GLIBC baseline:
conda install -y -c conda-forge "pyrealsense2=2.56.5=py310h4e94696_100_cpu"This Just Worked. The _cpu variant (vs _cuda) avoids pulling in cudatoolkit we don't need for camera I/O. The bundled librealsense2.so is independent of the system's librealsense2 package (PC2 happens to also have ROS Noetic's ros-noetic-librealsense2 2.50.0 installed for unrelated reasons; nothing references it from the teleimager env).
Side note: the conda-forge build doesn't expose pyrealsense2.__version__ (the attribute is missing). It works in every other respect; if you have code that depends on __version__, that line will need a getattr(rs, "__version__", "unknown") fallback.
24 — silencht/teleimager 6cff4ca needs logging_mp 0.1.x, not 0.2.x (mirror of Issue 17 on the Spark)
teleimager's image_client.py line 32 uses the old snake-case API:
logger_mp = logging_mp.get_logger(__name__, level=logging_mp.INFO)But teleimager's pyproject.toml doesn't pin logging_mp, so pip install -e .[server] resolved it to the latest 0.2.1, which renamed the helpers to camelCase (getLogger/basicConfig). Result: python -m teleimager.image_server --cf failed with AttributeError: module 'logging_mp' has no attribute 'get_logger'. Did you mean: 'getLogger'?
This is the same trap as Issue 17 on the Spark, in the opposite direction (Spark needed the snake_case API for xr_teleoperate's code; here PC2 needs it for teleimager's code). Same fix:
pip install "logging_mp==0.1.6"PC2 boots with a Unitree-supplied supervisor /etc/init.d/master_service (LSB init, auto-wrapped as master_service.service by systemd-sysv-generator). It polls every ~5 seconds and (re)starts the services listed in /unitree/etc/master_service/service/. Two of those — video_hub_pc4 and video_hub_pc4_chest — open /dev/video4 and /dev/video10 respectively. On this G1 (which only has the chest-mounted RealSense, no UVC head/chest cameras), one of the RealSense's stream nodes ends up at /dev/video4, so videohub_pc4 grabs it. teleimager then can't pull frames from the RealSense and shuts down 5 seconds later with head_camera ready timeout.
Three details that complicate "just stop the supervisor":
- The supervisor uses
KillMode=process.systemctl stop master_service.serviceonly kills the supervisor itself; child services live on. The systemd unit is auto-generated, so we can't easily changeKillMode=mixedwithout replacing the LSB init script — which we don't, per the no-edits-to-cloned-or-vendor-code rule. - The child services were started via
start-stop-daemon --start --background, which double-forks them and reparents to PID 1. Stopping master_service does not stop the orphans. - The supervisor respawns dead children on its 5-second poll. So killing children before stopping the supervisor is futile too.
What we did: pc2/pc2_stop_video_services.sh does both, in order — systemctl stop master_service.service (so it stops respawning), then start-stop-daemon --stop against each video child's PID file (the same mechanism the supervisor used to start them, taken from the Stop block of /unitree/etc/master_service/service/video_hub_pc4{,_chest}). pc2/pc2_restore_video_services.sh just runs systemctl start master_service.service afterwards — the supervisor wakes up, finds children missing on its next poll, and respawns them with their normal Start commands. ota_pipe (the third child of master_service) is never touched because it doesn't conflict with the camera.
The first time we tried teleimager without these scripts, we saw the head_camera ready timeout plus usbfs: usb_submit_urb returned -2 lines in dmesg at the same timestamps — kernel signal that two userspace processes were both trying to open the camera. With the stop script, the timeout is gone and teleimager streams cleanly.
When sanity-checking teleimager from the Spark, browsing https://192.168.123.164:60001/ and clicking Start loaded the page fine, completed the SDP offer/answer (we confirmed the offer reached PC2 via teleimager's [WebRTC] Preferred H264 log line), but the <video> element stayed black. tcpdump confirmed no WebRTC media UDP traffic was flowing back to the Spark. pyrealsense2 was producing frames at 30 FPS on PC2, ZMQ was carrying them to the Spark fine (we ran a no-window FPS-counter client from the Spark to confirm), and the H264 encoder was healthy (libx264 available; encoding a synthetic frame returned a normal 1.5KB keyframe). The problem was specifically the WebRTC media path Firefox-on-Linux ↔ aiortc-on-self-signed-HTTPS.
We did not fully root-cause this. The same URL on Chromium (also on the Spark, same network path) works first try.
What we did: the Quest 3 browser is Chromium-based, so this only matters for local-debug-from-the-Spark, where we now use Chromium. For the actual teleop, televuer/televuer.py directs the Quest browser to consume WebRTC directly from PC2 via WebRTCStereoVideoPlane(src=webrtc_url, iceServer=None, iceServers=[], …) — Chromium-class behavior, so we expect it to work the same as our Chromium sanity check.
If we ever need Firefox-on-Linux to work, the next thing to try is chrome://webrtc-internals/-style logging in Firefox via about:webrtc, plus possibly switching webrtc_codec to vp8 in pc2/cam_config_server.yaml. Not worth it right now — Chromium is the production target.
The defect. In --motion mode, G1_29_ArmController._ctrl_motor_state set the arm-blend weight (motor_cmd[kNotUsedJoint0].q, slot 29) to 1.0 instantly at launch (robot_arm.py:161). Combined with q_target = np.zeros(14) set in __init__ (line 64) and the publish thread starting immediately (lines 134–137), this means: from the moment xr_teleoperate launches, before any user keypress, the publish thread is already commanding the arms to slew to the L-shape (q=zeros), at the velocity ceiling enforced by clip_arm_q_target (= 20 rad/s = 1146 °/s in worst case). Upstream's wiki defends this design with: "under the constraints of the control program's clip_arm_q_target function, the target position and the current position will not have a significant difference" — but in --motion mode the loco controller parks the arms at the thighs (motion-control's idle pose), and xr_teleoperate's q_target = zeros is the L-shape, so the gap is large (e.g. ~120° at the shoulder). clip_arm_q_target then runs at its ceiling for the full slew, completing in ~80–100 ms. That is a snap.
Upstream's own xr_teleoperate exit code (ctrl_dual_arm_go_home, lines 229–231) does ramp the weight (1 → 0 over 2 s) when handing arm authority back to the loco controller. Unitree's official SDK example unitree_sdk2_python/example/g1/high_level/g1_arm7_sdk_dds_example.py likewise ramps the weight 1 → 0 in its "release arm_sdk" stage. xr_teleoperate just doesn't do the symmetric ramp on entry.
The fix (patches/xr_teleoperate/robot_arm.py.entry_weight_ramp.patch). Three small changes inside G1_29_ArmController, mirroring the structure of the existing exit ramp:
-
New
_entry_weight_rampmethod — exact mirror ofctrl_dual_arm_go_home's exit ramp shape (for weight in np.linspace(0, 1, num=151): … time.sleep(0.02)). Runs in its own short-lived daemon thread. -
__init__spawns that thread whenmotion_mode=True, after publishing thread is started. Right before that, it explicitly setsmotor_cmd[kNotUsedJoint0].q = 0.0to close a tiny race window between the publish thread starting to publish and the entry-ramp thread's first write — the lockdown loop above sets the slot to whatever PC1 reports in lowstate, and we don't trust that to be 0. -
ctrl_dual_arm_go_homecallsentry_weight_ramp_thread.join()at the top of the function. If the user pressesqwithin the first 3 s of launch (entry ramp still running), this waits for it to finish before the exit ramp starts, so the two ramps cannot ever both write the slot at the same time.
The publish thread (_ctrl_motor_state) is now back to upstream's structure — it never writes the weight slot. Slot 29's only writers, in order, are: the __init__ zero-init, the entry-ramp thread (0 → 1 over 3 s), then on q the exit ramp inside ctrl_dual_arm_go_home (1 → 0 over 2 s). They never overlap.
Why this works. Per Unitree's official arm_control_routine documentation, the blend formula is actual = motion_control_command × (1 − weight) + rt/arm_sdk_command × weight. With the weight ramping from 0 to 1 over 3 s while motion-control parks arms at thighs and xr_teleop_cmd points at L-shape, the actual position commanded interpolates linearly from thighs → L-shape over 3 s. Worst-case slew rate: 120° / 3 s = 40 °/s. Visibly slow, easy to abort.
A first attempt put the weight ramp inline in _ctrl_motor_state's 250 Hz publish loop. It worked at entry but broke on exit on the real robot — arms shook, elbow motors made loud noises during the 2-second exit ramp. Cause: the patched publish loop kept overwriting motor_cmd[29].q = 1.0 every 4 ms while ctrl_dual_arm_go_home was trying to ramp the same slot 1 → 0 every 20 ms. The two threads competed; PC1 saw a flickering weight; the loco controller and xr_teleop fought. Moving the entry ramp into a separate thread (and removing the publish-loop write entirely) solves this — only one writer is active at any moment.
Verification (full cycle, on the wire). Tested in --sim --motion mode with the launcher running inside a script(1) PTY so sshkeyboard could register a programmatically-sent q. The simulator doesn't implement the blend formula (unitree_sim_isaaclab/dds/g1_robot_dds.py forwards motor q/dq/tau/kp/kd directly without consulting slot 29) — but the sim publishes rt/lowstate so the launcher gets past its blocking init, and we subscribe to rt/arm_sdk from a separate process to capture exactly what the launcher publishes. Numbers from the 3 s entry ramp captured CSV:
| Phase | Result |
|---|---|
| Entry ramp (0 → 1) | 3.025 s linear ramp, slope 0.3306/s, max-deviation-from-linear 3.65 × 10⁻³ |
| Plateau at 1.0 | 200 samples, min = max = 1.000000, std = 0 |
| Exit ramp (1 → 0) | 2.022 s linear ramp, slope −0.4946/s, max-deviation-from-linear 5.36 × 10⁻³ |
| Post-exit | weight ≈ 0 as publish thread shuts down |
Real-robot validation: the 5 s version was tested end to end on real hardware first (clean entry slew, clean plateau, clean exit ramp, no motor noise on q). The 3 s version shortens the ramp but keeps the architecture identical; the wire trace above confirms the ramp shape on the same code path.
Scope. The patch only modifies G1_29_ArmController since that's the controller we use. The same defect exists in G1_23_ArmController, H1_2_ArmController, and H1_ArmController — the same three-part fix would apply if/when we use those.
28 — Building the cyclonedds Python wheel on PC2 needs CYCLONEDDS_HOME + libddsc symlinks (Issue-22 redux on PC2)
Same family of bug as Issue 22 (which we hit when building the cyclonedds Python wheel for the Spark's unitree_sim_env), but on PC2's teleimager env this time. pip install cyclonedds==0.10.2 on PC2 (aarch64, Ubuntu 20.04) fails to build because:
- The default headers it finds (
/usr/local/include/dds/) lack the IDL compiler headers (idl/retcode.h). - Even when the build succeeds, the resulting
_clayer.sohas its RPATH pointing at the conda env'slib/directory, but the cyclonedds C library lives somewhere else, soimport cycloneddsfails withlibddsc.so.0: cannot open shared object file.
PC2 already has a complete CycloneDDS 0.10.2 install at /home/unitree/cyclonedds_ws/install/cyclonedds/ (Unitree's own build, headers + IDL + libs). So:
# Build against Unitree's existing C SDK install:
CYCLONEDDS_HOME=/home/unitree/cyclonedds_ws/install/cyclonedds pip install cyclonedds==0.10.2 --no-cache-dir
# Then symlink the runtime libs into the conda env's lib/ so the Python wheel finds libddsc.so at import time:
CDDS=/home/unitree/cyclonedds_ws/install/cyclonedds/lib
ENV_LIB=/home/unitree/miniconda3/envs/teleimager/lib
for f in libddsc.so libddsc.so.0 libddsc.so.0.10.2 libcycloneddsidl.so libcycloneddsidl.so.0 libcycloneddsidl.so.0.10.2; do
[ -e "$CDDS/$f" ] && ln -sfn "$CDDS/$f" "$ENV_LIB/$f"
doneAfter that python -c "from cyclonedds.domain import DomainParticipant; DomainParticipant()" works in the teleimager env without any LD_LIBRARY_PATH games. Phase 5 in the install cookbook does this automatically. Same pattern as Issue 22 on the Spark side.
29 — inspire_sdkpy/__init__.py imports qt_tabs unconditionally; headless deploys still need a Qt binding
The Inspire Python SDK's package init (inspire_hand_ws/inspire_hand_sdk/inspire_sdkpy/__init__.py line 11) does:
from .qt_tabs import ImageTab, MainWindow, CurveTabqt_tabs.py in turn does import pyqtgraph as pg, and pyqtgraph at import time requires PyQt5 / PyQt6 / PySide2 / PySide6. So even a fully headless deploy of the FTP bridge (Headless_driver_double.py) cannot import inspire_sdkpy at all without a Qt binding present in the env.
pip install -e . from inspire_hand_sdk/ declares PyQt5 as a runtime dep — but on aarch64 PC2 there's no PyQt5 wheel and pip would build it from source over many minutes. Solution: install with --no-deps and add a Qt binding from conda-forge (which has prebuilt aarch64 packages):
pip install -e . --no-deps --no-build-isolation
conda install -y -c conda-forge pyqt pyqtgraph colorcet pyserialPhase 5 in the install cookbook does this. The bridge itself never instantiates a Qt application; the binding is loaded purely to satisfy the package-level import.
A cleaner upstream fix would be to make qt_tabs a lazy import in __init__.py. Out of scope for this project — we work around it.
The README path: Unitree RL Lab's README.md Installation section says "Install Isaac Lab by following the [installation guide]" which links to NVIDIA's pip_installation.rst. The included pip_python_virtual_env.rst runs:
conda create -n env_isaaclab python=3.11
conda activate env_isaaclabThen expects pip install --upgrade pip to work.
What happens on this Spark: same trap as Issues 1 and 4.2. The new env doesn't include pip by default (current conda-forge does not pull pip in as a transitive dep of python). After conda activate env_isaaclab, bare pip resolves to /usr/bin/pip (system Python 3.12), which Ubuntu 24.04 blocks via PEP 668 with error: externally-managed-environment. Every downstream pip command (isaacsim, torch, ./isaaclab.sh --install, ./unitree_rl_lab.sh -i) fails the same way.
What we did: added pip to the conda create spec:
- conda create -n env_isaaclab python=3.11
+ conda create -n env_isaaclab python=3.11 pip -yA related trap also worth flagging: Python 3.11 is mandatory, not just convention. Isaac Sim 5.1 hard-pins Requires-Python ==3.11.*. Trying Python 3.10 surfaces this clearly — pip silently filters all 5.x wheels by Python-version constraint and then reports Could not find a version that satisfies the requirement isaacsim==5.1.0 (from versions: 4.0.0.0, 4.1.0.0, 4.2.0.1, 4.2.0.2, 4.5.0.0). Neither NVIDIA's nor Unitree's README states the 3.11 requirement prominently in the install steps — it's in NVIDIA's pip_python_virtual_env.rst as a .. caution:: note that's easy to miss.
The README path: Unitree RL Lab's README badges declare IsaacLab-2.3.0 as the tested version. The install instruction is just git clone https://github.com/isaac-sim/IsaacLab.git — no --branch flag, no tag pin. Plain clone lands on main HEAD.
What actually happens: main HEAD (commit 3e73d6dd7 at our clone time) ships an updated isaaclab_rl.rsl_rl shim that requires rsl-rl-lib >= 5.0 with the new actor-config schema (cfg["actor"]["class_name"] is required). Unitree RL Lab's BasePPORunnerCfg (in tasks/locomotion/agents/rsl_rl_ppo_cfg.py and tasks/mimic/agents/rsl_rl_ppo_cfg.py) constructs RslRlPpoActorCriticCfg(init_noise_std=..., actor_hidden_dims=..., critic_hidden_dims=..., activation=...) — no class_name field anywhere.
Running ./unitree_rl_lab.sh -t --task Unitree-G1-29dof-Mimic-Gangnanm-Style on main-HEAD IsaacLab + the rsl-rl-lib 5.0.1 that ./isaaclab.sh --install pulls in dies with:
File ".../rsl_rl/algorithms/ppo.py", line 477, in construct_algorithm
actor_class: type[MLPModel] = resolve_callable(cfg["actor"].pop("class_name"))
KeyError: 'class_name'
What we did: added IsaacLab as a second clone (unitree_lab_isaac_lab/ submodule, separate from the existing IsaacLab/ that unitree_sim_env uses) and checked out the badge-declared tag:
git submodule add https://github.com/isaac-sim/IsaacLab.git unitree_lab_isaac_lab
git -C unitree_lab_isaac_lab checkout v2.3.0
cd unitree_lab_isaac_lab
./isaaclab.sh --installv2.3.0 (commit 3c6e67bb5, 2025-10-28) is close in time to Unitree RL Lab's last commit (4960b84, 2025-11-19) and its isaaclab_rl shim doesn't add the class_name requirement. Combined with the rsl-rl-lib pin in Issue 32, training initializes cleanly.
Why two separate IsaacLab clones? unitree_sim_env's setup has been working with IsaacLab/ on its develop-branch HEAD (commit 090aed181 from 2026-05-06) for weeks. Switching that clone's checkout to v2.3.0 would re-install Isaac Lab inside unitree_sim_env and risk breaking the teleop simulator that's currently green. A fresh second clone lets env_isaaclab pip-install editable from a different source folder, leaving unitree_sim_env's install untouched. Disk cost: another ~1 GB of repo files; zero risk to teleop.
Symptom. Even after switching to IsaacLab v2.3.0, training fails on rsl-rl-lib's PPO construction. The exact error depends on which rsl-rl-lib version pip installed by default:
- rsl-rl-lib 5.0.1 (the default that
./isaaclab.sh --installpulls in on main-HEAD IsaacLab):KeyError: 'class_name'inppo.pyline 477 — covered in Issue 31. - rsl-rl-lib 4.0.1: same
KeyError: 'class_name', line 486. The breaking API change happened before 4.0.1 already. - rsl-rl-lib 3.3.0 (without IsaacLab v2.3.0):
TypeError: PPO.__init__() got an unexpected keyword argument 'optimizer'— the OnPolicyRunner passesoptimizer=...to PPO but 3.3.0's PPO doesn't accept it.
So no rsl-rl-lib version on PyPI satisfies both class_name (added in 4.x) and optimizer (the calling-code expects it). What actually fixes it: switching IsaacLab to v2.3.0 changes which OnPolicyRunner is called (v2.3.0's shim wraps rsl_rl 3.3.0's API differently than main-HEAD's shim does). The right combination is IsaacLab v2.3.0 + rsl-rl-lib 3.3.0.
What we did:
pip install 'rsl-rl-lib==3.3.0'After this + IsaacLab v2.3.0, training initializes cleanly and the PPO learning loop runs end-to-end. Verified at iteration ~60 in 3 minutes with Mean reward climbing — the expected early-training pattern.
Why this is fragile: isaaclab_rl.rsl_rl.RslRlPpoActorCriticCfg is a thin shim between NVIDIA's config dataclasses and whatever rsl_rl version pip resolved. The verbatim ./isaaclab.sh --install doesn't pin rsl-rl-lib, so without manual intervention pip picks the latest, which mismatches whatever shim version you're on. The right pin is invisible from either upstream README — we had to find it by walking down the version list until the error pattern flipped.
33 — Unitree RL Lab's URDF spawn switch is "[Optional]" in the README but actually required for Method 2
The README path: Unitree RL Lab presents two ways to provide robot description files. Method 2 (URDF) is the recommended one for Isaacsim ≥ 5.0 (which is what we have). The README's three steps:
- Clone
unitree_ros. - Set
UNITREE_ROS_DIRinsource/.../assets/robots/unitree.py. - "[Optional]: change
robot_cfg.spawnif you want to use urdf files".
What actually happens: step 3 is not optional. Without it, robot_cfg.spawn stays on its default UnitreeUsdFileCfg, pointing at f"{UNITREE_MODEL_DIR}/G1/29dof/usd/g1_29dof_rev_1_0/g1_29dof_rev_1_0.usd". UNITREE_MODEL_DIR is for Method 1 (USD) — which we did NOT set up — so it stays at its placeholder string "path/to/unitree_model". Training crashes on first scene creation:
FileNotFoundError: USD file not found at path at:
'path/to/unitree_model/G1/29dof/usd/g1_29dof_rev_1_0/g1_29dof_rev_1_0.usd'
What we did: edited source/unitree_rl_lab/unitree_rl_lab/assets/robots/unitree.py to swap the spawn config from USD to URDF for both G1 29-DoF entries. Each block toggles which spawn=... line is commented:
- before: URDF block commented, USD block active
- after: URDF block active, USD block commented
Two blocks affected:
UNITREE_G1_29DOF_CFG(line 397) — used by the velocity locomotion task.UNITREE_G1_29DOF_MIMIC_CFG(line 531) — used by the Mimic motion-tracking tasks (our Stage 1 target).
The edit is documented inline in Phase 7's install procedure. It should become a real patch file at patches/unitree_rl_lab/unitree.py.patch for full reproducibility (TODO).
Why the README downplays it: the bootcamp content we've been working from (chapter 11 "Unitree RL Lab Mimic") uses Method 1 (USD) on its cloud GPU instance — so the bootcamp's commands run without this edit, because UNITREE_MODEL_DIR gets set there instead. Method 2 (URDF) is recommended for Isaacsim ≥ 5.0 but only fully wired when you make the spawn edits the README labels "[Optional]".
Symptom 1 — missing NPZ. First training attempt on a Mimic task dies with:
AssertionError: Invalid file path:
.../tasks/mimic/robots/g1_29dof/gangnanm_style/G1_gangnam_style_V01.bvh_60hz.npz
What's going on. Each Mimic task ships a .csv reference motion (~600 KB, e.g. retargeted from a BVH source) but expects a .npz at runtime (a sim-friendly cached version with per-body state pre-computed). The README is silent about this — the only hint is the unitree_rl_lab repo's most recent commit message: "doc: add comment to generate npz file befroe traning mimic task". The generator is at scripts/mimic/csv_to_npz.py: it loads Isaac Sim, replays the CSV motion through the configured robot articulation, and dumps a numpy archive of joint and body state at each frame.
What we did:
cd unitree_rl_lab
python scripts/mimic/csv_to_npz.py \
-f source/unitree_rl_lab/unitree_rl_lab/tasks/mimic/robots/g1_29dof/gangnanm_style/G1_gangnam_style_V01.bvh_60hz.csv \
--input_fps 60 --headlessThe NPZ lands next to the CSV. For Gangnanm-Style: 1618 frames × 29 joints × 35 bodies × {pos, quat, lin_vel, ang_vel} ≈ 3.3 MB.
Symptom 2 — hang in teardown. Once the NPZ is written, the script stops printing but does not exit. The python process remains alive at ~0% CPU / ~0% GPU, holding the env. Killing it is required:
pkill -KILL -f csv_to_npz.pyIsaac Sim 5.1's aarch64 shutdown path has a known slow / occasionally-stuck phase. It does not affect the NPZ on disk — that's already been written and is structurally valid.
Symptom 3 — CUDA assert at training time if NPZ and IsaacLab versions don't match. When we first generated the NPZ under IsaacLab main HEAD and then switched IsaacLab to v2.3.0 for training, training died after env construction with:
torch.AcceleratorError: CUDA error: device-side assert triggered
in commands.py:134:
self.motion.body_quat_w[self.time_steps, self.motion_anchor_body_index]
Cause: the NPZ stores body data per-body, in the order IsaacLab produced at NPZ-generation time. The number of bodies depends on which robot config (URDF / USD spawn, which fixed links collapse, etc.) IsaacLab built. We saw 32 bodies under main HEAD vs 35 bodies under v2.3.0. The training task computes motion_anchor_body_index from the current IsaacLab's robot config (35), then indexes into the old NPZ (which only has 32) — out of range, CUDA assert.
Fix: always regenerate the NPZ when changing IsaacLab version or robot spawn config. Phase 7's install procedure puts the NPZ generation as the last step before training so it always matches the rest of the env.
The prior project (a MuJoCo-based simulator on macOS, kept in a separate repo) hit some of the same surprises but on a different operating system. Where the issues differ we've called them out by environment. Two notable ones:
- vuer ImportError: the macOS project blamed
params_proto 3.3.0. On the Spark the same observable error came from OpenSSL 3.0 vs 3.6. Both can fire and we addressed both: pinnedparams_proto<3.0(resolved to 2.13.2) and force-loaded conda's libcrypto. Inspire_Controller_FTPThread-vs-Process: the macOS project replacedProcesswithThreadto avoid amultiprocessing.Arraycross-process bug specific to macOS-spawn. On Linux-fork the cross-process Array works; we have not needed that change. TheNameErrorwe hit (Issue 15) is unrelated.
For when a question is "what version of X did we install where?":
pinocchio 3.1.0(conda-forge — also bundles thecasadibinding sofrom pinocchio import casadi as cpinworks without extra steps)numpy 1.26.4cyclonedds 0.10.2(built from source)unitree_sdk2py 1.0.1(editable)televuer 4.0.0,vuer 0.0.60,params_proto 2.13.2(pinned)logging_mp 0.1.6(pinned for compatibility with xr_teleoperate at commit4c0afdb— see Issue 17)teleimager 1.5.0(editable)dex_retargeting 0.4.7(editable)nlopt 2.10.1(conda-forge)inspire_sdkpy 1.0.0(editable, frominspire_hand_ws/inspire_hand_sdk) +pyqt 5.15.x,pymodbus 3.6.9,pyserial(conda-forge)torch 2.3.0(CPU, aarch64 wheel — pulled in by dex_retargeting)matplotlib 3.7.5,rerun-sdk 0.20.1,meshcat 0.3.2,sshkeyboard 2.3.1(xr_teleoperate'srequirements.txt)pyzmq 27.x(needed by teleimager)
torch 2.9.0+cu130+torchvision 0.24.0(no torchaudio — see Issue 14)- 17 NVIDIA CUDA-13 runtime libraries
isaacsim 5.1.0.0+ 27 sub-libraries (app, asset, core, kernel, gui, robot, sensor, replicator, etc.)isaaclab 0.54.3+ 6 components (isaaclab_assets 0.2.4,isaaclab_contrib 0.0.2,isaaclab_mimic 1.0.16,isaaclab_rl 0.5.1,isaaclab_tasks 0.11.16) — installed editable fromIsaacLab/on the develop branch HEADcyclonedds 0.10.2(built from source againstcyclonedds/install/)unitree_sdk2py 1.0.1(editable)teleimager 1.5.0(editable, fromunitree_sim_isaaclab/teleimager/)rerun-sdk 0.20.1,onnxruntime 1.22.1,pyzmq 27.0.0,aiortc 1.14.0,numba 0.59.1numpy 1.26.4,opencv-python 4.11.0.86libstdcxx-ngpatch via conda-forge
torch 2.9.0+cu130+torchvision 0.24.0(no torchaudio, same reason as Issue 14)- 17 NVIDIA CUDA-13 runtime libraries (same set as
unitree_sim_env) isaacsim 5.1.0.0+ 27 sub-libraries (same asunitree_sim_env; ~13 GB of duplicate install — conda envs do not share)isaaclab 0.54.3+ 5 components — installed editable fromunitree_lab_isaac_lab/pinned to tag v2.3.0, distinct fromIsaacLab/(see Issue 31). Version numbers happen to matchunitree_sim_env's; different source folder.unitree_rl_lab 0.2.1(editable, fromunitree_rl_lab/)rsl-rl-lib 3.3.0(downgraded — see Issue 32). The other 4 frameworks./isaaclab.sh --installpulled in:rl_games 1.6.1,stable_baselines3 2.8.0,skrl 2.1.0,robomimic 0.4.0.argcomplete 3.6.3(pulled byunitree_rl_lab's installer; rl_lab's shell-completion setup hits non-fatal errors writing to/usr/local/share/zsh/site-functionswithout sudo — harmless)numpy 1.26.0,opencv-python-headless 4.11.0.86- Activate.d hook sets
LD_PRELOAD=/lib/aarch64-linux-gnu/libgomp.so.1andOMNI_KIT_ACCEPT_EULA=YES(same pattern asunitree_sim_env's hook)