HEI ReBot Lift

English | 中文 | français | Español

🚀 Overview

HEI ReBot Lift is a dual-arm lifting mobile robot project for embodied AI learning, reproduction, and real-robot validation. Its goal is to lower the barrier to building real robot learning systems. It follows the idea of truly reproducible open source: not only releasing code, but also organizing hardware materials, wiring, deployment steps, the VR teleoperation pipeline, dataset recording, ACT/VLA training, and real-robot rollout workflows, so builders can move from hardware assembly all the way to policy deployment. The robot consists of dual arms, a lift platform, a four-wheel O-type omnidirectional chassis, and three cameras. The software is built on LeRobot and covers MuJoCo/Pinocchio inverse kinematics, LeRobotDataset, imitation learning, and VLA policy deployment.

🚀 Dual-Arm Mobile Manipulation · 📖 Open Hardware + Software Stack · 🤖 LeRobot Ready

🚀 Quick Setup · 🦾 Hardware · 🎮 VR Teleoperation · 📷 Record Data · 🧠 Train ACT · ✨ Train VLA

✨ Features

Icon	Capability	Description
🦾	Dual-arm manipulation	Damiao dual arms and grippers for teleoperation, recording, and policy rollout
⬆️	Lift platform	Automatic homing on startup, with the upper limit defined as height.pos = 0
⭕	Omnidirectional base	Four-wheel O-type omnidirectional chassis with x/y/theta velocity control
🎮	VR teleoperation	Telegrip captures VR controller data; MuJoCo + Pinocchio/CasADi compute IK
📷	Three-camera data	front, left_wrist, and right_wrist visual inputs
🧠	Imitation learning / VLA	Supports LeRobotDataset, ACT, SmolVLA, and real-robot rollout

🤝 Get Your Own Robot / Join the Community

If you want to reproduce your own HEI ReBot Lift, you can follow the hardware materials, BOM, wiring notes, and software deployment documents gradually organized in this project to source parts and build the robot yourself. We also welcome builders and researchers interested in dual-arm mobile manipulation, VR teleoperation, LeRobot data collection, ACT/VLA training, and real-robot deployment to exchange ideas with us.

If you want to get your own robot faster, or would like to collaborate on hardware reproduction, teaching labs, data collection, algorithm validation, or application deployment, feel free to contact us.

WeChat community / collaboration: hgm159951   |   Email: hgm159951@163.com

Reproductions, discussions, issues, improvements, and real-robot test feedback are all welcome.

📁 Project Layout

hei-rebot-lift/
├── README.md
├── README_zh.md
├── README_Fr.md
├── README_es.md
├── LICENSE
├── community/                    # Community notes and collaboration records
├── hardware/                     # Hardware BOM, wiring, device binding, mechanical materials
├── media/                        # Images, videos, and README assets
├── docs/                         # Deployment and usage documentation
└── software/
    └── lerobot-hei-rebot-lift/   # Runnable LeRobot-based software project

The runnable software lives in:

software/lerobot-hei-rebot-lift/

All commands below assume you first enter this directory:

cd software/lerobot-hei-rebot-lift

🖼️ Showcase

⭐ Star History

🗺️ Roadmap & Latest Status

We will continue improving HEI ReBot Lift across hardware materials, software interfaces, data collection workflows, and mainstream embodied AI policy integrations. The table below summarizes the current status and links to the related documentation.

Module	Status	Current Progress	Related DOC
Robot body	✅ First version completed	Dual arms, lift platform, and four-wheel O-type omnidirectional base are integrated and tested as a complete system	Hardware
Damiao motor driver	✅ First version completed	damiao_u2can is implemented for dual arms, grippers, chassis, and lift motor control	Damiao U2CAN
Lift platform	✅ First version completed	Supports upper-limit homing on startup and height.pos position-target control	Robot Driver
Omnidirectional base	✅ First version completed	Supports x.vel, y.vel, and theta.vel commands with basic acceleration smoothing	Robot Driver
Three-camera vision	✅ First version completed	Supports front, left_wrist, and right_wrist OpenCV cameras with MJPG by default	Robot Driver
VR + MuJoCo IK	✅ First version completed	Telegrip + MuJoCo + Pinocchio/CasADi is connected to the real-robot control pipeline	VR MuJoCo IK
LeRobot integration	✅ First version completed	hei_rebot_lift robot/client/host is implemented with teleoperate, record, replay, evaluate, and rollout scripts	Examples
Data collection	✅ First version completed	Supports LeRobotDataset recording, resume recording, visualization, and bad-episode cleanup	Record Guide
ACT training and rollout	✅ Verified	Supports ACT training and real-robot rollout	Examples
SmolVLA / VLA	✅ Initial support	Supports SmolVLA training and real-robot rollout entry points	Examples
Open hardware materials	🚧 In progress	hardware/, media/, docs/, and community/ project structure is prepared	Hardware
Community and reproduction	🚧 Ongoing	WeChat group, email contact, and GitHub project entry are available	Community
Other mainstream VLA deployment reproduction	⏳ Coming soon	Plan to reproduce and test more mainstream VLA policies for training, inference, and real-robot deployment on HEI ReBot Lift	Not completed

🦾 Hardware

The hardware package now includes the overall BOM, full robot STEP assembly, 3D printed parts, and metal/CNC/sheet-metal manufacturing files.

Resource	File / Directory	Description
Hardware guide	hardware/README.md	Hardware directory index, reproduction order, and safety checklist
Overall BOM	hardware/HEI_ReBot_Lift_BOM.md / xlsx	Main purchasing and preparation checklist for the complete robot
Full robot assembly	hardware/Hei_robot_lift.STEP	Full STEP model for structure review and assembly reference
3D printed parts	hardware/3D_Printed_Parts/	STL files for printed covers, brackets, lift, chassis, and camera-related parts
Metal body parts list	hardware/Metal_Parts/HEI_Metal_Body_Parts_List.xlsx	Metal/CNC/sheet-metal part list
Metal CAD files	hardware/Metal_Parts/step/ / hardware/Metal_Parts/dwg/	STEP and DWG files for machining communication

Dual arms: left and right arms with 7 Damiao motors each. Joints 1-3 use DM4340P, joints 4-6 and gripper use DM4310
Chassis: four-wheel O-type omnidirectional mobile base, using DM4310 wheel motors
Lift: lead-screw lift platform using a DM4310 motor. On startup, the upper limit is homed as height.pos = 0
Cameras: three OpenCV cameras: front, left_wrist, right_wrist
Communication: ZMQ between robot-side host and computer-side client
Teleoperation: VR headset and controllers. Telegrip captures VR data; MuJoCo + Pinocchio/CasADi compute IK

🧩 Software Modules

software/lerobot-hei-rebot-lift/src/lerobot/robots/hei_rebot_lift/        Robot driver
software/lerobot-hei-rebot-lift/src/lerobot/motors/damiao_u2can/          Damiao U2CAN communication
software/lerobot-hei-rebot-lift/examples/hei_rebot_lift/                  Record, replay, evaluate, and rollout scripts
software/lerobot-hei-rebot-lift/examples/hei_rebot_lift/VR_mujoco_ik/     VR + MuJoCo + Pinocchio IK

⚡ Quick Setup

Create the LeRobot environment:

cd software/lerobot-hei-rebot-lift
conda create -n lerobot5 python=3.12 -y
conda activate lerobot5
pip install -e ".[core_scripts,training,pyzmq-dep]"

This full installation includes dataset recording/editing, Rerun visualization, keyboard input, ZMQ communication, and policy training. The Python module is imported as zmq, but its correct package name is pyzmq; do not install the unrelated zmq package.

For a Jetson that only runs the robot-side host, use the lighter installation:

pip install -e ".[hardware,pyzmq-dep]"

Create the VR/MuJoCo IK environment:

cd software/lerobot-hei-rebot-lift/examples/hei_rebot_lift/VR_mujoco_ik
conda env create -f environment.yml

Verify Pinocchio + CasADi:

conda activate hei-rebot-vr
env -u LD_LIBRARY_PATH python -c "import pinocchio as pin; from pinocchio import casadi as cpin; print(pin.__version__); print('casadi binding ok')"

🔌 Device Mapping

Stable udev device names are used by default:

/dev/hei_right_arm   Right arm U2CAN
/dev/hei_left_arm    Left arm U2CAN
/dev/hei_chassis     Chassis U2CAN
/dev/hei_lift        Lift motor U2CAN
/dev/hei_lift_io     Lift limit-switch serial port

Default cameras:

front       /dev/video0
left_wrist  /dev/video2
right_wrist /dev/video4

Find connected cameras:

cd software/lerobot-hei-rebot-lift
PYTHONPATH=src conda run --no-capture-output -n lerobot5 lerobot-find-cameras

🎮 Startup Flow

Start the robot-side host:

cd software/lerobot-hei-rebot-lift
PYTHONPATH=src conda run --no-capture-output -n lerobot5 hei-rebot-lift-host

Start Telegrip on the computer:

cd software/lerobot-hei-rebot-lift/examples/hei_rebot_lift/VR_mujoco_ik
./run_telegrip.sh

Open this URL in the VR headset browser:

https://COMPUTER_IP:8443

Start MuJoCo IK on the computer:

cd software/lerobot-hei-rebot-lift/examples/hei_rebot_lift/VR_mujoco_ik
./run_mujoco_ik.sh

Test teleoperation:

cd software/lerobot-hei-rebot-lift
PYTHONPATH=src conda run --no-capture-output -n lerobot5 python -u examples/hei_rebot_lift/teleoperate.py   --remote-ip 192.168.31.127

📷 Record Data

cd software/lerobot-hei-rebot-lift
PYTHONPATH=src conda run --no-capture-output -n lerobot5 python -u examples/hei_rebot_lift/record.py   --repo-id HGM/hei_rebot_lift_task1   --remote-ip 192.168.31.127   --num-episodes 5   --episode-time-sec 120   --reset-time-sec 30   --task-description "Pick up the yellow block from the floor and put it on the table in front"

By default, data is saved locally and is not pushed to the Hugging Face Hub. Add --push-to-hub when uploading is needed.

🧠 Train ACT

cd software/lerobot-hei-rebot-lift
PYTHONPATH=src conda run --no-capture-output -n lerobot5 lerobot-train   --dataset.repo_id=HGM/hei_rebot_lift_task1   --policy.type=act   --policy.device=cuda   --policy.push_to_hub=false   --output_dir=outputs/train/act_hei_rebot_lift_task1   --job_name=act_hei_rebot_lift_task1   --batch_size=8   --steps=10000   --save_freq=10000   --log_freq=200   --num_workers=4   --wandb.enable=false

✨ Train SmolVLA

cd software/lerobot-hei-rebot-lift
PYTHONPATH=src conda run --no-capture-output -n lerobot5 lerobot-train   --dataset.repo_id=HGM/hei_rebot_lift_task1   --policy.type=smolvla   --policy.device=cuda   --policy.push_to_hub=false   --output_dir=outputs/train/smolvla_hei_rebot_lift_task1   --job_name=smolvla_hei_rebot_lift_task1   --batch_size=1   --steps=1000   --save_freq=1000   --log_freq=50   --num_workers=2   --wandb.enable=false

🤖 Real-Robot Rollout

ACT rollout:

cd software/lerobot-hei-rebot-lift
PYTHONPATH=src conda run --no-capture-output -n lerobot5 python -u examples/hei_rebot_lift/rollout.py   --remote-ip 192.168.31.127   --model-id outputs/train/act_hei_rebot_lift_task1/checkpoints/010000/pretrained_model   --task "Pick up the yellow block from the floor and put it on the table in front"   --duration-sec 30   --inference sync

SmolVLA rollout:

cd software/lerobot-hei-rebot-lift
PYTHONPATH=src conda run --no-capture-output -n lerobot5 python -u examples/hei_rebot_lift/rollout.py   --remote-ip 192.168.31.127   --model-id outputs/train/smolvla_hei_rebot_lift_task1/checkpoints/001000/pretrained_model   --task "Pick up the yellow block from the floor and put it on the table in front"   --duration-sec 60   --fps 10   --inference rtc

📖 More Documentation

software/lerobot-hei-rebot-lift/README.md
software/lerobot-hei-rebot-lift/examples/hei_rebot_lift/README.md
software/lerobot-hei-rebot-lift/src/lerobot/robots/hei_rebot_lift/README.md
software/lerobot-hei-rebot-lift/examples/hei_rebot_lift/VR_mujoco_ik/README.md

🙏 References & Acknowledgments

The development of HEI ReBot Lift benefits from several excellent open-source projects and community efforts. In particular, we would like to thank:

• LeRobot: for providing a unified robot interface, the LeRobotDataset format, training utilities, and policy implementations such as ACT and SmolVLA, which form a strong foundation for real-robot data collection, training, and deployment.
• reBot / reBot-DevArm: for open robotic arm hardware, model resources, and practical references for embodied AI open-source projects. It also inspired the way this project organizes hardware materials, deployment documentation, and reproducible workflows.

This project customizes and extends ideas from these open-source ecosystems, aiming to further lower the barrier to learning, reproducing, collecting data with, and deploying policies on a dual-arm lifting mobile robot.

📄 License

This project is built on top of Hugging Face LeRobot and keeps the LeRobot dataset, training, policy, and robot-interface ecosystem. Please also follow the original LeRobot license requirements.