OpenPI Piper UMI

English | 中文

This repo fine-tunes pi0.5 for single-arm aloha-piper manipulation using single-gripper UMI-style data. The policy input is one RGB camera image, and the policy output is a future chunk of relative end-effector delta poses. During deployment, the client composes those delta EE poses into a target EE pose and sends that target to the aloha-piper arm to execute the task.

The project includes the full practical pipeline:

• UMI/Piper data conversion scripts.
• One OpenPI fine-tuning config for single-camera RGB + action7 pose data.
• A small runnable demo dataset.
• Training, policy server, offline eval, and aloha-piper deployment scripts.

Large full datasets and checkpoints are not included.

Install

git clone https://github.com/Agentic-Intelligence-Lab/openpi-piper-umi.git
cd openpi-piper-umi
sudo apt-get update
sudo apt-get install -y ffmpeg pkg-config \
  libavformat-dev libavcodec-dev libavdevice-dev libavutil-dev \
  libavfilter-dev libswscale-dev libswresample-dev
uv sync --group dev --frozen
source .venv/bin/activate

If uv sync fails while building av, the FFmpeg development libraries are missing. Re-run the apt-get install command above.

Demo Data

The repo includes two tiny LeRobot v2.1 datasets:

sample_data/stack_bowls_umi
sample_data/stack_bowls_piper

Each dataset has:

meta/info.json
meta/episodes.jsonl
meta/episodes_stats.jsonl
meta/tasks.jsonl
data/chunk-000/episode_000000.parquet
videos/chunk-000/observation.images.rgb/episode_000000.mp4

The action format is:

action7 = [dx, dy, dz, drx, dry, drz, clamp]

dx/dy/dz are local EE-frame translation deltas in meters. drx/dry/drz are xyz Euler rotation deltas in radians. clamp is the next clamp/gripper target in UMI units.

Training Config

The main config is:

pi05_umi_stack_bowls

It lives in:

src/openpi/training/config.py

To choose your training data, edit this block:

local_roots=(
    _umi_data_path("stack_bowls_umi"),
    _umi_data_path("stack_bowls_piper"),
)

Use any LeRobot datasets with the same RGB + action7 format. You can use one dataset, add more datasets, or point UMI_DATA_ROOT to another folder:

export UMI_DATA_ROOT=/path/to/my_lerobot_datasets

For real fine-tuning, set the pi0.5 base PyTorch checkpoint:

export PI05_BASE_PYTORCH_PATH=/path/to/pi05_base

That directory should contain model.safetensors.

Compute Norm Stats

bash scripts/prepare_umi_norm_stats.sh

The norm stats are saved to:

assets/pi05_umi_stack_bowls/umi_stack_bowls/norm_stats.json

Train

export UMI_DATA_ROOT=/path/to/my_lerobot_datasets
export PI05_BASE_PYTORCH_PATH=/path/to/pi05_base
RUN_NAME=my_run \
CUDA_VISIBLE_DEVICES=0 \
NPROC_PER_NODE=1 \
BATCH_SIZE=512 \
GRADIENT_ACCUMULATION_STEPS=8 \
NUM_TRAIN_STEPS=10000 \
SAVE_INTERVAL=1000 \
bash scripts/train.sh

The training script saves:

checkpoints/pi05_umi_stack_bowls/my_run/<step>/model.safetensors
checkpoints/pi05_umi_stack_bowls/my_run/<step>/optimizer.pt
checkpoints/pi05_umi_stack_bowls/my_run/<step>/assets/umi_stack_bowls/norm_stats.json

Offline Eval

CKPT=checkpoints/pi05_umi_stack_bowls/my_run/<step>
NORM=assets/pi05_umi_stack_bowls/umi_stack_bowls/norm_stats.json

python scripts/openpi_umi_offline_eval.py \
  --config-name pi05_umi_stack_bowls \
  --checkpoint-dir "$CKPT" \
  --norm-stats-path "$NORM" \
  --dataset-root sample_data/stack_bowls_piper \
  --disable-raw-ee6-eval \
  --episodes 1 \
  --episode-ids 0 \
  --max-anchors 5

Outputs go to:

outputs/umi_offline_eval/

This repo also includes two lightweight previous eval examples under:

outputs/umi_offline_eval/example_real_robot_ckpt
outputs/umi_offline_eval/example_mixed_umi_real_ckpt

Aloha-Piper

Set checkpoint paths:

CKPT=checkpoints/pi05_umi_stack_bowls/my_run/<step>
NORM=assets/pi05_umi_stack_bowls/umi_stack_bowls/norm_stats.json

Bring up CAN:

sudo ip link set can1 down || true
sudo ip link set can1 up type can bitrate 1000000
ip link show can1

Terminal 1, start policy server:

python scripts/openpi_umi_policy_server.py \
  --config-name pi05_umi_stack_bowls \
  --checkpoint-dir "$CKPT" \
  --norm-stats-path "$NORM" \
  --host 127.0.0.1 \
  --port 8000

Terminal 2, start execution:

RIGHT_WRIST_SERIAL=<your-realsense-serial> \
PIPER_CAN_NAME=can1 \
EXECUTE=1 \
GRIPPER=1 \
STEPS=0 \
CHUNK_DURATION_S=1.0 \
SPEED=20 \
bash scripts/run_aloha_piper_final.sh

CHUNK_DURATION_S controls how long the composed action chunk takes to execute. SPEED is the Piper controller speed gear.

Important Scripts

scripts/convert_fastumi_to_lerobot_v21_pose6.py      FastUMI -> pose6/action7 LeRobot
scripts/convert_piper_lerobot_to_umi_v21_pose6.py    Piper LeRobot -> pose6/action7 UMI LeRobot
scripts/prepare_umi_norm_stats.sh                    compute normalization stats
scripts/train.sh                                     training entry script
scripts/openpi_umi_policy_server.py                  checkpoint -> websocket policy server
scripts/openpi_umi_offline_eval.py                   recorded-data eval
scripts/piper_right_umi_delta_pose_client.py         RealSense + Piper client
scripts/run_aloha_piper_final.sh                     aloha-piper final-target deployment wrapper