Faster-Motus

English | 简体中文

9.59× faster Motus training iterations, optimized by AI agents.

Implementation	Time / training iteration (s) ↓	Speedup ↑
Baseline	7.468	1.00×
Faster-Motus	0.778	9.59×

Measured on 8 GPUs with the full RoboTwin 2.0 data. In deterministic mode, the training iterations of Faster-Motus are bitwise identical to the baseline.

Quick Start

Prepare the environment, weights and data as for upstream Motus. Set the paths in configs/robotwin.yaml, then source the Faster-Motus switches and launch upstream's script (8 GPUs, batch 8 per GPU):

source scripts/faster-motus.env.sh   # MOTUS_OPT_DS_HOOK=1 MOTUS_OPT_LANG_MMAP=1
bash scripts/train.sh

With the switches unset, the code runs the baseline code paths.

Agent Optimization History

Measured on eight GPUs with the RoboTwin 2.0 Recipe of Motus.

What changed

• DeepSpeed gradient hooks (MOTUS_OPT_DS_HOOK, preset): DeepSpeed 0.18's ZeRO-1/2 hook counts the parameters used in the backward pass, over all 1,361 trainable tensors, from every tensor's hook. That O(n²) Python work took 6.6 s of each 7.5 s iteration; the count is now cached per autograd graph task.
• umt5 text embeddings (MOTUS_OPT_LANG_MMAP, preset): each sample read a whole ~52 MB embedding file to use one of its ~100 embeddings. The files are now memory-mapped (identical tensors), and each loader worker keeps the parsed list.
• Data-loader workers (MOTUS_OPT_LOADER_REUSE, MOTUS_OPT_LOADER_SPEC, off by default): workers kept across epochs, and a speculative next epoch. They add no speed on the full data, and LOADER_SPEC is not recommended there; see the full-data results.

Validation

Numerical Consistency. We measured the numerical consistency of iteration loss during training between Faster-Motus and the baseline. Both runs used the full data in deterministic mode, with the same seed for each rank:

Comparison	Iterations	Result
Baseline vs Faster-Motus preset	1,100	bitwise identical on 8 ranks
Baseline vs all four switches	1,100	bitwise identical on 8 ranks

Upstream Motus documentation

Based on thu-ml/Motus at 88f6fe3, by Hongzhe Bi, Hengkai Tan, Shenghao Xie et al. Paper · Project page · Apache-2.0 license. The original README and upstream authorship are retained below.

Original Motus README: overview, installation, checkpoints, inference and training

Motus: A Unified Latent Action World Model

Table of Contents

• Table of Contents
• Overview
• Updates
• Requirements
• Installation
• Model Checkpoints
• Data Format
• Running Inference
• Training
• Troubleshooting
• Citation

Overview

Motus is a unified latent action world model that leverages existing pretrained models and rich, sharable motion information. Motus introduces a Mixture-of-Transformers (MoT) architecture to integrate three experts (understanding, action, and video generation) and adopts a UniDiffuser-style scheduler to enable flexible switching between different modeling modes (World Models, Vision-Language-Action Models, Inverse Dynamics Models, Video Generation Models, and Video-Action Joint Prediction Models). Motus further leverages optical flow to learn latent actions and adopts a three-phase training pipeline and six-layer data pyramid, thereby extracting pixel-level "delta action" and enabling large-scale action pretraining.

Component	Base Model	Parameters
VGM (Video Generation Model)	Wan2.2-5B	~5.00B
VLM (Vision-Language Model)	Qwen3-VL-2B	~2.13B
Action Expert	-	~641.5M
Understanding Expert	-	~253.5M
Total	-	~8B

Key Results (RoboTwin 2.0 Simulation. With 50 clean and 500 randomized data entries per task, we merge the data from all 50 tasks for multi-task training.):

• 87.02% average success rate (+15% over X-VLA, +45% over π₀.₅)

Updates

• [2025-12-16] Initial release of Motus with pretrained checkpoints and training code.
• [2025-12-20] Simple RoboTwin inference
• [2025-12-24] LeRobotDataset format support
• [2025-12-24] Optimized training scripts
• [2025-12-26] MultiLeRobotDataset format support
• [2025-12-27] RoboTwin raw dataset conversion
• [2025-12-27] Three-view image concatenation scripts

We welcome community members to help maintain and extend Motus. Welcome to join the Motus community and contribute together!

Requirements

Mode	VRAM	Recommended GPU
Inference (with pre-encoded T5)	> 24 GB	RTX 5090
Inference (without pre-encoded T5)	~ 41 GB	A100 (40GB) / A100 (80GB) / H100
Training	> 80 GB	A100 (80GB) / H100

Installation

# Clone the repository
git clone https://github.com/thu-ml/Motus.git
cd Motus

# Create conda environment
conda create -n motus python=3.10 -y
conda activate motus

# install torch (cuda12.8)
pip install torch==2.7.1 torchvision==0.22.1 --index-url https://download.pytorch.org/whl/cu128

# install flash 
pip install flash-attn --no-build-isolation

# Install motus dependencies
pip install -r requirements.txt

# (Optinal) Install lerobot dependencies
pip install --no-deps lerobot==0.3.2
pip install -r requirements/lerobot.txt

Model Checkpoints

We provide multiple checkpoints for different use cases:

Model	Use Case	Description	Checkpoint Path
Motus_Wan2_2_5B_pretrain	Pretrain / VGM Backbone	Stage 1 VGM pretrained checkpoint	motus-robotics/Motus_Wan2_2_5B_pretrain
Motus	Fine-Tuning	Stage 2 latent action pretrained checkpoint	motus-robotics/Motus
Motus_robotwin2	Inference / Fine-Tuning	Stage 3 RoboTwin2 fine-tuned checkpoint	motus-robotics/Motus_robotwin2

Download checkpoints:

# Create pretrained models directory
mkdir -p pretrained_models

# Download Motus checkpoints
huggingface-cli download motus-robotics/Motus_Wan2_2_5B_pretrain --local-dir ./pretrained_models/Motus_Wan2_2_5B_pretrain
huggingface-cli download motus-robotics/Motus --local-dir ./pretrained_models/Motus
huggingface-cli download motus-robotics/Motus_robotwin2 --local-dir ./pretrained_models/Motus_robotwin2

# Download foundation models
huggingface-cli download Qwen/Qwen3-VL-2B-Instruct --local-dir ./pretrained_models/Qwen3-VL-2B-Instruct
huggingface-cli download Wan-AI/Wan2.2-TI2V-5B --local-dir ./pretrained_models/Wan2.2-TI2V-5B

Update config paths in your embodiment-specific config file (e.g., configs/robotwin.yaml, configs/ac_one.yaml, or other embodiment configs):

model:
  wan:
    checkpoint_path: "./pretrained_models/Motus_Wan2_2_5B_pretrain"
    config_path: "./pretrained_models/Motus_Wan2_2_5B_pretrain"
    vae_path: "./pretrained_models/Wan2.2-TI2V-5B/Wan2.2_VAE.pth"
  vlm:
    checkpoint_path: "./pretrained_models/Qwen3-VL-2B-Instruct"
    config_path: "./pretrained_models/Qwen3-VL-2B-Instruct"

Data Format

Motus supports three types of datasets with specific directory structures for optimal training and inference.

📖 See detailed guide: Data Format Guide

Quick Overview:

• RoboTwin 2.0: Simulation data with clean/randomized splits
• Real-World: AC-One, Aloha-Agilex-2 robot data

Data Conversion Tools:

• RoboTwin Dataset Converter
• Multi-Camera Concatenation

Running Inference

📖 See detailed guide: Inference Guide

• RoboTwin 2.0: Evaluation Setup
• Real-World: Minimal inference without robot environment

Training

Motus follows a three-stage training pipeline:

Stage	Data	Training
Pretrained Foundation Models	Level 1: Web Data	VGM and VLM
Stage 1 (VGM Training)	Level 2: Egocentric Human Videos Level 3: Synthetic Data Level 5: Multi-Robot Task Trajectory	Only VGM
Stage 2 (Motus Pretraining)	Level 2: Egocentric Human Videos Level 3: Synthetic Data Level 4: Task-agnostic Data Level 5: Multi-Robot Task Trajectory	Motus (all 3 experts, with latent actions)
Stage 3 (Motus SFT)	Level 6: Target-Robot Task Trajectory	Motus (all 3 experts, with actions)

The six-layer data pyramid is shown in the figure here:

📖 See detailed guide: Training Guide

Data Preparation:

• RoboTwin Dataset Converter
• Multi-Camera Concatenation

Troubleshooting

📖 Detailed guides:

• Inference Issues
• Training Issues
• Data Format Issues

Citation

If you find our work helpful, please cite us:

@misc{bi2025motusunifiedlatentaction,
      title={Motus: A Unified Latent Action World Model}, 
      author={Hongzhe Bi and Hengkai Tan and Shenghao Xie and Zeyuan Wang and Shuhe Huang and Haitian Liu and Ruowen Zhao and Yao Feng and Chendong Xiang and Yinze Rong and Hongyan Zhao and Hanyu Liu and Zhizhong Su and Lei Ma and Hang Su and Jun Zhu},
      year={2025},
      eprint={2512.13030},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2512.13030}, 
}

Thank you!