RHOS

🏛️ About RHOS Lab

RHOS studies Embodied AI, Physical Reasoning, and Human Activity Understanding. We are building a knowledge and reasoning-driven system that enables intelligent agents and robots to perceive human activities, reason about human behavior logics, learn skills from human activities, and interact with the environment.

Our homepage: https://mvig-rhos.com

---

🔥 Latest Release

OpenD4RT

An unofficial PyTorch/GPU implementation of D4RT for 4D reconstruction and tracking, with WorldTrack evaluation, visualization tools, and released Hugging Face checkpoints.

🔗 Resources

• 💻 GitHub: Lijiaxin0111/Open-d4rt
• 🌿 RHOS Branch: OpenD4RT
• 🤗 Hugging Face: OpenD4RT Checkpoints

---

📚 Previous Works

Open4DHOI: Efficient and Scalable Monocular Human-Object Interaction Motion Reconstruction

🔗 Resources

• 📖 Paper: arXiv:2512.00960
• 🌐 Website: Open4DHOI
• 💻 GitHub: wenboran2002/open4dhoi_code
• 🤗 Dataset: Open4DHOI
• 📦 Demo Data: Google Drive
• 🌿 RHOS Branch: open4dhoi_code

L1Flow: L1 Sample Flow for Efficient Visuomotor Learning

🔗 Resources

• 📖 Paper: arXiv:2511.17898
• 🌐 Website: L1Flow Project Page
• 💻 GitHub: THyanNK/L1Flow
• 📦 Dataset: Diffusion Policy Training Data
• 🤗 Checkpoints: L1Flow Results
• 🌿 RHOS Branch: L1Flow

IPR-1: Interactive Physical Reasoner

🔗 Resources

• 📖 Paper: arXiv:2511.15407
• 💻 GitHub: RHOS/IPR-1 branch

RoboHiMan: A Hierarchical Evaluation Paradigm for Compositional Generalization in Long-Horizon Manipulation

🔗 Resources

• 📖 Paper: arXiv:2510.13149
• 🌐 Website: RoboHiMan
• 💻 GitHub: chenyt31/RoboHiMan
• 🌿 RHOS Branch: RoboHiMan

exUMI

🔗 Resources

• 📖 Paper: arXiv:2509.14688
• 🌐 Website: exUMI Project Page
• 💻 GitHub: silicx/exUMI
• 🌿 RHOS Branch: exUMI

SIME: Enhancing Policy Self-Improvement with Modal-level Exploration

🔗 Resources

• 📖 Paper: arXiv:2505.01396
• 🌐 Website: SIME Project Page
• 💻 GitHub: EricJin2002/SIME
• 📦 Dataset: robomimic Benchmark Instructions
• 🌿 RHOS Branch: SIME

Open3DHOI: Reconstructing In-the-Wild Open-Vocabulary Human-Object Interactions

🔗 Resources

• 📖 Paper: arXiv:2503.15898
• 🌐 Website: Open3DHOI Project Page
• 💻 GitHub: wenboran2002/open-3dhoi
• 🤗 Dataset: Open3DHOI
• 🌿 RHOS Branch: open-3dhoi

ImDy: Human Inverse Dynamics from Imitated Observations

🔗 Resources

• 🌐 Website: ImDy Project Page
• 💻 GitHub: Foruck/ImDy
• 🤗 Dataset: ImDy
• 📦 Checkpoints: Google Drive
• 🌿 RHOS Branch: ImDy

HDyS: Homogeneous Dynamics Space for Heterogeneous Humans

🔗 Resources

• 💻 GitHub: Foruck/HDyS
• 🌿 RHOS Branch: HDyS

ViFailback: Diagnose, Correct, and Learn from Manipulation Failures via Visual Symbols

🔗 Resources

• 🌐 Website: ViFailback
• 💻 GitHub: x1nyuzhou/ViFailback
• 🌿 RHOS Branch: ViFailback

HAJL: Human-Robot Joint Learning for Efficient Robot Manipulation Skill Acquisition

🔗 Resources

• 🌐 Website: Human-Robot Joint Learning
• 💻 GitHub: RHOS/HAJL branch

LLM-Inception: The Labyrinth of Links: Navigating the Associative Maze of Multi-modal LLMs

🔗 Resources

• 📖 Paper: arXiv:2410.01417
• 🌐 Website: LLM Inception
• 💻 GitHub: lihongcs/LLM_Inception
• 📦 Data: Google Drive
• 🤗 Demo: Hugging Face Space
• 🌿 RHOS Branch: LLM_Inception

Verb Mirage: Unveiling and Assessing Verb Concept Hallucinations in Multimodal Large Language Models

🔗 Resources

• 📖 Paper: arXiv:2412.04939
• 💻 GitHub: RHOS/Verb-Mirage branch

HAKE-AVA

Fine-grained spatio-temporal activity understanding based on AVA videos, as part of the HAKE project.

🔗 Resources

• 📖 Paper: HAKE-GIO
• 🌐 Website: HAKE
• 💻 GitHub: DirtyHarryLYL/HAKE-AVA
• 🤗 Dataset: HAKE-GIO
• 📦 Dataset Guide: HAKE-AVA DATASET.md
• 🌿 RHOS Branch: HAKE-AVA