Behavior

README.md

@@ -40,6 +40,7 @@ RPent is built for four kinds of users:
 
 - [2026/08] 🔥 RPent supports RoboCasa with RLDX-1 as manipulation model. See the [RoboCasa setup and Target50 guide](robots/robocasa/README.md) and [full documentation](https://rpent.readthedocs.io/en/latest/rst_source/usage/robocasa.html).
 - [2026/08] 🔥 RPent supports the non-reasoning mode, which reduces average execution time by ~40%.
+- [2026/08] 🔥 RPent supports BEHAVIOR with Pi0.5 for long-horizon household tasks. Doc: [BEHAVIOR](https://rpent.readthedocs.io/en/latest/rst_source/usage/behavior.html).
 - [2026/08] 🔥 RPent supports exploration mode for LIBERO. Doc: [LIBERO exploration mode](https://rpent.readthedocs.io/en/latest/rst_source/usage/libero.html#exploration-and-local-memory-evaluation).
 - [2026/08] 🔥 RPent supports RoboTwin with LingBot-VLA for dual-arm manipulation tasks. Doc: [RoboTwin](https://rpent.readthedocs.io/en/latest/rst_source/usage/robotwin.html).
 - [2026/07] 🔥 Our first RPent publication, [Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents](https://arxiv.org/abs/2607.08448), is released.
@@ -81,6 +82,7 @@ RPent is built for four kinds of users:
       <td style="text-align: left; padding-left: 8px;">
         <ul style="margin-left: 0; padding-left: 16px;">
           <li><a href="https://rpent.readthedocs.io/en/latest/rst_source/usage/libero.html">LIBERO-PRO</a> ✅</li>
+          <li><a href="https://rpent.readthedocs.io/en/latest/rst_source/usage/behavior.html">BEHAVIOR-1K</a> ✅</li>
           <li><a href="https://rpent.readthedocs.io/en/latest/rst_source/usage/robocasa.html">RoboCasa</a> ✅</li>
           <li><a href="https://rpent.readthedocs.io/en/latest/rst_source/usage/robotwin.html">RoboTwin</a> ✅</li>
         </ul>

README.zh-CN.md

@@ -40,6 +40,7 @@ RPent 面向以下四类用户：
 
 - [2026/08] 🔥 支持 RoboCasa，使用 RLDX-1 作为操作模型。参见 [RoboCasa 安装与 Target50 指南](robots/robocasa/README.md)和 [完整中文文档](https://rpent.readthedocs.io/zh-cn/latest/rst_source/usage/robocasa.html)。
 - [2026/08] 🔥 新增非推理（non-reasoning）模式，平均执行时间降低约 40%。
+- [2026/08] 🔥 支持 BEHAVIOR，使用 Pi0.5 处理家庭长程任务。文档：[BEHAVIOR](https://rpent.readthedocs.io/zh-cn/latest/rst_source/usage/behavior.html)。
 - [2026/08] 🔥 支持 LIBERO 探索模式。文档：[LIBERO 探索模式](https://rpent.readthedocs.io/zh-cn/latest/rst_source/usage/libero.html#memory)。
 - [2026/08] 🔥 支持 RoboTwin，使用 LingBot-VLA 处理双臂操作任务。文档：[RoboTwin](https://rpent.readthedocs.io/zh-cn/latest/rst_source/usage/robotwin.html)。
 - [2026/07] 🔥 RPent 首篇论文 [Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents](https://arxiv.org/abs/2607.08448) 发布。
@@ -81,6 +82,7 @@ RPent 面向以下四类用户：
       <td style="text-align: left; padding-left: 8px;">
         <ul style="margin-left: 0; padding-left: 16px;">
           <li><a href="https://rpent.readthedocs.io/zh-cn/latest/rst_source/usage/libero.html">LIBERO-PRO</a> ✅</li>
+          <li><a href="https://rpent.readthedocs.io/zh-cn/latest/rst_source/usage/behavior.html">BEHAVIOR-1K</a> ✅</li>
           <li><a href="https://rpent.readthedocs.io/zh-cn/latest/rst_source/usage/robocasa.html">RoboCasa</a> ✅</li>
           <li><a href="https://rpent.readthedocs.io/zh-cn/latest/rst_source/usage/robotwin.html">RoboTwin</a> ✅</li>
         </ul>

docs/source-en/index.rst

@@ -54,8 +54,8 @@ Welcome to RPent
       :link-type: doc
       :text-align: center
 
-      Drive the LIBERO / RoboCasa simulators or a Franka / SO-101 arm,
-      switch planners, and pick action primitives.
+      Drive the LIBERO / BEHAVIOR / RoboCasa simulators or a Franka /
+      SO-101 arm, switch planners, and pick action primitives.
 
    .. grid-item-card:: Development Tutorial
       :link: rst_source/development/architecture
@@ -87,6 +87,7 @@ Welcome to RPent
    LIBERO <rst_source/usage/libero>
    RoboCasa <rst_source/usage/robocasa>
    RoboTwin <rst_source/usage/robotwin>
+   BEHAVIOR <rst_source/usage/behavior>
    Franka <rst_source/usage/franka>
    SO-101 <rst_source/usage/so101>
    Advanced Deployment <rst_source/usage/advanced_deployment>

docs/source-en/rst_source/development/add_robot.rst

@@ -131,9 +131,9 @@ template, and output slug; ``runtime_components`` and ``frame_channels``
 describe the robot-specific rows and camera views rendered by the
 frontend. See ``robots/libero/robot_spec.py`` for the reference shape.
 
-That's the entire registration step — ``_resolve_robot(name)`` does an
-``importlib.import_module(f"robots.{name}")``, so dropping the package under
-``robots/`` on disk is enough. No central list to update.
+The standard source tree currently includes ``libero``, ``robocasa``,
+``robotwin``, and ``behavior`` robot packages. New robot packages should follow
+the same entry-point contract before being wired into a release.
 
 The sections below describe what each referenced module must contain.
 ``_add_cli_args`` / ``_parse_config`` are covered in §4 and the runtime hook

docs/source-en/rst_source/development/architecture.rst

@@ -158,10 +158,10 @@ Dashboard description, and three runner hooks (``add_cli_args`` /
 ``parse_config`` / ``init_runtime``). See :doc:`interfaces` for what each
 field must provide.
 
-The loader itself does not maintain a list of robot names. The
-current CLI restricts ``--robot`` to ``libero`` and ``robocasa``; adding a
-new name therefore also requires updating the CLI choices. See
-:doc:`add_robot` for the complete procedure.
+The standard source tree currently includes ``libero``, ``robocasa``,
+``robotwin``, and ``behavior`` robot packages. New integrations should follow
+the same ``RobotSpec`` / ``get_toolkit`` package shape described in
+:doc:`add_robot`.
 
 Planner, Toolkit, and RPC transports
 -------------------------------------

docs/source-en/rst_source/installation.rst

@@ -35,6 +35,7 @@ Other environment configurations are available when needed:
 
    pip install -e ".[robocasa]"    # RoboCasa
    pip install -e ".[robotwin]"    # RoboTwin
+   pip install -e ".[behavior]"    # BEHAVIOR
 
 ``.[libero-pro]`` is the recommended default.
 
@@ -51,6 +52,10 @@ Available extras:
      - LIBERO-PRO + openpi Pi0.5 VLA + SAM 3.0 + RLinf runtime
    * - ``.[libero-plus]``
      - LIBERO-plus + openpi Pi0.5 VLA + SAM 3.0 + RLinf runtime
+   * - ``.[behavior]``
+     - RPent-side BEHAVIOR dependencies; the full OmniGibson/Isaac Sim stack
+       uses the source-editable dual-venv workflow and licensed assets documented
+       in :doc:`usage/behavior`
    * - ``.[robocasa]``
      - RoboCasa365 simulator + the RLDX-1 VLA; see :doc:`usage/robocasa`
    * - ``.[robotwin]``

docs/source-en/rst_source/usage/behavior.rst

@@ -0,0 +1,296 @@
+BEHAVIOR
+========
+
+`BEHAVIOR-1K <https://behavior.stanford.edu/>`_ provides long-horizon household
+tasks in OmniGibson. RPent exposes ``turning_on_radio`` and
+``picking_up_trash`` as a standard sibling robot plugin under
+``robots/behavior``.
+
+The integration follows the same lightweight contract as LIBERO, RoboCasa, and
+RoboTwin: ``get_robot_spec()`` supplies CLI/config/runtime hooks and
+``get_toolkit()`` supplies the public tools. BEHAVIOR-specific lifecycle code
+stays inside ``robots/behavior``. The common ``--explore`` entry point supports
+BEHAVIOR while preserving its one-attempt-per-session environment lifecycle.
+
+Installation status
+-------------------
+
+BEHAVIOR is source-editable and uses two independent Python 3.10 environments:
+
+- the **RPent venv** runs the CLI, planner, Dashboard, and MemoryManager;
+- the **BEHAVIOR venv** runs RLinf, OmniGibson, Isaac Sim, and Pi0.5.
+
+The ``.[behavior]`` extra installs only RPent-side dependencies. It does not
+install the complete simulator, assets, or checkpoint, and a normal wheel does
+not promise a directly runnable BEHAVIOR stack. From a source checkout, run:
+
+.. code-block:: bash
+
+   python -m pip install -e ".[behavior]"
+   export RPENT_REPRO_ROOT="$PWD/.behavior-runtime"
+   export UV_CACHE_DIR="$RPENT_REPRO_ROOT/uv-cache"
+   behavior-install-runtime
+
+The installer keeps RPent editable in both venvs, clones the reviewed RLinf
+revision, invokes the official RLinf BEHAVIOR installer, applies the reviewed
+CUDA/OpenPI compatibility pins, verifies critical imports and CUDA, and writes
+freezes plus source identities under ``$RPENT_REPRO_ROOT/manifests``. Use a new
+``RPENT_REPRO_ROOT`` for a fresh install; the script refuses to overwrite a
+wrong or dirty RLinf checkout.
+
+Motion planning uses NVlabs/cuRobo v0.8.0 at commit
+``4ea77366ca48ee453e7df139e39fa6532af49f3b`` in the BEHAVIOR venv.
+The installer applies constraints before its final repin, retaining NumPy
+1.26.4, Torch 2.5.1+cu124 and Isaac Sim 4.5.0.0. Do not install cuRobo with
+an unconstrained resolver: a NumPy 2 upgrade is incompatible with this stack.
+
+Simulator assets
+----------------
+
+Accept the BEHAVIOR/OmniGibson licences, choose a dedicated data root, and use
+the standard asset command. It invokes the three official OmniGibson download
+functions in the BEHAVIOR venv rather than importing OmniGibson into the RPent
+environment:
+
+.. code-block:: bash
+
+   export OMNIGIBSON_DATA_PATH=/path/to/BEHAVIOR-1K-datasets
+   export BEHAVIOR_PYTHON="$RPENT_REPRO_ROOT/venvs/behavior/bin/python"
+   behavior-download-assets --accept-license --skip-existing
+
+Omit ``--accept-license`` to let the official downloader display its
+interactive licence prompt. The flag is an explicit non-interactive
+confirmation; do not use it unless you accept the licence terms.
+
+The final data root must contain:
+
+.. code-block:: text
+
+   BEHAVIOR-1K-datasets/
+     2025-challenge-task-instances/
+     behavior-1k-assets/
+       scenes/
+     omnigibson-robot-assets/
+     omnigibson.key
+
+Pi0.5 checkpoint
+----------------
+
+Download the reviewed checkpoint into a directory outside the source tree:
+
+.. code-block:: bash
+
+   export PI05_CHECKPOINT_PATH=/path/to/RLinf-Pi05-BEHAVIOR-1K-PT50-CS32
+   "$RPENT_REPRO_ROOT/venvs/rpent/bin/hf" download \
+     RLinf/RLinf-Pi05-BEHAVIOR-1K-PT50-CS32 \
+     --local-dir "$PI05_CHECKPOINT_PATH"
+
+``behavior-download-assets --verify`` verifies the required OmniGibson layout
+and the source-controlled checkpoint size/SHA binding. The shared #136 Pi0.5
+component receives head, left-wrist, right-wrist, and raw R1Pro proprio data.
+The raw RPC result is ``[1, 32, 23]``; the common client returns ``[32, 23]``.
+
+.. code-block:: bash
+
+   behavior-download-assets --verify
+
+DINOv2 configuration
+--------------------
+
+BEHAVIOR keeps a reviewed `DINOv2 <https://github.com/facebookresearch/dinov2>`_
+ViT-S/14 deployment for whole-image embeddings and episode-memory retrieval.
+Provide a DINOv2 source archive and the ``dinov2_vits14_pretrain.pth`` weights:
+
+.. code-block:: bash
+
+   export DINOV2_SOURCE_ARCHIVE=/path/to/dinov2-source.tar.gz
+   export DINOV2_WEIGHTS=/path/to/dinov2_vits14_pretrain.pth
+
+   curl -L \
+     https://github.com/facebookresearch/dinov2/archive/7764ea0f912e53c92e82eb78a2a1631e92725fc8.tar.gz \
+     -o "$DINOV2_SOURCE_ARCHIVE"
+   curl -L \
+     https://dl.fbaipublicfiles.com/dinov2/dinov2_vits14/dinov2_vits14_pretrain.pth \
+     -o "$DINOV2_WEIGHTS"
+
+DINOv2 is the shared visual-memory component. It is not a segmentation model,
+does not replace SAM3 masks, and does not replace current public observations
+or MemoryManager Markdown/YAML material. The accepted DINOv2 source revision
+and both asset SHA-256 identities are pinned in
+``robots/behavior/dino_v2/encoder.py``; the runtime rejects mismatched assets.
+
+The DINO episode-memory catalog is a rebuildable derivative of official
+BEHAVIOR demonstration data. Build it with the standard command below; do not
+edit the catalog by hand.
+
+.. code-block:: bash
+
+   behavior-build-memory \
+     --selection-manifest /path/to/selection_manifest.json \
+     --video-root /path/to/2025-challenge-demos \
+     --rollups-dir /path/to/episode_rollups \
+     --source-archive "$DINOV2_SOURCE_ARCHIVE" \
+     --weights "$DINOV2_WEIGHTS" \
+     --cuda-device 2 \
+     --output-dir /path/to/behavior-dino-catalog
+
+Task identity
+-------------
+
+Use ``--task-name`` and ``--public-seed``. Public seeds map to fixed official
+activity instances through ``robots/behavior/task_specs.py``.
+
+.. list-table::
+   :header-rows: 1
+   :widths: 24 42 16 18
+
+   * - Task
+     - Instruction
+     - Explore seeds
+     - Eval seeds
+   * - ``turning_on_radio``
+     - Turn on the radio receiver on the living-room table.
+     - ``0``
+     - ``1``-``9``
+   * - ``picking_up_trash``
+     - Put the three living-room soda cans into the kitchen trash can.
+     - ``0``-``9``
+     - ``10``-``19``
+
+One evaluation run
+------------------
+
+Bind each CUDA child to one physical GPU explicitly:
+
+.. code-block:: bash
+
+   "$RPENT_REPRO_ROOT/venvs/rpent/bin/rpent" --robot behavior \
+     --task-name turning_on_radio --public-seed 1 \
+     --behavior-mode eval \
+     --planner codex --model gpt-5.5 \
+     --behavior-repo "$RPENT_REPRO_ROOT/RLinf" \
+     --behavior-python "$RPENT_REPRO_ROOT/venvs/behavior/bin/python" \
+     --activity-instance-dir \
+       "$OMNIGIBSON_DATA_PATH/2025-challenge-task-instances" \
+     --policy-checkpoint "$PI05_CHECKPOINT_PATH" \
+     --behavior-env-cuda-device 0 \
+     --behavior-model-cuda-device 1 \
+     --dino-source-archive "$DINOV2_SOURCE_ARCHIVE" \
+     --dino-weights "$DINOV2_WEIGHTS" \
+     --memory-profile local \
+     --memory-dir /path/to/behavior-memory \
+     --behavior-memory-dir /path/to/reviewed-behavior-episode-memory
+
+The first environment load can take several minutes. The environment, VLA, and
+DINO are separate processes, and each receives only its explicitly selected GPU.
+
+Official MemoryManager
+----------------------
+
+BEHAVIOR uses the same Markdown/YAML ``MemoryManager`` format and common memory
+tools as the other robots. The DINO episode-memory catalog is a separate visual
+experience retrieval source; when configured, its advisory is attached to
+public tool receipts and remains historical guidance only.
+
+- Eval creates one ``MemoryManager`` with ``read_only`` access.
+- Explore creates one ``MemoryManager`` with ``inbox_write`` access scoped to
+  ``<memory-dir>/_internal/inbox/<recipe-tag>``.
+- ``MEMORY.md``, ``global/``, ``suite/``, and ``task_only/`` hold the published
+  corpus. Solved audit/recipe pairs are copied to ``task_only/``.
+- When merge processes a valid root-level draft, the cell inbox is archived to
+  ``_internal/merged/<recipe-tag>``. An inbox containing only invalid drafts
+  stays in place; conflicting prose is archived under ``_internal/conflicts/``.
+
+An absent or empty corpus is valid, but it contains no advice. Pass the same
+explicit ``--memory-dir`` to runs that should share reviewed memory.
+
+Use ``--behavior-memory-dir`` only for the reviewed DINO episode-memory catalog.
+Omitting it selects a legal empty episode catalog and does not download or
+silently substitute task-specific memory.
+
+Use the standard RPent Explore entry point for a bounded sequence of sessions:
+
+.. code-block:: bash
+
+   "$RPENT_REPRO_ROOT/venvs/rpent/bin/rpent" --robot behavior \
+     --behavior-mode explore \
+     --explore \
+     --explore-sessions 3 \
+     --task-name picking_up_trash --public-seed 0 \
+     --output-dir /path/to/behavior-explore \
+     --memory-dir /path/to/behavior-memory \
+     --planner codex --model gpt-5.5 \
+     --behavior-repo "$RPENT_REPRO_ROOT/RLinf" \
+     --behavior-python "$RPENT_REPRO_ROOT/venvs/behavior/bin/python" \
+     --activity-instance-dir \
+       "$OMNIGIBSON_DATA_PATH/2025-challenge-task-instances" \
+     --policy-checkpoint "$PI05_CHECKPOINT_PATH" \
+     --behavior-env-cuda-device 0 \
+     --behavior-model-cuda-device 1 \
+     --dino-source-archive "$DINOV2_SOURCE_ARCHIVE" \
+     --dino-weights "$DINOV2_WEIGHTS"
+
+For BEHAVIOR, one session is exactly one attempt. Each session starts a fresh
+environment sidecar, episode, and ``sessions/session_NNN`` output directory;
+the VLA and DINO sidecars stay shared across sessions. The planner cannot reset
+inside an invocation, and ``--explore-attempts-per-session`` values above zero
+are rejected.
+
+Runtime and Dashboard
+---------------------
+
+The runtime has four component roles:
+
+- ``env``: the task-scoped official BEHAVIOR/OmniGibson environment;
+- ``vla``: the shared ``rpent/robots/components/pi05_vla_server.py`` service;
+- ``dino``: the shared ``robots/behavior/dino_v2/server.py`` episode-memory
+  embedding service;
+- ``memory``: the task-scoped official MemoryManager.
+
+Start a Dashboard Session with:
+
+.. code-block:: bash
+
+   export RPENT_BEHAVIOR_PYTHON="$RPENT_REPRO_ROOT/venvs/behavior/bin/python"
+   "$RPENT_REPRO_ROOT/venvs/rpent/bin/rpent" \
+     --robot behavior --dashboard \
+     --task-name turning_on_radio --public-seed 1 \
+     --behavior-mode eval \
+     --behavior-repo "$RPENT_REPRO_ROOT/RLinf" \
+     --behavior-python "$RPENT_BEHAVIOR_PYTHON" \
+     --activity-instance-dir \
+       "$OMNIGIBSON_DATA_PATH/2025-challenge-task-instances" \
+     --policy-checkpoint "$PI05_CHECKPOINT_PATH" \
+     --dino-source-archive "$DINOV2_SOURCE_ARCHIVE" \
+     --dino-weights "$DINOV2_WEIGHTS" \
+     --memory-profile local --memory-dir /path/to/behavior-memory \
+     --output-dir /path/to/behavior-dashboard-run
+
+The Dashboard uses the common Start Session flow and head/left-wrist/right-
+wrist camera views. BEHAVIOR does not add robot-local manual buttons, a manual
+control backend, or ``env.dashboard_*`` RPC methods. The public contract
+registers nine executable planner primitives: ``pi0_nav_pick``, ``observe``,
+``pixel_to_world``, ``navigate_to``, ``move_to``, ``rotate_wrist``, ``close``,
+``open``, and ``press``. ``move_to(hand=both)`` coordinates both arms through
+cuRobo collision-checked trajectories; wrist rotation uses the same planner.
+Navigation executes a bounded straight base segment or rotation and rejects
+obstructed paths. RGB-D projections use the current physical camera frame;
+R1Pro has no movable head camera, so non-center ``head_view`` presets are rejected.
+``press`` advances an already aligned hand
+at most 2 cm for at most 10 seconds, stopping on external contact or episode end.
+Contact is not verified button contact; visual hand checks remain unverified.
+Planning, collision, tracking and duration failures are reported explicitly.
+Only raw ``info["done"]["success"]`` establishes task success. Motion primitives
+return their final observation for the next policy call and streaming video;
+VLA chunks additionally record each returned environment frame.
+
+The main logs are:
+
+.. code-block:: text
+
+   <output-dir>/run.log
+   <output-dir>/behavior_vla_server.log
+   <output-dir>/behavior_dino_server.log
+   <output-dir>/tasks/<task-run>/behavior_env_server.log
+   <output-dir>/tasks/<task-run>/episode.mp4
+   <output-dir>/tasks/<task-run>/terminal_receipt.json