Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ orphan: true
- Superseded by: None
- Roadmap: [Issue #2520](https://github.com/Motphys/UniLab/issues/2520);
FlashMimic subset extension:
[Issue #2521](https://github.com/Motphys/UniLab/issues/2521)
[Issue #2521](https://github.com/Motphys/UniLab/issues/2521); unified capability sets:
[Issue #2550](https://github.com/Motphys/UniLab/issues/2550)

## Context

Expand All @@ -32,19 +33,19 @@ before this integration; the authoritative adapter ADR and implementation live i

UniLab consumes the unified contract as device-only Genesis runtime DR:

- Genesis declares reset terms only in `supported_device_reset_terms`, and interval terms only
in `supported_device_interval_terms`. The host term sets are empty, including for portable
scenes and sessions without the exact CUDA/AMDGPU lifecycle.
- Manager capability negotiation selects the plane from the backend-bound tensor device. It never
treats default host capability wording as a fallback for a device-resident runtime.
- The packed reset device is declared before cold event-term binds, so owner terms negotiate
against the same plane that will commit after backend materialization.
- Genesis declares reset terms in `supported_reset_terms` and interval terms in
`supported_interval_terms`. These single sets enumerate supported terms without splitting them
into host/device public capability planes. They do not imply that Genesis accepts host payloads.
- Manager capability negotiation is plane-independent. The backend-bound tensor device still
determines operand placement, while the backend owns execution and any adapter-internal
conversion for an advertised term.
- The packed reset device is declared before cold event-term binds, so owner terms construct and
commit operands on the same device after backend materialization.
- Reset DR is limited to the audited device lifecycle (`body_mass`, `base_mass_delta`,
`body_ipos`, `base_com_offset`, `kp`, `kd`); interval DR is limited to `body_force`,
`body_torque`, world-frame root linear velocity delta, and root-local angular velocity delta.
Unsupported, host-plane, mixed-plane, and CPU/host payloads fail closed with the term and plane
named.
- Genesis owner YAML enables `pd_gains` only because that device-plane packed lane declares
Unsupported terms and CPU/host payloads fail closed with explicit backend errors.
- Genesis owner YAML enables `pd_gains` only because that device-resident packed lane declares
`kp`/`kd`. It does not re-enable broad portable host setters, geometry, gravity, DOF, or the
opaque legacy push descriptor. Startup body mass and the six-axis velocity-delta event use
their audited public device lifecycle; absolute `geom_friction` remains intentionally
Expand All @@ -55,21 +56,21 @@ UniLab consumes the unified contract as device-only Genesis runtime DR:

## Stable Contracts

- `DomainRandomizationCapabilities` plane sets are authoritative; aggregate projections keep
`dr.reset.*`/`dr.interval.*` and `dr.device.reset.*`/`dr.device.interval.*` independent.
- The single `DomainRandomizationCapabilities.supported_reset_terms` and
`supported_interval_terms` sets are authoritative; execution topology is not an alternate DR
term set.
- The Manager packed reset transaction declares its device before reset event construction.
- Genesis interval dispatch uses public `IntervalTermOp` values only; backend IDs are Torch int64
on the backend device and wrench payloads are float32 on that same device.
- Genesis owners cannot declare backend capability by comments or backend-name branches; owner
YAML may only enable terms whose plane the current backend declares.
YAML may only enable terms that the current backend declares.

## Alternatives Considered

- **Default to the historical host plane before packed-device declaration:** rejected. It made a
portable host setter look valid during cold binds and could accept a term that the eventual
device commit must reject.
- **Convert host operands inside each backend adapter:** rejected as the Manager contract. It
reintroduces hidden hot-path H2D detours and plane ambiguity at every adapter boundary.
- **Keep separate host/device capability sets:** rejected. It made a portable host setter look
valid during cold binds and leaked adapter execution topology into every Manager consumer.
- **Select capability sets from the Manager payload device:** rejected. Term support is a public
backend capability; payload placement and adapter-internal conversion are execution details.
- **Keep `events.pd_gains: null` until every Genesis reset term is device-capable:** rejected.
`kp`/`kd` have one audited lifecycle; unrelated unsupported terms already fail closed and need
no owner-level blanket disable.
Expand All @@ -82,16 +83,17 @@ UniLab consumes the unified contract as device-only Genesis runtime DR:
reset writes, and six-axis velocity kicks. Its foot-friction event stays disabled.
- Portable Genesis scenes have a smaller reset contract, and removed portable host runtime paths
have no compatibility alias in UniLab.
- Owners must sample device-resident operands on the Manager RNG/backend device. CPU-only Genesis
sessions do not advertise runtime DR.
- Owners must sample device-resident operands on the Manager RNG/backend device. Unified term
sets do not add CPU/host execution support to Genesis.
- Adding a Genesis reset or interval term requires first extending the audited device lifecycle
and capability declaration in `unisim-core`, then updating owners/docs here.

## Evidence In Repo

- `src/unilab/base/entity.py` — interval plane negotiation and device-resident interval ops.
- `src/unilab/base/entity.py` — plane-independent interval capability checks and device-resident
interval ops.
- `src/unilab/envs/manager_based_rl_env.py` — packed reset device declaration before event binds.
- `src/unilab/base/reset_state.py` — plane-aware capability checks.
- `src/unilab/base/reset_state.py` — unified reset-term capability checks.
- `src/unilab/conf/{ppo,sac}/task/g1_walk_flat/genesis.yaml` — walk owner parity.
- `src/unilab/conf/sac/task/g1_motion_tracking/genesis.yaml` — FlashMimic DR subset owner.
- `tests/base/test_genesis_backend.py`; `tests/envs/mdp/test_events.py`;
Expand Down
16 changes: 9 additions & 7 deletions docs/sphinx/source/en/2-user_guide/3-backends/5-genesis.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,11 @@ compensates on the materialize cold path — hot paths never parse XML:
reproduction of it.
- **Actuators**: `<position kp kv>` actuators import losslessly and are
driven with `control_dofs_position`; kp/kd reset randomization is supported
on the device plane (the DR capability sets declare only the measured device
reset terms: body mass, base-mass delta, body CoM / base-CoM delta, kp,
kd; the device interval set additionally covers body force, body torque,
and world-frame root linear plus root-local angular velocity deltas).
through the device-resident lifecycle. The unified DR capability sets declare
only the measured reset terms: body mass, base-mass delta, body CoM /
base-CoM delta, kp, kd; the interval set additionally covers body force,
body torque, and world-frame root linear plus root-local angular velocity
deltas.

## Prerequisites

Expand Down Expand Up @@ -159,9 +160,10 @@ degrading:
construction/plan time. Use the six-axis velocity-delta event, which commits
world-frame linear and root-local angular deltas through Genesis public
state without touching qpos, controls, solver state, or unrelated rows.
- **Host-plane runtime DR**: Genesis accepts reset and interval payloads only
on its backend-bound CUDA/AMDGPU device. Host payloads fail closed with the
term and plane named; there is no NumPy or host-bridge fallback.
- **Host-payload runtime DR**: Genesis accepts reset and interval payloads only
on its backend-bound CUDA/AMDGPU device. Host payloads fail closed with an
explicit backend error; the unified term sets do not imply host execution
support, and there is no NumPy or host-bridge fallback.
- **Repeated `gs.init`** in one process (see above).

## Cross-Backend Migration (sim2sim)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ These three paths correspond to three lifecycle classes:

1. Reset/interval randomization consists of `events:` manager terms in the owner YAML, executed uniformly by the manager lifecycle.
2. Manager-Based owners declare reset behavior through Hydra command/event terms. G1 motion reset perturbations belong to the device-resident `MotionCommandCfg`, while task owners add `EventTermCfg` reset and interval terms.
3. `TensorResetRandomizationPayload` (the torch-first reset payload) expresses curated reset terms; a backend must advertise every requested term on the matching host/device plane and own its derived-quantity obligation.
3. `TensorResetRandomizationPayload` (the torch-first reset payload) expresses curated reset terms; a backend must advertise every requested term in its unified capability set and own its derived-quantity and device-execution obligations.
4. `MotrixBackend` currently supports `base_mass_delta`, `base_com_offset`, `kp`, `kd`, and interval push; and it requires all model actuators to be position actuators during initialization.
5. Fixed mesh/tool identity is declared by `env.fixed_model_variants`; reset-time geometry fields remain behind backend capability declarations and never change that identity.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,23 +33,24 @@ provider protocol and the environment does not carry a DR manager.

Backend differences are explicit capabilities, not task-side branches:

- `DomainRandomizationCapabilities.supported_reset_terms` — host plane
(`torch.cpu` payloads)
- `supported_interval_terms` — host plane
- `supported_device_reset_terms` / `supported_device_interval_terms` — device
plane (CUDA payloads); an empty set fails closed
- `DomainRandomizationCapabilities.supported_reset_terms` — every reset term the
backend supports through the public DR path
- `supported_interval_terms` — every interval term the backend supports through
the public DR path
- fixed-variant layouts
- per-environment playback support
- `TensorLifecycleCapabilities.packed_reset` with
`SimBackend.bind_packed_reset()`: required for the Manager-Based reset and
startup DR lifecycle; a missing declaration fails closed at env
construction.

The payload tensors' device selects the plane: `torch.cpu` payloads negotiate
the host term sets and CUDA payloads the device term sets. An unadvertised
requested term fails closed with the backend, term, and plane named.
Manager code never imports MuJoCo or mjbatch and never accesses a backend model
or pool.
These are single, plane-independent term sets. Manager capability negotiation
does not inspect the payload device or pass a plane. CPU/GPU execution and any
required internal conversion belong to the backend adapter, not to separate
public capability sets. An unadvertised requested term fails closed with the
backend and term named; a backend may also reject payloads that violate its
device or lifecycle contract. Manager code never imports MuJoCo or mjbatch and
never accesses a backend model or pool.

## Reset Payload And Defaults

Expand Down Expand Up @@ -89,10 +90,10 @@ the former UniLab-side MuJoCo recompilation used to obtain inertia defaults.
Interval plans are term-descriptor based: `IntervalRandomizationPlan.ops`
carries `IntervalTermOp` entries from `unisim.dr.interval`. Payloads are
torch-first: `IntervalTermOp.payload` is a torch tensor or a
`TensorIntervalPayload` whose device selects the plane, and push is
producer-sampled on every backend — backend-internal global-RNG push sampling
no longer exists. Builtin payload contracts are enforced by
`IntervalTermOp.validate`; unknown backend-owned custom terms pass through to
`TensorIntervalPayload`; push is producer-sampled on every backend —
backend-internal global-RNG push sampling no longer exists. Builtin payload
contracts are enforced by `IntervalTermOp.validate`; unknown backend-owned
custom terms pass through to
that backend's handler table. Plans that cross spawn collectors carry sampling
ranges rather than tensors; device payloads are process-local.

Expand Down
10 changes: 5 additions & 5 deletions docs/sphinx/source/zh_CN/2-user_guide/3-backends/5-genesis.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,11 @@ materialize 冷路径补偿——热路径从不解析 XML:
contact sensor 变为 per-link net contact force 阈值(1 N)——这是对
geom 对 `found` 语义的近似,不是复现。
- **执行器**:`<position kp kv>` 执行器无损导入,用
`control_dofs_position` 驱动;kp/kd reset 随机化在 device plane 受支持
(DR 能力集只声明实测的 device reset 项:body mass、kp、kd,以及
base-mass delta、body CoM / base-CoM delta、kp、kd;device interval 集
另覆盖 body force、body torque,以及世界系 root 线速度与 root 本体系
角速度 delta)。
`control_dofs_position` 驱动;kp/kd reset 随机化通过 device-resident
生命周期受支持。统一 DR 能力集只声明实测的 reset 项:body mass、
base-mass delta、body CoM / base-CoM delta、kp、kd;interval 集另覆盖
body force、body torque,以及世界系 root 线速度与 root 本体系角速度
delta。

## 前置条件

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,21 +29,20 @@ Manager-Based event term 是 UniLab 唯一的 DR lifecycle。任务 provider 协

Backend 差异是显式 capability,不是 task-side 分支:

- `DomainRandomizationCapabilities.supported_reset_terms`——host 平面
(`torch.cpu` payload)
- `supported_interval_terms`——host 平面
- `supported_device_reset_terms` / `supported_device_interval_terms`——device
平面(CUDA payload);空集合 fail closed
- `DomainRandomizationCapabilities.supported_reset_terms`——backend 通过公开 DR
路径支持的全部 reset term
- `supported_interval_terms`——backend 通过公开 DR 路径支持的全部 interval term
- fixed-variant layout
- per-environment playback 支持
- `TensorLifecycleCapabilities.packed_reset` 与 `SimBackend.bind_packed_reset()`:
Manager-Based reset 与 startup DR 生命周期的必需 capability;缺失声明会在 env
构造时 fail closed。

payload tensor 的设备决定平面:`torch.cpu` payload 走 host term 集合协商,CUDA
payload 走 device term 集合。未在对应平面声明的请求 term 会携带 backend、term
与平面名称 fail closed。Manager code 不 import MuJoCo 或 mjbatch,也不访问
backend model/pool。
这两个集合是单一且与 plane 无关的 term 集合。Manager capability 协商不检查
payload 所在设备,也不传递 plane。CPU/GPU 执行及所需的内部转换由 backend
adapter 负责,不再拆成不同的公开 capability 集合。未声明的请求 term 会携带
backend 与 term 名称 fail closed;backend 仍会拒绝违反其设备或生命周期契约的
payload。Manager code 不 import MuJoCo 或 mjbatch,也不访问 backend model/pool。

## Reset Payload 与默认值

Expand Down Expand Up @@ -76,8 +75,8 @@ closed。该边界移除了 UniLab 侧为 inertia 默认值重新 compile MuJoCo

Interval plan 基于 term descriptor:`IntervalRandomizationPlan.ops` 携带来自
`unisim.dr.interval` 的 `IntervalTermOp`。payload 为 torch-first:
`IntervalTermOp.payload` 是 torch tensor 或 `TensorIntervalPayload`,其设备决定
平面;push 在所有后端统一为生产方采样——后端内部的全局 RNG push 采样已不存在。
`IntervalTermOp.payload` 是 torch tensor 或 `TensorIntervalPayload`;push 在所有
后端统一为生产方采样——后端内部的全局 RNG push 采样已不存在。
内置 payload contract 由 `IntervalTermOp.validate` 强制;未知 backend-owned
custom term 传给该 backend handler table。跨 spawn collector 的 plan 携带采样
范围而不是 tensor;device payload 是进程本地的。
Expand Down
4 changes: 2 additions & 2 deletions pyproject.rocm.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ classifiers = [
requires-python = ">=3.10,<3.14"
dependencies = [
"numpy",
"unisim-core==2.2.0",
"unisim-core==2.2.3",
"prettytable>=3.10",
"torch==2.14.0",
"triton-rocm==3.8.0 ; sys_platform == 'linux' and platform_machine == 'x86_64'",
Expand Down Expand Up @@ -68,7 +68,7 @@ mujoco = [
"mujoco~=3.11.0",
"mjbatch-uni~=0.2.5",
]
motrix = ["unisim-core[motrix]==2.2.0"]
motrix = ["unisim-core[motrix]==2.2.3"]
genesis = ["genesis-world==1.3.3"]
# Keep the optional tensor-ray extra aligned with pyproject.toml.
ray = ["uniray[warp]==0.1.0"]
Expand Down
6 changes: 3 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ classifiers = [
requires-python = ">=3.10,<3.14"
dependencies = [
"numpy",
"unisim-core==2.2.0",
"unisim-core==2.2.3",
"prettytable>=3.10",
# A range (not an exact pin) lets ROCm users substitute a ROCm torch
# build; uv.lock pins the tested CUDA builds via tool.uv.sources.
Expand Down Expand Up @@ -102,12 +102,12 @@ newton = [
"pyglet>=2.1.6,<3",
"imgui-bundle>=1.92.0",
]
motrix = ["unisim-core[motrix]==2.2.0"]
motrix = ["unisim-core[motrix]==2.2.3"]
genesis = ["genesis-world==1.3.3"]
# Wheels are CPython 3.12/3.13 Linux x86_64 only; the marker keeps uv lock
# resolvable for the other required-environments.
superdex = [
"unisim-core[superdex]==2.2.0 ; python_version >= '3.12' and sys_platform == 'linux' and platform_machine == 'x86_64'",
"unisim-core[superdex]==2.2.3 ; python_version >= '3.12' and sys_platform == 'linux' and platform_machine == 'x86_64'",
]
# Tensor ray perception is optional and is discovered through UniSim's public
# caster factory. The PyPI distribution is `uniray`; the import namespace is
Expand Down
17 changes: 3 additions & 14 deletions src/unilab/base/entity.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@
RESET_TERM_GEOM_FRICTION,
RESET_TERM_KD,
RESET_TERM_KP,
DataPlaneName,
IntervalRandomizationPlan,
IntervalTermOp,
)
Expand Down Expand Up @@ -3807,9 +3806,7 @@ def _bind_root_velocity_delta(self, *, angular: bool, term_name: str) -> None:
if angular
else INTERVAL_TERM_BODY_LINEAR_VELOCITY_DELTA
)
supported = capabilities.supports_interval_term(
term, plane=self._interval_capability_plane()
)
supported = capabilities.supports_interval_term(term)
if not supported:
raise self._capability_error(
"interval root velocity delta",
Expand Down Expand Up @@ -3965,15 +3962,12 @@ def bind_body_wrench(
capabilities = self._backend.get_dr_capabilities()
except (AttributeError, NotImplementedError) as exc:
raise self._capability_error("interval body wrench", str(exc)) from exc
plane = self._interval_capability_plane()
if not capabilities.supports_interval_term(INTERVAL_TERM_BODY_FORCE, plane=plane):
if not capabilities.supports_interval_term(INTERVAL_TERM_BODY_FORCE):
raise self._capability_error(
"interval body wrench",
f"EventManager term '{term_name}' requested an unsupported backend capability",
)
if torque and not capabilities.supports_interval_term(
INTERVAL_TERM_BODY_TORQUE, plane=plane
):
if torque and not capabilities.supports_interval_term(INTERVAL_TERM_BODY_TORQUE):
raise self._capability_error(
"interval body wrench",
f"EventManager term '{term_name}' requested an unsupported backend "
Expand All @@ -3993,11 +3987,6 @@ def _interval_operand_device(self) -> torch.device:
self._interval_operand_device_value = device
return device

def _interval_capability_plane(self) -> DataPlaneName:
"""Return the capability plane used by this entity's interval payloads."""

return "device" if self._interval_operand_device().type != "cpu" else "host"

def apply_body_wrench_to_sim(
self,
forces: np.ndarray | torch.Tensor,
Expand Down
Loading
Loading