feat: Newton 后端原生渲染器（离线 record + 交互式 interactive）（roadmap #1513）

docs/sphinx/source/en/1-getting_started/2-installation.md

@@ -170,10 +170,12 @@ uv sync --extra mujoco --extra motrix --extra mjwarp --extra genesis
 
 The `mujoco`, `mjwarp`, and `newton` extras share one MuJoCo 3.11 /
 MuJoCo-Warp 3.11 / Warp 1.16 line and are jointly installable in a single
-environment:
+environment; Newton's native ViewerGL rendering (offline record +
+interactive) additionally needs the `newton-render` extra:
 
 ```bash
 uv sync --extra mujoco --extra mjwarp --extra newton
+uv sync --extra newton --extra newton-render  # when native rendering is needed
 ```
 
 | Backend | Install path | Important prerequisites |

docs/sphinx/source/en/2-user_guide/3-backends/7-newton.md

@@ -16,8 +16,10 @@ registered for the `newton` backend), and the cross-backend contract audit
 algo trees (verdict TRANSFERABLE). Support levels: SAC is **Tested** — a
 full 5000-iteration training completed on 2026-09-06 on an RTX 4090 /
 torch 2.8.0+cu128 / newton 1.5.1 / mujoco-warp 3.11 (reward/mean 6.68 →
-242.3, episode length → 983, ~43k steps/s, 4m25s wall), with record
-playback validated on `model_5000.pt`; PPO remains **Configured**
+242.3, episode length → 983, ~43k steps/s, 4m25s wall), with playback
+validated on `model_5000.pt`: native ViewerGL offscreen record (800-frame
+1280x720 mp4) and an interactive ViewerGL window smoke on a live X
+display; PPO remains **Configured**
 (evidence limited to the owner configs, compose/contract checks, and
 fail-closed runtime/import boundaries; no training or playback claim yet).
 
@@ -45,8 +47,12 @@ MuJoCo-Warp 3.11 / Warp 1.16 line:
 # In a source checkout:
 uv sync --extra newton
 
+# Native ViewerGL rendering (offline record + interactive) additionally:
+uv sync --extra newton --extra newton-render
+
 # From PyPI:
 pip install "unilab[newton]"
+pip install "unilab[newton,newton-render]"
 ```
 
 The extra pins `newton==1.5.1`, `mujoco-warp==3.11.0`, `mujoco==3.11.0`, and
@@ -101,9 +107,18 @@ as `env.*` fields in the owner YAML:
 
 ## Playback and Rendering
 
-The Newton backend has no native UniLab viewer yet: the owner sets
-`training.play_render_mode: record`, so playback takes the record/headless
-path, and requesting a native interactive renderer fails closed.
+The Newton backend renders natively through the upstream
+`newton.viewer.ViewerGL` (unisim `newton-render` extra, `pyglet>=2.1.6,<3`
++ `imgui-bundle>=1.92.0`). The owner inherits the base config's
+`training.play_render_mode: auto`: with a display, `auto` resolves to
+`interactive` (the ViewerGL window); without one it resolves to `record`
+(`ViewerGL(headless=True)` offscreen rendering to mp4). Without the render
+dependencies, `record` falls back to the MuJoCo offline snapshot renderer
+(the plan diagnostic reports `mujoco-snapshot` vs the native
+`newton-viewer-gl`), while `interactive` fails closed when dependencies or
+a display are missing. Headless offscreen rendering still needs an OpenGL
+context: set `PYOPENGL_PLATFORM=egl` on display-less Linux hosts and
+`PYOPENGL_PLATFORM=glx` under Wayland.
 
 ```bash
 uv run eval --algo ppo --task g1_walk_flat --sim newton \
@@ -121,7 +136,8 @@ than silently degrading:
 - **Runtime PD-gain randomization**: Newton currently rejects it, and the
   owner keeps `events.pd_gains: null` to stay fail-closed until the adapter
   adds the capability.
-- **Native interactive rendering**: see Playback above.
+- **Camera kwargs on the native render path**: the native ViewerGL path
+  ignores `camera_kwargs` (the MuJoCo snapshot path still honors them).
 
 ## Cross-Backend Migration (sim2sim)
 

docs/sphinx/source/zh_CN/1-getting_started/2-installation.md

@@ -151,10 +151,12 @@ uv sync --extra mujoco --extra motrix --extra mjwarp --extra genesis
 ```
 
 `mujoco`、`mjwarp` 和 `newton` 三个 extra 共享同一条 MuJoCo 3.11 /
-MuJoCo-Warp 3.11 / Warp 1.16 版本线，可以组合进同一个环境：
+MuJoCo-Warp 3.11 / Warp 1.16 版本线，可以组合进同一个环境；Newton 的
+原生 ViewerGL 渲染（离线 record + 交互式）另需 `newton-render` extra：
 
 ```bash
 uv sync --extra mujoco --extra mjwarp --extra newton
+uv sync --extra newton --extra newton-render  # 需要原生渲染时
 ```
 
 | 后端 | 安装路径 | 重要前置条件 |

docs/sphinx/source/zh_CN/2-user_guide/3-backends/7-newton.md

@@ -16,7 +16,9 @@ mujoco↔newton（结论 TRANSFERABLE）。支持等级：SAC 为 **Tested**—
 2026-09-06 在 RTX 4090 / torch 2.8.0+cu128 / newton 1.5.1 / mujoco-warp
 3.11 上完成完整 5000-iteration 训练（reward/mean 6.68 → 242.3，
 episode length → 983，约 43k steps/s，wall 4m25s），并对
-`model_5000.pt` 做了 record 回放验证；PPO 保持 **Configured**（现有
+`model_5000.pt` 完成回放验证：原生 ViewerGL 离屏 record（800 帧
+1280x720 mp4）与交互式 ViewerGL 窗口冒烟（真实 X display）；PPO 保持
+**Configured**（现有
 证据限于 owner 配置、compose/contract 检查和 fail-closed 的
 runtime/import 边界，尚无训练或回放验证结论）。
 
@@ -42,8 +44,12 @@ MuJoCo-Warp 3.11 / Warp 1.16 系列：
 # 源码 checkout：
 uv sync --extra newton
 
+# 原生 ViewerGL 渲染（离线 record + 交互式 interactive）另需：
+uv sync --extra newton --extra newton-render
+
 # 从 PyPI 安装：
 pip install "unilab[newton]"
+pip install "unilab[newton,newton-render]"
 ```
 
 该 extra 精确钉定 `newton==1.5.1`、`mujoco-warp==3.11.0`、
@@ -95,9 +101,16 @@ Newton/MuJoCo-Warp 3.11 要求显式的设备与存储容量，对应 owner YAML
 
 ## Playback 与渲染
 
-Newton 后端尚无原生 UniLab viewer：owner 设置
-`training.play_render_mode: record`，回放走录制 / 无头路径；请求原生
-交互式渲染器会 fail-closed。
+Newton 后端通过上游 `newton.viewer.ViewerGL` 提供**原生渲染**（unisim
+`newton-render` extra，`pyglet>=2.1.6,<3` + `imgui-bundle>=1.92.0`）。
+owner 继承 base 配置的 `training.play_render_mode: auto`：有显示时
+`auto` 解析为 `interactive`（ViewerGL 交互窗口），无显示时解析为
+`record`（`ViewerGL(headless=True)` 离屏渲染写 mp4）。未安装渲染依赖
+时 `record` 回退到 MuJoCo 离线快照渲染（plan 诊断中的 renderer 为
+`mujoco-snapshot`，原生路径为 `newton-viewer-gl`）；`interactive` 在
+缺依赖或无显示时 fail-closed。headless 离屏渲染仍需 OpenGL context：
+无显示服务器的 Linux 主机设 `PYOPENGL_PLATFORM=egl`，Wayland 设
+`PYOPENGL_PLATFORM=glx`。
 
 ```bash
 uv run eval --algo ppo --task g1_walk_flat --sim newton \
@@ -113,7 +126,8 @@ uv run eval --algo sac --task g1_walk_flat --sim newton \
 
 - **PD gain 运行时随机化**：Newton 当前拒绝该能力，owner 显式设置
   `events.pd_gains: null` 保持 fail-closed，直到适配器补齐。
-- **原生交互式渲染**：见上文 Playback 一节。
+- **原生渲染路径的相机参数**：native ViewerGL 路径忽略
+  `camera_kwargs`（MuJoCo 快照路径仍生效）。
 
 ## 跨后端迁移（sim2sim）
 

docs/sphinx/source/zh_CN/5-reference/5-support_matrix.md

@@ -134,5 +134,5 @@ uv run scripts/generate_support_matrix.py --write
 - Validated isaacgym entrypoints are explicitly recorded in `_MAINTAINER_VALIDATED_ISAACGYM_ENTRYPOINT_TASKS` (real hardware via the external Python 3.8 worker runtime; not covered by repo CI).
 - Validated genesis entrypoints are explicitly recorded in `_MAINTAINER_VALIDATED_GENESIS_ENTRYPOINT_TASKS` (real hardware, genesis-world extra + CUDA; not covered by repo CI); near-risk coverage lives in `tests/base/test_genesis_backend.py` (fake runtime), `tests/base/test_genesis_runtime.py` (real-runtime slow lane), and the genesis env smoke in `tests/envs/locomotion/g1/test_g1_owner_contract.py`.
 - IsaacSim owner scope is intentionally not promoted to `Tested`; `_MAINTAINER_VALIDATED_ISAACSIM_ENTRYPOINT_TASKS` is empty until a maintainer records full training evidence. Rendering protocol coverage lives in `tests/base/test_isaacsim_backend.py`; it is not a substitute for successful real playback.
-- `newton` is an optional owner backed by Newton 1.5.1 and the MuJoCo-Warp 3.11 / Warp 1.16 line, which it shares with the `mujoco` / `mjwarp` extras (jointly installable in one environment). Validated newton entrypoints are explicitly recorded in `_MAINTAINER_VALIDATED_NEWTON_ENTRYPOINT_TASKS` (real hardware, newton extra + CUDA; not covered by repo CI); remaining cells rely on the G1 PPO/SAC owner configs, compose/contract checks, and fail-closed runtime/import boundaries.
+- `newton` is an optional owner backed by Newton 1.5.1 and the MuJoCo-Warp 3.11 / Warp 1.16 line, which it shares with the `mujoco` / `mjwarp` extras (jointly installable in one environment). Validated newton entrypoints are explicitly recorded in `_MAINTAINER_VALIDATED_NEWTON_ENTRYPOINT_TASKS` (real hardware, newton extra + CUDA; not covered by repo CI); remaining cells rely on the G1 PPO/SAC owner configs, compose/contract checks, and fail-closed runtime/import boundaries. Native ViewerGL playback (offscreen record + interactive) is wired through the unisim `newton-render` extra; without it, record falls back to the MuJoCo snapshot renderer and interactive stays fail-closed.
 <!-- END GENERATED SUPPORT MATRIX -->

pyproject.toml

@@ -154,6 +154,14 @@ newton = [
     # dependency, so the extra carries it explicitly.
     "trimesh>=3.21.7",
 ]
+newton-render = [
+    # Native ViewerGL rendering for the newton backend (offline record and
+    # interactive playback). Install together with the newton extra; bounds
+    # mirror unisim-core's newton-render extra and newton's own examples
+    # extra. Headless record still needs an OpenGL context (EGL/GLX).
+    "pyglet>=2.1.6,<3",
+    "imgui-bundle>=1.92.0",
+]
 motrix = ["motrixsim-core==0.8.2"]
 genesis = [
     # Genesis imports as ``genesis`` and owns the GPU physics implementation;

scripts/tools/support_matrix.py

@@ -65,9 +65,12 @@
 # repo CI). sac_torch g1_walk_flat: full 5000-iteration training completed
 # on 2026-09-06 (RTX 4090, torch 2.8.0+cu128, newton 1.5.1, mujoco-warp
 # 3.11; reward/mean 6.68 -> 242.3, episode length -> 983, ~43k steps/s,
-# 4m25s wall, run 2026-09-06_01-21-36_newton) plus record playback
-# validation on model_5000.pt (800-frame MuJoCo snapshot render, walking
-# gait confirmed). PPO remains ``Configured`` without runtime evidence.
+# 4m25s wall, run 2026-09-06_01-21-36_newton). Playback validated the same
+# day on model_5000.pt: native ViewerGL offscreen record (800-frame
+# 1280x720 mp4 via ``newton-viewer-gl``) and an interactive ViewerGL window
+# smoke on a live X display; the MuJoCo snapshot record path remains as the
+# no-render-deps fallback. PPO remains ``Configured`` without runtime
+# evidence.
 _MAINTAINER_VALIDATED_NEWTON_ENTRYPOINT_TASKS: frozenset[tuple[str, str]] = frozenset(
     {
         ("sac_torch", "g1_walk_flat"),
@@ -433,7 +436,7 @@ def render_support_matrix(root: Path | None = None) -> str:
             "- Validated isaacgym entrypoints are explicitly recorded in `_MAINTAINER_VALIDATED_ISAACGYM_ENTRYPOINT_TASKS` (real hardware via the external Python 3.8 worker runtime; not covered by repo CI).",
             "- Validated genesis entrypoints are explicitly recorded in `_MAINTAINER_VALIDATED_GENESIS_ENTRYPOINT_TASKS` (real hardware, genesis-world extra + CUDA; not covered by repo CI); near-risk coverage lives in `tests/base/test_genesis_backend.py` (fake runtime), `tests/base/test_genesis_runtime.py` (real-runtime slow lane), and the genesis env smoke in `tests/envs/locomotion/g1/test_g1_owner_contract.py`.",
             "- IsaacSim owner scope is intentionally not promoted to `Tested`; `_MAINTAINER_VALIDATED_ISAACSIM_ENTRYPOINT_TASKS` is empty until a maintainer records full training evidence. Rendering protocol coverage lives in `tests/base/test_isaacsim_backend.py`; it is not a substitute for successful real playback.",
-            "- `newton` is an optional owner backed by Newton 1.5.1 and the MuJoCo-Warp 3.11 / Warp 1.16 line, which it shares with the `mujoco` / `mjwarp` extras (jointly installable in one environment). Validated newton entrypoints are explicitly recorded in `_MAINTAINER_VALIDATED_NEWTON_ENTRYPOINT_TASKS` (real hardware, newton extra + CUDA; not covered by repo CI); remaining cells rely on the G1 PPO/SAC owner configs, compose/contract checks, and fail-closed runtime/import boundaries.",
+            "- `newton` is an optional owner backed by Newton 1.5.1 and the MuJoCo-Warp 3.11 / Warp 1.16 line, which it shares with the `mujoco` / `mjwarp` extras (jointly installable in one environment). Validated newton entrypoints are explicitly recorded in `_MAINTAINER_VALIDATED_NEWTON_ENTRYPOINT_TASKS` (real hardware, newton extra + CUDA; not covered by repo CI); remaining cells rely on the G1 PPO/SAC owner configs, compose/contract checks, and fail-closed runtime/import boundaries. Native ViewerGL playback (offscreen record + interactive) is wired through the unisim `newton-render` extra; without it, record falls back to the MuJoCo snapshot renderer and interactive stays fail-closed.",
         ]
     )
     return "\n".join(lines)

src/unilab/conf/ppo/task/g1_walk_flat/newton.yaml

@@ -11,9 +11,9 @@ defaults:
 training:
   task_name: G1WalkFlat
   sim_backend: newton
-  # Newton has no native UniLab viewer yet; eval uses the configured record /
-  # headless path and fails closed if a native renderer is requested.
-  play_render_mode: record
+  # Native ViewerGL rendering (offline record + interactive) is provided by
+  # the backend; playback inherits the base config's auto mode (interactive
+  # with a display, record otherwise).
 
 algo:
   num_envs: 2048

src/unilab/conf/sac/task/g1_walk_flat/newton.yaml

@@ -7,16 +7,16 @@
 # capacities: a null newton_device is intentional — the process device binder
 # injects the rank-local CUDA device before materialization. Runtime PD gain
 # randomization is rejected by the backend, so pd_gains stays disabled.
-# Newton has no native UniLab viewer yet; eval uses the configured record /
-# headless path and fails closed if a native renderer is requested.
+# Native ViewerGL rendering (offline record + interactive) is provided by the
+# backend; playback inherits the base config's auto mode (interactive with a
+# display, record otherwise).
 defaults:
   - /task/g1_walk_flat/base
   - _self_
 
 training:
   task_name: G1WalkFlat
   sim_backend: newton
-  play_render_mode: record
   # Newton compiles its Warp kernels on the materialize cold path, which
   # exceeds the runner's default 30 s collector tick-0 budget; raise it for
   # this backend (same precedent as the Genesis owner).

tests/envs/locomotion/g1/test_g1_owner_contract.py

@@ -586,6 +586,9 @@ def test_g1_owner_materializes_complete_plain_manager_cfg(
         assert env_cfg.newton_nconmax == 320
         assert env_cfg.newton_njmax == 512
         assert env_cfg.newton_capacity_check_steps == 1
+        # Native ViewerGL rendering (interactive viewer + offscreen record)
+        # is supported; playback stays on the base config's auto mode.
+        assert hydra_cfg.training.play_render_mode == "auto"
     if backend == "isaacgym":
         assert env_cfg.isaacgym_device_id == 0
         # Native rendering (viewer + camera-sensor record) is supported;