Conversation
|
Thanks for the pr. We will soon review it. |
0f49c70 to
42018d1
Compare
| """Return (pos, rot) of a bottle in world-ish coords (env 0 at origin).""" | ||
| import torch | ||
|
|
||
| pos, rot = env.scene_manager.layout_manager.get_instance_pose(env_idx, label=label) |
There was a problem hiding this comment.
Could this expose simulator object positions to the agent? Since the RoboDojo prompt defines the setup as perception-isolated, should object locations only come from camera or depth observations instead?
There was a problem hiding this comment.
Good catch ??these outputs are now grouped as privileged tools and can be removed through allowed_tool_groups (e05e361). We will wire that into the two modes: dev keeps full feedback for self-evolution, eval-fair excludes privileged groups (mirroring LIBERO's evaluation-only Flash planner). The mode switch + contract tests land in the next stage.
| ## 13. Reward/Score 明细（2026-08-20 补） | ||
|
|
||
| - 新增 RPC `get_reward_details`：返回逐项判定——每个 bottle 的 | ||
| `is_A_on_B_bottom`（底部贴合 dustbin 底部平面）、`grippers_open`、 |
There was a problem hiding this comment.
Is it intended to expose these per-object success predicates to the agent? I would expect the environment to use them internally for success computation, while the agent only observes the normal reward or final success signal.
There was a problem hiding this comment.
Same handling as the object-position thread: get_reward_details is marked privileged and is excluded when the eval-fair group filter is active. A contract test proving eval runs cannot read predicates is part of the next stage.
| name="sam3_server", | ||
| cmd=[ | ||
| sys.executable, | ||
| str(get_repo_root() / "robots" / "libero" / "sam3_server.py"), |
There was a problem hiding this comment.
Could we use the shared rpent/robots/components/sam3_server.py here instead of the LIBERO-specific path?
| """Initialize every RoboDojo component, or only ``components`` when given.""" | ||
| from robots.robodojo.env_client import RoboDojoEnvClient | ||
| from robots.robodojo.vla_client import RoboDojoVLAClient | ||
| from rpent.utils.sam3_client import Sam3Client |
There was a problem hiding this comment.
Could we use rpent.robots.components.sam3_client.Sam3Client here to stay consistent with the latest main?
| owned_daemons: dict[str, ProcessDaemon] = {} | ||
| primitives_kwargs: dict[str, Any] = {} | ||
|
|
||
| if "env" in selected: |
There was a problem hiding this comment.
Could we follow the starters and connectors pattern used in robots/libero/robot_spec.py here? The env, SAM3, and VLA lifecycle looks quite similar, and this could avoid duplicating the spawn and wait logic for each component.
| from rpent.utils.daemon import ProcessDaemon | ||
|
|
||
|
|
||
| DEFAULT_WORKSPACE = "/home/admin/robodojo_pro6000_ws" |
There was a problem hiding this comment.
Could we avoid using a developer-specific /home/admin/... path as the default workspace?
|
|
||
| .. code-block:: bash | ||
|
|
||
| uv pip install -e ".[rlinf,openpi,libero-pro,sam3]" # full install |
There was a problem hiding this comment.
This installation command seems out of sync with the latest main after #114: openpi is no longer a standalone extra. Could we align the RoboDojo installation with the new per-environment packaging structure?
| RPent x RoboDojo Integration Log | ||
| ================================ | ||
|
|
||
| Record period: 2026-08-20 ~ 2026-08-21. Workspace: |
There was a problem hiding this comment.
Do we need to keep this integration log in the user-facing documentation? It contains local workspace paths and historical experiment notes that are likely to become stale.
| call `stabilize` first — place the nearest arm's open gripper in the bottle's | ||
| path at table height to stop it before it falls off the table (a lost bottle | ||
| is unrecoverable). Only resume the task after the alarm clears. | ||
| - To place a held object into the dustbin, use `place_in_bin` (carry to the |
There was a problem hiding this comment.
Should this task-specific dustbin guidance live in the generic RoboDojo system prompt? It seems better suited to task-specific context or memory so unrelated RoboDojo tasks do not receive put_bottles instructions.
There was a problem hiding this comment.
Done in e05e361: the dustbin/placement, bottle-alarm and score guidance moved out of the generic system prompt into task-specific context, so unrelated RoboDojo tasks no longer receive it.
| - env_cfg: {{env_cfg_type}} | ||
| - action_type: {{action_type}} | ||
| - output_dir: {{output_dir}} | ||
| - scene (static hint): {{task_summary}}""" |
There was a problem hiding this comment.
Does this mean the agent knows which objects are in the task before looking at the camera? Should it discover them from the observations instead?
| ) | ||
|
|
||
|
|
||
| def get_toolkit( |
There was a problem hiding this comment.
Could we align get_toolkit with the current robot interface and pass config and MemoryManager here, as LIBERO, RoboCasa, and RoboTwin do?
| dashboard_events: DashboardEventSink, | ||
| ) -> None: | ||
| state = EnvState(get_output_dir()) | ||
| super().__init__(dashboard_events=dashboard_events, state=state) |
There was a problem hiding this comment.
It looks like the current Toolkit constructor requires memory. Could we pass the MemoryManager through here, consistent with the other robot toolkits?
| "--enable isaacsim.sensors.camera", | ||
| ] | ||
| + (["--random"] if getattr(args, "random", False) else []), | ||
| env=_subprocess_env({"CUDA_VISIBLE_DEVICES": str(args.sim_device)}), |
There was a problem hiding this comment.
ProcessDaemon currently takes env_overrides rather than env.
| else: | ||
| from rpent.utils.rpc import parse_endpoint | ||
|
|
||
| _, host, port = parse_endpoint(args.env_endpoint) |
There was a problem hiding this comment.
Could we use make_rpc_client() here like libero/robocasa/robotwin?
The CLI accepts [protocol://]host:port, but the current code discards the parsed protocol and always creates an HttpRpcClient.
| vision = obs.get("vision") or {} | ||
| return {name: {"width": 640, "height": 480} for name in vision} | ||
|
|
||
| def render_camera(self, camera_name: str | None = None) -> dict[str, Any]: |
There was a problem hiding this comment.
Could get_camera_meta follow the common camera_name-based interface? BaseEnvClient sends camera_name
| raise ValueError(f"unknown camera: {camera_name!r}") | ||
| return vision | ||
|
|
||
| def reset(self) -> dict[str, Any]: |
There was a problem hiding this comment.
Could we follow the current BaseEnvFacade contract here? New backends should preferably return reset=(obs, info) and step=(obs, reward, terminated, truncated, info), as LIBERO and RoboTwin do.
| } | ||
| return obs, reward, done, info | ||
|
|
||
| def chunk_step(self, flat_actions, *, return_all_frames: bool = False): |
There was a problem hiding this comment.
Since the RoboDojo Pi_05 backend already produces action chunks, could we implement the shared chunk_step contract here, similar to LIBERO and RoboTwin, instead of issuing one RPC step per policy action?
| return env_cfg | ||
|
|
||
| env = create_collect_env(_build_env_cfg(), simulation_app) | ||
| if _args.random: |
There was a problem hiding this comment.
Do we need this initial reset in the server? BaseEnvClient performs reset when the client connects, so RoboDojo currently appears to reset twice on startup. LIBERO and RoboTwin leave the episode reset to the normal client lifecycle, which seems especially important for --random reproducibility.
| return out | ||
|
|
||
|
|
||
| def back_project(primitives, state, row, col, camera="cam_head") -> dict: |
There was a problem hiding this comment.
Could back_project and segment be marked @readonly, consistent with LIBERO?
There was a problem hiding this comment.
Same change as the newer thread: @readonly added in adb621d with a contract test covering it.
| - The robot tools are exposed as an MCP HTTP server. Find its URL in this | ||
| session's startup log: a line like `I [mcp_http] HttpMcpServer ready at | ||
| http://127.0.0.1:<port>/mcp/`. | ||
| - Call tools with JSON-RPC over HTTP: `initialize` (stateless=true), |
There was a problem hiding this comment.
Could we remove the manual MCP URL / JSON-RPC workflow here? LIBERO, RoboCasa, and RoboTwin prompts simply instruct the planner to call the registered RPent tools, leaving MCP details to the runtime. This seems cleaner and less coupled to the current implementation.
There was a problem hiding this comment.
Done in e05e361: the prompt assumes planner-injected tools and calls them by name; the MCP URL / JSON-RPC discovery instructions were removed from system/user prompts and guides (zh/en).
| `success` heuristic is provisional; confirm holds from the wrist camera. | ||
| - Gripper semantics: 1 = close/hold, -1 = open. Keep the gripper closed while | ||
| carrying an object. | ||
| - `get_status` reports the step counter and step limit; the environment |
There was a problem hiding this comment.
The prompt refers to get_status, but I don't see get_status registered in RoboDojo's TOOLS_SPEC.
| the URL port) instead of writing a new client. | ||
| - Do NOT read the environment/rpent source code to understand the tools; | ||
| use `tools/list` for schemas and get on with the task. | ||
| - You are a text-only model: camera images are NOT visible to you. Trust |
There was a problem hiding this comment.
Should the shared RoboDojo prompt assume that every planner is text-only? The other robot prompts are planner-agnostic, and view_env_state already carries image data.
| "localhost", | ||
| ] | ||
| _log("spawning policy server: " + " ".join(cmd)) | ||
| proc = subprocess.Popen( |
There was a problem hiding this comment.
Should RoboDojoVLAFacade retain ownership of the spawned policy process and terminate it on close? RPent's other runtime components follow a clear spawn/own/cleanup lifecycle, while this Popen handle is currently discarded and may leave the nested policy server running.
|
P3 implemented in 8f96df3 (feat(robodojo): add eval-fair frozen Flash replay).
|
|
P4 documentation update in 2b205e8: rewrote the English/Chinese RoboDojo guide as an Adding a Robot Backend worked example covering registration/runtime ownership, Env contracts and main-thread dispatch, tool information groups, prompts/tasks, shared policy backends, Flash isolation, and an implementation/test checklist. Preserved the frozen-replay instructions. Filled RoboDojo entries in both README feature matrices and Sphinx overviews as experimental, with task-scope caveats, no handover, and limited low-Z scripted IK. No runtime code changed. Validation: ruff check --preview ., ruff format --check ., pre-commit run --all-files, and both make -C docs html LANG=en/zh builds with -W --keep-going -E passed. Python 3.11 full unit suite: 633 passed, 3 skipped, one dependency deprecation warning. GPU, real policy services, hardware, simulator E2E, and the other CI Python versions were not validated; no task-success claim is made. |
littleZ05
changed the title
P5 smoke result: failed at VLA startup
Logs
PR metadataUpdated title to the repository's Conventional Commit style (under 70 characters): |
P5 lazy-import regression fix and rerunLocal commit: OmegaConf is imported only when building RLinf model configuration; missing installation raises a backend-specific error identifying the policy Python environment. Torch and RLinf imports were already lazy. New contracts execute the XPolicyLab CLI with OmegaConf/RLinf/Torch blocked and verify the RLinf missing-OmegaConf error. Offline validation
Real smoke: failed at toolkit constructionCommand: The script records the full environment and exact CLI: worktree imports, gpt-6-astra/low, layout 1, GPU0, single sim, 1780-second timeout with 20-second kill grace, explicit source/Python/checkpoint paths, isolated memory and output paths. No research-main files were changed. Wall time: 55.93 seconds, exit code 1. No retry after failure.
Evidence (outside git)
The factory keyword mismatch and failure-path policy/video cleanup need separate fixes before the smoke sequence can pass. The import fix is committed locally but remains unpublished because the requested smoke gate failed. |
P5 toolkit interface fix and smoke rerunLocal commit: FixRoboDojo's factory now accepts the same required keyword arguments as LIBERO/RoboCasa/RoboTwin: The existing task/group tests now construct the real toolkit through the registry, checking memory and actual dispatch. New cases check all three missing required arguments and rejection of the obsolete keyword. Validation
Smoke: stopped at the first failureCommand: The script contains the exact environment/CLI (worktree PYTHONPATH/RPENT_REPO_ROOT, gpt-6-astra/low, layout 1, GPU0, single sim, explicit interpreter/source/checkpoint paths, isolated local memory, 1780-second timeout plus 20-second kill grace). Wall time 55.72 seconds, exit code 1.
Evidence outside git
Remaining blockers are the installed Codex SDK/model-catalog compatibility and policy child cleanup on failure. Research main and the 4090 were not touched; no evidence was committed. |
|
P5d rerun at local 0d883f7 (+ cd24e20), not pushed: Smoke still FAILED its cleanup gate: the XPolicyLab child survived parent shutdown holding 29772 MiB. After verifying this run?s task/port, manually terminated PID 964708; GPU0 returned to 20 MiB / 0%, no compute processes. Per stop-on-first-failure, fill_pen_holder and Flash replay were not run; no push. Offline validation of the actual 14-entry dev trace also blocks Flash export: Reproduction note: this installed Codex CLI requires supports_parallel_tool_calls in every model-catalog entry. The environment owner fixed the catalog before P5d (backup suffix (local backup)); no PR source/global configuration change was needed for that issue. Exact command/environment: evidence/pr96-p5d-20260918/run-dev.sh. Console/wall time: .../put_bottles_into_dustbin.console.log. Logs, states.json, flash_trace.json, transcript and videos: .../put_bottles_into_dustbin/. Detailed local report: evidence/pr96-p5d-20260918/results.md. Evidence remains outside git. |
|
P5e local fixes: 181950c adds idempotent owned-policy cleanup, atexit and SIGTERM unwinding through ProcessDaemon; 88b32a7 unifies recording/replay mask-centroid derivation with strict version-2 plans and tamper rejection. Prior cd24e20/0d883f7 remain local too. Offline validation: 647 passed / 3 skipped; focused 35 passed; Ruff, pre-commit and both strict Sphinx builds passed. Commands:
No push because the complete smoke gate is not green. Next: bound the pen-holder smoke completion or agree a planner budget within the 30-minute cap, then run the remaining Flash checks. Old box-center plans must be rerecorded, not silently converted. Evidence root: evidence/pr96-p5e-20260918/ (run-dev.sh, task.console.log, task/videos, states.json, flash_trace.json, service logs). Full commands/results: results.md there; all outside git. Reproduction note: the installed Codex CLI model catalog requires supports_parallel_tool_calls on every entry; the environment owner supplied it before this batch, and no global configuration was changed here. |
|
P5f closeout at 88b32a7: both remaining runtime smokes were executed serially on PRO 6000 GPU0, one simulator. Overall not green; no source changes or new push. The PR description now reflects current evidence rather than the superseded P5 startup failure. bash evidence/pr96-p5f-20260918/run-dev.sh fill_pen_holder
bash evidence/pr96-p5f-20260918/run-flash.sh
Next: profile fill planner/tool latency before selecting another bounded completion budget; record a genuinely held dev grasp (and a real same-anchor approach if retry is needed), then re-export. Do not weaken the Flash hold gate or synthesize a passing plan. Replay completion is not official task success. Same-revision prior validation remains 647 passed / 3 skipped, Ruff check/format, pre-commit, EN/ZH strict Sphinx passed; no source changes, so not rerun in P5f. Prior dustbin dev: 570.71s, exit 0, 615 frames/camera, cleanup passed. Reproduction prerequisite: installed Codex model-catalog entries must declare Exact runners/paths, timings, video frame reports and caveats: |
|
P6 fixes and bounded smoke results:
Validation: 658 passed / 3 skipped, Ruff check/format, pre-commit and EN/ZH strict Sphinx passed (Python 3.11). Includes real subprocess EOF/cancel/exit-0 tests, RobotWin lifecycle fakes, stale graph tick regression and dev/Flash error-artifact contracts. bash evidence/pr96-p6-20260918/run-dev.shDustbin layout 1, single sim, PRO 6000 GPU0 only; gpt-6-astra/low. Planner timeout 1500s/40 turns; outer timeout 1700s plus 20s grace (<30 min). These are smoke overrides, not defaults; the same bounded strategy is documented for fill. 990.66s, CLI/smoke exit 1. Planner ran one 10-chunk Lifecycle acceptance passed: env/VLA/SAM3 exits 0/0/0, no fatal errors, no Overall hold gate failed, so no plan export, Flash eval, or fill rerun. Successful replay on this revision remains unverified. No synthetic plan, weakened hold gate or replacement evaluation. Next: diagnose actual policy/scene grasp behavior and obtain a hold-accepted dev demonstration before export/eval. RobotWin/LingBot and Molmo GPU runs, hardware and other Python matrix legs were not tested here. Evidence (outside git): Environment reminder: installed Codex catalog entries require |
|
P7 hold follow-up: not green; local fix
Full commands, raw paths and limitations: |
|
P8 identity follow-up: not green; commits Added shared 5cm first-head-anchor consistency ( Real dev: Infrastructure passed: env/VLA/SAM3 0/0/0, three videos 49/49 decoded, no residual processes, GPU20MiB/0%, no provider disconnect. No plan export or Flash run after failed target hold. Remaining blocker is checkpoint target selection; inspect language conditioning before another trial. Full report/raw paths: |
P9 official task language (local, not pushed)Local Before-fix CPU execution of actual original methods returns Same-P8 smoke command: No export/Flash after failed target hold. Current Flash isolation/replay and real general_pickup scene remain unverified. No push or pull-rebase: |
get_task_language returned the raw general_pickup template, so callers could receive the literal placeholder instead of the description the environment publishes. Resolve language through RoboDojo's description manager, fall back to expanding labels from the layout descriptions, and reject empty or unresolved strings at the boundary. Observations expose the same resolved value, and pi0_pick defaults to it while still rejecting template markers in explicit overrides.
|
Pushed
The RPC and the public observations now resolve language through RoboDojo's Verified on this branch: |
| @@ -0,0 +1,297 @@ | |||
| Adding a Robot Backend | |||
There was a problem hiding this comment.
remove the Adding a Robot Backend section, it's already in add_robot doc
There was a problem hiding this comment.
Removed in 541a25f. The duplicated backend-integration steps ("register the backend", "define the environment contract", "assemble tools", "reuse the policy service") are gone from both index files, which now keep only what is specific to RoboDojo: the module list, capability scope and limitations, and the installation pointer, plus a pointer to development/add_robot.
| pi05_root = args.policy_root | ||
| if not pi05_root: | ||
| raise ValueError("--policy-root is required for a local XPolicyLab server") | ||
| launcher = os.path.join(pi05_root, "setup_eval_policy_server.sh") |
There was a problem hiding this comment.
setup_eval_policy_server.sh, is this provided by upstream or your local script? if it is latter, please use a more standard method.
There was a problem hiding this comment.
Upstream — the PR does not ship that script.
_spawn_policy_server resolves launcher = <policy-root>/setup_eval_policy_server.sh, and --policy-root points into RoboDojo's XPolicyLab submodule (XPolicyLab/policy/Pi_05/). RoboDojo tracks that submodule itself, e.g. 36bfcb7 [scripts] chore: bump XPolicyLab submodule to latest main. So this is the upstream-provided entry point; we only pass its arguments.
The /tmp log next to it was ours, and it is fixed in the follow-up commit on this branch: the policy server log now goes to <output-dir>/vla_server.log, and the RoboDojo launcher passes its run output directory down.
| "general": frozenset( | ||
| {"back_project", "segment", "move_to", "pi0_pick", "stabilize"} | ||
| ), | ||
| "privileged": frozenset({"get_reward_details", "get_safety_status"}), |
There was a problem hiding this comment.
just curious, what is the use of privileged tools?
There was a problem hiding this comment.
It is the eval-fair isolation gate.
TOOL_GROUPS classifies each RoboDojo tool as general, privileged or mixed, and get_toolkit(allowed_tool_groups=...) filters registration by group. In dev mode nothing is filtered, so the planner can read get_reward_details (reward breakdown, per-stage progress) and get_safety_status (rolling / off-table alarms) and use them to correct a failing run.
In Flash replay (--planner flash, mode eval-fair) robots/robodojo/toolkit.py intersects the allowed groups with {"general", "mixed"}, and env_server additionally drops env.get_reward_details, env.get_safety_status, env.is_success and env.reset from the RPC table. A replay therefore cannot consult reward, and cannot reset the episode. mixed covers tools such as view_env_state / set_gripper / place_in_bin that are needed on both paths, which is why they are not in general.
|
Pushed
Verification on this commit: |
The installation page deferred the whole environment build to upstream and never mentioned the pieces this integration actually depends on. Add a Python environments section that states why the RoboDojo simulator, Pi_05 and RPent interpreters have to stay separate, which upstream installer builds each one, the versions this backend is validated against, and that RPent reaches the child services through PYTHONPATH rather than being installed into the policy environment. Also record the placement settling budget the backend needs in official mode, note that each service writes into the run output directory, and add a verification section with one bounded development run, the files and logs it should leave behind, and the expected observation keys.
|
Pushed New "Python environments" section. States why the RoboDojo simulator, Pi_05 and RPent interpreters must stay separate (Isaac Sim pins
New "Verify the installation" section. One bounded development run, the logs it should leave ( The page still delegates the Isaac Sim build itself to the upstream installer, since duplicating it would drift. Verified on this commit: strict EN/ZH Sphinx ( |
4 participants
Summary
Add RoboDojo (Isaac Sim / IsaacLab, dual ARX-X5 arms) through
rpent --robot robodojo, using RPent's existing robot, runtime, perception, planner and memory interfaces. Infrastructure only; no evidence, checkpoints or research memories are included.rlinf/xpolicylabadapters; lazy backend dependencies, 14-DoF actions, three-camera WebSocket inputs and unchanged dual-armpi0_pickmonitoring.error; commonrun_diagnostics.jsonpreserves final errors, including finalization failures, for dev and Flash.Affected tasks
Task discovery uses the configured RoboDojo checkout. Integration targets include
put_bottles_into_dustbin,fill_pen_holder,stack_bowls_random; these are not benchmark-success claims.place_in_binremains dustbin-specific.Test plan
P6, Python 3.11:
.venv/bin/ruff check --preview .: passed..venv/bin/ruff format --check .: passed (253 files)..venv/bin/pre-commit run --all-files: passed..venv/bin/pytest tests/unit_tests -q: 658 passed, 3 skipped, one Starlette dependency deprecation warning, 29.14s.make -C docs html LANG=en SPHINXBUILD=<repo-checkout>/.venv/bin/sphinx-build SPHINXOPTS='-W --keep-going -E -q': passed; same command withLANG=zh: passed.P6 real smoke
Single simulator, only PRO 6000 GPU0, layout 1; worktree
PYTHONPATHandRPENT_REPO_ROOT. Codexgpt-6-astra, low effort,--planner-timeout-s 1500 --max-turns 40, outertimeout --signal=INT --kill-after=20s 1700s. This leaves startup/cleanup room below 30 minutes. These are smoke overrides, not defaults; the bilingual guide documents the same bounded policy forfill_pen_holder.The evidence-only wrapper narrows the dev request to a real single-target grasp-and-hold demonstration with at most three attempts and no resets, runs the existing hold predicate on actual returned state/perception, records server return codes and fully decodes three videos. It never changes tool return values or creates a synthetic plan. Required shutdown checks reject fatal errors, tracebacks and
[Error]after the shutdown marker; headless GLFW startup warnings are expected noise.990.66s outer wall time, CLI/smoke exit 1 (audit timer 990.26s; CLI loop 950.4s). Planner entered and executed one 10-chunk pick. Actual
pi0_pick.success=false, its step-limitterminated=false, both final grippers approximately open (0.998/0.999), and the unchanged hold predicate returned false. The environment's subsequent recordedterminated=trueis preserved separately; it does not establish a held object or official benchmark success. The planner stopped motion rather than resetting the ended episode. No approach waypoint or passing plan was fabricated.The provider then disconnected five times and failed with
stream disconnected before completion: stream closed before response.completed. Bothdev/transcript_put_bottles_into_dustbin_l1.jsonanddev/run_diagnostics.jsoncontain that exact error. This attempt did not report HTTP 503; the earlier interrupted run did.Shutdown acceptance passed: env/VLA/SAM3 each exited 0; no
Fatal Python error; env shutdown interval contained no[Error], traceback, stalePy_Graph, or worker-thread event-loop error. Headless GLFW initialization/plugin warnings were the only closing warnings and are explicitly allowed noise. All three videos fully decoded 513/513 frames each. All owned services/policy children exited without manual intervention; GPU0 returned to 20 MiB / 0%, no compute processes.Hold gate failed, so the batch stopped. No version-2 plan was exported and no P6 Flash or fill run was started. P6 eval-fair runtime isolation and successful replay remain unverified; offline Flash contracts pass, and historical P5f isolation evidence is below. Next step: investigate the real policy/scene grasp mismatch and obtain a genuine hold-accepted dev demonstration with a stable provider before evaluating it. Do not relax the hold gate.
Detailed evidence:
evidence/pr96-p6-20260918/results.md,dev-audit.json,dev.console.log, anddev/(service logs, states, trace, transcript and videos). Smoke instrumentation remains outside git; the PR contains only code/tests/documentation.P7 hold follow-up (local fix, not pushed)
Local
3f126b2stops policy chunks immediately on the unchanged pick heuristic and adds aperture/action diagnostics; 659 passed / 3 skipped, Ruff/format/pre-commit and EN/ZH strict Sphinx passed. PR code remainsc8bd725pending a green smoke.Command:
bash evidence/pr96-p7-20260918/run-dev.sh(single PRO6000 GPU0, layout1, planner1500s/40 turns, outer1700s+20s). 962.48s, CLI0 but smoke1: approach reached within3mm in8 steps; same white-cap query switched to a different horizontal bottle (~22.3cm anchor jump) twice. Planner finished stuck before any pick (0 chunks); hold not established, native task success=false. No provider disconnect.Infrastructure passed: env/VLA/SAM3 exit0, clean shutdown, all three videos20/20 decoded frames, no residual services, GPU20MiB/0%. No export or Flash run after this failure; runtime isolation/replay and early-stop hold remain unverified at the local revision. Next check distinctive target identity and approach visibility on saved RGB-D before another bounded trial; same query/confidence alone is not persistent identity. Evidence and exact commands:
evidence/pr96-p7-20260918/results.md(outside git).P8 identity follow-up (local, not pushed)
Local commits
3f126b2and963a30dare not pushed because the real hold gate remains red.AnchorTrackerandlocate_anchor: fixed 5cm world-centroid consistency against the first head observation, fresh accepted localization before every motion, and post-approach/pre-pick confirmation. Flash/export use the same guard; identity jumps revoke motion and are rejected. Added EN/ZH guidance and contracts.bash /home/admin/robodojo_runtime/evidence/pr96-p8-20260918/run-dev.sh, single PRO6000 GPU0/sim, gpt-6-astra low, planner1500s/40 turns. 1028.65s, CLI0 / smoke1. Identity passed: centroid [169,167] -> [170,167], 2.249mm; approach error 2.4mm. Pick executed 1 chunk/36 actions and original gate returnedpick_success=true, but wrist/head evidence showed the requested blue target still on table and a different white bottle in the gripper;held=false, native success=false. Planner stopped without retry./home/admin/robodojo_runtime/evidence/pr96-p8-20260918/results.md(outside git).P9 official task language (local, not pushed)
Local
3864c6afixes raw-template task-language RPC: prefer official description manager, otherwise fill every label from public layout descriptions, fail explicitly on unresolved/empty text; observations use the same resolver.pi0_picknow defaults to official language if prompt is omitted; explicit identity overrides remain supported and template markers are rejected. EN/ZH docs and contracts included. 676 passed /3 skipped, Ruff/format/pre-commit/strict bilingual Sphinx passed.Before-fix CPU execution of actual original methods returns
Pick up the <target> by 10 cm.Official HDF5 at the supplied location has root/instruction(not additional_info/instruction), valuePick up the lavender plastic shovel by 10 cm.However, official observations already used resolved descriptions, and P8's dustbin pick had an explicit blue-target override; that template defect is not established as P8's cause.Same-P8 smoke command:
bash /home/admin/robodojo_runtime/evidence/pr96-p9-20260918/run-dev.sh. Same dustbin/layout1/blue anchor/staging/arm/chunks/budgets; only omitted pick prompt to test official default. Actual RPC and VLA strings both:Pick up the bottles and throw them into the dustbin, using handover when needed.810.99s, CLI0/smoke1; 1chunk/33 policy actions (35 total), pick_success=true but held=false: white bottle grasped, designated blue target remains on table. Native success=false. No provider disconnect. Services0/0/0, videos46/46 decoded each, clean shutdown/no residual processes, GPU20MiB/0%.No export/Flash after failed target hold. Current Flash isolation/replay and real general_pickup scene remain unverified. No push or pull-rebase:
3f126b2,963a30d,3864c6aremain local pending green smoke; authenticated user littleZ05. Next investigate checkpoint instance-selection support, not lower gates. Evidence:/home/admin/robodojo_runtime/evidence/pr96-p9-20260918/results.mdandlanguage_probe.py(outside git).Prior runtime evidence (different revisions)
Local evidence directories:
evidence/pr96-p6-20260918/,pr96-p5f-20260918/,pr96-p5e-20260918/, andpr96-p5g-20260918/. They are host-local artifacts, not committed files.Environment note
The installed Codex CLI model-catalog schema requires
supports_parallel_tool_callsin every configured entry. The environment owner previously fixed~/.codex/codex-models.json(backup suffix(local backup)). No global model configuration changed here. Provider stream errors and HTTP 503 must be reported separately from robot/runtime defects.Known limitations
move_to,set_gripper,pi0_pick; unsupported/failed/unanchored records are rejected. No task plans are bundled.sam3_mask_centroid_floor_v1; old box-center/version-1 plans require rerecording.pi0_pick.successis a proprioceptive heuristic, not the official task predicate.