Sync fork to NVIDIA GR00T N1.7 - #12
Merged
Merged
Conversation
* GR00T N1.6 Release Co-authored-by: Abhishek Mishra <abhmishra@nvidia.com> Co-authored-by: Alisson Azzolini <aazzolini@nvidia.com> Co-authored-by: Curie Park <curiep@nvidia.com> Co-authored-by: Fengyuan Hu <fengyuanh@nvidia.com> Co-authored-by: Hassan Eslami <heslami@nvidia.com> Co-authored-by: Kaushil Prakashbhai Kundalia <kkundalia@nvidia.com> Co-authored-by: Rahul Chand <rchand@nvidia.com> Co-authored-by: runyud <runyud@nvidia.com> Co-authored-by: Scott Reed <sreed@nvidia.com> Co-authored-by: Wei Liu <liuw@nvidia.com> Co-authored-by: Xiaowei Jiang <xiaoweij@nvidia.com> Co-authored-by: You Liang Tan <youliangt@nvidia.com> Co-authored-by: Yu Fang <yufa@nvidia.com> Co-authored-by: Yuke Zhu <yukez@nvidia.com> Co-authored-by: Yunfan Jiang <yunfanj@nvidia.com> Co-authored-by: Zhe Zhang <zhezh@nvidia.com> Co-authored-by: Zhe Zhang <zhezh@nvidia.com> Co-authored-by: You Liang Tan <tan_you_liang@hotmail.com> * Add CI * Remove GitLab CI * fix gif and nit Signed-off-by: youliangt <youliangt@nvidia.com> * fix stacks and nits Signed-off-by: youliangt <youliangt@nvidia.com> --------- Signed-off-by: youliangt <youliangt@nvidia.com> Co-authored-by: Abhishek Mishra <abhmishra@nvidia.com> Co-authored-by: Alisson Azzolini <aazzolini@nvidia.com> Co-authored-by: Curie Park <curiep@nvidia.com> Co-authored-by: Hassan Eslami <heslami@nvidia.com> Co-authored-by: Kaushil Prakashbhai Kundalia <kkundalia@nvidia.com> Co-authored-by: Rahul Chand <rchand@nvidia.com> Co-authored-by: runyud <runyud@nvidia.com> Co-authored-by: Scott Reed <sreed@nvidia.com> Co-authored-by: Wei Liu <liuw@nvidia.com> Co-authored-by: Xiaowei Jiang <xiaoweij@nvidia.com> Co-authored-by: You Liang Tan <youliangt@nvidia.com> Co-authored-by: Yu Fang <yufa@nvidia.com> Co-authored-by: Yuke Zhu <yukez@nvidia.com> Co-authored-by: Yunfan Jiang <yunfanj@nvidia.com> Co-authored-by: You Liang Tan <tan_you_liang@hotmail.com>
* minro n1.6 doc fix Signed-off-by: youliangt <youliangt@nvidia.com> * shell Signed-off-by: youliangt <youliangt@nvidia.com> --------- Signed-off-by: youliangt <youliangt@nvidia.com>
* fix torch.distributed.error on single gpu * lint
Bumps [transformers](https://github.com/huggingface/transformers) from 4.51.3 to 4.53.0. - [Release notes](https://github.com/huggingface/transformers/releases) - [Commits](huggingface/transformers@v4.51.3...v4.53.0) --- updated-dependencies: - dependency-name: transformers dependency-version: 4.53.0 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* add extra batch size in option for replay * mark obs optional
Downgrade transformers package from version 4.53.0 to 4.51.3.
* initial commit for N1.6-DROID integration * Fix small bug * Added fix to run inference on blackwell gpus * Added control scripts for oss release * Gr00t N1.6-DROID release * fix path Signed-off-by: Nadun Ranawaka <nadun.kesara@gmail.com> * update docs Signed-off-by: Nadun Ranawaka <nadun.kesara@gmail.com> * update docs Signed-off-by: Nadun Ranawaka <nadun.kesara@gmail.com> * cleanup Signed-off-by: Nadun Ranawaka <nadun.kesara@gmail.com> --------- Signed-off-by: Nadun Ranawaka <nadun.kesara@gmail.com> Co-authored-by: Nadun Ranawaka <nranawakaara@nvidia.com>
Signed-off-by: youliangt <youliangt@nvidia.com>
We need to add apt-get update before running apt-get install
…d add fixture/...
…option over decord, solve related behavior sim import issue. (SO101 error is fixed in 489)
* update * fix ruff * adjust augment order * update img aug * clean code * clean code * update doc strings * update ruff * revert doc strings changes * revert some changes * address comments * fix ruff * rewrite cfg * update doc strings * add example * add mask desc
Co-authored-by: Gr00t Metrics Bot <gr00t-metrics-bot@noreply.nvidia.com> Co-authored-by: Isabel Zuluaga <izuluaga@nvidia.com> Co-authored-by: Jiajing Chen <jiajingc@nvidia.com> Co-authored-by: Mohammad Harrim <mharrim@nvidia.com> Co-authored-by: Ryan Halabi <ryhalabi@nvidia.com> Co-authored-by: Scott Reed <sreed@nvidia.com> Co-authored-by: Sean Gillen <sgillen@nvidia.com> Co-authored-by: SyncBot <syncbot@gitlab-master.nvidia.com> Co-authored-by: Xiaotong Chen <xiaotongc@nvidia.com> Co-authored-by: Yu Fang <yufa@nvidia.com> id: 5db28a3026371ff4625f62a1801061ef9a72a2ec
Co-authored-by: jiajingc-nv <jiajingc@nvidia.com> Co-authored-by: Ryan Halabi <ryhalabi@nvidia.com> id: ffa40fbd0c017241400ef59bf610b977461e50b0
- Testing improvements/fixes Co-authored-by: Fernando Castaneda Garcia-Rozas <fcastanedaga@nvidia.com> Co-authored-by: Jiajing Chen <jiajingc@nvidia.com> Co-authored-by: Ryan Halabi <ryhalabi@nvidia.com> id: 61756420c3b2f675176d25c7eab948e622c69eff
- Restores and expands RoboCasa benchmark coverage with Panda benchmark examples and RoboCasa GR1 tabletop metrics support. - Hardens runtime, data loading, and policy serving with safer stats handling, shard edge-case fixes, and stricter request payload validation. - Updates deployment and dependency handling with network install retries and cross-platform pyproject/lockfile alignment. - Strengthens GPU and distributed test coverage while reducing unnecessary model-loading overhead in tests. Co-authored-by: jiajingc-nv <jiajingc@nvidia.com> Co-authored-by: Ryan Halabi <ryhalabi@nvidia.com> id: f5579024e56d9dd17153188da59dba76abac119c
…handling, and stats caching. A few commits cover security fixes, dependency/setup cleanup, deployment fixes, and broader test coverage. Co-authored-by: Jiajing Chen <jiajingc@nvidia.com> Co-authored-by: Ryan Halabi <ryhalabi@nvidia.com> id: a7b5ed3f5b6aaa8ef1dd75b1bf1643f639f0135b
…h greater than 0 (NVIDIA#54) - RoboCasa365 support Co-authored-by: Ryan Halabi <ryhalabi@nvidia.com> Co-authored-by: XAgents <service_account_group_331993_bdeca170e8aa5b0958079a733070defa@noreply.gitlab-master.nvidia.com> id: b72ae0ddb8f8d950c28f1fbbd20aa06eeb97ab02
…atform install scripts Co-authored-by: XAgents <service_account_group_331993_bdeca170e8aa5b0958079a733070defa@noreply.gitlab-master.nvidia.com> id: 68f8c79a695eaa66f60cb8cc8699a8f13804c6ac
- Finetuning Updates - Torch Beta Parameter stability fixes. Co-authored-by: Jiajing Chen <jiajingc@nvidia.com> Co-authored-by: Ryan Halabi <ryhalabi@nvidia.com> id: 5597264a9fc6f1b13fbed672128f7792302ba75b
…wires them into ONNX export, TRT build, verification, and benchmark CLIs. - Tightens TRT export/build behavior: BF16-only pipeline config, static batch-size docs, ViT exported/built as FP32 while the rest stays BF16, strongly-typed precision validation, per-component precision inference from ONNX names, and full-pipeline build failure when any ONNX input/engine is missing. - Refactors TRT runtime lifecycle: trt_torch.Engine now has explicit close() cleanup instead of an atexit hook. - Simplifies video decoding to torchcodec-only, removing ffmpeg/decord/opencv/pyav dispatch paths, video_backend config/CLI args, av dependencies, and related obsolete tests. - Reworks rollout video recording to stream frames through ffmpeg, stabilize caption buffer height, support explicit record_video_keys, and avoid shape changes during H.264 encoding. - Hardens eval wrappers: env-prefix-to-embodiment mapping now fails with actionable errors, and MultiStepWrapper uses a typed AggregateMethod with fail-fast validation. - Updates N1.7 model processing: letterbox padding is disabled by default and made optional, image transform inputs are validated more strictly, single-model tyro union parsing is fixed, and Qwen3-VL RoPE inv_freq buffers are recomputed after load for deterministic behavior. - Adds explicit PolicyServer / PolicyClient cleanup and context-manager support, including server startup usage in run_gr00t_server.py and the DROID example client. - Cleans up training/data code: hides modality configs from tyro CLI, removes dataset video_backend plumbing, deletes dead flowmatching/eval-accuracy code, and removes custom pytest log banners. - Updates Docker and platform deps: global image venv under /opt/gr00t-venv, aarch64 wheel bootstrap support, Thor wheel LFS tracking, TensorRT libs pinned to NVIDIA index, dev deps for pytest-xdist/coverage/licensing, and platform pyprojects without av. - Updates examples/docs: DROID video saving now uses imageio instead of moviepy, SimplerEnv metrics/docs are refreshed, and deployment docs remove video-backend options. - Expands tests around Qwen3 RoPE/no-weight loading, TRT precision and missing-input handling, TRT engine lifecycle, deployment mode consistency, multistep aggregation, policy service lifecycle, and torchcodec video behavior; removes obsolete PyAV/backend-dispatch tests. Co-authored-by: Jiajing Chen <jiajingc@nvidia.com> Co-authored-by: Ryan Halabi <ryhalabi@nvidia.com> id: f12a6f10fcb60bf317397fe363a50b077b88ae58
- Single-source policy/export action horizons and batch-size metadata across rollout, DROID, ONNX/TRT, and benchmark paths to prevent wrapper/policy drift. - Add common metrics and finetune utilities, including dataset power-law weighting, configurable preprocessing flags, cache locking, and LFS materialization helpers. - Improve sim-eval defaults and episode-length accounting with self-describing length units and final-info handling. - Flatten combined rollout videos and include frame counts in captions. - Preserve policy serialization compatibility while keeping unsafe object-dtype payload rejection and numpy language-batch support. Co-authored-by: Jiajing Chen <jiajingc@nvidia.com> Co-authored-by: Ryan Halabi <ryhalabi@nvidia.com> id: 9848681a45e47cbcafad1537dd6dcc1be0fc1906
…e with results and demo media. - Documents TensorRT export modes, force legacy ONNX export with dynamo=False, wire dit_only verification, and add regression tests. Co-authored-by: Jiajing Chen <jiajingc@nvidia.com> Co-authored-by: Xiaotong Chen <xiaotongc@nvidia.com> id: c9a51ad2ab05e4394df7823025a48fb5a227f89b
- Updated GR00T N1.7 public docs, README release notes, FAQ wording, setup guides, terminology, and references to the public GitHub repo. - Migrated deployment and simulation/evaluation environments to Python 3.12, Torch 2.9, and TorchCodec 0.8, with related dependency and setup fixes. - Hardened TensorRT deployment flows with export metadata validation, unified engine/output directories, clearer missing-engine errors, export-mode binding, inference-mode cleanup, and runtime-shape guards. - Improved evaluation and simulation behavior across action/execution horizons, embodiment tags, SimplerEnv success detection, rollout cleanup, ffmpeg cleanup, LIBERO setup, and RoboCasa eval settings and results. - Refactored shared data and utility paths, including DROID frame correction, `parse_observation_gr00t`, and `dist_utils`. - Improved model, policy, video, and experiment robustness with clearer gated-backbone errors, Qwen3-VL RoPE safeguards, ZMQ shutdown handling, WORLD_SIZE/GPU reconciliation, and distributed failure propagation. - Fixed notebooks and examples so GR00T inference and deployment workflows run end-to-end with current tags, TRT commands, and memory guidance. - Added N1.7 functionality and release content, including `letter_box_transform` support, GA messaging, RoboLab release updates, and combined rollout video handling. Co-authored-by: Jiajing Chen <jiajingc@nvidia.com> Co-authored-by: Ryan Halabi <ryhalabi@nvidia.com> Co-authored-by: Xiaotong Chen <xiaotongc@nvidia.com> id: cfdbf736958b9844a7fbb2a30802d042d0a7f253
Co-authored-by: Ryan Halabi <ryhalabi@nvidia.com> id: c4d6114b48f184769a7210962a2353c4582891fe
- Regenerate the bundled DROID sample statistics for the rebuilt 17-dimensional layout and add regression coverage for feature dimensions. - Update the RoboLab GA and SO100/SO101 setup, fine-tuning, evaluation, and deployment instructions. - Correct the Hugging Face authentication command and SO100 client dependencies. Co-authored-by: Jiajing Chen <jiajingc@nvidia.com> Co-authored-by: Shane Reetz <sreetz@nvidia.com> Co-authored-by: Xiaotong Chen <xiaotongc@nvidia.com> id: d7283d5ceb2e8f9efcb7f175b0f88b9f1797114b
Remove unnecessary NumPy/PIL round trips from VLM image preprocessing while preserving input parity. Add regression coverage for tensor and NumPy inputs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move subprocess-, model-, video-, and checkpoint-heavy CPU tests into the dedicated serial phase. This avoids resource contention from 32 concurrent xdist workers while preserving the same test coverage and assertions. Co-authored-by: Ryan Halabi <ryhalabi@nvidia.com> id: 7f0406ca11ca8c615e8ea11069dce0734abea2fc
* add:rebot_arm_dm * fix:rebot arm inference * Apply suggestion from @ryhalabi Co-authored-by: ryhalabi <ryhalabi@nvidia.com> * Fix: rebot_arm_dm * Update examples/rebot-arm-dm/README.md Co-authored-by: ryhalabi <ryhalabi@nvidia.com> * Update examples/rebot-arm-dm/README.md Co-authored-by: ryhalabi <ryhalabi@nvidia.com> * Update examples/rebot-arm-dm/pyproject.toml Co-authored-by: ryhalabi <ryhalabi@nvidia.com> * Update examples/rebot-arm-dm/eval_rebot_arm_dm.py Co-authored-by: ryhalabi <ryhalabi@nvidia.com> --------- Co-authored-by: ryhalabi <ryhalabi@nvidia.com>
Validate every embodiment against the processor's default maximum action horizon, while limiting the base-model action-horizon check to pretraining embodiments. Derive the processor limit directly from its constructor to avoid duplicating configuration values. Add the complete Apache 2.0 license header to the reBot Arm evaluation script to satisfy license CI. Co-authored-by: Jiajing Chen <jiajingc@nvidia.com> id: f34cc2c481378e705f612a1df095dae30f4a6795
Co-authored-by: Blake McHale <bmchale@nvidia.com> id: 1139039af77b8eda2f61c1c2cab9b8c9fe8defc8
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
This was referenced Sep 10, 2026
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 229f4bde40
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
17 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace the fork's N1.6 source and customizations with NVIDIA's GR00T N1.7 at
51d4c89f72fda44cbf77285c6a8114b52676b8a1.The resulting Git tree exactly matches NVIDIA's commit. There are no Positronic additions in this PR. The diff includes removal of the old N1.6 fork customizations; those removals are part of resetting to upstream, not NVIDIA-authored changes.
Our N1.7 changes are isolated in PR #11, with two commits rebased onto this branch. Review that PR for the 12 files we changed.
Merge this PR first using a merge commit to preserve NVIDIA's ancestry. Then retarget PR #11 from
nvidia-n1.7tomain-positronicand merge it. No default branch has been updated directly.Validation
51d4c89:43617c11aa6c4ea1be47594882d797255c1a575b.