Skip to content

Upgrade GR00T integration to native N1.7 DROID behavior - #733

Open
vertix wants to merge 14 commits into
mainfrom
groot-1.7
Open

Upgrade GR00T integration to native N1.7 DROID behavior#733
vertix wants to merge 14 commits into
mainfrom
groot-1.7

Conversation

@vertix

@vertix vertix commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Upgrade Positronic to the published nvidia/GR00T-N1.7-DROID checkpoint. Use native image preprocessing, DROID tool poses and rotation representation. The checkpoint processor owns relative pose/joint conversion; Positronic executes its absolute joint targets with DROID control settings.

The default accepts one exterior camera plus wrist. Conversion and fine-tuning also support two exterior cameras plus wrist, using the checkpoint's architecture and preprocessing. The server checks the codec's camera layout before warmup and bounds images before transmission. Action values and training metadata share one field order; shared metadata paths and wire names are explicit.

Docker builds the pinned GR00T fork at be79d6244dda302ace1ff7a2cd55239aad2ad109. NVIDIA's update is separate in merged PR #12. The model and Positronic use separate Python 3.12 environments. Legacy PhAIL and spoons deployments require an explicit N1.6 image and retain its original Python 3.13 startup command, including dependency synchronization; desktop PhAIL inherits the shared legacy service.

Nebius launchers preserve the image environment and provide the Hugging Face credential for the gated backbone. Uncharged inference collects its result in the same simulation tick, and chunk expiry uses integer nanoseconds to maintain continuous 15 Hz commands. Recording uses the storage mirror context; ZMQ transport failures reset the client socket.

Validation

  • Docker: 292 affected vendor/policy tests passed (4 skipped); 21 GR00T adapter tests passed after endpoint cleanup. Type checking, lint and formatting passed.
  • Fork Docker CPU suite: 681 passed, 21 skipped, 49 GPU/device tests deselected. Full suite repeated after minimizing the fork patch at ee9dd54; all 21 Positronic GR00T adapter tests passed against that source.
  • Docker Compose: every legacy GR00T entrypoint matches the pre-upgrade command exactly. Image pins, the desktop phail command and port 8000 were checked without starting deployments; an unset N1.6 image pin is rejected.
  • Code-rule checks passed without waivers for the legacy startup fix.
  • Fresh make -C docker build-groot and both environment smoke checks passed before the review fixes, at fork 506ccc87314fb69ad7ac8d8960b002ce3a378395. Review fixes used current source mounted into those Docker test environments; a full image rebuild was not repeated.
  • Nebius L40S recording review: five successful RoboLab episodes across two scenes, with 81 inference calls checked for pose/action parity. Commands run continuously at 15 Hz and chunks start every second. Results: s3://inference/tmp/groot-n1.7-robolab-20260909T180755Z/.

The GPU smoke used pre-polish revisions, two scenes at seed 0 and excluded inference wall time. GPU fine-tuning, three-camera checkpoint inference and real hardware were not tested.

The default frame transform matches Franka and RoboLab. The bundled MuJoCo Panda and legacy sim_stack_cubes recordings have a separate 45 mm frame mismatch tracked by #550. Their default GR00T E2E configuration is not native-checkpoint pose-parity validation; the limitation is documented and its review thread remains open.

Mac build verification: Docker pre-commit checks passed; Makefile dry runs in both repositories target linux/amd64, including with DOCKER_DEFAULT_PLATFORM=linux/arm64. The platform/documentation correction does not change Python source; the CPU results above remain from ee9dd54. No full image rebuild was repeated for this correction.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 10, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-11T11:12:07.569883Z c41da6f New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ebd57ec685

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

)
if self._is_hub_model:
self.resolve(model_id)
checkpoint_dir = self.checkpoints_dir

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Strip the hf:// prefix before launching GR00T

When serving the default published checkpoint, this passes hf://nvidia/GR00T-N1.7-DROID to the upstream --model_path argument. Hugging Face loaders accept a repository ID such as nvidia/GR00T-N1.7-DROID or a filesystem path, not Positronic's hf:// source marker, so the documented default server cannot load; strip the prefix when constructing checkpoint_dir.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping the hf:// marker here. The pinned fork explicitly handles it in Gr00tPolicy.init: snapshot_download receives the repository ID after removeprefix("hf://"), and the model and processor then load from the downloaded local directory. run_gr00t_server.py forwards this argument to that constructor unchanged.

Source: https://github.com/Positronic-Robotics/gr00t/blob/9d17a03fc4d46cc738fd65ca6705eacf3f3c3d6f/gr00t/policy/gr00t_policy.py#L107

The fork's test_policy_resolves_local_and_hub_checkpoints covers both local and hf:// paths and verifies the stripped download ID. It passed in the 679-test Docker CPU suite. Leaving this thread open for review because the finding does not apply to the fork Positronic pins.

Comment thread positronic/vendors/gr00t/codecs.py Outdated
Comment thread positronic/vendors/gr00t/server.py Outdated
Comment thread positronic/vendors/gr00t/codecs.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d7c07fc5c4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread positronic/vendors/gr00t/server.py Outdated
Comment thread positronic/vendors/gr00t/server.py Outdated
Comment thread positronic/vendors/gr00t/server.py Outdated
Comment thread positronic/vendors/gr00t/server.py Outdated
Comment thread positronic/vendors/gr00t/train.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f80e384c1e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

image_mappings={gr00t.EXTERIOR_IMAGE: keys.EXTERIOR_IMAGE, gr00t.WRIST_IMAGE: keys.WRIST_IMAGE},
fps=15.0,
execution_horizon=15,
ee_frame=models.DROID_EE_FRAME,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Use a simulator-correct DROID frame

When this codec is used by the documented default sim_stack_cubes workflow, ChangeEEFrame applies models.DROID_EE_FRAME to every inference pose and training label. That constant is explicitly documented in positronic/drivers/roboarm/models.py:26-29 as correct on the Franka rig but 45 mm off in simulation, so the GR00T E2E workflow trains and evaluates against displaced tool poses. Align the simulator's default frame first or select a simulator-specific transform instead of applying this default universally.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deferred to existing issue #550 (#550), with the limitation documented in 6302abd. This is valid for the bundled MuJoCo Panda and its legacy sim_stack_cubes recordings: its default frame differs by 45 mm from the Franka frame used to define DROID_EE_FRAME. The current GR00T/Nebius E2E harness must not be treated as native-checkpoint pose-parity validation for that simulator.

The requested checkpoint validation used RoboLab, whose adapter explicitly reports eef_pose * DROID_EE_FRAME.inv; the codec composes DROID_EE_FRAME back, matching the native RoboLab observation. Its flange-to-EEF geometry is separately checked by simulator/robolab/validate.py. The five recorded RoboLab runs are unaffected. Changing this default globally would break the real-rig/RoboLab path. Issue #550 already covers the frame migration and legacy dataset handling; that migration is outside this checkpoint upgrade. Leaving this thread open for human review.

Comment thread positronic/vendors/gr00t/codecs.py Outdated
Comment thread positronic/vendors/gr00t/codecs.py Outdated
Comment thread positronic/vendors/gr00t/codecs.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6302abd7cd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread positronic/vendors/gr00t/server.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f8f82e9c5f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread positronic/vendors/gr00t/server.py
@vertix

vertix commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

Please retry the review of 7e36bbd. The automatic review reported Failed without findings. All 17 CI checks passed; the desktop PhAIL compose fix was validated with Docker Compose.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7e36bbd39f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docker/docker-compose.phail.desktop.yml

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5a19de90d6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread positronic/vendors/gr00t/server.py Outdated
Comment thread positronic/vendors/gr00t/server.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant