Serve the RoboLab leaderboard pi05 through the env-server (droid_jointpos) - #486
Merged
Merged
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8049ca9e9b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First policy served against the merged RoboLab env (#469): openpi's DROID jointpos model — the same recipe RoboLab's leaderboard runs for its pi0-family entries.
droid_jointposcodec (vendors/openpi/codecs.py):droid_obs+ absolute joint-position actions, gripper binarized at 0.5, no horizon — the chunk length is the open-loop cadence for every pi0-family variant (pi05 = 15, pi0 = 10), matching their client'sopen_loop_horizon= modelaction_horizon. The full-chunk cadence itself comes from the validity sentinel Close action chunks with a validity sentinel in codec decode #488 added for Trajectories never state how long their last action lasts —ChunkedSchedulegives it zero time #487, so this PR carries no timing mechanism of its own.server.droid_jointpospreset: configpi05_droid_jointpos, checkpointgs://openpi-assets-simeval/pi05_droid_jointpos(thepolicies/pi0_family/README.mdrecipe). The config lives in our openpi fork: Addpi05_droid_jointposconfig serving absolute joint actions openpi#7.test_full_chunk_executes_between_replanspins the cadence contract over the full wire stack: every action of every chunk lands on the wire, replans exactlychunk_lenperiods apart.Differential eval
Same checkpoint, same serving process (our server hosts openpi's
serve_policy.pyon :8001 and speaks the positronic wire on :8000). Two harnesses, 10 episodes per task per arm, on one L40S (2026-07-19):positronic eval run→ env-server proxy → RoboLab envpolicies/pi0_family/run.py --policy pi05 --remote-port 8001, no positronic code in the loopPer-task rates match within n=10 noise (RoboLab has no seed hook, so both arms draw fresh randomized scenes). BananaInBowl our-arm was also 9/10 on 2026-07-07 with the pre-rebase tree.
The eval ran with the pre-#488 tree, where an interim
hold_lastguard produced the same full-chunk cadence the sentinel now provides universally; the wire test above proves the cadence is unchanged on the rebased tree.Test plan
pytestsuite green locally (605 passed / 7 skipped);test_remote_envpins the full-stack cadence.~/eval_out/2026-07-19/, RoboLab-side logs underoutput/differential_2026-07-19/.