Skip to content

Move Franka's default frame to the flange and migrate existing datasets #550

Description

@vertix

Follow-up to #485 and #483.

Context

Under the frame contract, every embodiment's model defines a frame named default, the driver publishes poses in it, and every transform is computed against it. default belongs at the flange whenever the gripper can be changed, because recorded data outlives grippers: an episode anchored to a gripper frame becomes unplaceable the moment that gripper comes off, while one anchored to the flange stays valid forever.

Franka is exactly that case — the arm carries a Robotiq 2F-85 today, and default currently sits 103.4 mm out and 45° round from the flange, at a point that belongs to a Franka Hand we don't own.

Nothing is broken today. Each rig is self-consistent: sim data was recorded at the sim's EE point and is served back to it, real data likewise, and with no frame conversion in play the two never meet. What the current placement costs is portability — a checkpoint states its end-effector frame as a transform from default, and that sentence only means the same thing on two rigs if their defaults are in the same physical place. They are not:

fr3    link7 -> default:  xyz [0, 0, 0.2104]
panda  link7 -> default:  xyz [0, 0, 0.1654]

45 mm apart. So cfg.codecs.DROID_EE_FRAME, measured on the FR3, is right on the real arm and 45 mm off in sim. Moving both to the flange is what makes one checkpoint's declaration portable.

The risk this issue must manage: relabeling, not conversion

The recorded poses are correct in the frame they were recorded in. The danger is that moving default changes what the label means while the numbers stay put. Whether an episode survives depends on whether it names its own frame or inherits one — and real and sim differ, because of transform ordering in positronic/cfg/ds/internal.py:56-59:

REAL_ROBOT_TRANSFORM = Group(_ROBOT_SIGNAL_POINTERS, Identity(...), _REAL_MODEL, _RENAME_ROBOT_COMMAND)
SIM_ROBOT_TRANSFORM  = Group(_ROBOT_SIGNAL_POINTERS, _SIM_MODEL, Identity(...), _RENAME_ROBOT_COMMAND)
  • Real data is safe. Identity precedes _REAL_MODEL, so the episode's own control_frame wins and the bundled model only backfills. Old recordings say end_effector; Serve checkpoints in their own EE frame, declared by the server #485 added default alongside that link rather than renaming it, so the name still resolves to the same spot after the move. No pose conversion needed.
  • Sim data is not. _SIM_MODEL precedes Identity, so the bundled panda overwrites control_frame. It is default today. The moment default moves to the flange, every existing sim episode silently claims its poses are flange poses — off by however far the sim's EE point sits from its flange.

Once ee_frame is the only statement of where poses sit, this whole failure mode goes away: the value is anchored on default rather than on a name that can be overwritten, so a moved default cannot silently relabel an episode. Until then the fallback's assert is what catches it.

So the migration is a config change, not a rewrite of s3://PUBLIC@positronic-public/... (public, referenced from workflows/nebius/README.md). Pin the legacy sim datasets to the frame they were recorded in instead of letting the moved model relabel them.

Work items

  • Move Franka's default to the flange. positronic-franka must publish O_T_F (= O_T_EE · F_T_EE⁻¹, both already in the RobotState we read) and robot.hpp's IK must solve for the flange rather than F_T_EE. Not a relabel — fr3.urdf was generated on the arm, so its end_effector link is a recording of Desk's tool configuration:
    link8 -> end_effector:  xyz [0, 0, 0.1034]   rot -45° about Z
    
  • Move the sim Franka's default to its flange. bundled_panda_model()'s links end link6, link7, hand, left_finger, right_finger, end_effector — there is no link8, so the flange has to be identified rather than renamed.
  • Stop SIM_ROBOT_TRANSFORM relabeling legacy sim episodes (sim_stack_cubes, sim_pick_place). Either drop the overwrite or inject the frame those episodes were recorded in explicitly.
  • Confirm real datasets need nothing (phail). Expected to be a no-op given the ordering above — verify, don't assume.
  • Sunset control_frame. Serve checkpoints in their own EE frame, declared by the server #485 replaces it with ee_frame, a transform from the model's default to where the episode's poses actually are — one self-describing value instead of a site name plus an implied offset. control_frame survives there only as a legacy read path: ik.pose_anchor() falls back to it when an episode carries no ee_frame, and IK no longer uses it to choose a site. Once every dataset in use carries ee_frame, delete the fallback, the key, and the field from every robot_meta producer (franka.py, yam.py, so101/driver.py, models.py, the RoboLab launcher).
  • Delete Harness._assert_control_frame (positronic/policy/harness.py, carries the TODO(#550)). It is the only frame knowledge the harness holds, and both halves expire with the field: the control_frame == default half has nothing left to compare, and nothing at serving reads the model, so "this model declares default" belongs to whoever builds the model — asserted once at construction in each driver and in the RoboLab launcher, not re-checked on every observation by the thing running the policy.
  • Recompute cfg.codecs.DROID_EE_FRAME (positronic/drivers/roboarm/models.py). With default at the flange it becomes the authored 2F-85 geometry, xyz [0, 0, 0.018174023] and +90° about Z, instead of today's xyz [0, 0, -0.085225977] at 135°, which is the F_T_EE offset riding along. test_declared_droid_frame_matches_the_model_geometry pins the constant to the model, so it fails loudly when the move lands — that test is the reminder.
  • Update fine-tuning instructions and inference servers. Any checkpoint trained before the move speaks the old frame and must declare it: vendors/openpi/server.py (pipeline(ee_frame=)), cfg/codecs.py (compose(ee_frame=)), the vendor READMEs, workflows/nebius/README.md, docs/training-workflow.md. Note ee_frame takes a Transform3D now, not a frame name.
  • Update the frame contract doc (positronic/drivers/roboarm/README.md) — the per-rig placement table and the portability caveat above it — and drop the TODO(#550) in models.py.

Cross-environment migration and acceptance criteria

  • Update RoboLab, LIBERO (LIBERO reports its eef 38mm and 90° from where the shipped panda model says default is #557), and MolmoSpaces (Add the MolmoSpaces env-server integration #504) alongside the physical Franka and bundled MuJoCo simulator. Each adapter must convert between its native observation/control points and the shared flange reference using geometry from its actual environment model.
  • Make robot-base versus simulator-world coordinates explicit. Verify pose and Cartesian command conversions with the robot translated and rotated in the scene, as well as at several arm orientations.
  • Preserve each existing checkpoint's native inputs and command behavior through the migration. Coordinate adapter and policy-codec changes, including GR00T DROID and OpenPI LIBERO, and compare them with the native implementations. Recordings must retain the reference point and coordinate system their numbers describe; do not silently relabel old data.
  • Validate the declared transforms independently against each running simulator's geometry. Command/observation round trips alone are insufficient because matching conversion errors can cancel.

Depends on

The frame contract from #483default as the named anchor, ChangeEEFrame taking a transform, and the frame travelling with the data. This issue is the Franka-specific migration that follows it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions