Per-policy EE control frames: make checkpoints transferable across rigs with different frame conventions

[vertix]

Problem

"EE pose" has no universal meaning, and the disagreement is invisible until a policy misbehaves.

• Our convention: RobotState.ee_pose and robot_command.pose are the flange frame — control_frame='end_effector' in bundled_franka_model() (positronic/drivers/roboarm/models.py) and the real driver (positronic/drivers/roboarm/franka.py).
• DROID / RoboLab convention: the EE frame is the gripper base (Robotiq_2F_85/base_link) composed with a fixed rotation offset (EEF_OFFSET_ROT = (0.5, -0.5, 0.5, -0.5) in RoboLab's robolab/robots/droid.py). Between our flange and that frame sits the gripper mount (+45° about Z in our model) plus that offset — one constant rigid transform, but a large one.

Corrected 2026-08-04, on closing this via #485. As filed, this section led with the DROID-checkpoint case
and listed the RoboLab mislabeling as a secondary consequence. That was backwards. We serve no EE-space DROID
checkpoint — droid and droid_jointpos are both joint-space — so that case was never live, while #469
shipped the mislabeled robot_meta knowingly, behind a HACK pointing here. The two are swapped below.
Requirement 2 also turned out to be unachievable; see the comment thread.

The live bug: RoboLab episodes are recorded with the wrong frame. RobolabEnv._observe records robot_state.ee_pose from RoboLab's eef_pos/eef_quat — the DROID gripper frame — while the bundled_franka_model() serialized alongside it claims control_frame='end_effector' (flange). Every episode's statics therefore mislabel its poses by that ~10 cm transform, so offline IK over RoboLab episodes (ik.ik_joints_from_episode, IKJointsAction) solves the wrong target and the viewer's pose overlays are offset. Flagged by Codex on #469: #469 (comment) — #469 merged anyway, carrying a HACK at the mismatch site that points here.

The anticipated bug: EE-space DROID checkpoints on our Franka arm. Run one on our rig and every pose is misinterpreted twice: the policy reads our flange pose as if it were the DROID gripper frame, and its commands are executed as flange targets. Both directions are off by that constant transform, and nothing errors — the policy just behaves badly. No such checkpoint is deployed today (joint-space DROID policies, including the pi05_droid_jointpos leaderboard path, are immune because joints are unambiguous), so this is what the design must not preclude rather than something currently broken.

Also shaping the design:

1. We can't just redefine FK/IK globally. A customer trained a checkpoint on the phail dataset under the current flange convention; existing recordings and checkpoints must keep their meaning bit-for-bit.
2. Manual conversion is two moving pieces. Even when a conversion codec is wired by hand at serving time, correctness rests on two hand-maintained configs — training-side and serving-side — agreeing about which frame the checkpoint speaks. Forgetting one is silent.

This is the observation/command sibling of the control-law parity work in Positronic-Robotics/positronic-franka#8: same goal — our rig, RoboLab, and DROID should execute a policy identically — split across "how commands are tracked" (there) and "which frame poses live in" (here).

Visual explainer of the frame mismatch: https://claude.ai/code/artifact/9d580a55-decd-46c3-94b9-f9cade9051cf

Requirements

We want policies to be transferable: trained on data in one frame, deployable on any rig, with the system — not the operator — keeping the frames straight.

1. Transferability — a checkpoint trained in frame F runs correctly on any rig whose robot model defines F, regardless of that rig's canonical frame.
2. One owner for the frame — the checkpoint itself declares what frame it speaks. Nobody re-states it at serving time; there is no second config to forget. Not achievable, established while closing this. Most checkpoints we serve are third-party gs:// dirs (pi05_droid, pi05_droid_jointpos, pi05_libero) that will never carry a frame, and we do not own the training script that writes our own. The frame is knowledge about a checkpoint that lives outside it. Serve checkpoints in their own EE frame, declared by the server #485 meets the intent by another route: the frame is stated once, in the deployment entry that already names the checkpoint, and Pipeline.__init__ refuses a pipeline whose components declare it more than once.
3. Loud, early failure — a frame the rig doesn't know fails at session open / first observation, never mid-motion, never silently.
4. Training conversion — a dataset recorded in the rig's canonical frame can be re-expressed in the policy's frame at training time, and that choice is what stamps the checkpoint's frame.
5. Exactness for every action type — observations and actions convert exactly: absolute poses, relative actions, and driver-anchored deltas (CartesianDelta). Delta translations don't convert pose-independently, so conversion must anchor where the true pose is known — for CartesianDelta that pose exists only at the driver, at execution time.
6. Zero impact when unused — no frame declared → bit-for-bit today's behavior, for old checkpoints and old datasets.

Constraints

1. Harness stays stupid — no conversion logic in the harness; at most its existing plumbing of robot_meta into obs.
2. Server: no per-session state, no heavy per-request payloads — rig knowledge on the server means either session state (kills statelessness) or shipping the rig model each request (traffic). A tiny per-request item — e.g. the 7-number transform T — is acceptable.
3. One conversion implementation, ideally one integration point — the math (frame_transform, pose·T) lives in one module regardless; calling it from one place beats calling it from N drivers, but N is acceptable if the design wins elsewhere.
4. Frame registry is the robot model — frame names resolve against the rig's URDF sites (droid_eef is just a site); no parallel enum, registry, or config schema.

Deliberately not constraints: drivers may be frame-aware (they hold the one unique asset — the execution-time pose — enabling exact delta conversion), and where conversion runs is an open design axis: client-side codec, driver-side, or server-side applying a client-computed T.

Ruled out: manually wired serving-time codec

The original design here — ChangeEEFrame(to=) composed by hand into the serving config (remote(codec=change_ee_frame.override(to='droid_eef'))) — converts correctly but keeps the frame stated in two independent places with nothing enforcing agreement. That is exactly failure mode 3 above, so it fails requirement 2. Its building blocks survive as input to the design: the codec's conversion math, the droid_eef site, per-episode T from episode statics, and URDF-as-frame-registry.

Correction (2026-08-04). Requirement 2 is unachievable, so this reasoning does not hold, and #485 ships close to what this section rules out: the frame is stated in the serving config. Two things changed. It is a transform rather than a frame name, so the rig's model no longer has to cross the wire for it to mean anything; and it can only be stated once — Pipeline.__init__ refuses a pipeline whose components declare ee_frame more than once, and ChangeEEFrame's training encoder refuses a dataset another codec already moved. What stays unenforceable is agreement with how the checkpoint was actually trained, which for a third-party checkpoint is knowledge from a paper, not from any artifact we hold. #558 carries the remaining option: drop the ee_frame=None default so a deployment cannot silently omit it.

Work items

• Design the conversion placement — a client-side codec carrying a transform, not a frame name. Every embodiment anchors on a frame called default; ChangeEEFrame(T) states the policy's frame relative to it.
• Add the droid_eef site to the 2F-85 graft, validated against RoboLab's eef_frame — measured on the running RoboLab scene at 0.000037 mm / 0.0000°, rigid across four arm configurations.
• Training-side conversion that stamps the checkpoint's frame declaration — ChangeEEFrame.training_encoder re-expresses the episode's poses and writes ee_frame into its statics.
• Frame audit — moved to Frame audit: declare the EE frame every checkpoint we serve speaks #558. Answered there: phail and sim_stack sit in their rig's default (collected through data_collection.py, no conversion applied), both DROID deployments are joint-space and need none, lerobot/gr00t serve only phail checkpoints. libero is unanswered and blocked on LIBERO reports its eef 38mm and 90° from where the shipped panda model says default is #557.
• Flip the RoboLab launcher's robot_meta; remove the Add RoboLab (Isaac Lab DROID benchmark) behind the env-server #469 HACK and resolve the Codex thread — RobolabAdapter converts at models.DROID_EE_FRAME so the env reports at default, and robolab/validate.py checks that conversion against RoboLab's own scene.
• End-to-end validation against RoboLab: _check_flange_to_eef in robolab/validate.py compares the declared constant against panda_link8 → eef_frame as the running env reports it.

[vertix]

MolmoSpaces instance of this frame problem

Adding the MolmoSpaces case, now that #504 — Add the MolmoSpaces env-server integration is up.

Where the pose is read. env.py's observation path reports robot_state.ee_pose at the arm move-group's grasp site, not the flange:

arm = self._robot_view.get_move_group('arm')
eef_world = np.asarray(arm.leaf_frame_to_world, dtype=np.float64)  # 4x4 grasp-site world transform

For the DROID rig, MolmoSpaces builds that arm move group (FrankaFR3ArmGroup) with grasp_site_name="gripper/grasp_site" (molmo_spaces/robots/robot_views/franka_droid_view.py), so leaf_frame_to_world resolves the gripper/grasp_site site of the attached Robotiq 2F-85 — the TCP between the fingers.

What that frame is, measured. Loaded robots/franka_droid/model.xml @ allenai/molmospaces@c2f1b58 in MuJoCo and measured gripper/grasp_site relative to the flange (positronic link8 = molmo's attachment_site; both are link7 + (0 0 0.107) with identity rotation): (0, 0, 0.155) m, identity rotation, constant across joint configs (rigid).

Distinct from droid_eef. It is not the gripper-base frame droid_eef targets (18.17 mm Z, +90° Z) — it sits 155 mm out along the flange Z with no rotation. Δ 136.8 mm translation / 90° rotation. So the molmo frame needs its own site, not a reuse of droid_eef.

The mislabel. #504 serves static_meta=bundled_franka_model(), whose control_frame is end_effector (the flange), while the env reports poses at the grasp site — the exact secondary consequence "the RoboLab integration (#469) ships a mislabeled robot_meta" from this issue, now recurring in MolmoSpaces. It does not affect the joint-space pi05 DROID eval (joint commands + privileged success), but recorded episode statics mislabel the pose frame and offline IK over them solves the wrong target; the 3D viewer draws it wrong.

Fix — the MolmoSpaces sibling of the "flip the RoboLab launcher's robot_meta to declare droid_eef; remove the HACK" work item: #507 — Declare MolmoSpaces' grasp frame as the eval control frame (molmo_grasp), stacked on #504 and #485. It adds a molmo_grasp site (validated at 0.000 mm / 0 quat against MolmoSpaces' own model) and declares it as the served control frame, removing the HACK.

[vertix]

Note on requirement 2, for anyone reading this after it closes.

Requirement 2 asks that "the checkpoint itself declares what frame it speaks. Nobody re-states it at serving time; there is no second config to forget." The first half is not achievable and will not become achievable: most of what we serve is third-party (pi05_droid, pi05_droid_jointpos, pi05_libero, all published gs:// dirs) and will never carry a frame, and we do not own the training script that writes our own checkpoints. The frame is knowledge about a checkpoint that lives outside it.

What #485 delivers instead meets the intent by a different route:

• The frame is stated once, in the deployment entry that already names the checkpoint — one override block, so the two cannot drift apart.
• Pipeline.__init__ refuses a pipeline whose components declare ee_frame more than once, so stating it on both sides of the wire fails at config time rather than doubling the conversion silently.
• ChangeEEFrame's training encoder refuses a dataset that another codec already moved.

What remains uncovered is narrower than the failure mode this issue describes: nothing verifies that a stated frame matches how the checkpoint was actually trained. For a third-party checkpoint that is irreducible — the answer comes from a paper, not from an artifact we hold. #558 carries the remaining option, which is to make omission impossible (drop the ee_frame=None default so every deployment states its frame) rather than to make disagreement detectable.