Add MolmoSpaces adapter under simulator/molmo_spaces

.github/workflows/unit-test.yaml

@@ -36,7 +36,7 @@ jobs:
           PYTHONPATH: ${{ env.PYTHONPATH }}:$PWD
         run: >-
           uv run pytest --cov-report=html
-          --override-ini='testpaths=pimm/tests positronic/cfg/tests positronic/dataset/tests positronic/geom/tests positronic/offboard/tests positronic/policy/tests positronic/tests positronic/utils/tests'
+          --override-ini='testpaths=pimm/tests positronic/cfg/tests positronic/dataset/tests positronic/geom/tests positronic/offboard/tests positronic/policy/tests positronic/simulator/molmo_spaces/tests positronic/tests positronic/utils/tests'
 
       - name: Coverage summary (job summary)
         if: always()

CLAUDE.md

@@ -75,4 +75,6 @@
 # Infrastructure
 - Machines, Docker contexts and images: `docker/CONTEXTS.md`
 - Model-specific workflows: `positronic/vendors/{lerobot,gr00t,openpi}/README.md`
+- Inference serving, and the adapter/codec/wire-client separation of responsibilities (read BEFORE
+  writing a sim/rig adapter): `positronic/offboard/README.md`
 - Reconstructing previous runs: read `run_metadata_*.yaml` and episode `static.json` from output directory

positronic/offboard/README.md

@@ -2,6 +2,34 @@
 
 This package implements the protocol and utilities for offboard policy inference, allowing robots or simulators to stream observations to a remote server and receive actions.
 
+## Separation of responsibilities: adapter vs codec vs wire client
+
+Three layers touch an observation on its way to a model, and each owns exactly one concern.
+When writing a new sim/rig adapter, check this table before adding any transform to it:
+
+| Layer | Owns | Examples |
+|---|---|---|
+| **Adapter** (per sim/rig, e.g. `simulator/molmo_spaces/adapter.py`) | Rig semantics ONLY: mapping the rig's observation/action vocabulary onto positronic's raw keys | Camera-key mapping, gripper qpos → `[0, 1]` closure, decoded commands → the rig's action format |
+| **Codec** (per model family, `policy/codec.py` subclasses) | Model preprocessing: everything the checkpoint's input distribution requires | Resize-with-pad to model resolution, prompt normalization (e.g. DROID lowercasing), state assembly |
+| **Wire client** (`InferenceClient` / `RemotePolicy`) | Transport optimization, negotiated — never semantics | Downscaling frames to the server-advertised `image_sizes` (aspect-preserving, never upscaling), optional JPEG compression |
+
+Consequences:
+
+- **An adapter never resizes, pads, normalizes prompts, or otherwise preprocesses for the model.**
+  It passes frames and text through at native fidelity. If the same transform appears in an adapter
+  and a codec, the adapter's copy is the bug: a drifted duplicate silently changes eval inputs.
+- **Bandwidth is not the adapter's problem.** The client already downsizes to what the server says
+  it needs: every `Codec` advertises its expected input sizes via the reserved `image_sizes` meta
+  key (see `Codec.meta`), the server returns it in the session handshake, and the client fits
+  frames to it before sending. This is default-on — an adapter that resizes "to keep the wire
+  payload small" is duplicating it.
+- **Codecs run on either side of the wire.** positronic-native evals compose the codec around
+  `RemotePolicy` on the client (`cfg/policy.py` — the wire then carries model-sized encoded inputs,
+  and the client-side resize is disabled since `codec.meta` already reports `image_sizes`).
+  Thin-client deployments (a sim adapter in a foreign venv talking to a serverless endpoint) host
+  the codec on the server — the wire carries raw positronic keys, downsized by the negotiation
+  above. Both placements are supported; pick by where the dependencies can live.
+
 ## Protocol v1
 
 The unified WebSocket protocol is built to enable ANY hardware to connect to ANY model. All Positronic inference servers (LeRobot, GR00T, OpenPI) implement this protocol, allowing a single `.remote` policy client to work across all vendors.

positronic/policy/observation.py

@@ -18,14 +18,21 @@ class ObservationCodec(Codec):
         state: mapping from output state key to an ordered dict of {episode_key: dim} to concatenate.
         images: mapping from output image name to tuple (input_key, (width, height)).
         task_field: output key carrying the language prompt at inference; LeRobot training always uses ``task``.
+        lowercase_task: lowercase the task text at inference, for checkpoints trained on lowercased language
+            (the pretrained DROID models; MolmoSpaces' Pi baseline applies the same normalization).
     """
 
     def __init__(
-        self, state: dict[str, dict[str, int]], images: dict[str, tuple[str, tuple[int, int]]], task_field: str = 'task'
+        self,
+        state: dict[str, dict[str, int]],
+        images: dict[str, tuple[str, tuple[int, int]]],
+        task_field: str = 'task',
+        lowercase_task: bool = False,
     ):
         self._state = state
         self._image_configs = images
         self._task_field = task_field
+        self._lowercase_task = lowercase_task
 
         self._derive_transforms = {k: partial(self._derive_state, k) for k in state.keys()}
         self._derive_transforms.update({k: partial(self._derive_image, k) for k in images.keys()})
@@ -54,7 +61,8 @@ def encode(self, inputs: dict[str, Any]) -> dict[str, Any]:
         obs: dict[str, Any] = {}
 
         if 'task' in inputs:
-            obs[self._task_field] = inputs['task']
+            task = inputs['task']
+            obs[self._task_field] = task.lower() if self._lowercase_task else task
 
         for out_name, (input_key, (width, height)) in self._image_configs.items():
             if input_key not in inputs:

positronic/policy/tests/test_policy_io.py

@@ -59,12 +59,17 @@ def test_observation_encode_missing_state_inputs_raise():
 
 def test_observation_encode_task():
     enc = ObservationCodec(state={'observation.state': ['a']}, images={})
-    obs = enc.encode({'a': 1.0, 'task': 'test_task'})
-    assert obs['task'] == 'test_task'
+    obs = enc.encode({'a': 1.0, 'task': 'Test_Task'})
+    assert obs['task'] == 'Test_Task'  # untouched by default
 
     obs_no_task = enc.encode({'a': 1.0})
     assert 'task' not in obs_no_task
 
+    # DROID-style configs lowercase the prompt: those checkpoints were trained on lowercased language,
+    # so capitalized benchmark task text must not reach them mixed-case.
+    lower = ObservationCodec(state={'observation.state': ['a']}, images={}, lowercase_task=True)
+    assert lower.encode({'a': 1.0, 'task': 'Pick up the Cube'})['task'] == 'pick up the cube'
+
 
 def test_absolute_position_action_encode_decode_quat():
     # Identity rotation, known translation/grip

positronic/simulator/libero/make_fixture.py → positronic/simulator/libero/tests/make_fixture.py

@@ -10,7 +10,7 @@
 The e2e replay needs only each demo's action sequence and its initial full state — a few KB per episode, not the
 multi-GB benchmark. Run once on a box that has the demos, then commit the ``.npz`` next to the test::
 
-    uv run --no-project positronic/simulator/libero/make_fixture.py \
+    uv run --no-project positronic/simulator/libero/tests/make_fixture.py \
         --demo-path "$LIBERO_DATASETS/libero_spatial/<task>_demo.hdf5" \
         --out positronic/simulator/libero/tests/libero_spatial_task0.npz
 """

positronic/simulator/libero/tests/test_e2e.py

@@ -7,7 +7,7 @@
 
 The fixture is generated once on a LIBERO box::
 
-    uv run --no-project positronic/simulator/libero/make_fixture.py \
+    uv run --no-project positronic/simulator/libero/tests/make_fixture.py \
         --demo-path "$LIBERO_DATASETS/libero_spatial/<task>_demo.hdf5" \
         --out positronic/simulator/libero/tests/libero_spatial_task0.npz