Skip to content

Trajectories never state how long their last action lasts — ChunkedSchedule gives it zero time #487

Description

@vertix

Problem

ChunkedSchedule treats a trajectory as ending at its last action's timestamp:

# positronic/policy/wrappers.py — _Session.__call__
self._trajectory_end = result[-1]['timestamp'] if result else None

Re-inference triggers when now() >= trajectory_end — the exact instant the last action is due. The new chunk is anchored at that instant, and command channels are last-value-wins: TrajectoryPlayer.set() (drivers/roboarm/command.py) replaces the current trajectory wholesale. In zero-latency lockstep sim the replacement lands before the player runs the last waypoint, so a K-action chunk executes K−1 steps and spans K−1 control periods.

The reference implementations this pipeline mirrors (RoboLab's pi0_family client, openpi's clients) are index-pull loops: action = chunk[i], one per control step, re-infer when the pointer runs off the end. Every action gets exactly one period; the next obs is taken after the last step. Our scheduler is off by one against that: the trajectory never says how long its last action lasts, and the scheduler assumes zero.

On real hardware the bug is masked, not absent: the driver plays the last waypoint during the inference-latency window, so it gets ~latency instead of one full period.

Evidence

  • droid runs a 7-step cadence; the reference runs 8. ActionHorizon(8/15) keeps actions at 0..7dt (ts < horizon, codec.py), trajectory_end = 7dt, replan at 7dt → 7 executed per cycle in lockstep. Masked by delta-action robustness.
  • droid_jointpos (cadence = chunk length) made it visible: chunks executed 14 of 15 steps — a cadence the model never saw in training. Serve the RoboLab leaderboard pi05 through the env-server (droid_jointpos) #486 ships the hold_last workaround and validates it differentially against RoboLab's own client (per-task success rates match; see the PR table).

Design: the trajectory states its own validity, via a timestamp-only sentinel

The scheduler must not guess the period: action_fps meta is not guaranteed (no-codec/server-stamped paths), and trailing waypoint spacing silently guesses on single-action or non-uniform chunks. Instead, the codecs that own time semantics append a sentinel entry — a dict with only a timestamp, no command keys — marking when the chunk's validity ends:

  • ActionTimestamp appends {'timestamp': K * dt} after stamping a K-action chunk: "these actions cover K periods."
  • ActionHorizon — when it truncates — appends {'timestamp': horizon_sec}: the boundary it cut at is where validity ends. When nothing is truncated it appends nothing (the inner sentinel already ends the chunk).

Why this encoding works with zero changes elsewhere:

  • ChunkedSchedule is untouched: trajectory_end = result[-1]['timestamp'] becomes correct as written, because the last entry now is the validity end. Its normalization ({**r, 'timestamp': now + r.get('timestamp', 0.0)}) handles a keyless dict as-is.
  • The demux filters by key — [(ts, a[name]) for a in actions if name in a] (harness._emit_commands) — so the sentinel reaches no channel: drivers never see it, recordings gain no duplicate waypoints.
  • Why not a copy of the last action (what Serve the RoboLab leaderboard pi05 through the env-server (droid_jointpos) #486's hold_last does): a copy is only inert for absolute commands. TrajectoryPlayer deliberately accumulates due deltas (command.reduce), so a copied JointDelta double-applies — a universal copy would fix jointpos and corrupt droid. The sentinel is inert for every command type by construction.
  • Why not a Hold command type: every driver would need a new vocabulary word; the sentinel needs none.
  • Paths with no time codec (servers that stamp/truncate themselves) keep today's behavior exactly — nothing guesses on their behalf.

Caveat to verify: decode ordering in cfg/codecs.py::compose. The sentinel must be appended at a point where no later-decoding codec indexes entries by key (e.g. grip binarization doing a['grip']). Check the actual decode direction of the | chain; if some codec decodes after ActionTimestamp, either make it skip keyless entries or append the sentinel in the outermost time codec.

Open decision (small): whether ActionTimestamp appends the sentinel for single-action lists too (uniform semantics: one action = one period of validity) or leaves them reactive as today. The bare-dict (non-list) path stays untouched either way.

Ordering with #486

This issue lands first, on main — main has no hold_last, so nothing here removes it. After this merges, #486 rebases and:

  • drops the hold_last flag from ActionTimestamp, the compose parameter, and droid_jointpos's hold_last=True — the universal sentinel now provides the held period;
  • keeps test_hold_last_runs_full_chunk_between_replans as the acceptance test: its assertions state the reference contract (every action of every chunk lands on the wire, replans exactly chunk_len periods apart) and must pass with the flag gone. Rename it to describe the contract (full-chunk cadence), not the removed mechanism.

Consequences checklist

  • ActionTimestamp.decode: append the sentinel (list path; decide the single-action question above).
  • ActionHorizon.decode: append boundary sentinel when truncating.
  • ChunkedSchedule, TrajectoryPlayer, _emit_commands: no changes — verify, don't edit.
  • Codec unit tests (test_policy_io.py): expectations gain the sentinel entry.
  • TestChunkedSchedule (test_wrappers.py): scheduler semantics unchanged; tests feeding raw chunks without sentinels still describe valid (self-terminating-at-last-ts) trajectories and should pass — verify.
  • Golden pipeline (test_golden_pipeline.py): locks current timing; will fail by design. Review the diff — every chunk cycle should lengthen by exactly one period (its horizon setup goes 7→8 periods + latency) — then regenerate: GOLDEN=1 uv run pytest positronic/policy/tests/test_golden_pipeline.py -p no:cacheprovider -o "addopts=".
  • Cadence shifts one period for every deployed codec — droid 7→8 (now matching the reference), phail (customer checkpoint), sim_stack, libero. Intended, but don't compare new eval numbers against old ones blindly.

Validation

References

  • Serve the RoboLab leaderboard pi05 through the env-server (droid_jointpos) #486hold_last workaround + differential eval (superseded by the sentinel on rebase)
  • positronic/policy/wrappers.py ChunkedSchedule._Session — the fencepost (stays as-is)
  • positronic/drivers/roboarm/command.py TrajectoryPlayer — replacement + delta accumulation (why a copy can't be the universal hold)
  • positronic/policy/codec.py ActionTimestamp, ActionHorizon — where the sentinel lives
  • positronic/policy/harness.py _emit_commands — key-filtered demux (why the sentinel is driver-invisible)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions