[SaveAanchal]

Why

On the Mecka flagship recipe, Lance and zarr hold out different episodes, because Lance shuffles recording IDs and zarr shuffles episode hashes. The flagship's pinned seen-val video episode (692e771c) lands in train on Lance, so the run stops with a PinError. Episode order also differs, so the same global index gives a different frame on each format.

What changed

• Split and order: LanceEpisodeResolver.split_keys() maps each recording with exactly one aliased clip to its zarr episode_hash. MultiDataset shuffles and orders on those keys. Recordings with several clips or no alias still use their recording_id, and two recordings mapping to one key raise ValueError.
• Zarr: unchanged, because its resolver has no split_keys.
• Recipes: adds train_lance_mecka_flagship_6d_hpt, train_lance_mecka_flagship_6d_rdt and data/mecka_fold_flagship_lance_hpt_6d, with all video pins unchanged. Set EGOVERSE_LANCE_PILOT to the folder containing lance_v2, as in aria_pilot_lance.yaml.
• Prompt: the recipe sets annotations: {}, so the prompt is empty like zarr's.
• Checker: check_zarr_to_lance.py imported zarr_to_lance, which doesn't exist on this branch. It now uses steps_columns.

For you to decide

With annotations=None, LanceEpisodeResolver silently falls back to the task name as the prompt ("fold_clothes" here), while zarr gives an empty prompt on this data. I set annotations: {} in this recipe and didn't change the fallback. Other Lance recipes that leave it unset could hit the same mismatch. check_zarr_to_lance doesn't compare the annotation key, so it won't catch this.

Testing

Unit tests: tests/unit gives 1672 passed, 19 skipped, 1 failed. The failure is test_pi_loss_is_reduced_over_the_packed_width, which also fails on 853c113b4 here because openpi isn't installed. New tests in test_lance_dataset.py cover the split, ordering, fallback, duplicate-key error and the unchanged no-split_keys behavior.

Real flagship data (1,607 episodes on ICE), using the same split and order logic:

• Train (1,346 episodes, 271,247 frames) and seen-val (70 episodes, 13,744 frames) match zarr, in the same order.
• 692e771c is back in seen-val.
• Across 4,248 samples read through both loaders, actions and proprio differ by at most 1.5e-7, and prompts match. Images differ by about 0.9% mean absolute pixel value (AV1 vs JPEG).
• 128 of 128 sampled episodes passed check_zarr_to_lance.

Zarr vs Lance training (ICE, H100, same starting weights and batches at all 5,000 steps):

At step 5,000	HPT zarr	HPT Lance	RDT zarr	RDT Lance
Train loss, last 100 steps	0.1995	0.1997	0.0352	0.0355
Seen-operator validation loss	0.187	0.180	0.039	0.038
Unseen-operator validation loss	0.317	0.311	0.107	0.105
Seen-operator rotation error	29.7°	33.3°	17.3°	16.6°
Unseen-operator rotation error	37.9°	41.9°	31.1°	30.4°

• Train loss matches, and validation differences go both ways with no consistent format effect. The HPT rotation gap doesn't repeat on RDT and looks like noise, but this is one run per format, so it isn't proven.
• One checkpoint validated on both formats on a single H200: two zarr passes were identical, and Lance differed by a median of 0.23% and at most 0.64% across 51 metrics.
• RDT peaked at 84 GB of RAM on zarr and 77 GB on Lance at batch size 64.

Not tested:

• These runs used the same code from a scratch worktree on ICE. Compared with this branch, only the checker import, comments and lance_root differ (a fixed path there, EGOVERSE_LANCE_PILOT here), so the recipes haven't run from this exact checkout.
• The checker's new import was only confirmed to resolve locally, because skimage isn't installed here. The 128 checks ran with an aliased import of the same function.
• The comparison runs also used forced seeding and per-step tracing from that worktree, which aren't in this PR.

🤖 Generated with Claude Code