Inspect final-motion failures and keep released stress packs verifiable

CHANGELOG.md

@@ -1,25 +1,32 @@
 # Changelog
 
-## 0.11.0 — 2026-09-14
-
-- Add Scene Lab: CC0 captured geometry, 25,000 mesh-derived surface Gaussians, separate collision heightfield, two actual L40S OptiX renders, bounded edit recipes and independent native Blender readback.
-- Add the official LIBERO-Plus adapter and source-linked SmolVLA replay: paired baseline, camera viewpoint ID 609 and lighting ID 2124. Outcomes are success at 77 actions, step limit at 220, and success at 87. This is a diagnostic subset, not a benchmark score.
-- Add an OpenEnv 0.4.2 verifiable creation environment. Numeric actions build and independently reopen a native scene; completion claims do not earn reward.
-- Add a pinned Cosmos Policy recorder/access preflight. Actual Cosmos inference remains unavailable until the account receives access to its gated NVIDIA dependency. No prediction results are claimed.
-- Link all 27 cross-project skill trials from the homepage and Hugging Face Space; improve mobile layouts, video seeking and lazy Gaussian loading.
+## Unreleased
 
+## 0.12.0 — 2026-09-14
 
-## Unreleased
+- Inspect failure classifications in Stress Lab and jump to each recorded final motion window without dropping experiment totals. Show measured travel in millimetres and keep causal limits explicit.
+- Accept original released stress packs that predate `reliability.json`; new packs still seal and independently recompute it.
+- Reject identical-input claims when a repeat lacks consumed frames or both records lack camera hashes.
 
-- Say what a failure was, not only that it happened: split step-limit failures by whether the arm was still moving at the cut-off. All 14 recorded failures were still in motion, so the 160-action budget is binding on the reported success rate rather than the policy having given up.
-- Measure whether a repeat of the identical plan agrees, on the same L40S, at every level: 30/30 outcomes, action counts, bitwise physics states and actions, and 360/360 renders on the frames a policy call consumed. Report renders on consumed frames apart from recorded-only frames, because only the former can change an outcome. One recorded-only frame of 3,195 differed and did not propagate.
+- Say what a failure was, not only that it happened: split step-limit failures by whether the arm was still moving at the cut-off. All 14 recorded failures were still in motion, under the stated motion threshold; continued movement does not establish progress or success with a larger action budget.
+- Measure whether a repeat of the identical plan agrees, on the same L40S, at every level: 30/30 outcomes, action counts, numerically equal recorded robot states and actions, and 360/360 renders on the frames a policy call consumed. Report renders on consumed frames apart from recorded-only frames, because only the former can change an outcome. One recorded-only frame of 3,195 differed and did not propagate.
 - Seal the failure taxonomy inside the experiment pack and recompute it on verification, rather than only checking its hash. Keep the reproducibility aggregate beside the pack, since it is a property of two collections and belongs to neither.
 
 - Verify draft assets against local checksums and GitHub SHA-256 digests, retry
   missing uploads individually, and retain uploads whose response was lost.
   Preserve existing public releases and drafts; document recovery using the
   original tested CI artifacts. Record the completed 0.10.0 publication.
 
+## 0.11.0 — 2026-09-14
+
+- Add Scene Lab: CC0 captured geometry, 25,000 mesh-derived surface Gaussians, separate collision heightfield, two actual L40S OptiX renders, bounded edit recipes and independent native Blender readback.
+- Add the official LIBERO-Plus adapter and source-linked SmolVLA replay: paired baseline, camera viewpoint ID 609 and lighting ID 2124. Outcomes are success at 77 actions, step limit at 220, and success at 87. This is a diagnostic subset, not a benchmark score.
+- Add an OpenEnv 0.4.2 verifiable creation environment. Numeric actions build and independently reopen a native scene; completion claims do not earn reward.
+- Add a pinned Cosmos Policy recorder/access preflight. Actual Cosmos inference remains unavailable until the account receives access to its gated NVIDIA dependency. No prediction results are claimed.
+- Link all 27 cross-project skill trials from the homepage and Hugging Face Space; improve mobile layouts, video seeking and lazy Gaussian loading.
+
+
+
 ## 0.10.0 — One launch, six recorded futures
 
 - Add Solver Lab: six independent L40S/CUDA ballistic flights from Genesis 1.4.1

CITATION.cff

@@ -7,7 +7,7 @@ authors:
 repository-code: "https://github.com/noteflowai/robot-reel"
 url: "https://noteflowai.github.io/robot-reel/"
 license: Apache-2.0
-version: 0.11.0
+version: 0.12.0
 date-released: 2026-09-14
 abstract: "Robot Reel is a physical-AI replay lab that records MuJoCo and Newton physics and SmolVLA/LeRobot rollouts, verifies the recorded traces, and directs them into inspectable browser replays and editable Blender/OpenUSD films through a CLI and an MCP agent director."
 keywords:

README.md

@@ -190,17 +190,21 @@ report for independent verification with the 0.8.0+ installed CLI.
 [Compare paired outcomes](https://noteflowai.github.io/robot-reel/stress/#outcomes)
 · [Report method and CLI](docs/stress.md#compare-paired-outcomes).
 
+**[Inspect every failure](https://noteflowai.github.io/robot-reel/stress/#failures).** Filter the recorded classifications and jump to each final motion window, with measured end-effector travel.
+
+[![Recorded failure review: classified episodes, complete denominators and a jump to the final motion window.](docs/failure-review.png)](https://noteflowai.github.io/robot-reel/stress/#failures)
+
 **What the failures were.** A success rate does not say. All 14 failures here end
 at the step limit, and that covers a policy that froze and one still reaching when
 the budget expired. Measured: **every one was still in motion at the cut-off**,
 44.8 mm to 138.6 mm of end-effector travel over the final tenth of its episode,
-against a 1 mm stall threshold. So 160 actions is binding on the reported success
-rate, not a policy that gave up.
+against a 1 mm stall threshold. The recorded budget expired while the arm was moving. This does not show
+that extra actions would complete the task or that the motion made progress.
 
 **Whether a repeat agrees.** The paired groups blame a condition for an outcome
 flip, which only holds if the same seed and condition answer the same twice. The
 whole plan was re-run on the same L40S: **30 / 30 identical outcomes, action
-counts, physics states and actions**, and **360 / 360 identical renders on the
+counts, recorded robot states and actions**, and **360 / 360 identical renders on the
 frames a policy call consumed**. One of 3,195 recorded-only frames differed, on a
 frame no call consumed. That difference is reported rather than rounded away: it
 shows the renderer is not bitwise deterministic even on identical hardware, and it

README.zh-CN.md

@@ -174,14 +174,18 @@ Markdown，重新导入可恢复原始样本位置，也可用命令行与完整
 记录明确标注末帧停留，保留完整实验统计，个人备注与录制事实分开。
 [复盘流程与核验命令](docs/stress.md#share-a-moment-for-review)。
 
+**[逐次检查失败](https://noteflowai.github.io/robot-reel/stress/#failures)。** 按记录类别筛选，直接跳到每个回合的末段运动窗口，并查看以毫米表示的末端移动距离。
+
+[![真实失败分类与末段运动复盘界面](docs/failure-review.png)](https://noteflowai.github.io/robot-reel/stress/#failures)
+
 **失败究竟是什么。** 成功率不回答这个问题。这里 14 次失败全部止于步数上限，而这个标签
 同时涵盖「策略停住了」和「预算耗尽时仍在伸手」两种情形。实测结果是：**每一次失败在截断时
 都仍在运动**，末端在最后十分之一个回合内移动 44.8 mm 到 138.6 mm，而停滞阈值为 1 mm。
-所以 160 步的预算才是所报成功率的约束，而不是策略放弃了。
+记录预算耗尽时机械臂仍在运动，但这不能证明增加步数会成功，也不能证明该运动代表任务进展。
 
 **重跑是否给出同样的答案。** 配对分组把结果翻转归因于条件，这只在同一种子与条件两次给出
-同样答案时才成立。整个计划在同一块 L40S 上重跑了一遍：**结果、动作步数、逐位物理状态与
-动作均为 30 / 30 一致**，**策略调用实际消费的帧渲染 360 / 360 一致**。3,195 个仅记录帧中
+同样答案时才成立。整个计划在同一块 L40S 上重跑了一遍：**结果、动作步数、记录中的机器人状态与
+动作数值均为 30 / 30 一致**，**策略调用实际消费的帧渲染 360 / 360 一致**。3,195 个仅记录帧中
 有 1 帧不同，且该帧未被任何推理调用消费。这个差异被如实报告而非抹平：它说明即使在同一硬件上
 渲染也不是逐位确定的，而它没有传播仅仅是因为落点位置。
 [分类、可复现性及其局限](docs/stress.md#what-the-failures-were-and-whether-a-repeat-agrees)。

docs/stress.md

@@ -51,8 +51,8 @@ UI filter never removes pairs from the report. “Not completed” combines
 
 A success rate does not say what a failure was. All fourteen failures here end at
 the step limit, and that label covers two different things: a policy that stopped,
-and a policy still reaching when the budget expired. They call for opposite
-responses.
+and an arm still moving when the budget expired. This measurement describes
+the recorded motion; it does not identify which intervention would fix the task.
 
 ```bash
 python3 -m robot_reel.cli stress docs/stress --reliability
@@ -61,9 +61,9 @@ python3 -m robot_reel.cli stress docs/stress --reliability
 Measured over the published collection, **every one of the fourteen failures was
 still in motion at the cut-off**; none had stalled. End-effector travel over the
 final tenth of each episode ranges from 44.8 mm to 138.6 mm, against a stall
-threshold of 1 mm chosen to sit far below anything a moving arm produces. So the
-step limit of 160 actions is binding on the reported success rate rather than the
-policy having given up, and raising it would be expected to change that rate.
+threshold of 1 mm. The 160-action budget expired while motion continued.
+That does not establish progress toward the goal or that a larger budget would
+change the success rate; either conclusion would require a new controlled run.
 
 Failures also wander further to end up no further along: median path-to-
 displacement ratio 3.17 against 1.92 for successes. The ranges overlap, so that
@@ -87,16 +87,18 @@ python3 -m robot_reel.cli stress docs/stress --repeat artifacts/repro-gpu-30
 | --- | --- |
 | Same outcome | 30 / 30 trials |
 | Same action count | 30 / 30 trials |
-| Bitwise identical physics states | 30 / 30 trials |
-| Bitwise identical actions | 30 / 30 trials |
+| Numerically equal recorded robot states | 30 / 30 trials |
+| Numerically equal recorded actions | 30 / 30 trials |
 | Identical renders on frames a policy call consumed | 360 / 360 frames |
 | Identical renders on frames recorded only | 3194 / 3195 frames |
 
 The two render levels are separated because only the first can change an
 outcome. The policy is called once every ten steps, so most recorded frames never
 reach it. The single differing frame is the wrist view at frame 3 of
-`seed-05-camera`, which no policy call consumed; the physics states and actions
-of that trial are bitwise identical throughout.
+`seed-05-camera`, which no policy call consumed; the recorded robot states and actions
+of that trial compare numerically equal throughout. This compares the saved
+`state` and `action` arrays, not every internal MuJoCo variable or floating-point
+bit pattern (for example, numeric equality treats signed zeros as equal).
 
 That difference is worth stating rather than rounding away. It shows the renderer
 is not bitwise deterministic across runs even on identical hardware, and it did
@@ -383,3 +385,9 @@ the ordinary imageio-ffmpeg/Pillow dependencies. Neither check reruns a policy.
 
 See the included media notice for third-party attribution. Model weights and
 upstream simulation meshes are not redistributed in the public experiment.
+
+## Inspect final motion and retain legacy packs
+
+Stress Lab now groups the recorded outcomes and lets a reviewer jump to the final tenth of samples of any selected episode. It displays the exact sample window and end-effector path length, with a stated 1 mm threshold. Movement alone cannot establish progress or the effect of a larger action budget. The full matrix, denominators and attempt ledger remain available.
+
+The verifier accepts the original exact inventory of releases through v0.11.0, which predated `reliability.json`. New exports include the derived taxonomy and independently recompute it. A missing consumed frame, or absent camera hashes in both runs, cannot count as identical policy input.

docs/stress/METHODS.md

@@ -51,8 +51,8 @@ UI filter never removes pairs from the report. “Not completed” combines
 
 A success rate does not say what a failure was. All fourteen failures here end at
 the step limit, and that label covers two different things: a policy that stopped,
-and a policy still reaching when the budget expired. They call for opposite
-responses.
+and an arm still moving when the budget expired. This measurement describes
+the recorded motion; it does not identify which intervention would fix the task.
 
 ```bash
 python3 -m robot_reel.cli stress docs/stress --reliability
@@ -61,9 +61,9 @@ python3 -m robot_reel.cli stress docs/stress --reliability
 Measured over the published collection, **every one of the fourteen failures was
 still in motion at the cut-off**; none had stalled. End-effector travel over the
 final tenth of each episode ranges from 44.8 mm to 138.6 mm, against a stall
-threshold of 1 mm chosen to sit far below anything a moving arm produces. So the
-step limit of 160 actions is binding on the reported success rate rather than the
-policy having given up, and raising it would be expected to change that rate.
+threshold of 1 mm. The 160-action budget expired while motion continued.
+That does not establish progress toward the goal or that a larger budget would
+change the success rate; either conclusion would require a new controlled run.
 
 Failures also wander further to end up no further along: median path-to-
 displacement ratio 3.17 against 1.92 for successes. The ranges overlap, so that
@@ -87,16 +87,18 @@ python3 -m robot_reel.cli stress docs/stress --repeat artifacts/repro-gpu-30
 | --- | --- |
 | Same outcome | 30 / 30 trials |
 | Same action count | 30 / 30 trials |
-| Bitwise identical physics states | 30 / 30 trials |
-| Bitwise identical actions | 30 / 30 trials |
+| Numerically equal recorded robot states | 30 / 30 trials |
+| Numerically equal recorded actions | 30 / 30 trials |
 | Identical renders on frames a policy call consumed | 360 / 360 frames |
 | Identical renders on frames recorded only | 3194 / 3195 frames |
 
 The two render levels are separated because only the first can change an
 outcome. The policy is called once every ten steps, so most recorded frames never
 reach it. The single differing frame is the wrist view at frame 3 of
-`seed-05-camera`, which no policy call consumed; the physics states and actions
-of that trial are bitwise identical throughout.
+`seed-05-camera`, which no policy call consumed; the recorded robot states and actions
+of that trial compare numerically equal throughout. This compares the saved
+`state` and `action` arrays, not every internal MuJoCo variable or floating-point
+bit pattern (for example, numeric equality treats signed zeros as equal).
 
 That difference is worth stating rather than rounding away. It shows the renderer
 is not bitwise deterministic across runs even on identical hardware, and it did
@@ -383,3 +385,9 @@ the ordinary imageio-ffmpeg/Pillow dependencies. Neither check reruns a policy.
 
 See the included media notice for third-party attribution. Model weights and
 upstream simulation meshes are not redistributed in the public experiment.
+
+## Inspect final motion and retain legacy packs
+
+Stress Lab now groups the recorded outcomes and lets a reviewer jump to the final tenth of samples of any selected episode. It displays the exact sample window and end-effector path length, with a stated 1 mm threshold. Movement alone cannot establish progress or the effect of a larger action budget. The full matrix, denominators and attempt ledger remain available.
+
+The verifier accepts the original exact inventory of releases through v0.11.0, which predated `reliability.json`. New exports include the derived taxonomy and independently recompute it. A missing consumed frame, or absent camera hashes in both runs, cannot count as identical policy input.