Hydraulics and bucket calibrated against PWRI's ZX200 step tests; clay autopilot fixes

README.md

@@ -20,7 +20,7 @@ Initiated and maintained by **[DigitalTwin Technology GmbH](https://www.digitalt
 | Site | 128 m terrain with pad, haul road, stockpile and a 2.5 m excavation pit; 40+ physics props from NVIDIA SimReady (CC BY 4.0) with checked masses and colliders |
 | Real machines (open-licensed) | Hitachi ZX200 excavator, IHI (Kato) IC120 crawler carrier, Komatsu D37PXi dozer (articulated, from the PWRI OPERA project, Apache-2.0); 35 m tower crane, cones, barrels and jersey barriers (Gazebo, CC0) |
 | Soil | The pit filled with 2 m of MPM soil (Newton implicit MPM) on the GPU, two-way coupled to the machine, in four presets switchable live: dry sand, moist sand, loam and clay. Cohesion is checked against vertical-cut stability and McKyes cutting forces, and dug clods break up to their remoulded strength; only soil near the bucket is particles, resting soil elsewhere is a heightfield, and cut faces slump to the angle of repose. The sand steps at 30 Hz and the machine at 60 Hz; the dig cycle runs at 1.3× real time, and a trench dug in one lane (1.2×) reaches 1.7 m with 43–45° sides |
-| Excavator | The real Hitachi ZX200 (OPERA URDF geometry and joint limits, curved bucket with teeth) driven by its real hydraulic cylinders (Hitachi bores, 34.3 MPa main relief and port reliefs, pump flow shared between functions, bucket four-bar linkage) and swing motor (13.3 min⁻¹, 68.9 kN·m, braking on its cross-over reliefs), so force and speed vary with pose and the machine slows and stalls under load. Digging force at the teeth 149 kN against 151–158 kN published. Full dig–swing–load cycle into a dump truck, lifting while swinging like an operator: 15–18 s and about 1 t per pass |
+| Excavator | The real Hitachi ZX200 (OPERA URDF geometry and joint limits, curved bucket with teeth) driven by its real hydraulic cylinders (Hitachi bores, 34.3 MPa main relief and port reliefs, pump flow shared between functions, bucket four-bar linkage) and swing motor (braking on its cross-over reliefs, with friction), so force and speed vary with pose and the machine slows and stalls under load. Joint speeds per direction, swing speed (11.2 min⁻¹ at 1800 min⁻¹ eco) and swing friction are calibrated against PWRI's step tests on a real ZX200 (ISARC 2025); the swing torque (77.9 kN·m) and the bucket's angle offset are provisional ([phase2/RESULTS.md](phase2/RESULTS.md)). Digging force at the teeth 149 kN against 151–158 kN published. Full dig–swing–load cycle into a dump truck, lifting while swinging like an operator: 16–19 s per pass in a trench, about 20–21 s and 0.7–1.3 t per pass loading the carrier (clay lowest) |
 | Haul cycle | An IHI IC120 crawler carrier (PWRI OPERA model) is loaded by the excavator, drives the load to a spoil area, tips it out as MPM soil and comes back. The operator model sends it off when full, waits while it is away, and spreads passes along the vessel; sticky loads are shaken out. One excavator and one carrier deliver 124–181 t/h depending on soil, with the excavator waiting 16–26 % of the time. With two carriers (`--carriers 2`), they take turns at the excavator, a waiting place and gates keep them apart on the one-lane road, and loading goes up 12–27 % over 10 minutes. Over 25 minutes the gain holds at 19 % while the excavator has soil within reach ([phase2/RESULTS.md](phase2/RESULTS.md)) |
 | Spoil spreading | A Komatsu D37PXi dozer (PWRI OPERA model) spreads each dumped load in lanes fanned out from the dump heap and builds the fill layer by layer (0.3 m lifts), its blade pushing MPM soil under grade and load control as on the real machine; the tracks slip above traction. It works while the carrier is away and parks before it comes back, so the carrier never waits; mean blade forces of 5–28 kN per pass match the sliding-load estimate ([phase2/RESULTS.md](phase2/RESULTS.md)) |
 | Browser client | Levers and keyboard control, cylinder pressure gauges, stall warnings, live truck load, the carrier, the dozer and the spoil area, a "Send carrier" button; 60 fps |

phase2/bench_haul.py

@@ -2,11 +2,17 @@
 dumps it and comes back, for N simulated seconds. Reports what a site engineer would look at: loads, load
 mass, passes per load, loading time, the carrier's time away and its stages, the excavator's waiting time,
 and tonnes per hour delivered; the dozer's passes, blade forces and how flat it left the spread layer, and
-how long the carrier waited for it; plus the real-time factor.
+how long the carrier waited for it; plus the real-time factor. Each pass is logged too (`pass_log`): its strip,
+the most the bucket held while digging, what it held at the swing, what it carried to the carrier and what it
+left after the dump, so one run shows where a soil's loads go.
+
+A quick check stops after a number of passes instead (PASSES=25 is about 500 s): enough to compare two
+versions of the autopilot pass by pass, in a quarter of the time. Progress goes to stderr every 100 s.
 
     ../phase0/.venv-newton/bin/python bench_haul.py 600                 # dry sand
     SOIL=loam ../phase0/.venv-newton/bin/python bench_haul.py 600
     SOIL=loam CARRIERS=2 ../phase0/.venv-newton/bin/python bench_haul.py 600   # two carriers sharing the road
+    SOIL=clay PASSES=25 ../phase0/.venv-newton/bin/python bench_haul.py 1500   # quick check: the first 25 passes
 """
 
 import json
@@ -21,15 +27,44 @@
 secs = float(sys.argv[1]) if len(sys.argv) > 1 else 600.0
 soil = os.environ.get("SOIL", ex.soils.DEFAULT_SOIL)
 n_carriers = int(os.environ.get("CARRIERS", "1"))
+max_passes = int(os.environ.get("PASSES", "0")) or None
 sim = ex.ExcavatorSim(soil=soil, carriers=n_carriers)
 sim.auto = True
 fleet = sim.fleet
 loads, start_load, passes_at = [], {}, {}
 t0 = time.perf_counter()
 prev = {c.index: c.stage for c in fleet}
 idle_spot = 0.0
-while sim.sim_time < secs:
+pass_log, cur, prev_stage, next_report = [], None, None, 100.0
+DIG = ("penetrate", "drag", "drag-curl", "curl")
+while sim.sim_time < secs and (max_passes is None or sim.passes < max_passes):
     sim.step()
+    if sim.stage_name != prev_stage:
+        st, fill = sim.stage_name, round(sim._bucket_fill())
+        if st == "approach" and cur is not None:
+            pass_log.append(cur)
+            cur = None
+        if cur is None:
+            n = sim.passes + (st == "approach")  # the count goes up as the approach ends
+            f = sim._faces.get(n) if sim.adaptive_face else None
+            cur = {"pass": n, "t": round(sim.sim_time, 1), "face": None if f is None else [f["psi"], f["shift"], round(f["avail_m3"], 2)],
+                   "dig_max_kg": 0, "early_curl": False, "skips": len(sim.skipped)}
+        if prev_stage in DIG:
+            cur["dig_max_kg"] = max(cur["dig_max_kg"], fill)
+        if st == "curl" and prev_stage not in ("drag-curl", "curl"):
+            cur["early_curl"] = True  # curled before the planned curl point (the load was there)
+        if st == "swing to truck":
+            cur["at_swing_kg"] = fill
+        elif st == "dump":
+            cur["carried_kg"] = fill
+        elif st == "swing back":
+            cur["left_kg"] = fill
+        prev_stage = st
+    if cur is not None and prev_stage in DIG:
+        cur["dig_max_kg"] = max(cur["dig_max_kg"], round(sim.auto_fill))
+    if sim.sim_time >= next_report:
+        print(f"{sim.sim_time:.0f} s: {sim.passes} passes, {sum(x['load_kg'] for x in loads) / 1000:.1f} t in {len(loads)} loads", file=sys.stderr, flush=True)
+        next_report += 100.0
     if fleet.at_spot is None:
         idle_spot += sim.frame_dt  # no carrier at the loading spot
     for c in fleet:
@@ -42,12 +77,19 @@
             prev[c.index] = c.stage
 wall = time.perf_counter() - t0
 sim.measure()
+if cur is not None and "left_kg" in cur:
+    pass_log.append(cur)
+for x, nxt in zip(pass_log, pass_log[1:] + [{"skips": len(sim.skipped)}]):
+    x["skips"] = nxt["skips"] - x["skips"]  # stages skipped in this pass
+secs = sim.sim_time  # a quick check stops early
+done = [x for x in pass_log if "carried_kg" in x]
 trips = fleet.trips
 delivered_t = fleet.delivered_kg / 1000.0
 out = {
     "soil": soil,
     "carriers": n_carriers,
-    "sim_s": secs,
+    "sim_s": round(secs, 1),
+    "stopped_after_passes": max_passes,
     "realtime_factor": round(secs / wall, 3),
     "loads": loads,
     "trips": trips,
@@ -74,6 +116,11 @@
     "pit_left_t": round(float((sim.zones.count_np * sim._pit_cols).sum()) * sim.particle_mass / 1000, 1),
     "pit_done": sim.pit_done,
     "faces": [(f["psi"], f["shift"]) for f in sim.face_log],
+    "curl_gain_kg": round(sim.curl_gain),
+    "mean_carried_kg": round(float(np.mean([x["carried_kg"] for x in done]))) if done else None,
+    "mean_dumped_kg": round(float(np.mean([x["carried_kg"] - x["left_kg"] for x in done]))) if done else None,
+    "thin_passes": sum(1 for x in done if x["at_swing_kg"] < 0.5 * sim.full_kg),
+    "pass_log": pass_log,
     "dozer": {**{k: round(v, 2) for k, v in sim.dozer.counters.items()}, "mean_push_kN": sim.dozer.push_force},
     "spread": sim.dozer.spread_report(sim._surface),
 }