sim: the ToF numbered its columns backwards - #30
Merged
Merged
Conversation
The other half of `microduck`'s `robotd --sim`: MuJoCo holds the body, the daemon holds everything
else, and a TCP socket replaces the servo bus. Above `duck_control::io::RobotIo` the code is what
runs on a real robot — the 50 Hz loop, the ONNX policies, safety, fall detection, odometry,
kinematics, every IPC call — and it cannot tell.
uv run duck-body # then, elsewhere: robotd --sim 127.0.0.1:7801
Measured working end to end: the daemon reports `50.0 of 50.0 Hz · 0 missed`, detects a seated boot
from the sim's own joint angles, and `robotctl robot init` runs the sitstand policy until the duck is
upright and stays there.
Two mappings live here on purpose. The daemon indexes joints by `JOINT_NAMES`, which has fifteen
entries including `mouth`; the model actuates fourteen and has no mouth. And the policy observes
projected gravity in the trunk frame, while MuJoCo offers an orientation quaternion. Both conversions
belong on the side that knows the model, which is why the protocol carries the robot's own units.
Three things cost an hour each and are commented where they bit:
**`scene.xml`, not `scene_walk.xml`.** The walking scene includes the model the RL work trains
against, whose actuator default classes carry `contype="0" conaffinity="0"` — so the robot collides
with nothing and sinks through a floor the scene really does contain: trunk z went 0.120 to -0.105 in
one second. The daemon read that as a robot on its back, which it was. `scene.xml` includes
`robot_allcollisions.xml`.
**Start from a keyframe, not `qpos0`.** Every joint at zero is not a pose this robot is ever in; the
daemon measured 0.41 rad from its home frame, called it a seated boot and tried to rise. The scene
carries the real poses, and `STAND` matches `duck_control::DEFAULT_POSITION` — including the mirrored
right leg, which I got wrong by assuming symmetry before reading it.
**Torque on at startup, holding the pose.** `robotd` deliberately never enables torque when it
starts, because "a robotd restarted by an update must leave a standing robot standing" — on a real
robot the servos are already holding. Starting limp means the duck collapses before the first read.
`--limp` is there for when a duck on the floor is what you want, and `--keyframe SIT` is the boot
story the daemon is actually written for.
Real time, not as fast as possible: the daemon's health gate fails below 45 of 50 Hz, so a simulator
running at its own pace does not look wrong — it makes every duck unhealthy and starts rolling
releases back. It says so when it falls behind.
Two fixes for the same misreading: a duck that never stood up looked like one that had. **`trunk_z` in the sensor frame.** Deliberately outside the protocol — no robot can measure how high its own trunk is, and serde ignores it on the daemon side — because a tool asking "did it stand?" has no other way to know. A duck sitting on its bottom with a vertical trunk has gravity [0, 0, -1] exactly as a standing one does, so a check on orientation alone called 0.978 upright while the window showed it sitting. STAND is 0.12 m and SIT is 0.07. **A frame every N steps, counted.** The viewer synced on `data.time % 0.033 < dt`, which is float arithmetic on an accumulating value and fires when it feels like it; a window that renders the first frame and rarely again shows a pose the robot left seconds ago.
`sim_time` alongside `trunk_z`, outside the protocol for the same reason: the daemon's loop is wall-clock, so a simulator whose seconds are not seconds silently ruins a policy, and the "behind real time" warning cannot see slow drift because it resets its own reference each time it fires. Measured with it: 6.00 s of sim in 6.00 s of wall clock, 1.00x. Pacing is not the problem. The pacing loop no longer sleeps on every 2 ms step. `time.sleep` overshoots a 2 ms budget by more than the budget, so 500 sleeps a second is a timer rather than a clock; it now sleeps only when there is more than a step's worth of slack to wait for.
…torque Three things the working reference does that this did not. `model.opt.timestep = 0.005` — the scenes ship 0.002 and `scripts/infer_policy.py` overrides it, which with its decimation of 4 is exactly the 50 Hz the daemon runs at. The BAM actuator fit, the contact solref and the joint armature are all tuned at that step. `--keyframe HOME` is infer_policy's placement rather than a keyframe: trunk 0.125 m, upright, every joint at the home pose. And physics does not advance until the first `set_torque(true)`. This duck cannot hold any pose without a policy driving it — measured at both timesteps, from every placement — and `robotd` deliberately does not enable torque at startup, so the seconds before it would be spent falling over. Holding it is a hand steadying the robot until someone enables it.
The window is for watching a duck. Everything the panels would drive — the pose, the actuators, the policy — belongs to the daemon here, so they are two columns of controls that must not be touched.
`--ducks 4` builds one model holding four of them and serves each on its own port, from `--port` upwards. One `mj_step` advances all of them, which is the difference between a room with four robots in it and four robots on the same screen: they share a floor, and they can bump into each other. The scene already contains one duck, so the rest are attached to it under a name prefix with `MjSpec`. A fresh child spec for each — `attach` renames the spec it is given, so reusing one names the third duck `d2_d1_left_hip_yaw` and fails to compile. `Body` becomes a view onto a shared `World` rather than owning a model: its own actuators, joints and trunk, resolved by prefixed name. A duck nobody has enabled yet is put back where it was after every step, since physics is shared and one of them cannot simply not be advanced — a hand steadying one robot while another walks about is an ordinary thing for a room to contain. Poses are applied by joint name from a table read out of the scene's keyframes, because attaching a robot does not bring the scene's keyframes with it. Including the keyframe's own trunk height: a seated duck placed at standing height hovers above the floor and drops the moment it is enabled.
The lock is the bottleneck, not the solver: four daemons at 50 Hz make 400 requests a second, each wanting the world lock, and Python hands the GIL between every one of them — so a physics loop that takes and releases it 200 times a second loses. Four steps of 5 ms is 20 ms of world, exactly one control tick and the same decimation infer_policy.py uses, so nothing sees a sensor older than the tick it belongs to. And the viewer syncs at 30 rather than 60. `viewer.sync()` copies the scene on the physics thread, and with four ducks in it that is the difference between keeping real time and not.
Outside the protocol, like trunk_z and sim_time: no robot knows its own position in a room. It is here so a simulated radio can decide who is close enough to hear whom — the one thing a real BLE advertisement gets for free and a faked one has to be told.
An 8x8 VL53L5CX on the model's own `tof` site — 45 degrees per axis, 4 m, and noise that grows with
range, from ~/MISC/microduck_maploc's version of the same sensor. Answered on the body protocol as
`{"op":"tof"}`, at the sensor's own 15 Hz rather than the control loop's 50, because sixty-four ray
casts is the most expensive thing here and that is the rate the hardware runs at.
The status byte matters as much as the distance: a real sensor distinguishes nothing-out-there from
could-not-measure, and the consumers treat them differently — one is empty space to clear on a map,
the other is no information. A simulator reporting distances alone would let a bug through that
hardware finds.
Self-hits are reported rather than filtered. A real sensor sees the duck's own beak when the beak is
in front of it, and quietly skipping our own geometry would hide exactly the mounting problem this
is here to catch. Verified standing: 24 of 64 zones valid, the floor at 0.74 m from a site 0.251 m
up looking level, and open air above.
Six rooms in a 7x6 m floorplan with 50 cm walls, from ~/MISC/microduck_maploc — and its own comment says why it is the right room rather than any room: the doorways are deliberately off centre, because that asymmetry is what lets a pose be disambiguated from a 45-degree forward field of view. Which is exactly the field of view the duck has. `scene_apartment.xml` is `scene.xml` with the apartment included, so the duck, the floor, the visuals and the four keyframes every pose is read from all come along. 164 geoms, and no name collisions worth reporting. The difference it makes to the sensor, measured through `tofd`: on a bare floor 24 of 64 zones are valid and the rest is sky. In the apartment, 62 of 64 — a wall receding from 0.88 to 2.2 m with a gap where a doorway is, another at 0.8 m, and the floor at 0.41 m. That is the first depth frame in this project that a mapper could do something with.
The apartment lays its own floors — boxes whose top face is exactly z=0 — and `scene.xml`'s ground plane sits at z=0 too. Two surfaces in the same place is a floor that flickers as the renderer picks between them, which is what it looked like. The plane goes. The apartment's floors carry `contype="1"`, so a duck still stands on them; walk out of the flat and there is nothing there, which is a fair description of a flat.
Column 0 of the simulated sensor looked right; kinematics::tof, which reads the real sensor's buffer, takes column 0 as the sensor's left. Every frame reached maploc mirrored: walls seen at an angle were inked at their mirror image across the head's axis, phantom walls stood in open floor, and no loop ever closed on the twin. Replaying one session with the columns reversed took the wall error against the scene from 0.174 m to 0.058 m; a fresh run with this fix reads 0.031 m.
apirrone
self-requested a review
September 2, 2026 18:39
Collaborator
|
Good catch ! I'll test and merge this when I have time. Thanks for your contribution ! |
apirrone
approved these changes
Sep 2, 2026
apirrone
left a comment
apirrone
left a comment
Collaborator
There was a problem hiding this comment.
Indeed it was reversed :)
microai-lab
pushed a commit
to microai-lab/microduck_rl
that referenced
this pull request
Sep 11, 2026
sim: the ToF numbered its columns backwards
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
The
sim-body-serverbranch as @apirrone pushed it (eleven commits: a simulated body served to the real robotd over TCP, several ducks in one world, a depth sensor per duck, an apartment scene), plus one fix on top from running maploc on it.The fix: the ToF numbered its columns backwards
sim/tof.pygave column 0 of the 8x8 frame the azimuth −22.5°, looking right.kinematics::tofin microduck, which reads the real VL53L5CX buffer, takes column 0 as the sensor's left. Every simulated frame therefore reached maploc mirrored left/right. Walls seen at an angle were inked at their mirror image across the head's axis, phantom walls stood in open floor, and no loop closure ever fired on the twin.Proven before the change by replaying one recorded session through maploc's bench with the columns reversed, and confirmed by a fresh run after it. Map wall cells against the apartment's walls and furniture footprints:
The change is one sign:
azimuth = -centres[col]. The comment above it now states the column convention and why.Companion PR: pollen-robotics/microduck#202, which holds the scaffolding, the recordings and the bench numbers.