Conversation
`robotd-design.md` §9 deferred "the MuJoCo backend and the `RemoteIo` protocol". This is it, and it is small because the seam was already there: `duck_control::io::RobotIo` is six methods, `FakeIo` already implements them, and everything above — the loop, the policy, `Safety`, fall detection, odometry, kinematics, every IPC call and all of `robotctl` — runs unchanged and cannot tell. Three decisions worth the words they cost. **TCP, not a unix socket.** A unix path is capped at `SUN_LEN`, about 108 bytes, which a scratch directory blows straight through — that bit during validation this afternoon. And the simulator has to be reachable from outside whatever the daemons run in: a container on Linux, and on macOS a Linux VM with MuJoCo on the host beside it. A port crosses all of those. **Newline-delimited JSON.** A tick is fifteen joints each way, about a kilobyte, so 50 KB/s at 50 Hz — nothing against being able to read a frame with `nc` and write the other half in twenty lines of Python. The alternative is a packed struct shared between two repositories in two languages, which is the exact shape of thing that has already cost this project days when an offset was wrong and the failure was silent. **A dead simulator is one bad tick, not a dead duck.** MuJoCo compiles its model, so changing the number of ducks restarts it, and the ducks are expected to live through that. So a broken connection returns an error and reconnects on the next call — no backoff thread, because the control loop *is* the retry timer, and `robotd` already treats a failed read as a tick to skip. Two tests hold that: one hangs up mid-read, one answers with something that will not parse. Both must produce a fresh handshake rather than a wedged link, because a line protocol cannot resynchronise any other way. `TCP_NODELAY` is set, and that is not a micro-optimisation: Nagle delays a small write up to ~40 ms waiting for more to send, which is twice the tick, and it would look exactly like a slow simulator. The simulator reports in the robot's own units — radians, rad/s, mA, the IMU already in the trunk frame — because MuJoCo knows its own model's ordering and scaling, and a translation layer here would be a second place for that knowledge to drift.
`robotd --sim` referenced this file before it existed. It now says what the twin is, what it is not, and why — including the experiment that changed the plan halfway through. The attractive idea was to run the board's own aarch64 binaries on an x86 laptop under qemu-user: same bytes, perfect provenance. Measured twice, and the results point opposite ways. The daemon alone is fine — CI's real artifact, emulated, holds 50.0 of 50.0 Hz with zero missed ticks at 4.7% of one host core, because the policy is 0.029 ms against a 20 ms tick and the slow part of a real tick is a local socket here. Under systemd it is not: aarch64 systemd 257 boots in 8.7 s and then starts nothing, because qemu-user 8.2 cannot translate the new mount API its per-unit namespaces and credentials need. `robotd.service` dies with 226/NAMESPACE, journald and logind with 243/CREDENTIALS. Per-unit hardening is the entire reason to boot a container rather than run seven processes in a terminal, so the twin follows the *host's* architecture instead: an amd64 container and a native build on x86, and on Apple Silicon the robot's own signed artifact running natively. A colleague on a Mac gets the higher-fidelity twin for free. §1 is the part that decides whether any of this gets used: one command to get a duck, defaults that are the common case, and every duck a systemd unit so stopping one is `systemctl stop` rather than a key combination. That last rule was bought this afternoon by a container that had to be killed from a second terminal, because `Ctrl-]` is `AltGr + )` on a French keyboard. §6 lists what the twin is and is not, and checks itself against a week of real bugs: the videoflip that cost 22 fps, the 3A engine missing a stream-start event, an auto-exposure loop that converges once, an INT8 head collapsed to two values, a bus dropping reads. It would have caught none of them. That is the boundary, stated so nobody mistakes a green sim for a working robot.
One duck works end to end: `robotd --sim` against `microduck_rl`'s `duck-body`, 50.0 of 50.0 Hz with no missed ticks, a seated boot detected from the simulator's own joint angles, and the sitstand policy standing the duck up and holding it there. Getting there cost three hours, one per way of choosing a model that looks right and is not — and all three present identically, as a duck on its back. `scene_walk.xml` includes the RL training model, whose actuator default classes disable collision entirely, so the duck sinks through a floor the scene does contain. `qpos0` is every joint at zero, which is a shape this robot is never in. And a simulator that starts limp has its duck on the floor before the daemon's first read, because on a real robot the servos are already holding when the process comes up. Written down because the next person to point this at a scene will pick from six of them.
The last version of this was three terminals and a heredoc, which is a set of instructions rather than a tool. `scripts/duck-sim` and the duck is up: it builds what is missing, writes the params file, starts both halves, waits for each, and stands the duck up with the sitstand policy. Three things it exists so that nobody has to know, because none of them is guessable from the outside. Policies resolve under `/opt/robot/daemon/current` on a robot and live in this repo on a laptop, so every one has to be named explicitly. `ort` dlopens libonnxruntime and a laptop has no system one — the RL repo's venv does, so this finds it there. And a unix socket path is capped at about 108 bytes, which a scratch directory blows through, so the state directory is short by design. `drive` is the smallest client that sends a velocity intent, because walking has no `robotctl` subcommand — intents come from a pad or the console. It resends at 10 Hz, since `robotd` expires an intent 500 ms after they stop arriving, which is what makes a dropped connection stop the robot rather than run it into a wall. Stopping is by pidfile, never by name: `pkill -f robotd` also matches the shell about to start one, which killed this session's terminal three times in one afternoon. Verified end to end: up, `50.0 of 50.0 Hz · 0 missed`, upright at gravity z −0.974, still upright after walking, and down.
…stand Two bugs, both mine, both of the kind that makes a tool untrustworthy. **`down` left MuJoCo running.** The simulator was started as `( cd "$RL" && nohup python ... & echo $! > body.pid )`, and `A && B & C` parses as `(A && B) & C` — so the pidfile held the *subshell's* pid. Killing it reaped a wrapper and orphaned the window. Both halves now start with `setsid` and are stopped by process group, `down` waits for them to go and escalates to KILL, and it checks the port afterwards rather than announcing success: reporting "down" with a window still open is worse than reporting nothing. **A duck that never stood up looked like a success.** `robot init` ran as `>/dev/null 2>&1 || true`, which threw away the one command whose failure explains a duck sitting there doing nothing. It now fails loudly, and `up` checks afterwards whether the duck is actually upright — the daemon reporting healthy only means the loop is turning, and a duck that never rose is healthy and sitting. When it did not stand, the script says which of the two causes it was. If the simulator logged falling behind real time, that is the answer: the daemon's loop is wall-clock, so a MuJoCo running at half speed is a policy driving a robot that moves half as fast as it expects, and it cannot balance one — the viewer is the usual reason, and `DUCK_SIM_VIEWER=0` is the fix. If the simulator kept up, it is the policy, and the next thing to read is the log. `init` and `status` are there to retry and to ask, without restarting anything.
I claimed this twin stands a duck up. It does not, and the check that told me so was mine: `upright()` read gravity in the trunk frame and called -0.978 a success. A duck sitting on its bottom with a vertical trunk has gravity [0, 0, -1] as surely as a standing one does — so the script reported success twice, in front of the person who could see the window and could see it sitting. It now reads the trunk's height, which the simulator reports for exactly this purpose: 0.12 m standing, 0.07 seated, and 0.064 on a real run. Orientation is still checked, for the duck that stands and then falls over.
`robot init` enables torque and then position-ramps to the home pose over two seconds with nothing balancing. This duck cannot hold any pose without a policy driving it, so the ramp tipped it flat every single time: gravity in the trunk frame went -1.000 to +0.016 inside those two seconds, and it lay there. Every failure chased today — the seated duck, the frozen duck, the duck on its back — was that ramp. `robot.enable` hands it to the policy directly, and from a seated start it stands itself up: trunk 0.063 to 0.116 m, upright and steady. There is no robotctl subcommand for it, so the script makes the call. It starts seated now, because a seated duck is stable while it waits.
`up` runs the daemons as processes on your laptop. `boot` runs them the way a robot does — real unit files with their real User=, groups, RuntimeDirectory= and hardening, under a real init in a systemd-nspawn container — so `machinectl shell duck-a` puts you on the duck and `robotctl health` there is the same thing it is on hardware. MuJoCo stays on the host and the container shares the host's network namespace, so localhost is the same localhost on both sides of the wall. `rootfs` builds Debian 13 Trixie, the board's own userland, in about thirty seconds. Two things an Ubuntu host needs and does not have, both of which present as something other than what they are: Debian's archive keyring, fetched here — and staged in a world-traversable directory for the length of the build, because a keyring under a 0700 `~/.cache` cannot be read by the uid apt drops to *inside* the user namespace, and apt then calls the repository unsigned. That reads like a mirror problem and is a permissions one. And the rootfs is built as a tarball and unpacked with sudo, rather than written as a tree: writing the tree needs every parent directory traversable by the subuids mmdebstrap maps in, and `~/.cache` being 0700 is the user's business rather than this script's. A tar is one file in a directory we own, and the sudo that `boot` needs anyway is what gets the ownership right.
**`robotctl robot enable`.** The console has had this button since it existed and the CLI did not, which nobody noticed until a robot with no hands to hold it needed one. The difference from `init` is the whole point: `init` powers the joints and position-ramps to the home pose with nothing balancing, while `enable` hands the robot to its policy, which then holds it up. A biped cannot stand by being commanded to a pose — in simulation, where nobody is steadying it, `init` puts the duck on the floor and `enable` stands it up from sitting. `--toggle` is what Start does, and the daemon's own `reason` is printed because a client cannot know which way a toggle went. Three things the first boot got wrong, all visible the moment somebody logged in: `robotctl: command not found` — the release's bin is not on PATH. Symlinked into /usr/local/bin rather than added by a profile.d line, so `machinectl shell duck-a robotctl …` works too, which does not read a profile. The prompt said `antoine-Blade-14-RZ09-0508`. A duck should be called `duck-a`, so nspawn is told its hostname. And the container was given a params file that named a policy section with no policies in it. It now gets none at all: inside the container `RELEASE_DIR` is a real path, so every policy resolves exactly as it does on a robot. That is the reason to boot one. `boot` now also enables the policy from inside, over the container's own socket, and reports whether the duck actually stood.
`scripts/duck-sim boot 4` gives four ducks in one MuJoCo window and four containers to log into — `duck-a` through `duck-d`, each with its own body port, its own hostname and its own systemd. One rootfs, one overlay per duck: the 213 MB base is shared read-only and each duck gets its own upper, so the fourth costs megabytes and `robotctl configure` on one does not edit another's config. The port reaches each container through `--setenv`, which the robotd drop-in reads — the same unit file, told which body is its own. `down` stops and unmounts every duck this script could have started rather than the ones it remembers starting, so an interrupted boot does not leave a container running and an overlay mounted.
**Three ducks out of four were never enabled.** `machinectl shell` allocates a PTY and is unhappy with its output redirected, so the loop that stood them up failed silently for all but the first — and a duck that was never handed to its policy sits there looking like a duck whose policy failed. `systemd-run --machine --pipe --wait` runs the command in the container and returns its exit status, which is what the `||` was always assuming it had. And `boot` now measures the world's real-time factor and prints it. Measured here, four ducks with all four released run at 1.00x — so physics is not the reason anything felt slow, and guessing at that from the outside is exactly what this line is for. Below 1.0 the daemons' wall-clock loops are driving robots that move less than they expect, which no policy can balance; the message says what to try. `duck-sim simlog` follows the simulator's own log, where it reports falling behind.
Four daemons drove the same duck. The port reached each container through `systemd-nspawn --setenv`, which sets it for the container's PID 1 — and systemd does not pass its own environment on to the services it starts, so every robotd fell through to the `:-7801` default. The simulator's log says it plainly: seven connections to duck 0 and none to the others, and ducks b, c and d were never released, which is why they did not move at all rather than falling over. The drop-in naming the body is now written per duck into that duck's own overlay, before it boots. That is what the overlay is for, and it needs no environment to survive the trip.
`scripts/duck-sim boot 4` answered `boot: not found` from a script that defines boot forty lines earlier. The staging step is one single-quoted argument to `sudo sh -c`, and a comment inside it said "the container's PID 1" — which ended the quote. Everything after became unquoted shell, and it swallowed the brace closing that function along with the whole of the next one. `sh -n` sees nothing wrong, because the quotes balance again at the end of the block. That is the part worth remembering: syntax-checking a shell script does not tell you which of two readings the shell took. The prose moves above the block where apostrophes are free, the block carries a warning, and the edit that made this asserts there are none inside it.
The prompt inside every container read `antoine-Blade-14-RZ09-0508`. nspawn's `--hostname` sets the kernel hostname, and then systemd inside boots, reads `/etc/hostname` — which mmdebstrap filled in with the name of the machine that built the rootfs — and sets it straight back. The first boot log said so plainly: "Hostname set to <antoine-Blade-14-RZ09-0508>". Written per duck into its own overlay, beside the drop-in that names its body, for the same reason: what a duck is cannot depend on anything surviving the trip into the container.
The chorale has never been testable without several robots on a desk. It is now testable on a
laptop, and it works:
duck-a: soprano bar 16 beat 59.3 2 voices
duck-b: bass bar 16 beat 59.5 2 voices
`duck-ether` replaces `btd`'s radio and nothing above it. Presence is already an IPC contract on
`robotd`'s own socket — `chorale.subscribe` to be told what to advertise, `chorale.beacon` carrying
it, `chorale.heard` carrying what came back — and `btd` is a *client* of `robotd` rather than a
server, so this impersonates nothing and steals no socket path. It holds one connection per duck
exactly as `btd` does, and every duck's election, roster, beat and conductor deference runs
unmodified and cannot tell.
**Distance decides who hears whom**, because `ChoraleHeard` carries no signal strength: a real
scanner either sees an advertisement or does not. So the ether asks each simulator where its duck is
standing and delivers a beacon only within range. Cruder than a real radio and far more
controllable — "these two can hear each other and those two cannot" becomes a number, where on
hardware it means carrying robots into other rooms. `--rotate` changes every address on a timer,
because `from` is documented as an identity for de-duplication only and a real address moves
underneath you — the bug that cost this project a day, now available as a test.
**A voice each, and it is not decoration.** Two ducks heard nothing from each other for an hour of
debugging: `robotd` takes its chorale id from the seed recorded in the bank it plays from, and that
id is how a duck recognises its own beacon reflected back. Sharing a bank meant sharing an id, so
each dropped the other as its own reflection and sat there listening to nobody. One bank per duck,
seeded from its name, fixes the id and gives four ducks four voices — which is what a chorale is for.
`sounds::hardware_seed` learns `DUCK_IDENTITY` for the same reason: several ducks on one machine is
the one situation where deriving identity from hardware is wrong. Nothing on a robot sets it.
A container's `/run` is its own, so the socket at `/run/robotd.sock` inside one is invisible from outside — and the radio has to hold a connection to every duck, from outside all of them. So a host directory is bound in at `/run/duck`, robotd is told to listen there, and a symlink puts the socket back at the path every client in the container expects. One object, reachable from both sides. Three things a container needs that the process-mode duck got from a params file, and which belong in its own overlay instead: the drop-in naming its body and its socket, `[chorale] accept` (and only that key, so everything else still resolves from the release), and a voice bank of its own — the chorale id comes from the bank seed, and ducks sharing a bank drop each other as their own reflection. The ether runs under sudo, because robotd creates its socket 0660 root:robot and the person running this script is not in that group. Being in the *host* group would not help either: the container has its own.
`stop_one` killed the process *group* named by a pidfile, with a `sudo` fallback. Pidfiles go stale, pids are recycled, and a `boot` leaves behind the pidfiles an `up` wrote — so `down` ran `kill -TERM -<long-dead pid>` as root against whatever process group had inherited that number. It took out a login session, and the machine had to be rebooted to get it back. Two changes, and the first is the rule: **Nothing acts on a pid without confirming through /proc that the process is still the one the pidfile was written for.** The pidfile records what the process must be, `stop_one` checks `/proc/<pid>/cmdline` against it, and says so and does nothing when it does not match. The file is removed before anything is signalled, so a stale one cannot be acted on twice. Anything that is not a live pid above 1 is dropped. **No group kill anywhere.** Every process this script starts is a single process that exits on TERM; the group form bought nothing except the reach to do harm. The container-mode radio, which has to run as root, is a `systemd-run --unit=duck-ether` instead — `systemctl stop` needs no pid and cannot be wrong about which process it is. Verified: a pidfile pointing at an unrelated live process now prints "not stopping pid N: it is no longer robotd" and leaves it alone, and a pidfile containing 1 does not touch init.
Frames from the simulator instead of the sensor, over the same newline-delimited JSON link `duck_control::sim` uses for the servo bus. `tofd` publishes them exactly as it publishes real ones — same `tof.frame`, same 8x8, same per-zone statuses — so `robotd`, the theremin and the viewer cannot tell. Verified end to end: `sensor: "sim"`, 15 Hz, 24 of 64 zones valid, the floor at 0.74 m. **The fake stays at the loop level.** `sensor.rs` says in as many words that the off-board `Sensor` "is not a fake sensor and must never become one", because what it stands for is a vendor C library talking to a bus — so a frame arriving from somewhere else is a different question from a sensor that lies, and it gets its own loop beside `fake_loop`. A simulator that goes away is one missed frame and a reconnect, since MuJoCo restarts whenever the number of ducks changes. `TCP_NODELAY`, for the reason the body link needed it: Nagle would add tens of milliseconds to a 15 Hz request and response, which is most of a frame. `duck-sim` starts one per duck in both modes, and points each duck's theremin at its own — on a board that socket is `/run/tofd/tof.sock` and needs no saying, but here every duck has one and they must not share. In a container it is a unit with a drop-in, like robotd, because the units being the real thing is the reason to boot one.
`robotctl monitor` draws the depth frame from `tofd` and the pad from `padd`, and looks for them under /run — which is where they are on a board and is not where they are here. So a monitor with no ToF points was a client pointed at the wrong path, not a sensor that was never wired: `tofd` was publishing 48 valid zones on its socket the whole time. `ctl` now passes `--tof-socket` alongside `--robot-socket`, `scripts/duck-sim monitor` is the short way in, and `DUCK_SIM_DUCK=duck-b` picks another duck.
`tofd.service` in the container failed with `status=217/USER`: `User=tofd`, and no such user. The staging step copied only `robot.conf`, while the release's own postinstall installs every sysusers file it ships. Now so does this, plus the `i2c` group `tofd.service` asks for — which a board gets from provisioning rather than from a release. Worth the note for how it presented: a unit whose `User=` does not exist reports 217/USER and nothing about a missing user, and from the monitor it looks exactly like a sensor nobody wired. mediad, btd and padd would each have hit the same wall in turn, so all five files go in now.
`tofd.service` still died with 217/USER after every sysusers file was staged correctly. The files were there; nothing had read them. `systemd-sysusers.service` carries `ConditionNeedsUpdate=|/etc`, and staging writes /etc (the hostname, the drop-ins, the params) after /usr — so /etc looks newer, the condition is not met, and the unit is skipped for the life of that rootfs. The users never exist, and the first daemon with a `User=` of its own falls over. The release's postinstall runs `systemd-sysusers` directly rather than trusting the unit, and now so does this. Followed by a check that each expected user or group is actually in /etc/passwd or /etc/group, said plainly at staging time — three rounds of this were spent looking at a monitor, then at a socket path, then at a copy that had in fact worked.
`tofd.service` sets `RestrictAddressFamilies=AF_UNIX`, and rightly: on a robot the daemon talks to an I2C bus and its own socket, and TCP is not its business. In the twin the sensor is across a TCP connection, so the sandbox refuses it — with EAFNOSUPPORT, "Address family not supported by protocol", which reads like a broken network rather than a policy that is working exactly as written. The per-duck drop-in adds AF_INET, reset and re-stated rather than appended so the drop-in says what the daemon may do rather than what it may do *as well*. This is the shape of thing the twin exists to find, incidentally: the unit is right, the daemon is right, and the simulator is the one asking for something the hardening never had to allow.
DUCK_SIM_SCENE=apartment scripts/duck-sim boot 2 A bare name is one of the simulator's own scenes; anything with a slash is a path. The default stays a bare floor, where a depth frame is 24 valid zones and 40 of open sky. In the apartment it is 62 of 64 — a wall receding from 0.88 m to 2.2 m with a gap where a doorway is, another wall at 0.8 m, and the floor at 0.41 m. The first depth frame in this project a mapper could do anything with.
The chorale was visibly working and silent, and the reason was blunt: there is no `aplay` in the rootfs. `robotd` forks it to make a sound and — by design, so that a robot with no codec still runs — logs the failure and carries on. A duck singing into a container with no ALSA tools is exactly that case, and it looks like a sound bug rather than a missing package. So the rootfs gains `alsa-utils` and `libasound2-plugins`, `/etc/asound.conf` points the default PCM at PulseAudio, and each container is handed the host's audio socket at `/run/pulse-host` with `PULSE_SERVER` set for robotd. Through the host's audio server rather than a bound `/dev/snd`, because that is what lets four ducks be audible at once instead of the first one holding the device. A machine with no audio server is told, not failed: on a board a missing speaker is a warning. The package list is now recorded beside the rootfs and compared on every `rootfs`, so adding a package rebuilds instead of silently not taking effect — which is otherwise a round trip spent wondering why a pull changed nothing.
The package list gained `alsa-utils` and nothing rebuilt, so there was still no `aplay` and the ducks still sang silently. `boot` called `rootfs` only when the directory was missing — and deciding whether the existing rootfs is still the right one is the whole of what that function does. Guarding the call on the directory existing meant the check could never run. It is called unconditionally now; it returns immediately when the recorded package list matches. A function that exists to decide something must not be called only when the answer is already known.
`robotd` runs `aplay -q -D plughw:aic3104` — the board's codec, named explicitly from `[audio] device`. So every sound fails before ALSA's default, and therefore `/etc/asound.conf` and the host's audio server, is ever consulted. The failure is logged and skipped by design, so that a robot with no codec still runs, which is precisely why a silent duck looks like a sound bug rather than a device that is not there. `device = "default"` in both modes, which is what makes the asound.conf pointing at PulseAudio mean anything. Three rounds on this and each layer was real: no `aplay` in the rootfs, then a rootfs that never rebuilt because `boot` asked for it only when it was missing, and now the device name.
A perfect ether hides the bugs a real one causes. Four ducks in the twin converged on one piece every
time — simultaneous starts, staggered starts, it made no difference — because every duck was visible
to every other instantly and losslessly. On hardware, BLE discovery is slow and lossy, so two ducks
can be singing before the other two have noticed them, which is the split-brain the election has to
survive. The simulator was not modelling the one property that causes the bug.
`--discovery` makes a duck take a while to be *noticed*, per pair and timed from when it goes on the
air — per pair because it is the asymmetry that splits a flock, and one delay shared by everybody
cannot produce it. `--loss` drops a fraction of deliveries. Both come from a seeded splitmix, so a
split that happens once happens again: a flaky radio is only useful for debugging if its flakiness
repeats.
With `--discovery 90 --loss 0.3 --seed 3`, four ducks and a staggered start, on main's chorale:
duck-a: listening — 1 ducks in range (never sings)
duck-b: bass bar 4 beat 13.3 3 voices
duck-c: alto bar 4 beat 13.4 2 voices
duck-d: bass bar 2 beat 2.0 2 voices
Which is the field report — "sometimes nothing happens, sometimes 2 different songs" — with the
disagreeing rosters and a duplicated part thrown in. On a laptop, on demand.
The twin now produces the chorale's field symptom on demand: four ducks, a staggered start, and `duck-ether --discovery 90 --loss 0.3 --seed 3`. One duck never sings, two are on bar 4 with rosters of 3 and 2, and the fourth is on bar 2 with a duplicated part. "Sometimes nothing happens, sometimes two different songs", deterministically, on a laptop. Recorded with the caveat that matters: `chorale-election` merged does not fix *this* scenario (three timelines, bars 5, 12 and 8), but at `--discovery 20 --loss 0.4` both versions converge — so ninety seconds of discovery is harsher than that branch was written for, and this is not yet evidence about a robot. The sweep that would settle it is a loop over one number, which is the whole point of having built the thing.
`Source::Sim(addr)` beside `Test` and `Camera`: an `appsrc` fed by a thread reading length-prefixed
UYVY frames off a socket. Verified end to end — `capture rate fps=31.2 target=30`, the console
answering 200, and `source=Sim("127.0.0.1:7901")` in the log — so the encoder, the console and the
duck detector's raw tap all have a picture of an apartment now.
Raw and length-prefixed rather than JSON like the other two simulator links, because a 640x360 UYVY
frame is 460,800 bytes and 14 MB/s of base64 would be a joke. The tradeoff is that geometry cannot be
negotiated, so a frame of the wrong size is refused with a message naming both numbers rather than
pushed as a picture nobody can read.
`is-live` and `do-timestamp`, both deliberately: an `appsrc` is not live unless told, and a pipeline
that can be pulled faster than real time is one `webrtcsink` will pull faster than real time; without
timestamps every element downstream invents its own, which presents as a stream at the wrong speed
rather than as an error. The reader owns the reconnect, because MuJoCo restarts whenever the number
of ducks changes and a camera going away must not take the pipeline with it — which is also what
unplugging a real one looks like.
`360p30` is exactly 640x360, so the simulated camera fits a mode the robot already has and nothing
needed a new one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W5Ff83FWy3bU5LxvWNJu4U
In robotctl configure, `detect.enabled` one line from `chorale.accept` read as "detect what?". The section, its struct (DuckDetectorParams) and the robotctl namespace are named for the thing detected. A file that still says [detect] loads through a serde alias and the editor carries it to the new name on its next save, the way [imu_head] was handled. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… raw branch Since the H.264 stream branch (cf0a476) nothing reading the raw tee branch got a frame: every duck detector report said starved=20, and auto-exposure never logged a metering line, on a robot whose camera captured at 30 fps and whose console video was fine. A sink prerolls on its first buffer, and the bin does not finish going to PLAYING until every async sink has. The H.264 appsink sits behind a valve that is shut until media.stream asks, so its first buffer never came, the pipeline never completed its state change, and the raw appsink — which had prerolled — waited for PLAYING for ever with its first buffer in hand: no callbacks, no frames. webrtcsink is live and does not preroll, which is why the video track kept working and the fault was invisible from the console. async=false on the H.264 appsink, so a sink with nothing to show does not gate the pipeline. And a videoconvert in front of the encoder: the tee carries UYVY, which neither x264enc nor mpph264enc takes, so on a machine without mpph264enc the branch failed to link and mediad did not start at all — every laptop, and the sim twin. The test starts the real pipeline on the test pattern and asks for a frame; it fails without the async change and skips where the plugins are not installed (CI has neither webrtcsink nor an H.264 encoder). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t-pattern The test pattern runs small, and the key that picks it says what it does
…h-starved The valved H.264 sink held the whole pipeline in PAUSED, starving the…
# Conflicts: # mediad/src/main.rs
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`duckctl ssh` already resolves the robot's address and hands the terminal over; copying a file off a robot still meant `scp radxa@$(duckctl ip):…`, which is the substitution `ssh` exists to remove. The robot side is `scp`'s own `host:path` with the host left out, so `duckctl scp report.md :/tmp/` sends one up and `duckctl scp :/var/log/robotd.log .` brings one down. The rewrite is textual and only ever looks at the first character, which is what keeps `-r`, `-P 2222`, several sources and a path with a colon in the middle of it working without this tool having to understand any of them. Both directions, then, out of one command that answers to the name people reach for — rather than a `push` and a `pull` that nobody typing `scp` would find. A copy with no `:` anywhere in it is refused before the radio is turned on. `scp` performs that copy happily, locally, and nothing in its output says the robot was never involved; refusing it after the scan would also charge eight seconds for the diagnosis. The `exec` that makes `ssh` become `ssh` is now `become_program`, shared by both, so the progress meter, the key prompts and the exit status are `scp`'s own for the same reason they were ssh's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…from-hub The duck detector comes from the Hub, the way the policies do
duckctl: scp, with `:path` for the robot
A candidate whose properties read fails — unplugged between the read_dir and the read, so the file is gone — returned None for the whole search, and the real IMU sorting after it was never seen: no IMU head control and a frozen monitor panel for the rest of that pad's session. Skip it and keep walking.
A read failure fell straight back into open_imu: a chip that answers its ID but cannot stream was reopened in a tight loop, warning each time, on the bus the audio codec shares. Same backoff as the open-failure path, which the module docs always claimed.
With two pads connected, the event drain matched no pad id: a Start or Select from the second pad toggled the policy or shut the robot down while the sticks it combined with were the first pad's. The driving pad is the first one; events from any other are skipped, and after the drain the pad is re-read by id — a Disconnected dequeued this tick must yield one tick of "pad gone", not a silent handover to the other pad's sticks.
The contact odometry picked its anchor from four hardcoded corners: the v1.5 sole bbox, ±27.0 x ±20.6 mm on the foot-site Z = 0 plane, a placeholder since the port. The alpha sole is 54 mm long, sits 7 mm forward of the site, and its bottom is canted 4.9° about X in the site frame, so those corners float 2 mm above the real contact on the low side and 3 mm under the mesh on the high side. `anchors.rs` is GENERATED by microduck_rl/scripts/odom_anchor_points.py, which drops a grid of vertical rays onto the sole collision mesh: V15 the legacy corners (unchanged behaviour) ALPHA4 the flat contact patch's four corners, on the mesh ALPHA16 a 4x4 grid over the whole footprint, bevels included `Odometry` takes an `&'static AnchorSet`; `Odometry::alpha()` still uses V15, so robotd is unchanged until the sets have been compared (infer_policy --odom-compare in microduck_rl). `alpha_with(&ALPHA16)` is the switch. Cost, `update_cost_per_anchor_set` on an x86 laptop: 484 ns per update with 4 points per foot, 951 ns with 16. On the Zero 3's A55 expect a handful of µs — nothing next to a 20 ms tick. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014ZPSQ6nCsfqannMDXwAKZ3
Measured in the MuJoCo twin (infer_policy --odom-compare, perfect IMU, 3.2 m of keyboard-driven walking, 23.8 s): set final xy err max xy err final z err v15 17.3 mm 24.5 mm +3.7 mm alpha4 15.6 mm 21.6 mm +0.9 mm alpha16 9.1 mm 19.7 mm +0.5 mm Odometry::alpha() now runs the 16-point grid. V15 and ALPHA4 stay as named sets for alpha_with() and the cost/reference tests. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014ZPSQ6nCsfqannMDXwAKZ3
…or-sets Odom anchor sets
The velstand gait (velstand.onnx, pushed to pollen-robotics/microduck-policies and tagged v5) walks on a twist and stands still at zero command, so the walk-mode defaults become: walk = velstand.onnx, stand = none. With no standing network to hand back to, limp_fall ships off; voltage_adapt ships on. A daemon whose default names a file only v5 carries would fail to load its gait on a board that updated the daemon but kept an older set — unhealthy, rolled back. So the pin in Cargo.toml / seed-policies.sh is now a minimum rather than a floor: the seeder reads the installed set's .source record and moves an official set below the pin up to it. A set past the pin, from another repo, without a record, or with a non-numeric version is left alone as before. The fallback file list tracks robotd's defaults (alpha_walking / alpha_stand drop out, velstand comes in), per the xtask test that pins the two together. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…default policies: velstand is the default walk, set v5, the pin is a minimum
§6 argued that only the robot needs to offer a relay candidate: a connection needs one and not two, and `aiortc`'s TURN client does not work, so a Python consumer can never be that side. Both halves are true. The conclusion assumed something it did not check — that the two ends can address each other at all. An iPhone on a mobile network cannot. It has no IPv4 socket: it reaches a hostname through DNS64/NAT64, the STUN server reports an IPv4 reflexive address back, and the phone gathers a candidate saying so. But an ICE candidate is a bare literal, and the robot's relay candidate is a bare IPv4 literal on a board with no global IPv6 at all, which that phone cannot send a packet to. Measured on olducky over 4G: six sessions, `offering relay candidates relays=5` every one, `Ice connection state ... failed` every one, eight seconds apart. The same symptom a missing relay gives, and a different cause — which is why the robot-side endpoint fix did not move it. So the page mints credentials for itself. Only its own allocation bridges this: `turn.cloudflare.com` is a name, so it resolves over IPv6, and the relayed address Cloudflare hands back is IPv4, which the robot can reach. Confirmed from the phone before this was written — those same credentials in a Trickle ICE page gathered a `relay` candidate with an IPv4 address over 4G, where the robot's own candidates had paired with nothing. With the visitor's token rather than the robot's, which is the right way round twice: a robot's allowance should go on being watched rather than on watching, and a browser signed in with `hf_oauth` already holds a token of its own. `refreshRelays` runs on the connect path, which is already asynchronous and already says "connecting…". `newPeerConnection` reads what it stored and never fetches — the same rule the robot follows in `consumer-added`, and for the same reason: that handler cannot wait without delaying every session, LAN sessions included, which will never use a relay. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The endpoint `mediad::turn` fetched from was dead for three months before anybody noticed, and it was already dead when the default that named it shipped. Nothing in this repository would have caught that: the only symptom is a warning in a journal and a missing candidate type, and every check we have pairs two peers on one network, which never looks at a relay. One authenticated GET, asserting 200 and at least one turn:/turns: entry. Three deliberate choices. The URL is read out of turn.rs with sed rather than copied, because a check carrying its own endpoint drifts from what the daemon compiles in, and a green check on a URL no robot uses reads as proof while proving nothing. It is scheduled rather than on pull_request: the failure being guarded against is nobody touching this for months, which a PR trigger cannot see, and somebody else's outage must not block unrelated work. And a missing HF_TOKEN fails rather than skips, because a check that skips itself into permanent silence is precisely the failure mode it exists to end. It will fail until an HF_TOKEN secret is set on the repository. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…et-path-before-bind robotd: refuse a non-socket path before bind, on every platform
padd: only the driving pad's buttons may act
…search padd: an unreadable sibling does not end the IMU search
…-endpoint mediad(turn): the relay endpoint is the Space, not the alias in front of it
…ffers-its-own-relay console: offer a relay of this page's own, for a consumer with no IPv4
Three things this repository could not tell somebody who wanted to run a model too heavy for the board. There was no page about it at all. `spaces/` appears nowhere under `docs/` — not in the index, not linked, nothing. The only writing was a Space card published to Hugging Face and a lot of source comments. What writing existed argued the wrong way. `media.stream` was built when the relay endpoint was dead and WebRTC genuinely could not connect from a data centre, so `stream.rs` made its case at length and, once the dead-endpoint premise was removed, read as though outbound frames were the preferred design. They are not. WebRTC carries encrypted media, a control channel on the same session and a return path; `media.stream` has none of those and exists for a program consuming frames only on a long-running stream, where a relay's metered bandwidth is the cost that matters. And nothing recorded the rule for whoever changes this next, which is how it would have been re-derived backwards from the volume of prose in `stream.rs`. So: `docs/faq.md`, task-shaped rather than linear, because somebody arrives at this with a situation and not a curiosity. `AGENTS.md` with `CLAUDE.md` symlinked to it, carrying the transport rule and three others. `stream.rs` demoted to the fallback it is, with what it gives up stated rather than implied. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…rst-and-a-faq docs: WebRTC is the default transport, and a FAQ that says so
…ckoff tof: back off before reopening the head IMU after a read failure
A frame off the tee is the picture the sensor took, and this robot's camera is mounted a quarter turn off — so a snapshot came out sideways with nothing in the reply to say by how much. The geometry cannot recover it: a 180° mount is indistinguishable from an upright one, and a quarter turn is only a guess from the aspect ratio. `MediaFrameHeader` now carries `rotate`, degrees clockwise from upright, the same number `media.video` already tells a WebRTC peer — and zero when `--flip-in-pipeline` turned the pixels, the rule the detector's sampler and the JPEG streamer already follow. `valid_uyvy` refuses anything that is not a quarter turn, so a nonsense angle is rejected rather than silently ignored. The console's `/frame` applies it instead of reporting it. That route is the one consumer with nowhere to put an angle: a PNG opened in a browser carries no metadata a viewer will act on, so reporting it there would hand every human a sideways picture and no way to know why. It costs one rotation per request on a blocking thread, not one per frame in front of the encoder, which is what made `videoflip` expensive. The recipe in npu-bringup.md grew the `-vf transpose=1` it was missing; it produced a sideways image and said nothing about it. Assisted-by: Claude:claude-opus-5
The merge that gave `handle` a `rotate` argument updated the call sites that ask for a frame but not the two that never get that far, so `mediad` failed to compile as a test target. Assisted-by: Claude:claude-opus-5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
検証用のフォーク内PR。 上流へ出す前に CI（ubuntu + gstreamer）で
cargo clippy --workspace --all-targetsとcargo test --workspaceを通すために立てている。mediad/src/frame.rsは#[cfg(target_os = "linux")]配下で、pipelineが Linux 限定である以上必然的にそうなる。macOS では1行もコンパイルされない — 上流レビューで
Id::String/Id::Textの取り違えがcargo test --workspaceを緑のまま通り抜けたのと同じ構造。上流レビュー（pollen-robotics#214）で指摘された4点
Id::String→Id::Textrobotグループへchown（tof/paddと同じ形。group が無ければ warning）take()で上限。後から長さを見る形をやめたarchitecture.mdのサービス表（"no unix socket of its own" を更新）上流の変更に伴う設計差分
Framesが「常に最新を保持」から**「要求されたときだけ捕捉」**に変わっていた（1.84 MiB × 30fps の memcpy を避けるため）。そのため
media.frameの契約が変わる。FRAME_TIMEOUT= 500ms で打ち切り）。停止したカメラは「止まった時のフレーム」ではなくタイムアウトとして報告される
next_frameは condvar でブロックするのでspawn_blocking経由で呼んでいる。含めていないもの
#214のうちmedia.frameだけを切り出している。policy_enabled、required_files/max_artifact_bytes、LeRobot 録画、robotctl supportは別。