From 2dd630ad4269f7a4b904f6786f7c3245466a6a3e Mon Sep 17 00:00:00 2001 From: Pierre Rouanet Date: Mon, 14 Sep 2026 12:34:28 +0200 Subject: [PATCH] docs: WebRTC is the default transport, and a FAQ that says so MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three things this repository could not tell somebody who wanted to run a model too heavy for the board. There was no page about it at all. `spaces/` appears nowhere under `docs/` — not in the index, not linked, nothing. The only writing was a Space card published to Hugging Face and a lot of source comments. What writing existed argued the wrong way. `media.stream` was built when the relay endpoint was dead and WebRTC genuinely could not connect from a data centre, so `stream.rs` made its case at length and, once the dead-endpoint premise was removed, read as though outbound frames were the preferred design. They are not. WebRTC carries encrypted media, a control channel on the same session and a return path; `media.stream` has none of those and exists for a program consuming frames only on a long-running stream, where a relay's metered bandwidth is the cost that matters. And nothing recorded the rule for whoever changes this next, which is how it would have been re-derived backwards from the volume of prose in `stream.rs`. So: `docs/faq.md`, task-shaped rather than linear, because somebody arrives at this with a situation and not a curiosity. `AGENTS.md` with `CLAUDE.md` symlinked to it, carrying the transport rule and three others. `stream.rs` demoted to the fallback it is, with what it gives up stated rather than implied. Co-Authored-By: Claude Opus 5 --- AGENTS.md | 41 ++++++++++++++++++++++ CLAUDE.md | 1 + docs/README.md | 4 +++ docs/faq.md | 83 ++++++++++++++++++++++++++++++++++++++++++++ mediad/src/stream.rs | 40 +++++++++++---------- 5 files changed, 150 insertions(+), 19 deletions(-) create mode 100644 AGENTS.md create mode 120000 CLAUDE.md create mode 100644 docs/faq.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..7ddaf1b1 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,41 @@ +# Working in this repository + +`CONTRIBUTING.md` is the reference: building, testing, layout, conventions, releasing. This page +is the short list of things that are easy to get backwards, and where the answer lives when they +are not here. + +## Docs own mechanisms; one page each + +`docs/README.md` assigns every mechanism to one design doc. When a fact belongs to a page listed +there, every other page says one sentence and links. When two pages disagree, the one that does +not own the mechanism is the bug — and when behaviour and a design doc disagree, the doc is the +bug. [`docs/faq.md`](docs/faq.md) is the task-shaped front door for someone building against a +robot rather than changing it. + +## A consumer uses WebRTC. `media.stream` is the fallback + +The robot publishes H.264 over WebRTC, and that is the default for anything consuming a duck's +camera — it is encrypted end to end, it carries the control channel on the same session, and it +has a return path. It works from a data centre because the robot offers a relay candidate +(`remote-access-design.md` §6). + +`media.stream` — the robot dialling an outbound WebSocket and pushing frames to you — is the +fallback for a **program** consuming **frames only** on a **long-running** stream, where relay +metering is the thing that matters. It has no return path and no control channel. + +This is worth stating because the repository reads the other way round if you only follow the +code: `media.stream` was built when the relay endpoint was dead and WebRTC genuinely could not +connect from a data centre, so its module doc argues its own case at length. That endpoint is +fixed. Do not conclude from the volume of prose that it is the preferred path. + +## Never design around a version difference + +One user, one robot. An old component's limits are a question to raise, not something to route +around — bump `API_VERSION` and name the install consequence. A version skew is logged and served, +never refused; only a genuinely missing route or an unknown parameter may refuse. + +## Releases are how a fix reaches a robot + +`main` being fixed is not a robot being fixed. Robots on the stable channel move when a release is +cut, and a dev build from a branch is superseded by the next `daemon-dev-main` the board's +six-hourly check finds. `docs/design/updater-design.md` owns the mechanism. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 120000 index 00000000..47dc3e3d --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file diff --git a/docs/README.md b/docs/README.md index 7faf2e6b..7b654d14 100644 --- a/docs/README.md +++ b/docs/README.md @@ -3,6 +3,10 @@ The [README](../README.md) is the front door — what a microduck is, and where to go. If you have one in front of you and want to drive it, start at the [cheat sheet](robot/cheatsheet.md). +[`faq.md`](faq.md) is the other front door: task-shaped questions from somebody building +*against* a duck rather than changing it — running a model too heavy for the board, getting the +camera into their own program, why a Space cannot connect. + It is also where a **publisher** starts: [`policy-manifest.md`](policy-manifest.md) is the contract for a `manifest.json` beside a microduck `.onnx`, and it owns every field. The design docs give the reasoning and point at it. diff --git a/docs/faq.md b/docs/faq.md new file mode 100644 index 00000000..ef026610 --- /dev/null +++ b/docs/faq.md @@ -0,0 +1,83 @@ +# FAQ + +Questions that come up when you want a duck to do something it does not do out of the box. The +design docs say how the machinery works; this says which piece to reach for. + +## I want to run a model that is too heavy for the board + +Run it somewhere else and send it the camera. The board is a Radxa Zero 3 — the NPU takes a small +detector and little more — so anything larger belongs off the robot, and a Hugging Face Space on +paid hardware is the path with the least to build: it already has a GPU option, an account system +the robot shares, and a URL. + +**Use WebRTC.** The robot publishes H.264 over a WebRTC session, your Space consumes it, and the +rendezvous introduces the two so neither needs to know the other's address. `spaces/vision-demo/` +is the worked example. + +``` +your Space ──sign in, list robots──► rendezvous ◄──registers── the duck +your Space ◄═══════════ H.264 over WebRTC, media and control ════════════► the duck +``` + +Three things you get for free by staying on WebRTC, and they are the reason it is the default: +the media is encrypted end to end by DTLS-SRTP, the control channel rides the same session so you +can *drive* as well as watch, and there is a return path — audio, or a second video track — the +day you want one. + +## Does it work from a data centre? My Space is behind a NAT I do not control + +Yes. A robot behind a home router and a container behind a data centre's NAT usually cannot +hole-punch to each other, so the robot offers a **relay candidate** — short-lived Cloudflare +credentials it mints with its own account token — and your consumer uses it without holding any +credentials of its own. `remote-access-design.md` §6 is the mechanism. + +Two caveats worth knowing before you are surprised by them: + +- **A relay costs the robot owner's bandwidth**, metered per Hugging Face account at 10 GB a + month on the free tier. A continuous 720p stream is roughly a gigabyte an hour, so a Space that + watches all day will spend it. ICE only relays when it must — a direct pair is used whenever + one can be found — but "must" is common between a home and a data centre. +- **A browser consumer needs relay credentials of its own** when it has no IPv4 of its own; a + Python one does not. §6 has the case. + +## My consumer is a program and the stream runs all day. Is there something cheaper? + +There is a fallback, and it is a fallback rather than a second design: **`media.stream`**, where +the robot dials *your* WebSocket and pushes frames outbound. + +``` +your Space ──media.stream {url: "wss://…/frames"}──► rendezvous ──► the duck +the duck ══════════ frames, outbound wss, direct ══════════════► your Space +``` + +It costs nobody's relay, because an outbound connection from the robot needs no hole punched. Use +it when all three are true: the consumer is a **program** and not a person, the stream is +**long-running** enough for relay metering to matter, and you need **frames only**. + +What you give up by leaving WebRTC, which is why it is not the default: + +- **No return path at all.** Nothing reaches the robot on this transport — no audio, no second + track, no teleop loop. +- **No control channel.** Driving the robot means a JSON-RPC call over the rendezvous + (`spaces/shared/wire.py`), separately. +- **Encryption is your TLS, not DTLS-SRTP**, and it terminates at your server rather than at the + peer. + +The robot sends one text **hello** describing what is coming — `frames.encoding` is `h264` or +`jpeg` — then one binary message per frame. Branch on the hello rather than sniffing the bytes; +`spaces/vision-demo/receiver.py` is the reference receiver and `decoder_for` is the branch. + +## How does my Space find the robot, and what stops somebody else's reaching it? + +The account. The robot registers with the rendezvous holding its own Hugging Face token, your +Space signs the visitor in with `hf_oauth`, and the service only ever shows an account the robots +it owns. Nothing is configured on the robot and no robot-side gate is involved — +`remote-access-design.md` §7 is the argument for why that is enough. + +`spaces/shared/rendezvous.py` is the listing call: `ducks(token)` gives you what that account can +reach. + +## Can I test without a robot? + +`spaces/vision-demo/fake_duck.py` stands in for one, and `scripts/duck-sim` runs the real daemons +against a MuJoCo body ([`robot/simulation.md`](robot/simulation.md)). diff --git a/mediad/src/stream.rs b/mediad/src/stream.rs index 61656303..168c5061 100644 --- a/mediad/src/stream.rs +++ b/mediad/src/stream.rs @@ -1,17 +1,18 @@ //! Frames out to a WebSocket **this robot dials**, for a Space that runs a model on them. //! -//! # Why the robot dials, and why that is the whole idea +//! # This is the fallback, and WebRTC is the default //! -//! The goal is a Space on Hugging Face hardware processing this camera. The obvious route is the -//! one `vision-demo` takes — a WebRTC consumer pulls the stream through the rendezvous — and it -//! runs into the one thing WebRTC cannot do without help: a robot behind a home router and a -//! container behind a data centre's NAT need a **relay candidate** to pair -//! (`remote-access-design.md` §6). Signalling crosses; media needs somebody's relay. +//! **A consumer should use WebRTC**, which carries encrypted media, a control channel on the same +//! session and a return path, and which reaches a data centre because the robot offers a relay +//! candidate (`remote-access-design.md` §6). `docs/faq.md` is the decision, in the shape somebody +//! arrives at it. This module is for the narrow case WebRTC serves badly: a **program** consuming +//! **frames only** on a **long-running** stream, where a relay's metered bandwidth is the cost +//! that matters. //! -//! An outbound WebSocket needs nobody's. **The robot already proves this every second it is -//! reachable**: `relay.rs` holds an outbound HTTPS stream to a Space right now, and nothing about -//! a home router objects. So the frames go the same way the registration does — outward — and NAT -//! stops being a participant. +//! In that case an outbound WebSocket needs nobody's relay. **The robot already proves this every +//! second it is reachable**: `relay.rs` holds an outbound HTTPS stream to a Space right now, and +//! nothing about a home router objects. So the frames go the same way the registration does — +//! outward — and NAT stops being a participant. //! //! ```text //! Space ──media.stream {url: "wss://…/frames"}──► rendezvous ──► this robot @@ -22,18 +23,19 @@ //! this scale where relaying payload through a shared service would not: one small envelope per //! session, on a service the mini fleet also depends on, and the bytes go point to point. //! -//! **This is not a workaround for a relay that does not exist**, and it used to read like one. -//! `turn.rs` offers relay candidates and they work. The argument above is the argument either -//! way: a relay is metered per Hugging Face account, so every one of these frames would be spent -//! against an allowance (§6: 10 GB a month) that the robot's owner also needs for being -//! *watched* — and the shortest path between a board and a data centre is not through a third -//! one. A relay is the fallback for a session that cannot be made direct. This one can. +//! **This was written when the relay endpoint was dead and WebRTC could not connect from a data +//! centre at all.** That is fixed (§6), so the reason this exists is now the narrow one above and +//! not "the alternative does not work". What survives of the original argument is the cost: a +//! relay is metered per Hugging Face account at 10 GB a month, and a stream that runs all day +//! spends an allowance its owner also needs for being *watched*. //! //! # What it is not //! -//! Not a replacement for a relay candidate in general. There is no return media path, so nothing -//! here helps a browser *watch* a robot, carries audio, or closes a teleop loop — a viewer wants -//! WebRTC and §6 is still what it needs. This is for the case where the consumer is a program. +//! **Not a replacement for WebRTC, and not the path to reach for first.** There is no return +//! media path and no control channel, so nothing here helps a browser *watch* a robot, carries +//! audio, or closes a teleop loop; driving means a separate JSON-RPC call over the rendezvous. +//! Encryption is the receiver's TLS rather than DTLS-SRTP, terminating at a server instead of at +//! the peer. A consumer that is not all three of program, frames-only and long-running wants §6. //! //! # This half is portable, and that is deliberate //!