Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 23 additions & 5 deletions docs/design/remote-access-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -850,13 +850,31 @@ consumer behind whatever a cloud provider gives a container, srflx-to-srflx need
allow a hole to be punched — often they do, and often enough they do not. A relay always works, at
the cost of somebody's bandwidth, which is why ICE tries it last.

**Only the robot offers one**, and that is the part worth knowing before writing any of it: a
connection needs *one* relay candidate, not two. If the robot offers one, a consumer that can
reach the internet uses it — so credentials live on the robot and a consumer needs none. Which
matters more than it sounds, because `aiortc`'s STUN client works where its TURN client does not,
so a Python consumer *cannot* be the side that relays. `reachy_mini`'s #1182 established this
**One relay candidate is enough, and it is not always the robot's.** A connection needs *one*, not
two, and `aiortc`'s STUN client works where its TURN client does not — so a Python consumer cannot
be the side that relays, and the robot has to be. `reachy_mini`'s #1182 established that
arrangement and `mediad::turn` is the same one.

**What that argument left out is whether the two ends can address each other at all**, and an
iPhone on a mobile network is the case where they cannot. It has no IPv4 socket: it reaches a
*hostname* through DNS64/NAT64, the STUN server reports an IPv4 reflexive address back, and the
phone gathers a candidate saying so. But an ICE candidate is a bare literal, and the robot's relay
candidate is a bare IPv4 literal on a board with no global IPv6 at all — which that phone cannot
send a packet to. Measured on olducky: six sessions, `offering relay candidates relays=5` every
time, `Ice connection state … failed` every time, about eight seconds apart.

So **the console offers a relay of its own** (`refreshRelays` in `mediad/webclient/index.html`),
and only its own allocation can bridge this: `turn.cloudflare.com` is a name, so it resolves over
IPv6, and the relayed address Cloudflare hands back is IPv4, which the robot can reach. Confirmed
from the phone before it was written — the same credentials in a Trickle ICE page gathered a
`relay` candidate with an IPv4 address over 4G, where the robot's own candidates paired with
nothing.

The page mints them with **the visitor's** token, not the robot's, which is the right way round
twice over: a robot's allowance should go on being watched rather than on watching, and a browser
signed in with `hf_oauth` already holds a token of its own. A LAN session asks for none — there
are host candidates on both sides and nothing would use a relay.

The credentials are Cloudflare's, minted per account by a proxy Hugging Face hosts and
authenticated with **the same token the relay signs in with** — so a robot that belongs to
somebody can offer a relay and one that belongs to nobody cannot, which is the same line §2 draws
Expand Down
66 changes: 62 additions & 4 deletions mediad/webclient/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,13 @@ <h2>telemetry</h2>
? PARAMS.get("mode") === "remote"
: !SERVED_BY_ROBOT && location.protocol === "https:";

// Where this page mints its own relay credentials, addressed as the Space it is rather than
// through `turn.fastrtc.org` — that alias is a dangling delegation and answers nothing
// (`remote-access-design.md` §6). `?turn=` points this at a local copy or a fake.
const TURN_CREDENTIALS_URL = PARAMS.get("turn")
|| "https://fastrtc-turn-service.hf.space/credentials";
const TURN_TTL_SECONDS = 600;

// The Space the robot's relay registers with. `?rendezvous=` points this at a local copy.
const RENDEZVOUS = (PARAMS.get("rendezvous")
|| "https://pollen-robotics-reachy-mini-central.hf.space").replace(/\/$/, "");
Expand Down Expand Up @@ -429,6 +436,9 @@ <h2>telemetry</h2>
// The rendezvous half: the account token, a handle on the open event stream, and which robot of
// the account's this page is driving.
let token = null, stream = null, chosen = null;
// The relay servers this page holds for itself, in the `{urls, username, credential}` shape
// `RTCPeerConnection` takes. Empty until `refreshRelays` has run, and empty forever on a LAN.
let relays = [];
// Whether a session is wanted. Set by a person — the connect button, or picking another robot —
// and cleared when the robot ends one.
//
Expand Down Expand Up @@ -711,6 +721,10 @@ <h2>telemetry</h2>
$("account").hidden = false;
}
setState("connecting…");
// Before the session rather than during it: `newPeerConnection` runs in a message handler
// that cannot wait, and credentials are short-lived enough that minting them per connect
// is simpler than keeping a fresh set around.
await refreshRelays();
log("dim", "opening the rendezvous event stream…");
let spoke = false;
stream = await openRendezvous((data) => {
Expand Down Expand Up @@ -901,6 +915,46 @@ <h2>telemetry</h2>
}
}

/// Mint relay credentials for **this visitor**, so the page can offer a relay of its own.
///
/// §6 of `remote-access-design.md` argued only the robot needs to offer one: a connection needs
/// one relay candidate and not two, and `aiortc`'s TURN client does not work, so a Python
/// consumer could never be the side that relays. Both halves are still true, and the conclusion
/// was still wrong for a browser — because it assumed the two ends can address each other at all.
///
/// **An iPhone on a mobile network has no IPv4 socket.** It reaches a *hostname* through
/// DNS64/NAT64 and the STUN server reports an IPv4 reflexive address back, so it looks like it
/// has IPv4 and gathers a candidate saying so. But an ICE candidate is a bare literal, and the
/// robot's relay candidate is a bare IPv4 literal, which that phone cannot send a packet to.
/// Six sessions negotiated perfectly and carried nothing, which is the same symptom a missing
/// relay gives and a different cause.
///
/// Its own allocation is what bridges the two, and only its own can: `turn.cloudflare.com` is a
/// name, so it resolves over IPv6, and the relayed address Cloudflare hands back is IPv4 — which
/// the robot can reach. One relay candidate is still enough. It just has to be this end's.
///
/// Metered against the visitor's account rather than the robot owner's, which is the right way
/// round: a robot's allowance should go on being watched, not on watching.
async function refreshRelays() {
relays = [];
if (!REMOTE || !token) return;
try {
const response = await fetch(`${TURN_CREDENTIALS_URL}?ttl=${TURN_TTL_SECONDS}`, {
headers: { authorization: `Bearer ${token}` },
});
if (!response.ok) throw new Error(`HTTP ${response.status}`);
const offered = (await response.json()).iceServers || [];
// A `stun:` entry has no credentials and is not a relay; the STUN server is set below.
relays = offered.filter((server) => server.username && server.credential);
log("dim", `relay credentials: ${relays.length} server(s)`);
} catch (e) {
// Best effort, always. Without a relay this page still reaches every robot it could reach
// before, which is most of them — so this is a line in the log and not a failed connect.
log("bad", "!! no relay credentials:", e.message || String(e),
"— a session from a mobile network may negotiate and carry nothing");
}
}

function newPeerConnection() {
// **STUN only when it is needed, which is when this page is not on the robot's network.**
//
Expand All @@ -912,11 +966,15 @@ <h2>telemetry</h2>
// nothing.
//
// `stun.l.google.com` is what `webrtcsink` already defaults to on the robot side, so the two
// ends use one server rather than two. TURN is the next thing this needs — see
// `remote-access-design.md` §6, and `reachy_mini`'s #1182, where the *robot* fetches
// short-lived Cloudflare credentials with its own HF token.
// ends use one server rather than two.
//
// **And this page's own relay servers, when `refreshRelays` got any.** Reading them here rather
// than fetching is the same rule the robot follows in its `consumer-added` handler: this runs
// inside a signalling message handler, and an HTTP request in it would delay the answer to
// every session, including the LAN ones that will never use a relay. `relays` is filled on the
// connect path, which is already asynchronous and already says "connecting…".
pc = new RTCPeerConnection(REMOTE
? { iceServers: [{ urls: "stun:stun.l.google.com:19302" }] }
? { iceServers: [{ urls: "stun:stun.l.google.com:19302" }, ...relays] }
: undefined);

pc.ontrack = (event) => {
Expand Down
Loading