A demo Space that processes the duck's camera, and what it measured - #223
Closed
pierre-rouanet wants to merge 12 commits into
Closed
pierre-rouanet wants to merge 12 commits into
pierre-rouanet wants to merge 12 commits into
Conversation
`gr.LoginButton` raises `Cannot initialize OAuth due to a missing library` without the `oauth` extra, and `requirements.txt` asked for bare `gradio`. It would have worked anyway: a Hugging Face Space installs `gradio[oauth,mcp]` of its own accord, which is visible in the build log — so the app was standing on somebody else's default and would have broken the first time it ran anywhere else. Found by building the UI against the version the Space actually installs, with the consumer stubbed: `gradio[oauth]==6.26.0`, OpenCV, numpy, and a fake `ReachyCentralConsumer`. That exercises the import, the Blocks construction, the four states the status panel distinguishes and every filter through the render path — all without a robot, a token or a three-minute Space build. Everything Gradio 6 was checked for still exists under the same names, `height` included. Assisted-by: Claude:claude-opus-5[1m]
`cp "$SOURCE"/*` published whatever was lying around, and something always is: running the app from its own directory leaves a `__pycache__` beside it, which is a directory, which `cp` without `-r` refuses, which stopped the script after it had cloned and before it had pushed. Listing tracked files fixes both halves. Bytecode and scratch files cannot reach a Space, and a Space always corresponds to a commit here — a file that has not been committed does not deploy, which is the answer to "which version is live" the first time somebody asks. Assisted-by: Claude:claude-opus-5[1m]
…environment
`ResolutionImpossible`, and the resolver is right:
gradio 6.26 needs starlette >=1.0.1, huggingface-hub >=1.16
reachy-mini needs starlette <1.0.0 — every version, 1.10 included
A Gradio Space installs `gradio[oauth,mcp]==6.26.0` of its own accord, so the one
version that cannot coexist with this consumer is the one version such a Space
has. Not a matter of waiting for an upgrade either: the constraint is on every
`reachy-mini` release, not the current one.
Gradio 5.23 resolves with the same consumer in 117 packages, so the version is
the thing to control — which a Docker Space can and a Gradio Space cannot. That
is the second Space in this branch where the managed environment was the obstacle
rather than the shortcut; the console reached `sdk: docker` from the other
direction, an OAuth client id a static Space would not inject.
The pin is written down with the reason, because `gradio[oauth]>=5,<6` looks like
timidity and is arithmetic.
Assisted-by: Claude:claude-opus-5[1m]
Both of them are Docker now, so the parting line about Gradio installing requirements was wrong twice over — and the build log is the thing worth pointing at anyway. Assisted-by: Claude:claude-opus-5[1m]
`reachy_mini[central-consumer]` proved the transport and was the wrong
dependency, three times over: `reachy_mini/__init__.py` imports the daemon's
world, so a consumer drags in PyGObject — a source build wanting a C compiler and
cairo headers for GStreamer bindings it never touches — and the package pins
`starlette<1.0.0`, which no current Gradio can satisfy, and it ignores a data
channel labelled `control`, so the camera geometry could not be asked for at all.
We wrote the robot half of this protocol. The client half is 300 lines and
smaller than the workarounds: SSE in, `POST /send` out, aiortc answering the
robot's offer, frames as numpy. Verified against `olducky`:
frame 8 (720, 1280, 3) uint8
media.video: {'width': 1280, 'height': 720, 'rotate': 90,
'intrinsics': {'fx': 1809.52, 'cx': 640.0, 'calibrated': False}}
pc_state: connected ice_state: completed control: True
That second line is the point. The intrinsics come from the robot now, so the
demo's overlay stops being arithmetic repeated in two places — and the fifteen
lines that make DTLS complete against GStreamer's webrtcsink are copied with the
credit and the upstream link, because they are the reason this took an afternoon
rather than a week.
**And the teardown ordering is written down because remembering it did not
work.** `POST /send` is refused once the event stream has gone, so the farewell
has to precede the cancel. I noticed exactly this bug in their consumer an hour
before writing it here — said ours could not do it, which was true of the Rust
relay and not of this — and the first end-to-end run raised on the way out. It
now says goodbye first and raises nothing: a teardown that throws leaves a UI
holding a session it believes is alive.
Assisted-by: Claude:claude-opus-5[1m]
`ModuleNotFoundError: No module named 'consumer'` — because the script was in a scratchpad, reaching into a clone by absolute path, in a `/tmp` that gets wiped. A check that only runs from one directory on one machine is a check nobody runs twice. `spaces/vision-demo/check_consumer.py` imports its neighbour, is committed with it, and says in its own docstring what each stage of a successful run proves — including the one their consumer cannot reach, `media.video` answering on the control channel. Assisted-by: Claude:claude-opus-5[1m]
`git diff --quiet` ignores untracked files, so a publish whose only change was a new file said "the Space already serves this" and pushed nothing. That is the worst shape a wrong answer can take: it reads exactly like the right one, and the file it skipped was the check script somebody would go looking for. Staged first, compared as staged. Additions, deletions and modifications all count now. Assisted-by: Claude:claude-opus-5[1m]
`COPY app.py filters.py ./` built a perfect image without `consumer.py` in it, and the app died on import — the same `ModuleNotFoundError` the local check hit an hour earlier, from the same cause in a different place. That is three of these in one afternoon: `cp "$SOURCE"/*` published whatever was lying around, `git diff --quiet` ignored a file that was only ever added, and now a `COPY` naming two of three modules. Each is an enumeration somebody has to keep in step with the truth, and each failed silently in the direction of looking fine. `COPY *.py ./`, with the publisher already guaranteeing that only tracked files are there to be globbed. Assisted-by: Claude:claude-opus-5[1m]
Gradio decided it was not in a Space — `OAUTH_CLIENT_ID` is not in a Docker Space's environment — fell back to *mocked* OAuth, and refused to start without a local `huggingface-cli login`. The app crashed on `gr.Blocks.__exit__`, which is before any of it runs. That is the second time today a sign-in was blocked by platform behaviour rather than by code: the console's static Space never injected the client id either, and that one at least had a Docker escape. Here Docker *is* the escape and the variable still is not there. So the button goes and an `HF_TOKEN` secret takes its place, with the trade written down rather than glossed: a visitor's own token would reach their robots and nobody else's, which is what would make this Space safe to publish, and without one **the Space must stay private** — a visitor would otherwise reach the owner's robot with the owner's token. Still never the robot's own credential: the rendezvous maps a token to one peer, so sharing the robot's takes the robot off its own listing. A demo that will not start is worse than a demo with one credential. Verified the way the last three of these should have been: the Blocks builds against the Gradio the Space installs, the four status states render, every filter runs through the render path, and the status line now reports what the robot says about its own camera — 1280×720, mounted 90° off upright, fx 1810 px (nominal). Assisted-by: Claude:claude-opus-5[1m]
It printed nothing of its own. The consumer logs at INFO and nothing had configured a handler, so every line about the welcome, the producer, the session and the peer connection state went nowhere — leaving a private Space whose only diagnostic is a status pill somebody has to read out. `basicConfig` at INFO, aiortc and aioice turned down to WARNING so the log is readable, and the connection state logged on every change: `connecting` that never becomes `connected` is no candidate pair, `failed` is ICE having tried them all, and those two want different conversations. Assisted-by: Claude:claude-opus-5[1m]
The demo Space got as far as it can, and its log is the measurement §6 was
missing:
the rendezvous welcomed this consumer as 9d3a17dd (PierreRouanet)
found olducky (8a73eda9), asking for a session
session 4c7e9c60 started; waiting for the robot's offer
peer connection connecting (ice checking)
the session ended: the robot ended the session ← 8s later
peer connection failed (ice failed)
Signalling crosses perfectly and no candidate pair works. The robot offers host
and srflx, the Space offers its own, neither offers a `relay` — and eight seconds
in, `webrtcsink` times out a consumer whose ICE never connected, ends the session,
and the bridge forwards that, which is why the robot's farewell arrives before
ICE's own verdict.
This turns a guess into a fact. On one LAN the console works on host candidates;
from a 4G phone it failed and CGNAT was the suspicion; from a Space it fails too,
and a Space is not behind CGNAT. **TURN is not a fallback for awkward networks.**
It is the requirement for every consumer that is not on the robot's own wifi,
which is every cloud backend and therefore anything doing perception off-robot.
Nothing in the demo needs changing for it: it will connect the day the robot can
offer a relay candidate.
Assisted-by: Claude:claude-opus-5[1m]
Coverage72.94% lines on this branch, against a floor of 70%. Per-file |
pierre-rouanet
marked this pull request as draft
September 7, 2026 14:49
Member
Author
|
Superseded. Main grew its own What this PR was carrying has landed or been answered elsewhere:
The third publisher bug is the one thing main still has, so it goes in on its own: #284. Merging this as it stands would also revert |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Everything after #219 merged. A Gradio Space on Hugging Face hardware consumes the robot's camera through the rendezvous and runs OpenCV over each frame — and on its first run it measured the thing nothing else could.
The measurement
Signalling crosses perfectly and no candidate pair works. Three data points now instead of one: a browser on the robot's LAN connects on host candidates, a 4G phone fails and CGNAT was the suspicion, and a Hugging Face Space fails too — and a Space is not behind CGNAT. So TURN is not a fallback for awkward networks; it is the requirement for every consumer that is not on the robot's own wifi, which is every cloud backend and therefore anything doing perception off-robot. §6 carries the log.
Nothing in the demo needs changing for it: it will connect the day the robot can offer a relay candidate.
The consumer is ours
reachy_mini[central-consumer]proved the transport — it was the first non-browser client to pull frames out of a duck — and was the wrong dependency, three times over.reachy_mini/__init__.pyimports the daemon's world, so a consumer drags in PyGObject: a source build wanting a C compiler and cairo headers for GStreamer bindings it never touches. It pinsstarlette<1.0.0, which no current Gradio can satisfy (ResolutionImpossible, correctly). And it ignores a data channel labelledcontrol, so the camera geometry cannot be asked for.spaces/vision-demo/consumer.pyis 300 lines: SSE in,POST /sendout, aiortc answering the robot's offer, frames as numpy. Verified againstolducky:That second line is the payoff: the intrinsics come from the robot, so the demo's geometry overlay uses published numbers rather than repeating the arithmetic. The fifteen lines that make DTLS complete against GStreamer's
webrtcsinkare copied from their consumer with the credit and the upstream link (aiortc PR #1392) — they are the reason this took an afternoon rather than a week.Six dependencies instead of 117:
aiortc,av,httpx,gradio[oauth],opencv-python-headless,numpy.What the Space is
spaces/vision-demo, deployed byscripts/publish-space.sh— source in-repo for §5's reason, since a consumer tracks the rendezvous protocol, this project's method names and the camera's geometry. Docker rather thansdk: gradio, because a Gradio Space cannot choose its Gradio version and 6.26 is the one version that cannot work here.Five filters — raw, Canny edges, motion diff, sparse Lucas–Kanade flow, and an intrinsics overlay — with the pixels in
filters.pyimporting only OpenCV and numpy, so every one is exercised on a synthetic sideways frame pair without a WebRTC stack, a robot or a token.It authenticates with an
HF_TOKENsecret and must stay private. A visitor's own OAuth token would be better and would make it publishable; two attempts failed on platform behaviour rather than code — a static Space never injected the client id (§5.0), and a Docker Space does not putOAUTH_CLIENT_IDin the environment either, so Gradio mocks OAuth and refuses to start. A demo that will not start is worse than a demo with one credential.Four bugs of mine, three of them the same bug
A hand-kept list that has to match reality, failing silently in the direction of looking fine:
cp "$SOURCE"/*published whatever was lying around,git diff --quietreported "already serves this" for a file that was only ever added, andCOPY app.py filters.pybuilt a perfect image withoutconsumer.pyin it. All three now ask git or the filesystem what is actually there.The fourth:
stop()posted its farewell after cancelling the event stream, which the service refuses with a 400. I had diagnosed that exact bug in their consumer an hour earlier and said ours could not have it — true of the Rust relay, not of the Python I had yet to write. The ordering is a docstring now.