sdk: a python client, and a loop to write behaviours in - #271
Open
Nixxx19 wants to merge 3 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
stacked on #266, which is the socket this talks to. M5's finish line is "a server-side script can fetch a frame and send an intent in a few dozen lines", and this is the other half of it.
the investigation M5 says to do first — whether one client covers both webrtc and websocket — i think answers itself once you look at spaces/shared/control.py. that Rpc has already carried the same json-rpc over three transports: a datachannel through the rendezvous, a datachannel on the lan, and http with no webrtc at all. so the transport was never the hard part. the sdk binds a websocket to that same object rather than being a fourth implementation of the wire.
the part i think is actually worth having is
watch(). one call each is fine for a script that asks a question, but a behaviour is a loop, and what it looks at arrives on three streams at three different rates — state at the control rate, depth at 15 hz, frames at whatever you asked for. merging those is what every caller would otherwise write for itself:view.nearestfollowstof::Frame::zonerather than re-deriving the thresholds, including the bit that bites: the sensor returns negative distances on a failed convergence and still flags them valid, so taken at face value they are the nearest thing in the room. there's a test for exactly that.no perception in here.
view.frameis the jpeg as it arrived and what's in it is the caller's model to run.frames are worth a look too — the robot doesn't serve them, it dials out to a socket the sdk opens, so there's no relay candidate or ice anywhere.
drove it against the real agent route with a fake robotd behind it: health, move and stop round-trip, system.pairingPin comes back "not available over this transport", and the loop merges state and depth and picks the right nearest. ten tests on the wire and the depth rules, no robot needed.
couldn't run it against a full mediad locally — webrtcsink isn't packaged in any debian suite so the container has no pipeline — so the live check covers the route and the dispatcher, not the camera.