A DREAM Robotics Remote
- Type: Handheld remote (RIFT in your hand)
RITA is a 3.5" touch screen that shows the DREAM Robotics fleet and talks to it through RIFT: every robot online, what each one says about itself, and a button for everything it can do.
How far away: RITA hears the fleet's beacons, over ESP-NOW and Bluetooth LE (fleet_near.h, fleet_near_ble.h, the same files as NORA and WHIP). On DRIVE, beside the robot's avatar, she shows about how far it is (from its signal strength: about -45 dBm at 1 m, so a rough figure; walls and how you hold her change it), whether it's coming closer or going away, and which radio heard it. The KIDAs are heard over Bluetooth LE only. IDA has no radio of her own, and DREAM and ARM have no beacon.
STAY CLOSE: RITA sends the fleet's beacon ("hand"), over ESP-NOW and Bluetooth LE, once she's on a fleet network, so robots can tell how near she is. Her STAY button on NORA's and WHIP's pages makes them keep near her, about a metre away, going round things on the way: they slow down near something and sidestep round it toward open space (NORA's mecanum wheels, WHIP's crab), still heading for her, and back out of corners. Tested in simulation (round a wall in 47 of 50 pretend rooms, no bumps); not on the floor yet.
Colours: colour_scheme.xml comes from her avatar: navy, cream, burnt orange and CRT teal, on her site and her screen.
 ARIA |
 KIDA-01 |
 KIDA-00 |
 WHIP |
 NORA |
 MILA |
 NIKA |
|
 IDA |
 ARM |
 DREAM |
 RIFT |
 NINA |
 TINA |
 AURA |
 FLORA |
Website: https://cursedprograms.github.io/RITA/
Overview
- Splash: her TV-head avatar for 5 s at power-up.
- Home: every robot she drives, its round avatar with its name under it: WHIP, NORA, MILA, NIKA, KIDA-00, KIDA-01, IDA, ARM, ARIA and DREAM. Each avatar has a dot like a chatroom's: green when the robot is on the network, red (and the avatar dimmed) when it isn't.
- A controller for each robot: tap one and drive him or her, talking to the robot directly (their own web API, the one their PC controller uses). Hold a pad button to drive, let go and it stops. Each robot gets its own pad and buttons:
- WHIP (hexapod): W / A / S / D with curves, crab left / right, HELLO, DANCE, REST, AUTO, CRAB - / + (his crab offset, a degree a press).
- ARIA (drone): her state, battery and network, and the fleet's phrases. No flying: she flies from her app or a gamepad.
- NORA (mecanum): A / D slide, diagonals slide, TURN L / R, AUTO, UV, music, her wheel lock.
- MILA, NIKA, KIDA-00, KIDA-01 (tracks): WASD or TANK, switched at the top. The controls change, and so does the robot's own drive mode. The screen takes one finger at a time, so in TANK the tracks latch: tap the left track's FWD / STOP / BACK, then the right's, and each keeps going until you change it. LINK moves both together; the big STOP stops both.
- IDA (no WiFi): through NORA's IR link.
- Cameras (CAM): KIDA-00's camera as her own HUD shows it, with what she sees marked; KIDA-01's two cameras (CAM 0 / CAM 1: cam 0 has her YOLO boxes while her inference is on), each beside a small pad so you can drive while you watch; and DREAM's camera with her detections (people, objects, faces). They're MJPEG streams, read a little at a time so RITA never stalls.
- ARM (no WiFi): driven from her PC over USB, so RITA just says so.
- DREAM: a small mirror of her. RITA plays the clip DREAM is showing, from the SD card and from the same frame, so the two screens move together, and DREAM's voice comes out of the speaker connector (SOUND turns it off). Without an SD card RITA shows DREAM's avatar and what she's doing. See The SD card.
- STATS (on HOME): every robot's sensors in one table, TEMP, HUMIDITY, DISTANCE, BATTERY and WIFI (his or her signal), from each robot's own
/status, asked one robot at a time so she never stalls. The robots all name these differently (temp,temperature,hum,distance_cm,range_cm,distances_cm.front,battery_pct,battery_v...), so she finds them by name, and new sensors show up by themselves. Tap a robot for everything he or she reports. - Her own web page (http://rita.local, or her IP; http://192.168.4.1 while she hosts): in step with her, it shows what's on her screen right now and what she's doing (the robot she's driving, the pad button held, the camera, DREAM's state, her last message), her network, and the fleet with its sensors and an Open site button for each robot's own site (DREAM: her dashboard). It refreshes every second; driving stays on her touchscreen. The page is plain files in
scripts/esp32/web/(index.html,style.css,app.js);scripts/make_web.pyembeds them, andflash.batruns it before every compile. - QR codes: WEB on HOME shows one to her page (and, while she hosts, one that joins her WiFi "RITA" first). Each robot's STATS page has one to his or her own site.
- She hosts when she has to: her rung on the fleet's ladder is above NORA. With RIFT, DREAM, NINA, TINA and the controllers all off, she hosts "RITA" (192.168.4.1, up to 10 devices, LED blue) with the fleet list on :5000, so the robots join her; when a higher network is back she hands over and joins it.
- RIFT too: from a controller, RIFT: ASK / DO shows what the robot says about itself through RIFT and everything RIFT can do with it, as 1 s timed moves that ask "Let RITA move NORA?" first.
- Safety stays with the robots: their own stops (walls, tilt, the wheel locks) still hold. Their PIN (
ROBOT_PINinsecrets.h) signs RITA in as a controller; unlocking wheels is its own button (UNLOCK). - The fleet's WiFi ladder: her rung is under the controllers and above NORA. She joins the highest network she can see above her (RIFT, DREAM, NINA, TINA, the controllers); with none, she hosts "RITA" herself (see below). She looks again every minute and moves up when a higher one is back. She registers with the hub as
RITA(typeremote). - RGB LED, in colour (PWM): the network as a slow breath, green on RIFT's, amber on another (the list works, ask and do need RIFT), blue while she hosts, red with none; red blinking while the robot she's driving doesn't answer; on DREAM's screen it follows her (an orange pulse talking, teal listening, violet thinking, a dim blue breath asleep).
- KIDA-00's camera servo: a CAM slider under her pad and under her camera picture; drag it and the camera follows.
- TALK (on HOME): she joins the fleet's conversations in Brainfuck, each phrase beeped on her speaker in her own voice (a little higher than the fleet's), whenever she isn't playing DREAM's voice. RIFT calls
/chirp?u=<n>on her; she registerstalk:80while TALK is on. Tap it off and she keeps quiet; she remembers which. - Made for a power bank: the screen dims after two minutes without a touch (not while you watch a camera or DREAM), and the first touch only wakes it, so nothing gets pressed by accident.
- SD card, more of it: the robots' avatars as short looping clips on HOME (the built-in avatars stay, for a card-less RITA); an animated boot splash; a screensaver of DREAM's clips when RITA's been idle a while; her WiFi and robot list in a
rita.jsonon the card; a log of what she sent and what each robot answered; music through the speaker. - DREAM's mirror from
scripts/dream.pyand the C++dream.exetoo (today it's her page:app.py, the browser andbuilds/dream.exe). - RIFT's music together, the mission log and the robots' conversations.
- Start a robot's MIND mode from her (through its PC controller).
Hardware
| Part | Notes |
|---|---|
| ESP32-3248S035R | The 3.5" "Cheap Yellow Display": ESP32, 480x320 ST7796 screen, XPT2046 resistive touch, RGB LED, SD slot, speaker connector |
| USB cable | Power and flashing (micro-USB or USB-C, depending on the board revision) |
| Power bank | Any USB power bank: what she runs on in your hand (her screen dims when idle to make it last) |
| Speaker (optional) | On the speaker connector: DREAM's voice and her Brainfuck talk (or a headphone jack there) |
Software
- Arduino IDE (or arduino-cli) with the esp32 board package (3.x)
- Libraries: LovyanGFX, ArduinoJson (7.x)
ESP32-3248S035R
| Part | Pins |
|---|---|
| ST7796 screen (HSPI) | SCLK 14, MOSI 13, MISO 12, DC 2, CS 15, backlight 27 |
| XPT2046 touch (same bus) | CS 33, IRQ 36 |
| RGB LED (active low) | R 4, G 16, B 17 |
| microSD (VSPI, its own bus) | SCK 18, MISO 19, MOSI 23, CS 5 |
| Speaker connector (DAC 2, through the board's amplifier) | 26 |
Negative colours? Flip PANEL_INVERT in scripts/esp32/display.h. Red and blue swapped? PANEL_BGR.
View How to Run
- Copy
scripts/esp32/secrets.example.htosecrets.hand put in the fleet's WiFi passwords (the same as every robot's). - Flash her:
flash.bat(on a fresh PC it installs the Arduino command line tool, the ESP32 board package and her libraries by itself, then finds the port, compiles and flashes, talking you through it), or openscripts/esp32/esp32.inoin the Arduino IDE with ESP32 Dev Module and the Huge APP (3MB No OTA) partition scheme, like NORA and WHIP: one big slot, room for Bluetooth LE (2.1 MB today). - On the first boot, tap each corner as it lights up (touch calibration, kept in flash). Hold the screen while she starts to calibrate again.
- She finds the fleet's network by herself. Tap a robot to see and drive them.
- New firmware always goes over the cable: with one big app slot there's no room for an update over WiFi. This board doesn't reset into its flashing mode by itself, so hold BOOT while the screen flickers, until it goes black.
DREAM on RITA (optional)
With a microSD card RITA can show what DREAM is showing. Run make_sdcard.bat (needs ffmpeg): it converts DREAM's clips (../DREAM/videos) into sdcard/dream/ as MJPEG (480 x 270, 15 fps, back-to-back JPEGs: an ESP32 has no video decoder, but it draws JPEGs) with a frame index each. Copy everything in sdcard/ onto a FAT32 card and put it in RITA. Details: sdcard/README.md.
Tap DREAM on HOME. DREAM's page tells her own app which clip it started and when; DREAM serves that on a small plain-HTTP, read-only mirror (port 5015, DREAM/scripts/dream_mirror.py), because her dashboard is HTTPS and every TLS handshake costs an ESP32 about a second. RITA asks twice a second and draws the frame DREAM is on, so they move together however late RITA joined. When DREAM speaks, RITA streams the line (16 kHz, 8-bit) to the DAC on the speaker connector, from as far in as DREAM already is: a speaker there, or a headphone jack. SOUND turns it off and she remembers.
See her screens with no board
emulator\RITA-Emulator.exe is RITA emulated in C++ on the PC, ready to run (its DLLs are next to it). preview.bat (or preview.sh) builds it fresh as build/preview/rita_preview.exe. It runs her real esp32.ino, drawn by the real LovyanGFX, on the real fleet: nothing is made up. She joins the ladder the way the board does, seen from the PC (a fleet network is there when the PC's WiFi sees it or its hub answers; the PC's own WiFi is never switched), and her web calls are real HTTP to RIFT and the robots. Her web page runs on port 80, DREAM's voice and her Brainfuck phrases play on the PC's speakers, the robots' cameras are their real streams, and her settings are remembered between runs. What a PC hasn't got, ESP-NOW and Bluetooth LE, she simply doesn't hear. The repo's sdcard/ folder is her SD card. --hub HOST (or RITA_HUB) points her at a hub elsewhere. The window is her screen, the mouse is your finger and the console prints every web call she makes. Only preview.bat --shots uses a pretend fleet, so the screenshots always look the same.
| Key | Does |
|---|---|
| 1-9, 0 | HOME: open WHIP ... ARIA, DREAM |
| W A S D / arrows | hold to drive (the pad), let go to stop |
| Q / E | crab (WHIP) or turn on the spot (NORA) |
| Space | STOP |
| T | WASD / TANK |
| R | RIFT: ask and do |
| M | DREAM's sound on / off |
| C | CAM: the camera (KIDA-00, KIDA-01, DREAM) |
| V | KIDA-01: cam 0 / cam 1 |
| Backspace | back |
| Esc | quit |
preview.bat --shots writes these pictures to images/screens, and fails if holding a pad button doesn't drive, letting go doesn't stop, or a robot has more buttons than DRIVE shows. |
It needs g++ and SDL2 (Windows: MSYS2's mingw64), Pillow for the pictures, and the libraries flash.bat installs.
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Minds — DREAM companion · TINA engineer · NINA caretaker · RIFT fleet hub · LYCEA training school
Robots — NORA · MILA · WHIP · KIDA-00 · KIDA-01 · IDA · ARM · NIKA · ARIA · AURA weather · FLORA gardener
Remotes — RITA handheld remote