diff --git a/CLAUDE.md b/CLAUDE.md index 3b3920e0c..673bef72d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,3 +1,298 @@ # Claude Code Instructions Read `agents.md` in this directory for full instructions on developing Reachy Mini applications. + +--- + +## Project Overview + +**Reachy Mini** is a Python SDK and daemon for controlling the Reachy Mini robot, a small expressive robot with a 6-DOF head (Stewart platform), body rotation, and two antenna motors. The project is developed by [Pollen Robotics](https://github.com/pollen-robotics). + +- **Package name:** `reachy_mini` +- **Version:** 1.3.0 +- **License:** Apache 2.0 +- **Python:** >= 3.10 +- **Build system:** setuptools + +--- + +## Repository Structure + +``` +reachy_mini/ +├── src/reachy_mini/ # Main package source code +│ ├── reachy_mini.py # Core SDK class (ReachyMini) +│ ├── apps/ # App system (assistant, manager, templates) +│ ├── daemon/ # FastAPI daemon server +│ │ ├── app/ # Uvicorn entry point, API routers, dashboard +│ │ ├── backend/ # Robot, MuJoCo, Mockup backends +│ │ └── routers/ # REST API route handlers +│ ├── io/ # Zenoh communication (client/server) +│ ├── kinematics/ # IK/FK: analytical, Placo, neural network +│ ├── motion/ # Motion primitives (Move, Goto, RecordedMove) +│ ├── media/ # Audio/video: OpenCV, GStreamer, WebRTC +│ ├── tools/ # Camera calibration, motor reflash utilities +│ ├── utils/ # Hardware config, interpolation, wireless utils +│ ├── descriptions/ # URDF & MuJoCo robot models +│ └── assets/ # Firmware binaries, audio files, NN models +├── tests/ # Pytest test suite +├── examples/ # Runnable example scripts (23 files) +├── docs/ # Full Markdown documentation +│ └── source/ +│ ├── SDK/ # SDK guides (quickstart, python-sdk, core-concept) +│ ├── platforms/ # Platform-specific docs (Lite, Wireless, Simulation) +│ ├── examples/ # Example tutorials +│ └── API/ # Auto-generated API reference +├── skills/ # AI agent skill guides (12 files) +├── agents.md # Main AI development guide +├── agents.local.md.template # User config template +├── pyproject.toml # Build config, dependencies, tool settings +├── conftest.py # Pytest marker registration +├── MANIFEST.in # Package data inclusion +├── uv.lock # Dependency lock file (uv) +└── .pre-commit-config.yaml # Pre-commit hooks +``` + +--- + +## Development Setup + +### Install dependencies + +```bash +# Using uv (preferred): +uv pip install -e ".[dev]" + +# Using pip: +pip install -e ".[dev]" +``` + +### Optional extras + +| Extra | Purpose | +|-------|---------| +| `dev` | pytest, ruff, mypy, pre-commit | +| `mujoco` | MuJoCo physics simulation | +| `nn_kinematics` | ONNX neural network kinematics | +| `placo_kinematics` | Placo IK solver (Linux/macOS only) | +| `gstreamer` | GStreamer media (PyGObject) | +| `rerun` | Rerun 3D visualization | +| `wireless-version` | Wireless hardware support (Linux only) | +| `all` | Everything above | + +### Entry points (console scripts) + +| Command | Module | +|---------|--------| +| `reachy-mini-daemon` | `reachy_mini.daemon.app.main:main` | +| `reachy-mini-app-assistant` | `reachy_mini.apps.app:main` | +| `reachy-mini-reflash-motors` | `reachy_mini.tools.reflash_motors:main` | + +--- + +## Code Quality + +### Linting and formatting: Ruff (v0.12.0) + +```bash +# Check linting +ruff check src/ + +# Auto-fix +ruff check --fix src/ + +# Format +ruff format src/ +``` + +**Configuration (pyproject.toml):** +- Enabled rules: `I` (import sorting), `D` (docstrings) +- Ignored: `D203`, `D213` (conflicting docstring style rules) +- Excluded: `src/reachy_mini/__init__.py`, `build/`, `conftest.py`, `tests/`, `src/reachy_mini_dashboard/` + +### Type checking: MyPy (v1.18.2) + +```bash +mypy --install-types --non-interactive +``` + +- Target: Python 3.10 +- **Strict mode enabled** +- Scans `src/` directory +- `ignore_missing_imports = true` + +### Pre-commit hooks + +```bash +pre-commit install +pre-commit run --all-files +``` + +Runs `ruff-check` and `ruff-format` via the `astral-sh/ruff-pre-commit` hooks. + +--- + +## Testing + +### Running tests + +```bash +# Run all non-hardware tests (standard CI command): +pytest -vv -m 'not audio and not video and not audio_gstreamer and not video_gstreamer and not wireless and not wireless_gstreamer' --tb=short + +# Set this env var when running without a display: +MUJOCO_GL=disable pytest ... +``` + +### Test markers + +Tests that require physical hardware or specific system capabilities are marked: + +| Marker | Requires | +|--------|----------| +| `audio` | Audio hardware | +| `audio_gstreamer` | GStreamer audio | +| `video` | Video/camera hardware | +| `video_gstreamer` | GStreamer video | +| `wireless` | Wireless Reachy Mini connected | +| `wireless_gstreamer` | GStreamer on Wireless | + +### Test files + +| File | Tests | +|------|-------| +| `test_import.py` | Basic package import checks | +| `test_app.py` | App system functionality | +| `test_daemon.py` | Daemon server behavior | +| `test_analytical_kinematics.py` | Analytical IK/FK | +| `test_placo.py` | Placo kinematics solver | +| `test_collision.py` | Collision detection | +| `test_video.py` | Video/camera (marked: video) | +| `test_audio.py` | Audio playback/recording (marked: audio) | +| `test_wireless.py` | Wireless-specific features | + +Test fixtures include `ok_app/` and `faulty_app/` directories for app system testing. + +--- + +## CI/CD (GitHub Actions) + +| Workflow | Trigger | What it does | +|----------|---------|--------------| +| `pytest.yml` | PR on `src/`, `tests/`, `pyproject.toml` | Matrix tests: Ubuntu + macOS, Python 3.10, 10min timeout | +| `lint.yml` | Push/PR on `src/`, `tests/`, `pyproject.toml` | Ruff linting + MyPy type checking | +| `wheels.yml` | Release created | Build & publish to PyPI | +| `build_documentation.yml` | Documentation changes | Build docs | +| `build_pr_documentation.yml` | PR documentation changes | Build PR docs preview | +| `upload_pr_documentation.yml` | After PR docs build | Upload PR docs artifact | +| `uv-lock-check.yml` | Dependency changes | Validate uv.lock is up to date | + +--- + +## Architecture + +### Core SDK (`src/reachy_mini/reachy_mini.py`) + +The `ReachyMini` class is the primary interface. Key methods: + +- **Connection:** `ReachyMini()` with context manager support +- **Motion:** `goto_target()` (interpolated), `set_target()` (real-time) +- **State:** `wake_up()`, `goto_sleep()`, `get_current_joint_positions()`, `get_current_head_pose()` +- **Vision:** `look_at_image()`, `look_at_world()` +- **Motors:** `enable_motors()`, `disable_motors()`, `enable_gravity_compensation()` +- **Recording:** `start_recording()`, `stop_recording()`, `play_move()`, `async_play_move()` +- **Media:** `mini.media` property (audio/video access) + +### Daemon (`src/reachy_mini/daemon/`) + +FastAPI server exposing REST/WebSocket API at port 8000. + +**Backends:** +- `RobotBackend` - Real hardware via serial +- `MujocoBackend` - Full physics simulation +- `MockupSimBackend` - Lightweight mock + +**API routes:** `/api/apps/`, `/api/daemon/`, `/api/move/`, `/api/state/`, `/api/motors/`, `/api/kinematics/`, `/api/logs/`, `/api/cache/`, `/api/wifi_config/`, `/api/hf_auth/`, `/api/volume/`, `/api/update/` + +### Communication + +Uses **Zenoh** middleware for pub/sub messaging between SDK clients and the daemon, over localhost or network. + +### Kinematics + +Three solver implementations: +- `analytical_kinematics.py` - Fast analytical solution +- `placo_kinematics.py` - Constraint-based (Linux/macOS) +- `nn_kinematics.py` - Neural network via ONNX runtime + +### Media + +Pluggable backends for audio/video: +- **OpenCV** (Lite/default), **GStreamer** (Wireless), **WebRTC** (remote Wireless) +- Audio via `sounddevice` (Lite) or GStreamer (Wireless) + +--- + +## Key Conventions + +### Code style +- All source under `src/reachy_mini/` (src layout) +- Strict MyPy type checking - maintain full type annotations +- Ruff-formatted with import sorting +- Docstrings follow D211/D212 conventions + +### Robot safety limits +- Head pitch/roll: [-40, +40] degrees +- Head yaw: [-180, +180] degrees +- Body yaw: [-160, +160] degrees +- Yaw delta (head - body): max 65 degrees +- SDK clamps values automatically + +### Motor names +`body_rotation`, `stewart_1` through `stewart_6`, `right_antenna`, `left_antenna` + +### Interpolation methods +`linear`, `minjerk` (default), `ease`, `cartoon` + +### App development +- Always use Python for discoverable/shareable apps +- Use `reachy-mini-app-assistant create ` to scaffold +- Web UIs go in a `static/` subdirectory +- Create `plan.md` before implementing any app + +--- + +## Documentation + +Full SDK documentation lives in `docs/source/`: + +| Topic | Path | +|-------|------| +| Quickstart | `docs/source/SDK/quickstart.md` | +| Python SDK reference | `docs/source/SDK/python-sdk.md` | +| Core concepts | `docs/source/SDK/core-concept.md` | +| AI/LLM integration | `docs/source/SDK/integration.md` | +| Media architecture | `docs/source/SDK/media-architecture.md` | +| Troubleshooting | `docs/source/troubleshooting.md` | +| Platform guides | `docs/source/platforms/` | + +--- + +## Skills Reference (for AI agents) + +Detailed guides in `skills/` for specific topics: + +| Skill | When to use | +|-------|-------------| +| `setup-environment.md` | First session setup | +| `create-app.md` | Creating new apps | +| `control-loops.md` | Real-time reactive apps | +| `motion-philosophy.md` | Choosing goto_target vs set_target | +| `safe-torque.md` | Motor enable/disable patterns | +| `ai-integration.md` | LLM-powered apps | +| `symbolic-motion.md` | Choreography / mathematical motion | +| `interaction-patterns.md` | Antennas as buttons, head as controller | +| `debugging.md` | Troubleshooting crashes and connectivity | +| `testing-apps.md` | Testing before delivery | +| `rest-api.md` | HTTP/WebSocket API usage | +| `deep-dive-docs.md` | When to read full documentation | diff --git a/docs/HELP.md b/docs/HELP.md new file mode 100644 index 000000000..ebbf9c678 --- /dev/null +++ b/docs/HELP.md @@ -0,0 +1,114 @@ +# Reachy Mini -- First-Run Survival Guide + +Get from unboxing to your first successful interaction in under 15 minutes. + +--- + +## What Reachy Mini Is (and Is Not) + +Reachy Mini is a small expressive robot with: +- A **6-DOF head** (pitch, roll, yaw + x/y/z translation via a Stewart platform) +- A **body** that rotates around its vertical axis +- **Two antennas** that double as physical buttons + +It is **not** a humanoid arm robot. It does not pick things up. It expresses, reacts, looks, listens, and speaks. + +--- + +## Minimum Requirements + +| Requirement | Details | +|-------------|---------| +| **Python** | 3.10 -- 3.12 | +| **OS** | Linux, macOS, or Windows | +| **Power** | 7V / 5A power supply (included). USB alone will NOT power the motors. | +| **Connection** | Lite: USB-C cable. Wireless: WiFi on the same network. | +| **Git + Git LFS** | Required to clone repo and download model assets | + +--- + +## Golden Path: Box to First Interaction + +### 1. Assemble the robot +Follow the printed booklet or the [interactive digital guide](https://huggingface.co/spaces/pollen-robotics/Reachy_Mini_LITE_Assembly_Guide). Budget 2--3 hours. + +### 2. Power up and connect +- Plug in the **power supply** (wall outlet, not USB). +- **Lite:** Connect the USB-C cable to your computer. +- **Wireless:** Power on, connect to `reachy-mini-ap` WiFi (password: `reachy-mini`), then configure your home WiFi at `http://reachy-mini.local:8000/settings`. + +### 3. Install the SDK +```bash +# Install uv (fast Python package manager) +curl -LsSf https://astral.sh/uv/install.sh | sh # Linux/macOS +# Or: powershell -c "irm https://astral.sh/uv/install.ps1 | iex" # Windows + +# Create a virtual environment and install +uv venv reachy_mini_env --python 3.12 +source reachy_mini_env/bin/activate # Linux/macOS +# Or: reachy_mini_env\Scripts\activate # Windows +uv pip install reachy-mini +``` + +### 4. Start the daemon +```bash +# Lite (USB): +uv run reachy-mini-daemon + +# Simulation (no robot): +uv run reachy-mini-daemon --sim + +# Wireless: daemon is already running on the robot. +``` + +### 5. Verify the dashboard +Open http://localhost:8000 in your browser. If you see the Reachy Dashboard, you are ready. + +### 6. Run your first script +Save this as `hello.py` and run `python hello.py`: + +```python +from reachy_mini import ReachyMini + +with ReachyMini() as mini: + print("Connected!") + mini.goto_target(antennas=[0.5, -0.5], duration=0.5) + mini.goto_target(antennas=[-0.5, 0.5], duration=0.5) + mini.goto_target(antennas=[0, 0], duration=0.5) + print("Done!") +``` + +If the antennas wiggle, everything works. + +--- + +## First-Hour Failures and Fixes + +| Symptom | Cause | Fix | +|---------|-------|-----| +| Motors don't move | Power supply not connected | Plug in the 7V wall adapter. USB alone is not enough. | +| `Connection refused` | Daemon not running | Open a terminal and run `reachy-mini-daemon` | +| `PortAudio library not found` | Missing system dependency (Linux) | Run `sudo apt-get install libportaudio2` | +| `Permission denied` on serial (Linux) | USB permissions not set | See HELP_SOFTWARE_SETUP.md, "Linux USB Permissions" section | +| Dashboard not loading | Browser blocking localhost | Check browser privacy settings for local network access | +| Wireless robot not found | Not on the same WiFi network | Connect your computer to the same network as the robot | + +--- + +## What to Read Next + +| Goal | File | +|------|------| +| Understand the hardware | [HELP_HARDWARE.md](HELP_HARDWARE.md) | +| Detailed environment setup | [HELP_SOFTWARE_SETUP.md](HELP_SOFTWARE_SETUP.md) | +| Move the robot safely | [HELP_FIRST_MOTION.md](HELP_FIRST_MOTION.md) | +| Understand control modes | [HELP_CONTROL_MODES.md](HELP_CONTROL_MODES.md) | +| Something went wrong | [HELP_WHEN_THINGS_GO_WRONG.md](HELP_WHEN_THINGS_GO_WRONG.md) | + +--- + +## Community and Support + +- **Discord:** https://discord.gg/Y7FgMqHsub +- **GitHub Issues:** https://github.com/pollen-robotics/reachy_mini/issues +- **Community Apps:** https://huggingface.co/spaces?q=reachy_mini diff --git a/docs/HELP_CALIBRATION.md b/docs/HELP_CALIBRATION.md new file mode 100644 index 000000000..1acdd4fc1 --- /dev/null +++ b/docs/HELP_CALIBRATION.md @@ -0,0 +1,160 @@ +# Reachy Mini -- Calibration Guide + +Make precision feel achievable. Know when calibration is needed and how to fix drift. + +--- + +## When Calibration Is Required + +Reachy Mini's Dynamixel motors use absolute encoders, so **you typically do not need to calibrate joint positions.** The motors know where they are after power-on. + +However, calibration is relevant in these situations: + +| Situation | What to do | +|-----------|-----------| +| **Camera image is dark** | Adjust exposure (see below) | +| **Motor shows wrong position after reassembly** | Check motor orientation marks | +| **Head drifts or feels off-center** | Verify motor ID mapping and physical alignment | +| **Audio direction-of-arrival is inaccurate** | Microphone array calibration (firmware-level) | +| **Antenna appears rotated 90/180 degrees** | Physical repositioning (manufacturing offset) | + +--- + +## Camera Exposure Calibration + +The most common "calibration" task. If the camera image appears dark: + +### Quick Fix (Any OS) +Enable auto-exposure or increase exposure time using a camera control app: + +| OS | Application | +|----|------------| +| **macOS** | [CameraController](https://github.com/itaybre/CameraController) | +| **Linux** | `qv4l2` (install: `sudo apt install qv4l2`) | +| **Windows** | [Webcam Settings](https://www.softpedia.com/get/Internet/WebCam/Webcam-Settings-Tool.shtml) | + +### Programmatic Fix (Linux) +```bash +# Install v4l2 utilities +sudo apt install v4l-utils + +# List available controls +v4l2-ctl --list-ctrls + +# Set auto-exposure priority (fixes darkness) +v4l2-ctl --set-ctrl=auto_exposure_priority=1 +``` + +### From Python (OpenCV) +```python +import cv2 + +cap = cv2.VideoCapture(0) +cap.set(cv2.CAP_PROP_AUTO_EXPOSURE, 3) # Auto exposure on +# Or manual: +# cap.set(cv2.CAP_PROP_EXPOSURE, 200) +``` + +--- + +## Motor Alignment Verification + +After assembly or reassembly, verify motors are correctly oriented: + +### Step 1: Scan Motors +```bash +# On Lite (from the tools/ directory): +python src/reachy_mini/tools/scan_motors.py + +# On Wireless (SSH in first): +ssh pollen@reachy-mini.local # password: root +source /venvs/mini_daemon/bin/activate +python scan_motors.py +``` + +Expected output -- all 9 motors on baudrate 1,000,000: +``` +Found motors at baudrate 1000000: [10, 11, 12, 13, 14, 15, 16, 17, 18] +``` + +### Step 2: Check Motor IDs + +| Expected ID | Motor | +|-------------|-------| +| 10 | body_rotation | +| 11-16 | stewart_1 through stewart_6 | +| 17 | right_antenna | +| 18 | left_antenna | + +If a motor has the wrong ID or baudrate, use the **Reachy Mini Testbench app** or the reflash tool: +```bash +reachy-mini-reflash-motors +``` + +### Step 3: Verify Physical Alignment +Each motor has an **orientation mark**. During assembly, these marks must be aligned. If marks are misaligned, the motor will report incorrect positions and may trigger "Overload Error." + +--- + +## Antenna Repositioning + +If an antenna appears physically rotated (90 or 180 degrees off from where it should be): + +1. This is a **manufacturing offset**, not a software issue. +2. Follow the [antenna repositioning guide](https://drive.google.com/file/d/1FsmNpwELuXUbdhGHDMjG_CNpYXOMtR7A/view). +3. It involves loosening the antenna, rotating it to the correct position, and re-tightening. + +--- + +## PID Tuning (Advanced) + +If a motor is shaky or jittery (especially motors 10, 17, 18), you can tune the PID values: + +**Default PID values:** + +| Motor | P | I | D | +|-------|---|---|---| +| body_rotation (10) | 200 | 0 | 0 | +| stewart_1-6 (11-16) | 300 | 0 | 0 | +| antennas (17, 18) | 200 | 0 | 0 | + +**To reduce jitter:** +1. Lower P to ~180 on the affected motor. +2. If still jittery, increase D to ~10. + +PID values are configured in `src/reachy_mini/assets/config/hardware_config.yaml`. + +--- + +## After Transport + +After transporting the robot: + +1. **Visual inspection:** Check that no cables are disconnected or pinched. +2. **Power on** and open the dashboard to verify all motors are detected. +3. **Run a basic motion test:** + ```bash + python examples/minimal_demo.py + ``` +4. If motors are missing, check the physical connections at the foot PCB and head PCB. + +--- + +## Symptoms of Bad Calibration / Alignment + +| Symptom | Likely Cause | +|---------|-------------| +| Motor blinks red on startup | Orientation mark misaligned or motor overloaded | +| Head drifts to one side at rest | One Stewart motor may have incorrect ID or position | +| Antenna faces the wrong way | Manufacturing offset (reposition physically) | +| Camera image consistently dark | Auto-exposure disabled or wrong setting | +| Audio direction-of-arrival always off | Flat flex cable installed backwards | + +--- + +## Getting Help + +If basic checks do not resolve the issue: +1. Run the **Reachy Mini Testbench app** from the dashboard to scan and diagnose motors. +2. Check `docs/source/platforms/reachy_mini/motors_diagnosis.md` for the full motor troubleshooting flowchart. +3. Ask on Discord: https://discord.gg/Y7FgMqHsub diff --git a/docs/HELP_COMMON_ERRORS.md b/docs/HELP_COMMON_ERRORS.md new file mode 100644 index 000000000..a43fc9902 --- /dev/null +++ b/docs/HELP_COMMON_ERRORS.md @@ -0,0 +1,226 @@ +# Reachy Mini -- Common Errors Guide + +Searchable, blunt, fast. Find your error, get the fix, move on. + +--- + +## Connection Errors + +### `ConnectionError` / `Connection refused` / Timeout on connect + +**What it means:** The SDK cannot reach the daemon. + +**Fix:** +1. Is the daemon running? Start it: `reachy-mini-daemon` (Lite) or verify robot is powered on (Wireless). +2. Is another app already connected and controlling the robot? Stop it first. +3. For Wireless: Are your computer and robot on the same WiFi network? +4. Try opening http://localhost:8000 (Lite) or http://reachy-mini.local:8000 (Wireless). If the dashboard loads, the daemon is running. + +--- + +### `OSError: PortAudio library not found` + +**What it means:** Missing system audio library on Linux. + +**Fix:** +```bash +sudo apt-get install libportaudio2 +``` +Then restart the daemon. + +--- + +### Dashboard at `http://localhost:8000` doesn't load + +**What it means:** Daemon is not running or browser is blocking local access. + +**Fix:** +1. Check the daemon terminal for errors. +2. On macOS: Check System Settings > Privacy & Security > Local Network for browser permissions. +3. Make sure you are inside your virtual environment. +4. Update the SDK: `uv pip install -U reachy-mini` + +--- + +## Motor Errors + +### Motors don't move / No response + +**What it usually means:** Power supply is not connected. + +**Fix:** Plug in the 7V wall adapter. USB-C provides data only, not motor power. + +--- + +### Motor blinking red / `Overload Error` + +**What it means:** Motor is physically stuck or orientation marks are misaligned. + +**Fix:** +1. Check that motor orientation marks are aligned (assembly guide). +2. Use the **Reachy Mini Testbench app** to diagnose. +3. If the motor feels hard to turn even when powered off AND blinks red, it may be a hardware defect. Contact support. + +--- + +### `Motor '' hardware errors: ['Input Voltage Error']` + +**What it means:** Nothing wrong. Reachy Mini intentionally operates at the upper voltage range of the Dynamixel motors. This warning is expected and suppressed in normal operation. + +**Fix:** No action needed. + +--- + +### `Electrical Shock Error` + +**What it means:** Power supply issue or short circuit. + +**Fix:** +1. Check all cables from foot PCB to head for damage. +2. Inspect the power cable (black & red). +3. Inspect the 3-wire motor cables (300mm, 200mm, 100mm, 40mm). + +--- + +### Motors stop responding after a while + +**What it means:** Thermal protection (overheating) or power issue. + +**Fix:** +1. Power off, wait 30 seconds, power on. +2. Check power supply connection. +3. Update the SDK: `pip install -U reachy-mini` +4. If motor LED blinks red, see "Overload Error" above. + +--- + +### Motor is shaky / jittery + +**What it means:** PID values are causing overcorrection. Common on motors 10 (body), 17 and 18 (antennas). + +**Fix:** Tune PID values in `src/reachy_mini/assets/config/hardware_config.yaml`: +- Reduce P to ~180 on the affected motor. +- If still shaky, increase D to ~10. + +--- + +### `No motor found on port` / Missing motors + +**What it means:** Motor is not detected on the serial bus. + +**Fix:** +1. Check physical cable connections (especially at foot PCB and head PCB). +2. Run the motor scan script to identify which motors are found. +3. If consecutive motors are missing (e.g., 11-12-13 or 17-18), the issue is likely a disconnected cable to that chain. +4. Use `reachy-mini-reflash-motors` if a motor has the wrong baudrate or ID. + +--- + +## SDK / Python Errors + +### `ModuleNotFoundError: No module named 'reachy_mini'` + +**What it means:** SDK not installed or wrong virtual environment. + +**Fix:** +1. Activate your virtual environment: `source reachy_mini_env/bin/activate` +2. Install: `uv pip install reachy-mini` + +--- + +### `ValueError` on `goto_target` or `set_target` + +**What it means:** Invalid input shape or type. + +**Fix:** +- `head` must be a 4x4 numpy array (use `create_head_pose()` to build it). +- `antennas` must be a list/tuple of 2 floats (in radians). +- `body_yaw` must be a single float (in radians). +- `duration` must be positive. + +--- + +### `Warning: Circular buffer overrun` (Simulation) + +**What it means:** Video frames are being produced but not consumed, filling the buffer. + +**Fix:** If you don't need video, initialize with: +```python +with ReachyMini(media_backend="no_media") as mini: + ... +``` + +--- + +## Audio / Video Errors + +### Camera image is dark (Lite) + +**Fix:** Adjust exposure. See HELP_CALIBRATION.md for detailed instructions per OS. + +Quick fix: set `auto-exposure-priority=1` using your OS camera controls. + +--- + +### No microphone input (Wireless) + +**What it means:** Firmware too old or flat flex cable installed backwards. + +**Fix:** +1. Update to firmware 2.1.3+: run the update script at `src/reachy_mini/assets/firmware/update.sh`. +2. Check flat flex cable orientation (assembly guide slides 45--47). + +--- + +### Audio volume too low + +**Fix:** Update to SDK version 1.2.3 or later. + +On Linux, also check `alsamixer`: +1. Run `alsamixer` +2. Set PCM1 to 100% +3. Adjust global volume with PCM,0 + +--- + +### Face tracking feels slow + +**Fix:** +1. Ensure the face is well-lit. +2. The GStreamer backend may have lower latency than the default OpenCV backend. +3. On Wireless, remote (WebRTC) introduces additional latency compared to local execution. + +--- + +## Wireless-Specific Errors + +### WiFi access point (`reachy-mini-ap`) doesn't appear + +**Fix:** Check the switch on the head board. It must be in "debug" position, not "download." + +--- + +### Can't connect via USB-C cable (Wireless) + +**What it means:** Wireless units do NOT expose the robot over USB like the Lite version. + +**Fix:** Use WiFi. For a wired connection, use a USB-C-to-Ethernet adapter plus Ethernet cable. + +--- + +### App installations fail on Windows + +**Fix:** +```powershell +set HF_HUB_DISABLE_SYMLINKS_WARNING=1 +``` + +--- + +## When to Escalate + +Contact support or ask on [Discord](https://discord.gg/Y7FgMqHsub) when: +- A motor feels physically hard to turn when powered off AND blinks red (hardware defect). +- Cables appear physically damaged. +- The motor scan shows motors at unexpected baudrates or IDs after a fresh assembly. +- The problem persists after updating firmware and SDK to latest versions. diff --git a/docs/HELP_CONTROL_MODES.md b/docs/HELP_CONTROL_MODES.md new file mode 100644 index 000000000..615b50b9d --- /dev/null +++ b/docs/HELP_CONTROL_MODES.md @@ -0,0 +1,214 @@ +# Reachy Mini -- Control Modes Guide + +Clarify the mental models. Most bugs are mode misunderstandings, not code errors. + +--- + +## The Two Motion Methods + +Reachy Mini has exactly two ways to move: + +| Method | What it does | When to use | +|--------|-------------|-------------| +| `goto_target()` | Smooth interpolation over a duration | **Default choice.** Gestures, emotions, choreography. | +| `set_target()` | Sets position immediately, no interpolation | Real-time control loops (tracking, games, joystick). | + +### `goto_target()` -- Smooth, Blocking + +```python +from reachy_mini import ReachyMini +from reachy_mini.utils import create_head_pose + +with ReachyMini() as mini: + mini.goto_target( + head=create_head_pose(yaw=30, pitch=10, degrees=True), + duration=1.5, + method="minjerk" + ) +``` + +**Key behaviors:** +- Blocks until the motion completes. +- You cannot react to external input during the motion. +- The robot commits to completing the movement. +- Minimum useful duration: ~0.5 seconds. + +**Interpolation methods:** +| Method | Character | +|--------|-----------| +| `minjerk` | Natural, smooth (default) | +| `linear` | Constant speed | +| `ease` | Slow start and end | +| `cartoon` | Exaggerated, bouncy | + +### `set_target()` -- Instant, Non-blocking + +```python +import time +from reachy_mini import ReachyMini +from reachy_mini.utils import create_head_pose + +with ReachyMini() as mini: + while True: + pose = compute_target() # Your logic here + mini.set_target(head=pose) + time.sleep(0.01) # ~100Hz +``` + +**Key behaviors:** +- Returns immediately (non-blocking). +- Must be called continuously in a loop. +- You are responsible for smooth motion (send frequent updates). +- If you stop calling it, the robot holds the last sent position. + +--- + +## Decision Flowchart + +``` +Does your app need to react to input in real-time? +| ++-- NO --> Use goto_target() +| Simple code, guaranteed smooth motion. +| Good for: emotions, dances, scripted sequences. +| ++-- YES -> Use set_target() in a control loop + More complex, but fully reactive. + Good for: face tracking, games, recording, teleoperation. +``` + +--- + +## Motor Modes + +Separate from motion methods, motors have three torque states: + +| Mode | Method | Behavior | +|------|--------|----------| +| **Stiff** | `mini.enable_motors()` | Motors hold position. Normal operating mode. | +| **Limp** | `mini.disable_motors()` | No power. Head drops under gravity. | +| **Compliant** | `mini.enable_gravity_compensation()` | Soft. You can move the head by hand and it stays where you leave it. | + +### When to switch modes: + +| Use case | Sequence | +|----------|----------| +| Normal operation | `enable_motors()` then `goto_target()` / `set_target()` | +| Teaching by demonstration | `enable_gravity_compensation()` then `start_recording()` | +| Shutting down | `goto_target(SLEEP_HEAD_POSE)` then `disable_motors()` | +| Idle (save power) | `disable_motors()` | + +--- + +## Simulation vs Real Hardware + +| Behavior | Simulation (MuJoCo) | Real Hardware | +|----------|---------------------|---------------| +| Motor response | Instant, perfect | Subject to PID tuning, inertia | +| Gravity compensation | Works | Works (requires Placo kinematics) | +| Camera | Not available | Available | +| Audio | May not work | Available | +| Safety limits | Applied identically | Applied identically | +| Timing | May differ slightly | Real-world timing | + +**Important:** Code that works in simulation will work on hardware. But **timing-sensitive** code (control loops, audio sync) may need tuning on real hardware. + +To run without needing media (cameras/audio), initialize with: +```python +with ReachyMini(media_backend="no_media") as mini: + # Works in sim or on hardware without camera/mic access +``` + +--- + +## Common Mistakes + +### Mistake 1: Mixing `goto_target` and `set_target` +```python +# BAD: These fight each other +mini.goto_target(head=pose_a, duration=1.0) # Starts interpolation +mini.set_target(head=pose_b) # Interrupts mid-motion +``` + +Pick one method per behavior phase. If you need to transition between modes, complete the `goto_target` first. + +### Mistake 2: Calling `set_target` from multiple places +```python +# BAD: Race condition, jerky motion +def on_face_detected(face): + mini.set_target(head=look_at_face(face)) + +def idle_behavior(): + mini.set_target(head=breathing_pose) +``` + +**Fix:** One control loop, one `set_target()` call. Update target variables from callbacks. + +```python +# GOOD: Single control loop +def control_loop(): + while running: + pose = compute_final_pose() # Combines all inputs + mini.set_target(head=pose) + time.sleep(0.01) +``` + +### Mistake 3: `set_target` too slowly +If you call `set_target()` below 30Hz, motion will look jerky. Aim for 50--100Hz. + +### Mistake 4: Enabling motors without setting goal to current position +The robot will jump to the last goal position. See HELP_FIRST_MOTION.md for the safe enable pattern. + +--- + +## Control Loop Template + +For apps that need real-time reactivity: + +```python +import time +import threading +from reachy_mini import ReachyMini +from reachy_mini.utils import create_head_pose + +class MyApp: + def __init__(self): + self.stop = threading.Event() + self.target_yaw = 0.0 + self.target_pitch = 0.0 + + def control_loop(self, mini): + """The ONLY place that calls set_target.""" + while not self.stop.is_set(): + pose = create_head_pose( + yaw=self.target_yaw, + pitch=self.target_pitch, + degrees=True + ) + mini.set_target(head=pose) + time.sleep(0.01) # ~100Hz + + def update_from_sensor(self, yaw, pitch): + """Called from other threads.""" + self.target_yaw = yaw + self.target_pitch = pitch + +app = MyApp() +with ReachyMini() as mini: + thread = threading.Thread(target=app.control_loop, args=(mini,)) + thread.start() + # ... update targets from sensor callbacks ... + app.stop.set() + thread.join() +``` + +--- + +## Frequency Guidelines + +| Loop frequency | Quality | +|----------------|---------| +| 100 Hz | Excellent. Real-time tracking, games. | +| 50 Hz | Good. Most interactive apps. | +| 30 Hz | Minimum for acceptable smoothness. | +| < 30 Hz | Visibly jerky. Not recommended. | diff --git a/docs/HELP_DEVELOPMENT_WORKFLOW.md b/docs/HELP_DEVELOPMENT_WORKFLOW.md new file mode 100644 index 000000000..6b118da30 --- /dev/null +++ b/docs/HELP_DEVELOPMENT_WORKFLOW.md @@ -0,0 +1,217 @@ +# Reachy Mini -- Development Workflow Guide + +Turn tinkering into engineering. Build robust apps using proven patterns. + +--- + +## Recommended Project Structure + +Use the app assistant to scaffold new projects. Never create app folders manually: + +```bash +# Standard app: +reachy-mini-app-assistant create my_app ~/projects/my_app --publish + +# Conversation/LLM app: +reachy-mini-app-assistant create --template conversation my_ai_app ~/projects/my_ai_app --publish +``` + +This generates the correct metadata, entry points, and folder structure. The `--publish` flag prepares it for sharing on Hugging Face. + +### Resulting structure: +``` +my_app/ +├── my_app/ +│ └── main.py # Entry point +├── static/ # Web UI files (if applicable) +├── pyproject.toml # Package metadata +└── plan.md # Your implementation plan (create this) +``` + +--- + +## Development Loop + +### 1. Plan before coding + +Create `plan.md` in your app directory: +- What the app should do +- Which SDK features it uses (`goto_target` vs `set_target`, media, etc.) +- Known constraints (Lite only? Wireless only? Need camera?) + +### 2. Start with simulation + +```bash +# Terminal 1: Start simulator +reachy-mini-daemon --sim + +# Terminal 2: Run your app +python my_app/main.py +``` + +Simulation catches logic errors without risking hardware. Use `media_backend="no_media"` if your app does not need camera/audio. + +### 3. Test on hardware + +Once simulation works, connect to the real robot: +- Lite: `reachy-mini-daemon` (no `--sim` flag) +- Wireless: Daemon is already running + +Timing and PID behavior will differ slightly from simulation. Tune durations and control loop frequencies on real hardware. + +### 4. Test on Wireless (if applicable) + +SSH into the robot and run your app there: +```bash +ssh pollen@reachy-mini.local # password: root +cd my_app +python my_app/main.py +``` + +This reproduces exactly what the dashboard does when launching an installed app. + +--- + +## Logging and Debugging + +### Add logging to your app +```python +import logging +logging.basicConfig(level=logging.DEBUG) +logger = logging.getLogger(__name__) + +logger.debug(f"Head pose: {mini.get_current_head_pose()}") +logger.info("Starting main loop") +``` + +### Daemon logs +```bash +# Lite: Run daemon with verbose output +reachy-mini-daemon --verbose + +# Wireless: Check system logs +ssh pollen@reachy-mini.local +journalctl -u reachy-mini-daemon.service + +# Restart daemon on Wireless +systemctl restart reachy-mini-daemon.service +``` + +### Check motor control loop health +```python +print(mini.client.get_status()) +``` + +You should see ~50Hz (~20ms period). If the period is much higher, the control loop is too slow (CPU overloaded or high USB latency). + +--- + +## Version Control Advice + +Even for personal projects, use Git: + +```bash +cd my_app +git init +git add . +git commit -m "Initial scaffold from app assistant" +``` + +### Commit often in small increments +Small commits make debugging easier. If something breaks, `git diff` between the working and broken state is usually enough to spot the issue. + +### When something breaks after changes + +Use git history to narrow down the problem: + +```bash +# See recent commits +git log --oneline -10 + +# Go back to a known-good state +git checkout +python my_app/main.py # Does it work? + +# Compare the diff +git diff +``` + +--- + +## Reference Apps + +Study these production-quality apps to learn patterns: + +| App | Key Patterns | +|-----|-------------| +| [reachy_mini_conversation_app](https://github.com/pollen-robotics/reachy_mini_conversation_app) | LLM integration, control loops, pose fusion, threading | +| [marionette](https://huggingface.co/spaces/RemiFabre/marionette) | Motion recording, safe torque, HuggingFace datasets | +| [fire_nation_attacked](https://huggingface.co/spaces/RemiFabre/fire_nation_attacked) | Head-as-controller, game logic, leaderboards | +| [spaceship_game](https://huggingface.co/spaces/apirrone/spaceship_game) | Head-as-joystick, antenna buttons | +| [reachy_mini_radio](https://huggingface.co/spaces/pollen-robotics/reachy_mini_radio) | Antenna interaction pattern | +| [reachy_mini_simon](https://huggingface.co/spaces/apirrone/reachy_mini_simon) | No-GUI pattern (antenna to start) | +| [hand_tracker_v2](https://huggingface.co/spaces/pollen-robotics/hand_tracker_v2) | Camera-based control loop | +| [reachy_mini_dances_library](https://github.com/pollen-robotics/reachy_mini_dances_library) | Symbolic motion definition | + +Clone them for local study: +```bash +git clone https://github.com/pollen-robotics/reachy_mini_conversation_app +``` + +--- + +## Adding New Behaviors + +### Using `goto_target` (choreographed) +```python +# Define a sequence of poses +poses = [ + (create_head_pose(yaw=30, degrees=True), [0.5, -0.5], 0.8), + (create_head_pose(pitch=-20, degrees=True), [0, 0], 0.6), + (create_head_pose(), [0, 0], 1.0), +] + +for head, antennas, duration in poses: + mini.goto_target(head=head, antennas=antennas, duration=duration) +``` + +### Using recorded moves +```python +from reachy_mini.motion.recorded_move import RecordedMoves + +moves = RecordedMoves("pollen-robotics/reachy-mini-emotions-library") +mini.play_move(moves.get("happy"), initial_goto_duration=1.0) +``` + +### Using `set_target` (reactive) +See the control loop template in [HELP_CONTROL_MODES.md](HELP_CONTROL_MODES.md). + +--- + +## Publishing Your App + +Once your app works: + +1. Push to a Hugging Face Space: + ```bash + # The app assistant handles this if you used --publish + ``` + +2. Other users can install it directly from their dashboard. + +3. Community apps: https://huggingface.co/spaces?q=reachy_mini + +--- + +## Platform Compatibility Checklist + +Before sharing, verify your app works on intended platforms: + +| Check | How | +|-------|-----| +| Simulation | `reachy-mini-daemon --sim` + your app | +| Lite hardware | USB connection + your app | +| Wireless local | SSH + run on the robot | +| Wireless remote | Run on laptop, robot over WiFi | + +Not all apps need all platforms. A camera-tracking app won't work in simulation. A simple dance works everywhere. diff --git a/docs/HELP_FIRST_MOTION.md b/docs/HELP_FIRST_MOTION.md new file mode 100644 index 000000000..d254ae5a3 --- /dev/null +++ b/docs/HELP_FIRST_MOTION.md @@ -0,0 +1,199 @@ +# Reachy Mini -- First Motion Guide + +Safely move something as fast as possible. Understand what happens when you do. + +--- + +## Prerequisites + +Before attempting motion: +1. Power supply is connected (USB alone will NOT power motors) +2. Daemon is running (`reachy-mini-daemon` for Lite, automatic for Wireless) +3. Dashboard shows robot is connected: http://localhost:8000 + +--- + +## Your First Movement: Antenna Wiggle + +This is the safest possible first motion. Antennas are lightweight and cannot collide with anything. + +```python +from reachy_mini import ReachyMini + +with ReachyMini() as mini: + # Wiggle right antenna up, left antenna down + mini.goto_target(antennas=[0.5, -0.5], duration=0.5) + # Reverse + mini.goto_target(antennas=[-0.5, 0.5], duration=0.5) + # Return to neutral + mini.goto_target(antennas=[0, 0], duration=0.5) +``` + +Antenna values are in **radians**. Typical range: roughly -3.0 to +3.0. + +--- + +## Your Second Movement: Head Nod + +```python +from reachy_mini import ReachyMini +from reachy_mini.utils import create_head_pose + +with ReachyMini() as mini: + # Nod down (positive pitch) + mini.goto_target( + head=create_head_pose(pitch=15, degrees=True), + duration=1.0 + ) + # Return to neutral + mini.goto_target( + head=create_head_pose(), + duration=1.0 + ) +``` + +`create_head_pose()` with no arguments creates the neutral (home) position. + +--- + +## Your Third Movement: Full Combo + +```python +from reachy_mini import ReachyMini +from reachy_mini.utils import create_head_pose +import numpy as np + +with ReachyMini() as mini: + # Look right + turn body right + spread antennas + mini.goto_target( + head=create_head_pose(yaw=-30, degrees=True), + body_yaw=np.deg2rad(-20), + antennas=[0.5, -0.5], + duration=1.5, + method="minjerk" + ) + # Return to neutral + mini.goto_target( + head=create_head_pose(), + body_yaw=0.0, + antennas=[0, 0], + duration=1.5 + ) +``` + +--- + +## How to Stop Motion Immediately + +### From code: +```python +# Disable all motors -- head will go limp (gravity will pull it down) +mini.disable_motors() +``` + +### From the dashboard: +Open http://localhost:8000 and use the motor controls to disable torque. + +### Physical: +Unplug the power adapter. Motors will immediately lose torque. + +**Important:** Disabling motors causes the head to drop under gravity. This is normal and not harmful, but be prepared for it. + +--- + +## Safety Limits (Automatic) + +The SDK automatically clamps all values to safe ranges. You cannot command the robot into self-collision through software. + +| Joint | Safe Range | +|-------|-----------| +| Head pitch | -40 to +40 degrees | +| Head roll | -40 to +40 degrees | +| Head yaw | -180 to +180 degrees | +| Body yaw | -160 to +160 degrees | +| Head-body yaw difference | Max 65 degrees | + +If you send a value outside these limits, the SDK silently clamps it to the nearest safe value. Your code will not crash. + +--- + +## Motor Enable / Disable + +```python +# Motors ON -- robot holds position (stiff) +mini.enable_motors() + +# Motors OFF -- robot goes limp +mini.disable_motors() + +# Gravity compensation -- robot is "soft", you can move it by hand +# and it will hold where you leave it +mini.enable_gravity_compensation() +``` + +### Safe Enable Pattern + +When enabling motors, set the goal to the current position first. This prevents the head from jumping to a stale goal position: + +```python +# Read where the head actually is +head_pose = mini.get_current_head_pose() +_, antennas = mini.get_current_joint_positions() + +# Set the goal to the current position (very short duration) +mini.goto_target( + head=head_pose, + antennas=list(antennas) if antennas is not None else None, + duration=0.05 +) + +# Now safely enable +mini.disable_motors() # Full reset (handles mixed motor state edge case) +mini.enable_motors() +``` + +### Safe Disable Pattern + +Go to a safe resting position before disabling torque: + +```python +import numpy as np +from reachy_mini.reachy_mini import SLEEP_HEAD_POSE + +antenna_angle = np.deg2rad(15) +mini.goto_target( + SLEEP_HEAD_POSE, + antennas=[-antenna_angle, antenna_angle], + duration=1.0 +) +mini.disable_motors() +``` + +--- + +## Interpolation Methods + +`goto_target` supports four interpolation styles: + +| Method | Character | Best for | +|--------|-----------|----------| +| `minjerk` | Smooth, natural acceleration/deceleration | Default. Most gestures. | +| `linear` | Constant speed | Mechanical, predictable motion | +| `ease` | Slow start and end | Gentle transitions | +| `cartoon` | Exaggerated, bouncy overshoot | Playful, expressive motion | + +```python +mini.goto_target( + head=create_head_pose(yaw=30, degrees=True), + duration=1.0, + method="cartoon" # Try each one to see the difference +) +``` + +--- + +## What to Read Next + +- [HELP_CONTROL_MODES.md](HELP_CONTROL_MODES.md) -- Understanding `goto_target` vs `set_target` +- [HELP_SAFETY.md](HELP_SAFETY.md) -- Physical safety around the robot +- [HELP_COMMON_ERRORS.md](HELP_COMMON_ERRORS.md) -- If something goes wrong during your first motion diff --git a/docs/HELP_HARDWARE.md b/docs/HELP_HARDWARE.md new file mode 100644 index 000000000..41675a2d7 --- /dev/null +++ b/docs/HELP_HARDWARE.md @@ -0,0 +1,111 @@ +# Reachy Mini -- Hardware Guide + +Prevent physical damage. Understand what you are working with. + +--- + +## Specifications at a Glance + +| Spec | Value | +|------|-------| +| **Dimensions** | 30 x 20 x 15.5 cm (extended) | +| **Mass** | 1.35 kg (Lite) / 1.475 kg (Wireless) | +| **Materials** | ABS, PC, Aluminium, Steel | +| **Power input** | 6.8 -- 7.6V DC | +| **Camera** | Sony IMX708, 12MP, 120 degree wide angle, autofocus | +| **Microphones** | 4-mic MEMS array (Seeed reSpeaker XMOS XVF3800), 16 kHz | +| **Speaker** | 5W @ 4 Ohms | +| **Compute (Wireless)** | Raspberry Pi CM4 (4GB RAM, 16GB flash, WiFi) | +| **Battery (Wireless)** | LiFePO4, 2000mAh, 6.4V, 12.8Wh | + +--- + +## Degrees of Freedom + +| Component | DOF | Details | +|-----------|-----|---------| +| **Head** | 6 | 3 rotations (pitch, roll, yaw) + 3 translations (x, y, z) via Stewart platform | +| **Body** | 1 | Rotation around vertical axis | +| **Antennas** | 2 | 1 rotation each (right and left) | +| **Total** | 9 | | + +--- + +## Motor Details + +| Motor | ID | Type | Purpose | +|-------|----|------|---------| +| `body_rotation` | 10 | XC330-M288-PG | Base rotation | +| `stewart_1` | 11 | XL330-M288-T | Head platform | +| `stewart_2` | 12 | XL330-M288-T | Head platform | +| `stewart_3` | 13 | XL330-M288-T | Head platform | +| `stewart_4` | 14 | XL330-M288-T | Head platform | +| `stewart_5` | 15 | XL330-M288-T | Head platform | +| `stewart_6` | 16 | XL330-M288-T | Head platform | +| `right_antenna` | 17 | XL330-M077-T | Right antenna | +| `left_antenna` | 18 | XL330-M077-T | Left antenna | + +Serial baudrate: 1,000,000 bps. + +--- + +## Power + +- **Input voltage:** 6.8 -- 7.6V via the included power adapter. +- **USB-C does NOT power the motors.** You must use the wall adapter for any motion. +- **Wireless battery:** LiFePO4 with overcharge, overdischarge, overcurrent, and short-circuit protection. Built-in temperature sensor. +- **Battery indicator:** LED color (green -> orange -> red). There is no precise battery percentage readout. + +--- + +## Safe Boot and Shutdown + +### Power On +1. Plug in the power adapter (Lite) or press the ON button (Wireless). +2. Wait for the daemon to start (Wireless: ~30 seconds). +3. Verify via the dashboard at `http://localhost:8000` (Lite) or `http://reachy-mini.local:8000` (Wireless). + +### Power Off +1. **Wireless:** Press the OFF button. Wait 5 seconds before unplugging. +2. **Lite:** Close the daemon process, then unplug USB and power. + +### Restart (fixes many issues) +Press OFF, wait 5 seconds, press ON. + +--- + +## Cable and Assembly Notes + +- **USB cable in the head:** Leave enough slack for full head rotation. A tight cable will restrict motion and eventually fatigue the connector. +- **Motor orientation marks:** Each motor has an alignment mark that must match during assembly. Misalignment causes "Overload Error" on startup. +- **Flat flex cable (mic array):** Must be installed the correct way (see assembly guide slides 45--47). Reversed connection = no microphone input. +- **Antenna positioning:** If an antenna appears rotated 90 or 180 degrees, it is a manufacturing offset. Follow the [antenna repositioning guide](https://drive.google.com/file/d/1FsmNpwELuXUbdhGHDMjG_CNpYXOMtR7A/view) to correct it. + +--- + +## DO NOT Do This + +- **DO NOT** power motors from USB alone. They will not respond, and you may think something is broken. +- **DO NOT** force the head past its physical stops. The Stewart platform has mechanical limits. +- **DO NOT** leave the robot powered with motors enabled and unattended for extended periods. Motors may overheat. +- **DO NOT** disconnect cables while the robot is powered on. +- **DO NOT** expose to liquids. No ingress protection rating. +- **DO NOT** use a power supply outside the 6.8 -- 7.6V range. Higher voltage will trigger "Input Voltage Error" on motors (which is intentionally suppressed in the firmware since the robot operates near the upper limit). + +--- + +## Storage and Transport + +- Power off and unplug before moving. +- Store in a dry environment at room temperature. +- Use the original packaging for transport when possible. +- The head can rest in its natural sleep position during storage. + +--- + +## Wireless-Specific Notes + +- **Switch position on the head board:** Must be in "debug" position, not "download". If the WiFi access point does not appear, check this switch. +- **SSH access:** `ssh pollen@reachy-mini.local` (password: `root`). +- **System health check:** Run `reachyminios_check` after SSH login. +- **USB-C on Wireless:** This is an output port (for USB devices like a flash drive). It does NOT provide a tethered connection like the Lite version. diff --git a/docs/HELP_SAFETY.md b/docs/HELP_SAFETY.md new file mode 100644 index 000000000..94abc868e --- /dev/null +++ b/docs/HELP_SAFETY.md @@ -0,0 +1,144 @@ +# Reachy Mini -- Safety Guide + +Reduce risk of injury and equipment damage. Build trust through transparency. + +--- + +## Physical Safety + +### Pinch Points + +The Stewart platform mechanism (6 actuators connecting body to head) creates pinch points during motion. Keep fingers away from the gap between head and body while the robot is active. + +**Specific locations:** +- Between the 6 Stewart platform rods and the head/body shells +- Between the body shell and the base during body rotation +- Near antenna joints when antennas are moving + +### For Children and Pets + +- **Supervise** children under 12 when the robot is powered on. +- Antennas are lightweight and cannot cause injury, but small children may pull on them. +- The robot is not waterproof or food-safe. Keep away from liquids. +- The power adapter cable is a trip hazard. Route it safely. + +--- + +## Emergency Stop + +### From software: +```python +mini.disable_motors() +``` +This immediately removes torque from all motors. The head will drop under gravity to its rest position. + +### From the dashboard: +Navigate to http://localhost:8000 and disable motors through the interface. + +### Physical: +Unplug the power adapter. All motors instantly lose torque. + +**Note:** Disabling motors is always safe. The head dropping to rest position is normal and will not damage the robot. Gentle collisions between head and body are expected and harmless. + +--- + +## Operating Limits + +### Joint Limits (enforced by software) + +| Joint | Range | +|-------|-------| +| Head pitch | -40 to +40 degrees | +| Head roll | -40 to +40 degrees | +| Head yaw | -180 to +180 degrees | +| Body yaw | -160 to +160 degrees | +| Head-body yaw difference | Max 65 degrees | + +The SDK automatically clamps values to these ranges. You cannot command the robot past these limits through the API. + +### Electrical Limits + +| Parameter | Value | +|-----------|-------| +| Input voltage | 6.8 -- 7.6V DC | +| Power supply | Use ONLY the included adapter | +| USB-C | Data only (Lite) or output only (Wireless). Does NOT supply motor power. | + +### Thermal Limits + +Motors have built-in thermal protection. If a motor overheats: +1. It will stop responding (thermal shutdown). +2. Power off the robot. +3. Wait 5+ minutes for cooling. +4. Power back on. + +**Prevention:** Avoid running motors at high torque for extended periods without breaks. + +--- + +## Recommended Operating Time + +- **Continuous use (active motion):** Up to 2 hours before checking motor temperatures. +- **Idle with motors enabled:** Motors draw current even when holding position. Disable motors during long idle periods. +- **Wireless battery life:** Approximately 1--2 hours depending on activity level. Charge when the LED turns red. + +--- + +## What the Robot Can Safely Do + +- Head can touch the body shell during some motions. This is expected. +- Antennas can be pushed by hand (they are semi-compliant). This is by design -- they are used as physical buttons. +- The robot can be moved by hand when motors are disabled or in gravity compensation mode. + +--- + +## What You Should NOT Do + +| Action | Risk | +|--------|------| +| Use a power supply above 7.6V | Motor damage, electrical shock errors | +| Run motors continuously for hours at high torque | Overheating, thermal shutdown | +| Force the head past its physical stops | Mechanical damage to Stewart platform | +| Disconnect cables while powered on | Motor errors, potential damage | +| Operate in wet or humid conditions | No ingress protection | +| Leave unattended with motors enabled and active motion | Overheating risk | +| Place heavy objects on the head | Stewart platform not designed for external loads | + +--- + +## Power Supply Safety + +- Use **only** the included 7V/5A power adapter. +- Ensure the power cable is not pinched or bent sharply. +- Unplug when not in use. +- The Wireless battery has built-in protections (overcharge, overdischarge, overcurrent, short circuit, temperature sensor). +- There is no way to check exact battery percentage. Rely on the LED indicator (green > orange > red). + +--- + +## Handling and Transport + +- Power off completely before moving the robot. +- Support the head when moving -- it is the heaviest part relative to its mount. +- Use the original packaging for shipping. +- After transport, run a basic motion test before complex operations. + +--- + +## If Something Seems Wrong + +1. **Motors making unusual sounds:** Power off immediately. Check for cables caught in mechanisms. +2. **Burning smell:** Power off immediately. Unplug. Do not power on until inspected. +3. **Motor locked and blinking red:** May be overloaded or defective. See HELP_COMMON_ERRORS.md. +4. **Robot falls over:** The base is designed to be stable, but ensure it is on a flat surface. The robot is not designed for inclined surfaces. + +For any safety concern, power off first and assess second. + +--- + +## Contact for Safety Issues + +If you believe there is a safety defect: +- Email: sales@pollen-robotics.com +- Include photos and description of the issue +- Include your order/invoice number diff --git a/docs/HELP_SOFTWARE_SETUP.md b/docs/HELP_SOFTWARE_SETUP.md new file mode 100644 index 000000000..a5c9f5919 --- /dev/null +++ b/docs/HELP_SOFTWARE_SETUP.md @@ -0,0 +1,207 @@ +# Reachy Mini -- Software Setup Guide + +Eliminate environment hell. Get a clean, working installation on any OS. + +--- + +## Prerequisites + +| Tool | Version | Check | Purpose | +|------|---------|-------|---------| +| **Python** | 3.10 -- 3.12 | `python --version` | Run SDK and apps | +| **Git** | Latest | `git --version` | Clone repositories | +| **Git LFS** | Latest | `git lfs version` | Download model assets | +| **uv** (recommended) | Latest | `uv --version` | Fast package management | + +--- + +## Step 1: Install uv (Recommended Package Manager) + +**Linux / macOS:** +```bash +curl -LsSf https://astral.sh/uv/install.sh | sh +``` + +**Windows (PowerShell):** +```powershell +powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex" +``` + +Close and reopen your terminal, then verify: +```bash +uv --version +``` + +--- + +## Step 2: Install Python + +```bash +uv python install 3.12 --default +``` + +Verify: +```bash +python --version +# Should show Python 3.12.x +``` + +--- + +## Step 3: Install Git and Git LFS + +**Linux:** +```bash +sudo apt install git git-lfs +``` + +**macOS:** +```bash +brew install git git-lfs +``` + +**Windows:** Download from https://git-scm.com/install/windows + +Then initialize Git LFS: +```bash +git lfs install +``` + +--- + +## Step 4: Create a Virtual Environment + +```bash +uv venv reachy_mini_env --python 3.12 +``` + +Activate it: +```bash +# Linux / macOS: +source reachy_mini_env/bin/activate + +# Windows: +reachy_mini_env\Scripts\activate +``` + +You should see `(reachy_mini_env)` at the start of your prompt. + +**Important:** You must activate this environment every time you open a new terminal. + +--- + +## Step 5: Install Reachy Mini + +**Standard install (most users):** +```bash +uv pip install reachy-mini +``` + +**With simulation support:** +```bash +uv pip install "reachy-mini[mujoco]" +``` + +**From source (developers):** +```bash +git clone https://github.com/pollen-robotics/reachy_mini && cd reachy_mini +uv sync +# Or with simulation: +uv sync --extra mujoco +``` + +--- + +## Step 6: Linux USB Permissions + +If you are using a Lite (USB-connected) robot on Linux, you need to grant serial port access: + +```bash +echo 'SUBSYSTEM=="usb", ATTRS{idVendor}=="1a86", ATTRS{idProduct}=="55d3", MODE="0666", GROUP="dialout" +SUBSYSTEM=="usb", ATTRS{idVendor}=="38fb", ATTRS{idProduct}=="1001", MODE="0666", GROUP="dialout"' \ +| sudo tee /etc/udev/rules.d/99-reachy-mini.rules + +sudo udevadm control --reload-rules && sudo udevadm trigger +sudo usermod -aG dialout $USER +``` + +**Log out and log back in** for group changes to take effect. + +--- + +## Step 7: Linux Audio Dependency + +```bash +sudo apt-get install libportaudio2 +``` + +Without this, you will get `OSError: PortAudio library not found` when the SDK tries to initialize audio. + +--- + +## OS-Specific Notes + +### macOS +- MuJoCo simulation requires `mjpython` launcher: + ```bash + mjpython -m reachy_mini.daemon.app.main --sim + ``` +- `uv` may have compatibility issues with MuJoCo on macOS. Use `pip` as a fallback. + +### Windows +- Before activating a virtual environment for the first time, enable script execution: + ```powershell + Set-ExecutionPolicy RemoteSigned + ``` + (Run this once in an Administrator PowerShell.) +- App installations from Hugging Face may fail due to symlink permissions. Set: + ```powershell + set HF_HUB_DISABLE_SYMLINKS_WARNING=1 + ``` + +--- + +## Health Check Script + +After installation, run this to verify everything works: + +```bash +python -c " +from reachy_mini import ReachyMini +print('SDK import: OK') +from reachy_mini.utils import create_head_pose +print('Utilities import: OK') +import numpy as np +pose = create_head_pose(yaw=10, pitch=5, degrees=True) +print('Head pose creation: OK') +print('All checks passed.') +" +``` + +If any import fails, your virtual environment is not correctly set up or the package is not installed. + +--- + +## Connecting to the Robot + +| Setup | Daemon Command | Dashboard URL | +|-------|----------------|---------------| +| **Lite (USB)** | `reachy-mini-daemon` | http://localhost:8000 | +| **Wireless** | Already running on robot | http://reachy-mini.local:8000 | +| **Simulation** | `reachy-mini-daemon --sim` | http://localhost:8000 | + +The daemon must be running in a terminal before any Python scripts will work. Keep that terminal open. + +--- + +## Upgrading + +```bash +# Via uv: +uv pip install -U reachy-mini + +# Via pip: +pip install -U reachy-mini +``` + +For Wireless robots, use the dashboard Settings page to check for system updates. diff --git a/docs/HELP_WHEN_THINGS_GO_WRONG.md b/docs/HELP_WHEN_THINGS_GO_WRONG.md new file mode 100644 index 000000000..d53bf381f --- /dev/null +++ b/docs/HELP_WHEN_THINGS_GO_WRONG.md @@ -0,0 +1,186 @@ +# Reachy Mini -- When Things Go Wrong + +Something is not working. That is okay. This guide will help you figure out what happened and get back on track. + +--- + +## "The robot doesn't move" + +This is the most common first-time issue. Work through these checks in order: + +### Check 1: Is the power supply plugged in? +The USB-C cable provides data only. Motors require the 7V wall adapter. + +**Test:** Is the power LED on? If no, plug in the power supply. + +### Check 2: Is the daemon running? +- **Lite:** Open a terminal and run `reachy-mini-daemon`. Keep it running. +- **Wireless:** The daemon starts automatically when powered on. +- **Test:** Open http://localhost:8000 (Lite) or http://reachy-mini.local:8000 (Wireless). If the dashboard loads, the daemon is running. + +### Check 3: Is another app already controlling the robot? +Only one client can control the robot at a time. Stop any running apps from the dashboard before running your script. + +### Check 4: Are motors enabled? +```python +mini.enable_motors() +``` +Motors start disabled on some initialization paths. Explicitly enable them. + +### Check 5: Is your virtual environment activated? +Look for `(reachy_mini_env)` at the start of your terminal prompt. If missing: +```bash +source reachy_mini_env/bin/activate +``` + +--- + +## "The robot moves weirdly" + +### Jerky / stuttering motion +- Are you calling `set_target()` from multiple threads? Use a single control loop. +- Is your control loop running below 30Hz? Speed it up to 50--100Hz. +- Are you mixing `goto_target()` and `set_target()`? Pick one per behavior phase. +- Check the motor control loop frequency: + ```python + print(mini.client.get_status()) + ``` + It should show ~50Hz (~20ms period). Much higher means CPU or USB latency issues. + +### Head jumps when motors are enabled +You enabled motors without first setting the goal to the current position. Use the safe enable pattern: +```python +head_pose = mini.get_current_head_pose() +_, antennas = mini.get_current_joint_positions() +mini.goto_target(head=head_pose, antennas=list(antennas) if antennas else None, duration=0.05) +mini.disable_motors() +mini.enable_motors() +``` + +### Motor is shaky when holding position +PID values may need tuning. In `src/reachy_mini/assets/config/hardware_config.yaml`, try lowering P to ~180 on the affected motor (commonly 10, 17, or 18). + +### Head drifts to one side +Check that all Stewart motors (11--16) are detected and have correct IDs. Run the motor scan script. + +--- + +## "The robot worked yesterday" + +### Step 1: Update and restart +The single most effective fix: +- **Wireless:** Press OFF, wait 5 seconds, press ON. Then check for updates in dashboard Settings. +- **Lite:** Update the SDK: `uv pip install -U reachy-mini`. Restart the daemon. + +### Step 2: Check what changed +```bash +# If your code is in Git: +git log --oneline -5 +git diff HEAD~1 +``` + +Did you change your code? Did you update a dependency? Did you change your Python environment? + +### Step 3: Run the minimal demo +```bash +python examples/minimal_demo.py +``` +If this works, the problem is in your app code, not the robot or SDK. + +### Step 4: Check hardware +- Are all cables still connected? Cables can work loose, especially in the head. +- Is the power supply providing consistent power? Try a different outlet. + +--- + +## "I think I broke it" + +Take a breath. Reachy Mini is more resilient than it looks. + +### Motor blinking red +This usually means the motor detected an overload, not that it is broken. Try: +1. Power off completely. +2. Wait 30 seconds. +3. Power on. +4. Run the Testbench app from the dashboard to check motor status. + +### A motor won't respond +1. Check the cable to that motor. +2. Run `reachy-mini-reflash-motors` to reset the motor firmware. +3. If the motor feels physically stuck and hard to turn by hand (when powered off), it may be a hardware defect. Contact support. + +### An antenna is pointing the wrong way +This is a known manufacturing variation. It is not broken -- the antenna was mounted at a different offset. Follow the [antenna repositioning guide](https://drive.google.com/file/d/1FsmNpwELuXUbdhGHDMjG_CNpYXOMtR7A/view). + +### Head dropped suddenly +If you called `disable_motors()` or the power was interrupted, the head will drop under gravity. This is normal. It does not damage the robot. + +### Something smells like it's burning +**Power off immediately and unplug.** This is not normal. Do not power on again until you have inspected all cables. Contact support if you find damage. + +--- + +## "My code keeps crashing" + +### Debug checklist +- [ ] Daemon is running +- [ ] No other apps are connected +- [ ] Virtual environment is activated +- [ ] `reachy-mini` is installed in the current environment +- [ ] Basic connection test passes: `python -c "from reachy_mini import ReachyMini; print('OK')"` +- [ ] Minimal demo runs: `python examples/minimal_demo.py` +- [ ] Motors are enabled + +### Common code issues + +| Error | Cause | Fix | +|-------|-------|-----| +| `ModuleNotFoundError` | Wrong venv or not installed | Activate venv, `uv pip install reachy-mini` | +| `ConnectionError` | Daemon not running | Start daemon | +| `ValueError` on motion | Wrong input format | Use `create_head_pose()`, pass radians for antennas | +| `TypeError` | Wrong argument types | Check function signatures in SDK docs | + +### Read the traceback +Python tracebacks read bottom-to-top. The last line is the actual error. The lines above show where in your code it happened. Start from the bottom. + +--- + +## Recovery Procedures + +### Full reset (Wireless) +If nothing else works: +1. SSH in: `ssh pollen@reachy-mini.local` (password: `root`) +2. Restart the daemon: `systemctl restart reachy-mini-daemon.service` +3. If that doesn't help, reboot: `sudo reboot` +4. Last resort: [Reflash the Raspberry Pi ISO](../source/platforms/reachy_mini/reflash_the_rpi_ISO.md) + +### Full reset (Lite) +1. Close the daemon. +2. Unplug USB and power. +3. Wait 10 seconds. +4. Reconnect power, then USB. +5. Start the daemon. +6. Test with `python examples/minimal_demo.py`. + +### Reinstall the SDK +```bash +uv pip uninstall reachy-mini +uv pip install reachy-mini +``` + +--- + +## Getting Human Help + +If you have worked through this guide and are still stuck: + +1. **Discord** (fastest): https://discord.gg/Y7FgMqHsub +2. **GitHub Issues:** https://github.com/pollen-robotics/reachy_mini/issues + +When asking for help, include: +- What you expected to happen +- What actually happened +- The full error message / traceback +- Your OS, Python version, and SDK version (`pip show reachy-mini`) +- Whether the minimal demo (`examples/minimal_demo.py`) works +- Lite or Wireless version diff --git a/docs/advanced/HELP_ARCHITECTURE.md b/docs/advanced/HELP_ARCHITECTURE.md new file mode 100644 index 000000000..0d0f0c0e8 --- /dev/null +++ b/docs/advanced/HELP_ARCHITECTURE.md @@ -0,0 +1,229 @@ +# Reachy Mini -- System Architecture + +How the robot thinks. A mental map for contributors. + +--- + +## High-Level Block Diagram + +``` +┌──────────────────────────────────────────────────────────────────────┐ +│ USER CODE / APP │ +│ (Python script, ReachyMiniApp subclass, or HTTP/WebSocket client) │ +└────────────────────────────┬─────────────────────────────────────────┘ + │ + ┌──────────────┴──────────────┐ + │ SDK (ReachyMini) │ + │ reachy_mini/reachy_mini.py │ + │ │ + │ - goto_target / set_target │ + │ - look_at_image / world │ + │ - enable/disable motors │ + │ - media (camera, audio) │ + │ - recording / playback │ + └──────────────┬──────────────┘ + │ + ┌──────────────┴──────────────┐ + │ ZENOH TRANSPORT LAYER │ + │ io/zenoh_client.py │ + │ io/zenoh_server.py │ + │ │ + │ Pub/Sub over localhost:7447 │ + │ or network (peer scouting) │ + └──────────────┬──────────────┘ + │ +┌────────────────────────────┴─────────────────────────────────────────┐ +│ DAEMON │ +│ daemon/app/main.py (FastAPI + Uvicorn, port 8000) │ +│ │ +│ ┌─────────────────┐ ┌─────────────────┐ ┌──────────────────────┐│ +│ │ REST API │ │ WebSocket API │ │ Dashboard (HTML) ││ +│ │ /api/move/ │ │ /api/state/ws │ │ / ││ +│ │ /api/state/ │ │ /api/move/ws │ │ /settings ││ +│ │ /api/motors/ │ │ │ │ ││ +│ │ /api/apps/ │ │ │ │ ││ +│ │ /api/daemon/ │ │ │ │ ││ +│ └────────┬────────┘ └────────┬────────┘ └──────────────────────┘│ +│ │ │ │ +│ ┌────────┴────────────────────┴─────────┐ │ +│ │ DAEMON ORCHESTRATOR │ │ +│ │ daemon/daemon.py │ │ +│ │ │ │ +│ │ State: NOT_INITIALIZED → STARTING │ │ +│ │ → RUNNING ↔ ERROR → STOPPING │ │ +│ │ → STOPPED │ │ +│ └────────────────────┬──────────────────┘ │ +│ │ │ +│ ┌────────────────────┴──────────────────┐ │ +│ │ BACKEND (runs in own thread) │ │ +│ │ │ │ +│ │ ┌─────────────┐ ┌─────────────────┐ │ │ +│ │ │RobotBackend │ │ MujocoBackend │ │ │ +│ │ │(real HW) │ │ (physics sim) │ │ │ +│ │ └──────┬──────┘ └────────┬────────┘ │ │ +│ │ │ ┌────────┴────────┐ │ │ +│ │ │ │MockupSimBackend │ │ │ +│ │ │ │(no physics) │ │ │ +│ │ │ └─────────────────┘ │ │ +│ └─────────┼────────────────────────────┘ │ +│ │ │ +│ ┌─────────┴──────────┐ ┌────────────────────┐ │ +│ │ KINEMATICS ENGINE │ │ MEDIA MANAGER │ │ +│ │ (pluggable) │ │ (pluggable) │ │ +│ │ │ │ │ │ +│ │ Analytical (Rust) │ │ OpenCV / SoundDev │ │ +│ │ Placo (Python) │ │ GStreamer │ │ +│ │ NN (ONNX) │ │ WebRTC │ │ +│ └────────────────────┘ └────────────────────┘ │ +└──────────────────────────────────────────────────────────────────────┘ + │ + ┌──────────────┴──────────────┐ + │ HARDWARE / SIM │ + │ │ + │ 9 Dynamixel motors │ + │ Camera (IMX708) │ + │ 4-mic array (XMOS XVF3800) │ + │ Speaker (5W) │ + │ IMU (BMI088, Wireless only) │ + └─────────────────────────────┘ +``` + +--- + +## Core Subsystems and Responsibilities + +### 1. SDK Client (`reachy_mini.py`) + +**Responsibility:** User-facing Python API. Translates high-level intent into commands. + +- Owns the `ReachyMini` context manager +- Sends commands via `ZenohClient` (pub/sub) or REST (HTTP) +- Receives state updates via Zenoh subscriptions +- Manages media backend selection and lifecycle +- Performs `look_at_image/world` coordinate transforms using camera calibration matrix +- Does NOT perform IK/FK -- that is delegated to the daemon + +### 2. Zenoh Transport (`io/`) + +**Responsibility:** Reliable pub/sub messaging between SDK and daemon. + +- `ZenohClient`: Publishes commands to `{prefix}/command`, receives joint positions and pose +- `ZenohServer`: Subscribes to commands, publishes state at control loop frequency +- Supports two network modes: + - **Localhost** (`tcp/localhost:7447`): Single machine, no discovery + - **Network** (`tcp/0.0.0.0:7447`): LAN with multicast/gossip scouting +- Heartbeat monitoring: client checks liveness every 1 second + +### 3. Daemon (`daemon/`) + +**Responsibility:** Hardware abstraction, safety enforcement, app lifecycle. + +- FastAPI server (port 8000) for REST/WebSocket API +- `Daemon` class manages backend lifecycle (start/stop/restart) +- `AppManager` runs user apps as subprocesses with process isolation +- Publishes daemon status every 1 second +- Mounts 10+ API routers for different concerns + +### 4. Backend (`daemon/backend/`) + +**Responsibility:** Motor control loop. The only code that talks to hardware. + +- Abstract `Backend` base class defines the control interface +- Three implementations: `RobotBackend`, `MujocoBackend`, `MockupSimBackend` +- Runs the 50Hz control loop in a dedicated thread +- Owns all state: current positions, target positions, motor mode +- Manages move execution (goto_target, play_move) with RLock serialization +- Performs IK/FK computation using pluggable kinematics engine + +### 5. Kinematics (`kinematics/`) + +**Responsibility:** Forward and inverse kinematics for the 6-DOF Stewart platform head. + +- Three pluggable engines with the same interface (IK + FK methods) +- `AnalyticalKinematics` (Rust FFI): Default, fastest (~1ms) +- `PlacoKinematics` (constraint solver): Most accurate, supports gravity compensation and collision detection (~50ms) +- `NNKinematics` (ONNX): Fast inference (~5ms) +- Safety limits enforced at the IK level (head pitch/roll, yaw delta) + +### 6. Motion (`motion/`) + +**Responsibility:** Time-parameterized trajectory generation. + +- `Move` abstract base: `evaluate(t) -> (head_pose, antennas, body_yaw)` +- `GotoMove`: Interpolated motion with 4 methods (minjerk, linear, ease, cartoon) +- `RecordedMove`: Playback from HuggingFace datasets with binary search interpolation +- Moves are deterministic functions of time -- no state, no side effects + +### 7. Media (`media/`) + +**Responsibility:** Camera frames and audio I/O. + +- `MediaManager` selects backend based on hardware variant and connection mode +- Camera: `get_frame()` returns BGR numpy array +- Audio: Ring-buffer recording and callback-based playback +- Direction of Arrival from XMOS mic array +- Four backends: OpenCV+SoundDevice (default), GStreamer, WebRTC, no_media + +### 8. Apps (`apps/`) + +**Responsibility:** Discoverable app ecosystem. + +- `ReachyMiniApp` base class with lifecycle (run, stop_event) +- `AppManager` runs apps as isolated subprocesses +- Sources: HuggingFace Spaces, local installs, curated store +- App assistant CLI for scaffolding and publishing + +--- + +## Data Ownership + +| Data | Owner | Published via | +|------|-------|---------------| +| Current joint positions | Backend (read from hardware) | Zenoh `{prefix}/joint_positions` @ 50Hz | +| Current head pose | Backend (computed via FK) | Zenoh `{prefix}/head_pose` @ 50Hz | +| Target joint positions | Backend (computed via IK from target pose) | Internal only | +| Target head pose | SDK client or move executor | Zenoh `{prefix}/command` | +| Motor control mode | Backend | REST `/api/motors/status` | +| Daemon state | Daemon orchestrator | Zenoh `{prefix}/daemon_status` @ 1Hz | +| IMU data | Backend (Wireless only) | Zenoh `{prefix}/imu_data` @ 50Hz | +| Camera frames | MediaManager (client-side) | Not published (local to client) | +| Audio samples | MediaManager (client-side) | Not published (local to client) | +| Recorded motion data | Backend | Zenoh `{prefix}/recorded_data` (on stop) | + +--- + +## Real-Time vs Non-Real-Time Paths + +### Real-Time (50Hz control loop, deterministic timing) +- Backend `_update()` method: read joints → FK → IK → write targets → publish +- Must complete within 20ms per iteration +- Runs in dedicated daemon thread, no GIL contention from asyncio +- Uses `multiprocessing.Event.wait()` for cross-platform precise timing + +### Soft Real-Time (100Hz motion playback) +- `play_move()` loop: evaluate trajectory → set targets → sleep +- Runs as asyncio task, subject to event loop scheduling +- Acceptable jitter: up to 5ms + +### Non-Real-Time (human-scale latency acceptable) +- REST API requests (state queries, motor mode changes) +- WebSocket state streaming (configurable, default 10Hz) +- App lifecycle management (start/stop/install) +- Dashboard rendering +- Dataset downloading and caching + +--- + +## Key Architectural Decisions + +1. **Client-server split via Zenoh.** The SDK (client) never touches hardware directly. This allows running AI workloads on a powerful laptop while the daemon runs on a Raspberry Pi CM4. + +2. **Backend runs in its own thread.** The control loop must not be blocked by FastAPI request handling, move execution, or app management. + +3. **IK lives in the daemon, not the client.** The client sends desired poses; the daemon computes joint angles. This centralizes safety enforcement and allows the daemon to reject infeasible poses. + +4. **Moves are pure functions of time.** `Move.evaluate(t)` takes a timestamp and returns a target. No internal state mutation. This makes moves composable and testable. + +5. **Single move at a time.** An RLock prevents concurrent goto_target and play_move calls. `set_target` is blocked while a move is running. This prevents race conditions on target joint positions. + +6. **Media is client-side.** Camera frames and audio are not routed through the daemon (except on Wireless via GStreamer/WebRTC). This avoids adding latency to the perception pipeline. diff --git a/docs/advanced/HELP_CONFIGURATION_MODEL.md b/docs/advanced/HELP_CONFIGURATION_MODEL.md new file mode 100644 index 000000000..1391d79d5 --- /dev/null +++ b/docs/advanced/HELP_CONFIGURATION_MODEL.md @@ -0,0 +1,258 @@ +# Reachy Mini -- Configuration Model + +Separate code from tuning. Know what is hardcoded, what is configurable, and when configs take effect. + +--- + +## Configuration Layers + +``` +┌─────────────────────────────────────────────────┐ +│ HARDCODED CONSTANTS (code changes required) │ +│ Safety limits, coordinate frames, poses │ +├─────────────────────────────────────────────────┤ +│ BOOT-TIME CONFIG (daemon restart required) │ +│ Hardware YAML, kinematics engine, serial port │ +├─────────────────────────────────────────────────┤ +│ RUNTIME CONFIG (takes effect immediately) │ +│ Motor mode, targets, recording, app start/stop │ +└─────────────────────────────────────────────────┘ +``` + +--- + +## Hardcoded Constants (Code Changes Required) + +### Safety Limits + +**Location:** `kinematics/analytical_kinematics.py`, enforced in IK + +| Parameter | Value | Source | +|-----------|-------|--------| +| Head pitch range | [-40, +40] degrees | Mechanical constraint | +| Head roll range | [-40, +40] degrees | Mechanical constraint | +| Head yaw range | [-180, +180] degrees | Full rotation | +| Body yaw range | [-160, +160] degrees | Mechanical constraint | +| Max yaw delta (head - body) | 65 degrees | Collision avoidance | + +These are not configurable. Changing them risks physical damage to the robot. + +### Predefined Poses + +**Location:** `daemon/backend/abstract.py` + +```python +INIT_HEAD_POSE = np.eye(4) # Neutral position (identity matrix) +SLEEP_HEAD_POSE = [specific 4x4 matrix] # Head down resting position +SLEEP_HEAD_JOINT_POSITIONS = [0, -0.98, ...] # Joint-space sleep pose +SLEEP_ANTENNAS_JOINT_POSITIONS = [-3.05, 3.05]# Antennas folded +``` + +### Camera Calibration + +**Location:** `reachy_mini.py` + +```python +T_head_cam = [ + Position: [0.0437, 0, 0.0512] meters + Rotation: 90 deg roll, then 90 deg pitch +] +``` + +### Kinematics Tolerances + +**Location:** `daemon/backend/abstract.py` + +```python +_fk_kin_tolerance = 1e-3 # ~0.25 degrees for FK convergence +_ik_kin_tolerance = { + "rad": 2e-3, # ~0.1 degrees for IK + "m": 0.5e-3, # 0.5mm position tolerance +} +``` + +--- + +## Boot-Time Configuration (Daemon Restart Required) + +### Hardware Config YAML + +**Location:** `src/reachy_mini/assets/config/hardware_config.yaml` + +**Format:** +```yaml +version: beta +serial: + baudrate: 1000000 + +motors: + body_rotation: + id: 10 + offset: 0 + angle_limit: + lower: 0 + upper: 4095 + return_delay_time: 0 + shutdown_error: 52 + operating_mode: 3 + pid: + p: 200 + i: 0 + d: 0 + + stewart_1: + id: 11 + offset: 1024 + angle_limit: + lower: 1502 + upper: 2958 + pid: + p: 300 + i: 0 + d: 0 + # ... stewart_2 through stewart_6, right_antenna, left_antenna +``` + +**What you can tune:** +- **PID gains** (p, i, d): Reduce P to ~180 and increase D to ~10 to reduce motor jitter +- **Angle limits**: Hardware position bounds in servo ticks (0-4095) +- **Offset**: Encoder zero-point offset per motor + +**What you should not change:** +- Motor IDs (10-18) -- must match physical wiring +- Baudrate -- must match firmware +- Operating mode -- position control (3) is the standard mode + +**Custom config path:** +```bash +reachy-mini-daemon --hardware-config-filepath /path/to/my_config.yaml +``` + +### Daemon CLI Arguments + +**Location:** `daemon/app/main.py` `Args` dataclass + +| Argument | Default | Type | Notes | +|----------|---------|------|-------| +| `--sim` | False | flag | MuJoCo simulation mode | +| `--mockup-sim` | False | flag | Lightweight mock mode | +| `--scene` | "empty" | str | MuJoCo scene: "empty", "minimal" | +| `--headless` | False | flag | No GUI (sim only) | +| `--serialport` | "auto" | str | Serial port path or "auto" | +| `--kinematics-engine` | "AnalyticalKinematics" | str | "AnalyticalKinematics", "Placo", "NN" | +| `--check-collision` | False | flag | Enable collision detection (Placo only) | +| `--use-audio` | True | flag | Enable audio hardware | +| `--log-level` | "INFO" | str | Python logging level | +| `--fastapi-host` | "0.0.0.0" | str | Bind address | +| `--fastapi-port` | 8000 | int | HTTP port | +| `--robot-name` | "reachy_mini" | str | Zenoh topic prefix | +| `--wake-up-on-start` | True | flag | Play wake-up animation | +| `--goto-sleep-on-stop` | True | flag | Play sleep animation on shutdown | +| `--preload-datasets` | False | flag | Pre-download motion datasets | +| `--dataset-update-interval-hours` | 24.0 | float | Auto-update check interval | +| `--wireless-version` | False | flag | Enable wireless-specific features | +| `--localhost-only` | None | bool | Force Zenoh to localhost mode | + +These are set once at daemon startup. Changing them requires a daemon restart. + +### Zenoh Network Configuration + +**Determined at startup based on `localhost_only` flag:** + +| Mode | Client Config | Server Config | +|------|--------------|---------------| +| Localhost | `tcp/localhost:7447`, no scouting | `tcp/localhost:7447`, no multicast | +| Network | Peer mode, multicast + gossip | `tcp/0.0.0.0:7447`, multicast + gossip | + +--- + +## Runtime Configuration (Immediate Effect) + +### Motor Control Mode + +Changed via REST API or SDK, takes effect on next control loop tick (20ms): + +```python +# SDK +mini.enable_motors() +mini.disable_motors() +mini.enable_gravity_compensation() + +# REST +POST /api/motors/set_mode/enabled +POST /api/motors/set_mode/disabled +POST /api/motors/set_mode/gravity_compensation +``` + +### Motion Targets + +Changed on every call, applied at control loop frequency: + +```python +mini.set_target(head=pose, antennas=[0.5, -0.5], body_yaw=0.3) +mini.goto_target(head=pose, duration=1.0, method="minjerk") +``` + +### Media Backend + +Selected at `ReachyMini()` construction time: + +```python +with ReachyMini(media_backend="default") as mini: # OpenCV + SoundDevice +with ReachyMini(media_backend="gstreamer") as mini: # GStreamer +with ReachyMini(media_backend="webrtc") as mini: # WebRTC (remote Wireless) +with ReachyMini(media_backend="no_media") as mini: # No camera/audio +``` + +Auto-detection happens if not specified: checks daemon for wireless flag, then checks local camera availability. + +### Recording + +Start/stop at any time: + +```python +mini.start_recording() +# ... robot moves ... +data = mini.stop_recording() +``` + +### App Lifecycle + +Start/stop apps without restarting daemon: + +``` +POST /api/apps/start-app/{app_name} +POST /api/apps/stop-current-app +``` + +--- + +## Safe Defaults + +Every configuration has a safe default that works out of the box: + +| Parameter | Default | Why safe | +|-----------|---------|----------| +| Kinematics | AnalyticalKinematics | Fastest, fits in timing budget | +| Collision check | False | Avoid false positives during development | +| Media backend | "default" (auto) | Works on all platforms | +| Wake up on start | True | Robot visually confirms it's ready | +| Sleep on stop | True | Robot goes to safe rest position | +| Serial port | "auto" | Discovers correct port automatically | +| Control loop | 50Hz | Proven frequency for Dynamixel motors | +| Motion playback | 100Hz | 2x oversampling for smooth motion | +| Audio sample rate | 16000 Hz | ReSpeaker native rate | + +--- + +## Configuration File Locations + +| File | Location | Purpose | +|------|----------|---------| +| Hardware config | `src/reachy_mini/assets/config/hardware_config.yaml` | Motor PID, limits, IDs | +| Kinematics data | `src/reachy_mini/assets/config/kinematics_data.json` | Stewart platform geometry | +| URDF models | `src/reachy_mini/descriptions/reachy_mini/urdf/` | Robot model for Placo | +| MuJoCo scenes | `src/reachy_mini/descriptions/reachy_mini/mjcf/scenes/` | Simulation environments | +| ONNX models | `src/reachy_mini/assets/models/` | Neural network kinematics | +| Audio assets | `src/reachy_mini/assets/sounds/` | Wake-up, sleep, dance sounds | +| Firmware | `src/reachy_mini/assets/firmware/` | Motor firmware binaries | diff --git a/docs/advanced/HELP_DATA_FLOW.md b/docs/advanced/HELP_DATA_FLOW.md new file mode 100644 index 000000000..d43cd13f4 --- /dev/null +++ b/docs/advanced/HELP_DATA_FLOW.md @@ -0,0 +1,285 @@ +# Reachy Mini -- Data Flow + +Make debugging logical instead of magical. Know where data comes from, where it goes, and where to probe. + +--- + +## Primary Data Pipeline + +``` +┌─────────┐ ┌──────────┐ ┌─────────┐ ┌──────────┐ ┌─────────┐ +│ HARDWARE │────►│ BACKEND │────►│ ZENOH │────►│ SDK │────►│ USER │ +│ │ │ │ │ SERVER │ │ CLIENT │ │ CODE │ +│ Motors │ 50Hz│ FK comp │ 50Hz│ Publish │ sub │ Cache │ get │ │ +│ IMU │────►│ State │────►│ Topics │────►│ Pose │────►│ │ +│ Camera │ │ update │ │ │ │ Joints │ │ │ +└─────────┘ └──────────┘ └─────────┘ └──────────┘ └─────────┘ + ▲ │ + │ ┌───────────────────┘ + │ │ send_command() + │ ▼ + ┌────┴─────┐ ┌──────────┐ + │ IK comp │◄─│ ZENOH │ + │ Target │ │ CLIENT │ + │ joints │ │ command │ + └──────────┘ └──────────┘ +``` + +--- + +## Zenoh Topics (Pub/Sub) + +All topics are prefixed with the robot name (default: `reachy_mini`). + +### Published by Daemon (Server → Client) + +| Topic | Frequency | Format | Contents | +|-------|-----------|--------|----------| +| `{prefix}/joint_positions` | 50Hz | JSON | `{"head_joint_positions": [7], "antennas_joint_positions": [2]}` | +| `{prefix}/head_pose` | 50Hz | JSON | `{"head_pose": [16]}` (flattened 4x4 matrix) | +| `{prefix}/imu_data` | 50Hz | JSON | `{"accelerometer": [3], "gyroscope": [3], "quaternion": [4], "temperature": float}` | +| `{prefix}/daemon_status` | 1Hz | JSON | DaemonStatus with state, backend_status, errors | +| `{prefix}/task_progress` | On event | JSON | `{"uuid": str, "finished": bool, "error": str\|null}` | +| `{prefix}/recorded_data` | On stop | JSON | Array of timestamped joint position frames | + +### Published by Client (Client → Server) + +| Topic | Frequency | Format | Contents | +|-------|-----------|--------|----------| +| `{prefix}/command` | On demand | JSON | Command dict (see below) | +| `{prefix}/task` | On demand | JSON | TaskRequest (goto_target, play_move) | + +### Command Message Structure + +```json +{ + "torque": true, // Enable/disable motors + "ids": ["stewart_1", "right_antenna"], // Specific motors (null = all) + "head_pose": [16 floats], // Target 4x4 pose (flattened) + "head_joint_positions": [7 floats], // Target joint positions + "body_yaw": 0.5, // Target body yaw (rad) + "antennas_joint_positions": [2 floats],// Target antenna angles (rad) + "gravity_compensation": true, // Toggle mode + "automatic_body_yaw": true, // Auto body tracking + "start_recording": true, // Begin recording + "stop_recording": true, // Stop and publish + "set_target_record": {...} // Recording frame data +} +``` + +All fields are optional. Only present fields are applied. + +--- + +## REST/WebSocket Data Flow + +### State Reading + +``` +GET /api/state/full + → Daemon reads backend.current_head_pose, current_joint_positions, etc. + → Serializes to FullState Pydantic model + → Returns JSON + +WebSocket /api/state/ws/full?frequency=10 + → Every 100ms: read backend state → serialize → send +``` + +### Motion Commands + +``` +POST /api/move/goto + Body: {head_pose, antennas, body_yaw, duration, interpolation} + → Router creates GotoMove + → Launches asyncio.Task with backend.goto_target() + → Returns MoveUUID immediately + → Task runs play_move at 100Hz, setting targets each tick + → Control loop picks up targets at 50Hz + → On completion: publishes to WebSocket /api/move/ws/updates + +POST /api/move/set_target + Body: {target_head_pose, target_antennas, target_body_yaw} + → If move is running: returns {"status": "ignored"} + → Otherwise: sets backend.target_head_pose, ik_required=True + → Returns {"status": "ok"} + → Next control loop tick: IK computed, joints written to hardware +``` + +--- + +## Camera Data Flow + +Camera data does NOT flow through the daemon (except Wireless WebRTC): + +### Lite (Direct USB) +``` +Camera (USB) → OpenCV VideoCapture → mini.media.get_frame() → User code +``` + +### Wireless Local (on CM4) +``` +Camera (CSI) → GStreamer pipeline → Unix socket → mini.media.get_frame() → User code +``` + +### Wireless Remote +``` +Camera (CSI) → GStreamer → WebRTC encode → Network → WebRTC decode → mini.media.get_frame() → User code +``` + +--- + +## Audio Data Flow + +### Recording (Microphone → User) +``` +Mic array → SoundDevice InputStream callback → Ring buffer (deque) + → mini.media.get_audio_sample() → User code +``` + +Buffer: deque with 60-second max capacity. Overflow drops oldest chunks. + +### Playback (User → Speaker) +``` +User code → mini.media.push_audio_sample(data) → Output buffer (list) + → SoundDevice OutputStream callback → Speaker +``` + +Push is non-blocking. Callback drains buffer in real-time. Silence padding on underrun. + +--- + +## Recording Data Flow + +``` +1. mini.start_recording() + → ZenohClient sends {"start_recording": true} + → ZenohServer._handle_command() sets backend.is_recording = True + +2. Control loop (50Hz): + → backend.append_record({timestamp, head_joints, antennas, head_pose, body_yaw}) + → Guarded by _rec_lock + +3. mini.stop_recording() + → ZenohClient sends {"stop_recording": true} + → Backend: acquire _rec_lock, swap buffer, publish to Zenoh + → ZenohClient: wait for recorded_data message (timeout 5s) + → Returns recorded data to user +``` + +--- + +## Where to Probe When Things Go Wrong + +### "Robot doesn't respond to commands" + +``` +Probe points (in order): +1. Is daemon running? + → GET /api/daemon/status + → Check state == "running" + +2. Is Zenoh connected? + → SDK: mini.client._is_alive should be True + → Check heartbeat: client receives joint_positions within 1s + +3. Is control loop running? + → GET /api/daemon/status → backend_status.control_loop_stats + → mean_control_loop_frequency should be ~50Hz + +4. Are motors enabled? + → GET /api/motors/status → mode should be "enabled" + +5. Is a move blocking commands? + → GET /api/move/running → should be empty list +``` + +### "Motion is jerky" + +``` +Probe points: +1. Control loop overruns? + → backend_status.control_loop_stats.max_control_loop_interval + → Should be <25ms. If >50ms, something is blocking the loop. + +2. How often is set_target being called? + → Add logging: timestamp each set_target call + → Should be 50-100Hz for smooth motion + +3. Multiple set_target sources? + → Search code for all set_target/goto_target calls + → Must be exactly ONE place calling set_target in your control loop + +4. IK failures? + → Check daemon logs for "IK error" warnings + → Frequent IK failures = jerky motion (old target retained) +``` + +### "State looks stale" + +``` +Probe points: +1. Zenoh subscriber receiving? + → SDK: check mini.client._last_head_pose changes on each read + +2. Backend publishing? + → Check daemon logs for publish errors + +3. Network latency (Wireless)? + → Compare timestamps between backend and client + → >100ms latency = noticeable lag +``` + +### "Audio/video not working" + +``` +Probe points: +1. Media backend selection + → Log mini.media.backend value + → Ensure correct backend for your platform + +2. Camera device detected? + → OpenCV: cv2.VideoCapture(device_id).isOpened() + → GStreamer: check pipeline element creation + +3. Audio device detected? + → SoundDevice: sd.query_devices() → look for "Reachy Mini Audio" +``` + +--- + +## Message Lifecycle: goto_target (Complete Path) + +``` +1. User calls mini.goto_target(head=pose, duration=1.0, method="minjerk") + +2. SDK creates GotoTaskRequest, assigns UUID + → ZenohClient publishes to {prefix}/task + +3. ZenohServer receives task + → Deserializes GotoTaskRequest + → Creates asyncio.Task running backend.goto_target() + +4. Backend.goto_target(): + a. Creates GotoMove(start=current_pose, target=pose, duration=1.0, method=minjerk) + b. Acquires _play_move_lock + c. Enters play_move loop at 100Hz: + - t = time.time() - t0 + - head, antennas, body_yaw = move.evaluate(t) + - backend.target_head_pose = head (sets ik_required=True) + - await asyncio.sleep(0.01) + +5. Backend control loop (50Hz, separate thread): + a. Reads current joint positions from hardware + b. Computes FK → updates current_head_pose + c. Sees ik_required=True → computes IK from target_head_pose + d. Writes target_head_joint_positions to motors + e. Publishes updated joint_positions and head_pose via Zenoh + +6. Move completes (t >= duration): + a. Releases _play_move_lock + b. Publishes TaskProgress(uuid, finished=True) via Zenoh + +7. SDK receives TaskProgress + → Sets event on task UUID + → goto_target() returns to user +``` diff --git a/docs/advanced/HELP_ERROR_HANDLING.md b/docs/advanced/HELP_ERROR_HANDLING.md new file mode 100644 index 000000000..fd68b646b --- /dev/null +++ b/docs/advanced/HELP_ERROR_HANDLING.md @@ -0,0 +1,241 @@ +# Reachy Mini -- Error Handling + +Make failures graceful and recoverable. A system that fails well feels robust. + +--- + +## Error Categories + +### 1. Hardware Errors (Motor Controller) + +**Source:** `RobotBackend.read_hardware_errors()`, checked every 1 second. + +| Error Flag | Meaning | Severity | +|-----------|---------|----------| +| Input Voltage Error | Voltage outside motor spec range | **Ignore** -- intentional (robot runs at upper range) | +| Overheating Error | Motor thermal shutdown | **Recover** -- power off, wait 5min, power on | +| Electrical Shock Error | Short circuit or power supply issue | **Investigate** -- check cables | +| Overload Error | Motor physically stuck | **Investigate** -- check orientation marks, cables | + +**Handling pattern:** +```python +# In RobotBackend._update() +hardware_errors = self.read_hardware_errors() +for motor_name, errors in hardware_errors.items(): + # Voltage errors are filtered out (expected) + logger.error(f"Motor '{motor_name}' hardware errors: {errors}") +``` + +Hardware errors are logged but do not stop the control loop. The robot continues operating with degraded motors. + +### 2. Communication Errors (Serial / Zenoh) + +**Motor communication lost:** +```python +# In RobotBackend._update() +if last_alive + 1.0 < time.time(): + self.error = "No response from the robot's motor for the last second." + raise RuntimeError(self.error) + # → Backend thread exits → Daemon state → ERROR +``` + +This is the most critical error. If the motor controller does not respond for 1 second, the backend assumes the serial connection is broken and shuts down. + +**Zenoh connection lost (client-side):** +```python +# In ZenohClient.check_alive() (background thread, 1Hz) +_is_alive = is_connected() # Waits 1s for joint_positions message +# If False, next send_command() raises ConnectionError +``` + +### 3. Kinematics Errors (IK Failures) + +**Unreachable pose:** +```python +# In Backend.update_target_head_joints_from_ik() +try: + joints = self.head_kinematics.ik(pose, body_yaw=body_yaw) + if joints is None or np.any(np.isnan(joints)): + raise ValueError("Collision detected or head pose not achievable!") +except ValueError as e: + log_throttling.by_time(logger, interval=0.5).warning(f"IK error: {e}") + # Previous valid target retained -- robot holds last good position +``` + +IK failures are **not fatal**. The control loop continues with the last valid target. This happens when: +- User requests a pose outside the workspace +- Collision detection rejects the pose (Placo only) +- Numerical issues in the solver + +### 4. Application Errors (App Crashes) + +**Detection:** `AppManager.monitor_process()` watches stdout/stderr. + +```python +# App exits with non-zero code +if process.returncode != 0: + last_stderr = captured_stderr[-10:] # Last 10 lines + status.state = AppState.ERROR + status.error = "\n".join(last_stderr) +``` + +**Recovery:** Post-app cleanup returns robot to init pose: +```python +await backend.goto_target(INIT_HEAD_POSE, antennas=[0.0, 0.0], duration=1.0) +``` + +### 5. Task Errors (Async Motion) + +**Detection:** asyncio task wrapper catches exceptions: +```python +async def wrapped_task(): + error = None + try: + await task_coroutine() + except Exception as e: + error = str(e) + progress = TaskProgress(uuid=task_uuid, finished=True, error=error) + task_progress_pub.put(progress.model_dump_json()) +``` + +**Client-side propagation:** +```python +def wait_for_task_completion(uid, timeout=5.0): + if not tasks[uid].event.wait(timeout): + raise TimeoutError("Task did not complete in time.") + if tasks[uid].error is not None: + raise Exception(f"Task failed: {tasks[uid].error}") +``` + +--- + +## Retry vs Fail-Fast Rules + +| Error Type | Strategy | Rationale | +|-----------|----------|-----------| +| IK failure | **Retain previous target** | Transient -- next target may be reachable | +| Motor communication timeout (< 1s) | **Retry next loop** | Brief glitches happen on serial bus | +| Motor communication timeout (> 1s) | **Fail-fast: shutdown** | Sustained loss means hardware disconnected | +| Zenoh connection lost | **Fail-fast: raise ConnectionError** | Client should reconnect or alert user | +| App crash | **Fail-fast: mark ERROR** | Do not auto-restart (may crash again) | +| goto_target move lock busy | **Fail silently** | Another move is running; caller can retry | +| set_target while move running | **Drop command** | Move has priority; command would conflict | +| Dataset download failure | **Return empty, log warning** | Non-critical; app can work without new data | + +--- + +## Escalation Paths + +``` +Level 1: Log warning (no user action needed) +├── IK failure (throttled to every 0.5s) +├── Control loop overrun (single occurrence) +├── Hardware error: Input Voltage Error +└── Circular buffer overrun (no video consumer) + +Level 2: Log error (user should investigate) +├── Hardware error: Overload, Overheating, Electrical Shock +├── Motor jitter (PID tuning needed) +├── App process crashed +└── Control loop sustained overruns + +Level 3: Daemon stops (user must restart) +├── Motor communication lost > 1 second +├── Backend initialization failure (wrong serial port, etc.) +├── Backend thread unhandled exception +└── Daemon stop timeout (5 seconds) + +Level 4: User must intervene physically +├── Motor thermal shutdown (wait 5 minutes) +├── Cable damage (inspect and replace) +├── Power supply failure (check connections) +└── Board switch in wrong position (Wireless) +``` + +--- + +## User-Visible vs Internal Errors + +### User-Visible (Surfaced to SDK / REST API) + +| Error | How surfaced | Recovery | +|-------|-------------|----------| +| Connection lost | `ConnectionError` from SDK | Recreate `ReachyMini()` context | +| Motor timeout | `GET /api/daemon/status` → `error` field | Restart daemon | +| Task failure | `wait_for_task_completion()` raises | Check error message, fix command | +| App crash | `GET /api/apps/current-app-status` → `error` | Fix app code, restart | +| Invalid input | `ValueError` from `goto_target` / `set_target` | Fix input format | + +### Internal (Logged but not surfaced) + +| Error | Where logged | Why not surfaced | +|-------|-------------|-----------------| +| IK failure | Daemon log (warning) | Transient, self-healing | +| Control loop overrun | Daemon log (debug) | Brief jitters are normal | +| Hardware voltage error | Daemon log (filtered out) | Expected by design | +| Recording buffer overflow | Daemon log (warning) | Self-correcting (drops old data) | + +--- + +## Error Handling Patterns in Code + +### Pattern 1: Graceful degradation (IK) +```python +try: + result = compute() +except ValueError: + log_throttling.warning("...") # Don't spam logs + # Keep previous state -- system continues +``` + +### Pattern 2: Fatal shutdown (motor loss) +```python +if not_responding_for_too_long: + self.error = "descriptive message" + raise RuntimeError(self.error) # Backend thread exits + # Daemon catches and transitions to ERROR state +``` + +### Pattern 3: Subprocess isolation (apps) +```python +process = await asyncio.create_subprocess_exec(...) +returncode = await process.wait() +if returncode != 0: + status.state = AppState.ERROR + status.error = last_stderr_lines +# Daemon is NOT affected -- continues running +``` + +### Pattern 4: Lock-based conflict avoidance (moves) +```python +if not self._try_start_move(): + return # Another move running -- silently skip +try: + await execute_move() +finally: + self._end_move() # Always release lock +``` + +### Pattern 5: Timeout with escalation (daemon lifecycle) +```python +await daemon.stop() +backend_thread.join(timeout=5.0) +if backend_thread.is_alive(): + state = DaemonState.ERROR # Could not stop cleanly +else: + state = DaemonState.STOPPED +``` + +--- + +## Writing Error-Resilient Extensions + +When adding new code to Reachy Mini: + +1. **Never swallow exceptions silently.** At minimum, log them. +2. **Never let exceptions escape the control loop.** Catch at the boundary and log. +3. **Use `log_throttling`** for high-frequency error paths (avoids log flood). +4. **Always release locks in `finally` blocks.** +5. **Set `self.error` before raising** in backend code, so the daemon can report it. +6. **Prefer returning `None` over raising** for query methods that may fail. +7. **Use timeouts on all blocking operations** (joins, event waits, network calls). diff --git a/docs/advanced/HELP_EXTENSION_POINTS.md b/docs/advanced/HELP_EXTENSION_POINTS.md new file mode 100644 index 000000000..0632a2f1c --- /dev/null +++ b/docs/advanced/HELP_EXTENSION_POINTS.md @@ -0,0 +1,272 @@ +# Reachy Mini -- Extension Points + +Enable new features without refactoring core systems. Turn users into contributors. + +--- + +## Where New Behaviors Plug In + +### 1. New App (Most Common Extension) + +The app system is the primary extension point. Apps are isolated subprocesses that use the SDK. + +**How to add:** +```bash +reachy-mini-app-assistant create my_app ~/projects/my_app --publish +``` + +**What you get:** +- Scaffolded `main.py` with `ReachyMiniApp` subclass +- `pyproject.toml` with `reachy_mini_apps` entry point +- Optional `static/` directory for web UI +- Publishable to HuggingFace Spaces + +**What not to modify:** The app system itself (`apps/manager.py`, `apps/sources/`). Apps interact exclusively through the SDK or REST API. + +**Lifecycle your app must respect:** +```python +class MyApp(ReachyMiniApp): + def run(self, reachy_mini: ReachyMini, stop_event: threading.Event) -> None: + while not stop_event.is_set(): + # Your logic here + time.sleep(0.01) +``` + +The `stop_event` is set when the user stops your app from the dashboard. You must check it regularly and exit cleanly. + +--- + +### 2. New Kinematics Engine + +The kinematics system is pluggable via a common interface. + +**Interface to implement:** +```python +class MyKinematics: + def __init__(self, automatic_body_yaw: bool = True): + ... + + def ik( + self, + pose: np.ndarray, # 4x4 SE(3) matrix + body_yaw: float = 0.0, + check_collision: bool = False, + no_iterations: int = 0, + ) -> np.ndarray: # [7]: [body_yaw, stewart_1-6] + ... + + def fk( + self, + joint_angles: np.ndarray, # [7]: [body_yaw, stewart_1-6] + check_collision: bool = False, + no_iterations: int = 3, + ) -> np.ndarray: # 4x4 SE(3) matrix + ... +``` + +**Where to add it:** +1. Create `src/reachy_mini/kinematics/my_kinematics.py` +2. Add import in `src/reachy_mini/kinematics/__init__.py` with try/except fallback +3. Register the name in `daemon/app/main.py` `Args.kinematics_engine` choices +4. Add selection logic in `daemon/daemon.py` `_setup_backend()` + +**What not to modify:** The backend control loop structure or the `ik()`/`fk()` interface signatures. + +**Constraints your engine must satisfy:** +- IK must return 7-element array `[body_yaw, stewart_1, stewart_2, ..., stewart_6]` +- FK must return 4x4 homogeneous transformation matrix +- IK should return `None` or raise `ValueError` for unreachable poses +- Must enforce safety limits (head pitch/roll: [-40,+40]deg, yaw delta: max 65deg) +- IK latency must fit within 20ms control loop budget (or only recompute when target changes) + +--- + +### 3. New Motion Primitive + +Extend the motion system by subclassing `Move`. + +**Interface to implement:** +```python +from reachy_mini.motion.move import Move + +class MyMove(Move): + @property + def duration(self) -> float: + return self._duration + + @property + def sound_path(self) -> Path | None: + return self._sound_path # Optional synchronized audio + + def evaluate(self, t: float) -> tuple[ + np.ndarray | None, # 4x4 head pose (None = don't change) + np.ndarray | None, # [2] antenna angles in radians + float | None, # body yaw in radians + ]: + # Your trajectory computation here + # t is in [0, self.duration] + ... +``` + +**How to use it:** +```python +move = MyMove(duration=2.0) +await backend.play_move(move, play_frequency=100.0) +``` + +**What not to modify:** The `Move` base class, the `play_move()` loop, or the interpolation utilities. + +**Design constraints:** +- `evaluate()` must be a pure function of `t` -- no side effects, no state mutation +- Must complete in <1ms per call (called 100 times per second) +- Return `None` for any component you don't want to control + +--- + +### 4. New Backend + +For alternative simulation engines or hardware platforms. + +**Interface to implement:** Subclass `Backend` (from `daemon/backend/abstract.py`) and implement: + +```python +class MyBackend(Backend): + def run(self) -> None: + """Control loop. Must call self.ready.set() when initialized. + Must check self.should_stop.is_set() to exit.""" + ... + + def get_status(self) -> MyBackendStatus: + ... + + def get_present_head_joint_positions(self) -> np.ndarray: # [7] + ... + + def get_present_antenna_joint_positions(self) -> np.ndarray: # [2] + ... + + def get_motor_control_mode(self) -> MotorControlMode: + ... + + def set_motor_control_mode(self, mode: MotorControlMode) -> None: + ... + + def set_motor_torque_ids(self, ids: list[str], on: bool) -> None: + ... +``` + +**Where to add it:** +1. Create `src/reachy_mini/daemon/backend/my_backend/backend.py` +2. Add selection logic in `daemon/daemon.py` `_setup_backend()` +3. Add CLI flag in `daemon/app/main.py` `Args` dataclass + +**Critical contract:** Your `run()` method must: +- Run in a loop until `self.should_stop.is_set()` +- Call `self.ready.set()` after initialization (daemon waits 2s for this) +- Read targets set by the parent class (`self.target_head_pose`, etc.) +- Update current state (`self.current_head_pose`, etc.) +- Call `self.update_head_kinematics_model()` each iteration +- Publish state via the Zenoh publishers provided by `ZenohServer` + +--- + +### 5. New Media Backend + +For alternative camera/audio hardware. + +**Camera interface:** +```python +from reachy_mini.media.camera_base import CameraBase + +class MyCamera(CameraBase): + def open(self) -> None: ... + def read(self) -> np.ndarray | None: ... # BGR uint8 frame + def close(self) -> None: ... +``` + +**Audio interface:** +```python +from reachy_mini.media.audio_base import AudioBase + +class MyAudio(AudioBase): + def start_recording(self) -> None: ... + def get_audio_sample(self) -> np.ndarray | None: ... # float32 + def stop_recording(self) -> None: ... + def start_playing(self) -> None: ... + def push_audio_sample(self, data: np.ndarray) -> None: ... + def stop_playing(self) -> None: ... + def play_sound(self, sound_file: str) -> None: ... +``` + +**Where to add it:** +1. Create implementation files in `src/reachy_mini/media/` +2. Add a new `MediaBackend` enum value in `media/media_manager.py` +3. Add the selection case in `MediaManager.__init__()` + +--- + +### 6. New REST API Endpoint + +For exposing new daemon capabilities. + +**How to add:** +1. Create a router file in `src/reachy_mini/daemon/app/routers/my_router.py` +2. Use FastAPI patterns: + ```python + from fastapi import APIRouter, Request + router = APIRouter(prefix="/my_feature", tags=["my_feature"]) + + @router.get("/status") + async def get_status(request: Request): + backend = request.app.state.daemon.backend + return {"status": "ok"} + ``` +3. Mount in `daemon/app/main.py` `create_app()`: + ```python + app.include_router(my_router.router, prefix="/api") + ``` + +**What not to modify:** Existing router files (add new ones instead). + +--- + +## What is Stable vs Experimental + +### Stable (safe to build extensions on) + +| Component | Why stable | +|-----------|-----------| +| `ReachyMini` class public API | Versioned, documented, widely used | +| `Move` abstract class | Simple contract, unlikely to change | +| `Backend` abstract methods | Core architecture, three implementations depend on it | +| `CameraBase` / `AudioBase` interfaces | Clean abstractions with multiple implementations | +| REST API endpoints | Used by dashboard, web clients, external tools | +| Zenoh topic structure | SDK client depends on exact topics | +| Hardware config YAML format | Motor configuration must be backward compatible | + +### Experimental (may change between versions) + +| Component | Why experimental | +|-----------|----------------| +| Placo gravity compensation | Empirical torque constants, may need recalibration | +| WebRTC media backend | Platform support limited (Linux client only) | +| GStreamer pipeline details | Hardware-dependent configuration | +| App assistant templates | Evolving based on user feedback | +| Dashboard WebSocket message format | Not formally versioned | +| Raw packet interface | Direct hardware access, use at own risk | +| MuJoCo scene files | Scene format may change with MuJoCo updates | + +--- + +## Extension Checklist + +Before submitting an extension: + +- [ ] Implements the documented interface (no monkey-patching) +- [ ] Does not modify existing module files (add new files instead) +- [ ] Respects timing constraints (see HELP_TIMING_AND_REALTIME.md) +- [ ] Has unit tests (see HELP_TESTING_STRATEGY.md) +- [ ] Does not break existing tests: `pytest -vv -m 'not audio and not video...'` +- [ ] Passes linting: `ruff check src/` and `ruff format src/` +- [ ] Passes type checking: `mypy --install-types --non-interactive` +- [ ] Documented in this file (update extension point section) diff --git a/docs/advanced/HELP_MODULE_BOUNDARIES.md b/docs/advanced/HELP_MODULE_BOUNDARIES.md new file mode 100644 index 000000000..64708899d --- /dev/null +++ b/docs/advanced/HELP_MODULE_BOUNDARIES.md @@ -0,0 +1,233 @@ +# Reachy Mini -- Module Boundaries + +What each module is allowed to do. What it must never do. Where the lines are. + +--- + +## Module Ownership Map + +### `reachy_mini.py` (SDK Client) + +**Owns:** +- User-facing Python API surface +- Connection mode selection (auto, localhost, network) +- Camera calibration transforms (`look_at_image`, `look_at_world`) +- Media backend lifecycle +- `create_head_pose()` convenience function + +**Must never:** +- Compute IK/FK directly +- Write to motor hardware +- Enforce safety limits (that is the daemon's job) +- Manage daemon state transitions +- Run the control loop + +**Public interface:** `ReachyMini` class with documented methods. Everything else is internal. + +--- + +### `daemon/daemon.py` (Daemon Orchestrator) + +**Owns:** +- Daemon lifecycle state machine (NOT_INITIALIZED → RUNNING → STOPPED) +- Backend creation and teardown +- ZenohServer initialization +- WebRTC setup (Wireless) +- Status publishing (1Hz) + +**Must never:** +- Read/write motor hardware directly (that is Backend's job) +- Compute kinematics +- Process motion commands +- Manage app subprocesses (that is AppManager's job) + +**Public interface:** `start()`, `stop()`, `restart()`, `get_status()` + +--- + +### `daemon/backend/abstract.py` (Backend Base) + +**Owns:** +- Current and target joint positions/poses (all state variables) +- Control loop execution (`run()`) +- IK/FK computation dispatch +- Move execution lock (`_play_move_lock`) +- Recording buffer management +- Motor control mode transitions +- Wake-up and sleep animations + +**Must never:** +- Decide which backend to instantiate (Daemon does that) +- Manage network connections (ZenohServer does that) +- Handle HTTP/WebSocket requests (routers do that) +- Know about the app system + +**Public interface:** Abstract methods (`run`, `get_status`, `get_present_*`, `set_motor_control_mode`, `set_motor_torque_ids`), plus shared concrete methods (`goto_target`, `play_move`, `set_target`, `start_recording`, `stop_recording`). + +--- + +### `daemon/backend/robot/backend.py` (Robot Backend) + +**Owns:** +- Serial communication with motor controller (via `ReachyMiniPyControlLoop`) +- Hardware error detection and reporting +- PID gain application +- Gravity compensation torque computation +- IMU reading (Wireless) +- Motor operation mode switching (position vs torque vs current-limiting) + +**Must never:** +- Define the control interface (Backend base does that) +- Perform kinematics (pluggable engine does that) +- Handle network or API concerns + +--- + +### `daemon/backend/mujoco/backend.py` (MuJoCo Backend) + +**Owns:** +- MuJoCo physics simulation step +- Rendering and UDP frame streaming +- Scene management (empty, minimal) +- Joint-to-MuJoCo actuator mapping (including antenna sign flip) + +**Must never:** +- Touch real hardware +- Manage network connections + +--- + +### `kinematics/` (Kinematics Engines) + +**Owns:** +- Forward kinematics: joints → 4x4 pose +- Inverse kinematics: 4x4 pose → joints +- Safety limit enforcement (max yaw delta, pitch/roll bounds) +- Collision detection (Placo only) +- Gravity torque computation (Placo only) + +**Must never:** +- Read/write motor positions directly +- Know about Zenoh, REST, or any transport layer +- Manage timing or control loop frequency + +**Public interface per engine:** `ik(pose, body_yaw) -> joints`, `fk(joints) -> pose` + +--- + +### `motion/` (Motion Primitives) + +**Owns:** +- Time-parameterized trajectory evaluation +- Interpolation methods (minjerk, linear, ease, cartoon) +- Recorded move parsing and binary-search playback +- HuggingFace dataset loading and caching + +**Must never:** +- Call `set_target` or `goto_target` on the backend +- Know about hardware, network, or daemon state +- Maintain mutable state between evaluations + +**Public interface:** `Move.evaluate(t) -> (head_pose, antennas, body_yaw)`, `Move.duration` + +--- + +### `media/` (Media Manager) + +**Owns:** +- Camera frame acquisition +- Audio recording and playback buffers +- Direction of arrival from mic array +- Backend selection logic (OpenCV, GStreamer, WebRTC) +- Device detection (auto-find "Reachy Mini Audio" or "reSpeaker") + +**Must never:** +- Send motor commands +- Know about robot state or kinematics +- Participate in the control loop timing + +**Public interface:** `get_frame()`, `start_recording()`, `get_audio_sample()`, `push_audio_sample()`, `get_DoA()` + +--- + +### `io/` (Zenoh Transport) + +**Owns:** +- Zenoh session and publisher/subscriber lifecycle +- Topic key expressions (`{prefix}/command`, `{prefix}/joint_positions`, etc.) +- Message serialization/deserialization (JSON) +- Connection liveness monitoring (heartbeat) + +**Must never:** +- Interpret command semantics (ZenohServer just dispatches) +- Perform kinematics or motion computation +- Know about specific backend implementations + +**Public interface:** `ZenohClient.send_command()`, `ZenohClient.send_task_request()`, `ZenohServer._handle_command()`, `ZenohServer._handle_task_request()` + +--- + +### `apps/` (App System) + +**Owns:** +- App discovery (HuggingFace, local, installed) +- App installation/removal +- Subprocess execution and monitoring +- App scaffolding (app assistant CLI) + +**Must never:** +- Import or call backend code directly +- Access motor hardware +- Modify daemon state (except through the REST API) + +**Public interface:** `AppManager.start_app()`, `stop_current_app()`, `install_new_app()`, `list_available_apps()` + +--- + +### `daemon/routers/` (API Routers) + +**Owns:** +- HTTP request validation (Pydantic models) +- Response serialization +- WebSocket connection lifecycle +- Background job tracking (UUIDs) + +**Must never:** +- Access hardware directly +- Compute kinematics +- Own state -- all state lives in the backend or daemon + +**Each router is a thin translation layer** between HTTP semantics and backend method calls. + +--- + +## Boundary Violations to Watch For + +| Violation | Why it's dangerous | What to do instead | +|-----------|-------------------|-------------------| +| SDK computing IK locally | Safety limits bypassed, duplicate logic | Send target pose to daemon, let daemon compute IK | +| Router modifying backend state directly | Bypasses thread synchronization | Call async backend methods through daemon | +| Motion calling set_target | Creates coupling between trajectory and transport | Motion returns targets; backend applies them | +| Backend knowing about FastAPI | Prevents backend reuse in other contexts | Backend exposes methods; routers call them | +| Kinematics engine reading motor positions | Breaks pluggability | Backend passes positions to kinematics | +| App importing backend modules | Breaks process isolation | App uses ReachyMini SDK or REST API | + +--- + +## Stable vs Experimental APIs + +### Stable (safe to depend on) +- `ReachyMini` class public methods +- `create_head_pose()` utility +- `Move` abstract interface +- `RecordedMoves` dataset loading +- REST API endpoints (`/api/move/`, `/api/state/`, `/api/motors/`) +- `ReachyMiniApp` base class + +### Experimental (may change) +- Placo kinematics gravity compensation +- WebRTC media backend +- GStreamer pipeline configurations +- App assistant templates +- Dashboard WebSocket streaming format +- Raw packet interface (`/api/move/ws/raw/write`) diff --git a/docs/advanced/HELP_PERFORMANCE_BUDGETS.md b/docs/advanced/HELP_PERFORMANCE_BUDGETS.md new file mode 100644 index 000000000..2472330a7 --- /dev/null +++ b/docs/advanced/HELP_PERFORMANCE_BUDGETS.md @@ -0,0 +1,225 @@ +# Reachy Mini -- Performance Budgets + +Stop slow creep. Make tradeoffs explicit instead of accidental. + +--- + +## Control Loop Budget (20ms total) + +The backend control loop runs at 50Hz. Every component in the loop has a time budget. + +``` +┌──────────────────────────────────────────────────┐ +│ 20ms CONTROL LOOP BUDGET │ +├──────────────┬───────┬───────────────────────────┤ +│ Component │ Budget│ Actual (measured) │ +├──────────────┼───────┼───────────────────────────┤ +│ Read joints │ 5ms │ ~2ms (serial I/O) │ +│ FK compute │ 3ms │ ~1ms (analytical) │ +│ IK compute │ 5ms │ ~1ms (analytical) │ +│ │ │ ~5ms (NN) │ +│ │ │ ~50ms (Placo) ← OVERRUN │ +│ Write targets│ 2ms │ ~0.2ms (serial write) │ +│ Publish state│ 2ms │ ~0.5ms (Zenoh JSON) │ +│ Error checks │ 1ms │ ~0.1ms (every 1s only) │ +│ Margin │ 2ms │ ~15ms (analytical default) │ +├──────────────┼───────┼───────────────────────────┤ +│ TOTAL │ 20ms │ ~5ms (analytical) │ +└──────────────┴───────┴───────────────────────────┘ +``` + +**Overrun behavior:** If the loop exceeds 20ms, a debug warning is logged and the next iteration starts immediately (minimum 1ms sleep). Sustained overruns (>1 second without a response) trigger a fatal error. + +**How to monitor:** +```python +# Via SDK +status = mini.client.get_status() +stats = status["backend_status"]["control_loop_stats"] +print(f"Mean frequency: {stats['mean_control_loop_frequency']:.1f} Hz") +print(f"Max interval: {stats['max_control_loop_interval']*1000:.1f} ms") + +# Via REST +# GET /api/daemon/status → backend_status.control_loop_stats +``` + +**Target:** Mean frequency ~50Hz, max interval <25ms. + +--- + +## IK Engine Latency Budget + +| Engine | Latency | Fits in 20ms? | Notes | +|--------|---------|---------------|-------| +| AnalyticalKinematics (Rust) | ~1ms | Yes (15ms margin) | Default. Always safe. | +| NNKinematics (ONNX) | ~5ms | Yes (10ms margin) | Good alternative. | +| PlacoKinematics | ~50ms | **No** | Only recomputes when target changes. OK for goto_target. Risky for set_target at high frequency. | + +**Rule:** If you are calling `set_target()` at >10Hz, use AnalyticalKinematics. PlacoKinematics is for goto_target (interpolated) use cases where IK is computed infrequently. + +--- + +## Motion Playback Budget (10ms per tick) + +The `play_move()` loop runs at 100Hz. + +| Component | Budget | Typical | +|-----------|--------|---------| +| `move.evaluate(t)` | 5ms | <1ms (polynomial or lookup) | +| Set targets | 1ms | <0.1ms (attribute assignment) | +| asyncio.sleep overhead | 2ms | ~1ms | +| Margin | 2ms | ~8ms | + +**If evaluate() exceeds 5ms:** Motion will visibly stutter. Pre-compute trajectories if possible. + +--- + +## CPU Budget by Subsystem + +### Raspberry Pi CM4 (Wireless -- 4-core ARM, 4GB RAM) + +| Subsystem | CPU Budget | Rationale | +|-----------|-----------|-----------| +| Backend control loop | 1 core, <25% | Must be deterministic | +| Daemon (FastAPI + async) | 1 core | Request handling, app management | +| GStreamer media pipeline | 1 core | Audio/video encoding | +| User app | 1 core (remaining) | App code runs as subprocess | + +**Total:** All 4 cores allocated. On Wireless, heavy AI workloads (LLMs, vision models) should run on the user's laptop, not on the CM4. + +### Developer Laptop (Lite / Simulation) + +CPU is typically not a constraint. The daemon uses <10% of a modern laptop CPU. + +--- + +## Memory Budget + +### Raspberry Pi CM4 (4GB total) + +| Component | Budget | Notes | +|-----------|--------|-------| +| OS + system | 500MB | Linux base | +| Daemon | 300MB | Python runtime + libraries | +| GStreamer pipelines | 200MB | Audio/video buffers | +| App subprocess | 500MB | Depends on app | +| App venvs | 1.5GB | Per-app virtual environments | +| Available | 1GB | Headroom | + +**Watch out for:** +- ONNX runtime loading (NNKinematics): ~200MB for model + runtime +- Large HuggingFace datasets: Downloaded to disk, loaded incrementally +- Audio buffers: SoundDevice ring buffer capped at 60 seconds (~2MB @ 16kHz stereo) + +### Recording Memory + +Motion recording stores one frame per control loop tick (50Hz): + +``` +Per frame: ~200 bytes (7 joints + 2 antennas + 4x4 pose + timestamp) +1 minute: ~600KB +10 minutes: ~6MB +``` + +This is bounded by the in-memory list. For very long recordings, consider streaming to disk. + +--- + +## Network Bandwidth Budget + +### Zenoh (Localhost) + +| Channel | Direction | Size per message | Frequency | Bandwidth | +|---------|-----------|-----------------|-----------|-----------| +| joint_positions | Server→Client | ~200B JSON | 50Hz | ~10 KB/s | +| head_pose | Server→Client | ~300B JSON | 50Hz | ~15 KB/s | +| daemon_status | Server→Client | ~500B JSON | 1Hz | ~0.5 KB/s | +| commands | Client→Server | ~200B JSON | Varies | <10 KB/s | +| **Total** | | | | **~35 KB/s** | + +Negligible on localhost. Over WiFi, adds <1ms latency typically. + +### WebRTC (Wireless Remote) + +| Stream | Direction | Bandwidth | +|--------|-----------|-----------| +| Video (H.264) | Robot→Laptop | 2-5 Mbps | +| Audio (Opus) | Robot→Laptop | 64 kbps | +| Audio (Opus) | Laptop→Robot | 64 kbps | +| **Total** | | **2-5 Mbps** | + +Requires stable WiFi. Packet loss >5% will cause visible artifacts. + +### MuJoCo Rendering (Simulation) + +| Stream | Direction | Bandwidth | +|--------|-----------|-----------| +| JPEG frames (UDP) | Daemon→Client | ~2 Mbps @ 25Hz | + +Localhost only. No network traversal. + +--- + +## Thermal Budget + +### Continuous Operation + +| Condition | Duration | Risk | +|-----------|----------|------| +| Active motion (moderate) | Up to 2 hours | Low -- motors share load | +| High-torque hold (static) | Up to 30 minutes | Medium -- single motor may overheat | +| Gravity compensation | Up to 1 hour | Medium -- continuous current draw | +| Motors disabled (idle) | Unlimited | None -- no power draw | + +**Thermal protection:** Dynamixel motors have built-in thermal shutdown. If triggered: +1. Motor stops responding +2. Hardware error flag set (Overheating) +3. Power off, wait 5+ minutes, power on + +**Prevention:** +- Disable motors during idle periods +- Use gravity compensation instead of position hold for recording sessions +- Avoid sending targets far from current position repeatedly (causes high torque) + +### Battery (Wireless) + +| Activity | Approximate Runtime | +|----------|-------------------| +| Active (motors + compute + WiFi) | 1-2 hours | +| Idle (daemon running, motors disabled) | 3-4 hours | + +No precise battery percentage available. Rely on LED indicator (green → orange → red). + +--- + +## Performance Monitoring Checklist + +To verify the system is within budget: + +```python +with ReachyMini() as mini: + status = mini.client.get_status() + stats = status["backend_status"]["control_loop_stats"] + + freq = stats["mean_control_loop_frequency"] + max_dt = stats["max_control_loop_interval"] + errors = stats["nb_error"] + + assert freq > 45, f"Control loop too slow: {freq:.1f} Hz" + assert max_dt < 0.025, f"Max interval too high: {max_dt*1000:.1f} ms" + assert errors == 0, f"Communication errors: {errors}" +``` + +Run this periodically during development to catch performance regressions early. + +--- + +## When Budget is Exceeded + +| Symptom | Likely cause | Fix | +|---------|-------------|-----| +| Control loop < 45Hz | Heavy IK (Placo) or CPU contention | Switch to AnalyticalKinematics | +| Max interval > 50ms | GC pause or blocking I/O in loop | Profile, reduce allocations | +| Motor errors > 0 | Serial timeout from overrun | Reduce control loop work | +| App sluggish on Wireless | CM4 out of CPU | Offload AI to laptop | +| WebRTC video stutters | WiFi bandwidth or latency | Move closer to router, reduce resolution | +| Memory >3.5GB on CM4 | App or model too large | Reduce model size, stream data | diff --git a/docs/advanced/HELP_STATE_MODEL.md b/docs/advanced/HELP_STATE_MODEL.md new file mode 100644 index 000000000..aae96749a --- /dev/null +++ b/docs/advanced/HELP_STATE_MODEL.md @@ -0,0 +1,224 @@ +# Reachy Mini -- State Model + +Make behavior predictable. Most robotics bugs are illegal state transitions, not math errors. + +--- + +## State Machines Overview + +Reachy Mini has three independent state machines that operate concurrently: + +1. **Daemon State** -- Is the daemon running? +2. **Motor Control Mode** -- Are motors stiff, limp, or compliant? +3. **App State** -- Is a user app running? + +--- + +## 1. Daemon State + +``` + start() + NOT_INITIALIZED ──────────► STARTING + │ + ┌───────┴────────┐ + │ backend.ready │ backend fails + │ within 2s │ or timeout + ▼ ▼ + RUNNING ERROR + │ │ + stop() │ │ restart() + ▼ │ + STOPPING ◄───────────┘ + │ + ┌───────┴────────┐ + │ thread joins │ timeout or + │ within 5s │ exception + ▼ ▼ + STOPPED ERROR +``` + +**Enum: `DaemonState`** +``` +NOT_INITIALIZED -- Created but not started +STARTING -- Backend initializing, waiting for ready event +RUNNING -- Control loop active, accepting commands +STOPPING -- Shutdown in progress (optional goto_sleep) +STOPPED -- Clean shutdown complete +ERROR -- Unrecoverable failure +``` + +**Transition rules:** +- Only `start()` can move from NOT_INITIALIZED or STOPPED to STARTING +- Only the backend ready event (within 2s timeout) moves STARTING to RUNNING +- `stop()` moves RUNNING to STOPPING, then STOPPED (or ERROR on timeout) +- `restart()` calls `stop()` then `start()` sequentially +- Any unhandled exception in the backend thread transitions to ERROR + +**Who can trigger transitions:** +- REST API: `/api/daemon/start`, `/api/daemon/stop`, `/api/daemon/restart` +- Daemon lifespan (FastAPI startup/shutdown) +- Backend thread crash (automatic ERROR transition) + +--- + +## 2. Motor Control Mode + +``` + enable_motors() + DISABLED ◄──────────────────────► ENABLED + │ │ + │ disable_motors() │ + │◄─────────────────────────────────┤ + │ │ + │ enable_gravity_ │ + │ compensation() │ + └──────────────────► GRAVITY_COMPENSATION + │ + disable_ │ + motors() │ + ◄──────────────┘ +``` + +**Enum: `MotorControlMode`** +``` +Enabled -- Torque ON, position control (operating mode 3) +Disabled -- Torque OFF, motors limp +GravityCompensation -- Torque ON, current control (operating mode 0) + Requires Placo kinematics +``` + +**Transition details:** + +| From | To | What happens internally | +|------|----|------------------------| +| Disabled → Enabled | `enable_motors()` | Set goal to current position, then enable torque | +| Enabled → Disabled | `disable_motors()` | Remove torque from all motors | +| Disabled → GravityComp | `enable_gravity_compensation()` | Switch to torque mode (op mode 0), enable torque, start gravity torque loop | +| GravityComp → Enabled | `enable_motors()` | Disable torque, switch to position mode (op mode 3), set goal to current, re-enable | +| GravityComp → Disabled | `disable_motors()` | Remove torque | +| Enabled → GravityComp | `enable_gravity_compensation()` | Disable torque, switch to torque mode, re-enable | + +**Constraints:** +- GravityCompensation requires PlacoKinematics. Raises `RuntimeError` with AnalyticalKinematics or NNKinematics. +- Motor mode transitions are serialized (one at a time via backend method calls). + +**Who can trigger transitions:** +- SDK: `mini.enable_motors()`, `mini.disable_motors()`, `mini.enable_gravity_compensation()` +- REST: `POST /api/motors/set_mode/{mode}` +- Zenoh: Command with `"torque": true/false` or `"gravity_compensation": true` +- WebRTC: Data channel command `set_motor_mode` + +--- + +## 3. App State + +``` + start_app() + (no app) ──────────────────► STARTING + │ + ┌────────┴────────┐ + │ app runs │ app crashes + │ successfully │ on startup + ▼ ▼ + RUNNING ERROR + │ + ┌────────┴────────┐ + │ app finishes │ stop_current_app() + │ normally │ (SIGINT, 20s timeout) + ▼ ▼ + DONE STOPPING + │ + ┌────────┴────────┐ + │ process exits │ force kill + ▼ ▼ + DONE DONE +``` + +**Enum: `AppState`** +``` +STARTING -- Subprocess launched, waiting for first output +RUNNING -- App producing output, no errors detected +ERROR -- App crashed or reported error +STOPPING -- SIGINT sent, waiting for graceful exit +DONE -- Process exited (success or after forced kill) +``` + +**Constraints:** +- Only one app can run at a time. `start_app()` fails if `is_app_running()` returns true. +- `set_target` commands from the SDK are blocked while a move is running (not while an app is running -- apps use the SDK internally). + +**Post-app recovery:** +When an app stops, the daemon returns the robot to `INIT_HEAD_POSE` with antennas at [0, 0] over 1 second. + +--- + +## 4. Move Execution State + +This is not a formal state machine but a critical guard: + +``` + IDLE (no move running) + │ + │ goto_target() or play_move() + │ acquires _play_move_lock + ▼ + MOVE_RUNNING + │ + │ - set_target commands are BLOCKED + │ - new goto_target/play_move calls FAIL silently + │ - control loop continues at 50Hz + │ + │ move.evaluate(t) reaches duration + │ releases _play_move_lock + ▼ + IDLE +``` + +**Guard mechanism:** `_play_move_lock` (RLock) with `_active_move_depth` counter. + +- `_try_start_move()`: Non-blocking acquire. Returns False if locked by another thread. +- `is_move_running`: Property checking `_active_move_depth > 0`. +- ZenohServer `_handle_command()`: Logs warning and drops command if `is_move_running`. + +--- + +## What Happens on Failure + +### Backend thread crashes +1. Exception stored in `backend.error` +2. `backend.close()` called for cleanup +3. Daemon state transitions to ERROR +4. Status published with error message +5. SDK `send_command()` raises `ConnectionError` (heartbeat fails within 1s) + +### Motor communication lost +1. `RobotBackend._update()` detects no response for 1+ second +2. Sets error: `"No response from the robot's motor for the last second."` +3. Raises `RuntimeError`, causing backend thread to exit +4. Daemon transitions to ERROR + +### IK computation fails +1. `update_target_head_joints_from_ik()` catches `ValueError` +2. Warning logged (throttled to 0.5s interval) +3. Previous valid target positions retained +4. Control loop continues -- does NOT crash + +### App subprocess crashes +1. Monitor task detects non-zero exit code +2. Last 10 stderr lines captured +3. App state transitions to ERROR +4. Robot returned to init pose via recovery procedure +5. Daemon continues running normally + +--- + +## State Queries + +| What you want to know | How to check | +|------------------------|-------------| +| Is daemon running? | `GET /api/daemon/status` → `state == "running"` | +| Are motors enabled? | `GET /api/motors/status` → `mode` | +| Is a move executing? | `GET /api/move/running` → list of UUIDs | +| Is an app running? | `GET /api/apps/current-app-status` → `state` | +| Is the backend ready? | `GET /api/daemon/status` → `backend_status.ready` | +| Last control loop frequency? | `GET /api/daemon/status` → `backend_status.control_loop_stats` | diff --git a/docs/advanced/HELP_TESTING_STRATEGY.md b/docs/advanced/HELP_TESTING_STRATEGY.md new file mode 100644 index 000000000..4d0948011 --- /dev/null +++ b/docs/advanced/HELP_TESTING_STRATEGY.md @@ -0,0 +1,226 @@ +# Reachy Mini -- Testing Strategy + +Increase capability without fear. Know what to test and how. + +--- + +## Test Pyramid + +``` + ┌─────────┐ + │ Hardware │ Requires physical robot + │ In Loop │ (marked tests, manual) + ─┤ ├─ + / └─────────┘ \ + / \ + ┌──────────────────┐ + │ Integration │ Daemon + backend + SDK + │ (test_daemon) │ MuJoCo sim, no hardware + ─┤ ├─ + / └──────────────────┘ \ + / \ + ┌──────────────────────────┐ + │ Unit Tests │ Kinematics, collision, + │ (test_analytical_kin, │ imports, app lifecycle + │ test_collision, etc.) │ + └──────────────────────────┘ +``` + +--- + +## Running Tests + +### Standard CI command (no hardware required) + +```bash +MUJOCO_GL=disable pytest -vv \ + -m 'not audio and not video and not audio_gstreamer and not video_gstreamer and not wireless and not wireless_gstreamer' \ + --tb=short +``` + +This runs all tests except those requiring physical hardware. Set `MUJOCO_GL=disable` when running without a display (CI, SSH, headless). + +### Run a specific test file + +```bash +pytest tests/test_analytical_kinematics.py -vv +``` + +### Run tests matching a pattern + +```bash +pytest -k "daemon" -vv +``` + +--- + +## Test Markers + +Tests that require specific hardware or system capabilities are marked. CI excludes all of these. + +| Marker | What it requires | Example test | +|--------|-----------------|-------------| +| `audio` | Audio hardware (speaker + mic) | `test_audio.py` | +| `audio_gstreamer` | GStreamer audio pipeline | `test_audio.py` | +| `video` | Camera hardware | `test_video.py` | +| `video_gstreamer` | GStreamer video pipeline | `test_video.py` | +| `wireless` | Wireless Reachy Mini connected | `test_wireless.py` | +| `wireless_gstreamer` | GStreamer on Wireless version | `test_wireless.py` | + +To run hardware tests (when connected to a robot): + +```bash +pytest -m "audio" -vv # Audio tests only +pytest -m "wireless" -vv # Wireless tests only +``` + +--- + +## Test Categories + +### Import Tests (`test_import.py`) + +**What they verify:** Package structure, all public modules importable. + +```python +def test_import(): + from reachy_mini import ReachyMini + from reachy_mini.utils import create_head_pose + # ... verifies core imports work +``` + +**When they fail:** Broken imports, missing dependencies, circular imports. + +### Kinematics Tests (`test_analytical_kinematics.py`) + +**What they verify:** IK-FK round-trip accuracy. + +```python +def test_analytical_kinematics(): + # IK: identity pose → joint angles + joints = kin.ik(np.eye(4)) + # FK: joint angles → recovered pose + recovered = kin.fk(joints, no_iterations=10) + # Must match within tolerance + np.testing.assert_allclose(recovered, np.eye(4), atol=1e-2) + +def test_analytical_kinematics_with_yaw(): + # Same but with body_yaw = π/4 + # Verifies head-body coupling +``` + +**Tolerance:** `atol=1e-2` (~0.57 degrees). This is generous to account for numerical precision. + +**When they fail:** Kinematics engine bug, geometry data file corruption, tolerance regression. + +### Collision Tests (`test_collision.py`) + +**What they verify:** Placo kinematics rejects unreachable poses. + +```python +def test_collision(): + # Reachable pose → IK returns solution + assert kin.ik(create_head_pose()) is not None + + # Unreachable pose → IK returns None + assert kin.ik(create_head_pose(x=20, y=20, mm=True)) is None +``` + +**Requires:** `placo_kinematics` extra installed. + +### Daemon Tests (`test_daemon.py`) + +**What they verify:** Full daemon lifecycle with MuJoCo simulation. + +```python +async def test_daemon_start_stop(): + daemon = Daemon() + await daemon.start(sim=True, headless=True, wake_up_on_start=False) + # Daemon is RUNNING, backend is ready + await daemon.stop(goto_sleep_on_stop=False) + # Daemon is STOPPED + +async def test_daemon_client_disconnection(): + # Start daemon, connect client, stop daemon + # Verify client detects disconnection +``` + +**Pattern:** Uses `asyncio` with `pytest-asyncio`. Runs MuJoCo in headless mode. + +### App Tests (`test_app.py`) + +**What they verify:** App lifecycle (install, start, stop, remove). + +```python +def test_app_manager(): + # Install ok_app → verify listed + # Start ok_app → verify RUNNING + # Wait for completion → verify DONE + # Remove ok_app → verify gone + +def test_faulty_app(): + # Install faulty_app → start → verify ERROR state +``` + +**Test fixtures:** `tests/ok_app/` (working app) and `tests/faulty_app/` (crashes on purpose). + +--- + +## What to Test for New Features + +### New kinematics engine +1. **IK-FK round-trip** for neutral pose, tilted pose, and extreme yaw +2. **Safety limits** enforced (pitch/roll clamped to [-40, +40]) +3. **Yaw delta constraint** (max 65 degrees head-body difference) +4. **Unreachable pose** returns None or raises ValueError +5. **Timing** -- IK completes within 20ms budget + +### New motion primitive +1. **Boundary values** -- `evaluate(0)` matches start, `evaluate(duration)` matches end +2. **Continuity** -- no jumps between consecutive evaluations +3. **Duration property** is finite and positive +4. **None handling** -- returns None for uncontrolled components + +### New backend +1. **Start/stop lifecycle** -- clean startup, graceful shutdown +2. **State publishing** -- joint_positions and head_pose published at expected frequency +3. **Motor mode transitions** -- enable → disable → gravity compensation +4. **Move execution** -- goto_target and play_move work through the backend + +### New API endpoint +1. **Happy path** -- returns expected response +2. **Invalid input** -- returns 400 with descriptive error +3. **Backend not running** -- returns 503 or appropriate error +4. **Concurrent access** -- no race conditions + +--- + +## CI Pipeline + +| Workflow | Trigger | What runs | +|----------|---------|-----------| +| `pytest.yml` | PR touching `src/`, `tests/`, `pyproject.toml` | Full test suite (Ubuntu + macOS, Python 3.10), excludes hardware markers | +| `lint.yml` | Push/PR touching `src/`, `tests/`, `pyproject.toml` | Ruff v0.12.0 + MyPy strict mode | + +### Local pre-commit checks + +```bash +# Install hooks (one-time) +pre-commit install + +# Run manually +pre-commit run --all-files +``` + +This runs `ruff-check` and `ruff-format`. Code that fails pre-commit will fail CI. + +--- + +## Testing Discipline + +1. **Every IK/FK change needs a round-trip test.** If you modify kinematics math, prove it with a test. +2. **Every new motion type needs boundary tests.** Verify start, end, and mid-point behavior. +3. **New REST endpoints need at least a happy-path test.** Use FastAPI's `TestClient`. +4. **Run the full test suite before pushing.** Don't rely solely on CI. +5. **Mark hardware-dependent tests.** Use `@pytest.mark.audio`, `@pytest.mark.video`, etc. +6. **Tests must be deterministic.** No random seeds, no timing-dependent assertions (use generous timeouts). diff --git a/docs/advanced/HELP_TIMING_AND_REALTIME.md b/docs/advanced/HELP_TIMING_AND_REALTIME.md new file mode 100644 index 000000000..e0a54835e --- /dev/null +++ b/docs/advanced/HELP_TIMING_AND_REALTIME.md @@ -0,0 +1,232 @@ +# Reachy Mini -- Timing and Real-Time Constraints + +Protect determinism. Know what must be fast and what can be slow. + +--- + +## Timing Budget Overview + +``` + HARD REAL-TIME + ┌──────────────────────────────┐ + │ Backend Control Loop │ + │ Period: 20ms (50Hz) │ + │ Budget: <20ms per iteration │ + │ │ + │ Read joints: ~2ms │ + │ FK compute: ~1ms │ + │ IK compute: ~1ms (analyt.) │ + │ Write targets: ~0.2ms │ + │ Publish state: ~0.5ms │ + │ Margin: ~15ms │ + └──────────────────────────────┘ + + SOFT REAL-TIME + ┌──────────────────────────────┐ + │ Motion Playback │ + │ Period: 10ms (100Hz) │ + │ Runs as asyncio task │ + │ Jitter: up to ~5ms OK │ + │ │ + │ Evaluate trajectory: <1ms │ + │ Set targets: <0.1ms │ + │ asyncio.sleep: remainder │ + └──────────────────────────────┘ + + BEST EFFORT + ┌──────────────────────────────┐ + │ REST API, Dashboard, Apps │ + │ WebSocket state streaming │ + │ App lifecycle management │ + │ Dataset downloading │ + │ Audio playback callbacks │ + └──────────────────────────────┘ +``` + +--- + +## Control Loop (50Hz -- Hard Real-Time) + +**Location:** `Backend.run()` in each backend subclass + +**What happens every 20ms:** + +| Step | Duration | Notes | +|------|----------|-------| +| Read joint positions from hardware | ~2ms | Serial communication | +| Update FK (kinematics model) | ~1ms | Analytical default | +| Compute IK if target changed | ~1ms | Only when `ik_required=True` | +| Write target positions to motors | ~0.2ms | Serial write | +| Publish joint_positions via Zenoh | ~0.5ms | JSON serialization + pub | +| Publish head_pose via Zenoh | ~0.5ms | Same | +| Hardware error check | ~0.1ms | Only every 1 second | +| **Total** | **~5ms** | **15ms margin** | + +**Timing mechanism (RobotBackend):** +```python +period = 1.0 / 50.0 # 20ms + +while not should_stop.is_set(): + start = time.time() + _update() + took = time.time() - start + sleep_time = max(period - took, 0.001) + next_call_event.clear() + next_call_event.wait(sleep_time) # Interruptible, cross-platform +``` + +If `_update()` takes longer than 20ms, a warning is logged and the loop runs as fast as possible (minimum 1ms sleep). + +**What makes it deterministic:** +- Runs in a dedicated `threading.Thread` (daemon thread) +- No asyncio involvement -- pure threading + blocking I/O +- Motor controller uses `multiprocessing.Event.wait()` for precise timing +- No GC pressure: minimal allocations per iteration + +--- + +## Motion Playback (100Hz -- Soft Real-Time) + +**Location:** `Backend.play_move()` + +```python +async def play_move(move, play_frequency=100.0): + t0 = time.time() + sleep_period = 1.0 / play_frequency # 10ms + + while time.time() - t0 < move.duration: + t = time.time() - t0 + head, antennas, body_yaw = move.evaluate(t) + # Set targets (picked up by control loop at next 50Hz tick) + elapsed = time.time() - t0 - t + await asyncio.sleep(max(sleep_period - elapsed, 0.001)) +``` + +**Why 100Hz, not 50Hz?** +The motion playback runs at 2x the control loop frequency to ensure no target update is missed. The control loop reads the latest target on each tick. + +**What can cause jitter:** +- asyncio event loop congestion (other coroutines running) +- GIL contention from backend thread +- Heavy REST API requests during playback +- Acceptable: up to ~5ms jitter (smoothed by motor PID) + +--- + +## IK Engine Latency Comparison + +| Engine | IK Latency | FK Latency | Notes | +|--------|-----------|-----------|-------| +| AnalyticalKinematics (Rust) | ~1ms | ~1ms | Default. Fits easily in 20ms budget. | +| NNKinematics (ONNX) | ~5ms | ~5ms | Fits in budget with less margin. | +| PlacoKinematics (Python) | ~50ms | ~50ms | **Exceeds budget.** Used only with special care. | + +When PlacoKinematics is selected, the control loop will overrun its 20ms budget on every IK computation. This is mitigated by only recomputing IK when `ik_required=True` (target changed). + +--- + +## Publishing Frequencies + +| Data | Frequency | Publisher | Consumer | +|------|-----------|-----------|----------| +| Joint positions | 50Hz | Backend → ZenohServer | SDK client, dashboard | +| Head pose | 50Hz | Backend → ZenohServer | SDK client, dashboard | +| IMU data | 50Hz | Backend → ZenohServer | SDK client (Wireless) | +| Daemon status | 1Hz | Daemon orchestrator | SDK client, dashboard | +| Task progress | On completion | ZenohServer | SDK client | +| Recorded data | On stop_recording | Backend | SDK client | +| WebSocket state | Configurable (default 10Hz) | State router | Dashboard, web clients | + +--- + +## Threading Model + +``` +┌─────────────────────────────────────────────────────────┐ +│ PROCESS: reachy-mini-daemon │ +│ │ +│ Main Thread (asyncio event loop) │ +│ ├── FastAPI request handlers │ +│ ├── WebSocket connections │ +│ ├── App subprocess monitoring (asyncio.Task) │ +│ ├── Motion playback (asyncio.Task via play_move) │ +│ ├── Dataset updater (asyncio.Task) │ +│ └── Health check timeout (asyncio.Task) │ +│ │ +│ Backend Thread (daemon thread) │ +│ └── Control loop @ 50Hz (_update method) │ +│ ├── Serial I/O to motor controller │ +│ ├── FK/IK computation │ +│ └── Zenoh publishing │ +│ │ +│ Status Publisher Thread (daemon thread) │ +│ └── Publishes daemon_status every 1s │ +│ │ +│ MuJoCo Rendering Thread (daemon thread, sim only) │ +│ └── UDP frame streaming @ 25Hz │ +│ │ +│ Zenoh Background Threads (managed by zenoh library) │ +│ └── Subscriber callbacks │ +│ │ +│ APP SUBPROCESS (separate process, isolated) │ +│ └── User's ReachyMiniApp.run() │ +│ └── User's control loop (set_target @ 50-100Hz) │ +└─────────────────────────────────────────────────────────┘ +``` + +**Critical synchronization points:** + +| Lock | Type | Protects | Contention | +|------|------|----------|------------| +| `_play_move_lock` | RLock | Move execution serialization | asyncio tasks vs Zenoh commands | +| `_rec_lock` | Lock | Recording buffer swap | Backend thread vs API calls | +| `ZenohServer._lock` | Lock | Command processing | Zenoh subscriber callback vs API | +| `busy_lock` (daemon router) | Lock | Daemon start/stop operations | Concurrent REST requests | + +--- + +## User-Side Control Loop Guidelines + +When building apps with `set_target()`: + +| Frequency | Quality | CPU Impact | +|-----------|---------|------------| +| 100Hz (10ms) | Excellent. Matches motion playback. | Low | +| 50Hz (20ms) | Good. Matches control loop. | Very low | +| 30Hz (33ms) | Minimum acceptable. May appear slightly jerky. | Negligible | +| <30Hz | Visibly jerky. Not recommended. | - | +| >100Hz | Diminishing returns. Control loop is 50Hz anyway. | Wasteful | + +**Recommended pattern:** +```python +import time + +while not stop_event.is_set(): + target = compute_target() + mini.set_target(head=target) + time.sleep(0.01) # 100Hz +``` + +Using `time.sleep()` is sufficient. The motor PID controller smooths any jitter between user updates and the 50Hz hardware loop. + +--- + +## Where Latency Matters Most + +1. **Control loop period (20ms):** Overruns degrade motion quality. Sustained overruns cause "No response from motor" errors. +2. **IK computation:** Must fit within the control loop budget. AnalyticalKinematics (~1ms) is the safe default. +3. **Zenoh message delivery:** Typically <1ms on localhost. Network mode adds ~1-5ms. +4. **Audio playback latency:** SoundDevice callback-based, ~100ms end-to-end. Not suitable for real-time audio effects. +5. **Camera frame latency:** OpenCV: ~33ms@30fps. GStreamer: lower. WebRTC: +50-100ms network latency. + +--- + +## What Contributors Must Not Break + +1. **Never add blocking I/O to the backend `_update()` method.** This is the 50Hz loop. File I/O, network requests, or heavy computation here will cause motor communication timeouts. + +2. **Never call `set_target()` from the backend thread.** Targets are set by the asyncio event loop (via Zenoh commands or move playback). The backend thread reads them. + +3. **Never use `time.sleep()` in asyncio coroutines.** Use `await asyncio.sleep()`. Blocking sleep in a coroutine blocks the entire event loop. + +4. **Never allocate large objects in the control loop.** Numpy array creation, JSON parsing, and string formatting all trigger GC. Pre-allocate buffers.