Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions docs/tools/ai-agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,59 @@ Ethos Lua script for FrSky radios. Source in `src/<folder>/`, deployed to `scrip

The same agent can deploy to a USB-connected radio with [`ethos_deploy.py`](vscode.md#7-the-deploy-tool) and read the radio's `print()` output over serial (`--radio --debug-only --for 30` stops after 30 s, so a non-interactive agent isn't left waiting).

### Driving a real radio

Recent firmware answers the same **remote control protocol** as the simulator on its USB serial port, so an agent can operate a physical radio the way it operates the simulator: screenshot, read it, press a key or tap, repeat. `ethos-tools` has a client for it, [`simulation/radio_remote.py`](https://github.com/FrSkyRC/ethos-tools/blob/main/simulation/radio_remote.md), which works like `run_wasm.js --serve`: one process holds the connection, and each command is a one-shot call that prints JSON.

| Requirement | Notes |
| --- | --- |
| **Ethos 26 nightly from 2026-10-16 or later** | Earlier firmware has no remote control over USB |
| **Python 3.8+** | `pip install pyserial hidapi` (`hidapi` only for `mode`) |
| **The radio in Serial USB mode** | Choose *Serial* when plugging in, or switch over HID with `mode serial` (the [`81 68` request](vscode.md#4-talking-to-the-radio-over-usb)) |

```bash
python radio_remote.py mode serial # radio drops off the bus, COM/tty port appears
python radio_remote.py --serve & # finds the port by VID/PID, then waits...
# ...for you to ACCEPT the connection prompt on the radio screen (60 s, --accept-timeout)
python radio_remote.py status # board, firmware, protocol version
python radio_remote.py press SYS
python radio_remote.py screenshot shots/system.png
python radio_remote.py quit
```

| Command | Effect |
| --- | --- |
| `screenshot [out.png] [main\|top\|glasses]` | Save a display as PNG (800×480 on an X20S) |
| `press KEY [holdMs]` | `SYS` `MDL` `DISP` `PAGE` `RTN` `ENTER` `UP` `DOWN` `LEFT` `RIGHT`; trims are keys too: `T1+` ... `T6-` |
| `tap X Y`, `longpress X Y`, `swipe X1 Y1 X2 Y2`, `wheel N` | Touch and rotary encoder |
| `ls PATH`, `read PATH [out]`, `write SRC PATH`, `rm PATH` | The radio's storage (`/scripts`, `/models`, ...), without switching to storage mode |
| `trims`, `load PATH`, `text STRING`, `fswitch I` | Trim values, load a model, type on the on-screen keyboard, press a function switch |

The server listens on port 8766, so it can run next to a `run_wasm.js` simulator on 8765.

What differs from the simulator:

- **Someone has to accept each connection on the radio.** Before that, every command except `HELLO` is refused. A new serial connection asks again, which is why one server process keeps the port open. An agent can't do this step, so start the server while you're at the radio.
- **No sticks, pots or switches.** The protocol defines commands for them, but FrSky's web simulator doesn't send them to a physical radio, and they haven't been tested on one. A radio bound to a powered model would move its servos or motor, so don't try them with RF on.
- **No macros or model JSON.** Those are simulator-only functions.
- **Deploy over the protocol.** `write` copies a script to `/scripts/<folder>/` while the radio stays in serial mode. Scripts still load at boot, so restart the radio before testing the new version.

*Verified on an X20S running Ethos 26.1.3 (nightly, protocol version 6) on Windows: connect and accept, screenshots, keys, trims, tap, rotary encoder, and listing, reading, writing and deleting files. On Windows, open the port with overlapped I/O as pyserial does. A synchronous handle (Node's `fs.open('\\\\.\\COM6')`, for example) gets `HELLO` through, then later commands time out.*

The protocol itself (frame format, the version 6 payloads, tiled screenshots, connection events) is documented in [`radio_remote.md`](https://github.com/FrSkyRC/ethos-tools/blob/main/simulation/radio_remote.md#protocol-notes-for-a-physical-radio).

### Through the web simulator instead

The [web simulator](web-simulator.md) can connect to a radio by Web Serial (Chrome or Edge) and has a **Connect Claude** dialog. It gives an MCP server URL that FrSky's server relays to the open browser tab:

```bash
claude mcp add --transport http ethos-simulator https://ethos-simulator.frsky-rc.com/mcp/<token>
```

With a radio connected to that tab, its `screenshot`, `pressKey`, `touch` and `swipe` tools reach the radio. `runMacro`, `getModelJson` and `applyModelJson` only work with the simulator. It needs no install, but the browser tab has to stay open and the traffic goes through FrSky's server. `radio_remote.py` is local only and also gives file access.

*This option was read from the web simulator's code (October 2026), not tested.*

!!! warning "`ELECTRON_RUN_AS_NODE` breaks FrSky Suite's CLI"
Agents running inside VS Code inherit `ELECTRON_RUN_AS_NODE=1`. With it set, `FrSky Suite.exe --get-path SCRIPTS` runs as plain Node.js and fails with `bad option`. Unset the variable before calling Suite (details on [FrSky Suite](frsky-suite.md#command-line)), or use `ethos_deploy.py`, which talks to the radio over USB HID and doesn't need Suite.

Expand Down
2 changes: 1 addition & 1 deletion docs/tools/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ You can develop most of a script without touching a radio. Pick the environment
| [FrSky Suite](frsky-suite.md) | FrSky's desktop app | Simulator with a real folder on disk, installing scripts on a radio, backups |
| [VS Code project template](vscode-template.md) | VS Code, the Ethos extension, Python | Ready-made project: ▶ deploy to simulator or USB radio, live `print()` output, i18n, release zips from git tags |
| [VS Code deployments](vscode.md) | | How that tooling works: USB HID, drive discovery, safe copying |
| [`ethos-tools` + an AI agent](ai-agents.md) | Node.js + a clone or Claude Code plugin | Letting Claude (or another agent) boot a radio, deploy your script, screenshot and drive it, and read its errors |
| [`ethos-tools` + an AI agent](ai-agents.md) | Node.js + a clone or Claude Code plugin | Letting Claude (or another agent) boot a radio, deploy your script, screenshot and drive it, and read its errors. Also drives a [real radio over USB](ai-agents.md#driving-a-real-radio) |

Whatever you use, read [Debugging](debugging.md) for how to see `print` output and Lua errors.

Expand Down
2 changes: 1 addition & 1 deletion docs/tools/web-simulator.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,4 +78,4 @@ Two habits:
- Before relying on a new API, check its **since** version on its [API page](../api/index.md), then load an older release in the simulator to confirm your fallback works.

!!! note "Command-line and AI use"
The same WebAssembly builds can run headless from a terminal with [`ethos-tools`](ai-agents.md), which also lets an AI agent drive the radio.
The same WebAssembly builds can run headless from a terminal with [`ethos-tools`](ai-agents.md), which also lets an AI agent drive the radio. The web simulator can also connect to a real radio by Web Serial and pair with Claude through its **Connect Claude** dialog: see [Through the web simulator instead](ai-agents.md#through-the-web-simulator-instead).
Loading