Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,13 @@ had changed in them - which is the gap this file exists to close.

### Added

- **Debug a node's firmware while the run goes on.** `node.debug`, and
"Debug this node" on a node's row menu, say how to attach gdb or VS Code to
a running node. A native node gets its process, binary and whether it has
symbols, and a breakpoint now holds it rather than getting it marked down:
the rest of the mesh carries on, and it rejoins when let go. An emulated
board is restarted with QEMU's or Renode's GDB stub, optionally held at
reset so boot can be stepped through. See `docs/debugging-firmware.md`.
- **Script input at an instant of simulated time.** `console.type`,
`console.cli` and the `companion.*` verbs that write to a node take `at_ms`:
the call is held and runs when the run reaches that instant, so a script
Expand Down
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ lives here and **is** maintained - unlike the working notes below.
| `install.md` | per platform, including the macOS and Windows signing caveats |
| `firmware-build-settings.md` | what each setting in a build's own window does, and what it costs |
| `native-and-emulated.md` | where the real firmware ends and the simulation begins, and the two ways a node can run |
| `debugging-firmware.md` | attaching gdb or an IDE to a running node, native or emulated |

## Machine-specific, and deliberately so

Expand Down
71 changes: 71 additions & 0 deletions docs/debugging-firmware.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Debugging a node's firmware

`node.debug` makes a running node debuggable and says how to attach. The
firmware is real, so the debugger is the ordinary one: gdb, lldb, or VS Code
with the C/C++ extension.

## Native nodes

A native node is an ordinary process. Ask for it:

```
node.debug {"node": "West Lomond"}
```

The answer has the process ID, the binary, whether the binary has symbols, a
gdb command line and a VS Code `attach` configuration:

```
gdb -p 2104750 ~/.cache/meshbench/firmware/native/.../meshcore-simple_repeater-linux-amd64
```

A native node normally has to answer every tick within 30 seconds or it is
marked down. Once `node.debug` has been called it is held instead. A
breakpoint stops it, the rest of the mesh carries on, and when the debugger
lets it go it catches up to the run's clock and rejoins. Its clock jumps by
however long it was stopped, so `sim.state` reports the run as not
reproducible from then on.

- `symbols` is `debug` when the binary carries DWARF, `symbols` when it has a
symbol table only, and `stripped` when it has neither. For source-level
debugging, build MeshCore from a checkout with `meshbench dev`.
- On Linux, attaching to a process that is not the debugger's own child needs
ptrace permission: `kernel.yama.ptrace_scope` set to 0, or the debugger run
as root.
- A native node runs `setup()` as soon as it starts, so a fault during boot
cannot be caught by attaching afterwards.

`node.debug {"node": ..., "off": true}` puts the deadline back.

## Emulated boards

An emulated board's firmware runs inside QEMU or Renode, and the way in is the
emulator's GDB stub. `node.debug` restarts the board with one:

```
node.debug {"node": "Deck", "wait": true}
```

- `gdb_port` is where the stub listens. Pass `port` to choose it.
- `wait` holds the CPU at reset until a debugger continues, so boot can be
stepped through.
- `debugger` is the gdb that speaks the board's architecture:
`xtensa-esp32-elf-gdb` or `xtensa-esp32s3-elf-gdb` for the ESP32 boards
under QEMU, `arm-none-eabi-gdb` (or `gdb-multiarch`) for the nRF52 boards
under Renode.

The image a board runs is a flashable `.bin` or `.uf2` with no symbols. Point
the debugger at the `firmware.elf` the same build produced, for a PlatformIO
build `.pio/build/<env>/firmware.elf`:

```
arm-none-eabi-gdb -ex 'target remote 127.0.0.1:3333' .pio/build/RAK_4631_repeater/firmware.elf
```

The answer includes a VS Code configuration for this, with
`miDebuggerServerAddress` set to the stub.

QEMU's stub listens on loopback only. Renode's listens on every interface, so
only ask for one on a machine you trust the network around.

`node.debug {"node": ..., "off": true}` restarts the board without the stub.
2 changes: 1 addition & 1 deletion docs/scripting-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ and a verb added tomorrow is usable today.

### Why two layers rather than one

**260**<!--verbdoc:total--> addressable methods hand-written across two
**261**<!--verbdoc:total--> addressable methods hand-written across two
languages is twice that many things to keep in step. Keeping them in step is what the verb manifest (`docs/verbs.json`) is for: a
hand-written surface that names a verb the tree no longer has is exactly the
drift it catches, where once a tool could call a deleted verb and nothing
Expand Down
9 changes: 5 additions & 4 deletions docs/scripting-verbs.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Every verb, and the call that covers it

The wire underneath [scripting-api.md](scripting-api.md). The store registers
**222**<!--verbdoc:public--> public verbs a script can call and
**38**<!--verbdoc:internal--> internal callbacks it cannot, **260**<!--verbdoc:total-->
**223**<!--verbdoc:public--> public verbs a script can call and
**38**<!--verbdoc:internal--> internal callbacks it cannot, **261**<!--verbdoc:total-->
in total, plus the two the socket answers itself, with what each reads, what
each returns, and which façade call reaches it.

Expand Down Expand Up @@ -77,8 +77,8 @@ Two verbs are **not** in this table:

## What this table shows about the surface

- **222**<!--verbdoc:public--> public verbs, plus **38**<!--verbdoc:internal-->
the workbench keeps to itself, **260**<!--verbdoc:total--> registered in
- **223**<!--verbdoc:public--> public verbs, plus **38**<!--verbdoc:internal-->
the workbench keeps to itself, **261**<!--verbdoc:total--> registered in
total. The façade covers the public verbs, over roughly 60 calls once
objects and properties absorb them.
- **The naming is not regular.** `node.*` and `nodes.*` are both node verbs and
Expand Down Expand Up @@ -131,6 +131,7 @@ Two verbs are **not** in this table:
| `node.antenna` | *a bare string*, `node` string | `node`, `pattern`, `gain_dbi_peak`, `beamwidth_deg`, `front_to_back_db`, `bearing_deg`, `downtilt_deg`, `polarisation`, `feedline_db`, `peak_dbi` | `node.antenna` |
| `node.boardview` 🪟 | *a bare string*, `node` string, `tab` string | `node`, `board`, `tab` | `wb.board_view(node)` |
| `node.card` | *a bare string*, `node` string, `fitted` bool, `file` string, `wipe` bool | `node`, `slot`, `fitted`, `file`, `own_file`, `bytes`, `required_by_firmware`, `board_has_slot`, `wiped` | `node.card(fitted=\|file=\|wipe=)` |
| `node.debug` | *a bare string*, `node` string, `wait` bool, `port` number, `off` bool | `node`, `debugging`, `backend`, `pid`, `binary`, `symbols`, `gdb_port`, `wait`, `debugger`, `image`, `attach`, `vscode`, `note` | `node.debug(wait=, port=, off=)` |
| `node.energy` | *a bare string*, `node` string | `node` | `node.energy()` |
| `node.output` | *a bare string*, `node` string, `source` string, `lines` number | `node`, `source`, `lines`, `total`, `path`, `tail`, `note`, `tracing` | `node.output(source)` |
| `node.output_window` 🪟 | *a bare string*, `node` string, `source` string | `node`, `source` | `node.output_window(source)` |
Expand Down
29 changes: 27 additions & 2 deletions docs/verb-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
Generated. Run `tools/verbdoc/verbdoc.py` to rewrite it and
`tools/verbdoc/verbdoc.py --check` to fail when it is stale.

The store registers 260 verbs: 222 a script may call and
The store registers 261 verbs: 223 a script may call and
38 the workbench calls on itself, which the socket refuses. Of those,
260 say what they are for and 0 do not yet; the ones that
261 say what they are for and 0 do not yet; the ones that
do not are marked, and what is printed for them is read out of the handler
rather than said by it.

Expand Down Expand Up @@ -490,6 +490,31 @@ Report or change what is in one node's card slot: whether a card is fitted, whic

**Client** `node.card(fitted=|file=|wipe=)`

### `node.debug`

Make a running node debuggable and say how to attach: a native node is held rather than marked down while a breakpoint stops it, and an emulated board is restarted with its emulator's GDB stub.

**Takes**

| parameter | type | | what |
|---|---|---|---|
| `node` | string | required, primary | the node to debug; refused when absent, unknown, or not running firmware |
| `wait` | bool | optional | for an emulated board, hold its CPU at reset until a debugger continues, so boot can be stepped through; refused for a native node, which has already booted by the time anything can attach |
| `port` | number | optional | for an emulated board, the port its GDB stub listens on; absent picks a free one |
| `off` | bool | optional | stop debugging: a native node is given its tick deadline back, an emulated board is restarted without the stub |

**Answers** `node`, `debugging`, `backend`, `pid`, `binary`, `symbols`, `gdb_port`, `wait`, `debugger`, `image`, `attach`, `vscode`, `note`. `attach` is a debugger command line and `vscode` a launch configuration to paste. A native node has `pid`, `binary` and `symbols`: `debug` with DWARF, `symbols` with a symbol table only, `stripped` with neither. An emulated board has `gdb_port`, `wait`, the `debugger` for its architecture and the `image` it runs, which has no symbols: the debugger wants the firmware.elf the same build produced. From the first call, `sim.state` reports the run as not reproducible, because a debugger stopping a node moves its clock.

**Example** - attach gdb to a native repeater

```json
{"id":1,"method":"node.debug","params":{"node":"West Lomond"}}
```

Not made by the test suite: this call needs more than the two-node headless session the runnable examples go to.

**Client** `node.debug(wait=, port=, off=)`

### `node.energy`

Run a year of sun and battery at one named node, at the duty cycle the run measured for it rather than one typed into a form, for the node window that has a name and no selection to work from.
Expand Down
49 changes: 49 additions & 0 deletions docs/verbs.json
Original file line number Diff line number Diff line change
Expand Up @@ -2953,6 +2953,55 @@
"runnable": true
}
},
"node.debug": {
"what": "make a running node debuggable and say how to attach: a native node is held rather than marked down while a breakpoint stops it, and an emulated board is restarted with its emulator's GDB stub",
"params": [
{
"name": "node",
"type": "string",
"required": true,
"primary": true,
"what": "the node to debug; refused when absent, unknown, or not running firmware"
},
{
"name": "wait",
"type": "bool",
"what": "for an emulated board, hold its CPU at reset until a debugger continues, so boot can be stepped through; refused for a native node, which has already booted by the time anything can attach"
},
{
"name": "port",
"type": "number",
"what": "for an emulated board, the port its GDB stub listens on; absent picks a free one"
},
{
"name": "off",
"type": "bool",
"what": "stop debugging: a native node is given its tick deadline back, an emulated board is restarted without the stub"
}
],
"returns": [
"node",
"debugging",
"backend",
"pid",
"binary",
"symbols",
"gdb_port",
"wait",
"debugger",
"image",
"attach",
"vscode",
"note"
],
"answers": "`attach` is a debugger command line and `vscode` a launch configuration to paste. A native node has `pid`, `binary` and `symbols`: `debug` with DWARF, `symbols` with a symbol table only, `stripped` with neither. An emulated board has `gdb_port`, `wait`, the `debugger` for its architecture and the `image` it runs, which has no symbols: the debugger wants the firmware.elf the same build produced. From the first call, `sim.state` reports the run as not reproducible, because a debugger stopping a node moves its clock.",
"example": {
"params": {
"node": "West Lomond"
},
"what": "attach gdb to a native repeater"
}
},
"node.energy": {
"what": "run a year of sun and battery at one named node, at the duty cycle the run measured for it rather than one typed into a form, for the node window that has a name and no selection to work from",
"params": [
Expand Down
Loading