diff --git a/.claude/skills/openpocket-flash/SKILL.md b/.claude/skills/openpocket-flash/SKILL.md new file mode 100644 index 0000000..16a8dc4 --- /dev/null +++ b/.claude/skills/openpocket-flash/SKILL.md @@ -0,0 +1,100 @@ +--- +name: openpocket-flash +description: Build, dry-run and flash an OpenPocket package to a connected Donner Pocket X or M-VAVE BlackBox from the command line (node tools/op.mjs flash, over USB MIDI, no browser). Use to check that a built package fits the connected pedal, or, only when the captain explicitly instructs it, to write it. Covers the safety rules and the SD-card recovery. +--- + +# Flash a pedal from the command line + +`node tools/op.mjs flash` (or `make flash ARGS="..."`) runs the same update protocol as the browser installer +(`docs/index.html`): the code is `docs/pxota.js` and `docs/pxpkg.js`, shared with the page; `tools/flash.mjs` only adds the +command line. Run everything from the repository root. The command runs on the host, not in the container: the pedal is a USB +device. + +## Safety rules (read first) + +1. **Never flash without the captain's explicit instruction** for this pedal and this package. A task that says "build" or + "test" is not that instruction. Without it, stop after the dry run and report. +2. **Always dry-run first**, and read its output: pedal, model and package must be the ones intended. +3. Never put `--confirm-flash` in a script, alias or default. It is typed for one flash, on the captain's say-so. +4. Keep the pedal plugged in and powered until the command prints `done`. Close the browser installer and any other MIDI + program first; they hold the port. +5. Only one pedal on the bus. The command refuses when none or several answer. +6. A failure prints the recovery pointer. Do not retry in a loop: report the output to the captain. + +## 1. Set up once + +``` +node tools/op.mjs doctor # Node 18+ (Docker only for building) +npm ci --prefix tools # the one dependency, @julusian/midi (prebuilt RtMidi), into tools/node_modules +``` + +On Linux the MIDI library needs ALSA (`libasound2`); the pedal's USB MIDI port must be visible to your user. + +## 2. Build the package + +``` +node tools/op.mjs package # docs/firmware/PocketX_9NN.fwsc, BlackBox_9NN.fwsc, manifest.json, build/SHA256SUMS +``` + +Details and failures: the `openpocket-toolchain` skill. To flash a package built elsewhere (for example a test image from +another branch), keep its `.fwsc` next to the `manifest.json` that was built with it and pass both. + +## 3. Dry run (writes nothing) + +``` +node tools/op.mjs flash --dry-run +node tools/op.mjs flash --dry-run --package build-x/PocketX_925.fwsc --manifest build-x/manifest.json +``` + +It checks the package against the manifest (size, SHA-256, identity, UFW and CRC structure), finds the pedal by asking each +MIDI input for its identity (a read-only query), and refuses a package for the other model. The pedal's flash is not touched. +Success ends with `dry run ok: would update PocketX_923 -> PocketX_924`. + +Refusals (exit 2, nothing sent to the pedal): no `--confirm-flash` and no `--dry-run`; package not listed in the manifest; +SHA-256, size or identity differs from the manifest; wrong model; no pedal; several pedals; pedal already in update mode +(power-cycle it). + +## 4. Flash (only on the captain's explicit instruction) + +``` +node tools/op.mjs flash --confirm-flash [--package FILE] [--manifest FILE] +``` + +Same checks as the dry run, then: the pedal stages its update loader (step 1), the loader writes the app area (step 2), the +pedal restarts, and the command reads the version it reports. `done: the pedal reports PocketX_924.` is success (exit 0); a +version that differs from the package is a failure. It takes about a minute. Report the full output to the captain. + +## 4b. A pedal that restarts in a loop (only on the captain's explicit instruction) + +``` +node tools/op.mjs flash --rescue --confirm-flash [--package FILE] [--manifest FILE] [--timeout SECONDS] +make flash-rescue ARGS="--confirm-flash" +``` + +The installer page's rescue mode, on the command line. Start it first, then plug the pedal in or let it restart. It polls every +MIDI port every 100 ms; the moment the pedal answers its identity query it sends the upgrade command (which stops the sound +processing, so the restarts stop) and installs the package of the pedal's model. `--package` pins one package: a pedal of +the other model is refused before anything is sent. Without `--package` the manifest's package for the pedal's model is used. +It waits `--timeout` seconds (default 180); an install already running is never cut short. A pedal that vanishes before the +update finished has had nothing written: it is waited for again. `--rescue --dry-run` waits for the pedal and checks the model, +writing nothing. Reached states: the running firmware's MIDI port, and an OpenPocket update loader left half-done (finished only with `--package`: a loader does not report its model; a foreign loader is refused). Not reached +(same as the page): a firmware that crashes before it answers, and the chip's mask-ROM loader `WL80UBOOT` (USB 4C4A:8057, no +MIDI); both need the SD-card recovery. Timeout is exit 1 with the recovery text. One pedal on the bus: the first to answer is used. + +## 5. Recovery + +A failure (exit 1) prints `FAILED: ...` and the recovery text. In order: + +1. Unplug USB, power-cycle the pedal, run `--dry-run` again to see what it reports. A pedal in update mode (`ota-...`) is + finished by the browser installer's resume button, or by this command after a power-cycle. +2. If it stays dead, the SD-card recovery in the README (`flash/`: fm1buddy on an ESP32-S3; the image comes from your factory + backup with `make appimage DUMP=fm1-1.bin` or `makeimage.bat`). Only the captain does this. + +Never publish or commit a pedal backup (`fm1-1.bin`, `fm1-300.bin`): they hold the official firmware. + +## Tests + +`node tests/test_flash.mjs` (part of `node tools/op.mjs test`) drives the real command against the simulated pedal that runs +the pedal's own `ota.c` and loader code: update, dry run writes nothing, wrong model, damaged package, missing confirmation, +no pedal, two pedals, failure with the recovery text, and `--rescue` against a pedal that restarts in a loop (answers on the +Nth restart, never answers, vanishes mid-transfer, wrong model, missing confirmation). Real hardware is not covered by any test. diff --git a/.claude/skills/openpocket-rescue/SKILL.md b/.claude/skills/openpocket-rescue/SKILL.md new file mode 100644 index 0000000..b86d027 --- /dev/null +++ b/.claude/skills/openpocket-rescue/SKILL.md @@ -0,0 +1,68 @@ +--- +name: openpocket-rescue +description: Rescue a Donner Pocket X / M-VAVE BlackBox that restarts in a loop or no longer boots, from a Mac with no extra hardware (make rescue, tools/uboot_rescue.py, over the chip's USB UBOOT boot state). Use when the installer's rescue mode cannot finish, the pedal loops after a bad image, or a flash left the app area broken. Covers when to use it, the safety rules, who runs the sudo write, and how to read the result. +--- + +# Rescue a looping or bricked pedal + +The pedal's mask ROM always enters the USB boot state ("UBOOT": USB `4C4A:8057`, shown as `WL80UBOOT1.00`) when the app is bad, +and it cannot be damaged. `tools/uboot_rescue.py` waits for that state, loads JieLi's RAM loader and rewrites the app area from +the official package. Full walkthrough and the record of the first real run: `docs/RESCUE.md`. + +## When to use it + +- The pedal restarts in a loop and the installer page's rescue mode (or `node tools/op.mjs flash`) cannot finish. Try those first: + they need no root. +- The pedal stays in the boot state (no loop, no sound) after an interrupted flash: the rescue finishes the job. +- Not for Bluetooth or firmware behaviour questions, and not on a pedal that works. + +## Rules (read first) + +1. **Read-only first.** `make rescue` writes nothing to the flash (the loader goes to RAM only) and saves a backup of the whole flash + to `~/OpenPocket-rescue`. Read its output before anything else. +2. **The captain runs the write.** `make rescue-write` and every `sudo` command need the captain's password and instruction. An + agent runs neither: prepare the exact command and give it with the `!` prefix. +3. The write range is only whole 4 KiB sectors in `[0x4000, end of the package)`, `[0x4000, 0xA3000)` for BlackBox. Never write + the head `[0, 0x4000)` or anything above the package. The tool refuses; do not work around it. +4. Never chip erase (`FB02`) and never burn a key (`FC12`). The tool cannot send them; do not add them. +5. Stop on any `REFUSED` (wrong device, chip key not `0x980F`, flash not 1 MiB, head differs from the package). Nothing was written; + report the message. +6. Keep the first backup. Never commit or publish a backup or a package: they hold the official firmware. +7. After a restore, do not reinstall the Bluetooth test image until the cause of the loop is known. + +## How + +``` +brew install libusb # once +make doctor # rescue lines: libusb, official package, loader, root +make rescue # read-only: INQUIRY, loader to RAM, key, flash id, backup, head check, sectors that differ +make rescue-write # only on the captain's instruction, after the read-only run passed +``` + +Options go after `ARGS=` (`make rescue ARGS="--package FILE.fwsc"`; `--image flash.bin`, `--loader FILE`, `--backup-dir DIR`, +`--attempts N`). Plug the pedal straight into the Mac (no hub). Start the command first; it waits for the pedal and starts by +itself the moment it appears (the pedal is in UBOOT about 44 s of every 70 s while it loops). Needs the official BlackBox package +(default `~/Documents/BlackBox (2).fwsc`) and Python 3. + +## Reading the result + +| Output | Meaning | +| --- | --- | +| `INQUIRY: WL82 UBOOT1.00 1.00` | the right device | +| `loader running: chip key 0x980f, flash id ... (1024 KiB)` | the checks passed; record the flash id | +| `backup saved: ...dump.bin` | the rollback copy is complete on disk | +| `head [0, 0x4000) matches the image` | the package fits this pedal | +| `N of 160 sectors ... differ` | what a write would do; a rerun after a partial write shows fewer | +| `Read-only run finished, nothing was written` | exit 0, the read-only run is done | +| `Done: N sectors written and verified` | exit 0 after `--write`; power-cycle the pedal | +| `the pedal went away ... waiting for it again` | the window closed; it resumes by itself | +| exit 1 `ERROR` | setup: libusb, `sudo`, missing package; the message names the fix | +| exit 2 `REFUSED` | a safety check stopped it, nothing written | +| `LIBUSB_ERROR_ACCESS` | run with `sudo` (`make rescue` does); if it persists as root, report it | + +Save the output of a real run (`make rescue 2>&1 | tee rescue.log`) and add its numbers to `docs/RESCUE.md`. + +## Tests + +`make rescue-check` runs the tool against a simulated UBOOT pedal (protocol, guards, resume, no hardware); it is part of +`make test`. Hardware behaviour is not covered by any test. diff --git a/.claude/skills/openpocket-simulator/SKILL.md b/.claude/skills/openpocket-simulator/SKILL.md new file mode 100644 index 0000000..d4f7b2c --- /dev/null +++ b/.claude/skills/openpocket-simulator/SKILL.md @@ -0,0 +1,74 @@ +--- +name: openpocket-simulator +description: Start and drive the hosted OpenPocket simulated pedal (node tools/op.mjs sim, make sim): the firmware's real screen, editor protocol and preset store running natively behind a local HTTP API, with a code-drawn front panel of the pedal as its page. Use to press or hold footswitches and buttons, turn the knobs, read the screen as a PNG, read the pedal state, send SysEx, dump or reset the flash, or to use the installer and editor pages against a simulated pedal in a browser. Use for verifying UI, editor-protocol or preset-store changes without a pedal, and for trying a preset or a screen. Use to hear a preset through bundled guitar loops, a file or a computer input. Not for USB or Bluetooth (not simulated). +--- + +# OpenPocket simulator + +The pedal's own `ui.h`, `px_editor.h` and `px_store.h` run natively (`tests/edsim.c`) behind HTTP. Full reference: +`docs/SIMULATOR.md` (every endpoint, the state fields, what is not simulated). Run from the repository root. Needs Node 18+ +and a C compiler (`node tools/op.mjs doctor` says whether `cc` is there); no Docker. + +## Start + +``` +node tools/op.mjs sim --port 0 --reset & # prints: OpenPocket simulator (pedal) listening on http://127.0.0.1:PORT/ +``` + +`--port 0` picks a free port: read the URL from the first line of output. Without `--port` it is 8766. The flash lives in +`build/sim/flash.bin` (`--flash FILE`); without `--reset` a session continues where it stopped. The first start compiles +the simulator (a few seconds). Stop it with Ctrl-C or by killing the process **you** started. + +## Drive (with `U=http://127.0.0.1:PORT`) + +| Do | Request | +| --- | --- | +| A footswitch or button | `curl -X POST $U/api/press/NAME`: `fs_up` `fs_dn` `fx` `home` `push` `mod` `amp` `cab` `dly` `rev`; `push_long` (SAVE), `dpush` (MOVE), `home_long` (tuner) | +| Hold a button | `curl -X POST $U/api/down/NAME`, later `$U/api/up/NAME`: the same ten names, timed as the firmware does (HOME 0.7 s, PUSH 0.8 s, double click 350 ms) | +| Turn the knob | `curl -X POST $U/api/knob/3` (negative: other way; `?fast` for steps of 5) | +| Volume, battery, tuner pitch | `POST $U/api/volume/55`, `/api/battery/3640[/USB_MV]`, `/api/tuner/110` | +| Bypass, load in the status text | `POST $U/api/bypass/1`, `/api/load/17` | +| Read the state | `curl -s $U/api/state` (slot, name, mode, focused block, the whole sound, `frame`, `held`, `knob`) | +| Read the screen | `curl -s "$U/api/screen.png?scale=2" -o screen.png`, then look at it; `screen.rgb565` is the raw pixels | +| SysEx | `curl -s -X POST $U/api/sysex -d 'f0 7d 4f 50 01 f7'` returns `{"reply":"hex"}` (`docs/EDITOR-PROTOCOL.md`) | +| Dump / reset the flash | `curl -s $U/api/flash -o flash.bin`; `curl -X POST $U/api/reset` | + +Every request is applied before it is answered. The pedal's pages: `chain` (PLAY) -> `block` -> `edit`; `push_long` opens +`save` (knob picks the slot, `push` saves); `home_long` opens the tuner, any control leaves it. A worked path to change a +setting: `press/amp`, `knob/2`, `press/push`, `knob/8`, then `state` shows `preset.p[5][0]` up by 8 and `dirty: 1`. +Prove a screen claim by reading `screen.png`, not only the state. + +## Browser + +`$U/` is the pedal's front panel (`tools/sim/pedal.html`) with the live screen in its LCD: press and hold the buttons and +footswitches, drag or wheel the knobs, click PARA for PUSH, or use the keys listed on the page. Scripts on the page have +`window.pedal` (`down`, `up`, `tap`, `turn`, `master`, `state`); a control is `[data-control=NAME]`, `#master` or `#para`. +`$U/plain` is the plain page with the screen, every control as a button, the battery, the tuner pitch and the state. +`$U/site/index.html` (installer) and `$U/site/editor.html` +(editor) are `docs/` with a virtual Web MIDI injected, so Connect finds the simulated pedal in any browser; drive them with +`chrome-devtools-axi` or Playwright. For the installer use the update flow instead of the pedal: +`node tools/op.mjs sim --update FLASH.bin` (an image such as `flash_old.bin` from `python3 tests/fixtures.py DIR`). + +## Sound + +The skin's Sound section runs the real engine (`engine/px_dsp.c`, built to `tools/sim/px_engine.wasm`) in an AudioWorklet: +pick a bundled loop (`assets/loops/`), a dropped file or a device, press Play (the first click starts the audio; a device +starts muted), and the preset on the screen is what you hear. `window.pedal.audio()` reports it (`running`, `heard`, +`muted`, ...), and `#in-meter`, `#out-meter` carry `data-db`. Without a browser, `node tools/sim/render.mjs OUTDIR [preset]` +renders the loops to WAV. After any change to `engine/`, run `node tools/op.mjs wasm` (Docker) and commit the new binary: +`tests/test_audio.mjs` fails otherwise. `docs/SIMULATOR.md` ("The sound") has the reasons and limits. + +## Limits (say so when verifying) + +No LEDs or bypass footswitch (the firmware has none: the skin's backlights are +its own reading of the state), no USB or Bluetooth. `press/NAME` has no timing, `down` and `up` do. The panel's TUNER line +is the stock firmware's: in OpenPocket both footswitches are two presses, and the tuner is a long press of HOME. +`press()` and the `button_*` functions in `tests/edsim.c` mirror the control dispatch of `firmware/openpocket.c`: if that +loop changes, change them too. The proof is the firmware's real code on a PC, not the pedal. + +## Tests + +`node tests/test_sim.mjs` (also in `node tools/op.mjs test`) covers the API, the flash file and the shim. +`node tests/test_pedal_input.mjs` covers the skin's input mapping, and `node tests/test_pedal_page.mjs OUTDIR` drives the skin +in Chromium and saves screenshots (needs Playwright). `node tests/test_audio.mjs` covers the engine and the loops, and +`node tests/test_audio_page.mjs OUTDIR` the Sound section in Chromium with a fake audio input (needs Playwright). diff --git a/.claude/skills/openpocket-toolchain/SKILL.md b/.claude/skills/openpocket-toolchain/SKILL.md index 0e989f1..9c708ac 100644 --- a/.claude/skills/openpocket-toolchain/SKILL.md +++ b/.claude/skills/openpocket-toolchain/SKILL.md @@ -24,8 +24,10 @@ Run commands from the repository root. | Build the firmware, the update loader and the installer packages | `node tools/op.mjs build` | | Build, then write `build/SHA256SUMS` | `node tools/op.mjs package` | | Host tests (engine, preset store, screen, simulated pedal, installer, editor) | `node tools/op.mjs test` | +| The simulated pedal on a local port (host `cc`, no Docker; skill `openpocket-simulator`) | `node tools/op.mjs sim` | | Remove every build output (needs no Docker) | `node tools/op.mjs clean` | | A shell in the container | `node tools/op.mjs shell` | +| Dry-run or flash a built package to a connected pedal (host, USB MIDI) | `node tools/op.mjs flash`: see the `openpocket-flash` skill | Outputs: `firmware/openpocket.bin`, `docs/firmware/PocketX_9NN.fwsc`, `BlackBox_9NN.fwsc`, `manifest.json`. All of them are gitignored. Two clean builds give identical bytes; a changed checksum means the source or a pin changed. @@ -50,8 +52,9 @@ Follow `references/updating-pins.md`. Never bypass the checksum check. ## Safety rules of this repository -- **Never flash a device from an agent.** No `flash`, no WebUSB or MIDI write to a real pedal. Only the captain installs, - from the browser installer, by hand. Hardware checks are in `docs/MIDI-TEST.md` when present. +- **Never flash a device unless the captain explicitly instructs it.** `node tools/op.mjs flash --dry-run` is allowed; `--confirm-flash` + and any other write to a real pedal only on the captain's say-so, following the `openpocket-flash` skill. Builds and tests never + touch a pedal. Hardware checks are in `docs/MIDI-TEST.md` when present. - **Never publish an unreviewed package as the installer default.** A test package is a separate, labelled file and is not listed in the default `docs/firmware/manifest.json`. `PUBLISH-*.bat` is the maintainer's tool: do not run it. - **Keep the official-firmware analysis out of the repository.** Facts about the vendor's firmware (offsets, tables, diff --git a/.claude/skills/verify-pedal/SKILL.md b/.claude/skills/verify-pedal/SKILL.md new file mode 100644 index 0000000..b109aa3 --- /dev/null +++ b/.claude/skills/verify-pedal/SKILL.md @@ -0,0 +1,73 @@ +--- +name: verify-pedal +description: Verify an OpenPocket build on the real, USB-connected pedal without a person reading its screen: flash the Bluetooth test image, wait for boot, read boot state, last-boot report and radio counters over USB MIDI (node tools/op.mjs log), run the Mac Bluetooth scan, and report pass or fail with the reason in words. Use after a firmware/ble or boot change when a pedal is attached. For checks without a pedal, use the verify skill. +--- + +# Verify on the real pedal + +`node tools/op.mjs log` asks the pedal for the text its screen shows (the `DIAG` request, read only) and judges it. Only the +Bluetooth test image answers `DIAG`; the release firmware does not. The pedal is one shared device: one driver at a time. + +## Launch + +The pedal must run the build under test. Follow the `openpocket-flash` skill: its rules on authority apply, and +`--confirm-flash` never goes in a script. One flash per attempt, no sudo. + +``` +npm ci --prefix tools # once +node tools/op.mjs ble # build/ble/*.fwsc, manifest.json +node tools/op.mjs flash --dry-run --manifest build/ble/manifest.json +node tools/op.mjs flash --confirm-flash --manifest build/ble/manifest.json # ends with "done: the pedal reports ..." +``` + +Ready means the log says stage `OK`, about 7 s after start. `--wait` below waits for that. A pedal that restarts again +and again is a crash loop: stop, do not flash again, and report (see `features/boot-failure.md`). + +## Doctor + +`node tools/op.mjs log --count 1` is read only and says how far the pedal answers. Exit 2 means the MIDI library is missing. +Check first that no other agent is flashing: `ps ax | grep -E 'op.mjs (flash|log)'`. + +## Drive + +``` +node tools/op.mjs log --wait 40 --scan 30 # the whole check: two readings, the scan, a third reading, the verdict +node tools/op.mjs log --wait 40 # state only, no scan +node tools/op.mjs log --follow --interval 2 # watch it; add --json for one object per reading +``` + +| Verdict | Exit | Meaning | +| --- | --- | --- | +| `PASS` | 0 | the Mac saw the advertisement | +| `RUNNING` | 0 | radio events rise, no scan was run: not proof of packets on the air | +| `FAIL` | 1 | the reason says which step, or that nothing was seen | +| `PENDING`, `UNKNOWN` | 3 | not decided: still starting, scanner unusable, or less could be read (below) | + +Report the verdict line as it stands. Field meanings: `docs/BLE-TEST.md`, and printed by the command. + +## What it cannot read + +| Pedal state | What the log gives | Fallback | +| --- | --- | --- | +| red `CRASH` screen, or the 12 s `LAST BOOT FAILED` report | no port, or a silent port, while it shows | read again after it: the next boot gives the same report as `last boot`; a screen photo meanwhile | +| main loop stuck, firmware failed before USB service | port there, nothing answers | screen photo; power cycle or the UBOOT rescue | +| boot loader, update mode | identity only, or no port | finish the update; `openpocket-flash` skill | +| release firmware, or a test image without `DIAG` | version and DSP load, says the rest is missing | screen photo | +| stock firmware | identity only | nothing to read | + +The log never shows sound, footswitches or the drawing on the screen. + +## Evidence + +Save the output with `tee` to a folder outside the repository, with the package SHA-256 from the flash log. Proof standards: +the flash log, the `log --wait 40 --scan 30` output and the verdict, from the real pedal. The scan listens only; it never +connects or pairs. `RUNNING` is not `PASS`. `r` stays 0 with a Mac scan, so it says nothing here. A boot-failure claim needs +the `last boot` line from the real pedal, not the simulator. + +## Cleanup + +Nothing stays running; stop `--follow` with Ctrl-C. Restore the release firmware only when the task says so: +`node tools/op.mjs flash --confirm-flash` after `node tools/op.mjs package`. `build/ble-scan` is rebuilt when missing. +Evidence stays in the folder you chose. + +Features: `features/README.md`. Keep them true with `/maintain-verification-skill`. diff --git a/.claude/skills/verify-pedal/features/README.md b/.claude/skills/verify-pedal/features/README.md new file mode 100644 index 0000000..cc5af42 --- /dev/null +++ b/.claude/skills/verify-pedal/features/README.md @@ -0,0 +1,8 @@ +# Pedal feature map + +| Feature | File | Proof | +| --- | --- | --- | +| Log channel | `log-channel.md` | parsed fields from the real pedal, rising counters | +| Flash and boot | `flash-and-boot.md` | `done: the pedal reports ...`, then stage `OK` | +| Bluetooth advertising | `ble-advertising.md` | verdict `PASS` | +| Boot failure report | `boot-failure.md` | `last boot FAILED` with the stage and the reset or crash | diff --git a/.claude/skills/verify-pedal/features/ble-advertising.md b/.claude/skills/verify-pedal/features/ble-advertising.md new file mode 100644 index 0000000..83b36a3 --- /dev/null +++ b/.claude/skills/verify-pedal/features/ble-advertising.md @@ -0,0 +1,26 @@ +# Bluetooth advertising + +The test image sends `OpenPocket` with the BLE-MIDI service UUID, at 100 ms, connectable. + +## Sub-features + +- Controller start-up: `STARTING n`, `FAILED s c`, `ADV ...`. +- Radio counters: `i` about 10 a second is the advertising rate; `t`, `s`, and `V` (stored radio records) are reported as notes. +- The Mac scan: `tools/ble-scan.swift`, built into `build/ble-scan` on first use. + +## How to get to it (user POV) + +Pedal running the test image, within a few metres of the Mac, Bluetooth on. + +## Driving it with tools/op.mjs + +`node tools/op.mjs log --wait 40 --scan 30`: 30 s filtered on `03B80E5A-EDE8-4B33-A751-6CE34EC4C700`, then 30 s unfiltered, +matching the name or the UUID. Proof: `verdict PASS`, with the name and RSSI. + +## Gotchas + +- `others` above 0 is what shows the scanner works when the target is not seen; a scanner with no authorization gives `UNKNOWN`. +- The first run may ask macOS for Bluetooth permission for the terminal. +- The name is in the scan response, the UUID in the advertising data: a sighting may carry only one. +- `ADV` only means the controller accepted the command. +- The pedal stops advertising while it has a link, so a scan then sees nothing. Read the `link` line first: `LINK 1` means a central holds it, and `i` then counts connection events (16.7 a second at the Chocolate's 60 ms), not advertising. diff --git a/.claude/skills/verify-pedal/features/boot-failure.md b/.claude/skills/verify-pedal/features/boot-failure.md new file mode 100644 index 0000000..a9408db --- /dev/null +++ b/.claude/skills/verify-pedal/features/boot-failure.md @@ -0,0 +1,22 @@ +# Boot failure report + +The `LAST BOOT FAILED` screen, readable from the computer on the next boot. + +## Sub-features + +- `last boot` line: stage, reset cause, audio counters; after a crash, PC, RETS, vector, uptime. +- Safe mode: after a failed boot Bluetooth stays off for one boot (`this boot ... SAFE MODE`). + +## How to get to it (user POV) + +Nothing to do: it is there on the first boot that gets USB up after the failure. + +## Driving it with tools/op.mjs + +`node tools/op.mjs log --wait 60`. Look the addresses up in `firmware/build-ble/symbols.txt` of the same build. + +## Gotchas + +- Not yet seen on hardware: only the simulator has produced this line (`tests/test_log.mjs`). Do not break a pedal to test it. +- While the report or the red crash screen is up, USB is down: `--wait` rides that out; the crash itself cannot be streamed. +- Only one previous boot is kept. If the safe-mode boot fails as well, the line describes that boot. diff --git a/.claude/skills/verify-pedal/features/flash-and-boot.md b/.claude/skills/verify-pedal/features/flash-and-boot.md new file mode 100644 index 0000000..8292665 --- /dev/null +++ b/.claude/skills/verify-pedal/features/flash-and-boot.md @@ -0,0 +1,23 @@ +# Flash and boot + +Write the test image, and see it start. + +## Sub-features + +- Flash over USB MIDI (`openpocket-flash` skill). +- Boot stages and the boot guard: `this boot` line (`STAGE`, `GUARD`, reset cause). + +## How to get to it (user POV) + +Pedal running OpenPocket, plugged in, no installer page open. + +## Driving it with tools/op.mjs + +Dry run, flash, then `node tools/op.mjs log --wait 40`. Proof: `done: the pedal reports BlackBox_936`, then +`this boot pedal clock N s (wraps every 179 s), stage OK, normal`. + +## Gotchas + +- The version stays `v36` between test builds: only the package SHA-256 tells them apart. +- The boot after a flash shows reset by `POWER-ON`. +- The stage is `FIRST 6 S` until 6 s after start; the boot guard clears then. diff --git a/.claude/skills/verify-pedal/features/log-channel.md b/.claude/skills/verify-pedal/features/log-channel.md new file mode 100644 index 0000000..93585d9 --- /dev/null +++ b/.claude/skills/verify-pedal/features/log-channel.md @@ -0,0 +1,24 @@ +# Log channel + +The pedal's state as text over USB MIDI: this boot, the last boot, the status line the screen scrolls. + +## Sub-features + +- `DIAG`, command 23 (`docs/EDITOR-PROTOCOL.md`), read only; `firmware/px_editor.h`, `firmware/ble/ble_diag.h`. +- `tools/log.mjs`: parsing, meanings, verdict. It falls back to `INFO` and `PING`, then to the identity query. + +## How to get to it (user POV) + +Plug the pedal in over USB and run the command. No screen, no browser. + +## Driving it with tools/op.mjs + +`node tools/op.mjs log --count 2 --interval 2`: `i` rises between the readings. +`--json` gives the same as objects, with `raw` holding each line as the pedal sent it. + +## Gotchas + +- The pedal clock in `this boot` wraps every 179 s, so it is not the uptime after that; the event rate uses this computer's clock. +- Do not run it during a flash: both talk to the same port. +- The line is the screen's own text (`ble_test_line`), so a new field shows in `raw` before the parser knows it. +- `tests/test_log.mjs` runs this against the real editor code; it says nothing about USB on hardware. diff --git a/.claude/skills/verify/SKILL.md b/.claude/skills/verify/SKILL.md index 80c449c..49527d6 100644 --- a/.claude/skills/verify/SKILL.md +++ b/.claude/skills/verify/SKILL.md @@ -1,17 +1,19 @@ --- name: verify -description: Verify OpenPocket (guitar multi-effects firmware for the Donner Pocket X / M-VAVE BlackBox) without a pedal: build the firmware and installer packages in the pinned container, run the host tests and simulators, and drive the web installer (docs/index.html) and editor (docs/editor.html) in headless Chromium against simulated pedals. Use after any change to firmware/, engine/, loader/, tools/, tests/ or docs/*.html|js, and before claiming a change works. Only a person with a real pedal can verify sound and USB behaviour on hardware. +description: Verify OpenPocket (guitar multi-effects firmware for the Donner Pocket X / M-VAVE BlackBox) without a pedal: build the firmware and installer packages in the pinned container, run the host tests and simulators, and drive the web installer (docs/index.html) and editor (docs/editor.html) in headless Chromium against simulated pedals. Use after any change to firmware/, engine/, loader/, tools/, tests/ or docs/*.html|js, and before claiming a change works. For a connected pedal, use the verify-pedal skill; only a person can verify sound. --- # Verify OpenPocket No user session exists to drive: the product is firmware that runs on a pedal, plus two static web pages. "Driving it as a user" means (1) building what a user installs, (2) running the pedal's own code on the PC in simulators, and (3) clicking -through the installer and editor pages with a mocked Web MIDI device. Never flash or contact a real device from here. +through the installer and editor pages with a mocked Web MIDI device. Never flash or contact a real device from here: that +is the `verify-pedal` skill. ## Launch -Nothing stays running except short-lived test servers, which the scripts start and stop themselves. +Nothing stays running except short-lived test servers, which the scripts start and stop themselves, and the hosted +simulator when you start it (`node tools/op.mjs sim`, skill `openpocket-simulator`): stop the one you started. ``` node tools/op.mjs doctor # Node, Docker, image (toolchain skill: .claude/skills/openpocket-toolchain) @@ -37,6 +39,10 @@ Run it first whenever anything looks off. | Installer page, update flow | `pages.sh` (runs `tests/test_page.mjs`) | `build/page/page_1..4*.png`, last line `page test passed` | | Editor page | `pages.sh` (runs `tests/test_editor_page.mjs`) | `build/editor/editor_1..6*.png`, last line `editor page test passed` | | Screen layout | `cc ... tests/uisim.c` (see toolchain skill), `build/uisim build/ui` | PPM pages, `never drawn` absent from the log | +| The pedal as a user plays it: footswitches, knob, pages, save, tuner, the live screen | `node tools/op.mjs sim`, then the HTTP API (skill `openpocket-simulator`, `docs/SIMULATOR.md`) | `GET /api/state` and `/api/screen.png` after each press; the flash file `build/sim/flash.bin` survives a restart | +| The pedal skin: every control by mouse, touch and keyboard, the backlights, the live LCD | `pages.sh` (runs `tests/test_pedal_page.mjs`) | `build/pedal/pedal-skin*.png`, last line `pedal page: ok` | +| The sound in the simulator: the engine on known signals, against its native build; the input picker, meters, bypass, the preset heard | `node tests/test_audio.mjs`, `pages.sh` (runs `tests/test_audio_page.mjs`) | every line `ok`; `build/audio/sound.png`; `node tools/sim/render.mjs DIR` renders WAV files to listen to | +| Editor and installer pages by hand or with `chrome-devtools-axi` | `sim` serves them at `/site/editor.html` and `/site/index.html` with a virtual Web MIDI | Connect finds "OpenPocket"; an edit shows in `/api/state` | | MIDI musical messages | see `features/midi-usb.md` | host simulator tests when the USB-MIDI feature is in; hardware: `docs/MIDI-TEST.md` | Features: `features/README.md` and one file per feature there. A proof that drives only one entry point is incomplete when @@ -45,9 +51,9 @@ the map lists more. ## Evidence Screenshots and logs go to `build/` (gitignored); copy what you cite to a folder outside the repo. Proof standards: run the -real code (the simulators compile the firmware's own `px_editor.h`, `px_store.h`, `ota.c`), not a re-implementation; +real code (the simulators compile the firmware's own `px_editor.h`, `px_store.h`, `ui.h`, `ota.c`), not a re-implementation; a mocked boundary is only Web MIDI/USB (already isolated by the tests); state what was exercised and what was not. -**Not verifiable here:** audio, DSP load and glitches, USB enumeration on a real host, Bluetooth, flashing, the screen on +**Not verifiable here:** audio on the pedal, its DSP load and glitches, USB enumeration on a real host, Bluetooth, flashing, the screen on the real LCD. Say so, and point to `docs/MIDI-TEST.md` (hardware checklist) when it exists. ## Cleanup diff --git a/.claude/skills/verify/features/README.md b/.claude/skills/verify/features/README.md index 4f153b8..642cb96 100644 --- a/.claude/skills/verify/features/README.md +++ b/.claude/skills/verify/features/README.md @@ -3,7 +3,7 @@ | File | Feature | Driver | | --- | --- | --- | | [build-and-packages.md](build-and-packages.md) | Firmware build, update loader, `.fwsc` packages, `manifest.json` | `op.mjs package` | -| [installer-page.md](installer-page.md) | Web installer `docs/index.html`: connect, detect, install, rescue | `pages.sh` | -| [editor-page.md](editor-page.md) | Web editor `docs/editor.html`: presets, blocks, import/export | `pages.sh` | -| [screen-ui.md](screen-ui.md) | Pedal screen pages (play, edit, save, tuner) | `uisim` via `op.mjs test` | +| [installer-page.md](installer-page.md) | Web installer `docs/index.html`: connect, detect, install, rescue | `pages.sh`; by hand: `op.mjs sim --update` | +| [editor-page.md](editor-page.md) | Web editor `docs/editor.html`: presets, blocks, import/export | `pages.sh`; by hand: `op.mjs sim` | +| [screen-ui.md](screen-ui.md) | Pedal screen pages (play, edit, save, tuner) | `uisim` via `op.mjs test`; live: `op.mjs sim` | | [midi-usb.md](midi-usb.md) | USB-MIDI: update/editor SysEx today; musical MIDI with the MIDI feature | simulators; hardware by the captain | diff --git a/.claude/skills/verify/features/editor-page.md b/.claude/skills/verify/features/editor-page.md index acf6aa5..c1a9646 100644 --- a/.claude/skills/verify/features/editor-page.md +++ b/.claude/skills/verify/features/editor-page.md @@ -1,13 +1,16 @@ # Editor page (docs/editor.html) ## Sub-features -52 presets (load, save, save as, move, factory reset), the current sound (blocks, types, parameters, order by dragging, name), JSON export/import, live follow of changes made on the pedal (WATCH), unsaved-change prompts, phone layout. +52 presets (load, save, save as, move, factory reset), the current sound (blocks, types, parameters, order by dragging, name), JSON export/import, live follow of changes made on the pedal (WATCH), unsaved-change prompts, phone layout; v37: the NAM models panel (A2 `.nam` files into the 16 user slots, `docs/pxnam.js`, protocol 2). ## How to get to it (user POV) Open the editor from the installer page, Connect, edit. ## Driving it with Playwright -`sh .claude/skills/verify/scripts/pages.sh`; `tests/test_editor_page.mjs` talks to `build/edsim` (the real `px_editor.h` and `px_store.h`) through a fake Web MIDI. End state: every line `ok`, `editor page test passed`, `build/editor/editor_1..6*.png`. Protocol only (no browser): `node tools/op.mjs test` runs `tests/test_editor.mjs`. +`sh .claude/skills/verify/scripts/pages.sh`; `tests/test_editor_page.mjs` talks to `build/edsim` (the real `px_editor.h` and `px_store.h`) through a fake Web MIDI. End state: every line `ok`, `editor page test passed`, `build/editor/editor_1..6*.png` (`editor_6_nam.png`: the NAM panel). Protocol only (no browser): `node tools/op.mjs test` runs `tests/test_editor.mjs`. + +## Driving it against the hosted simulator +`node tools/op.mjs sim --port 0` serves the page at `/site/editor.html` with a virtual Web MIDI: open it with `chrome-devtools-axi`, press Connect (the port is "OpenPocket"), edit, and compare with `GET /api/state` and `GET /api/screen.png` (the pedal's own screen follows an edit; a footswitch pressed over `POST /api/press/fs_up` moves the editor's selection through WATCH). Skill: `openpocket-simulator`. The real editor code over the same shim runs in `tests/test_sim.mjs`. ## Gotchas - Protocol is in `docs/EDITOR-PROTOCOL.md`; a change to a command needs both `px_editor.h` and `docs/pxedit.js`. diff --git a/.claude/skills/verify/features/installer-page.md b/.claude/skills/verify/features/installer-page.md index 6c5aa79..329ab7c 100644 --- a/.claude/skills/verify/features/installer-page.md +++ b/.claude/skills/verify/features/installer-page.md @@ -9,6 +9,9 @@ Open the page in Chrome/Edge, press Connect, plug the pedal in, press Install. ## Driving it with Playwright `sh .claude/skills/verify/scripts/pages.sh`; `tests/test_page.mjs` serves `docs/` plus fixture packages, bridges a fake Web MIDI to `build/devsim` (the pedal's real `ota.c` and `ldr_core.c`) and clicks Connect, Install, rescue. End state: "Done: the pedal now runs PocketX_924", the simulated flash holds the new app, screenshots `build/page/*.png`. +## Driving it against the hosted simulator +`node tools/op.mjs sim --update FLASH.bin --port 0` serves `/site/index.html` with a virtual Web MIDI over `tests/devsim.c` on that image (make one and fake packages with `python3 tests/fixtures.py DIR`; the page also needs its packages in `docs/firmware/`). `GET /api/info` shows the steps the pedal took in `log`. The image is never written; the result goes to `build/sim/update-dump.bin`. Skill: `openpocket-simulator`. + ## Gotchas - Fixtures are version 924 packages made by `tests/fixtures.py`, not the real build. - Nothing proves a real pedal: only the captain installs, by hand. diff --git a/.claude/skills/verify/features/midi-usb.md b/.claude/skills/verify/features/midi-usb.md index 084755c..0f3e904 100644 --- a/.claude/skills/verify/features/midi-usb.md +++ b/.claude/skills/verify/features/midi-usb.md @@ -9,6 +9,8 @@ Plug the pedal's USB-C into a computer; a DAW or the browser sees a MIDI port. ## Driving it without hardware `node tools/op.mjs test` (devsim and edsim run the SysEx paths); once the MIDI feature is in, its simulator test (`tests/`) covers the parser and CC map. The pages scripts cover the SysEx clients. +Any SysEx by hand: `curl -s -X POST $U/api/sysex -d 'f0 7d 4f 50 01 f7'` against `node tools/op.mjs sim` (the editor protocol, answered by the real `px_editor.h`); `POST /api/midi` plus `GET /api/events` for unanswered or pushed messages (WATCH). + ## Only a person with a pedal can verify Enumeration on a real host, a DAW sending PC/CC, the pedal answering, audio while MIDI flows, the update still working. Use `docs/MIDI-TEST.md` when it exists; do not claim these from simulators. diff --git a/.claude/skills/verify/features/screen-ui.md b/.claude/skills/verify/features/screen-ui.md index ab4be94..a8f4a74 100644 --- a/.claude/skills/verify/features/screen-ui.md +++ b/.claude/skills/verify/features/screen-ui.md @@ -9,6 +9,12 @@ Knob, PUSH, HOME, footswitches and block buttons on the pedal. ## Driving it with uisim `node tools/op.mjs test` builds and runs `tests/uisim.c` (the real `ui.h`), writing every page to `build/ui/*.ppm` and a log in which `never drawn` must not appear. View a PPM by converting it (`sips -s format png file.ppm --out file.png` on macOS). +## Driving it live with the hosted simulator +`node tools/op.mjs sim --port 0 --reset` runs the same `ui.h` on a real pedal state (`tests/edsim.c`). Press buttons (`POST /api/press/NAME`, `/api/knob/N`), then `GET /api/screen.png?scale=2` and look at it; `GET /api/state` gives the page (`mode`), the focused block and the sound. This reaches what `uisim`'s script does not: any sequence a user can play, the save flow into a slot and the flash file after it, the tuner with `POST /api/tuner/HZ`, the battery and volume. Skill: `openpocket-simulator`. + +## Driving it on the pedal skin +`$U/` shows the same screen in the drawn front panel (`tools/sim/pedal.html`), with the firmware's hold timings. `node tests/test_pedal_page.mjs OUTDIR` drives every control in Chromium and saves screenshots; it needs Playwright. Backlights are the skin's reading of the state, not firmware behaviour. + ## Gotchas - Colours and fonts are checked as drawn pixels, not on the real LCD. - New screens must be added to `tests/uisim.c` or they are never checked. diff --git a/.claude/skills/verify/scripts/pages.sh b/.claude/skills/verify/scripts/pages.sh index ce645e8..cad7447 100755 --- a/.claude/skills/verify/scripts/pages.sh +++ b/.claude/skills/verify/scripts/pages.sh @@ -1,6 +1,7 @@ #!/bin/sh -# Drives docs/index.html (installer) and docs/editor.html (editor) in headless Chromium against the simulated pedals -# (devsim.c / edsim.c) with a mocked Web MIDI device. Never touches a real pedal. Evidence: build/page/*.png, build/editor/*.png. +# Drives docs/index.html (installer), docs/editor.html (editor) and the pedal skin in headless Chromium against the simulated +# pedals (devsim.c / edsim.c) with a mocked Web MIDI device. Never touches a real pedal. Evidence: build/page/*.png, +# build/editor/*.png, build/pedal/*.png. # PW_DIR=/path/with/node_modules sh .claude/skills/verify/scripts/pages.sh (default PW_DIR: ~/.cache/openpocket-pw) set -e cd "$(dirname "$0")/../../../.." @@ -15,3 +16,5 @@ cc -O1 -Wall -Wno-unused-function -o build/edsim tests/edsim.c engine/px_dsp.c f export NODE_PATH="$PW_DIR/node_modules" node tests/test_page.mjs build/page node tests/test_editor_page.mjs build/editor +node tests/test_pedal_page.mjs build/pedal +node tests/test_audio_page.mjs build/audio diff --git a/.dockerignore b/.dockerignore index e241596..bb28666 100644 --- a/.dockerignore +++ b/.dockerignore @@ -1,6 +1,7 @@ .git build firmware/build +firmware/build-ble docs/firmware/*.fwsc node_modules fm1-1.bin diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 1fba492..56ebd77 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -12,8 +12,6 @@ jobs: - uses: actions/checkout@v4 - name: Build the container, the firmware and the installer packages run: node tools/op.mjs package - - name: Host tests - run: node tools/op.mjs test - uses: actions/upload-artifact@v4 with: name: openpocket-build @@ -22,3 +20,22 @@ jobs: docs/firmware/*.fwsc docs/firmware/manifest.json build/SHA256SUMS + - name: The simulator's engine as WebAssembly (test branch only) + run: node tools/op.mjs wasm + - uses: actions/upload-artifact@v4 + with: + name: openpocket-wasm + path: | + tools/sim/px_engine.wasm + tools/sim/px_engine.wasm.sources + - name: Build the MIDI test channel package (never the installer default) + run: node tools/op.mjs package-test + - uses: actions/upload-artifact@v4 + with: + name: openpocket-midi-test + path: | + firmware/openpocket-midi-test.bin + docs/firmware/midi-test/ + build/SHA256SUMS-midi-test + - name: Host tests + run: node tools/op.mjs test diff --git a/.gitignore b/.gitignore index cd0c4ea..b4ecd66 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,10 @@ build/ firmware/build/ firmware/openpocket.bin +firmware/openpocket-*.bin +docs/firmware/midi-test/ +firmware/build-ble/ +firmware/openpocket-ble.bin docs/firmware/*.fwsc docs/firmware/manifest.json *.o @@ -9,4 +13,6 @@ __pycache__/ fm1-1.bin fm1-300.bin *.dump.bin +*.fcp openpocket.bundle +node_modules/ diff --git a/Dockerfile b/Dockerfile index 05f2449..71b0a73 100644 --- a/Dockerfile +++ b/Dockerfile @@ -7,5 +7,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ && rm -rf /var/lib/apt/lists/* COPY firmware/setup.sh /usr/local/bin/op-setup-toolchain RUN JIELI_DEST=/opt/jieli sh /usr/local/bin/op-setup-toolchain +# the engine as WebAssembly for the simulated pedal's sound (engine/wasm.sh) +RUN apt-get update && apt-get install -y --no-install-recommends clang lld \ + && rm -rf /var/lib/apt/lists/* ENV JIELI_TOOLCHAIN=/opt/jieli/toolchain OP_IN_CONTAINER=1 WORKDIR /src diff --git a/Makefile b/Makefile index 591ee81..ab466b8 100644 --- a/Makefile +++ b/Makefile @@ -2,19 +2,42 @@ # Docker command lines; Windows calls it from the .bat files. See docs/BUILDING.md. # make doctor check the prerequisites (Node, Docker, the image) # make build firmware/openpocket.bin, the update loader and docs/firmware/*.fwsc + manifest.json +# make ble the Bluetooth MIDI test image: firmware/openpocket-ble.bin, build/ble/*.fwsc (docs/BLE-TEST.md) # make test the host tests (tests/run_tests.sh) # make package build + build/SHA256SUMS of the outputs +# make package-test the MIDI test channel package (docs/firmware/midi-test/, never the installer default) # make clean remove every build output (needs no Docker) # make shell a shell in the build container # make image (re)build the container image # make serve the installer and editor on http://localhost:8765/ (offline install) +# make serve-ble the same, with the Bluetooth test packages (build/ble) to install; builds them first if missing +# make flash ARGS="--dry-run" write a built package to a connected pedal (ARGS="--confirm-flash" to really write) +# make flash-rescue ARGS="--confirm-flash" wait for a pedal that restarts in a loop and install (docs/BUILDING.md) +# make log ARGS="--follow" read a connected pedal's state over USB MIDI, with the fields explained +# make sim [ARGS="--port 9000 --reset"] the hosted simulated pedal on http://127.0.0.1:8766/ (docs/SIMULATOR.md) +# make wasm the engine as WebAssembly for the simulator's sound (tools/sim/px_engine.wasm, committed) # make render IN=guitar.wav [OUT=out.wav] [PRESET=1] a WAV through the engine # make appimage [DUMP=fm1-1.bin] fm1-300.bin for the SD-card recovery +# make rescue [ARGS=..] restore a looping BlackBox from its UBOOT state, read-only (needs sudo and libusb: docs/RESCUE.md) +# make rescue-write [ARGS=..] the same, writing the official firmware back (run `make rescue` first) +# make rescue-check the UBOOT rescue tool (tools/uboot_rescue.py) against a simulated pedal; needs only Python 3 # `make NATIVE=1 build` skips Docker (toolchain installed with firmware/setup.sh). -.PHONY: doctor build test package clean shell image serve render appimage -doctor build test package clean shell image serve: +.PHONY: doctor build ble test package package-test clean shell image serve serve-ble wasm render appimage flash flash-rescue log rescue rescue-write rescue-check sim +doctor build ble test package package-test clean shell image serve wasm rescue-check: @node tools/op.mjs $@ +serve-ble: + @node tools/op.mjs serve ble render: @node tools/op.mjs render "$(IN)" "$(OUT)" "$(PRESET)" appimage: @node tools/op.mjs appimage "$(DUMP)" +flash log: + @node tools/op.mjs $@ $(ARGS) +rescue: + @node tools/op.mjs rescue $(ARGS) +rescue-write: + @node tools/op.mjs rescue --write $(ARGS) +flash-rescue: + @node tools/op.mjs flash --rescue $(ARGS) +sim: + @node tools/op.mjs sim $(ARGS) diff --git a/NOTICE.md b/NOTICE.md index c9a68b7..e833ce5 100644 --- a/NOTICE.md +++ b/NOTICE.md @@ -8,7 +8,17 @@ OpenPocket is GPL-3.0-only (`LICENSE`), except the parts below, which keep their | `loader/` (update loader) | felucca-bsp `loader/`, changed for the Pocket X flash layout | MPL-2.0 | | `docs/pxota.js`, `docs/pxpkg.js`, `tools/pxpkg.py`, `tools/mkloader.py`, `tools/lz4blk.py` | adapted from [Felucca](https://github.com/hugelton/Felucca)'s web installer and build tools (as used by CLIPPER and SLOOP) | GPL-3.0 | | `firmware/sdk/` (I2S and DAC drivers as LLVM bitcode, SPL, cfg files) | JieLi AC79 SDK V1.2.0 (fw-AC79_AIoT_SDK) | Apache-2.0 (`LICENSES/Apache-2.0.txt`) | +| `firmware/px_cacheram.h` (data-cache ways as RAM, for the NAM network's vector-unit buffers) | [melodee](https://github.com/keremimo/melodee) `firmware/hal/fm1_cache.h`, Kerem Kilic (Ellic Studio), after the AC79 SDK's cache way setup | GPL-3.0 | +| `third_party/jieli-ble/` (JieLi's Bluetooth LE controller, 33 objects of `btctrler.a` and 3 of `cpu.a`, as published LLVM bitcode, used only by the Bluetooth test image `make ble`) | JieLi AC79 SDK V1.2.0 (fw-AC79_AIoT_SDK, commit `0617f4c`), `cpu/wl82/liba/`; see `third_party/jieli-ble/README.md` | Apache-2.0 (`third_party/jieli-ble/LICENSE-JieLi-Apache-2.0`, `LICENSES/Apache-2.0.txt`) | +| `firmware/ble/ble_cfg.c` | values and names from the same SDK's `apps/common/config/log_config/lib_btctrler_config.c` | Apache-2.0 | | `firmware/fonts.h` (bitmaps generated from Inter) | [Inter](https://rsms.me/inter/), Rasmus Andersson | SIL OFL 1.1 (`LICENSES/OFL-1.1-Inter.txt`) | +| `assets/loops/*.wav` (guitar loops for the simulator) | cut from [Guitar-TECHS](https://zenodo.org/records/14963133), credits in `assets/loops/ATTRIBUTION.md` | CC BY 4.0 | + +The factory NAM models (`engine/px_nam_factory.h`, v37) are A2-Lite networks trained by OpenPocket from Neural Amp +Modeler community captures ([pelennor2170/NAM_models](https://github.com/pelennor2170/NAM_models), GPL v3; authors +in `docs/NAM.md`) played with DI guitar from [Alec-Wright/Automated-GuitarAmpModelling](https://github.com/Alec-Wright/Automated-GuitarAmpModelling) +(GPL v3). The NAM A2 architecture is from Neural Amp Modeler (Steven Atkinson, MIT); `engine/px_nam.c`, +`tools/nam_a2.py` and `tools/train_a2.py` are written for OpenPocket from the published architecture. Sound models: the names refer to the gear that inspired each voicing (circuit analyses published by ElectroSmash, Electric Druid, R.G. Keen / GEOFEX, Rob Robinette, Duncan's Tone Stack Calculator; speaker data sheets; manufacturers' diff --git a/README.md b/README.md index 5e58bde..417b6fb 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ !!! THIS PROJECT IS A PROOF OF CONCEPT, DONT EXPECT FULL FUNCTION AND SOUND QUALITY !!! Install here: https://isod89.github.io/openpocket/ (Donner Pocket X; M-VAVE BlackBox: v31 has the fix for issue #1, being tested) -NEW WEB EDITOR: https://isod89.github.io/openpocket/editor.html · What's new since v33: [docs/WHATS-NEW-v34-v36.md](docs/WHATS-NEW-v34-v36.md) +NEW WEB EDITOR: https://isod89.github.io/openpocket/editor.html · **v37: neural amp models (NAM A2), load your own from TONE3000**: [docs/WHATS-NEW-v37.md](docs/WHATS-NEW-v37.md) · since v33: [docs/WHATS-NEW-v34-v36.md](docs/WHATS-NEW-v34-v36.md) **Open-source guitar multi-effects firmware for the Donner Pocket X and the M-VAVE BlackBox.** Written from scratch for the pedal's own chip (JieLi AC791N, two cores), installable from the browser. @@ -12,11 +12,15 @@ Written from scratch for the pedal's own chip (JieLi AC791N, two cores), install ## What it does -- **12-block chain, 77 models**, in any order you like (double-click PUSH to move a block; the order is saved with +- **12-block chain, 80 models**, in any order you like (double-click PUSH to move a block; the order is saved with the preset): GATE, COMP, PITCH, SPECIAL FX, SYNTH, DRIVE, AMP, CAB, EQ, MOD, DELAY, REVERB - **Synths, keyboards and other guitars** from a normal guitar (no MIDI pickup): organ, church organ, string machine, choir, mono synth with pitch tracking and a resonant ladder filter, PWM synth, bass synth, 12-string, acoustic, sitar (bridge buzz + sympathetic strings), baritone +- **Neural amp models (v37)**: the AMP block's NAM type runs Neural Amp Modeler **A2** networks on the pedal: 12 + factory models trained for it (Twin, AC15, Super 59, Dirty Shirley, JCM900, JCM2000, JVM, Mark IV, Pro Mod, 5150, + 6505+) and 16 user slots you fill from the web editor with A2 captures (TONE3000...). All 24 factory presets use + them. Details: [docs/NAM.md](docs/NAM.md) - **8 amps fitted to captures of the real ones** (v34): Twin Reverb, AC15 Top Boost, JCM900, JCM2000, Mark IV, Dirty Shirley, Pro Mod, 5150. A computer sets the engine's numbers (tube stages oversampled ×4, tone stack, power amp, sag) so that it matches a Neural Amp Modeler capture of each amp on real guitar playing; same CPU cost as a @@ -25,7 +29,10 @@ Written from scratch for the pedal's own chip (JieLi AC791N, two cores), install Greenback, T75, Alnico Blue, Jensen, Cream, DI) with a MIC position - **Pro levels (v36)**: every amp, block and preset at the same loudness (-23 LUFS, measured on real guitar); 24 factory presets round the fitted amps, 28 USER slots; a PRESET LVL per preset; soft output limiter -- What changed since v33: [docs/WHATS-NEW-v34-v36.md](docs/WHATS-NEW-v34-v36.md) +- **MIDI over USB-C (v37)**: Program Change selects a preset, Control Change switches blocks and sets their models and + settings, the pedal tells the computer when you change the sound, MIDI clock sets the delay time. Class-compliant, no driver; + Bluetooth MIDI is only in the test image ([docs/BLE-TEST.md](docs/BLE-TEST.md)). Reference: [docs/MIDI.md](docs/MIDI.md), hardware test: [docs/MIDI-TEST.md](docs/MIDI-TEST.md) +- What changed: [v37](docs/WHATS-NEW-v37.md), [since v33](docs/WHATS-NEW-v34-v36.md) - **New screen (v36)**: a PLAY page readable from a distance (preset number and name, amp, chain in colour) and an EDIT page with big values and bars - **8 reverbs rebuilt in v35**: an 8-line feedback delay network with early reflections and moving lines (no @@ -38,7 +45,7 @@ Written from scratch for the pedal's own chip (JieLi AC791N, two cores), install (the note compresses on a hard attack and blooms back) - **24-bit audio** in and out, guitar read from the left input channel only, like the official firmware (the right channel only carries noise: mixing it in was the constant hiss on high-gain amps) -- **USB update**: the pedal shows up as a USB-MIDI device and takes updates from the [web installer](docs/index.html) +- **USB update**: the pedal shows up as a USB-MIDI device and takes updates from the [web installer](docs/index.html), or from the command line (`make flash`, see [docs/BUILDING.md](docs/BUILDING.md)) Full list of models, presets and controls: [docs/SOUNDS-AND-CONTROLS.md](docs/SOUNDS-AND-CONTROLS.md). Development notes (how the audio path was found, what changed in each version): [docs/AUDIO-SDK.md](docs/AUDIO-SDK.md). @@ -60,8 +67,9 @@ setting was still the official firmware's (it survives a warm reset), so the ped the DSP slower and crackle, so v33 leaves them as the boot loader does again.) Being tested on the BlackBox. For a pedal that restarts in a loop, the page has a **rescue mode** (step 2): tick it, press Connect, plug the pedal in; the install starts the moment the pedal answers and halts its DSP at once. A BlackBox can always go back to its official firmware from the page (step 3). It is the same JieLi update system as the M-VAVE FM-1, where this method is -used by Felucca, CLIPPER and SLOOP. Keep the SD-card recovery at hand (`flash/`: fm1buddy on an ESP32-S3, -`makeimage.bat`). +used by Felucca, CLIPPER and SLOOP. If the pedal still loops, restore it from a Mac with no extra hardware: `make rescue` (read-only check and backup), then +`make rescue-write` (`docs/RESCUE.md`; it needs `brew install libusb`, sudo and the official BlackBox file). The second route is the +SD-card recovery (`flash/`: fm1buddy on an ESP32-S3, `makeimage.bat`). **Without internet**: double-click `INSTALL-OFFLINE.bat` (Windows; nothing to install: a small PowerShell server, `tools/serve_offline.ps1`) or run `make serve` (macOS, Linux; `tools/serve.py`, needs Python 3). It serves the installer @@ -111,6 +119,9 @@ replaces the files with this folder's. on a simulated flash: every command, pushes both ways, a power cycle; then the editor page in Chromium (sliders, drag and drop, save, move, export / import). +To play with the simulated pedal yourself (screen, footswitches, knob, the installer and editor pages, and its sound, +with bundled guitar loops, a file or an input of your computer), run `make sim` and open `http://127.0.0.1:8766/`; see [`docs/SIMULATOR.md`](docs/SIMULATOR.md). + ## Layout | | | @@ -122,6 +133,7 @@ replaces the files with this folder's. | `tools/` | package builder, loader wrapper, font converter, flash image tools | | `tests/` | host tests and the simulated pedal | | `flash/` | SD-card recovery with fm1buddy (ESP32-S3) | +| `firmware/ble/`, `third_party/jieli-ble/` | the optional Bluetooth MIDI test image (`make ble`, [docs/BLE-TEST.md](docs/BLE-TEST.md)) and JieLi's Bluetooth controller it links | ## Licence diff --git a/assets/loops/ATTRIBUTION.md b/assets/loops/ATTRIBUTION.md new file mode 100644 index 0000000..1c39ce1 --- /dev/null +++ b/assets/loops/ATTRIBUTION.md @@ -0,0 +1,13 @@ +# The bundled guitar loops + +These recordings are not under this repository's GPL-3.0. They are licensed under the [Creative Commons Attribution 4.0 International licence](https://creativecommons.org/licenses/by/4.0/), as the dataset's Zenodo record states ([https://zenodo.org/records/14963133](https://zenodo.org/records/14963133), read on 2026-11-05). All are the dry direct-input channel of the guitar, with no amp. + +The dataset's authors ask users to cite their paper: Guitar-TECHS, ICASSP 2025, doi:10.1109/ICASSP49660.2025.10887996. + +`prepare.py` records how each file was made. BPM values come from the dataset's MIDI files. + +- `chug.wav`: "Low riff", cut from Guitar-TECHS (directinput_02.wav of P3_music) by Hegel Emmanuel Pedroza Villalobos, Termeh Taheri, Wallace Abreu, Ryan Corey and Iran R. Roman, https://zenodo.org/records/14963133, CC BY 4.0. Changed: cut to a loop, crossfaded, resampled to 44.1 kHz, converted to mono 16-bit, normalised. +- `riff.wav`: "Clean riff", cut from Guitar-TECHS (directinput_08.wav of P3_music) by Hegel Emmanuel Pedroza Villalobos, Termeh Taheri, Wallace Abreu, Ryan Corey and Iran R. Roman, https://zenodo.org/records/14963133, CC BY 4.0. Changed: cut to a loop, crossfaded, resampled to 44.1 kHz, converted to mono 16-bit, normalised. +- `arpeggio.wav`: "Arpeggios", cut from Guitar-TECHS (directinput_09.wav of P3_music) by Hegel Emmanuel Pedroza Villalobos, Termeh Taheri, Wallace Abreu, Ryan Corey and Iran R. Roman, https://zenodo.org/records/14963133, CC BY 4.0. Changed: cut to a loop, crossfaded, resampled to 44.1 kHz, converted to mono 16-bit, normalised. +- `chords.wav`: "Chords", cut from Guitar-TECHS (directinput_11.wav of P3_music) by Hegel Emmanuel Pedroza Villalobos, Termeh Taheri, Wallace Abreu, Ryan Corey and Iran R. Roman, https://zenodo.org/records/14963133, CC BY 4.0. Changed: cut to a loop, crossfaded, resampled to 44.1 kHz, converted to mono 16-bit, normalised. +- `lead.wav`: "Lead phrase", cut from Guitar-TECHS (directinput_12.wav of P3_music) by Hegel Emmanuel Pedroza Villalobos, Termeh Taheri, Wallace Abreu, Ryan Corey and Iran R. Roman, https://zenodo.org/records/14963133, CC BY 4.0. Changed: cut to a loop, crossfaded, resampled to 44.1 kHz, converted to mono 16-bit, normalised. diff --git a/assets/loops/arpeggio.wav b/assets/loops/arpeggio.wav new file mode 100644 index 0000000..39c5c1d Binary files /dev/null and b/assets/loops/arpeggio.wav differ diff --git a/assets/loops/chords.wav b/assets/loops/chords.wav new file mode 100644 index 0000000..dfa5537 Binary files /dev/null and b/assets/loops/chords.wav differ diff --git a/assets/loops/chug.wav b/assets/loops/chug.wav new file mode 100644 index 0000000..635348f Binary files /dev/null and b/assets/loops/chug.wav differ diff --git a/assets/loops/lead.wav b/assets/loops/lead.wav new file mode 100644 index 0000000..1e624f0 Binary files /dev/null and b/assets/loops/lead.wav differ diff --git a/assets/loops/manifest.json b/assets/loops/manifest.json new file mode 100644 index 0000000..6c366a1 --- /dev/null +++ b/assets/loops/manifest.json @@ -0,0 +1,94 @@ +{ + "loops": [ + { + "file": "chug.wav", + "title": "Low riff", + "kind": "chugging rhythm", + "signal": "dry", + "author": "Hegel Emmanuel Pedroza Villalobos, Termeh Taheri, Wallace Abreu, Ryan Corey and Iran R. Roman", + "source": "https://zenodo.org/records/14963133", + "source_file": "P3_music.zip: audio/directinput/directinput_02.wav, 14.66 s to 20.054 s", + "license": "CC-BY-4.0", + "license_url": "https://creativecommons.org/licenses/by/4.0/", + "attribution": "\"Low riff\", cut from Guitar-TECHS (directinput_02.wav of P3_music) by Hegel Emmanuel Pedroza Villalobos, Termeh Taheri, Wallace Abreu, Ryan Corey and Iran R. Roman, https://zenodo.org/records/14963133, CC BY 4.0. Changed: cut to a loop, crossfaded, resampled to 44.1 kHz, converted to mono 16-bit, normalised.", + "sha256": "dc070d2ddad9daa3c0f19cda2e184bcd08c620e769bcead8d719dc45a92a255b", + "seconds": 5.395, + "bpm": 90, + "bars": 2, + "sample_rate": 44100, + "channels": 1 + }, + { + "file": "riff.wav", + "title": "Clean riff", + "kind": "clean riff", + "signal": "dry", + "author": "Hegel Emmanuel Pedroza Villalobos, Termeh Taheri, Wallace Abreu, Ryan Corey and Iran R. Roman", + "source": "https://zenodo.org/records/14963133", + "source_file": "P3_music.zip: audio/directinput/directinput_08.wav, 15.57 s to 19.987 s", + "license": "CC-BY-4.0", + "license_url": "https://creativecommons.org/licenses/by/4.0/", + "attribution": "\"Clean riff\", cut from Guitar-TECHS (directinput_08.wav of P3_music) by Hegel Emmanuel Pedroza Villalobos, Termeh Taheri, Wallace Abreu, Ryan Corey and Iran R. Roman, https://zenodo.org/records/14963133, CC BY 4.0. Changed: cut to a loop, crossfaded, resampled to 44.1 kHz, converted to mono 16-bit, normalised.", + "sha256": "ac61ab74cb344782086bba9d10d020745190361dd66bd2b8a8b47b18f94f9bad", + "seconds": 4.423, + "bpm": 110, + "bars": 2, + "sample_rate": 44100, + "channels": 1 + }, + { + "file": "arpeggio.wav", + "title": "Arpeggios", + "kind": "arpeggios", + "signal": "dry", + "author": "Hegel Emmanuel Pedroza Villalobos, Termeh Taheri, Wallace Abreu, Ryan Corey and Iran R. Roman", + "source": "https://zenodo.org/records/14963133", + "source_file": "P3_music.zip: audio/directinput/directinput_09.wav, 18.078 s to 23.546 s", + "license": "CC-BY-4.0", + "license_url": "https://creativecommons.org/licenses/by/4.0/", + "attribution": "\"Arpeggios\", cut from Guitar-TECHS (directinput_09.wav of P3_music) by Hegel Emmanuel Pedroza Villalobos, Termeh Taheri, Wallace Abreu, Ryan Corey and Iran R. Roman, https://zenodo.org/records/14963133, CC BY 4.0. Changed: cut to a loop, crossfaded, resampled to 44.1 kHz, converted to mono 16-bit, normalised.", + "sha256": "1545446fb05a3af5bc369d4d5933b1da4b7af4e5c21e2588877d9a886e9af5bd", + "seconds": 5.465, + "bpm": 89, + "bars": 2, + "sample_rate": 44100, + "channels": 1 + }, + { + "file": "chords.wav", + "title": "Chords", + "kind": "chords", + "signal": "dry", + "author": "Hegel Emmanuel Pedroza Villalobos, Termeh Taheri, Wallace Abreu, Ryan Corey and Iran R. Roman", + "source": "https://zenodo.org/records/14963133", + "source_file": "P3_music.zip: audio/directinput/directinput_11.wav, 2.377 s to 7.487 s", + "license": "CC-BY-4.0", + "license_url": "https://creativecommons.org/licenses/by/4.0/", + "attribution": "\"Chords\", cut from Guitar-TECHS (directinput_11.wav of P3_music) by Hegel Emmanuel Pedroza Villalobos, Termeh Taheri, Wallace Abreu, Ryan Corey and Iran R. Roman, https://zenodo.org/records/14963133, CC BY 4.0. Changed: cut to a loop, crossfaded, resampled to 44.1 kHz, converted to mono 16-bit, normalised.", + "sha256": "60629bb17273731f292e0991e0da7d096f1e69b2d45faffaaf17884c0b804bff", + "seconds": 5.113, + "bpm": 93, + "bars": 2, + "sample_rate": 44100, + "channels": 1 + }, + { + "file": "lead.wav", + "title": "Lead phrase", + "kind": "lead phrase", + "signal": "dry", + "author": "Hegel Emmanuel Pedroza Villalobos, Termeh Taheri, Wallace Abreu, Ryan Corey and Iran R. Roman", + "source": "https://zenodo.org/records/14963133", + "source_file": "P3_music.zip: audio/directinput/directinput_12.wav, 3.275 s to 13.566 s", + "license": "CC-BY-4.0", + "license_url": "https://creativecommons.org/licenses/by/4.0/", + "attribution": "\"Lead phrase\", cut from Guitar-TECHS (directinput_12.wav of P3_music) by Hegel Emmanuel Pedroza Villalobos, Termeh Taheri, Wallace Abreu, Ryan Corey and Iran R. Roman, https://zenodo.org/records/14963133, CC BY 4.0. Changed: cut to a loop, crossfaded, resampled to 44.1 kHz, converted to mono 16-bit, normalised.", + "sha256": "7b8125718beff2f432dc56aadc3b33d48197ab5a2e0162de72d30745793fc13f", + "seconds": 10.287, + "bpm": 140, + "bars": 6, + "sample_rate": 44100, + "channels": 1 + } + ] +} diff --git a/assets/loops/prepare.py b/assets/loops/prepare.py new file mode 100644 index 0000000..3e6eb04 --- /dev/null +++ b/assets/loops/prepare.py @@ -0,0 +1,94 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-3.0-only +# Makes the bundled loops from the Guitar-TECHS dataset's direct-input takes, and their manifest.json and ATTRIBUTION.md. +# The WAV files are committed, so nothing needs this script to run: it is the record of how they were made. +# curl -LO https://zenodo.org/records/14963133/files/P3_music.zip && unzip P3_music.zip +# python3 assets/loops/prepare.py P3_music (needs numpy and scipy) +# Each loop runs from a note to the same note some bars later, found by comparing the spectra around both. The end point +# is moved to where the two waveforms line up, then the end's first 20 ms are crossfaded into the start. +import hashlib, json, os, sys, wave +import numpy as np +import scipy.io.wavfile +import scipy.signal + +HERE = os.path.dirname(os.path.abspath(__file__)) +RATE = 44100 +PEAK = 10 ** (-3 / 20) +FADE = int(0.020 * RATE) +ALIGN = int(0.008 * RATE) +AUTHOR = "Hegel Emmanuel Pedroza Villalobos, Termeh Taheri, Wallace Abreu, Ryan Corey and Iran R. Roman" +SOURCE = "https://zenodo.org/records/14963133" +LICENSE, LICENSE_URL = "CC-BY-4.0", "https://creativecommons.org/licenses/by/4.0/" + +# take, seconds from, to, BPM of the take's MIDI file, bars +LOOPS = [ + ("chug", "Low riff", "chugging rhythm", 2, (14.66, 20.054, 90, 2)), + ("riff", "Clean riff", "clean riff", 8, (15.57, 19.987, 110, 2)), + ("arpeggio", "Arpeggios", "arpeggios", 9, (18.078, 23.546, 89, 2)), + ("chords", "Chords", "chords", 11, (2.377, 7.487, 93, 2)), + ("lead", "Lead phrase", "lead phrase", 12, (3.275, 13.566, 140, 6)), +] + + +def take(root, number): + _, x = scipy.io.wavfile.read(os.path.join(root, "audio", "directinput", f"directinput_{number:02d}.wav")) + x = x[:, 0] if x.ndim > 1 else x + return scipy.signal.resample_poly(x / float(2 ** 31), 147, 160) + + +def loop(x, start, end): + s, e = round(start * RATE), round(end * RATE) + window = x[s : s + 2048] + lags = range(-ALIGN, ALIGN + 1) + e += max(lags, key=lambda lag: float(np.dot(window, x[e + lag : e + lag + 2048]))) + y = x[s:e].copy() + fade = 0.5 - 0.5 * np.cos(np.pi * np.arange(FADE) / FADE) + y[:FADE] = y[:FADE] * fade + x[e : e + FADE] * (1 - fade) + return y * (PEAK / np.abs(y).max()) + + +def write(path, y): + with wave.open(path, "wb") as w: + w.setnchannels(1) + w.setsampwidth(2) + w.setframerate(RATE) + w.writeframes(np.round(y * 32767).astype(" **Status: runs on a real BlackBox.** A Mac connected, found the service, and a Control Change switched a block and a Program Change +> loaded a preset, with the state read back over USB. A Chocolate Plus on H paired by itself and its footswitches A to D changed +> DRIVE, MOD, DELAY and REVERB. The audio under a link is not measured: read +> [What is not verified](#what-is-not-verified) before you flash it. + +**First hardware run (fixed).** The first image crashed on every boot, before any advertisement: the hardware stack-limit +guard (`fm1_guard.h`, EMU bit 3) faults any stack pointer outside `[_ustack_lo, _sstack_top]`, and the controller's task +stacks (`ble_os.c`) are in ordinary RAM below that window. The test image is now built with that one guard off +(`OP_GUARD_STACK=0` in `firmware/build.sh`); the write, bus and code-range guards stay on. The normal firmware keeps all four. +`firmware/app.ld` now refuses to link the image if the stack guard is on while the task stacks lie outside the window, so the +mistake cannot come back unnoticed (`OP_BLE=1 OP_GUARD_STACK=1 sh firmware/build.sh` shows the refusal). + +## Build + +``` +make ble # firmware/openpocket-ble.bin and build/ble/PocketX_936.fwsc, BlackBox_936.fwsc, manifest.json +``` + +(`node tools/op.mjs ble` is the same; `OP_BLE=1 sh firmware/build.sh` inside the container.) It prints the SHA-256 of the image and the two packages. Build outputs +are `firmware/build-ble/` and `build/ble/`; `make clean` removes them. + +## Flash it + +This is the pedal's normal update path, with the test packages instead of the release ones. Your pedal must already run +OpenPocket (the stock-firmware scan was skipped on purpose). + +1. `make serve-ble` builds the test packages if they are missing and opens the offline installer on + `http://localhost:8765/` with `build/ble/` as its package folder. (Chrome or Edge, USB cable to the pedal.) +2. **Connect**, pick the pedal, **Install**, as for any update. The pedal restarts into the test image; its start-up + screen reads `v36 BLE TEST`. +3. To go back to the normal firmware: run `make serve` (the release packages) and **Install** again, or use the + **Back to the official firmware** step. The SD-card recovery (`make appimage`) works as before. + +The status text at the bottom right of the screen shows the Bluetooth state: `STARTING` with the stage and the pedal's +usual `C1 42%`, then `ADVERTISING` (the proof counters, scrolling) or `FAILED` (the failure, scrolling, with its short +code first so that it is always on screen). See [Reading the status text](#reading-the-status-text). + +## What you should see + +Open a scanner (nRF Connect, LightBlue, or macOS *Audio MIDI Setup*; use active scanning, which phones do by default): + +| In the scanner | Value | +| --- | --- | +| Name | `OpenPocket` (in the scan response) | +| Service UUID | BLE-MIDI `03B80E5A-EDE8-4B33-A751-6CE34EC4C700` (in the advertising data) | +| Type | connectable undirected (`ADV_IND`), 100 ms interval, channels 37, 38, 39 | +| Address | `02:4F:50:42:4C:54`, a fixed locally administered address set by this image (not the stock firmware's) | + +The pedal has one link and stops advertising while it is in use, so a scanner sees nothing then. A Chocolate Plus with its switch on +H connects by itself within seconds of the pedal starting. To reach the pedal from a Mac, put the Chocolate on O. +`tools/chocolate/README.md` sets its footswitches up to switch the effects. + +If the screen reads `FAILED`, or nothing appears within a few seconds, please report the screen text, the pedal model, +what the scanner shows and whether the pedal still works as a normal OpenPocket (audio, screen, USB MIDI). + +## The link + +| Part | What it is | +| --- | --- | +| GATT | GAP (`0x1800`, device name), GATT (`0x1801`), and the MIDI service with its characteristic `7772E5DB-3868-4112-A1A9-F2669D106BF3`: read (empty), write without response, notify | +| MIDI in | `firmware/px_ble_midi.h` reads the header, timestamps, running status and several messages in one write, and ignores the timestamps. SysEx and system common messages are skipped | +| MIDI out | Nothing yet: notify can be enabled, but the pedal sends nothing | +| Pairing | Just Works with bonding, accepted when the central asks. The pedal never asks, and the characteristic does not need it. Keys stay in RAM, so a bond ends at a power cycle | +| Connection | The central's parameters are kept. The pedal does not ask for others: the Chocolate's 60 ms, latency 0, 4 s timeout held | + +The log channel (`node tools/op.mjs log`) adds two lines, and the status area scrolls the first one while connected: + +`link LINK 1 n2 A A1B2C3D4E5F6 I48 L0 T400 W7 P1 D0 LAST 2 24 127 STUB 3 low_power_request` + +| Field | Meaning | +| --- | --- | +| `LINK` | 1 while connected | +| `n` | links so far, counting ones that failed to establish | +| `A` | the central's address | +| `I` `L` `T` | interval (1.25 ms units), latency, timeout (10 ms units) | +| `W` | writes to the MIDI characteristic | +| `P` | Just Works pairings accepted | +| `D` | the last disconnection reason in hex: `13` the central closed it, `8` timeout, `3E` the link never established | +| `LAST` | the last message: type (1 Program Change, 2 Control Change), data 1, data 2 | +| `STUB` | calls of the stand-ins (`ble_stubs.c`), and the last one's name | + +`trace EV 43E0100 4050013 ST name name` lists the last host events (packet type, event code, subevent or status, status; without Number Of Completed Packets) and the stand-ins called. While a link is up, `i` in the `ADV` line counts connection events, so about 16.7 a second means a 60 ms link. + +## Reading the status text + +| Text | Meaning | +| --- | --- | +| `STARTING n` | the last stage reached: 1 task created, 2 controller init called, 3 radio front end replay entered, 4 waiting for the controller task, 5 setting the advertisement, 6 enabling it, 7 the replay returned, 8 the baseband asked for its interrupts. The markers are set by two tasks, so 3 and 7 can appear after 4; the number shown is the last one set, not the highest | +| `FAILED s c` | step `s` failed with return value `c`. The status area is only 11 characters wide, so the whole line scrolls like a marquee (2 characters a second, resting 2 s on the start, then looping): `FAILED 2 -7 step btctrler_task_init stage init`. The short code is first, so it can be read before the scroll starts. The usual `C1 42%` is not shown while it has failed. | +| `ADVERTISING` | the controller accepted the command to advertise; it does not mean a packet was sent. The status shows the proof counters instead, as a marquee (below) | + +Failing steps: `1` `task_create` failed (`c` is -1); `2` `btctrler_task_init` returned `c` (not 0); `3` +`btctrler_task_ready` returned `c` (0: the controller task did not answer); `4` `ll_hci_adv_enable` returned `c` (not +0); `5` still `STARTING` after 5 s, and `c` is the stage it stalled in (the list above). At a timeout the line ends with `wait XXXXXXXX`: the address of the semaphore or mutex the controller task is blocked on (look it up in `firmware/build-ble/symbols.txt`; a value below 0x10 is a task state instead). Example: `FAILED 5 3` only means the last marker passed was the entry of the radio replay; it does not say the replay hung. (In the second hardware run it was a deadlock after the radio start-up: see "Found on hardware" below.) The codes and the text come from `firmware/ble/ble_status.h`, checked on the PC by +`tests/test_ble_status.c`. Please report the whole `FAILED s c` text. + +### The advertising line (proof counters) + +The fourth flash showed `ADVERTISING` but neither a Mac nor a phone scanner saw the pedal, so while advertising the status area scrolls a line of counters and registers, with the numbers first: `ADV i12 e5 r0 t340 s0 CON 2001 IEN 101 IPND 0 STAT 80 IC45 3 IC29 3 BEN 1`. + +| Field | Meaning | What it tells | +| --- | --- | --- | +| `i` | baseband event interrupts (IRQ 45), counted by `ble_isr.S` before the library's own handler runs | above 0 and rising: the radio runs and events happen. 0: no event ever reaches the CPU | +| `r` | receive interrupts (IRQ 29) | above 0: a scanner's request was heard, so the antenna works both ways | +| `e` | HCI packets the controller returned (`hci_packet_handler`) | any controller answer | +| `t` | library software timers that fired | the controller's tick source runs | +| `s` | calls of the slot-timer stand-ins (`slot_timer_*`) | above 0: the link scheduler asked for a timer this image does not have (see below) | +| `CON IEN IPND STAT` | baseband registers `0x28000`, `0x28028`, `0x28030`, `0x28038` in hex | enable, interrupt enables, pending, busy flags | +| `IC45 IC29` | the interrupt configuration of the two lines (bit 0 enabled, bits 3:1 priority) | 0: the line is off | +| `BEN` | BLE block enable `0x2FC80` | bit 0 clear: the block is off | + +Reading the next flash: `i` and `t` rising means the radio runs and the cause is on the air side (power, trims, address). `i` 0 with `IC45` 0 means the interrupt was never enabled; `i` 0 with `IC45` 3 and `IEN` 0 means the baseband never enabled its event; `t` 0 means the library's timers never fire. + +**Checked against the Core Specification** (`firmware/ble/ble_adv.h`, tested on the PC): the advertising data is two whole AD structures (flags 0x06, the complete 128-bit UUID list, little endian), 21 of 31 bytes; the scan response is the complete local name, 21 bytes; the PDU is ADV_SCAN_IND at 100 ms on channels 37, 38 and 39 with filter policy 0. One fault was found and fixed: the controller's public address was never set (the library reads it from a stored item this image does not have), so it was all zeros, which is not a valid device address. It is now `02:4F:50:42:4C:54` (locally administered, unicast; not an IEEE-assigned address). The library's own-address type is fixed at public in its HCI layer, so a random static address cannot be used from here. + +**Open suspicion, not a fix:** the link scheduler (`link_task.c`) arms its timer through `slot_timer_set` and its siblings, which are among the empty stand-ins in `ble_stubs.c` (the library's slot timer is not part of the linked archive). Whether a plain advertising event depends on them is not known. `s` counts the calls: if it is above 0 the next step is a real slot timer. + +## Reading the pedal from a computer + +`node tools/op.mjs log` reads all of this over USB MIDI, so nobody has to read the scrolling text: it prints each field with its meaning and a verdict. It needs `npm ci --prefix tools` once. + +``` +node tools/op.mjs log --wait 40 --scan 30 # after a flash: wait for the boot, read, scan from this Mac, read, judge +node tools/op.mjs log --follow --json # keep reading, one JSON object per reading +``` + +The request is `DIAG` ([EDITOR-PROTOCOL.md](EDITOR-PROTOCOL.md)), which only this image answers, so the normal firmware is byte for byte what it was. It reads memory and the same registers as the status text, and writes nothing. Its three lines are: + +| Line | Example | Meaning | +| --- | --- | --- | +| this boot | `UP 137120 SAFE 0 STAGE 8 RST 1 GUARD 0` | the pedal's clock in ms, which wraps every 179 s, so it is the uptime only in the first 179 s; safe mode; boot stage (8 = OK); reset source bits (`POWER-ON`, `VDDIO LOW`, `WATCHDOG`, `VCM`, `LONG PRESS`, `1.2V`, `SOFT`, from bit 0); failed boots counted | +| last boot | `LAST OK`, or `LAST FAILED STAGE 7 BLOCKS 9 LOAD 42 PEAK 61 NAN 0 CPU1 1 LOOPS 0 CRASH VEC 3F PC 2000DC8 RETS 2001234 DBG 0 EMU 0 UP 4100` | the fields of the `LAST BOOT FAILED` screen; `CRASH ...` only after a crash | +| status | `ADV i12 e5 ...`, `FAILED 5 3 step timeout ...`, `STARTING 4`, `OFF` | the text the status area scrolls, from the same function | + +It cannot read a pedal that is not running its main loop. While the red crash screen or the 12 s `LAST BOOT FAILED` report is up, USB is down; the next boot gives the same report through this channel. A pedal stuck before USB service stays silent: only the screen shows what it is doing. With firmware that has no `DIAG`, the command gives the version and the DSP load and says what is missing. Exit codes: 0 pass or running, 1 fail, 3 undecided. `RUNNING` means the radio events rise; only a sighting by the scan is `PASS`. + +## Found on hardware + +The second flash showed `FAILED 5 3`. The cause, found by reading the vendor code against the linked image: the controller's link-layer start-up calls `thread_create`, which calls `crc_get_32bit`, which waits for ever on a lock that the vendor function `__crc16_mutex_init()` creates and the SDK's own start-up calls. This image never called it, so the lock stayed "taken" and the controller task slept for ever. `ble_test_start` now calls it. Two more defects were fixed in the same change, because the first one would have exposed them: `ll_hci_adv_set_params` reads six bytes from its direct-address argument (it was NULL, which the bus guard is documented to fault on; now a zero array), and the context switch saved `r4`-`r9` while the compiler preserves `r4`-`r15` (now all of them, the list the compiler's own prologues use). None of the three has run on a pedal yet. + +The third flash froze: the screen was drawn but the controls were dead, and the pedal reset by watchdog after about 8 s (`LAST BOOT FAILED`, `RESET BY: WATCHDOG`, `LOOPS 0`). The scheduler is cooperative, so a vendor call that never returns holds the whole main loop. The scout report (data/openpocket-ble-hang-diagnosis/report.md) found two such places on the advertising path: a vendor assertion that fails because the empty stand-in for `low_power_get` returned a null handle (a failed assertion parks in `P33_SYSTEM_RESET`, an endless loop), and, earlier, an unbounded wait on baseband status bit 1 (`0x28038`) that the repository's own baseband header says can stay set. `low_power_get` now returns a non-null handle. The second wait is untested; if it is the one, the crash screen below names it. + +**Stall watch.** The 10 kHz timer interrupt counts while a `ble_poll()` runs. After 4 s it makes the next tick take the fatal path, so the pedal shows the normal crash screen with `VEC 3F` and the `PC` and `RETS` of the code that did not return (look them up in `firmware/build-ble/symbols.txt`; `PC 02000DC8` is a vendor assertion and `RETS` names it; a `PC` inside `ble_hw_disable` is the status-bit wait). It cannot fire while interrupts are off, and it watches only `ble_poll()`; the radio replay is excluded because it is long and its waits are bounded. + +**After a failed boot Bluetooth stays off.** The next boot shows `LAST BOOT FAILED` and runs the normal firmware (status `OFF SAFE nn%`), so the boot guard never counts two failures in a row and the pedal does not fall back to the loader. The flasher works in that state. The boot after that one tries Bluetooth again. + +## How it works + +``` +openpocket.c (OP_BLE_TEST) ble_test_start() after the screen is up, ble_test_poll() in the main loop +firmware/ble/ble_test.c a task that starts the controller and the host +firmware/ble/ble_midi.c the GATT table, the host's callbacks, pairing, advertising +firmware/px_ble_midi.h a BLE MIDI write -> USB-MIDI event packets (tests/midisim.c) +firmware/ble/ble_os.c the operating-system services the library calls (below) +firmware/ble/ble_rf.c the Wi-Fi front-end half of the radio start-up (below) +firmware/ble/ble_cfg.c configuration: LE only, advertising, one slave link, a GATT server +firmware/ble/ble_stubs.c empty stand-ins for the parts that are not built +third_party/jieli-ble/ JieLi's controller, as published (Apache-2.0): see its README.md +``` + +* **The controller is JieLi's**, from the Apache-2.0 SDK libraries (`btctrler.a`, 33 of its objects plus 3 helpers from + `cpu.a`): HCI front end, link layer, advertising, the LE radio driver. Only the objects the image reaches are linked. + The LE part of the host stack (`btstack.a`) runs as its own task; the classic profiles are stand-ins. + The OpenPocket code around it is small and separate from the vendor objects (`firmware/ble/`), which are kept + unmodified under `third_party/jieli-ble/` with the licence text and the source URL. +* **No RTOS.** The library expects tasks, queues, semaphores and timers. `ble_os.c` provides a cooperative scheduler: + the controller's one task runs from `ble_poll()` in the main loop, switches only where it waits (queue, semaphore, + delay), and is woken by the two BLE interrupts (IRQ 45 and 29, priority 2: the numbers and priority the stock firmware + uses). The heap is a 24 KB static pool; each task has a 4 KB stack. Against the normal image it needs about 44 KB more RAM (of which the heap + and stacks are 36 KB) and 131 KB more flash; the link fits with about 15 KB of the 96 KB RAM region free. +* **Radio start-up.** The library's `bt_rf_init` calls the Wi-Fi library's `wf_rf_common_init` first. Linking that would + pull in the Wi-Fi MAC/SDIO stack, so `ble_rf.c` runs the already committed capture of the stock start-up instead + (`firmware/bsp/chip/ble_rf_tables_v15.h`, run by `fm1_ble_rf.h`), with this unit's trims measured by `fm1_rf_cal.h`. + The BLE baseband itself is initialised by the library (`le_hw_init`), which makes the same register changes as the stock + firmware's baseband start-up in the analysis (bits 13 and 14 set, bit 4 cleared, IRQs 45 and 29 at priority 2). +* **Nothing is saved.** The library asks for stored items (`syscfg_read`): none are returned. It tries to store calibration + results (`syscfg_write`): the call is accepted and ignored. The VM (`0xA5000`) and BTIF (`0xBE000`) areas are not read + and not written by this image. + +## What is not verified + +On a real pedal the radio, the link, the GATT table, pairing with the Chocolate and the MIDI path have run. These have not: + +1. **Sound under a link.** No one has listened for crackle or measured the DSP load with a link up, at the Chocolate's 60 ms or a Mac's 30 ms. +2. **Long runs.** The longest link so far is minutes. The stock firmware's 3 s radio maintenance timer is not here, so a link may degrade as the pedal warms. +3. **Reconnection after a power cycle.** Keys are in RAM, so a central that keeps its bond finds a pedal that has forgotten it. +4. **Failed first connections.** A Mac's first attempts can end with reason `3E` before one holds; it retries by itself. The cause is not known. +5. **Flash writes while Bluetooth runs.** The library's interrupt code runs from flash (execute in place). Saving a preset during a link could fetch code while the flash is busy. Do not save presets while testing. +6. **Regulation.** The pedal was certified with the stock firmware. This image uses the standard channels, the 1M PHY and the library's own power table; it is a hobby experiment, not a product. + +## Recovery + +If the image does not start, the existing safety net applies: two failed boots go back to the update loader +(`UBOOT`), and the web installer, the SD-card recovery (`make appimage`) and the official firmware's own updater all work as +before. The image does not touch the flash outside the application area. + +## Found on hardware: the radio was silent for two reasons (fixed) + +The fifth flash showed `ADVERTISING` and event interrupts at the advertising rate, but no scanner saw the pedal. The cause was in our glue, found by comparing the stock BlackBox_016 start-up with this image: + +1. The library writes the modem's transmit I/Q trims (`keep_BT_CMPUK1/2/3`, `keep_BT_OFSI`, `keep_BT_OFSQ`, which stock fills from the Wi-Fi front-end calibration) into `0x2FC10` and `0x2FC08`. Here nothing set them, so the I and Q gains were zero. `ble_rf.c` now sets the neutral values (`0x80` for the two gains). +2. `wf_rf_common_init` was `void`, so the library read a stray register value as its result and switched on the Wi-Fi/Bluetooth coexistence arbiter (`0x14000` bit 1). It now returns 0, as stock does on a first start. + +After the fix a Mac scan saw `OpenPocket-BLE-TEST` with the BLE-MIDI service UUID (206 sightings in 60 s, RSSI -51 to -36 dBm, not connectable) and the scan-request counter `r` rose. The stored radio records are also read now: the VM is plain flash (the XIP window decrypts it, so it must be read raw, `ble_vm.h`), VM 106 to 108 go to the replay, and VM 187 (the unit's calibration) replaces the measured one. The status line shows them as `V ` (`V F BD B0B` on the test pedal). `bt_max_pwr_set(10, 5, 8, 9)` is called after the controller's own init. Which of these beyond the two fixes matters was not isolated. `t` and `e` stay 0 by design here (no software timer on the advertising path, no host packet handler), so they are not health signs. + +## Found on hardware: the link + +Connectable advertising needed the real link-layer members (`RF_ble5`, `multilink_schedule`, `ll_data_extension`, `ll_multiple_phys`) in place of their stand-ins. Two more faults showed up on the pedal: + +1. `dev_hci_set_event_mask` was a stand-in, so the host never received LE meta events: a central connected and the pedal looked idle, with `i` at 16.7 a second and no advertisement on the air. The real one is linked now. +2. After a disconnection the advertisement must be set up again, as the SDK does. Enabling it alone left the pedal nearly silent. diff --git a/docs/BUILDING.md b/docs/BUILDING.md index de42e5f..e9dd366 100644 --- a/docs/BUILDING.md +++ b/docs/BUILDING.md @@ -28,21 +28,41 @@ Apple silicon and other ARM machines Docker emulates it (Rosetta or QEMU); you c | --- | --- | | `make doctor` | check Node, Docker and the image, one line each (exit 1 when the build cannot work) | | `make build` | `firmware/build.sh` in the container: `firmware/openpocket.bin`, the update loader, `docs/firmware/PocketX_9NN.fwsc`, `BlackBox_9NN.fwsc`, `manifest.json` | +| `make ble` | the Bluetooth MIDI test image (`OP_BLE=1 firmware/build.sh`): `firmware/openpocket-ble.bin`, `build/ble/*.fwsc`. Own target, own output folders; [BLE-TEST.md](BLE-TEST.md) | +| `make serve-ble` | `make serve` with `build/ble/` as the installer's package folder, to flash the test image | | `make package` | `build`, then `build/SHA256SUMS` | +| `make package-test` | the MIDI test channel (`OP_CHANNEL=midi-test`): `firmware/openpocket-midi-test.bin` and `docs/firmware/midi-test/` (v37 MIDI TEST, its own `manifest.json`), with `build/SHA256SUMS-midi-test`. The release files are not touched and the installer offers it only with `?channel=midi-test` | | `make test` | `tests/run_tests.sh` (engine, preset store, screen, simulated pedal, installer, editor; the browser page tests also run when Playwright is installed) | | `make clean` | remove every build output (on the host; needs neither Docker nor the image) | | `make shell` | a shell in the container (as your user) | | `make serve` | the installer and editor on `http://localhost:8765/` (offline install; needs Python 3 on the host) | +| `make flash ARGS="--dry-run"` | write a built package to a connected pedal over USB MIDI, no browser (`--confirm-flash` really writes; see below) | +| `make sim [ARGS="--port 9000"]` | the simulated pedal on `http://127.0.0.1:8766/`: screen, controls, state, an HTTP API, the installer and editor pages on a virtual MIDI port (`docs/SIMULATOR.md`; needs a C compiler on the host, no Docker) | +| `make log ARGS="--follow"` | read a connected pedal's state over USB MIDI, with the fields explained and a verdict (`docs/BLE-TEST.md`, "Reading the pedal from a computer") | | `make render IN=guitar.wav [OUT=..] [PRESET=1]` | a DI recording through the engine | | `make appimage DUMP=fm1-1.bin` | `fm1-300.bin` for the SD-card recovery | +| `make rescue` / `make rescue-write` | restore a looping BlackBox from its UBOOT state from a Mac (read-only first; asks for sudo; `ARGS="--package FILE"` for options): `docs/RESCUE.md` | +| `make rescue-check` | the UBOOT rescue tool (`tools/uboot_rescue.py`, see `docs/RESCUE.md`) against a simulated pedal; needs only Python 3 | `make NATIVE=1 build` skips Docker and uses a toolchain installed on the machine (`sh firmware/setup.sh` installs it in `~/.jieli`; Linux x86-64 only). `tests/run_tests.sh` also runs directly on macOS and Linux with only a C compiler, Node and -Python 3. +Python 3. Its NAM check (`tests/test_nam.py`, the fixed-point network against the float reference) also needs NumPy +and is skipped without it (the container has none). + +The NAM factory models (`engine/px_nam_factory.h`, v37) are generated, not hand-written: `tools/train_a2.py` trains each +A2-Lite network from a capture (needs `pip install jax optax numpy scipy`, about 15 minutes a model on two CPU cores), +then `tools/mk_nam_factory.py LIST.txt DIR docs/nam_di.wav` writes the header with each model's level trim +([NAM.md](NAM.md)). The `.bat` files `makeimage.bat` and `render.bat` only check that Node is installed and call `tools/op.mjs`. `build.bat` does the same when Node and Docker are there, and otherwise builds in WSL (`build.bat docker` or `build.bat wsl` forces one way). `INSTALL-OFFLINE.bat` needs nothing installed: it serves `docs\` with PowerShell (`tools/serve_offline.ps1`), and falls back to Python (`tools/serve.py`) or WSL only if PowerShell cannot open a port. `tests/test_op.mjs` (part of `make test`) checks the Docker command lines, the friendly errors and `make clean` without Docker, using a fake `docker`. `PUBLISH-GITHUB.bat` and `PUBLISH-SOURCE.bat` (PowerShell `tools/publish_github.ps1`) are the maintainer's release tool and still need Windows; nothing else does. +## Flashing from the command line + +`make flash ARGS="--dry-run"` (or `node tools/op.mjs flash --dry-run`) finds the pedal over USB MIDI and checks the built package without writing; `--confirm-flash` instead of `--dry-run` writes it. It runs on the host (not in the container), speaks the same update protocol as the installer page (`docs/pxota.js`, shared with the page), and checks the package against `docs/firmware/manifest.json` (size, SHA-256, identity), refuses the wrong model, and verifies the version the pedal reports afterwards. `--package FILE --manifest FILE` flash a package built elsewhere. Install its one dependency once with `npm ci --prefix tools`. The rules for agents (dry run first, never flash without the captain's explicit instruction, recovery) are in `.claude/skills/openpocket-flash/SKILL.md`. On failure the command prints the SD-card recovery pointer (README, `flash/`). `tests/test_flash.mjs` runs it against the simulated pedal; no test touches real hardware. + +**A pedal that restarts in a loop.** `make flash-rescue ARGS="--confirm-flash"` (or `node tools/op.mjs flash --rescue --confirm-flash [--package FILE] [--timeout SECONDS]`) is the installer page's rescue mode on the command line, and uses the same code (`Updater.rescueWithin` in `docs/pxota.js`). Start it, then plug the pedal in or let it restart. It asks every MIDI port for the pedal's identity every 100 ms; the moment one answers, it stops the pedal's sound processing (which ends the restarts) and installs the package of that pedal's model, printing `pedal answered: ...` and one line per step. If the pedal disappears before the update finished, nothing has been written yet: it waits for the pedal again, and an OpenPocket update loader left behind is finished when `--package` names the package (a loader does not report its model; without `--package` the timeout message says so). It waits 180 s by default (`--timeout`); an install that has started is never cut short. `--rescue --dry-run` only waits for the pedal and checks the model. The same refusals apply (no `--confirm-flash`, wrong model, damaged package). Only a pedal whose firmware answers the identity query for a moment is caught, as on the page; a pedal that crashes before it answers, or sits in the chip's mask-ROM boot loader (`WL80UBOOT`, USB 4C4A:8057, no MIDI), is out of reach and needs the SD-card recovery. Connect only one pedal: the first to answer is used. + ## What is pinned - The base image `debian:stable-slim`, by digest (`Dockerfile`). @@ -57,7 +77,7 @@ old one. The toolchain is JieLi's proprietary software. It is **not** in this repository and is not redistributed by it: the image build downloads it from JieLi's own server (`jl-update.oss-cn-shenzhen.aliyuncs.com`, the file `https://pkgman.jieliapp.com/s/linux-toolchain` redirects to). If you publish a container image, do not include the toolchain layer. The Apache-2.0 SDK parts in -`firmware/sdk/` are covered by `NOTICE.md`. +`firmware/sdk/` and `third_party/jieli-ble/` are covered by `NOTICE.md`. ## Reproducibility diff --git a/docs/EDITOR-PROTOCOL.md b/docs/EDITOR-PROTOCOL.md index 1a7a9c6..c589d94 100644 --- a/docs/EDITOR-PROTOCOL.md +++ b/docs/EDITOR-PROTOCOL.md @@ -1,7 +1,8 @@ # OpenPocket editor protocol (SysEx over USB-MIDI) The web editor (`docs/editor.html`, `docs/pxedit.js`) talks to the pedal with this protocol. The firmware side is -`firmware/px_editor.h`; `tests/edsim.c` runs the same code on a PC for the tests. Protocol version 1 (OpenPocket v32). +`firmware/px_editor.h`; `tests/edsim.c` runs the same code on a PC for the tests. Protocol version 2 (OpenPocket v37; version 1: v32 to v36, +the same without the NAM commands 19..22). ## Framing @@ -42,8 +43,41 @@ The 12 blocks, by index: 0 GATE, 1 COMP, 2 PITCH, 3 SFX, 4 DRIVE, 5 AMP, 6 CAB, | 16 SAVE | slot | slot, rc. Saves the current sound into the slot, writes the flash, the slot becomes the current one (as the pedal's SAVE) | | 17 WATCH | on (0/1) | on. While on, the pedal pushes `STATE` (cmd 5) when the sound, the slot or the focused block changes on the pedal; it stops by itself 5 s after the last request | | 18 PING | — | DSP load % (0..127), flags (bit 0: bypass, bit 1: second core) | +| 19 NAM_LIST | — | factory models (12), user slots (16), then per model (factory first, then the user slots): present (0/1), name string (11 characters at most; an empty user slot reads "U5 EMPTY") | +| 20 NAM_WRITE | piece (2 bytes: bits 0-6, 7-13), 128 packed bytes | the 2 piece bytes, rc (0 ok, 1 invalid). Piece i holds bytes 112 i .. 112 i + 111 of a model (the last piece fewer), packed 7 bit: groups of a byte of high bits (bit j = bit 7 of byte j) then 7 bytes. Stored in RAM only | +| 21 NAM_COMMIT | slot (0..15) | slot, rc (0 ok, 1 invalid: header, a weight that is not a number, CRC; 2 flash). Checks the pieces received and writes them to the user slot | +| 22 NAM_ERASE | slot (0..15) | slot, rc (0 ok, 1 invalid, 2 flash). Empties the user slot | +| 23 DIAG | — | n (5), then n strings: this boot, the last boot, the status line, the link, the trace. Read only. Only the Bluetooth test image answers: other firmware gives no reply. `docs/BLE-TEST.md`, "Reading the pedal from a computer" | -Writing the flash mutes the sound for a moment, as saving on the pedal does. A whole import sends `SLOT_PUT` without +### NAM models (protocol 2) + +The AMP block's ninth type, `NAM`, plays a neural amp model (NAM A2, its 3-channel "Lite" size, 1871 weights: +`engine/px_nam.h`). Its first parameter, MODEL, picks one of 28 models: 0..11 the factory ones (in the firmware), +12..27 the 16 user slots in the pedal's flash (`firmware/px_models.h`, 0xC0000..0xE0000, kept by firmware updates). +`TEXTS` of AMP type 8, parameter 0 gives each value's model name; the editor asks for it again after a change of +the user slots. + +A user model is 7548 bytes, little endian: a 64-byte header, then the 1871 weights as float32 in the `.nam` +file's order. + +| Offset | Size | Field | +| --- | --- | --- | +| 0 | 4 | magic `0x4D4E504F` ("OPNM") | +| 4 | 2 | version, 1 | +| 6 | 2 | weights, 1871 | +| 8 | 12 | name, ASCII, 0-ended (11 characters at most) | +| 20 | 2 | level trim, int16, 0.01 dB (added to the model's output) | +| 22 | 2 | CRC16 of the weights (polynomial 0x1021, initial value 0) | +| 24 | 40 | 0 | + +The editor (`docs/pxnam.js`) reads a `.nam` file, takes its A2-Lite part (an A2 file is a "SlimmableContainer" +with several sizes; a plain A2-Lite WaveNet also works), measures its loudness on `docs/nam_di.wav` and sets the +level trim so that every model plays at -20 LUFS on that clip, then sends 68 `NAM_WRITE` and one `NAM_COMMIT`. +A1 models (the older "standard" / "lite" / "feather" WaveNets) and other architectures are refused: the pedal's CPU +runs A2-Lite only. + +Writing the flash (`SLOT_PUT` with bit 1, `COMMIT`, `SAVE`, `NAM_COMMIT`, `NAM_ERASE`) mutes the sound for a moment, as +saving on the pedal does. A whole import sends `SLOT_PUT` without bit 1 for every slot, then one `COMMIT`. ## Preset files diff --git a/docs/MIDI-TEST.md b/docs/MIDI-TEST.md new file mode 100644 index 0000000..28d5809 --- /dev/null +++ b/docs/MIDI-TEST.md @@ -0,0 +1,112 @@ +# MIDI over USB-C: hardware test plan + +For the first run of **OpenPocket v37 MIDI TEST** on a real pedal. Nothing here was tried on hardware yet: the pedal is the only +way to check the USB behaviour and the sound. The design is in [MIDI.md](MIDI.md). Takes about 45 minutes. Tick as you go and +send back the "report" block at the end. + +**Do this on one pedal first.** The Donner Pocket X is the better first choice: the M-VAVE BlackBox install is still being +tested (issue #1: a restart loop after power cycles in some versions, rescue mode on the installer page is the cure). + +## 0. Before you start + +- [ ] Know your way back (section 7). Have your official firmware file for the pedal at hand, and the SD-card recovery + (`flash/`, `fm1-1.bin`) if you made one. +- [ ] Chrome or Edge on a computer, a USB **data** cable, the pedal on and charged. +- [ ] A MIDI monitor: macOS: Audio MIDI Setup (Window > Show MIDI Studio) and the free MIDI Monitor; Windows: MIDI-OX or + Bome MIDI Translator; Linux: `aseqdump -l` then `aseqdump -p "OpenPocket"`. +- [ ] A DAW or a MIDI controller that can send Program Change and Control Change (Ableton, Logic, Reaper, GarageBand with a + MIDI app, or a free web tool such as the Web MIDI Monitor and sender pages). +- [ ] The package: a build of the pull request gives `docs/firmware/midi-test/` (`node tools/op.mjs package-test`), or the CI + artifact. Serve it with `node tools/op.mjs serve` (offline) and open `http://localhost:8765/index.html?channel=midi-test`, + or use the page you were given. + +## 1. Install the test build + +- [ ] Open the installer with `?channel=midi-test`. A warning banner says "TEST CHANNEL" and the package line says `v37 (midi-test)`. +- [ ] Connect the pedal, check that it is found, press Install, wait for "Done". Do not unplug while it works. +- [ ] The pedal restarts. The boot screen shows **v37 MIDI TEST**. +- [ ] The sound is normal: play a few presets, each block of one preset, the tuner (hold HOME). Note the number at the bottom + right ("C1 42%"): baseline DSP load with the same preset: ______ %. +- [ ] Open the web editor (`editor.html`): it connects, shows the presets, and a knob turned on the pedal moves on the page. + +## 2. The MIDI page + +- [ ] Hold FX for about a second: the **MIDI** page opens. Rows: CHANNEL OMNI, SEND ON, CLOCK > DELAY OFF. "USB: NO HOST" + or "HOST CONNECTED" is right. +- [ ] A short press of FX (not held) still cycles the effects (DRV, SFX, SYN...). It acts when you let go. +- [ ] Knob changes the row, PUSH goes to the next row, after the last row it leaves. HOME leaves too. +- [ ] Change CHANNEL to 3, leave. The screen shows "SAVING..." briefly, the sound mutes for a fraction of a second. Power + the pedal off and on: the page still says 3. + +Set it back to OMNI for the next steps. + +## 3. Over USB-C: the pedal receives + +Plug the pedal into the computer (the pedal should appear as a MIDI port named **OpenPocket**) and send: + +- [ ] **Program Change 0, 1, 2, 11**: the pedal shows preset 01, 02, 03, 12 each time, the sound follows. Program Change 60 does nothing. +- [ ] **CC 25 = 0 / 127** (AMP off / on): the AMP tile on the screen goes dark and back, the sound loses and gets its amp. +- [ ] **CC 77 sweep 0 to 127** (AMP setting 1, GAIN): the gain bar moves and the sound changes smoothly. No crackle, no + glitch, no mute. Also try CC 102 to 105 (DELAY settings), CC 106 (REVERB). +- [ ] **CC 57 = 0, 64, 127** (AMP model): the model changes (first, middle, last). +- [ ] **CC 7 sweep** (volume): the VOL number at the top follows. +- [ ] The MIDI page's "LAST:" line shows what you just sent (open it with FX held; it is read-only for the host while open). +- [ ] CHANNEL 3: a message on channel 1 does nothing, on channel 3 works. Back to OMNI. +- [ ] While the **SAVE page** (hold PUSH) is open, a Program Change from the computer is ignored. HOME to leave. +- [ ] **Clock:** CLOCK > DELAY ON, select a preset with the delay on (press DLY), start the DAW playing at 120 BPM: the delay + TIME shows about 500 ms (the nearest step is 498 ms) and follows a change of tempo to 90 BPM (about 670 ms). With + CLOCK > DELAY OFF nothing changes. + +## 4. Over USB-C: the pedal sends + +With the MIDI monitor listening to the pedal's port and SEND ON: + +- [ ] Footswitch up / down: one **Program Change** per press, with the slot number minus 1. +- [ ] Turn the knob on a block's page: **Control Change** messages with the numbers of [MIDI.md](MIDI.md). Switch a block: 127 / 0. +- [ ] Nothing is sent at power-on, and nothing when the computer was unplugged and is plugged back in. +- [ ] SEND OFF: the pedal sends nothing; it still receives. +- [ ] A change sent **to** the pedal is not sent back (watch the monitor while moving a DAW fader: no echo). +- [ ] Loop test: a DAW track that echoes everything it receives (MIDI through) back to the pedal does not make it chatter. + +## 5. Audio with MIDI flowing + +- [ ] Play a high-gain preset continuously for 5 minutes while the DAW sends a CC sweep and clock: no crackle, no dropout, no + mute, no "BYP" at the bottom right. Note the load: ______ % (baseline above). A rise of more than a few percent is worth reporting. +- [ ] Same with a long note flood (a keyboard or a DAW sending 100 notes a second): the pedal ignores the notes and the sound + stays clean. +- [ ] While the flood runs, use the **web editor** (change a preset, rename, save): it keeps working. (Before this release a + busy MIDI port could stop the editor.) + +## 6. The update path still works + +- [ ] On the normal installer page (without `?channel=`), Connect: the pedal is found and shows `OpenPocket v37`. Press Install + (this puts **v36** back) and wait for "Done": the boot screen shows v36, the sound is normal. +- [ ] Open the test page again and install v37 MIDI TEST once more: it also works in this direction. +- [ ] The editor still connects on both versions. + +## 7. Way back and recovery + +1. **To the previous OpenPocket:** the normal installer page (no `?channel=`), Install (v36). +2. **To the official firmware:** the installer page, step 3 "Back to the official firmware", with your official `.fwsc`. +3. **A pedal that restarts in a loop:** the installer's rescue mode (step 2), plug the pedal in; it installs by itself. +4. **A pedal that does not start at all:** the SD-card recovery (`flash/`, `makeimage.bat` / `node tools/op.mjs appimage`). + +This build does not change the boot, update-entry or recovery code, so all four work as for v36. + +## 8. Report back + +Please send, even if everything works: + +``` +Pedal: Pocket X / BlackBox Computer + OS: +Installed v37 MIDI TEST: yes/no Boot screen showed v37 MIDI TEST: yes/no +USB port name seen: Apps tried (DAW, monitor): +Section 2 (page): ok / what failed +Section 3 (receive): ok / what failed (which CC or PC numbers) +Section 4 (send): ok / what failed +Section 5: DSP load baseline ___ % with MIDI ___ %, any crackle / dropout / BYP: +Section 6 (update, both ways): ok / what failed +Anything strange on screen or in sound: +``` + +For a failure, add what the MIDI page's "LAST:" line showed and the exact bytes your monitor showed. A photo of the screen helps. diff --git a/docs/MIDI.md b/docs/MIDI.md new file mode 100644 index 0000000..991159f --- /dev/null +++ b/docs/MIDI.md @@ -0,0 +1,191 @@ +# OpenPocket MIDI over USB-C + +OpenPocket answers MIDI Program Change and Control Change from a computer, a DAW or a controller over the USB-C cable, +and tells the computer when you change the sound on the pedal. This page is the design and the reference. The hardware +checklist for the first test is [MIDI-TEST.md](MIDI-TEST.md). + +**Status.** Written and tested on a PC (host simulators, see "How it was checked"). Not yet tried on a real pedal. It ships +as a separate test package (OpenPocket v37 MIDI TEST), not in the normal installer. **Bluetooth MIDI is only in the separate +Bluetooth test image** ([BLE-TEST.md](BLE-TEST.md)). + +## What the pedal does + +**Receives** (USB, from the host): + +| Message | What happens | +| --- | --- | +| Program Change 0 to 51 | Loads preset slot 1 to 52, as the footswitches do (unsaved changes are dropped). 52 and above are ignored | +| Control Change, the map below | Switches or flips a block on or off, picks a block's model, sets one of its 5 settings, or sets the MASTER volume | +| MIDI clock (24 per quarter note), when CLOCK > DELAY is on | Sets the delay TIME to one quarter note | +| Start, Continue, Stop | Restart the tempo measurement | +| 64 (sustain) | With a SYN PAD on in the preset: 64 to 127 catches the chord being played (as the left footswitch); 0 to 63 nothing | +| Everything else | Ignored: notes, pitch bend, bank select (0 and 32), modulation, "all notes off", system messages | + +**Sends** (when SEND is on and a host is connected): + +| What you do on the pedal | What the computer receives | +| --- | --- | +| Change the preset (footswitch, knob, SAVE to another slot) | One Program Change with the slot number (0 to 51) | +| Turn a setting, switch a block, pick a model | One Control Change per change, with the value from the map | + +A change made by the host is not sent back. Changes made from the web editor are sent like changes made on the pedal. The +sound the pedal starts on is not sent at power-on. Nothing is sent while no host is connected, and nothing old is sent +when one connects. + +**Channel.** CHANNEL is OMNI (receive on any channel; send on channel 1) or 1 to 16 (receive and send on that channel only). +The clock and the transport bytes have no channel and are always taken. + +## The Control Change map + +Value scaling: a setting is 0 to 100 on the pedal and 0 to 127 in MIDI (`v * 127 / 100`, rounded; the way back is exact). +On or off: 64 and above is on, below 64 is off; the pedal sends 127 or 0. A model number n of N models is the slice +`n * 128 / N` to `(n + 1) * 128 / N - 1` of 0 to 127; the pedal sends the middle of the slice. Send the highest value to get the +last model. + +| Block | On / off | Model | Setting 1 | 2 | 3 | 4 | 5 | +| --- | --- | --- | --- | --- | --- | --- | --- | +| GATE | 20 | 52 | 33 | 34 | 35 | 36 | 37 | +| COMP | 21 | 53 | 39 | 40 | 41 | 42 | 43 | +| PITCH | 22 | 54 | 44 | 45 | 46 | 47 | 48 | +| SFX | 23 | 55 | 49 | 50 | 51 | 70 | 71 | +| DRIVE | 24 | 56 | 72 | 73 | 74 | 75 | 76 | +| AMP | 25 | 57 | 77 | 78 | 79 | 80 | 81 | +| CAB | 26 | 58 | 82 | 83 | 84 | 85 | 86 | +| EQ | 27 | 59 | 87 | 88 | 89 | 90 | 91 | +| MOD | 28 | 60 | 92 | 93 | 94 | 95 | 96 | +| DELAY | 29 | 61 | 97 | 102 | 103 | 104 | 105 | +| REVERB | 30 | 62 | 106 | 107 | 108 | 109 | 110 | +| SYN | 31 | 63 | 111 | 112 | 113 | 114 | 115 | + +**Toggles (footswitches).** CC 8 to 19 flip a block with **value 0** and ignore every other value, so a footswitch that sends +127 then 0 for a tap flips it once. The pedal holds the state, so a preset change cannot put the footswitch out of step, as +it can with the on/off numbers above. + +| CC | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| Block | GATE | COMP | PITCH | SFX | DRIVE | AMP | CAB | EQ | MOD | DELAY | REVERB | SYN | + +The order is the order of the blocks above: CC 8 + n flips what CC 20 + n switches. The pedal never sends these numbers: it reports a +block with its on/off number. + +CC **7** (volume) sets the MASTER volume, 0 to 127 to 0 to 100, like the MASTER knob (turning the knob takes over again). + +What the "setting" numbers mean depends on the model (the screen shows the names): for the AMP they are GAIN, BASS, MID, +TREBLE, PRESENCE; setting 3 of GATE is the preset LEVEL (50 = 0 dB); setting 1 of DELAY is TIME. Only the settings the model +has do anything; the others are stored and ignored. See [SOUNDS-AND-CONTROLS.md](SOUNDS-AND-CONTROLS.md) and +[EDITOR-PROTOCOL.md](EDITOR-PROTOCOL.md). + +The numbers were chosen so that nothing special is touched: 0 and 32 (bank), 6 and 38 (data entry), 64 to 69 (pedals), 98 to 101 +(NRPN and RPN) and 120 to 127 (channel mode, "all notes off") are never used, and 7 is the only standard one. A few settings +use 33 to 51, which are the low halves of controllers 1 to 19; a controller that sends 14-bit pairs on those would move them +by accident. Map them to one of the free numbers in your DAW instead. + +The map is in one place in the source, `px_cc_param()` and its siblings in `firmware/px_midi.h`; a test checks that every +number is unique, reserved ones are free, and a value sent by one pedal is applied by another to the same sound. + +## Tempo for the delay + +Turn **CLOCK > DELAY** on (default off). When the host sends MIDI clock, the pedal times 24 clocks, and sets the delay TIME to +the quarter note once two measurements in a row agree within 3 ms (so a drifting host does not make the delay jump). Quarter +notes shorter than 200 ms or longer than 2 s (above 300 or below 30 BPM) are ignored, and the TIME knob only reaches 2 s. The +nearest TIME step is chosen: 12.25 ms steps up to 1 s, 50 ms steps above it. REVERSE has its own range (250 ms to 1 s). To +follow a tempo with another note value, set the delay's own TIME; this is not a division setting. + +## Settings and the screen + +**Hold the FX button for 0.8 s** (a short press still cycles the effects, now when you release it) to open the **MIDI page**: + +| Row | Values | Default | +| --- | --- | --- | +| CHANNEL | OMNI, 1 to 16 | OMNI | +| SEND | ON, OFF | ON | +| CLOCK > DELAY | ON, OFF | OFF | + +The knob changes the row, PUSH goes to the next one (after the last it leaves), HOME leaves. The page also shows whether a +host is connected and the last Program or Control Change received ("LAST: CC 25 = 127"), which is what to look at first when +nothing happens. Leaving saves the settings only if you changed something (one flash write, about 0.3 s of silence, like SAVE). + +The settings are one byte in the header of the preset store (`firmware/px_store.h`). Older copies have 0 there, which means the +defaults. A power cut while saving leaves the old or the new settings, never a broken store (tested). + +While the SAVE page, the MIDI page, the tuner or the block mover is open, the host's Program Change and Control Change are read +and ignored, so a DAW cannot change the preset under a save in progress. + +## How it is wired, and what stays separate + +The USB device is felucca-bsp's `chip/usb.c` as before, **unchanged**: a class-compliant USB-MIDI interface (no driver on macOS, +Windows 10 and later, Linux, iOS or Android; the port is named "OpenPocket"). It is polled at 2 kHz from a timer interrupt. This +work does not touch the descriptors, `ota.c`, the loader or the boot, update-entry and recovery paths. + +There are two kinds of traffic on the one cable, and they never meet: + +1. **SysEx** (`F0 ... F7`) belongs to the update and the editor only: the installer's update protocol (`ota.c`), the editor's + frames `F0 7D 4F 50 ...` ([EDITOR-PROTOCOL.md](EDITOR-PROTOCOL.md)) and the rescue key. `usb.c` collects SysEx into one frame + buffer for them. A SysEx that is not theirs (a General MIDI reset from a DAW, a patch dump) is taken out of that buffer and + thrown away by `ota_take`; a SysEx longer than the buffer is dropped. A test sends a foreign SysEx to the simulated pedal + before an update, and the update still works. +2. **Musical messages** (Program Change, Control Change, clock and transport) go into a 64-entry ring that `usb.c` fills and + `firmware/px_midi_io.h` empties every pass of the main loop. Nothing sends SysEx into that ring and nothing in it is SysEx. + +One fix came with this: until now nothing emptied that ring. `usb.c` leaves the host's USB packet unread while the ring is nearly +full, so a DAW sending clock or notes would, after a few dozen messages, also have stopped the editor's and the installer's SysEx, +which arrive in the same packets. The ring is now always drained (messages the pedal does not use are dropped), and the +overflow flag `usb.c` sets is cleared once the ring is empty. A test floods the ring and checks it recovers. + +Code map: `engine/px_dsp.c` has `px_dly_value_for_ms` (the TIME value nearest a delay time); `firmware/px_midi.h` is the pure +logic (decoder, CC map, what to send, settings byte, clock), with no hardware; `firmware/px_midi_io.h` is the glue to the rings +and the pedal's state; `firmware/ui.h` has the MIDI page; `firmware/px_store.h` keeps the settings byte. + +## Cost and safety + +- **CPU and audio.** The MIDI code runs on the main loop (CPU0), not in the audio interrupt and not on the DSP core. Per pass it + reads the ring (usually empty) and compares the 84 on/model/setting bytes of the preset with the last values sent. The binary + grows by about 3.3 KB. The DSP load was **not measured on a pedal** (no hardware here): the hardware checklist asks for the + "C1 xx%" reading with and without MIDI traffic. Nothing was added to the audio path. +- **Flash.** The only new write is the settings save, in the preset store's own area, through the existing commit. +- **Update and recovery paths.** Unchanged. The tests of the installer, the update loader and the editor run as before. +- **Never flash from here.** The test package is built and checked on a PC; the captain installs it. + +## Bluetooth MIDI + +Bluetooth MIDI is not in this release. The Bluetooth test image (`make ble`, [BLE-TEST.md](BLE-TEST.md)) is a connectable BLE MIDI +peripheral on JieLi's Apache-2.0 controller and host libraries. A write to the characteristic goes through +`firmware/px_ble_midi.h` into the same `px_midi_decode` and `px_midi_in` as USB, so Program Change, Control Change, the clock and the +page guards mean the same on both roads. It only receives: the pedal's own changes are not sent over Bluetooth. + +Before this can leave the test image it needs a setting to switch the radio off (saved in the store; default off, so a pedal that +never uses it pays nothing), a measurement of the DSP load and glitches with a link up, and bonds that survive a power cycle. The +transmit power must not exceed what the stock firmware uses. + +## How it was checked (and what was not) + +Run `node tools/op.mjs test` (see [BUILDING.md](BUILDING.md)): + +- `tests/midisim.c`: the packet decoder (channel filter, bad status, bad data bytes, SysEx never decoded, cable number), the CC + map (unique, reserved numbers free, exact inverse, value scaling round trips), what the pedal sends (Program Change, one + Control Change per change, no echo, SEND off), the settings byte, MIDI clock (120 BPM, jitter, gaps, a changing tempo), and + that a sound sent by one pedal is applied unchanged by another. +- `tests/midiio.c`: the real glue code against the ring: Program Change and Control Change change the sound, are ignored while + SAVE, tuner or the MIDI page is open, channel filtering, a flood is drained and recovers, clock sync on and off. +- `tests/test_store.c`: the settings survive a power cycle, a preset save and a power cut at every step. +- `tests/uisim.c`: the MIDI page is drawn completely, the channel wraps, leaving saves only if changed. +- `tests/test_installer.mjs`: an update still succeeds after a foreign SysEx. +- `tests/test_op.mjs`, `tests/test_page.mjs`: the test package command and the installer's test channel. + +**Not checked, because it needs the pedal:** the USB enumeration and the port name on each OS, a real DAW or app sending, the +timing of replies, the DSP load and any dropout with MIDI flowing, the real screen, the update path on the new build, power +cycles. That is what [MIDI-TEST.md](MIDI-TEST.md) is for. + +## The test package + +`node tools/op.mjs package-test` (or `make package-test`) builds **OpenPocket v37 MIDI TEST** into `docs/firmware/midi-test/` +(`PocketX_937.fwsc`, `BlackBox_937.fwsc`, its own `manifest.json`) and `firmware/openpocket-midi-test.bin`. The boot screen and +the editor say `v37 MIDI TEST`. The normal files (`docs/firmware/manifest.json`, v36) are not touched, and the normal installer +page never offers it. To install it, open the installer with `?channel=midi-test` (a banner says it is a test build). To go back, +open the normal page and press Install (v36), or use the official-firmware way back on the same page. + +## Licences and references + +The USB code is felucca-bsp's (MPL-2.0, see [NOTICE.md](../NOTICE.md)); the MIDI code here is GPL-3.0-only like the rest of +OpenPocket. The CC map is OpenPocket's own. Facts about the official firmware come from a private analysis of the captain's +own device, used only to learn how it behaves, and no vendor code, bytes or tables are reproduced. diff --git a/docs/NAM.md b/docs/NAM.md new file mode 100644 index 0000000..9f4ea72 --- /dev/null +++ b/docs/NAM.md @@ -0,0 +1,125 @@ +# Neural amp models (NAM A2) on the pedal (v37) + +Since v37 the AMP block has a ninth type, **NAM**, that plays a [Neural Amp Modeler](https://www.neuralampmodeler.com/) +network instead of an amp built from circuit stages. It plays the networks of NAM's **A2** architecture, at its +3-channel size, **A2-Lite**. That is the size the pedal's CPU can run. + +## The knobs + +| Knob | What it does | +| --- | --- | +| MODEL | which network: the 12 factory models, then the 16 user slots (`U1 EMPTY` ... until you load one) | +| GAIN | the level into the network, ±12 dB around the level of the captures (noon) | +| BASS, MID, TREBLE | an EQ after the network: ±10 dB at 120 Hz, ±8 dB at 800 Hz, ±10 dB above 2.5 kHz | + +The models are amp captures without a cabinet, so keep a CAB after the AMP. Every model plays at the same loudness +(-20 LUFS on the reference clip `docs/nam_di.wav`), so switching models keeps the volume. + +## Factory models + +Twelve A2-Lite networks, trained for the pedal at 44.1 kHz (`tools/train_a2.py`). Each one learnt to sound like a +community capture of a real amp, playing real DI guitar. Accuracy is ESR, the error on 20 s of guitar the training never +heard: 0.01 is very close, and below about 0.05 the two are hard to tell apart. + +| MODEL | Amp | Capture (author) | ESR | +| --- | --- | --- | --- | +| TWIN | Fender Twin Reverb (normal, bright) | Tim R Fender TwinVerb Norm Bright (Tim R) | 0.006 | +| AC15 | Vox AC15 Top Boost | Phillipe P VOXAC15-TopBoost (Phillipe P) | 0.045 | +| SUPER 59 | Magnatone Super 59 Mk II | Tim R Magnatone Super 59 Mkii Ch1 Br (Tim R) | 0.009 | +| SHIRL CL | Friedman Dirty Shirley Mini, clean | Sascha S DirtyShirleyMini_Clean_B1_M6_T7_MV10_G4 (Sascha S) | 0.014 | +| JCM900 | Marshall JCM900 50 W Dual Verb | Tim R JCM90050WDualVerbCHAG6 (Tim R) | 0.011 | +| SHIRLEY | Friedman Dirty Shirley Mini, crunch | Sascha S DirtyShirleyMini_crunch_G6 (Sascha S) | 0.033 | +| JCM2000 | Marshall JCM2000 DSL, lead | Tim R JCM2000 L2 G6 (Tim R) | 0.042 | +| MARK IV | Mesa Boogie Mark IV | Roman A LT_MESA_MARKIV_1 (Roman A) | 0.041 | +| JVM OD2 | Marshall JVM, OD2 | Phillipe P JVM-OD2-RD-NoDrive-ESR0,003 (Phillipe P) | 0.056 | +| PRO MOD | Splawn Pro Mod, 2nd gear | Tim R Splawn Pro Mod 2nd Gear - G12 (Tim R) | 0.052 | +| 5150 | Peavey 5150 (block letter), boosted | Helga B 5150 BlockLetter - Boosted (Helga B) | 0.064 | +| 6505 RED | Peavey 6505+ red channel, MXR drive | Helga B 6505+ Red ch - MXR Drive (Helga B) | 0.124 | + +High-gain captures are harder for a network this small: their error is higher (the 6505+ most), while the clean +and crunch models are very close. A larger A2 would do better, but would not fit the pedal's CPU. + +The factory presets use them; the fitted amps of v34 are still there as the other 8 AMP types. + +## Your own models (TONE3000 and others) + +1. Get a capture in the **A2** format: on [TONE3000](https://www.tone3000.com/) and in NAM's own trainer, A2 files + are offered beside the older formats. The pedal reads the A2-Lite part of an A2 file. The older A1 files + ("standard", "lite", "feather", "nano") do not run on the pedal and are refused with that reason. +2. Open the web editor (`editor.html`), connect the pedal, and under **NAM models** pick a free slot and the `.nam` file. + The editor checks it, measures its level and sends it to the pedal (about 2 seconds; the sound mutes a moment + while the flash is written). +3. In a preset, set the AMP type to **NAM** and turn **MODEL** to that slot. + +Or press **Browse TONE3000** in the same panel: TONE3000's own tone browser opens in a pop-up (sign in there; it shows +A2 captures only), and the chosen tone comes back with its creator and its A2 models; **Load** puts one into the +user slot chosen beside it. This uses TONE3000's public API ([tone3000.com/api](https://www.tone3000.com/api)), its +"Select" flow: OAuth 2.0 with PKCE and OpenPocket's publishable key (`T3K_CLIENT_ID` in `docs/pxt3k.js`), the sign-in +coming back through `docs/t3k.html`. The access token lives in the page only while it is open. It needs that key and +`https://isod89.github.io/openpocket/t3k.html` registered as a redirect URI in TONE3000's settings. + +The 16 user slots are in the pedal's flash at 0xC0000..0xE0000, outside the area a firmware update rewrites: your +models stay through updates. Clear a slot from the same panel. + +Most captures are made at 48 kHz. The pedal runs at 44.1 kHz, so such a model plays with its tone about 1.5 +semitones lower (the amp's filters sit 8% lower). The editor says so when it loads one. The factory models are +trained at 44.1 kHz and are not affected. + +## How it runs + +- `engine/px_nam.c`: the network in fixed point (activations with 20 fraction bits, weights with 24, 64-bit sums), + checked against a float reference (`tools/nam_a2.py`) for every factory model by `tests/test_nam.py`: about 99 dB + signal to error. +- Blocks of 32 frames, layer by layer. On the CPU, each output channel of a layer is computed for 4 frames at a time + straight from the layer's history ring (`rdot`): for each kernel tap, the 4 frames' 3 channels sit side by side in + the ring (12 words), so a weight is read once for 4 frames and no input is copied (where a tap's 4 frames wrap + round the ring, those 12 words are copied first). Then LeakyReLU, the head sum and the residual (`post`). On the + pedal these are hand-written (pi32v2 assembly: multiply-adds paired with loads, 3 words moved by one instruction, + no branch per sample), because this CPU has no data cache in front of its RAM and a taken branch costs about 8 + cycles. At start-up they are run against the C versions on a test pattern, and the C versions are used if they + differ by a single bit. On this CPU a multiply-add paired with the load of the value it needs next waits for it + (3.2 cycles, against 2.2 without the wait), so the routines for 6, 15 and 16 taps (`rdq`, written by + `tools/gen_rdq.py`) are unrolled with each value loaded 2 to 4 steps before its use. The start-up timing compares + them with the first `rdot` and with gathering each frame's inputs into one row first (then the hardware-loop dot + routine), and keeps the fastest. +- The dot products can also run on the chip's vector unit (in its FFT block: one command computes a row of weights + against many X rows, beside the CPU; each frame's inputs to the layer are then gathered into one X row). The block + is cut in two halves, and half B of a layer never needs the next layer's half A: so while the unit computes one + half, the CPU gathers or posts the other, and may compute the last 4 or 8 frames of a half itself. The unit costs + about 4.8 cycles a multiply-add on the pedal, about as much as the CPU. Its inputs (the X rows and a copy of the + weights, 13 KB) sit in seven of the data-cache ways turned into RAM at boot (`firmware/px_cacheram.h`, melodee's + routine; the data cache only serves the flash), so that it does not read the RAM the CPUs work in; the start-up + check runs the unit there too, and the ordinary RAM is used if it differs. +- Two cores: frame t of a layer needs only frames t and earlier of the layer before. So CPU1 takes the first 12 or 16 + frames of each 32-frame block through all 23 layers without ever waiting, and CPU0 the rest, waiting before each + layer until CPU1 has posted the one before (one counter). With a NAM amp in two-core mode the audio block runs on + CPU0 (in the audio interrupt) as a pipeline (`dsp_pipe` in `firmware/px_audio.h`, `px_engine_part`): CPU1 first runs + the effects after the amp on the previous block's network output while CPU0 runs the effects before the amp on this + block, then both run the network (one block, 2.9 ms, more latency; `tests/engine/pipe_test.c` checks it gives the + one-core output one block later). In the pipeline the roles are the other way round (`pxn_serve`): CPU1 comes to + the network after its effects, so it takes the last frames of each 32-frame block and waits per layer for CPU0, + which takes the first frames and never waits, and up to 12 more of them in the first blocks by how late CPU1 came + in the blocks before. Without a NAM amp the whole block runs on CPU1 as before. +- Each layer's history ring has its first 3 frames mirrored after its end, so 4 frames can be read straight from any + position: the tap pointers just move on by 4 frames per group. On the pedal a layer's whole block of groups runs in one call + (`pxn_rdl6/15/16_asm`, generated by `tools/gen_rdq.py --rdl`), its tap table on the stack. +- At start-up the pedal times the ways to run it (CPU alone; the unit one command after the other; the unit and the + CPU at once, the CPU taking 0, 4 or 8 frames; two cores, with or without the unit) on the first factory model and + keeps the fastest (`pxn_tune`). The PC test runs every mode with a simulated unit that finishes its commands late, + at random, and a simulated second core that moves on only while the first waits for it: the output must be exactly + the CPU's. +- 1,730 multiply-adds per sample in 23 layers; the history buffers take 83 KB of RAM, a model 7.5 KB. +- The load shows at the bottom right of the screen (`C1 nn%`). + +## Training (`tools/train_a2.py`) + +`train_a2.py --capture CAPTURE.nam --di DI.wav --val DI2.wav --out MODEL.nam --name NAME` trains an A2-Lite network +(JAX) on a capture: the DI goes through the capture (resampled to its rate and back), the A2-Lite learns the result at +44.1 kHz. Loss: pre-emphasised ESR plus a multi-resolution spectral term. `tools/mk_nam_factory.py` writes the +factory models into `engine/px_nam_factory.h`, each with its level trim. + +Credits: the captures are from the Neural Amp Modeler community collection +[pelennor2170/NAM_models](https://github.com/pelennor2170/NAM_models) (GNU GPL v3), by the authors named in the +table; the DI guitar is from [Alec-Wright/Automated-GuitarAmpModelling](https://github.com/Alec-Wright/Automated-GuitarAmpModelling) +(GNU GPL v3). The A2 architecture is Steven Atkinson's (Neural Amp Modeler, MIT); the pedal's code is written for +OpenPocket from the published architecture. diff --git a/docs/RESCUE.md b/docs/RESCUE.md new file mode 100644 index 0000000..971829e --- /dev/null +++ b/docs/RESCUE.md @@ -0,0 +1,104 @@ +# Rescuing a crash-looping BlackBox from a Mac (no extra hardware) + +Use this when the pedal restarts in a loop and the installer's rescue mode cannot finish. Between crashes the pedal sits +about 44 seconds in the JieLi mask-ROM USB boot state ("UBOOT": USB 4C4A:8057, shown by macOS as `WL80UBOOT1.00`). The +mask ROM is chip ROM, so it cannot be damaged. `tools/uboot_rescue.py` waits for that state, loads JieLi's RAM loader and +rewrites the app area of the flash from the official package, then restarts the pedal. + +## One-time setup (macOS) + +``` +brew install libusb +``` + +That is all. The tool is one Python 3 file that calls libusb through `ctypes` (no `pip install`). It needs two files: + +- The official package, `~/Documents/BlackBox (2).fwsc` (BlackBox_016) by default, or `--package FILE.fwsc`, or a raw flash + image with `--image flash.bin`. Every CRC of a package is checked before it is used. +- JieLi's `wl82loader.bin`, downloaded once into the git-ignored `build/cache/` from kagaimiq/jl-uboot-tool (commit + `c32b28e`, the same pin as `flash/installer-fm1buddy-pocketx.bat`) and checked against SHA-256 + `d41da6126760c9d66660bcc0cac8d27d221806c5e369a8036921efe68dca5376` on every run. It is not in this repository. `make clean` + removes the cache; `--loader FILE` uses a copy you downloaded yourself. + +## The commands + +`make rescue` and `make rescue-write` (or `node tools/op.mjs rescue [--write]`, the entry point next to `flash`) run the tool with +`sudo` for you and pass options on (`make rescue ARGS="--package FILE.fwsc"`). `make doctor` shows whether libusb, the package and +the loader are in place. The `sudo python3 tools/uboot_rescue.py` lines below are the same thing written out. + +macOS keeps its own mass-storage driver on the pedal in this state, and libusb needs root to take the interface from it. +Plug the pedal straight into the Mac (no hub), start the command, and it waits for the pedal and starts by itself the moment +it appears. + +1. **Read-only first.** Nothing is written to the pedal's flash; the loader goes to RAM only. + +``` +make rescue # = sudo python3 tools/uboot_rescue.py +``` + +It checks that the pedal answers INQUIRY as `WL82 UBOOT1.00`, loads the loader, reads the chip key (must be `0x980F`) and +the flash id (must be a 1 MiB flash), saves a backup of the whole flash to `~/OpenPocket-rescue/blackbox-