diff --git a/.claude/skills/verify/features/editor-page.md b/.claude/skills/verify/features/editor-page.md index acf6aa5..2e8da84 100644 --- a/.claude/skills/verify/features/editor-page.md +++ b/.claude/skills/verify/features/editor-page.md @@ -1,13 +1,13 @@ # Editor page (docs/editor.html) ## Sub-features -52 presets (load, save, save as, move, factory reset), the current sound (blocks, types, parameters, order by dragging, name), JSON export/import, live follow of changes made on the pedal (WATCH), unsaved-change prompts, phone layout. +52 presets (load, save, save as, move, factory reset), the current sound (blocks, types, parameters, order by dragging, name), JSON export/import, live follow of changes made on the pedal (WATCH), unsaved-change prompts, phone layout; v37: the NAM models panel (A2 `.nam` files into the 16 user slots, `docs/pxnam.js`, protocol 2). ## How to get to it (user POV) Open the editor from the installer page, Connect, edit. ## Driving it with Playwright -`sh .claude/skills/verify/scripts/pages.sh`; `tests/test_editor_page.mjs` talks to `build/edsim` (the real `px_editor.h` and `px_store.h`) through a fake Web MIDI. End state: every line `ok`, `editor page test passed`, `build/editor/editor_1..6*.png`. Protocol only (no browser): `node tools/op.mjs test` runs `tests/test_editor.mjs`. +`sh .claude/skills/verify/scripts/pages.sh`; `tests/test_editor_page.mjs` talks to `build/edsim` (the real `px_editor.h` and `px_store.h`) through a fake Web MIDI. End state: every line `ok`, `editor page test passed`, `build/editor/editor_1..6*.png` (`editor_6_nam.png`: the NAM panel). Protocol only (no browser): `node tools/op.mjs test` runs `tests/test_editor.mjs`. ## Gotchas - Protocol is in `docs/EDITOR-PROTOCOL.md`; a change to a command needs both `px_editor.h` and `docs/pxedit.js`. diff --git a/NOTICE.md b/NOTICE.md index c9a68b7..de5e074 100644 --- a/NOTICE.md +++ b/NOTICE.md @@ -8,8 +8,15 @@ OpenPocket is GPL-3.0-only (`LICENSE`), except the parts below, which keep their | `loader/` (update loader) | felucca-bsp `loader/`, changed for the Pocket X flash layout | MPL-2.0 | | `docs/pxota.js`, `docs/pxpkg.js`, `tools/pxpkg.py`, `tools/mkloader.py`, `tools/lz4blk.py` | adapted from [Felucca](https://github.com/hugelton/Felucca)'s web installer and build tools (as used by CLIPPER and SLOOP) | GPL-3.0 | | `firmware/sdk/` (I2S and DAC drivers as LLVM bitcode, SPL, cfg files) | JieLi AC79 SDK V1.2.0 (fw-AC79_AIoT_SDK) | Apache-2.0 (`LICENSES/Apache-2.0.txt`) | +| `firmware/px_cacheram.h` (data-cache ways as RAM, for the NAM network's vector-unit buffers) | [melodee](https://github.com/keremimo/melodee) `firmware/hal/fm1_cache.h`, Kerem Kilic (Ellic Studio), after the AC79 SDK's cache way setup | GPL-3.0 | | `firmware/fonts.h` (bitmaps generated from Inter) | [Inter](https://rsms.me/inter/), Rasmus Andersson | SIL OFL 1.1 (`LICENSES/OFL-1.1-Inter.txt`) | +The factory NAM models (`engine/px_nam_factory.h`, v37) are A2-Lite networks trained by OpenPocket from Neural Amp +Modeler community captures ([pelennor2170/NAM_models](https://github.com/pelennor2170/NAM_models), GPL v3; authors +in `docs/NAM.md`) played with DI guitar from [Alec-Wright/Automated-GuitarAmpModelling](https://github.com/Alec-Wright/Automated-GuitarAmpModelling) +(GPL v3). The NAM A2 architecture is from Neural Amp Modeler (Steven Atkinson, MIT); `engine/px_nam.c`, +`tools/nam_a2.py` and `tools/train_a2.py` are written for OpenPocket from the published architecture. + Sound models: the names refer to the gear that inspired each voicing (circuit analyses published by ElectroSmash, Electric Druid, R.G. Keen / GEOFEX, Rob Robinette, Duncan's Tone Stack Calculator; speaker data sheets; manufacturers' manuals). All trademarks belong to their owners; OpenPocket is not affiliated with Donner, M-VAVE or any gear maker. diff --git a/README.md b/README.md index 5e58bde..580bf78 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ !!! THIS PROJECT IS A PROOF OF CONCEPT, DONT EXPECT FULL FUNCTION AND SOUND QUALITY !!! Install here: https://isod89.github.io/openpocket/ (Donner Pocket X; M-VAVE BlackBox: v31 has the fix for issue #1, being tested) -NEW WEB EDITOR: https://isod89.github.io/openpocket/editor.html · What's new since v33: [docs/WHATS-NEW-v34-v36.md](docs/WHATS-NEW-v34-v36.md) +NEW WEB EDITOR: https://isod89.github.io/openpocket/editor.html · **v37: neural amp models (NAM A2), load your own from TONE3000**: [docs/WHATS-NEW-v37.md](docs/WHATS-NEW-v37.md) · since v33: [docs/WHATS-NEW-v34-v36.md](docs/WHATS-NEW-v34-v36.md) **Open-source guitar multi-effects firmware for the Donner Pocket X and the M-VAVE BlackBox.** Written from scratch for the pedal's own chip (JieLi AC791N, two cores), installable from the browser. @@ -12,11 +12,15 @@ Written from scratch for the pedal's own chip (JieLi AC791N, two cores), install ## What it does -- **12-block chain, 77 models**, in any order you like (double-click PUSH to move a block; the order is saved with +- **12-block chain, 78 models**, in any order you like (double-click PUSH to move a block; the order is saved with the preset): GATE, COMP, PITCH, SPECIAL FX, SYNTH, DRIVE, AMP, CAB, EQ, MOD, DELAY, REVERB - **Synths, keyboards and other guitars** from a normal guitar (no MIDI pickup): organ, church organ, string machine, choir, mono synth with pitch tracking and a resonant ladder filter, PWM synth, bass synth, 12-string, acoustic, sitar (bridge buzz + sympathetic strings), baritone +- **Neural amp models (v37)**: the AMP block's NAM type runs Neural Amp Modeler **A2** networks on the pedal: 12 + factory models trained for it (Twin, AC15, Super 59, Dirty Shirley, JCM900, JCM2000, JVM, Mark IV, Pro Mod, 5150, + 6505+) and 16 user slots you fill from the web editor with A2 captures (TONE3000...). All 24 factory presets use + them. Details: [docs/NAM.md](docs/NAM.md) - **8 amps fitted to captures of the real ones** (v34): Twin Reverb, AC15 Top Boost, JCM900, JCM2000, Mark IV, Dirty Shirley, Pro Mod, 5150. A computer sets the engine's numbers (tube stages oversampled ×4, tone stack, power amp, sag) so that it matches a Neural Amp Modeler capture of each amp on real guitar playing; same CPU cost as a @@ -25,7 +29,7 @@ Written from scratch for the pedal's own chip (JieLi AC791N, two cores), install Greenback, T75, Alnico Blue, Jensen, Cream, DI) with a MIC position - **Pro levels (v36)**: every amp, block and preset at the same loudness (-23 LUFS, measured on real guitar); 24 factory presets round the fitted amps, 28 USER slots; a PRESET LVL per preset; soft output limiter -- What changed since v33: [docs/WHATS-NEW-v34-v36.md](docs/WHATS-NEW-v34-v36.md) +- What changed: [v37](docs/WHATS-NEW-v37.md), [since v33](docs/WHATS-NEW-v34-v36.md) - **New screen (v36)**: a PLAY page readable from a distance (preset number and name, amp, chain in colour) and an EDIT page with big values and bars - **8 reverbs rebuilt in v35**: an 8-line feedback delay network with early reflections and moving lines (no diff --git a/docs/AUDIO-SDK.md b/docs/AUDIO-SDK.md index 3ebe3ad..259b0e8 100644 --- a/docs/AUDIO-SDK.md +++ b/docs/AUDIO-SDK.md @@ -262,3 +262,30 @@ Driver setup: `iis_open` IIS_PORTA, channel_in = BIT(1), channel_out = BIT(0), m - Screen: PLAY (number and name large, amp / cab, the chain in colour, the selected block) and EDIT (the block on its colour, model, 5 settings with value + bar), new Inter fonts (26 and 58 px). `tests/uisim.c` draws every page on the PC and checks that each full page paints every pixel. + +## v37: NAM A2-Lite, user models +- AMP type 8, NAM: a Neural Amp Modeler A2-Lite network (3 channels, 23 dilated layers, LeakyReLU, 16-tap head; + 1871 weights, about 1,730 multiply-adds a sample) in fixed point, `engine/px_nam.c` ([NAM.md](NAM.md)). Activations + Q20, weights Q24, 64-bit sums shifted by a constant. Against the float reference (`tools/nam_a2.py`): 83 to 100 dB + signal to error on the factory models (`tests/test_nam.py`). +- This CPU has no data cache in front of the SRAM, a taken branch costs about 8 cycles, and it issues two + instructions together (a 32-bit multiply-add with a 16-bit load). A first version (a loop over taps per frame, + 16-frame chunks) measured 223 % of real time on the pedal. Now: blocks of 32 frames, layer by layer. Gather: each + frame's inputs to the layer into one row of X (`[u, 1.0, taps x 3 channels]`, 3 words a move), from the layer's + ring (interleaved [time][channel], (kernel - 1) x dilation + 32 frames). Dot: the layer's 3 weight rows against 4 X + rows at a time (`pxn_dot3x4_asm`: a weight read once for 4 frames, 4 multiply-adds paired with 4 loads, 5 issues, + in the hardware loop). Post: LeakyReLU (predicated, no branch), the head sum, the residual 1x1 written straight + into the next layer's ring. The head is a 50-long row (bias, 16 taps x 3) over the head's ring. The hand routines + are in the flash with their callers; `px_nam_check()` compares them with the C ones at boot (pattern with both + signs, rows of 20 and 47, with and without the residual) and keeps C on any difference. +- Memory: the rings (83.4 KB) in the POOL region beside the engine, the model (weights converted, 8 KB) and the + block buffers (7 KB) in RAM. A direct call between the flash (0x02xxxxxx) and the RAM code (0x01C0xxxx) is out of + the call instruction's +-4 MiB: `tools/check_calls.py`, run by `firmware/build.sh` before linking, refuses any such + call and names it (a first v37 build put the audio path in RAM: the linker refused 478 calls, naming only one). +- User models: 16 slots of 8 KiB at 0xC0000 (`firmware/px_models.h`), written by the editor (protocol 2); the update + loader keeps that range (`loader/ldr_core.c`, `LDR_NAM_LO/HI`). The weights are read from the flash (SPI, with the + audio held) when a preset picks the model. +- Level: every model plays at the same loudness: a trim per model (-20 LUFS on `docs/nam_di.wav`, measured by the + editor for user models and by `tools/mk_nam_factory.py` for the factory ones), then one constant that puts NAM + + 1x12 at GAIN noon at the fitted amps' -23 LUFS. +- 24 factory presets rebuilt on the 12 factory NAM models, levelled by `tools/level_presets.py`; USER slots: NAM TWIN. diff --git a/docs/BUILDING.md b/docs/BUILDING.md index de42e5f..bf389a1 100644 --- a/docs/BUILDING.md +++ b/docs/BUILDING.md @@ -38,7 +38,13 @@ Apple silicon and other ARM machines Docker emulates it (Rosetta or QEMU); you c `make NATIVE=1 build` skips Docker and uses a toolchain installed on the machine (`sh firmware/setup.sh` installs it in `~/.jieli`; Linux x86-64 only). `tests/run_tests.sh` also runs directly on macOS and Linux with only a C compiler, Node and -Python 3. +Python 3. Its NAM check (`tests/test_nam.py`, the fixed-point network against the float reference) also needs NumPy +and is skipped without it (the container has none). + +The NAM factory models (`engine/px_nam_factory.h`, v37) are generated, not hand-written: `tools/train_a2.py` trains each +A2-Lite network from a capture (needs `pip install jax optax numpy scipy`, about 15 minutes a model on two CPU cores), +then `tools/mk_nam_factory.py LIST.txt DIR docs/nam_di.wav` writes the header with each model's level trim +([NAM.md](NAM.md)). The `.bat` files `makeimage.bat` and `render.bat` only check that Node is installed and call `tools/op.mjs`. `build.bat` does the same when Node and Docker are there, and otherwise builds in WSL (`build.bat docker` or `build.bat wsl` forces one way). `INSTALL-OFFLINE.bat` needs nothing installed: it serves `docs\` with PowerShell (`tools/serve_offline.ps1`), and falls back to Python (`tools/serve.py`) or WSL only if PowerShell cannot open a port. `tests/test_op.mjs` (part of `make test`) checks the Docker command lines, the friendly errors and `make clean` without Docker, using a fake `docker`. `PUBLISH-GITHUB.bat` and `PUBLISH-SOURCE.bat` (PowerShell `tools/publish_github.ps1`) are the maintainer's release tool and still need Windows; nothing else does. diff --git a/docs/EDITOR-PROTOCOL.md b/docs/EDITOR-PROTOCOL.md index 1a7a9c6..f622a23 100644 --- a/docs/EDITOR-PROTOCOL.md +++ b/docs/EDITOR-PROTOCOL.md @@ -1,7 +1,8 @@ # OpenPocket editor protocol (SysEx over USB-MIDI) The web editor (`docs/editor.html`, `docs/pxedit.js`) talks to the pedal with this protocol. The firmware side is -`firmware/px_editor.h`; `tests/edsim.c` runs the same code on a PC for the tests. Protocol version 1 (OpenPocket v32). +`firmware/px_editor.h`; `tests/edsim.c` runs the same code on a PC for the tests. Protocol version 2 (OpenPocket v37; version 1: v32 to v36, +the same without the NAM commands 19..22). ## Framing @@ -42,8 +43,40 @@ The 12 blocks, by index: 0 GATE, 1 COMP, 2 PITCH, 3 SFX, 4 DRIVE, 5 AMP, 6 CAB, | 16 SAVE | slot | slot, rc. Saves the current sound into the slot, writes the flash, the slot becomes the current one (as the pedal's SAVE) | | 17 WATCH | on (0/1) | on. While on, the pedal pushes `STATE` (cmd 5) when the sound, the slot or the focused block changes on the pedal; it stops by itself 5 s after the last request | | 18 PING | — | DSP load % (0..127), flags (bit 0: bypass, bit 1: second core) | +| 19 NAM_LIST | — | factory models (12), user slots (16), then per model (factory first, then the user slots): present (0/1), name string (11 characters at most; an empty user slot reads "U5 EMPTY") | +| 20 NAM_WRITE | piece (2 bytes: bits 0-6, 7-13), 128 packed bytes | the 2 piece bytes, rc (0 ok, 1 invalid). Piece i holds bytes 112 i .. 112 i + 111 of a model (the last piece fewer), packed 7 bit: groups of a byte of high bits (bit j = bit 7 of byte j) then 7 bytes. Stored in RAM only | +| 21 NAM_COMMIT | slot (0..15) | slot, rc (0 ok, 1 invalid: header, a weight that is not a number, CRC; 2 flash). Checks the pieces received and writes them to the user slot | +| 22 NAM_ERASE | slot (0..15) | slot, rc (0 ok, 1 invalid, 2 flash). Empties the user slot | -Writing the flash mutes the sound for a moment, as saving on the pedal does. A whole import sends `SLOT_PUT` without +### NAM models (protocol 2) + +The AMP block's ninth type, `NAM`, plays a neural amp model (NAM A2, its 3-channel "Lite" size, 1871 weights: +`engine/px_nam.h`). Its first parameter, MODEL, picks one of 28 models: 0..11 the factory ones (in the firmware), +12..27 the 16 user slots in the pedal's flash (`firmware/px_models.h`, 0xC0000..0xE0000, kept by firmware updates). +`TEXTS` of AMP type 8, parameter 0 gives each value's model name; the editor asks for it again after a change of +the user slots. + +A user model is 7548 bytes, little endian: a 64-byte header, then the 1871 weights as float32 in the `.nam` +file's order. + +| Offset | Size | Field | +| --- | --- | --- | +| 0 | 4 | magic `0x4D4E504F` ("OPNM") | +| 4 | 2 | version, 1 | +| 6 | 2 | weights, 1871 | +| 8 | 12 | name, ASCII, 0-ended (11 characters at most) | +| 20 | 2 | level trim, int16, 0.01 dB (added to the model's output) | +| 22 | 2 | CRC16 of the weights (polynomial 0x1021, initial value 0) | +| 24 | 40 | 0 | + +The editor (`docs/pxnam.js`) reads a `.nam` file, takes its A2-Lite part (an A2 file is a "SlimmableContainer" +with several sizes; a plain A2-Lite WaveNet also works), measures its loudness on `docs/nam_di.wav` and sets the +level trim so that every model plays at -20 LUFS on that clip, then sends 68 `NAM_WRITE` and one `NAM_COMMIT`. +A1 models (the older "standard" / "lite" / "feather" WaveNets) and other architectures are refused: the pedal's CPU +runs A2-Lite only. + +Writing the flash (`SLOT_PUT` with bit 1, `COMMIT`, `SAVE`, `NAM_COMMIT`, `NAM_ERASE`) mutes the sound for a moment, as +saving on the pedal does. A whole import sends `SLOT_PUT` without bit 1 for every slot, then one `COMMIT`. ## Preset files diff --git a/docs/NAM.md b/docs/NAM.md new file mode 100644 index 0000000..9f4ea72 --- /dev/null +++ b/docs/NAM.md @@ -0,0 +1,125 @@ +# Neural amp models (NAM A2) on the pedal (v37) + +Since v37 the AMP block has a ninth type, **NAM**, that plays a [Neural Amp Modeler](https://www.neuralampmodeler.com/) +network instead of an amp built from circuit stages. It plays the networks of NAM's **A2** architecture, at its +3-channel size, **A2-Lite**. That is the size the pedal's CPU can run. + +## The knobs + +| Knob | What it does | +| --- | --- | +| MODEL | which network: the 12 factory models, then the 16 user slots (`U1 EMPTY` ... until you load one) | +| GAIN | the level into the network, ±12 dB around the level of the captures (noon) | +| BASS, MID, TREBLE | an EQ after the network: ±10 dB at 120 Hz, ±8 dB at 800 Hz, ±10 dB above 2.5 kHz | + +The models are amp captures without a cabinet, so keep a CAB after the AMP. Every model plays at the same loudness +(-20 LUFS on the reference clip `docs/nam_di.wav`), so switching models keeps the volume. + +## Factory models + +Twelve A2-Lite networks, trained for the pedal at 44.1 kHz (`tools/train_a2.py`). Each one learnt to sound like a +community capture of a real amp, playing real DI guitar. Accuracy is ESR, the error on 20 s of guitar the training never +heard: 0.01 is very close, and below about 0.05 the two are hard to tell apart. + +| MODEL | Amp | Capture (author) | ESR | +| --- | --- | --- | --- | +| TWIN | Fender Twin Reverb (normal, bright) | Tim R Fender TwinVerb Norm Bright (Tim R) | 0.006 | +| AC15 | Vox AC15 Top Boost | Phillipe P VOXAC15-TopBoost (Phillipe P) | 0.045 | +| SUPER 59 | Magnatone Super 59 Mk II | Tim R Magnatone Super 59 Mkii Ch1 Br (Tim R) | 0.009 | +| SHIRL CL | Friedman Dirty Shirley Mini, clean | Sascha S DirtyShirleyMini_Clean_B1_M6_T7_MV10_G4 (Sascha S) | 0.014 | +| JCM900 | Marshall JCM900 50 W Dual Verb | Tim R JCM90050WDualVerbCHAG6 (Tim R) | 0.011 | +| SHIRLEY | Friedman Dirty Shirley Mini, crunch | Sascha S DirtyShirleyMini_crunch_G6 (Sascha S) | 0.033 | +| JCM2000 | Marshall JCM2000 DSL, lead | Tim R JCM2000 L2 G6 (Tim R) | 0.042 | +| MARK IV | Mesa Boogie Mark IV | Roman A LT_MESA_MARKIV_1 (Roman A) | 0.041 | +| JVM OD2 | Marshall JVM, OD2 | Phillipe P JVM-OD2-RD-NoDrive-ESR0,003 (Phillipe P) | 0.056 | +| PRO MOD | Splawn Pro Mod, 2nd gear | Tim R Splawn Pro Mod 2nd Gear - G12 (Tim R) | 0.052 | +| 5150 | Peavey 5150 (block letter), boosted | Helga B 5150 BlockLetter - Boosted (Helga B) | 0.064 | +| 6505 RED | Peavey 6505+ red channel, MXR drive | Helga B 6505+ Red ch - MXR Drive (Helga B) | 0.124 | + +High-gain captures are harder for a network this small: their error is higher (the 6505+ most), while the clean +and crunch models are very close. A larger A2 would do better, but would not fit the pedal's CPU. + +The factory presets use them; the fitted amps of v34 are still there as the other 8 AMP types. + +## Your own models (TONE3000 and others) + +1. Get a capture in the **A2** format: on [TONE3000](https://www.tone3000.com/) and in NAM's own trainer, A2 files + are offered beside the older formats. The pedal reads the A2-Lite part of an A2 file. The older A1 files + ("standard", "lite", "feather", "nano") do not run on the pedal and are refused with that reason. +2. Open the web editor (`editor.html`), connect the pedal, and under **NAM models** pick a free slot and the `.nam` file. + The editor checks it, measures its level and sends it to the pedal (about 2 seconds; the sound mutes a moment + while the flash is written). +3. In a preset, set the AMP type to **NAM** and turn **MODEL** to that slot. + +Or press **Browse TONE3000** in the same panel: TONE3000's own tone browser opens in a pop-up (sign in there; it shows +A2 captures only), and the chosen tone comes back with its creator and its A2 models; **Load** puts one into the +user slot chosen beside it. This uses TONE3000's public API ([tone3000.com/api](https://www.tone3000.com/api)), its +"Select" flow: OAuth 2.0 with PKCE and OpenPocket's publishable key (`T3K_CLIENT_ID` in `docs/pxt3k.js`), the sign-in +coming back through `docs/t3k.html`. The access token lives in the page only while it is open. It needs that key and +`https://isod89.github.io/openpocket/t3k.html` registered as a redirect URI in TONE3000's settings. + +The 16 user slots are in the pedal's flash at 0xC0000..0xE0000, outside the area a firmware update rewrites: your +models stay through updates. Clear a slot from the same panel. + +Most captures are made at 48 kHz. The pedal runs at 44.1 kHz, so such a model plays with its tone about 1.5 +semitones lower (the amp's filters sit 8% lower). The editor says so when it loads one. The factory models are +trained at 44.1 kHz and are not affected. + +## How it runs + +- `engine/px_nam.c`: the network in fixed point (activations with 20 fraction bits, weights with 24, 64-bit sums), + checked against a float reference (`tools/nam_a2.py`) for every factory model by `tests/test_nam.py`: about 99 dB + signal to error. +- Blocks of 32 frames, layer by layer. On the CPU, each output channel of a layer is computed for 4 frames at a time + straight from the layer's history ring (`rdot`): for each kernel tap, the 4 frames' 3 channels sit side by side in + the ring (12 words), so a weight is read once for 4 frames and no input is copied (where a tap's 4 frames wrap + round the ring, those 12 words are copied first). Then LeakyReLU, the head sum and the residual (`post`). On the + pedal these are hand-written (pi32v2 assembly: multiply-adds paired with loads, 3 words moved by one instruction, + no branch per sample), because this CPU has no data cache in front of its RAM and a taken branch costs about 8 + cycles. At start-up they are run against the C versions on a test pattern, and the C versions are used if they + differ by a single bit. On this CPU a multiply-add paired with the load of the value it needs next waits for it + (3.2 cycles, against 2.2 without the wait), so the routines for 6, 15 and 16 taps (`rdq`, written by + `tools/gen_rdq.py`) are unrolled with each value loaded 2 to 4 steps before its use. The start-up timing compares + them with the first `rdot` and with gathering each frame's inputs into one row first (then the hardware-loop dot + routine), and keeps the fastest. +- The dot products can also run on the chip's vector unit (in its FFT block: one command computes a row of weights + against many X rows, beside the CPU; each frame's inputs to the layer are then gathered into one X row). The block + is cut in two halves, and half B of a layer never needs the next layer's half A: so while the unit computes one + half, the CPU gathers or posts the other, and may compute the last 4 or 8 frames of a half itself. The unit costs + about 4.8 cycles a multiply-add on the pedal, about as much as the CPU. Its inputs (the X rows and a copy of the + weights, 13 KB) sit in seven of the data-cache ways turned into RAM at boot (`firmware/px_cacheram.h`, melodee's + routine; the data cache only serves the flash), so that it does not read the RAM the CPUs work in; the start-up + check runs the unit there too, and the ordinary RAM is used if it differs. +- Two cores: frame t of a layer needs only frames t and earlier of the layer before. So CPU1 takes the first 12 or 16 + frames of each 32-frame block through all 23 layers without ever waiting, and CPU0 the rest, waiting before each + layer until CPU1 has posted the one before (one counter). With a NAM amp in two-core mode the audio block runs on + CPU0 (in the audio interrupt) as a pipeline (`dsp_pipe` in `firmware/px_audio.h`, `px_engine_part`): CPU1 first runs + the effects after the amp on the previous block's network output while CPU0 runs the effects before the amp on this + block, then both run the network (one block, 2.9 ms, more latency; `tests/engine/pipe_test.c` checks it gives the + one-core output one block later). In the pipeline the roles are the other way round (`pxn_serve`): CPU1 comes to + the network after its effects, so it takes the last frames of each 32-frame block and waits per layer for CPU0, + which takes the first frames and never waits, and up to 12 more of them in the first blocks by how late CPU1 came + in the blocks before. Without a NAM amp the whole block runs on CPU1 as before. +- Each layer's history ring has its first 3 frames mirrored after its end, so 4 frames can be read straight from any + position: the tap pointers just move on by 4 frames per group. On the pedal a layer's whole block of groups runs in one call + (`pxn_rdl6/15/16_asm`, generated by `tools/gen_rdq.py --rdl`), its tap table on the stack. +- At start-up the pedal times the ways to run it (CPU alone; the unit one command after the other; the unit and the + CPU at once, the CPU taking 0, 4 or 8 frames; two cores, with or without the unit) on the first factory model and + keeps the fastest (`pxn_tune`). The PC test runs every mode with a simulated unit that finishes its commands late, + at random, and a simulated second core that moves on only while the first waits for it: the output must be exactly + the CPU's. +- 1,730 multiply-adds per sample in 23 layers; the history buffers take 83 KB of RAM, a model 7.5 KB. +- The load shows at the bottom right of the screen (`C1 nn%`). + +## Training (`tools/train_a2.py`) + +`train_a2.py --capture CAPTURE.nam --di DI.wav --val DI2.wav --out MODEL.nam --name NAME` trains an A2-Lite network +(JAX) on a capture: the DI goes through the capture (resampled to its rate and back), the A2-Lite learns the result at +44.1 kHz. Loss: pre-emphasised ESR plus a multi-resolution spectral term. `tools/mk_nam_factory.py` writes the +factory models into `engine/px_nam_factory.h`, each with its level trim. + +Credits: the captures are from the Neural Amp Modeler community collection +[pelennor2170/NAM_models](https://github.com/pelennor2170/NAM_models) (GNU GPL v3), by the authors named in the +table; the DI guitar is from [Alec-Wright/Automated-GuitarAmpModelling](https://github.com/Alec-Wright/Automated-GuitarAmpModelling) +(GNU GPL v3). The A2 architecture is Steven Atkinson's (Neural Amp Modeler, MIT); the pedal's code is written for +OpenPocket from the published architecture. diff --git a/docs/SOUNDS-AND-CONTROLS.md b/docs/SOUNDS-AND-CONTROLS.md index f1bc9db..f9e52d6 100644 --- a/docs/SOUNDS-AND-CONTROLS.md +++ b/docs/SOUNDS-AND-CONTROLS.md @@ -1,8 +1,8 @@ -# OpenPocket — sounds and controls (v36) +# OpenPocket — sounds and controls (v37)  -## The chain (12 blocks, 77 models) +## The chain (12 blocks, 78 models) Default order: `GT → CMP → PIT → SFX → SYN → DRV → AMP → CAB → EQ → MOD → DLY → REV` Since v29 every preset has its own block order: **double-click PUSH** on the chain to grab the selected block, turn @@ -19,6 +19,12 @@ captures of the real units (v34); the effects follow published circuit analyses R.G. Keen, Boss/Roland/EHX manuals). They are digital models, not copies. The product names are trademarks of their owners, used only to describe the sound. +### AMP — NAM: neural amp models (v37; MODEL, GAIN, BASS, MID, TREBLE) +The ninth AMP type plays Neural Amp Modeler A2 networks: 12 factory models (TWIN, AC15, SUPER 59, SHIRL CL, JCM900, +SHIRLEY, JCM2000, MARK IV, JVM OD2, PRO MOD, 5150, 6505 RED) and 16 user slots filled from the web editor with A2 +captures. MODEL picks the network, GAIN sets the level into it (±12 dB), BASS / MID / TREBLE are an EQ after it. The +models have no cabinet: keep a CAB after the AMP. All the factory presets use NAM. Details: [NAM.md](NAM.md). + ### AMP — 8 amps, fitted to captures of the real ones (GAIN, BASS, MID, TREBLE, PRESENCE) Since v34 there are fewer amps, and each one is fitted by computer to a capture of the real amp (method and references: [FITTED-MODELS.md](FITTED-MODELS.md)). With the knobs at noon it sounds like the capture. @@ -106,8 +112,8 @@ plate). DECAY sets the real decay time (RT60) of the lows: ROOM 0.3–1.2 s, HAL PLATE 0.6–5 s, SPRING 1.5–4 s, SHIMMER 2–8 s, AMBIENT 6–20 s. They run at 22 kHz in 16 bits (like the classic studio units), in a quarter of v34's memory. -## Factory presets (v36): 24 sounds, all at the same loudness, then 28 USER slots -Built round the fitted amps and drives, which sound like their captures near noon. Every preset is levelled to the +## Factory presets (v37): 24 sounds, all at the same loudness, then 28 USER slots +Built round the NAM models (v37; v36's were built round the fitted amps) and the fitted drives. Every preset is levelled to the same loudness (-23 LUFS, EBU R128, measured on real DI guitar by `tools/level_presets.py`): going from a clean to a high-gain preset no longer jumps by 20 dB. @@ -120,7 +126,7 @@ high-gain preset no longer jumps by 20 dB. | | 05 SURF SPRING | 11 70S ROCK | | 21 DOOM | | | | 06 AMBIENT SWELL | 12 BOUTIQUE | | | | -25–52: **USER 25 … USER 52**, a clean TWIN + 1x12 + a little room, to build your own. +25–52: **USER 25 … USER 52**, a clean NAM TWIN + 1x12 + a little room, to build your own. **Levels (v36)**: every amp plays at the same loudness at GAIN noon and moves only +-3 dB over the whole knob; every block type, with its knobs at noon, keeps the loudness when switched on; each preset has its own **PRESET LVL** diff --git a/docs/WHATS-NEW-v37.md b/docs/WHATS-NEW-v37.md new file mode 100644 index 0000000..5e53e3d --- /dev/null +++ b/docs/WHATS-NEW-v37.md @@ -0,0 +1,59 @@ +# What's new in OpenPocket v37: neural amp models (NAM A2) + +## Neural amps on the pedal +- **A new AMP type, NAM**, plays Neural Amp Modeler networks of the **A2** architecture (its "Lite" size) on the pedal + itself. The pedal no longer plays an approximation fitted to a capture: it plays the network. +- **12 factory models**, trained for the pedal from community captures of real amps (Fender Twin Reverb, Vox AC15, + Magnatone Super 59, Friedman Dirty Shirley clean and crunch, Marshall JCM900, JCM2000 and JVM, Mesa Mark IV, Splawn + Pro Mod, Peavey 5150 and 6505+). All at the same loudness. +- **Knobs:** MODEL, GAIN (±12 dB into the network), BASS, MID, TREBLE (an EQ after it). +- Details, accuracy and credits: [NAM.md](NAM.md). + +## Load your own captures (TONE3000) +- The web editor has a **NAM models** panel: pick a user slot, choose an **A2** `.nam` file (from TONE3000, or from + NAM's trainer), and it goes to the pedal in about 2 seconds. 16 user slots, kept through firmware updates. +- The editor checks the file, refuses A1 captures with the reason, and sets the model's level so it plays as loud as + the others. +- On the pedal, MODEL shows the model's name; the PLAY page shows it as the amp. +- **Browse TONE3000 from the editor:** the button opens TONE3000's own tone browser (sign in there, A2 captures only), + then the editor shows the chosen tone (title, gear, creator) and its A2 models, and loads one into a user slot. It + uses TONE3000's public API (its "Select" flow, OAuth with PKCE); nothing is stored by the editor. + +## New factory presets +- **All 24 factory presets are rebuilt around the NAM models**, levelled to the same loudness as before. The 28 USER + slots start from a clean NAM Twin. +- The 8 fitted amps of v34 stay available as AMP types, and your saved presets keep the amp they use. + +## Faster engine +- The NAM network has hand-written routines for the pedal's CPU, checked against the C versions at every start-up: + each layer is read straight from its history ring (no copy), 4 frames at a time, a weight read once for the 4, + multiply-adds paired with loads. +- It can use the chip's **vector unit** (in its FFT block) for the dot products beside the CPU, and the **second + core**: with a NAM amp, CPU0 runs the block and CPU1 takes the first frames of each network block (earlier frames + never need later ones, so CPU1 never waits). At start-up the pedal times the ways it can run the network and keeps + the fastest. +- A NAM amp replaces the fitted amp's ×4-oversampled stages, so it adds less than its own cost. +- Effects run over a whole block, one after another, instead of sample by sample (same output, about 20% cheaper). + In front of a NAM amp the drive runs at ×2 oversampling instead of ×4. +- With a NAM amp the two cores work as a pipeline: CPU1 runs the effects after the amp on the previous block while + CPU0 runs the effects before it, then both run the network. This adds one block of latency (2.9 ms). +- The network reads each layer's history without any per-tap position arithmetic: every ring's first frames are + mirrored after its end, and each layer's whole block runs in one hand-written call. +- In the pipeline CPU1 goes straight from its effects to the network and takes the last frames of each network + block; CPU0 takes more frames when CPU1 comes late, so neither core waits for the other. +- **Load on the pedal** (% of real time, measured): the network alone 66%, the 23 factory presets with a NAM amp + 69–90% (most 75–83%). The status corner shows `2C` and the load while both cores run a NAM preset. Above 93% for + 8 blocks in a row the pedal bypasses the effects (`BYP`) until the next change, as before (85% on one core). + +## For developers +- Editor protocol 2: `NAM_LIST`, `NAM_WRITE`, `NAM_COMMIT`, `NAM_ERASE` ([EDITOR-PROTOCOL.md](EDITOR-PROTOCOL.md)). +- `tools/train_a2.py` (JAX) trains an A2-Lite network from a capture; `tools/mk_nam_factory.py` builds the factory + set; `tests/test_nam.py` checks the fixed-point engine against a float reference. +- Memory: the NAM history buffers (83 KB) sit in the POOL region beside the engine, so that the ordinary RAM keeps + about 47 KB free (for Bluetooth later). +- `tools/check_calls.py` (run by `firmware/build.sh`): refuses a direct call between the flash code and the RAM code, + which is out of the CPU's call range, and names it. + +## Please report +The CPU load cannot be measured off the pedal. Please send the `C1 nn%` shown at the bottom right on a few factory +presets (for example 14 HAIR METAL, 16 SPACE LEAD and 06 AMBIENT SWELL), and say if you hear crackles. diff --git a/docs/editor.html b/docs/editor.html index d56bb47..63af993 100644 --- a/docs/editor.html +++ b/docs/editor.html @@ -74,6 +74,29 @@ .row.cur{background:var(--panel);box-shadow:inset 3px 0 0 var(--acc)} .row.cur .n{color:var(--acc)} .slotbar{display:flex;flex-wrap:wrap;gap:6px;align-items:center} +/* v37: TONE3000 (the Select flow) */ +.t3k{display:flex;align-items:center;gap:10px;flex-wrap:wrap;margin:6px 0 8px} +.t3k .logo{font-weight:800;letter-spacing:.04em;font-size:15px;color:#fff} +.t3kcard{border:1px solid #2a2e2a;border-radius:10px;padding:10px;margin:0 0 10px;display:grid;gap:8px} +.t3kcard .top{display:grid;grid-template-columns:64px minmax(0,1fr);gap:10px;align-items:center} +.t3kcard .img{width:64px;height:64px;border-radius:8px;object-fit:cover;background:#222} +.t3kcard .ttl{font-weight:700;overflow:hidden;text-overflow:ellipsis} +.t3kcard .sub{color:var(--dim);font-size:12.5px} +.t3kcard .by{display:flex;align-items:center;gap:6px;font-size:13px;min-width:0} +.t3kcard .by img{width:20px;height:20px;border-radius:50%;background:#222} +.t3kcard .mrow{display:grid;grid-template-columns:minmax(0,1fr) auto;gap:8px;align-items:center;font-size:13px} +.t3kcard .mrow span{overflow:hidden;text-overflow:ellipsis;white-space:nowrap} +.t3kcard .foot{display:flex;gap:8px;align-items:center;flex-wrap:wrap;font-size:12.5px;color:var(--dim)} +.t3kcard .logo{font-weight:800;letter-spacing:.04em;color:#fff} +.t3kcard select{background:#1c1f1c;color:var(--txt);border:1px solid #333;border-radius:7px;padding:5px} +/* v37: NAM models */ +.namlist{max-height:360px} +.namlist .row{grid-template-columns:34px minmax(0,1fr) auto;cursor:default} +.namlist .row .acts{display:flex;gap:4px} +.namlist .row.empty .nm{color:var(--dim);font-weight:500} +.side .note{margin:0;font-size:12.5px;line-height:1.45;color:var(--dim)} +.side .note code{font-size:12px} +.a2{font-size:10px;letter-spacing:.06em;color:var(--acc);border:1px solid var(--acc);border-radius:4px;padding:0 4px;margin-left:6px;vertical-align:1px} .slotbar .lbl{font-size:12.5px;color:var(--dim);margin-right:auto} /* ---- the screen ---- */ @@ -174,6 +197,15 @@
The AMP block's NAM type plays these neural captures (pick one with its
+ MODEL setting). Load your own: an A2 capture (.nam) into a user slot, or browse TONE3000 and pick one
+ there.