From 910bc3a4c967afdc1dcced1e361b84a2675319b6 Mon Sep 17 00:00:00 2001 From: Will Shields Date: Fri, 4 Sep 2026 17:02:17 -0700 Subject: [PATCH 01/21] Archive the completed v1 roadmap, open the V2 gate board docs/ROADMAP.md becomes the forward-looking V2 board: five workstreams (12-16) with an explicit status/gate model, where every workstream passes design entry, engineering, device behavior, claim truth, and release in order. The completed v1 phase narrative, feasibility analysis, and exit criteria move unedited to docs/history/ROADMAP_V1.md at the stable v1.0.7 tag, so v1 questions still have an answer without the active board carrying 1,000 lines of finished history. CLAUDE.md, AGENTS.md, and the README indexes point at the V2 board and its design-entry documents first, and at the v1 archive only when a question is actually about phases 0-11. version.h's versioning comment and build.yml's one ROADMAP reference follow the same move; neither changes behavior. Co-Authored-By: Claude Opus 5 --- .github/workflows/build.yml | 2 +- AGENTS.md | 20 +- CLAUDE.md | 52 +- README.md | 21 +- docs/README.md | 22 +- docs/ROADMAP.md | 1036 +++++------------------------------ docs/history/ROADMAP_V1.md | 21 + docs/research/V2_DESIGN.md | 21 +- src/version.h | 11 +- 9 files changed, 240 insertions(+), 966 deletions(-) create mode 100644 docs/history/ROADMAP_V1.md diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 6c2585a..9d7c0e7 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -78,7 +78,7 @@ jobs: # Rolling "latest main" build, separate from the versioned vX.Y.Z # releases in release.yml. Overwrites the same dev-latest tag/release # on every merge so there's always a one-click, no-login download for - # SD-drop/Launcher testing — see docs/ROADMAP.md Distribution section. + # SD-drop/Launcher testing — see the root README's install guidance. # Only on pushes to main (not PRs, not every commit on a branch). - name: Move dev-latest tag if: github.ref == 'refs/heads/main' && github.event_name == 'push' diff --git a/AGENTS.md b/AGENTS.md index 9ec4dac..03f4458 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -14,10 +14,12 @@ should be fixed to match, not the other way around. Also relevant, referenced from CLAUDE.md: - [docs/STATUS.md](docs/STATUS.md) — current status, what's hardware-verified, and what's still open. -- [docs/DESIGN.md](docs/DESIGN.md) — the "why" behind RF/architecture - decisions. -- [docs/ROADMAP.md](docs/ROADMAP.md) — phase-by-phase build order and - scope. +- [docs/DESIGN.md](docs/DESIGN.md) — the "why" behind the shipped v1 + RF/architecture foundation. +- [docs/ROADMAP.md](docs/ROADMAP.md) — active V2 workstreams, gates, and + release policy. +- [docs/research/V2_DESIGN.md](docs/research/V2_DESIGN.md) — V2 product + boundaries and design direction; read it before V2 architecture work. - [CHANGELOG.md](CHANGELOG.md) — short, ongoing changelog; full pre-2026-08-29 session-by-session decisions log is archived at [docs/history/CHANGELOG.md](docs/history/CHANGELOG.md). @@ -25,8 +27,8 @@ Also relevant, referenced from CLAUDE.md: - [docs/HARDWARE_TESTING.md](docs/HARDWARE_TESTING.md) — repeatable bench matrix and Phase 7 memory acceptance rules. -**Don't read `docs/history/PROGRESS.md` or `docs/history/CHANGELOG.md` -end-to-end by default.** They're a frozen pre-2026-08-29 development log, -not required context for every task — `docs/STATUS.md` already gives the -current-state summary. Search them for the specific date/version/topic you -need instead of reading front to back. +**Don't read `docs/history/PROGRESS.md`, `docs/history/CHANGELOG.md`, or +`docs/history/ROADMAP_V1.md` end-to-end by default.** They're historical +records, not required context for active V2 work — `docs/STATUS.md` and the +active roadmap give the current-state summary. Search history for the specific +date/version/topic you need instead of reading front to back. diff --git a/CLAUDE.md b/CLAUDE.md index 236d345..fd95e82 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -28,9 +28,10 @@ here depends on that choice except the specific API calls. ## Proposed layout -`[x]` created, `[ ]` proposed but deferred until its build-order phase -(docs/DESIGN.md §9) — see docs/ROADMAP.md/docs/STATUS.md for why the -task/queue files aren't scaffolded yet. +`[x]` created, `[ ]` a deferred legacy candidate. V1's build order is frozen +at `docs/history/ROADMAP_V1.md`; active V2 workstreams and gates live in +`docs/ROADMAP.md`, with detailed product decisions in +`docs/research/V2_DESIGN.md`. ``` src/ @@ -73,7 +74,7 @@ src/ --- added during phase 8 foundation, not in the original proposal --- [x] discovery_plan.h # fixed, source-backed candidate tuples [x] scan_observation.h # fixed CAD result record + probe.csv formatter - --- added during phase 11, out of sequence (see docs/ROADMAP.md) --- + --- added during V1 Phase 11, out of sequence (see docs/history/ROADMAP_V1.md) --- [x] cell_plan.h # Cell: 869-894MHz cell-band bin math [x] cell_observation.h # fixed RSSI-reading record + cell.csv formatter --- present since earlier phases but never listed here until the @@ -81,6 +82,9 @@ src/ [x] radio_task.h # (listed above as radio_task.cpp/.h) [x] energy_plan.h # Sweep's frequency-bin math (phase 9) [x] energy_observation.h # Pass-A working stats + energy.csv record + [x] focus_plan.h # Phase 12 one-bin Focus request contract + [x] focus_observation.h # Phase 12 bounded RSSI stats + focus.csv row + [x] focus_runtime.h # Phase 12 restore-before-publish state contract [x] pass_b_plan.h # Pass-B SF/BW candidate table + confidence [x] region_plan.h # Region enum shared by Sweep's band choice [x] region_settings.h / .cpp # System > Tuning > Region, /loratrace/region.txt @@ -99,6 +103,9 @@ src/ [x] bench_fault.h / .cpp # bench-image-only fault/override hooks test/ [x] test_channel_plans/ # host-native unit tests, pio test -e native + [x] test_focus_plan/ # one-bin request/source/selection math (phase 12) + [x] test_focus_observation/ # bounded RSSI summaries + focus.csv (phase 12) + [x] test_focus_runtime/ # bounded request terminal/restore rules (phase 12) [x] test_detection/ # queue-record + CSV, fixtures are REAL captured packets [x] test_gps_parse/ # NMEA -> fix, mostly about REFUSING a bad position [x] test_keyboard/ # raw TCA8418 event byte -> KeyAction, allowlist-only @@ -120,9 +127,12 @@ docs/ [x] STATUS.md # current version, hardware-verified state, open items [x] DESIGN.md [x] ROADMAP.md + [x] research/V2_DESIGN.md # V2 product direction and permanent boundaries + [x] research/phase12-survey-truth-design.md # active Workstream 12 design entry [x] LOG_GUIDE.md [x] HARDWARE_TESTING.md [x] BRAND.md + [x] history/ROADMAP_V1.md # immutable v1.0.7 roadmap snapshot pointer [x] history/PROGRESS.md # frozen pre-2026-08-29 status doc, unedited [x] history/CHANGELOG.md # frozen pre-2026-08-29 session log, unedited [x] CHANGELOG.md # short, terse, ongoing — points at docs/history/ for the rest @@ -153,17 +163,18 @@ docs/ a web-UI-only setting or a silent default. Established 2026-08-25 after Phase 5's menu grew a third item (verbose debug) the same day it shipped, with no framework change to absorb it — see - docs/history/CHANGELOG.md and docs/ROADMAP.md's Phase 6 (UI architecture - redesign). + docs/history/CHANGELOG.md and the V1 Phase 6 entry in + docs/history/ROADMAP_V1.md (UI architecture redesign). Doesn't apply to one-shot boot-time config (channel overrides, `config.txt`) — this is about anything that changes runtime behavior while the device is already running. -- **Bump `src/version.h` when a phase lands.** `MAJOR.MINOR` tracks the - build-order phase (docs/ROADMAP.md Versioning: v0.1.x = phase 1, v0.2.x = - phase 2, ...), `PATCH` for fixes adding no phase scope. It is bumped by - hand on purpose — it asserts "this phase is reached", and a number that - auto-increments every build asserts nothing. `release.yml` now fails a - tag whose version doesn't match the header, so a mismatch can't ship. +- **Bump `src/version.h` only when its release gate lands.** In the completed + v1 series, `MAJOR.MINOR` tracks the reached build-order phase and `PATCH` + fixes scope; V2 uses the workstream/release policy in `docs/ROADMAP.md`. + The number is bumped by hand on purpose — it asserts that a gate is reached, + and a number that auto-increments every build asserts nothing. + `release.yml` now fails a tag whose version doesn't match the header, so a + mismatch can't ship. Build *provenance* is the automated half: `scripts/build_rev.py` injects `FIRMWARE_BUILD_REV` (git short SHA, `-dirty` when the tree is modified) into every build, because a dozen `dev-latest` binaries otherwise share @@ -206,14 +217,15 @@ docs/ See [docs/STATUS.md](docs/STATUS.md) for the current version, what's hardware-verified, and what's still open — the single current-state doc, -kept instead of duplicating status prose here. Build order is -`docs/DESIGN.md` §9; `docs/ROADMAP.md` has the phase-by-phase scope. - -**Don't read `docs/history/PROGRESS.md` or `docs/history/CHANGELOG.md` -end-to-end by default** — they're a frozen pre-2026-08-29 development log, -not required context for every task. Grep them for a specific date/topic -instead of reading front to back; root `CHANGELOG.md` covers what's -changed since. +kept instead of duplicating status prose here. Active V2 gates are in +`docs/ROADMAP.md`; `docs/research/V2_DESIGN.md` is the product/design source. +The completed V1 build order remains in `docs/history/ROADMAP_V1.md`. + +**Don't read `docs/history/PROGRESS.md`, `docs/history/CHANGELOG.md`, or +`docs/history/ROADMAP_V1.md` end-to-end by default** — they are historical +records, not required context for active V2 work. Search them for a specific +date, version, or topic instead; root `CHANGELOG.md` covers what's changed +since. Three hard-won rules from Phases 1–2, worth not relearning: - **The IO expander's P0 powers the GPS as well as switching the RF antenna diff --git a/README.md b/README.md index dfc5844..b1aff0b 100644 --- a/README.md +++ b/README.md @@ -64,24 +64,23 @@ not a laptop-tethered tool. ## Version -No tagged `vX.Y.Z` release has been published yet — `main` builds a -rolling +The stable release is **v1.0.7**. `main` also builds a rolling [`dev-latest`](https://github.com/d3mocide/LoRaTrace-RX/releases/tag/dev-latest) -prerelease on every merge (see [Install](#flash-from-your-browser) below), -which is what the **dev-latest** badge above tracks. The firmware's own -semantic version lives in `src/version.h` and is bumped by hand when a -build-order phase lands — see -**[docs/STATUS.md](docs/STATUS.md)** for the current version, what's -hardware-verified, and what's still open, rather than a number restated -here that can drift out of sync. +prerelease on every merge for hardware iteration (see +[Install](#flash-from-your-browser)); it is not a stable release. The firmware +semantic version lives in `src/version.h` and is bumped by hand only when its +release gate lands. See **[docs/STATUS.md](docs/STATUS.md)** for current +hardware verification and open work rather than treating this summary as a +release record. ## Documentation | Doc | For | |---|---| | **[docs/STATUS.md](docs/STATUS.md)** | Current version, what's hardware-verified, what's still open. Start here. | -| [docs/DESIGN.md](docs/DESIGN.md) | Hardware, RF parameters, and architecture rationale — read before making architecture changes. | -| [docs/ROADMAP.md](docs/ROADMAP.md) | Build-order phases, MVP-Beta scope, and an honest feasibility read against this hardware's real limits. | +| [docs/DESIGN.md](docs/DESIGN.md) | Shipped v1 hardware, RF parameters, and architecture rationale — read before changing those foundations. | +| [docs/ROADMAP.md](docs/ROADMAP.md) | Active V2 workstreams, gates, and release policy. | +| [docs/research/V2_DESIGN.md](docs/research/V2_DESIGN.md) | V2 product direction and permanent boundaries. | | [docs/LOG_GUIDE.md](docs/LOG_GUIDE.md) | Operator guide to run folders, CSV fields, identity observations, health checks, and privacy-aware export. | | [docs/HARDWARE_TESTING.md](docs/HARDWARE_TESTING.md) | Repeatable device-validation matrix and Phase 7 memory acceptance rules. | | [docs/BRAND.md](docs/BRAND.md) | Naming, tone, and on-device UI copy conventions. | diff --git a/docs/README.md b/docs/README.md index 4db3207..cfbf129 100644 --- a/docs/README.md +++ b/docs/README.md @@ -8,11 +8,12 @@ material once you're past that. - **[STATUS.md](STATUS.md)** — where the project actually is right now: current version, what's hardware-verified, what's still open. -- **[DESIGN.md](DESIGN.md)** — hardware, RF parameters, and architecture - rationale. Read before making architecture changes; don't re-derive - decisions already made here. -- **[ROADMAP.md](ROADMAP.md)** — build-order phases, MVP-Beta scope, and - an honest feasibility assessment against this hardware's real limits. +- **[DESIGN.md](DESIGN.md)** — shipped v1 hardware, RF parameters, and + architecture rationale. Read before changing those foundations. +- **[ROADMAP.md](ROADMAP.md)** — active V2 workstreams, gates, and release + policy. Start here for the next implementation decision. +- **[research/V2_DESIGN.md](research/V2_DESIGN.md)** — V2 product direction, + permanent boundaries, and rationale behind the active roadmap. - **[LOG_GUIDE.md](LOG_GUIDE.md)** — operator guide to run folders, CSV fields, identity observations, health checks, and privacy-aware export. - **[HARDWARE_TESTING.md](HARDWARE_TESTING.md)** — repeatable @@ -25,11 +26,12 @@ material once you're past that. Not onboarding reading — these are raw evidence and in-progress notes, kept for reference rather than written to be read start to finish. -- **`history/`** — `PROGRESS.md` and `CHANGELOG.md` as they stood before - the 2026-08-29 documentation restructuring: a frozen, unedited - session-by-session development log. Search it for a specific - date/topic rather than reading it front to back; `STATUS.md` above and - the root `CHANGELOG.md` are what stay current going forward. +- **`history/`** — the immutable v1.0.7 roadmap snapshot pointer + ([`ROADMAP_V1.md`](history/ROADMAP_V1.md)), plus `PROGRESS.md` and + `CHANGELOG.md` as they stood before the 2026-08-29 documentation + restructuring. Search history for a specific date/topic rather than + reading it front to back; `STATUS.md` above and the root `CHANGELOG.md` + are what stay current going forward. - **`research/`** — design/investigation notes written during specific phases (e.g. Phase 8/9 sweep design), not maintained after the phase they were written for. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 3521de7..14c88fa 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -1,903 +1,139 @@ -# LoRaTrace RX — Roadmap - -This roadmap operationalizes the build order already decided in -`docs/DESIGN.md` §9. It doesn't change any RF or architecture decision — it adds -scope boundaries, exit criteria, and an honest read on what this specific -hardware can and can't do, so "MVP-Beta" means something concrete instead -of a vibe. - -## What "MVP-Beta" means here - -The smallest version of this firmware that's actually useful as a field -tool: **Meshtastic War Drive, end to end.** RX locked to the LongFast (US) -channel, every detection GPS-stamped and written to SD, running -unattended on battery. That's docs/DESIGN.md §9 phases 1–2. Everything past -that (MeshCore, discovery sweeps, energy sweep, UI, upload) is real -project scope, but it's post-MVP-Beta. - -## Hardware feasibility — reality check - -Read this before assuming any phase below "just works." It's an honest -accounting against the ESP32-S3FN8's actual limits (CLAUDE.md hardware -assumptions), not a pitch. - -**Solid — low risk:** -- RX-only detection on a known channel (`HOME_LISTEN`) for Meshtastic and - MeshCore. Continuous single-channel RX is cheap: no polling, no - scanning, minimal CPU. This is the easy 80% of the project's value. -- The Core 1 (radio) / Core 0 (GPS+SD+UI) split with a FreeRTOS queue. - Standard ESP32-S3 pattern, and it's the right call given SD's latency - spikes — keeping those off the radio task is the one architectural - decision that actually matters for not dropping packets. -- GPS fusion (NMEA over UART) and small-struct SD logging. Neither needs - meaningful RAM; TinyGPSPlus-class parsers and a ~40B detection struct - flushed often are well inside an 8MB-flash, no-PSRAM budget. -- Keyboard + small-display UI, *if* it avoids a full framebuffer (see - below). - -**Achievable, but needs deliberate care:** -- **No PSRAM means ~512KB SRAM total, shared with the RTOS, WiFi stack (if - enabled), display driver, and every task's stack.** docs/DESIGN.md's own - estimate of 250–380KB free heap is a guess pending `ESP.getFreeHeap()` - on real hardware — treat every RAM-hungry decision below as provisional - until that number is in hand. -- **WiFi AP + web UI (Phase 3).** This used to sit in the "lowest priority" - tier below, gated behind a real free-heap number under full load. That - number now exists: `run0007`/`run0011` (session.csv) measured heap_free - settling at ~304KB with radio + GPS + logger + display all running, - flat with no decline across a 2.5-hour run — real headroom, not the - provisional estimate this gate was originally waiting on. Built - on-demand (off by default, toggled by an operator gesture) specifically - so its RAM/CPU/RF-noise cost is never present during an actual drive - unless asked for — see docs/history/CHANGELOG.md for the spike this was still worth - gating behind before building the rest of the feature on top of it. -- **Display:** a naive full framebuffer at 240×135×16bpp is ~65KB — a - meaningful bite out of a few-hundred-KB heap. Use direct-to-panel - partial-window writes (LovyanGFX/M5GFX style) instead of buffering a - whole frame, especially once WiFi is in the picture. - **Superseded 2026-08-25 (Phase 6's own hardware bench pass):** real - glass proved direct-to-panel wrong for this UI — every intermediate draw - call is visible the instant it happens, causing real flicker/tearing a - build report can't catch. The actual fix was an off-screen - `Arduino_Canvas_Indexed` framebuffer, just an indexed 1-byte/pixel one - (~32KB, since this UI only ever uses 6 colours) rather than a full RGB565 - one (~63KB) — see docs/history/CHANGELOG.md's v0.6.2 -> v0.6.3 entry for the full - root-cause and the near-miss (an `SPIClass::beginTransaction()` deadlock) - ruled out along the way. -- **`ENERGY_SWEEP` (Reticulum / General Exploration):** a single SX1262 - cannot listen to the whole 868–923MHz band at once. The design's - sweep-then-return-to-listen approach is the right shape for this - hardware, but it's a fundamental physical tradeoff, not a bug to code - around: short bursts on a non-home frequency *will* be missed during the - portion of the duty cycle spent elsewhere. Set that expectation in the - UI/docs rather than implying continuous full-band coverage. -- **Front-end rolloff near 923–928MHz:** the module's tuned range tops out - around 923MHz; the top of US ISM (923–928MHz) is outside it. This is a - hardware fact, not fixable in firmware — it specifically dents General - Exploration's full-band sweep, since every named profile's actual - channel already sits inside 868–923 (docs/DESIGN.md §1). Needs an empirical - RSSI noise-floor sweep once hardware is in hand, then document the - sensitivity gap rather than pretend it isn't there. -- **Meshtastic's 104-slot hash space:** Phase 1–2 only lock to the - LongFast US default (slot 20). That's most of the real-world traffic, - but a mesh running a non-default channel name lands on a different slot - and is invisible to `HOME_LISTEN`. `DISCOVERY_SWEEP` (Phase 8) is what - closes that gap — until then, be precise in docs/UI that "Meshtastic - profile" means "default channel," not "all Meshtastic traffic." - -**Genuinely open — blocks real functionality, not just polish:** -- microSD bus (SPI vs SDMMC) on this board revision — docs/DESIGN.md §7, - unresolved. Blocks finalizing the Logger task's pin/driver choice. -- Meshtastic's exact sync-word register value, and MeshCore's - encryption/PSK model — docs/DESIGN.md §7. Detection (RSSI/SF/BW/timing) works - without these; reliable protocol-level filtering and payload decode - don't. Don't hardcode a guessed value for either (CLAUDE.md house rule). - -**Deliberately out of scope:** -- TX/injection of any kind — permanent non-goal per CLAUDE.md house rules, - not a phase-ordering question. - -## Phases - -Each phase maps 1:1 to docs/DESIGN.md §9. "Blocking unknowns" cross-reference -docs/DESIGN.md §7 items that must be resolved (or explicitly deferred) before -the phase can be called done, not just started. - -### Phase 0 — Project scaffold -**Status:** done (this change). -PlatformIO project, pin map, RF parameter tables, phase-1 bring-up code, -this roadmap, and the tracking doc. - -### Phase 1 — RadioLib bring-up -**Goal:** prove the radio path is alive. -**Deliverable:** `src/main.cpp` — SPI init, IO-expander P0 high, SX1262 up, -hardcoded RX on 906.875MHz/SF11/BW250/CR8, detections printed to Serial. -**Exit criteria:** a real Meshtastic LongFast (US) packet in the air shows -up on the serial monitor with plausible RSSI/SNR. -**Blocking unknowns:** none functionally — but the IO-expander register -map and SPI host assignment (see docs/history/PROGRESS.md) are unverified against real -hardware and are the first suspects if the radio stays silent. -**Status:** complete, hardware-verified. See docs/STATUS.md for current -state and docs/history/CHANGELOG.md for the full session history. - -### Phase 2 — `HOME_LISTEN` + task/queue architecture + GPS + SD (= MVP-Beta) -**Goal:** the smallest genuinely useful field tool. -**Deliverable:** `radio_task`, `gps_task`, `logger_task` per the proposed -layout in CLAUDE.md; FreeRTOS queue Core 1 → Core 0; GPS-stamped -detections batched to SD per the §8 log schema. -**Exit criteria:** unattended run — power on, GPS fix acquired, detections -logged with correct lat/lon, no dropped packets attributable to SD -latency, no crash from heap exhaustion over a multi-hour run. -**Blocking unknowns:** none left. microSD is confirmed on the shared SPI -bus (arbitrated by `spi_bus.h`), and GPS reached a fix on hardware -2026-08-23 — the last piece that had never been proven. -**Status:** see docs/STATUS.md for current state and -docs/history/CHANGELOG.md for the full session history. - -### Phase 3 — Web Command Center (WiFi AP + web UI) -**Goal:** get data and control off the device without ejecting the SD card. -**Deliverable:** `wifi_task` — an on-demand (off by default, operator- -toggled), WPA2-protected WiFi AP hosting a single embedded web page (no -LittleFS/SPIFFS, no new `lib_deps` — built-in `WiFi.h`/`WebServer.h` only, -see docs/history/PROGRESS.md) with three tabs: a live status dashboard (the same -counters the Serial `[status]` line and `ui_task`'s pages already expose), -a run browser to download `detections.csv`/`session.csv` per run, and a -settings form that writes `/loratrace/config.txt` (`config.h`'s existing -format/validation, reused not reimplemented) — applied on next boot, not -hot-reloaded, so `radio_task`'s real-time critical section is never touched -by another task. `ui_task` gains a long-press-any-key gesture to toggle the -AP (no keymap needed, matching its existing "any key" discipline) and a -WIFI status page. -**Blocking unknowns:** none for the feature itself — pulled forward ahead -of Phase 4 at the user's request, prioritizing operator convenience over -protocol breadth. The one thing worth confirming on real hardware before -trusting this at length is the heap/counter spike described in -docs/history/PROGRESS.md: `ESP.getFreeHeap()` before/after `WiFi.softAP()` with the -full Phase 2 task set already running, and `radioCrcErrorCount()`/ -`radioQueueDropCount()`/`radioBusMissCount()` staying at 0 with the AP -active — the actual go/no-go this phase used to be gated behind, now -answerable instead of estimated (see the heap numbers above). - -### Phase 4 — MeshCore profile -**Deliverable:** same `HOME_LISTEN` engine, MeshCore US-narrow table -(910.525MHz/SF7/BW62.5/CR5) wired in as a second selectable profile — the -first phase where "selectable" is real, not aspirational. This is where -docs/DESIGN.md §5's keyboard-gated profile switch (operator-selected, mutually -exclusive — Meshtastic and MeshCore never listen at once) actually gets -built, deliberately deferred from Phase 3 (docs/history/CHANGELOG.md) so it's designed -and tested against a real second channel table instead of a stub. -**Blocking unknowns:** none for basic detection; MeshCore's -encryption/PSK model (§7) still blocks payload decode, not detection. -**Status:** see docs/STATUS.md for current state and -docs/history/CHANGELOG.md for the full session history. - -### Phase 5 — On-device menu UI -**Deliverable:** a real menu on `ui_task`'s display, replacing the timed -hold-gestures Phase 3/4 built as a stopgap. Pulled forward ahead of -`DISCOVERY_SWEEP`/`ENERGY_SWEEP` at the user's request — same precedent as -WiFi's Phase-3 pull-forward ahead of MeshCore — because the menu/settings -framework built here is what the rest of the project gets built around, not -because it was blocking anything below it. - -Scope, decided with the user up front rather than assumed: plain `,`/`.` -move a selection (no Fn chord — the Cardputer-ADV has no dedicated arrow -keys; directional intent there is Fn held + `;`/`,`/`.`/`/`, confirmed -against M5Stack's own keyboard API docs, a dedicated Cardputer-ADV keyboard -reference, and `bmorcelli/Launcher`'s own shipped interface code), Enter -selects, Backspace goes back. Toggles only this phase — profile switch -(Meshtastic/MeshCore, reusing Phase 4's `radioRequestProfileSwitch()` -unchanged) and WiFi on/off (reusing Phase 3's `wifiToggle()` unchanged) — -no on-device numeric editing of channel params; that stays on the existing -web UI Settings tab / `config.txt`. A new read-only CHANNEL status page -shows the actual active freq/SF/BW/CR/sync word, closing a real gap (that -was previously visible only over Serial or the web UI). The two old -hold-gestures are removed, not kept as parallel shortcuts, so there's -exactly one way to trigger each action. - -This is what turns "decode the keyboard" from a 56-key project of its own -into a small, low-risk slice: only four specific keys need to be identified -correctly, not the whole QWERTY layout — see `src/keyboard.h` for the full -sourcing (TCA8418 raw-event encoding, the raw-value-to-physical-position -formula, and the physical-position-to-character table, each independently -cited) and docs/DESIGN.md for the citation-level writeup. - -**Blocking unknowns:** none for the menu/toggle mechanism itself — every -action it triggers already exists and was already exercised in Phase 3/4. -The keyboard decode is sourced from three independent references but, -per this project's own standing rule, not yet bench-verified against real -hardware (see docs/history/PROGRESS.md's Phase 5 checklist): press each of the four keys -once and confirm the firmware recognizes exactly that key and no other. - -### Phase 6 — UI architecture redesign -**Goal:** rework `ui_task` before `DISCOVERY_SWEEP` (or anything after it) -adds another ad hoc entry to a framework that was never designed to hold -more than a couple of toggles. Pulled forward ahead of `DISCOVERY_SWEEP` at -the user's request (2026-08-25) — same restructuring precedent as WiFi's -Phase-3 pull-forward and the menu UI's own Phase-5 pull-forward, both -already in this doc's history. Unlike those two, this isn't new subsystem -bring-up: it's a scoped rework of `ui_task.cpp`/`.h`, the one file the next -two phases would otherwise keep bolting onto. -**Why now, not later:** Phase 5 shipped exactly two menu items by design -(docs/history/CHANGELOG.md/docs/DESIGN.md §9) and had already grown a third — the verbose -serial-debug toggle — by the end of the same day, with no framework change -and a stale in-code comment (`ui_task.cpp`'s "the menu has exactly two -items this phase") as the only trace of the drift. `DISCOVERY_SWEEP` would -add at least a fourth item (a sweep trigger) and `ENERGY_SWEEP` a fifth -after it; fixing the shape once beats letting two more features each bolt -on their own row. -**Deliverable:** -- **Grouped menu**, replacing the flat, fixed-size list `MENU_ITEM_COUNT` - indexes into today. Root categories (e.g. Profile, Radio Mode, System) - open short sub-lists, same four input keys throughout (`,`/`.` move, - Enter act, backtick/ESC back/close) — a shape borrowed structurally from - M5PORKCHOP's (github.com/0ct0sec/M5PORKCHOP) "Sirloin-style grouped - modal" `Menu` class (`src/ui/menu.h` in that repo: `RootItem` entries are - either `DIRECT` actions or `GROUP`s that open a `MenuItem` sub-list) — - reviewed for that structural idea only, not its content or aesthetic - (see note below). -- **Toast/notice layer** for transient feedback — a toggle firing, later a - sweep hit — that isn't tied to whichever carousel/menu row happens to be - on screen. Real RAM cost gets measured and reported before it ships, - same discipline as every other RAM-hungry addition this project has made - (WiFi AP's ~55-56KB measurement is the precedent, not an estimate). -- **Redesigned status pages.** The current five (RADIO/CHANNEL/GPS/ - SYSTEM/WIFI) are single-column stacked text (`ui_task.cpp` - `drawRadioPage()` etc.) with real unused width on a 240px panel. Group - related values into blocks instead of one value per line. (As-proposed - this assumed direct-to-panel partial-window writes, no framebuffer — the - real hardware bench pass overturned that; see the Hardware feasibility - section above and this phase's Status note below.) -- **Adopt docs/BRAND.md's on-device labels** (`Watch`/`Probe`/`Sweep` for - HOME_LISTEN/DISCOVERY_SWEEP/ENERGY_SWEEP; plain profile names — - `Meshtastic`/`MeshCore`/`Reticulum`/`Spectrum` — grouped under one - "Profile" menu row, not branded per-profile names. docs/BRAND.md's Interface - Naming table went through two revisions the same day mid-implementation: - first a "Mesh Trace" family name over Meshtastic/MeshCore sub-profiles, - then walked back entirely once it was clear that branding every profile - its own "___ Trace" name made four settings on one sniffer read like - four separate tools — "Profile" was already this doc's preferred word - for the axis before either revision) as the strings the UI actually - displays. Kept as a separate UI-label layer, not a rename of the - internal identifiers: `detection.h`'s `missionProfileName()` keeps - emitting `meshtastic`/`meshcore` into `detections.csv` exactly as every - already-logged run expects — docs/DESIGN.md §8's own "don't concatenate runs - across a format change without checking the header" rule applies here - too, and there's no reason to risk it for a cosmetic rename. -- **Design reference, not a dependency:** M5PORKCHOP was cloned locally - (read-only) and reviewed for its menu grouping, its `NoticeKind`/ - `NoticeChannel` toast abstraction (`src/ui/display.h`), and how it - organizes a stats/status screen (`src/ui/swine_stats.h`). Its - gamification layer (XP/levels/achievements, `src/core/xp.h`), the piglet - mascot/mood system (`src/piglet/`), and its aesthetic and voice are - explicitly **not** part of this redesign — docs/BRAND.md's own guardrails - already rule out mascots, gamified framing, and anything outside the - "field instrument, not a consumer app" positioning. Structure was worth - learning from; the pig was not coming with it. -**Blocking unknowns:** the toast layer's actual heap cost, unmeasured -until built. Whether the ST7789V2's partial-window writes can carry the -denser block layout above without needing a framebuffer — an assumption to -confirm during implementation, not before. -**Status:** see docs/STATUS.md for current state and -docs/history/CHANGELOG.md for the full session history (canvas/framebuffer rework, -bugs found/fixed, and the still-open WiFi AP heap/counter re-measurement). - -### Phase 7 — Device optimization - -**Status:** complete 2026-08-27 as v0.7.0. Hardware evidence, measured -budgets, and the operator-approved same-build repetition waiver are recorded -in docs/history/PROGRESS.md. - -**Goal:** turn the current memory assumptions into measured budgets before -new scan states add load to the no-PSRAM device. - -**Deliverable:** internal-heap fragmentation metrics, stack high-water marks -for all five tasks, lifecycle checkpoints around the display canvas/WiFi/CSV -downloads, a repeatable real-device matrix (`docs/HARDWARE_TESTING.md`), and -measured optimizations that preserve radio/GPS/SD/UI behavior. - -**Epic priority order:** - -1. **P0 — Observability and baseline.** Append largest-free-block, heap block - counts, and every task's stack watermark to `session.csv`; bracket known - large/transient allocations in serial; run the full WiFi+canvas workload. -2. **P1 — Task stacks.** Right-size only stacks whose worst-case watermark - proves excess headroom. Preserve at least 25% and 1KB after the change. -3. **P2 — WiFi lifecycle and requests.** Measure repeated AP cycles and CSV - downloads; reclaim the off-state task/server cost or remove request-time - churn only where the measurements identify a persistent cost. -4. **P3 — Canvas allocation.** Keep the verified indexed canvas unless it is - the limiting allocation. Any tiled/partial alternative must repeat the - full real-glass regression pass; moving it to static RAM is not a saving. -5. **P4 — Final budget.** Record normal, WiFi-on, largest-block, and per-task - stack budgets for Phases 8/9, explicitly accept or reject the provisional - 2.5 KB transient-scan result buffer, then complete a combined-load soak. - -**Exit criteria:** every `docs/HARDWARE_TESTING.md` stage passes on one identified -build; no continuing decline in current free heap or largest block after -warm-up; `queue_drop`/`row_drop`/`bus_miss` stay 0 under combined load; task -stack margins satisfy the acceptance rule; final budgets and results are -recorded in `docs/history/PROGRESS.md`. - -**Scope guardrail:** Phase 7 does not add Discovery/Energy behavior and does -not optimize from compile-time RAM percentages alone. One lever changes per -measurement cycle so gains and regressions remain attributable. - -### Phase 8 — `DISCOVERY_SWEEP` - -**Operator label:** Probe. - -**Implementation status:** bounded acquisition slice started 2026-08-27. The -source-backed, versioned fixed candidate-plan layer is documented in -`research/phase8-discovery-research.md` and implemented in -`src/discovery_plan.h`; radio-owned CAD/receive-on-hit, observation queues, -durable Probe output, and the on-device trigger are now implemented. Hardware -validation, transient mode, and deterministic cancellation/fault coverage -remain open. This is not a phase-complete release. - -**Deliverable:** a bounded-duration, radio-task-owned CAD sweep of a curated -candidate list per active profile — non-default Meshtastic slots and sourced -legacy MeshCore tuples. Every complete, cancel, timeout, and failure path -restores the resolved home configuration and reports total time away from -Watch. - -Packet-bearing hits keep using the existing `Detection` pipeline. CAD-only -observations use a separate fixed-size, non-blocking queue; cumulative retry, -drop, recovery, and home-away values belong in the run summary/health log, -not every observation. - -Durable Probe writes to SD. An explicitly selected transient mode may run -without SD only if Phase 7 accepts a fixed result-buffer ceiling of 2.5 KB. -It reuses the live measurement buffer, retains one result, stores no raw or -historical stream, replaces the prior result on the next scan, and displays -`NOT SAVED`. If the budget fails, Probe requires SD. - -**Blocking unknowns:** curated candidate lists should be weighted by -MeshMapper-observed frequencies ([[meshmapper-pipeline]], per CLAUDE.md) -where available, not scraped defaults alone. CAD `symNum` tuning (§7) -needs bench testing against Semtech AN1200.48 before trusting false- -positive/miss rates. New sweep results use the existing carousel and Probe -controls, not a reason to reopen UI architecture a second time. - -Phase 8 ships with bounded, versioned built-in plans plus the existing -per-profile home override. Persistent operator-edited candidate lists are a -post-Phase-8 enhancement requiring their own bounded schema, validation, -deduplication, provenance, and device/web editing design; they do not block -Phase 8 completion. - -**Exit criteria:** correct built-in LongFast CR 4/5 is host- and OTA-verified; -a known alternate transmitter is found; CAD false-positive/miss rates are -measured; 1,000 Probe cycles pass through a deterministic automated bench -mode; deterministic cancellation/fault injection covers every acquisition -state; queues and memory remain stable; every exit restores Watch. - -### Phase 9 — `ENERGY_SWEEP` - -**Operator label:** Sweep. - -**Deliverable:** a truthful frequency-binned energy map across the supported -868–923MHz front end, followed by selective LoRa CAD only at energy peaks, -operator-selected bins, or a sparse sourced subset. This realizes Reticulum -and General Exploration without claiming that energy is LoRa or that an -off-grid CAD hit is Reticulum; those results are labeled `unknown LoRa -candidate` until stronger evidence exists. - -Each bin retains only bounded streaming statistics. Durable energy peaks use -a schema that cannot be confused with packet detections. Transient mode uses -the same Phase 7-gated 2.5 KB ceiling and `NOT SAVED` behavior as Probe. - -**Blocking unknowns:** 923–928MHz front-end rolloff — **resolved, -2026-09-01, no rolloff found.** A passive floor-only pass and a real -injected-carrier pass (both real hardware, see STATUS.md for the full -evidence and the bench tooling) agree: a real transmitted signal near the -923MHz ceiling registers within ~2dB of the same strength as one at -912.8MHz, and captures at least as reliably. Not a gap in coverage. - -**Exit criteria:** timing and home-away duration are measured (**done**, -2026-09-02) — 3/3 real US-region sweeps: EA (device-measured away time) -3386-3438ms, home channel restored every time. Quiet-band behavior is -characterized with WiFi off/on (**done**, 2026-09-02) — 3 matched pairs, -zero peaks and no meaningful timing difference (EA within ~50ms) whether -the AP was on or off. CAD never promotes energy alone to LoRa (**done**, -2026-09-02) — 10 real `BENCH_PASS_B_CAD` attempts at the noisiest known -combo, 5 came back `CAD_DETECTED`, the real-packet-promotion counter -(`PBD`) never moved once, confirming empirically (not just by code -inspection) that a bare CAD hit never becomes a Detection. See -`docs/STATUS.md` for the full numbers and how each was measured -(`WIFI_SET`/`EA` are new Serial Control additions this session). -Injected low/mid/high carriers land in the correct bins (**done**, -2026-09-02) — `scripts/phase9_bin_accuracy_bench.py`, the dedicated test -`research/LoRaTrace-Phases-7-10-Design.md`'s own hardware matrix already -specified ("Sweep calibration... known signals at low/mid/high bins"). -Real sourced candidates at the low, mid, and high ends of the US region -(`LONG_SLOW` 905.3125MHz, `LONG_MODERATE` 912.8125MHz, `SHORT_SLOW` -920.625MHz): 11/24 attempts landed a real elevated reading via -`BENCH_SWEEP_FLOOR`, and every single hit was at the exact pre-computed -bin index for that candidate's frequency — never a neighbor, never a -wrong bin. Sub-100% hit rate is expected and not a correctness problem: -a live Sweep only dwells ~40ms per bin, so a repeatedly-fired pulse has -to land inside that window by chance each lap (same timing reality the -923MHz-edge work already established); what matters is that a hit is -always at the *right* bin, which held 11/11 times. - -8-hour endurance soak (**done**, 2026-09-03, scoped down from 24h — see -STATUS.md for why) — `scripts/phase9_soak.py`, production firmware. -First run found a real bug: 5 of 4,148 laps failed, initially documented -here as harmless dropped-response noise, which was wrong — pulling -`session.csv` off the SD card afterward showed all 5 were the identical -`Guru Meditation Error... Stack canary watchpoint triggered (logger)` -crash, `logger_task`'s 5,120-byte stack genuinely overflowing under -sustained operation and silently rebooting the device each time (also -explaining that run's "WiFi went quiet" finding — the device rebooted, -not a WiFi bug). Fixed (8,192 bytes). A second full 8-hour run with the -fix in place: **4,353/4,353 laps, 0 failures, 0 crashes**, one -continuous run directory on the SD card confirming no reboot of any -kind. Real `session.csv` heap data from that run confirms bounded -memory directly: a clean step function (flat, one legitimate one-time -jump when WiFi switched on, flat again), no drift in either phase. The -recurring slow-lap timing tail (19-35s vs the ~3.4s median) turned out -to be fully explained, not a bug: 100% correlated with `wp>0` (Pass B -correctly running its full CAD sequence on a real Sweep peak). The same -real data also caught `radio_task`'s own stack margin sitting at 20.0% -headroom (below the 25%-or-1KB house rule, though never actually -overflowing) — fixed proactively (4,096 → 6,144) the same way, verified -clean with a focused 2-hour run covering where the old watermark hit its -floor. Full writeup in `docs/STATUS.md`. - -**All five exit criteria closed. Status: complete, hardware-verified, -`v0.9.0`.** - -**Region setting (`v0.8.9`, added and hardware-verified 2026-09-01, out -of sequence — same "appended, not inserted" convention as Cell below):** -the full 868-923MHz range Sweep always scanned is the module's hardware -ceiling, not a legal requirement — nothing legal for unlicensed US -LoRa-type operation sits below 902MHz (47 CFR § 15.247). System > Region -(`region_plan.h`, `energy_plan.h`'s `energySweepBandForRegion()`) lets -Sweep default to 902-923MHz (US) instead of the full range (Global), -roughly halving scan time for the common case, with Global one menu -cycle away. Persisted to `/loratrace/region.txt` -(`region_settings.h`/`.cpp`, mirrors `display_settings.h`/`.cpp`'s -shape exactly). Confirmed on real hardware: the menu row cycles and -persists across reboot, and both US and Global sweeps show the correct -band on the frequency bar with the expected scan duration. - -**Follow-ups identified but explicitly out of scope for this change** -(operator conversation, 2026-09-01) — larger, not scheduled: -- **Cell regionalization.** Cell's A/B block markers (§5a of - `docs/DESIGN.md`) cite 47 CFR § 22.905, an FCC-only structure. A - non-US operator has no equivalent band to swap in — Cell would need - to stop presenting FCC-specific claims outside the US, not gain a - parallel per-region table. -- **`channel_plans.h` per-region presets.** Meshtastic/MeshCore's - hardcoded "US default" channel tables would need real per-region - frequency-plan tables (Meshtastic alone ships a dozen-plus regions), - each individually sourced per CLAUDE.md's citation house rule — a - research project, not a mechanical change. - -### Phase 10 — Field Analyzer (closed 2026-09-03 — the `v1.0.x` release gate) - -**Deliverable:** Meter, truthful frequency waterfall, bounded live Scope, -recent captures, and a passive node roster over data acquired by Watch, -Probe, and Sweep. Field Analyzer is not another mission profile and never -controls the SX1262 directly. - -Scope uses an explicit bounded `SCOPE_ACQUIRE` request owned by the radio -task. It samples one displayed frequency, exposes `Watch paused`, and restores -the resolved home configuration on complete, cancel, timeout, or failure. -Ordinary analyzer page changes consume snapshots and never retune the radio. - -Analyzer storage is fixed-size and reuses the existing indexed canvas. The -initial incremental ceiling is 8 KB beyond that canvas, subject to Phase 7 and -the measured Phase 8/9 costs. Waterfall plot columns use deterministic, -host-tested aggregation of real frequency bins; Scope is never presented as -a spectrum. - -**Exit criteria:** bin-to-pixel and scope-source truthfulness are verified; -bounded memory and deterministic roster eviction hold; a worst-case UI/radio -run has no drops, deadlocks, or watchdog resets; outdoor readability and -minimum-brightness rendering are tested as separate conditions. - -**Decided 2026-09-03, with Phase 9 hardware evidence in hand: Phase 10 is -required for `v1.0.x`.** `v1.0.x` is not tagged until Field Analyzer's own -exit criteria above are also closed, matching the original "all four -profiles + UI stable" framing this doc has used since the 2026-08-25 -renumbering. Work starts now under an interim `v0.10.x` line, the same -convention Phase 8/9 used while in progress (see Versioning below). - -**Gate satisfied the same day: promoted to `v1.0.0`, 2026-09-03.** All -five exit criteria above hardware-confirmed (`v0.10.1`), plus real -hardware-confirmed UI polish beyond them (`v0.10.2`/`v0.10.3`, below). -Phase 11 (Cell) was never part of this gate — added out of sequence, -appended after Phase 10 rather than inserted into it, outside the -original four-profile scope this promotion tracks — its own two open -items (docs/STATUS.md) are known, tracked gaps post-`v1.0`, not blockers, -an explicit call rather than a silent omission. - -**Implementation status (`v0.10.0`, 2026-09-03):** -- **Data layer — done, host-verified.** `waterfall.h`/`scope_trace.h`/ - `capture_history.h`/`node_roster.h` (WaterfallHistory, ScopeTrace, - CaptureHistory, NodeRoster) plus `analyzer_state.h`/`.cpp`'s mutex-guarded - cross-task snapshot layer. 202+ host-native test cases - (`pio test -e native`). Row-at-a-time waterfall snapshot API deliberately - replaces a first cut that returned the whole ~5.5KB struct — that version - crashed `ui_task`'s 4096B stack on real hardware (Waterfall page, - 2026-09-03); see `analyzer_state.h`'s own comment on - `analyzerWaterfallRowSnapshot()`. -- **`SCOPE_ACQUIRE` — done, hardware-verified.** Bounded, radio-owned, - mutual-exclusion against Probe/Sweep/Cell/bench triggers in both - directions, home-config restore on complete/cancel/timeout/failure - (`radio_task.cpp`/`.h`). -- **On-device UI — done, hardware-verified**, and materially reshaped from - the original one-page-per-view plan during this session at the operator's - request: rather than five analyzer views navigable directly on the main - carousel, they're gated behind a single **Analyze** hub card (page-local - row selection, UP/DOWN — `;`/`.` — to browse, SELECT to open, BACK to - return); Probe/Sweep/Cell were folded the same way into a second **Tools** - hub, both real scope additions beyond this section's original text, not a - deviation from the exit criteria above. Iterated live against - `docs/research/analyzer-preview.html` (an interactive Canvas mockup of - the exact device palette/coordinates/state machine, built specifically so - hub UX could be worked out before each real flash) before being ported to - firmware. Two real bugs were caught and fixed along the way: a whole-struct - Waterfall snapshot overflowing `ui_task`'s stack (above), and - `drawFooterStatus()` computing the card-count footer from the raw - `UiPage` enum ordinal instead of the main-carousel-relative position, - which would have shown "14/14" for Tools and "8/14" for Analyze instead of - the intended "2/6"/"3/6" — fixed via new `mainCarouselPosition()`/ - `mainCarouselCount()` accessors. -- **Memory/telemetry — landed, not yet confirmed against a real multi-hour - run.** `ANALYZER_STATIC_BYTES` (`analyzer_budget.h`, split out of - `analyzer_state.h` so it stays reachable without pulling in FreeRTOS.h, - same "keep it pure" reasoning as `session_log.h`'s own header comment) is - a compile-time `sizeof()` sum, not a runtime measurement — the four - analyzer structures commit **6,728 bytes** against this section's - 8,192-byte incremental ceiling (82.1% used, 1,464 bytes headroom). - `session_log.h`'s `analyzer_static_bytes` column reports it on every - health row (added append-only, per this project's CSV-schema - convention). Real linked-firmware RAM delta since Phase 10 work began: - +6,472 bytes, within compiler-alignment noise of the computed number. -**All five exit criteria closed, 2026-09-03:** -- **Worst-case UI/radio run — closed.** WiFi on, Sweep repeat mode - running continuously, Waterfall open the whole time, for a full 60 - minutes. A background Serial Control watch (231 `STATUS` polls at 15s - intervals) recorded **zero** dropped/unanswered requests and **zero** - `task_wdt`/`Guru Meditation` signatures for the entire hour. Confirms - the exit criterion directly, not just "nothing looked wrong." -- **Outdoor and minimum-brightness readability — closed (operator - check, 2026-09-03):** confirmed good both in direct window sunlight and - indoors. -- **A real, hardware-found Waterfall bug surfaced by the worst-case - run — found and fixed same day.** During the run, Pass A found 50 - energy peaks (`PBA=50` cumulative, `STATUS`'s own field) worth - triggering Pass B CAD on, yet Waterfall showed nothing the entire hour. - Root cause: `analyzerNoteSweepComplete()` (`analyzer_state.cpp`) read - `radio_task.cpp`'s live `energyPeakBinMask` to build each Waterfall row, - but in repeat mode `radio_task`'s own do-while loop (`radio_task.cpp` - ~1443) calls straight back into `performEnergySweep()` for the next lap - with no delay — whose first line resets that same mask. `logger_task` - (Core 0) only polls `radioEnergySweepCount()` on its own ~100ms cadence, - so it almost always lost that race against Core 1 and read an - already-cleared mask: every repeat-mode Waterfall row reported quiet - regardless of what Pass A actually found. `energy.csv` itself was never - affected — `enqueueEnergyObservation()` pushes each peak to a queue the - instant Pass A finds it, independent of this mask, so all 50 peaks (with - bin/frequency/RSSI/`pass_b_confidence`) are real rows on the card. - **Fix:** a second, stable buffer (`energyPeakBinMaskAtComplete`) - snapshotted atomically at `energySweepCount++`, before the next lap can - touch it; a new accessor, - `radioEnergyPeakBinSetAtLastComplete()`, is what - `analyzerNoteSweepComplete()` reads now. `drawSweepOccupancy()` - (ui_pages.cpp, the Sweep page's own live occupancy ticks) is untouched - — it still reads the live mask on purpose, so it keeps rendering ticks - progressively as an active sweep runs. **Hardware-confirmed the same - day**: operator re-ran Waterfall during a live repeat Sweep against - real MeshCore traffic and confirmed hits now appear on the display. -- **Bench SD card / boot-loop finding — found then resolved same day - (2026-09-03):** the bench Cardputer briefly hit a 100%-reproducible - `task_wdt` abort during SD bring-up on every cold boot, isolated to the - SD card (identical crash reproduced on the last tagged release, - `v0.9.0`/`b84d88c`, with no Phase 10 code at all — not a regression from - this work). Operator swapped the card; the fix was confirmed two ways: - a 20-minute passive Serial Control watch (zero `task_wdt`/`Guru - Meditation`, `SD=1` throughout), then the card itself pulled and - inspected directly — `run0006`'s real `session.csv` covers a clean - 29-minute boot, `analyzer_static_bytes=6728` on every row, `sd=ok` - throughout, zero row/queue/bus drops, heap settling once early then - flat for the rest of the run. This closes Stage 4's memory/telemetry - confirmation for real, not just at compile time. Full writeup: - `docs/STATUS.md`'s Phase 10 section. -- **Post-closure UI polish, same day (`v0.10.2`/`v0.10.3`), hardware- - confirmed:** real scope beyond the five exit criteria above, not a - deviation from them — Waterfall gained a frequency axis (lo/center/hi - MHz + reference ticks, later merged into the plot box's own bottom - border to remove a redundant line and reclaim height) and an Enter key - that starts/stops repeat Sweep straight from the page, sharing - `WATERFALL_SWEEP_REPEAT_TOGGLE` with the Tools/Sweep card's own R key - rather than a parallel mechanism. Meter — previously three lines of - text and a lot of unused screen — gained a real bar gauge (`drawMeterBar()`, - filled proportionally, deliberately not `drawFreqBar()`'s position-marker - convention since signal strength is a quantity, not a location), an SNR - line, and a right-column SF/BW/CR block, all three watch-sourced only - from data (`CaptureSummary`) this page already had and never showed. Bar - range widened -30 -> 0dBm same day after a real -16dBm reading clipped - flat against the original ceiling. Every layout change was workshopped - in `docs/research/analyzer-preview.html` before reaching real hardware, - same tool that already caught this session's real Waterfall footer- - collision bug once. - -### Phase 11 — Cell (added out of sequence, 2026-09-01) - -**Operator label:** Cell — same single-word register as Watch/Probe/Sweep -(docs/BRAND.md), not a "___ Trace" name. Named "Cell Trace" for its first -same-day revision; renamed once the operator noticed that collided with the -product name's own "Trace" and read as a bigger decode claim than the -feature (RSSI presence only) actually makes. - -Not part of the original four-profile plan (docs/DESIGN.md §3/§9) — added at -the operator's request after real wardriving runs kept picking up energy in -the 869-894MHz North American Cellular downlink band near cell towers. -Unlike WiFi's Phase-3 pull-forward or the UI's Phase-5/6 pull-forwards -(both of which *reordered* the existing sequence), this is *appended* after -Phase 10 rather than inserted into it — Phase 9/10 keep their numbers and -their own in-progress status is unaffected. - -**Deliberately not a fifth mission profile.** The SX1262 only demodulates -FSK/GFSK/MSK/LoRa/OOK — it cannot decode GSM/CDMA/LTE, so there is no -protocol to detect a channel for, and no HOME_LISTEN table to give it. It is -instead a third bounded, radio-owned action alongside Probe (`DISCOVERY_SWEEP`) -and Sweep (`ENERGY_SWEEP`): retune across a curated 101-bin, 250kHz grid -covering 869-894MHz (`cell_plan.h`), sample RSSI at each bin -(`radio.getRSSI(false)`, the same primitive Sweep's Pass A uses), log every -bin — not threshold-filtered — to its own `cell.csv` (`cell_observation.h`). -No CAD is attempted (CAD is a LoRa-preamble correlator; it will never fire on -a cellular carrier) and no packet read is attempted (there is nothing to -decode). This keeps the feature honest: it is a coarse RF-presence/strength -survey ("a strong carrier sits near this frequency, here"), not a cell-tower -identifier — no cell ID, LAC/TAC, or MCC/MNC is or can be extracted. - -**Deliberately isolated from `ENERGY_SWEEP`'s Pass A/B engine** -(`performEnergySweep()`), not a parameterized reuse of it: Pass A's -35.0dB noise-floor margin (`energy_observation.h`) was bench-calibrated -against a LoRa/RF-quiet environment for *sparse* peak detection, not -continuous cellular-strength carriers — reusing it here would dress up a -guess as a calibration. `performCellSweep()` (`radio_task.cpp`) is its own -function; it shares only the generic streaming-RSSI-statistics primitives -(`EnergyBinStats`/`energyBinStatsAddSample()`/`energyRssiDbmToFixed()`, -`energy_observation.h`) that Pass A itself uses, not Pass A's acquisition -loop or its calibrated threshold. - -**869-894MHz** is FCC Part 22 Cellular Radiotelephone Service downlink / -3GPP Band 5 downlink — a regulatory band edge, not a carrier-specific -channel plan (no individual ARFCN/channel table is hardcoded, consistent -with CLAUDE.md's house rule against hardcoding unverified RF parameters). -It sits entirely inside the Cap LoRa-1262's tuned 868-923MHz front end -(docs/DESIGN.md §1), so unlike General Exploration's 923-928MHz top end, -there is no front-end rolloff caveat here. - -**Operator surface:** same shape as Probe/Sweep, not a menu row (`ui_menu.h`'s -`CELL_TOGGLE`, same SD-required/start/cancel logic) — a global hotkey (C, -`keyboard.h`'s `KEY_RAW_C_PRESS`) and a dedicated carousel results card -(`UiPage::CELL`, `ui_pages.cpp`'s `drawCellPage()`), inserted as carousel -position 4 and pushing CHANNEL/GPS/SYSTEM's digit-jump keys from 4/5/6 to -5/6/7 (`keyboard.h`'s `KEY_RAW_7_PRESS`). An earlier same-day revision of -this feature shipped it as a root-level menu row with no card, before the -operator asked for the full Probe/Sweep treatment instead — reverted before -merge, not kept as a parallel entry point. - -**Repeat mode (R), added 2026-09-01:** identical in shape to Sweep's own -repeat — `radioRequestCellSweepRepeat()` re-runs `performCellSweep()` -back-to-back until stopped, with a lap counter on the Cell card. This also -changed Sweep's own R: it was previously a global hotkey (fired from -anywhere, including with the menu open); R is now page-gated by -`ui_task.cpp` to the Sweep/Cell cards specifically, matching how Enter -(`SELECT`) already dispatches per-page. C/S/P remain global. Probe -deliberately has no repeat mode — operator decision: "Repeat only on the -Sweeps." - -**Implementation status:** code + host-native tests (`test_cell_plan`, -`test_cell_observation`, and `test_session_log`'s extended coverage) landed -2026-09-01. Single-shot Cell is now **partially hardware-verified** -(2026-09-01, `v0.8.6`): the C key runs a scan end-to-end with a real -MHz/dBm reading and correct home-channel restore, and the mutual-exclusion -guard against Sweep holds in both directions — see `docs/STATUS.md`. Still -open: a real cell-band RSSI reading confirmed rising near a known tower -(vs. floor noise the whole way through), and `cell.csv`/`session.csv`'s new -columns rendering correctly. Repeat mode (above, `v0.8.7`) is -**hardware-verified** the same day: Sweep-repeat still works after -becoming page-gated, Cell-repeat works with a correctly-placed lap -counter, R is a confirmed no-op on Probe/other pages/menu-open, and mutual -exclusion holds across a real repeat chain in both directions — see -`docs/STATUS.md`. - -**FCC A/B block markers (`v0.8.8`, added and hardware-verified -2026-09-01):** the Cell frequency bar now labels the FCC's own downlink -sub-band split (`cell_plan.h`'s `CELL_BAND_BLOCKS`, cited to 47 CFR -§ 22.905) — Block A (869-880MHz + 890-891.5MHz) and Block B (880-890MHz + -891.5-894MHz), rendered as a two-shade tick row under the bar with letter -labels on the two segments wide enough to hold one. Regulatory block letter -only — see the "Non-goals" line below for why no carrier name is shown. -Confirmed on real glass: renders cleanly with no overlap against the -surrounding lo/hi labels or the disclaimer line. - -**Non-goals, same as the rest of this project:** no GSM/CDMA/LTE decode of -any kind (impossible on this hardware, not just out of scope), no per-carrier -or per-channel identification, no claim of tower triangulation from a single -reading. - -## Distribution - -Two install paths, both real, serving different audiences: - -- **Direct flash (`pio run --target upload`)** — the dev-iteration path. - Fastest feedback loop, direct serial access for debugging. Primary - method through Phases 1–2 while bring-up is still being bench-verified. -- **SD-drop via [bmorcelli/Launcher](https://github.com/bmorcelli/Launcher)** - — the preferred *end-user* install method once builds are stable. - Matches docs/BRAND.md's "field instrument, not a laptop-tethered tool" - positioning: swap firmware without reflashing. - -This costs us nothing extra to support. Launcher installs a plain -PlatformIO app binary (`.pio/build/cardputer-adv/firmware.bin`, standard -ESP32 image starting with the `0xE9` magic byte) straight off a FAT32 SD -card — no merged image, no manifest. Launcher's own dynamic partition -manager carves out or resizes an OTA app slot to fit whatever we hand it. - -**In practice, SD-drop is already the primary test path**, not just the -eventual end-user one — it's how this gets tested on hardware that's -already running a Launcher install the operator doesn't want to disturb -with a direct USB flash. CI reflects that: every merge to `main` publishes -a rolling `dev-latest` prerelease at a stable URL -(`LoRaTraceRX-dev.bin` — see Versioning below) specifically so there's -always a current build to drop on the card without waiting on a version -tag. Direct flash remains useful when iterating fast enough that even the -CI round-trip is overhead, or for the deepest debugging (upload errors, -brick recovery). - -**Confirmed:** returning from a running LoRaTrace RX build back to -Launcher is a manual restart + button combo — not something our firmware -needs to implement. No return-to-launcher hook belongs in `ui_task`. -**2026-08-22, confirmed against Launcher's own source (not a guess):** the -combo is "press any key during Launcher's own ~5s post-reset boot window, -before it auto-chains back into whatever ran last" — or, to remove the -timing pressure entirely, enable Launcher's own Settings → "Boot to -Launcher" toggle, which makes it always stop at its menu on reset instead -of auto-booting the last app. See docs/history/PROGRESS.md "Open questions — Launcher -distribution" for the full read of Launcher's boot logic. - -**Follows from this:** Launcher owns the flash partition table, not our -static `platformio.ini` partition CSV — that only governs direct-flash -installs. Any state we want to survive a user switching firmwares back -and forth (settings, last-used profile, calibration data) has to live on -SD, not in a custom NVS/data partition, since a custom partition isn't -guaranteed to survive a later Launcher install. This is already -docs/DESIGN.md's "SD is the datastore" philosophy — see docs/history/CHANGELOG.md — it just -now extends to config, not only detection logs. - -**Binary size:** no documented hard ceiling was found for how much flash -Launcher leaves free per app on an 8MB device, especially once Launcher -itself plus other installed firmwares share the same flash (that's the -whole point of a multi-firmware launcher — see docs/history/PROGRESS.md open -questions). Rather than target an arbitrary number like "under 4MB," -treat it as an ongoing discipline: keep the binary as lean as the feature -set allows, and measure the real number instead of assuming one. - -- **Measured, not estimated:** the Phase 1 scaffold (RadioLib + IO-expander - init, no GPS/SD/display/WiFi yet) compiles to **312KB** — 9.5% of the - ~3.19MB app partition our own `default_8MB.csv` direct-flash scheme - allocates (`pio run`, logged in CI now — see below). Adding SD, GPS - parsing, and a display library will grow this, but from a 312KB - baseline there's a lot of room before size becomes a real constraint - either for direct flash or for coexisting with Launcher + other apps. - **2026-08-22 update:** adding the boot-status splash's display library - (GFX Library for Arduino) plus SD's own footprint brought this to - **406KB** (12.2% of the same partition) — a real +94KB data point, not - an estimate, confirming there's still plenty of headroom. -- **WiFi is the one feature with a real size lever**, same as it's the one - feature with the real RAM lever (`lwIP` + the WiFi driver stack is - typically the single biggest chunk of a "full" Arduino-ESP32 build). That - go/no-go already happened (Phase 3, shipped — see CLAUDE.md's Status - section for the real numbers this note used to be waiting on). -- `RadioLib` compiles in support for many radio families by default; - disabling the ones we don't use (`RADIOLIB_EXCLUDE_*` build flags, only - SX126x needed here) is a cheap, real size reduction worth doing before - Phase 9 (`ENERGY_SWEEP`, the other RAM/flash-hungry feature after WiFi), - not just a nice-to-have. -- CI now measures the actual `firmware.bin` size on every build (see - below) — track it there instead of trusting an estimate. - -## Versioning - -Formalizes the phase-mapped table below into an actual scheme CI and bug -reports can use. - -- **Format:** `vMAJOR.MINOR.PATCH` (e.g. `v0.2.1`). MAJOR.MINOR tracks the - build-order phase reached; PATCH increments for fixes that don't add new - phase scope. -- **Source of truth:** `src/version.h` (`FIRMWARE_VERSION`), printed on - the boot banner (Serial) and on `ui_task`'s SYSTEM status page (on-device, - since Phase 2). A bug report against a specific build should always be - traceable to this string. -- **Release trigger:** pushing a `vX.Y.Z` git tag. `src/version.h` must be - bumped to match before tagging; release CI rejects a tag/header mismatch. -- **CI:** `.github/workflows/build.yml` runs `pio run` (+ `pio test`) on - every push/PR — catches build breaks before they land, independent of - tagging. `.github/workflows/release.yml` runs only on a `vX.Y.Z` tag - push: builds, renames the output to `LoRaTraceRX-.bin` - (Launcher/SD-drop-friendly naming, per its "use simple characters" SD - guidance), and attaches it to a **draft** GitHub Release. -- **Rolling dev build, separate from the tagged scheme:** every push to - `main` also force-moves a `dev-latest` tag and republishes a prerelease - there with a fixed filename (`LoRaTraceRX-dev.bin`) — a stable, - no-tagging-required download for day-to-day hardware testing. It's - explicitly *not* versioned or draft-gated the way real releases are: - it can be broken, it reflects whatever's on `main` at that moment, and - its own release notes point back at the exact commit. Cutting a real - `vX.Y.Z` tag is a separate, deliberate step for when a phase is actually - bench-verified — see docs/STATUS.md for current status before trusting - either. - -| Version | Corresponds to | +# LoRaTrace RX — V2 Roadmap + +This is the active, forward-looking gate board for LoRaTrace RX. It begins +from stable `v1.0.7` and governs V2 work only. The completed v1 phase +narrative, feasibility analysis, exit criteria, and versioning history remain +at [history/ROADMAP_V1.md](history/ROADMAP_V1.md). + +Start with [STATUS.md](STATUS.md) for what is true on hardware now and +[research/V2_DESIGN.md](research/V2_DESIGN.md) for V2 product boundaries. +This document records what may enter implementation next, what proof it needs, +and what earns a release. + +## Permanent boundaries + +V2 preserves the shipped foundation: + +- Receive-only; no transmit, beacon, injection, decryption, keys, payload + display, or protocol-client behavior. +- One radio-owner task on Core 1; at most one bounded acquisition action owns + the SX1262. SD, display, GPS, and WiFi never block its real-time path. +- Fixed/static storage, bounded queues, streaming metrics, and SD as the + datastore. New capability may not silently add unbounded RAM, SD backlog, + task lifetime, or radio-away time. +- Watch remains the default. Every acquisition completion, cancel, timeout, + and failure restores the resolved home configuration and records its result. +- RSSI/CAD/packet evidence never becomes a protocol identity merely by + inference. A missed dwell is not a quiet frequency. +- WiFi remains opt-in and browser acquisition control remains out of scope + without a separate security decision. + +The detailed rationale and product wording live in +[research/V2_DESIGN.md](research/V2_DESIGN.md); do not duplicate it here. + +## Status and gate model + +| Status | Meaning | |---|---| -| v0.1.x | Phase 1 (serial bring-up) | -| v0.2.x | Phase 2 (MVP-Beta: Meshtastic War Drive complete) | -| v0.3.x | Phase 3 (Web Command Center: WiFi AP + web UI) | -| v0.4.x | Phase 4 (MeshCore) | -| v0.5.x | Phase 5 (on-device menu UI) | -| v0.6.x | Phase 6 (UI architecture redesign) | -| v0.7.x | Phase 7 (device optimization) | -| v0.8.x | Phase 8 (discovery sweep) | -| v0.9.x | Phase 9 (energy sweep: Reticulum + General Exploration) | -| v0.10.x | Phase 10 (Field Analyzer), in progress | -| v1.0.x | reached 2026-09-03 — Phase 10 (Field Analyzer)'s exit criteria closed same day, the doc's own gate for this promotion (decided 2026-09-03), see Phase 10 above | - -**Phase 11 (Cell) is a deliberate exception to this table.** -It landed as a PATCH bump (`v0.8.6`, not `v0.9.x`) because it is not the -*next* build-order phase — Phase 9 (`ENERGY_SWEEP`) is still in progress, and -jumping MINOR to 9 (or past it to 11) would misrepresent Phase 9/10 as -reached when they aren't. If a later phase completes Phase 9/10 first, this -table's normal MAJOR.MINOR-tracks-phase-reached rule resumes as before; Phase -11 doesn't get its own `v0.11.x` line unless a future revision of this table -decides it should. +| **Not entered** | Entry decisions are not yet locked. | +| **Design entry** | Scope and the measurement plan are being locked; there is no release claim. | +| **Engineering** | Code and host validation are in progress. | +| **Hardware pending** | The implementation gate is met, but device proof is incomplete. | +| **Closed** | Every applicable gate has accepted evidence. | -**Renumbered 2026-08-24** (same restructuring precedent as WiFi's Phase-3 -pull-forward): the on-device UI overhaul moved from a trailing "Phase 7 -polish" step to Phase 5, at the user's request, pushing `DISCOVERY_SWEEP` -and `ENERGY_SWEEP` down one each. What `v1.0` means moved with it — it used -to be "Phase 7, all profiles + UI stable," and Phase 7 was `ENERGY_SWEEP` -at the time, not UI. +Every workstream passes these gates in order: -**Renumbered again 2026-08-25**, same precedent a second time: reviewing -Phase 5's own menu against what `DISCOVERY_SWEEP` would add to it surfaced -that the menu had already grown past its documented two-item scope (a -third, `Debug`, row landed the same day with no framework change) — see -docs/history/CHANGELOG.md for the full session. Rather than let -`DISCOVERY_SWEEP` and `ENERGY_SWEEP` each bolt on their own ad hoc entry, a -UI architecture redesign now sits at Phase 6, pushing `DISCOVERY_SWEEP` to -7 and `ENERGY_SWEEP` to 8. `v1.0`'s meaning (all four profiles + UI stable) -is unchanged by this move — it's still a placeholder for the same total -phase count, not a re-litigated decision — revisit if it stops fitting once -Phase 6/7/8 are actually in hand. - -**Renumbered again 2026-08-26:** the measured ~32KB Phase-6 display canvas -now overlaps WiFi's ~55–56KB runtime cost, while only the logger task had a -stack watermark and the post-canvas combined-load gate was still open. -Device optimization therefore became Phase 7, before either new scan state. -`DISCOVERY_SWEEP` moved to Phase 8 and `ENERGY_SWEEP` to Phase 9; their scope -did not change. `v1.0` still meant all four profiles and their UI were stable -at that point. - -**Phase 10 added to the plan 2026-08-26:** Field Analyzer is accepted as -planned post-Sweep scope, including bounded radio-owned Scope acquisition. -This does not silently move the release gate: after Phase 9 hardware evidence -exists, explicitly decide whether Field Analyzer is part of `v1.0.x` or the -first post-v1.0 phase. - -## Non-goals - -- Transmit or injection of any kind, beyond the one-time antenna-switch - init GPIO write. Permanent, per CLAUDE.md house rules — not a phase to - eventually reach. -- Auto-detecting mission profile. Operator-selected via keyboard, by - design (docs/DESIGN.md §5). -- Full protocol decode/decrypt, payload display, or key handling. LoRaTrace - remains a metadata-first passive field instrument; safe cleartext radio - headers are the boundary. +| Gate | Required proof | +|---|---| +| **Design entry** | Scope, non-goals, operator promise, unresolved decisions, log-schema impact, and worst-case memory/queue/SD/radio-away budget are explicit. | +| **Engineering** | Host tests cover plans, bounds, state transitions, CSV formatting, and coverage math; native tests and the production build pass. | +| **Device behavior** | A real device proves request/refuse/cancel/timeout/failure paths, mutual exclusion, home restore, UI behavior, and fresh SD output. | +| **Claim truth** | Controlled RF timing or RTL-SDR ground truth validates every RF/coverage claim; absence of an observation is never relabeled as silence. | +| **Release** | WiFi-off/on resource evidence, redacted field summary, `STATUS.md` reconciliation, and operator release notes are complete. | + +Focus Survey and Field Missions also require a before/after measurement of +Watch packet opportunity. Displaying a radio-away duration does not make the +cost acceptable; the measurement is part of the decision. + +## V2 workstreams + +| Workstream | Status | Outcome and phase-specific exit gate | +|---|---|---| +| **12 — Survey truth** | **Engineering** | Define coverage vocabulary and persistent per-survey evidence, then add bounded Focus Survey. The active [design-entry and acceptance plan](research/phase12-survey-truth-design.md) locks the one-bin first slice and controlled matrix; radio-away budgets and sampled/repeated thresholds remain measurement-gated. | +| **13 — Field Missions** | Not entered | Add Drive, Stationary, and Investigate as explicit recipes with visible WATCHING/SURVEYING/RESTORING and `mission.csv` accounting. Prove transitions do not hide radio-away time or weaken action arbitration. | +| **14 — Companion analysis** | Not entered | Deliver an offline, reproducible tool that reads copied run folders without changing original evidence. Test deterministic reports, multi-run comparison, coverage warnings, and privacy-safe export behavior. | +| **15 — Field markers and sharing** | Not entered | Add fixed, safe marker presets and `marker.csv`, then integrate redacted sharing. Prove markers cannot affect radio behavior and realistic exports remove selected location/identity detail. | +| **16 — Cell closeout** | Deferred bonus | Close the existing V1 Phase 11 evidence gap: a real tower-adjacent RSSI rise plus fresh SD verification of `cell.csv` and Cell's appended `session.csv` fields. This preserves V1 history; it does not renumber it. | + +Rigorously sourced region packs are later candidates, not Workstream 16 and +not V2.0 blockers. Each proposed pack needs a separate entry gate with source +quality, regulatory/range rationale, fixed-table validation, and realistic +hardware access. + +## Current work — Workstream 12 + +The workstream is in Engineering. Its bench-only first slice now exercises one +bounded radio request, fixed statistics, and `focus.csv` persistence; it has +not added a production control or coverage/activity claim. The plan sets the +following constraints: + +- One selected Sweep/Waterfall bin or fixed preset per request; no free-form + frequency entry, arbitrary range, or multi-bin list in the first slice. +- `focus.csv` is an append-only result record, not raw sample history. +- A fixed histogram/statistics accumulator supplies median, P90, and maximum + without heap allocation; actual static/queue/row budgets must be measured. +- `insufficient`, `sampled`, and `repeated` cannot be displayed until a + controlled transmitter/RTL-SDR matrix selects their pass/time thresholds. +- Portland metro, Oregon is the privacy-preserving field-validation area; + exact location and raw GPS-bearing artifacts remain private. + +Engineering may build a bench-only raw-counter prototype so the matrix can +measure the open decisions. It may not present a coverage label or a “no +activity” conclusion until the Device/claim gate closes. + +## V2.0 composition release + +`v2.0.0` is earned only when Workstreams 12–15 pass together on one identified +build. It repeats cross-feature risks rather than aggregating old checklists: + +- every bounded radio action mutually excludes correctly; +- complete, cancel, timeout, and failure restore resolved home listening; +- append-only CSVs remain readable by the companion; +- WiFi-off/on resource trends remain healthy; and +- a field workflow demonstrates Watch-first driving, deliberate investigation, + known radio-away cost, and an explainable offline report. + +Workstream 16 (Cell closeout) is intentionally not a V2.0 blocker. + +## Version and release policy + +- `src/version.h` is the semantic-version source of truth. A release tag must + match it; CI rejects a mismatch. Each tagged release needs operator-facing + notes in `docs/RELEASE_NOTES.md`. +- `v1.0.x` is the stable maintenance line. The completed phase-number mapping + belongs to [the v1 archive](history/ROADMAP_V1.md). +- A closed core V2 workstream may earn the next stable minor release: + `v1.1.0` through `v1.4.0` for Workstreams 12–15. Workstream numbers remain + roadmap identities, not version components. +- `v2.0.0` requires the composition release gate above. +- Workstream 16 evidence closeout without an operator-facing behavior change + updates status only. A later Cell improvement may earn a `v2.1.0`-class + release after its own gates. +- A rolling `dev-latest` build remains for day-to-day hardware testing; it is + not a release gate and must be identified by build revision. + +## Evidence and history + +Use this order when deciding a gate: + +1. `docs/STATUS.md` for current accepted facts and open hardware work. +2. The workstream's design-entry document for scope, measurement method, and + acceptance criteria. +3. `docs/hardware-results/` for location-redacted evidence summaries; keep raw + serial/CSV/GPS artifacts in its git-ignored `private/` area. +4. `CHANGELOG.md` for terse post-v1 decision history. +5. [history/ROADMAP_V1.md](history/ROADMAP_V1.md) only for v1 Phase 0–11 + questions or original v1 gate rationale. + +Historic code comments and changelog entries that mention a V1 Phase refer to +the archive unless they name a later research or hardware-results record. diff --git a/docs/history/ROADMAP_V1.md b/docs/history/ROADMAP_V1.md new file mode 100644 index 0000000..3ed2f65 --- /dev/null +++ b/docs/history/ROADMAP_V1.md @@ -0,0 +1,21 @@ +# V1 roadmap archive + +The complete v1 roadmap is frozen at the stable +[`v1.0.7` release tag](https://github.com/d3mocide/LoRaTrace-RX/blob/v1.0.7/docs/ROADMAP.md), +commit `1d8ea360fda8dc3d159ea1c0e626edcbedb42114`. + +This archive intentionally points to that immutable source instead of copying +978 lines of completed phase narrative into the live documentation tree. The +tag's `docs/ROADMAP.md` content SHA-256 is: + +```text +bedf2e9f736b1da88c1a4161277e7e46ba8f21fcf39b216bb85fb7a54223048d +``` + +Use it for v1 Phases 0–11, historic feasibility decisions, distribution +details, phase-number versioning, and their original exit criteria. The live +[V2 roadmap](../ROADMAP.md) contains only active workstreams and gates. + +Historical comments and changelog entries that mention a v1 "Phase N" or the +old roadmap refer to this tagged snapshot unless they explicitly link to a +later research or hardware-results record. diff --git a/docs/research/V2_DESIGN.md b/docs/research/V2_DESIGN.md index 5d7aa28..49c443e 100644 --- a/docs/research/V2_DESIGN.md +++ b/docs/research/V2_DESIGN.md @@ -1,7 +1,7 @@ # LoRaTrace RX — Version 2 Design **Status:** draft for product decisions and phased validation. -**Baseline:** v1.0.0, released 2026-09-03. +**Baseline:** v1.0.7, released 2026-09-04. **Purpose:** describe the post-v1 direction without rewriting the hardware-verified v1 design, roadmap, or status record. ## 1. Product thesis @@ -214,23 +214,25 @@ transmitter, or packet decoder into the Cardputer. ## 6. Proposed build order -The following is a priority order, not a committed release calendar. A phase -lands only after its scope, static-memory cost, host tests, and applicable -hardware evidence are complete. Version numbers should be decided in the -canonical roadmap when the first post-v1 phase is accepted. +The following is a priority order, not a committed release calendar. A +workstream lands only after its scope, static-memory cost, host tests, and +applicable hardware evidence are complete. The canonical gate and version +policy live in `docs/ROADMAP.md`. | Workstream | Outcome | Why it comes here | |---|---|---| -| **11. Cell closeout** | Validate a real tower-adjacent RSSI rise and verify cell/session output on SD. | Finishes existing, partially verified scope before broadening the product. | | **12. Survey truth** | Coverage vocabulary, persistent per-survey evidence, and bounded Focus Survey. | Directly fixes the most important interpretation gap revealed by V1 testing. | | **13. Field Missions** | Drive, Stationary, and Investigate recipes with visible Watch availability and radio-time accounting. | Turns individual tools into a coherent operator workflow. | | **14. Companion analysis** | Offline run report, comparison, and export bundle. | Delivers more value from existing and V2 logs without consuming device heap. | | **15. Field markers and privacy-aware sharing** | Preset annotations and shareable, redacted report options. | Makes collected data interpretable and responsibly reusable. | -| **16. Region packs** | First rigorously sourced, validated non-US profile pack(s). | High value, but research-heavy and easy to get wrong if rushed. | +| **16. Cell closeout** | Validate the existing Cell feature's real tower-adjacent RSSI rise and fresh `cell.csv`/`session.csv` output. | A useful bonus feature, deliberately deferred so it does not block the core V2 field-survey workflow. | The following stay as later candidates until a concrete field question justifies -them: richer passive node-activity dossiers, antenna/install comparative -surveys, battery/endurance guidance, and additional lab correlation tooling. +them: rigorously sourced region packs, richer passive node-activity dossiers, +antenna/install comparative surveys, battery/endurance guidance, and additional +lab correlation tooling. Region packs require their own source-quality, +regulatory/range, and real-hardware-access entry gate; they are not a V2.0 +release dependency. ## 7. Architecture and resource rules @@ -321,4 +323,3 @@ observation coverage, device health, and field context. It does **not** need to become a transmitter, decoder, protocol client, cloud platform, or unlimited-band analyzer to achieve that. - diff --git a/src/version.h b/src/version.h index 3ff43ae..858a756 100644 --- a/src/version.h +++ b/src/version.h @@ -4,11 +4,12 @@ // // TWO different things live here, deliberately kept separate: // -// 1. FIRMWARE_VERSION — the semantic version, bumped BY HAND. MAJOR.MINOR -// tracks the build-order phase reached (docs/ROADMAP.md Versioning); PATCH is -// for fixes that add no phase scope. This is a *statement* that a phase -// was reached, so it must not auto-increment — a number that changes on -// every build asserts nothing. +// 1. FIRMWARE_VERSION — the semantic version, bumped BY HAND. In completed +// v1, MAJOR.MINOR tracked the build-order phase reached and PATCH covered +// fixes adding no phase scope; V2 uses docs/ROADMAP.md's workstream/release +// policy. This is a *statement* that a release gate was reached, so it +// must not auto-increment — a number that changes on every build asserts +// nothing. // // **Bump this when a phase lands, and before pushing the matching // `vX.Y.Z` tag.** CI enforces the tag/version match in release.yml, so a From ded5720bc20199ac9da5c60521b2eeed33d918a9 Mon Sep 17 00:00:00 2001 From: Will Shields Date: Fri, 4 Sep 2026 17:02:40 -0700 Subject: [PATCH 02/21] Workstream 12: bounded Focus Survey, bench evidence, controlled matrix Focus asks one narrow question -- at this selected frequency, what did the receiver observe for how long -- and answers it with one bounded, radio-owned request that restores Watch on every exit path. It is bench-image only: there is no operator control yet, `coverage` is deliberately blank in focus.csv, and no coverage or activity label can be displayed until its thresholds are earned. Firmware: a Core-1 one-bin request with a fixed 141-bucket RSSI histogram (median/P90/peak with no heap allocation), a Core-0 focus.csv writer, and a restore-before-publish terminal row. A request is bounded in wall-clock time as well as in samples -- every sample waits on the shared SPI bus, so a contended bus could otherwise stretch a pass past its dwell with the radio away from home and nothing to stop it. Mutual exclusion with Probe, Sweep, Cell, and Scope is enforced in both directions. Also fixes a latent silent-drop in Serial Control's STATUS: its argument buffer was hand-sized to 240 bytes against a frame budget of ~230, and an over-long frame is dropped rather than truncated, so a long session's wider counters would have lost the whole frame -- newest fields first, with no error. The budget is now derived from the frame size and tested at saturation. Evidence, all on real hardware: - Cancel, injected failure, timeout, and four-way arbitration each restore home before their durable row is written. - The controlled dwell matrix ran 900 trials with zero transport errors, zero queue or row drops, and home restored every time. 14 of 15 arms separated a controlled source from ambient 30/30 vs 0/30. - Budgets are measured off the built image rather than estimated: 188 B of working state against a 256 B target, a 189 B worst-case row against a 256 B buffer, 1,072 B radioTask frame in a 6,144 B stack. The matrix's most consequential result is a limit, not a capability: detection tracks the source's airtime against Focus's sample spacing, not dwell length. With a fixed 8 samples the spacing is dwell/7, so a 2,000 ms dwell observes eight instants rather than 2,000 ms, and worst-case detection degraded monotonically as source airtime approached that spacing until one arm stopped separating entirely. That is the design's own "observation time is not coverage" rule with numbers behind it, and it means a later slice must scale the sample budget with the dwell before any operator-facing dwell or observation figure can mean what it reads. Consequently no single fixed RSSI condition separates every arm, so the qualifying condition gets a documented candidate rather than an accepted constant, and the coverage thresholds stay unselected. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 125 +++++ docs/STATUS.md | 77 ++- .../2026-09-04-phase12-focus-matrix.md | 130 +++++ docs/research/phase12-survey-truth-design.md | 459 ++++++++++++++++++ scripts/phase12_focus_behavior_bench.py | 222 +++++++++ scripts/phase12_focus_bench.py | 136 ++++++ scripts/phase12_focus_matrix.py | 389 +++++++++++++++ scripts/phase12_focus_matrix_report.py | 163 +++++++ scripts/phase12_watch_opportunity.py | 256 ++++++++++ src/bench_fault.cpp | 42 ++ src/bench_fault.h | 18 + src/focus_observation.h | 213 ++++++++ src/focus_plan.h | 100 ++++ src/focus_runtime.h | 76 +++ src/logger_task.cpp | 52 +- src/logger_task.h | 6 +- src/main.cpp | 11 +- src/radio_task.cpp | 267 +++++++++- src/radio_task.h | 16 +- src/serial_control.cpp | 169 ++++++- src/serial_control_protocol.h | 56 ++- test/test_focus_observation/test_main.cpp | 156 ++++++ test/test_focus_plan/test_main.cpp | 83 ++++ test/test_focus_runtime/test_main.cpp | 74 +++ .../test_main.cpp | 78 +++ 25 files changed, 3337 insertions(+), 37 deletions(-) create mode 100644 docs/hardware-results/2026-09-04-phase12-focus-matrix.md create mode 100644 docs/research/phase12-survey-truth-design.md create mode 100644 scripts/phase12_focus_behavior_bench.py create mode 100644 scripts/phase12_focus_bench.py create mode 100644 scripts/phase12_focus_matrix.py create mode 100644 scripts/phase12_focus_matrix_report.py create mode 100644 scripts/phase12_watch_opportunity.py create mode 100644 src/focus_observation.h create mode 100644 src/focus_plan.h create mode 100644 src/focus_runtime.h create mode 100644 test/test_focus_observation/test_main.cpp create mode 100644 test/test_focus_plan/test_main.cpp create mode 100644 test/test_focus_runtime/test_main.cpp diff --git a/CHANGELOG.md b/CHANGELOG.md index 56dbbb5..e967c53 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,103 @@ project (not a log of how it got there), see [docs/STATUS.md](docs/STATUS.md). ## 2026-09-05 +- Ran Workstream 12's §6.2 controlled dwell matrix: 900 trials in 54.7 min, + zero transport errors, zero drops, home restored on every trial; 14 of 15 + arms separated a controlled source from ambient 30/30 vs 0/30. The finding + that matters is a limit — **detection tracks the source's airtime against + Focus's sample spacing, not dwell length**. With a fixed 8 samples, spacing + is dwell/7, so a 2000 ms dwell observes eight instants rather than 2000 ms; + worst-case detection degraded monotonically as source airtime approached + that spacing and one arm stopped separating. That is §3's "observation time + is not coverage" rule with numbers behind it, and it means a later slice + should scale samples with dwell. Bin-center offset turned out not to be the + dominant term after all. No single fixed RSSI condition separates every arm, + so §3.1 gets a candidate rather than a constant. Summary in + `docs/hardware-results/2026-09-04-phase12-focus-matrix.md`. + +- Two methodology bugs found by running the §6.2 matrix rather than by + reading it. A single armed pulse at a fixed delay cannot cover every dwell + arm — at 100 ms the pulse began after the window closed, and source-on was + indistinguishable from source-off across 18 trials. Re-arming continuously + fixed that but at SF12's ~275 ms airtime the sends overlapped and their tail + bled into the next trial, making source-off read -26 dBm. The runner now + paces the burst by the transmitter's own `TX_DONE` and waits for the tail to + clear before a trial ends: 100 ms arms then separate cleanly (-25/-27 dBm + against -97/-101 dBm). Both failure modes are recorded in the design doc so + the next harness doesn't rediscover them. + +- Added `scripts/phase12_watch_opportunity.py` for §6.3, the measurement that + gates Focus's maximum radio-away budget: Watch-only versus Watch-with-Focus + against one independently timed pulse train, reported as Wilson intervals + plus away time, completed requests, and restored-Watch gap distribution. It + refuses to run unless the device's home channel matches the reference train, + since otherwise both arms hear nothing and their equality would read as a + reassuring result, and it reports the away fraction alongside the loss + because a small loss at a small duty cycle predicts nothing about a larger + one. + +- Built Workstream 12's §6.2 matrix tooling: `scripts/phase12_focus_matrix.py` + runs paired source-on/source-off Focus trials across dwell arms and writes + one durable JSONL row each, and `scripts/phase12_focus_matrix_report.py` does + the offline analysis — per-arm distributions, the lowest separating 1 dB + condition or an explicit "these overlap", and Wilson 95% intervals rather + than bare proportions. Neither can emit a coverage label. Validated on + hardware with a 4-trial smoke run at 912.750 MHz: source-on P90 -66/-68 dBm + against -97/-98 dBm quiet, 573 ms radio-away, no drops. + +- Found that none of §6.1's three fixture frequencies sits on a Focus bin + center. Focus tunes at the home channel's bandwidth (125 kHz here), so the + offset changes what a pass can observe: low and mid are +62.5 kHz off bins 13 + and 43, and the high position sits exactly on a bin boundary, 125 kHz from + either neighbour — outside the passband. A null there would measure geometry, + not sensitivity, so the matrix now also runs two exactly bin-centered + controls from the same transmitter table, and every trial records its offset. + +- Closed Workstream 12's last Engineering-gate item by measuring the Focus + budgets off the built image instead of estimating them + (`docs/research/phase12-survey-truth-design.md` §4.2): 12 B request, 40 B + result, 148 B histogram, 188 B working state against a 256 B target, 159 B + of Focus statics, a 1,072 B `radioTask` frame in a 6,144 B stack, and a + 189 B worst-case `focus.csv` row against a 256 B buffer that drops rather + than truncates. Everything landed inside the bounds §4.1 had set, so + nothing needed resizing. The row buffer now comes from one shared + `FOCUS_CSV_ROW_MAX` and a test formats the saturated row, so the writer and + its budget can't drift the way `STATUS`'s hand-sized buffer did. + +- Phase 12: bounded a Focus request in wall-clock time, not just in samples. + Each sample waits on the shared SPI bus, so a contended bus could stretch a + pass past its dwell with the radio away from home and nothing to stop it — + `focusRuntimeTimeout()` existed but no code path reached it. A request now + carries a dwell-plus-slack deadline and terminates as `timeout`, restoring + home like every other exit. Bench-only additions make the remaining Device + gate reproducible: `BENCH_FOCUS`'s optional 4th field arms a one-shot + sample-loop stall (production only times out under real contention), and + `BENCH_ACTION` starts/cancels/reports Cell and Scope, which are menu-only in + production, so Focus's mutual exclusion against them can be driven by a + fixture. Verified on hardware the same day: a stalled request timed out, + restored home, and wrote its durable row inside a 1,573 ms radio-away + window, and Cell and Scope each arbitrated with Focus in both directions. + +- Fixed a latent silent-drop in Serial Control's `STATUS`: its argument buffer + was hand-sized to 240 bytes against a frame budget of ~230, and + `serialControlFormatFrame()` drops an over-long frame rather than truncating + — so a long session's wider counters would have lost the whole status frame, + newest fields first, with no error. The argument budget is now derived from + the frame size, the frame limit is 384, and a host test formats at + saturation. The formatter also builds in place instead of staging through a + second full-size buffer, which nets less stack on the 4KB UI task than + before. + +- Phase 12 engineering: wired one bounded, bench-only Focus Survey through + Core 1, the Core-0 `focus.csv` writer, and restore-before-publish terminal + state. Added framed `BENCH_FOCUS` and GPS-free compact + `BENCH_FOCUS_RESULT` readback, durable-write health counters, and the + explicit-transmit-gated `scripts/phase12_focus_bench.py` plus + non-transmitting `scripts/phase12_focus_behavior_bench.py` fixture + harnesses. Quiet/source-on, cancel, injected-failure, and Probe/Sweep + arbitration hardware checks completed with successful home restore and SD + rows; they do not set a coverage/activity threshold. + - `pages.yml`'s "Download stable release assets" step now retries (6x/20s) instead of failing hard: a maintainer publishing a draft release before release.yml's build job finishes uploading assets makes `release: @@ -78,6 +175,34 @@ project (not a log of how it got there), see [docs/STATUS.md](docs/STATUS.md). ## 2026-09-04 +- Archived the completed v1 roadmap at the immutable `v1.0.7` tag and + streamlined `docs/ROADMAP.md` into the active V2 gate board. Updated + `CLAUDE.md`, `AGENTS.md`, and documentation indexes so future work opens + V2 gates/design first and searches v1 history only when needed. + +- Opened V2 Workstream 12 (Survey truth) at Design entry: one selected-bin + Focus Survey, fixed streaming statistics, durable `focus.csv` contract, + controlled transmitter/RTL-SDR truthfulness matrix, and Portland metro, + Oregon as the privacy-preserving field-validation area. No firmware scope + has entered implementation; radio-away and coverage thresholds remain + evidence-gated. + +- Began Workstream 12 Engineering with host-tested, radio-free Focus request + and `focus.csv` contracts: a single sourced bin, 1 dB bounded RSSI histogram + (median/P90/peak), explicit completion states, and an intentionally blank + `coverage` column. No menu, radio task, logger, or coverage conclusion is + wired yet. + +- Tightened the Focus bench-request contract before radio integration: exactly + one pass, 2--2,000 ms dwell, and 2--64 samples. Added a host-tested state + machine that records restoration before a terminal result and makes restore + failure override partial/cancelled work. + +- Wired the bench-only Focus vertical slice: a Core-1 one-bin RSSI request, + mutually exclusive ownership/restoration, four-row Core-1-to-Core-0 queue, + durable `focus.csv`, and framed `BENCH_FOCUS bin:dwell_ms:samples` control. + The production image rejects the command; hardware evidence is still open. + - Ran a `/code-review` pass and then a whole-project audit (`docs/research/2026-09-04-project-audit.md`), and fixed what it found (`v1.0.6`). Radio-ownership bugs in v1.0.5's new capture window: diff --git a/docs/STATUS.md b/docs/STATUS.md index 5bbccea..ada9490 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -3,14 +3,16 @@ The one place "where is this project right now" lives. Replaces status prose that used to be duplicated (and drifting) across `CLAUDE.md`, `PROGRESS.md`, and `README.md`. For *how* we got here, see -`docs/history/CHANGELOG.md`; for the phase-by-phase build order, see -[ROADMAP.md](ROADMAP.md). +`docs/history/CHANGELOG.md`; for active V2 workstream gates, see +[ROADMAP.md](ROADMAP.md); for the completed v1 phase record, see +[ROADMAP_V1.md](history/ROADMAP_V1.md). ## Current version -**v1.0.6** (`src/version.h`). `MAJOR.MINOR` tracks the build-order phase -*reached*, not the phase in progress — see ROADMAP.md's Versioning -section. Phase 9 (`ENERGY_SWEEP`/"Sweep") reached 2026-09-03: all five +**v1.0.7** (`src/version.h`). In the completed v1 series, `MAJOR.MINOR` +tracks the build-order phase *reached*, not the phase in progress — see +ROADMAP.md's Versioning section for the V2 workstream policy. Phase 9 +(`ENERGY_SWEEP`/"Sweep") reached 2026-09-03: all five ROADMAP.md exit criteria closed, including two full 8-hour endurance soaks that caught and fixed a real `logger_task` stack overflow and a proactive `radio_task` stack margin fix — see "What's hardware-verified" @@ -33,6 +35,71 @@ sequence, outside the original four-profile scope); its own two open items (below) are known, tracked gaps post-`v1.0`, an explicit operator call, not an oversight. +`v1.0.7` is the stable V2 planning baseline. It is an operator-navigation +PATCH release (Tools and Analyze moved into the menu and Activity became the +read-only bounded-action status page); it does not change any completed phase +gate or the two open Cell evidence items. + +## V2 entry state + +The V2 product direction is adopted for planning, with its canonical workstream +and gate policy in `docs/ROADMAP.md` and the detailed design in +`docs/research/V2_DESIGN.md`. **Workstream 12 (Survey truth) is in +Engineering.** Its bench-only first slice locks one-bin request, bounded RSSI +summary, and `focus.csv` contracts; it has not added a production control or +coverage label. The active +[design-entry and acceptance plan](research/phase12-survey-truth-design.md) +keeps radio-away budgets and coverage thresholds open until controlled evidence +selects them. + +The bench image now has one bounded Core-1 Focus request and a Core-0 +`focus.csv` writer. Two paired 500 ms/eight-sample smoke checks at US Sweep +bin 43 (912.750 MHz) observed P90/peak at -101.0/-101.0 dBm while quiet and +-82.0/-82.0 then -87.0/-87.0 dBm during capped controlled 912.8125 MHz Heltec +pulses—a 14--19 dB rise. Every request restored home listening and durably +wrote its row. This is only fixture/transport evidence, not a calibration, +coverage label, or activity claim. The repeatable bench harness is +`scripts/phase12_focus_bench.py`; its pulse path requires explicit +`--with-pulse --allow-transmit`. + +The non-transmitting Focus behavior fixture also proved cancelled and injected +failed rows restore before their durable write, and proved two-way refusal +with Probe and Sweep. + +A Focus request is now bounded in wall-clock time as well as in samples: past +a dwell-plus-slack deadline it stops sampling and terminates as `timeout`, +restoring home like every other exit path. That path and two-way Cell/Scope +refusal are hardware-verified as of 2026-09-04 on bench build `3e31daa-dirty`. +An injected 1,500 ms sample-loop stall against a 100 ms request (1,100 ms +deadline) produced a `timeout` row with successful home restore and 1,573 ms +radio-away time; Focus and Cell, then Focus and Scope, each refused the other +while it owned RX. Six terminal requests wrote six durable rows with zero +queue or row drops. Reaching those paths on demand needs bench-image-only +entrypoints (the stall is `BENCH_FOCUS`'s optional fourth field; +`BENCH_ACTION` starts and cancels the otherwise menu-only Cell and Scope); +production rejects both. No coverage/activity claim is enabled. + +The §6.2 controlled dwell matrix ran on 2026-09-04: 900 trials (5 positions x +3 dwell arms x 30 source-on and 30 source-off) in 54.7 min with zero transport +errors, zero queue or row drops, and successful home restore on every trial. +14 of 15 arms separated a controlled source from ambient completely (30/30 vs +0/30). Its most consequential result is a limit, not a capability: **detection +tracks the source's airtime against Focus's sample spacing, not dwell length** +— with a fixed 8 samples a 2000 ms dwell observes eight instants, not 2000 ms, +and one arm with a 94 ms source stopped separating there entirely. Bin-center +offset was not the dominant term. Radio-away measured dwell plus ~73 ms, worst +case 2,139 ms. No single fixed RSSI condition separates every arm, so the +qualifying condition has a candidate (`p90 >= -90 dBm`) but not an accepted +constant, and the coverage thresholds remain unselected. Full location-redacted +summary: [hardware-results/2026-09-04-phase12-focus-matrix.md](hardware-results/2026-09-04-phase12-focus-matrix.md). +The §6.3 Watch-opportunity comparison has tooling but has not run, so Focus's +maximum radio-away budget is still unapproved. + +The existing Phase 11 Cell feature remains partially hardware-verified and +visible in "What's still open." It is deliberately scheduled as **V2 +Workstream 16 — Cell closeout**, an optional post-core-V2 bonus: it does not +block Workstreams 12–15 or the V2.0 composition release. + ## What's hardware-verified Phases 0-9 are complete and hardware-verified: radio bring-up (Phase 1), diff --git a/docs/hardware-results/2026-09-04-phase12-focus-matrix.md b/docs/hardware-results/2026-09-04-phase12-focus-matrix.md new file mode 100644 index 0000000..e8800eb --- /dev/null +++ b/docs/hardware-results/2026-09-04-phase12-focus-matrix.md @@ -0,0 +1,130 @@ +# Phase 12 §6.2 controlled dwell matrix — 900 trials + +**Build:** `V=1.0.7;R=3e31daa-dirty;BENCH=1` (bench image; production rejects +every command used here). +**Date:** 2026-09-04. **Location:** bench, indoors. No coordinates, route, or +observed third-party identity appears here or in the tracked artifacts. +**Fixture:** repo-owned Heltec V4 R8 controlled transmitter, output capped at +-9 dBm, quieted on every exit path. +**Raw evidence:** `private/phase12-matrix-20260904T223047Z.{jsonl,log}` +(git-ignored). Re-derive this summary with +`scripts/phase12_focus_matrix_report.py`. + +900 trials — 5 positions x 3 dwell arms x 30 source-on and 30 source-off, +alternating — completed in 54.7 min with **zero transport errors, zero queue +drops, zero dropped durable rows, and successful home restore on every trial**. +Each trial was one independently logged bounded Focus request. + +## Result by arm + +Ordered by the transmitter's measured airtime, which is what the outcome +actually tracks. `margin` is the gap between the worst source-on reading and +the strongest source-off reading; a negative margin means the sets overlap. + +| Position | tx airtime | offset from bin center | dwell | source-on min | source-off max | margin | +|---|---|---|---|---|---|---| +| low (905.3125) | 1751 ms | +62.5 kHz | 100 ms | -29.0 | -94.0 | 65.0 dB | +| low | 1751 ms | +62.5 kHz | 500 ms | -29.0 | -95.0 | 66.0 dB | +| low | 1751 ms | +62.5 kHz | 2000 ms | -29.0 | -75.0 | 46.0 dB | +| mid (912.8125) | 1026 ms | +62.5 kHz | 100 ms | -29.0 | -97.0 | 68.0 dB | +| mid | 1026 ms | +62.5 kHz | 500 ms | -29.0 | -98.0 | 69.0 dB | +| mid | 1026 ms | +62.5 kHz | 2000 ms | -32.0 | -92.0 | 60.0 dB | +| low-aligned (908.750) | 286 ms | 0 kHz | 100 ms | -55.0 | -97.0 | 42.0 dB | +| low-aligned | 286 ms | 0 kHz | 500 ms | -61.0 | -97.0 | 36.0 dB | +| low-aligned | 286 ms | 0 kHz | 2000 ms | -71.0 | -97.0 | 26.0 dB | +| mid-aligned (918.500) | 148 ms | 0 kHz | 100 ms | -32.0 | -97.0 | 65.0 dB | +| mid-aligned | 148 ms | 0 kHz | 500 ms | -87.0 | -92.0 | 5.0 dB | +| mid-aligned | 148 ms | 0 kHz | 2000 ms | -33.0 | -97.0 | 64.0 dB | +| high (920.625) | 94 ms | +125 kHz | 100 ms | -45.0 | -97.0 | 52.0 dB | +| high | 94 ms | +125 kHz | 500 ms | -47.0 | -96.0 | 49.0 dB | +| **high** | 94 ms | +125 kHz | **2000 ms** | **-99.0** | **-96.0** | **-3.0 dB** | + +In the 14 arms that separated, every one separated completely: 30/30 source-on +detected and 0/30 source-off, Wilson 95% intervals [0.886, 1.000] and +[0.000, 0.114]. + +## What the matrix established + +**1. Detection tracks airtime against sample spacing, not dwell length.** +Focus takes a fixed 8 instantaneous RSSI samples spread across the dwell, so +sample spacing is `dwell / (samples - 1)`: 14 ms, 71 ms, and 286 ms for the +three arms. Detection holds while the source's airtime is comfortably longer +than that spacing and degrades as it approaches it: + +| tx airtime | vs 286 ms spacing (2000 ms arm) | worst-case source-on across dwells | +|---|---|---| +| 1751 ms, 1026 ms | much longer | flat: -29, -29, -29 / -29, -29, -32 | +| 286 ms | comparable | degrades: -55, -61, -71 | +| 148 ms | shorter | erratic: -32, -87, -33 | +| 94 ms | much shorter | fails at 2000 ms: -45, -47, -99 | + +This is the practical form of the §3 contract's warning that observation time +must not be read as time covering a frequency. **A 2000 ms dwell does not +observe 2000 ms — it observes eight instants.** Holding the sample count fixed +while lengthening the dwell makes the pass strictly worse at catching bursts, +which is the opposite of what "longer observation" suggests to a reader. Any +operator-facing presentation of dwell or observation time has to account for +this, and a later slice should scale the sample budget with the dwell rather +than leaving it fixed at 8. + +**2. Bin-center offset had no detectable effect at these signal levels.** This +was the concern that motivated adding the two exactly-centered control +positions: the `high` position sits 125 kHz from its bin center, outside the +125 kHz home-channel passband half-width. It still separated by 52 dB and +49 dB at the two shorter dwells, while the perfectly centered `low-aligned` +position separated by only 42/36/26 dB. Offset is therefore not what +distinguishes these arms — airtime is. This does not prove offset is +irrelevant at realistic signal levels; it proves it is not the dominant term +at a source this strong. + +**3. Radio-away time is dwell plus a small fixed overhead.** Measured 173 ms, +573 ms, and 2073 ms for the 100/500/2000 ms arms — a consistent ~73 ms of +retune-and-restore overhead, with a worst case of 2139 ms across all 900 +trials. No trial lost radio ownership or failed to restore home listening. + +## What it did not establish + +**No single fixed RSSI condition separates every arm.** Pooled across all 900 +trials the sets overlap (source-on min -99.0 dBm against source-off max +-75.0 dBm), and they still overlap with the one failing arm excluded +(-87.0 against -75.0). The nearest thing to a clean global condition is +`p90 >= -90 dBm`, which across the 14 separating arms misses no source-on +trial and takes 1 ambient false hit in 420. That is a **candidate** for §3.1, +not an accepted constant, and it carries three limits: + +- The bench source reads around -26 dBm against a -100 dBm ambient floor — + far stronger than realistic traffic. This separates a strong known source + from ambient; it is **not** a sensitivity limit and must not be shown as one. +- Source-off trials are ambient observation, not a calibrated false-hit rate: + the control path is not known quiet (§6.1). The single -75 dBm ambient + reading is a real received event, not necessarily noise. +- A condition chosen on `p90` of 8 samples inherits finding 1: it describes + eight instants, so its meaning changes with the dwell it ran at. + +**The coverage thresholds remain unselected.** `FOCUS_MIN_VALID_PASSES`, +`FOCUS_MIN_OBSERVATION_MS`, and the `repeated` pair are about pass counts and +accumulated time across repeated requests; a single-pass matrix cannot supply +them. `insufficient`/`sampled`/`repeated` stay undisplayable, and `coverage` +stays blank in `focus.csv`. + +**The §6.3 Watch-opportunity comparison has not run**, so the maximum +radio-away budget is still unapproved. + +## Method notes worth not relearning + +Two harness defects were found by running this, each of which produced +confident, wrong-looking data before it was caught: + +1. **One armed pulse at a fixed delay cannot cover every dwell arm.** At + 100 ms the pulse began after the window closed, and source-on was + indistinguishable from source-off across 18 trials — readable as "a 100 ms + dwell detects nothing." +2. **Re-arming faster than the airtime overlaps transmissions**, and the tail + bleeds into the next trial: source-*off* trials read -26 dBm — readable as + "the receiver hears a transmitter that is switched off." + +The runner now paces its burst by the transmitter's own `TX_DONE` and waits +for the tail to clear before a trial ends. A source-on trial consequently means +*the transmitter was radiating during the window*, which is the condition the +qualifying-RSSI question needs — and explicitly **not** a catch-probability +estimate for intermittent traffic. diff --git a/docs/research/phase12-survey-truth-design.md b/docs/research/phase12-survey-truth-design.md new file mode 100644 index 0000000..02f30a3 --- /dev/null +++ b/docs/research/phase12-survey-truth-design.md @@ -0,0 +1,459 @@ +# Phase 12 Survey Truth — design entry and acceptance plan + +**Status:** design entry active; it permits a bench-only raw-counter prototype, +not operator-facing coverage labels or release scope. +**Baseline:** LoRaTrace RX `v1.0.7` on Cardputer-Adv + Cap LoRa-1262. +**Field-validation area:** Portland metro, Oregon. This is a coarse test-area +label only; no coordinates, route, node identity, or claim about local RF +occupancy belongs in tracked evidence. + +## 1. Why this phase exists + +Phase 9 established that a normal Sweep's short bin dwell can miss genuine +traffic. The real-traffic dwell comparison was inconclusive, and the later +capture investigation showed that packet capture is governed by time parked on +the home channel, not by making a full-band bin dwell slightly longer. Neither +result licenses the UI to call an empty bin quiet. + +Phase 12 therefore adds a deliberate, bounded answer to a narrower question: + +> At this selected frequency, what did the receiver observe for how long, and +> what did that observation contain? + +It does not try to infer a protocol, prove absence, turn Sweep into background +retuning, or improve packet capture by accident. + +## 2. Scope and non-goals + +The first Focus Survey slice is deliberately small: + +- **One selected frequency per request.** The source is one completed Sweep + or Waterfall bin, or one fixed menu preset. Free-form frequency entry, + contiguous windows, and multi-bin requests are deferred until this slice's + actual static-memory and radio-away measurements support them. +- **One bounded radio-owned request.** It snapshots the resolved home channel, + gathers fixed streaming statistics, writes one durable result row, and + restores Watch on complete, cancel, timeout, and failure. +- **No raw RF samples or packet bytes.** The radio task retains only fixed + accumulators; logger/GPS/SD remain outside its real-time path. +- **No identity claim.** RSSI and CAD/packet events, if a later slice adds + them, remain observations or unknown LoRa candidates under the existing + evidence rules—not Meshtastic, MeshCore, Reticulum, LoRaWAN, or cellular + identity by inference. +- **No automatic action.** Focus is operator-selected and menu reachable; + Drive/Stationary/Investigate recipes belong to Workstream 13. + +## 3. Observation and coverage contract + +The result presents *coverage* separately from *activity*. + +| Term | Definition for Phase 12 | Must not mean | +|---|---|---| +| Requested pass | One requested dwell at the selected frequency with a declared time and sample budget. | A successful measurement. | +| Valid pass | A requested pass that configured the selected frequency, produced its required samples, and was neither cancelled nor radio-error terminated. | A quiet channel. | +| Observation time | Measured elapsed time from the first accepted sample through the last accepted sample, summed only across valid passes. | Time continuously covering a frequency outside those windows. | +| Observed activity | A documented qualifying RSSI condition or independently recorded packet/CAD event during a valid pass. | Protocol identity or continuous activity. | +| No observation | No qualifying event occurred during the valid windows. | “Quiet,” “empty,” or “absent.” | +| Insufficient / sampled / repeated | Coverage labels derived only from valid-pass count and accumulated observation time. | Signal strength, likelihood of absence, or confidence in identity. | + +The firmware must store the underlying counts and durations even after a label +is chosen. The display may say `N of M passes` and elapsed observation time; +it must never replace those values with a single confidence word. + +### 3.1 Labels not yet frozen + +The first implementation must not hardcode arbitrary thresholds. Before the +Device/claim gate closes, the controlled matrix below selects: + +1. `FOCUS_MIN_VALID_PASSES` and `FOCUS_MIN_OBSERVATION_MS` for **sampled**; +2. `FOCUS_REPEATED_VALID_PASSES` and `FOCUS_REPEATED_OBSERVATION_MS` for + **repeated**; and +3. one documented qualifying RSSI condition for the activity count. + +Until those constants are accepted, a bench-only prototype may report only `N +valid passes`, requested/completed status, total observation time, and RSSI +summary. It may not show a coverage label or a “no activity” conclusion. + +## 4. Bounded data design + +`focus.csv` is append-only and purpose-specific. Existing detection, session, +probe, energy, and Cell records keep their current meanings and schemas. + +The planned one-row-per-selected-bin result contains the shared run/GPS +columns, then: + +```text +rx_uptime_ms,profile,focus_id,selection_source,selection_bin_index, +freq_mhz,requested_passes,valid_passes,requested_dwell_ms, +observation_ms,requested_samples,sample_count,rssi_median_dbm, +rssi_p90_dbm,rssi_peak_dbm,qualifying_count,coverage,request_status, +home_restore,wifi_on,radio_status +``` + +`coverage` is empty until §3.1 closes. `request_status` distinguishes complete, +cancelled, timeout, and failed; a cancelled or failed request cannot borrow a +coverage label from its partial samples. `home_restore` is recorded separately +because “complete” without Watch recovery is not a valid acquisition result. + +### 4.1 Fixed statistics budget + +The initial implementation uses one selected bin and a fixed 1 dB RSSI +histogram over a documented bounded receiver range, plus count, sum, maximum, +elapsed-time, and saturation flags. This gives median and P90 without raw +sample retention or a heap allocation. The exact histogram range, bucket type, +request/result struct sizes, queue depth, and worst-case CSV row length are +part of the Engineering-gate budget; the target is a single result state under +256 bytes of static SRAM before queue/control overhead. + +This is a target, not a claim of current usage. A wider focus set, dynamic +container, second result history, or per-event allocation is out of scope for +this slice and requires a new budget decision. + +### 4.2 Measured budget (2026-09-04) + +Counted from the built production image and a host measurement of the +formatter, not estimated. Struct sizes are additionally held by `static_assert` +so a schema change cannot quietly exceed them. + +| Item | Measured | Bound | +|---|---|---| +| `FocusRequest` | 12 B | 16 B | +| `FocusObservation` (queue/result record) | 40 B | 48 B | +| `FocusRssiHistogram` (141 one-byte buckets, -140..0 dBm) | 148 B | 160 B | +| One result's working state (histogram + record) | 188 B | 224 B, under §4.1's 256 B target | +| All Focus statics in the production image | 159 B | — | +| Request queue item storage | 1 x 12 B | — | +| Result queue item storage | 4 x 40 B = 160 B | — | +| `radioTask` stack frame | 1,072 B | 6,144 B task stack | +| Worst-case `focus.csv` row | 189 B | 256 B buffer (`FOCUS_CSV_ROW_MAX`) | +| SD write rate | one row per terminal request | — | + +Notes on what those numbers do and do not cover: + +- The stack frame is the whole radio task's, measured with `-fstack-usage`; + every bounded action is inlined into it, so it is a shared ceiling rather + than Focus's own cost. Focus's objects account for at most 216 B of it. + The device's own measured high-water is `radio_stack_free` in `session.csv`, + which this budget has not yet read back off a card. +- The worst-case row uses every field at its widest, including widths a + bounded request cannot produce (a 65,535 ms dwell, a 255-pass count). An + over-long row is dropped rather than truncated—`focusObservationFormatCsv()` + returns 0 and the logger counts a dropped row—so the 66 B of margin is + deliberate and regression-tested at the maximum. +- The 326 B header is written straight to the file (`File::println`) with no + intermediate buffer, so it is not bound by the row budget. +- Measured radio-away time across the 2026-09-04 behavior run was 74 ms + (an injected failure, which never sampled) to 1,573 ms (a 1,500 ms injected + stall). That is fixture timing, not the §6.3 Watch-opportunity decision. + +## 5. Radio and UI contract + +`radio_task` owns Focus exactly as it owns Probe, Sweep, Cell, and Scope. At +most one bounded acquisition action owns the SX1262. A Focus request is visibly +refused when another action owns the radio; it never waits behind it. Every +exit path restores the resolved home configuration before publishing its final +state. + +The on-device menu supplies the runtime control required by `CLAUDE.md`. The +status surface and Activity page show the selected frequency, source, valid/ +requested passes, observation time, request state, and recovery result. +`WATCHING`, `SURVEYING`, and `RESTORING` remain mutually exclusive visible +states. Browser observation remains read-only; adding browser acquisition +control would require a separate security decision. + +Focus's radio-away measurement starts when it leaves resolved home listening +and ends only after `restoreHomeListen()` reports success or failure. It is a +new Focus-specific health/status value; Sweep's `EA` is evidence for Sweep, +not a substitute for this measurement. + +The sample budget alone does not bound that measurement. Every sample waits on +the shared SPI bus, so a contended bus stretches a pass past its dwell while +the radio is away from home. A request therefore also carries a wall-clock +deadline (`focusRequestTimeoutMs()`: dwell plus a slack constant): past it the +request stops sampling and terminates as `timeout`, restoring home like any +other exit path. A recovered timeout stays a timeout—it does not borrow +`complete` from a restore that worked—and it contributes no valid pass and no +observation time. The slack is a bench-slice bound; §6 may revise it. + +## 6. Controlled measurement matrix + +The matrix establishes whether the product's bookkeeping and wording are true. +It does not seek a universal RF sensitivity number. + +### 6.1 Fixture and ground truth + +- Use the existing repository-owned Heltec V4 R8 controlled transmitter and + bench protocol; it is the timing source, not the Cardputer. +- Exercise three sourced US Sweep positions already used by the Phase 9 + low/mid/high harness: 905.3125, 912.8125, and 920.625 MHz. This checks + selection and bin translation across the accessible US range without + inventing a Portland channel plan. +- **None of those three sits on a Focus bin center, and the offset is part of + the measurement.** Focus tunes at the resolved home channel's bandwidth + (125 kHz on this bench), so where the transmitter sits inside the bin + changes what a pass can observe. Measured against the US 250 kHz grid: + 905.3125 and 912.8125 MHz are +62.5 kHz from bins 13 and 43, at the edge of + a 125 kHz passband's half-width; 920.625 MHz sits exactly on a bin boundary, + 125 kHz from either bin 74 or 75 — outside it. A weak or absent rise at the + high position would therefore measure the offset, not the receiver, so the + matrix also runs two exactly bin-centered controls from the same repo-owned + candidate table: `LONG_TURBO` 908.750 MHz (bin 27) and `MESH_OREGON` + 918.500 MHz (bin 66). Every trial records its own offset; an arm that fails + to separate is reported as unresolved, never as a quiet frequency. +- Record transmitter send timestamps and a time-aligned RTL-SDR capture when + making an activity-detection claim. The receiver's own `focus.csv` cannot + prove a missed event. +- Run source-on and source-off controls in alternating order. A source-off + antenna-in-room run is ambient observation only—not a calibrated false-hit + rate—unless the fixture/control path itself is known quiet. +- **A source-on trial means the transmitter was radiating for the whole + window.** One armed pulse at a fixed delay cannot be scheduled against every + arm: measured airtimes across this candidate table span ~10 ms (SF8/BW250) + to ~275 ms (SF12/BW125), and the receiver's window opens tens of ms after + the request is accepted, so a delay that lands inside a 2,000 ms dwell + misses a 100 ms one entirely. The first matrix attempt measured exactly + that: at 100 ms, source-on and source-off were indistinguishable because the + pulse began after the window closed. The runner therefore re-arms + continuously for the trial's duration, paced by the transmitter's own + `TX_DONE` — a fixed interval shorter than the airtime queues overlapping + sends whose tail bleeds into the next trial, which the second attempt then + measured as source-off trials reading -26 dBm. + This makes the matrix a measurement of the qualifying RSSI condition under a + known-present source. It is explicitly **not** a catch-probability estimate + for intermittent traffic; §1 already records that comparison as inconclusive + and this is not a second attempt at it. +- The bench source is also far stronger than realistic traffic (source-on + reads near -26 dBm against a -100 dBm floor at these antenna separations). + A threshold selected here separates a strong known source from ambient; it + is not a sensitivity limit and must not be presented as one. + +The SX1262 documentation and its CAD application note remain the authoritative +hardware references; device-specific timing and RSSI behavior are accepted +only from this board's measured matrix, not copied from a datasheet. + +### 6.2 Candidate dwell matrix + +Test three candidate dwell budgets—100 ms, 500 ms, and 2 s—at each fixture +frequency. They are measurement arms, not shipped defaults. For each arm: + +1. run at least 30 source-on and 30 source-off requests in randomized or + alternating order; +2. retain transmitter log, Cardputer framed status/boot identity, fresh + `focus.csv`, `session.csv`, and SDR correlation where applicable; +3. verify count/time arithmetic against timestamps and the independent source; +4. record request completion, cancellation latency, timeout, failure, restore, + queue/row drops, and actual radio-away time; and +5. report activity opportunity as a binomial proportion with a 95% interval, + never as a certainty. Use Wilson or exact intervals rather than a symmetric + normal approximation at small counts. + +`scripts/phase12_focus_matrix.py` runs the arms and writes one JSONL row per +trial; it requires explicit `--allow-transmit`, quiets the transmitter on every +exit path, and rejects a trial that loses home restore, misses its sample +count, or drops a queued or durable row. It selects no threshold. +`scripts/phase12_focus_matrix_report.py` is the offline, deterministic analysis +over that file: per arm it prints the source-on and source-off distributions, +the lowest 1 dB condition that separates them (or reports that they overlap), +and Wilson 95% intervals for both detection proportions. Neither script can +emit a coverage label. + +An arm is rejected if it loses radio ownership, cannot restore home listening, +has nonzero unexplained queue/row drops, or its durable row disagrees with the +measured request. No hit-rate target is set until the source waveform, link +budget, and qualifying condition are locked; a deceptively precise percentage +would answer a different question than coverage. + +### 6.3 Watch-opportunity measurement + +Before accepting a dwell/pass budget, compare: + +- Watch-only baseline at the active home channel against the independent + transmitter/reference count; and +- the same interval with repeated, explicitly bounded Focus requests. + +Report received/reference packets with a 95% interval, Focus away time, +completed requests, and the distribution of time between restored Watch +windows. The acceptance decision is an operator product decision after the +measurement; it must not imply that an away-time display makes the loss free. + +`scripts/phase12_watch_opportunity.py` runs both arms against one independently +timed pulse train and reports exactly those figures. Two constraints make the +comparison mean anything, and the script enforces the first and reports the +second: + +- The reference train must be receivable by Watch, so it uses the one + transmitter candidate matching this bench's resolved home channel + (`MESH_OREGON`, 918.5 MHz SF8/BW125/CR4-5/sync 0x2B) and refuses to run if + the device's home channel is something else. Otherwise both arms measure + nothing and their equality would look like a reassuring result. +- The loss figure is only interpretable next to the **away fraction**: the + share of the arm's wall time Focus actually held the radio. Host round-trip + latency inflates the idle gaps between requests, so a small measured loss at + a small away fraction says nothing about a larger duty cycle. The script + prints both and refuses to reduce them to one number. + +### 6.4 Matrix result (2026-09-04) + +900 trials completed with zero transport errors, zero queue or row drops, and +successful home restore on every trial. The full location-redacted summary is +[docs/hardware-results/2026-09-04-phase12-focus-matrix.md](../hardware-results/2026-09-04-phase12-focus-matrix.md); +the three results that change this design are: + +1. **Detection tracks the source's airtime against sample spacing, not dwell + length.** With a fixed 8 samples, spacing is `dwell / 7` — 14/71/286 ms for + the three arms. Sources with airtime far above the spacing were caught in + every arm; as airtime approached and fell below it, worst-case source-on + degraded monotonically and one arm stopped separating entirely. This is §3's + "observation time is not coverage" warning made concrete: **a 2000 ms dwell + observes eight instants, not 2000 ms**, and lengthening the dwell at a fixed + sample count makes a pass *worse* at catching bursts. A later slice should + scale the sample budget with the dwell; §4's schema and any operator-facing + dwell/observation display must not imply continuous coverage. +2. **Bin-center offset was not the dominant term.** The +125 kHz `high` + position still separated by ~50 dB at the two shorter dwells while a + perfectly centered control separated by less. The §6.1 offset concern is + not what distinguishes these arms at this source strength. +3. **Radio-away is dwell plus ~73 ms**, worst case 2,139 ms over 900 trials. + +And the two the matrix could not settle: + +- **No single fixed RSSI condition separates every arm.** Pooled, the sets + overlap (source-on min -99.0 dBm vs source-off max -75.0 dBm). `p90 >= -90 + dBm` is the nearest candidate — no missed source-on and 1 ambient false hit + in 420 across the 14 separating arms — but it is a candidate only, measured + against a source ~70 dB above ambient, and its meaning is tied to the sample + spacing of the arm it ran at. +- **Coverage thresholds remain unselected.** They are about pass counts and + accumulated time across repeated requests, which a single-pass matrix cannot + supply. `coverage` stays blank and the labels stay undisplayable. + +## 7. Portland metro field validation + +After controlled acceptance, run a small stationary field validation in the +Portland metro, Oregon area. It checks usability and evidence durability, not +local spectrum identity or coverage of the city. + +- Select only conditions already visible in a completed Sweep/Waterfall or a + fixed preset; record their source in `focus.csv`. +- Use a small number of operator-chosen stops; retain exact coordinates and raw + CSVs privately. Commit only a location-redacted summary with build revision, + rough conditions, requested/completed passes, radio-away time, health + counters, and caveats. +- Confirm on-device selection/cancel/recovery feedback, SD removal/error + behavior, and WiFi-off/on resource behavior under real GPS/SD workload. +- Treat a no-activity result as a bounded observation at that stop. Do not use + it to map a quiet area or publish a protocol or operator identity claim. + +## 8. Gate checklist and implementation order + +### Design entry — active + +- [x] V2 boundaries, one-bin first slice, result vocabulary, fixed-statistics + direction, controlled matrix, and Portland field-validation handling are + recorded here. +- [~] Choose the qualifying RSSI condition and coverage thresholds from the + matrix rather than from a desk estimate. §6.4 supplies a *candidate* RSSI + condition (`p90 >= -90 dBm`) and, more importantly, shows why one fixed + number is not sufficient on its own: the condition's meaning depends on + sample spacing, and no single threshold separates all 900 trials. The + coverage thresholds are untouched by this matrix and stay open. +- [ ] Approve the maximum radio-away budget after the Watch-opportunity result. +- [x] Count actual static SRAM, stack frame, queue, row-length, and SD-rate + budgets (§4.2). Counted after the vertical slice rather than before it, on + the built image instead of on paper; the numbers land inside the bounds this + section set, so nothing had to be resized. + +### Engineering gate + +- [x] Pure one-bin request, histogram/percentile, CSV, and restore-before- + publish state tests pass. The fixed RSSI histogram is 141 one-byte buckets + (-140 through 0 dBm); + its 148-byte working state plus the 40-byte result record total 188 bytes, + below the 256-byte one-result target before queue/control overhead. +- [x] Focus plan/statistics/runtime and framed-control native tests pass, and + both production and bench firmware build. Focus is now linked into the + bounded bench image; the production image retains the same source but + rejects every bench trigger. +- [x] Fixed request/result/queue/CSV budgets are measured and accepted: see + §4.2. Every struct is inside its `static_assert` bound, one result's working + state is 188 B against the 256 B target, and the worst-case row is 189 B + against a 256 B buffer that drops rather than truncates. +- [x] A bounded bench harness extends the existing framed Serial Control + pattern; no ad-hoc USB text parsing is treated as evidence. +- [x] A bench-only prototype now has a Core-1-owned one-bin request, Core-0 + `focus.csv` queue/writer, and restore-before-publish terminal row. It is + entered only by the bench-image `BENCH_FOCUS bin:dwell_ms:samples` framed + command; production rejects that command. `BENCH_FOCUS_RESULT` returns only + the latest compact, GPS-free fixed-point summary to make the fixture + reproducible; the durable record remains `focus.csv`. +- [x] Two paired smoke checks on the bench image confirmed quiet/source-on + requests at US Sweep bin 43 (912.750 MHz), each 500 ms/8 samples. The + controlled Heltec `LONG_MODERATE` pulse (912.8125 MHz, capped -9 dBm) + completed inside each source-on window. Quiet P90/peak was -101.0/-101.0 + dBm; source-on P90/peak was -82.0/-82.0 then -87.0/-87.0 dBm, a 14--19 dB + rise. All rows reported successful home restore and durable `focus.csv` + writes. These are transport/RSSI smoke checks, **not** a threshold, + calibration, coverage, or activity-detection claim. The reusable + `scripts/phase12_focus_bench.py` requires `--with-pulse --allow-transmit` + before it may arm the transmitter. +- [x] The non-transmitting `scripts/phase12_focus_behavior_bench.py` hardware + fixture confirmed two cancelled Focus requests, an injected post-retune + failure, and two-way refusal with Probe and Sweep. Every Focus terminal row + was durably written only after home restore; the failed row retained the + injected operation error after recovery. +- [x] The request is bounded in wall-clock time, not only in samples (§5), and + the timeout path is reachable on demand: `BENCH_FOCUS`'s optional fourth + field arms a one-shot bench sample-loop stall, since production only times + out under real bus contention a fixture cannot arrange. Cell and Scope are + menu-only actions in production, so the bench-only `BENCH_ACTION` opcode + starts, cancels, and reports them through the same request functions the + menu calls—without it, Focus's mutual exclusion against them has no + reproducible fixture. Both are bench-image gated; production rejects them. +- [x] Sizing `STATUS` by hand had left its 240-byte argument buffer past the + frame's own ~230-byte budget once Phase 12 added six fields, and an + over-long frame is dropped silently rather than truncated—losing exactly the + newest fields the fixtures read. The argument budget is now derived from the + frame size, host-tested at saturation, and the frame limit raised to 384. + +### Device, claim, and release gates + +- [x] Hardware proves timeout plus mutual exclusion against Cell and Scope. + `scripts/phase12_focus_behavior_bench.py` drove all four on the bench image + (`V=1.0.7;R=3e31daa-dirty;BENCH=1`, 2026-09-04): a 1,500 ms injected stall + against a 100 ms/1,100 ms-deadline request terminated as `timeout` + (`RS=2`) with successful home restore and 1,573 ms total radio-away time, + and Focus and Cell, then Focus and Scope, each refused the other while it + owned RX. Six terminal requests across the run wrote six durable rows with + zero queue or row drops (`FW` 0->6, `FD`/`FL` 0), which is the logger's + post-write counter, not an enqueue count. Timing evidence is the receiver's + own; it is not a coverage, calibration, or activity claim. +- [~] The controlled matrix is complete with durable SD evidence and a + location-redacted summary + ([2026-09-04-phase12-focus-matrix.md](../hardware-results/2026-09-04-phase12-focus-matrix.md)). + The Watch-opportunity comparison has tooling + (`scripts/phase12_watch_opportunity.py`) but has not run, so the maximum + radio-away budget stays unapproved. +- [ ] WiFi-off/on resource matrix, Portland field validation, `STATUS.md`, + `LOG_GUIDE.md`, release notes, and any companion-schema update reconcile. + +Implementation proceeds in two bounded parts. The Engineering foundation now +has pure plan/statistics/CSV tests. The bench request is one pass, 2--2,000 ms +and 2--64 RSSI samples; the matrix gets 30 trials from 30 durable requests, +not one long radio-away loop. Its next slice adds the radio request/recovery +path, logger/health integration, and the bench harness with no coverage label. +Only after §6 selects the thresholds and Watch budget may the operator menu, +Activity surface, durable final schema, and field validation enter the +Device/claim gates. + +## Sources + +- Project evidence: `docs/STATUS.md` (Phase 9 dwell, capture-window, and + RTL-SDR findings); `src/radio_task.cpp` / `.h` (bounded ownership and Sweep + away-time precedent); `scripts/phase9_bin_accuracy_bench.py` and + `scripts/bench_harness.py` (existing controlled-fixture pattern). +- [Semtech SX1262 resources](https://www.semtech.com/products/wireless-rf/lora-connect/sx1262) + — datasheet and SX126x CAD application note catalogue; use them for hardware + API/reference behavior, not as a substitute for board measurements. +- [NIST confidence intervals for proportions](https://www.itl.nist.gov/div898/handbook/prc/section2/prc241.htm) + — Wilson and exact interval guidance for small-count activity-opportunity + measurements. diff --git a/scripts/phase12_focus_behavior_bench.py b/scripts/phase12_focus_behavior_bench.py new file mode 100644 index 0000000..f62b583 --- /dev/null +++ b/scripts/phase12_focus_behavior_bench.py @@ -0,0 +1,222 @@ +#!/usr/bin/env python3 +"""Non-transmitting Phase 12 Focus cancel/failure/timeout/arbitration checks. + +Uses only framed Serial Control and the bench image. It records Focus's +durable row after cancellation, a one-shot injected failure, and an injected +sample-loop stall past the request deadline, then verifies mutual refusal with +Probe, Sweep, Cell, and Scope in both directions. Nothing here transmits, and +none of it sets a coverage or activity threshold. +""" + +import argparse +import pathlib +import sys +import time + +import serial + +from bench_harness import (CARD_MARKER, Endpoint, ResultWriter, card_status, + parse_fields, require_ack, wait_for) + + +FOCUS_ARGUMENT = "43:2000:64" # US 912.750 MHz bin; long enough to arbitrate. +# 100ms dwell -> a 1,100ms deadline (focus_plan.h), so a 1,500ms bench stall +# overruns it deterministically instead of waiting on real bus contention. +FOCUS_TIMEOUT_ARGUMENT = "43:100:4:1500" +FOCUS_TIMEOUT_STALL_MS = 1500 +# Stall, deadline, and one restore: past this the request was not bounded. +FOCUS_TIMEOUT_AWAY_LIMIT_MS = 4000 + + +def expect_error(card, command, argument, payload): + opcode, actual = card.request(command, argument, timeout=5.0) + if opcode != "ERROR" or actual != payload: + raise RuntimeError(f"{command} expected ERROR {payload}, got {opcode} {actual}") + + +def start_focus(card): + card.send("BENCH_FOCUS", FOCUS_ARGUMENT) + # The framed request is already on USB; allow Core 1 to claim its mailbox + # before sending the conflicting command, without relying on debug text. + time.sleep(0.35) + + +def cancel_focus(card, prior_written): + card.send("BENCH_FOCUS_CANCEL", "-") + terminal = wait_for( + card, + lambda status: status.get("FS") == "4" + and int(status.get("FW", "0")) > prior_written, + 10.0, + "cancelled Focus durable row", + ) + result = parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "HEALTH")) + if result.get("RS") != "1" or result.get("HR") != "1": + raise RuntimeError(f"cancelled Focus did not restore: {result}") + return terminal, result + + +def action_state(card): + return parse_fields(require_ack(card, "BENCH_ACTION", "STATE")) + + +def wait_for_action_idle(card, key, timeout, description): + deadline = time.monotonic() + timeout + last = None + while time.monotonic() < deadline: + last = action_state(card) + if last.get(key) == "0": + return last + time.sleep(0.1) + raise TimeoutError(f"timed out waiting for {description}; last state={last}") + + +def check_refuses_focus(card, action, key, results, event): + """Prove the named bounded action refuses a Focus request while it owns RX.""" + require_ack(card, "BENCH_ACTION", f"{action}:START") + time.sleep(0.35) + busy = action_state(card) + if busy.get(key) != "1": + raise RuntimeError(f"{action} did not take the radio: {busy}") + expect_error(card, "BENCH_FOCUS", "43:500:8", "UNAVAILABLE") + require_ack(card, "BENCH_ACTION", f"{action}:CANCEL") + terminal = wait_for_action_idle(card, key, 20.0, f"{action} to release the radio") + record(results, event, busy=busy, terminal=terminal) + + +def check_focus_refuses(card, action, key, results, event): + """Prove Focus refuses the named bounded action while Focus owns RX.""" + before = card_status(card) + start_focus(card) + expect_error(card, "BENCH_ACTION", f"{action}:START", "UNAVAILABLE") + during = action_state(card) + if during.get(key) != "0": + raise RuntimeError(f"{action} started while Focus owned RX: {during}") + terminal, result = cancel_focus(card, int(before["FW"])) + record(results, event, during=during, status=terminal, result=result) + + +def record(writer, event, **fields): + if writer is not None: + writer.write({"event": event, **fields}) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--cardputer-port", required=True) + parser.add_argument("--log", required=True, help="append-only raw framed-control log") + parser.add_argument("--results", help="append-only JSONL summary path") + args = parser.parse_args() + + log_path = pathlib.Path(args.log) + log_path.parent.mkdir(parents=True, exist_ok=True) + results = ResultWriter(args.results) if args.results else None + with log_path.open("a", encoding="utf-8") as log: + card = Endpoint("cardputer", args.cardputer_port, CARD_MARKER, log) + try: + identity = require_ack(card, "HELLO", "-", timeout=15.0) + initial = card_status(card) + if initial.get("SD") != "1": + raise RuntimeError(f"Focus requires an SD-backed result row: {initial}") + record(results, "boot", identity=identity, status=initial) + + start_focus(card) + card.send("PROBE_START", "-") + time.sleep(0.05) + cancel_status, cancel_result = cancel_focus(card, int(initial["FW"])) + # Native USB can fragment a one-way ERROR frame. The framed + # terminal STATUS is stronger evidence: a request accepted while + # Focus owned RX would have entered/raced Probe after cancel. + if cancel_status.get("B") != "IDLE": + raise RuntimeError(f"Probe started while Focus owned RX: {cancel_status}") + record(results, "focus_refuses_probe", status=cancel_status, result=cancel_result) + + before_sweep = card_status(card) + start_focus(card) + card.send("SWEEP_START", "-") + time.sleep(0.05) + cancel_status, cancel_result = cancel_focus(card, int(before_sweep["FW"])) + if cancel_status.get("W") != "IDLE": + raise RuntimeError(f"Sweep started while Focus owned RX: {cancel_status}") + record(results, "focus_refuses_sweep", status=cancel_status, result=cancel_result) + + require_ack(card, "BENCH_FAULT", "AFTER_RETUNE:FAIL") + before_failure = card_status(card) + card.send("BENCH_FOCUS", "43:500:8") + failed = wait_for( + card, + lambda status: status.get("FS") == "6" + and int(status.get("FW", "0")) > int(before_failure["FW"]), + 10.0, + "failed Focus durable row", + ) + failure_result = parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "HEALTH")) + if failure_result.get("RS") != "3" or failure_result.get("HR") != "1" or failure_result.get("E") == "0": + raise RuntimeError(f"failure did not retain operation error plus restore: {failure_result}") + record(results, "focus_failure_restores", status=failed, result=failure_result) + require_ack(card, "BENCH_FAULT", "CLEAR") + + card.send("PROBE_START", "-") + time.sleep(0.35) + expect_error(card, "BENCH_FOCUS", "43:500:8", "UNAVAILABLE") + card.send("PROBE_CANCEL", "-") + probe_terminal = wait_for( + card, lambda status: status.get("B") in {"CANCELLED", "COMPLETE", "FAILED"}, + 12.0, "Probe terminal", + ) + record(results, "probe_refuses_focus", status=probe_terminal) + + card.send("SWEEP_START", "-") + time.sleep(0.35) + expect_error(card, "BENCH_FOCUS", "43:500:8", "UNAVAILABLE") + card.send("SWEEP_CANCEL", "-") + sweep_terminal = wait_for( + card, lambda status: status.get("W") in {"CANCELLED", "COMPLETE", "FAILED"}, + 12.0, "Sweep terminal", + ) + record(results, "sweep_refuses_focus", status=sweep_terminal) + + # Timeout: a stalled sample loop must stop at its own deadline, + # restore Watch, and publish `timeout` -- not `complete` borrowed + # from a restore that worked, and not an unbounded radio-away hold. + before_timeout = card_status(card) + card.send("BENCH_FOCUS", FOCUS_TIMEOUT_ARGUMENT) + timed_out = wait_for( + card, + lambda status: status.get("FS") == "5" + and int(status.get("FW", "0")) > int(before_timeout["FW"]), + 15.0, + "timed-out Focus durable row", + ) + timeout_result = parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "HEALTH")) + if timeout_result.get("RS") != "2" or timeout_result.get("HR") != "1": + raise RuntimeError(f"timeout did not restore home: {timeout_result}") + away_ms = int(timed_out.get("FA", "0")) + if not FOCUS_TIMEOUT_STALL_MS <= away_ms <= FOCUS_TIMEOUT_AWAY_LIMIT_MS: + raise RuntimeError(f"timeout radio-away time was not bounded: {away_ms}ms") + record(results, "focus_timeout_restores", status=timed_out, + result=timeout_result, away_ms=away_ms) + + check_focus_refuses(card, "CELL", "CELL", results, "focus_refuses_cell") + check_refuses_focus(card, "CELL", "CELL", results, "cell_refuses_focus") + check_focus_refuses(card, "SCOPE", "SCOPE", results, "focus_refuses_scope") + check_refuses_focus(card, "SCOPE", "SCOPE", results, "scope_refuses_focus") + + print("Focus cancel/failure/timeout and Probe/Sweep/Cell/Scope " + "mutual-exclusion checks passed.") + finally: + # A failure cannot leave the one-shot injector armed for later work. + try: + require_ack(card, "BENCH_FAULT", "CLEAR", timeout=5.0) + finally: + card.close() + if results is not None: + results.close() + + +if __name__ == "__main__": + try: + main() + except (OSError, serial.SerialException, RuntimeError, TimeoutError, ValueError) as error: + print(f"phase12 Focus behavior bench: {error}", file=sys.stderr) + sys.exit(2) diff --git a/scripts/phase12_focus_bench.py b/scripts/phase12_focus_bench.py new file mode 100644 index 0000000..9cd570a --- /dev/null +++ b/scripts/phase12_focus_bench.py @@ -0,0 +1,136 @@ +#!/usr/bin/env python3 +"""Measure one bounded Focus bin quietly, then against one controlled pulse. + +This is Phase 12 engineering evidence, not a coverage-calibration tool. It +uses the shared framed transport, leaves the Heltec quiet on every exit path, +and requires explicit --allow-transmit before it can arm a pulse. Results +contain only the GPS-free compact Focus summary; `focus.csv` is the durable +device-side evidence. +""" + +import argparse +import pathlib +import sys + +import serial + +from bench_harness import (CARD_MARKER, TX_MARKER, Endpoint, ResultWriter, + card_status, parse_fields, require_ack, wait_for) + + +TX_CANDIDATE = "LONG_MODERATE" # 912.8125 MHz, SF11/BW125/CR4/8, sync 0x2B + + +def request_focus(card, bin_index, dwell_ms, samples, prior_status): + """Request one Focus pass and require its SD commit before readback.""" + before_written = int(prior_status.get("FW", "0")) + before_enqueued = int(prior_status.get("FO", "0")) + require_ack(card, "BENCH_FOCUS", f"{bin_index}:{dwell_ms}:{samples}") + terminal = wait_for( + card, + lambda status: status.get("FS") == "3" + and int(status.get("FO", "0")) > before_enqueued + and int(status.get("FW", "0")) > before_written, + 12.0, + "Focus completion, restore, and focus.csv commit", + ) + result = parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "-")) + if result.get("RS") != "0" or result.get("HR") != "1": + raise RuntimeError(f"Focus did not complete with home restore: {result}") + if int(result.get("N", "0")) != samples: + raise RuntimeError(f"Focus sample count mismatch: expected {samples}, got {result}") + return terminal, result + + +def record(writer, event, status, result, **extra): + if writer is not None: + writer.write({"event": event, "status": status, "result": result, **extra}) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--cardputer-port", required=True) + parser.add_argument("--heltec-port", required=True) + parser.add_argument("--log", required=True, help="append-only raw framed-control log") + parser.add_argument("--results", help="append-only JSONL summary path") + parser.add_argument("--bin", type=int, default=43, + help="US 250 kHz bin (default 43 = 912.750 MHz)") + parser.add_argument("--dwell-ms", type=int, default=500) + parser.add_argument("--samples", type=int, default=8) + parser.add_argument("--with-pulse", action="store_true", help="also measure one Heltec pulse") + parser.add_argument("--allow-transmit", action="store_true", help="required with --with-pulse") + parser.add_argument("--arm-delay-ms", type=int, default=150) + args = parser.parse_args() + if args.with_pulse and not args.allow_transmit: + parser.error("--with-pulse requires explicit --allow-transmit") + if not 0 <= args.bin <= 65535: + parser.error("--bin must fit uint16") + if not 2 <= args.dwell_ms <= 2000: + parser.error("--dwell-ms must be 2..2000") + if not 2 <= args.samples <= 64: + parser.error("--samples must be 2..64") + if not 10 <= args.arm_delay_ms <= 5000: + parser.error("--arm-delay-ms must be 10..5000") + + log_path = pathlib.Path(args.log) + log_path.parent.mkdir(parents=True, exist_ok=True) + results = ResultWriter(args.results) if args.results else None + with log_path.open("a", encoding="utf-8") as log: + card = Endpoint("cardputer", args.cardputer_port, CARD_MARKER, log) + transmitter = None + try: + require_ack(card, "HELLO", "-", timeout=15.0) + transmitter = Endpoint("heltec", args.heltec_port, TX_MARKER, log) + require_ack(transmitter, "HELLO", "-") + require_ack(transmitter, "QUIET", "-") + + quiet_status, quiet = request_focus( + card, args.bin, args.dwell_ms, args.samples, card_status(card) + ) + print(f"quiet: {quiet}") + record(results, "quiet", quiet_status, quiet) + if not args.with_pulse: + return + + require_ack(transmitter, "CONFIG", TX_CANDIDATE) + on_before = card_status(card) + require_ack(card, "BENCH_FOCUS", f"{args.bin}:{args.dwell_ms}:{args.samples}") + require_ack(transmitter, "ARM", str(args.arm_delay_ms)) + on_status = wait_for( + card, + lambda status: status.get("FS") == "3" + and int(status.get("FO", "0")) > int(on_before.get("FO", "0")) + and int(status.get("FW", "0")) > int(on_before.get("FW", "0")), + 12.0, + "pulsed Focus completion, restore, and focus.csv commit", + ) + on = parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "-")) + if on.get("RS") != "0" or on.get("HR") != "1": + raise RuntimeError(f"pulsed Focus did not restore: {on}") + if int(on.get("N", "0")) != args.samples: + raise RuntimeError(f"pulsed Focus sample count mismatch: {on}") + transmitter.request("STATUS", "-") # Drains TX_STARTED/TX_DONE evidence. + events = {opcode for _, opcode, _ in transmitter.observed} + if not {"TX_STARTED", "TX_DONE"}.issubset(events): + raise RuntimeError(f"transmitter pulse incomplete: {sorted(events)}") + delta = {field: int(on[field]) - int(quiet[field]) for field in ("MED", "P90", "MAX")} + print(f"pulse: {on}; delta_tenths_db={delta}") + record(results, "pulse", on_status, on, delta_tenths_db=delta, + tx_candidate=TX_CANDIDATE) + finally: + if transmitter is not None: + try: + require_ack(transmitter, "QUIET", "-", timeout=5.0) + finally: + transmitter.close() + card.close() + if results is not None: + results.close() + + +if __name__ == "__main__": + try: + main() + except (OSError, serial.SerialException, RuntimeError, TimeoutError, ValueError) as error: + print(f"phase12 Focus bench: {error}", file=sys.stderr) + sys.exit(2) diff --git a/scripts/phase12_focus_matrix.py b/scripts/phase12_focus_matrix.py new file mode 100644 index 0000000..4417e53 --- /dev/null +++ b/scripts/phase12_focus_matrix.py @@ -0,0 +1,389 @@ +#!/usr/bin/env python3 +"""Phase 12 §6.2 controlled dwell matrix: paired source-on/source-off Focus trials. + +Collects the evidence the coverage vocabulary needs and nothing more. Each +trial is one independently logged, bounded Focus request — the matrix gets its +30 trials from 30 durable requests, never from one long radio-away loop +(docs/research/phase12-survey-truth-design.md §6.2). It records raw RSSI +summaries per trial and selects no threshold; that is the report script's job, +run offline against this output. + +The transmitter is repo-owned (bench/heltec-v4r8-transmitter) and is left quiet +on every exit path. Arming it requires explicit --allow-transmit. + +Nothing here decides a coverage label, and an absent observation is recorded as +a measured no-observation at a stated frequency offset — never as "quiet". +""" + +import argparse +import pathlib +import random +import sys +import threading +import time + +import serial + +from bench_harness import (CARD_MARKER, TX_MARKER, Endpoint, ResultWriter, + card_status, parse_fields, require_ack, wait_for) + + +# US Sweep band (energy_plan.h): bin centers are lo + index * step. +US_BAND_LO_MHZ = 902.0 +BIN_STEP_MHZ = 0.25 + +# Sourced from bench/heltec-v4r8-transmitter/src/main.cpp's CANDIDATES table; +# no frequency is invented here. `bin` is the Sweep bin Focus is asked for, and +# `offset_khz` is how far the transmitter actually sits from that bin's center. +# +# That offset is a first-class result field, not a footnote: Focus tunes at the +# resolved home channel's bandwidth (125 kHz on this bench), so a position +# 125 kHz off center is outside the passband half-width and a weak or absent +# rise there measures the offset, not the receiver's sensitivity. The three +# §6.1 positions are kept because they are the sourced low/mid/high set; the +# two `-aligned` positions exist so an arm can be run without that confound. +POSITIONS = { + # §6.1's sourced low/mid/high set. + "low": {"candidate": "LONG_SLOW", "tx_mhz": 905.3125, "bin": 13}, + "mid": {"candidate": "LONG_MODERATE", "tx_mhz": 912.8125, "bin": 43}, + "high": {"candidate": "SHORT_SLOW", "tx_mhz": 920.625, "bin": 74}, + # Exactly bin-centered alternatives from the same candidate table. + "low-aligned": {"candidate": "LONG_TURBO", "tx_mhz": 908.750, "bin": 27}, + "mid-aligned": {"candidate": "MESH_OREGON", "tx_mhz": 918.500, "bin": 66}, +} + +DEFAULT_DWELLS_MS = (100, 500, 2000) +DEFAULT_POSITIONS = ("low", "mid", "high") + + +def bin_center_mhz(bin_index): + return US_BAND_LO_MHZ + bin_index * BIN_STEP_MHZ + + +def position_offset_khz(position): + spec = POSITIONS[position] + return round((spec["tx_mhz"] - bin_center_mhz(spec["bin"])) * 1000.0, 1) + + +class PulseBurst: + """Keeps the transmitter firing for as long as a trial's window is open. + + A single armed pulse cannot be scheduled reliably against a short dwell. + Candidate airtimes across this table span roughly 30 ms (SF8/BW250) to + ~1 s (SF12/BW125), and the receiver's window opens tens of ms after the + request is accepted — so one pulse at a fixed delay lands inside a 2,000 ms + dwell and misses a 100 ms one entirely. That is exactly what the first + attempt at this matrix measured: at 100 ms, source-on and source-off were + indistinguishable because the pulse started after the window had closed. + + Firing continuously for the trial's duration makes "source on" mean the + transmitter was actually radiating while Focus was listening, which is the + condition §3.1's qualifying-RSSI question needs. It deliberately does NOT + measure the catch probability of intermittent traffic — §1 already records + that the dwell-versus-real-traffic comparison was inconclusive, and this + matrix is not a second attempt at it. + """ + + def __init__(self, transmitter, gap_s): + self.transmitter = transmitter + self.gap_s = gap_s + self._stop = threading.Event() + self._thread = None + self.fired = 0 + self.failed = 0 + + def _await_tx_done(self, timeout_s=4.0): + """Block until the in-flight transmission reports TX_DONE. + + Paced by the transmitter's own completion rather than a fixed cadence. + A fixed interval shorter than the airtime queues overlapping sends — + measured airtime here ranges from ~10 ms (SF8/BW250) to ~275 ms + (SF12/BW125) — and the tail then bleeds into the next trial. The first + burst attempt did exactly that: source-off trials inherited a + transmission and read -26 dBm, identical to source-on. + """ + deadline = time.monotonic() + timeout_s + while time.monotonic() < deadline: + if any(opcode == "TX_DONE" for _, opcode, _ in self.transmitter.observed): + return True + # STATUS is idempotent; the round trip is what drains the + # asynchronous TX_STARTED/TX_DONE frames into `observed`. + try: + self.transmitter.request("STATUS", "-", timeout=1.0) + except (RuntimeError, TimeoutError, OSError): + pass + return False + + def _run(self): + while not self._stop.is_set(): + try: + self.transmitter.observed.clear() + require_ack(self.transmitter, "ARM", "0", timeout=1.5) + self.fired += 1 + self._await_tx_done() + except (RuntimeError, TimeoutError, OSError): + self.failed += 1 + self._stop.wait(self.gap_s) + + def __enter__(self): + self._thread = threading.Thread(target=self._run, daemon=True) + self._thread.start() + return self + + def __exit__(self, *exc): + self._stop.set() + if self._thread is not None: + self._thread.join(timeout=8.0) + # The trial is over, but a transmission may still be in the air. A + # source-off trial that inherits it is not a source-off trial. + self._await_tx_done() + return False + + +def counters(status): + """The health counters a trial must not disturb.""" + return {field: int(status.get(field, "0")) for field in ("FO", "FW", "FD", "FL")} + + +def run_focus_request(card, spec, dwell_ms, samples, start): + # The long timeout is transport, not RF: the device's native USB CDC + # occasionally truncates an outbound frame under sustained round trips, and + # require_ack re-sends once a second within its own timeout. The command is + # idempotent on the device (one cached response per sequence), so a re-send + # cannot start a second survey. + require_ack(card, "BENCH_FOCUS", f"{spec['bin']}:{dwell_ms}:{samples}", timeout=12.0) + terminal = wait_for( + card, + lambda status: status.get("FS") == "3" + and int(status.get("FO", "0")) > start["FO"] + and int(status.get("FW", "0")) > start["FW"], + max(12.0, dwell_ms / 1000.0 + 10.0), + "Focus completion, home restore, and focus.csv commit", + ) + return terminal, parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "-", timeout=12.0)) + + +def run_trial(card, transmitter, position, dwell_ms, samples, source_on, gap_ms): + """One bounded Focus request, with the transmitter radiating throughout it or not at all.""" + spec = POSITIONS[position] + before = card_status(card) + start = counters(before) + + pulses = 0 + if source_on: + with PulseBurst(transmitter, gap_ms / 1000.0) as burst: + terminal, result = run_focus_request(card, spec, dwell_ms, samples, start) + pulses = burst.fired + if burst.fired == 0: + raise RuntimeError("the transmitter never fired during a source-on trial") + else: + terminal, result = run_focus_request(card, spec, dwell_ms, samples, start) + + # §6.2: an arm is rejected if it loses ownership, cannot restore home, or + # has unexplained queue/row drops. Fail the trial, not the whole matrix's + # credibility after the fact. + if result.get("RS") != "0" or result.get("HR") != "1": + raise RuntimeError(f"trial did not complete with home restore: {result}") + if int(result.get("N", "0")) != samples: + raise RuntimeError(f"trial sample count mismatch: expected {samples}, got {result}") + end = counters(terminal) + if end["FD"] != start["FD"] or end["FL"] != start["FL"]: + raise RuntimeError(f"trial dropped a queued or durable row: {start} -> {end}") + + if source_on: + transmitter.observed.clear() + + return { + "position": position, + "candidate": spec["candidate"], + "tx_mhz": spec["tx_mhz"], + "bin": spec["bin"], + "bin_center_mhz": round(bin_center_mhz(spec["bin"]), 4), + "offset_khz": position_offset_khz(position), + "dwell_ms": dwell_ms, + "samples": samples, + "source_on": source_on, + # Tenths of a dBm, exactly as the device reported them. + "median_dbm_x10": int(result["MED"]), + "p90_dbm_x10": int(result["P90"]), + "peak_dbm_x10": int(result["MAX"]), + "sample_count": int(result["N"]), + "observation_ms": int(result["OBS"]), + "focus_id": int(result["ID"]), + "radio_away_ms": int(terminal.get("FA", "0")), + "request_status": result["RS"], + "home_restore": result["HR"], + "radio_status": int(result["E"]), + "pulses_fired": pulses, + } + + +def settle(card, timeout_s=20.0): + """Wait until no Focus owns the radio, so a retried trial isn't refused. + + A trial can fail with its request already accepted (the ACK truncated in + transit). Starting the next one immediately would be refused as + UNAVAILABLE and look like an arbitration bug rather than a transport one. + """ + deadline = time.monotonic() + timeout_s + while time.monotonic() < deadline: + try: + state = card_status(card).get("FS") + except (RuntimeError, TimeoutError): + time.sleep(0.5) + continue + # 1 = SURVEYING, 2 = RESTORING (focus_runtime.h). + if state not in {"1", "2"}: + time.sleep(0.3) + return True + time.sleep(0.3) + return False + + +def run_trial_with_retry(card, transmitter, position, dwell_ms, samples, source_on, + gap_ms, results, attempts): + """A transport failure is a harness event, not an RF result — retry it, loudly. + + The retry is a genuinely new bounded request, and the failure is recorded + rather than swallowed: a run whose trials silently retried would overstate + how cleanly the transport behaved. + """ + for attempt in range(1, attempts + 1): + try: + row = run_trial(card, transmitter, position, dwell_ms, samples, + source_on, gap_ms) + if attempt > 1: + row["retried_attempts"] = attempt + return row + except (RuntimeError, TimeoutError) as error: + results.write({"event": "trial_error", "position": position, + "dwell_ms": dwell_ms, "source_on": source_on, + "attempt": attempt, "error": str(error)}) + print(f" trial attempt {attempt}/{attempts} failed: {error}", flush=True) + if attempt == attempts: + raise + if not settle(card): + raise RuntimeError("Focus never returned to idle after a failed trial") + if transmitter is not None: + try: + require_ack(transmitter, "QUIET", "-", timeout=5.0) + transmitter.observed.clear() + except (RuntimeError, TimeoutError): + pass + + +def trial_order(trials, source_order, rng): + """Alternating by default; --order random shuffles within the arm (§6.2).""" + plan = [True, False] * trials + if source_order == "random": + rng.shuffle(plan) + return plan + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--cardputer-port", required=True) + parser.add_argument("--heltec-port", required=True) + parser.add_argument("--log", required=True, help="append-only raw framed-control log") + parser.add_argument("--results", required=True, help="append-only JSONL trial rows") + parser.add_argument("--position", action="append", choices=sorted(POSITIONS), + help=f"repeatable; default {' '.join(DEFAULT_POSITIONS)}") + parser.add_argument("--dwell-ms", action="append", type=int, + help=f"repeatable; default {' '.join(str(d) for d in DEFAULT_DWELLS_MS)}") + parser.add_argument("--trials", type=int, default=30, + help="trials per source state per arm (§6.2 requires at least 30)") + parser.add_argument("--samples", type=int, default=8) + parser.add_argument("--order", choices=("alternating", "random"), default="alternating") + parser.add_argument("--seed", type=int, default=20260904, help="--order random seed") + parser.add_argument("--pulse-gap-ms", type=int, default=30, + help="idle gap between completed transmissions during a source-on " + "trial; the burst is paced by TX_DONE, not by this alone") + parser.add_argument("--attempts", type=int, default=3, + help="attempts per trial before the run gives up (transport retries)") + parser.add_argument("--allow-transmit", action="store_true", + help="required: this matrix arms the controlled transmitter") + args = parser.parse_args() + + if not args.allow_transmit: + parser.error("this matrix arms the transmitter; pass --allow-transmit") + positions = args.position or list(DEFAULT_POSITIONS) + dwells = args.dwell_ms or list(DEFAULT_DWELLS_MS) + for dwell in dwells: + if not 2 <= dwell <= 2000: + parser.error(f"--dwell-ms {dwell} outside the bounded 2..2000 request range") + if not 2 <= args.samples <= 64: + parser.error("--samples must be 2..64") + if args.trials < 1: + parser.error("--trials must be positive") + if args.trials < 30: + print(f"warning: {args.trials} trials per state is below §6.2's 30; " + "the result is a smoke check, not matrix evidence", file=sys.stderr) + + rng = random.Random(args.seed) + log_path = pathlib.Path(args.log) + log_path.parent.mkdir(parents=True, exist_ok=True) + results = ResultWriter(args.results) + completed = 0 + started = time.monotonic() + + with log_path.open("a", encoding="utf-8") as log: + card = Endpoint("cardputer", args.cardputer_port, CARD_MARKER, log) + transmitter = None + try: + identity = require_ack(card, "HELLO", "-", timeout=15.0) + boot = card_status(card) + if boot.get("SD") != "1": + raise RuntimeError(f"the matrix needs durable focus.csv rows: {boot}") + transmitter = Endpoint("heltec", args.heltec_port, TX_MARKER, log) + require_ack(transmitter, "HELLO", "-") + require_ack(transmitter, "QUIET", "-") + results.write({"event": "boot", "identity": identity, "status": boot, + "positions": positions, "dwells_ms": dwells, + "trials_per_state": args.trials, "samples": args.samples, + "order": args.order, "seed": args.seed}) + + for position in positions: + require_ack(transmitter, "CONFIG", POSITIONS[position]["candidate"]) + for dwell in dwells: + plan = trial_order(args.trials, args.order, rng) + print(f"[{position} {dwell}ms] {len(plan)} trials " + f"(offset {position_offset_khz(position):+.1f} kHz)", flush=True) + for index, source_on in enumerate(plan): + row = run_trial_with_retry(card, transmitter, position, dwell, + args.samples, source_on, + args.pulse_gap_ms, results, + args.attempts) + row.update({"event": "trial", "arm_index": index}) + results.write(row) + completed += 1 + state = "on " if source_on else "off" + print(f" [{index + 1:3d}/{len(plan)}] source={state} " + f"p90={row['p90_dbm_x10'] / 10:.1f}dBm " + f"peak={row['peak_dbm_x10'] / 10:.1f}dBm " + f"away={row['radio_away_ms']}ms", flush=True) + # Let the device's USB CDC buffer drain between trials. + time.sleep(0.15) + elapsed = time.monotonic() - started + print(f"{completed} trials completed in {elapsed / 60:.1f} min. " + "Raw rows only — run phase12_focus_matrix_report.py to analyze.") + finally: + # A transmitter left armed after a failure is the one outcome this + # script must never produce. + if transmitter is not None: + try: + require_ack(transmitter, "QUIET", "-", timeout=5.0) + finally: + transmitter.close() + card.close() + results.close() + + +if __name__ == "__main__": + try: + main() + except KeyboardInterrupt: + print("interrupted; partial rows are already durable", file=sys.stderr) + sys.exit(130) + except (OSError, serial.SerialException, RuntimeError, TimeoutError, ValueError) as error: + print(f"phase12 Focus matrix: {error}", file=sys.stderr) + sys.exit(2) diff --git a/scripts/phase12_focus_matrix_report.py b/scripts/phase12_focus_matrix_report.py new file mode 100644 index 0000000..3c02daf --- /dev/null +++ b/scripts/phase12_focus_matrix_report.py @@ -0,0 +1,163 @@ +#!/usr/bin/env python3 +"""Offline, deterministic report over phase12_focus_matrix.py trial rows. + +Reads the JSONL the matrix runner wrote and answers one question per arm: does +a fixed RSSI condition separate source-on from source-off trials, and with what +confidence? It reports proportions as Wilson 95% intervals rather than bare +percentages (docs/research/phase12-survey-truth-design.md §6.2 item 5, NIST +1.3.5 guidance) because a "30/30" at these counts is not the certainty it looks +like. + +It touches no device and mutates no input. Given the same file it prints the +same report, so a result can be re-derived rather than remembered. + +What it deliberately does NOT do: name a coverage label, call an arm's +source-off trials "quiet", or pick thresholds the operator has not accepted. A +separating threshold printed here is a *candidate* for §3.1, and it is only +meaningful for the frequency offset its arm actually ran at. +""" + +import argparse +import collections +import json +import math +import pathlib +import statistics +import sys + + +def wilson_interval(successes, total, z=1.959963984540054): + """Two-sided Wilson score interval; correct at 0/n and n/n where normal isn't.""" + if total == 0: + return (0.0, 1.0) + p = successes / total + denom = 1.0 + z * z / total + center = (p + z * z / (2 * total)) / denom + margin = z * math.sqrt(p * (1 - p) / total + z * z / (4 * total * total)) / denom + return (max(0.0, center - margin), min(1.0, center + margin)) + + +def load_trials(path): + trials, meta = [], None + with path.open(encoding="utf-8") as handle: + for line_number, line in enumerate(handle, 1): + line = line.strip() + if not line: + continue + try: + row = json.loads(line) + except json.JSONDecodeError as error: + raise SystemExit(f"{path}:{line_number}: {error}") + if row.get("event") == "boot" and meta is None: + meta = row + elif row.get("event") == "trial": + trials.append(row) + return trials, meta + + +def best_threshold(on_values, off_values): + """Lowest 1 dB threshold that admits every source-on trial and no source-off one. + + Returns (threshold_dbm_x10, margin_db) or None when the two sets overlap — + and an overlap is a real answer: it says this arm cannot support a fixed + RSSI condition, which is exactly what §3.1 needs to know before choosing + one. + """ + if not on_values or not off_values: + return None + if min(on_values) <= max(off_values): + return None + # Any cut in the gap separates; take the midpoint, rounded to whole dB. + low, high = max(off_values), min(on_values) + threshold = int(round((low + high) / 2.0 / 10.0)) * 10 + threshold = max(low + 1, min(high, threshold)) + return threshold, (high - low) / 10.0 + + +def summarize(values): + if not values: + return "n/a" + return (f"min {min(values) / 10:7.1f} med {statistics.median(values) / 10:7.1f} " + f"max {max(values) / 10:7.1f}") + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("results", type=pathlib.Path, help="JSONL from phase12_focus_matrix.py") + parser.add_argument("--metric", choices=("p90", "peak", "median"), default="p90", + help="RSSI summary the candidate condition is built on") + args = parser.parse_args() + + if not args.results.exists(): + raise SystemExit(f"no such results file: {args.results}") + trials, meta = load_trials(args.results) + if not trials: + raise SystemExit(f"{args.results} contains no trial rows") + + field = {"p90": "p90_dbm_x10", "peak": "peak_dbm_x10", "median": "median_dbm_x10"}[args.metric] + + if meta: + print(f"build : {meta.get('identity', 'unknown')}") + print(f"plan : positions={meta.get('positions')} dwells_ms={meta.get('dwells_ms')} " + f"trials/state={meta.get('trials_per_state')} order={meta.get('order')}") + print(f"metric : {args.metric} (dBm)") + print(f"trial rows: {len(trials)}\n") + + arms = collections.defaultdict(lambda: {"on": [], "off": [], "away": [], "rows": []}) + for row in trials: + arm = arms[(row["position"], row["dwell_ms"])] + arm["on" if row["source_on"] else "off"].append(row[field]) + arm["away"].append(row["radio_away_ms"]) + arm["rows"].append(row) + + unresolved = [] + for (position, dwell), arm in sorted(arms.items(), key=lambda item: (item[0][0], item[0][1])): + sample = arm["rows"][0] + on, off = arm["on"], arm["off"] + print(f"=== {position} @ {dwell} ms " + f"(tx {sample['tx_mhz']:.4f} MHz -> bin {sample['bin']} " + f"center {sample['bin_center_mhz']:.3f}, offset {sample['offset_khz']:+.1f} kHz)") + print(f" source-on n={len(on):3d} {summarize(on)}") + print(f" source-off n={len(off):3d} {summarize(off)}") + print(f" radio-away min {min(arm['away'])} ms, max {max(arm['away'])} ms") + + selection = best_threshold(on, off) + if selection is None: + overlap = "no trials" if not on or not off else ( + f"source-on min {min(on) / 10:.1f} <= source-off max {max(off) / 10:.1f}") + print(f" candidate condition: NONE — the two sets overlap ({overlap}).") + print(" This arm does not support a fixed RSSI condition; it is not evidence " + "that the frequency was quiet.") + unresolved.append((position, dwell)) + else: + threshold, margin = selection + hits_on = sum(1 for value in on if value >= threshold) + hits_off = sum(1 for value in off if value >= threshold) + lo_on, hi_on = wilson_interval(hits_on, len(on)) + lo_off, hi_off = wilson_interval(hits_off, len(off)) + print(f" candidate condition: {args.metric} >= {threshold / 10:.1f} dBm " + f"(separating margin {margin:.1f} dB)") + print(f" source-on detected {hits_on}/{len(on)} " + f"95% CI [{lo_on:.3f}, {hi_on:.3f}]") + print(f" source-off detected {hits_off}/{len(off)} " + f"95% CI [{lo_off:.3f}, {hi_off:.3f}]") + print() + + print("Interpretation limits:") + print(" - A separating threshold is a candidate for §3.1, not an accepted constant,") + print(" and it is valid only at the frequency offset its arm ran at.") + print(" - Source-off trials are ambient observation on this bench, not a calibrated") + print(" false-hit rate (§6.1): the control path is not known quiet.") + print(" - No arm here licenses a coverage label or a 'no activity' conclusion.") + if unresolved: + print(f" - {len(unresolved)} arm(s) did not separate: " + + ", ".join(f"{p}@{d}ms" for p, d in unresolved)) + return 0 + + +if __name__ == "__main__": + try: + sys.exit(main()) + except (OSError, ValueError, KeyError) as error: + print(f"phase12 Focus matrix report: {error}", file=sys.stderr) + sys.exit(2) diff --git a/scripts/phase12_watch_opportunity.py b/scripts/phase12_watch_opportunity.py new file mode 100644 index 0000000..e5ff260 --- /dev/null +++ b/scripts/phase12_watch_opportunity.py @@ -0,0 +1,256 @@ +#!/usr/bin/env python3 +"""Phase 12 §6.3: what does interleaving Focus cost Watch packet opportunity? + +Two arms against an identical, independently timed pulse train on the home +channel: Watch alone, then Watch with repeated bounded Focus requests. Reports +received/reference for each as a Wilson 95% interval, plus Focus away time, +completed requests, and the gaps between restored Watch windows. + +This measures a cost. It does not decide whether that cost is acceptable — +that is an operator product decision after the measurement, and a displayed +away-time does not make the loss free (docs/research/phase12-survey-truth-design.md +§6.3). + +The transmitter must be configured to a candidate that matches the receiver's +resolved home channel, or Watch cannot hear the reference train at all and +both arms measure nothing. The script checks the frequency and refuses +otherwise. Requires explicit --allow-transmit; the transmitter is quieted on +every exit path. +""" + +import argparse +import math +import pathlib +import statistics +import sys +import threading +import time + +import serial + +from bench_harness import (CARD_MARKER, TX_MARKER, Endpoint, ResultWriter, + card_status, parse_fields, require_ack) + + +# bench/heltec-v4r8-transmitter/src/main.cpp: 918.5 MHz, SF8, BW125, CR4/5, +# sync 0x2B — the one candidate that matches this bench's resolved home +# channel, which is what makes Watch able to receive the reference train. +HOME_CANDIDATE = "MESH_OREGON" +HOME_CANDIDATE_MHZ = 918.5 + + +def wilson_interval(successes, total, z=1.959963984540054): + if total == 0: + return (0.0, 1.0) + p = successes / total + denom = 1.0 + z * z / total + center = (p + z * z / (2 * total)) / denom + margin = z * math.sqrt(p * (1 - p) / total + z * z / (4 * total * total)) / denom + return (max(0.0, center - margin), min(1.0, center + margin)) + + +class PulseTrain: + """Fires the reference train on its own schedule, independent of Focus. + + The transmitter is the timing source (§6.1): if the pulse cadence were + driven by the receiver's request loop, an arm that spent longer away would + also be sent fewer packets, and the comparison would measure the harness. + """ + + def __init__(self, transmitter, interval_s): + self.transmitter = transmitter + self.interval_s = interval_s + self._stop = threading.Event() + self._thread = None + self.sent = 0 + self.failed = 0 + + def _run(self): + next_fire = time.monotonic() + while not self._stop.is_set(): + now = time.monotonic() + if now < next_fire: + self._stop.wait(min(0.05, next_fire - now)) + continue + next_fire += self.interval_s + try: + require_ack(self.transmitter, "ARM", "0", timeout=1.5) + self.sent += 1 + except (RuntimeError, TimeoutError, OSError): + # One dropped ARM must not end the train; it is counted so the + # reference total stays honest. + self.failed += 1 + + def __enter__(self): + self._thread = threading.Thread(target=self._run, daemon=True) + self._thread.start() + return self + + def __exit__(self, *exc): + self._stop.set() + if self._thread is not None: + self._thread.join(timeout=5.0) + return False + + +def run_arm(card, results, label, duration_s, train, focus_request): + """One timed arm. `focus_request` is None for the Watch-only baseline.""" + before = card_status(card) + start_rxp = int(before.get("RXP", "0")) + start_sent = train.sent + started = time.monotonic() + + focus_completed = 0 + focus_refused = 0 + away_ms_total = 0 + watch_gaps_s = [] + last_restore = started + + while time.monotonic() - started < duration_s: + if focus_request is None: + time.sleep(0.25) + continue + bin_index, dwell_ms, samples = focus_request + opcode, payload = card.request("BENCH_FOCUS", f"{bin_index}:{dwell_ms}:{samples}", + timeout=5.0) + if opcode != "ACK": + focus_refused += 1 + continue + deadline = time.monotonic() + dwell_ms / 1000.0 + 10.0 + while time.monotonic() < deadline: + status = card_status(card) + if status.get("FS") == "3": + break + else: + raise RuntimeError("a Focus request did not reach a terminal state in this arm") + result = parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "HEALTH")) + if result.get("HR") != "1": + raise RuntimeError(f"Focus failed to restore home listening: {result}") + focus_completed += 1 + away_ms_total += int(card_status(card).get("FA", "0")) + now = time.monotonic() + watch_gaps_s.append(now - last_restore) + last_restore = now + + elapsed = time.monotonic() - started + after = card_status(card) + received = int(after.get("RXP", "0")) - start_rxp + reference = train.sent - start_sent + low, high = wilson_interval(min(received, reference), reference) + + row = { + "event": "arm", + "arm": label, + "elapsed_s": round(elapsed, 2), + "reference_pulses": reference, + "received_packets": received, + "received_fraction": round(received / reference, 4) if reference else None, + "wilson95": [round(low, 4), round(high, 4)], + "focus_completed": focus_completed, + "focus_refused": focus_refused, + "focus_away_ms_total": away_ms_total, + "away_fraction": round(away_ms_total / (elapsed * 1000.0), 4) if elapsed else None, + "watch_gap_s_median": round(statistics.median(watch_gaps_s), 2) if watch_gaps_s else None, + "watch_gap_s_max": round(max(watch_gaps_s), 2) if watch_gaps_s else None, + "crc_errors": int(after.get("RXC", "0")) - int(before.get("RXC", "0")), + "train_failed_arms": train.failed, + } + results.write(row) + return row + + +def describe(row): + fraction = "n/a" if row["received_fraction"] is None else f"{row['received_fraction']:.3f}" + print(f" {row['arm']:<14s} received {row['received_packets']}/{row['reference_pulses']} " + f"= {fraction} 95% CI [{row['wilson95'][0]:.3f}, {row['wilson95'][1]:.3f}]") + if row["focus_completed"]: + print(f" Focus {row['focus_completed']} completed, " + f"{row['focus_refused']} refused, away {row['focus_away_ms_total']} ms " + f"({row['away_fraction']:.1%} of the arm)") + print(f" restored-Watch gaps: median {row['watch_gap_s_median']}s, " + f"max {row['watch_gap_s_max']}s") + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--cardputer-port", required=True) + parser.add_argument("--heltec-port", required=True) + parser.add_argument("--log", required=True) + parser.add_argument("--results", required=True) + parser.add_argument("--duration-s", type=int, default=180, help="per arm") + parser.add_argument("--pulse-interval-s", type=float, default=2.0) + parser.add_argument("--bin", type=int, default=43, help="Focus bin for the interleaved arm") + parser.add_argument("--dwell-ms", type=int, default=2000) + parser.add_argument("--samples", type=int, default=8) + parser.add_argument("--allow-transmit", action="store_true") + args = parser.parse_args() + + if not args.allow_transmit: + parser.error("this comparison arms the transmitter; pass --allow-transmit") + if args.pulse_interval_s < 0.5: + parser.error("--pulse-interval-s below 0.5 risks overlapping airtime at SF8") + if not 2 <= args.dwell_ms <= 2000: + parser.error("--dwell-ms must be 2..2000") + + log_path = pathlib.Path(args.log) + log_path.parent.mkdir(parents=True, exist_ok=True) + results = ResultWriter(args.results) + + with log_path.open("a", encoding="utf-8") as log: + card = Endpoint("cardputer", args.cardputer_port, CARD_MARKER, log) + transmitter = None + try: + identity = require_ack(card, "HELLO", "-", timeout=15.0) + status = card_status(card) + home_khz = int(status.get("F", "0")) + if abs(home_khz - HOME_CANDIDATE_MHZ * 1000) > 1: + raise RuntimeError( + f"home channel is {home_khz} kHz but the reference train is " + f"{HOME_CANDIDATE} at {HOME_CANDIDATE_MHZ} MHz; Watch could not hear it " + "and both arms would measure nothing") + transmitter = Endpoint("heltec", args.heltec_port, TX_MARKER, log) + require_ack(transmitter, "HELLO", "-") + require_ack(transmitter, "QUIET", "-") + require_ack(transmitter, "CONFIG", HOME_CANDIDATE) + results.write({"event": "boot", "identity": identity, "status": status, + "duration_s": args.duration_s, + "pulse_interval_s": args.pulse_interval_s, + "focus": {"bin": args.bin, "dwell_ms": args.dwell_ms, + "samples": args.samples}}) + + print(f"reference train: {HOME_CANDIDATE} every {args.pulse_interval_s}s, " + f"{args.duration_s}s per arm") + with PulseTrain(transmitter, args.pulse_interval_s) as train: + baseline = run_arm(card, results, "watch-only", args.duration_s, train, None) + describe(baseline) + focus = run_arm(card, results, "watch+focus", args.duration_s, train, + (args.bin, args.dwell_ms, args.samples)) + describe(focus) + + if baseline["received_fraction"] is not None and focus["received_fraction"] is not None: + delta = focus["received_fraction"] - baseline["received_fraction"] + print(f"\ndelta: {delta:+.3f} received fraction with Focus interleaved") + print("Read this against away_fraction, not alone: a small loss at a small") + print("away fraction says nothing about a larger one. Overlapping intervals") + print("mean the arms are not distinguishable at this trial count.") + print("\nThis is the measurement §6.3 asks for. Accepting a radio-away budget " + "is a separate operator decision.") + finally: + if transmitter is not None: + try: + require_ack(transmitter, "QUIET", "-", timeout=5.0) + finally: + transmitter.close() + card.close() + results.close() + + +if __name__ == "__main__": + try: + main() + except KeyboardInterrupt: + print("interrupted; completed arms are already durable", file=sys.stderr) + sys.exit(130) + except (OSError, serial.SerialException, RuntimeError, TimeoutError, ValueError) as error: + print(f"phase12 Watch opportunity: {error}", file=sys.stderr) + sys.exit(2) diff --git a/src/bench_fault.cpp b/src/bench_fault.cpp index 9fe0ad2..9c14e3d 100644 --- a/src/bench_fault.cpp +++ b/src/bench_fault.cpp @@ -28,6 +28,11 @@ volatile uint16_t sweepSettleMs = ENERGY_SWEEP_SETTLE_DEFAULT_MS; volatile EnergyObservationResult passBCadLastResult = EnergyObservationResult::RADIO_ERROR; volatile bool passBCadHaveResult = false; +// Wide enough to overrun focusRequestTimeoutMs() for the longest legal +// request (2,000ms dwell + 1,000ms slack) and no wider. +constexpr uint32_t FOCUS_BENCH_STALL_MAX_MS = 4000; +volatile uint16_t focusStallMs = 0; + #if defined(LORATRACE_BENCH_FAULTS) // energy_plan.h's own ENERGY_BIN_RESERVED_COUNT -- the same hard ceiling // Pass A's bin loop is already bounded by, so this array is sized to hold @@ -271,6 +276,43 @@ bool benchRssiWindowTriggerAllowed() { #endif } +bool benchFocusSurveyTriggerAllowed() { +#if !defined(LORATRACE_BENCH_FAULTS) + return false; +#else + return true; +#endif +} + +bool benchFocusStallConfigure(uint32_t stall_ms) { +#if !defined(LORATRACE_BENCH_FAULTS) + (void)stall_ms; + return false; +#else + if (stall_ms > FOCUS_BENCH_STALL_MAX_MS) return false; + focusStallMs = (uint16_t)stall_ms; + return true; +#endif +} + +uint16_t benchFocusStallTakeMs() { +#if !defined(LORATRACE_BENCH_FAULTS) + return 0; +#else + const uint16_t stall = focusStallMs; + focusStallMs = 0; + return stall; +#endif +} + +bool benchArbitrationTriggerAllowed() { +#if !defined(LORATRACE_BENCH_FAULTS) + return false; +#else + return true; +#endif +} + void benchPassBCadRecordResult(EnergyObservationResult result) { #if defined(LORATRACE_BENCH_FAULTS) passBCadLastResult = result; diff --git a/src/bench_fault.h b/src/bench_fault.h index 9b48a27..ab26ded 100644 --- a/src/bench_fault.h +++ b/src/bench_fault.h @@ -113,6 +113,24 @@ bool benchSweepFloorQuery(uint16_t bin, int16_t &rssi_avg_dbm_x10); // holds still long enough to. Same production/bench split as the rest of // this file. bool benchRssiWindowTriggerAllowed(); +// Phase 12 bench-only Focus request gate. Production keeps Focus unreachable +// until its on-device control and claim gate are ready. +bool benchFocusSurveyTriggerAllowed(); + +// Bench-image-only one-shot stall injected into the Focus sample loop, in ms +// (0-4000). It is the only way to reach the timeout path deliberately: +// production Focus only times out when the SPI bus is genuinely contended, +// which a bench fixture cannot arrange on demand. Consumed once by the next +// accepted request; production stores nothing and always reports 0. +bool benchFocusStallConfigure(uint32_t stall_ms); +uint16_t benchFocusStallTakeMs(); + +// Bench-image-only gate for entering Cell and Scope from framed control. +// Both are UI-only actions in production, so a fixture otherwise cannot prove +// Focus's mutual exclusion against them +// (docs/research/phase12-survey-truth-design.md §8's Device gate). It grants +// no new radio behavior -- it calls the same request functions the menu does. +bool benchArbitrationTriggerAllowed(); // Bench-image-only readback of the most recent BENCH_PASS_B_CAD attempt's // raw result (research/phase9-sweep-pass-b-design.md's standing diff --git a/src/focus_observation.h b/src/focus_observation.h new file mode 100644 index 0000000..cee292b --- /dev/null +++ b/src/focus_observation.h @@ -0,0 +1,213 @@ +#pragma once +// LoRaTrace RX — Phase 12 Focus Survey bounded RSSI statistics and durable +// row contract. No coverage label is produced until its thresholds are earned +// by the controlled matrix in phase12-survey-truth-design.md. + +#include +#include +#include + +#include "detection.h" // missionProfileName() +#include "focus_plan.h" + +constexpr int16_t FOCUS_RSSI_NO_SAMPLE_DBM_X10 = 32767; +constexpr int16_t FOCUS_RSSI_HISTOGRAM_MIN_DBM = -140; +constexpr int16_t FOCUS_RSSI_HISTOGRAM_MAX_DBM = 0; +constexpr uint16_t FOCUS_RSSI_HISTOGRAM_BUCKET_COUNT = + (uint16_t)(FOCUS_RSSI_HISTOGRAM_MAX_DBM - FOCUS_RSSI_HISTOGRAM_MIN_DBM + 1); + +constexpr uint8_t FOCUS_RSSI_FLAG_SAMPLE_OVERFLOW = 0x01; +constexpr uint8_t FOCUS_RSSI_FLAG_BUCKET_OVERFLOW = 0x02; +constexpr uint8_t FOCUS_RSSI_FLAG_UNDERFLOW = 0x04; +constexpr uint8_t FOCUS_RSSI_FLAG_OVERFLOW = 0x08; + +// A fixed 1dB histogram is compact enough for one result while retaining no +// raw samples. Values beyond the documented receiver range are clamped to an +// endpoint and flagged, so summaries never pretend that the range is wider. +struct FocusRssiHistogram { + uint8_t bucket_counts[FOCUS_RSSI_HISTOGRAM_BUCKET_COUNT] = {}; + uint16_t sample_count = 0; + int16_t peak_dbm_x10 = FOCUS_RSSI_NO_SAMPLE_DBM_X10; + uint8_t flags = 0; +}; + +static_assert(sizeof(FocusRssiHistogram) <= 160, + "Focus histogram exceeds its one-result static SRAM budget"); + +inline int16_t focusRssiDbmX10ToNearestDbm(int16_t value_dbm_x10) { + // C++ integer division truncates toward zero, so handle negative values + // explicitly to retain half-away-from-zero 1dB binning. + return value_dbm_x10 >= 0 ? (int16_t)((value_dbm_x10 + 5) / 10) + : (int16_t)((value_dbm_x10 - 5) / 10); +} + +inline void focusHistogramAddSample(FocusRssiHistogram &histogram, + int16_t rssi_dbm_x10) { + if (histogram.sample_count == UINT16_MAX) { + histogram.flags |= FOCUS_RSSI_FLAG_SAMPLE_OVERFLOW; + return; + } + + if (histogram.sample_count == 0 || rssi_dbm_x10 > histogram.peak_dbm_x10) { + histogram.peak_dbm_x10 = rssi_dbm_x10; + } + histogram.sample_count++; + + int16_t dbm = focusRssiDbmX10ToNearestDbm(rssi_dbm_x10); + if (dbm < FOCUS_RSSI_HISTOGRAM_MIN_DBM) { + dbm = FOCUS_RSSI_HISTOGRAM_MIN_DBM; + histogram.flags |= FOCUS_RSSI_FLAG_UNDERFLOW; + } else if (dbm > FOCUS_RSSI_HISTOGRAM_MAX_DBM) { + dbm = FOCUS_RSSI_HISTOGRAM_MAX_DBM; + histogram.flags |= FOCUS_RSSI_FLAG_OVERFLOW; + } + + const uint16_t index = (uint16_t)(dbm - FOCUS_RSSI_HISTOGRAM_MIN_DBM); + if (histogram.bucket_counts[index] == UINT8_MAX) { + histogram.flags |= FOCUS_RSSI_FLAG_BUCKET_OVERFLOW; + } else { + histogram.bucket_counts[index]++; + } +} + +constexpr bool focusHistogramHasExactQuantiles(const FocusRssiHistogram &histogram) { + return histogram.sample_count > 0 && + (histogram.flags & (FOCUS_RSSI_FLAG_SAMPLE_OVERFLOW | + FOCUS_RSSI_FLAG_BUCKET_OVERFLOW)) == 0; +} + +// Returns a 1dB-quantized tenths-of-dBm value, or NO_SAMPLE when the retained +// buckets cannot faithfully represent the count (or no sample was accepted). +inline int16_t focusHistogramPercentileDbmX10(const FocusRssiHistogram &histogram, + uint8_t percentile) { + if (!focusHistogramHasExactQuantiles(histogram) || percentile == 0 || percentile > 100) { + return FOCUS_RSSI_NO_SAMPLE_DBM_X10; + } + const uint32_t rank = ((uint32_t)histogram.sample_count * percentile + 99u) / 100u; + uint32_t cumulative = 0; + for (uint16_t i = 0; i < FOCUS_RSSI_HISTOGRAM_BUCKET_COUNT; ++i) { + cumulative += histogram.bucket_counts[i]; + if (cumulative >= rank) { + return (int16_t)((FOCUS_RSSI_HISTOGRAM_MIN_DBM + (int16_t)i) * 10); + } + } + return FOCUS_RSSI_NO_SAMPLE_DBM_X10; +} + +inline int16_t focusHistogramMedianDbmX10(const FocusRssiHistogram &histogram) { + return focusHistogramPercentileDbmX10(histogram, 50); +} + +inline int16_t focusHistogramP90DbmX10(const FocusRssiHistogram &histogram) { + return focusHistogramPercentileDbmX10(histogram, 90); +} + +enum class FocusRequestStatus : uint8_t { + COMPLETE = 0, + CANCELLED, + TIMEOUT, + FAILED, +}; + +inline const char *focusRequestStatusName(FocusRequestStatus status) { + switch (status) { + case FocusRequestStatus::COMPLETE: return "complete"; + case FocusRequestStatus::CANCELLED: return "cancelled"; + case FocusRequestStatus::TIMEOUT: return "timeout"; + case FocusRequestStatus::FAILED: return "failed"; + default: return "unknown"; + } +} + +// Queue/log record. GPS/run fields stay formatter parameters so radio +// ownership remains bounded. +struct FocusObservation { + uint32_t rx_millis = 0; + uint32_t observation_ms = 0; + float freq_mhz = 0.0f; + uint16_t focus_id = 0; + uint16_t selection_bin_index = 0; + uint16_t requested_dwell_ms = 0; + uint16_t requested_samples = 0; + uint16_t sample_count = 0; + uint16_t qualifying_count = 0; + int16_t rssi_median_dbm_x10 = FOCUS_RSSI_NO_SAMPLE_DBM_X10; + int16_t rssi_p90_dbm_x10 = FOCUS_RSSI_NO_SAMPLE_DBM_X10; + int16_t rssi_peak_dbm_x10 = FOCUS_RSSI_NO_SAMPLE_DBM_X10; + int16_t radio_status = 0; + uint8_t profile = 0; // MissionProfile + uint8_t requested_passes = 0; + uint8_t valid_passes = 0; + FocusSelectionSource selection_source = FocusSelectionSource::PRESET; + FocusRequestStatus request_status = FocusRequestStatus::FAILED; + bool home_restore = false; + bool wifi_on = false; +}; + +static_assert(sizeof(FocusObservation) <= 48, + "FocusObservation exceeds the Phase 12 one-result queue budget"); +static_assert(sizeof(FocusRssiHistogram) + sizeof(FocusObservation) <= 224, + "Focus working state must leave headroom below the 256B target"); + +// The logger's row buffer, defined here so the writer and its budget test +// cannot drift apart. An over-long row is dropped, not truncated +// (focusObservationFormatCsv() returns 0), so the margin is deliberate: +// measured worst case is 189 bytes with every field at its widest, including +// widths the bounded request cannot actually produce. +constexpr size_t FOCUS_CSV_ROW_MAX = 256; + +constexpr const char *FOCUS_CSV_HEADER = + "timestamp_utc,lat,lon,fix_quality,run,rx_uptime_ms,profile,focus_id," + "selection_source,selection_bin_index,freq_mhz,requested_passes,valid_passes," + "requested_dwell_ms,observation_ms,requested_samples,sample_count," + "rssi_median_dbm,rssi_p90_dbm,rssi_peak_dbm,qualifying_count,coverage," + "request_status,home_restore,wifi_on,radio_status"; + +inline void focusFormatRssiOrBlank(int16_t rssi_dbm_x10, char *out, size_t out_size) { + if (rssi_dbm_x10 == FOCUS_RSSI_NO_SAMPLE_DBM_X10) { + if (out_size > 0) out[0] = '\0'; + return; + } + snprintf(out, out_size, "%.1f", (double)rssi_dbm_x10 / 10.0); +} + +inline size_t focusObservationFormatCsv(const FocusObservation &observation, + char *out, size_t out_size, + const char *timestamp_utc, bool has_fix, + double lat, double lon, uint8_t fix_quality, + uint16_t run) { + if (out == nullptr || out_size == 0) return 0; + + char latbuf[16], lonbuf[16], medianbuf[12], p90buf[12], peakbuf[12]; + if (has_fix) { + snprintf(latbuf, sizeof(latbuf), "%.6f", lat); + snprintf(lonbuf, sizeof(lonbuf), "%.6f", lon); + } else { + latbuf[0] = '\0'; + lonbuf[0] = '\0'; + } + focusFormatRssiOrBlank(observation.rssi_median_dbm_x10, medianbuf, sizeof(medianbuf)); + focusFormatRssiOrBlank(observation.rssi_p90_dbm_x10, p90buf, sizeof(p90buf)); + focusFormatRssiOrBlank(observation.rssi_peak_dbm_x10, peakbuf, sizeof(peakbuf)); + + // The doubled comma before request_status is intentional: coverage is an + // empty persisted field until controlled measurements select its policy. + const int n = snprintf( + out, out_size, + "%s,%s,%s,%u,%u,%lu,%s,%u,%s,%u,%.3f,%u,%u,%u,%lu,%u,%u,%s,%s,%s,%u,,%s,%u,%u,%d", + timestamp_utc ? timestamp_utc : "", latbuf, lonbuf, + (unsigned)fix_quality, (unsigned)run, (unsigned long)observation.rx_millis, + missionProfileName(observation.profile), (unsigned)observation.focus_id, + focusSelectionSourceName(observation.selection_source), + (unsigned)observation.selection_bin_index, (double)observation.freq_mhz, + (unsigned)observation.requested_passes, (unsigned)observation.valid_passes, + (unsigned)observation.requested_dwell_ms, (unsigned long)observation.observation_ms, + (unsigned)observation.requested_samples, (unsigned)observation.sample_count, + medianbuf, p90buf, peakbuf, (unsigned)observation.qualifying_count, + focusRequestStatusName(observation.request_status), + (unsigned)observation.home_restore, (unsigned)observation.wifi_on, + (int)observation.radio_status); + + if (n < 0 || (size_t)n >= out_size) return 0; + return (size_t)n; +} diff --git a/src/focus_plan.h b/src/focus_plan.h new file mode 100644 index 0000000..7367616 --- /dev/null +++ b/src/focus_plan.h @@ -0,0 +1,100 @@ +#pragma once +// LoRaTrace RX — Phase 12 Focus Survey request contract. This is pure +// selection math only: it does not own the radio or start an acquisition. + +#include + +#include "energy_plan.h" + +// Phase 12 starts with exactly one selected frequency. A wider request needs +// a new static-RAM and radio-away budget; it must not quietly grow here. +constexpr uint8_t FOCUS_SELECTED_BIN_COUNT = 1; +// The first bench prototype makes one measurement per request. The controlled +// matrix supplies its 30 trials by issuing 30 independently logged requests, +// not by monopolizing Watch for a hidden multi-pass loop. +constexpr uint8_t FOCUS_BENCH_REQUESTED_PASSES = 1; +constexpr uint16_t FOCUS_BENCH_DWELL_MIN_MS = 2; +constexpr uint16_t FOCUS_BENCH_DWELL_MAX_MS = 2000; +constexpr uint16_t FOCUS_BENCH_SAMPLES_MIN = 2; +constexpr uint16_t FOCUS_BENCH_SAMPLES_MAX = 64; + +// A bounded request must be bounded in wall-clock time, not just in samples: +// each sample waits on the shared SPI bus (radio_task.cpp's 250ms BUS_WAIT), +// so contention can stretch a nominal dwell well past it while the radio is +// away from home. Past this deadline the request stops sampling and +// terminates as `timeout` instead of running long. The slack absorbs a few +// bus waits plus scheduling jitter; it is a bench-slice bound, and §6's +// matrix (docs/research/phase12-survey-truth-design.md) may revise it before +// any operator-facing control exists. +constexpr uint16_t FOCUS_REQUEST_TIMEOUT_SLACK_MS = 1000; + +enum class FocusSelectionSource : uint8_t { + SWEEP_BIN = 0, + WATERFALL_BIN, + PRESET, +}; + +inline const char *focusSelectionSourceName(FocusSelectionSource source) { + switch (source) { + case FocusSelectionSource::SWEEP_BIN: return "sweep"; + case FocusSelectionSource::WATERFALL_BIN: return "waterfall"; + case FocusSelectionSource::PRESET: return "preset"; + default: return "unknown"; + } +} + +constexpr bool focusSelectionSourceIsKnown(FocusSelectionSource source) { + return source == FocusSelectionSource::SWEEP_BIN || + source == FocusSelectionSource::WATERFALL_BIN || + source == FocusSelectionSource::PRESET; +} + +// The 100ms/500ms/2s bench arms are not production defaults, and coverage +// thresholds remain unselected. Bounds make this bench-only request finite. +struct FocusRequest { + Region region = Region::GLOBAL; + EnergyBinStep bin_step = ENERGY_SWEEP_DEFAULT_STEP; + uint16_t selection_bin_index = 0; + uint16_t requested_dwell_ms = 0; + uint16_t requested_samples = 0; + uint8_t requested_passes = 0; + FocusSelectionSource selection_source = FocusSelectionSource::PRESET; +}; + +static_assert(sizeof(FocusRequest) <= 16, + "FocusRequest must stay a small fixed radio-control payload"); + +constexpr bool focusRequestHasValidBin(const FocusRequest &request) { + return request.selection_bin_index < + energyBinCount(energySweepBandForRegion(request.region), request.bin_step); +} + +constexpr bool focusRequestIsValid(const FocusRequest &request) { + return FOCUS_SELECTED_BIN_COUNT == 1 && + focusSelectionSourceIsKnown(request.selection_source) && + focusRequestHasValidBin(request) && + request.requested_passes == FOCUS_BENCH_REQUESTED_PASSES && + request.requested_dwell_ms >= FOCUS_BENCH_DWELL_MIN_MS && + request.requested_dwell_ms <= FOCUS_BENCH_DWELL_MAX_MS && + request.requested_samples >= FOCUS_BENCH_SAMPLES_MIN && + request.requested_samples <= FOCUS_BENCH_SAMPLES_MAX; +} + +// Deadline measured from the first sample-loop tick, not from the request's +// acceptance: queue latency is Core 1's scheduling, not the dwell's cost. +constexpr uint32_t focusRequestTimeoutMs(const FocusRequest &request) { + return (uint32_t)request.requested_dwell_ms + FOCUS_REQUEST_TIMEOUT_SLACK_MS; +} + +// Worst-case time a legal request may hold the radio before restore begins. +// Restore is bounded separately by restoreHomeListen()'s own path. +constexpr uint32_t FOCUS_MAX_SAMPLING_MS = + (uint32_t)FOCUS_BENCH_DWELL_MAX_MS + FOCUS_REQUEST_TIMEOUT_SLACK_MS; +static_assert(FOCUS_MAX_SAMPLING_MS <= 3000, + "Focus must not grow an unbudgeted radio-away window"); + +inline float focusRequestFrequencyMhz(const FocusRequest &request) { + return energyBinFrequencyMhz(request.selection_bin_index, + energySweepBandForRegion(request.region), + request.bin_step); +} diff --git a/src/focus_runtime.h b/src/focus_runtime.h new file mode 100644 index 0000000..fdeb242 --- /dev/null +++ b/src/focus_runtime.h @@ -0,0 +1,76 @@ +#pragma once +// Phase 12 radio-task lifecycle bookkeeping. Hardware calls remain outside +// this header; it makes the terminal row impossible to publish before home +// restore has been recorded. + +#include + +#include "focus_observation.h" // FocusRequestStatus + +enum class FocusRuntimeState : uint8_t { + IDLE, + SURVEYING, + RESTORING, + COMPLETE, + CANCELLED, + TIMEOUT, + FAILED, +}; + +struct FocusRuntime { + FocusRequest request; + FocusRuntimeState state = FocusRuntimeState::IDLE; + uint16_t valid_passes = 0; + uint32_t observation_ms = 0; + bool cancelled = false; + bool timed_out = false; + bool failed = false; +}; + +inline bool focusRuntimeBegin(FocusRuntime &runtime, const FocusRequest &request) { + if (runtime.state == FocusRuntimeState::SURVEYING || + runtime.state == FocusRuntimeState::RESTORING || !focusRequestIsValid(request)) { + return false; + } + runtime = FocusRuntime{}; + runtime.request = request; + runtime.state = FocusRuntimeState::SURVEYING; + return true; +} + +inline void focusRuntimeNoteValidPass(FocusRuntime &runtime, uint32_t observation_ms) { + if (runtime.state != FocusRuntimeState::SURVEYING) return; + runtime.valid_passes++; + runtime.observation_ms += observation_ms; +} + +inline void focusRuntimeCancel(FocusRuntime &runtime) { + if (runtime.state == FocusRuntimeState::SURVEYING) runtime.cancelled = true; +} + +inline void focusRuntimeTimeout(FocusRuntime &runtime) { + if (runtime.state == FocusRuntimeState::SURVEYING) runtime.timed_out = true; +} + +inline void focusRuntimeFail(FocusRuntime &runtime) { + if (runtime.state == FocusRuntimeState::SURVEYING) runtime.failed = true; +} + +inline void focusRuntimeBeginRestore(FocusRuntime &runtime) { + if (runtime.state == FocusRuntimeState::SURVEYING) runtime.state = FocusRuntimeState::RESTORING; +} + +inline FocusRequestStatus focusRuntimeFinishRestore(FocusRuntime &runtime, bool restored) { + if (!restored) runtime.failed = true; + if (runtime.failed) runtime.state = FocusRuntimeState::FAILED; + else if (runtime.cancelled) runtime.state = FocusRuntimeState::CANCELLED; + else if (runtime.timed_out) runtime.state = FocusRuntimeState::TIMEOUT; + else runtime.state = FocusRuntimeState::COMPLETE; + + switch (runtime.state) { + case FocusRuntimeState::COMPLETE: return FocusRequestStatus::COMPLETE; + case FocusRuntimeState::CANCELLED: return FocusRequestStatus::CANCELLED; + case FocusRuntimeState::TIMEOUT: return FocusRequestStatus::TIMEOUT; + default: return FocusRequestStatus::FAILED; + } +} diff --git a/src/logger_task.cpp b/src/logger_task.cpp index b0d8208..aded767 100644 --- a/src/logger_task.cpp +++ b/src/logger_task.cpp @@ -10,6 +10,7 @@ #include "cell_observation.h" #include "detection.h" #include "energy_observation.h" +#include "focus_observation.h" #include "gps_task.h" #include "memory_stats.h" #include "serial_control.h" @@ -30,6 +31,7 @@ constexpr const char *PROBE_LEAF = "probe.csv"; constexpr const char *ENERGY_LEAF = "energy.csv"; constexpr const char *NODES_LEAF = "nodes.csv"; constexpr const char *CELL_LEAF = "cell.csv"; +constexpr const char *FOCUS_LEAF = "focus.csv"; // Resolved once, on the first successful mount of this power-on. uint16_t runIndex = 0; @@ -39,6 +41,7 @@ char probePath[RUN_PATH_MAX]; char energyPath[RUN_PATH_MAX]; char nodesPath[RUN_PATH_MAX]; char cellPath[RUN_PATH_MAX]; +char focusPath[RUN_PATH_MAX]; // ~2KB holds a few complete raw-frame rows. Sized to keep a single flush // short (see the header): bigger buffers mean longer bus holds, which is @@ -67,6 +70,7 @@ QueueHandle_t scanObservationQueue = nullptr; QueueHandle_t energyObservationQueue = nullptr; QueueHandle_t identityQueue = nullptr; QueueHandle_t cellObservationQueue = nullptr; +QueueHandle_t focusObservationQueue = nullptr; bool sdReady = false; bool initialSdMounted = false; volatile bool sdRetryRequested = false; @@ -99,6 +103,9 @@ volatile uint32_t identityRowsDropped = 0; volatile uint32_t maxCellMs = 0; volatile uint32_t cellRowsWritten = 0; volatile uint32_t cellRowsDropped = 0; +volatile uint32_t maxFocusMs = 0; +volatile uint32_t focusRowsWritten = 0; +volatile uint32_t focusRowsDropped = 0; // Highest runNNNN index already on the card, or 0 if there are none. // Assumes the caller holds the bus and SD is mounted. @@ -185,6 +192,9 @@ bool openLogsLocked(bool remount) { if (runFilePath(cellPath, sizeof(cellPath), LOG_DIR, runIndex, CELL_LEAF) == 0) { return false; } + if (runFilePath(focusPath, sizeof(focusPath), LOG_DIR, runIndex, FOCUS_LEAF) == 0) { + return false; + } } if (!ensureCsvLocked(detectionsPath, LOG_CSV_HEADER)) return false; @@ -202,6 +212,7 @@ bool openLogsLocked(bool remount) { // Same durability tier as probePath/energyPath: Cell is mission // output too when an operator runs it. if (!ensureCsvLocked(cellPath, CELL_CSV_HEADER)) return false; + if (!ensureCsvLocked(focusPath, FOCUS_CSV_HEADER)) return false; return true; } @@ -523,6 +534,35 @@ void appendCellObservation(const CellObservation &observation) { } } +void appendFocusObservation(const FocusObservation &observation) { + if (!sdReady) { + focusRowsDropped++; + return; + } + GpsFix fix; + const bool haveFix = gpsGetFix(fix, pdMS_TO_TICKS(50)); + const uint32_t now = millis(); + const bool fresh = haveFix && gpsFixIsFresh(fix, now, FIX_MAX_AGE_MS); + char timestamp[24]; + detectionFormatTimestamp(timestamp, sizeof(timestamp), haveFix && fix.has_time, fix.year, + fix.month, fix.day, fix.hour, fix.minute, fix.second); + char row[FOCUS_CSV_ROW_MAX]; + const size_t n = focusObservationFormatCsv( + observation, row, sizeof(row), timestamp, fresh, fix.lat, fix.lon, + haveFix ? fix.fix_quality : 0, runIndex); + if (n == 0) { + focusRowsDropped++; + return; + } + row[n] = '\n'; + const WriteResult result = appendToFile(focusPath, row, n + 1, maxFocusMs); + if (result == WriteResult::OK) focusRowsWritten++; + else { + focusRowsDropped++; + if (result == WriteResult::FILE_ERROR) sdReady = false; + } +} + void appendNodeIdentity(const NodeIdentity &identity) { if (!sdReady) { identityRowsDropped++; @@ -615,6 +655,12 @@ void loggerTask(void *) { appendCellObservation(observation); } } + if (focusObservationQueue != nullptr) { + FocusObservation observation; + if (xQueueReceive(focusObservationQueue, &observation, 0) == pdTRUE) { + appendFocusObservation(observation); + } + } const uint32_t analyzerSweepRuns = radioEnergySweepCount(); if (analyzerSweepRuns != lastAnalyzerSweepSeen) { @@ -659,12 +705,14 @@ void loggerTask(void *) { } // namespace bool loggerTaskStart(QueueHandle_t queue, QueueHandle_t scanQueue, QueueHandle_t energyQueue, - QueueHandle_t nodesQueue, QueueHandle_t cellQueue, bool mountedAtBoot) { + QueueHandle_t nodesQueue, QueueHandle_t cellQueue, + QueueHandle_t focusQueue, bool mountedAtBoot) { detectionQueue = queue; scanObservationQueue = scanQueue; energyObservationQueue = energyQueue; identityQueue = nodesQueue; cellObservationQueue = cellQueue; + focusObservationQueue = focusQueue; initialSdMounted = mountedAtBoot; sdReady = false; sdRetryRequested = false; @@ -743,6 +791,8 @@ uint32_t loggerCellRowsWritten() { uint32_t loggerCellRowsDropped() { return cellRowsDropped; } +uint32_t loggerFocusRowsWritten() { return focusRowsWritten; } +uint32_t loggerFocusRowsDropped() { return focusRowsDropped; } void loggerDebugToggle() { loggerDebugSetEnabled(!debugVerbose); diff --git a/src/logger_task.h b/src/logger_task.h index fa473a2..0fe8861 100644 --- a/src/logger_task.h +++ b/src/logger_task.h @@ -42,6 +42,7 @@ #include "cell_observation.h" #include "energy_observation.h" +#include "focus_observation.h" #include "node_identity.h" #include "scan_observation.h" @@ -54,7 +55,8 @@ // it down and immediately remounting it. Returns false if the task could // not be created. bool loggerTaskStart(QueueHandle_t queue, QueueHandle_t scanQueue, QueueHandle_t energyQueue, - QueueHandle_t identityQueue, QueueHandle_t cellQueue, bool initialSdMounted); + QueueHandle_t identityQueue, QueueHandle_t cellQueue, + QueueHandle_t focusQueue, bool initialSdMounted); // True once SD is mounted and the log file is writable. When false the // task keeps draining the queue and discarding — a missing card must not @@ -89,6 +91,8 @@ uint32_t loggerIdentityRowsWritten(); uint32_t loggerIdentityRowsDropped(); uint32_t loggerCellRowsWritten(); uint32_t loggerCellRowsDropped(); +uint32_t loggerFocusRowsWritten(); +uint32_t loggerFocusRowsDropped(); // This power-on's run index, or 0 before SD has mounted. Matches the // runNNNN directory the logs are being written into. diff --git a/src/main.cpp b/src/main.cpp index 54ee1b4..27af9e5 100644 --- a/src/main.cpp +++ b/src/main.cpp @@ -33,6 +33,7 @@ #include "board_pins.h" #include "capture_settings.h" #include "cell_observation.h" +#include "focus_observation.h" #include "channel_plans.h" #include "config.h" #include "detection.h" @@ -115,6 +116,7 @@ constexpr UBaseType_t ENERGY_OBSERVATION_QUEUE_DEPTH = 32; // covers a full sweep between logger drains — a starting choice, not a // measured one, same as the other queue depths here. constexpr UBaseType_t CELL_OBSERVATION_QUEUE_DEPTH = 16; +constexpr UBaseType_t FOCUS_OBSERVATION_QUEUE_DEPTH = 4; // How long the completed boot checklist stays on screen before uiTaskStart() // takes over the panel with the main status pages. constexpr uint32_t BOOT_CHECKLIST_HOLD_MS = 1000; @@ -123,6 +125,7 @@ QueueHandle_t scanObservationQueue = nullptr; QueueHandle_t energyObservationQueue = nullptr; QueueHandle_t identityQueue = nullptr; QueueHandle_t cellObservationQueue = nullptr; +QueueHandle_t focusObservationQueue = nullptr; void splashLine(const String &msg, uint16_t color = SPLASH_FG) { if (!displayReady) return; @@ -406,6 +409,10 @@ void setup() { if (cellObservationQueue == nullptr) { fatal(F("FATAL: could not allocate the cell queue."), F("FATAL: cell queue alloc")); } + focusObservationQueue = xQueueCreate(FOCUS_OBSERVATION_QUEUE_DEPTH, sizeof(FocusObservation)); + if (focusObservationQueue == nullptr) { + fatal(F("FATAL: could not allocate the Focus queue."), F("FATAL: Focus queue alloc")); + } // Consumers before producer: the logger must be draining before the // radio starts filling, or the first burst is dropped for no reason. @@ -427,7 +434,7 @@ void setup() { } if (!loggerTaskStart(detectionQueue, scanObservationQueue, energyObservationQueue, identityQueue, - cellObservationQueue, sdMounted)) { + cellObservationQueue, focusObservationQueue, sdMounted)) { fatal(F("FATAL: logger task failed to start."), F("FATAL: logger task")); } // No splash line on success: this is RTOS resource allocation, not a @@ -441,7 +448,7 @@ void setup() { // override the same way this boot resolved `bootProfile`'s. if (!radioTaskStart(activeChannel, bootProfile, channelOverrides, detectionQueue, scanObservationQueue, energyObservationQueue, identityQueue, - cellObservationQueue)) { + cellObservationQueue, focusObservationQueue)) { { SerialLock lock(pdMS_TO_TICKS(200)); if (lock.held()) { diff --git a/src/radio_task.cpp b/src/radio_task.cpp index 088c10d..8b7d179 100644 --- a/src/radio_task.cpp +++ b/src/radio_task.cpp @@ -48,6 +48,13 @@ portMUX_TYPE activeChannelMux = portMUX_INITIALIZER_UNLOCKED; ChannelParams activeChannel; MissionProfile activeProfile = MissionProfile::MESHTASTIC; +// Focus finishes on Core 1 but its bench-only compact result readback is +// served by Serial Control on Core 0. A 40-byte critical copy avoids a torn +// summary; it never carries GPS/run data and does not affect radio ownership. +portMUX_TYPE focusResultMux = portMUX_INITIALIZER_UNLOCKED; +FocusObservation lastFocusObservation; +bool haveLastFocusObservation = false; + // Both writers and readers go through these so the locking cannot be // forgotten at a call site. Radio-task-internal code that already knows it // owns the radio still reads the raw fields directly; only the @@ -218,6 +225,20 @@ volatile uint32_t energyFailureCount = 0; volatile uint32_t energyRecoveryCount = 0; volatile uint32_t energyLastAwayMs = 0; +QueueHandle_t focusRequestQueue = nullptr; +QueueHandle_t focusObservationQueue = nullptr; +volatile bool focusActive = false; +// Reservation closes the cross-core window between accepting a Focus request +// and Core 1 taking it from its mailbox. Other bounded actions must refuse in +// this state, rather than queue behind Focus and run after it. +volatile bool focusPending = false; +volatile bool focusCancelRequested = false; +volatile FocusRuntimeState focusState = FocusRuntimeState::IDLE; +volatile uint32_t focusLastAwayMs = 0; +volatile uint32_t focusObservationCount = 0; +volatile uint32_t focusObservationDropCount = 0; +uint16_t focusNextId = 0; + // Phase 9 Pass B (research/phase9-sweep-pass-b-design.md). Cumulative // across sweeps, same convention as the counters above. volatile uint32_t passBAttemptCount = 0; @@ -551,6 +572,24 @@ bool energyAbortPending() { return false; } +bool focusAbortPending() { + return focusCancelRequested || + (profileSwitchQueue != nullptr && uxQueueMessagesWaiting(profileSwitchQueue) > 0) || + (pauseQueue != nullptr && uxQueueMessagesWaiting(pauseQueue) > 0); +} + +bool applyFocusBenchFault(BenchFaultPoint point, FocusRuntime &runtime) { + BenchFaultAction action; + if (!benchFaultTake(point, action)) return false; + if (action == BenchFaultAction::CANCEL) { + focusRuntimeCancel(runtime); + } else { + lastError = RADIOLIB_ERR_UNKNOWN; + focusRuntimeFail(runtime); + } + return true; +} + bool cellAbortPending() { if (cellCancelRequested) return true; if (profileSwitchQueue != nullptr && uxQueueMessagesWaiting(profileSwitchQueue) > 0) return true; @@ -650,6 +689,156 @@ bool restoreHomeListen(const ChannelParams &homeChannel, MissionProfile homeProf return true; } +void enqueueFocusObservation(const FocusRequest &request, const FocusRssiHistogram &histogram, + FocusRuntime &runtime, uint16_t focusId, bool restored, + int16_t operationStatus) { + FocusObservation observation; + observation.rx_millis = millis(); + observation.observation_ms = runtime.observation_ms; + observation.freq_mhz = focusRequestFrequencyMhz(request); + observation.focus_id = focusId; + observation.selection_bin_index = request.selection_bin_index; + observation.requested_dwell_ms = request.requested_dwell_ms; + observation.requested_samples = request.requested_samples; + observation.sample_count = histogram.sample_count; + observation.rssi_median_dbm_x10 = focusHistogramMedianDbmX10(histogram); + observation.rssi_p90_dbm_x10 = focusHistogramP90DbmX10(histogram); + observation.rssi_peak_dbm_x10 = histogram.peak_dbm_x10; + observation.profile = (uint8_t)activeProfile; + observation.requested_passes = request.requested_passes; + observation.valid_passes = (uint8_t)runtime.valid_passes; + observation.selection_source = request.selection_source; + observation.request_status = focusRuntimeFinishRestore(runtime, restored); + observation.home_restore = restored; + // Recovery must overwrite the live radio error to resume Watch, but the + // Focus row reports the request operation that led to this terminal state. + observation.radio_status = operationStatus; + portENTER_CRITICAL(&focusResultMux); + lastFocusObservation = observation; + haveLastFocusObservation = true; + portEXIT_CRITICAL(&focusResultMux); + focusObservationCount++; + if (focusObservationQueue == nullptr || + xQueueSend(focusObservationQueue, &observation, 0) != pdTRUE) { + focusObservationDropCount++; + } +} + +void performFocusSurvey(const FocusRequest &request) { + if (focusActive || discoveryActive || energyActive || cellActive || scopeActive || + benchPassBCadActive || benchRssiWindowActive) { + focusPending = false; + return; + } + FocusRuntime runtime; + if (!focusRuntimeBegin(runtime, request)) { + focusPending = false; + return; + } + focusActive = true; + focusPending = false; + focusState = FocusRuntimeState::SURVEYING; + focusCancelRequested = false; + const uint32_t awayStarted = millis(); + const ChannelParams homeChannel = activeChannel; + const MissionProfile homeProfile = activeProfile; + const uint16_t focusId = ++focusNextId; + FocusRssiHistogram histogram; + const float frequency = focusRequestFrequencyMhz(request); + // Consumed here, not at the delay below, so an armed stall cannot survive + // a request that never reaches sampling and fire on a later one. Bench + // images only; production always returns 0 (bench_fault.h). + const uint16_t stallMs = benchFocusStallTakeMs(); + + const bool skipSampling = applyFocusBenchFault(BenchFaultPoint::BEFORE_RETUNE, runtime); + int beginState = RADIOLIB_ERR_NONE; + if (!skipSampling) { + SpiBusLock lock(BUS_WAIT); + if (!lock.held()) { + busMissCount++; + beginState = RADIOLIB_ERR_SPI_CMD_TIMEOUT; + } else { + beginState = radio.begin(frequency, homeChannel.bw_khz, homeChannel.sf, + homeChannel.cr_denom, homeChannel.sync_word); + if (beginState == RADIOLIB_ERR_NONE) beginState = radio.startReceive(); + } + lastError = beginState; + } + if (!skipSampling && beginState != RADIOLIB_ERR_NONE) { + lastError = beginState; + focusRuntimeFail(runtime); + } else if (!skipSampling && + !applyFocusBenchFault(BenchFaultPoint::AFTER_RETUNE, runtime)) { + const uint32_t passStarted = millis(); + // The dwell schedule alone does not bound radio-away time: every + // sample waits on the shared SPI bus, so a contended bus stretches + // the pass. Past this the request stops sampling and terminates as + // `timeout` rather than holding the radio (focus_plan.h). + const uint32_t deadline = passStarted + focusRequestTimeoutMs(request); + if (stallMs > 0) vTaskDelay(pdMS_TO_TICKS(stallMs)); + uint32_t firstSample = 0; + uint32_t lastSample = 0; + uint16_t passSamples = 0; + for (uint16_t sample = 0; sample < request.requested_samples; ++sample) { + if (focusAbortPending()) { + focusRuntimeCancel(runtime); + break; + } + if ((int32_t)(millis() - deadline) >= 0) { + focusRuntimeTimeout(runtime); + break; + } + const uint32_t target = passStarted + + (uint32_t)sample * request.requested_dwell_ms / (request.requested_samples - 1); + while ((int32_t)(millis() - target) < 0) { + if (focusAbortPending()) { + focusRuntimeCancel(runtime); + break; + } + if ((int32_t)(millis() - deadline) >= 0) { + focusRuntimeTimeout(runtime); + break; + } + vTaskDelay(1); + } + if (runtime.cancelled || runtime.timed_out) break; + float rssi = 0.0f; + bool gotSample = false; + { + SpiBusLock lock(BUS_WAIT); + if (lock.held()) { + rssi = radio.getRSSI(false); + gotSample = true; + } else { + busMissCount++; + } + } + if (!gotSample) { + focusRuntimeFail(runtime); + break; + } + const uint32_t sampledAt = millis(); + if (passSamples == 0) firstSample = sampledAt; + lastSample = sampledAt; + passSamples++; + focusHistogramAddSample(histogram, energyRssiDbmToFixed(rssi)); + } + if (!runtime.cancelled && !runtime.failed && passSamples == request.requested_samples) { + focusRuntimeNoteValidPass(runtime, lastSample - firstSample); + } + } + focusRuntimeBeginRestore(runtime); + int16_t operationStatus = (int16_t)lastError; + const bool restored = restoreHomeListen(homeChannel, homeProfile); + // A failed recovery is more actionable than the original request error. + if (!restored) operationStatus = (int16_t)lastError; + enqueueFocusObservation(request, histogram, runtime, focusId, restored, operationStatus); + focusState = runtime.state; + focusLastAwayMs = millis() - awayStarted; + focusCancelRequested = false; + focusActive = false; +} + void performDiscoverySweep() { if (discoveryActive || cellActive || scopeActive) return; @@ -1744,6 +1933,14 @@ void radioTask(void *) { continue; } + FocusRequest focusRequest; + if (focusRequestQueue != nullptr && + xQueueReceive(focusRequestQueue, &focusRequest, 0) == pdTRUE) { + if (!paused) performFocusSurvey(focusRequest); + else focusPending = false; // a concurrent pause must not reserve Focus forever + continue; + } + bool cellReq; if (cellSweepQueue != nullptr && xQueueReceive(cellSweepQueue, &cellReq, 0) == pdTRUE) { if (cellReq && !paused) { @@ -1825,7 +2022,7 @@ void radioTask(void *) { bool radioTaskStart(const ChannelParams &channel, MissionProfile profile, const ProfileOverrides &overrides, QueueHandle_t queue, QueueHandle_t scanQueue, QueueHandle_t energyQueue, - QueueHandle_t nodesQueue, QueueHandle_t cellQueue) { + QueueHandle_t nodesQueue, QueueHandle_t cellQueue, QueueHandle_t focusQueue) { // Boot-time and single-threaded (the task below does not exist yet), so // the lock is uncontended here -- taken anyway so every write to this // state goes through one path. @@ -1836,6 +2033,7 @@ bool radioTaskStart(const ChannelParams &channel, MissionProfile profile, energyObservationQueue = energyQueue; identityQueue = nodesQueue; cellObservationQueue = cellQueue; + focusObservationQueue = focusQueue; // Depth-1 mailbox for radioRequestProfileSwitch(). Created here, not // lazily, so a switch request right after boot can't race a @@ -1852,6 +2050,9 @@ bool radioTaskStart(const ChannelParams &channel, MissionProfile profile, energySweepQueue = xQueueCreate(1, sizeof(bool)); if (energySweepQueue == nullptr) return false; + focusRequestQueue = xQueueCreate(1, sizeof(FocusRequest)); + if (focusRequestQueue == nullptr) return false; + cellSweepQueue = xQueueCreate(1, sizeof(bool)); if (cellSweepQueue == nullptr) return false; @@ -2035,7 +2236,7 @@ bool radioRequestDiscoverySweep() { // none can preempt another; see radioRequestEnergySweep()'s/ // radioRequestCellSweep()'s/radioRequestScopeAcquire()'s matching // guards. - if (tracePaused || energyActive || cellActive || scopeActive) return false; + if (tracePaused || energyActive || cellActive || scopeActive || focusActive || focusPending) return false; const bool start = true; xQueueOverwrite(discoveryQueue, &start); xTaskNotifyGive(radioTaskHandle); @@ -2053,7 +2254,7 @@ bool radioRequestEnergySweep() { xTaskNotifyGive(radioTaskHandle); return true; } - if (tracePaused || discoveryActive || cellActive || scopeActive) return false; + if (tracePaused || discoveryActive || cellActive || scopeActive || focusActive || focusPending) return false; const bool start = true; xQueueOverwrite(energySweepQueue, &start); xTaskNotifyGive(radioTaskHandle); @@ -2082,7 +2283,8 @@ bool radioRequestEnergySweepRepeat() { } return true; } - if (energyActive || tracePaused || discoveryActive || cellActive || scopeActive) return false; + if (energyActive || tracePaused || discoveryActive || cellActive || scopeActive || focusActive || + focusPending) return false; energyRepeatActive = true; energyRepeatCount = 0; const bool start = true; @@ -2136,7 +2338,8 @@ bool radioRequestBenchPassBCadTrigger(uint8_t comboIndex) { if (!benchPassBCadTriggerAllowed()) return false; if (radioTaskHandle == nullptr || benchPassBCadQueue == nullptr) return false; if (comboIndex >= PASS_B_SF_BW_CANDIDATE_COUNT) return false; - if (benchPassBCadActive || energyActive || discoveryActive || tracePaused || scopeActive) { + if (benchPassBCadActive || energyActive || discoveryActive || tracePaused || scopeActive || + focusActive || focusPending) { return false; } xQueueOverwrite(benchPassBCadQueue, &comboIndex); @@ -2153,7 +2356,7 @@ bool radioRequestBenchRssiWindow(uint32_t freq_khz) { if (radioTaskHandle == nullptr || benchRssiWindowQueue == nullptr) return false; if (freq_khz < 860000UL || freq_khz > 930000UL) return false; if (benchRssiWindowActive || benchPassBCadActive || energyActive || discoveryActive || - cellActive || tracePaused || scopeActive) { + cellActive || tracePaused || scopeActive || focusActive || focusPending) { return false; } xQueueOverwrite(benchRssiWindowQueue, &freq_khz); @@ -2242,6 +2445,51 @@ uint32_t radioEnergyLastAwayMs() { return energyLastAwayMs; } +bool radioRequestFocusSurvey(const FocusRequest &request) { + if (!focusRequestIsValid(request) || radioTaskHandle == nullptr || focusRequestQueue == nullptr) { + return false; + } + if (focusActive) { + focusCancelRequested = true; + xTaskNotifyGive(radioTaskHandle); + return true; + } + if (focusPending || tracePaused || discoveryActive || energyActive || cellActive || scopeActive || + benchPassBCadActive || benchRssiWindowActive || + (discoveryQueue != nullptr && uxQueueMessagesWaiting(discoveryQueue) > 0) || + (energySweepQueue != nullptr && uxQueueMessagesWaiting(energySweepQueue) > 0) || + (cellSweepQueue != nullptr && uxQueueMessagesWaiting(cellSweepQueue) > 0) || + (scopeAcquireQueue != nullptr && uxQueueMessagesWaiting(scopeAcquireQueue) > 0) || + (benchPassBCadQueue != nullptr && uxQueueMessagesWaiting(benchPassBCadQueue) > 0) || + (benchRssiWindowQueue != nullptr && uxQueueMessagesWaiting(benchRssiWindowQueue) > 0)) { + return false; + } + if (xQueueOverwrite(focusRequestQueue, &request) != pdPASS) return false; + focusPending = true; + xTaskNotifyGive(radioTaskHandle); + return true; +} + +bool radioCancelFocusSurvey() { + if (!focusActive) return false; + focusCancelRequested = true; + xTaskNotifyGive(radioTaskHandle); + return true; +} + +bool radioFocusSurveyIsActive() { return focusActive; } +FocusRuntimeState radioFocusSurveyState() { return focusState; } +uint32_t radioFocusLastAwayMs() { return focusLastAwayMs; } +uint32_t radioFocusObservationCount() { return focusObservationCount; } +uint32_t radioFocusObservationDropCount() { return focusObservationDropCount; } +bool radioFocusLastObservation(FocusObservation &out) { + portENTER_CRITICAL(&focusResultMux); + const bool available = haveLastFocusObservation; + if (available) out = lastFocusObservation; + portEXIT_CRITICAL(&focusResultMux); + return available; +} + uint32_t radioPassBAttemptCount() { return passBAttemptCount; } @@ -2259,7 +2507,7 @@ bool radioRequestCellSweep() { } // Mutually exclusive with Probe, Sweep, and Scope — same convention as // their own guards above. - if (tracePaused || discoveryActive || energyActive || scopeActive) return false; + if (tracePaused || discoveryActive || energyActive || scopeActive || focusActive || focusPending) return false; const bool start = true; xQueueOverwrite(cellSweepQueue, &start); xTaskNotifyGive(radioTaskHandle); @@ -2285,7 +2533,8 @@ bool radioRequestCellSweepRepeat() { } return true; } - if (cellActive || tracePaused || discoveryActive || energyActive || scopeActive) return false; + if (cellActive || tracePaused || discoveryActive || energyActive || scopeActive || focusActive || + focusPending) return false; cellRepeatActive = true; cellRepeatCount = 0; const bool start = true; @@ -2359,7 +2608,7 @@ bool radioRequestScopeAcquire(uint32_t freq_khz) { } // Mutually exclusive with Probe, Sweep, and Cell — same convention as // their own guards above. - if (tracePaused || discoveryActive || energyActive || cellActive) return false; + if (tracePaused || discoveryActive || energyActive || cellActive || focusActive || focusPending) return false; xQueueOverwrite(scopeAcquireQueue, &freq_khz); xTaskNotifyGive(radioTaskHandle); return true; diff --git a/src/radio_task.h b/src/radio_task.h index 88f8221..824a0bc 100644 --- a/src/radio_task.h +++ b/src/radio_task.h @@ -23,6 +23,8 @@ #include "channel_plans.h" #include "detection.h" #include "energy_observation.h" +#include "focus_observation.h" +#include "focus_runtime.h" #include "node_identity.h" #include "region_plan.h" #include "scan_observation.h" @@ -45,7 +47,8 @@ bool radioTaskStart(const ChannelParams &channel, MissionProfile profile, const ProfileOverrides &overrides, QueueHandle_t queue, QueueHandle_t scanQueue, QueueHandle_t energyQueue, - QueueHandle_t identityQueue, QueueHandle_t cellQueue); + QueueHandle_t identityQueue, QueueHandle_t cellQueue, + QueueHandle_t focusQueue); void radioIdentityCaptureSetEnabled(bool enabled); bool radioIdentityCaptureIsEnabled(); @@ -239,6 +242,17 @@ uint32_t radioEnergyFailureCount(); uint32_t radioEnergyRecoveryCount(); uint32_t radioEnergyLastAwayMs(); +bool radioRequestFocusSurvey(const FocusRequest &request); +bool radioCancelFocusSurvey(); +bool radioFocusSurveyIsActive(); +FocusRuntimeState radioFocusSurveyState(); +uint32_t radioFocusLastAwayMs(); +uint32_t radioFocusObservationCount(); +uint32_t radioFocusObservationDropCount(); +// Copies the last completed Focus result under a short cross-core lock. +// It contains no GPS/run fields; Core 0 alone adds those while writing CSV. +bool radioFocusLastObservation(FocusObservation &out); + // Phase 9 Pass B (research/phase9-sweep-pass-b-design.md): CAD attempts run // at the first PASS_B_MAX_PEAKS_PER_SWEEP Pass-A peaks this sweep, and how // many of those attempts promoted a real packet to Detection diff --git a/src/serial_control.cpp b/src/serial_control.cpp index 702485e..8fd7e37 100644 --- a/src/serial_control.cpp +++ b/src/serial_control.cpp @@ -112,11 +112,14 @@ void sendStatus(uint16_t sequence) { // question that R= (Probe's recovery count, not a packet count // despite the letter) could not answer; distinct names on purpose so // a host script can't confuse them the way this session just did. - char argument[200] = {}; + // Sized from the protocol's own frame budget rather than by hand: this + // buffer was 240 against a 230-byte budget, which silently dropped the + // whole STATUS frame once the counters grew (serial_control_protocol.h). + char argument[SERIAL_CONTROL_ARGUMENT_MAX + 1] = {}; snprintf(argument, sizeof(argument), "P=%s;T=%u;B=%s;SD=%u;F=%lu;R=%lu;I=%u;N=%u;C=%u,%u,%u,%u;M=%04X;" "W=%s;WI=%u;WN=%u;WP=%u;PBA=%lu;PBD=%lu;BPC=%u;RW=%u;WIFI=%u;EA=%lu;" - "RXP=%lu;RXC=%lu", + "RXP=%lu;RXC=%lu;FS=%u;FA=%lu;FO=%lu;FD=%lu;FW=%lu;FL=%lu", profile, radioIsTracePaused() ? 0U : 1U, probe, loggerSdReady() ? 1U : 0U, (unsigned long)(channel.freq_mhz * 1000.0f), (unsigned long)radioDiscoveryRecoveryCount(), @@ -132,7 +135,12 @@ void sendStatus(uint16_t sequence) { (unsigned long)radioPassBAttemptCount(), (unsigned long)radioPassBDetectionCount(), radioBenchPassBCadIsActive() ? 1U : 0U, radioBenchRssiWindowIsActive() ? 1U : 0U, wifiIsEnabled() ? 1U : 0U, (unsigned long)radioEnergyLastAwayMs(), - (unsigned long)radioPacketCount(), (unsigned long)radioCrcErrorCount()); + (unsigned long)radioPacketCount(), (unsigned long)radioCrcErrorCount(), + (unsigned)radioFocusSurveyState(), (unsigned long)radioFocusLastAwayMs(), + (unsigned long)radioFocusObservationCount(), + (unsigned long)radioFocusObservationDropCount(), + (unsigned long)loggerFocusRowsWritten(), + (unsigned long)loggerFocusRowsDropped()); sendFrame(sequence, SerialControlOpcode::STATUS, argument); } @@ -320,6 +328,161 @@ void handleFrame(const SerialControlFrame &frame) { } break; } + case SerialControlOpcode::BENCH_FOCUS: { + if (!benchFocusSurveyTriggerAllowed()) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "UNSUPPORTED"); + break; + } + if (!loggerSdReady()) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "SD_REQUIRED"); + break; + } + char argument[40] = {}; + if (strlen(frame.argument) >= sizeof(argument)) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "BAD_ARGUMENT"); + break; + } + strcpy(argument, frame.argument); + char *a = argument; + char *b = strchr(a, ':'); + if (b == nullptr) { sendFrame(frame.sequence, SerialControlOpcode::ERROR, "BAD_ARGUMENT"); break; } + *b++ = '\0'; + char *c = strchr(b, ':'); + if (c == nullptr) { sendFrame(frame.sequence, SerialControlOpcode::ERROR, "BAD_ARGUMENT"); break; } + *c++ = '\0'; + // Optional 4th field: a one-shot sample-loop stall in ms, the only + // way to reach Focus's timeout on demand (bench_fault.h). Omitting + // it keeps the original bin:dwell:samples grammar working. + char *d = strchr(c, ':'); + if (d != nullptr) *d++ = '\0'; + uint32_t bin, dwell, samples, stall = 0; + if (!serialControlParseUint32(a, bin) || !serialControlParseUint32(b, dwell) || + !serialControlParseUint32(c, samples) || bin > UINT16_MAX || dwell > UINT16_MAX || + samples > UINT16_MAX || + (d != nullptr && !serialControlParseUint32(d, stall))) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "BAD_ARGUMENT"); + break; + } + if (d != nullptr && !benchFocusStallConfigure(stall)) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "BAD_ARGUMENT"); + break; + } + FocusRequest request; + request.region = radioEnergySweepRegion(); + request.selection_source = FocusSelectionSource::SWEEP_BIN; + request.selection_bin_index = (uint16_t)bin; + request.requested_dwell_ms = (uint16_t)dwell; + request.requested_samples = (uint16_t)samples; + request.requested_passes = FOCUS_BENCH_REQUESTED_PASSES; + const bool accepted = radioRequestFocusSurvey(request); + // A refused request never reaches Core 1's one-shot take, so the + // stall would otherwise fire on whichever request ran next. + if (!accepted) benchFocusStallConfigure(0); + sendFrame(frame.sequence, accepted ? SerialControlOpcode::ACK : SerialControlOpcode::ERROR, + accepted ? "QUEUED" : "UNAVAILABLE"); + break; + } + case SerialControlOpcode::BENCH_FOCUS_CANCEL: + if (!benchFocusSurveyTriggerAllowed()) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "UNSUPPORTED"); + } else if (radioCancelFocusSurvey()) { + sendFrame(frame.sequence, SerialControlOpcode::ACK, "CANCEL_QUEUED"); + } else { + sendFrame(frame.sequence, SerialControlOpcode::ACK, "IDLE"); + } + break; + case SerialControlOpcode::BENCH_FOCUS_RESULT: { + if (!benchFocusSurveyTriggerAllowed()) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "UNSUPPORTED"); + break; + } + FocusObservation observation; + if (!radioFocusLastObservation(observation)) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "NO_RESULT"); + break; + } + if (strcmp(frame.argument, "HEALTH") == 0) { + char argument[32] = {}; + snprintf(argument, sizeof(argument), "RS=%u;HR=%u;E=%d", + (unsigned)observation.request_status, + observation.home_restore ? 1U : 0U, + (int)observation.radio_status); + sendFrame(frame.sequence, SerialControlOpcode::ACK, argument); + break; + } + // Fixed-point dBm fields preserve exact device values without + // locale/float formatting or leaking CSV's GPS/run columns. + char argument[144] = {}; + snprintf(argument, sizeof(argument), + "ID=%u;BIN=%u;F=%lu;N=%u;MED=%d;P90=%d;MAX=%d;OBS=%lu;RS=%u;HR=%u;E=%d", + (unsigned)observation.focus_id, + (unsigned)observation.selection_bin_index, + (unsigned long)(observation.freq_mhz * 1000.0f), + (unsigned)observation.sample_count, + (int)observation.rssi_median_dbm_x10, + (int)observation.rssi_p90_dbm_x10, + (int)observation.rssi_peak_dbm_x10, + (unsigned long)observation.observation_ms, + (unsigned)observation.request_status, + observation.home_restore ? 1U : 0U, + (int)observation.radio_status); + sendFrame(frame.sequence, SerialControlOpcode::ACK, argument); + break; + } + case SerialControlOpcode::BENCH_ACTION: { + // Cell and Scope are menu-only actions in production, so a fixture + // has no other way to prove Focus refuses them and is refused by + // them (docs/research/phase12-survey-truth-design.md §8). This + // calls the same request functions the menu does -- both toggle to + // cancel while their own action is active -- and adds no radio + // behavior of its own. + if (!benchArbitrationTriggerAllowed()) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "UNSUPPORTED"); + break; + } + if (strcmp(frame.argument, "STATE") == 0) { + char argument[32] = {}; + snprintf(argument, sizeof(argument), "CELL=%u;SCOPE=%u;FOCUS=%u", + radioCellSweepIsActive() ? 1U : 0U, + radioScopeAcquireIsActive() ? 1U : 0U, + radioFocusSurveyIsActive() ? 1U : 0U); + sendFrame(frame.sequence, SerialControlOpcode::ACK, argument); + break; + } + const bool cell = strncmp(frame.argument, "CELL:", 5) == 0; + const bool scope = strncmp(frame.argument, "SCOPE:", 6) == 0; + if (!cell && !scope) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "BAD_ARGUMENT"); + break; + } + const char *verb = frame.argument + (cell ? 5 : 6); + const bool start = strcmp(verb, "START") == 0; + if (!start && strcmp(verb, "CANCEL") != 0) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "BAD_ARGUMENT"); + break; + } + const bool active = cell ? radioCellSweepIsActive() : radioScopeAcquireIsActive(); + if (start && active) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "ACTIVE"); + break; + } + if (!start && !active) { + sendFrame(frame.sequence, SerialControlOpcode::ACK, "IDLE"); + break; + } + // Scope parks at one frequency; the resolved home channel keeps the + // fixture from inventing one and keeps the hop short. + const uint32_t scopeFreqKhz = (uint32_t)(radioActiveChannel().freq_mhz * 1000.0f); + const bool accepted = cell ? radioRequestCellSweep() + : radioRequestScopeAcquire(scopeFreqKhz); + if (!accepted) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "UNAVAILABLE"); + } else { + sendFrame(frame.sequence, SerialControlOpcode::ACK, + start ? "QUEUED" : "CANCEL_QUEUED"); + } + break; + } case SerialControlOpcode::BENCH_PASS_B_CAD_RESULT: { EnergyObservationResult result; if (!benchPassBCadLastResult(result)) { diff --git a/src/serial_control_protocol.h b/src/serial_control_protocol.h index 92036e6..241e3a9 100644 --- a/src/serial_control_protocol.h +++ b/src/serial_control_protocol.h @@ -9,15 +9,26 @@ #include #include -// 160 was tight even before WIFI_SET/EA (2026-09-02): STATUS's own -// argument alone can realistically approach 155-170 bytes once cumulative -// counters (PBA/PBD) grow past a few digits during a long-running session -// (the Phase 9 24-hour soak exit criterion, ROADMAP.md, being the exact -// case that would hit this) -- serialControlFormatFrame() silently drops -// the frame (returns 0, nothing sent) rather than truncating, so this -// would have failed quietly. 256 gives real headroom, not just enough for -// today's field set. -constexpr size_t SERIAL_CONTROL_FRAME_MAX = 256; +// STATUS's argument is what sizes this, and it only ever grows: 160 was tight +// before WIFI_SET/EA (2026-09-02) and 256 became tight again once Phase 12 +// added six FS/FA/FO/FD/FW/FL fields (2026-09-05). Counted against a +// long-running session's realistic digit counts -- not all-uint32-max -- +// STATUS reaches ~240 bytes of argument, past 256's ~230-byte budget. +// serialControlFormatFrame() drops an over-long frame silently (returns 0, +// nothing sent) rather than truncating, so overrunning this loses exactly +// the newest fields, with no error to notice. 384 restores real headroom. +constexpr size_t SERIAL_CONTROL_FRAME_MAX = 384; + +// Frames are "@LTRX/1 ", so an argument +// budget has to account for the rest. Derived, not eyeballed: the previous +// 240-byte STATUS buffer was set by hand and silently exceeded its frame. +constexpr size_t SERIAL_CONTROL_OPCODE_NAME_MAX = 23; // BENCH_PASS_B_CAD_RESULT +constexpr size_t SERIAL_CONTROL_FRAME_FIXED_BYTES = + 8 /* "@LTRX/1 " */ + 5 /* sequence */ + 1 + SERIAL_CONTROL_OPCODE_NAME_MAX + + 1 + 5 /* " " + 4 CRC digits */; +// Longest argument that still frames, for any opcode, with room for the NUL. +constexpr size_t SERIAL_CONTROL_ARGUMENT_MAX = + SERIAL_CONTROL_FRAME_MAX - SERIAL_CONTROL_FRAME_FIXED_BYTES - 1; // Enum value names below are the actual wire opcode strings (via // serialControlOpcodeName()/serialControlOpcodeFromName()) and stay stable @@ -44,6 +55,10 @@ enum class SerialControlOpcode : uint8_t { BENCH_RSSI_WINDOW, BENCH_RSSI_RESULT, BENCH_PASS_B_CAD_RESULT, + BENCH_FOCUS, + BENCH_FOCUS_CANCEL, + BENCH_FOCUS_RESULT, + BENCH_ACTION, KEY_DUMP, SD_RETRY, WIFI_SET, @@ -90,6 +105,10 @@ inline const char *serialControlOpcodeName(SerialControlOpcode opcode) { case SerialControlOpcode::BENCH_RSSI_WINDOW: return "BENCH_RSSI_WINDOW"; case SerialControlOpcode::BENCH_RSSI_RESULT: return "BENCH_RSSI_RESULT"; case SerialControlOpcode::BENCH_PASS_B_CAD_RESULT: return "BENCH_PASS_B_CAD_RESULT"; + case SerialControlOpcode::BENCH_FOCUS: return "BENCH_FOCUS"; + case SerialControlOpcode::BENCH_FOCUS_CANCEL: return "BENCH_FOCUS_CANCEL"; + case SerialControlOpcode::BENCH_FOCUS_RESULT: return "BENCH_FOCUS_RESULT"; + case SerialControlOpcode::BENCH_ACTION: return "BENCH_ACTION"; case SerialControlOpcode::KEY_DUMP: return "KEY_DUMP"; case SerialControlOpcode::SD_RETRY: return "SD_RETRY"; case SerialControlOpcode::WIFI_SET: return "WIFI_SET"; @@ -120,6 +139,10 @@ inline SerialControlOpcode serialControlOpcodeFromName(const char *name) { if (strcmp(name, "BENCH_RSSI_WINDOW") == 0) return SerialControlOpcode::BENCH_RSSI_WINDOW; if (strcmp(name, "BENCH_RSSI_RESULT") == 0) return SerialControlOpcode::BENCH_RSSI_RESULT; if (strcmp(name, "BENCH_PASS_B_CAD_RESULT") == 0) return SerialControlOpcode::BENCH_PASS_B_CAD_RESULT; + if (strcmp(name, "BENCH_FOCUS") == 0) return SerialControlOpcode::BENCH_FOCUS; + if (strcmp(name, "BENCH_FOCUS_CANCEL") == 0) return SerialControlOpcode::BENCH_FOCUS_CANCEL; + if (strcmp(name, "BENCH_FOCUS_RESULT") == 0) return SerialControlOpcode::BENCH_FOCUS_RESULT; + if (strcmp(name, "BENCH_ACTION") == 0) return SerialControlOpcode::BENCH_ACTION; if (strcmp(name, "KEY_DUMP") == 0) return SerialControlOpcode::KEY_DUMP; if (strcmp(name, "SD_RETRY") == 0) return SerialControlOpcode::SD_RETRY; if (strcmp(name, "WIFI_SET") == 0) return SerialControlOpcode::WIFI_SET; @@ -208,11 +231,14 @@ inline size_t serialControlFormatFrame(char *out, size_t outSize, uint16_t seque argument == nullptr || strchr(argument, ' ') != nullptr) { return 0; } - char body[SERIAL_CONTROL_FRAME_MAX] = {}; - const int bodyLen = snprintf(body, sizeof(body), "@LTRX/1 %u %s %s", + // Built in place: a second SERIAL_CONTROL_FRAME_MAX staging buffer cost + // the 4KB UI task stack as much as the frame itself, and the CRC covers + // the body prefix either way. + const int bodyLen = snprintf(out, outSize, "@LTRX/1 %u %s %s", (unsigned)sequence, serialControlOpcodeName(opcode), argument); - if (bodyLen < 0 || (size_t)bodyLen >= sizeof(body)) return 0; - const int total = snprintf(out, outSize, "%s %04X", body, - (unsigned)serialControlCrc16(body, (size_t)bodyLen)); - return total >= 0 && (size_t)total < outSize ? (size_t)total : 0; + if (bodyLen < 0 || (size_t)bodyLen >= outSize) return 0; + const int crcLen = snprintf(out + bodyLen, outSize - (size_t)bodyLen, " %04X", + (unsigned)serialControlCrc16(out, (size_t)bodyLen)); + if (crcLen < 0 || (size_t)(bodyLen + crcLen) >= outSize) return 0; + return (size_t)(bodyLen + crcLen); } diff --git a/test/test_focus_observation/test_main.cpp b/test/test_focus_observation/test_main.cpp new file mode 100644 index 0000000..68308f6 --- /dev/null +++ b/test/test_focus_observation/test_main.cpp @@ -0,0 +1,156 @@ +#include + +#include + +#include "../../src/focus_observation.h" + +void test_focus_working_state_stays_below_one_result_budget() { + TEST_ASSERT_EQUAL_size_t(148, sizeof(FocusRssiHistogram)); + TEST_ASSERT_EQUAL_size_t(40, sizeof(FocusObservation)); + TEST_ASSERT_EQUAL_size_t(188, sizeof(FocusRssiHistogram) + sizeof(FocusObservation)); +} + +void test_focus_histogram_reports_median_p90_and_peak_without_raw_samples() { + FocusRssiHistogram histogram; + focusHistogramAddSample(histogram, -1000); + focusHistogramAddSample(histogram, -900); + focusHistogramAddSample(histogram, -800); + focusHistogramAddSample(histogram, -700); + focusHistogramAddSample(histogram, -600); + + TEST_ASSERT_EQUAL_UINT16(5, histogram.sample_count); + TEST_ASSERT_EQUAL_INT16(-800, focusHistogramMedianDbmX10(histogram)); + TEST_ASSERT_EQUAL_INT16(-600, focusHistogramP90DbmX10(histogram)); + TEST_ASSERT_EQUAL_INT16(-600, histogram.peak_dbm_x10); +} + +void test_focus_histogram_rounds_to_one_db_and_flags_clamped_input() { + FocusRssiHistogram histogram; + focusHistogramAddSample(histogram, -1005); // nearest 1dB bin: -101dBm + focusHistogramAddSample(histogram, -1600); // below documented range + focusHistogramAddSample(histogram, 50); // above documented range + + TEST_ASSERT_EQUAL_INT16(-1010, focusHistogramMedianDbmX10(histogram)); + TEST_ASSERT_TRUE(histogram.flags & FOCUS_RSSI_FLAG_UNDERFLOW); + TEST_ASSERT_TRUE(histogram.flags & FOCUS_RSSI_FLAG_OVERFLOW); +} + +void test_focus_histogram_refuses_quantiles_after_bucket_saturation() { + FocusRssiHistogram histogram; + for (uint16_t i = 0; i < 256; ++i) focusHistogramAddSample(histogram, -800); + TEST_ASSERT_TRUE(histogram.flags & FOCUS_RSSI_FLAG_BUCKET_OVERFLOW); + TEST_ASSERT_EQUAL_INT16(FOCUS_RSSI_NO_SAMPLE_DBM_X10, + focusHistogramMedianDbmX10(histogram)); +} + +void test_focus_request_status_names_are_explicit() { + TEST_ASSERT_EQUAL_STRING("complete", focusRequestStatusName(FocusRequestStatus::COMPLETE)); + TEST_ASSERT_EQUAL_STRING("cancelled", focusRequestStatusName(FocusRequestStatus::CANCELLED)); + TEST_ASSERT_EQUAL_STRING("timeout", focusRequestStatusName(FocusRequestStatus::TIMEOUT)); + TEST_ASSERT_EQUAL_STRING("failed", focusRequestStatusName(FocusRequestStatus::FAILED)); +} + +void test_focus_csv_with_fix_persists_blank_coverage_and_raw_counts() { + FocusObservation observation; + observation.rx_millis = 555000; + observation.observation_ms = 470; + observation.freq_mhz = 912.75f; + observation.focus_id = 7; + observation.selection_bin_index = 43; + observation.requested_dwell_ms = 500; + observation.requested_samples = 4; + observation.sample_count = 4; + observation.qualifying_count = 0; + observation.rssi_median_dbm_x10 = -920; + observation.rssi_p90_dbm_x10 = -850; + observation.rssi_peak_dbm_x10 = -850; + observation.profile = (uint8_t)MissionProfile::GENERAL_EXPLORATION; + observation.requested_passes = 1; + observation.valid_passes = 1; + observation.selection_source = FocusSelectionSource::SWEEP_BIN; + observation.request_status = FocusRequestStatus::COMPLETE; + observation.home_restore = true; + + char row[256]; + const size_t n = focusObservationFormatCsv(observation, row, sizeof(row), + "2026-09-04T10:00:00Z", true, + 45.5, -122.6, 1, 3); + TEST_ASSERT_TRUE(n > 0); + TEST_ASSERT_EQUAL_STRING( + "2026-09-04T10:00:00Z,45.500000,-122.600000,1,3,555000,general,7,sweep,43," + "912.750,1,1,500,470,4,4,-92.0,-85.0,-85.0,0,,complete,1,0,0", row); +} + +void test_focus_csv_without_fix_or_samples_keeps_unknown_values_blank() { + FocusObservation observation; + observation.rx_millis = 777; + observation.profile = (uint8_t)MissionProfile::RETICULUM; + observation.request_status = FocusRequestStatus::FAILED; + observation.radio_status = -2; + + char row[256]; + TEST_ASSERT_TRUE(focusObservationFormatCsv(observation, row, sizeof(row), "", false, + 0.0, 0.0, 0, 1) > 0); + TEST_ASSERT_NOT_NULL(strstr(row, ",,,0,1,777,reticulum,")); + TEST_ASSERT_NULL(strstr(row, "0.000000")); + TEST_ASSERT_NOT_NULL(strstr(row, ",0,0,,,,0,,failed,")); +} + +void test_focus_csv_truncation_is_reported() { + FocusObservation observation; + char row[8]; + TEST_ASSERT_EQUAL_size_t(0, focusObservationFormatCsv(observation, row, sizeof(row), + "timestamp", false, + 0.0, 0.0, 0, 1)); +} + +void test_worst_case_row_fits_the_logger_buffer_with_margin() { + // Every field at its widest, including widths the bounded request cannot + // actually produce. An over-long row is dropped rather than truncated, so + // this must hold at the maximum, not at today's typical bench values. + FocusObservation observation; + observation.rx_millis = 4294967295u; + observation.observation_ms = 4294967295u; + observation.freq_mhz = 928.1234f; + observation.focus_id = 65535; + observation.selection_bin_index = 65535; + observation.requested_dwell_ms = 65535; + observation.requested_samples = 65535; + observation.sample_count = 65535; + observation.qualifying_count = 65535; + observation.rssi_median_dbm_x10 = -1405; + observation.rssi_p90_dbm_x10 = -1405; + observation.rssi_peak_dbm_x10 = -1405; + observation.radio_status = -32768; + observation.profile = 200; // longest name path: "unknown" + observation.requested_passes = 255; + observation.valid_passes = 255; + observation.selection_source = FocusSelectionSource::WATERFALL_BIN; // "waterfall" + observation.request_status = FocusRequestStatus::CANCELLED; // "cancelled" + observation.home_restore = true; + observation.wifi_on = true; + + char row[FOCUS_CSV_ROW_MAX]; + const size_t n = focusObservationFormatCsv(observation, row, sizeof(row), + "2026-09-04T21:57:36Z", true, + -179.987654, -179.987654, 255, 65535); + TEST_ASSERT_TRUE(n > 0); + // The logger appends a newline at row[n], so n + 1 must still fit. + TEST_ASSERT_TRUE(n + 1 <= FOCUS_CSV_ROW_MAX); + // Measured at 189 bytes; fail loudly if a schema change eats the margin. + TEST_ASSERT_TRUE(n <= 200); +} + +int main(int argc, char **argv) { + UNITY_BEGIN(); + RUN_TEST(test_focus_working_state_stays_below_one_result_budget); + RUN_TEST(test_focus_histogram_reports_median_p90_and_peak_without_raw_samples); + RUN_TEST(test_focus_histogram_rounds_to_one_db_and_flags_clamped_input); + RUN_TEST(test_focus_histogram_refuses_quantiles_after_bucket_saturation); + RUN_TEST(test_focus_request_status_names_are_explicit); + RUN_TEST(test_focus_csv_with_fix_persists_blank_coverage_and_raw_counts); + RUN_TEST(test_focus_csv_without_fix_or_samples_keeps_unknown_values_blank); + RUN_TEST(test_focus_csv_truncation_is_reported); + RUN_TEST(test_worst_case_row_fits_the_logger_buffer_with_margin); + return UNITY_END(); +} diff --git a/test/test_focus_plan/test_main.cpp b/test/test_focus_plan/test_main.cpp new file mode 100644 index 0000000..602d76e --- /dev/null +++ b/test/test_focus_plan/test_main.cpp @@ -0,0 +1,83 @@ +#include + +#include "../../src/focus_plan.h" + +void test_focus_request_is_small_and_starts_with_one_selected_bin() { + TEST_ASSERT_EQUAL_UINT8(1, FOCUS_SELECTED_BIN_COUNT); + TEST_ASSERT_TRUE(sizeof(FocusRequest) <= 16); +} + +void test_focus_selection_sources_are_explicit_and_stable() { + TEST_ASSERT_EQUAL_STRING("sweep", focusSelectionSourceName(FocusSelectionSource::SWEEP_BIN)); + TEST_ASSERT_EQUAL_STRING("waterfall", focusSelectionSourceName(FocusSelectionSource::WATERFALL_BIN)); + TEST_ASSERT_EQUAL_STRING("preset", focusSelectionSourceName(FocusSelectionSource::PRESET)); +} + +void test_focus_request_resolves_a_sourced_energy_bin_frequency() { + FocusRequest request; + request.region = Region::US; + request.bin_step = EnergyBinStep::KHZ_250; + request.selection_bin_index = 13; + request.requested_dwell_ms = 100; + request.requested_samples = 4; + request.requested_passes = FOCUS_BENCH_REQUESTED_PASSES; + request.selection_source = FocusSelectionSource::SWEEP_BIN; + + TEST_ASSERT_TRUE(focusRequestIsValid(request)); + TEST_ASSERT_FLOAT_WITHIN(0.001f, 905.25f, focusRequestFrequencyMhz(request)); +} + +void test_focus_request_rejects_empty_or_out_of_band_work() { + FocusRequest request; + request.region = Region::US; + request.bin_step = EnergyBinStep::KHZ_250; + request.selection_bin_index = energyBinCount(ENERGY_SWEEP_BAND_US, request.bin_step); + request.requested_dwell_ms = 100; + request.requested_samples = FOCUS_BENCH_SAMPLES_MIN; + request.requested_passes = FOCUS_BENCH_REQUESTED_PASSES; + TEST_ASSERT_FALSE(focusRequestIsValid(request)); + + request.selection_bin_index = 0; + request.requested_dwell_ms = FOCUS_BENCH_DWELL_MIN_MS - 1; + TEST_ASSERT_FALSE(focusRequestIsValid(request)); + request.requested_dwell_ms = FOCUS_BENCH_DWELL_MAX_MS + 1; + TEST_ASSERT_FALSE(focusRequestIsValid(request)); + request.requested_dwell_ms = 100; + request.requested_samples = FOCUS_BENCH_SAMPLES_MIN - 1; + TEST_ASSERT_FALSE(focusRequestIsValid(request)); + request.requested_samples = FOCUS_BENCH_SAMPLES_MIN; + request.requested_passes = 0; + TEST_ASSERT_FALSE(focusRequestIsValid(request)); +} + +void test_focus_request_is_bounded_in_time_not_only_in_samples() { + FocusRequest request; + request.region = Region::US; + request.bin_step = EnergyBinStep::KHZ_250; + request.selection_bin_index = 43; + request.requested_dwell_ms = 500; + request.requested_samples = 8; + request.requested_passes = FOCUS_BENCH_REQUESTED_PASSES; + request.selection_source = FocusSelectionSource::SWEEP_BIN; + + TEST_ASSERT_TRUE(focusRequestIsValid(request)); + // The deadline must exceed the dwell it bounds, or a healthy request + // would report `timeout` instead of what it actually observed. + TEST_ASSERT_TRUE(focusRequestTimeoutMs(request) > request.requested_dwell_ms); + TEST_ASSERT_EQUAL_UINT32(500u + FOCUS_REQUEST_TIMEOUT_SLACK_MS, + focusRequestTimeoutMs(request)); + + request.requested_dwell_ms = FOCUS_BENCH_DWELL_MAX_MS; + TEST_ASSERT_EQUAL_UINT32(FOCUS_MAX_SAMPLING_MS, focusRequestTimeoutMs(request)); + TEST_ASSERT_TRUE(FOCUS_MAX_SAMPLING_MS <= 3000); +} + +int main(int argc, char **argv) { + UNITY_BEGIN(); + RUN_TEST(test_focus_request_is_small_and_starts_with_one_selected_bin); + RUN_TEST(test_focus_selection_sources_are_explicit_and_stable); + RUN_TEST(test_focus_request_resolves_a_sourced_energy_bin_frequency); + RUN_TEST(test_focus_request_rejects_empty_or_out_of_band_work); + RUN_TEST(test_focus_request_is_bounded_in_time_not_only_in_samples); + return UNITY_END(); +} diff --git a/test/test_focus_runtime/test_main.cpp b/test/test_focus_runtime/test_main.cpp new file mode 100644 index 0000000..c0f5809 --- /dev/null +++ b/test/test_focus_runtime/test_main.cpp @@ -0,0 +1,74 @@ +#include + +#include "../../src/focus_runtime.h" + +FocusRequest validRequest() { + FocusRequest request; + request.region = Region::US; + request.selection_bin_index = 13; + request.requested_dwell_ms = 100; + request.requested_samples = 4; + request.requested_passes = FOCUS_BENCH_REQUESTED_PASSES; + return request; +} + +void test_focus_runtime_only_starts_a_bounded_valid_request() { + FocusRuntime runtime; + TEST_ASSERT_TRUE(focusRuntimeBegin(runtime, validRequest())); + TEST_ASSERT_EQUAL((int)FocusRuntimeState::SURVEYING, (int)runtime.state); + TEST_ASSERT_FALSE(focusRuntimeBegin(runtime, validRequest())); +} + +void test_focus_runtime_publishes_complete_only_after_restore() { + FocusRuntime runtime; + TEST_ASSERT_TRUE(focusRuntimeBegin(runtime, validRequest())); + focusRuntimeNoteValidPass(runtime, 93); + focusRuntimeBeginRestore(runtime); + TEST_ASSERT_EQUAL((int)FocusRequestStatus::COMPLETE, + (int)focusRuntimeFinishRestore(runtime, true)); + TEST_ASSERT_EQUAL_UINT16(1, runtime.valid_passes); + TEST_ASSERT_EQUAL_UINT32(93, runtime.observation_ms); +} + +void test_focus_runtime_restore_failure_overrides_cancel_and_timeout() { + FocusRuntime runtime; + TEST_ASSERT_TRUE(focusRuntimeBegin(runtime, validRequest())); + focusRuntimeCancel(runtime); + focusRuntimeTimeout(runtime); + focusRuntimeBeginRestore(runtime); + TEST_ASSERT_EQUAL((int)FocusRequestStatus::FAILED, + (int)focusRuntimeFinishRestore(runtime, false)); +} + +void test_focus_runtime_timeout_publishes_timeout_not_complete() { + FocusRuntime runtime; + TEST_ASSERT_TRUE(focusRuntimeBegin(runtime, validRequest())); + focusRuntimeTimeout(runtime); + focusRuntimeBeginRestore(runtime); + // A recovered timeout is still a timeout: it must not borrow `complete` + // from a restore that worked, and it recorded no valid pass. + TEST_ASSERT_EQUAL((int)FocusRequestStatus::TIMEOUT, + (int)focusRuntimeFinishRestore(runtime, true)); + TEST_ASSERT_EQUAL_UINT16(0, runtime.valid_passes); + TEST_ASSERT_EQUAL_UINT32(0, runtime.observation_ms); +} + +void test_focus_runtime_cancel_outranks_a_concurrent_timeout() { + FocusRuntime runtime; + TEST_ASSERT_TRUE(focusRuntimeBegin(runtime, validRequest())); + focusRuntimeCancel(runtime); + focusRuntimeTimeout(runtime); + focusRuntimeBeginRestore(runtime); + TEST_ASSERT_EQUAL((int)FocusRequestStatus::CANCELLED, + (int)focusRuntimeFinishRestore(runtime, true)); +} + +int main(int argc, char **argv) { + UNITY_BEGIN(); + RUN_TEST(test_focus_runtime_only_starts_a_bounded_valid_request); + RUN_TEST(test_focus_runtime_publishes_complete_only_after_restore); + RUN_TEST(test_focus_runtime_restore_failure_overrides_cancel_and_timeout); + RUN_TEST(test_focus_runtime_timeout_publishes_timeout_not_complete); + RUN_TEST(test_focus_runtime_cancel_outranks_a_concurrent_timeout); + return UNITY_END(); +} diff --git a/test/test_serial_control_protocol/test_main.cpp b/test/test_serial_control_protocol/test_main.cpp index 8726134..ab30dd3 100644 --- a/test/test_serial_control_protocol/test_main.cpp +++ b/test/test_serial_control_protocol/test_main.cpp @@ -98,6 +98,78 @@ void test_bench_pass_b_cad_opcode_round_trips() { TEST_ASSERT_EQUAL_STRING("7", frame.argument); } +void test_bench_focus_opcode_round_trips() { + char line[SERIAL_CONTROL_FRAME_MAX]; + TEST_ASSERT_TRUE(serialControlFormatFrame(line, sizeof(line), 24, + SerialControlOpcode::BENCH_FOCUS, + "43:500:8") > 0); + SerialControlFrame frame; + TEST_ASSERT_TRUE(serialControlParseFrame(line, frame)); + TEST_ASSERT_EQUAL_INT((int)SerialControlOpcode::BENCH_FOCUS, (int)frame.opcode); + TEST_ASSERT_EQUAL_STRING("43:500:8", frame.argument); +} + +void test_bench_focus_result_opcode_round_trips() { + char line[SERIAL_CONTROL_FRAME_MAX]; + TEST_ASSERT_TRUE(serialControlFormatFrame(line, sizeof(line), 25, + SerialControlOpcode::BENCH_FOCUS_RESULT, + "-") > 0); + SerialControlFrame frame; + TEST_ASSERT_TRUE(serialControlParseFrame(line, frame)); + TEST_ASSERT_EQUAL_INT((int)SerialControlOpcode::BENCH_FOCUS_RESULT, (int)frame.opcode); + TEST_ASSERT_EQUAL_STRING("-", frame.argument); +} + +void test_bench_focus_cancel_opcode_round_trips() { + char line[SERIAL_CONTROL_FRAME_MAX]; + TEST_ASSERT_TRUE(serialControlFormatFrame(line, sizeof(line), 26, + SerialControlOpcode::BENCH_FOCUS_CANCEL, + "-") > 0); + SerialControlFrame frame; + TEST_ASSERT_TRUE(serialControlParseFrame(line, frame)); + TEST_ASSERT_EQUAL_INT((int)SerialControlOpcode::BENCH_FOCUS_CANCEL, (int)frame.opcode); + TEST_ASSERT_EQUAL_STRING("-", frame.argument); +} + +void test_bench_action_opcode_round_trips() { + char line[SERIAL_CONTROL_FRAME_MAX]; + TEST_ASSERT_TRUE(serialControlFormatFrame(line, sizeof(line), 27, + SerialControlOpcode::BENCH_ACTION, + "CELL:START") > 0); + SerialControlFrame frame; + TEST_ASSERT_TRUE(serialControlParseFrame(line, frame)); + TEST_ASSERT_EQUAL_INT((int)SerialControlOpcode::BENCH_ACTION, (int)frame.opcode); + TEST_ASSERT_EQUAL_STRING("CELL:START", frame.argument); +} + +void test_argument_budget_covers_the_longest_opcode_name() { + // SERIAL_CONTROL_ARGUMENT_MAX is derived from this; a longer opcode name + // added later would quietly overrun every buffer sized from it. + for (int value = (int)SerialControlOpcode::HELLO; value <= (int)SerialControlOpcode::ERROR; + ++value) { + const char *name = serialControlOpcodeName((SerialControlOpcode)value); + TEST_ASSERT_TRUE(strlen(name) <= SERIAL_CONTROL_OPCODE_NAME_MAX); + } +} + +void test_formatter_frames_a_saturated_status_argument() { + // STATUS is the longest frame the device emits, and an over-long one is + // dropped silently rather than truncated -- so the budget has to hold at + // the maximum, not at today's typical field widths. + char argument[SERIAL_CONTROL_ARGUMENT_MAX + 2]; + memset(argument, 'x', sizeof(argument) - 1); + argument[sizeof(argument) - 1] = '\0'; + + char line[SERIAL_CONTROL_FRAME_MAX]; + TEST_ASSERT_EQUAL_UINT(0, serialControlFormatFrame(line, sizeof(line), 65535, + SerialControlOpcode::BENCH_PASS_B_CAD_RESULT, + argument)); + argument[SERIAL_CONTROL_ARGUMENT_MAX] = '\0'; + TEST_ASSERT_TRUE(serialControlFormatFrame(line, sizeof(line), 65535, + SerialControlOpcode::BENCH_PASS_B_CAD_RESULT, + argument) > 0); +} + int main(int argc, char **argv) { UNITY_BEGIN(); RUN_TEST(test_valid_frame_round_trips); @@ -109,5 +181,11 @@ int main(int argc, char **argv) { RUN_TEST(test_sweep_opcodes_round_trip); RUN_TEST(test_sd_retry_opcode_round_trips); RUN_TEST(test_bench_pass_b_cad_opcode_round_trips); + RUN_TEST(test_bench_focus_opcode_round_trips); + RUN_TEST(test_bench_focus_result_opcode_round_trips); + RUN_TEST(test_bench_focus_cancel_opcode_round_trips); + RUN_TEST(test_bench_action_opcode_round_trips); + RUN_TEST(test_argument_budget_covers_the_longest_opcode_name); + RUN_TEST(test_formatter_frames_a_saturated_status_argument); return UNITY_END(); } From 2c2577f5a4252ac6791f50c78c3317649fb8c2a6 Mon Sep 17 00:00:00 2001 From: Will Shields Date: Fri, 4 Sep 2026 17:03:13 -0700 Subject: [PATCH 03/21] Replace Phase 12's stale implementation order with ordered next steps MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The closing paragraph still described the radio request/recovery path and bench harness as upcoming work; both shipped. It also predated the matrix, so it implied thresholds could be selected next -- but §6.4 showed the sampling design has to change first, and any threshold or radio-away budget chosen before that change would be measured against sampling the product will not ship. Co-Authored-By: Claude Opus 5 --- docs/research/phase12-survey-truth-design.md | 43 ++++++++++++++++---- 1 file changed, 35 insertions(+), 8 deletions(-) diff --git a/docs/research/phase12-survey-truth-design.md b/docs/research/phase12-survey-truth-design.md index 02f30a3..2f4599d 100644 --- a/docs/research/phase12-survey-truth-design.md +++ b/docs/research/phase12-survey-truth-design.md @@ -436,14 +436,41 @@ local spectrum identity or coverage of the city. - [ ] WiFi-off/on resource matrix, Portland field validation, `STATUS.md`, `LOG_GUIDE.md`, release notes, and any companion-schema update reconcile. -Implementation proceeds in two bounded parts. The Engineering foundation now -has pure plan/statistics/CSV tests. The bench request is one pass, 2--2,000 ms -and 2--64 RSSI samples; the matrix gets 30 trials from 30 durable requests, -not one long radio-away loop. Its next slice adds the radio request/recovery -path, logger/health integration, and the bench harness with no coverage label. -Only after §6 selects the thresholds and Watch budget may the operator menu, -Activity surface, durable final schema, and field validation enter the -Device/claim gates. +### Next, in order + +The Engineering gate is closed and the bench slice is hardware-proven, so what +remains is ordered by what blocks what. **The sampling change comes first: it +invalidates any threshold or budget chosen before it.** + +1. **Scale the sample budget with the dwell** (§6.4 finding 1). Today's fixed + 8 samples mean a long dwell observes a handful of instants, so a dwell or + observation-time figure does not mean what an operator would read it to + mean. Decide what a pass should sample — a target spacing, a rate, or a + dwell-proportional count — inside the existing static-RAM budget (§4.2 has + 66 B of row margin and 68 B under the working-state target). Re-run the + affected §6.2 arms afterwards; the shorter-airtime positions are the ones + that move. +2. **Re-select the qualifying RSSI condition** against the new sampling, and + at a source closer to realistic levels than this bench's ~-26 dBm. §6.4's + `p90 >= -90 dBm` is a candidate measured under the old sampling and a very + strong source; it should not survive into the product unexamined. +3. **Run §6.3's Watch-opportunity comparison** + (`scripts/phase12_watch_opportunity.py`) and approve, or refuse, a maximum + radio-away budget. Measured away time is already known (dwell + ~73 ms); + what is unknown is what repeated requests cost Watch. +4. **Select the coverage thresholds.** These are about valid-pass counts and + accumulated observation time across repeated requests, which no + single-pass matrix can supply — it needs its own repeated-request + measurement, once 1--3 have settled what a pass is worth. +5. Only then: the operator menu control, the Activity/status surface, the + durable final schema with a populated `coverage`, `LOG_GUIDE.md`'s + operator-facing `focus.csv` section, and Portland field validation. Until + an operator control exists, `focus.csv` is deliberately absent from the + operator log guide — documenting a file nobody can produce would be worse + than omitting it. + +A release is further out than the closed gates suggest: Focus is reachable +only from the bench image, and step 1 is a design change, not a tuning pass. ## Sources From 781221c2411a5d2a721be8de577a8c4d6150b204 Mon Sep 17 00:00:00 2001 From: Will Shields Date: Fri, 4 Sep 2026 17:10:43 -0700 Subject: [PATCH 04/21] UI Mockups --- docs/UI-Design_Guide.html | 3136 ++++++++++++++++++++++++++++++++++ docs/UI-Recommendations.html | 1404 +++++++++++++++ 2 files changed, 4540 insertions(+) create mode 100644 docs/UI-Design_Guide.html create mode 100644 docs/UI-Recommendations.html diff --git a/docs/UI-Design_Guide.html b/docs/UI-Design_Guide.html new file mode 100644 index 0000000..404d68a --- /dev/null +++ b/docs/UI-Design_Guide.html @@ -0,0 +1,3136 @@ + + + + + +LoRaTrace RX · Master UI Design Guide & Workbench + + + + + + + + +
+ + + + + +
+
+ ★ Baseline Spec (v1.0.7 Master Guide) + → Proposed UI Improvements & Labs (New) +
+ +
Firmware v1.0.7 Baseline · ST7789V2 240×135
+

Master UI Design Guide

+ + +
+ Panel: ST7789V2 240×135 SPI + Color: 16-bit RGB565 (6 tokens) + Core: Core 0 (UI/Logging/GPS) + Radio: Core 1 (SX1262, never blocked) + Heap: 512KB SRAM (No PSRAM) + Scale: 2.2× Canvas / 1:1 Target +
+ + +
+
+
+ Real-Time Simulation +

Live Hardware Workbench

+
+ CURRENT: RADIO (1/5) +
+ +
+
+
+ +
+
+ Simulated ST7789V2 Display (2.2× scale). Drive with hardware keys: , (prev), / (next), ; (up), . (down), Enter (select), ` (back), 1–5, P, S, C, 8. +
+ + +
+
+
Simulation & Loops: RADIO
+ LOOP: IDLE +
+
+ +
+
Simulates real-time hardware telemetry and background animation loops.
+
+
+ +
+ +
+

Hardware Keys TCA8418

+
+ + + + + + +
+
Prev/Next cycle main carousel (1/5 to 5/5)
+
+ + +
+

Main Carousel (5 Pages) v1.0.7 Order

+
+ + + + + +
+
+ + +
+

Tools Hub Menu or P / S / C

+
+ + + +
+
+ + +
+

Analyze Suite Menu or Key 8

+
+ + + + + +
+
+ + +
+

Menu & Overlays Root / Slider / Toast

+
+ + + + +
+
+ + +
+

Preset Scenarios Global State

+
+ + + +
+
+ +
+ State: RADIO · Carousel Pos: 1/5 · Active Bounded Action: NONE +
+
+
+
+ + +
+
+
+ 01 / ARCHITECTURE +

Hardware Display Architecture

+
+

+ LoRaTrace RX executes on the M5Cardputer-Adv (ESP32-S3FN8). The display is an ST7789V2 driven over dedicated SPI. The system is strictly budget-constrained: 512KB SRAM with zero external PSRAM. +

+ +
+
+

Dedicated Core 0 UI Thread

+

The UI task runs on Core 0 and redraws at ~10-20Hz. The SX1262 radio task runs strictly pinned to Core 1 and never blocks on display SPI, SD logging, or keyboard I/O.

+
+
+

Zero-Allocation Rendering

+

Every page draw function (drawRadioPage, drawActivityPage, etc.) executes with zero dynamic heap allocations. Memory is pre-allocated or statically budgeted in task stacks.

+
+
+

No Full-Screen Blanking

+

drawPage() wipes only the content canvas (HEADER_H to FOOTER_Y) and performs direct overwriting. Full fillScreen() calls are banned during page transitions to eliminate visible black flashes.

+
+
+

Fixed-Pitch Bitmaps

+

Uses Adafruit-GFX / M5GFX built-in bitmap fonts: Size 1 is 6×8px (up to 40×16 chars); Size 2 is 12×16px (up to 20×8 chars). All positions are integer-aligned.

+
+
+
+ + +
+
+
+ 02 / COLOR TOKENS +

True RGB565 Hardware Palette

+
+

+ The ST7789V2 renders in 16-bit RGB565. Only six functional color tokens exist in production firmware (src/ui_pages.cpp). No arbitrary gradients or strength-tier rainbows are used. +

+ +
+
+
+
+
COL_BG
+
0x0000 · #000000
+
Primary screen background, inverted menu text.
+
+
+ +
+
+
+
COL_FG
+
0xFFFF · #FFFFFF
+
Primary text, headline numbers, active menu selection.
+
+
+ +
+
+
+
COL_DIM
+
0xBDF7 · #BDBEBD (~75% Grey)
+
Labels, secondary values, frequency tracks, quiet bins.
+
+
+ +
+
+
+
COL_GOOD
+
0x07E0 · #00FF00
+
Active GPS 3D fix, zero drops, battery OK, completed scans.
+
+
+ +
+
+
+
COL_WARN
+
0xFFE0 · #FFFF00
+
Standby, scanning, paused watch, repeat mode, peaks.
+
+
+ +
+
+
+
COL_BAD
+
0xF800 · #FF0000
+
Queue drops, SD unmounted, GPS checksum errors, failures.
+
+
+
+
+ + +
+
+
+ 03 / GEOMETRY +

Screen Layout & Character Grid

+
+

+ Display area is strictly 240×135 pixels. The layout is divided into three fixed horizontal zones: +

+ +
+
+

Header Chrome: y=0..12

+

Height 12px. Contains breadcrumb / page title (left, max 25 chars size 1), status dots (x=155..177), and battery icon with % (x=214..238). Border hairline at y=12.

+
+
+

Content Canvas: y=13..124

+

Height 112px. Where all page-specific UI widgets, tables, graphs, and statistics are drawn. Standard left column starts at x=2; right column aligns at x=170.

+
+
+

Footer Bar: y=125..135

+

Height 10px. Anchors active profile name at x=2 (e.g. Meshtastic) and carousel position at right (e.g. [1/5]). Replaced during menu display.

+
+
+
+ + +
+
+
+ 04 / CHROME +

Header Chrome & Battery Anatomy

+
+

+ Drawn on every page by drawHeader() and drawBattery(). The 25-character breadcrumb limit guarantees that deep menu paths never collide with the status dots cluster. +

+ +
+
+
+
RADIO
+
+
+
+
82%
+
+
+
+
+ +
Breadcrumb safe limit: 25 chars size 1 (≤150px)
+
Dots: [SD log] [GPS fix] [WiFi AP] at x=155, 166, 175
+
+ +
+ + + + + + + + + + + +
ElementCoords (1×)Size & ColorSource Data
Page Breadcrumb(2, 2)Size 1 · COL_FGpageName(page) / Menu path
Status Dots(157, 6), (166, 6), (175, 6)r=2px · G/Y/RSD state, GPS fix, WiFi active
Battery Pct(x_batt-2, 2) right-alignedSize 1 · COL_GOODbatteryPercent()
Battery Glyph(214, 2) 22×9pxRect + Nub · COL_GOODbatteryMilliVolts()
Header Hairline(0, 12) w=2401px Line · COL_DIMFixed horizontal divider
+
+
+
+ + +
+
+
+ 05 / MAIN CAROUSEL 1/5 +

Plate: RADIO (Primary Wardriving)

+
+

+ The default home screen. Shows real-time packet ingress, SD logging volume, queue drop count, and radio integrity stats. If watch is paused, STANDBY renders in bold yellow. +

+ +
+
+
RADIO
+
84%
+
+ +
rx 142
+
log 142
+
drop 0
+ +
crc
+
0
+ +
miss
+
0
+ +
sd
+
ok
+ +
run
+
r12
+ +
flush 18 max 42ms
+ +
Meshtastic
+
1/5
+
+ +
+ + + + + + + + + + +
MetricCoordsSize & TokenBehavior
rx counter(2, 18)Size 2 · COL_FGPackets decoded from SX1262
log counter(2, 38)Size 2 · COL_FGRows written to SD card
drop counter(2, 58)Size 2 · GOOD / BADGreen if 0, Red if >0
STANDBY indicator(2, 78)Size 2 · COL_WARNShows when radio is paused/active away
crc / miss / sd / runx=170 right colSize 1 statBlocksRadio CRC fails, SPI bus miss, run dir
flush / max ms(2, 106)Size 1 · COL_DIMSD batch write latency diagnostics
+
+
+
+ + +
+
+
+ 06 / MAIN CAROUSEL 2/5 +

Plate: ACTIVITY (Background Action Mirror)

+
+

+ Added in v1.0.7 (slot 2). A dedicated, read-only mirror of running bounded actions (Probe, Sweep, Cell, Scope). When idle, shows a compact summary of the last result for all four tools without redundant hero clutter. +

+ +
+
+
ACTIVITY
+
84%
+
+ +
Probe
+
1/8 hits
+ +
Sweep
+
3pk 915MHz
+ +
Cell
+
881MHz -84dB
+ +
Scope
+
-96dBm
+ +
Meshtastic
+
2/5
+
+ +
+ + + + + + + + + + +
State / RowCoords (1×)Font & ColorDescription
Active Mode (Hero)(2, 20)Size 2 · COL_WARN"PROBE", "SWEEP", "CELL", or "SCOPE"
Active Progress Bar(2, 66) 200×14Rect fill · WARNCandidate progress or frequency bar
Idle Row 1: Probe(0, 22)Size 2 · FG/DIMShows "IDLE" or "N/M hits"
Idle Row 2: Sweep(0, 46)Size 2 · FG/DIMShows "IDLE", "Npk FreqMHz", or "none found"
Idle Row 3: Cell(0, 70)Size 2 · FG/DIMShows "IDLE", "FreqMHz dBm", or "none found"
Idle Row 4: Scope(0, 94)Size 2 · FG/DIMShows "IDLE" or latest sampled "NdBm"
+
+
+
+ + +
+
+
+ 07 / MAIN CAROUSEL 3/5 +

Plate: CHANNEL (RF Parameters)

+
+

+ Confirms actual hardware tuning: frequency, Spreading Factor (SF), Bandwidth (BW), Coding Rate (CR), and LoRa Sync Word. The frequency bar shows exact position within the tuned 868–923MHz front-end. +

+ +
+
+
CHANNEL
+
83%
+
+ +
906.875 MHz
+
SF11 BW250.0
+ +
+
+
868
+
923
+ +
CR4/5 sync 0x2B
+ +
mode
+
Watch
+
airtime
+
~420ms
+ +
Meshtastic
+
3/5
+
+ +
+ + + + + + + + + +
FieldCoordsFontRemarks
Frequency(2, 18)Size 2 · COL_FGTuned center frequency in MHz
Modulation(2, 44)Size 2 · COL_FGSF (7-12) and BW (kHz)
Frequency Bar(2, 74) w=108Track + MarkerMarker is COL_GOOD, bounds 868..923MHz
Coding & Sync(2, 106)Size 1 · COL_DIMCoding rate (e.g. CR4/5) and hex sync word
Mode & Airtimex=170 right colSize 1 statBlocksRadio task mode ("Watch") and packet ToA
+
+
+
+ + +
+
+
+ 08 / MAIN CAROUSEL 4/5 +

Plate: GPS (Fix & Constellation)

+
+

+ Real-time GPS telemetry parsed from NMEA sentences. Renders 3D/2D fix status, 5-decimal coordinates, altitude, satellites used/in-view, and talker constellation signal strength bars. +

+ +
+
+
GPS
+
83%
+
+ +
3D FIX
+
37.77490
+
-122.41940
+ +
2026-09-04 23:14:02Z
+ +
sats
+
11
+
qual
+
1
+ +
+
GP
+
GL
+
GA
+
+ +
Meshtastic
+
4/5
+
+ +
+ + + + + + + + + +
ElementCoordsColor TokenBehavior
Fix Type(2, 18)GOOD / WARN / BAD"3D FIX" (Green), "NO FIX" (Yellow/Red)
Coordinates(2, 40), (2, 60)Size 2 · COL_FGLatitude & Longitude formatted to 5 decimals
Timestamp(2, 84)Size 1 · COL_DIMUTC ISO timestamp from GPS NMEA sentence
Sats & Qualx=170 right colSize 1 statBlocksSatellites used in solution and fix quality
Talker Bars(170, 70) 4 barsRects · COL_GOODGP (GPS), GL (GLONASS), GA (Galileo), BD (BeiDou)
+
+
+
+ + +
+
+
+ 09 / MAIN CAROUSEL 5/5 +

Plate: SYSTEM (Heap & Diagnostics)

+
+

+ Critical system telemetry: device uptime, free internal heap (against the 512KB SRAM budget), minimum recorded free heap, battery voltage, SPI bus contention counter, WiFi AP clients, and firmware version. +

+ +
+
+
SYSTEM
+
83%
+
+ +
48 min
+
312k heap
+ +
+
+
+ +
min heap
+
298k
+ +
batt
+
3.98V
+ +
bus
+
0
+ +
wifi
+
OFF
+ +
keys tca8418 health 96
+
v1.0.7 (prod-release)
+ +
Meshtastic
+
5/5
+
+ +
+ + + + + + + + + + +
MetricCoordsTokenRules
Uptime(2, 18)Size 2 · COL_FGElapsed minutes since power on
Free Heap(2, 40)GOOD / WARN / BAD3-tier: Green <80%, Yellow 80-90%, Red >90%
Heap Fill Bar(2, 60) 108×8Rect · Match HeapFills proportionally to SRAM consumed (512KB base)
Min Heap / Battx=136 / x=205Size 1 statBlocksLowest watermark and raw battery voltage
Bus / WiFix=136 / x=205Size 1 statBlocksSPI bus contention and WiFi AP status
Key / Log / Version(2, 78), (2, 90)Size 1 · COL_DIMTCA8418 driver check, session log rows, build tag
+
+
+
+ + +
+
+
+ 10 / TOOLS HUB +

Plate: PROBE (Discovery CAD Sweep)

+
+

+ Executes fast Preamble Channel Activity Detection (CAD) across source-backed profile tuples. Scans without decoding full packets, providing rapid presence detection while keeping radio away-time bounded. +

+ +
+
+
PROBE
+
83%
+
+ +
COMPLETE
+
candidates 8/8 away 124ms
+ +
US LongFast, EU Narrow
+
hits 2 free 6 timeout 0 err 0
+ +
Meshtastic
+
+ +
+ + + + + + + + +
FieldCoordsTokenNotes
Headline Status(2, 20)Size 2 · Status"SCANNING" (Warn), "COMPLETE" (Good), "IDLE" (Dim)
Candidate Progress(2, 43)Size 1 · COL_FGCurrent/Total candidate count + away-time ms
Hit Labels(2, 62)Size 1 · COL_FGHuman-readable names of profiles detected
CAD Stats(2, 96)Size 1 · DIM / BADHits, free channels, timeouts, and hardware errors
+
+
+
+ + +
+
+
+ 11 / TOOLS HUB +

Plate: SWEEP (RF Energy & Pass-B)

+
+

+ Two-pass wideband energy acquisition. Pass-A takes RSSI samples across 85 frequency bins (902–923MHz in US). Bins exceeding threshold trigger Pass-B selective CAD. Supports ambient background repeat mode. +

+ +
+
+
SWEEP
+
83%
+
+ +
REPEATING
+
bins 85/85 away 340ms
+ +
+
+
902
+
923
+ +
listening to the noise
+ +
peaks
+
4
+
best MHz
+
915.0
+
rssi
+
-88dB
+
lap
+
14
+ +
Meshtastic
+
+ +
+ + + + + + + + + +
ElementCoordsTokenNotes
Headline(2, 20)Size 2 · Status"SCANNING", "REPEATING", "COMPLETE", "IDLE"
Bin Progress(2, 43)Size 1 · COL_FGBins swept and radio away-time
Frequency Bar(2, 74) w=108Track + MarkerMarker shows tuned bin position across 902..923MHz
Peaks & Best MHzx=170 right colSize 1 statBlocksCount of energy peaks and strongest frequency
Lap Counter(170, 102)Size 1 · COL_WARNShows total repeat cycles completed
+
+
+
+ + +
+
+
+ 12 / TOOLS HUB (PHASE 11) +

Plate: CELL (Cellular Band Energy)

+
+

+ Phase 11 cellular uplink band scanner (869–894MHz, 101 bins). Displays FCC Block A/B partitions (47 CFR § 22.905). Strictly measures RF presence without decoding or tower tracking. +

+ +
+
+
CELL
+
83%
+
+ +
COMPLETE
+
bins 101/101
+ +
+
+
869
+
894
+ +
+
+
+
+
+
A
+
B
+ +
best MHz
+
881.5
+
rssi
+
-76dB
+ +
Meshtastic
+
+ +
+ + + + + + + + +
WidgetCoordsTokenRules
Frequency Bounds(2, 74) w=108869..894MHzScoped tightly to cellular allocation
FCC Blocks(2, 92)DIM / FGBlock A (Dim) and Block B (White); letter labels
Strongest Signalx=170 right colSize 1 statBlocksStrongest detected frequency and peak dBm
Disclaimer(2, 108)Size 1 · COL_DIM"presence only — no cell ID / decode"
+
+
+
+ + +
+
+
+ 13 / ANALYZE SUITE +

Plate: METER (Real-Time Signal Level)

+
+

+ Dedicated signal strength meter. Features an honest filled bar gauge (-120 to 0 dBm), signal SNR readout (watch-sourced), and live channel modulation parameters (SF/BW/CR) on the right. +

+ +
+
+
METER
+
82%
+
+ +
-72 dBm
+
watch @ 906.875MHz
+
SNR: +8.5dB
+
2s ago
+ +
SF11
+
250.0kHz
+
CR4/5
+ +
+
+
+
-120
+
0
+ +
Meshtastic
+
+ +
+ + + + + + + + + +
ElementCoords (1×)TokenDesign Rationale
Signal dBm(2, 20)Size 2 · COL_FGPeak or packet RSSI in dBm
SNR Readout(2, 54)Size 1 · COL_DIMDecoded packet Signal-to-Noise Ratio (dB)
Elapsed Age(2, 66)FG / WARN"live - watch paused" (Warn) or "Ns ago"
Channel Paramsx=230 right-alignedSize 1 · COL_DIMSF, Bandwidth kHz, and Coding Rate denominator
Bar Gauge(2, 84) 232×12Border Dim, Fill FG-120 to 0 dBm (widened from -30 to avoid clipping)
+
+
+
+ + +
+
+
+ 14 / ANALYZE SUITE +

Plate: WATERFALL (Historical Spectrogram)

+
+

+ Stores up to 24 rows of historical frequency bin sweeps. Features a labeled MHz frequency axis along the bottom border, top-row SCANNING badge, and Enter-to-repeat sweep control. +

+ +
+
+
WATERFALL
+
82%
+
+ +
12 HITS
+
SCANNING
+
24 rows, newest first 85 bins
+ +
+
+
+
+
+
+
+
+ +
902.0
+
912.5
+
923.0
+ +
Meshtastic
+
+ +
+ + + + + + + + + +
ComponentGeometryColorDetails
Hit Summary(2, 18)Size 2 · WARN / DIMTotal hits across buffer or "QUIET"
Scanning Badgex=234 right-alignedSize 1 · COL_WARNRenders when repeat Sweep is running
Spectrogram Plot(2, 46) 232×60Box Border DimUp to 24 rows @ 4px height; hits in COL_WARN
Frequency Axisy=106 (border)Ticks + Labels902.0, 912.5, 923.0 MHz (plus 20/40/60/80% ticks)
Enter Key ActionInteractiveToggle RepeatStarts/stops repeat Sweep without leaving page
+
+
+
+ + +
+
+
+ 15 / ANALYZE SUITE +

Plate: SCOPE (Time-Domain RSSI Trace)

+
+

+ Fixed-frequency RF power oscilloscope. Samples raw RSSI at up to 50Hz (20ms intervals) into a 240-sample trace buffer. Visualizes packet preamble energy envelopes. +

+ +
+
+
SCOPE
+
82%
+
+ +
CAPTURED
+
906.875MHz 20ms/sample
+ +
+ + + +
+
-30
+
-120
+ +
Meshtastic
+
+ +
+ + + + + + + + +
FieldCoordsTokenRules
Headline Word(2, 18)GOOD / WARN / DIM"CAPTURING", "CAPTURED", or dim "IDLE"
Trace Metadata(2, 36)Size 1 · COL_FGTuned frequency and sample interval
Trace Plot Area(2, 60) 232×50COL_GOOD line240 samples scaled between -120 and -30 dBm
Hold ReversionRESULT_HOLD_MSTimerReverts from CAPTURED to IDLE after 8 seconds
+
+
+
+ + +
+
+
+ 16 / ANALYZE SUITE +

Plate: CAPTURES (Packet Ring Buffer)

+
+

+ Maintains an in-memory circular ring buffer of the 8 most recent decoded LoRa packet headers. Flags off-grid channels and displays node hex addresses, RSSI, and elapsed age. +

+ +
+
+
CAPTURES
+
82%
+
+ +
meshtastic !a4b892f0 906.9MHz -68dBm 2s
+
meshtastic !38c11e04 910.5MHz -94dBm 14s
+
meshcore 906.9MHz -82dBm 28s
+
meshtastic !f102bb88 906.9MHz -75dBm 42s
+
meshtastic !12aa045e 906.9MHz -89dBm 65s
+
meshtastic !88319eac 906.9MHz -102dBm 120s
+ +
Meshtastic
+
+ +
+ + + + + + + + +
PropertyValueFormatDescription
Max Entries8 packetsCircular FIFOOldest packets drop off automatically
Row Height13px pitchSize 1 fontFits up to 8 rows comfortably in content zone
Color CodingFG / WARNText TokenYellow if packet frequency was off-grid; white if standard
Line FormatStringTemplate{profile} [!nodeId] {freq}MHz {rssi}dBm {age}s
+
+
+
+ + +
+
+
+ 17 / ANALYZE SUITE +

Plate: NODES (Observed Node Roster)

+
+

+ Deduplicated roster of up to 24 observed nodes sorted by recency. Tracks transmission count per node, last seen RSSI, and elapsed seconds. +

+ +
+
+
NODES
+
82%
+
+ +
meshtastic !a4b892f0 x14 -68dBm 2s
+
meshtastic !38c11e04 x3 -94dBm 14s
+
meshtastic !f102bb88 x8 -75dBm 42s
+
meshtastic !12aa045e x1 -89dBm 65s
+
meshtastic !88319eac x5 -102dBm 120s
+ +
Meshtastic
+
+ +
+ + + + + + + + +
AttributeSpecificationDetails
Max Nodes24 nodesStatic array in node_roster.h
SortingRecency (ageS asc)Most recently heard nodes stay pinned to the top
Hit MultiplierxN countTracks packet density emitted by that node
Node Address!xxxxxxxx8-character hex address formatted with bang prefix
+
+
+
+ + +
+
+
+ 18 / NAVIGATION +

Plate: Menu Hierarchy & Sliders

+
+

+ Grouped hierarchical menu with inverted color selection bars (24px row pitch) and live interactive slider widgets (e.g. ST7789V2 LEDC backlight brightness). +

+ +
+
+
MENU > System
+
82%
+
+ +
+ Tools > +
+ +
+ Display > +
+ +
+ WiFi Toggle + OFF +
+
+ +
+
+
    +
  • Tools > (Probe, Sweep, Cell, Trace)
    • +
  • Analyze > (Meter, Waterfall, Scope, Captures, Nodes)
    • +
  • Profile > (Meshtastic, MeshCore, Reticulum, General)
    • +
  • System > +
      +
    • Display > +
        +
      • Brightness (Slider 10-100%)
        • +
      • Dim Timeout (Off, 30s, 60s, 300s)
        • +
      +
      • +
    • WiFi AP Toggle
      • +
    • Reboot
      • +
    +
    • +
+
+
+
+
+ + +
+
+
+ 19 / OVERLAYS +

Plate: Toast Notifications

+
+

+ Transient bottom-docked banner (16px height on hardware, 35px at 2.2×). Slides up, holds for TOAST_DURATION_MS (2000ms), and automatically dismisses without disrupting background operations. +

+ +
+
+
RADIO
+
82%
+
+ +
+ Profile: Meshtastic + OK +
+
+
+ +
+ + + + + + + + +
PropertyHardware ValueRemarks
Toast Height16pxExact height of size 2 font; docks flush to bottom
Duration2000msConfigured in ui_task_shared.h
Slide Time150msSmooth linear easing during entry and exit
Non-BlockingCore 0 tickRadio packet reception continues unimpeded
+
+
+
+ + +
+
+
+ 20 / INPUT ERGONOMICS +

Cardputer TCA8418 Key Matrix

+
+

+ Driven via TCA8418 I2C keyboard scanner. Four primary navigation keys form an inverted-T cluster, supplemented by Enter, Back, and direct single-key shortcuts. +

+ + + + + + + + + + + + + + + + +
Physical KeyMatrix CodePrimary Context ActionTool / Analyzer Subpage Action
, (Comma)KEY_PREVPrevious Carousel PageExit to Menu
/ (Slash)KEY_NEXTNext Carousel PageExit to Menu
; (Semicolon)KEY_UPMenu Cursor Up / Slider IncreaseCycle Previous Subpage
. (Period)KEY_DOWNMenu Cursor Down / Slider DecreaseCycle Next Subpage
EnterKEY_SELECTEnter Submenu / Execute ActionTrigger Scan / Repeat Toggle
` (Grave)KEY_BACKExit Submenu to ParentExit to Menu / Radio
1 .. 5HOTKEYDirect Jump to Carousel Pages 1-5Jump directly to Carousel Page
P, S, CHOTKEYDirect Jump to Probe, Sweep, CellRestart Tool Run
8HOTKEYDirect Jump to Analyze SuiteCycle Analyze Tools
+
+ + +
+
+
+ 21 / V2 ROADMAP +

Future UI Evolution & Proposals

+
+

+ Identified improvement areas and candidate enhancements under evaluation for Workstream 12–15: +

+ +
+
+

Direct Sunlight Contrast

+

COL_DIM (~75% grey) provides sufficient outdoor contrast while preserving visual hierarchy. High-contrast monochrome mode is under evaluation for high-glare environments.

+
+
+

Workstream 12 Focus Visuals

+

Adding dedicated single-bin dwell and statistical distribution indicators (P90/P50/Floor) when Workstream 12 transitions to production.

+
+
+

Activity Hero State Refinement

+

Maintaining clear hero status during active background scans while preserving instant access to last-result telemetry when idle.

+
+
+

Waterfall Persistence

+

Evaluating optional ring buffer persistence to SD for long-term RF landscape comparison across wardrive sessions.

+
+
+ +
+
+
Interactive Prototype Available
+
Explore V2 UI Proposals & Sunlight/Night Vision Labs
+
Test the glanceable radio ownership pill, live 30s ingress sparkline, and WS12 focus dwell gauge.
+
+ Launch V2 Labs → +
+
+ +
+ LoRaTrace RX Firmware UI Design Guide · Master Specification v1.0.7 · Maintained in docs/UI-Design_Guide.html +
+
+
+ + + + + + diff --git a/docs/UI-Recommendations.html b/docs/UI-Recommendations.html new file mode 100644 index 0000000..6c06f23 --- /dev/null +++ b/docs/UI-Recommendations.html @@ -0,0 +1,1404 @@ + + + + + +LoRaTrace RX · UI Recommendations & V2 Proposals + + + + + + + +
+ + + + + +
+
LoRaTrace RX · V2 Hardware UI Recommendations
+

Proposed UI Improvements & Live Labs

+ + +
+ ← Baseline Spec (v1.0.7 Master Guide) + ★ Proposed Improvements (Live Labs) +
+ + + + + +
+
+
+

Proposals Simulator Workbench

+
+ Strict 1:1 hardware font scale (Adafruit GFX Size 1 = 6×8px, Size 2 = 12×16px on 240×135 TFT) +
+
+ 1:1 HARDWARE ACCURATE +
+ +
+ +
+
+ +
+
+ Active Screen: Slot 2 · Split RF Activity Dashboard +
+ + +
+
+
Simulation Loops: ACTIVITY
+ LOOP: IDLE +
+
+ +
+
+
+ + +
+
+

1. Select Screen to Test

+
+ + + + +
+
Switch between candidate screen designs to inspect layouts with authentic 1:1 font sizing and clean header alignment.
+
+ +
+

2. Zero-Cost Palette Switcher (Lab)

+
+ + + + +
+
Swaps the 6-entry RGB565 lookup table. Proves direct sunlight readability and night stealth with 0 RAM allocation.
+
+ +
+

3. Header & Battery State Sim

+
+ + + +
+
Tests the authentic v1.0.7 header: battery outline, terminal nib, dynamic fill bars, and status dots.
+
+
+
+
+ + +
+ Carousel Slot 2 Upgrade +

Slot 2: Live Split Activity & RF Field Dashboard

+

+ Replaces the old 4-line static text with a high-density, real-time RF triage dashboard that stays readable at 240×135. +

+ +
+
+

Current v1.0.7 Slot 2

+
    +
  • Displays 4 static text rows: Last Sweep peak, Last Probe hit, Last Cell rise, Scope status.
    • +
  • No rolling time-series or packet throughput history.
    • +
  • Screen is completely blanked during active scans.
    • +
+
+
+

Proposed Split Dashboard

+
    +
  • Top Half (Live 30s Sparkline): Rolling packet rate bar graph (1 sample/sec, 30-byte ring buffer) with standard Size 1 header labels.
    • +
  • Bottom Half (3 Triage Cards): +
      +
    • SWEEP PK: 906.8 MHz (Size 2 font) @ -64dBm.
      • +
    • CAD HIT: CH 0 (Size 2 font) @ +8.5dB SNR.
      • +
    • AWAY T: 1.4s (Size 2 font) with 60s budget limit.
      • +
    +
    • +
+
+
+
+ + +
+ Workstream 12 Plate +

Workstream 12: Dedicated Focus Survey Display Plate

+

+ Direct UI specification for active Workstream 12. Provides honest single-channel dwell timing and statistical distribution without guessing. +

+ +
+
+

V1 Foundation (Missing WS12 UI)

+
    +
  • V1 only supported wide-band 85-bin Sweep or CAD candidate hopping.
    • +
  • No visual plate for single-channel dwell or sample confidence.
    • +
+
+
+

Proposed Focus Survey Screen

+
    +
  • Large Target MHz Header: 906.875 MHz [CH 0] (Size 2 bold font).
    • +
  • Dwell Progress Arc / Countdown: 08.4s / 10.0s with filled bar.
    • +
  • Statistical Bracket Gauge (-120 to -40 dBm): Clean linear scale marking Noise Floor (-116), P50 Median (-98), and P90 (-82).
    • +
  • Operator Truth Badge: [SAMPLING] → [REPEATED] (High confidence) or [INSUFFICIENT] (Zero evidence labeled truthfully, never guessed).
    • +
  • Instant Abort: [ESC] Abort & Return Home.
    • +
+
+
+
+ + +
+ Analyze Suite Modal +

Captures Ring: Non-Decrypting Packet Inspector Modal

+

+ Pressing [Enter] on any packet in the Captures ring brings up a forensic RF breakdown without violating our strict RX-only / no-decryption boundaries. +

+ +
+
+

Current Captures Screen

+
    +
  • Scrollable list of 16-entry ring buffer.
    • +
  • Only shows: Timestamp, Node ID, RSSI, and truncated string.
    • +
  • No way to inspect RF signal quality, AFC frequency offset, or framing headers.
    • +
+
+
+

Proposed Inspector Modal

+
    +
  • Detailed Signal Quality: RSSI (-78dBm), SNR (+9.2dB), Estimated Frequency Error / AFC (+1.4kHz).
    • +
  • LoRa Framing Details: Modulation (SF11/BW250/CR4:5), Payload Length (48 bytes), CRC OK / FAIL.
    • +
  • Raw Byte Hex Nibble Stream: Formatted hex view of raw over-the-air bytes (e.g. FF FF FF FF ...) without payload decryption or client protocol emulation.
    • +
+
+
+
+ + +
+ Display Foundations +

Zero-Cost Hardware Palette Switching (Sunlight & Night)

+

+ Direct sunlight washed out low-contrast greys on the ST7789V2 TFT; white screens ruin night vision. We propose 4 instant hardware color profiles with zero RAM footprint. +

+ +
+
+

Current v1.0.7 Palette

+
    +
  • Fixed black background (COL_BG = 0x0000).
    • +
  • Dim grey (COL_DIM = 0xBDF7) washes out under direct noon sunlight on windshield dashboard mounts.
    • +
  • White screen brightness impairs dark adaptation during nighttime wardrive operations.
    • +
+
+
+

Proposed Dynamic Lookups

+
    +
  • High-Sunlight Inverted: Crisp paper-white background with dark black ink and saturated blue/amber accents. Maximum ambient reflection contrast.
    • +
  • Tactical Red Night: True aviation red-on-black (#ff2828 on #000000) preserving rod rhodopsin for dark adaptation and stealth.
    • +
  • Amber CRT: Warm monochrome amber for reduced eye strain during extended night bench monitoring.
    • +
  • Zero RAM Cost: Swaps the existing 6-element uint16_t COL[] table in flash memory during redraws.
    • +
+
+
+
+ + +
+ Feasibility Analysis +

Hardware Memory & Performance Safety Budget

+

+ Verification against our strict Phase 7 memory limits: 0 bytes heap allocation, ≥40KB free SRAM safety floor, and Core 0 non-blocking execution. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Proposed FeatureStatic RAM CostHeap AllocationsExecution BudgetSafety Gate Status
P2: Slot 2 Live 30s Sparkline30 bytes (static ring)0 bytes< 0.5 msPASSED (< 32B budget)
P3: WS12 Focus Survey Display16 bytes (stat struct)0 bytes< 0.8 msPASSED (Bound verified)
P4: Captures Packet Inspector0 bytes (reads existing ring)0 bytes< 1.0 ms (on-demand)PASSED (No heap buffers)
P5: Zero-Cost Palette Switching48 bytes (flash table)0 bytes0 ms (pointer swap)PASSED (Zero SRAM)
+
+ + +
+
+ + + + + + From c699fb098c9919b52ef91720686663ccbde09af9 Mon Sep 17 00:00:00 2001 From: Will Shields Date: Fri, 4 Sep 2026 17:28:02 -0700 Subject: [PATCH 05/21] Raise the Focus bench sample ceiling; warn off /dev/ttyACM* targeting The 2026-09-04 matrix showed detection tracks the source's airtime against dwell/(samples-1), so choosing a real sampling policy needs to sweep spacings down to ~10 ms across a 2,000 ms dwell -- 201 samples, past the old ceiling of 64. The new 208 is bounded by the histogram rather than by RAM: buckets are uint8_t, so a pass whose samples all land in one 1 dB bucket saturates at 255 and the histogram then refuses to report a percentile at all. The histogram is a fixed 141 bytes whatever the sample count, so this costs no static RAM. Also documents a bench hazard that cost this session a wrong flash. The Cardputer and the bench Heltec enumerate identically and their ttyACMn numbering swaps between resets; targeting a raw path put LoRaTrace on the Heltec, which then failed its IO-expander init and read exactly like a dead receiver. The framed harnesses are self-checking, but flashing is not, so HARDWARE_TESTING.md now specifies /dev/serial/by-id and records the failure signature. Co-Authored-By: Claude Opus 5 --- docs/HARDWARE_TESTING.md | 34 ++++++++++++++++++++++++++++++ scripts/phase12_focus_matrix.py | 4 ++-- src/focus_plan.h | 16 +++++++++++++- test/test_focus_plan/test_main.cpp | 16 +++++++++++++- 4 files changed, 66 insertions(+), 4 deletions(-) diff --git a/docs/HARDWARE_TESTING.md b/docs/HARDWARE_TESTING.md index 4dee267..bb1d5b5 100644 --- a/docs/HARDWARE_TESTING.md +++ b/docs/HARDWARE_TESTING.md @@ -162,3 +162,37 @@ Opening the monitor can reset the board through DTR, which creates a new run directory. Start the retained serial capture at that reset and use the new run number. Launcher SD-drop remains valid for release-like verification, but direct USB flashing is preferred while Phase 7 is iterating. + +### Never target a board by `/dev/ttyACM*` + +With the Cardputer-Adv and the bench Heltec both attached, they enumerate as +identical `Espressif USB JTAG serial debug unit` devices and **the `ttyACMn` +numbering swaps between resets.** Address a board by its stable per-device +path instead, which is keyed to the USB serial: + +```text +ls /dev/serial/by-id/ +pio run -e cardputer-adv-bench -t upload \ + --upload-port "/dev/serial/by-id/usb-Espressif_USB_JTAG_serial_debug_unit_-if00" +``` + +Identify which MAC is which once, by resetting each and reading its boot +banner: the transmitter prints `[lttx] Heltec V4 R8 deterministic +transmitter`, the receiver prints `LoRaTrace RX v...`. The framed harnesses +are self-checking (they speak `@LTRX/1` and `@LTTX/1` and simply fail to +handshake if pointed at the wrong board), but **flashing is not** — it writes +whatever image you named to whatever is on that path. + +The signature of getting this wrong is unmistakable and worth recognizing at +a glance, because it looks like a dead receiver: + +```text +LoRaTrace RX v1.0.7 (...) — tasks + GPS + SD logging + WiFi + ... +FATAL: IO expander init failed — no I2C ACK at 0x43. + Antenna switch off and GPS unpowered. +``` + +That is LoRaTrace running on the *Heltec*, which has no PI4IOE5V6408 at +0x43. Repeated RTS resets will not clear it, because the banner is real +firmware booting correctly on the wrong board. Recover by reflashing +`bench/heltec-v4r8-transmitter` to that board (2026-09-04). diff --git a/scripts/phase12_focus_matrix.py b/scripts/phase12_focus_matrix.py index 4417e53..a2622db 100644 --- a/scripts/phase12_focus_matrix.py +++ b/scripts/phase12_focus_matrix.py @@ -311,8 +311,8 @@ def main(): for dwell in dwells: if not 2 <= dwell <= 2000: parser.error(f"--dwell-ms {dwell} outside the bounded 2..2000 request range") - if not 2 <= args.samples <= 64: - parser.error("--samples must be 2..64") + if not 2 <= args.samples <= 208: + parser.error("--samples must be 2..208 (focus_plan.h's bench ceiling)") if args.trials < 1: parser.error("--trials must be positive") if args.trials < 30: diff --git a/src/focus_plan.h b/src/focus_plan.h index 7367616..3447d0d 100644 --- a/src/focus_plan.h +++ b/src/focus_plan.h @@ -16,7 +16,21 @@ constexpr uint8_t FOCUS_BENCH_REQUESTED_PASSES = 1; constexpr uint16_t FOCUS_BENCH_DWELL_MIN_MS = 2; constexpr uint16_t FOCUS_BENCH_DWELL_MAX_MS = 2000; constexpr uint16_t FOCUS_BENCH_SAMPLES_MIN = 2; -constexpr uint16_t FOCUS_BENCH_SAMPLES_MAX = 64; +// Raised from 64 to sweep sample spacing at a fixed dwell: the 2026-09-04 +// matrix showed detection tracks the source's airtime against +// dwell/(samples-1), not against dwell, so selecting a real sampling policy +// needs spacings down to ~10 ms across a 2,000 ms dwell (201 samples). +// +// The ceiling is set by the histogram, not by RAM: bucket_counts are uint8_t, +// so a pass whose samples all land in one 1 dB bucket saturates at 255 and +// focusHistogramHasExactQuantiles() then refuses to report a percentile at +// all. Staying below that keeps every quantile exact in the worst case. The +// histogram itself is a fixed 141 bytes regardless of sample count, so this +// costs no additional static RAM (docs/research/phase12-survey-truth-design.md +// §4.2). +constexpr uint16_t FOCUS_BENCH_SAMPLES_MAX = 208; +static_assert(FOCUS_BENCH_SAMPLES_MAX < 255, + "a pass must not be able to saturate a uint8 histogram bucket"); // A bounded request must be bounded in wall-clock time, not just in samples: // each sample waits on the shared SPI bus (radio_task.cpp's 250ms BUS_WAIT), diff --git a/test/test_focus_plan/test_main.cpp b/test/test_focus_plan/test_main.cpp index 602d76e..c221e63 100644 --- a/test/test_focus_plan/test_main.cpp +++ b/test/test_focus_plan/test_main.cpp @@ -1,6 +1,6 @@ #include -#include "../../src/focus_plan.h" +#include "../../src/focus_observation.h" void test_focus_request_is_small_and_starts_with_one_selected_bin() { TEST_ASSERT_EQUAL_UINT8(1, FOCUS_SELECTED_BIN_COUNT); @@ -45,11 +45,24 @@ void test_focus_request_rejects_empty_or_out_of_band_work() { request.requested_dwell_ms = 100; request.requested_samples = FOCUS_BENCH_SAMPLES_MIN - 1; TEST_ASSERT_FALSE(focusRequestIsValid(request)); + request.requested_samples = FOCUS_BENCH_SAMPLES_MAX + 1; + TEST_ASSERT_FALSE(focusRequestIsValid(request)); request.requested_samples = FOCUS_BENCH_SAMPLES_MIN; request.requested_passes = 0; TEST_ASSERT_FALSE(focusRequestIsValid(request)); } +void test_sample_ceiling_keeps_every_quantile_exact() { + // All samples landing in one 1 dB bucket is the worst case; above 255 the + // bucket saturates and the histogram refuses to report a percentile. + FocusRssiHistogram histogram; + for (uint16_t i = 0; i < FOCUS_BENCH_SAMPLES_MAX; ++i) { + focusHistogramAddSample(histogram, -1000); + } + TEST_ASSERT_TRUE(focusHistogramHasExactQuantiles(histogram)); + TEST_ASSERT_EQUAL_INT16(-1000, focusHistogramP90DbmX10(histogram)); +} + void test_focus_request_is_bounded_in_time_not_only_in_samples() { FocusRequest request; request.region = Region::US; @@ -79,5 +92,6 @@ int main(int argc, char **argv) { RUN_TEST(test_focus_request_resolves_a_sourced_energy_bin_frequency); RUN_TEST(test_focus_request_rejects_empty_or_out_of_band_work); RUN_TEST(test_focus_request_is_bounded_in_time_not_only_in_samples); + RUN_TEST(test_sample_ceiling_keeps_every_quantile_exact); return UNITY_END(); } From d0a9ec44a9e1b8a8453e4076bc341573f1bd0948 Mon Sep 17 00:00:00 2001 From: Will Shields Date: Fri, 4 Sep 2026 18:07:44 -0700 Subject: [PATCH 06/21] Pin the bench home channel into the image; harden the bench harnesses The bench receiver's channel lived only in config.txt on the bench SD card, so replacing a failed card silently moved home from 918.5/SF8/BW125 to the built-in 906.875/SF11/BW250 default. That is not a cosmetic difference: Focus receives at the *home* bandwidth, so measurements would have continued at a different bandwidth than the run they were being compared against, with nothing in the output to show it -- and at BW250 the +-125 kHz bin-offset analysis inverts, since that offset falls inside the passband instead of outside it. The bench image now pins the channel through build flags and says so at boot. Production is untouched: benchHomeChannelOverride() returns false and changes nothing without both LORATRACE_BENCH_FAULTS and the -D flags. Harness fixes, each from a failure this session rather than from review: - The matrix waits for SD to mount instead of failing on one early poll. SD readiness lags HELLO by seconds, so a healthy device read as a broken one. - It refuses to run unless the resolved home channel matches --expect-home-khz, which is what would have caught the card swap immediately. - --samples is repeatable, so a spacing sweep is one process with one pre-flight gate rather than several invocations that could each half-run. - HELLO gets a 45s window. Opening native USB-CDC resets the board, but when that reset lands varies, and the fixed 8.5s settle could expire before boot had even started -- which read as an unresponsive device. Co-Authored-By: Claude Opus 5 --- platformio.ini | 10 ++++++ scripts/phase12_focus_behavior_bench.py | 2 +- scripts/phase12_focus_bench.py | 2 +- scripts/phase12_focus_matrix.py | 47 +++++++++++++++++++------ scripts/phase12_watch_opportunity.py | 2 +- src/bench_fault.cpp | 15 ++++++++ src/bench_fault.h | 12 +++++++ src/main.cpp | 11 ++++++ 8 files changed, 87 insertions(+), 14 deletions(-) diff --git a/platformio.ini b/platformio.ini index 2b4a1c7..7f459b2 100644 --- a/platformio.ini +++ b/platformio.ini @@ -89,6 +89,16 @@ extends = env:cardputer-adv build_flags = ${env:cardputer-adv.build_flags} -D LORATRACE_BENCH_FAULTS=1 + ; Pins the bench receiver's home channel into the image so it no longer + ; depends on config.txt surviving on the bench SD card. Focus receives at + ; the home bandwidth, so a card swap that reverts home to the built-in + ; default silently changes a measurement's bandwidth. Matches the channel + ; the 2026-09-04 dwell matrix ran at; production is unaffected. + -D LORATRACE_BENCH_HOME_FREQ_MHZ=918.5f + -D LORATRACE_BENCH_HOME_SF=8 + -D LORATRACE_BENCH_HOME_BW_KHZ=125.0f + -D LORATRACE_BENCH_HOME_CR=5 + -D LORATRACE_BENCH_HOME_SYNC=0x2B ; Host-native unit test env — for pure-logic headers (channel_plans.h and ; onward: fingerprint.h, log-schema formatting) that don't touch hardware. diff --git a/scripts/phase12_focus_behavior_bench.py b/scripts/phase12_focus_behavior_bench.py index f62b583..ec0d184 100644 --- a/scripts/phase12_focus_behavior_bench.py +++ b/scripts/phase12_focus_behavior_bench.py @@ -114,7 +114,7 @@ def main(): with log_path.open("a", encoding="utf-8") as log: card = Endpoint("cardputer", args.cardputer_port, CARD_MARKER, log) try: - identity = require_ack(card, "HELLO", "-", timeout=15.0) + identity = require_ack(card, "HELLO", "-", timeout=45.0) initial = card_status(card) if initial.get("SD") != "1": raise RuntimeError(f"Focus requires an SD-backed result row: {initial}") diff --git a/scripts/phase12_focus_bench.py b/scripts/phase12_focus_bench.py index 9cd570a..deaa936 100644 --- a/scripts/phase12_focus_bench.py +++ b/scripts/phase12_focus_bench.py @@ -79,7 +79,7 @@ def main(): card = Endpoint("cardputer", args.cardputer_port, CARD_MARKER, log) transmitter = None try: - require_ack(card, "HELLO", "-", timeout=15.0) + require_ack(card, "HELLO", "-", timeout=45.0) transmitter = Endpoint("heltec", args.heltec_port, TX_MARKER, log) require_ack(transmitter, "HELLO", "-") require_ack(transmitter, "QUIET", "-") diff --git a/scripts/phase12_focus_matrix.py b/scripts/phase12_focus_matrix.py index a2622db..1e1d81f 100644 --- a/scripts/phase12_focus_matrix.py +++ b/scripts/phase12_focus_matrix.py @@ -292,7 +292,10 @@ def main(): help=f"repeatable; default {' '.join(str(d) for d in DEFAULT_DWELLS_MS)}") parser.add_argument("--trials", type=int, default=30, help="trials per source state per arm (§6.2 requires at least 30)") - parser.add_argument("--samples", type=int, default=8) + parser.add_argument("--samples", action="append", type=int, + help="repeatable; sample count per pass (spacing is dwell/(n-1))") + parser.add_argument("--expect-home-khz", type=int, default=None, + help="refuse to run unless the resolved home channel matches") parser.add_argument("--order", choices=("alternating", "random"), default="alternating") parser.add_argument("--seed", type=int, default=20260904, help="--order random seed") parser.add_argument("--pulse-gap-ms", type=int, default=30, @@ -311,8 +314,10 @@ def main(): for dwell in dwells: if not 2 <= dwell <= 2000: parser.error(f"--dwell-ms {dwell} outside the bounded 2..2000 request range") - if not 2 <= args.samples <= 208: - parser.error("--samples must be 2..208 (focus_plan.h's bench ceiling)") + sample_counts = args.samples or [8] + for count in sample_counts: + if not 2 <= count <= 208: + parser.error(f"--samples {count} outside focus_plan.h's bench range 2..208") if args.trials < 1: parser.error("--trials must be positive") if args.trials < 30: @@ -330,27 +335,47 @@ def main(): card = Endpoint("cardputer", args.cardputer_port, CARD_MARKER, log) transmitter = None try: - identity = require_ack(card, "HELLO", "-", timeout=15.0) - boot = card_status(card) - if boot.get("SD") != "1": - raise RuntimeError(f"the matrix needs durable focus.csv rows: {boot}") + # Generous: opening native USB-CDC resets the board, but *when* + # that reset lands varies, and the harness's fixed settle can + # expire before boot even starts. HELLO is idempotent and retried + # once a second, so a wide window costs nothing on a healthy boot. + identity = require_ack(card, "HELLO", "-", timeout=45.0) + # SD readiness lags HELLO: the logger task mounts the card and opens + # its logs several seconds into boot, so polling once can read SD=0 + # on a perfectly healthy device (and a boot-time status also predates + # config.txt's channel override). + boot = wait_for(card, lambda status: status.get("SD") == "1", 30.0, + "SD to mount for durable focus.csv rows") + # A boot that missed the card also missed config.txt's channel + # override and silently resolves a different home channel -- and + # Focus tunes at the *home* bandwidth, so trials would land at a + # different bandwidth than the run they are being compared with, + # with nothing in the output to show it. Observed 2026-09-04: + # a no-SD boot came up SF11/BW250 instead of SF8/BW125. + if args.expect_home_khz and int(boot.get("F", "0")) != args.expect_home_khz: + raise RuntimeError( + f"home channel is {boot.get('F')} kHz, expected {args.expect_home_khz}; " + "this boot resolved a different channel (check config.txt applied)") transmitter = Endpoint("heltec", args.heltec_port, TX_MARKER, log) require_ack(transmitter, "HELLO", "-") require_ack(transmitter, "QUIET", "-") results.write({"event": "boot", "identity": identity, "status": boot, "positions": positions, "dwells_ms": dwells, - "trials_per_state": args.trials, "samples": args.samples, + "trials_per_state": args.trials, "samples": sample_counts, "order": args.order, "seed": args.seed}) for position in positions: require_ack(transmitter, "CONFIG", POSITIONS[position]["candidate"]) for dwell in dwells: + for samples in sample_counts: plan = trial_order(args.trials, args.order, rng) - print(f"[{position} {dwell}ms] {len(plan)} trials " - f"(offset {position_offset_khz(position):+.1f} kHz)", flush=True) + spacing = dwell / (samples - 1) + print(f"[{position} {dwell}ms x{samples} samples] {len(plan)} trials " + f"(offset {position_offset_khz(position):+.1f} kHz, " + f"{spacing:.0f}ms spacing)", flush=True) for index, source_on in enumerate(plan): row = run_trial_with_retry(card, transmitter, position, dwell, - args.samples, source_on, + samples, source_on, args.pulse_gap_ms, results, args.attempts) row.update({"event": "trial", "arm_index": index}) diff --git a/scripts/phase12_watch_opportunity.py b/scripts/phase12_watch_opportunity.py index e5ff260..04a66fb 100644 --- a/scripts/phase12_watch_opportunity.py +++ b/scripts/phase12_watch_opportunity.py @@ -200,7 +200,7 @@ def main(): card = Endpoint("cardputer", args.cardputer_port, CARD_MARKER, log) transmitter = None try: - identity = require_ack(card, "HELLO", "-", timeout=15.0) + identity = require_ack(card, "HELLO", "-", timeout=45.0) status = card_status(card) home_khz = int(status.get("F", "0")) if abs(home_khz - HOME_CANDIDATE_MHZ * 1000) > 1: diff --git a/src/bench_fault.cpp b/src/bench_fault.cpp index 9c14e3d..fd1eb25 100644 --- a/src/bench_fault.cpp +++ b/src/bench_fault.cpp @@ -305,6 +305,21 @@ uint16_t benchFocusStallTakeMs() { #endif } +bool benchHomeChannelOverride(ProfileOverrides &overrides) { +#if !defined(LORATRACE_BENCH_FAULTS) || !defined(LORATRACE_BENCH_HOME_FREQ_MHZ) + (void)overrides; + return false; +#else + overrides.meshtastic_set = true; + overrides.meshtastic.freq_mhz = LORATRACE_BENCH_HOME_FREQ_MHZ; + overrides.meshtastic.sf = LORATRACE_BENCH_HOME_SF; + overrides.meshtastic.bw_khz = LORATRACE_BENCH_HOME_BW_KHZ; + overrides.meshtastic.cr_denom = LORATRACE_BENCH_HOME_CR; + overrides.meshtastic.sync_word = LORATRACE_BENCH_HOME_SYNC; + return true; +#endif +} + bool benchArbitrationTriggerAllowed() { #if !defined(LORATRACE_BENCH_FAULTS) return false; diff --git a/src/bench_fault.h b/src/bench_fault.h index ab26ded..068df9e 100644 --- a/src/bench_fault.h +++ b/src/bench_fault.h @@ -2,6 +2,7 @@ #include +#include "channel_plans.h" #include "energy_observation.h" // Deterministic fault hooks used only by the dedicated cardputer-adv-bench @@ -125,6 +126,17 @@ bool benchFocusSurveyTriggerAllowed(); bool benchFocusStallConfigure(uint32_t stall_ms); uint16_t benchFocusStallTakeMs(); +// Bench-image-only home-channel override, applied after the SD config load. +// The bench's channel lives in config.txt on the bench SD card, so replacing +// a failed card silently moves `home` to the built-in default -- and Focus +// receives at the *home* bandwidth, so a bench measurement would quietly +// change bandwidth with nothing in its output to show it (2026-09-04: a card +// swap moved home from 918.5/SF8/BW125 to 906.875/SF11/BW250). +// +// Returns false, changing nothing, unless the bench image was built with +// -D LORATRACE_BENCH_HOME_* flags. Production always returns false. +bool benchHomeChannelOverride(ProfileOverrides &overrides); + // Bench-image-only gate for entering Cell and Scope from framed control. // Both are UI-only actions in production, so a fixture otherwise cannot prove // Focus's mutual exclusion against them diff --git a/src/main.cpp b/src/main.cpp index 27af9e5..391b8f3 100644 --- a/src/main.cpp +++ b/src/main.cpp @@ -30,6 +30,7 @@ #include "analyzer_state.h" #include "backlight.h" #include "battery.h" +#include "bench_fault.h" #include "board_pins.h" #include "capture_settings.h" #include "cell_observation.h" @@ -367,6 +368,16 @@ void setup() { // needed for this read. bool sdMounted = false; loadProfileOverridesFromSD(channelOverrides, PIN_SD_CS, sharedSpi(), &sdMounted); + // Bench images only, and only when built with the -D flags; production + // changes nothing here (bench_fault.h). Announced loudly so a bench + // channel can never be mistaken for the card's own configuration. + if (benchHomeChannelOverride(channelOverrides)) { + SerialLock lock(pdMS_TO_TICKS(200)); + if (lock.held()) { + Serial.println(F("[bench] home channel overridden by build flags, " + "ignoring config.txt's Meshtastic block.")); + } + } // Last profile an operator actually selected via the menu, not always // Meshtastic — see profile_state.h for why this used to be hardcoded. MissionProfile bootProfile = MissionProfile::MESHTASTIC; From f4572ac94c9e06cad42f5429b5497f6ecee0ce6d Mon Sep 17 00:00:00 2001 From: Will Shields Date: Fri, 4 Sep 2026 19:17:27 -0700 Subject: [PATCH 07/21] Derive Focus's sample count from the dwell, at a measured 20 ms spacing The first bench slice fixed the sample count at 8, which made a longer dwell strictly worse at catching bursts: spacing grows with the dwell, so a 2,000 ms pass observed eight instants rather than 2,000 ms. A 360-trial sweep held the dwell at 2,000 ms and varied only the spacing. A 94 ms source was missed at 286 ms and 100 ms spacing (8/15 and 2/15 trials) and caught in 15/15 at both 50 ms and 20 ms, with the worst-case reading improving from -97 dBm to -66 dBm between those two -- so 50 ms is the edge of working, not a safe choice. Detection collapses once spacing approaches the source's airtime. 20 ms keeps roughly a 2x margin against the ~40-50 ms airtime of the fastest realistic mesh traffic. That margin is a judgement about what is worth observing; the measurement itself says only that spacing at or above the airtime fails and half of it works. Finer sampling costs nothing measurable: radio-away was 2,073-2,075 ms across every arm whether the pass took 8 samples or 101, and the histogram is a fixed 141 bytes regardless of sample count. The cost of a longer dwell is the dwell. focusSamplesForDwell() rounds the interval count up, because truncating would let a 30 ms dwell take two samples 30 ms apart and quietly violate the constant it derives from. It is split into two constexpr functions because the device toolchain builds as C++11, where a constexpr body must be a single return -- the native test environment accepts locals and does not catch it. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 12 ++++ .../2026-09-04-phase12-focus-matrix.md | 60 +++++++++++++++++++ src/focus_plan.h | 40 +++++++++++++ test/test_focus_plan/test_main.cpp | 17 ++++++ 4 files changed, 129 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index e967c53..f008590 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,18 @@ project (not a log of how it got there), see [docs/STATUS.md](docs/STATUS.md). ## 2026-09-05 +- Selected Focus's sampling policy from measurement instead of reasoning: + `FOCUS_SAMPLE_SPACING_MS = 20`, with the sample count derived from the dwell + (`ceil(dwell/spacing) + 1`) rather than fixed at 8. A 360-trial sweep held + dwell at 2,000 ms and varied only the spacing: a 94 ms source was missed at + 286 ms and 100 ms spacing (8/15 and 2/15) and caught 15/15 at both 50 ms and + 20 ms, with worst-case reading improving from -97 to -66 dBm between them. + Detection collapses once spacing approaches the source's airtime. Finer + sampling turned out to be free -- radio-away measured 2,073-2,075 ms whether + a pass took 8 samples or 101 -- so the only real argument for coarse + sampling was one nobody had checked. Evidence appended to + `docs/hardware-results/2026-09-04-phase12-focus-matrix.md`. + - Ran Workstream 12's §6.2 controlled dwell matrix: 900 trials in 54.7 min, zero transport errors, zero drops, home restored on every trial; 14 of 15 arms separated a controlled source from ambient 30/30 vs 0/30. The finding diff --git a/docs/hardware-results/2026-09-04-phase12-focus-matrix.md b/docs/hardware-results/2026-09-04-phase12-focus-matrix.md index e8800eb..5aa9e09 100644 --- a/docs/hardware-results/2026-09-04-phase12-focus-matrix.md +++ b/docs/hardware-results/2026-09-04-phase12-focus-matrix.md @@ -128,3 +128,63 @@ for the tail to clear before a trial ends. A source-on trial consequently means *the transmitter was radiating during the window*, which is the condition the qualifying-RSSI question needs — and explicitly **not** a catch-probability estimate for intermittent traffic. + + +--- + +# Follow-up: sample-spacing sweep (360 trials) + +**Raw evidence:** `private/phase12-spacing-20260905T010723Z.{jsonl,log}`. +360 trials, 31.5 min, zero transport errors, zero drops, home restored on +every trial. Same fixture and build family; home channel pinned to +918.5 MHz / SF8 / BW125 by bench build flags rather than by the SD card. + +Finding 1 above said detection tracks the source's airtime against sample +spacing. This sweep tests that directly: dwell held at 2,000 ms while the +sample count varies, so spacing is the only thing changing. + +| source airtime | 286 ms spacing (8) | 100 ms (21) | 50 ms (41) | 20 ms (101) | +|---|---|---|---|---| +| 94 ms (`high`) | 8/15, -3.0 dB | 2/15, -1.0 dB | **15/15, +2.0 dB** | **15/15, +24.0 dB** | +| 148 ms (`mid-aligned`) | 15/15, +53 dB | 15/15, +52 dB | 15/15, +54 dB | 15/15, +54 dB | +| 286 ms (`low-aligned`) | 15/15, +26 dB | 15/15, +25 dB | 15/15, +35 dB | 15/15, +36 dB | + +`detects` counts source-on trials reading above that arm's strongest ambient +reading; `margin` is worst source-on minus strongest source-off. + +**Detection collapses once spacing approaches the source's airtime.** The +94 ms source is missed at 286 ms and 100 ms spacing and caught in every trial +at 50 ms and 20 ms. Its worst-case reading also improves sharply between those +two (-97 dBm at 50 ms, -66 dBm at 20 ms), so 50 ms is the edge of working +rather than a safe choice. The two longer-airtime sources separate at every +spacing tested, and their worst case still improves as spacing tightens +(-71 to -64 dBm for the 286 ms source). + +**Finer sampling costs nothing measurable.** Radio-away time was 2,073-2,075 ms +across all four arms, whether the pass took 8 samples or 101: + +| samples | spacing | radio-away (min/max) | observation_ms | +|---|---|---|---| +| 8 | 286 ms | 2,074 / 2,075 ms | 2,000 ms | +| 21 | 100 ms | 2,074 / 2,075 ms | 2,000 ms | +| 41 | 50 ms | 2,073 / 2,074 ms | 2,000 ms | +| 101 | 20 ms | 2,074 / 2,074 ms | 2,000 ms | + +The cost of a longer dwell is the dwell. The sample count inside it is free, +and the histogram is a fixed 141 bytes regardless (§4.2), so there is no +memory argument for coarse sampling either. + +## Selected policy + +`FOCUS_SAMPLE_SPACING_MS = 20`, with `focusSamplesForDwell()` deriving the +count as `ceil(dwell / spacing) + 1`. Rounding up matters: truncating would +let a 30 ms dwell take two samples 30 ms apart and violate the constant it is +derived from. This keeps roughly a 2x margin against the ~40-50 ms airtime of +the fastest realistic mesh traffic, which is a judgement about what is worth +observing rather than a measured limit -- the measurement says only that +spacing at or above the airtime fails and half of it works. + +The `p90 >= -90 dBm` qualifying-condition candidate from the main matrix was +measured under the old 8-sample policy and a source ~70 dB above ambient. It +should be re-selected against this sampling and a weaker source before any +operator-facing use. diff --git a/src/focus_plan.h b/src/focus_plan.h index 3447d0d..754561b 100644 --- a/src/focus_plan.h +++ b/src/focus_plan.h @@ -32,6 +32,43 @@ constexpr uint16_t FOCUS_BENCH_SAMPLES_MAX = 208; static_assert(FOCUS_BENCH_SAMPLES_MAX < 255, "a pass must not be able to saturate a uint8 histogram bucket"); +// How far apart a pass's RSSI samples may fall, in ms. This is the constant +// that decides what a pass can observe at all, and it was measured, not +// chosen: at a fixed 2,000 ms dwell, a 94 ms source was missed at 286 ms and +// 100 ms spacing (8/15 and 2/15 trials) and caught 15/15 at both 50 ms and +// 20 ms, with the worst-case reading improving from -97 dBm to -66 dBm +// between those two (docs/hardware-results/2026-09-04-phase12-focus-matrix.md). +// Detection collapses once spacing approaches the source's airtime, so 20 ms +// keeps roughly a 2x margin against the ~40-50 ms airtime of the fastest +// realistic mesh traffic. +// +// Finer sampling is free in the only currency that matters here: measured +// radio-away time was 2,073-2,075 ms across every arm, whether the pass took +// 8 samples or 101. The cost of a longer dwell is the dwell itself. +constexpr uint16_t FOCUS_SAMPLE_SPACING_MS = 20; + +// Samples a dwell should take under that policy. Fixing the sample count +// instead -- as the first bench slice did at 8 -- makes a longer dwell +// strictly worse at catching bursts, because the spacing grows with it: a +// 2,000 ms dwell then observes eight instants, not 2,000 ms, which is exactly +// what the coverage vocabulary must never imply (§3). +// Rounds the interval count up, so actual spacing is at most the policy +// rather than at least it -- truncating would let a 30 ms dwell take two +// samples 30 ms apart and quietly violate the constant it is derived from. +// Split in two because the device toolchain builds this as C++11 (pinned +// Arduino-ESP32 2.0.17), where a constexpr body must be a single return -- +// the native test environment's newer standard accepts locals and will not +// catch it. +constexpr uint16_t focusDerivedSampleCount(uint16_t dwell_ms) { + return (uint16_t)((dwell_ms + FOCUS_SAMPLE_SPACING_MS - 1) / FOCUS_SAMPLE_SPACING_MS + 1); +} + +constexpr uint16_t focusSamplesForDwell(uint16_t dwell_ms) { + return focusDerivedSampleCount(dwell_ms) < FOCUS_BENCH_SAMPLES_MIN + ? FOCUS_BENCH_SAMPLES_MIN + : focusDerivedSampleCount(dwell_ms); +} + // A bounded request must be bounded in wall-clock time, not just in samples: // each sample waits on the shared SPI bus (radio_task.cpp's 250ms BUS_WAIT), // so contention can stretch a nominal dwell well past it while the radio is @@ -94,6 +131,9 @@ constexpr bool focusRequestIsValid(const FocusRequest &request) { request.requested_samples <= FOCUS_BENCH_SAMPLES_MAX; } +static_assert(focusSamplesForDwell(FOCUS_BENCH_DWELL_MAX_MS) <= FOCUS_BENCH_SAMPLES_MAX, + "the sampling policy must stay inside the request's sample bound"); + // Deadline measured from the first sample-loop tick, not from the request's // acceptance: queue latency is Core 1's scheduling, not the dwell's cost. constexpr uint32_t focusRequestTimeoutMs(const FocusRequest &request) { diff --git a/test/test_focus_plan/test_main.cpp b/test/test_focus_plan/test_main.cpp index c221e63..ee2750a 100644 --- a/test/test_focus_plan/test_main.cpp +++ b/test/test_focus_plan/test_main.cpp @@ -85,6 +85,22 @@ void test_focus_request_is_bounded_in_time_not_only_in_samples() { TEST_ASSERT_TRUE(FOCUS_MAX_SAMPLING_MS <= 3000); } +void test_sampling_policy_scales_with_dwell_and_stays_bounded() { + // The measured policy: spacing stays fixed, so the count follows the dwell. + TEST_ASSERT_EQUAL_UINT16(6, focusSamplesForDwell(100)); + TEST_ASSERT_EQUAL_UINT16(26, focusSamplesForDwell(500)); + TEST_ASSERT_EQUAL_UINT16(101, focusSamplesForDwell(2000)); + + // Actual spacing must never exceed the policy at any legal dwell -- that + // is the property the 2026-09-04 sweep bought, not the sample counts. + for (uint16_t dwell = FOCUS_BENCH_DWELL_MIN_MS; dwell <= FOCUS_BENCH_DWELL_MAX_MS; ++dwell) { + const uint16_t samples = focusSamplesForDwell(dwell); + TEST_ASSERT_TRUE(samples >= FOCUS_BENCH_SAMPLES_MIN); + TEST_ASSERT_TRUE(samples <= FOCUS_BENCH_SAMPLES_MAX); + TEST_ASSERT_TRUE(dwell / (samples - 1) <= FOCUS_SAMPLE_SPACING_MS); + } +} + int main(int argc, char **argv) { UNITY_BEGIN(); RUN_TEST(test_focus_request_is_small_and_starts_with_one_selected_bin); @@ -93,5 +109,6 @@ int main(int argc, char **argv) { RUN_TEST(test_focus_request_rejects_empty_or_out_of_band_work); RUN_TEST(test_focus_request_is_bounded_in_time_not_only_in_samples); RUN_TEST(test_sample_ceiling_keeps_every_quantile_exact); + RUN_TEST(test_sampling_policy_scales_with_dwell_and_stays_bounded); return UNITY_END(); } From 0f7bc7a779f683d45738276e4fb17e5a2d3bebd9 Mon Sep 17 00:00:00 2001 From: Will Shields Date: Fri, 4 Sep 2026 21:00:17 -0700 Subject: [PATCH 08/21] Add a WiFi control bridge to the bench transmitter Bench measurements need real path loss between the transmitter and the receiver, and USB does not reach that far. The transmitter now optionally joins a network and serves the identical @LTTX/1 framed protocol over TCP, so a host can drive it from across a house. The harness needed no change at all: bench_harness.py opens ports through pyserial's serial_for_url(), which already accepts socket://host:port. Replies follow whichever transport issued the command and stay there. That is not a detail: ARM's TX_STARTED/TX_DONE are emitted from loop() long after the command returns, so a bridge-driven pulse would otherwise send its completion evidence out a USB port nobody is reading, and the harness gates every source-on trial on exactly those frames. Credentials live in NVS, set once over USB by scripts/heltec_wifi_setup.py, which prompts locally so a password never reaches a transcript or a shell history. WIFI_SSID/WIFI_PASS are refused over the bridge itself: a credential change should not be possible through the transport it authorizes, and a wrong one would disconnect the sender mid-command. WiFi.setSleep(false) is applied after association rather than before. Setting it ahead of WiFi.begin() is silently discarded, leaving the station in modem sleep -- which keeps the association looking perfect to the AP while dropping broadcast frames, so the board answers no ARP and appears dead to everything except its own logs. The fixture is mains powered; there is nothing to save. NET_TEST dials outward and reports the result, which is what finally distinguished "our firmware is broken" from "the network is blocking these two clients". It showed the board reaching the gateway and the internet while unable to reach the host, in both directions -- wireless client isolation, not a defect. Documented so the next person spends minutes on it, not an evening. SECURITY: the bridge has no authentication, and its commands key a transmitter. It is off unless explicitly enabled, output stays capped at TX_POWER_DBM, and every transmit path is bounded -- but anything that can reach the port can fire a pulse. Trusted networks only; --disable when idle. Co-Authored-By: Claude Opus 5 --- bench/heltec-v4r8-transmitter/src/main.cpp | 249 ++++++++++++++++++++- scripts/heltec_wifi_setup.py | 119 ++++++++++ 2 files changed, 366 insertions(+), 2 deletions(-) create mode 100644 scripts/heltec_wifi_setup.py diff --git a/bench/heltec-v4r8-transmitter/src/main.cpp b/bench/heltec-v4r8-transmitter/src/main.cpp index f23b8d2..321ef4a 100644 --- a/bench/heltec-v4r8-transmitter/src/main.cpp +++ b/bench/heltec-v4r8-transmitter/src/main.cpp @@ -1,6 +1,8 @@ #include +#include #include #include +#include #include #include @@ -68,7 +70,7 @@ constexpr uint32_t BUTTON_LONG_PRESS_MS = 1500; constexpr uint32_t BUTTON_PANIC_STOP_MS = 5000; constexpr uint16_t FIRE_BUTTON_DELAY_MS = 250; // same delay phase8_bench.py's own ARM calls typically use -enum class MenuScreen : uint8_t { HOME = 0, CANDIDATE, FIRE, SWEEP, BEACON, COUNT }; +enum class MenuScreen : uint8_t { HOME = 0, CANDIDATE, FIRE, SWEEP, BEACON, WIFI, COUNT }; // Bench-only field/range-test beacon: periodic capped-count TX pulses at // the selected candidate, for verifying Cardputer detection/range with a @@ -123,6 +125,53 @@ bool buttonActionFired = false; bool buttonPanicFired = false; uint32_t buttonPressedAtMs = 0; bool beaconActive = false; + +// --- Optional WiFi control bridge --------------------------------------- +// +// Exists so the transmitter can be physically separated from the receiver +// (across a room, outdoors) while a host still drives it: bench measurements +// need real path loss between the two boards, and USB does not reach. The +// bridge speaks the exact same @LTTX/1 framed protocol as USB, so the host +// harness needs no changes -- bench_harness.py already opens ports through +// serial_for_url(), which accepts socket://host:port. +// +// SECURITY: this listens on the local network and its commands key a +// transmitter. It is off unless explicitly enabled, but once enabled there is +// no authentication -- anything that can reach the port can trigger a pulse. +// That is acceptable only because this is a bench fixture on a trusted LAN, +// output is capped at TX_POWER_DBM, and every transmit path is bounded. Do +// not expose the port beyond a trusted network, and turn the bridge off +// (WIFI_OFF) when the fixture is not in use. +// +// Credentials live in NVS, set once over USB, and are never stored in the +// repository. The frame grammar splits on spaces, so an SSID or password +// containing a space cannot be carried by WIFI_SSID/WIFI_PASS. +constexpr uint16_t WIFI_BRIDGE_PORT = 4227; +constexpr char WIFI_NVS_NAMESPACE[] = "lttxwifi"; +constexpr uint32_t WIFI_RETRY_INTERVAL_MS = 15000; + +WiFiServer bridgeServer(WIFI_BRIDGE_PORT); +WiFiClient bridgeClient; +char bridgeInput[FRAME_MAX] = {}; +size_t bridgeInputLength = 0; +bool wifiEnabled = false; +bool bridgeListening = false; +uint32_t lastWifiAttemptMs = 0; +String wifiSsid; +String wifiPass; + +// Replies follow whichever transport issued the command, and stay there until +// another command arrives -- ARM's asynchronous TX_STARTED/TX_DONE are emitted +// from loop() long after the command returned, and a host driving the bridge +// must receive them rather than have them go out the USB port nobody is +// reading. +Print *replyStream = &Serial; + +// Defined below pollSerial(); declared here because handle() dispatches the +// WIFI_* commands before those definitions are in scope. +void wifiPersist(); +void wifiStart(); +void wifiStop(); uint16_t beaconPulseCount = 0; uint32_t beaconNextAtMs = 0; @@ -133,6 +182,7 @@ const char *screenLabel(MenuScreen screen) { case MenuScreen::FIRE: return "FIRE"; case MenuScreen::SWEEP: return "SWEEP"; case MenuScreen::BEACON: return "BEACON"; + case MenuScreen::WIFI: return "WIFI"; default: return "?"; } } @@ -141,6 +191,12 @@ void drawMenu() { display.clearBuffer(); display.setFont(u8g2_font_6x10_tf); display.drawStr(0, 10, screenLabel(currentScreen)); + // Right-aligned so it never collides with the longest screen label. + const char *wifiBadge = !wifiEnabled ? "" : (WiFi.status() == WL_CONNECTED ? "wifi:ok" + : "wifi:.."); + if (wifiBadge[0] != '\0') { + display.drawStr(128 - (int)(strlen(wifiBadge) * 6), 10, wifiBadge); + } char line1[24] = {}; char line2[24] = {}; @@ -183,6 +239,22 @@ void drawMenu() { (unsigned)BEACON_MAX_PULSES); snprintf(line3, sizeof(line3), "%s", active->name); break; + case MenuScreen::WIFI: + if (!wifiEnabled) { + snprintf(line1, sizeof(line1), "Bridge: OFF"); + snprintf(line2, sizeof(line2), "USB control only"); + } else if (WiFi.status() == WL_CONNECTED) { + snprintf(line1, sizeof(line1), "Bridge: LISTENING"); + snprintf(line2, sizeof(line2), "%s", wifiSsid.c_str()); + snprintf(line3, sizeof(line3), "%s", WiFi.localIP().toString().c_str()); + snprintf(line4, sizeof(line4), "port %u", (unsigned)WIFI_BRIDGE_PORT); + haveLine4 = true; + } else { + snprintf(line1, sizeof(line1), "Bridge: JOINING"); + snprintf(line2, sizeof(line2), "%s", wifiSsid.c_str()); + snprintf(line3, sizeof(line3), "not connected"); + } + break; default: break; } @@ -264,7 +336,7 @@ void reply(uint16_t sequence, const char *event, const char *argument) { const int n = snprintf(body, sizeof(body), "@LTTX/1 %u %s %s", (unsigned)sequence, event, argument); if (n < 0 || (size_t)n >= sizeof(body)) return; - Serial.printf("%s %04X\n", body, (unsigned)crc16(body, (size_t)n)); + replyStream->printf("%s %04X\n", body, (unsigned)crc16(body, (size_t)n)); } bool configure(const Candidate *candidate) { @@ -500,17 +572,184 @@ void handle(char *line) { } else { reply(sequence, "ERROR", "BAD_ARG"); } + } else if (strcmp(command, "WIFI_SSID") == 0) { + // USB only on purpose: credentials must not be settable through the + // bridge they authorize, and a wrong value taken over the bridge would + // disconnect the very transport that sent it. + if (replyStream != &Serial) { + reply(sequence, "ERROR", "USB_ONLY"); + } else if (strlen(argument) == 0 || strlen(argument) > 32) { + reply(sequence, "ERROR", "BAD_SSID"); + } else { + wifiSsid = argument; + wifiPersist(); + reply(sequence, "ACK", "SSID_SET"); + } + } else if (strcmp(command, "WIFI_PASS") == 0) { + if (replyStream != &Serial) { + reply(sequence, "ERROR", "USB_ONLY"); + } else if (strlen(argument) > 63) { + reply(sequence, "ERROR", "BAD_PASS"); + } else { + // "-" clears it, for an open network. + wifiPass = (strcmp(argument, "-") == 0) ? "" : argument; + wifiPersist(); + reply(sequence, "ACK", "PASS_SET"); + } + } else if (strcmp(command, "WIFI_ON") == 0) { + if (wifiSsid.length() == 0) { + reply(sequence, "ERROR", "NO_SSID"); + } else { + wifiEnabled = true; + wifiPersist(); + wifiStart(); + reply(sequence, "ACK", "ENABLED"); + } + } else if (strcmp(command, "WIFI_OFF") == 0) { + wifiEnabled = false; + wifiPersist(); + // Answer before tearing the transport down, or a bridge client never + // learns its own command succeeded. + reply(sequence, "ACK", "DISABLED"); + wifiStop(); + } else if (strcmp(command, "NET_TEST") == 0) { + // Diagnostic only: does an *outbound* connection work? A station can + // be associated, hold a lease, and serve nothing reachable if inbound + // frames are being dropped somewhere. Outbound is board-initiated and + // does not depend on receiving a broadcast ARP, so comparing the two + // directions says whether a reverse-dialled bridge would help. + char target[48] = {}; + if (strlen(argument) >= sizeof(target)) { + reply(sequence, "ERROR", "BAD_TARGET"); + } else { + strcpy(target, argument); + char *colon = strrchr(target, ':'); + uint16_t port = 0; + if (colon == nullptr || !parseU16(colon + 1, port)) { + reply(sequence, "ERROR", "BAD_TARGET"); + } else { + *colon = '\0'; + WiFiClient probe; + probe.setTimeout(4); + const bool ok = probe.connect(target, port); + char detail[64] = {}; + snprintf(detail, sizeof(detail), "%s TO=%s:%u FROM=%s", + ok ? "OPEN" : "REFUSED", target, (unsigned)port, + WiFi.localIP().toString().c_str()); + probe.stop(); + reply(sequence, ok ? "ACK" : "ERROR", detail); + } + } + } else if (strcmp(command, "WIFI_STATUS") == 0) { + char status[80] = {}; + snprintf(status, sizeof(status), "EN=%u;CONN=%u;IP=%s;PORT=%u;SSID=%s", + wifiEnabled ? 1U : 0U, WiFi.status() == WL_CONNECTED ? 1U : 0U, + WiFi.status() == WL_CONNECTED ? WiFi.localIP().toString().c_str() : "-", + (unsigned)WIFI_BRIDGE_PORT, + wifiSsid.length() ? wifiSsid.c_str() : "-"); + reply(sequence, "STATUS", status); } else { reply(sequence, "ERROR", "UNSUPPORTED"); } } +void wifiPersist() { + Preferences prefs; + if (!prefs.begin(WIFI_NVS_NAMESPACE, false)) return; + prefs.putString("ssid", wifiSsid); + prefs.putString("pass", wifiPass); + prefs.putBool("enabled", wifiEnabled); + prefs.end(); +} + +void wifiLoadPersisted() { + Preferences prefs; + if (!prefs.begin(WIFI_NVS_NAMESPACE, true)) return; + wifiSsid = prefs.getString("ssid", ""); + wifiPass = prefs.getString("pass", ""); + wifiEnabled = prefs.getBool("enabled", false); + prefs.end(); +} + +void wifiStop() { + if (bridgeClient) bridgeClient.stop(); + if (bridgeListening) { + bridgeServer.stop(); + bridgeListening = false; + } + WiFi.disconnect(true); + WiFi.mode(WIFI_OFF); + // A transport that vanished must not keep receiving asynchronous replies. + replyStream = &Serial; +} + +void wifiStart() { + if (wifiSsid.length() == 0) return; + WiFi.mode(WIFI_STA); + WiFi.begin(wifiSsid.c_str(), wifiPass.c_str()); + lastWifiAttemptMs = millis(); +} + +void pollWifi() { + if (!wifiEnabled) return; + + if (WiFi.status() != WL_CONNECTED) { + if (bridgeListening) { + bridgeServer.stop(); + bridgeListening = false; + if (replyStream != &Serial) replyStream = &Serial; + } + if (millis() - lastWifiAttemptMs >= WIFI_RETRY_INTERVAL_MS) wifiStart(); + return; + } + if (!bridgeListening) { + // Disabled *after* association, not before: setting it ahead of + // WiFi.begin() is silently discarded when the connection is + // established, which leaves the station in modem sleep. A sleeping + // station keeps its association -- the AP shows a healthy client with + // full signal -- but misses broadcast frames, so it never sees an ARP + // request and becomes unreachable for ping and for this server, while + // looking perfect from the router's side. Cost this bench an evening + // (2026-09-05). The fixture is mains powered; there is nothing to save. + WiFi.setSleep(false); + bridgeServer.begin(); + bridgeServer.setNoDelay(true); // Frames are tiny; Nagle would add latency. + bridgeListening = true; + Serial.printf("[lttx] bridge listening on %s:%u\n", + WiFi.localIP().toString().c_str(), (unsigned)WIFI_BRIDGE_PORT); + } + + if (!bridgeClient || !bridgeClient.connected()) { + WiFiClient incoming = bridgeServer.available(); + if (incoming) { + if (bridgeClient) bridgeClient.stop(); + bridgeClient = incoming; + bridgeInputLength = 0; + } + } + while (bridgeClient && bridgeClient.available() > 0) { + const int c = bridgeClient.read(); + if (c < 0) break; + if (c == '\n') { + bridgeInput[bridgeInputLength] = '\0'; + replyStream = &bridgeClient; + handle(bridgeInput); + bridgeInputLength = 0; + } else if (c != '\r' && bridgeInputLength + 1 < sizeof(bridgeInput)) { + bridgeInput[bridgeInputLength++] = (char)c; + } else if (bridgeInputLength + 1 >= sizeof(bridgeInput)) { + bridgeInputLength = 0; + } + } +} + void pollSerial() { while (Serial.available() > 0) { const int c = Serial.read(); if (c < 0) return; if (c == '\n') { input[inputLength] = '\0'; + replyStream = &Serial; handle(input); inputLength = 0; } else if (c != '\r' && inputLength + 1 < sizeof(input)) { @@ -654,6 +893,11 @@ void setup() { delay(2); Serial.println("[lttx] Heltec V4 R8 deterministic transmitter"); + wifiLoadPersisted(); + if (wifiEnabled) { + Serial.printf("[lttx] wifi bridge enabled, joining '%s'\n", wifiSsid.c_str()); + wifiStart(); + } if (configure(active)) { Serial.println("[lttx] radio ready; output capped at -9 dBm"); } else { @@ -666,6 +910,7 @@ void setup() { void loop() { pollSerial(); + pollWifi(); pollButton(); serviceBeacon(); if (armed && (int32_t)(millis() - armedAtMs) >= 0) transmitArmedPacket(); diff --git a/scripts/heltec_wifi_setup.py b/scripts/heltec_wifi_setup.py new file mode 100644 index 0000000..1846ce0 --- /dev/null +++ b/scripts/heltec_wifi_setup.py @@ -0,0 +1,119 @@ +#!/usr/bin/env python3 +"""Configure the bench transmitter's WiFi control bridge, over USB. + +Run this yourself: it prompts for the network password locally and sends it +straight to the board, so the credential never passes through a transcript, +a shell history entry, or a log file. The raw framed exchange is deliberately +not echoed for the WIFI_PASS command. + +The bridge exists so the transmitter can sit far enough from the receiver to +give real path loss while a host still drives it. Once it reports an IP, +point any bench harness at it with: + + --heltec-port socket://:4227 + +The harness needs no change for that: bench_harness.py opens ports through +pyserial's serial_for_url(), which accepts socket:// URLs. + +SECURITY: the bridge has no authentication. Anything that can reach the port +can key the transmitter (bounded, and capped at the firmware's -9 dBm, but +still a transmitter). Enable it on a trusted network only, and run +`--disable` when the fixture is not in use. +""" + +import argparse +import getpass +import io +import pathlib +import sys + +import serial + +sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent)) +from bench_harness import TX_MARKER, Endpoint, parse_fields, require_ack + + +def show_status(tx): + opcode, payload = tx.request("WIFI_STATUS", "-", timeout=6.0) + if opcode != "STATUS": + raise RuntimeError(f"WIFI_STATUS failed: {opcode} {payload}") + fields = parse_fields(payload) + enabled = fields.get("EN") == "1" + connected = fields.get("CONN") == "1" + print(f" enabled : {'yes' if enabled else 'no'}") + print(f" connected : {'yes' if connected else 'no'}") + print(f" ssid : {fields.get('SSID')}") + if connected: + print(f" reachable : socket://{fields.get('IP')}:{fields.get('PORT')}") + return fields + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument("--port", required=True, + help="the transmitter's USB path (use /dev/serial/by-id/...)") + parser.add_argument("--ssid", help="prompted for if omitted") + parser.add_argument("--disable", action="store_true", + help="turn the bridge off and forget nothing else") + parser.add_argument("--status", action="store_true", help="report and exit") + args = parser.parse_args() + + # Frames are space-delimited, so a credential containing a space cannot be + # carried. Better to say so up front than to have the board reject it. + log = io.StringIO() + tx = Endpoint("heltec", args.port, TX_MARKER, log) + try: + identity = require_ack(tx, "HELLO", "-", timeout=25.0) + print(f"transmitter: {identity}\n") + + if args.status: + show_status(tx) + return 0 + + if args.disable: + print(require_ack(tx, "WIFI_OFF", "-", timeout=6.0)) + print("bridge disabled.\n") + show_status(tx) + return 0 + + ssid = args.ssid or input("SSID: ").strip() + if not ssid or " " in ssid: + raise SystemExit("SSID must be non-empty and contain no spaces " + "(the frame grammar splits on spaces)") + password = getpass.getpass("Password (blank for an open network): ") + if " " in password: + raise SystemExit("password cannot contain a space " + "(the frame grammar splits on spaces)") + + require_ack(tx, "WIFI_SSID", ssid, timeout=6.0) + require_ack(tx, "WIFI_PASS", password if password else "-", timeout=6.0) + require_ack(tx, "WIFI_ON", "-", timeout=6.0) + print("\nenabled; waiting for the join to complete...") + + import time + deadline = time.monotonic() + 30.0 + while time.monotonic() < deadline: + fields = parse_fields(tx.request("WIFI_STATUS", "-", timeout=6.0)[1]) + if fields.get("CONN") == "1": + print() + show_status(tx) + print("\nUse that socket:// URL as --heltec-port. The transmitter can now " + "be moved away from the receiver.") + return 0 + time.sleep(2.0) + print("\ndid not join within 30s:") + show_status(tx) + print("\nCheck the SSID and password, and that the network is 2.4 GHz — " + "the ESP32-S3 has no 5 GHz radio.") + return 1 + finally: + tx.close() + + +if __name__ == "__main__": + try: + sys.exit(main()) + except (OSError, serial.SerialException, RuntimeError, TimeoutError, ValueError) as error: + print(f"heltec wifi setup: {error}", file=sys.stderr) + sys.exit(2) From fa7928169f7820acf82bb2bf928b4e342ebf8c34 Mon Sep 17 00:00:00 2001 From: Will Shields Date: Fri, 4 Sep 2026 21:25:22 -0700 Subject: [PATCH 09/21] Measure Focus's qualifying RSSI condition at field levels, and reject it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The 900-trial matrix proposed `p90 >= -90 dBm` while the bench source sat ~70 dB above ambient -- a regime where every metric separates and any threshold looks defensible. Repeating the question at a realistic level (transmitter outdoors, driven over the new WiFi control bridge; receiver antenna fitted, so the ambient floor is real; source peak ~-85 dBm against -99/-100 dBm) across 120 trials, the candidate fails outright. So does every floor-relative variant: peak-median, peak-p90, p90-median. One of ten metric/position combinations separates, by 2.0 dB, which is inside ordinary RSSI variance and would not survive a temperature change or a different antenna. Absolute and relative families fail together, so this is not a matter of having picked the wrong statistic. Two mechanisms explain it. At low SNR `p90` collapses even when the source is present for most of the window -- at one position the transmitter radiated 57-71% of the pass while `p90` still read -94 to -97 dBm against a -100 floor, because the instantaneous RSSI during a weak transmission mostly fails to stand above noise and only its best moments do. And real MeshOregon traffic reached -94 dBm during source-off control trials, which is not measurement error but an actual transmission that no RSSI condition can distinguish from a controlled fixture. Coverage reporting is untouched: valid passes, observation time, and the median/P90/peak summary remain honest. What the evidence refuses is the step from "RSSI was elevated" to "something transmitted" -- exactly the step §3's contract was written to prevent. The design was right and the constant was wrong, which is the outcome that measurement is for. `coverage` stays blank and the activity count stays unpopulated. Two routes remain untested: a per-pass count of samples above an adaptive floor, for which `FocusObservation` already reserves `qualifying_count` and never populates it; and CAD or packet evidence, which §3 already contemplates as the alternative basis for observed activity. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 15 ++++ docs/STATUS.md | 13 ++++ .../2026-09-04-phase12-focus-matrix.md | 74 +++++++++++++++++++ docs/research/phase12-survey-truth-design.md | 22 ++++-- 4 files changed, 118 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f008590..fbeb792 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,21 @@ project (not a log of how it got there), see [docs/STATUS.md](docs/STATUS.md). ## 2026-09-05 +- Measured Focus's qualifying RSSI condition at a realistic signal level and + **rejected it**. The `p90 >= -90 dBm` candidate came from a bench source ~70 + dB above ambient, where every metric separates and the choice looks easy. + Repeated with the transmitter outdoors (source peak ~-85 dBm against a + -99/-100 dBm floor, 120 trials, zero drops), it fails — and so does every + floor-relative variant tested: one of ten metric/position combinations + separates, by 2 dB, which is inside ordinary RSSI variance. Two reasons: at + low SNR `p90` collapses even when the source radiates most of the window, + and real MeshOregon traffic reached -94 dBm during control trials, which no + RSSI condition can distinguish from a controlled source. Coverage reporting + is unaffected; what the evidence refuses is the step from "RSSI was + elevated" to "something transmitted", which §3 forbids anyway. Remaining + routes: a per-pass count above an adaptive floor (`qualifying_count` is + reserved and unpopulated), or CAD/packet evidence. + - Selected Focus's sampling policy from measurement instead of reasoning: `FOCUS_SAMPLE_SPACING_MS = 20`, with the sample count derived from the dwell (`ceil(dwell/spacing) + 1`) rather than fixed at 8. A 360-trial sweep held diff --git a/docs/STATUS.md b/docs/STATUS.md index ada9490..c3f4098 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -95,6 +95,19 @@ summary: [hardware-results/2026-09-04-phase12-focus-matrix.md](hardware-results/ The §6.3 Watch-opportunity comparison has tooling but has not run, so Focus's maximum radio-away budget is still unapproved. +The qualifying RSSI condition was then measured at a realistic signal level +(transmitter outdoors on the WiFi control bridge, receiver antenna fitted: +source peak ~-85 dBm against a -99/-100 dBm ambient floor) and **rejected**. +The `p90 >= -90 dBm` candidate came from a source ~70 dB hot; across 120 +trials it fails, as does every floor-relative variant — one of ten +metric/position combinations separates, by 2 dB, which is inside ordinary RSSI +variance. An RSSI summary statistic cannot carry an activity claim at field +levels. Coverage reporting is unaffected: valid passes, observation time and +the RSSI summary stay honest. `coverage` stays blank, the activity count stays +unpopulated, and the remaining routes are a per-pass count above an adaptive +floor (`qualifying_count`, already reserved in the schema) or CAD/packet +evidence. + The existing Phase 11 Cell feature remains partially hardware-verified and visible in "What's still open." It is deliberately scheduled as **V2 Workstream 16 — Cell closeout**, an optional post-core-V2 bonus: it does not diff --git a/docs/hardware-results/2026-09-04-phase12-focus-matrix.md b/docs/hardware-results/2026-09-04-phase12-focus-matrix.md index 5aa9e09..7d2263a 100644 --- a/docs/hardware-results/2026-09-04-phase12-focus-matrix.md +++ b/docs/hardware-results/2026-09-04-phase12-focus-matrix.md @@ -188,3 +188,77 @@ The `p90 >= -90 dBm` qualifying-condition candidate from the main matrix was measured under the old 8-sample policy and a source ~70 dB above ambient. It should be re-selected against this sampling and a weaker source before any operator-facing use. + + +--- + +# Follow-up 2: qualifying condition at a realistic signal level (120 trials) + +**Raw evidence:** `private/phase12-step2-20260905T035952Z.{jsonl,log}`. +120 trials (2 positions x 30 source-on + 30 source-off), 9.4 min, zero +transport errors, zero drops, home restored on every trial. 2,000 ms dwell at +the measured 20 ms sampling policy (101 samples per pass). + +The earlier matrix chose `p90 >= -90 dBm` as a candidate qualifying condition +while the source sat ~70 dB above ambient. This run repeats the question at a +level a field deployment might actually see: transmitter on an outdoor porch, +driven over the WiFi control bridge, receiver with its antenna fitted so the +ambient floor is real. + +**Configuration:** source-on peak reads about -85 dBm against a -99/-100 dBm +ambient floor, versus -24 dBm against -98 dBm on the bench. Receiver antenna +fitted; transmitter at roughly one building's distance with a small stubby +antenna. + +## The candidate does not survive + +| metric | low-aligned (908.75) | mid-aligned (918.5) | +|---|---|---| +| `p90` (the previous candidate) | overlap | overlap | +| `peak` | separates by **2.0 dB** | overlap | +| `peak - median` | separates by 2.0 dB | overlap (-2.0 dB) | +| `peak - p90` | 1.0 dB, 3 false positives in 30 | overlap (-3.0 dB) | +| `p90 - median` | overlap | overlap | + +One of ten combinations separates, by 2 dB. That is inside ordinary RSSI +variance and would not survive a temperature change, a different antenna, or a +different room. **`p90 >= -90 dBm` is rejected**, and so is every relative +(floor-referenced) variant tested here. Both families fail for the same +reason, so this is not an artifact of picking the wrong statistic. + +Two observations explain it: + +- **`p90` collapses at low SNR even when the source is present most of the + time.** At `low-aligned` the transmitter radiated 57-71% of the window, yet + `p90` still read -94 to -97 dBm against a -100 floor. If the received signal + were steady, `p90` would equal `peak`. It does not: at these levels the + instantaneous RSSI during a weak transmission mostly fails to stand above + noise, and only its best moments do. That is why `peak` outperforms every + other summary here, and why none of them is reliable. +- **Real traffic is indistinguishable from the controlled source.** At + `mid-aligned` (918.5 MHz, a live MeshOregon channel) source-off trials + reached -94 dBm. That is not measurement error; it is a real transmission + arriving during a control trial. An RSSI condition cannot tell a controlled + fixture from a neighbour's node, and should not be expected to. + +## What this means + +An RSSI *summary statistic* cannot carry an activity claim at realistic signal +levels. Coverage reporting is unaffected: valid passes, observation time, and +the median/P90/peak summary itself remain honest and useful. What the evidence +refuses is the inference from "RSSI was elevated" to "something transmitted" — +precisely the inference §3's contract was written to prevent. The design was +right; the constant was wrong. + +Two untested routes remain, in order of cost: + +1. **A count, not a statistic.** `peak` is one sample and therefore noise-prone; + counting how many of a pass's 101 samples exceed an adaptive floor + integrates instead. `FocusObservation` already reserves `qualifying_count` + for this and never populates it. Needs firmware, and can reuse this fixture. +2. **CAD or packet evidence**, which §3 already contemplates as the alternative + basis for observed activity. A different primitive, and a larger change. + +Until one of those is measured, `coverage` stays blank, the activity count +stays unpopulated, and Focus reports what it observed rather than what it +concludes. diff --git a/docs/research/phase12-survey-truth-design.md b/docs/research/phase12-survey-truth-design.md index 2f4599d..cc02150 100644 --- a/docs/research/phase12-survey-truth-design.md +++ b/docs/research/phase12-survey-truth-design.md @@ -351,12 +351,22 @@ local spectrum identity or coverage of the city. - [x] V2 boundaries, one-bin first slice, result vocabulary, fixed-statistics direction, controlled matrix, and Portland field-validation handling are recorded here. -- [~] Choose the qualifying RSSI condition and coverage thresholds from the - matrix rather than from a desk estimate. §6.4 supplies a *candidate* RSSI - condition (`p90 >= -90 dBm`) and, more importantly, shows why one fixed - number is not sufficient on its own: the condition's meaning depends on - sample spacing, and no single threshold separates all 900 trials. The - coverage thresholds are untouched by this matrix and stay open. +- [x] Choose the qualifying RSSI condition from measurement rather than a desk + estimate — **measured and rejected**. §6.4's `p90 >= -90 dBm` candidate came + from a source ~70 dB above ambient. Repeated at a realistic level (source + peak ~-85 dBm against a -99/-100 dBm floor, 120 trials), it fails, and so + does every floor-relative variant: one of ten metric/position combinations + separates, by 2 dB, which is inside ordinary RSSI variance. An RSSI summary + statistic cannot support an activity claim at field levels. See + [the evidence summary](../hardware-results/2026-09-04-phase12-focus-matrix.md). + Coverage reporting is unaffected; what is refused is the step from "RSSI was + elevated" to "something transmitted", which is the step §3 forbids anyway. + Two routes remain untested: a per-pass **count** of samples above an adaptive + floor (`qualifying_count` is already reserved in the schema and never + populated), and CAD/packet evidence as §3 already contemplates. +- [ ] Coverage thresholds (`sampled`/`repeated`) remain unselected. They are + about pass counts and accumulated time across repeated requests, which no + single-pass measurement can supply. - [ ] Approve the maximum radio-away budget after the Watch-opportunity result. - [x] Count actual static SRAM, stack frame, queue, row-length, and SD-rate budgets (§4.2). Counted after the vertical slice rather than before it, on From 0138a549e1dc3dd075723fe268b16e6bafd33fe0 Mon Sep 17 00:00:00 2001 From: Will Shields Date: Fri, 4 Sep 2026 21:52:24 -0700 Subject: [PATCH 10/21] Measure what Focus costs Watch, and find the cost is exactly its away time MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two 240 s arms against one independently timed reference train, transmitter outdoors on the WiFi bridge so the link is real rather than a bench-adjacent one. Watch alone received 0.883 of the train; Watch with Focus interleaved at a 48.1% away fraction received 0.463. The 95% intervals do not overlap. The useful part is the arithmetic. If Focus merely stops the receiver hearing anything while away and costs nothing else, the predicted reception is the baseline scaled by the time left at home: 0.883 x (1 - 0.481) = 0.459. Measured 0.463, well inside the interval. So the loss is accounted for entirely by away time -- no extra penalty from retuning, from recovery, or from any lingering effect after home listening is restored. That makes the away duration Focus already records an honest proxy for what a request costs, which is what would make an operator-facing away-time display meaningful rather than decorative. It equally means there is no mitigation: half the listening time away is half the packets, and time between restored Watch windows reached 7.45 s. The budget decision itself is deliberately not made here. §6.3 requires it to be an operator product judgement taken after the measurement, not implied by it; this supplies the exchange rate and stops there. Limits are recorded with the evidence: a 0.883 baseline rather than a perfect link, one duty cycle at one dwell, and a fixed-interval fixture rather than real bursty traffic. Also hardens the harness's timeouts the way the matrix runner was hardened, after native USB-CDC truncation cost that run an abort. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 12 ++++ docs/STATUS.md | 11 ++++ .../2026-09-04-phase12-focus-matrix.md | 61 +++++++++++++++++++ docs/research/phase12-survey-truth-design.md | 23 +++++-- scripts/phase12_watch_opportunity.py | 8 ++- 5 files changed, 107 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fbeb792..a0fe299 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,18 @@ project (not a log of how it got there), see [docs/STATUS.md](docs/STATUS.md). ## 2026-09-05 +- Ran Workstream 12's §6.3 Watch-opportunity comparison. Two 240 s arms + against one independently timed reference train: Watch alone received 0.883, + Watch with Focus interleaved at a 48.1% away fraction received 0.463, + intervals non-overlapping, zero CRC errors either side. The loss is + proportional to away time and nothing else — 0.459 predicted from the + baseline against 0.463 measured — which means Focus's recorded away duration + is an honest proxy for what a request costs Watch, with no hidden retune or + recovery penalty. It also means there is no mitigation: half the listening + time away is half the packets, with restored-Watch gaps reaching 7.45 s. The + away-time budget decision stays open on purpose; the measurement gives the + exchange rate, not the policy. + - Measured Focus's qualifying RSSI condition at a realistic signal level and **rejected it**. The `p90 >= -90 dBm` candidate came from a bench source ~70 dB above ambient, where every metric separates and the choice looks easy. diff --git a/docs/STATUS.md b/docs/STATUS.md index c3f4098..4786817 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -108,6 +108,17 @@ unpopulated, and the remaining routes are a per-pass count above an adaptive floor (`qualifying_count`, already reserved in the schema) or CAD/packet evidence. +The §6.3 Watch-opportunity comparison also completed: two 240 s arms against +one independently timed reference train. Watch alone received 0.883 of the +train; with Focus interleaved at a 48.1% away fraction it received 0.463, with +non-overlapping 95% intervals. The loss is proportional to away time and +nothing more (0.459 predicted against 0.463 measured), so **Focus's recorded +radio-away duration is an honest proxy for what a request costs Watch** — no +hidden retune or recovery penalty — and equally, there is no mitigation: half +the time away is half the packets. Time between restored Watch windows ran to +a 7.45 s maximum. The maximum away-time budget is deliberately still +unapproved; the measurement supplies the exchange rate, not the policy. + The existing Phase 11 Cell feature remains partially hardware-verified and visible in "What's still open." It is deliberately scheduled as **V2 Workstream 16 — Cell closeout**, an optional post-core-V2 bonus: it does not diff --git a/docs/hardware-results/2026-09-04-phase12-focus-matrix.md b/docs/hardware-results/2026-09-04-phase12-focus-matrix.md index 7d2263a..284e728 100644 --- a/docs/hardware-results/2026-09-04-phase12-focus-matrix.md +++ b/docs/hardware-results/2026-09-04-phase12-focus-matrix.md @@ -262,3 +262,64 @@ Two untested routes remain, in order of cost: Until one of those is measured, `coverage` stays blank, the activity count stays unpopulated, and Focus reports what it observed rather than what it concludes. + + +--- + +# Follow-up 3: Watch-opportunity comparison (§6.3) + +**Raw evidence:** `private/phase12-watch-20260905T042817Z.{jsonl,log}`. +Two 240 s arms against one independently timed reference train (`MESH_OREGON`, +918.5 MHz SF8/BW125, one pulse every 2 s), transmitter outdoors on the WiFi +control bridge, receiver on its resolved home channel. Zero CRC errors in +either arm. Focus requests were 2,000 ms dwell at the measured 20 ms sampling +policy (101 samples). + +| arm | reference | received | fraction | 95% CI | +|---|---|---|---|---| +| Watch only | 120 | 106 | 0.883 | [0.814, 0.929] | +| Watch + Focus | 121 | 56 | **0.463** | [0.377, 0.551] | + +Focus completed 56 requests, was refused none, and held the radio for +116,142 ms — **48.1% of its arm**. Time between restored Watch windows: +median 4.04 s, maximum 7.45 s. The intervals do not overlap, so the +difference is not a small-sample artifact. + +## The cost is proportional, with nothing hidden + +If Focus simply stops the receiver hearing anything while away, and costs +nothing else, the predicted reception is the baseline scaled by the time +remaining at home: `0.883 x (1 - 0.481) = 0.459`. Measured: **0.463**. The +difference is well inside the interval. + +That is the useful part of this result. It says the loss is entirely +accounted for by away time — there is no additional penalty from retuning, +from recovery, or from any lingering effect once home listening is restored. +**The radio-away duration Focus already records is an honest proxy for what a +request costs Watch**, which is what makes an operator-facing away-time +display meaningful rather than decorative. + +The other half is that there is no mitigation either. Half the listening time +away is half the packets. A displayed cost is still a cost. + +## Limits + +- The baseline is 0.883, not 1.0: about 12% of reference pulses were missed + with no Focus running. These are relative figures against a real link, not + absolute capture rates, and the comparison is valid only because both arms + saw the same link and the same train. +- One duty cycle (48%) at one dwell (2,000 ms) at one pulse interval (2 s). + Proportionality held here; it is not established across other duty cycles, + and a much shorter dwell has proportionally more retune overhead per unit + of observation. +- The reference train is a controlled fixture, not real mesh traffic. Real + traffic is bursty and correlated in ways a fixed 2 s interval is not. + +## What it does not decide + +A maximum radio-away budget is an operator product decision, and this +measurement deliberately does not make it. What it supplies is the exchange +rate: **at this dwell, Watch packet opportunity falls in direct proportion to +the fraction of time Focus holds the radio.** Choosing what fraction is +acceptable — and whether Focus should bound it automatically rather than +leaving it to whoever is pressing the button — remains open. diff --git a/docs/research/phase12-survey-truth-design.md b/docs/research/phase12-survey-truth-design.md index cc02150..5ac5a7f 100644 --- a/docs/research/phase12-survey-truth-design.md +++ b/docs/research/phase12-survey-truth-design.md @@ -367,7 +367,17 @@ local spectrum identity or coverage of the city. - [ ] Coverage thresholds (`sampled`/`repeated`) remain unselected. They are about pass counts and accumulated time across repeated requests, which no single-pass measurement can supply. -- [ ] Approve the maximum radio-away budget after the Watch-opportunity result. +- [~] Approve the maximum radio-away budget. **The measurement is complete** + ([evidence](../hardware-results/2026-09-04-phase12-focus-matrix.md)): at a + 2,000 ms dwell and 48.1% away fraction, Watch reception fell from 0.883 to + 0.463 of a reference train, with non-overlapping 95% intervals. The loss is + proportional to away time and nothing else — predicted 0.459 against a + measured 0.463 — so Focus's recorded away duration is an honest proxy for + what a request costs, and there is no hidden retune or recovery penalty. + **The budget decision itself remains open**, deliberately: this supplies the + exchange rate, not the policy, and §6.3 requires the decision to be an + operator product judgement made after the measurement rather than implied + by it. - [x] Count actual static SRAM, stack frame, queue, row-length, and SD-rate budgets (§4.2). Counted after the vertical slice rather than before it, on the built image instead of on paper; the numbers land inside the bounds this @@ -437,12 +447,13 @@ local spectrum identity or coverage of the city. zero queue or row drops (`FW` 0->6, `FD`/`FL` 0), which is the logger's post-write counter, not an enqueue count. Timing evidence is the receiver's own; it is not a coverage, calibration, or activity claim. -- [~] The controlled matrix is complete with durable SD evidence and a - location-redacted summary +- [x] The controlled matrix and the Watch-opportunity comparison are both + complete, with durable evidence and a location-redacted summary ([2026-09-04-phase12-focus-matrix.md](../hardware-results/2026-09-04-phase12-focus-matrix.md)). - The Watch-opportunity comparison has tooling - (`scripts/phase12_watch_opportunity.py`) but has not run, so the maximum - radio-away budget stays unapproved. + Between them they settled the sampling policy, rejected the qualifying RSSI + condition, and measured Watch's cost. What they left open is recorded above: + the away-time budget decision, the coverage thresholds, and an activity + basis that is not an RSSI summary. - [ ] WiFi-off/on resource matrix, Portland field validation, `STATUS.md`, `LOG_GUIDE.md`, release notes, and any companion-schema update reconcile. diff --git a/scripts/phase12_watch_opportunity.py b/scripts/phase12_watch_opportunity.py index 04a66fb..a772939 100644 --- a/scripts/phase12_watch_opportunity.py +++ b/scripts/phase12_watch_opportunity.py @@ -111,8 +111,11 @@ def run_arm(card, results, label, duration_s, train, focus_request): time.sleep(0.25) continue bin_index, dwell_ms, samples = focus_request + # Generous like the matrix runner's: native USB-CDC occasionally + # truncates a reply under sustained round trips, and the command is + # idempotent on the device, so re-sends cannot start a second survey. opcode, payload = card.request("BENCH_FOCUS", f"{bin_index}:{dwell_ms}:{samples}", - timeout=5.0) + timeout=12.0) if opcode != "ACK": focus_refused += 1 continue @@ -121,9 +124,10 @@ def run_arm(card, results, label, duration_s, train, focus_request): status = card_status(card) if status.get("FS") == "3": break + time.sleep(0.2) else: raise RuntimeError("a Focus request did not reach a terminal state in this arm") - result = parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "HEALTH")) + result = parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "HEALTH", timeout=12.0)) if result.get("HR") != "1": raise RuntimeError(f"Focus failed to restore home listening: {result}") focus_completed += 1 From 8128be71a9cf5bf942b5d3cc959e1a24a0a754e0 Mon Sep 17 00:00:00 2001 From: Will Shields Date: Fri, 4 Sep 2026 22:27:26 -0700 Subject: [PATCH 11/21] Count samples above an adaptive floor; it works where statistics did not Every RSSI summary statistic was rejected at field levels, and the follow-up named one untested idea: `peak` is a single sample and therefore noise-prone, so count the samples above an adaptive floor instead of taking an extreme of them. A pass with one stray strong sample and a genuinely busy pass have the same peak; only a count tells them apart, and there is a host test for exactly that case. Bench images now retain the pass histogram and report a ladder of counts at median + 2/4/6/8/10/15/20 dB. That is deliberate: it lets one run evaluate any candidate rule offline rather than reflashing per guess, and it keeps the rule relative to each pass's own floor, since an absolute threshold is precisely what the previous measurement rejected. Across 120 trials at the same porch configuration, `C6 >= 2` -- two or more samples at the pass median plus 6 dB -- detected 57/60 source-on trials, 95% CI [0.863, 0.983], against 1/60 source-off. `C4 >= 5` performs identically, so the ladder has a plateau rather than one lucky point. The single flagged control trial read -63 dBm against a -101 dBm median with six elevated samples. Nothing 38 dB above its own noise floor is noise: that is a real transmission that was not ours, correctly caught by the detector and mislabelled by an experiment that cannot tell "our fixture is quiet" from "the band is quiet". The 1.7% false rate is an upper bound, not a measurement against silence. The three misses are all within a couple of dB of the floor, which is where a detector should miss. This is a candidate, not a constant, and `qualifying_count` stays unpopulated. The threshold is a count out of 101 samples, and the sampling policy now scales sample count with dwell, so two samples is 2% of a 2,000 ms pass and 33% of a 100 ms one. Expressing it as a fraction of accepted samples, or validating it per dwell, is the next measurement rather than a decision to take at a keyboard. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 15 ++++ docs/STATUS.md | 11 +++ .../2026-09-04-phase12-focus-matrix.md | 75 +++++++++++++++++++ docs/research/phase12-survey-truth-design.md | 15 +++- scripts/phase12_focus_matrix.py | 19 ++++- src/focus_observation.h | 34 +++++++++ src/radio_task.cpp | 22 ++++++ src/radio_task.h | 4 + src/serial_control.cpp | 32 ++++++++ src/serial_control_protocol.h | 5 ++ test/test_focus_observation/test_main.cpp | 33 ++++++++ 11 files changed, 259 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a0fe299..a4e8b08 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,21 @@ project (not a log of how it got there), see [docs/STATUS.md](docs/STATUS.md). ## 2026-09-05 +- Found an activity basis that survives field levels. After every RSSI summary + statistic was rejected, the remaining idea was to count samples above an + adaptive floor rather than take an extreme of them — `peak` is one sample and + noise-prone; a count integrates. Bench images now report a ladder of counts + at median + 2/4/6/8/10/15/20 dB, so one run evaluates any such rule offline + instead of reflashing per candidate. Across 120 trials at the same porch + configuration, `C6 >= 2` detected 57/60 source-on (95% CI [0.863, 0.983]) + against 1/60 source-off — and that one flagged control trial read -63 dBm + against a -101 dBm median, a real transmission that was not ours, so the + false rate is an upper bound rather than a measurement against silence. + Still a candidate, not a constant: the threshold is a count out of 101 + samples, and since sample count now scales with dwell, "2 samples" is 2% at + 2,000 ms and 33% at 100 ms. `qualifying_count` stays unpopulated until that + is expressed as a fraction or validated per dwell. + - Ran Workstream 12's §6.3 Watch-opportunity comparison. Two 240 s arms against one independently timed reference train: Watch alone received 0.883, Watch with Focus interleaved at a 48.1% away fraction received 0.463, diff --git a/docs/STATUS.md b/docs/STATUS.md index 4786817..da213aa 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -119,6 +119,17 @@ the time away is half the packets. Time between restored Watch windows ran to a 7.45 s maximum. The maximum away-time budget is deliberately still unapproved; the measurement supplies the exchange rate, not the policy. +A per-pass **count** above an adaptive floor then succeeded where the summary +statistics failed. Counting samples at or above the pass's own median plus +6 dB, a threshold of two detected 57/60 source-on trials (95% CI +[0.863, 0.983]) at the same field-level configuration, with a single flagged +control trial that read -63 dBm against a -101 dBm median — a real +transmission that was not ours, so the 1.7% false rate is an upper bound. It +is a candidate, not a constant: the threshold is a count out of 101 samples +and the sampling policy scales sample count with dwell, so it must become a +fraction of accepted samples or be validated per dwell first. +`qualifying_count` stays unpopulated until then. + The existing Phase 11 Cell feature remains partially hardware-verified and visible in "What's still open." It is deliberately scheduled as **V2 Workstream 16 — Cell closeout**, an optional post-core-V2 bonus: it does not diff --git a/docs/hardware-results/2026-09-04-phase12-focus-matrix.md b/docs/hardware-results/2026-09-04-phase12-focus-matrix.md index 284e728..a9ada37 100644 --- a/docs/hardware-results/2026-09-04-phase12-focus-matrix.md +++ b/docs/hardware-results/2026-09-04-phase12-focus-matrix.md @@ -323,3 +323,78 @@ rate: **at this dwell, Watch packet opportunity falls in direct proportion to the fraction of time Focus holds the radio.** Choosing what fraction is acceptable — and whether Focus should bound it automatically rather than leaving it to whoever is pressing the button — remains open. + + +--- + +# Follow-up 4: a count survives where every summary statistic failed + +**Raw evidence:** `private/phase12-counts-20260905T050037Z.{jsonl,log}`. +120 trials (2 positions x 30 source-on + 30 source-off), 11.4 min, zero +transport errors, zero drops, home restored on every trial. Same porch +configuration and 2,000 ms / 101-sample passes as Follow-up 2, which rejected +every RSSI summary statistic at this signal level. + +Follow-up 2 ended by naming one untested idea: `peak` is a single sample and +therefore noise-prone, so **count** the samples above an adaptive floor rather +than taking an extreme of them. The bench image now reports a ladder of counts +at median + 2/4/6/8/10/15/20 dB, so one run evaluates any such rule offline. + +## Result + +Pooled across both positions, best operating point **`C6 >= 2`** — at least two +of a pass's samples at or above that pass's own median plus 6 dB: + +| | count | 95% CI | +|---|---|---| +| detected, source on | **57/60 (95.0%)** | [0.863, 0.983] | +| detected, source off | **1/60 (1.7%)** | [0.003, 0.089] | + +Per position: + +| position | detect | false | +|---|---|---| +| low-aligned (908.75) | 30/30 | 1/30 | +| mid-aligned (918.5) | 27/30 | 0/30 | + +`C4 >= 5` performs identically (57/60 and 1/60); the ladder has a broad +plateau rather than one lucky point, which is what a real effect looks like. + +For contrast, at this same signal level Follow-up 2 found `p90` and `peak`, +absolute and floor-relative, could not separate the two populations at all. + +## The one false positive is not a false positive + +The single source-off trial flagged by `C6 >= 2` read `peak = -63 dBm` against +a -101 dBm median — 38 dB above its own floor, with six elevated samples. +Nothing 38 dB over noise is noise. That is a real transmission that was not +ours, correctly identified by the detector and mislabelled by the experiment, +which cannot distinguish "our fixture is quiet" from "the band is quiet". The +true false-alarm rate against genuine silence may therefore be 0/60; this +evidence cannot separate the two, and the reported 1.7% should be read as an +upper bound. + +The three misses are all mid-aligned at `peak = -95/-96 dBm`, within a couple +of dB of the floor. A detector that misses signals at the noise floor is +behaving correctly; one that claimed them would be the problem. + +## What still has to be settled before this is a constant + +**The threshold is a count out of 101 samples — about 2% — and the sample +count now scales with dwell.** Under the measured 20 ms sampling policy a +100 ms pass takes 6 samples, where "2 samples" is 33% rather than 2%. A fixed +count therefore means something completely different at another dwell. The +rule must be expressed as a fraction of accepted samples, or validated +separately per dwell, before it can be written into firmware. That is the next +measurement, not a detail to decide at a keyboard. + +Also unestablished: one distance, one indoor/outdoor geometry, two +frequencies, one 2,000 ms dwell. And the margin is relative to the pass's own +median, which assumes the median represents the floor — true when the source +is present for a minority of samples, and less true as occupancy rises toward +50%. + +Until those close, `qualifying_count` stays unpopulated and `coverage` stays +blank. What this run establishes is narrower and still worth having: **an +adaptive count is a viable basis for an activity claim where every summary +statistic measured here was not.** diff --git a/docs/research/phase12-survey-truth-design.md b/docs/research/phase12-survey-truth-design.md index 5ac5a7f..0dd977c 100644 --- a/docs/research/phase12-survey-truth-design.md +++ b/docs/research/phase12-survey-truth-design.md @@ -361,9 +361,18 @@ local spectrum identity or coverage of the city. [the evidence summary](../hardware-results/2026-09-04-phase12-focus-matrix.md). Coverage reporting is unaffected; what is refused is the step from "RSSI was elevated" to "something transmitted", which is the step §3 forbids anyway. - Two routes remain untested: a per-pass **count** of samples above an adaptive - floor (`qualifying_count` is already reserved in the schema and never - populated), and CAD/packet evidence as §3 already contemplates. + The count route named there has since been **measured and survives**: a + per-pass count of samples above the pass's own median separates where every + summary statistic failed. `C6 >= 2` (two or more samples at median + 6 dB) + detected 57/60 source-on trials, 95% CI [0.863, 0.983], with one flagged + source-off trial that read -63 dBm against a -101 dBm median — a real + transmission, not ours, so the 1.7% false rate is an upper bound rather than + a measurement against true silence. It is a **viable candidate, not yet a + constant**: the threshold is a count out of 101 samples, and the sampling + policy now scales sample count with dwell, so "2 samples" means 2% at a + 2,000 ms pass and 33% at a 100 ms one. Expressing it as a fraction of + accepted samples, or validating per dwell, is the next measurement. + CAD/packet evidence remains the alternative basis §3 contemplates. - [ ] Coverage thresholds (`sampled`/`repeated`) remain unselected. They are about pass counts and accumulated time across repeated requests, which no single-pass measurement can supply. diff --git a/scripts/phase12_focus_matrix.py b/scripts/phase12_focus_matrix.py index 1e1d81f..d002841 100644 --- a/scripts/phase12_focus_matrix.py +++ b/scripts/phase12_focus_matrix.py @@ -160,7 +160,18 @@ def run_focus_request(card, spec, dwell_ms, samples, start): max(12.0, dwell_ms / 1000.0 + 10.0), "Focus completion, home restore, and focus.csv commit", ) - return terminal, parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "-", timeout=12.0)) + result = parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "-", timeout=12.0)) + # Counts above the pass's own median, laddered, so any adaptive activity + # rule can be evaluated offline from one run. Optional: a bench image + # without it simply reports nothing and the trial row omits the counts. + counts = {} + try: + opcode, payload = card.request("BENCH_FOCUS_COUNTS", "-", timeout=12.0) + if opcode == "ACK": + counts = parse_fields(payload) + except (RuntimeError, TimeoutError): + pass + return terminal, result, counts def run_trial(card, transmitter, position, dwell_ms, samples, source_on, gap_ms): @@ -172,12 +183,12 @@ def run_trial(card, transmitter, position, dwell_ms, samples, source_on, gap_ms) pulses = 0 if source_on: with PulseBurst(transmitter, gap_ms / 1000.0) as burst: - terminal, result = run_focus_request(card, spec, dwell_ms, samples, start) + terminal, result, counts = run_focus_request(card, spec, dwell_ms, samples, start) pulses = burst.fired if burst.fired == 0: raise RuntimeError("the transmitter never fired during a source-on trial") else: - terminal, result = run_focus_request(card, spec, dwell_ms, samples, start) + terminal, result, counts = run_focus_request(card, spec, dwell_ms, samples, start) # §6.2: an arm is rejected if it loses ownership, cannot restore home, or # has unexplained queue/row drops. Fail the trial, not the whole matrix's @@ -215,6 +226,8 @@ def run_trial(card, transmitter, position, dwell_ms, samples, source_on, gap_ms) "home_restore": result["HR"], "radio_status": int(result["E"]), "pulses_fired": pulses, + # C = samples at or above the pass's median + N dB. + "counts_above_median": {k: int(v) for k, v in counts.items() if k.startswith("C")}, } diff --git a/src/focus_observation.h b/src/focus_observation.h index cee292b..54cd328 100644 --- a/src/focus_observation.h +++ b/src/focus_observation.h @@ -102,6 +102,40 @@ inline int16_t focusHistogramP90DbmX10(const FocusRssiHistogram &histogram) { return focusHistogramPercentileDbmX10(histogram, 90); } +// How many accepted samples read at or above `threshold_dbm_x10`. +// +// A count is a different instrument from a summary statistic. `peak` is one +// sample and therefore noise-prone; a count integrates over the pass, which +// is what the 2026-09-05 field-level measurement suggested was missing when +// every summary statistic failed to separate a known source from ambient +// (docs/hardware-results/2026-09-04-phase12-focus-matrix.md). Pure, so the +// rule can be evaluated on the host before any of it is trusted on-device. +inline uint16_t focusHistogramCountAtLeast(const FocusRssiHistogram &histogram, + int16_t threshold_dbm_x10) { + const int16_t threshold_dbm = focusRssiDbmX10ToNearestDbm(threshold_dbm_x10); + if (threshold_dbm > FOCUS_RSSI_HISTOGRAM_MAX_DBM) return 0; + const int16_t lowest = threshold_dbm < FOCUS_RSSI_HISTOGRAM_MIN_DBM + ? FOCUS_RSSI_HISTOGRAM_MIN_DBM + : threshold_dbm; + uint16_t total = 0; + for (uint16_t i = (uint16_t)(lowest - FOCUS_RSSI_HISTOGRAM_MIN_DBM); + i < FOCUS_RSSI_HISTOGRAM_BUCKET_COUNT; ++i) { + total = (uint16_t)(total + histogram.bucket_counts[i]); + } + return total; +} + +// The same count expressed relative to the pass's own median, so the rule +// adapts to the ambient floor where it ran instead of assuming one. An +// absolute threshold was measured and rejected at field levels; this is the +// candidate that replaced it, and it is still a candidate. +inline uint16_t focusHistogramCountAboveMedian(const FocusRssiHistogram &histogram, + int16_t margin_dbm_x10) { + const int16_t median = focusHistogramMedianDbmX10(histogram); + if (median == FOCUS_RSSI_NO_SAMPLE_DBM_X10) return 0; + return focusHistogramCountAtLeast(histogram, (int16_t)(median + margin_dbm_x10)); +} + enum class FocusRequestStatus : uint8_t { COMPLETE = 0, CANCELLED, diff --git a/src/radio_task.cpp b/src/radio_task.cpp index 8b7d179..0481800 100644 --- a/src/radio_task.cpp +++ b/src/radio_task.cpp @@ -54,6 +54,12 @@ MissionProfile activeProfile = MissionProfile::MESHTASTIC; portMUX_TYPE focusResultMux = portMUX_INITIALIZER_UNLOCKED; FocusObservation lastFocusObservation; bool haveLastFocusObservation = false; +#if defined(LORATRACE_BENCH_FAULTS) +// Bench images retain the pass's full histogram so a host can evaluate any +// candidate activity rule offline, from one run, instead of reflashing per +// guess. Production keeps only the fixed summary that reaches focus.csv. +FocusRssiHistogram lastFocusHistogram; +#endif // Both writers and readers go through these so the locking cannot be // forgotten at a call site. Radio-task-internal code that already knows it @@ -715,6 +721,9 @@ void enqueueFocusObservation(const FocusRequest &request, const FocusRssiHistogr observation.radio_status = operationStatus; portENTER_CRITICAL(&focusResultMux); lastFocusObservation = observation; +#if defined(LORATRACE_BENCH_FAULTS) + lastFocusHistogram = histogram; +#endif haveLastFocusObservation = true; portEXIT_CRITICAL(&focusResultMux); focusObservationCount++; @@ -2482,6 +2491,19 @@ FocusRuntimeState radioFocusSurveyState() { return focusState; } uint32_t radioFocusLastAwayMs() { return focusLastAwayMs; } uint32_t radioFocusObservationCount() { return focusObservationCount; } uint32_t radioFocusObservationDropCount() { return focusObservationDropCount; } +bool radioFocusLastHistogram(FocusRssiHistogram &out) { +#if !defined(LORATRACE_BENCH_FAULTS) + (void)out; + return false; +#else + portENTER_CRITICAL(&focusResultMux); + const bool available = haveLastFocusObservation; + if (available) out = lastFocusHistogram; + portEXIT_CRITICAL(&focusResultMux); + return available; +#endif +} + bool radioFocusLastObservation(FocusObservation &out) { portENTER_CRITICAL(&focusResultMux); const bool available = haveLastFocusObservation; diff --git a/src/radio_task.h b/src/radio_task.h index 824a0bc..27abec4 100644 --- a/src/radio_task.h +++ b/src/radio_task.h @@ -252,6 +252,10 @@ uint32_t radioFocusObservationDropCount(); // Copies the last completed Focus result under a short cross-core lock. // It contains no GPS/run fields; Core 0 alone adds those while writing CSV. bool radioFocusLastObservation(FocusObservation &out); +// Bench images only: the last pass's full RSSI histogram, so a host can test +// candidate activity rules offline rather than one reflash per candidate. +// Production always returns false. +bool radioFocusLastHistogram(FocusRssiHistogram &out); // Phase 9 Pass B (research/phase9-sweep-pass-b-design.md): CAD attempts run // at the first PASS_B_MAX_PEAKS_PER_SWEEP Pass-A peaks this sweep, and how diff --git a/src/serial_control.cpp b/src/serial_control.cpp index 8fd7e37..9265ddf 100644 --- a/src/serial_control.cpp +++ b/src/serial_control.cpp @@ -429,6 +429,38 @@ void handleFrame(const SerialControlFrame &frame) { sendFrame(frame.sequence, SerialControlOpcode::ACK, argument); break; } + case SerialControlOpcode::BENCH_FOCUS_COUNTS: { + // A ladder of counts above the pass's own median, so one run lets + // a host evaluate any adaptive activity rule offline instead of + // reflashing per candidate margin. Relative, not absolute: an + // absolute threshold was measured and rejected at field levels + // (docs/hardware-results/2026-09-04-phase12-focus-matrix.md). + if (!benchFocusSurveyTriggerAllowed()) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "UNSUPPORTED"); + break; + } + FocusRssiHistogram histogram; + if (!radioFocusLastHistogram(histogram)) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "NO_RESULT"); + break; + } + static const int16_t MARGINS_DBM_X10[] = {20, 40, 60, 80, 100, 150, 200}; + char argument[SERIAL_CONTROL_ARGUMENT_MAX + 1] = {}; + int written = snprintf(argument, sizeof(argument), "N=%u;MED=%d;P90=%d;MAX=%d", + (unsigned)histogram.sample_count, + (int)focusHistogramMedianDbmX10(histogram), + (int)focusHistogramP90DbmX10(histogram), + (int)histogram.peak_dbm_x10); + for (size_t i = 0; i < sizeof(MARGINS_DBM_X10) / sizeof(MARGINS_DBM_X10[0]); ++i) { + if (written < 0 || (size_t)written >= sizeof(argument)) break; + written += snprintf(argument + written, sizeof(argument) - (size_t)written, + ";C%d=%u", (int)(MARGINS_DBM_X10[i] / 10), + (unsigned)focusHistogramCountAboveMedian(histogram, + MARGINS_DBM_X10[i])); + } + sendFrame(frame.sequence, SerialControlOpcode::ACK, argument); + break; + } case SerialControlOpcode::BENCH_ACTION: { // Cell and Scope are menu-only actions in production, so a fixture // has no other way to prove Focus refuses them and is refused by diff --git a/src/serial_control_protocol.h b/src/serial_control_protocol.h index 241e3a9..a995f7b 100644 --- a/src/serial_control_protocol.h +++ b/src/serial_control_protocol.h @@ -23,6 +23,8 @@ constexpr size_t SERIAL_CONTROL_FRAME_MAX = 384; // budget has to account for the rest. Derived, not eyeballed: the previous // 240-byte STATUS buffer was set by hand and silently exceeded its frame. constexpr size_t SERIAL_CONTROL_OPCODE_NAME_MAX = 23; // BENCH_PASS_B_CAD_RESULT +// A host test walks every opcode name against this, so adding a longer one +// fails there rather than silently overrunning buffers sized from it. constexpr size_t SERIAL_CONTROL_FRAME_FIXED_BYTES = 8 /* "@LTRX/1 " */ + 5 /* sequence */ + 1 + SERIAL_CONTROL_OPCODE_NAME_MAX + 1 + 5 /* " " + 4 CRC digits */; @@ -58,6 +60,7 @@ enum class SerialControlOpcode : uint8_t { BENCH_FOCUS, BENCH_FOCUS_CANCEL, BENCH_FOCUS_RESULT, + BENCH_FOCUS_COUNTS, BENCH_ACTION, KEY_DUMP, SD_RETRY, @@ -108,6 +111,7 @@ inline const char *serialControlOpcodeName(SerialControlOpcode opcode) { case SerialControlOpcode::BENCH_FOCUS: return "BENCH_FOCUS"; case SerialControlOpcode::BENCH_FOCUS_CANCEL: return "BENCH_FOCUS_CANCEL"; case SerialControlOpcode::BENCH_FOCUS_RESULT: return "BENCH_FOCUS_RESULT"; + case SerialControlOpcode::BENCH_FOCUS_COUNTS: return "BENCH_FOCUS_COUNTS"; case SerialControlOpcode::BENCH_ACTION: return "BENCH_ACTION"; case SerialControlOpcode::KEY_DUMP: return "KEY_DUMP"; case SerialControlOpcode::SD_RETRY: return "SD_RETRY"; @@ -142,6 +146,7 @@ inline SerialControlOpcode serialControlOpcodeFromName(const char *name) { if (strcmp(name, "BENCH_FOCUS") == 0) return SerialControlOpcode::BENCH_FOCUS; if (strcmp(name, "BENCH_FOCUS_CANCEL") == 0) return SerialControlOpcode::BENCH_FOCUS_CANCEL; if (strcmp(name, "BENCH_FOCUS_RESULT") == 0) return SerialControlOpcode::BENCH_FOCUS_RESULT; + if (strcmp(name, "BENCH_FOCUS_COUNTS") == 0) return SerialControlOpcode::BENCH_FOCUS_COUNTS; if (strcmp(name, "BENCH_ACTION") == 0) return SerialControlOpcode::BENCH_ACTION; if (strcmp(name, "KEY_DUMP") == 0) return SerialControlOpcode::KEY_DUMP; if (strcmp(name, "SD_RETRY") == 0) return SerialControlOpcode::SD_RETRY; diff --git a/test/test_focus_observation/test_main.cpp b/test/test_focus_observation/test_main.cpp index 68308f6..3871766 100644 --- a/test/test_focus_observation/test_main.cpp +++ b/test/test_focus_observation/test_main.cpp @@ -141,6 +141,37 @@ void test_worst_case_row_fits_the_logger_buffer_with_margin() { TEST_ASSERT_TRUE(n <= 200); } +void test_sample_counts_integrate_where_a_peak_would_not() { + FocusRssiHistogram histogram; + // A quiet pass with one stray strong sample: peak says -40, but only one + // sample in 50 is anywhere near it. A count is what tells those apart. + for (int i = 0; i < 49; ++i) focusHistogramAddSample(histogram, -1000); + focusHistogramAddSample(histogram, -400); + TEST_ASSERT_EQUAL_INT16(-400, histogram.peak_dbm_x10); + TEST_ASSERT_EQUAL_UINT16(50, focusHistogramCountAtLeast(histogram, -1000)); + TEST_ASSERT_EQUAL_UINT16(1, focusHistogramCountAtLeast(histogram, -900)); + TEST_ASSERT_EQUAL_UINT16(1, focusHistogramCountAboveMedian(histogram, 60)); + TEST_ASSERT_EQUAL_UINT16(0, focusHistogramCountAtLeast(histogram, -300)); + + // A genuinely busy pass reads the same peak but many elevated samples. + FocusRssiHistogram busy; + for (int i = 0; i < 25; ++i) focusHistogramAddSample(busy, -1000); + for (int i = 0; i < 25; ++i) focusHistogramAddSample(busy, -400); + TEST_ASSERT_EQUAL_INT16(-400, busy.peak_dbm_x10); + TEST_ASSERT_EQUAL_UINT16(25, focusHistogramCountAboveMedian(busy, 60)); +} + +void test_count_thresholds_clamp_outside_the_histogram_range() { + FocusRssiHistogram histogram; + for (int i = 0; i < 10; ++i) focusHistogramAddSample(histogram, -1000); + // Below the range counts everything; above it counts nothing. + TEST_ASSERT_EQUAL_UINT16(10, focusHistogramCountAtLeast(histogram, -2000)); + TEST_ASSERT_EQUAL_UINT16(0, focusHistogramCountAtLeast(histogram, 100)); + // No samples at all must not report a phantom count. + FocusRssiHistogram empty; + TEST_ASSERT_EQUAL_UINT16(0, focusHistogramCountAboveMedian(empty, 60)); +} + int main(int argc, char **argv) { UNITY_BEGIN(); RUN_TEST(test_focus_working_state_stays_below_one_result_budget); @@ -152,5 +183,7 @@ int main(int argc, char **argv) { RUN_TEST(test_focus_csv_without_fix_or_samples_keeps_unknown_values_blank); RUN_TEST(test_focus_csv_truncation_is_reported); RUN_TEST(test_worst_case_row_fits_the_logger_buffer_with_margin); + RUN_TEST(test_sample_counts_integrate_where_a_peak_would_not); + RUN_TEST(test_count_thresholds_clamp_outside_the_histogram_range); return UNITY_END(); } From a4bc3351a88f2ab755b6e13f5fd07fb15c80ea13 Mon Sep 17 00:00:00 2001 From: Will Shields Date: Fri, 4 Sep 2026 23:18:33 -0700 Subject: [PATCH 12/21] Bound the count rule: a fraction of samples, and occupancy is the limit MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two sweeps, 480 trials, no arm failures, same porch configuration. The dwell sweep answers whether the rule is a count or a fraction. Roughly 4-5% of accepted samples transfers between 500 ms and 2,000 ms passes, so it is a fraction. It does not rescue a 100 ms pass: with six samples no threshold both catches the source and rejects ambient, at any margin. That is a floor on the instrument rather than a tuning problem, and it means a short pass may report coverage honestly while reporting no activity at all. The occupancy sweep matters more, because it bounds what the rule can claim. Holding dwell fixed and varying only the source's duty within the pass, detection runs 90-93% at 43-57% occupancy and collapses to 37-43% at 28.6%. A single SF8 packet inside a 2,000 ms pass is 3-7% occupancy -- far below where this already misses more than half the time. So the rule detects a persistently occupied channel; it does not detect individual packets. That is not a defect to tune away, it is the honest scope of sampling energy rather than detecting a signal, and it makes CAD or packet reception the better-supported basis for §3's observed activity. §3 has contemplated that alternative from the start; it is now an evidenced preference rather than a hunch. Median-as-floor survived every occupancy tested -- the pass median read -101 dBm for source-on and source-off alike. The reason bounds the result: at these levels most samples read like noise even while the transmitter radiates, the same effect that made p90 useless. A strong source at the same occupancy would pull the median up and break the assumption, and this sweep did not reach that regime or license any claim about it. qualifying_count stays unpopulated. What to populate it with is now a design decision with evidence behind it. Co-Authored-By: Claude Opus 5 --- .../2026-09-04-phase12-focus-matrix.md | 79 +++++++++++++++++++ scripts/phase12_focus_matrix.py | 3 + 2 files changed, 82 insertions(+) diff --git a/docs/hardware-results/2026-09-04-phase12-focus-matrix.md b/docs/hardware-results/2026-09-04-phase12-focus-matrix.md index a9ada37..5a6407b 100644 --- a/docs/hardware-results/2026-09-04-phase12-focus-matrix.md +++ b/docs/hardware-results/2026-09-04-phase12-focus-matrix.md @@ -398,3 +398,82 @@ Until those close, `qualifying_count` stays unpopulated and `coverage` stays blank. What this run establishes is narrower and still worth having: **an adaptive count is a viable basis for an activity claim where every summary statistic measured here was not.** + + +--- + +# Follow-up 5: what the count rule actually detects + +**Raw evidence:** `private/phase12-dwell-20260905T053804Z.jsonl` (240 trials) +and `private/phase12-occ-20260905T053804Z.jsonl` (240 trials). Zero arm +failures. Same porch configuration. + +Follow-up 4 left two questions: is the rule a fixed count or a fraction of +samples, and does "median as the floor" survive rising occupancy. Both were +measured rather than argued. + +## 1. A fraction, with a hard minimum sample count + +Sample count now scales with dwell (20 ms spacing), so the same rule was +re-derived at each dwell: + +| dwell | samples/pass | best rule | as a fraction | detect | false | +|---|---|---|---|---|---| +| 100 ms | 6 | `C4 >= 1` | 16.7% | **23/60 (38%)** | 1/60 | +| 500 ms | 26 | `C6 >= 1` | 3.8% | **60/60 (100%)** | 2/60 | +| 2,000 ms | 101 | `C4 >= 5` | 5.0% | 57/60 (95%) | 1/60 | + +A fraction of roughly **4-5% of accepted samples** transfers between the +500 ms and 2,000 ms passes. It does not rescue the 100 ms pass, which fails at +every threshold: with six samples there is no rule that both catches the +source and rejects ambient. That is a floor on the instrument, not a tuning +problem — **a short pass can report coverage honestly, but must not report +activity at all.** 500 ms / 26 samples was the strongest arm measured. + +## 2. Median-as-floor held, but was not tested where it should break + +Across source duty from 28.6% to 57.2%, the pass median read -101.0 dBm for +source-on and source-off alike. It never tracked the signal, so the adaptive +floor is sound over the range tested. + +The reason is worth stating, because it bounds the result: at these levels +most samples read like noise even while the transmitter is radiating, which is +the same effect that made `p90` useless in Follow-up 2. A **strong** source at +the same occupancy would pull the median up and break the assumption. This +sweep did not reach that regime and does not license a claim about it. + +## 3. Detection is a function of occupancy, and that is the limitation + +Same dwell, same rule, only the source's duty within the window varied: + +| source duty | detect (`C6 >= 5`, n=30) | 95% CI | +|---|---|---| +| 57.2% | 27/30 (90%) | [0.74, 0.97] | +| 42.9% | 28/30 (93%) | [0.79, 0.98] | +| 28.6% | 11/30 (37%) | [0.22, 0.54] | +| 28.6% | 13/30 (43%) | [0.27, 0.61] | + +Detection collapses below roughly 40% occupancy. **This rule detects a +persistently occupied channel; it does not detect individual packets.** A +single SF8 packet inside a 2,000 ms pass is on the order of 3-7% occupancy — +far below where this measurement shows the rule already failing more than half +the time. + +That is not a defect to tune away. It is the honest scope of an RSSI-sampling +instrument: it can say "this frequency was busy while I listened", and it +cannot say "a transmission occurred". The latter needs a primitive that +detects a signal rather than sampling energy — CAD, or actual packet +reception, which §3 has contemplated as the alternative basis from the start. + +## Consequences for the phase + +- Any activity condition is a **fraction** of accepted samples (~4-5%), never + a fixed count, and applies only above a minimum sample count. 6 samples is + below it; 26 is comfortably above. +- An activity claim, if one is ever surfaced, must be worded as channel + occupancy during the pass, not as evidence that something transmitted. +- Detecting individual packets needs CAD or packet reception. That is now the + better-supported route for §3's "observed activity", and this sweep is the + reason to prefer it rather than a hunch. +- `qualifying_count` remains unpopulated. What to populate it with is now a + design decision with evidence behind it rather than an open guess. diff --git a/scripts/phase12_focus_matrix.py b/scripts/phase12_focus_matrix.py index d002841..12274d1 100644 --- a/scripts/phase12_focus_matrix.py +++ b/scripts/phase12_focus_matrix.py @@ -226,6 +226,9 @@ def run_trial(card, transmitter, position, dwell_ms, samples, source_on, gap_ms) "home_restore": result["HR"], "radio_status": int(result["E"]), "pulses_fired": pulses, + # Recorded because it sets the source's duty cycle within the window, + # which is the independent variable of an occupancy sweep. + "pulse_gap_ms": gap_ms, # C = samples at or above the pass's median + N dB. "counts_above_median": {k: int(v) for k, v in counts.items() if k.startswith("C")}, } From b74b4b5b314f43e783717a8d43e657ced4f93ec6 Mon Sep 17 00:00:00 2001 From: Will Shields Date: Fri, 4 Sep 2026 23:18:57 -0700 Subject: [PATCH 13/21] Reconcile STATUS and CHANGELOG with the dwell and occupancy sweeps Both files were left behind by a scripted edit that failed partway through after the design doc had already been updated, so a4bc335 recorded the finding in one place and not the other two. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 15 +++++++++++++++ docs/STATUS.md | 16 ++++++++++++---- 2 files changed, 27 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a4e8b08..4db4a2c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,21 @@ project (not a log of how it got there), see [docs/STATUS.md](docs/STATUS.md). ## 2026-09-05 +- Bounded the count rule with two more sweeps (480 trials, no arm failures). + It is a **fraction** of accepted samples (~4-5%), which transfers between + 500 ms and 2,000 ms passes; no threshold rescues a 100 ms pass, whose six + samples cannot both catch the source and reject ambient — so a short pass + may report coverage and must not report activity. The occupancy sweep is the + consequential one: detection runs 90-93% while the source fills 43-57% of + the pass and collapses to 37-43% at 28.6%. A single SF8 packet inside a + 2,000 ms pass is 3-7% occupancy, far below that. **The rule detects a + persistently occupied channel, not individual packets**, which is the honest + scope of an RSSI-sampling instrument and makes CAD or packet reception the + better-supported basis for §3's observed activity. Median-as-floor held at + every occupancy tested, but only because a weak source leaves most samples + reading like noise; a strong source would break it and that regime was not + reached. + - Found an activity basis that survives field levels. After every RSSI summary statistic was rejected, the remaining idea was to count samples above an adaptive floor rather than take an extreme of them — `peak` is one sample and diff --git a/docs/STATUS.md b/docs/STATUS.md index da213aa..1291746 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -125,10 +125,18 @@ statistics failed. Counting samples at or above the pass's own median plus [0.863, 0.983]) at the same field-level configuration, with a single flagged control trial that read -63 dBm against a -101 dBm median — a real transmission that was not ours, so the 1.7% false rate is an upper bound. It -is a candidate, not a constant: the threshold is a count out of 101 samples -and the sampling policy scales sample count with dwell, so it must become a -fraction of accepted samples or be validated per dwell first. -`qualifying_count` stays unpopulated until then. +is a candidate, not a constant. + +Two further sweeps (480 trials) then bounded it. The rule is a **fraction** of +accepted samples (~4-5%), transferring between 500 ms and 2,000 ms passes, but +no threshold works at 100 ms — six samples cannot both catch the source and +reject ambient, so a short pass may report coverage and must not report +activity. More importantly, detection tracks how much of the pass the source +occupies: 90-93% at 43-57% occupancy, 37-43% at 28.6%. A single SF8 packet in +a 2,000 ms pass is 3-7% occupancy. **The rule detects a persistently occupied +channel, not individual packets**, which makes CAD or packet reception the +better-supported basis for an activity claim. `qualifying_count` stays +unpopulated; what to populate it with is now an evidenced design decision. The existing Phase 11 Cell feature remains partially hardware-verified and visible in "What's still open." It is deliberately scheduled as **V2 From 9d5a286911842f06b36d13f9dd9f39d1d114681f Mon Sep 17 00:00:00 2001 From: Will Shields Date: Sat, 5 Sep 2026 07:39:49 -0700 Subject: [PATCH 14/21] Second baseline withdraws the occupancy conclusion Follow-up 5 concluded that the count rule "detects a persistently occupied channel, not individual packets", and argued from there that CAD or packet reception was the better-supported basis for an activity claim. Both came from a single link at roughly 12 dB SNR. Repeating the SNR-sensitive tests at roughly 25 dB overturns it. At 28.6% occupancy, detection went from 11/30 to 26/30. The cliff moved, so it was never the statistical limit it was presented as -- it was where that particular link stopped delivering samples above the margin. Detection depends on occupancy and SNR together, and single-packet detection is untested rather than excluded: a 150 ms packet at 20 ms sampling yields about seven elevated samples, above the four or five the best rules require. The link change is worth recording on its own. A matched whip replaced the stubby and both antennas were stood vertical; vertical alignment mattered more than the antenna, worth about 13 dB at 918.5 MHz where the horizontal pair had been sitting in each other's pattern null. Two conclusions survived both links. The 100 ms floor is real -- six samples per pass detected 23/60 at 12 dB and 28/60 at 25 dB, so thirteen decibels bought almost nothing and it is a sampling limit rather than a link limit. And the rule is a fraction of accepted samples, 3.8-5.0% at 12 dB and 4.0-7.7% at 25 dB, not a fixed count. Recorded as a method note as well: three conclusions were drawn from one link, two held and the most consequential inverted. Where a conclusion depends on signal level it needs two levels before it goes in the evidence file. Co-Authored-By: Claude Opus 5 --- .../2026-09-04-phase12-focus-matrix.md | 72 +++++++++++++++++++ 1 file changed, 72 insertions(+) diff --git a/docs/hardware-results/2026-09-04-phase12-focus-matrix.md b/docs/hardware-results/2026-09-04-phase12-focus-matrix.md index 5a6407b..5ed7fda 100644 --- a/docs/hardware-results/2026-09-04-phase12-focus-matrix.md +++ b/docs/hardware-results/2026-09-04-phase12-focus-matrix.md @@ -477,3 +477,75 @@ reception, which §3 has contemplated as the alternative basis from the start. reason to prefer it rather than a hunch. - `qualifying_count` remains unpopulated. What to populate it with is now a design decision with evidence behind it rather than an open guess. + + +--- + +# Follow-up 6: a second baseline, and a correction + +**Raw evidence:** `private/phase12-b2-{occ,counts,dwell}-20260905T134743Z.jsonl` +(600 trials, no arm failures). Same porch position and geometry as Follow-up 5; +the only change is the link. + +Every conclusion in Follow-ups 4 and 5 came from one link at roughly 12 dB SNR. +This repeats the SNR-sensitive tests at roughly 25 dB, to separate properties +of Focus from properties of that link. + +**Link change:** the transmitter's stubby was replaced with a matched whip and +both antennas were stood vertical. Vertical alignment mattered more than the +antenna: at 918.5 MHz the horizontal pair was losing about 13 dB, consistent +with each whip sitting in the other's pattern null. Source peak moved from +-85/-86 dBm to -77/-66 dBm at 908.75 and from -87/-89 to -69/-75 at 918.5, +against an unchanged -98/-101 dBm floor. + +## The occupancy cliff was an SNR limit — Follow-up 5's conclusion is withdrawn + +| source duty | detect at ~12 dB | detect at ~25 dB | +|---|---|---| +| 57.2% | 27/30 (90%) | 30/30 (100%) | +| 42.9% | 28/30 (93%) | 29/30 (97%) | +| **28.6%** | **11/30 (37%)** | **26/30 (87%)** | +| 21.5% | not tested | 22/30 (73%) | + +Follow-up 5 concluded that "the rule detects a persistently occupied channel, +not individual packets", and reasoned from there that CAD or packet reception +was the better-supported basis for an activity claim. **That conclusion does +not survive a second link.** At 28.6% occupancy, 13 dB of link improvement +took detection from 37% to 87%. The cliff moved, so it was not the statistical +limit it was presented as; it was where *that* link stopped delivering samples +above the margin. + +What can be said now is narrower and conditional: **detection depends on +occupancy and SNR together**, and the occupancy at which it fails is a +property of the link, not of the instrument. Single-packet detection is +untested and is no longer ruled out — a 150 ms packet at 20 ms sampling yields +roughly seven elevated samples, comfortably above the four or five the best +rules require, provided the samples clear the margin. The lowest occupancy +measured is 21.5%, so anything below that remains unmeasured in both +directions. + +## What did survive + +- **The 100 ms floor is real.** Six samples per pass detected 23/60 at 12 dB + and 28/60 at 25 dB. Thirteen decibels bought almost nothing, so this is a + sampling limit rather than a link limit: a short pass may report coverage + and must not report activity. +- **The rule is a fraction.** Best thresholds landed at 3.8-5.0% of samples at + 12 dB and 4.0-7.7% at 25 dB. Roughly stable, and clearly not a fixed count. +- **500 ms / 26 samples remains the strongest arm** at both SNRs (60/60 each), + with zero false positives at 25 dB. + +## Method note + +This is why a single link is not a baseline. Three conclusions were drawn from +the first one; two held and one inverted. The one that inverted was the most +consequential and the most confidently worded, and it would have sent the next +slice toward CAD on the strength of an artifact. Where a conclusion depends on +signal level, it now needs at least two levels before it goes in this file. + +## Next measurement + +Drive occupancy below 21.5% at the stronger link — a single short-airtime +pulse per pass is about 5% — and find where detection actually fails when the +link is not the binding constraint. That, not a CAD rewrite, is the open +question for §3's activity basis. From 7237186920cd932a377778dac5c4a23bde8c0ade Mon Sep 17 00:00:00 2001 From: Will Shields Date: Sat, 5 Sep 2026 07:40:13 -0700 Subject: [PATCH 15/21] Reconcile STATUS and CHANGELOG with the withdrawn occupancy conclusion Same partial-failure as b74b4b5: a scripted multi-file edit asserted against STATUS text that had already been rewritten, so 9d5a286 carried the correction in the design doc and evidence file but left the withdrawn claim standing in the two places most likely to be read first. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 15 +++++++++++++++ docs/STATUS.md | 15 +++++++++------ 2 files changed, 24 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4db4a2c..2e0253a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,21 @@ project (not a log of how it got there), see [docs/STATUS.md](docs/STATUS.md). ## 2026-09-05 +- Ran a second baseline at ~25 dB SNR (600 trials) and **withdrew** the + occupancy conclusion drawn from the first. Replacing the transmitter's + stubby with a matched whip and standing both antennas vertical gained ~13 dB + — vertical alignment mattered more than the antenna, the horizontal pair + having sat in each other's pattern null at 918.5 MHz. At 28.6% occupancy + that took detection from 37% to 87%, so the "cliff" was a property of the + first link, not a statistical limit of the instrument. The claim that the + rule "detects a persistently occupied channel, not individual packets" is + therefore withdrawn, along with the argument that the evidence favours CAD + over sampling for §3's activity basis. Two conclusions did survive both + links: the 100 ms six-sample floor (13 dB bought almost nothing) and the + rule's fraction-of-samples form. Recorded as a method note too — three + conclusions came from one link, two held and the most confident one + inverted. + - Bounded the count rule with two more sweeps (480 trials, no arm failures). It is a **fraction** of accepted samples (~4-5%), which transfers between 500 ms and 2,000 ms passes; no threshold rescues a 100 ms pass, whose six diff --git a/docs/STATUS.md b/docs/STATUS.md index 1291746..e441a2d 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -131,12 +131,15 @@ Two further sweeps (480 trials) then bounded it. The rule is a **fraction** of accepted samples (~4-5%), transferring between 500 ms and 2,000 ms passes, but no threshold works at 100 ms — six samples cannot both catch the source and reject ambient, so a short pass may report coverage and must not report -activity. More importantly, detection tracks how much of the pass the source -occupies: 90-93% at 43-57% occupancy, 37-43% at 28.6%. A single SF8 packet in -a 2,000 ms pass is 3-7% occupancy. **The rule detects a persistently occupied -channel, not individual packets**, which makes CAD or packet reception the -better-supported basis for an activity claim. `qualifying_count` stays -unpopulated; what to populate it with is now an evidenced design decision. +activity. Detection also appeared to collapse below ~40% occupancy — but a second +baseline at ~25 dB SNR (600 trials) **withdrew that conclusion**: 13 dB of +link improvement took detection at 28.6% occupancy from 37% to 87%, so the +cliff belonged to the first link, not to the instrument. Detection depends on +occupancy and SNR together, and single-packet detection is untested rather +than excluded. The 100 ms floor did survive both links (13 dB bought almost +nothing there), as did the fraction-of-samples form. `qualifying_count` stays +unpopulated, and the open question is where detection fails once the link is +not the binding constraint. The existing Phase 11 Cell feature remains partially hardware-verified and visible in "What's still open." It is deliberately scheduled as **V2 From 9be6b65e33f49b69cf4a60923caf1bdcb4f1b651 Mon Sep 17 00:00:00 2001 From: Will Shields Date: Sat, 5 Sep 2026 10:37:01 -0700 Subject: [PATCH 16/21] Log a Sweep/Waterfall sampling review as a candidate workstream Workstream 12's measurements bear on Sweep whether or not that was the intent. Phase 9 already recorded that a short bin dwell can miss genuine traffic; what 12 adds is the quantitative form -- detection needs airtime above the sampling spacing, six samples per pass is below a usable floor and 13 dB did not rescue it, and a count beats the extreme statistics Pass A currently thresholds on. Parked as a candidate rather than acted on, for two reasons written into the entry. Sweep aims at persistent energy across 200+ bins, where short dwells are a reasonable trade and Pass B's CAD step already exists because a Pass-A peak is not packet evidence -- the architecture encodes the distinction 12 measured. And 12 itself drew three conclusions from one link, two of which held while the most confident inverted on a second, so any claim about Sweep's sensitivity needs the same two-baseline discipline before it is written down. Entry criteria are concrete rather than a call to go and look: a controlled per-bin sensitivity measurement at two or more signal levels, reusing 12's transmitter fixture and benchSweepFloorQuery's existing readback, plus a decision on whether Waterfall should distinguish "sampled and quiet" from "barely sampled" in what it draws. Also adds the matrix runner's single-pulse mode, which places one armed transmission inside a long window instead of a burst -- the only way to ask whether a single packet is detectable, since a burst answers a different question however its duty is tuned. Co-Authored-By: Claude Opus 5 --- docs/ROADMAP.md | 43 ++++++++++++++++++++++++++ scripts/phase12_focus_matrix.py | 54 +++++++++++++++++++++++++++++---- 2 files changed, 91 insertions(+), 6 deletions(-) diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 14c88fa..8325bcb 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -64,6 +64,49 @@ cost acceptable; the measurement is part of the decision. | **14 — Companion analysis** | Not entered | Deliver an offline, reproducible tool that reads copied run folders without changing original evidence. Test deterministic reports, multi-run comparison, coverage warnings, and privacy-safe export behavior. | | **15 — Field markers and sharing** | Not entered | Add fixed, safe marker presets and `marker.csv`, then integrate redacted sharing. Prove markers cannot affect radio behavior and realistic exports remove selected location/identity detail. | | **16 — Cell closeout** | Deferred bonus | Close the existing V1 Phase 11 evidence gap: a real tower-adjacent RSSI rise plus fresh SD verification of `cell.csv` and Cell's appended `session.csv` fields. This preserves V1 history; it does not renumber it. | +| **17 — Sweep/Waterfall sampling review** | Candidate | Re-evaluate whether Sweep's per-bin sampling and Waterfall's presentation can support what they imply, using the measurement apparatus Workstream 12 built. Entry needs a two-baseline sensitivity measurement, not an argument from analogy. See below. | + +## Candidate — Workstream 17 (Sweep/Waterfall sampling review) + +Raised 2026-09-05 out of Workstream 12's measurements, and deliberately parked +rather than acted on. Phase 9 already recorded that "a normal Sweep's short bin +dwell can miss genuine traffic"; what Workstream 12 adds is the quantitative +form of that statement, plus a fixture capable of testing it. + +What transfers from Workstream 12's evidence +([2026-09-04-phase12-focus-matrix.md](hardware-results/2026-09-04-phase12-focus-matrix.md)): + +- Detection requires the source's airtime to exceed the sampling spacing. + Sweep dwells on each bin for tens of ms with a handful of samples — a + shorter, sparser pass than the 100 ms / 6-sample arm that failed at both + measured signal levels. +- Six samples per pass is below a usable floor, and 13 dB of link improvement + did not rescue it. That was a sampling limit, not a link limit. +- A count of samples above an adaptive floor outperformed every extreme + statistic. Pass A currently keeps a per-bin average and peak and thresholds + on them; the peak is the noise-prone half of that pair. + +What that does **not** establish, and why this is a candidate rather than a +finding: + +- **Sweep is aimed at persistent energy, not packets.** Short per-bin dwells + are a reasonable trade when covering 200+ bins, and Pass B's CAD step exists + precisely because a Pass-A energy peak is not packet evidence. The + architecture already encodes the distinction Workstream 12 measured. +- **One link is not a baseline.** Workstream 12 drew three conclusions from a + single link; two held and the most confident one inverted on a second. Any + claim about Sweep's sensitivity needs the same two-baseline discipline + before it is written down. +- Waterfall is downstream of whatever Sweep's sampling delivers, so it is not + a separate question. Its own risk is presentational: a cell empty because + nothing transmitted looks identical to a cell empty because the pass did not + sample long enough — the §3 "absence is not silence" trap in a new surface. + +Entry would need: a controlled sensitivity measurement of Pass A's per-bin +sampling at two or more signal levels, reusing Workstream 12's transmitter +fixture and `benchSweepFloorQuery`'s existing per-bin floor readback; and a +decision about whether Waterfall should distinguish "sampled and quiet" from +"barely sampled" in what it draws. Rigorously sourced region packs are later candidates, not Workstream 16 and not V2.0 blockers. Each proposed pack needs a separate entry gate with source diff --git a/scripts/phase12_focus_matrix.py b/scripts/phase12_focus_matrix.py index 12274d1..811d9b5 100644 --- a/scripts/phase12_focus_matrix.py +++ b/scripts/phase12_focus_matrix.py @@ -174,14 +174,52 @@ def run_focus_request(card, spec, dwell_ms, samples, start): return terminal, result, counts -def run_trial(card, transmitter, position, dwell_ms, samples, source_on, gap_ms): - """One bounded Focus request, with the transmitter radiating throughout it or not at all.""" +def run_trial(card, transmitter, position, dwell_ms, samples, source_on, gap_ms, + single_pulse_delay_ms=None): + """One bounded Focus request, with the transmitter radiating during it or not at all. + + Two source modes. The burst keeps the transmitter radiating for the whole + window, which is what the qualifying-condition work needed. `single_pulse` + arms exactly one transmission placed inside the window instead, which is + the only way to ask whether a *single packet* is detectable: a burst + answers a different question no matter how its duty is tuned. A single + pulse can only be placed reliably when the window is long relative to the + timing jitter, so this mode is for long dwells, not for 100 ms passes. + """ spec = POSITIONS[position] before = card_status(card) start = counters(before) pulses = 0 - if source_on: + if source_on and single_pulse_delay_ms is not None: + transmitter.observed.clear() + require_ack(card, "BENCH_FOCUS", f"{spec['bin']}:{dwell_ms}:{samples}", timeout=12.0) + require_ack(transmitter, "ARM", str(single_pulse_delay_ms), timeout=5.0) + terminal = wait_for( + card, + lambda status: status.get("FS") == "3" + and int(status.get("FO", "0")) > start["FO"] + and int(status.get("FW", "0")) > start["FW"], + max(12.0, dwell_ms / 1000.0 + 10.0), + "Focus completion, home restore, and focus.csv commit", + ) + result = parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "-", timeout=12.0)) + counts = {} + try: + opcode, payload = card.request("BENCH_FOCUS_COUNTS", "-", timeout=12.0) + if opcode == "ACK": + counts = parse_fields(payload) + except (RuntimeError, TimeoutError): + pass + # The pulse must actually have happened, or a miss is the fixture's + # fault rather than the receiver's. + transmitter.request("STATUS", "-", timeout=5.0) + fired = {op for _, op, _ in transmitter.observed} & {"TX_STARTED", "TX_DONE"} + if fired != {"TX_STARTED", "TX_DONE"}: + raise RuntimeError(f"single pulse incomplete: {sorted(fired)}") + transmitter.observed.clear() + pulses = 1 + elif source_on: with PulseBurst(transmitter, gap_ms / 1000.0) as burst: terminal, result, counts = run_focus_request(card, spec, dwell_ms, samples, start) pulses = burst.fired @@ -229,6 +267,7 @@ def run_trial(card, transmitter, position, dwell_ms, samples, source_on, gap_ms) # Recorded because it sets the source's duty cycle within the window, # which is the independent variable of an occupancy sweep. "pulse_gap_ms": gap_ms, + "single_pulse_delay_ms": single_pulse_delay_ms, # C = samples at or above the pass's median + N dB. "counts_above_median": {k: int(v) for k, v in counts.items() if k.startswith("C")}, } @@ -257,7 +296,7 @@ def settle(card, timeout_s=20.0): def run_trial_with_retry(card, transmitter, position, dwell_ms, samples, source_on, - gap_ms, results, attempts): + gap_ms, results, attempts, single_pulse_delay_ms=None): """A transport failure is a harness event, not an RF result — retry it, loudly. The retry is a genuinely new bounded request, and the failure is recorded @@ -267,7 +306,7 @@ def run_trial_with_retry(card, transmitter, position, dwell_ms, samples, source_ for attempt in range(1, attempts + 1): try: row = run_trial(card, transmitter, position, dwell_ms, samples, - source_on, gap_ms) + source_on, gap_ms, single_pulse_delay_ms) if attempt > 1: row["retried_attempts"] = attempt return row @@ -317,6 +356,9 @@ def main(): parser.add_argument("--pulse-gap-ms", type=int, default=30, help="idle gap between completed transmissions during a source-on " "trial; the burst is paced by TX_DONE, not by this alone") + parser.add_argument("--single-pulse-delay-ms", type=int, default=None, + help="arm exactly one pulse this many ms after the request is " + "accepted, instead of a burst; for long dwells only") parser.add_argument("--attempts", type=int, default=3, help="attempts per trial before the run gives up (transport retries)") parser.add_argument("--allow-transmit", action="store_true", @@ -393,7 +435,7 @@ def main(): row = run_trial_with_retry(card, transmitter, position, dwell, samples, source_on, args.pulse_gap_ms, results, - args.attempts) + args.attempts, args.single_pulse_delay_ms) row.update({"event": "trial", "arm_index": index}) results.write(row) completed += 1 From b749ae7322c948561b689ad48ad2d952530df955 Mon Sep 17 00:00:00 2001 From: Will Shields Date: Sat, 5 Sep 2026 10:55:10 -0700 Subject: [PATCH 17/21] A single packet is detectable; reconcile the design doc, three commits stale One armed 148 ms transmission placed inside a 2,000 ms pass -- 7.4% occupancy -- was detected 29/30 with 0/30 false positives at the stronger link. That settles the question the occupancy withdrawal left open: an RSSI-sampling pass can support a packet-level activity signal. A 148 ms packet sampled every 20 ms yields about seven elevated samples against the four the rule needs, so nothing about the instrument prevented this. The first link did. It is not unconditional. The same rule at the weaker link failed on sources occupying four times as much of the pass, so the rate depends on link quality the device cannot know. And the companion position carrying real traffic measured worse rather than better: a -57 dBm event in a source-off control both produced false positives and suppressed counts by lifting the median, which means Focus is least reliable exactly where a band is busiest. Wording an activity indication that survives that is a product decision now, not an open measurement. A placement analysis was attempted and discarded rather than reported. It tried to time each pulse against its window from log timestamps, but TX_STARTED reaches the host only when the harness next polls the socket, so the timestamps record host polling rather than RF timing -- and the output said so, classifying "missed the window" trials as detected at 71-97% while "inside" detected 0/2. Timing a pulse against the window needs a device-side timestamp. Also reconciles the design doc, which had silently missed its edit in each of the last three documentation commits: in both a4bc335 and 9d5a286 the assert that failed was the design doc's, not STATUS's, and the follow-up fixes patched the wrong file. It now carries the consolidated current position -- the rule's fractional form, the 100 ms sampling floor, the withdrawn occupancy claim, and this result -- rather than a state from three commits ago. The governing gate document being the stale one is the worst place for that error to land. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 15 ++++ docs/STATUS.md | 16 ++++- .../2026-09-04-phase12-focus-matrix.md | 68 +++++++++++++++++++ docs/research/phase12-survey-truth-design.md | 35 ++++++++-- 4 files changed, 125 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2e0253a..632de83 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,21 @@ project (not a log of how it got there), see [docs/STATUS.md](docs/STATUS.md). ## 2026-09-05 +- Measured the positive case the withdrawal left open: **a single packet is + detectable**. One armed 148 ms transmission placed inside a 2,000 ms pass — + 7.4% occupancy — was caught 29/30 with 0/30 false positives at the stronger + link. A 148 ms packet sampled every 20 ms yields about seven elevated + samples against the four the rule needs, so nothing about the instrument + prevented it; the first link did. Not unconditional, though: the same rule + at the weaker link failed on sources filling four times as much of the pass, + and the companion position carrying real traffic measured *worse* — a + -57 dBm event in a control trial both produced false positives and + suppressed counts by lifting the median, so Focus is least reliable exactly + where a band is busiest. Also discarded a placement analysis that tried to + time the pulse against the window from log timestamps: `TX_STARTED` reaches + the host only when the harness next polls, so it records host polling rather + than RF timing, and its output was self-contradictory. + - Ran a second baseline at ~25 dB SNR (600 trials) and **withdrew** the occupancy conclusion drawn from the first. Replacing the transmitter's stubby with a matched whip and standing both antennas vertical gained ~13 dB diff --git a/docs/STATUS.md b/docs/STATUS.md index e441a2d..af619da 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -137,9 +137,19 @@ link improvement took detection at 28.6% occupancy from 37% to 87%, so the cliff belonged to the first link, not to the instrument. Detection depends on occupancy and SNR together, and single-packet detection is untested rather than excluded. The 100 ms floor did survive both links (13 dB bought almost -nothing there), as did the fraction-of-samples form. `qualifying_count` stays -unpopulated, and the open question is where detection fails once the link is -not the binding constraint. +nothing there), as did the fraction-of-samples form. + +That open question is now answered: one armed 148 ms packet inside a 2,000 ms +pass — 7.4% occupancy — was detected **29/30 with 0/30 false positives** at +the stronger link. An RSSI-sampling pass can support a packet-level activity +signal, but not unconditionally: the same rule at the weaker link failed on +sources occupying four times as much of the pass, and a position carrying real +traffic measured worse rather than better (a -57 dBm event in a control trial +both raised false positives and suppressed counts by lifting the median). So +Focus is least reliable where a band is busiest, and detection depends on link +quality the device cannot know. `qualifying_count` stays unpopulated: wording +an activity indication that stays true under those conditions is now a product +decision rather than an open measurement. The existing Phase 11 Cell feature remains partially hardware-verified and visible in "What's still open." It is deliberately scheduled as **V2 diff --git a/docs/hardware-results/2026-09-04-phase12-focus-matrix.md b/docs/hardware-results/2026-09-04-phase12-focus-matrix.md index 5ed7fda..57332ef 100644 --- a/docs/hardware-results/2026-09-04-phase12-focus-matrix.md +++ b/docs/hardware-results/2026-09-04-phase12-focus-matrix.md @@ -549,3 +549,71 @@ Drive occupancy below 21.5% at the stronger link — a single short-airtime pulse per pass is about 5% — and find where detection actually fails when the link is not the binding constraint. That, not a CAD rewrite, is the open question for §3's activity basis. + + +--- + +# Follow-up 7: a single packet is detectable + +**Raw evidence:** `private/phase12-singlepulse-20260905T173222Z.jsonl` +(120 trials, no arm failures). Same ~25 dB link as Follow-up 6. One armed +transmission placed inside each 2,000 ms pass instead of a burst — a burst +answers a different question however its duty is tuned. + +| position | packet | occupancy | detect (`C6 >= 4`) | false | +|---|---|---|---|---| +| mid-aligned | 148 ms | 7.4% | **29/30 (97%)**, [0.83, 0.99] | **0/30** | +| low-aligned | 286 ms | 14.3% | 20/30 (67%), [0.49, 0.81] | 2/30 | + +Source-on peak at mid-aligned: median -74 dBm against a -97 dBm floor. + +**This settles the question Follow-up 5 got wrong.** That entry concluded the +rule "detects a persistently occupied channel, not individual packets". +Follow-up 6 withdrew it on the grounds that the occupancy cliff was a property +of a weak link; this measures the positive case directly. At an adequate link, +a single packet occupying 7% of the pass is detected essentially every time, +with no false positives in thirty controls. + +The arithmetic behind it is unremarkable in hindsight: a 148 ms packet sampled +every 20 ms yields roughly seven elevated samples, comfortably above the four +the rule requires. Nothing about the instrument prevented this. The first +link did. + +## low-aligned was contaminated, and that is informative + +The weaker result at low-aligned comes with a source-off control that peaked +at **-57 dBm** — 44 dB above its own floor. That is real traffic on 908.75 +during the run, not noise, and it cuts both ways: it produces the two false +positives, and by lifting the median in affected passes it suppresses counts +and produces misses. An adaptive floor is only as good as the assumption that +the floor is quiet. + +Two consequences worth carrying forward. A frequency with real activity is +harder to measure on, not easier, so controlled work should prefer quiet +positions. And in the field the same effect applies in reverse: Focus will be +least reliable exactly where a band is busiest. + +## Limits + +- One link (~25 dB SNR) and one dwell. At the ~12 dB link of Follow-ups 4-5 + this would be substantially worse; the rate is not a property of Focus alone. +- The runner verifies that the pulse fired, not precisely *when* it fired + relative to the window. An attempt to classify placement from log timestamps + was invalid and discarded: `TX_STARTED` reaches the host only when the + harness next polls the socket, so its timestamp records host polling rather + than RF timing. 29/30 detection implies placement was mostly correct, but it + is inferred rather than measured. Timing the pulse against the window needs + a device-side timestamp, not a host-side one. +- 30 trials per cell. The intervals are wide enough to matter: [0.83, 0.99] + is consistent with a true rate anywhere from about six-in-seven to + essentially always. + +## Where this leaves the activity claim + +An RSSI-sampling pass **can** support a packet-level activity signal at an +adequate link, using a count above an adaptive floor. What it cannot do is +support it *unconditionally* — the same rule at a weaker link failed on +sources occupying four times as much of the window. Any operator-facing +activity indication has to be honest that it is a function of link quality, +which the device cannot know. That is a wording and product problem now, +rather than an open measurement. diff --git a/docs/research/phase12-survey-truth-design.md b/docs/research/phase12-survey-truth-design.md index 0dd977c..c1ac85b 100644 --- a/docs/research/phase12-survey-truth-design.md +++ b/docs/research/phase12-survey-truth-design.md @@ -367,12 +367,35 @@ local spectrum identity or coverage of the city. detected 57/60 source-on trials, 95% CI [0.863, 0.983], with one flagged source-off trial that read -63 dBm against a -101 dBm median — a real transmission, not ours, so the 1.7% false rate is an upper bound rather than - a measurement against true silence. It is a **viable candidate, not yet a - constant**: the threshold is a count out of 101 samples, and the sampling - policy now scales sample count with dwell, so "2 samples" means 2% at a - 2,000 ms pass and 33% at a 100 ms one. Expressing it as a fraction of - accepted samples, or validating per dwell, is the next measurement. - CAD/packet evidence remains the alternative basis §3 contemplates. + a measurement against true silence. Three further campaigns (1,200 trials) then + characterised it. + + **Form.** It is a *fraction* of accepted samples, roughly 4-8%, which + transfers between 500 ms and 2,000 ms passes. No threshold rescues a 100 ms + pass: six samples cannot both catch the source and reject ambient, and 13 dB + of link improvement barely moved it (23/60 to 28/60), so that is a sampling + limit rather than a link limit. **A short pass may report coverage and must + not report activity.** 500 ms / 26 samples was the strongest arm at both + measured links. + + **Scope, after a correction.** An earlier reading of one link concluded the + rule "detects a persistently occupied channel, not individual packets". A + second baseline withdrew it: at 28.6% occupancy, 13 dB took detection from + 37% to 87%, so the apparent cliff belonged to that link. The positive case + was then measured directly — one armed 148 ms packet inside a 2,000 ms pass, + 7.4% occupancy, detected **29/30 with 0/30 false positives**. A 148 ms + packet sampled every 20 ms yields about seven elevated samples against the + four the rule needs, so nothing about the instrument prevented it. + + **The condition that remains.** Detection is a function of link quality the + device cannot know: the same rule at the weaker link failed on sources + occupying four times as much of the pass. And a position carrying real + traffic measured *worse*, not better — a -57 dBm event in a control trial + both produced false positives and suppressed counts by lifting the median, + so Focus is least reliable exactly where a band is busiest. Wording an + activity indication that stays true under those conditions is a product + decision now, not an open measurement. CAD or packet reception remains §3's + alternative, neither favoured nor excluded by this evidence. - [ ] Coverage thresholds (`sampled`/`repeated`) remain unselected. They are about pass counts and accumulated time across repeated requests, which no single-pass measurement can supply. From 2a3a3529fb10852ed9a94eff5fc72d736b6cf4e8 Mon Sep 17 00:00:00 2001 From: Will Shields Date: Sat, 5 Sep 2026 11:10:47 -0700 Subject: [PATCH 18/21] Workstream 17 entry: Pass A reads a floor ~14 dB low as shipped version.h records an 11 dB under-read on Cell from sampling RSSI before the AGC settles after a light retune, and says outright that whether Sweep shares it is unknown. It does, and larger: adding a 5 ms settle moves Pass A's silent floor for bin 66 from -115.0 to -101.5 dBm, and 20 ms holds it at -101.7. Focus independently measures the ambient floor at that frequency as -99 to -101 dBm across hundreds of passes, so the settled readings are the real ones and the shipped configuration is about 14 dB low. It matters because Pass A's peak decision compares a bin against a noise floor plus a margin, and both terms come from under-read samples. The margin constant was calibrated against those same values so it is not obviously wrong, but it is calibrated on a scale that does not correspond to real dBm. Deliberately does not claim the other half. Whether Pass A can flag a bin carrying traffic is still open: even at 20 ms settle the source produced only +3.5 dB where Focus reads the same transmitter 25 dB above the floor, and the candidate explanations -- partial overlap with a 3 ms bin visit, bridge latency making "on air during this specific 3 ms" unguaranteeable, and a FULL begin() row that reads lower than the settled light path -- are untested. Two earlier attempts were invalid because firing only for a lap's duration radiates after the lap ends, which read as a silent band in every arm; both are recorded so the next attempt does not repeat them. Co-Authored-By: Claude Opus 5 --- .../2026-09-05-ws17-sweep-settle.md | 66 +++++ scripts/ws17_sweep_sensitivity_bench.py | 238 ++++++++++++++++++ 2 files changed, 304 insertions(+) create mode 100644 docs/hardware-results/2026-09-05-ws17-sweep-settle.md create mode 100644 scripts/ws17_sweep_sensitivity_bench.py diff --git a/docs/hardware-results/2026-09-05-ws17-sweep-settle.md b/docs/hardware-results/2026-09-05-ws17-sweep-settle.md new file mode 100644 index 0000000..3bc66b8 --- /dev/null +++ b/docs/hardware-results/2026-09-05-ws17-sweep-settle.md @@ -0,0 +1,66 @@ +# Workstream 17 — Pass A reads a floor ~14 dB low as shipped + +**Build:** bench image, `V=1.0.7;BENCH=1`. **Date:** 2026-09-05. +**Link:** Workstream 12's porch fixture at its stronger setting, matched whips +vertical, transmitter on `MESH_OREGON` (918.5 MHz -> Sweep bin 66 of 85). +**Raw:** `private/ws17-retune-probe.log`, `private/ws17-check*.jsonl`. + +Entry measurement for the candidate workstream. `src/version.h` records an +11 dB under-read on Cell caused by sampling RSSI before the AGC settles after +a light retune, and states plainly that "Whether Sweep has the same under-read +is unknown". This measures it. + +Each row: two silent laps and three laps with the transmitter radiating, +median of `BENCH_SWEEP_FLOOR` for bin 66. + +| Pass A configuration | silent floor | source on | delta | +|---|---|---|---| +| **LIGHT + 0 ms settle (shipped)** | **-115.0 dBm** | -114.8 | +0.2 dB | +| LIGHT + 5 ms settle | -101.5 dBm | -102.5 | -1.0 dB | +| LIGHT + 20 ms settle | -101.7 dBm | -98.2 | +3.5 dB | +| FULL `begin()` per bin | -112.3 dBm | -112.5 | -0.2 dB | + +## Established + +**Pass A as shipped reads the noise floor about 14 dB below a settled +receiver.** Adding a 5 ms settle moves the silent floor from -115.0 to +-101.5 dBm, and 20 ms holds it at -101.7. Focus independently measures the +ambient floor at this frequency as -99 to -101 dBm across hundreds of passes, +which is where the settled readings land and is 14 dB above the shipped one. + +That answers `version.h`'s open question: Sweep has the same defect Cell had, +and larger. It matters because Pass A's peak decision compares a bin against a +noise floor plus a margin, and both terms are being computed from under-read +samples. `ENERGY_DEFAULT_THRESHOLD_MARGIN_DBM_X10` was calibrated against +those same under-read values, so the margin is not obviously wrong — but it is +calibrated on a scale that does not correspond to real dBm, which makes any +absolute reasoning about Sweep sensitivity unsound until this is settled. + +## Not established, and not to be inferred from this + +**Whether Pass A can flag a bin carrying traffic remains open.** The +source-present column is not a clean measurement and should not be read as +one. Even at 20 ms settle the source produced only +3.5 dB, when Focus reads +the same transmitter at -69 to -75 dBm against a -100 floor. Candidate +explanations, none yet tested: the source is on air for only part of each +lap's 3 ms visit to bin 66, so a median across three laps washes it out; the +firing pipeline's ~1.5 s bridge latency makes "source on during this specific +3 ms" hard to guarantee; and the `FULL begin()` row is internally odd, reading +a *lower* floor (-112.3) than the settled light path despite skipping the +settle only because a full begin already takes longer. + +Two earlier attempts at this measurement were invalid for the same reason and +are recorded so the next one does not repeat them: firing only for a lap's +duration radiates entirely after the lap ends, and reads as a silent band in +every arm. + +## Next + +- Establish the floor result properly: more laps, both link levels, and a + settle sweep finer than 0/5/20 ms, to find where the reading stabilises. +- Only then ask the detection question, with the source's presence during the + bin visit *verified* rather than assumed — which needs a device-side + timestamp, not a host-side one. +- Decide separately whether the margin constant should be recalibrated against + settled readings, or whether Pass A should settle before sampling. Those are + different fixes with different costs to lap time. diff --git a/scripts/ws17_sweep_sensitivity_bench.py b/scripts/ws17_sweep_sensitivity_bench.py new file mode 100644 index 0000000..5fb8328 --- /dev/null +++ b/scripts/ws17_sweep_sensitivity_bench.py @@ -0,0 +1,238 @@ +#!/usr/bin/env python3 +"""Workstream 17: can Pass A flag a bin that is carrying traffic? + +v1.0.3 already settled that a full-band sweep cannot *decode* a packet -- that +needs the receiver parked for the whole airtime, which no 85-bin lap can offer, +and increasing samples per bin was tried and reverted on that reasoning. + +This asks a different question. Detecting a packet as *energy* needs only that +the bin's short sampling window overlaps the transmission, not that the whole +airtime is captured. Pass A visits each bin for about 3 ms (4 samples, 1 ms +apart) once per ~833 ms lap, so for a 148 ms packet on that bin the geometry +predicts an overlap chance near (148 + 3) / 833, roughly 18%. That is a +falsifiable number, and it is far from the "fraction of a percent" that 3 ms of +833 ms suggests at a glance. + +Three arms per run: + + persistent the transmitter radiates continuously. A control on the + instrument: if Pass A cannot flag the bin here, the sparse arm + measures nothing and the threshold margin is the story. + sparse one packet per interval, so the overlap is a lottery with known + odds. This is the measurement. + silent no transmission. Establishes the bin's own floor distribution, + which is what "flagged" has to be judged against. + +Reports the per-lap flag rate for each arm with Wilson intervals, against the +geometric prediction. It selects no threshold and changes no firmware. +""" + +import argparse +import json +import math +import pathlib +import statistics +import sys +import threading +import time + +import serial + +from bench_harness import (CARD_MARKER, TX_MARKER, Endpoint, ResultWriter, + card_status, parse_fields, require_ack) + + +BAND_HI_MHZ = 923.0 # fixed hardware ceiling regardless of region (energy_plan.h) +BIN_STEP_MHZ = 0.25 +SWEEP_TERMINAL_TIMEOUT_S = 90.0 +# Measured ARM-to-TX_STARTED latency over the WiFi control bridge. +PIPELINE_PRIME_S = 2.5 + + +def wilson(successes, total, z=1.959963984540054): + if total == 0: + return (0.0, 1.0) + p = successes / total + denom = 1.0 + z * z / total + centre = (p + z * z / (2 * total)) / denom + margin = z * math.sqrt(p * (1 - p) / total + z * z / (4 * total * total)) / denom + return (max(0.0, centre - margin), min(1.0, centre + margin)) + + +def bin_for_freq(freq_mhz, total_bins): + lo = BAND_HI_MHZ - (total_bins - 1) * BIN_STEP_MHZ + return max(0, min(total_bins - 1, round((freq_mhz - lo) / BIN_STEP_MHZ))) + + +class Firer: + """Fires the transmitter on its own schedule for the duration of a lap. + + `interval_s` of None means fire as fast as each transmission completes, + which is the persistent arm. Otherwise one pulse per interval, giving the + sparse arm its known duty. + """ + + def __init__(self, transmitter, interval_s): + self.transmitter = transmitter + self.interval_s = interval_s + self._stop = threading.Event() + self._thread = None + self.fired = 0 + + def _run(self): + while not self._stop.is_set(): + try: + require_ack(self.transmitter, "ARM", "0", timeout=2.0) + self.fired += 1 + except (RuntimeError, TimeoutError, OSError): + pass + self._stop.wait(self.interval_s if self.interval_s else 0.05) + + def __enter__(self): + self._thread = threading.Thread(target=self._run, daemon=True) + self._thread.start() + return self + + def __exit__(self, *exc): + self._stop.set() + if self._thread is not None: + self._thread.join(timeout=8.0) + return False + + +def run_lap(card, target_bin): + """One full Pass A lap; returns that bin's averaged RSSI in dBm.""" + require_ack(card, "SWEEP_START", "-", timeout=12.0) + deadline = time.monotonic() + SWEEP_TERMINAL_TIMEOUT_S + while time.monotonic() < deadline: + state = card_status(card).get("W") + if state in {"COMPLETE", "CANCELLED", "FAILED"}: + if state != "COMPLETE": + raise RuntimeError(f"sweep ended {state}") + break + time.sleep(0.1) + else: + raise TimeoutError("sweep did not reach a terminal state") + opcode, payload = card.request("BENCH_SWEEP_FLOOR", str(target_bin), timeout=8.0) + if opcode != "ACK": + raise RuntimeError(f"BENCH_SWEEP_FLOOR failed: {opcode} {payload}") + # Replies as "BIN=;FLOOR=". + fields = parse_fields(payload) + if "FLOOR" not in fields: + raise RuntimeError(f"BENCH_SWEEP_FLOOR gave no floor: {payload}") + return int(fields["FLOOR"]) / 10.0 + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument("--cardputer-port", required=True) + parser.add_argument("--heltec-port", required=True) + parser.add_argument("--log", required=True) + parser.add_argument("--results", required=True) + parser.add_argument("--candidate", default="MESH_OREGON") + parser.add_argument("--freq-mhz", type=float, default=918.5) + parser.add_argument("--airtime-ms", type=float, default=148.0, + help="measured airtime of the candidate, for the prediction") + parser.add_argument("--laps", type=int, default=30, help="laps per arm") + parser.add_argument("--sparse-interval-s", type=float, default=0.8) + parser.add_argument("--allow-transmit", action="store_true") + args = parser.parse_args() + if not args.allow_transmit: + parser.error("this bench arms the transmitter; pass --allow-transmit") + + log_path = pathlib.Path(args.log) + log_path.parent.mkdir(parents=True, exist_ok=True) + results = ResultWriter(args.results) + with log_path.open("a", encoding="utf-8") as log: + card = Endpoint("cardputer", args.cardputer_port, CARD_MARKER, log) + transmitter = None + try: + identity = require_ack(card, "HELLO", "-", timeout=45.0) + if "BENCH=1" not in identity: + raise RuntimeError(f"needs the bench image for BENCH_SWEEP_FLOOR: {identity}") + status = card_status(card) + total_bins = int(status.get("WN", "0")) or 85 + target = bin_for_freq(args.freq_mhz, total_bins) + transmitter = Endpoint("heltec", args.heltec_port, TX_MARKER, log) + require_ack(transmitter, "HELLO", "-", timeout=25.0) + require_ack(transmitter, "QUIET", "-", timeout=8.0) + require_ack(transmitter, "CONFIG", args.candidate, timeout=10.0) + + print(f"target {args.freq_mhz} MHz -> bin {target} of {total_bins}") + results.write({"event": "boot", "identity": identity, "target_bin": target, + "total_bins": total_bins, "candidate": args.candidate, + "freq_mhz": args.freq_mhz, "airtime_ms": args.airtime_ms, + "laps": args.laps, "sparse_interval_s": args.sparse_interval_s}) + + floors = {} + for arm, interval in (("silent", None), ("persistent", None), + ("sparse", args.sparse_interval_s)): + print(f"\n[{arm}] {args.laps} laps") + values = [] + for lap in range(args.laps): + if arm == "silent": + fired = 0 + floor = run_lap(card, target) + else: + with Firer(transmitter, interval) as firer: + # The control bridge adds roughly 1.5 s between ARM + # and the transmission actually starting. Firing + # only for the lap's duration therefore radiates + # entirely after the lap has ended, which reads as + # a silent band -- it did, identically across all + # three arms, before this wait was added. Prime the + # pipeline first so the source is genuinely on air + # while Pass A is sampling. + time.sleep(PIPELINE_PRIME_S) + floor = run_lap(card, target) + fired = firer.fired + values.append(floor) + results.write({"event": "lap", "arm": arm, "lap": lap, + "bin": target, "floor_dbm": floor, "pulses": fired}) + print(f" [{lap + 1:3d}/{args.laps}] bin {target} floor {floor:6.1f} dBm" + f" pulses {fired}", flush=True) + floors[arm] = values + require_ack(transmitter, "QUIET", "-", timeout=8.0) + + # "Flagged" is judged against the bin's own silent distribution, not + # an absolute number: Phase 12 rejected absolute thresholds at field + # levels, and the same objection applies here. + base = statistics.median(floors["silent"]) + spread = max(floors["silent"]) - base + threshold = base + max(3.0, spread + 1.0) + print(f"\nsilent floor median {base:.1f} dBm, max {max(floors['silent']):.1f} dBm") + print(f"flag threshold: {threshold:.1f} dBm (silent median + margin)\n") + predicted = (args.airtime_ms + 3.0) / 833.0 + for arm in ("silent", "persistent", "sparse"): + hits = sum(1 for v in floors[arm] if v >= threshold) + lo, hi = wilson(hits, len(floors[arm])) + extra = "" + if arm == "sparse": + extra = f" geometric prediction ~{predicted:.1%}" + print(f" {arm:11s} flagged {hits}/{len(floors[arm])} " + f"= {hits / len(floors[arm]):5.1%} 95% CI [{lo:.3f},{hi:.3f}]{extra}") + results.write({"event": "summary", "arm": arm, "hits": hits, + "laps": len(floors[arm]), "threshold_dbm": threshold, + "wilson95": [round(lo, 4), round(hi, 4)], + "floor_median": statistics.median(floors[arm]), + "floor_max": max(floors[arm])}) + finally: + if transmitter is not None: + try: + require_ack(transmitter, "QUIET", "-", timeout=8.0) + finally: + transmitter.close() + card.close() + results.close() + + +if __name__ == "__main__": + try: + main() + except KeyboardInterrupt: + print("interrupted; completed laps are already durable", file=sys.stderr) + sys.exit(130) + except (OSError, serial.SerialException, RuntimeError, TimeoutError, ValueError) as error: + print(f"ws17 sweep sensitivity: {error}", file=sys.stderr) + sys.exit(2) From e8e3dd4b52ab6bf9eefcb8d014f6aae5c279a0e1 Mon Sep 17 00:00:00 2001 From: Will Shields Date: Sat, 5 Sep 2026 11:23:10 -0700 Subject: [PATCH 19/21] Pass A's floor stabilises by 3 ms, at about 15% of lap time Eight settle values, five laps each, five bins, transmitter idle. Silence is the design: the floor is a property of the receiver and its retune, so measuring it with no source removes every timing problem that invalidated the first attempts at Sweep's detection question. Readings are flat from 3 ms through 40 ms. The shipped 0 ms configuration under-reads by 9.5 dB at bin 0 and up to 17.6 dB at bin 40; 1 ms recovers only about half. The cost is one settle per bin times 85 bins, so 3 ms predicts 0.26 s added per lap and the measured 1.70 -> 1.74 s is that within noise, against 5.60 s at 40 ms. Roughly 15% of lap time for 10-18 dB of accuracy. A designed control failed and is recorded as such. Bin 0 was meant to be unaffected because it takes a full begin(), and it moved 9.5 dB. The settle is skipped only when the global FULL retune mode is on -- the condition is !benchSweepRetuneFullEveryBin(), not a per-bin test -- so in shipped LIGHT mode bin 0 gets both the full begin and the settle. It shifted least, which is consistent with a full begin settling partially, but that is an observation rather than a demonstrated mechanism, and the clean control is FULL against LIGHT+3 ms across all bins. Not acted on. A 3 ms settle is the obvious fix and cannot be made alone: the margin constant was calibrated against under-read values, so moving the settle without recalibrating the margin changes Sweep's peak decisions in an untested direction. Also unexplained: the settled floor is not flat across the band, bin 40 sitting about 8 dB above bins 20 and 66 at every settle from 2 ms up. Co-Authored-By: Claude Opus 5 --- .../2026-09-05-ws17-sweep-settle.md | 72 +++++++++ scripts/ws17_sweep_floor_settle.py | 143 ++++++++++++++++++ 2 files changed, 215 insertions(+) create mode 100644 scripts/ws17_sweep_floor_settle.py diff --git a/docs/hardware-results/2026-09-05-ws17-sweep-settle.md b/docs/hardware-results/2026-09-05-ws17-sweep-settle.md index 3bc66b8..e15271a 100644 --- a/docs/hardware-results/2026-09-05-ws17-sweep-settle.md +++ b/docs/hardware-results/2026-09-05-ws17-sweep-settle.md @@ -64,3 +64,75 @@ every arm. - Decide separately whether the margin constant should be recalibrated against settled readings, or whether Pass A should settle before sampling. Those are different fixes with different costs to lap time. + + +--- + +# Established: the floor stabilises by 3 ms, at ~15% of lap time + +**Raw:** `private/ws17-floor-20260905T181609Z.jsonl`. Eight settle values, +five laps each, five bins, **transmitter idle throughout**. Silence is the +point: the floor is a property of the receiver and its retune, so measuring it +with no source removes the timing problems that invalidated the first attempts +at the detection question. + +| settle | lap time | bin 0 | bin 20 | bin 40 | bin 66 | bin 84 | +|---|---|---|---|---|---|---| +| **0 ms (shipped)** | 1.70 s | -112.2 | -115.1 | -111.6 | -115.0 | -113.6 | +| 1 ms | 1.62 s | -105.3 | -108.4 | -104.2 | -108.5 | -107.0 | +| 2 ms | 3.76 s | -102.8 | -102.3 | -95.1 | -102.0 | -98.8 | +| **3 ms** | 1.74 s | -102.1 | -101.5 | -94.5 | -101.9 | -98.5 | +| 5 ms | 2.01 s | -102.7 | -102.0 | -94.5 | -102.7 | -99.3 | +| 10 ms | 2.10 s | -102.5 | -101.9 | -94.0 | -102.2 | -98.6 | +| 20 ms | 4.13 s | -102.0 | -101.8 | -94.5 | -102.3 | -98.3 | +| 40 ms | 5.60 s | -102.7 | -102.6 | -94.0 | -102.3 | -98.7 | + +**Three milliseconds is enough.** Readings are flat from 3 ms through 40 ms. +The shipped 0 ms configuration under-reads by 9.5 dB (bin 0) to 17.6 dB +(bin 40), and 1 ms recovers only about half of it. + +**The cost is small.** A settle is paid once per bin, so 85 bins multiply it: +3 ms predicts 0.26 s added per lap, and the measured 1.70 -> 1.74 s is that +within noise. 40 ms costs 5.60 s, which is why this is worth pinning down +rather than picking a generous value. Roughly 15% of lap time buys 10-18 dB of +floor accuracy. (The 2 ms and 20 ms lap times are out of trend and are +believed to be timing noise in the host's lap measurement, which includes +status polling; they do not affect the floor readings.) + +## A prediction that was wrong, and what it means + +This measurement was designed with bin 0 as a control: bin 0 takes a full +`begin()` while later bins take the light retune, so if the under-read were a +settle artifact, bin 0 should not move. **It moved 9.5 dB.** Re-reading +`performEnergySweep()`, the settle is skipped only when the *global* FULL +retune mode is on — the condition is `!benchSweepRetuneFullEveryBin()`, not a +per-bin test — so in the shipped LIGHT mode bin 0 receives the full begin +*and* the settle. It was never a control. + +Bin 0 did shift least (+9.5 dB against +12.5 to +17.6 elsewhere), which is +consistent with a full begin providing partial settling on its own, but that +is an observation and not a demonstrated mechanism. The clean version of this +control would compare FULL mode against LIGHT+3 ms across all bins. + +## Also unexplained + +The settled floor is not flat across the band: bin 40 reads about 8 dB above +bins 20 and 66 at every settle value from 2 ms up. That could be real ambient +energy near 912 MHz or a front-end response, and this run does not +distinguish them. It matters because Pass A compares each bin against a floor +plus a margin, and a band-dependent floor is a different situation from a flat +one. + +## What follows + +- A ~3 ms settle in Pass A is cheap and recovers most of the error. It is the + obvious fix, but it cannot be made alone: the shipped margin constant was + calibrated against under-read values, so changing the settle without + recalibrating the margin changes Sweep's peak decisions in an untested + direction. +- This is one environment and one link. The floor is a receiver property so it + should transfer, but it has not been checked at a second location. +- The detection question — can Pass A flag a bin carrying traffic — remains + open and is now better approached with real repeater traffic on two known + channels (bins 34 and 66), using the other 83 bins as within-run controls, + than with a fixture whose presence during a 3 ms visit cannot be guaranteed. diff --git a/scripts/ws17_sweep_floor_settle.py b/scripts/ws17_sweep_floor_settle.py new file mode 100644 index 0000000..9b8f342 --- /dev/null +++ b/scripts/ws17_sweep_floor_settle.py @@ -0,0 +1,143 @@ +#!/usr/bin/env python3 +"""Workstream 17: how far does Pass A's settle time move its noise floor? + +Needs no transmitter. That is the point: the floor is a property of the +receiver and its retune, so measuring it silently removes every source-timing +problem that invalidated the first attempts at the detection question. + +Pass A retunes each bin with standby/setFrequency/startReceive and then samples +immediately -- `ENERGY_SWEEP_SETTLE_DEFAULT_MS` is 0. `src/version.h` records +an 11 dB under-read on Cell from exactly that, and leaves Sweep's case open. +A first probe put Sweep's silent floor 14 dB below a settled reading; this +establishes it properly and finds where the reading stabilises. + +It also measures two things any fix has to trade against each other: + + lap time a settle is paid once per bin, so 85 bins multiply it. This is + the cost side of "make Pass A settle" versus "recalibrate the + margin against under-read values". + bin 0 bin 0 takes a full begin(), later bins take the light retune. + If the under-read is a settle artifact, bin 0 should be + unaffected while the rest are not -- which localises the cause + rather than inferring it. + +Reports floors per bin per settle value. It selects nothing and changes no +firmware; the shipped default is restored on exit. +""" + +import argparse +import pathlib +import statistics +import sys +import time + +import serial + +from bench_harness import (CARD_MARKER, Endpoint, ResultWriter, card_status, + parse_fields, require_ack) + +SWEEP_TERMINAL_TIMEOUT_S = 120.0 +DEFAULT_SETTLES_MS = (0, 1, 2, 3, 5, 10, 20, 40) +DEFAULT_BINS = (0, 20, 40, 66, 84) + + +def run_lap(card): + """One Pass A lap; returns its wall-clock duration in seconds.""" + started = time.monotonic() + require_ack(card, "SWEEP_START", "-", timeout=12.0) + deadline = time.monotonic() + SWEEP_TERMINAL_TIMEOUT_S + while time.monotonic() < deadline: + state = card_status(card).get("W") + if state in {"COMPLETE", "CANCELLED", "FAILED"}: + if state != "COMPLETE": + raise RuntimeError(f"sweep ended {state}") + return time.monotonic() - started + time.sleep(0.05) + raise TimeoutError("sweep did not reach a terminal state") + + +def read_bin(card, index): + opcode, payload = card.request("BENCH_SWEEP_FLOOR", str(index), timeout=8.0) + if opcode != "ACK": + raise RuntimeError(f"BENCH_SWEEP_FLOOR({index}) failed: {opcode} {payload}") + fields = parse_fields(payload) + return int(fields["FLOOR"]) / 10.0 + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument("--cardputer-port", required=True) + parser.add_argument("--log", required=True) + parser.add_argument("--results", required=True) + parser.add_argument("--laps", type=int, default=5, help="laps per settle value") + parser.add_argument("--settles-ms", type=int, action="append", + help=f"repeatable; default {' '.join(map(str, DEFAULT_SETTLES_MS))}") + parser.add_argument("--bins", type=int, action="append", + help=f"repeatable; default {' '.join(map(str, DEFAULT_BINS))}") + args = parser.parse_args() + settles = args.settles_ms or list(DEFAULT_SETTLES_MS) + bins = args.bins or list(DEFAULT_BINS) + + log_path = pathlib.Path(args.log) + log_path.parent.mkdir(parents=True, exist_ok=True) + results = ResultWriter(args.results) + with log_path.open("a", encoding="utf-8") as log: + card = Endpoint("cardputer", args.cardputer_port, CARD_MARKER, log) + try: + identity = require_ack(card, "HELLO", "-", timeout=45.0) + if "BENCH=1" not in identity: + raise RuntimeError(f"needs the bench image: {identity}") + results.write({"event": "boot", "identity": identity, + "settles_ms": settles, "bins": bins, "laps": args.laps}) + require_ack(card, "BENCH_SWEEP_RETUNE", "LIGHT", timeout=8.0) + + table = {} + print(f"{'settle':>7s} {'lap':>8s} " + " ".join(f"bin{b:<4d}" for b in bins)) + for settle in settles: + require_ack(card, "BENCH_SWEEP_SETTLE", str(settle), timeout=8.0) + per_bin = {b: [] for b in bins} + durations = [] + for _ in range(args.laps): + durations.append(run_lap(card)) + for b in bins: + per_bin[b].append(read_bin(card, b)) + med = {b: statistics.median(v) for b, v in per_bin.items()} + table[settle] = med + lap_s = statistics.median(durations) + results.write({"event": "settle", "settle_ms": settle, + "lap_seconds": round(lap_s, 3), + "floors": {str(b): med[b] for b in bins}, + "raw": {str(b): per_bin[b] for b in bins}}) + print(f"{settle:5d}ms {lap_s:7.2f}s " + + " ".join(f"{med[b]:7.1f}" for b in bins), flush=True) + + # bin 0 takes a full begin(); the rest take the light retune. If the + # under-read is a settle artifact it should show that asymmetry. + print() + base = table[max(settles)] + worst = table[min(settles)] + print(f"shift from {min(settles)}ms to {max(settles)}ms settle, per bin:") + for b in bins: + tag = " <- full begin() bin" if b == 0 else "" + print(f" bin {b:3d}: {worst[b]:7.1f} -> {base[b]:7.1f} " + f"({base[b] - worst[b]:+5.1f} dB){tag}") + finally: + # Never leave the bench image on a non-default sweep configuration. + try: + require_ack(card, "BENCH_SWEEP_SETTLE", "0", timeout=8.0) + require_ack(card, "BENCH_SWEEP_RETUNE", "LIGHT", timeout=8.0) + finally: + card.close() + results.close() + + +if __name__ == "__main__": + try: + main() + except KeyboardInterrupt: + print("interrupted; completed settle values are already durable", file=sys.stderr) + sys.exit(130) + except (OSError, serial.SerialException, RuntimeError, TimeoutError, ValueError) as error: + print(f"ws17 floor/settle: {error}", file=sys.stderr) + sys.exit(2) From 44a4c946b700827d97ff19ff6b84e6fce90bc8a1 Mon Sep 17 00:00:00 2001 From: Will Shields Date: Sat, 5 Sep 2026 11:43:21 -0700 Subject: [PATCH 20/21] Pass A misses real repeater traffic because of the settle under-read Fully passive: 68 laps with two live repeaters as the sources and the bench transmitter explicitly quieted, so this injected nothing into a live mesh. Settle alternated 0 ms and 3 ms lap by lap, and seven control bins were read in the same lap as the two traffic bins, so the comparison needs no external ground truth. The margin constant is 35 dB above the noise floor. Applying it to what was measured: at the shipped 0 ms settle the best traffic reading sat 19.9 dB above the floor and would not have been flagged; at 3 ms the same traffic sat 46.3 dB above and would have been. Pass A would have missed both repeaters as shipped and caught both with the settle. The reason a 13 dB floor error becomes a functional failure is that the signal is under-read much harder than the floor: the same repeater burst read -95.3 dBm at 0 ms and -56.5 dBm at 3 ms. That matches version.h's note that the under-read grows with signal strength, and it means the two errors do not cancel -- the excursion the margin tests collapses from 46 dB to 20 dB and falls under the threshold. Deliberately claims no rate. Two detections in 34 laps per configuration is a tiny count, the hit rate is bounded by a ~3 ms bin visit coinciding with a burst rather than by sensitivity, and the 0 ms versus 3 ms hit counts are not distinguishable at that size. The evidence here is the magnitude of the excursion, not the number of hits. Recommendation recorded but not applied: settle and re-derive the margin as one change. Shipping the settle alone would apply a margin calibrated against under-read values to correctly-read ones. Workstream 17 moves to Design entry. The run ended at lap 68 of 200 on a USB-CDC transport timeout; this harness lacks the retry hardening the Focus matrix runner has. Co-Authored-By: Claude Opus 5 --- docs/ROADMAP.md | 2 +- .../2026-09-05-ws17-sweep-settle.md | 76 ++++++++++ scripts/ws17_sweep_traffic_bench.py | 134 ++++++++++++++++++ 3 files changed, 211 insertions(+), 1 deletion(-) create mode 100644 scripts/ws17_sweep_traffic_bench.py diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 8325bcb..fb539ae 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -64,7 +64,7 @@ cost acceptable; the measurement is part of the decision. | **14 — Companion analysis** | Not entered | Deliver an offline, reproducible tool that reads copied run folders without changing original evidence. Test deterministic reports, multi-run comparison, coverage warnings, and privacy-safe export behavior. | | **15 — Field markers and sharing** | Not entered | Add fixed, safe marker presets and `marker.csv`, then integrate redacted sharing. Prove markers cannot affect radio behavior and realistic exports remove selected location/identity detail. | | **16 — Cell closeout** | Deferred bonus | Close the existing V1 Phase 11 evidence gap: a real tower-adjacent RSSI rise plus fresh SD verification of `cell.csv` and Cell's appended `session.csv` fields. This preserves V1 history; it does not renumber it. | -| **17 — Sweep/Waterfall sampling review** | Candidate | Re-evaluate whether Sweep's per-bin sampling and Waterfall's presentation can support what they imply, using the measurement apparatus Workstream 12 built. Entry needs a two-baseline sensitivity measurement, not an argument from analogy. See below. | +| **17 — Sweep/Waterfall sampling review** | **Design entry** | Re-evaluate whether Sweep's per-bin sampling and Waterfall's presentation can support what they imply, using the measurement apparatus Workstream 12 built. Entry needs a two-baseline sensitivity measurement, not an argument from analogy. See below. | ## Candidate — Workstream 17 (Sweep/Waterfall sampling review) diff --git a/docs/hardware-results/2026-09-05-ws17-sweep-settle.md b/docs/hardware-results/2026-09-05-ws17-sweep-settle.md index e15271a..2d36985 100644 --- a/docs/hardware-results/2026-09-05-ws17-sweep-settle.md +++ b/docs/hardware-results/2026-09-05-ws17-sweep-settle.md @@ -136,3 +136,79 @@ one. open and is now better approached with real repeater traffic on two known channels (bins 34 and 66), using the other 83 bins as within-run controls, than with a fixture whose presence during a 3 ms visit cannot be guaranteed. + + +--- + +# The under-read is not cosmetic: Pass A misses real traffic because of it + +**Raw:** `private/ws17-traffic-20260905T182652Z.jsonl`. 68 laps, **fully +passive — this run transmitted nothing.** The sources are two live repeaters +on known channels; the bench transmitter was explicitly quieted. Settle +alternated 0 ms and 3 ms lap by lap so both configurations saw the same +traffic. Ended early at lap 68 of 200 on a USB-CDC transport timeout: this +script lacks the retry hardening the Focus matrix runner has. + +Bins 34 (MeshCore, 910.525 MHz) and 66 (MeshOregon, 918.5 MHz) carry traffic; +seven bins spread across the band are read in the *same lap* as controls, so +the comparison needs no external ground truth. + +| | control median | control max | bin 34 max | bin 66 max | +|---|---|---|---|---| +| 0 ms (shipped), 34 laps | -115.2 | -112.3 | **-95.3** | **-113.8** | +| 3 ms, 34 laps | -102.8 | -96.4 | **-56.5** | **-50.8** | + +Laps where a traffic bin exceeded the control maximum: 2/34 and 0/34 at 0 ms; +2/34 and 2/34 at 3 ms. Adjacent bins (33/35/65/67) were clean at 0/34 except +one 1/34 excursion at bin 67, so energy lands in the intended bin. + +## Pass A's own threshold would reject what it caught + +`ENERGY_DEFAULT_THRESHOLD_MARGIN_DBM_X10` is 350, i.e. a bin must sit **35 dB** +above the noise floor to be flagged as a peak. Applying that to what was +actually measured: + +| | floor | best traffic reading | excursion | flagged? | +|---|---|---|---|---| +| **0 ms (shipped)** | -115.2 | bin 34 at -95.3 | **19.9 dB** | **no** | +| | | bin 66 at -113.8 | 1.4 dB | no | +| **3 ms** | -102.8 | bin 34 at -56.5 | **46.3 dB** | yes | +| | | bin 66 at -50.8 | 52.0 dB | yes | + +**As shipped, Pass A would not have flagged either repeater in this run. With +a 3 ms settle it would have flagged both.** That is the same traffic on +alternating laps, so it is not a difference in conditions. + +This is what makes the settle a defect rather than a calibration curiosity. +The floor measurement showed the noise floor read 9.5-17.6 dB low; this shows +the *signal* is under-read far more — the same repeater burst read -95.3 dBm +at 0 ms and -56.5 dBm at 3 ms, a 39 dB difference — which matches +`version.h`'s note that the under-read grows with signal strength. A floor +that is 13 dB low and a signal that is 39 dB low do not cancel: the excursion +that the margin tests collapses from 46 dB to 20 dB, and falls under the +threshold. + +## What this does and does not support + +Supported: at the shipped settle, real traffic that Pass A physically sampled +did not reach its own flagging threshold, and a 3 ms settle fixed that in the +same run. The mechanism is consistent across the floor sweep, this run, and +the prior Cell finding. + +Not supported: any rate claim. Two detections in 34 laps per configuration is +a tiny count, the hit rate is limited by the ~3 ms bin visit coinciding with a +burst rather than by sensitivity, and 0 ms versus 3 ms hit counts (2 and 0 +against 2 and 2) are not distinguishable at that size. **The decisive evidence +here is the magnitude of the excursion, not the number of hits.** Traffic was +also uncontrolled; alternating laps controls for drift but not for a burst +happening to fall in one arm. + +Also unchanged: one environment, one link, one repeater pair. + +## Recommendation + +Pass A should settle before sampling, and the margin must be re-derived in the +same change. 3 ms costs about 0.26 s of a 1.7 s lap. Shipping the settle alone +would leave a 35 dB margin calibrated against under-read values now being +applied to correctly-read ones, which changes peak decisions in an untested +direction — the fix is one change, not two. diff --git a/scripts/ws17_sweep_traffic_bench.py b/scripts/ws17_sweep_traffic_bench.py new file mode 100644 index 0000000..130a962 --- /dev/null +++ b/scripts/ws17_sweep_traffic_bench.py @@ -0,0 +1,134 @@ +#!/usr/bin/env python3 +"""Workstream 17: does Pass A flag bins that are actually carrying traffic? + +Fully passive. The sources are real repeaters on known channels, so this +transmits nothing -- which is both correct etiquette on a live mesh and a more +realistic test than a fixture whose presence during a 3 ms bin visit cannot be +guaranteed. The bench transmitter is explicitly quieted and never armed. + +The design needs no external ground truth. Two bins are known to carry traffic +and the rest are not, so the quiet bins are the control *within the same lap*: + + bin 34 MeshCore default, 910.525 MHz (reported as the busier of the two) + bin 66 MeshOregon, 918.5 MHz + others controls, sampled across the band + +If Pass A sees traffic, 34 and 66 should be elevated relative to the controls +more often than the controls are relative to each other. If they are not +distinguishable, that is an answer too. + +Settle alternates 0 ms (shipped) and 3 ms (the value the floor measurement +found sufficient) lap by lap, so both configurations see the same traffic +rather than consecutive blocks that could differ in activity. That makes this +a direct test of whether the settle fix improves real detection, not just the +noise floor. +""" + +import argparse +import pathlib +import statistics +import sys +import time + +import serial + +from bench_harness import (CARD_MARKER, TX_MARKER, Endpoint, ResultWriter, + card_status, parse_fields, require_ack) + +SWEEP_TERMINAL_TIMEOUT_S = 120.0 +TRAFFIC_BINS = (34, 66) +# Immediate neighbours, to show whether energy lands only in the intended bin. +ADJACENT_BINS = (33, 35, 65, 67) +CONTROL_BINS = (5, 15, 25, 45, 55, 75, 80) + + +def run_lap(card): + require_ack(card, "SWEEP_START", "-", timeout=12.0) + deadline = time.monotonic() + SWEEP_TERMINAL_TIMEOUT_S + while time.monotonic() < deadline: + state = card_status(card).get("W") + if state in {"COMPLETE", "CANCELLED", "FAILED"}: + if state != "COMPLETE": + raise RuntimeError(f"sweep ended {state}") + return + time.sleep(0.05) + raise TimeoutError("sweep did not reach a terminal state") + + +def read_bin(card, index): + opcode, payload = card.request("BENCH_SWEEP_FLOOR", str(index), timeout=8.0) + if opcode != "ACK": + raise RuntimeError(f"BENCH_SWEEP_FLOOR({index}) failed: {opcode} {payload}") + return int(parse_fields(payload)["FLOOR"]) / 10.0 + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument("--cardputer-port", required=True) + parser.add_argument("--heltec-port", help="quieted if given; never armed") + parser.add_argument("--log", required=True) + parser.add_argument("--results", required=True) + parser.add_argument("--laps", type=int, default=120, help="total laps, alternating settle") + parser.add_argument("--settles-ms", type=int, action="append", + help="repeatable; default 0 and 3, alternated lap by lap") + args = parser.parse_args() + settles = args.settles_ms or [0, 3] + watched = sorted(set(TRAFFIC_BINS + ADJACENT_BINS + CONTROL_BINS)) + + log_path = pathlib.Path(args.log) + log_path.parent.mkdir(parents=True, exist_ok=True) + results = ResultWriter(args.results) + with log_path.open("a", encoding="utf-8") as log: + card = Endpoint("cardputer", args.cardputer_port, CARD_MARKER, log) + transmitter = None + try: + identity = require_ack(card, "HELLO", "-", timeout=45.0) + if "BENCH=1" not in identity: + raise RuntimeError(f"needs the bench image: {identity}") + if args.heltec_port: + # Not a source here. Quieted explicitly so a leftover beacon + # from earlier work cannot be mistaken for repeater traffic. + transmitter = Endpoint("heltec", args.heltec_port, TX_MARKER, log) + require_ack(transmitter, "HELLO", "-", timeout=25.0) + require_ack(transmitter, "QUIET", "-", timeout=8.0) + print("bench transmitter quieted; this run transmits nothing") + require_ack(card, "BENCH_SWEEP_RETUNE", "LIGHT", timeout=8.0) + results.write({"event": "boot", "identity": identity, "settles_ms": settles, + "traffic_bins": list(TRAFFIC_BINS), "adjacent": list(ADJACENT_BINS), + "controls": list(CONTROL_BINS), "laps": args.laps}) + + started = time.monotonic() + for lap in range(args.laps): + settle = settles[lap % len(settles)] + require_ack(card, "BENCH_SWEEP_SETTLE", str(settle), timeout=8.0) + run_lap(card) + floors = {b: read_bin(card, b) for b in watched} + results.write({"event": "lap", "lap": lap, "settle_ms": settle, + "floors": {str(b): v for b, v in floors.items()}, + "elapsed_s": round(time.monotonic() - started, 1)}) + if lap % 10 == 0 or lap == args.laps - 1: + ctl = statistics.median([floors[b] for b in CONTROL_BINS]) + print(f" [{lap + 1:4d}/{args.laps}] settle {settle}ms " + f"bin34 {floors[34]:7.1f} bin66 {floors[66]:7.1f} " + f"controls {ctl:7.1f}", flush=True) + print(f"\n{args.laps} laps in {(time.monotonic() - started) / 60:.1f} min") + finally: + try: + require_ack(card, "BENCH_SWEEP_SETTLE", "0", timeout=8.0) + finally: + if transmitter is not None: + transmitter.close() + card.close() + results.close() + + +if __name__ == "__main__": + try: + main() + except KeyboardInterrupt: + print("interrupted; completed laps are already durable", file=sys.stderr) + sys.exit(130) + except (OSError, serial.SerialException, RuntimeError, TimeoutError, ValueError) as error: + print(f"ws17 sweep traffic: {error}", file=sys.stderr) + sys.exit(2) From c16a7b5b29b698f0cd02978c961931581a4cdfff Mon Sep 17 00:00:00 2001 From: Will Shields Date: Sat, 5 Sep 2026 17:15:11 -0700 Subject: [PATCH 21/21] Confirm at 300 laps: shipped Pass A flags neither busy channel 300 laps, zero skipped, 62.7 minutes, transmitting nothing -- two live repeaters as the sources, alternating 0/3 ms settle so both configurations saw the same traffic, and seven control bins read in the same lap. Judged by Sweep's own 35 dB threshold: the shipped 0 ms configuration flagged neither traffic bin in 150 laps, with a best excursion of 19.0 dB. The settled configuration flagged both, 3/150 each, with a best excursion of 51.6 dB. The shipped arm was not a near miss; it was short of its own threshold by a factor of two in dB. Specificity is clean in both arms. No adjacent bin and no control bin was ever flagged -- about 1,650 non-traffic bin observations per arm, zero false flags. When the settled configuration flags something it flags the two channels that actually carry traffic. The flag rates remain statistically unseparated and the writeup says so: 0/150 is [0.000, 0.025] and 3/150 is [0.007, 0.057], overlapping on [0.007, 0.025]. Separating those needs about 300 laps per arm and this run was sized before that arithmetic was done. It does not weaken the conclusion, because the rate follows from the margin arithmetic rather than evidencing it -- a configuration whose best excursion is 19.0 dB cannot clear 35 dB at any sample size. Also recorded: the ~2% rate at 3 ms is a coincidence rate set by the ~3 ms per-bin visit, not a sensitivity figure, and it binds a fixed Sweep too. Fix still unapplied: settle and re-derive the margin as one change. Co-Authored-By: Claude Opus 5 --- .../2026-09-05-ws17-sweep-settle.md | 62 +++++++++++++++++++ scripts/ws17_sweep_traffic_bench.py | 42 ++++++++++--- 2 files changed, 96 insertions(+), 8 deletions(-) diff --git a/docs/hardware-results/2026-09-05-ws17-sweep-settle.md b/docs/hardware-results/2026-09-05-ws17-sweep-settle.md index 2d36985..261cc8b 100644 --- a/docs/hardware-results/2026-09-05-ws17-sweep-settle.md +++ b/docs/hardware-results/2026-09-05-ws17-sweep-settle.md @@ -212,3 +212,65 @@ same change. 3 ms costs about 0.26 s of a 1.7 s lap. Shipping the settle alone would leave a 35 dB margin calibrated against under-read values now being applied to correctly-read ones, which changes peak decisions in an untested direction — the fix is one change, not two. + + +--- + +# Confirmation at 300 laps + +**Raw:** `private/ws17-traffic2-20260905T231056Z.jsonl`. 300 laps, **0 skipped**, +62.7 minutes, fully passive — the bench transmitter was quieted and this run +transmitted nothing. Same two live repeaters, same alternating 0/3 ms settle, +same within-lap controls. The harness gained retry-on-transport-failure and +skip-a-lap-rather-than-abort after the previous attempt died at lap 68. + +Judged by Sweep's own rule: a bin is flagged when it sits 35 dB +(`ENERGY_DEFAULT_THRESHOLD_MARGIN_DBM_X10`) above the lap's noise floor. + +| | bin 34 (MeshCore) | bin 66 (MeshOregon) | adjacent + control bins | best traffic excursion | +|---|---|---|---|---| +| **0 ms (shipped)** | 0/150, CI [0.000, 0.025] | 0/150, CI [0.000, 0.025] | **0 flagged** | **19.0 dB** | +| **3 ms** | 3/150, CI [0.007, 0.057] | 3/150, CI [0.007, 0.057] | **0 flagged** | **51.6 dB** | + +## What this establishes + +**The shipped configuration never came close to flagging traffic it was +physically sampling.** Its best excursion across 150 laps was 19.0 dB against +a 35 dB threshold — not a near miss, a factor of two in dB terms. The settled +configuration reached 51.6 dB on the same traffic in interleaved laps. + +**Specificity is clean.** Across both arms, no adjacent bin (33/35/65/67) and +no control bin (5/15/25/45/55/75/80) was ever flagged — roughly 1,650 +non-traffic bin observations per arm with zero false flags. When the settled +configuration flags something, it flags the two channels that actually carry +traffic and nothing else. + +**Both repeaters behaved alike at 3 ms** (3/150 each), where the shorter +earlier run had seen MeshOregon twice and MeshCore twice. The per-session +difference in activity the operator described shows up as variance between +runs rather than as a systematic difference here. + +## What this does not establish + +**The flag *rates* are still not statistically separated.** 0/150 gives +[0.000, 0.025] and 3/150 gives [0.007, 0.057]; those overlap on [0.007, +0.025]. Separating a 0% rate from a 2% one needs roughly 300 laps per arm, not +150, and this run was sized before that arithmetic was done. + +That gap does not weaken the conclusion, because the rate is a *consequence* +of the margin arithmetic rather than independent evidence for it: a +configuration whose best excursion is 19.0 dB cannot flag a 35 dB threshold at +any sample size. The rate would only become the load-bearing evidence if the +excursion result were ambiguous, and it is not. + +**The ~2% flag rate at 3 ms is a coincidence rate, not a sensitivity figure.** +Pass A visits each bin for about 3 ms per lap, so it can only flag traffic +that happens to be transmitting during that window. That bound applies equally +to a fixed Sweep and is a property of the design, not of the settle. + +## Status + +This closes Workstream 17's entry question. Pass A's settle under-read is not a +calibration curiosity: it is the difference between flagging the two busiest +channels in the band and flagging nothing at all. The fix — settle, and +re-derive the margin in the same change — remains unapplied and untested. diff --git a/scripts/ws17_sweep_traffic_bench.py b/scripts/ws17_sweep_traffic_bench.py index 130a962..70ed297 100644 --- a/scripts/ws17_sweep_traffic_bench.py +++ b/scripts/ws17_sweep_traffic_bench.py @@ -55,11 +55,25 @@ def run_lap(card): raise TimeoutError("sweep did not reach a terminal state") -def read_bin(card, index): - opcode, payload = card.request("BENCH_SWEEP_FLOOR", str(index), timeout=8.0) - if opcode != "ACK": - raise RuntimeError(f"BENCH_SWEEP_FLOOR({index}) failed: {opcode} {payload}") - return int(parse_fields(payload)["FLOOR"]) / 10.0 +def read_bin(card, index, attempts=3): + """Read one bin's floor, retrying transport failures. + + Native USB-CDC occasionally truncates a reply under sustained round trips; + an earlier run of this bench died at lap 68 of 200 for exactly that. The + query is idempotent -- it reads state the sweep already recorded -- so a + re-send cannot disturb the measurement. + """ + last = None + for attempt in range(attempts): + try: + opcode, payload = card.request("BENCH_SWEEP_FLOOR", str(index), timeout=12.0) + if opcode != "ACK": + raise RuntimeError(f"BENCH_SWEEP_FLOOR({index}): {opcode} {payload}") + return int(parse_fields(payload)["FLOOR"]) / 10.0 + except (RuntimeError, TimeoutError, KeyError, ValueError) as error: + last = error + time.sleep(0.3) + raise RuntimeError(f"BENCH_SWEEP_FLOOR({index}) failed {attempts}x: {last}") def main(): @@ -99,11 +113,23 @@ def main(): "controls": list(CONTROL_BINS), "laps": args.laps}) started = time.monotonic() + skipped = 0 for lap in range(args.laps): settle = settles[lap % len(settles)] require_ack(card, "BENCH_SWEEP_SETTLE", str(settle), timeout=8.0) - run_lap(card) - floors = {b: read_bin(card, b) for b in watched} + # A lost lap is a lost sample, not a lost run: at a ~6% hit + # rate the value is in the count of laps, so aborting on one + # transport failure discards everything gathered so far. + try: + run_lap(card) + floors = {b: read_bin(card, b) for b in watched} + except (RuntimeError, TimeoutError) as error: + skipped += 1 + results.write({"event": "lap_error", "lap": lap, + "settle_ms": settle, "error": str(error)}) + print(f" [{lap + 1:4d}/{args.laps}] skipped: {error}", flush=True) + time.sleep(1.0) + continue results.write({"event": "lap", "lap": lap, "settle_ms": settle, "floors": {str(b): v for b, v in floors.items()}, "elapsed_s": round(time.monotonic() - started, 1)}) @@ -112,7 +138,7 @@ def main(): print(f" [{lap + 1:4d}/{args.laps}] settle {settle}ms " f"bin34 {floors[34]:7.1f} bin66 {floors[66]:7.1f} " f"controls {ctl:7.1f}", flush=True) - print(f"\n{args.laps} laps in {(time.monotonic() - started) / 60:.1f} min") + print(f"\n{args.laps} laps in {(time.monotonic() - started) / 60:.1f} min, {skipped} skipped") finally: try: require_ack(card, "BENCH_SWEEP_SETTLE", "0", timeout=8.0)