diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 6c2585a..9d7c0e7 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -78,7 +78,7 @@ jobs: # Rolling "latest main" build, separate from the versioned vX.Y.Z # releases in release.yml. Overwrites the same dev-latest tag/release # on every merge so there's always a one-click, no-login download for - # SD-drop/Launcher testing — see docs/ROADMAP.md Distribution section. + # SD-drop/Launcher testing — see the root README's install guidance. # Only on pushes to main (not PRs, not every commit on a branch). - name: Move dev-latest tag if: github.ref == 'refs/heads/main' && github.event_name == 'push' diff --git a/AGENTS.md b/AGENTS.md index 9ec4dac..03f4458 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -14,10 +14,12 @@ should be fixed to match, not the other way around. Also relevant, referenced from CLAUDE.md: - [docs/STATUS.md](docs/STATUS.md) — current status, what's hardware-verified, and what's still open. -- [docs/DESIGN.md](docs/DESIGN.md) — the "why" behind RF/architecture - decisions. -- [docs/ROADMAP.md](docs/ROADMAP.md) — phase-by-phase build order and - scope. +- [docs/DESIGN.md](docs/DESIGN.md) — the "why" behind the shipped v1 + RF/architecture foundation. +- [docs/ROADMAP.md](docs/ROADMAP.md) — active V2 workstreams, gates, and + release policy. +- [docs/research/V2_DESIGN.md](docs/research/V2_DESIGN.md) — V2 product + boundaries and design direction; read it before V2 architecture work. - [CHANGELOG.md](CHANGELOG.md) — short, ongoing changelog; full pre-2026-08-29 session-by-session decisions log is archived at [docs/history/CHANGELOG.md](docs/history/CHANGELOG.md). @@ -25,8 +27,8 @@ Also relevant, referenced from CLAUDE.md: - [docs/HARDWARE_TESTING.md](docs/HARDWARE_TESTING.md) — repeatable bench matrix and Phase 7 memory acceptance rules. -**Don't read `docs/history/PROGRESS.md` or `docs/history/CHANGELOG.md` -end-to-end by default.** They're a frozen pre-2026-08-29 development log, -not required context for every task — `docs/STATUS.md` already gives the -current-state summary. Search them for the specific date/version/topic you -need instead of reading front to back. +**Don't read `docs/history/PROGRESS.md`, `docs/history/CHANGELOG.md`, or +`docs/history/ROADMAP_V1.md` end-to-end by default.** They're historical +records, not required context for active V2 work — `docs/STATUS.md` and the +active roadmap give the current-state summary. Search history for the specific +date/version/topic you need instead of reading front to back. diff --git a/CHANGELOG.md b/CHANGELOG.md index 56dbbb5..632de83 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,202 @@ project (not a log of how it got there), see [docs/STATUS.md](docs/STATUS.md). ## 2026-09-05 +- Measured the positive case the withdrawal left open: **a single packet is + detectable**. One armed 148 ms transmission placed inside a 2,000 ms pass — + 7.4% occupancy — was caught 29/30 with 0/30 false positives at the stronger + link. A 148 ms packet sampled every 20 ms yields about seven elevated + samples against the four the rule needs, so nothing about the instrument + prevented it; the first link did. Not unconditional, though: the same rule + at the weaker link failed on sources filling four times as much of the pass, + and the companion position carrying real traffic measured *worse* — a + -57 dBm event in a control trial both produced false positives and + suppressed counts by lifting the median, so Focus is least reliable exactly + where a band is busiest. Also discarded a placement analysis that tried to + time the pulse against the window from log timestamps: `TX_STARTED` reaches + the host only when the harness next polls, so it records host polling rather + than RF timing, and its output was self-contradictory. + +- Ran a second baseline at ~25 dB SNR (600 trials) and **withdrew** the + occupancy conclusion drawn from the first. Replacing the transmitter's + stubby with a matched whip and standing both antennas vertical gained ~13 dB + — vertical alignment mattered more than the antenna, the horizontal pair + having sat in each other's pattern null at 918.5 MHz. At 28.6% occupancy + that took detection from 37% to 87%, so the "cliff" was a property of the + first link, not a statistical limit of the instrument. The claim that the + rule "detects a persistently occupied channel, not individual packets" is + therefore withdrawn, along with the argument that the evidence favours CAD + over sampling for §3's activity basis. Two conclusions did survive both + links: the 100 ms six-sample floor (13 dB bought almost nothing) and the + rule's fraction-of-samples form. Recorded as a method note too — three + conclusions came from one link, two held and the most confident one + inverted. + +- Bounded the count rule with two more sweeps (480 trials, no arm failures). + It is a **fraction** of accepted samples (~4-5%), which transfers between + 500 ms and 2,000 ms passes; no threshold rescues a 100 ms pass, whose six + samples cannot both catch the source and reject ambient — so a short pass + may report coverage and must not report activity. The occupancy sweep is the + consequential one: detection runs 90-93% while the source fills 43-57% of + the pass and collapses to 37-43% at 28.6%. A single SF8 packet inside a + 2,000 ms pass is 3-7% occupancy, far below that. **The rule detects a + persistently occupied channel, not individual packets**, which is the honest + scope of an RSSI-sampling instrument and makes CAD or packet reception the + better-supported basis for §3's observed activity. Median-as-floor held at + every occupancy tested, but only because a weak source leaves most samples + reading like noise; a strong source would break it and that regime was not + reached. + +- Found an activity basis that survives field levels. After every RSSI summary + statistic was rejected, the remaining idea was to count samples above an + adaptive floor rather than take an extreme of them — `peak` is one sample and + noise-prone; a count integrates. Bench images now report a ladder of counts + at median + 2/4/6/8/10/15/20 dB, so one run evaluates any such rule offline + instead of reflashing per candidate. Across 120 trials at the same porch + configuration, `C6 >= 2` detected 57/60 source-on (95% CI [0.863, 0.983]) + against 1/60 source-off — and that one flagged control trial read -63 dBm + against a -101 dBm median, a real transmission that was not ours, so the + false rate is an upper bound rather than a measurement against silence. + Still a candidate, not a constant: the threshold is a count out of 101 + samples, and since sample count now scales with dwell, "2 samples" is 2% at + 2,000 ms and 33% at 100 ms. `qualifying_count` stays unpopulated until that + is expressed as a fraction or validated per dwell. + +- Ran Workstream 12's §6.3 Watch-opportunity comparison. Two 240 s arms + against one independently timed reference train: Watch alone received 0.883, + Watch with Focus interleaved at a 48.1% away fraction received 0.463, + intervals non-overlapping, zero CRC errors either side. The loss is + proportional to away time and nothing else — 0.459 predicted from the + baseline against 0.463 measured — which means Focus's recorded away duration + is an honest proxy for what a request costs Watch, with no hidden retune or + recovery penalty. It also means there is no mitigation: half the listening + time away is half the packets, with restored-Watch gaps reaching 7.45 s. The + away-time budget decision stays open on purpose; the measurement gives the + exchange rate, not the policy. + +- Measured Focus's qualifying RSSI condition at a realistic signal level and + **rejected it**. The `p90 >= -90 dBm` candidate came from a bench source ~70 + dB above ambient, where every metric separates and the choice looks easy. + Repeated with the transmitter outdoors (source peak ~-85 dBm against a + -99/-100 dBm floor, 120 trials, zero drops), it fails — and so does every + floor-relative variant tested: one of ten metric/position combinations + separates, by 2 dB, which is inside ordinary RSSI variance. Two reasons: at + low SNR `p90` collapses even when the source radiates most of the window, + and real MeshOregon traffic reached -94 dBm during control trials, which no + RSSI condition can distinguish from a controlled source. Coverage reporting + is unaffected; what the evidence refuses is the step from "RSSI was + elevated" to "something transmitted", which §3 forbids anyway. Remaining + routes: a per-pass count above an adaptive floor (`qualifying_count` is + reserved and unpopulated), or CAD/packet evidence. + +- Selected Focus's sampling policy from measurement instead of reasoning: + `FOCUS_SAMPLE_SPACING_MS = 20`, with the sample count derived from the dwell + (`ceil(dwell/spacing) + 1`) rather than fixed at 8. A 360-trial sweep held + dwell at 2,000 ms and varied only the spacing: a 94 ms source was missed at + 286 ms and 100 ms spacing (8/15 and 2/15) and caught 15/15 at both 50 ms and + 20 ms, with worst-case reading improving from -97 to -66 dBm between them. + Detection collapses once spacing approaches the source's airtime. Finer + sampling turned out to be free -- radio-away measured 2,073-2,075 ms whether + a pass took 8 samples or 101 -- so the only real argument for coarse + sampling was one nobody had checked. Evidence appended to + `docs/hardware-results/2026-09-04-phase12-focus-matrix.md`. + +- Ran Workstream 12's §6.2 controlled dwell matrix: 900 trials in 54.7 min, + zero transport errors, zero drops, home restored on every trial; 14 of 15 + arms separated a controlled source from ambient 30/30 vs 0/30. The finding + that matters is a limit — **detection tracks the source's airtime against + Focus's sample spacing, not dwell length**. With a fixed 8 samples, spacing + is dwell/7, so a 2000 ms dwell observes eight instants rather than 2000 ms; + worst-case detection degraded monotonically as source airtime approached + that spacing and one arm stopped separating. That is §3's "observation time + is not coverage" rule with numbers behind it, and it means a later slice + should scale samples with dwell. Bin-center offset turned out not to be the + dominant term after all. No single fixed RSSI condition separates every arm, + so §3.1 gets a candidate rather than a constant. Summary in + `docs/hardware-results/2026-09-04-phase12-focus-matrix.md`. + +- Two methodology bugs found by running the §6.2 matrix rather than by + reading it. A single armed pulse at a fixed delay cannot cover every dwell + arm — at 100 ms the pulse began after the window closed, and source-on was + indistinguishable from source-off across 18 trials. Re-arming continuously + fixed that but at SF12's ~275 ms airtime the sends overlapped and their tail + bled into the next trial, making source-off read -26 dBm. The runner now + paces the burst by the transmitter's own `TX_DONE` and waits for the tail to + clear before a trial ends: 100 ms arms then separate cleanly (-25/-27 dBm + against -97/-101 dBm). Both failure modes are recorded in the design doc so + the next harness doesn't rediscover them. + +- Added `scripts/phase12_watch_opportunity.py` for §6.3, the measurement that + gates Focus's maximum radio-away budget: Watch-only versus Watch-with-Focus + against one independently timed pulse train, reported as Wilson intervals + plus away time, completed requests, and restored-Watch gap distribution. It + refuses to run unless the device's home channel matches the reference train, + since otherwise both arms hear nothing and their equality would read as a + reassuring result, and it reports the away fraction alongside the loss + because a small loss at a small duty cycle predicts nothing about a larger + one. + +- Built Workstream 12's §6.2 matrix tooling: `scripts/phase12_focus_matrix.py` + runs paired source-on/source-off Focus trials across dwell arms and writes + one durable JSONL row each, and `scripts/phase12_focus_matrix_report.py` does + the offline analysis — per-arm distributions, the lowest separating 1 dB + condition or an explicit "these overlap", and Wilson 95% intervals rather + than bare proportions. Neither can emit a coverage label. Validated on + hardware with a 4-trial smoke run at 912.750 MHz: source-on P90 -66/-68 dBm + against -97/-98 dBm quiet, 573 ms radio-away, no drops. + +- Found that none of §6.1's three fixture frequencies sits on a Focus bin + center. Focus tunes at the home channel's bandwidth (125 kHz here), so the + offset changes what a pass can observe: low and mid are +62.5 kHz off bins 13 + and 43, and the high position sits exactly on a bin boundary, 125 kHz from + either neighbour — outside the passband. A null there would measure geometry, + not sensitivity, so the matrix now also runs two exactly bin-centered + controls from the same transmitter table, and every trial records its offset. + +- Closed Workstream 12's last Engineering-gate item by measuring the Focus + budgets off the built image instead of estimating them + (`docs/research/phase12-survey-truth-design.md` §4.2): 12 B request, 40 B + result, 148 B histogram, 188 B working state against a 256 B target, 159 B + of Focus statics, a 1,072 B `radioTask` frame in a 6,144 B stack, and a + 189 B worst-case `focus.csv` row against a 256 B buffer that drops rather + than truncates. Everything landed inside the bounds §4.1 had set, so + nothing needed resizing. The row buffer now comes from one shared + `FOCUS_CSV_ROW_MAX` and a test formats the saturated row, so the writer and + its budget can't drift the way `STATUS`'s hand-sized buffer did. + +- Phase 12: bounded a Focus request in wall-clock time, not just in samples. + Each sample waits on the shared SPI bus, so a contended bus could stretch a + pass past its dwell with the radio away from home and nothing to stop it — + `focusRuntimeTimeout()` existed but no code path reached it. A request now + carries a dwell-plus-slack deadline and terminates as `timeout`, restoring + home like every other exit. Bench-only additions make the remaining Device + gate reproducible: `BENCH_FOCUS`'s optional 4th field arms a one-shot + sample-loop stall (production only times out under real contention), and + `BENCH_ACTION` starts/cancels/reports Cell and Scope, which are menu-only in + production, so Focus's mutual exclusion against them can be driven by a + fixture. Verified on hardware the same day: a stalled request timed out, + restored home, and wrote its durable row inside a 1,573 ms radio-away + window, and Cell and Scope each arbitrated with Focus in both directions. + +- Fixed a latent silent-drop in Serial Control's `STATUS`: its argument buffer + was hand-sized to 240 bytes against a frame budget of ~230, and + `serialControlFormatFrame()` drops an over-long frame rather than truncating + — so a long session's wider counters would have lost the whole status frame, + newest fields first, with no error. The argument budget is now derived from + the frame size, the frame limit is 384, and a host test formats at + saturation. The formatter also builds in place instead of staging through a + second full-size buffer, which nets less stack on the 4KB UI task than + before. + +- Phase 12 engineering: wired one bounded, bench-only Focus Survey through + Core 1, the Core-0 `focus.csv` writer, and restore-before-publish terminal + state. Added framed `BENCH_FOCUS` and GPS-free compact + `BENCH_FOCUS_RESULT` readback, durable-write health counters, and the + explicit-transmit-gated `scripts/phase12_focus_bench.py` plus + non-transmitting `scripts/phase12_focus_behavior_bench.py` fixture + harnesses. Quiet/source-on, cancel, injected-failure, and Probe/Sweep + arbitration hardware checks completed with successful home restore and SD + rows; they do not set a coverage/activity threshold. + - `pages.yml`'s "Download stable release assets" step now retries (6x/20s) instead of failing hard: a maintainer publishing a draft release before release.yml's build job finishes uploading assets makes `release: @@ -78,6 +274,34 @@ project (not a log of how it got there), see [docs/STATUS.md](docs/STATUS.md). ## 2026-09-04 +- Archived the completed v1 roadmap at the immutable `v1.0.7` tag and + streamlined `docs/ROADMAP.md` into the active V2 gate board. Updated + `CLAUDE.md`, `AGENTS.md`, and documentation indexes so future work opens + V2 gates/design first and searches v1 history only when needed. + +- Opened V2 Workstream 12 (Survey truth) at Design entry: one selected-bin + Focus Survey, fixed streaming statistics, durable `focus.csv` contract, + controlled transmitter/RTL-SDR truthfulness matrix, and Portland metro, + Oregon as the privacy-preserving field-validation area. No firmware scope + has entered implementation; radio-away and coverage thresholds remain + evidence-gated. + +- Began Workstream 12 Engineering with host-tested, radio-free Focus request + and `focus.csv` contracts: a single sourced bin, 1 dB bounded RSSI histogram + (median/P90/peak), explicit completion states, and an intentionally blank + `coverage` column. No menu, radio task, logger, or coverage conclusion is + wired yet. + +- Tightened the Focus bench-request contract before radio integration: exactly + one pass, 2--2,000 ms dwell, and 2--64 samples. Added a host-tested state + machine that records restoration before a terminal result and makes restore + failure override partial/cancelled work. + +- Wired the bench-only Focus vertical slice: a Core-1 one-bin RSSI request, + mutually exclusive ownership/restoration, four-row Core-1-to-Core-0 queue, + durable `focus.csv`, and framed `BENCH_FOCUS bin:dwell_ms:samples` control. + The production image rejects the command; hardware evidence is still open. + - Ran a `/code-review` pass and then a whole-project audit (`docs/research/2026-09-04-project-audit.md`), and fixed what it found (`v1.0.6`). Radio-ownership bugs in v1.0.5's new capture window: diff --git a/CLAUDE.md b/CLAUDE.md index 236d345..fd95e82 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -28,9 +28,10 @@ here depends on that choice except the specific API calls. ## Proposed layout -`[x]` created, `[ ]` proposed but deferred until its build-order phase -(docs/DESIGN.md §9) — see docs/ROADMAP.md/docs/STATUS.md for why the -task/queue files aren't scaffolded yet. +`[x]` created, `[ ]` a deferred legacy candidate. V1's build order is frozen +at `docs/history/ROADMAP_V1.md`; active V2 workstreams and gates live in +`docs/ROADMAP.md`, with detailed product decisions in +`docs/research/V2_DESIGN.md`. ``` src/ @@ -73,7 +74,7 @@ src/ --- added during phase 8 foundation, not in the original proposal --- [x] discovery_plan.h # fixed, source-backed candidate tuples [x] scan_observation.h # fixed CAD result record + probe.csv formatter - --- added during phase 11, out of sequence (see docs/ROADMAP.md) --- + --- added during V1 Phase 11, out of sequence (see docs/history/ROADMAP_V1.md) --- [x] cell_plan.h # Cell: 869-894MHz cell-band bin math [x] cell_observation.h # fixed RSSI-reading record + cell.csv formatter --- present since earlier phases but never listed here until the @@ -81,6 +82,9 @@ src/ [x] radio_task.h # (listed above as radio_task.cpp/.h) [x] energy_plan.h # Sweep's frequency-bin math (phase 9) [x] energy_observation.h # Pass-A working stats + energy.csv record + [x] focus_plan.h # Phase 12 one-bin Focus request contract + [x] focus_observation.h # Phase 12 bounded RSSI stats + focus.csv row + [x] focus_runtime.h # Phase 12 restore-before-publish state contract [x] pass_b_plan.h # Pass-B SF/BW candidate table + confidence [x] region_plan.h # Region enum shared by Sweep's band choice [x] region_settings.h / .cpp # System > Tuning > Region, /loratrace/region.txt @@ -99,6 +103,9 @@ src/ [x] bench_fault.h / .cpp # bench-image-only fault/override hooks test/ [x] test_channel_plans/ # host-native unit tests, pio test -e native + [x] test_focus_plan/ # one-bin request/source/selection math (phase 12) + [x] test_focus_observation/ # bounded RSSI summaries + focus.csv (phase 12) + [x] test_focus_runtime/ # bounded request terminal/restore rules (phase 12) [x] test_detection/ # queue-record + CSV, fixtures are REAL captured packets [x] test_gps_parse/ # NMEA -> fix, mostly about REFUSING a bad position [x] test_keyboard/ # raw TCA8418 event byte -> KeyAction, allowlist-only @@ -120,9 +127,12 @@ docs/ [x] STATUS.md # current version, hardware-verified state, open items [x] DESIGN.md [x] ROADMAP.md + [x] research/V2_DESIGN.md # V2 product direction and permanent boundaries + [x] research/phase12-survey-truth-design.md # active Workstream 12 design entry [x] LOG_GUIDE.md [x] HARDWARE_TESTING.md [x] BRAND.md + [x] history/ROADMAP_V1.md # immutable v1.0.7 roadmap snapshot pointer [x] history/PROGRESS.md # frozen pre-2026-08-29 status doc, unedited [x] history/CHANGELOG.md # frozen pre-2026-08-29 session log, unedited [x] CHANGELOG.md # short, terse, ongoing — points at docs/history/ for the rest @@ -153,17 +163,18 @@ docs/ a web-UI-only setting or a silent default. Established 2026-08-25 after Phase 5's menu grew a third item (verbose debug) the same day it shipped, with no framework change to absorb it — see - docs/history/CHANGELOG.md and docs/ROADMAP.md's Phase 6 (UI architecture - redesign). + docs/history/CHANGELOG.md and the V1 Phase 6 entry in + docs/history/ROADMAP_V1.md (UI architecture redesign). Doesn't apply to one-shot boot-time config (channel overrides, `config.txt`) — this is about anything that changes runtime behavior while the device is already running. -- **Bump `src/version.h` when a phase lands.** `MAJOR.MINOR` tracks the - build-order phase (docs/ROADMAP.md Versioning: v0.1.x = phase 1, v0.2.x = - phase 2, ...), `PATCH` for fixes adding no phase scope. It is bumped by - hand on purpose — it asserts "this phase is reached", and a number that - auto-increments every build asserts nothing. `release.yml` now fails a - tag whose version doesn't match the header, so a mismatch can't ship. +- **Bump `src/version.h` only when its release gate lands.** In the completed + v1 series, `MAJOR.MINOR` tracks the reached build-order phase and `PATCH` + fixes scope; V2 uses the workstream/release policy in `docs/ROADMAP.md`. + The number is bumped by hand on purpose — it asserts that a gate is reached, + and a number that auto-increments every build asserts nothing. + `release.yml` now fails a tag whose version doesn't match the header, so a + mismatch can't ship. Build *provenance* is the automated half: `scripts/build_rev.py` injects `FIRMWARE_BUILD_REV` (git short SHA, `-dirty` when the tree is modified) into every build, because a dozen `dev-latest` binaries otherwise share @@ -206,14 +217,15 @@ docs/ See [docs/STATUS.md](docs/STATUS.md) for the current version, what's hardware-verified, and what's still open — the single current-state doc, -kept instead of duplicating status prose here. Build order is -`docs/DESIGN.md` §9; `docs/ROADMAP.md` has the phase-by-phase scope. - -**Don't read `docs/history/PROGRESS.md` or `docs/history/CHANGELOG.md` -end-to-end by default** — they're a frozen pre-2026-08-29 development log, -not required context for every task. Grep them for a specific date/topic -instead of reading front to back; root `CHANGELOG.md` covers what's -changed since. +kept instead of duplicating status prose here. Active V2 gates are in +`docs/ROADMAP.md`; `docs/research/V2_DESIGN.md` is the product/design source. +The completed V1 build order remains in `docs/history/ROADMAP_V1.md`. + +**Don't read `docs/history/PROGRESS.md`, `docs/history/CHANGELOG.md`, or +`docs/history/ROADMAP_V1.md` end-to-end by default** — they are historical +records, not required context for active V2 work. Search them for a specific +date, version, or topic instead; root `CHANGELOG.md` covers what's changed +since. Three hard-won rules from Phases 1–2, worth not relearning: - **The IO expander's P0 powers the GPS as well as switching the RF antenna diff --git a/README.md b/README.md index dfc5844..b1aff0b 100644 --- a/README.md +++ b/README.md @@ -64,24 +64,23 @@ not a laptop-tethered tool. ## Version -No tagged `vX.Y.Z` release has been published yet — `main` builds a -rolling +The stable release is **v1.0.7**. `main` also builds a rolling [`dev-latest`](https://github.com/d3mocide/LoRaTrace-RX/releases/tag/dev-latest) -prerelease on every merge (see [Install](#flash-from-your-browser) below), -which is what the **dev-latest** badge above tracks. The firmware's own -semantic version lives in `src/version.h` and is bumped by hand when a -build-order phase lands — see -**[docs/STATUS.md](docs/STATUS.md)** for the current version, what's -hardware-verified, and what's still open, rather than a number restated -here that can drift out of sync. +prerelease on every merge for hardware iteration (see +[Install](#flash-from-your-browser)); it is not a stable release. The firmware +semantic version lives in `src/version.h` and is bumped by hand only when its +release gate lands. See **[docs/STATUS.md](docs/STATUS.md)** for current +hardware verification and open work rather than treating this summary as a +release record. ## Documentation | Doc | For | |---|---| | **[docs/STATUS.md](docs/STATUS.md)** | Current version, what's hardware-verified, what's still open. Start here. | -| [docs/DESIGN.md](docs/DESIGN.md) | Hardware, RF parameters, and architecture rationale — read before making architecture changes. | -| [docs/ROADMAP.md](docs/ROADMAP.md) | Build-order phases, MVP-Beta scope, and an honest feasibility read against this hardware's real limits. | +| [docs/DESIGN.md](docs/DESIGN.md) | Shipped v1 hardware, RF parameters, and architecture rationale — read before changing those foundations. | +| [docs/ROADMAP.md](docs/ROADMAP.md) | Active V2 workstreams, gates, and release policy. | +| [docs/research/V2_DESIGN.md](docs/research/V2_DESIGN.md) | V2 product direction and permanent boundaries. | | [docs/LOG_GUIDE.md](docs/LOG_GUIDE.md) | Operator guide to run folders, CSV fields, identity observations, health checks, and privacy-aware export. | | [docs/HARDWARE_TESTING.md](docs/HARDWARE_TESTING.md) | Repeatable device-validation matrix and Phase 7 memory acceptance rules. | | [docs/BRAND.md](docs/BRAND.md) | Naming, tone, and on-device UI copy conventions. | diff --git a/bench/heltec-v4r8-transmitter/src/main.cpp b/bench/heltec-v4r8-transmitter/src/main.cpp index f23b8d2..321ef4a 100644 --- a/bench/heltec-v4r8-transmitter/src/main.cpp +++ b/bench/heltec-v4r8-transmitter/src/main.cpp @@ -1,6 +1,8 @@ #include +#include #include #include +#include #include #include @@ -68,7 +70,7 @@ constexpr uint32_t BUTTON_LONG_PRESS_MS = 1500; constexpr uint32_t BUTTON_PANIC_STOP_MS = 5000; constexpr uint16_t FIRE_BUTTON_DELAY_MS = 250; // same delay phase8_bench.py's own ARM calls typically use -enum class MenuScreen : uint8_t { HOME = 0, CANDIDATE, FIRE, SWEEP, BEACON, COUNT }; +enum class MenuScreen : uint8_t { HOME = 0, CANDIDATE, FIRE, SWEEP, BEACON, WIFI, COUNT }; // Bench-only field/range-test beacon: periodic capped-count TX pulses at // the selected candidate, for verifying Cardputer detection/range with a @@ -123,6 +125,53 @@ bool buttonActionFired = false; bool buttonPanicFired = false; uint32_t buttonPressedAtMs = 0; bool beaconActive = false; + +// --- Optional WiFi control bridge --------------------------------------- +// +// Exists so the transmitter can be physically separated from the receiver +// (across a room, outdoors) while a host still drives it: bench measurements +// need real path loss between the two boards, and USB does not reach. The +// bridge speaks the exact same @LTTX/1 framed protocol as USB, so the host +// harness needs no changes -- bench_harness.py already opens ports through +// serial_for_url(), which accepts socket://host:port. +// +// SECURITY: this listens on the local network and its commands key a +// transmitter. It is off unless explicitly enabled, but once enabled there is +// no authentication -- anything that can reach the port can trigger a pulse. +// That is acceptable only because this is a bench fixture on a trusted LAN, +// output is capped at TX_POWER_DBM, and every transmit path is bounded. Do +// not expose the port beyond a trusted network, and turn the bridge off +// (WIFI_OFF) when the fixture is not in use. +// +// Credentials live in NVS, set once over USB, and are never stored in the +// repository. The frame grammar splits on spaces, so an SSID or password +// containing a space cannot be carried by WIFI_SSID/WIFI_PASS. +constexpr uint16_t WIFI_BRIDGE_PORT = 4227; +constexpr char WIFI_NVS_NAMESPACE[] = "lttxwifi"; +constexpr uint32_t WIFI_RETRY_INTERVAL_MS = 15000; + +WiFiServer bridgeServer(WIFI_BRIDGE_PORT); +WiFiClient bridgeClient; +char bridgeInput[FRAME_MAX] = {}; +size_t bridgeInputLength = 0; +bool wifiEnabled = false; +bool bridgeListening = false; +uint32_t lastWifiAttemptMs = 0; +String wifiSsid; +String wifiPass; + +// Replies follow whichever transport issued the command, and stay there until +// another command arrives -- ARM's asynchronous TX_STARTED/TX_DONE are emitted +// from loop() long after the command returned, and a host driving the bridge +// must receive them rather than have them go out the USB port nobody is +// reading. +Print *replyStream = &Serial; + +// Defined below pollSerial(); declared here because handle() dispatches the +// WIFI_* commands before those definitions are in scope. +void wifiPersist(); +void wifiStart(); +void wifiStop(); uint16_t beaconPulseCount = 0; uint32_t beaconNextAtMs = 0; @@ -133,6 +182,7 @@ const char *screenLabel(MenuScreen screen) { case MenuScreen::FIRE: return "FIRE"; case MenuScreen::SWEEP: return "SWEEP"; case MenuScreen::BEACON: return "BEACON"; + case MenuScreen::WIFI: return "WIFI"; default: return "?"; } } @@ -141,6 +191,12 @@ void drawMenu() { display.clearBuffer(); display.setFont(u8g2_font_6x10_tf); display.drawStr(0, 10, screenLabel(currentScreen)); + // Right-aligned so it never collides with the longest screen label. + const char *wifiBadge = !wifiEnabled ? "" : (WiFi.status() == WL_CONNECTED ? "wifi:ok" + : "wifi:.."); + if (wifiBadge[0] != '\0') { + display.drawStr(128 - (int)(strlen(wifiBadge) * 6), 10, wifiBadge); + } char line1[24] = {}; char line2[24] = {}; @@ -183,6 +239,22 @@ void drawMenu() { (unsigned)BEACON_MAX_PULSES); snprintf(line3, sizeof(line3), "%s", active->name); break; + case MenuScreen::WIFI: + if (!wifiEnabled) { + snprintf(line1, sizeof(line1), "Bridge: OFF"); + snprintf(line2, sizeof(line2), "USB control only"); + } else if (WiFi.status() == WL_CONNECTED) { + snprintf(line1, sizeof(line1), "Bridge: LISTENING"); + snprintf(line2, sizeof(line2), "%s", wifiSsid.c_str()); + snprintf(line3, sizeof(line3), "%s", WiFi.localIP().toString().c_str()); + snprintf(line4, sizeof(line4), "port %u", (unsigned)WIFI_BRIDGE_PORT); + haveLine4 = true; + } else { + snprintf(line1, sizeof(line1), "Bridge: JOINING"); + snprintf(line2, sizeof(line2), "%s", wifiSsid.c_str()); + snprintf(line3, sizeof(line3), "not connected"); + } + break; default: break; } @@ -264,7 +336,7 @@ void reply(uint16_t sequence, const char *event, const char *argument) { const int n = snprintf(body, sizeof(body), "@LTTX/1 %u %s %s", (unsigned)sequence, event, argument); if (n < 0 || (size_t)n >= sizeof(body)) return; - Serial.printf("%s %04X\n", body, (unsigned)crc16(body, (size_t)n)); + replyStream->printf("%s %04X\n", body, (unsigned)crc16(body, (size_t)n)); } bool configure(const Candidate *candidate) { @@ -500,17 +572,184 @@ void handle(char *line) { } else { reply(sequence, "ERROR", "BAD_ARG"); } + } else if (strcmp(command, "WIFI_SSID") == 0) { + // USB only on purpose: credentials must not be settable through the + // bridge they authorize, and a wrong value taken over the bridge would + // disconnect the very transport that sent it. + if (replyStream != &Serial) { + reply(sequence, "ERROR", "USB_ONLY"); + } else if (strlen(argument) == 0 || strlen(argument) > 32) { + reply(sequence, "ERROR", "BAD_SSID"); + } else { + wifiSsid = argument; + wifiPersist(); + reply(sequence, "ACK", "SSID_SET"); + } + } else if (strcmp(command, "WIFI_PASS") == 0) { + if (replyStream != &Serial) { + reply(sequence, "ERROR", "USB_ONLY"); + } else if (strlen(argument) > 63) { + reply(sequence, "ERROR", "BAD_PASS"); + } else { + // "-" clears it, for an open network. + wifiPass = (strcmp(argument, "-") == 0) ? "" : argument; + wifiPersist(); + reply(sequence, "ACK", "PASS_SET"); + } + } else if (strcmp(command, "WIFI_ON") == 0) { + if (wifiSsid.length() == 0) { + reply(sequence, "ERROR", "NO_SSID"); + } else { + wifiEnabled = true; + wifiPersist(); + wifiStart(); + reply(sequence, "ACK", "ENABLED"); + } + } else if (strcmp(command, "WIFI_OFF") == 0) { + wifiEnabled = false; + wifiPersist(); + // Answer before tearing the transport down, or a bridge client never + // learns its own command succeeded. + reply(sequence, "ACK", "DISABLED"); + wifiStop(); + } else if (strcmp(command, "NET_TEST") == 0) { + // Diagnostic only: does an *outbound* connection work? A station can + // be associated, hold a lease, and serve nothing reachable if inbound + // frames are being dropped somewhere. Outbound is board-initiated and + // does not depend on receiving a broadcast ARP, so comparing the two + // directions says whether a reverse-dialled bridge would help. + char target[48] = {}; + if (strlen(argument) >= sizeof(target)) { + reply(sequence, "ERROR", "BAD_TARGET"); + } else { + strcpy(target, argument); + char *colon = strrchr(target, ':'); + uint16_t port = 0; + if (colon == nullptr || !parseU16(colon + 1, port)) { + reply(sequence, "ERROR", "BAD_TARGET"); + } else { + *colon = '\0'; + WiFiClient probe; + probe.setTimeout(4); + const bool ok = probe.connect(target, port); + char detail[64] = {}; + snprintf(detail, sizeof(detail), "%s TO=%s:%u FROM=%s", + ok ? "OPEN" : "REFUSED", target, (unsigned)port, + WiFi.localIP().toString().c_str()); + probe.stop(); + reply(sequence, ok ? "ACK" : "ERROR", detail); + } + } + } else if (strcmp(command, "WIFI_STATUS") == 0) { + char status[80] = {}; + snprintf(status, sizeof(status), "EN=%u;CONN=%u;IP=%s;PORT=%u;SSID=%s", + wifiEnabled ? 1U : 0U, WiFi.status() == WL_CONNECTED ? 1U : 0U, + WiFi.status() == WL_CONNECTED ? WiFi.localIP().toString().c_str() : "-", + (unsigned)WIFI_BRIDGE_PORT, + wifiSsid.length() ? wifiSsid.c_str() : "-"); + reply(sequence, "STATUS", status); } else { reply(sequence, "ERROR", "UNSUPPORTED"); } } +void wifiPersist() { + Preferences prefs; + if (!prefs.begin(WIFI_NVS_NAMESPACE, false)) return; + prefs.putString("ssid", wifiSsid); + prefs.putString("pass", wifiPass); + prefs.putBool("enabled", wifiEnabled); + prefs.end(); +} + +void wifiLoadPersisted() { + Preferences prefs; + if (!prefs.begin(WIFI_NVS_NAMESPACE, true)) return; + wifiSsid = prefs.getString("ssid", ""); + wifiPass = prefs.getString("pass", ""); + wifiEnabled = prefs.getBool("enabled", false); + prefs.end(); +} + +void wifiStop() { + if (bridgeClient) bridgeClient.stop(); + if (bridgeListening) { + bridgeServer.stop(); + bridgeListening = false; + } + WiFi.disconnect(true); + WiFi.mode(WIFI_OFF); + // A transport that vanished must not keep receiving asynchronous replies. + replyStream = &Serial; +} + +void wifiStart() { + if (wifiSsid.length() == 0) return; + WiFi.mode(WIFI_STA); + WiFi.begin(wifiSsid.c_str(), wifiPass.c_str()); + lastWifiAttemptMs = millis(); +} + +void pollWifi() { + if (!wifiEnabled) return; + + if (WiFi.status() != WL_CONNECTED) { + if (bridgeListening) { + bridgeServer.stop(); + bridgeListening = false; + if (replyStream != &Serial) replyStream = &Serial; + } + if (millis() - lastWifiAttemptMs >= WIFI_RETRY_INTERVAL_MS) wifiStart(); + return; + } + if (!bridgeListening) { + // Disabled *after* association, not before: setting it ahead of + // WiFi.begin() is silently discarded when the connection is + // established, which leaves the station in modem sleep. A sleeping + // station keeps its association -- the AP shows a healthy client with + // full signal -- but misses broadcast frames, so it never sees an ARP + // request and becomes unreachable for ping and for this server, while + // looking perfect from the router's side. Cost this bench an evening + // (2026-09-05). The fixture is mains powered; there is nothing to save. + WiFi.setSleep(false); + bridgeServer.begin(); + bridgeServer.setNoDelay(true); // Frames are tiny; Nagle would add latency. + bridgeListening = true; + Serial.printf("[lttx] bridge listening on %s:%u\n", + WiFi.localIP().toString().c_str(), (unsigned)WIFI_BRIDGE_PORT); + } + + if (!bridgeClient || !bridgeClient.connected()) { + WiFiClient incoming = bridgeServer.available(); + if (incoming) { + if (bridgeClient) bridgeClient.stop(); + bridgeClient = incoming; + bridgeInputLength = 0; + } + } + while (bridgeClient && bridgeClient.available() > 0) { + const int c = bridgeClient.read(); + if (c < 0) break; + if (c == '\n') { + bridgeInput[bridgeInputLength] = '\0'; + replyStream = &bridgeClient; + handle(bridgeInput); + bridgeInputLength = 0; + } else if (c != '\r' && bridgeInputLength + 1 < sizeof(bridgeInput)) { + bridgeInput[bridgeInputLength++] = (char)c; + } else if (bridgeInputLength + 1 >= sizeof(bridgeInput)) { + bridgeInputLength = 0; + } + } +} + void pollSerial() { while (Serial.available() > 0) { const int c = Serial.read(); if (c < 0) return; if (c == '\n') { input[inputLength] = '\0'; + replyStream = &Serial; handle(input); inputLength = 0; } else if (c != '\r' && inputLength + 1 < sizeof(input)) { @@ -654,6 +893,11 @@ void setup() { delay(2); Serial.println("[lttx] Heltec V4 R8 deterministic transmitter"); + wifiLoadPersisted(); + if (wifiEnabled) { + Serial.printf("[lttx] wifi bridge enabled, joining '%s'\n", wifiSsid.c_str()); + wifiStart(); + } if (configure(active)) { Serial.println("[lttx] radio ready; output capped at -9 dBm"); } else { @@ -666,6 +910,7 @@ void setup() { void loop() { pollSerial(); + pollWifi(); pollButton(); serviceBeacon(); if (armed && (int32_t)(millis() - armedAtMs) >= 0) transmitArmedPacket(); diff --git a/docs/HARDWARE_TESTING.md b/docs/HARDWARE_TESTING.md index 4dee267..bb1d5b5 100644 --- a/docs/HARDWARE_TESTING.md +++ b/docs/HARDWARE_TESTING.md @@ -162,3 +162,37 @@ Opening the monitor can reset the board through DTR, which creates a new run directory. Start the retained serial capture at that reset and use the new run number. Launcher SD-drop remains valid for release-like verification, but direct USB flashing is preferred while Phase 7 is iterating. + +### Never target a board by `/dev/ttyACM*` + +With the Cardputer-Adv and the bench Heltec both attached, they enumerate as +identical `Espressif USB JTAG serial debug unit` devices and **the `ttyACMn` +numbering swaps between resets.** Address a board by its stable per-device +path instead, which is keyed to the USB serial: + +```text +ls /dev/serial/by-id/ +pio run -e cardputer-adv-bench -t upload \ + --upload-port "/dev/serial/by-id/usb-Espressif_USB_JTAG_serial_debug_unit_-if00" +``` + +Identify which MAC is which once, by resetting each and reading its boot +banner: the transmitter prints `[lttx] Heltec V4 R8 deterministic +transmitter`, the receiver prints `LoRaTrace RX v...`. The framed harnesses +are self-checking (they speak `@LTRX/1` and `@LTTX/1` and simply fail to +handshake if pointed at the wrong board), but **flashing is not** — it writes +whatever image you named to whatever is on that path. + +The signature of getting this wrong is unmistakable and worth recognizing at +a glance, because it looks like a dead receiver: + +```text +LoRaTrace RX v1.0.7 (...) — tasks + GPS + SD logging + WiFi + ... +FATAL: IO expander init failed — no I2C ACK at 0x43. + Antenna switch off and GPS unpowered. +``` + +That is LoRaTrace running on the *Heltec*, which has no PI4IOE5V6408 at +0x43. Repeated RTS resets will not clear it, because the banner is real +firmware booting correctly on the wrong board. Recover by reflashing +`bench/heltec-v4r8-transmitter` to that board (2026-09-04). diff --git a/docs/README.md b/docs/README.md index 4db3207..cfbf129 100644 --- a/docs/README.md +++ b/docs/README.md @@ -8,11 +8,12 @@ material once you're past that. - **[STATUS.md](STATUS.md)** — where the project actually is right now: current version, what's hardware-verified, what's still open. -- **[DESIGN.md](DESIGN.md)** — hardware, RF parameters, and architecture - rationale. Read before making architecture changes; don't re-derive - decisions already made here. -- **[ROADMAP.md](ROADMAP.md)** — build-order phases, MVP-Beta scope, and - an honest feasibility assessment against this hardware's real limits. +- **[DESIGN.md](DESIGN.md)** — shipped v1 hardware, RF parameters, and + architecture rationale. Read before changing those foundations. +- **[ROADMAP.md](ROADMAP.md)** — active V2 workstreams, gates, and release + policy. Start here for the next implementation decision. +- **[research/V2_DESIGN.md](research/V2_DESIGN.md)** — V2 product direction, + permanent boundaries, and rationale behind the active roadmap. - **[LOG_GUIDE.md](LOG_GUIDE.md)** — operator guide to run folders, CSV fields, identity observations, health checks, and privacy-aware export. - **[HARDWARE_TESTING.md](HARDWARE_TESTING.md)** — repeatable @@ -25,11 +26,12 @@ material once you're past that. Not onboarding reading — these are raw evidence and in-progress notes, kept for reference rather than written to be read start to finish. -- **`history/`** — `PROGRESS.md` and `CHANGELOG.md` as they stood before - the 2026-08-29 documentation restructuring: a frozen, unedited - session-by-session development log. Search it for a specific - date/topic rather than reading it front to back; `STATUS.md` above and - the root `CHANGELOG.md` are what stay current going forward. +- **`history/`** — the immutable v1.0.7 roadmap snapshot pointer + ([`ROADMAP_V1.md`](history/ROADMAP_V1.md)), plus `PROGRESS.md` and + `CHANGELOG.md` as they stood before the 2026-08-29 documentation + restructuring. Search history for a specific date/topic rather than + reading it front to back; `STATUS.md` above and the root `CHANGELOG.md` + are what stay current going forward. - **`research/`** — design/investigation notes written during specific phases (e.g. Phase 8/9 sweep design), not maintained after the phase they were written for. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 3521de7..fb539ae 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -1,903 +1,182 @@ -# LoRaTrace RX — Roadmap - -This roadmap operationalizes the build order already decided in -`docs/DESIGN.md` §9. It doesn't change any RF or architecture decision — it adds -scope boundaries, exit criteria, and an honest read on what this specific -hardware can and can't do, so "MVP-Beta" means something concrete instead -of a vibe. - -## What "MVP-Beta" means here - -The smallest version of this firmware that's actually useful as a field -tool: **Meshtastic War Drive, end to end.** RX locked to the LongFast (US) -channel, every detection GPS-stamped and written to SD, running -unattended on battery. That's docs/DESIGN.md §9 phases 1–2. Everything past -that (MeshCore, discovery sweeps, energy sweep, UI, upload) is real -project scope, but it's post-MVP-Beta. - -## Hardware feasibility — reality check - -Read this before assuming any phase below "just works." It's an honest -accounting against the ESP32-S3FN8's actual limits (CLAUDE.md hardware -assumptions), not a pitch. - -**Solid — low risk:** -- RX-only detection on a known channel (`HOME_LISTEN`) for Meshtastic and - MeshCore. Continuous single-channel RX is cheap: no polling, no - scanning, minimal CPU. This is the easy 80% of the project's value. -- The Core 1 (radio) / Core 0 (GPS+SD+UI) split with a FreeRTOS queue. - Standard ESP32-S3 pattern, and it's the right call given SD's latency - spikes — keeping those off the radio task is the one architectural - decision that actually matters for not dropping packets. -- GPS fusion (NMEA over UART) and small-struct SD logging. Neither needs - meaningful RAM; TinyGPSPlus-class parsers and a ~40B detection struct - flushed often are well inside an 8MB-flash, no-PSRAM budget. -- Keyboard + small-display UI, *if* it avoids a full framebuffer (see - below). - -**Achievable, but needs deliberate care:** -- **No PSRAM means ~512KB SRAM total, shared with the RTOS, WiFi stack (if - enabled), display driver, and every task's stack.** docs/DESIGN.md's own - estimate of 250–380KB free heap is a guess pending `ESP.getFreeHeap()` - on real hardware — treat every RAM-hungry decision below as provisional - until that number is in hand. -- **WiFi AP + web UI (Phase 3).** This used to sit in the "lowest priority" - tier below, gated behind a real free-heap number under full load. That - number now exists: `run0007`/`run0011` (session.csv) measured heap_free - settling at ~304KB with radio + GPS + logger + display all running, - flat with no decline across a 2.5-hour run — real headroom, not the - provisional estimate this gate was originally waiting on. Built - on-demand (off by default, toggled by an operator gesture) specifically - so its RAM/CPU/RF-noise cost is never present during an actual drive - unless asked for — see docs/history/CHANGELOG.md for the spike this was still worth - gating behind before building the rest of the feature on top of it. -- **Display:** a naive full framebuffer at 240×135×16bpp is ~65KB — a - meaningful bite out of a few-hundred-KB heap. Use direct-to-panel - partial-window writes (LovyanGFX/M5GFX style) instead of buffering a - whole frame, especially once WiFi is in the picture. - **Superseded 2026-08-25 (Phase 6's own hardware bench pass):** real - glass proved direct-to-panel wrong for this UI — every intermediate draw - call is visible the instant it happens, causing real flicker/tearing a - build report can't catch. The actual fix was an off-screen - `Arduino_Canvas_Indexed` framebuffer, just an indexed 1-byte/pixel one - (~32KB, since this UI only ever uses 6 colours) rather than a full RGB565 - one (~63KB) — see docs/history/CHANGELOG.md's v0.6.2 -> v0.6.3 entry for the full - root-cause and the near-miss (an `SPIClass::beginTransaction()` deadlock) - ruled out along the way. -- **`ENERGY_SWEEP` (Reticulum / General Exploration):** a single SX1262 - cannot listen to the whole 868–923MHz band at once. The design's - sweep-then-return-to-listen approach is the right shape for this - hardware, but it's a fundamental physical tradeoff, not a bug to code - around: short bursts on a non-home frequency *will* be missed during the - portion of the duty cycle spent elsewhere. Set that expectation in the - UI/docs rather than implying continuous full-band coverage. -- **Front-end rolloff near 923–928MHz:** the module's tuned range tops out - around 923MHz; the top of US ISM (923–928MHz) is outside it. This is a - hardware fact, not fixable in firmware — it specifically dents General - Exploration's full-band sweep, since every named profile's actual - channel already sits inside 868–923 (docs/DESIGN.md §1). Needs an empirical - RSSI noise-floor sweep once hardware is in hand, then document the - sensitivity gap rather than pretend it isn't there. -- **Meshtastic's 104-slot hash space:** Phase 1–2 only lock to the - LongFast US default (slot 20). That's most of the real-world traffic, - but a mesh running a non-default channel name lands on a different slot - and is invisible to `HOME_LISTEN`. `DISCOVERY_SWEEP` (Phase 8) is what - closes that gap — until then, be precise in docs/UI that "Meshtastic - profile" means "default channel," not "all Meshtastic traffic." - -**Genuinely open — blocks real functionality, not just polish:** -- microSD bus (SPI vs SDMMC) on this board revision — docs/DESIGN.md §7, - unresolved. Blocks finalizing the Logger task's pin/driver choice. -- Meshtastic's exact sync-word register value, and MeshCore's - encryption/PSK model — docs/DESIGN.md §7. Detection (RSSI/SF/BW/timing) works - without these; reliable protocol-level filtering and payload decode - don't. Don't hardcode a guessed value for either (CLAUDE.md house rule). - -**Deliberately out of scope:** -- TX/injection of any kind — permanent non-goal per CLAUDE.md house rules, - not a phase-ordering question. - -## Phases - -Each phase maps 1:1 to docs/DESIGN.md §9. "Blocking unknowns" cross-reference -docs/DESIGN.md §7 items that must be resolved (or explicitly deferred) before -the phase can be called done, not just started. - -### Phase 0 — Project scaffold -**Status:** done (this change). -PlatformIO project, pin map, RF parameter tables, phase-1 bring-up code, -this roadmap, and the tracking doc. - -### Phase 1 — RadioLib bring-up -**Goal:** prove the radio path is alive. -**Deliverable:** `src/main.cpp` — SPI init, IO-expander P0 high, SX1262 up, -hardcoded RX on 906.875MHz/SF11/BW250/CR8, detections printed to Serial. -**Exit criteria:** a real Meshtastic LongFast (US) packet in the air shows -up on the serial monitor with plausible RSSI/SNR. -**Blocking unknowns:** none functionally — but the IO-expander register -map and SPI host assignment (see docs/history/PROGRESS.md) are unverified against real -hardware and are the first suspects if the radio stays silent. -**Status:** complete, hardware-verified. See docs/STATUS.md for current -state and docs/history/CHANGELOG.md for the full session history. - -### Phase 2 — `HOME_LISTEN` + task/queue architecture + GPS + SD (= MVP-Beta) -**Goal:** the smallest genuinely useful field tool. -**Deliverable:** `radio_task`, `gps_task`, `logger_task` per the proposed -layout in CLAUDE.md; FreeRTOS queue Core 1 → Core 0; GPS-stamped -detections batched to SD per the §8 log schema. -**Exit criteria:** unattended run — power on, GPS fix acquired, detections -logged with correct lat/lon, no dropped packets attributable to SD -latency, no crash from heap exhaustion over a multi-hour run. -**Blocking unknowns:** none left. microSD is confirmed on the shared SPI -bus (arbitrated by `spi_bus.h`), and GPS reached a fix on hardware -2026-08-23 — the last piece that had never been proven. -**Status:** see docs/STATUS.md for current state and -docs/history/CHANGELOG.md for the full session history. - -### Phase 3 — Web Command Center (WiFi AP + web UI) -**Goal:** get data and control off the device without ejecting the SD card. -**Deliverable:** `wifi_task` — an on-demand (off by default, operator- -toggled), WPA2-protected WiFi AP hosting a single embedded web page (no -LittleFS/SPIFFS, no new `lib_deps` — built-in `WiFi.h`/`WebServer.h` only, -see docs/history/PROGRESS.md) with three tabs: a live status dashboard (the same -counters the Serial `[status]` line and `ui_task`'s pages already expose), -a run browser to download `detections.csv`/`session.csv` per run, and a -settings form that writes `/loratrace/config.txt` (`config.h`'s existing -format/validation, reused not reimplemented) — applied on next boot, not -hot-reloaded, so `radio_task`'s real-time critical section is never touched -by another task. `ui_task` gains a long-press-any-key gesture to toggle the -AP (no keymap needed, matching its existing "any key" discipline) and a -WIFI status page. -**Blocking unknowns:** none for the feature itself — pulled forward ahead -of Phase 4 at the user's request, prioritizing operator convenience over -protocol breadth. The one thing worth confirming on real hardware before -trusting this at length is the heap/counter spike described in -docs/history/PROGRESS.md: `ESP.getFreeHeap()` before/after `WiFi.softAP()` with the -full Phase 2 task set already running, and `radioCrcErrorCount()`/ -`radioQueueDropCount()`/`radioBusMissCount()` staying at 0 with the AP -active — the actual go/no-go this phase used to be gated behind, now -answerable instead of estimated (see the heap numbers above). - -### Phase 4 — MeshCore profile -**Deliverable:** same `HOME_LISTEN` engine, MeshCore US-narrow table -(910.525MHz/SF7/BW62.5/CR5) wired in as a second selectable profile — the -first phase where "selectable" is real, not aspirational. This is where -docs/DESIGN.md §5's keyboard-gated profile switch (operator-selected, mutually -exclusive — Meshtastic and MeshCore never listen at once) actually gets -built, deliberately deferred from Phase 3 (docs/history/CHANGELOG.md) so it's designed -and tested against a real second channel table instead of a stub. -**Blocking unknowns:** none for basic detection; MeshCore's -encryption/PSK model (§7) still blocks payload decode, not detection. -**Status:** see docs/STATUS.md for current state and -docs/history/CHANGELOG.md for the full session history. - -### Phase 5 — On-device menu UI -**Deliverable:** a real menu on `ui_task`'s display, replacing the timed -hold-gestures Phase 3/4 built as a stopgap. Pulled forward ahead of -`DISCOVERY_SWEEP`/`ENERGY_SWEEP` at the user's request — same precedent as -WiFi's Phase-3 pull-forward ahead of MeshCore — because the menu/settings -framework built here is what the rest of the project gets built around, not -because it was blocking anything below it. - -Scope, decided with the user up front rather than assumed: plain `,`/`.` -move a selection (no Fn chord — the Cardputer-ADV has no dedicated arrow -keys; directional intent there is Fn held + `;`/`,`/`.`/`/`, confirmed -against M5Stack's own keyboard API docs, a dedicated Cardputer-ADV keyboard -reference, and `bmorcelli/Launcher`'s own shipped interface code), Enter -selects, Backspace goes back. Toggles only this phase — profile switch -(Meshtastic/MeshCore, reusing Phase 4's `radioRequestProfileSwitch()` -unchanged) and WiFi on/off (reusing Phase 3's `wifiToggle()` unchanged) — -no on-device numeric editing of channel params; that stays on the existing -web UI Settings tab / `config.txt`. A new read-only CHANNEL status page -shows the actual active freq/SF/BW/CR/sync word, closing a real gap (that -was previously visible only over Serial or the web UI). The two old -hold-gestures are removed, not kept as parallel shortcuts, so there's -exactly one way to trigger each action. - -This is what turns "decode the keyboard" from a 56-key project of its own -into a small, low-risk slice: only four specific keys need to be identified -correctly, not the whole QWERTY layout — see `src/keyboard.h` for the full -sourcing (TCA8418 raw-event encoding, the raw-value-to-physical-position -formula, and the physical-position-to-character table, each independently -cited) and docs/DESIGN.md for the citation-level writeup. - -**Blocking unknowns:** none for the menu/toggle mechanism itself — every -action it triggers already exists and was already exercised in Phase 3/4. -The keyboard decode is sourced from three independent references but, -per this project's own standing rule, not yet bench-verified against real -hardware (see docs/history/PROGRESS.md's Phase 5 checklist): press each of the four keys -once and confirm the firmware recognizes exactly that key and no other. - -### Phase 6 — UI architecture redesign -**Goal:** rework `ui_task` before `DISCOVERY_SWEEP` (or anything after it) -adds another ad hoc entry to a framework that was never designed to hold -more than a couple of toggles. Pulled forward ahead of `DISCOVERY_SWEEP` at -the user's request (2026-08-25) — same restructuring precedent as WiFi's -Phase-3 pull-forward and the menu UI's own Phase-5 pull-forward, both -already in this doc's history. Unlike those two, this isn't new subsystem -bring-up: it's a scoped rework of `ui_task.cpp`/`.h`, the one file the next -two phases would otherwise keep bolting onto. -**Why now, not later:** Phase 5 shipped exactly two menu items by design -(docs/history/CHANGELOG.md/docs/DESIGN.md §9) and had already grown a third — the verbose -serial-debug toggle — by the end of the same day, with no framework change -and a stale in-code comment (`ui_task.cpp`'s "the menu has exactly two -items this phase") as the only trace of the drift. `DISCOVERY_SWEEP` would -add at least a fourth item (a sweep trigger) and `ENERGY_SWEEP` a fifth -after it; fixing the shape once beats letting two more features each bolt -on their own row. -**Deliverable:** -- **Grouped menu**, replacing the flat, fixed-size list `MENU_ITEM_COUNT` - indexes into today. Root categories (e.g. Profile, Radio Mode, System) - open short sub-lists, same four input keys throughout (`,`/`.` move, - Enter act, backtick/ESC back/close) — a shape borrowed structurally from - M5PORKCHOP's (github.com/0ct0sec/M5PORKCHOP) "Sirloin-style grouped - modal" `Menu` class (`src/ui/menu.h` in that repo: `RootItem` entries are - either `DIRECT` actions or `GROUP`s that open a `MenuItem` sub-list) — - reviewed for that structural idea only, not its content or aesthetic - (see note below). -- **Toast/notice layer** for transient feedback — a toggle firing, later a - sweep hit — that isn't tied to whichever carousel/menu row happens to be - on screen. Real RAM cost gets measured and reported before it ships, - same discipline as every other RAM-hungry addition this project has made - (WiFi AP's ~55-56KB measurement is the precedent, not an estimate). -- **Redesigned status pages.** The current five (RADIO/CHANNEL/GPS/ - SYSTEM/WIFI) are single-column stacked text (`ui_task.cpp` - `drawRadioPage()` etc.) with real unused width on a 240px panel. Group - related values into blocks instead of one value per line. (As-proposed - this assumed direct-to-panel partial-window writes, no framebuffer — the - real hardware bench pass overturned that; see the Hardware feasibility - section above and this phase's Status note below.) -- **Adopt docs/BRAND.md's on-device labels** (`Watch`/`Probe`/`Sweep` for - HOME_LISTEN/DISCOVERY_SWEEP/ENERGY_SWEEP; plain profile names — - `Meshtastic`/`MeshCore`/`Reticulum`/`Spectrum` — grouped under one - "Profile" menu row, not branded per-profile names. docs/BRAND.md's Interface - Naming table went through two revisions the same day mid-implementation: - first a "Mesh Trace" family name over Meshtastic/MeshCore sub-profiles, - then walked back entirely once it was clear that branding every profile - its own "___ Trace" name made four settings on one sniffer read like - four separate tools — "Profile" was already this doc's preferred word - for the axis before either revision) as the strings the UI actually - displays. Kept as a separate UI-label layer, not a rename of the - internal identifiers: `detection.h`'s `missionProfileName()` keeps - emitting `meshtastic`/`meshcore` into `detections.csv` exactly as every - already-logged run expects — docs/DESIGN.md §8's own "don't concatenate runs - across a format change without checking the header" rule applies here - too, and there's no reason to risk it for a cosmetic rename. -- **Design reference, not a dependency:** M5PORKCHOP was cloned locally - (read-only) and reviewed for its menu grouping, its `NoticeKind`/ - `NoticeChannel` toast abstraction (`src/ui/display.h`), and how it - organizes a stats/status screen (`src/ui/swine_stats.h`). Its - gamification layer (XP/levels/achievements, `src/core/xp.h`), the piglet - mascot/mood system (`src/piglet/`), and its aesthetic and voice are - explicitly **not** part of this redesign — docs/BRAND.md's own guardrails - already rule out mascots, gamified framing, and anything outside the - "field instrument, not a consumer app" positioning. Structure was worth - learning from; the pig was not coming with it. -**Blocking unknowns:** the toast layer's actual heap cost, unmeasured -until built. Whether the ST7789V2's partial-window writes can carry the -denser block layout above without needing a framebuffer — an assumption to -confirm during implementation, not before. -**Status:** see docs/STATUS.md for current state and -docs/history/CHANGELOG.md for the full session history (canvas/framebuffer rework, -bugs found/fixed, and the still-open WiFi AP heap/counter re-measurement). - -### Phase 7 — Device optimization - -**Status:** complete 2026-08-27 as v0.7.0. Hardware evidence, measured -budgets, and the operator-approved same-build repetition waiver are recorded -in docs/history/PROGRESS.md. - -**Goal:** turn the current memory assumptions into measured budgets before -new scan states add load to the no-PSRAM device. - -**Deliverable:** internal-heap fragmentation metrics, stack high-water marks -for all five tasks, lifecycle checkpoints around the display canvas/WiFi/CSV -downloads, a repeatable real-device matrix (`docs/HARDWARE_TESTING.md`), and -measured optimizations that preserve radio/GPS/SD/UI behavior. - -**Epic priority order:** - -1. **P0 — Observability and baseline.** Append largest-free-block, heap block - counts, and every task's stack watermark to `session.csv`; bracket known - large/transient allocations in serial; run the full WiFi+canvas workload. -2. **P1 — Task stacks.** Right-size only stacks whose worst-case watermark - proves excess headroom. Preserve at least 25% and 1KB after the change. -3. **P2 — WiFi lifecycle and requests.** Measure repeated AP cycles and CSV - downloads; reclaim the off-state task/server cost or remove request-time - churn only where the measurements identify a persistent cost. -4. **P3 — Canvas allocation.** Keep the verified indexed canvas unless it is - the limiting allocation. Any tiled/partial alternative must repeat the - full real-glass regression pass; moving it to static RAM is not a saving. -5. **P4 — Final budget.** Record normal, WiFi-on, largest-block, and per-task - stack budgets for Phases 8/9, explicitly accept or reject the provisional - 2.5 KB transient-scan result buffer, then complete a combined-load soak. - -**Exit criteria:** every `docs/HARDWARE_TESTING.md` stage passes on one identified -build; no continuing decline in current free heap or largest block after -warm-up; `queue_drop`/`row_drop`/`bus_miss` stay 0 under combined load; task -stack margins satisfy the acceptance rule; final budgets and results are -recorded in `docs/history/PROGRESS.md`. - -**Scope guardrail:** Phase 7 does not add Discovery/Energy behavior and does -not optimize from compile-time RAM percentages alone. One lever changes per -measurement cycle so gains and regressions remain attributable. - -### Phase 8 — `DISCOVERY_SWEEP` - -**Operator label:** Probe. - -**Implementation status:** bounded acquisition slice started 2026-08-27. The -source-backed, versioned fixed candidate-plan layer is documented in -`research/phase8-discovery-research.md` and implemented in -`src/discovery_plan.h`; radio-owned CAD/receive-on-hit, observation queues, -durable Probe output, and the on-device trigger are now implemented. Hardware -validation, transient mode, and deterministic cancellation/fault coverage -remain open. This is not a phase-complete release. - -**Deliverable:** a bounded-duration, radio-task-owned CAD sweep of a curated -candidate list per active profile — non-default Meshtastic slots and sourced -legacy MeshCore tuples. Every complete, cancel, timeout, and failure path -restores the resolved home configuration and reports total time away from -Watch. - -Packet-bearing hits keep using the existing `Detection` pipeline. CAD-only -observations use a separate fixed-size, non-blocking queue; cumulative retry, -drop, recovery, and home-away values belong in the run summary/health log, -not every observation. - -Durable Probe writes to SD. An explicitly selected transient mode may run -without SD only if Phase 7 accepts a fixed result-buffer ceiling of 2.5 KB. -It reuses the live measurement buffer, retains one result, stores no raw or -historical stream, replaces the prior result on the next scan, and displays -`NOT SAVED`. If the budget fails, Probe requires SD. - -**Blocking unknowns:** curated candidate lists should be weighted by -MeshMapper-observed frequencies ([[meshmapper-pipeline]], per CLAUDE.md) -where available, not scraped defaults alone. CAD `symNum` tuning (§7) -needs bench testing against Semtech AN1200.48 before trusting false- -positive/miss rates. New sweep results use the existing carousel and Probe -controls, not a reason to reopen UI architecture a second time. - -Phase 8 ships with bounded, versioned built-in plans plus the existing -per-profile home override. Persistent operator-edited candidate lists are a -post-Phase-8 enhancement requiring their own bounded schema, validation, -deduplication, provenance, and device/web editing design; they do not block -Phase 8 completion. - -**Exit criteria:** correct built-in LongFast CR 4/5 is host- and OTA-verified; -a known alternate transmitter is found; CAD false-positive/miss rates are -measured; 1,000 Probe cycles pass through a deterministic automated bench -mode; deterministic cancellation/fault injection covers every acquisition -state; queues and memory remain stable; every exit restores Watch. - -### Phase 9 — `ENERGY_SWEEP` - -**Operator label:** Sweep. - -**Deliverable:** a truthful frequency-binned energy map across the supported -868–923MHz front end, followed by selective LoRa CAD only at energy peaks, -operator-selected bins, or a sparse sourced subset. This realizes Reticulum -and General Exploration without claiming that energy is LoRa or that an -off-grid CAD hit is Reticulum; those results are labeled `unknown LoRa -candidate` until stronger evidence exists. - -Each bin retains only bounded streaming statistics. Durable energy peaks use -a schema that cannot be confused with packet detections. Transient mode uses -the same Phase 7-gated 2.5 KB ceiling and `NOT SAVED` behavior as Probe. - -**Blocking unknowns:** 923–928MHz front-end rolloff — **resolved, -2026-09-01, no rolloff found.** A passive floor-only pass and a real -injected-carrier pass (both real hardware, see STATUS.md for the full -evidence and the bench tooling) agree: a real transmitted signal near the -923MHz ceiling registers within ~2dB of the same strength as one at -912.8MHz, and captures at least as reliably. Not a gap in coverage. - -**Exit criteria:** timing and home-away duration are measured (**done**, -2026-09-02) — 3/3 real US-region sweeps: EA (device-measured away time) -3386-3438ms, home channel restored every time. Quiet-band behavior is -characterized with WiFi off/on (**done**, 2026-09-02) — 3 matched pairs, -zero peaks and no meaningful timing difference (EA within ~50ms) whether -the AP was on or off. CAD never promotes energy alone to LoRa (**done**, -2026-09-02) — 10 real `BENCH_PASS_B_CAD` attempts at the noisiest known -combo, 5 came back `CAD_DETECTED`, the real-packet-promotion counter -(`PBD`) never moved once, confirming empirically (not just by code -inspection) that a bare CAD hit never becomes a Detection. See -`docs/STATUS.md` for the full numbers and how each was measured -(`WIFI_SET`/`EA` are new Serial Control additions this session). -Injected low/mid/high carriers land in the correct bins (**done**, -2026-09-02) — `scripts/phase9_bin_accuracy_bench.py`, the dedicated test -`research/LoRaTrace-Phases-7-10-Design.md`'s own hardware matrix already -specified ("Sweep calibration... known signals at low/mid/high bins"). -Real sourced candidates at the low, mid, and high ends of the US region -(`LONG_SLOW` 905.3125MHz, `LONG_MODERATE` 912.8125MHz, `SHORT_SLOW` -920.625MHz): 11/24 attempts landed a real elevated reading via -`BENCH_SWEEP_FLOOR`, and every single hit was at the exact pre-computed -bin index for that candidate's frequency — never a neighbor, never a -wrong bin. Sub-100% hit rate is expected and not a correctness problem: -a live Sweep only dwells ~40ms per bin, so a repeatedly-fired pulse has -to land inside that window by chance each lap (same timing reality the -923MHz-edge work already established); what matters is that a hit is -always at the *right* bin, which held 11/11 times. - -8-hour endurance soak (**done**, 2026-09-03, scoped down from 24h — see -STATUS.md for why) — `scripts/phase9_soak.py`, production firmware. -First run found a real bug: 5 of 4,148 laps failed, initially documented -here as harmless dropped-response noise, which was wrong — pulling -`session.csv` off the SD card afterward showed all 5 were the identical -`Guru Meditation Error... Stack canary watchpoint triggered (logger)` -crash, `logger_task`'s 5,120-byte stack genuinely overflowing under -sustained operation and silently rebooting the device each time (also -explaining that run's "WiFi went quiet" finding — the device rebooted, -not a WiFi bug). Fixed (8,192 bytes). A second full 8-hour run with the -fix in place: **4,353/4,353 laps, 0 failures, 0 crashes**, one -continuous run directory on the SD card confirming no reboot of any -kind. Real `session.csv` heap data from that run confirms bounded -memory directly: a clean step function (flat, one legitimate one-time -jump when WiFi switched on, flat again), no drift in either phase. The -recurring slow-lap timing tail (19-35s vs the ~3.4s median) turned out -to be fully explained, not a bug: 100% correlated with `wp>0` (Pass B -correctly running its full CAD sequence on a real Sweep peak). The same -real data also caught `radio_task`'s own stack margin sitting at 20.0% -headroom (below the 25%-or-1KB house rule, though never actually -overflowing) — fixed proactively (4,096 → 6,144) the same way, verified -clean with a focused 2-hour run covering where the old watermark hit its -floor. Full writeup in `docs/STATUS.md`. - -**All five exit criteria closed. Status: complete, hardware-verified, -`v0.9.0`.** - -**Region setting (`v0.8.9`, added and hardware-verified 2026-09-01, out -of sequence — same "appended, not inserted" convention as Cell below):** -the full 868-923MHz range Sweep always scanned is the module's hardware -ceiling, not a legal requirement — nothing legal for unlicensed US -LoRa-type operation sits below 902MHz (47 CFR § 15.247). System > Region -(`region_plan.h`, `energy_plan.h`'s `energySweepBandForRegion()`) lets -Sweep default to 902-923MHz (US) instead of the full range (Global), -roughly halving scan time for the common case, with Global one menu -cycle away. Persisted to `/loratrace/region.txt` -(`region_settings.h`/`.cpp`, mirrors `display_settings.h`/`.cpp`'s -shape exactly). Confirmed on real hardware: the menu row cycles and -persists across reboot, and both US and Global sweeps show the correct -band on the frequency bar with the expected scan duration. - -**Follow-ups identified but explicitly out of scope for this change** -(operator conversation, 2026-09-01) — larger, not scheduled: -- **Cell regionalization.** Cell's A/B block markers (§5a of - `docs/DESIGN.md`) cite 47 CFR § 22.905, an FCC-only structure. A - non-US operator has no equivalent band to swap in — Cell would need - to stop presenting FCC-specific claims outside the US, not gain a - parallel per-region table. -- **`channel_plans.h` per-region presets.** Meshtastic/MeshCore's - hardcoded "US default" channel tables would need real per-region - frequency-plan tables (Meshtastic alone ships a dozen-plus regions), - each individually sourced per CLAUDE.md's citation house rule — a - research project, not a mechanical change. - -### Phase 10 — Field Analyzer (closed 2026-09-03 — the `v1.0.x` release gate) - -**Deliverable:** Meter, truthful frequency waterfall, bounded live Scope, -recent captures, and a passive node roster over data acquired by Watch, -Probe, and Sweep. Field Analyzer is not another mission profile and never -controls the SX1262 directly. - -Scope uses an explicit bounded `SCOPE_ACQUIRE` request owned by the radio -task. It samples one displayed frequency, exposes `Watch paused`, and restores -the resolved home configuration on complete, cancel, timeout, or failure. -Ordinary analyzer page changes consume snapshots and never retune the radio. - -Analyzer storage is fixed-size and reuses the existing indexed canvas. The -initial incremental ceiling is 8 KB beyond that canvas, subject to Phase 7 and -the measured Phase 8/9 costs. Waterfall plot columns use deterministic, -host-tested aggregation of real frequency bins; Scope is never presented as -a spectrum. - -**Exit criteria:** bin-to-pixel and scope-source truthfulness are verified; -bounded memory and deterministic roster eviction hold; a worst-case UI/radio -run has no drops, deadlocks, or watchdog resets; outdoor readability and -minimum-brightness rendering are tested as separate conditions. - -**Decided 2026-09-03, with Phase 9 hardware evidence in hand: Phase 10 is -required for `v1.0.x`.** `v1.0.x` is not tagged until Field Analyzer's own -exit criteria above are also closed, matching the original "all four -profiles + UI stable" framing this doc has used since the 2026-08-25 -renumbering. Work starts now under an interim `v0.10.x` line, the same -convention Phase 8/9 used while in progress (see Versioning below). - -**Gate satisfied the same day: promoted to `v1.0.0`, 2026-09-03.** All -five exit criteria above hardware-confirmed (`v0.10.1`), plus real -hardware-confirmed UI polish beyond them (`v0.10.2`/`v0.10.3`, below). -Phase 11 (Cell) was never part of this gate — added out of sequence, -appended after Phase 10 rather than inserted into it, outside the -original four-profile scope this promotion tracks — its own two open -items (docs/STATUS.md) are known, tracked gaps post-`v1.0`, not blockers, -an explicit call rather than a silent omission. - -**Implementation status (`v0.10.0`, 2026-09-03):** -- **Data layer — done, host-verified.** `waterfall.h`/`scope_trace.h`/ - `capture_history.h`/`node_roster.h` (WaterfallHistory, ScopeTrace, - CaptureHistory, NodeRoster) plus `analyzer_state.h`/`.cpp`'s mutex-guarded - cross-task snapshot layer. 202+ host-native test cases - (`pio test -e native`). Row-at-a-time waterfall snapshot API deliberately - replaces a first cut that returned the whole ~5.5KB struct — that version - crashed `ui_task`'s 4096B stack on real hardware (Waterfall page, - 2026-09-03); see `analyzer_state.h`'s own comment on - `analyzerWaterfallRowSnapshot()`. -- **`SCOPE_ACQUIRE` — done, hardware-verified.** Bounded, radio-owned, - mutual-exclusion against Probe/Sweep/Cell/bench triggers in both - directions, home-config restore on complete/cancel/timeout/failure - (`radio_task.cpp`/`.h`). -- **On-device UI — done, hardware-verified**, and materially reshaped from - the original one-page-per-view plan during this session at the operator's - request: rather than five analyzer views navigable directly on the main - carousel, they're gated behind a single **Analyze** hub card (page-local - row selection, UP/DOWN — `;`/`.` — to browse, SELECT to open, BACK to - return); Probe/Sweep/Cell were folded the same way into a second **Tools** - hub, both real scope additions beyond this section's original text, not a - deviation from the exit criteria above. Iterated live against - `docs/research/analyzer-preview.html` (an interactive Canvas mockup of - the exact device palette/coordinates/state machine, built specifically so - hub UX could be worked out before each real flash) before being ported to - firmware. Two real bugs were caught and fixed along the way: a whole-struct - Waterfall snapshot overflowing `ui_task`'s stack (above), and - `drawFooterStatus()` computing the card-count footer from the raw - `UiPage` enum ordinal instead of the main-carousel-relative position, - which would have shown "14/14" for Tools and "8/14" for Analyze instead of - the intended "2/6"/"3/6" — fixed via new `mainCarouselPosition()`/ - `mainCarouselCount()` accessors. -- **Memory/telemetry — landed, not yet confirmed against a real multi-hour - run.** `ANALYZER_STATIC_BYTES` (`analyzer_budget.h`, split out of - `analyzer_state.h` so it stays reachable without pulling in FreeRTOS.h, - same "keep it pure" reasoning as `session_log.h`'s own header comment) is - a compile-time `sizeof()` sum, not a runtime measurement — the four - analyzer structures commit **6,728 bytes** against this section's - 8,192-byte incremental ceiling (82.1% used, 1,464 bytes headroom). - `session_log.h`'s `analyzer_static_bytes` column reports it on every - health row (added append-only, per this project's CSV-schema - convention). Real linked-firmware RAM delta since Phase 10 work began: - +6,472 bytes, within compiler-alignment noise of the computed number. -**All five exit criteria closed, 2026-09-03:** -- **Worst-case UI/radio run — closed.** WiFi on, Sweep repeat mode - running continuously, Waterfall open the whole time, for a full 60 - minutes. A background Serial Control watch (231 `STATUS` polls at 15s - intervals) recorded **zero** dropped/unanswered requests and **zero** - `task_wdt`/`Guru Meditation` signatures for the entire hour. Confirms - the exit criterion directly, not just "nothing looked wrong." -- **Outdoor and minimum-brightness readability — closed (operator - check, 2026-09-03):** confirmed good both in direct window sunlight and - indoors. -- **A real, hardware-found Waterfall bug surfaced by the worst-case - run — found and fixed same day.** During the run, Pass A found 50 - energy peaks (`PBA=50` cumulative, `STATUS`'s own field) worth - triggering Pass B CAD on, yet Waterfall showed nothing the entire hour. - Root cause: `analyzerNoteSweepComplete()` (`analyzer_state.cpp`) read - `radio_task.cpp`'s live `energyPeakBinMask` to build each Waterfall row, - but in repeat mode `radio_task`'s own do-while loop (`radio_task.cpp` - ~1443) calls straight back into `performEnergySweep()` for the next lap - with no delay — whose first line resets that same mask. `logger_task` - (Core 0) only polls `radioEnergySweepCount()` on its own ~100ms cadence, - so it almost always lost that race against Core 1 and read an - already-cleared mask: every repeat-mode Waterfall row reported quiet - regardless of what Pass A actually found. `energy.csv` itself was never - affected — `enqueueEnergyObservation()` pushes each peak to a queue the - instant Pass A finds it, independent of this mask, so all 50 peaks (with - bin/frequency/RSSI/`pass_b_confidence`) are real rows on the card. - **Fix:** a second, stable buffer (`energyPeakBinMaskAtComplete`) - snapshotted atomically at `energySweepCount++`, before the next lap can - touch it; a new accessor, - `radioEnergyPeakBinSetAtLastComplete()`, is what - `analyzerNoteSweepComplete()` reads now. `drawSweepOccupancy()` - (ui_pages.cpp, the Sweep page's own live occupancy ticks) is untouched - — it still reads the live mask on purpose, so it keeps rendering ticks - progressively as an active sweep runs. **Hardware-confirmed the same - day**: operator re-ran Waterfall during a live repeat Sweep against - real MeshCore traffic and confirmed hits now appear on the display. -- **Bench SD card / boot-loop finding — found then resolved same day - (2026-09-03):** the bench Cardputer briefly hit a 100%-reproducible - `task_wdt` abort during SD bring-up on every cold boot, isolated to the - SD card (identical crash reproduced on the last tagged release, - `v0.9.0`/`b84d88c`, with no Phase 10 code at all — not a regression from - this work). Operator swapped the card; the fix was confirmed two ways: - a 20-minute passive Serial Control watch (zero `task_wdt`/`Guru - Meditation`, `SD=1` throughout), then the card itself pulled and - inspected directly — `run0006`'s real `session.csv` covers a clean - 29-minute boot, `analyzer_static_bytes=6728` on every row, `sd=ok` - throughout, zero row/queue/bus drops, heap settling once early then - flat for the rest of the run. This closes Stage 4's memory/telemetry - confirmation for real, not just at compile time. Full writeup: - `docs/STATUS.md`'s Phase 10 section. -- **Post-closure UI polish, same day (`v0.10.2`/`v0.10.3`), hardware- - confirmed:** real scope beyond the five exit criteria above, not a - deviation from them — Waterfall gained a frequency axis (lo/center/hi - MHz + reference ticks, later merged into the plot box's own bottom - border to remove a redundant line and reclaim height) and an Enter key - that starts/stops repeat Sweep straight from the page, sharing - `WATERFALL_SWEEP_REPEAT_TOGGLE` with the Tools/Sweep card's own R key - rather than a parallel mechanism. Meter — previously three lines of - text and a lot of unused screen — gained a real bar gauge (`drawMeterBar()`, - filled proportionally, deliberately not `drawFreqBar()`'s position-marker - convention since signal strength is a quantity, not a location), an SNR - line, and a right-column SF/BW/CR block, all three watch-sourced only - from data (`CaptureSummary`) this page already had and never showed. Bar - range widened -30 -> 0dBm same day after a real -16dBm reading clipped - flat against the original ceiling. Every layout change was workshopped - in `docs/research/analyzer-preview.html` before reaching real hardware, - same tool that already caught this session's real Waterfall footer- - collision bug once. - -### Phase 11 — Cell (added out of sequence, 2026-09-01) - -**Operator label:** Cell — same single-word register as Watch/Probe/Sweep -(docs/BRAND.md), not a "___ Trace" name. Named "Cell Trace" for its first -same-day revision; renamed once the operator noticed that collided with the -product name's own "Trace" and read as a bigger decode claim than the -feature (RSSI presence only) actually makes. - -Not part of the original four-profile plan (docs/DESIGN.md §3/§9) — added at -the operator's request after real wardriving runs kept picking up energy in -the 869-894MHz North American Cellular downlink band near cell towers. -Unlike WiFi's Phase-3 pull-forward or the UI's Phase-5/6 pull-forwards -(both of which *reordered* the existing sequence), this is *appended* after -Phase 10 rather than inserted into it — Phase 9/10 keep their numbers and -their own in-progress status is unaffected. - -**Deliberately not a fifth mission profile.** The SX1262 only demodulates -FSK/GFSK/MSK/LoRa/OOK — it cannot decode GSM/CDMA/LTE, so there is no -protocol to detect a channel for, and no HOME_LISTEN table to give it. It is -instead a third bounded, radio-owned action alongside Probe (`DISCOVERY_SWEEP`) -and Sweep (`ENERGY_SWEEP`): retune across a curated 101-bin, 250kHz grid -covering 869-894MHz (`cell_plan.h`), sample RSSI at each bin -(`radio.getRSSI(false)`, the same primitive Sweep's Pass A uses), log every -bin — not threshold-filtered — to its own `cell.csv` (`cell_observation.h`). -No CAD is attempted (CAD is a LoRa-preamble correlator; it will never fire on -a cellular carrier) and no packet read is attempted (there is nothing to -decode). This keeps the feature honest: it is a coarse RF-presence/strength -survey ("a strong carrier sits near this frequency, here"), not a cell-tower -identifier — no cell ID, LAC/TAC, or MCC/MNC is or can be extracted. - -**Deliberately isolated from `ENERGY_SWEEP`'s Pass A/B engine** -(`performEnergySweep()`), not a parameterized reuse of it: Pass A's -35.0dB noise-floor margin (`energy_observation.h`) was bench-calibrated -against a LoRa/RF-quiet environment for *sparse* peak detection, not -continuous cellular-strength carriers — reusing it here would dress up a -guess as a calibration. `performCellSweep()` (`radio_task.cpp`) is its own -function; it shares only the generic streaming-RSSI-statistics primitives -(`EnergyBinStats`/`energyBinStatsAddSample()`/`energyRssiDbmToFixed()`, -`energy_observation.h`) that Pass A itself uses, not Pass A's acquisition -loop or its calibrated threshold. - -**869-894MHz** is FCC Part 22 Cellular Radiotelephone Service downlink / -3GPP Band 5 downlink — a regulatory band edge, not a carrier-specific -channel plan (no individual ARFCN/channel table is hardcoded, consistent -with CLAUDE.md's house rule against hardcoding unverified RF parameters). -It sits entirely inside the Cap LoRa-1262's tuned 868-923MHz front end -(docs/DESIGN.md §1), so unlike General Exploration's 923-928MHz top end, -there is no front-end rolloff caveat here. - -**Operator surface:** same shape as Probe/Sweep, not a menu row (`ui_menu.h`'s -`CELL_TOGGLE`, same SD-required/start/cancel logic) — a global hotkey (C, -`keyboard.h`'s `KEY_RAW_C_PRESS`) and a dedicated carousel results card -(`UiPage::CELL`, `ui_pages.cpp`'s `drawCellPage()`), inserted as carousel -position 4 and pushing CHANNEL/GPS/SYSTEM's digit-jump keys from 4/5/6 to -5/6/7 (`keyboard.h`'s `KEY_RAW_7_PRESS`). An earlier same-day revision of -this feature shipped it as a root-level menu row with no card, before the -operator asked for the full Probe/Sweep treatment instead — reverted before -merge, not kept as a parallel entry point. - -**Repeat mode (R), added 2026-09-01:** identical in shape to Sweep's own -repeat — `radioRequestCellSweepRepeat()` re-runs `performCellSweep()` -back-to-back until stopped, with a lap counter on the Cell card. This also -changed Sweep's own R: it was previously a global hotkey (fired from -anywhere, including with the menu open); R is now page-gated by -`ui_task.cpp` to the Sweep/Cell cards specifically, matching how Enter -(`SELECT`) already dispatches per-page. C/S/P remain global. Probe -deliberately has no repeat mode — operator decision: "Repeat only on the -Sweeps." - -**Implementation status:** code + host-native tests (`test_cell_plan`, -`test_cell_observation`, and `test_session_log`'s extended coverage) landed -2026-09-01. Single-shot Cell is now **partially hardware-verified** -(2026-09-01, `v0.8.6`): the C key runs a scan end-to-end with a real -MHz/dBm reading and correct home-channel restore, and the mutual-exclusion -guard against Sweep holds in both directions — see `docs/STATUS.md`. Still -open: a real cell-band RSSI reading confirmed rising near a known tower -(vs. floor noise the whole way through), and `cell.csv`/`session.csv`'s new -columns rendering correctly. Repeat mode (above, `v0.8.7`) is -**hardware-verified** the same day: Sweep-repeat still works after -becoming page-gated, Cell-repeat works with a correctly-placed lap -counter, R is a confirmed no-op on Probe/other pages/menu-open, and mutual -exclusion holds across a real repeat chain in both directions — see -`docs/STATUS.md`. - -**FCC A/B block markers (`v0.8.8`, added and hardware-verified -2026-09-01):** the Cell frequency bar now labels the FCC's own downlink -sub-band split (`cell_plan.h`'s `CELL_BAND_BLOCKS`, cited to 47 CFR -§ 22.905) — Block A (869-880MHz + 890-891.5MHz) and Block B (880-890MHz + -891.5-894MHz), rendered as a two-shade tick row under the bar with letter -labels on the two segments wide enough to hold one. Regulatory block letter -only — see the "Non-goals" line below for why no carrier name is shown. -Confirmed on real glass: renders cleanly with no overlap against the -surrounding lo/hi labels or the disclaimer line. - -**Non-goals, same as the rest of this project:** no GSM/CDMA/LTE decode of -any kind (impossible on this hardware, not just out of scope), no per-carrier -or per-channel identification, no claim of tower triangulation from a single -reading. - -## Distribution - -Two install paths, both real, serving different audiences: - -- **Direct flash (`pio run --target upload`)** — the dev-iteration path. - Fastest feedback loop, direct serial access for debugging. Primary - method through Phases 1–2 while bring-up is still being bench-verified. -- **SD-drop via [bmorcelli/Launcher](https://github.com/bmorcelli/Launcher)** - — the preferred *end-user* install method once builds are stable. - Matches docs/BRAND.md's "field instrument, not a laptop-tethered tool" - positioning: swap firmware without reflashing. - -This costs us nothing extra to support. Launcher installs a plain -PlatformIO app binary (`.pio/build/cardputer-adv/firmware.bin`, standard -ESP32 image starting with the `0xE9` magic byte) straight off a FAT32 SD -card — no merged image, no manifest. Launcher's own dynamic partition -manager carves out or resizes an OTA app slot to fit whatever we hand it. - -**In practice, SD-drop is already the primary test path**, not just the -eventual end-user one — it's how this gets tested on hardware that's -already running a Launcher install the operator doesn't want to disturb -with a direct USB flash. CI reflects that: every merge to `main` publishes -a rolling `dev-latest` prerelease at a stable URL -(`LoRaTraceRX-dev.bin` — see Versioning below) specifically so there's -always a current build to drop on the card without waiting on a version -tag. Direct flash remains useful when iterating fast enough that even the -CI round-trip is overhead, or for the deepest debugging (upload errors, -brick recovery). - -**Confirmed:** returning from a running LoRaTrace RX build back to -Launcher is a manual restart + button combo — not something our firmware -needs to implement. No return-to-launcher hook belongs in `ui_task`. -**2026-08-22, confirmed against Launcher's own source (not a guess):** the -combo is "press any key during Launcher's own ~5s post-reset boot window, -before it auto-chains back into whatever ran last" — or, to remove the -timing pressure entirely, enable Launcher's own Settings → "Boot to -Launcher" toggle, which makes it always stop at its menu on reset instead -of auto-booting the last app. See docs/history/PROGRESS.md "Open questions — Launcher -distribution" for the full read of Launcher's boot logic. - -**Follows from this:** Launcher owns the flash partition table, not our -static `platformio.ini` partition CSV — that only governs direct-flash -installs. Any state we want to survive a user switching firmwares back -and forth (settings, last-used profile, calibration data) has to live on -SD, not in a custom NVS/data partition, since a custom partition isn't -guaranteed to survive a later Launcher install. This is already -docs/DESIGN.md's "SD is the datastore" philosophy — see docs/history/CHANGELOG.md — it just -now extends to config, not only detection logs. - -**Binary size:** no documented hard ceiling was found for how much flash -Launcher leaves free per app on an 8MB device, especially once Launcher -itself plus other installed firmwares share the same flash (that's the -whole point of a multi-firmware launcher — see docs/history/PROGRESS.md open -questions). Rather than target an arbitrary number like "under 4MB," -treat it as an ongoing discipline: keep the binary as lean as the feature -set allows, and measure the real number instead of assuming one. - -- **Measured, not estimated:** the Phase 1 scaffold (RadioLib + IO-expander - init, no GPS/SD/display/WiFi yet) compiles to **312KB** — 9.5% of the - ~3.19MB app partition our own `default_8MB.csv` direct-flash scheme - allocates (`pio run`, logged in CI now — see below). Adding SD, GPS - parsing, and a display library will grow this, but from a 312KB - baseline there's a lot of room before size becomes a real constraint - either for direct flash or for coexisting with Launcher + other apps. - **2026-08-22 update:** adding the boot-status splash's display library - (GFX Library for Arduino) plus SD's own footprint brought this to - **406KB** (12.2% of the same partition) — a real +94KB data point, not - an estimate, confirming there's still plenty of headroom. -- **WiFi is the one feature with a real size lever**, same as it's the one - feature with the real RAM lever (`lwIP` + the WiFi driver stack is - typically the single biggest chunk of a "full" Arduino-ESP32 build). That - go/no-go already happened (Phase 3, shipped — see CLAUDE.md's Status - section for the real numbers this note used to be waiting on). -- `RadioLib` compiles in support for many radio families by default; - disabling the ones we don't use (`RADIOLIB_EXCLUDE_*` build flags, only - SX126x needed here) is a cheap, real size reduction worth doing before - Phase 9 (`ENERGY_SWEEP`, the other RAM/flash-hungry feature after WiFi), - not just a nice-to-have. -- CI now measures the actual `firmware.bin` size on every build (see - below) — track it there instead of trusting an estimate. - -## Versioning - -Formalizes the phase-mapped table below into an actual scheme CI and bug -reports can use. - -- **Format:** `vMAJOR.MINOR.PATCH` (e.g. `v0.2.1`). MAJOR.MINOR tracks the - build-order phase reached; PATCH increments for fixes that don't add new - phase scope. -- **Source of truth:** `src/version.h` (`FIRMWARE_VERSION`), printed on - the boot banner (Serial) and on `ui_task`'s SYSTEM status page (on-device, - since Phase 2). A bug report against a specific build should always be - traceable to this string. -- **Release trigger:** pushing a `vX.Y.Z` git tag. `src/version.h` must be - bumped to match before tagging; release CI rejects a tag/header mismatch. -- **CI:** `.github/workflows/build.yml` runs `pio run` (+ `pio test`) on - every push/PR — catches build breaks before they land, independent of - tagging. `.github/workflows/release.yml` runs only on a `vX.Y.Z` tag - push: builds, renames the output to `LoRaTraceRX-.bin` - (Launcher/SD-drop-friendly naming, per its "use simple characters" SD - guidance), and attaches it to a **draft** GitHub Release. -- **Rolling dev build, separate from the tagged scheme:** every push to - `main` also force-moves a `dev-latest` tag and republishes a prerelease - there with a fixed filename (`LoRaTraceRX-dev.bin`) — a stable, - no-tagging-required download for day-to-day hardware testing. It's - explicitly *not* versioned or draft-gated the way real releases are: - it can be broken, it reflects whatever's on `main` at that moment, and - its own release notes point back at the exact commit. Cutting a real - `vX.Y.Z` tag is a separate, deliberate step for when a phase is actually - bench-verified — see docs/STATUS.md for current status before trusting - either. - -| Version | Corresponds to | +# LoRaTrace RX — V2 Roadmap + +This is the active, forward-looking gate board for LoRaTrace RX. It begins +from stable `v1.0.7` and governs V2 work only. The completed v1 phase +narrative, feasibility analysis, exit criteria, and versioning history remain +at [history/ROADMAP_V1.md](history/ROADMAP_V1.md). + +Start with [STATUS.md](STATUS.md) for what is true on hardware now and +[research/V2_DESIGN.md](research/V2_DESIGN.md) for V2 product boundaries. +This document records what may enter implementation next, what proof it needs, +and what earns a release. + +## Permanent boundaries + +V2 preserves the shipped foundation: + +- Receive-only; no transmit, beacon, injection, decryption, keys, payload + display, or protocol-client behavior. +- One radio-owner task on Core 1; at most one bounded acquisition action owns + the SX1262. SD, display, GPS, and WiFi never block its real-time path. +- Fixed/static storage, bounded queues, streaming metrics, and SD as the + datastore. New capability may not silently add unbounded RAM, SD backlog, + task lifetime, or radio-away time. +- Watch remains the default. Every acquisition completion, cancel, timeout, + and failure restores the resolved home configuration and records its result. +- RSSI/CAD/packet evidence never becomes a protocol identity merely by + inference. A missed dwell is not a quiet frequency. +- WiFi remains opt-in and browser acquisition control remains out of scope + without a separate security decision. + +The detailed rationale and product wording live in +[research/V2_DESIGN.md](research/V2_DESIGN.md); do not duplicate it here. + +## Status and gate model + +| Status | Meaning | |---|---| -| v0.1.x | Phase 1 (serial bring-up) | -| v0.2.x | Phase 2 (MVP-Beta: Meshtastic War Drive complete) | -| v0.3.x | Phase 3 (Web Command Center: WiFi AP + web UI) | -| v0.4.x | Phase 4 (MeshCore) | -| v0.5.x | Phase 5 (on-device menu UI) | -| v0.6.x | Phase 6 (UI architecture redesign) | -| v0.7.x | Phase 7 (device optimization) | -| v0.8.x | Phase 8 (discovery sweep) | -| v0.9.x | Phase 9 (energy sweep: Reticulum + General Exploration) | -| v0.10.x | Phase 10 (Field Analyzer), in progress | -| v1.0.x | reached 2026-09-03 — Phase 10 (Field Analyzer)'s exit criteria closed same day, the doc's own gate for this promotion (decided 2026-09-03), see Phase 10 above | - -**Phase 11 (Cell) is a deliberate exception to this table.** -It landed as a PATCH bump (`v0.8.6`, not `v0.9.x`) because it is not the -*next* build-order phase — Phase 9 (`ENERGY_SWEEP`) is still in progress, and -jumping MINOR to 9 (or past it to 11) would misrepresent Phase 9/10 as -reached when they aren't. If a later phase completes Phase 9/10 first, this -table's normal MAJOR.MINOR-tracks-phase-reached rule resumes as before; Phase -11 doesn't get its own `v0.11.x` line unless a future revision of this table -decides it should. - -**Renumbered 2026-08-24** (same restructuring precedent as WiFi's Phase-3 -pull-forward): the on-device UI overhaul moved from a trailing "Phase 7 -polish" step to Phase 5, at the user's request, pushing `DISCOVERY_SWEEP` -and `ENERGY_SWEEP` down one each. What `v1.0` means moved with it — it used -to be "Phase 7, all profiles + UI stable," and Phase 7 was `ENERGY_SWEEP` -at the time, not UI. - -**Renumbered again 2026-08-25**, same precedent a second time: reviewing -Phase 5's own menu against what `DISCOVERY_SWEEP` would add to it surfaced -that the menu had already grown past its documented two-item scope (a -third, `Debug`, row landed the same day with no framework change) — see -docs/history/CHANGELOG.md for the full session. Rather than let -`DISCOVERY_SWEEP` and `ENERGY_SWEEP` each bolt on their own ad hoc entry, a -UI architecture redesign now sits at Phase 6, pushing `DISCOVERY_SWEEP` to -7 and `ENERGY_SWEEP` to 8. `v1.0`'s meaning (all four profiles + UI stable) -is unchanged by this move — it's still a placeholder for the same total -phase count, not a re-litigated decision — revisit if it stops fitting once -Phase 6/7/8 are actually in hand. - -**Renumbered again 2026-08-26:** the measured ~32KB Phase-6 display canvas -now overlaps WiFi's ~55–56KB runtime cost, while only the logger task had a -stack watermark and the post-canvas combined-load gate was still open. -Device optimization therefore became Phase 7, before either new scan state. -`DISCOVERY_SWEEP` moved to Phase 8 and `ENERGY_SWEEP` to Phase 9; their scope -did not change. `v1.0` still meant all four profiles and their UI were stable -at that point. - -**Phase 10 added to the plan 2026-08-26:** Field Analyzer is accepted as -planned post-Sweep scope, including bounded radio-owned Scope acquisition. -This does not silently move the release gate: after Phase 9 hardware evidence -exists, explicitly decide whether Field Analyzer is part of `v1.0.x` or the -first post-v1.0 phase. - -## Non-goals - -- Transmit or injection of any kind, beyond the one-time antenna-switch - init GPIO write. Permanent, per CLAUDE.md house rules — not a phase to - eventually reach. -- Auto-detecting mission profile. Operator-selected via keyboard, by - design (docs/DESIGN.md §5). -- Full protocol decode/decrypt, payload display, or key handling. LoRaTrace - remains a metadata-first passive field instrument; safe cleartext radio - headers are the boundary. +| **Not entered** | Entry decisions are not yet locked. | +| **Design entry** | Scope and the measurement plan are being locked; there is no release claim. | +| **Engineering** | Code and host validation are in progress. | +| **Hardware pending** | The implementation gate is met, but device proof is incomplete. | +| **Closed** | Every applicable gate has accepted evidence. | + +Every workstream passes these gates in order: + +| Gate | Required proof | +|---|---| +| **Design entry** | Scope, non-goals, operator promise, unresolved decisions, log-schema impact, and worst-case memory/queue/SD/radio-away budget are explicit. | +| **Engineering** | Host tests cover plans, bounds, state transitions, CSV formatting, and coverage math; native tests and the production build pass. | +| **Device behavior** | A real device proves request/refuse/cancel/timeout/failure paths, mutual exclusion, home restore, UI behavior, and fresh SD output. | +| **Claim truth** | Controlled RF timing or RTL-SDR ground truth validates every RF/coverage claim; absence of an observation is never relabeled as silence. | +| **Release** | WiFi-off/on resource evidence, redacted field summary, `STATUS.md` reconciliation, and operator release notes are complete. | + +Focus Survey and Field Missions also require a before/after measurement of +Watch packet opportunity. Displaying a radio-away duration does not make the +cost acceptable; the measurement is part of the decision. + +## V2 workstreams + +| Workstream | Status | Outcome and phase-specific exit gate | +|---|---|---| +| **12 — Survey truth** | **Engineering** | Define coverage vocabulary and persistent per-survey evidence, then add bounded Focus Survey. The active [design-entry and acceptance plan](research/phase12-survey-truth-design.md) locks the one-bin first slice and controlled matrix; radio-away budgets and sampled/repeated thresholds remain measurement-gated. | +| **13 — Field Missions** | Not entered | Add Drive, Stationary, and Investigate as explicit recipes with visible WATCHING/SURVEYING/RESTORING and `mission.csv` accounting. Prove transitions do not hide radio-away time or weaken action arbitration. | +| **14 — Companion analysis** | Not entered | Deliver an offline, reproducible tool that reads copied run folders without changing original evidence. Test deterministic reports, multi-run comparison, coverage warnings, and privacy-safe export behavior. | +| **15 — Field markers and sharing** | Not entered | Add fixed, safe marker presets and `marker.csv`, then integrate redacted sharing. Prove markers cannot affect radio behavior and realistic exports remove selected location/identity detail. | +| **16 — Cell closeout** | Deferred bonus | Close the existing V1 Phase 11 evidence gap: a real tower-adjacent RSSI rise plus fresh SD verification of `cell.csv` and Cell's appended `session.csv` fields. This preserves V1 history; it does not renumber it. | +| **17 — Sweep/Waterfall sampling review** | **Design entry** | Re-evaluate whether Sweep's per-bin sampling and Waterfall's presentation can support what they imply, using the measurement apparatus Workstream 12 built. Entry needs a two-baseline sensitivity measurement, not an argument from analogy. See below. | + +## Candidate — Workstream 17 (Sweep/Waterfall sampling review) + +Raised 2026-09-05 out of Workstream 12's measurements, and deliberately parked +rather than acted on. Phase 9 already recorded that "a normal Sweep's short bin +dwell can miss genuine traffic"; what Workstream 12 adds is the quantitative +form of that statement, plus a fixture capable of testing it. + +What transfers from Workstream 12's evidence +([2026-09-04-phase12-focus-matrix.md](hardware-results/2026-09-04-phase12-focus-matrix.md)): + +- Detection requires the source's airtime to exceed the sampling spacing. + Sweep dwells on each bin for tens of ms with a handful of samples — a + shorter, sparser pass than the 100 ms / 6-sample arm that failed at both + measured signal levels. +- Six samples per pass is below a usable floor, and 13 dB of link improvement + did not rescue it. That was a sampling limit, not a link limit. +- A count of samples above an adaptive floor outperformed every extreme + statistic. Pass A currently keeps a per-bin average and peak and thresholds + on them; the peak is the noise-prone half of that pair. + +What that does **not** establish, and why this is a candidate rather than a +finding: + +- **Sweep is aimed at persistent energy, not packets.** Short per-bin dwells + are a reasonable trade when covering 200+ bins, and Pass B's CAD step exists + precisely because a Pass-A energy peak is not packet evidence. The + architecture already encodes the distinction Workstream 12 measured. +- **One link is not a baseline.** Workstream 12 drew three conclusions from a + single link; two held and the most confident one inverted on a second. Any + claim about Sweep's sensitivity needs the same two-baseline discipline + before it is written down. +- Waterfall is downstream of whatever Sweep's sampling delivers, so it is not + a separate question. Its own risk is presentational: a cell empty because + nothing transmitted looks identical to a cell empty because the pass did not + sample long enough — the §3 "absence is not silence" trap in a new surface. + +Entry would need: a controlled sensitivity measurement of Pass A's per-bin +sampling at two or more signal levels, reusing Workstream 12's transmitter +fixture and `benchSweepFloorQuery`'s existing per-bin floor readback; and a +decision about whether Waterfall should distinguish "sampled and quiet" from +"barely sampled" in what it draws. + +Rigorously sourced region packs are later candidates, not Workstream 16 and +not V2.0 blockers. Each proposed pack needs a separate entry gate with source +quality, regulatory/range rationale, fixed-table validation, and realistic +hardware access. + +## Current work — Workstream 12 + +The workstream is in Engineering. Its bench-only first slice now exercises one +bounded radio request, fixed statistics, and `focus.csv` persistence; it has +not added a production control or coverage/activity claim. The plan sets the +following constraints: + +- One selected Sweep/Waterfall bin or fixed preset per request; no free-form + frequency entry, arbitrary range, or multi-bin list in the first slice. +- `focus.csv` is an append-only result record, not raw sample history. +- A fixed histogram/statistics accumulator supplies median, P90, and maximum + without heap allocation; actual static/queue/row budgets must be measured. +- `insufficient`, `sampled`, and `repeated` cannot be displayed until a + controlled transmitter/RTL-SDR matrix selects their pass/time thresholds. +- Portland metro, Oregon is the privacy-preserving field-validation area; + exact location and raw GPS-bearing artifacts remain private. + +Engineering may build a bench-only raw-counter prototype so the matrix can +measure the open decisions. It may not present a coverage label or a “no +activity” conclusion until the Device/claim gate closes. + +## V2.0 composition release + +`v2.0.0` is earned only when Workstreams 12–15 pass together on one identified +build. It repeats cross-feature risks rather than aggregating old checklists: + +- every bounded radio action mutually excludes correctly; +- complete, cancel, timeout, and failure restore resolved home listening; +- append-only CSVs remain readable by the companion; +- WiFi-off/on resource trends remain healthy; and +- a field workflow demonstrates Watch-first driving, deliberate investigation, + known radio-away cost, and an explainable offline report. + +Workstream 16 (Cell closeout) is intentionally not a V2.0 blocker. + +## Version and release policy + +- `src/version.h` is the semantic-version source of truth. A release tag must + match it; CI rejects a mismatch. Each tagged release needs operator-facing + notes in `docs/RELEASE_NOTES.md`. +- `v1.0.x` is the stable maintenance line. The completed phase-number mapping + belongs to [the v1 archive](history/ROADMAP_V1.md). +- A closed core V2 workstream may earn the next stable minor release: + `v1.1.0` through `v1.4.0` for Workstreams 12–15. Workstream numbers remain + roadmap identities, not version components. +- `v2.0.0` requires the composition release gate above. +- Workstream 16 evidence closeout without an operator-facing behavior change + updates status only. A later Cell improvement may earn a `v2.1.0`-class + release after its own gates. +- A rolling `dev-latest` build remains for day-to-day hardware testing; it is + not a release gate and must be identified by build revision. + +## Evidence and history + +Use this order when deciding a gate: + +1. `docs/STATUS.md` for current accepted facts and open hardware work. +2. The workstream's design-entry document for scope, measurement method, and + acceptance criteria. +3. `docs/hardware-results/` for location-redacted evidence summaries; keep raw + serial/CSV/GPS artifacts in its git-ignored `private/` area. +4. `CHANGELOG.md` for terse post-v1 decision history. +5. [history/ROADMAP_V1.md](history/ROADMAP_V1.md) only for v1 Phase 0–11 + questions or original v1 gate rationale. + +Historic code comments and changelog entries that mention a V1 Phase refer to +the archive unless they name a later research or hardware-results record. diff --git a/docs/STATUS.md b/docs/STATUS.md index 5bbccea..af619da 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -3,14 +3,16 @@ The one place "where is this project right now" lives. Replaces status prose that used to be duplicated (and drifting) across `CLAUDE.md`, `PROGRESS.md`, and `README.md`. For *how* we got here, see -`docs/history/CHANGELOG.md`; for the phase-by-phase build order, see -[ROADMAP.md](ROADMAP.md). +`docs/history/CHANGELOG.md`; for active V2 workstream gates, see +[ROADMAP.md](ROADMAP.md); for the completed v1 phase record, see +[ROADMAP_V1.md](history/ROADMAP_V1.md). ## Current version -**v1.0.6** (`src/version.h`). `MAJOR.MINOR` tracks the build-order phase -*reached*, not the phase in progress — see ROADMAP.md's Versioning -section. Phase 9 (`ENERGY_SWEEP`/"Sweep") reached 2026-09-03: all five +**v1.0.7** (`src/version.h`). In the completed v1 series, `MAJOR.MINOR` +tracks the build-order phase *reached*, not the phase in progress — see +ROADMAP.md's Versioning section for the V2 workstream policy. Phase 9 +(`ENERGY_SWEEP`/"Sweep") reached 2026-09-03: all five ROADMAP.md exit criteria closed, including two full 8-hour endurance soaks that caught and fixed a real `logger_task` stack overflow and a proactive `radio_task` stack margin fix — see "What's hardware-verified" @@ -33,6 +35,127 @@ sequence, outside the original four-profile scope); its own two open items (below) are known, tracked gaps post-`v1.0`, an explicit operator call, not an oversight. +`v1.0.7` is the stable V2 planning baseline. It is an operator-navigation +PATCH release (Tools and Analyze moved into the menu and Activity became the +read-only bounded-action status page); it does not change any completed phase +gate or the two open Cell evidence items. + +## V2 entry state + +The V2 product direction is adopted for planning, with its canonical workstream +and gate policy in `docs/ROADMAP.md` and the detailed design in +`docs/research/V2_DESIGN.md`. **Workstream 12 (Survey truth) is in +Engineering.** Its bench-only first slice locks one-bin request, bounded RSSI +summary, and `focus.csv` contracts; it has not added a production control or +coverage label. The active +[design-entry and acceptance plan](research/phase12-survey-truth-design.md) +keeps radio-away budgets and coverage thresholds open until controlled evidence +selects them. + +The bench image now has one bounded Core-1 Focus request and a Core-0 +`focus.csv` writer. Two paired 500 ms/eight-sample smoke checks at US Sweep +bin 43 (912.750 MHz) observed P90/peak at -101.0/-101.0 dBm while quiet and +-82.0/-82.0 then -87.0/-87.0 dBm during capped controlled 912.8125 MHz Heltec +pulses—a 14--19 dB rise. Every request restored home listening and durably +wrote its row. This is only fixture/transport evidence, not a calibration, +coverage label, or activity claim. The repeatable bench harness is +`scripts/phase12_focus_bench.py`; its pulse path requires explicit +`--with-pulse --allow-transmit`. + +The non-transmitting Focus behavior fixture also proved cancelled and injected +failed rows restore before their durable write, and proved two-way refusal +with Probe and Sweep. + +A Focus request is now bounded in wall-clock time as well as in samples: past +a dwell-plus-slack deadline it stops sampling and terminates as `timeout`, +restoring home like every other exit path. That path and two-way Cell/Scope +refusal are hardware-verified as of 2026-09-04 on bench build `3e31daa-dirty`. +An injected 1,500 ms sample-loop stall against a 100 ms request (1,100 ms +deadline) produced a `timeout` row with successful home restore and 1,573 ms +radio-away time; Focus and Cell, then Focus and Scope, each refused the other +while it owned RX. Six terminal requests wrote six durable rows with zero +queue or row drops. Reaching those paths on demand needs bench-image-only +entrypoints (the stall is `BENCH_FOCUS`'s optional fourth field; +`BENCH_ACTION` starts and cancels the otherwise menu-only Cell and Scope); +production rejects both. No coverage/activity claim is enabled. + +The §6.2 controlled dwell matrix ran on 2026-09-04: 900 trials (5 positions x +3 dwell arms x 30 source-on and 30 source-off) in 54.7 min with zero transport +errors, zero queue or row drops, and successful home restore on every trial. +14 of 15 arms separated a controlled source from ambient completely (30/30 vs +0/30). Its most consequential result is a limit, not a capability: **detection +tracks the source's airtime against Focus's sample spacing, not dwell length** +— with a fixed 8 samples a 2000 ms dwell observes eight instants, not 2000 ms, +and one arm with a 94 ms source stopped separating there entirely. Bin-center +offset was not the dominant term. Radio-away measured dwell plus ~73 ms, worst +case 2,139 ms. No single fixed RSSI condition separates every arm, so the +qualifying condition has a candidate (`p90 >= -90 dBm`) but not an accepted +constant, and the coverage thresholds remain unselected. Full location-redacted +summary: [hardware-results/2026-09-04-phase12-focus-matrix.md](hardware-results/2026-09-04-phase12-focus-matrix.md). +The §6.3 Watch-opportunity comparison has tooling but has not run, so Focus's +maximum radio-away budget is still unapproved. + +The qualifying RSSI condition was then measured at a realistic signal level +(transmitter outdoors on the WiFi control bridge, receiver antenna fitted: +source peak ~-85 dBm against a -99/-100 dBm ambient floor) and **rejected**. +The `p90 >= -90 dBm` candidate came from a source ~70 dB hot; across 120 +trials it fails, as does every floor-relative variant — one of ten +metric/position combinations separates, by 2 dB, which is inside ordinary RSSI +variance. An RSSI summary statistic cannot carry an activity claim at field +levels. Coverage reporting is unaffected: valid passes, observation time and +the RSSI summary stay honest. `coverage` stays blank, the activity count stays +unpopulated, and the remaining routes are a per-pass count above an adaptive +floor (`qualifying_count`, already reserved in the schema) or CAD/packet +evidence. + +The §6.3 Watch-opportunity comparison also completed: two 240 s arms against +one independently timed reference train. Watch alone received 0.883 of the +train; with Focus interleaved at a 48.1% away fraction it received 0.463, with +non-overlapping 95% intervals. The loss is proportional to away time and +nothing more (0.459 predicted against 0.463 measured), so **Focus's recorded +radio-away duration is an honest proxy for what a request costs Watch** — no +hidden retune or recovery penalty — and equally, there is no mitigation: half +the time away is half the packets. Time between restored Watch windows ran to +a 7.45 s maximum. The maximum away-time budget is deliberately still +unapproved; the measurement supplies the exchange rate, not the policy. + +A per-pass **count** above an adaptive floor then succeeded where the summary +statistics failed. Counting samples at or above the pass's own median plus +6 dB, a threshold of two detected 57/60 source-on trials (95% CI +[0.863, 0.983]) at the same field-level configuration, with a single flagged +control trial that read -63 dBm against a -101 dBm median — a real +transmission that was not ours, so the 1.7% false rate is an upper bound. It +is a candidate, not a constant. + +Two further sweeps (480 trials) then bounded it. The rule is a **fraction** of +accepted samples (~4-5%), transferring between 500 ms and 2,000 ms passes, but +no threshold works at 100 ms — six samples cannot both catch the source and +reject ambient, so a short pass may report coverage and must not report +activity. Detection also appeared to collapse below ~40% occupancy — but a second +baseline at ~25 dB SNR (600 trials) **withdrew that conclusion**: 13 dB of +link improvement took detection at 28.6% occupancy from 37% to 87%, so the +cliff belonged to the first link, not to the instrument. Detection depends on +occupancy and SNR together, and single-packet detection is untested rather +than excluded. The 100 ms floor did survive both links (13 dB bought almost +nothing there), as did the fraction-of-samples form. + +That open question is now answered: one armed 148 ms packet inside a 2,000 ms +pass — 7.4% occupancy — was detected **29/30 with 0/30 false positives** at +the stronger link. An RSSI-sampling pass can support a packet-level activity +signal, but not unconditionally: the same rule at the weaker link failed on +sources occupying four times as much of the pass, and a position carrying real +traffic measured worse rather than better (a -57 dBm event in a control trial +both raised false positives and suppressed counts by lifting the median). So +Focus is least reliable where a band is busiest, and detection depends on link +quality the device cannot know. `qualifying_count` stays unpopulated: wording +an activity indication that stays true under those conditions is now a product +decision rather than an open measurement. + +The existing Phase 11 Cell feature remains partially hardware-verified and +visible in "What's still open." It is deliberately scheduled as **V2 +Workstream 16 — Cell closeout**, an optional post-core-V2 bonus: it does not +block Workstreams 12–15 or the V2.0 composition release. + ## What's hardware-verified Phases 0-9 are complete and hardware-verified: radio bring-up (Phase 1), diff --git a/docs/UI-Design_Guide.html b/docs/UI-Design_Guide.html new file mode 100644 index 0000000..404d68a --- /dev/null +++ b/docs/UI-Design_Guide.html @@ -0,0 +1,3136 @@ + + + + + +LoRaTrace RX · Master UI Design Guide & Workbench + + + + + + + + +
+ + + + + +
+
+ ★ Baseline Spec (v1.0.7 Master Guide) + → Proposed UI Improvements & Labs (New) +
+ +
Firmware v1.0.7 Baseline · ST7789V2 240×135
+

Master UI Design Guide

+ + +
+ Panel: ST7789V2 240×135 SPI + Color: 16-bit RGB565 (6 tokens) + Core: Core 0 (UI/Logging/GPS) + Radio: Core 1 (SX1262, never blocked) + Heap: 512KB SRAM (No PSRAM) + Scale: 2.2× Canvas / 1:1 Target +
+ + +
+
+
+ Real-Time Simulation +

Live Hardware Workbench

+
+ CURRENT: RADIO (1/5) +
+ +
+
+
+ +
+
+ Simulated ST7789V2 Display (2.2× scale). Drive with hardware keys: , (prev), / (next), ; (up), . (down), Enter (select), ` (back), 1–5, P, S, C, 8. +
+ + +
+
+
Simulation & Loops: RADIO
+ LOOP: IDLE +
+
+ +
+
Simulates real-time hardware telemetry and background animation loops.
+
+
+ +
+ +
+

Hardware Keys TCA8418

+
+ + + + + + +
+
Prev/Next cycle main carousel (1/5 to 5/5)
+
+ + +
+

Main Carousel (5 Pages) v1.0.7 Order

+
+ + + + + +
+
+ + +
+

Tools Hub Menu or P / S / C

+
+ + + +
+
+ + +
+

Analyze Suite Menu or Key 8

+
+ + + + + +
+
+ + +
+

Menu & Overlays Root / Slider / Toast

+
+ + + + +
+
+ + +
+

Preset Scenarios Global State

+
+ + + +
+
+ +
+ State: RADIO · Carousel Pos: 1/5 · Active Bounded Action: NONE +
+
+
+
+ + +
+
+
+ 01 / ARCHITECTURE +

Hardware Display Architecture

+
+

+ LoRaTrace RX executes on the M5Cardputer-Adv (ESP32-S3FN8). The display is an ST7789V2 driven over dedicated SPI. The system is strictly budget-constrained: 512KB SRAM with zero external PSRAM. +

+ +
+
+

Dedicated Core 0 UI Thread

+

The UI task runs on Core 0 and redraws at ~10-20Hz. The SX1262 radio task runs strictly pinned to Core 1 and never blocks on display SPI, SD logging, or keyboard I/O.

+
+
+

Zero-Allocation Rendering

+

Every page draw function (drawRadioPage, drawActivityPage, etc.) executes with zero dynamic heap allocations. Memory is pre-allocated or statically budgeted in task stacks.

+
+
+

No Full-Screen Blanking

+

drawPage() wipes only the content canvas (HEADER_H to FOOTER_Y) and performs direct overwriting. Full fillScreen() calls are banned during page transitions to eliminate visible black flashes.

+
+
+

Fixed-Pitch Bitmaps

+

Uses Adafruit-GFX / M5GFX built-in bitmap fonts: Size 1 is 6×8px (up to 40×16 chars); Size 2 is 12×16px (up to 20×8 chars). All positions are integer-aligned.

+
+
+
+ + +
+
+
+ 02 / COLOR TOKENS +

True RGB565 Hardware Palette

+
+

+ The ST7789V2 renders in 16-bit RGB565. Only six functional color tokens exist in production firmware (src/ui_pages.cpp). No arbitrary gradients or strength-tier rainbows are used. +

+ +
+
+
+
+
COL_BG
+
0x0000 · #000000
+
Primary screen background, inverted menu text.
+
+
+ +
+
+
+
COL_FG
+
0xFFFF · #FFFFFF
+
Primary text, headline numbers, active menu selection.
+
+
+ +
+
+
+
COL_DIM
+
0xBDF7 · #BDBEBD (~75% Grey)
+
Labels, secondary values, frequency tracks, quiet bins.
+
+
+ +
+
+
+
COL_GOOD
+
0x07E0 · #00FF00
+
Active GPS 3D fix, zero drops, battery OK, completed scans.
+
+
+ +
+
+
+
COL_WARN
+
0xFFE0 · #FFFF00
+
Standby, scanning, paused watch, repeat mode, peaks.
+
+
+ +
+
+
+
COL_BAD
+
0xF800 · #FF0000
+
Queue drops, SD unmounted, GPS checksum errors, failures.
+
+
+
+
+ + +
+
+
+ 03 / GEOMETRY +

Screen Layout & Character Grid

+
+

+ Display area is strictly 240×135 pixels. The layout is divided into three fixed horizontal zones: +

+ +
+
+

Header Chrome: y=0..12

+

Height 12px. Contains breadcrumb / page title (left, max 25 chars size 1), status dots (x=155..177), and battery icon with % (x=214..238). Border hairline at y=12.

+
+
+

Content Canvas: y=13..124

+

Height 112px. Where all page-specific UI widgets, tables, graphs, and statistics are drawn. Standard left column starts at x=2; right column aligns at x=170.

+
+
+

Footer Bar: y=125..135

+

Height 10px. Anchors active profile name at x=2 (e.g. Meshtastic) and carousel position at right (e.g. [1/5]). Replaced during menu display.

+
+
+
+ + +
+
+
+ 04 / CHROME +

Header Chrome & Battery Anatomy

+
+

+ Drawn on every page by drawHeader() and drawBattery(). The 25-character breadcrumb limit guarantees that deep menu paths never collide with the status dots cluster. +

+ +
+
+
+
RADIO
+
+
+
+
82%
+
+
+
+
+ +
Breadcrumb safe limit: 25 chars size 1 (≤150px)
+
Dots: [SD log] [GPS fix] [WiFi AP] at x=155, 166, 175
+
+ +
+ + + + + + + + + + + +
ElementCoords (1×)Size & ColorSource Data
Page Breadcrumb(2, 2)Size 1 · COL_FGpageName(page) / Menu path
Status Dots(157, 6), (166, 6), (175, 6)r=2px · G/Y/RSD state, GPS fix, WiFi active
Battery Pct(x_batt-2, 2) right-alignedSize 1 · COL_GOODbatteryPercent()
Battery Glyph(214, 2) 22×9pxRect + Nub · COL_GOODbatteryMilliVolts()
Header Hairline(0, 12) w=2401px Line · COL_DIMFixed horizontal divider
+
+
+
+ + +
+
+
+ 05 / MAIN CAROUSEL 1/5 +

Plate: RADIO (Primary Wardriving)

+
+

+ The default home screen. Shows real-time packet ingress, SD logging volume, queue drop count, and radio integrity stats. If watch is paused, STANDBY renders in bold yellow. +

+ +
+
+
RADIO
+
84%
+
+ +
rx 142
+
log 142
+
drop 0
+ +
crc
+
0
+ +
miss
+
0
+ +
sd
+
ok
+ +
run
+
r12
+ +
flush 18 max 42ms
+ +
Meshtastic
+
1/5
+
+ +
+ + + + + + + + + + +
MetricCoordsSize & TokenBehavior
rx counter(2, 18)Size 2 · COL_FGPackets decoded from SX1262
log counter(2, 38)Size 2 · COL_FGRows written to SD card
drop counter(2, 58)Size 2 · GOOD / BADGreen if 0, Red if >0
STANDBY indicator(2, 78)Size 2 · COL_WARNShows when radio is paused/active away
crc / miss / sd / runx=170 right colSize 1 statBlocksRadio CRC fails, SPI bus miss, run dir
flush / max ms(2, 106)Size 1 · COL_DIMSD batch write latency diagnostics
+
+
+
+ + +
+
+
+ 06 / MAIN CAROUSEL 2/5 +

Plate: ACTIVITY (Background Action Mirror)

+
+

+ Added in v1.0.7 (slot 2). A dedicated, read-only mirror of running bounded actions (Probe, Sweep, Cell, Scope). When idle, shows a compact summary of the last result for all four tools without redundant hero clutter. +

+ +
+
+
ACTIVITY
+
84%
+
+ +
Probe
+
1/8 hits
+ +
Sweep
+
3pk 915MHz
+ +
Cell
+
881MHz -84dB
+ +
Scope
+
-96dBm
+ +
Meshtastic
+
2/5
+
+ +
+ + + + + + + + + + +
State / RowCoords (1×)Font & ColorDescription
Active Mode (Hero)(2, 20)Size 2 · COL_WARN"PROBE", "SWEEP", "CELL", or "SCOPE"
Active Progress Bar(2, 66) 200×14Rect fill · WARNCandidate progress or frequency bar
Idle Row 1: Probe(0, 22)Size 2 · FG/DIMShows "IDLE" or "N/M hits"
Idle Row 2: Sweep(0, 46)Size 2 · FG/DIMShows "IDLE", "Npk FreqMHz", or "none found"
Idle Row 3: Cell(0, 70)Size 2 · FG/DIMShows "IDLE", "FreqMHz dBm", or "none found"
Idle Row 4: Scope(0, 94)Size 2 · FG/DIMShows "IDLE" or latest sampled "NdBm"
+
+
+
+ + +
+
+
+ 07 / MAIN CAROUSEL 3/5 +

Plate: CHANNEL (RF Parameters)

+
+

+ Confirms actual hardware tuning: frequency, Spreading Factor (SF), Bandwidth (BW), Coding Rate (CR), and LoRa Sync Word. The frequency bar shows exact position within the tuned 868–923MHz front-end. +

+ +
+
+
CHANNEL
+
83%
+
+ +
906.875 MHz
+
SF11 BW250.0
+ +
+
+
868
+
923
+ +
CR4/5 sync 0x2B
+ +
mode
+
Watch
+
airtime
+
~420ms
+ +
Meshtastic
+
3/5
+
+ +
+ + + + + + + + + +
FieldCoordsFontRemarks
Frequency(2, 18)Size 2 · COL_FGTuned center frequency in MHz
Modulation(2, 44)Size 2 · COL_FGSF (7-12) and BW (kHz)
Frequency Bar(2, 74) w=108Track + MarkerMarker is COL_GOOD, bounds 868..923MHz
Coding & Sync(2, 106)Size 1 · COL_DIMCoding rate (e.g. CR4/5) and hex sync word
Mode & Airtimex=170 right colSize 1 statBlocksRadio task mode ("Watch") and packet ToA
+
+
+
+ + +
+
+
+ 08 / MAIN CAROUSEL 4/5 +

Plate: GPS (Fix & Constellation)

+
+

+ Real-time GPS telemetry parsed from NMEA sentences. Renders 3D/2D fix status, 5-decimal coordinates, altitude, satellites used/in-view, and talker constellation signal strength bars. +

+ +
+
+
GPS
+
83%
+
+ +
3D FIX
+
37.77490
+
-122.41940
+ +
2026-09-04 23:14:02Z
+ +
sats
+
11
+
qual
+
1
+ +
+
GP
+
GL
+
GA
+
+ +
Meshtastic
+
4/5
+
+ +
+ + + + + + + + + +
ElementCoordsColor TokenBehavior
Fix Type(2, 18)GOOD / WARN / BAD"3D FIX" (Green), "NO FIX" (Yellow/Red)
Coordinates(2, 40), (2, 60)Size 2 · COL_FGLatitude & Longitude formatted to 5 decimals
Timestamp(2, 84)Size 1 · COL_DIMUTC ISO timestamp from GPS NMEA sentence
Sats & Qualx=170 right colSize 1 statBlocksSatellites used in solution and fix quality
Talker Bars(170, 70) 4 barsRects · COL_GOODGP (GPS), GL (GLONASS), GA (Galileo), BD (BeiDou)
+
+
+
+ + +
+
+
+ 09 / MAIN CAROUSEL 5/5 +

Plate: SYSTEM (Heap & Diagnostics)

+
+

+ Critical system telemetry: device uptime, free internal heap (against the 512KB SRAM budget), minimum recorded free heap, battery voltage, SPI bus contention counter, WiFi AP clients, and firmware version. +

+ +
+
+
SYSTEM
+
83%
+
+ +
48 min
+
312k heap
+ +
+
+
+ +
min heap
+
298k
+ +
batt
+
3.98V
+ +
bus
+
0
+ +
wifi
+
OFF
+ +
keys tca8418 health 96
+
v1.0.7 (prod-release)
+ +
Meshtastic
+
5/5
+
+ +
+ + + + + + + + + + +
MetricCoordsTokenRules
Uptime(2, 18)Size 2 · COL_FGElapsed minutes since power on
Free Heap(2, 40)GOOD / WARN / BAD3-tier: Green <80%, Yellow 80-90%, Red >90%
Heap Fill Bar(2, 60) 108×8Rect · Match HeapFills proportionally to SRAM consumed (512KB base)
Min Heap / Battx=136 / x=205Size 1 statBlocksLowest watermark and raw battery voltage
Bus / WiFix=136 / x=205Size 1 statBlocksSPI bus contention and WiFi AP status
Key / Log / Version(2, 78), (2, 90)Size 1 · COL_DIMTCA8418 driver check, session log rows, build tag
+
+
+
+ + +
+
+
+ 10 / TOOLS HUB +

Plate: PROBE (Discovery CAD Sweep)

+
+

+ Executes fast Preamble Channel Activity Detection (CAD) across source-backed profile tuples. Scans without decoding full packets, providing rapid presence detection while keeping radio away-time bounded. +

+ +
+
+
PROBE
+
83%
+
+ +
COMPLETE
+
candidates 8/8 away 124ms
+ +
US LongFast, EU Narrow
+
hits 2 free 6 timeout 0 err 0
+ +
Meshtastic
+
+ +
+ + + + + + + + +
FieldCoordsTokenNotes
Headline Status(2, 20)Size 2 · Status"SCANNING" (Warn), "COMPLETE" (Good), "IDLE" (Dim)
Candidate Progress(2, 43)Size 1 · COL_FGCurrent/Total candidate count + away-time ms
Hit Labels(2, 62)Size 1 · COL_FGHuman-readable names of profiles detected
CAD Stats(2, 96)Size 1 · DIM / BADHits, free channels, timeouts, and hardware errors
+
+
+
+ + +
+
+
+ 11 / TOOLS HUB +

Plate: SWEEP (RF Energy & Pass-B)

+
+

+ Two-pass wideband energy acquisition. Pass-A takes RSSI samples across 85 frequency bins (902–923MHz in US). Bins exceeding threshold trigger Pass-B selective CAD. Supports ambient background repeat mode. +

+ +
+
+
SWEEP
+
83%
+
+ +
REPEATING
+
bins 85/85 away 340ms
+ +
+
+
902
+
923
+ +
listening to the noise
+ +
peaks
+
4
+
best MHz
+
915.0
+
rssi
+
-88dB
+
lap
+
14
+ +
Meshtastic
+
+ +
+ + + + + + + + + +
ElementCoordsTokenNotes
Headline(2, 20)Size 2 · Status"SCANNING", "REPEATING", "COMPLETE", "IDLE"
Bin Progress(2, 43)Size 1 · COL_FGBins swept and radio away-time
Frequency Bar(2, 74) w=108Track + MarkerMarker shows tuned bin position across 902..923MHz
Peaks & Best MHzx=170 right colSize 1 statBlocksCount of energy peaks and strongest frequency
Lap Counter(170, 102)Size 1 · COL_WARNShows total repeat cycles completed
+
+
+
+ + +
+
+
+ 12 / TOOLS HUB (PHASE 11) +

Plate: CELL (Cellular Band Energy)

+
+

+ Phase 11 cellular uplink band scanner (869–894MHz, 101 bins). Displays FCC Block A/B partitions (47 CFR § 22.905). Strictly measures RF presence without decoding or tower tracking. +

+ +
+
+
CELL
+
83%
+
+ +
COMPLETE
+
bins 101/101
+ +
+
+
869
+
894
+ +
+
+
+
+
+
A
+
B
+ +
best MHz
+
881.5
+
rssi
+
-76dB
+ +
Meshtastic
+
+ +
+ + + + + + + + +
WidgetCoordsTokenRules
Frequency Bounds(2, 74) w=108869..894MHzScoped tightly to cellular allocation
FCC Blocks(2, 92)DIM / FGBlock A (Dim) and Block B (White); letter labels
Strongest Signalx=170 right colSize 1 statBlocksStrongest detected frequency and peak dBm
Disclaimer(2, 108)Size 1 · COL_DIM"presence only — no cell ID / decode"
+
+
+
+ + +
+
+
+ 13 / ANALYZE SUITE +

Plate: METER (Real-Time Signal Level)

+
+

+ Dedicated signal strength meter. Features an honest filled bar gauge (-120 to 0 dBm), signal SNR readout (watch-sourced), and live channel modulation parameters (SF/BW/CR) on the right. +

+ +
+
+
METER
+
82%
+
+ +
-72 dBm
+
watch @ 906.875MHz
+
SNR: +8.5dB
+
2s ago
+ +
SF11
+
250.0kHz
+
CR4/5
+ +
+
+
+
-120
+
0
+ +
Meshtastic
+
+ +
+ + + + + + + + + +
ElementCoords (1×)TokenDesign Rationale
Signal dBm(2, 20)Size 2 · COL_FGPeak or packet RSSI in dBm
SNR Readout(2, 54)Size 1 · COL_DIMDecoded packet Signal-to-Noise Ratio (dB)
Elapsed Age(2, 66)FG / WARN"live - watch paused" (Warn) or "Ns ago"
Channel Paramsx=230 right-alignedSize 1 · COL_DIMSF, Bandwidth kHz, and Coding Rate denominator
Bar Gauge(2, 84) 232×12Border Dim, Fill FG-120 to 0 dBm (widened from -30 to avoid clipping)
+
+
+
+ + +
+
+
+ 14 / ANALYZE SUITE +

Plate: WATERFALL (Historical Spectrogram)

+
+

+ Stores up to 24 rows of historical frequency bin sweeps. Features a labeled MHz frequency axis along the bottom border, top-row SCANNING badge, and Enter-to-repeat sweep control. +

+ +
+
+
WATERFALL
+
82%
+
+ +
12 HITS
+
SCANNING
+
24 rows, newest first 85 bins
+ +
+
+
+
+
+
+
+
+ +
902.0
+
912.5
+
923.0
+ +
Meshtastic
+
+ +
+ + + + + + + + + +
ComponentGeometryColorDetails
Hit Summary(2, 18)Size 2 · WARN / DIMTotal hits across buffer or "QUIET"
Scanning Badgex=234 right-alignedSize 1 · COL_WARNRenders when repeat Sweep is running
Spectrogram Plot(2, 46) 232×60Box Border DimUp to 24 rows @ 4px height; hits in COL_WARN
Frequency Axisy=106 (border)Ticks + Labels902.0, 912.5, 923.0 MHz (plus 20/40/60/80% ticks)
Enter Key ActionInteractiveToggle RepeatStarts/stops repeat Sweep without leaving page
+
+
+
+ + +
+
+
+ 15 / ANALYZE SUITE +

Plate: SCOPE (Time-Domain RSSI Trace)

+
+

+ Fixed-frequency RF power oscilloscope. Samples raw RSSI at up to 50Hz (20ms intervals) into a 240-sample trace buffer. Visualizes packet preamble energy envelopes. +

+ +
+
+
SCOPE
+
82%
+
+ +
CAPTURED
+
906.875MHz 20ms/sample
+ +
+ + + +
+
-30
+
-120
+ +
Meshtastic
+
+ +
+ + + + + + + + +
FieldCoordsTokenRules
Headline Word(2, 18)GOOD / WARN / DIM"CAPTURING", "CAPTURED", or dim "IDLE"
Trace Metadata(2, 36)Size 1 · COL_FGTuned frequency and sample interval
Trace Plot Area(2, 60) 232×50COL_GOOD line240 samples scaled between -120 and -30 dBm
Hold ReversionRESULT_HOLD_MSTimerReverts from CAPTURED to IDLE after 8 seconds
+
+
+
+ + +
+
+
+ 16 / ANALYZE SUITE +

Plate: CAPTURES (Packet Ring Buffer)

+
+

+ Maintains an in-memory circular ring buffer of the 8 most recent decoded LoRa packet headers. Flags off-grid channels and displays node hex addresses, RSSI, and elapsed age. +

+ +
+
+
CAPTURES
+
82%
+
+ +
meshtastic !a4b892f0 906.9MHz -68dBm 2s
+
meshtastic !38c11e04 910.5MHz -94dBm 14s
+
meshcore 906.9MHz -82dBm 28s
+
meshtastic !f102bb88 906.9MHz -75dBm 42s
+
meshtastic !12aa045e 906.9MHz -89dBm 65s
+
meshtastic !88319eac 906.9MHz -102dBm 120s
+ +
Meshtastic
+
+ +
+ + + + + + + + +
PropertyValueFormatDescription
Max Entries8 packetsCircular FIFOOldest packets drop off automatically
Row Height13px pitchSize 1 fontFits up to 8 rows comfortably in content zone
Color CodingFG / WARNText TokenYellow if packet frequency was off-grid; white if standard
Line FormatStringTemplate{profile} [!nodeId] {freq}MHz {rssi}dBm {age}s
+
+
+
+ + +
+
+
+ 17 / ANALYZE SUITE +

Plate: NODES (Observed Node Roster)

+
+

+ Deduplicated roster of up to 24 observed nodes sorted by recency. Tracks transmission count per node, last seen RSSI, and elapsed seconds. +

+ +
+
+
NODES
+
82%
+
+ +
meshtastic !a4b892f0 x14 -68dBm 2s
+
meshtastic !38c11e04 x3 -94dBm 14s
+
meshtastic !f102bb88 x8 -75dBm 42s
+
meshtastic !12aa045e x1 -89dBm 65s
+
meshtastic !88319eac x5 -102dBm 120s
+ +
Meshtastic
+
+ +
+ + + + + + + + +
AttributeSpecificationDetails
Max Nodes24 nodesStatic array in node_roster.h
SortingRecency (ageS asc)Most recently heard nodes stay pinned to the top
Hit MultiplierxN countTracks packet density emitted by that node
Node Address!xxxxxxxx8-character hex address formatted with bang prefix
+
+
+
+ + +
+
+
+ 18 / NAVIGATION +

Plate: Menu Hierarchy & Sliders

+
+

+ Grouped hierarchical menu with inverted color selection bars (24px row pitch) and live interactive slider widgets (e.g. ST7789V2 LEDC backlight brightness). +

+ +
+
+
MENU > System
+
82%
+
+ +
+ Tools > +
+ +
+ Display > +
+ +
+ WiFi Toggle + OFF +
+
+ +
+
+
    +
  • Tools > (Probe, Sweep, Cell, Trace)
    • +
  • Analyze > (Meter, Waterfall, Scope, Captures, Nodes)
    • +
  • Profile > (Meshtastic, MeshCore, Reticulum, General)
    • +
  • System > +
      +
    • Display > +
        +
      • Brightness (Slider 10-100%)
        • +
      • Dim Timeout (Off, 30s, 60s, 300s)
        • +
      +
      • +
    • WiFi AP Toggle
      • +
    • Reboot
      • +
    +
    • +
+
+
+
+
+ + +
+
+
+ 19 / OVERLAYS +

Plate: Toast Notifications

+
+

+ Transient bottom-docked banner (16px height on hardware, 35px at 2.2×). Slides up, holds for TOAST_DURATION_MS (2000ms), and automatically dismisses without disrupting background operations. +

+ +
+
+
RADIO
+
82%
+
+ +
+ Profile: Meshtastic + OK +
+
+
+ +
+ + + + + + + + +
PropertyHardware ValueRemarks
Toast Height16pxExact height of size 2 font; docks flush to bottom
Duration2000msConfigured in ui_task_shared.h
Slide Time150msSmooth linear easing during entry and exit
Non-BlockingCore 0 tickRadio packet reception continues unimpeded
+
+
+
+ + +
+
+
+ 20 / INPUT ERGONOMICS +

Cardputer TCA8418 Key Matrix

+
+

+ Driven via TCA8418 I2C keyboard scanner. Four primary navigation keys form an inverted-T cluster, supplemented by Enter, Back, and direct single-key shortcuts. +

+ + + + + + + + + + + + + + + + +
Physical KeyMatrix CodePrimary Context ActionTool / Analyzer Subpage Action
, (Comma)KEY_PREVPrevious Carousel PageExit to Menu
/ (Slash)KEY_NEXTNext Carousel PageExit to Menu
; (Semicolon)KEY_UPMenu Cursor Up / Slider IncreaseCycle Previous Subpage
. (Period)KEY_DOWNMenu Cursor Down / Slider DecreaseCycle Next Subpage
EnterKEY_SELECTEnter Submenu / Execute ActionTrigger Scan / Repeat Toggle
` (Grave)KEY_BACKExit Submenu to ParentExit to Menu / Radio
1 .. 5HOTKEYDirect Jump to Carousel Pages 1-5Jump directly to Carousel Page
P, S, CHOTKEYDirect Jump to Probe, Sweep, CellRestart Tool Run
8HOTKEYDirect Jump to Analyze SuiteCycle Analyze Tools
+
+ + +
+
+
+ 21 / V2 ROADMAP +

Future UI Evolution & Proposals

+
+

+ Identified improvement areas and candidate enhancements under evaluation for Workstream 12–15: +

+ +
+
+

Direct Sunlight Contrast

+

COL_DIM (~75% grey) provides sufficient outdoor contrast while preserving visual hierarchy. High-contrast monochrome mode is under evaluation for high-glare environments.

+
+
+

Workstream 12 Focus Visuals

+

Adding dedicated single-bin dwell and statistical distribution indicators (P90/P50/Floor) when Workstream 12 transitions to production.

+
+
+

Activity Hero State Refinement

+

Maintaining clear hero status during active background scans while preserving instant access to last-result telemetry when idle.

+
+
+

Waterfall Persistence

+

Evaluating optional ring buffer persistence to SD for long-term RF landscape comparison across wardrive sessions.

+
+
+ +
+
+
Interactive Prototype Available
+
Explore V2 UI Proposals & Sunlight/Night Vision Labs
+
Test the glanceable radio ownership pill, live 30s ingress sparkline, and WS12 focus dwell gauge.
+
+ Launch V2 Labs → +
+
+ +
+ LoRaTrace RX Firmware UI Design Guide · Master Specification v1.0.7 · Maintained in docs/UI-Design_Guide.html +
+
+
+ + + + + + diff --git a/docs/UI-Recommendations.html b/docs/UI-Recommendations.html new file mode 100644 index 0000000..6c06f23 --- /dev/null +++ b/docs/UI-Recommendations.html @@ -0,0 +1,1404 @@ + + + + + +LoRaTrace RX · UI Recommendations & V2 Proposals + + + + + + + +
+ + + + + +
+
LoRaTrace RX · V2 Hardware UI Recommendations
+

Proposed UI Improvements & Live Labs

+ + +
+ ← Baseline Spec (v1.0.7 Master Guide) + ★ Proposed Improvements (Live Labs) +
+ + + + + +
+
+
+

Proposals Simulator Workbench

+
+ Strict 1:1 hardware font scale (Adafruit GFX Size 1 = 6×8px, Size 2 = 12×16px on 240×135 TFT) +
+
+ 1:1 HARDWARE ACCURATE +
+ +
+ +
+
+ +
+
+ Active Screen: Slot 2 · Split RF Activity Dashboard +
+ + +
+
+
Simulation Loops: ACTIVITY
+ LOOP: IDLE +
+
+ +
+
+
+ + +
+
+

1. Select Screen to Test

+
+ + + + +
+
Switch between candidate screen designs to inspect layouts with authentic 1:1 font sizing and clean header alignment.
+
+ +
+

2. Zero-Cost Palette Switcher (Lab)

+
+ + + + +
+
Swaps the 6-entry RGB565 lookup table. Proves direct sunlight readability and night stealth with 0 RAM allocation.
+
+ +
+

3. Header & Battery State Sim

+
+ + + +
+
Tests the authentic v1.0.7 header: battery outline, terminal nib, dynamic fill bars, and status dots.
+
+
+
+
+ + +
+ Carousel Slot 2 Upgrade +

Slot 2: Live Split Activity & RF Field Dashboard

+

+ Replaces the old 4-line static text with a high-density, real-time RF triage dashboard that stays readable at 240×135. +

+ +
+
+

Current v1.0.7 Slot 2

+
    +
  • Displays 4 static text rows: Last Sweep peak, Last Probe hit, Last Cell rise, Scope status.
    • +
  • No rolling time-series or packet throughput history.
    • +
  • Screen is completely blanked during active scans.
    • +
+
+
+

Proposed Split Dashboard

+
    +
  • Top Half (Live 30s Sparkline): Rolling packet rate bar graph (1 sample/sec, 30-byte ring buffer) with standard Size 1 header labels.
    • +
  • Bottom Half (3 Triage Cards): +
      +
    • SWEEP PK: 906.8 MHz (Size 2 font) @ -64dBm.
      • +
    • CAD HIT: CH 0 (Size 2 font) @ +8.5dB SNR.
      • +
    • AWAY T: 1.4s (Size 2 font) with 60s budget limit.
      • +
    +
    • +
+
+
+
+ + +
+ Workstream 12 Plate +

Workstream 12: Dedicated Focus Survey Display Plate

+

+ Direct UI specification for active Workstream 12. Provides honest single-channel dwell timing and statistical distribution without guessing. +

+ +
+
+

V1 Foundation (Missing WS12 UI)

+
    +
  • V1 only supported wide-band 85-bin Sweep or CAD candidate hopping.
    • +
  • No visual plate for single-channel dwell or sample confidence.
    • +
+
+
+

Proposed Focus Survey Screen

+
    +
  • Large Target MHz Header: 906.875 MHz [CH 0] (Size 2 bold font).
    • +
  • Dwell Progress Arc / Countdown: 08.4s / 10.0s with filled bar.
    • +
  • Statistical Bracket Gauge (-120 to -40 dBm): Clean linear scale marking Noise Floor (-116), P50 Median (-98), and P90 (-82).
    • +
  • Operator Truth Badge: [SAMPLING] → [REPEATED] (High confidence) or [INSUFFICIENT] (Zero evidence labeled truthfully, never guessed).
    • +
  • Instant Abort: [ESC] Abort & Return Home.
    • +
+
+
+
+ + +
+ Analyze Suite Modal +

Captures Ring: Non-Decrypting Packet Inspector Modal

+

+ Pressing [Enter] on any packet in the Captures ring brings up a forensic RF breakdown without violating our strict RX-only / no-decryption boundaries. +

+ +
+
+

Current Captures Screen

+
    +
  • Scrollable list of 16-entry ring buffer.
    • +
  • Only shows: Timestamp, Node ID, RSSI, and truncated string.
    • +
  • No way to inspect RF signal quality, AFC frequency offset, or framing headers.
    • +
+
+
+

Proposed Inspector Modal

+
    +
  • Detailed Signal Quality: RSSI (-78dBm), SNR (+9.2dB), Estimated Frequency Error / AFC (+1.4kHz).
    • +
  • LoRa Framing Details: Modulation (SF11/BW250/CR4:5), Payload Length (48 bytes), CRC OK / FAIL.
    • +
  • Raw Byte Hex Nibble Stream: Formatted hex view of raw over-the-air bytes (e.g. FF FF FF FF ...) without payload decryption or client protocol emulation.
    • +
+
+
+
+ + +
+ Display Foundations +

Zero-Cost Hardware Palette Switching (Sunlight & Night)

+

+ Direct sunlight washed out low-contrast greys on the ST7789V2 TFT; white screens ruin night vision. We propose 4 instant hardware color profiles with zero RAM footprint. +

+ +
+
+

Current v1.0.7 Palette

+
    +
  • Fixed black background (COL_BG = 0x0000).
    • +
  • Dim grey (COL_DIM = 0xBDF7) washes out under direct noon sunlight on windshield dashboard mounts.
    • +
  • White screen brightness impairs dark adaptation during nighttime wardrive operations.
    • +
+
+
+

Proposed Dynamic Lookups

+
    +
  • High-Sunlight Inverted: Crisp paper-white background with dark black ink and saturated blue/amber accents. Maximum ambient reflection contrast.
    • +
  • Tactical Red Night: True aviation red-on-black (#ff2828 on #000000) preserving rod rhodopsin for dark adaptation and stealth.
    • +
  • Amber CRT: Warm monochrome amber for reduced eye strain during extended night bench monitoring.
    • +
  • Zero RAM Cost: Swaps the existing 6-element uint16_t COL[] table in flash memory during redraws.
    • +
+
+
+
+ + +
+ Feasibility Analysis +

Hardware Memory & Performance Safety Budget

+

+ Verification against our strict Phase 7 memory limits: 0 bytes heap allocation, ≥40KB free SRAM safety floor, and Core 0 non-blocking execution. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Proposed FeatureStatic RAM CostHeap AllocationsExecution BudgetSafety Gate Status
P2: Slot 2 Live 30s Sparkline30 bytes (static ring)0 bytes< 0.5 msPASSED (< 32B budget)
P3: WS12 Focus Survey Display16 bytes (stat struct)0 bytes< 0.8 msPASSED (Bound verified)
P4: Captures Packet Inspector0 bytes (reads existing ring)0 bytes< 1.0 ms (on-demand)PASSED (No heap buffers)
P5: Zero-Cost Palette Switching48 bytes (flash table)0 bytes0 ms (pointer swap)PASSED (Zero SRAM)
+
+ + +
+
+ + + + + + diff --git a/docs/hardware-results/2026-09-04-phase12-focus-matrix.md b/docs/hardware-results/2026-09-04-phase12-focus-matrix.md new file mode 100644 index 0000000..57332ef --- /dev/null +++ b/docs/hardware-results/2026-09-04-phase12-focus-matrix.md @@ -0,0 +1,619 @@ +# Phase 12 §6.2 controlled dwell matrix — 900 trials + +**Build:** `V=1.0.7;R=3e31daa-dirty;BENCH=1` (bench image; production rejects +every command used here). +**Date:** 2026-09-04. **Location:** bench, indoors. No coordinates, route, or +observed third-party identity appears here or in the tracked artifacts. +**Fixture:** repo-owned Heltec V4 R8 controlled transmitter, output capped at +-9 dBm, quieted on every exit path. +**Raw evidence:** `private/phase12-matrix-20260904T223047Z.{jsonl,log}` +(git-ignored). Re-derive this summary with +`scripts/phase12_focus_matrix_report.py`. + +900 trials — 5 positions x 3 dwell arms x 30 source-on and 30 source-off, +alternating — completed in 54.7 min with **zero transport errors, zero queue +drops, zero dropped durable rows, and successful home restore on every trial**. +Each trial was one independently logged bounded Focus request. + +## Result by arm + +Ordered by the transmitter's measured airtime, which is what the outcome +actually tracks. `margin` is the gap between the worst source-on reading and +the strongest source-off reading; a negative margin means the sets overlap. + +| Position | tx airtime | offset from bin center | dwell | source-on min | source-off max | margin | +|---|---|---|---|---|---|---| +| low (905.3125) | 1751 ms | +62.5 kHz | 100 ms | -29.0 | -94.0 | 65.0 dB | +| low | 1751 ms | +62.5 kHz | 500 ms | -29.0 | -95.0 | 66.0 dB | +| low | 1751 ms | +62.5 kHz | 2000 ms | -29.0 | -75.0 | 46.0 dB | +| mid (912.8125) | 1026 ms | +62.5 kHz | 100 ms | -29.0 | -97.0 | 68.0 dB | +| mid | 1026 ms | +62.5 kHz | 500 ms | -29.0 | -98.0 | 69.0 dB | +| mid | 1026 ms | +62.5 kHz | 2000 ms | -32.0 | -92.0 | 60.0 dB | +| low-aligned (908.750) | 286 ms | 0 kHz | 100 ms | -55.0 | -97.0 | 42.0 dB | +| low-aligned | 286 ms | 0 kHz | 500 ms | -61.0 | -97.0 | 36.0 dB | +| low-aligned | 286 ms | 0 kHz | 2000 ms | -71.0 | -97.0 | 26.0 dB | +| mid-aligned (918.500) | 148 ms | 0 kHz | 100 ms | -32.0 | -97.0 | 65.0 dB | +| mid-aligned | 148 ms | 0 kHz | 500 ms | -87.0 | -92.0 | 5.0 dB | +| mid-aligned | 148 ms | 0 kHz | 2000 ms | -33.0 | -97.0 | 64.0 dB | +| high (920.625) | 94 ms | +125 kHz | 100 ms | -45.0 | -97.0 | 52.0 dB | +| high | 94 ms | +125 kHz | 500 ms | -47.0 | -96.0 | 49.0 dB | +| **high** | 94 ms | +125 kHz | **2000 ms** | **-99.0** | **-96.0** | **-3.0 dB** | + +In the 14 arms that separated, every one separated completely: 30/30 source-on +detected and 0/30 source-off, Wilson 95% intervals [0.886, 1.000] and +[0.000, 0.114]. + +## What the matrix established + +**1. Detection tracks airtime against sample spacing, not dwell length.** +Focus takes a fixed 8 instantaneous RSSI samples spread across the dwell, so +sample spacing is `dwell / (samples - 1)`: 14 ms, 71 ms, and 286 ms for the +three arms. Detection holds while the source's airtime is comfortably longer +than that spacing and degrades as it approaches it: + +| tx airtime | vs 286 ms spacing (2000 ms arm) | worst-case source-on across dwells | +|---|---|---| +| 1751 ms, 1026 ms | much longer | flat: -29, -29, -29 / -29, -29, -32 | +| 286 ms | comparable | degrades: -55, -61, -71 | +| 148 ms | shorter | erratic: -32, -87, -33 | +| 94 ms | much shorter | fails at 2000 ms: -45, -47, -99 | + +This is the practical form of the §3 contract's warning that observation time +must not be read as time covering a frequency. **A 2000 ms dwell does not +observe 2000 ms — it observes eight instants.** Holding the sample count fixed +while lengthening the dwell makes the pass strictly worse at catching bursts, +which is the opposite of what "longer observation" suggests to a reader. Any +operator-facing presentation of dwell or observation time has to account for +this, and a later slice should scale the sample budget with the dwell rather +than leaving it fixed at 8. + +**2. Bin-center offset had no detectable effect at these signal levels.** This +was the concern that motivated adding the two exactly-centered control +positions: the `high` position sits 125 kHz from its bin center, outside the +125 kHz home-channel passband half-width. It still separated by 52 dB and +49 dB at the two shorter dwells, while the perfectly centered `low-aligned` +position separated by only 42/36/26 dB. Offset is therefore not what +distinguishes these arms — airtime is. This does not prove offset is +irrelevant at realistic signal levels; it proves it is not the dominant term +at a source this strong. + +**3. Radio-away time is dwell plus a small fixed overhead.** Measured 173 ms, +573 ms, and 2073 ms for the 100/500/2000 ms arms — a consistent ~73 ms of +retune-and-restore overhead, with a worst case of 2139 ms across all 900 +trials. No trial lost radio ownership or failed to restore home listening. + +## What it did not establish + +**No single fixed RSSI condition separates every arm.** Pooled across all 900 +trials the sets overlap (source-on min -99.0 dBm against source-off max +-75.0 dBm), and they still overlap with the one failing arm excluded +(-87.0 against -75.0). The nearest thing to a clean global condition is +`p90 >= -90 dBm`, which across the 14 separating arms misses no source-on +trial and takes 1 ambient false hit in 420. That is a **candidate** for §3.1, +not an accepted constant, and it carries three limits: + +- The bench source reads around -26 dBm against a -100 dBm ambient floor — + far stronger than realistic traffic. This separates a strong known source + from ambient; it is **not** a sensitivity limit and must not be shown as one. +- Source-off trials are ambient observation, not a calibrated false-hit rate: + the control path is not known quiet (§6.1). The single -75 dBm ambient + reading is a real received event, not necessarily noise. +- A condition chosen on `p90` of 8 samples inherits finding 1: it describes + eight instants, so its meaning changes with the dwell it ran at. + +**The coverage thresholds remain unselected.** `FOCUS_MIN_VALID_PASSES`, +`FOCUS_MIN_OBSERVATION_MS`, and the `repeated` pair are about pass counts and +accumulated time across repeated requests; a single-pass matrix cannot supply +them. `insufficient`/`sampled`/`repeated` stay undisplayable, and `coverage` +stays blank in `focus.csv`. + +**The §6.3 Watch-opportunity comparison has not run**, so the maximum +radio-away budget is still unapproved. + +## Method notes worth not relearning + +Two harness defects were found by running this, each of which produced +confident, wrong-looking data before it was caught: + +1. **One armed pulse at a fixed delay cannot cover every dwell arm.** At + 100 ms the pulse began after the window closed, and source-on was + indistinguishable from source-off across 18 trials — readable as "a 100 ms + dwell detects nothing." +2. **Re-arming faster than the airtime overlaps transmissions**, and the tail + bleeds into the next trial: source-*off* trials read -26 dBm — readable as + "the receiver hears a transmitter that is switched off." + +The runner now paces its burst by the transmitter's own `TX_DONE` and waits +for the tail to clear before a trial ends. A source-on trial consequently means +*the transmitter was radiating during the window*, which is the condition the +qualifying-RSSI question needs — and explicitly **not** a catch-probability +estimate for intermittent traffic. + + +--- + +# Follow-up: sample-spacing sweep (360 trials) + +**Raw evidence:** `private/phase12-spacing-20260905T010723Z.{jsonl,log}`. +360 trials, 31.5 min, zero transport errors, zero drops, home restored on +every trial. Same fixture and build family; home channel pinned to +918.5 MHz / SF8 / BW125 by bench build flags rather than by the SD card. + +Finding 1 above said detection tracks the source's airtime against sample +spacing. This sweep tests that directly: dwell held at 2,000 ms while the +sample count varies, so spacing is the only thing changing. + +| source airtime | 286 ms spacing (8) | 100 ms (21) | 50 ms (41) | 20 ms (101) | +|---|---|---|---|---| +| 94 ms (`high`) | 8/15, -3.0 dB | 2/15, -1.0 dB | **15/15, +2.0 dB** | **15/15, +24.0 dB** | +| 148 ms (`mid-aligned`) | 15/15, +53 dB | 15/15, +52 dB | 15/15, +54 dB | 15/15, +54 dB | +| 286 ms (`low-aligned`) | 15/15, +26 dB | 15/15, +25 dB | 15/15, +35 dB | 15/15, +36 dB | + +`detects` counts source-on trials reading above that arm's strongest ambient +reading; `margin` is worst source-on minus strongest source-off. + +**Detection collapses once spacing approaches the source's airtime.** The +94 ms source is missed at 286 ms and 100 ms spacing and caught in every trial +at 50 ms and 20 ms. Its worst-case reading also improves sharply between those +two (-97 dBm at 50 ms, -66 dBm at 20 ms), so 50 ms is the edge of working +rather than a safe choice. The two longer-airtime sources separate at every +spacing tested, and their worst case still improves as spacing tightens +(-71 to -64 dBm for the 286 ms source). + +**Finer sampling costs nothing measurable.** Radio-away time was 2,073-2,075 ms +across all four arms, whether the pass took 8 samples or 101: + +| samples | spacing | radio-away (min/max) | observation_ms | +|---|---|---|---| +| 8 | 286 ms | 2,074 / 2,075 ms | 2,000 ms | +| 21 | 100 ms | 2,074 / 2,075 ms | 2,000 ms | +| 41 | 50 ms | 2,073 / 2,074 ms | 2,000 ms | +| 101 | 20 ms | 2,074 / 2,074 ms | 2,000 ms | + +The cost of a longer dwell is the dwell. The sample count inside it is free, +and the histogram is a fixed 141 bytes regardless (§4.2), so there is no +memory argument for coarse sampling either. + +## Selected policy + +`FOCUS_SAMPLE_SPACING_MS = 20`, with `focusSamplesForDwell()` deriving the +count as `ceil(dwell / spacing) + 1`. Rounding up matters: truncating would +let a 30 ms dwell take two samples 30 ms apart and violate the constant it is +derived from. This keeps roughly a 2x margin against the ~40-50 ms airtime of +the fastest realistic mesh traffic, which is a judgement about what is worth +observing rather than a measured limit -- the measurement says only that +spacing at or above the airtime fails and half of it works. + +The `p90 >= -90 dBm` qualifying-condition candidate from the main matrix was +measured under the old 8-sample policy and a source ~70 dB above ambient. It +should be re-selected against this sampling and a weaker source before any +operator-facing use. + + +--- + +# Follow-up 2: qualifying condition at a realistic signal level (120 trials) + +**Raw evidence:** `private/phase12-step2-20260905T035952Z.{jsonl,log}`. +120 trials (2 positions x 30 source-on + 30 source-off), 9.4 min, zero +transport errors, zero drops, home restored on every trial. 2,000 ms dwell at +the measured 20 ms sampling policy (101 samples per pass). + +The earlier matrix chose `p90 >= -90 dBm` as a candidate qualifying condition +while the source sat ~70 dB above ambient. This run repeats the question at a +level a field deployment might actually see: transmitter on an outdoor porch, +driven over the WiFi control bridge, receiver with its antenna fitted so the +ambient floor is real. + +**Configuration:** source-on peak reads about -85 dBm against a -99/-100 dBm +ambient floor, versus -24 dBm against -98 dBm on the bench. Receiver antenna +fitted; transmitter at roughly one building's distance with a small stubby +antenna. + +## The candidate does not survive + +| metric | low-aligned (908.75) | mid-aligned (918.5) | +|---|---|---| +| `p90` (the previous candidate) | overlap | overlap | +| `peak` | separates by **2.0 dB** | overlap | +| `peak - median` | separates by 2.0 dB | overlap (-2.0 dB) | +| `peak - p90` | 1.0 dB, 3 false positives in 30 | overlap (-3.0 dB) | +| `p90 - median` | overlap | overlap | + +One of ten combinations separates, by 2 dB. That is inside ordinary RSSI +variance and would not survive a temperature change, a different antenna, or a +different room. **`p90 >= -90 dBm` is rejected**, and so is every relative +(floor-referenced) variant tested here. Both families fail for the same +reason, so this is not an artifact of picking the wrong statistic. + +Two observations explain it: + +- **`p90` collapses at low SNR even when the source is present most of the + time.** At `low-aligned` the transmitter radiated 57-71% of the window, yet + `p90` still read -94 to -97 dBm against a -100 floor. If the received signal + were steady, `p90` would equal `peak`. It does not: at these levels the + instantaneous RSSI during a weak transmission mostly fails to stand above + noise, and only its best moments do. That is why `peak` outperforms every + other summary here, and why none of them is reliable. +- **Real traffic is indistinguishable from the controlled source.** At + `mid-aligned` (918.5 MHz, a live MeshOregon channel) source-off trials + reached -94 dBm. That is not measurement error; it is a real transmission + arriving during a control trial. An RSSI condition cannot tell a controlled + fixture from a neighbour's node, and should not be expected to. + +## What this means + +An RSSI *summary statistic* cannot carry an activity claim at realistic signal +levels. Coverage reporting is unaffected: valid passes, observation time, and +the median/P90/peak summary itself remain honest and useful. What the evidence +refuses is the inference from "RSSI was elevated" to "something transmitted" — +precisely the inference §3's contract was written to prevent. The design was +right; the constant was wrong. + +Two untested routes remain, in order of cost: + +1. **A count, not a statistic.** `peak` is one sample and therefore noise-prone; + counting how many of a pass's 101 samples exceed an adaptive floor + integrates instead. `FocusObservation` already reserves `qualifying_count` + for this and never populates it. Needs firmware, and can reuse this fixture. +2. **CAD or packet evidence**, which §3 already contemplates as the alternative + basis for observed activity. A different primitive, and a larger change. + +Until one of those is measured, `coverage` stays blank, the activity count +stays unpopulated, and Focus reports what it observed rather than what it +concludes. + + +--- + +# Follow-up 3: Watch-opportunity comparison (§6.3) + +**Raw evidence:** `private/phase12-watch-20260905T042817Z.{jsonl,log}`. +Two 240 s arms against one independently timed reference train (`MESH_OREGON`, +918.5 MHz SF8/BW125, one pulse every 2 s), transmitter outdoors on the WiFi +control bridge, receiver on its resolved home channel. Zero CRC errors in +either arm. Focus requests were 2,000 ms dwell at the measured 20 ms sampling +policy (101 samples). + +| arm | reference | received | fraction | 95% CI | +|---|---|---|---|---| +| Watch only | 120 | 106 | 0.883 | [0.814, 0.929] | +| Watch + Focus | 121 | 56 | **0.463** | [0.377, 0.551] | + +Focus completed 56 requests, was refused none, and held the radio for +116,142 ms — **48.1% of its arm**. Time between restored Watch windows: +median 4.04 s, maximum 7.45 s. The intervals do not overlap, so the +difference is not a small-sample artifact. + +## The cost is proportional, with nothing hidden + +If Focus simply stops the receiver hearing anything while away, and costs +nothing else, the predicted reception is the baseline scaled by the time +remaining at home: `0.883 x (1 - 0.481) = 0.459`. Measured: **0.463**. The +difference is well inside the interval. + +That is the useful part of this result. It says the loss is entirely +accounted for by away time — there is no additional penalty from retuning, +from recovery, or from any lingering effect once home listening is restored. +**The radio-away duration Focus already records is an honest proxy for what a +request costs Watch**, which is what makes an operator-facing away-time +display meaningful rather than decorative. + +The other half is that there is no mitigation either. Half the listening time +away is half the packets. A displayed cost is still a cost. + +## Limits + +- The baseline is 0.883, not 1.0: about 12% of reference pulses were missed + with no Focus running. These are relative figures against a real link, not + absolute capture rates, and the comparison is valid only because both arms + saw the same link and the same train. +- One duty cycle (48%) at one dwell (2,000 ms) at one pulse interval (2 s). + Proportionality held here; it is not established across other duty cycles, + and a much shorter dwell has proportionally more retune overhead per unit + of observation. +- The reference train is a controlled fixture, not real mesh traffic. Real + traffic is bursty and correlated in ways a fixed 2 s interval is not. + +## What it does not decide + +A maximum radio-away budget is an operator product decision, and this +measurement deliberately does not make it. What it supplies is the exchange +rate: **at this dwell, Watch packet opportunity falls in direct proportion to +the fraction of time Focus holds the radio.** Choosing what fraction is +acceptable — and whether Focus should bound it automatically rather than +leaving it to whoever is pressing the button — remains open. + + +--- + +# Follow-up 4: a count survives where every summary statistic failed + +**Raw evidence:** `private/phase12-counts-20260905T050037Z.{jsonl,log}`. +120 trials (2 positions x 30 source-on + 30 source-off), 11.4 min, zero +transport errors, zero drops, home restored on every trial. Same porch +configuration and 2,000 ms / 101-sample passes as Follow-up 2, which rejected +every RSSI summary statistic at this signal level. + +Follow-up 2 ended by naming one untested idea: `peak` is a single sample and +therefore noise-prone, so **count** the samples above an adaptive floor rather +than taking an extreme of them. The bench image now reports a ladder of counts +at median + 2/4/6/8/10/15/20 dB, so one run evaluates any such rule offline. + +## Result + +Pooled across both positions, best operating point **`C6 >= 2`** — at least two +of a pass's samples at or above that pass's own median plus 6 dB: + +| | count | 95% CI | +|---|---|---| +| detected, source on | **57/60 (95.0%)** | [0.863, 0.983] | +| detected, source off | **1/60 (1.7%)** | [0.003, 0.089] | + +Per position: + +| position | detect | false | +|---|---|---| +| low-aligned (908.75) | 30/30 | 1/30 | +| mid-aligned (918.5) | 27/30 | 0/30 | + +`C4 >= 5` performs identically (57/60 and 1/60); the ladder has a broad +plateau rather than one lucky point, which is what a real effect looks like. + +For contrast, at this same signal level Follow-up 2 found `p90` and `peak`, +absolute and floor-relative, could not separate the two populations at all. + +## The one false positive is not a false positive + +The single source-off trial flagged by `C6 >= 2` read `peak = -63 dBm` against +a -101 dBm median — 38 dB above its own floor, with six elevated samples. +Nothing 38 dB over noise is noise. That is a real transmission that was not +ours, correctly identified by the detector and mislabelled by the experiment, +which cannot distinguish "our fixture is quiet" from "the band is quiet". The +true false-alarm rate against genuine silence may therefore be 0/60; this +evidence cannot separate the two, and the reported 1.7% should be read as an +upper bound. + +The three misses are all mid-aligned at `peak = -95/-96 dBm`, within a couple +of dB of the floor. A detector that misses signals at the noise floor is +behaving correctly; one that claimed them would be the problem. + +## What still has to be settled before this is a constant + +**The threshold is a count out of 101 samples — about 2% — and the sample +count now scales with dwell.** Under the measured 20 ms sampling policy a +100 ms pass takes 6 samples, where "2 samples" is 33% rather than 2%. A fixed +count therefore means something completely different at another dwell. The +rule must be expressed as a fraction of accepted samples, or validated +separately per dwell, before it can be written into firmware. That is the next +measurement, not a detail to decide at a keyboard. + +Also unestablished: one distance, one indoor/outdoor geometry, two +frequencies, one 2,000 ms dwell. And the margin is relative to the pass's own +median, which assumes the median represents the floor — true when the source +is present for a minority of samples, and less true as occupancy rises toward +50%. + +Until those close, `qualifying_count` stays unpopulated and `coverage` stays +blank. What this run establishes is narrower and still worth having: **an +adaptive count is a viable basis for an activity claim where every summary +statistic measured here was not.** + + +--- + +# Follow-up 5: what the count rule actually detects + +**Raw evidence:** `private/phase12-dwell-20260905T053804Z.jsonl` (240 trials) +and `private/phase12-occ-20260905T053804Z.jsonl` (240 trials). Zero arm +failures. Same porch configuration. + +Follow-up 4 left two questions: is the rule a fixed count or a fraction of +samples, and does "median as the floor" survive rising occupancy. Both were +measured rather than argued. + +## 1. A fraction, with a hard minimum sample count + +Sample count now scales with dwell (20 ms spacing), so the same rule was +re-derived at each dwell: + +| dwell | samples/pass | best rule | as a fraction | detect | false | +|---|---|---|---|---|---| +| 100 ms | 6 | `C4 >= 1` | 16.7% | **23/60 (38%)** | 1/60 | +| 500 ms | 26 | `C6 >= 1` | 3.8% | **60/60 (100%)** | 2/60 | +| 2,000 ms | 101 | `C4 >= 5` | 5.0% | 57/60 (95%) | 1/60 | + +A fraction of roughly **4-5% of accepted samples** transfers between the +500 ms and 2,000 ms passes. It does not rescue the 100 ms pass, which fails at +every threshold: with six samples there is no rule that both catches the +source and rejects ambient. That is a floor on the instrument, not a tuning +problem — **a short pass can report coverage honestly, but must not report +activity at all.** 500 ms / 26 samples was the strongest arm measured. + +## 2. Median-as-floor held, but was not tested where it should break + +Across source duty from 28.6% to 57.2%, the pass median read -101.0 dBm for +source-on and source-off alike. It never tracked the signal, so the adaptive +floor is sound over the range tested. + +The reason is worth stating, because it bounds the result: at these levels +most samples read like noise even while the transmitter is radiating, which is +the same effect that made `p90` useless in Follow-up 2. A **strong** source at +the same occupancy would pull the median up and break the assumption. This +sweep did not reach that regime and does not license a claim about it. + +## 3. Detection is a function of occupancy, and that is the limitation + +Same dwell, same rule, only the source's duty within the window varied: + +| source duty | detect (`C6 >= 5`, n=30) | 95% CI | +|---|---|---| +| 57.2% | 27/30 (90%) | [0.74, 0.97] | +| 42.9% | 28/30 (93%) | [0.79, 0.98] | +| 28.6% | 11/30 (37%) | [0.22, 0.54] | +| 28.6% | 13/30 (43%) | [0.27, 0.61] | + +Detection collapses below roughly 40% occupancy. **This rule detects a +persistently occupied channel; it does not detect individual packets.** A +single SF8 packet inside a 2,000 ms pass is on the order of 3-7% occupancy — +far below where this measurement shows the rule already failing more than half +the time. + +That is not a defect to tune away. It is the honest scope of an RSSI-sampling +instrument: it can say "this frequency was busy while I listened", and it +cannot say "a transmission occurred". The latter needs a primitive that +detects a signal rather than sampling energy — CAD, or actual packet +reception, which §3 has contemplated as the alternative basis from the start. + +## Consequences for the phase + +- Any activity condition is a **fraction** of accepted samples (~4-5%), never + a fixed count, and applies only above a minimum sample count. 6 samples is + below it; 26 is comfortably above. +- An activity claim, if one is ever surfaced, must be worded as channel + occupancy during the pass, not as evidence that something transmitted. +- Detecting individual packets needs CAD or packet reception. That is now the + better-supported route for §3's "observed activity", and this sweep is the + reason to prefer it rather than a hunch. +- `qualifying_count` remains unpopulated. What to populate it with is now a + design decision with evidence behind it rather than an open guess. + + +--- + +# Follow-up 6: a second baseline, and a correction + +**Raw evidence:** `private/phase12-b2-{occ,counts,dwell}-20260905T134743Z.jsonl` +(600 trials, no arm failures). Same porch position and geometry as Follow-up 5; +the only change is the link. + +Every conclusion in Follow-ups 4 and 5 came from one link at roughly 12 dB SNR. +This repeats the SNR-sensitive tests at roughly 25 dB, to separate properties +of Focus from properties of that link. + +**Link change:** the transmitter's stubby was replaced with a matched whip and +both antennas were stood vertical. Vertical alignment mattered more than the +antenna: at 918.5 MHz the horizontal pair was losing about 13 dB, consistent +with each whip sitting in the other's pattern null. Source peak moved from +-85/-86 dBm to -77/-66 dBm at 908.75 and from -87/-89 to -69/-75 at 918.5, +against an unchanged -98/-101 dBm floor. + +## The occupancy cliff was an SNR limit — Follow-up 5's conclusion is withdrawn + +| source duty | detect at ~12 dB | detect at ~25 dB | +|---|---|---| +| 57.2% | 27/30 (90%) | 30/30 (100%) | +| 42.9% | 28/30 (93%) | 29/30 (97%) | +| **28.6%** | **11/30 (37%)** | **26/30 (87%)** | +| 21.5% | not tested | 22/30 (73%) | + +Follow-up 5 concluded that "the rule detects a persistently occupied channel, +not individual packets", and reasoned from there that CAD or packet reception +was the better-supported basis for an activity claim. **That conclusion does +not survive a second link.** At 28.6% occupancy, 13 dB of link improvement +took detection from 37% to 87%. The cliff moved, so it was not the statistical +limit it was presented as; it was where *that* link stopped delivering samples +above the margin. + +What can be said now is narrower and conditional: **detection depends on +occupancy and SNR together**, and the occupancy at which it fails is a +property of the link, not of the instrument. Single-packet detection is +untested and is no longer ruled out — a 150 ms packet at 20 ms sampling yields +roughly seven elevated samples, comfortably above the four or five the best +rules require, provided the samples clear the margin. The lowest occupancy +measured is 21.5%, so anything below that remains unmeasured in both +directions. + +## What did survive + +- **The 100 ms floor is real.** Six samples per pass detected 23/60 at 12 dB + and 28/60 at 25 dB. Thirteen decibels bought almost nothing, so this is a + sampling limit rather than a link limit: a short pass may report coverage + and must not report activity. +- **The rule is a fraction.** Best thresholds landed at 3.8-5.0% of samples at + 12 dB and 4.0-7.7% at 25 dB. Roughly stable, and clearly not a fixed count. +- **500 ms / 26 samples remains the strongest arm** at both SNRs (60/60 each), + with zero false positives at 25 dB. + +## Method note + +This is why a single link is not a baseline. Three conclusions were drawn from +the first one; two held and one inverted. The one that inverted was the most +consequential and the most confidently worded, and it would have sent the next +slice toward CAD on the strength of an artifact. Where a conclusion depends on +signal level, it now needs at least two levels before it goes in this file. + +## Next measurement + +Drive occupancy below 21.5% at the stronger link — a single short-airtime +pulse per pass is about 5% — and find where detection actually fails when the +link is not the binding constraint. That, not a CAD rewrite, is the open +question for §3's activity basis. + + +--- + +# Follow-up 7: a single packet is detectable + +**Raw evidence:** `private/phase12-singlepulse-20260905T173222Z.jsonl` +(120 trials, no arm failures). Same ~25 dB link as Follow-up 6. One armed +transmission placed inside each 2,000 ms pass instead of a burst — a burst +answers a different question however its duty is tuned. + +| position | packet | occupancy | detect (`C6 >= 4`) | false | +|---|---|---|---|---| +| mid-aligned | 148 ms | 7.4% | **29/30 (97%)**, [0.83, 0.99] | **0/30** | +| low-aligned | 286 ms | 14.3% | 20/30 (67%), [0.49, 0.81] | 2/30 | + +Source-on peak at mid-aligned: median -74 dBm against a -97 dBm floor. + +**This settles the question Follow-up 5 got wrong.** That entry concluded the +rule "detects a persistently occupied channel, not individual packets". +Follow-up 6 withdrew it on the grounds that the occupancy cliff was a property +of a weak link; this measures the positive case directly. At an adequate link, +a single packet occupying 7% of the pass is detected essentially every time, +with no false positives in thirty controls. + +The arithmetic behind it is unremarkable in hindsight: a 148 ms packet sampled +every 20 ms yields roughly seven elevated samples, comfortably above the four +the rule requires. Nothing about the instrument prevented this. The first +link did. + +## low-aligned was contaminated, and that is informative + +The weaker result at low-aligned comes with a source-off control that peaked +at **-57 dBm** — 44 dB above its own floor. That is real traffic on 908.75 +during the run, not noise, and it cuts both ways: it produces the two false +positives, and by lifting the median in affected passes it suppresses counts +and produces misses. An adaptive floor is only as good as the assumption that +the floor is quiet. + +Two consequences worth carrying forward. A frequency with real activity is +harder to measure on, not easier, so controlled work should prefer quiet +positions. And in the field the same effect applies in reverse: Focus will be +least reliable exactly where a band is busiest. + +## Limits + +- One link (~25 dB SNR) and one dwell. At the ~12 dB link of Follow-ups 4-5 + this would be substantially worse; the rate is not a property of Focus alone. +- The runner verifies that the pulse fired, not precisely *when* it fired + relative to the window. An attempt to classify placement from log timestamps + was invalid and discarded: `TX_STARTED` reaches the host only when the + harness next polls the socket, so its timestamp records host polling rather + than RF timing. 29/30 detection implies placement was mostly correct, but it + is inferred rather than measured. Timing the pulse against the window needs + a device-side timestamp, not a host-side one. +- 30 trials per cell. The intervals are wide enough to matter: [0.83, 0.99] + is consistent with a true rate anywhere from about six-in-seven to + essentially always. + +## Where this leaves the activity claim + +An RSSI-sampling pass **can** support a packet-level activity signal at an +adequate link, using a count above an adaptive floor. What it cannot do is +support it *unconditionally* — the same rule at a weaker link failed on +sources occupying four times as much of the window. Any operator-facing +activity indication has to be honest that it is a function of link quality, +which the device cannot know. That is a wording and product problem now, +rather than an open measurement. diff --git a/docs/hardware-results/2026-09-05-ws17-sweep-settle.md b/docs/hardware-results/2026-09-05-ws17-sweep-settle.md new file mode 100644 index 0000000..261cc8b --- /dev/null +++ b/docs/hardware-results/2026-09-05-ws17-sweep-settle.md @@ -0,0 +1,276 @@ +# Workstream 17 — Pass A reads a floor ~14 dB low as shipped + +**Build:** bench image, `V=1.0.7;BENCH=1`. **Date:** 2026-09-05. +**Link:** Workstream 12's porch fixture at its stronger setting, matched whips +vertical, transmitter on `MESH_OREGON` (918.5 MHz -> Sweep bin 66 of 85). +**Raw:** `private/ws17-retune-probe.log`, `private/ws17-check*.jsonl`. + +Entry measurement for the candidate workstream. `src/version.h` records an +11 dB under-read on Cell caused by sampling RSSI before the AGC settles after +a light retune, and states plainly that "Whether Sweep has the same under-read +is unknown". This measures it. + +Each row: two silent laps and three laps with the transmitter radiating, +median of `BENCH_SWEEP_FLOOR` for bin 66. + +| Pass A configuration | silent floor | source on | delta | +|---|---|---|---| +| **LIGHT + 0 ms settle (shipped)** | **-115.0 dBm** | -114.8 | +0.2 dB | +| LIGHT + 5 ms settle | -101.5 dBm | -102.5 | -1.0 dB | +| LIGHT + 20 ms settle | -101.7 dBm | -98.2 | +3.5 dB | +| FULL `begin()` per bin | -112.3 dBm | -112.5 | -0.2 dB | + +## Established + +**Pass A as shipped reads the noise floor about 14 dB below a settled +receiver.** Adding a 5 ms settle moves the silent floor from -115.0 to +-101.5 dBm, and 20 ms holds it at -101.7. Focus independently measures the +ambient floor at this frequency as -99 to -101 dBm across hundreds of passes, +which is where the settled readings land and is 14 dB above the shipped one. + +That answers `version.h`'s open question: Sweep has the same defect Cell had, +and larger. It matters because Pass A's peak decision compares a bin against a +noise floor plus a margin, and both terms are being computed from under-read +samples. `ENERGY_DEFAULT_THRESHOLD_MARGIN_DBM_X10` was calibrated against +those same under-read values, so the margin is not obviously wrong — but it is +calibrated on a scale that does not correspond to real dBm, which makes any +absolute reasoning about Sweep sensitivity unsound until this is settled. + +## Not established, and not to be inferred from this + +**Whether Pass A can flag a bin carrying traffic remains open.** The +source-present column is not a clean measurement and should not be read as +one. Even at 20 ms settle the source produced only +3.5 dB, when Focus reads +the same transmitter at -69 to -75 dBm against a -100 floor. Candidate +explanations, none yet tested: the source is on air for only part of each +lap's 3 ms visit to bin 66, so a median across three laps washes it out; the +firing pipeline's ~1.5 s bridge latency makes "source on during this specific +3 ms" hard to guarantee; and the `FULL begin()` row is internally odd, reading +a *lower* floor (-112.3) than the settled light path despite skipping the +settle only because a full begin already takes longer. + +Two earlier attempts at this measurement were invalid for the same reason and +are recorded so the next one does not repeat them: firing only for a lap's +duration radiates entirely after the lap ends, and reads as a silent band in +every arm. + +## Next + +- Establish the floor result properly: more laps, both link levels, and a + settle sweep finer than 0/5/20 ms, to find where the reading stabilises. +- Only then ask the detection question, with the source's presence during the + bin visit *verified* rather than assumed — which needs a device-side + timestamp, not a host-side one. +- Decide separately whether the margin constant should be recalibrated against + settled readings, or whether Pass A should settle before sampling. Those are + different fixes with different costs to lap time. + + +--- + +# Established: the floor stabilises by 3 ms, at ~15% of lap time + +**Raw:** `private/ws17-floor-20260905T181609Z.jsonl`. Eight settle values, +five laps each, five bins, **transmitter idle throughout**. Silence is the +point: the floor is a property of the receiver and its retune, so measuring it +with no source removes the timing problems that invalidated the first attempts +at the detection question. + +| settle | lap time | bin 0 | bin 20 | bin 40 | bin 66 | bin 84 | +|---|---|---|---|---|---|---| +| **0 ms (shipped)** | 1.70 s | -112.2 | -115.1 | -111.6 | -115.0 | -113.6 | +| 1 ms | 1.62 s | -105.3 | -108.4 | -104.2 | -108.5 | -107.0 | +| 2 ms | 3.76 s | -102.8 | -102.3 | -95.1 | -102.0 | -98.8 | +| **3 ms** | 1.74 s | -102.1 | -101.5 | -94.5 | -101.9 | -98.5 | +| 5 ms | 2.01 s | -102.7 | -102.0 | -94.5 | -102.7 | -99.3 | +| 10 ms | 2.10 s | -102.5 | -101.9 | -94.0 | -102.2 | -98.6 | +| 20 ms | 4.13 s | -102.0 | -101.8 | -94.5 | -102.3 | -98.3 | +| 40 ms | 5.60 s | -102.7 | -102.6 | -94.0 | -102.3 | -98.7 | + +**Three milliseconds is enough.** Readings are flat from 3 ms through 40 ms. +The shipped 0 ms configuration under-reads by 9.5 dB (bin 0) to 17.6 dB +(bin 40), and 1 ms recovers only about half of it. + +**The cost is small.** A settle is paid once per bin, so 85 bins multiply it: +3 ms predicts 0.26 s added per lap, and the measured 1.70 -> 1.74 s is that +within noise. 40 ms costs 5.60 s, which is why this is worth pinning down +rather than picking a generous value. Roughly 15% of lap time buys 10-18 dB of +floor accuracy. (The 2 ms and 20 ms lap times are out of trend and are +believed to be timing noise in the host's lap measurement, which includes +status polling; they do not affect the floor readings.) + +## A prediction that was wrong, and what it means + +This measurement was designed with bin 0 as a control: bin 0 takes a full +`begin()` while later bins take the light retune, so if the under-read were a +settle artifact, bin 0 should not move. **It moved 9.5 dB.** Re-reading +`performEnergySweep()`, the settle is skipped only when the *global* FULL +retune mode is on — the condition is `!benchSweepRetuneFullEveryBin()`, not a +per-bin test — so in the shipped LIGHT mode bin 0 receives the full begin +*and* the settle. It was never a control. + +Bin 0 did shift least (+9.5 dB against +12.5 to +17.6 elsewhere), which is +consistent with a full begin providing partial settling on its own, but that +is an observation and not a demonstrated mechanism. The clean version of this +control would compare FULL mode against LIGHT+3 ms across all bins. + +## Also unexplained + +The settled floor is not flat across the band: bin 40 reads about 8 dB above +bins 20 and 66 at every settle value from 2 ms up. That could be real ambient +energy near 912 MHz or a front-end response, and this run does not +distinguish them. It matters because Pass A compares each bin against a floor +plus a margin, and a band-dependent floor is a different situation from a flat +one. + +## What follows + +- A ~3 ms settle in Pass A is cheap and recovers most of the error. It is the + obvious fix, but it cannot be made alone: the shipped margin constant was + calibrated against under-read values, so changing the settle without + recalibrating the margin changes Sweep's peak decisions in an untested + direction. +- This is one environment and one link. The floor is a receiver property so it + should transfer, but it has not been checked at a second location. +- The detection question — can Pass A flag a bin carrying traffic — remains + open and is now better approached with real repeater traffic on two known + channels (bins 34 and 66), using the other 83 bins as within-run controls, + than with a fixture whose presence during a 3 ms visit cannot be guaranteed. + + +--- + +# The under-read is not cosmetic: Pass A misses real traffic because of it + +**Raw:** `private/ws17-traffic-20260905T182652Z.jsonl`. 68 laps, **fully +passive — this run transmitted nothing.** The sources are two live repeaters +on known channels; the bench transmitter was explicitly quieted. Settle +alternated 0 ms and 3 ms lap by lap so both configurations saw the same +traffic. Ended early at lap 68 of 200 on a USB-CDC transport timeout: this +script lacks the retry hardening the Focus matrix runner has. + +Bins 34 (MeshCore, 910.525 MHz) and 66 (MeshOregon, 918.5 MHz) carry traffic; +seven bins spread across the band are read in the *same lap* as controls, so +the comparison needs no external ground truth. + +| | control median | control max | bin 34 max | bin 66 max | +|---|---|---|---|---| +| 0 ms (shipped), 34 laps | -115.2 | -112.3 | **-95.3** | **-113.8** | +| 3 ms, 34 laps | -102.8 | -96.4 | **-56.5** | **-50.8** | + +Laps where a traffic bin exceeded the control maximum: 2/34 and 0/34 at 0 ms; +2/34 and 2/34 at 3 ms. Adjacent bins (33/35/65/67) were clean at 0/34 except +one 1/34 excursion at bin 67, so energy lands in the intended bin. + +## Pass A's own threshold would reject what it caught + +`ENERGY_DEFAULT_THRESHOLD_MARGIN_DBM_X10` is 350, i.e. a bin must sit **35 dB** +above the noise floor to be flagged as a peak. Applying that to what was +actually measured: + +| | floor | best traffic reading | excursion | flagged? | +|---|---|---|---|---| +| **0 ms (shipped)** | -115.2 | bin 34 at -95.3 | **19.9 dB** | **no** | +| | | bin 66 at -113.8 | 1.4 dB | no | +| **3 ms** | -102.8 | bin 34 at -56.5 | **46.3 dB** | yes | +| | | bin 66 at -50.8 | 52.0 dB | yes | + +**As shipped, Pass A would not have flagged either repeater in this run. With +a 3 ms settle it would have flagged both.** That is the same traffic on +alternating laps, so it is not a difference in conditions. + +This is what makes the settle a defect rather than a calibration curiosity. +The floor measurement showed the noise floor read 9.5-17.6 dB low; this shows +the *signal* is under-read far more — the same repeater burst read -95.3 dBm +at 0 ms and -56.5 dBm at 3 ms, a 39 dB difference — which matches +`version.h`'s note that the under-read grows with signal strength. A floor +that is 13 dB low and a signal that is 39 dB low do not cancel: the excursion +that the margin tests collapses from 46 dB to 20 dB, and falls under the +threshold. + +## What this does and does not support + +Supported: at the shipped settle, real traffic that Pass A physically sampled +did not reach its own flagging threshold, and a 3 ms settle fixed that in the +same run. The mechanism is consistent across the floor sweep, this run, and +the prior Cell finding. + +Not supported: any rate claim. Two detections in 34 laps per configuration is +a tiny count, the hit rate is limited by the ~3 ms bin visit coinciding with a +burst rather than by sensitivity, and 0 ms versus 3 ms hit counts (2 and 0 +against 2 and 2) are not distinguishable at that size. **The decisive evidence +here is the magnitude of the excursion, not the number of hits.** Traffic was +also uncontrolled; alternating laps controls for drift but not for a burst +happening to fall in one arm. + +Also unchanged: one environment, one link, one repeater pair. + +## Recommendation + +Pass A should settle before sampling, and the margin must be re-derived in the +same change. 3 ms costs about 0.26 s of a 1.7 s lap. Shipping the settle alone +would leave a 35 dB margin calibrated against under-read values now being +applied to correctly-read ones, which changes peak decisions in an untested +direction — the fix is one change, not two. + + +--- + +# Confirmation at 300 laps + +**Raw:** `private/ws17-traffic2-20260905T231056Z.jsonl`. 300 laps, **0 skipped**, +62.7 minutes, fully passive — the bench transmitter was quieted and this run +transmitted nothing. Same two live repeaters, same alternating 0/3 ms settle, +same within-lap controls. The harness gained retry-on-transport-failure and +skip-a-lap-rather-than-abort after the previous attempt died at lap 68. + +Judged by Sweep's own rule: a bin is flagged when it sits 35 dB +(`ENERGY_DEFAULT_THRESHOLD_MARGIN_DBM_X10`) above the lap's noise floor. + +| | bin 34 (MeshCore) | bin 66 (MeshOregon) | adjacent + control bins | best traffic excursion | +|---|---|---|---|---| +| **0 ms (shipped)** | 0/150, CI [0.000, 0.025] | 0/150, CI [0.000, 0.025] | **0 flagged** | **19.0 dB** | +| **3 ms** | 3/150, CI [0.007, 0.057] | 3/150, CI [0.007, 0.057] | **0 flagged** | **51.6 dB** | + +## What this establishes + +**The shipped configuration never came close to flagging traffic it was +physically sampling.** Its best excursion across 150 laps was 19.0 dB against +a 35 dB threshold — not a near miss, a factor of two in dB terms. The settled +configuration reached 51.6 dB on the same traffic in interleaved laps. + +**Specificity is clean.** Across both arms, no adjacent bin (33/35/65/67) and +no control bin (5/15/25/45/55/75/80) was ever flagged — roughly 1,650 +non-traffic bin observations per arm with zero false flags. When the settled +configuration flags something, it flags the two channels that actually carry +traffic and nothing else. + +**Both repeaters behaved alike at 3 ms** (3/150 each), where the shorter +earlier run had seen MeshOregon twice and MeshCore twice. The per-session +difference in activity the operator described shows up as variance between +runs rather than as a systematic difference here. + +## What this does not establish + +**The flag *rates* are still not statistically separated.** 0/150 gives +[0.000, 0.025] and 3/150 gives [0.007, 0.057]; those overlap on [0.007, +0.025]. Separating a 0% rate from a 2% one needs roughly 300 laps per arm, not +150, and this run was sized before that arithmetic was done. + +That gap does not weaken the conclusion, because the rate is a *consequence* +of the margin arithmetic rather than independent evidence for it: a +configuration whose best excursion is 19.0 dB cannot flag a 35 dB threshold at +any sample size. The rate would only become the load-bearing evidence if the +excursion result were ambiguous, and it is not. + +**The ~2% flag rate at 3 ms is a coincidence rate, not a sensitivity figure.** +Pass A visits each bin for about 3 ms per lap, so it can only flag traffic +that happens to be transmitting during that window. That bound applies equally +to a fixed Sweep and is a property of the design, not of the settle. + +## Status + +This closes Workstream 17's entry question. Pass A's settle under-read is not a +calibration curiosity: it is the difference between flagging the two busiest +channels in the band and flagging nothing at all. The fix — settle, and +re-derive the margin in the same change — remains unapplied and untested. diff --git a/docs/history/ROADMAP_V1.md b/docs/history/ROADMAP_V1.md new file mode 100644 index 0000000..3ed2f65 --- /dev/null +++ b/docs/history/ROADMAP_V1.md @@ -0,0 +1,21 @@ +# V1 roadmap archive + +The complete v1 roadmap is frozen at the stable +[`v1.0.7` release tag](https://github.com/d3mocide/LoRaTrace-RX/blob/v1.0.7/docs/ROADMAP.md), +commit `1d8ea360fda8dc3d159ea1c0e626edcbedb42114`. + +This archive intentionally points to that immutable source instead of copying +978 lines of completed phase narrative into the live documentation tree. The +tag's `docs/ROADMAP.md` content SHA-256 is: + +```text +bedf2e9f736b1da88c1a4161277e7e46ba8f21fcf39b216bb85fb7a54223048d +``` + +Use it for v1 Phases 0–11, historic feasibility decisions, distribution +details, phase-number versioning, and their original exit criteria. The live +[V2 roadmap](../ROADMAP.md) contains only active workstreams and gates. + +Historical comments and changelog entries that mention a v1 "Phase N" or the +old roadmap refer to this tagged snapshot unless they explicitly link to a +later research or hardware-results record. diff --git a/docs/research/V2_DESIGN.md b/docs/research/V2_DESIGN.md index 5d7aa28..49c443e 100644 --- a/docs/research/V2_DESIGN.md +++ b/docs/research/V2_DESIGN.md @@ -1,7 +1,7 @@ # LoRaTrace RX — Version 2 Design **Status:** draft for product decisions and phased validation. -**Baseline:** v1.0.0, released 2026-09-03. +**Baseline:** v1.0.7, released 2026-09-04. **Purpose:** describe the post-v1 direction without rewriting the hardware-verified v1 design, roadmap, or status record. ## 1. Product thesis @@ -214,23 +214,25 @@ transmitter, or packet decoder into the Cardputer. ## 6. Proposed build order -The following is a priority order, not a committed release calendar. A phase -lands only after its scope, static-memory cost, host tests, and applicable -hardware evidence are complete. Version numbers should be decided in the -canonical roadmap when the first post-v1 phase is accepted. +The following is a priority order, not a committed release calendar. A +workstream lands only after its scope, static-memory cost, host tests, and +applicable hardware evidence are complete. The canonical gate and version +policy live in `docs/ROADMAP.md`. | Workstream | Outcome | Why it comes here | |---|---|---| -| **11. Cell closeout** | Validate a real tower-adjacent RSSI rise and verify cell/session output on SD. | Finishes existing, partially verified scope before broadening the product. | | **12. Survey truth** | Coverage vocabulary, persistent per-survey evidence, and bounded Focus Survey. | Directly fixes the most important interpretation gap revealed by V1 testing. | | **13. Field Missions** | Drive, Stationary, and Investigate recipes with visible Watch availability and radio-time accounting. | Turns individual tools into a coherent operator workflow. | | **14. Companion analysis** | Offline run report, comparison, and export bundle. | Delivers more value from existing and V2 logs without consuming device heap. | | **15. Field markers and privacy-aware sharing** | Preset annotations and shareable, redacted report options. | Makes collected data interpretable and responsibly reusable. | -| **16. Region packs** | First rigorously sourced, validated non-US profile pack(s). | High value, but research-heavy and easy to get wrong if rushed. | +| **16. Cell closeout** | Validate the existing Cell feature's real tower-adjacent RSSI rise and fresh `cell.csv`/`session.csv` output. | A useful bonus feature, deliberately deferred so it does not block the core V2 field-survey workflow. | The following stay as later candidates until a concrete field question justifies -them: richer passive node-activity dossiers, antenna/install comparative -surveys, battery/endurance guidance, and additional lab correlation tooling. +them: rigorously sourced region packs, richer passive node-activity dossiers, +antenna/install comparative surveys, battery/endurance guidance, and additional +lab correlation tooling. Region packs require their own source-quality, +regulatory/range, and real-hardware-access entry gate; they are not a V2.0 +release dependency. ## 7. Architecture and resource rules @@ -321,4 +323,3 @@ observation coverage, device health, and field context. It does **not** need to become a transmitter, decoder, protocol client, cloud platform, or unlimited-band analyzer to achieve that. - diff --git a/docs/research/phase12-survey-truth-design.md b/docs/research/phase12-survey-truth-design.md new file mode 100644 index 0000000..c1ac85b --- /dev/null +++ b/docs/research/phase12-survey-truth-design.md @@ -0,0 +1,539 @@ +# Phase 12 Survey Truth — design entry and acceptance plan + +**Status:** design entry active; it permits a bench-only raw-counter prototype, +not operator-facing coverage labels or release scope. +**Baseline:** LoRaTrace RX `v1.0.7` on Cardputer-Adv + Cap LoRa-1262. +**Field-validation area:** Portland metro, Oregon. This is a coarse test-area +label only; no coordinates, route, node identity, or claim about local RF +occupancy belongs in tracked evidence. + +## 1. Why this phase exists + +Phase 9 established that a normal Sweep's short bin dwell can miss genuine +traffic. The real-traffic dwell comparison was inconclusive, and the later +capture investigation showed that packet capture is governed by time parked on +the home channel, not by making a full-band bin dwell slightly longer. Neither +result licenses the UI to call an empty bin quiet. + +Phase 12 therefore adds a deliberate, bounded answer to a narrower question: + +> At this selected frequency, what did the receiver observe for how long, and +> what did that observation contain? + +It does not try to infer a protocol, prove absence, turn Sweep into background +retuning, or improve packet capture by accident. + +## 2. Scope and non-goals + +The first Focus Survey slice is deliberately small: + +- **One selected frequency per request.** The source is one completed Sweep + or Waterfall bin, or one fixed menu preset. Free-form frequency entry, + contiguous windows, and multi-bin requests are deferred until this slice's + actual static-memory and radio-away measurements support them. +- **One bounded radio-owned request.** It snapshots the resolved home channel, + gathers fixed streaming statistics, writes one durable result row, and + restores Watch on complete, cancel, timeout, and failure. +- **No raw RF samples or packet bytes.** The radio task retains only fixed + accumulators; logger/GPS/SD remain outside its real-time path. +- **No identity claim.** RSSI and CAD/packet events, if a later slice adds + them, remain observations or unknown LoRa candidates under the existing + evidence rules—not Meshtastic, MeshCore, Reticulum, LoRaWAN, or cellular + identity by inference. +- **No automatic action.** Focus is operator-selected and menu reachable; + Drive/Stationary/Investigate recipes belong to Workstream 13. + +## 3. Observation and coverage contract + +The result presents *coverage* separately from *activity*. + +| Term | Definition for Phase 12 | Must not mean | +|---|---|---| +| Requested pass | One requested dwell at the selected frequency with a declared time and sample budget. | A successful measurement. | +| Valid pass | A requested pass that configured the selected frequency, produced its required samples, and was neither cancelled nor radio-error terminated. | A quiet channel. | +| Observation time | Measured elapsed time from the first accepted sample through the last accepted sample, summed only across valid passes. | Time continuously covering a frequency outside those windows. | +| Observed activity | A documented qualifying RSSI condition or independently recorded packet/CAD event during a valid pass. | Protocol identity or continuous activity. | +| No observation | No qualifying event occurred during the valid windows. | “Quiet,” “empty,” or “absent.” | +| Insufficient / sampled / repeated | Coverage labels derived only from valid-pass count and accumulated observation time. | Signal strength, likelihood of absence, or confidence in identity. | + +The firmware must store the underlying counts and durations even after a label +is chosen. The display may say `N of M passes` and elapsed observation time; +it must never replace those values with a single confidence word. + +### 3.1 Labels not yet frozen + +The first implementation must not hardcode arbitrary thresholds. Before the +Device/claim gate closes, the controlled matrix below selects: + +1. `FOCUS_MIN_VALID_PASSES` and `FOCUS_MIN_OBSERVATION_MS` for **sampled**; +2. `FOCUS_REPEATED_VALID_PASSES` and `FOCUS_REPEATED_OBSERVATION_MS` for + **repeated**; and +3. one documented qualifying RSSI condition for the activity count. + +Until those constants are accepted, a bench-only prototype may report only `N +valid passes`, requested/completed status, total observation time, and RSSI +summary. It may not show a coverage label or a “no activity” conclusion. + +## 4. Bounded data design + +`focus.csv` is append-only and purpose-specific. Existing detection, session, +probe, energy, and Cell records keep their current meanings and schemas. + +The planned one-row-per-selected-bin result contains the shared run/GPS +columns, then: + +```text +rx_uptime_ms,profile,focus_id,selection_source,selection_bin_index, +freq_mhz,requested_passes,valid_passes,requested_dwell_ms, +observation_ms,requested_samples,sample_count,rssi_median_dbm, +rssi_p90_dbm,rssi_peak_dbm,qualifying_count,coverage,request_status, +home_restore,wifi_on,radio_status +``` + +`coverage` is empty until §3.1 closes. `request_status` distinguishes complete, +cancelled, timeout, and failed; a cancelled or failed request cannot borrow a +coverage label from its partial samples. `home_restore` is recorded separately +because “complete” without Watch recovery is not a valid acquisition result. + +### 4.1 Fixed statistics budget + +The initial implementation uses one selected bin and a fixed 1 dB RSSI +histogram over a documented bounded receiver range, plus count, sum, maximum, +elapsed-time, and saturation flags. This gives median and P90 without raw +sample retention or a heap allocation. The exact histogram range, bucket type, +request/result struct sizes, queue depth, and worst-case CSV row length are +part of the Engineering-gate budget; the target is a single result state under +256 bytes of static SRAM before queue/control overhead. + +This is a target, not a claim of current usage. A wider focus set, dynamic +container, second result history, or per-event allocation is out of scope for +this slice and requires a new budget decision. + +### 4.2 Measured budget (2026-09-04) + +Counted from the built production image and a host measurement of the +formatter, not estimated. Struct sizes are additionally held by `static_assert` +so a schema change cannot quietly exceed them. + +| Item | Measured | Bound | +|---|---|---| +| `FocusRequest` | 12 B | 16 B | +| `FocusObservation` (queue/result record) | 40 B | 48 B | +| `FocusRssiHistogram` (141 one-byte buckets, -140..0 dBm) | 148 B | 160 B | +| One result's working state (histogram + record) | 188 B | 224 B, under §4.1's 256 B target | +| All Focus statics in the production image | 159 B | — | +| Request queue item storage | 1 x 12 B | — | +| Result queue item storage | 4 x 40 B = 160 B | — | +| `radioTask` stack frame | 1,072 B | 6,144 B task stack | +| Worst-case `focus.csv` row | 189 B | 256 B buffer (`FOCUS_CSV_ROW_MAX`) | +| SD write rate | one row per terminal request | — | + +Notes on what those numbers do and do not cover: + +- The stack frame is the whole radio task's, measured with `-fstack-usage`; + every bounded action is inlined into it, so it is a shared ceiling rather + than Focus's own cost. Focus's objects account for at most 216 B of it. + The device's own measured high-water is `radio_stack_free` in `session.csv`, + which this budget has not yet read back off a card. +- The worst-case row uses every field at its widest, including widths a + bounded request cannot produce (a 65,535 ms dwell, a 255-pass count). An + over-long row is dropped rather than truncated—`focusObservationFormatCsv()` + returns 0 and the logger counts a dropped row—so the 66 B of margin is + deliberate and regression-tested at the maximum. +- The 326 B header is written straight to the file (`File::println`) with no + intermediate buffer, so it is not bound by the row budget. +- Measured radio-away time across the 2026-09-04 behavior run was 74 ms + (an injected failure, which never sampled) to 1,573 ms (a 1,500 ms injected + stall). That is fixture timing, not the §6.3 Watch-opportunity decision. + +## 5. Radio and UI contract + +`radio_task` owns Focus exactly as it owns Probe, Sweep, Cell, and Scope. At +most one bounded acquisition action owns the SX1262. A Focus request is visibly +refused when another action owns the radio; it never waits behind it. Every +exit path restores the resolved home configuration before publishing its final +state. + +The on-device menu supplies the runtime control required by `CLAUDE.md`. The +status surface and Activity page show the selected frequency, source, valid/ +requested passes, observation time, request state, and recovery result. +`WATCHING`, `SURVEYING`, and `RESTORING` remain mutually exclusive visible +states. Browser observation remains read-only; adding browser acquisition +control would require a separate security decision. + +Focus's radio-away measurement starts when it leaves resolved home listening +and ends only after `restoreHomeListen()` reports success or failure. It is a +new Focus-specific health/status value; Sweep's `EA` is evidence for Sweep, +not a substitute for this measurement. + +The sample budget alone does not bound that measurement. Every sample waits on +the shared SPI bus, so a contended bus stretches a pass past its dwell while +the radio is away from home. A request therefore also carries a wall-clock +deadline (`focusRequestTimeoutMs()`: dwell plus a slack constant): past it the +request stops sampling and terminates as `timeout`, restoring home like any +other exit path. A recovered timeout stays a timeout—it does not borrow +`complete` from a restore that worked—and it contributes no valid pass and no +observation time. The slack is a bench-slice bound; §6 may revise it. + +## 6. Controlled measurement matrix + +The matrix establishes whether the product's bookkeeping and wording are true. +It does not seek a universal RF sensitivity number. + +### 6.1 Fixture and ground truth + +- Use the existing repository-owned Heltec V4 R8 controlled transmitter and + bench protocol; it is the timing source, not the Cardputer. +- Exercise three sourced US Sweep positions already used by the Phase 9 + low/mid/high harness: 905.3125, 912.8125, and 920.625 MHz. This checks + selection and bin translation across the accessible US range without + inventing a Portland channel plan. +- **None of those three sits on a Focus bin center, and the offset is part of + the measurement.** Focus tunes at the resolved home channel's bandwidth + (125 kHz on this bench), so where the transmitter sits inside the bin + changes what a pass can observe. Measured against the US 250 kHz grid: + 905.3125 and 912.8125 MHz are +62.5 kHz from bins 13 and 43, at the edge of + a 125 kHz passband's half-width; 920.625 MHz sits exactly on a bin boundary, + 125 kHz from either bin 74 or 75 — outside it. A weak or absent rise at the + high position would therefore measure the offset, not the receiver, so the + matrix also runs two exactly bin-centered controls from the same repo-owned + candidate table: `LONG_TURBO` 908.750 MHz (bin 27) and `MESH_OREGON` + 918.500 MHz (bin 66). Every trial records its own offset; an arm that fails + to separate is reported as unresolved, never as a quiet frequency. +- Record transmitter send timestamps and a time-aligned RTL-SDR capture when + making an activity-detection claim. The receiver's own `focus.csv` cannot + prove a missed event. +- Run source-on and source-off controls in alternating order. A source-off + antenna-in-room run is ambient observation only—not a calibrated false-hit + rate—unless the fixture/control path itself is known quiet. +- **A source-on trial means the transmitter was radiating for the whole + window.** One armed pulse at a fixed delay cannot be scheduled against every + arm: measured airtimes across this candidate table span ~10 ms (SF8/BW250) + to ~275 ms (SF12/BW125), and the receiver's window opens tens of ms after + the request is accepted, so a delay that lands inside a 2,000 ms dwell + misses a 100 ms one entirely. The first matrix attempt measured exactly + that: at 100 ms, source-on and source-off were indistinguishable because the + pulse began after the window closed. The runner therefore re-arms + continuously for the trial's duration, paced by the transmitter's own + `TX_DONE` — a fixed interval shorter than the airtime queues overlapping + sends whose tail bleeds into the next trial, which the second attempt then + measured as source-off trials reading -26 dBm. + This makes the matrix a measurement of the qualifying RSSI condition under a + known-present source. It is explicitly **not** a catch-probability estimate + for intermittent traffic; §1 already records that comparison as inconclusive + and this is not a second attempt at it. +- The bench source is also far stronger than realistic traffic (source-on + reads near -26 dBm against a -100 dBm floor at these antenna separations). + A threshold selected here separates a strong known source from ambient; it + is not a sensitivity limit and must not be presented as one. + +The SX1262 documentation and its CAD application note remain the authoritative +hardware references; device-specific timing and RSSI behavior are accepted +only from this board's measured matrix, not copied from a datasheet. + +### 6.2 Candidate dwell matrix + +Test three candidate dwell budgets—100 ms, 500 ms, and 2 s—at each fixture +frequency. They are measurement arms, not shipped defaults. For each arm: + +1. run at least 30 source-on and 30 source-off requests in randomized or + alternating order; +2. retain transmitter log, Cardputer framed status/boot identity, fresh + `focus.csv`, `session.csv`, and SDR correlation where applicable; +3. verify count/time arithmetic against timestamps and the independent source; +4. record request completion, cancellation latency, timeout, failure, restore, + queue/row drops, and actual radio-away time; and +5. report activity opportunity as a binomial proportion with a 95% interval, + never as a certainty. Use Wilson or exact intervals rather than a symmetric + normal approximation at small counts. + +`scripts/phase12_focus_matrix.py` runs the arms and writes one JSONL row per +trial; it requires explicit `--allow-transmit`, quiets the transmitter on every +exit path, and rejects a trial that loses home restore, misses its sample +count, or drops a queued or durable row. It selects no threshold. +`scripts/phase12_focus_matrix_report.py` is the offline, deterministic analysis +over that file: per arm it prints the source-on and source-off distributions, +the lowest 1 dB condition that separates them (or reports that they overlap), +and Wilson 95% intervals for both detection proportions. Neither script can +emit a coverage label. + +An arm is rejected if it loses radio ownership, cannot restore home listening, +has nonzero unexplained queue/row drops, or its durable row disagrees with the +measured request. No hit-rate target is set until the source waveform, link +budget, and qualifying condition are locked; a deceptively precise percentage +would answer a different question than coverage. + +### 6.3 Watch-opportunity measurement + +Before accepting a dwell/pass budget, compare: + +- Watch-only baseline at the active home channel against the independent + transmitter/reference count; and +- the same interval with repeated, explicitly bounded Focus requests. + +Report received/reference packets with a 95% interval, Focus away time, +completed requests, and the distribution of time between restored Watch +windows. The acceptance decision is an operator product decision after the +measurement; it must not imply that an away-time display makes the loss free. + +`scripts/phase12_watch_opportunity.py` runs both arms against one independently +timed pulse train and reports exactly those figures. Two constraints make the +comparison mean anything, and the script enforces the first and reports the +second: + +- The reference train must be receivable by Watch, so it uses the one + transmitter candidate matching this bench's resolved home channel + (`MESH_OREGON`, 918.5 MHz SF8/BW125/CR4-5/sync 0x2B) and refuses to run if + the device's home channel is something else. Otherwise both arms measure + nothing and their equality would look like a reassuring result. +- The loss figure is only interpretable next to the **away fraction**: the + share of the arm's wall time Focus actually held the radio. Host round-trip + latency inflates the idle gaps between requests, so a small measured loss at + a small away fraction says nothing about a larger duty cycle. The script + prints both and refuses to reduce them to one number. + +### 6.4 Matrix result (2026-09-04) + +900 trials completed with zero transport errors, zero queue or row drops, and +successful home restore on every trial. The full location-redacted summary is +[docs/hardware-results/2026-09-04-phase12-focus-matrix.md](../hardware-results/2026-09-04-phase12-focus-matrix.md); +the three results that change this design are: + +1. **Detection tracks the source's airtime against sample spacing, not dwell + length.** With a fixed 8 samples, spacing is `dwell / 7` — 14/71/286 ms for + the three arms. Sources with airtime far above the spacing were caught in + every arm; as airtime approached and fell below it, worst-case source-on + degraded monotonically and one arm stopped separating entirely. This is §3's + "observation time is not coverage" warning made concrete: **a 2000 ms dwell + observes eight instants, not 2000 ms**, and lengthening the dwell at a fixed + sample count makes a pass *worse* at catching bursts. A later slice should + scale the sample budget with the dwell; §4's schema and any operator-facing + dwell/observation display must not imply continuous coverage. +2. **Bin-center offset was not the dominant term.** The +125 kHz `high` + position still separated by ~50 dB at the two shorter dwells while a + perfectly centered control separated by less. The §6.1 offset concern is + not what distinguishes these arms at this source strength. +3. **Radio-away is dwell plus ~73 ms**, worst case 2,139 ms over 900 trials. + +And the two the matrix could not settle: + +- **No single fixed RSSI condition separates every arm.** Pooled, the sets + overlap (source-on min -99.0 dBm vs source-off max -75.0 dBm). `p90 >= -90 + dBm` is the nearest candidate — no missed source-on and 1 ambient false hit + in 420 across the 14 separating arms — but it is a candidate only, measured + against a source ~70 dB above ambient, and its meaning is tied to the sample + spacing of the arm it ran at. +- **Coverage thresholds remain unselected.** They are about pass counts and + accumulated time across repeated requests, which a single-pass matrix cannot + supply. `coverage` stays blank and the labels stay undisplayable. + +## 7. Portland metro field validation + +After controlled acceptance, run a small stationary field validation in the +Portland metro, Oregon area. It checks usability and evidence durability, not +local spectrum identity or coverage of the city. + +- Select only conditions already visible in a completed Sweep/Waterfall or a + fixed preset; record their source in `focus.csv`. +- Use a small number of operator-chosen stops; retain exact coordinates and raw + CSVs privately. Commit only a location-redacted summary with build revision, + rough conditions, requested/completed passes, radio-away time, health + counters, and caveats. +- Confirm on-device selection/cancel/recovery feedback, SD removal/error + behavior, and WiFi-off/on resource behavior under real GPS/SD workload. +- Treat a no-activity result as a bounded observation at that stop. Do not use + it to map a quiet area or publish a protocol or operator identity claim. + +## 8. Gate checklist and implementation order + +### Design entry — active + +- [x] V2 boundaries, one-bin first slice, result vocabulary, fixed-statistics + direction, controlled matrix, and Portland field-validation handling are + recorded here. +- [x] Choose the qualifying RSSI condition from measurement rather than a desk + estimate — **measured and rejected**. §6.4's `p90 >= -90 dBm` candidate came + from a source ~70 dB above ambient. Repeated at a realistic level (source + peak ~-85 dBm against a -99/-100 dBm floor, 120 trials), it fails, and so + does every floor-relative variant: one of ten metric/position combinations + separates, by 2 dB, which is inside ordinary RSSI variance. An RSSI summary + statistic cannot support an activity claim at field levels. See + [the evidence summary](../hardware-results/2026-09-04-phase12-focus-matrix.md). + Coverage reporting is unaffected; what is refused is the step from "RSSI was + elevated" to "something transmitted", which is the step §3 forbids anyway. + The count route named there has since been **measured and survives**: a + per-pass count of samples above the pass's own median separates where every + summary statistic failed. `C6 >= 2` (two or more samples at median + 6 dB) + detected 57/60 source-on trials, 95% CI [0.863, 0.983], with one flagged + source-off trial that read -63 dBm against a -101 dBm median — a real + transmission, not ours, so the 1.7% false rate is an upper bound rather than + a measurement against true silence. Three further campaigns (1,200 trials) then + characterised it. + + **Form.** It is a *fraction* of accepted samples, roughly 4-8%, which + transfers between 500 ms and 2,000 ms passes. No threshold rescues a 100 ms + pass: six samples cannot both catch the source and reject ambient, and 13 dB + of link improvement barely moved it (23/60 to 28/60), so that is a sampling + limit rather than a link limit. **A short pass may report coverage and must + not report activity.** 500 ms / 26 samples was the strongest arm at both + measured links. + + **Scope, after a correction.** An earlier reading of one link concluded the + rule "detects a persistently occupied channel, not individual packets". A + second baseline withdrew it: at 28.6% occupancy, 13 dB took detection from + 37% to 87%, so the apparent cliff belonged to that link. The positive case + was then measured directly — one armed 148 ms packet inside a 2,000 ms pass, + 7.4% occupancy, detected **29/30 with 0/30 false positives**. A 148 ms + packet sampled every 20 ms yields about seven elevated samples against the + four the rule needs, so nothing about the instrument prevented it. + + **The condition that remains.** Detection is a function of link quality the + device cannot know: the same rule at the weaker link failed on sources + occupying four times as much of the pass. And a position carrying real + traffic measured *worse*, not better — a -57 dBm event in a control trial + both produced false positives and suppressed counts by lifting the median, + so Focus is least reliable exactly where a band is busiest. Wording an + activity indication that stays true under those conditions is a product + decision now, not an open measurement. CAD or packet reception remains §3's + alternative, neither favoured nor excluded by this evidence. +- [ ] Coverage thresholds (`sampled`/`repeated`) remain unselected. They are + about pass counts and accumulated time across repeated requests, which no + single-pass measurement can supply. +- [~] Approve the maximum radio-away budget. **The measurement is complete** + ([evidence](../hardware-results/2026-09-04-phase12-focus-matrix.md)): at a + 2,000 ms dwell and 48.1% away fraction, Watch reception fell from 0.883 to + 0.463 of a reference train, with non-overlapping 95% intervals. The loss is + proportional to away time and nothing else — predicted 0.459 against a + measured 0.463 — so Focus's recorded away duration is an honest proxy for + what a request costs, and there is no hidden retune or recovery penalty. + **The budget decision itself remains open**, deliberately: this supplies the + exchange rate, not the policy, and §6.3 requires the decision to be an + operator product judgement made after the measurement rather than implied + by it. +- [x] Count actual static SRAM, stack frame, queue, row-length, and SD-rate + budgets (§4.2). Counted after the vertical slice rather than before it, on + the built image instead of on paper; the numbers land inside the bounds this + section set, so nothing had to be resized. + +### Engineering gate + +- [x] Pure one-bin request, histogram/percentile, CSV, and restore-before- + publish state tests pass. The fixed RSSI histogram is 141 one-byte buckets + (-140 through 0 dBm); + its 148-byte working state plus the 40-byte result record total 188 bytes, + below the 256-byte one-result target before queue/control overhead. +- [x] Focus plan/statistics/runtime and framed-control native tests pass, and + both production and bench firmware build. Focus is now linked into the + bounded bench image; the production image retains the same source but + rejects every bench trigger. +- [x] Fixed request/result/queue/CSV budgets are measured and accepted: see + §4.2. Every struct is inside its `static_assert` bound, one result's working + state is 188 B against the 256 B target, and the worst-case row is 189 B + against a 256 B buffer that drops rather than truncates. +- [x] A bounded bench harness extends the existing framed Serial Control + pattern; no ad-hoc USB text parsing is treated as evidence. +- [x] A bench-only prototype now has a Core-1-owned one-bin request, Core-0 + `focus.csv` queue/writer, and restore-before-publish terminal row. It is + entered only by the bench-image `BENCH_FOCUS bin:dwell_ms:samples` framed + command; production rejects that command. `BENCH_FOCUS_RESULT` returns only + the latest compact, GPS-free fixed-point summary to make the fixture + reproducible; the durable record remains `focus.csv`. +- [x] Two paired smoke checks on the bench image confirmed quiet/source-on + requests at US Sweep bin 43 (912.750 MHz), each 500 ms/8 samples. The + controlled Heltec `LONG_MODERATE` pulse (912.8125 MHz, capped -9 dBm) + completed inside each source-on window. Quiet P90/peak was -101.0/-101.0 + dBm; source-on P90/peak was -82.0/-82.0 then -87.0/-87.0 dBm, a 14--19 dB + rise. All rows reported successful home restore and durable `focus.csv` + writes. These are transport/RSSI smoke checks, **not** a threshold, + calibration, coverage, or activity-detection claim. The reusable + `scripts/phase12_focus_bench.py` requires `--with-pulse --allow-transmit` + before it may arm the transmitter. +- [x] The non-transmitting `scripts/phase12_focus_behavior_bench.py` hardware + fixture confirmed two cancelled Focus requests, an injected post-retune + failure, and two-way refusal with Probe and Sweep. Every Focus terminal row + was durably written only after home restore; the failed row retained the + injected operation error after recovery. +- [x] The request is bounded in wall-clock time, not only in samples (§5), and + the timeout path is reachable on demand: `BENCH_FOCUS`'s optional fourth + field arms a one-shot bench sample-loop stall, since production only times + out under real bus contention a fixture cannot arrange. Cell and Scope are + menu-only actions in production, so the bench-only `BENCH_ACTION` opcode + starts, cancels, and reports them through the same request functions the + menu calls—without it, Focus's mutual exclusion against them has no + reproducible fixture. Both are bench-image gated; production rejects them. +- [x] Sizing `STATUS` by hand had left its 240-byte argument buffer past the + frame's own ~230-byte budget once Phase 12 added six fields, and an + over-long frame is dropped silently rather than truncated—losing exactly the + newest fields the fixtures read. The argument budget is now derived from the + frame size, host-tested at saturation, and the frame limit raised to 384. + +### Device, claim, and release gates + +- [x] Hardware proves timeout plus mutual exclusion against Cell and Scope. + `scripts/phase12_focus_behavior_bench.py` drove all four on the bench image + (`V=1.0.7;R=3e31daa-dirty;BENCH=1`, 2026-09-04): a 1,500 ms injected stall + against a 100 ms/1,100 ms-deadline request terminated as `timeout` + (`RS=2`) with successful home restore and 1,573 ms total radio-away time, + and Focus and Cell, then Focus and Scope, each refused the other while it + owned RX. Six terminal requests across the run wrote six durable rows with + zero queue or row drops (`FW` 0->6, `FD`/`FL` 0), which is the logger's + post-write counter, not an enqueue count. Timing evidence is the receiver's + own; it is not a coverage, calibration, or activity claim. +- [x] The controlled matrix and the Watch-opportunity comparison are both + complete, with durable evidence and a location-redacted summary + ([2026-09-04-phase12-focus-matrix.md](../hardware-results/2026-09-04-phase12-focus-matrix.md)). + Between them they settled the sampling policy, rejected the qualifying RSSI + condition, and measured Watch's cost. What they left open is recorded above: + the away-time budget decision, the coverage thresholds, and an activity + basis that is not an RSSI summary. +- [ ] WiFi-off/on resource matrix, Portland field validation, `STATUS.md`, + `LOG_GUIDE.md`, release notes, and any companion-schema update reconcile. + +### Next, in order + +The Engineering gate is closed and the bench slice is hardware-proven, so what +remains is ordered by what blocks what. **The sampling change comes first: it +invalidates any threshold or budget chosen before it.** + +1. **Scale the sample budget with the dwell** (§6.4 finding 1). Today's fixed + 8 samples mean a long dwell observes a handful of instants, so a dwell or + observation-time figure does not mean what an operator would read it to + mean. Decide what a pass should sample — a target spacing, a rate, or a + dwell-proportional count — inside the existing static-RAM budget (§4.2 has + 66 B of row margin and 68 B under the working-state target). Re-run the + affected §6.2 arms afterwards; the shorter-airtime positions are the ones + that move. +2. **Re-select the qualifying RSSI condition** against the new sampling, and + at a source closer to realistic levels than this bench's ~-26 dBm. §6.4's + `p90 >= -90 dBm` is a candidate measured under the old sampling and a very + strong source; it should not survive into the product unexamined. +3. **Run §6.3's Watch-opportunity comparison** + (`scripts/phase12_watch_opportunity.py`) and approve, or refuse, a maximum + radio-away budget. Measured away time is already known (dwell + ~73 ms); + what is unknown is what repeated requests cost Watch. +4. **Select the coverage thresholds.** These are about valid-pass counts and + accumulated observation time across repeated requests, which no + single-pass matrix can supply — it needs its own repeated-request + measurement, once 1--3 have settled what a pass is worth. +5. Only then: the operator menu control, the Activity/status surface, the + durable final schema with a populated `coverage`, `LOG_GUIDE.md`'s + operator-facing `focus.csv` section, and Portland field validation. Until + an operator control exists, `focus.csv` is deliberately absent from the + operator log guide — documenting a file nobody can produce would be worse + than omitting it. + +A release is further out than the closed gates suggest: Focus is reachable +only from the bench image, and step 1 is a design change, not a tuning pass. + +## Sources + +- Project evidence: `docs/STATUS.md` (Phase 9 dwell, capture-window, and + RTL-SDR findings); `src/radio_task.cpp` / `.h` (bounded ownership and Sweep + away-time precedent); `scripts/phase9_bin_accuracy_bench.py` and + `scripts/bench_harness.py` (existing controlled-fixture pattern). +- [Semtech SX1262 resources](https://www.semtech.com/products/wireless-rf/lora-connect/sx1262) + — datasheet and SX126x CAD application note catalogue; use them for hardware + API/reference behavior, not as a substitute for board measurements. +- [NIST confidence intervals for proportions](https://www.itl.nist.gov/div898/handbook/prc/section2/prc241.htm) + — Wilson and exact interval guidance for small-count activity-opportunity + measurements. diff --git a/platformio.ini b/platformio.ini index 2b4a1c7..7f459b2 100644 --- a/platformio.ini +++ b/platformio.ini @@ -89,6 +89,16 @@ extends = env:cardputer-adv build_flags = ${env:cardputer-adv.build_flags} -D LORATRACE_BENCH_FAULTS=1 + ; Pins the bench receiver's home channel into the image so it no longer + ; depends on config.txt surviving on the bench SD card. Focus receives at + ; the home bandwidth, so a card swap that reverts home to the built-in + ; default silently changes a measurement's bandwidth. Matches the channel + ; the 2026-09-04 dwell matrix ran at; production is unaffected. + -D LORATRACE_BENCH_HOME_FREQ_MHZ=918.5f + -D LORATRACE_BENCH_HOME_SF=8 + -D LORATRACE_BENCH_HOME_BW_KHZ=125.0f + -D LORATRACE_BENCH_HOME_CR=5 + -D LORATRACE_BENCH_HOME_SYNC=0x2B ; Host-native unit test env — for pure-logic headers (channel_plans.h and ; onward: fingerprint.h, log-schema formatting) that don't touch hardware. diff --git a/scripts/heltec_wifi_setup.py b/scripts/heltec_wifi_setup.py new file mode 100644 index 0000000..1846ce0 --- /dev/null +++ b/scripts/heltec_wifi_setup.py @@ -0,0 +1,119 @@ +#!/usr/bin/env python3 +"""Configure the bench transmitter's WiFi control bridge, over USB. + +Run this yourself: it prompts for the network password locally and sends it +straight to the board, so the credential never passes through a transcript, +a shell history entry, or a log file. The raw framed exchange is deliberately +not echoed for the WIFI_PASS command. + +The bridge exists so the transmitter can sit far enough from the receiver to +give real path loss while a host still drives it. Once it reports an IP, +point any bench harness at it with: + + --heltec-port socket://:4227 + +The harness needs no change for that: bench_harness.py opens ports through +pyserial's serial_for_url(), which accepts socket:// URLs. + +SECURITY: the bridge has no authentication. Anything that can reach the port +can key the transmitter (bounded, and capped at the firmware's -9 dBm, but +still a transmitter). Enable it on a trusted network only, and run +`--disable` when the fixture is not in use. +""" + +import argparse +import getpass +import io +import pathlib +import sys + +import serial + +sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent)) +from bench_harness import TX_MARKER, Endpoint, parse_fields, require_ack + + +def show_status(tx): + opcode, payload = tx.request("WIFI_STATUS", "-", timeout=6.0) + if opcode != "STATUS": + raise RuntimeError(f"WIFI_STATUS failed: {opcode} {payload}") + fields = parse_fields(payload) + enabled = fields.get("EN") == "1" + connected = fields.get("CONN") == "1" + print(f" enabled : {'yes' if enabled else 'no'}") + print(f" connected : {'yes' if connected else 'no'}") + print(f" ssid : {fields.get('SSID')}") + if connected: + print(f" reachable : socket://{fields.get('IP')}:{fields.get('PORT')}") + return fields + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument("--port", required=True, + help="the transmitter's USB path (use /dev/serial/by-id/...)") + parser.add_argument("--ssid", help="prompted for if omitted") + parser.add_argument("--disable", action="store_true", + help="turn the bridge off and forget nothing else") + parser.add_argument("--status", action="store_true", help="report and exit") + args = parser.parse_args() + + # Frames are space-delimited, so a credential containing a space cannot be + # carried. Better to say so up front than to have the board reject it. + log = io.StringIO() + tx = Endpoint("heltec", args.port, TX_MARKER, log) + try: + identity = require_ack(tx, "HELLO", "-", timeout=25.0) + print(f"transmitter: {identity}\n") + + if args.status: + show_status(tx) + return 0 + + if args.disable: + print(require_ack(tx, "WIFI_OFF", "-", timeout=6.0)) + print("bridge disabled.\n") + show_status(tx) + return 0 + + ssid = args.ssid or input("SSID: ").strip() + if not ssid or " " in ssid: + raise SystemExit("SSID must be non-empty and contain no spaces " + "(the frame grammar splits on spaces)") + password = getpass.getpass("Password (blank for an open network): ") + if " " in password: + raise SystemExit("password cannot contain a space " + "(the frame grammar splits on spaces)") + + require_ack(tx, "WIFI_SSID", ssid, timeout=6.0) + require_ack(tx, "WIFI_PASS", password if password else "-", timeout=6.0) + require_ack(tx, "WIFI_ON", "-", timeout=6.0) + print("\nenabled; waiting for the join to complete...") + + import time + deadline = time.monotonic() + 30.0 + while time.monotonic() < deadline: + fields = parse_fields(tx.request("WIFI_STATUS", "-", timeout=6.0)[1]) + if fields.get("CONN") == "1": + print() + show_status(tx) + print("\nUse that socket:// URL as --heltec-port. The transmitter can now " + "be moved away from the receiver.") + return 0 + time.sleep(2.0) + print("\ndid not join within 30s:") + show_status(tx) + print("\nCheck the SSID and password, and that the network is 2.4 GHz — " + "the ESP32-S3 has no 5 GHz radio.") + return 1 + finally: + tx.close() + + +if __name__ == "__main__": + try: + sys.exit(main()) + except (OSError, serial.SerialException, RuntimeError, TimeoutError, ValueError) as error: + print(f"heltec wifi setup: {error}", file=sys.stderr) + sys.exit(2) diff --git a/scripts/phase12_focus_behavior_bench.py b/scripts/phase12_focus_behavior_bench.py new file mode 100644 index 0000000..ec0d184 --- /dev/null +++ b/scripts/phase12_focus_behavior_bench.py @@ -0,0 +1,222 @@ +#!/usr/bin/env python3 +"""Non-transmitting Phase 12 Focus cancel/failure/timeout/arbitration checks. + +Uses only framed Serial Control and the bench image. It records Focus's +durable row after cancellation, a one-shot injected failure, and an injected +sample-loop stall past the request deadline, then verifies mutual refusal with +Probe, Sweep, Cell, and Scope in both directions. Nothing here transmits, and +none of it sets a coverage or activity threshold. +""" + +import argparse +import pathlib +import sys +import time + +import serial + +from bench_harness import (CARD_MARKER, Endpoint, ResultWriter, card_status, + parse_fields, require_ack, wait_for) + + +FOCUS_ARGUMENT = "43:2000:64" # US 912.750 MHz bin; long enough to arbitrate. +# 100ms dwell -> a 1,100ms deadline (focus_plan.h), so a 1,500ms bench stall +# overruns it deterministically instead of waiting on real bus contention. +FOCUS_TIMEOUT_ARGUMENT = "43:100:4:1500" +FOCUS_TIMEOUT_STALL_MS = 1500 +# Stall, deadline, and one restore: past this the request was not bounded. +FOCUS_TIMEOUT_AWAY_LIMIT_MS = 4000 + + +def expect_error(card, command, argument, payload): + opcode, actual = card.request(command, argument, timeout=5.0) + if opcode != "ERROR" or actual != payload: + raise RuntimeError(f"{command} expected ERROR {payload}, got {opcode} {actual}") + + +def start_focus(card): + card.send("BENCH_FOCUS", FOCUS_ARGUMENT) + # The framed request is already on USB; allow Core 1 to claim its mailbox + # before sending the conflicting command, without relying on debug text. + time.sleep(0.35) + + +def cancel_focus(card, prior_written): + card.send("BENCH_FOCUS_CANCEL", "-") + terminal = wait_for( + card, + lambda status: status.get("FS") == "4" + and int(status.get("FW", "0")) > prior_written, + 10.0, + "cancelled Focus durable row", + ) + result = parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "HEALTH")) + if result.get("RS") != "1" or result.get("HR") != "1": + raise RuntimeError(f"cancelled Focus did not restore: {result}") + return terminal, result + + +def action_state(card): + return parse_fields(require_ack(card, "BENCH_ACTION", "STATE")) + + +def wait_for_action_idle(card, key, timeout, description): + deadline = time.monotonic() + timeout + last = None + while time.monotonic() < deadline: + last = action_state(card) + if last.get(key) == "0": + return last + time.sleep(0.1) + raise TimeoutError(f"timed out waiting for {description}; last state={last}") + + +def check_refuses_focus(card, action, key, results, event): + """Prove the named bounded action refuses a Focus request while it owns RX.""" + require_ack(card, "BENCH_ACTION", f"{action}:START") + time.sleep(0.35) + busy = action_state(card) + if busy.get(key) != "1": + raise RuntimeError(f"{action} did not take the radio: {busy}") + expect_error(card, "BENCH_FOCUS", "43:500:8", "UNAVAILABLE") + require_ack(card, "BENCH_ACTION", f"{action}:CANCEL") + terminal = wait_for_action_idle(card, key, 20.0, f"{action} to release the radio") + record(results, event, busy=busy, terminal=terminal) + + +def check_focus_refuses(card, action, key, results, event): + """Prove Focus refuses the named bounded action while Focus owns RX.""" + before = card_status(card) + start_focus(card) + expect_error(card, "BENCH_ACTION", f"{action}:START", "UNAVAILABLE") + during = action_state(card) + if during.get(key) != "0": + raise RuntimeError(f"{action} started while Focus owned RX: {during}") + terminal, result = cancel_focus(card, int(before["FW"])) + record(results, event, during=during, status=terminal, result=result) + + +def record(writer, event, **fields): + if writer is not None: + writer.write({"event": event, **fields}) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--cardputer-port", required=True) + parser.add_argument("--log", required=True, help="append-only raw framed-control log") + parser.add_argument("--results", help="append-only JSONL summary path") + args = parser.parse_args() + + log_path = pathlib.Path(args.log) + log_path.parent.mkdir(parents=True, exist_ok=True) + results = ResultWriter(args.results) if args.results else None + with log_path.open("a", encoding="utf-8") as log: + card = Endpoint("cardputer", args.cardputer_port, CARD_MARKER, log) + try: + identity = require_ack(card, "HELLO", "-", timeout=45.0) + initial = card_status(card) + if initial.get("SD") != "1": + raise RuntimeError(f"Focus requires an SD-backed result row: {initial}") + record(results, "boot", identity=identity, status=initial) + + start_focus(card) + card.send("PROBE_START", "-") + time.sleep(0.05) + cancel_status, cancel_result = cancel_focus(card, int(initial["FW"])) + # Native USB can fragment a one-way ERROR frame. The framed + # terminal STATUS is stronger evidence: a request accepted while + # Focus owned RX would have entered/raced Probe after cancel. + if cancel_status.get("B") != "IDLE": + raise RuntimeError(f"Probe started while Focus owned RX: {cancel_status}") + record(results, "focus_refuses_probe", status=cancel_status, result=cancel_result) + + before_sweep = card_status(card) + start_focus(card) + card.send("SWEEP_START", "-") + time.sleep(0.05) + cancel_status, cancel_result = cancel_focus(card, int(before_sweep["FW"])) + if cancel_status.get("W") != "IDLE": + raise RuntimeError(f"Sweep started while Focus owned RX: {cancel_status}") + record(results, "focus_refuses_sweep", status=cancel_status, result=cancel_result) + + require_ack(card, "BENCH_FAULT", "AFTER_RETUNE:FAIL") + before_failure = card_status(card) + card.send("BENCH_FOCUS", "43:500:8") + failed = wait_for( + card, + lambda status: status.get("FS") == "6" + and int(status.get("FW", "0")) > int(before_failure["FW"]), + 10.0, + "failed Focus durable row", + ) + failure_result = parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "HEALTH")) + if failure_result.get("RS") != "3" or failure_result.get("HR") != "1" or failure_result.get("E") == "0": + raise RuntimeError(f"failure did not retain operation error plus restore: {failure_result}") + record(results, "focus_failure_restores", status=failed, result=failure_result) + require_ack(card, "BENCH_FAULT", "CLEAR") + + card.send("PROBE_START", "-") + time.sleep(0.35) + expect_error(card, "BENCH_FOCUS", "43:500:8", "UNAVAILABLE") + card.send("PROBE_CANCEL", "-") + probe_terminal = wait_for( + card, lambda status: status.get("B") in {"CANCELLED", "COMPLETE", "FAILED"}, + 12.0, "Probe terminal", + ) + record(results, "probe_refuses_focus", status=probe_terminal) + + card.send("SWEEP_START", "-") + time.sleep(0.35) + expect_error(card, "BENCH_FOCUS", "43:500:8", "UNAVAILABLE") + card.send("SWEEP_CANCEL", "-") + sweep_terminal = wait_for( + card, lambda status: status.get("W") in {"CANCELLED", "COMPLETE", "FAILED"}, + 12.0, "Sweep terminal", + ) + record(results, "sweep_refuses_focus", status=sweep_terminal) + + # Timeout: a stalled sample loop must stop at its own deadline, + # restore Watch, and publish `timeout` -- not `complete` borrowed + # from a restore that worked, and not an unbounded radio-away hold. + before_timeout = card_status(card) + card.send("BENCH_FOCUS", FOCUS_TIMEOUT_ARGUMENT) + timed_out = wait_for( + card, + lambda status: status.get("FS") == "5" + and int(status.get("FW", "0")) > int(before_timeout["FW"]), + 15.0, + "timed-out Focus durable row", + ) + timeout_result = parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "HEALTH")) + if timeout_result.get("RS") != "2" or timeout_result.get("HR") != "1": + raise RuntimeError(f"timeout did not restore home: {timeout_result}") + away_ms = int(timed_out.get("FA", "0")) + if not FOCUS_TIMEOUT_STALL_MS <= away_ms <= FOCUS_TIMEOUT_AWAY_LIMIT_MS: + raise RuntimeError(f"timeout radio-away time was not bounded: {away_ms}ms") + record(results, "focus_timeout_restores", status=timed_out, + result=timeout_result, away_ms=away_ms) + + check_focus_refuses(card, "CELL", "CELL", results, "focus_refuses_cell") + check_refuses_focus(card, "CELL", "CELL", results, "cell_refuses_focus") + check_focus_refuses(card, "SCOPE", "SCOPE", results, "focus_refuses_scope") + check_refuses_focus(card, "SCOPE", "SCOPE", results, "scope_refuses_focus") + + print("Focus cancel/failure/timeout and Probe/Sweep/Cell/Scope " + "mutual-exclusion checks passed.") + finally: + # A failure cannot leave the one-shot injector armed for later work. + try: + require_ack(card, "BENCH_FAULT", "CLEAR", timeout=5.0) + finally: + card.close() + if results is not None: + results.close() + + +if __name__ == "__main__": + try: + main() + except (OSError, serial.SerialException, RuntimeError, TimeoutError, ValueError) as error: + print(f"phase12 Focus behavior bench: {error}", file=sys.stderr) + sys.exit(2) diff --git a/scripts/phase12_focus_bench.py b/scripts/phase12_focus_bench.py new file mode 100644 index 0000000..deaa936 --- /dev/null +++ b/scripts/phase12_focus_bench.py @@ -0,0 +1,136 @@ +#!/usr/bin/env python3 +"""Measure one bounded Focus bin quietly, then against one controlled pulse. + +This is Phase 12 engineering evidence, not a coverage-calibration tool. It +uses the shared framed transport, leaves the Heltec quiet on every exit path, +and requires explicit --allow-transmit before it can arm a pulse. Results +contain only the GPS-free compact Focus summary; `focus.csv` is the durable +device-side evidence. +""" + +import argparse +import pathlib +import sys + +import serial + +from bench_harness import (CARD_MARKER, TX_MARKER, Endpoint, ResultWriter, + card_status, parse_fields, require_ack, wait_for) + + +TX_CANDIDATE = "LONG_MODERATE" # 912.8125 MHz, SF11/BW125/CR4/8, sync 0x2B + + +def request_focus(card, bin_index, dwell_ms, samples, prior_status): + """Request one Focus pass and require its SD commit before readback.""" + before_written = int(prior_status.get("FW", "0")) + before_enqueued = int(prior_status.get("FO", "0")) + require_ack(card, "BENCH_FOCUS", f"{bin_index}:{dwell_ms}:{samples}") + terminal = wait_for( + card, + lambda status: status.get("FS") == "3" + and int(status.get("FO", "0")) > before_enqueued + and int(status.get("FW", "0")) > before_written, + 12.0, + "Focus completion, restore, and focus.csv commit", + ) + result = parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "-")) + if result.get("RS") != "0" or result.get("HR") != "1": + raise RuntimeError(f"Focus did not complete with home restore: {result}") + if int(result.get("N", "0")) != samples: + raise RuntimeError(f"Focus sample count mismatch: expected {samples}, got {result}") + return terminal, result + + +def record(writer, event, status, result, **extra): + if writer is not None: + writer.write({"event": event, "status": status, "result": result, **extra}) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--cardputer-port", required=True) + parser.add_argument("--heltec-port", required=True) + parser.add_argument("--log", required=True, help="append-only raw framed-control log") + parser.add_argument("--results", help="append-only JSONL summary path") + parser.add_argument("--bin", type=int, default=43, + help="US 250 kHz bin (default 43 = 912.750 MHz)") + parser.add_argument("--dwell-ms", type=int, default=500) + parser.add_argument("--samples", type=int, default=8) + parser.add_argument("--with-pulse", action="store_true", help="also measure one Heltec pulse") + parser.add_argument("--allow-transmit", action="store_true", help="required with --with-pulse") + parser.add_argument("--arm-delay-ms", type=int, default=150) + args = parser.parse_args() + if args.with_pulse and not args.allow_transmit: + parser.error("--with-pulse requires explicit --allow-transmit") + if not 0 <= args.bin <= 65535: + parser.error("--bin must fit uint16") + if not 2 <= args.dwell_ms <= 2000: + parser.error("--dwell-ms must be 2..2000") + if not 2 <= args.samples <= 64: + parser.error("--samples must be 2..64") + if not 10 <= args.arm_delay_ms <= 5000: + parser.error("--arm-delay-ms must be 10..5000") + + log_path = pathlib.Path(args.log) + log_path.parent.mkdir(parents=True, exist_ok=True) + results = ResultWriter(args.results) if args.results else None + with log_path.open("a", encoding="utf-8") as log: + card = Endpoint("cardputer", args.cardputer_port, CARD_MARKER, log) + transmitter = None + try: + require_ack(card, "HELLO", "-", timeout=45.0) + transmitter = Endpoint("heltec", args.heltec_port, TX_MARKER, log) + require_ack(transmitter, "HELLO", "-") + require_ack(transmitter, "QUIET", "-") + + quiet_status, quiet = request_focus( + card, args.bin, args.dwell_ms, args.samples, card_status(card) + ) + print(f"quiet: {quiet}") + record(results, "quiet", quiet_status, quiet) + if not args.with_pulse: + return + + require_ack(transmitter, "CONFIG", TX_CANDIDATE) + on_before = card_status(card) + require_ack(card, "BENCH_FOCUS", f"{args.bin}:{args.dwell_ms}:{args.samples}") + require_ack(transmitter, "ARM", str(args.arm_delay_ms)) + on_status = wait_for( + card, + lambda status: status.get("FS") == "3" + and int(status.get("FO", "0")) > int(on_before.get("FO", "0")) + and int(status.get("FW", "0")) > int(on_before.get("FW", "0")), + 12.0, + "pulsed Focus completion, restore, and focus.csv commit", + ) + on = parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "-")) + if on.get("RS") != "0" or on.get("HR") != "1": + raise RuntimeError(f"pulsed Focus did not restore: {on}") + if int(on.get("N", "0")) != args.samples: + raise RuntimeError(f"pulsed Focus sample count mismatch: {on}") + transmitter.request("STATUS", "-") # Drains TX_STARTED/TX_DONE evidence. + events = {opcode for _, opcode, _ in transmitter.observed} + if not {"TX_STARTED", "TX_DONE"}.issubset(events): + raise RuntimeError(f"transmitter pulse incomplete: {sorted(events)}") + delta = {field: int(on[field]) - int(quiet[field]) for field in ("MED", "P90", "MAX")} + print(f"pulse: {on}; delta_tenths_db={delta}") + record(results, "pulse", on_status, on, delta_tenths_db=delta, + tx_candidate=TX_CANDIDATE) + finally: + if transmitter is not None: + try: + require_ack(transmitter, "QUIET", "-", timeout=5.0) + finally: + transmitter.close() + card.close() + if results is not None: + results.close() + + +if __name__ == "__main__": + try: + main() + except (OSError, serial.SerialException, RuntimeError, TimeoutError, ValueError) as error: + print(f"phase12 Focus bench: {error}", file=sys.stderr) + sys.exit(2) diff --git a/scripts/phase12_focus_matrix.py b/scripts/phase12_focus_matrix.py new file mode 100644 index 0000000..811d9b5 --- /dev/null +++ b/scripts/phase12_focus_matrix.py @@ -0,0 +1,472 @@ +#!/usr/bin/env python3 +"""Phase 12 §6.2 controlled dwell matrix: paired source-on/source-off Focus trials. + +Collects the evidence the coverage vocabulary needs and nothing more. Each +trial is one independently logged, bounded Focus request — the matrix gets its +30 trials from 30 durable requests, never from one long radio-away loop +(docs/research/phase12-survey-truth-design.md §6.2). It records raw RSSI +summaries per trial and selects no threshold; that is the report script's job, +run offline against this output. + +The transmitter is repo-owned (bench/heltec-v4r8-transmitter) and is left quiet +on every exit path. Arming it requires explicit --allow-transmit. + +Nothing here decides a coverage label, and an absent observation is recorded as +a measured no-observation at a stated frequency offset — never as "quiet". +""" + +import argparse +import pathlib +import random +import sys +import threading +import time + +import serial + +from bench_harness import (CARD_MARKER, TX_MARKER, Endpoint, ResultWriter, + card_status, parse_fields, require_ack, wait_for) + + +# US Sweep band (energy_plan.h): bin centers are lo + index * step. +US_BAND_LO_MHZ = 902.0 +BIN_STEP_MHZ = 0.25 + +# Sourced from bench/heltec-v4r8-transmitter/src/main.cpp's CANDIDATES table; +# no frequency is invented here. `bin` is the Sweep bin Focus is asked for, and +# `offset_khz` is how far the transmitter actually sits from that bin's center. +# +# That offset is a first-class result field, not a footnote: Focus tunes at the +# resolved home channel's bandwidth (125 kHz on this bench), so a position +# 125 kHz off center is outside the passband half-width and a weak or absent +# rise there measures the offset, not the receiver's sensitivity. The three +# §6.1 positions are kept because they are the sourced low/mid/high set; the +# two `-aligned` positions exist so an arm can be run without that confound. +POSITIONS = { + # §6.1's sourced low/mid/high set. + "low": {"candidate": "LONG_SLOW", "tx_mhz": 905.3125, "bin": 13}, + "mid": {"candidate": "LONG_MODERATE", "tx_mhz": 912.8125, "bin": 43}, + "high": {"candidate": "SHORT_SLOW", "tx_mhz": 920.625, "bin": 74}, + # Exactly bin-centered alternatives from the same candidate table. + "low-aligned": {"candidate": "LONG_TURBO", "tx_mhz": 908.750, "bin": 27}, + "mid-aligned": {"candidate": "MESH_OREGON", "tx_mhz": 918.500, "bin": 66}, +} + +DEFAULT_DWELLS_MS = (100, 500, 2000) +DEFAULT_POSITIONS = ("low", "mid", "high") + + +def bin_center_mhz(bin_index): + return US_BAND_LO_MHZ + bin_index * BIN_STEP_MHZ + + +def position_offset_khz(position): + spec = POSITIONS[position] + return round((spec["tx_mhz"] - bin_center_mhz(spec["bin"])) * 1000.0, 1) + + +class PulseBurst: + """Keeps the transmitter firing for as long as a trial's window is open. + + A single armed pulse cannot be scheduled reliably against a short dwell. + Candidate airtimes across this table span roughly 30 ms (SF8/BW250) to + ~1 s (SF12/BW125), and the receiver's window opens tens of ms after the + request is accepted — so one pulse at a fixed delay lands inside a 2,000 ms + dwell and misses a 100 ms one entirely. That is exactly what the first + attempt at this matrix measured: at 100 ms, source-on and source-off were + indistinguishable because the pulse started after the window had closed. + + Firing continuously for the trial's duration makes "source on" mean the + transmitter was actually radiating while Focus was listening, which is the + condition §3.1's qualifying-RSSI question needs. It deliberately does NOT + measure the catch probability of intermittent traffic — §1 already records + that the dwell-versus-real-traffic comparison was inconclusive, and this + matrix is not a second attempt at it. + """ + + def __init__(self, transmitter, gap_s): + self.transmitter = transmitter + self.gap_s = gap_s + self._stop = threading.Event() + self._thread = None + self.fired = 0 + self.failed = 0 + + def _await_tx_done(self, timeout_s=4.0): + """Block until the in-flight transmission reports TX_DONE. + + Paced by the transmitter's own completion rather than a fixed cadence. + A fixed interval shorter than the airtime queues overlapping sends — + measured airtime here ranges from ~10 ms (SF8/BW250) to ~275 ms + (SF12/BW125) — and the tail then bleeds into the next trial. The first + burst attempt did exactly that: source-off trials inherited a + transmission and read -26 dBm, identical to source-on. + """ + deadline = time.monotonic() + timeout_s + while time.monotonic() < deadline: + if any(opcode == "TX_DONE" for _, opcode, _ in self.transmitter.observed): + return True + # STATUS is idempotent; the round trip is what drains the + # asynchronous TX_STARTED/TX_DONE frames into `observed`. + try: + self.transmitter.request("STATUS", "-", timeout=1.0) + except (RuntimeError, TimeoutError, OSError): + pass + return False + + def _run(self): + while not self._stop.is_set(): + try: + self.transmitter.observed.clear() + require_ack(self.transmitter, "ARM", "0", timeout=1.5) + self.fired += 1 + self._await_tx_done() + except (RuntimeError, TimeoutError, OSError): + self.failed += 1 + self._stop.wait(self.gap_s) + + def __enter__(self): + self._thread = threading.Thread(target=self._run, daemon=True) + self._thread.start() + return self + + def __exit__(self, *exc): + self._stop.set() + if self._thread is not None: + self._thread.join(timeout=8.0) + # The trial is over, but a transmission may still be in the air. A + # source-off trial that inherits it is not a source-off trial. + self._await_tx_done() + return False + + +def counters(status): + """The health counters a trial must not disturb.""" + return {field: int(status.get(field, "0")) for field in ("FO", "FW", "FD", "FL")} + + +def run_focus_request(card, spec, dwell_ms, samples, start): + # The long timeout is transport, not RF: the device's native USB CDC + # occasionally truncates an outbound frame under sustained round trips, and + # require_ack re-sends once a second within its own timeout. The command is + # idempotent on the device (one cached response per sequence), so a re-send + # cannot start a second survey. + require_ack(card, "BENCH_FOCUS", f"{spec['bin']}:{dwell_ms}:{samples}", timeout=12.0) + terminal = wait_for( + card, + lambda status: status.get("FS") == "3" + and int(status.get("FO", "0")) > start["FO"] + and int(status.get("FW", "0")) > start["FW"], + max(12.0, dwell_ms / 1000.0 + 10.0), + "Focus completion, home restore, and focus.csv commit", + ) + result = parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "-", timeout=12.0)) + # Counts above the pass's own median, laddered, so any adaptive activity + # rule can be evaluated offline from one run. Optional: a bench image + # without it simply reports nothing and the trial row omits the counts. + counts = {} + try: + opcode, payload = card.request("BENCH_FOCUS_COUNTS", "-", timeout=12.0) + if opcode == "ACK": + counts = parse_fields(payload) + except (RuntimeError, TimeoutError): + pass + return terminal, result, counts + + +def run_trial(card, transmitter, position, dwell_ms, samples, source_on, gap_ms, + single_pulse_delay_ms=None): + """One bounded Focus request, with the transmitter radiating during it or not at all. + + Two source modes. The burst keeps the transmitter radiating for the whole + window, which is what the qualifying-condition work needed. `single_pulse` + arms exactly one transmission placed inside the window instead, which is + the only way to ask whether a *single packet* is detectable: a burst + answers a different question no matter how its duty is tuned. A single + pulse can only be placed reliably when the window is long relative to the + timing jitter, so this mode is for long dwells, not for 100 ms passes. + """ + spec = POSITIONS[position] + before = card_status(card) + start = counters(before) + + pulses = 0 + if source_on and single_pulse_delay_ms is not None: + transmitter.observed.clear() + require_ack(card, "BENCH_FOCUS", f"{spec['bin']}:{dwell_ms}:{samples}", timeout=12.0) + require_ack(transmitter, "ARM", str(single_pulse_delay_ms), timeout=5.0) + terminal = wait_for( + card, + lambda status: status.get("FS") == "3" + and int(status.get("FO", "0")) > start["FO"] + and int(status.get("FW", "0")) > start["FW"], + max(12.0, dwell_ms / 1000.0 + 10.0), + "Focus completion, home restore, and focus.csv commit", + ) + result = parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "-", timeout=12.0)) + counts = {} + try: + opcode, payload = card.request("BENCH_FOCUS_COUNTS", "-", timeout=12.0) + if opcode == "ACK": + counts = parse_fields(payload) + except (RuntimeError, TimeoutError): + pass + # The pulse must actually have happened, or a miss is the fixture's + # fault rather than the receiver's. + transmitter.request("STATUS", "-", timeout=5.0) + fired = {op for _, op, _ in transmitter.observed} & {"TX_STARTED", "TX_DONE"} + if fired != {"TX_STARTED", "TX_DONE"}: + raise RuntimeError(f"single pulse incomplete: {sorted(fired)}") + transmitter.observed.clear() + pulses = 1 + elif source_on: + with PulseBurst(transmitter, gap_ms / 1000.0) as burst: + terminal, result, counts = run_focus_request(card, spec, dwell_ms, samples, start) + pulses = burst.fired + if burst.fired == 0: + raise RuntimeError("the transmitter never fired during a source-on trial") + else: + terminal, result, counts = run_focus_request(card, spec, dwell_ms, samples, start) + + # §6.2: an arm is rejected if it loses ownership, cannot restore home, or + # has unexplained queue/row drops. Fail the trial, not the whole matrix's + # credibility after the fact. + if result.get("RS") != "0" or result.get("HR") != "1": + raise RuntimeError(f"trial did not complete with home restore: {result}") + if int(result.get("N", "0")) != samples: + raise RuntimeError(f"trial sample count mismatch: expected {samples}, got {result}") + end = counters(terminal) + if end["FD"] != start["FD"] or end["FL"] != start["FL"]: + raise RuntimeError(f"trial dropped a queued or durable row: {start} -> {end}") + + if source_on: + transmitter.observed.clear() + + return { + "position": position, + "candidate": spec["candidate"], + "tx_mhz": spec["tx_mhz"], + "bin": spec["bin"], + "bin_center_mhz": round(bin_center_mhz(spec["bin"]), 4), + "offset_khz": position_offset_khz(position), + "dwell_ms": dwell_ms, + "samples": samples, + "source_on": source_on, + # Tenths of a dBm, exactly as the device reported them. + "median_dbm_x10": int(result["MED"]), + "p90_dbm_x10": int(result["P90"]), + "peak_dbm_x10": int(result["MAX"]), + "sample_count": int(result["N"]), + "observation_ms": int(result["OBS"]), + "focus_id": int(result["ID"]), + "radio_away_ms": int(terminal.get("FA", "0")), + "request_status": result["RS"], + "home_restore": result["HR"], + "radio_status": int(result["E"]), + "pulses_fired": pulses, + # Recorded because it sets the source's duty cycle within the window, + # which is the independent variable of an occupancy sweep. + "pulse_gap_ms": gap_ms, + "single_pulse_delay_ms": single_pulse_delay_ms, + # C = samples at or above the pass's median + N dB. + "counts_above_median": {k: int(v) for k, v in counts.items() if k.startswith("C")}, + } + + +def settle(card, timeout_s=20.0): + """Wait until no Focus owns the radio, so a retried trial isn't refused. + + A trial can fail with its request already accepted (the ACK truncated in + transit). Starting the next one immediately would be refused as + UNAVAILABLE and look like an arbitration bug rather than a transport one. + """ + deadline = time.monotonic() + timeout_s + while time.monotonic() < deadline: + try: + state = card_status(card).get("FS") + except (RuntimeError, TimeoutError): + time.sleep(0.5) + continue + # 1 = SURVEYING, 2 = RESTORING (focus_runtime.h). + if state not in {"1", "2"}: + time.sleep(0.3) + return True + time.sleep(0.3) + return False + + +def run_trial_with_retry(card, transmitter, position, dwell_ms, samples, source_on, + gap_ms, results, attempts, single_pulse_delay_ms=None): + """A transport failure is a harness event, not an RF result — retry it, loudly. + + The retry is a genuinely new bounded request, and the failure is recorded + rather than swallowed: a run whose trials silently retried would overstate + how cleanly the transport behaved. + """ + for attempt in range(1, attempts + 1): + try: + row = run_trial(card, transmitter, position, dwell_ms, samples, + source_on, gap_ms, single_pulse_delay_ms) + if attempt > 1: + row["retried_attempts"] = attempt + return row + except (RuntimeError, TimeoutError) as error: + results.write({"event": "trial_error", "position": position, + "dwell_ms": dwell_ms, "source_on": source_on, + "attempt": attempt, "error": str(error)}) + print(f" trial attempt {attempt}/{attempts} failed: {error}", flush=True) + if attempt == attempts: + raise + if not settle(card): + raise RuntimeError("Focus never returned to idle after a failed trial") + if transmitter is not None: + try: + require_ack(transmitter, "QUIET", "-", timeout=5.0) + transmitter.observed.clear() + except (RuntimeError, TimeoutError): + pass + + +def trial_order(trials, source_order, rng): + """Alternating by default; --order random shuffles within the arm (§6.2).""" + plan = [True, False] * trials + if source_order == "random": + rng.shuffle(plan) + return plan + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--cardputer-port", required=True) + parser.add_argument("--heltec-port", required=True) + parser.add_argument("--log", required=True, help="append-only raw framed-control log") + parser.add_argument("--results", required=True, help="append-only JSONL trial rows") + parser.add_argument("--position", action="append", choices=sorted(POSITIONS), + help=f"repeatable; default {' '.join(DEFAULT_POSITIONS)}") + parser.add_argument("--dwell-ms", action="append", type=int, + help=f"repeatable; default {' '.join(str(d) for d in DEFAULT_DWELLS_MS)}") + parser.add_argument("--trials", type=int, default=30, + help="trials per source state per arm (§6.2 requires at least 30)") + parser.add_argument("--samples", action="append", type=int, + help="repeatable; sample count per pass (spacing is dwell/(n-1))") + parser.add_argument("--expect-home-khz", type=int, default=None, + help="refuse to run unless the resolved home channel matches") + parser.add_argument("--order", choices=("alternating", "random"), default="alternating") + parser.add_argument("--seed", type=int, default=20260904, help="--order random seed") + parser.add_argument("--pulse-gap-ms", type=int, default=30, + help="idle gap between completed transmissions during a source-on " + "trial; the burst is paced by TX_DONE, not by this alone") + parser.add_argument("--single-pulse-delay-ms", type=int, default=None, + help="arm exactly one pulse this many ms after the request is " + "accepted, instead of a burst; for long dwells only") + parser.add_argument("--attempts", type=int, default=3, + help="attempts per trial before the run gives up (transport retries)") + parser.add_argument("--allow-transmit", action="store_true", + help="required: this matrix arms the controlled transmitter") + args = parser.parse_args() + + if not args.allow_transmit: + parser.error("this matrix arms the transmitter; pass --allow-transmit") + positions = args.position or list(DEFAULT_POSITIONS) + dwells = args.dwell_ms or list(DEFAULT_DWELLS_MS) + for dwell in dwells: + if not 2 <= dwell <= 2000: + parser.error(f"--dwell-ms {dwell} outside the bounded 2..2000 request range") + sample_counts = args.samples or [8] + for count in sample_counts: + if not 2 <= count <= 208: + parser.error(f"--samples {count} outside focus_plan.h's bench range 2..208") + if args.trials < 1: + parser.error("--trials must be positive") + if args.trials < 30: + print(f"warning: {args.trials} trials per state is below §6.2's 30; " + "the result is a smoke check, not matrix evidence", file=sys.stderr) + + rng = random.Random(args.seed) + log_path = pathlib.Path(args.log) + log_path.parent.mkdir(parents=True, exist_ok=True) + results = ResultWriter(args.results) + completed = 0 + started = time.monotonic() + + with log_path.open("a", encoding="utf-8") as log: + card = Endpoint("cardputer", args.cardputer_port, CARD_MARKER, log) + transmitter = None + try: + # Generous: opening native USB-CDC resets the board, but *when* + # that reset lands varies, and the harness's fixed settle can + # expire before boot even starts. HELLO is idempotent and retried + # once a second, so a wide window costs nothing on a healthy boot. + identity = require_ack(card, "HELLO", "-", timeout=45.0) + # SD readiness lags HELLO: the logger task mounts the card and opens + # its logs several seconds into boot, so polling once can read SD=0 + # on a perfectly healthy device (and a boot-time status also predates + # config.txt's channel override). + boot = wait_for(card, lambda status: status.get("SD") == "1", 30.0, + "SD to mount for durable focus.csv rows") + # A boot that missed the card also missed config.txt's channel + # override and silently resolves a different home channel -- and + # Focus tunes at the *home* bandwidth, so trials would land at a + # different bandwidth than the run they are being compared with, + # with nothing in the output to show it. Observed 2026-09-04: + # a no-SD boot came up SF11/BW250 instead of SF8/BW125. + if args.expect_home_khz and int(boot.get("F", "0")) != args.expect_home_khz: + raise RuntimeError( + f"home channel is {boot.get('F')} kHz, expected {args.expect_home_khz}; " + "this boot resolved a different channel (check config.txt applied)") + transmitter = Endpoint("heltec", args.heltec_port, TX_MARKER, log) + require_ack(transmitter, "HELLO", "-") + require_ack(transmitter, "QUIET", "-") + results.write({"event": "boot", "identity": identity, "status": boot, + "positions": positions, "dwells_ms": dwells, + "trials_per_state": args.trials, "samples": sample_counts, + "order": args.order, "seed": args.seed}) + + for position in positions: + require_ack(transmitter, "CONFIG", POSITIONS[position]["candidate"]) + for dwell in dwells: + for samples in sample_counts: + plan = trial_order(args.trials, args.order, rng) + spacing = dwell / (samples - 1) + print(f"[{position} {dwell}ms x{samples} samples] {len(plan)} trials " + f"(offset {position_offset_khz(position):+.1f} kHz, " + f"{spacing:.0f}ms spacing)", flush=True) + for index, source_on in enumerate(plan): + row = run_trial_with_retry(card, transmitter, position, dwell, + samples, source_on, + args.pulse_gap_ms, results, + args.attempts, args.single_pulse_delay_ms) + row.update({"event": "trial", "arm_index": index}) + results.write(row) + completed += 1 + state = "on " if source_on else "off" + print(f" [{index + 1:3d}/{len(plan)}] source={state} " + f"p90={row['p90_dbm_x10'] / 10:.1f}dBm " + f"peak={row['peak_dbm_x10'] / 10:.1f}dBm " + f"away={row['radio_away_ms']}ms", flush=True) + # Let the device's USB CDC buffer drain between trials. + time.sleep(0.15) + elapsed = time.monotonic() - started + print(f"{completed} trials completed in {elapsed / 60:.1f} min. " + "Raw rows only — run phase12_focus_matrix_report.py to analyze.") + finally: + # A transmitter left armed after a failure is the one outcome this + # script must never produce. + if transmitter is not None: + try: + require_ack(transmitter, "QUIET", "-", timeout=5.0) + finally: + transmitter.close() + card.close() + results.close() + + +if __name__ == "__main__": + try: + main() + except KeyboardInterrupt: + print("interrupted; partial rows are already durable", file=sys.stderr) + sys.exit(130) + except (OSError, serial.SerialException, RuntimeError, TimeoutError, ValueError) as error: + print(f"phase12 Focus matrix: {error}", file=sys.stderr) + sys.exit(2) diff --git a/scripts/phase12_focus_matrix_report.py b/scripts/phase12_focus_matrix_report.py new file mode 100644 index 0000000..3c02daf --- /dev/null +++ b/scripts/phase12_focus_matrix_report.py @@ -0,0 +1,163 @@ +#!/usr/bin/env python3 +"""Offline, deterministic report over phase12_focus_matrix.py trial rows. + +Reads the JSONL the matrix runner wrote and answers one question per arm: does +a fixed RSSI condition separate source-on from source-off trials, and with what +confidence? It reports proportions as Wilson 95% intervals rather than bare +percentages (docs/research/phase12-survey-truth-design.md §6.2 item 5, NIST +1.3.5 guidance) because a "30/30" at these counts is not the certainty it looks +like. + +It touches no device and mutates no input. Given the same file it prints the +same report, so a result can be re-derived rather than remembered. + +What it deliberately does NOT do: name a coverage label, call an arm's +source-off trials "quiet", or pick thresholds the operator has not accepted. A +separating threshold printed here is a *candidate* for §3.1, and it is only +meaningful for the frequency offset its arm actually ran at. +""" + +import argparse +import collections +import json +import math +import pathlib +import statistics +import sys + + +def wilson_interval(successes, total, z=1.959963984540054): + """Two-sided Wilson score interval; correct at 0/n and n/n where normal isn't.""" + if total == 0: + return (0.0, 1.0) + p = successes / total + denom = 1.0 + z * z / total + center = (p + z * z / (2 * total)) / denom + margin = z * math.sqrt(p * (1 - p) / total + z * z / (4 * total * total)) / denom + return (max(0.0, center - margin), min(1.0, center + margin)) + + +def load_trials(path): + trials, meta = [], None + with path.open(encoding="utf-8") as handle: + for line_number, line in enumerate(handle, 1): + line = line.strip() + if not line: + continue + try: + row = json.loads(line) + except json.JSONDecodeError as error: + raise SystemExit(f"{path}:{line_number}: {error}") + if row.get("event") == "boot" and meta is None: + meta = row + elif row.get("event") == "trial": + trials.append(row) + return trials, meta + + +def best_threshold(on_values, off_values): + """Lowest 1 dB threshold that admits every source-on trial and no source-off one. + + Returns (threshold_dbm_x10, margin_db) or None when the two sets overlap — + and an overlap is a real answer: it says this arm cannot support a fixed + RSSI condition, which is exactly what §3.1 needs to know before choosing + one. + """ + if not on_values or not off_values: + return None + if min(on_values) <= max(off_values): + return None + # Any cut in the gap separates; take the midpoint, rounded to whole dB. + low, high = max(off_values), min(on_values) + threshold = int(round((low + high) / 2.0 / 10.0)) * 10 + threshold = max(low + 1, min(high, threshold)) + return threshold, (high - low) / 10.0 + + +def summarize(values): + if not values: + return "n/a" + return (f"min {min(values) / 10:7.1f} med {statistics.median(values) / 10:7.1f} " + f"max {max(values) / 10:7.1f}") + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("results", type=pathlib.Path, help="JSONL from phase12_focus_matrix.py") + parser.add_argument("--metric", choices=("p90", "peak", "median"), default="p90", + help="RSSI summary the candidate condition is built on") + args = parser.parse_args() + + if not args.results.exists(): + raise SystemExit(f"no such results file: {args.results}") + trials, meta = load_trials(args.results) + if not trials: + raise SystemExit(f"{args.results} contains no trial rows") + + field = {"p90": "p90_dbm_x10", "peak": "peak_dbm_x10", "median": "median_dbm_x10"}[args.metric] + + if meta: + print(f"build : {meta.get('identity', 'unknown')}") + print(f"plan : positions={meta.get('positions')} dwells_ms={meta.get('dwells_ms')} " + f"trials/state={meta.get('trials_per_state')} order={meta.get('order')}") + print(f"metric : {args.metric} (dBm)") + print(f"trial rows: {len(trials)}\n") + + arms = collections.defaultdict(lambda: {"on": [], "off": [], "away": [], "rows": []}) + for row in trials: + arm = arms[(row["position"], row["dwell_ms"])] + arm["on" if row["source_on"] else "off"].append(row[field]) + arm["away"].append(row["radio_away_ms"]) + arm["rows"].append(row) + + unresolved = [] + for (position, dwell), arm in sorted(arms.items(), key=lambda item: (item[0][0], item[0][1])): + sample = arm["rows"][0] + on, off = arm["on"], arm["off"] + print(f"=== {position} @ {dwell} ms " + f"(tx {sample['tx_mhz']:.4f} MHz -> bin {sample['bin']} " + f"center {sample['bin_center_mhz']:.3f}, offset {sample['offset_khz']:+.1f} kHz)") + print(f" source-on n={len(on):3d} {summarize(on)}") + print(f" source-off n={len(off):3d} {summarize(off)}") + print(f" radio-away min {min(arm['away'])} ms, max {max(arm['away'])} ms") + + selection = best_threshold(on, off) + if selection is None: + overlap = "no trials" if not on or not off else ( + f"source-on min {min(on) / 10:.1f} <= source-off max {max(off) / 10:.1f}") + print(f" candidate condition: NONE — the two sets overlap ({overlap}).") + print(" This arm does not support a fixed RSSI condition; it is not evidence " + "that the frequency was quiet.") + unresolved.append((position, dwell)) + else: + threshold, margin = selection + hits_on = sum(1 for value in on if value >= threshold) + hits_off = sum(1 for value in off if value >= threshold) + lo_on, hi_on = wilson_interval(hits_on, len(on)) + lo_off, hi_off = wilson_interval(hits_off, len(off)) + print(f" candidate condition: {args.metric} >= {threshold / 10:.1f} dBm " + f"(separating margin {margin:.1f} dB)") + print(f" source-on detected {hits_on}/{len(on)} " + f"95% CI [{lo_on:.3f}, {hi_on:.3f}]") + print(f" source-off detected {hits_off}/{len(off)} " + f"95% CI [{lo_off:.3f}, {hi_off:.3f}]") + print() + + print("Interpretation limits:") + print(" - A separating threshold is a candidate for §3.1, not an accepted constant,") + print(" and it is valid only at the frequency offset its arm ran at.") + print(" - Source-off trials are ambient observation on this bench, not a calibrated") + print(" false-hit rate (§6.1): the control path is not known quiet.") + print(" - No arm here licenses a coverage label or a 'no activity' conclusion.") + if unresolved: + print(f" - {len(unresolved)} arm(s) did not separate: " + + ", ".join(f"{p}@{d}ms" for p, d in unresolved)) + return 0 + + +if __name__ == "__main__": + try: + sys.exit(main()) + except (OSError, ValueError, KeyError) as error: + print(f"phase12 Focus matrix report: {error}", file=sys.stderr) + sys.exit(2) diff --git a/scripts/phase12_watch_opportunity.py b/scripts/phase12_watch_opportunity.py new file mode 100644 index 0000000..a772939 --- /dev/null +++ b/scripts/phase12_watch_opportunity.py @@ -0,0 +1,260 @@ +#!/usr/bin/env python3 +"""Phase 12 §6.3: what does interleaving Focus cost Watch packet opportunity? + +Two arms against an identical, independently timed pulse train on the home +channel: Watch alone, then Watch with repeated bounded Focus requests. Reports +received/reference for each as a Wilson 95% interval, plus Focus away time, +completed requests, and the gaps between restored Watch windows. + +This measures a cost. It does not decide whether that cost is acceptable — +that is an operator product decision after the measurement, and a displayed +away-time does not make the loss free (docs/research/phase12-survey-truth-design.md +§6.3). + +The transmitter must be configured to a candidate that matches the receiver's +resolved home channel, or Watch cannot hear the reference train at all and +both arms measure nothing. The script checks the frequency and refuses +otherwise. Requires explicit --allow-transmit; the transmitter is quieted on +every exit path. +""" + +import argparse +import math +import pathlib +import statistics +import sys +import threading +import time + +import serial + +from bench_harness import (CARD_MARKER, TX_MARKER, Endpoint, ResultWriter, + card_status, parse_fields, require_ack) + + +# bench/heltec-v4r8-transmitter/src/main.cpp: 918.5 MHz, SF8, BW125, CR4/5, +# sync 0x2B — the one candidate that matches this bench's resolved home +# channel, which is what makes Watch able to receive the reference train. +HOME_CANDIDATE = "MESH_OREGON" +HOME_CANDIDATE_MHZ = 918.5 + + +def wilson_interval(successes, total, z=1.959963984540054): + if total == 0: + return (0.0, 1.0) + p = successes / total + denom = 1.0 + z * z / total + center = (p + z * z / (2 * total)) / denom + margin = z * math.sqrt(p * (1 - p) / total + z * z / (4 * total * total)) / denom + return (max(0.0, center - margin), min(1.0, center + margin)) + + +class PulseTrain: + """Fires the reference train on its own schedule, independent of Focus. + + The transmitter is the timing source (§6.1): if the pulse cadence were + driven by the receiver's request loop, an arm that spent longer away would + also be sent fewer packets, and the comparison would measure the harness. + """ + + def __init__(self, transmitter, interval_s): + self.transmitter = transmitter + self.interval_s = interval_s + self._stop = threading.Event() + self._thread = None + self.sent = 0 + self.failed = 0 + + def _run(self): + next_fire = time.monotonic() + while not self._stop.is_set(): + now = time.monotonic() + if now < next_fire: + self._stop.wait(min(0.05, next_fire - now)) + continue + next_fire += self.interval_s + try: + require_ack(self.transmitter, "ARM", "0", timeout=1.5) + self.sent += 1 + except (RuntimeError, TimeoutError, OSError): + # One dropped ARM must not end the train; it is counted so the + # reference total stays honest. + self.failed += 1 + + def __enter__(self): + self._thread = threading.Thread(target=self._run, daemon=True) + self._thread.start() + return self + + def __exit__(self, *exc): + self._stop.set() + if self._thread is not None: + self._thread.join(timeout=5.0) + return False + + +def run_arm(card, results, label, duration_s, train, focus_request): + """One timed arm. `focus_request` is None for the Watch-only baseline.""" + before = card_status(card) + start_rxp = int(before.get("RXP", "0")) + start_sent = train.sent + started = time.monotonic() + + focus_completed = 0 + focus_refused = 0 + away_ms_total = 0 + watch_gaps_s = [] + last_restore = started + + while time.monotonic() - started < duration_s: + if focus_request is None: + time.sleep(0.25) + continue + bin_index, dwell_ms, samples = focus_request + # Generous like the matrix runner's: native USB-CDC occasionally + # truncates a reply under sustained round trips, and the command is + # idempotent on the device, so re-sends cannot start a second survey. + opcode, payload = card.request("BENCH_FOCUS", f"{bin_index}:{dwell_ms}:{samples}", + timeout=12.0) + if opcode != "ACK": + focus_refused += 1 + continue + deadline = time.monotonic() + dwell_ms / 1000.0 + 10.0 + while time.monotonic() < deadline: + status = card_status(card) + if status.get("FS") == "3": + break + time.sleep(0.2) + else: + raise RuntimeError("a Focus request did not reach a terminal state in this arm") + result = parse_fields(require_ack(card, "BENCH_FOCUS_RESULT", "HEALTH", timeout=12.0)) + if result.get("HR") != "1": + raise RuntimeError(f"Focus failed to restore home listening: {result}") + focus_completed += 1 + away_ms_total += int(card_status(card).get("FA", "0")) + now = time.monotonic() + watch_gaps_s.append(now - last_restore) + last_restore = now + + elapsed = time.monotonic() - started + after = card_status(card) + received = int(after.get("RXP", "0")) - start_rxp + reference = train.sent - start_sent + low, high = wilson_interval(min(received, reference), reference) + + row = { + "event": "arm", + "arm": label, + "elapsed_s": round(elapsed, 2), + "reference_pulses": reference, + "received_packets": received, + "received_fraction": round(received / reference, 4) if reference else None, + "wilson95": [round(low, 4), round(high, 4)], + "focus_completed": focus_completed, + "focus_refused": focus_refused, + "focus_away_ms_total": away_ms_total, + "away_fraction": round(away_ms_total / (elapsed * 1000.0), 4) if elapsed else None, + "watch_gap_s_median": round(statistics.median(watch_gaps_s), 2) if watch_gaps_s else None, + "watch_gap_s_max": round(max(watch_gaps_s), 2) if watch_gaps_s else None, + "crc_errors": int(after.get("RXC", "0")) - int(before.get("RXC", "0")), + "train_failed_arms": train.failed, + } + results.write(row) + return row + + +def describe(row): + fraction = "n/a" if row["received_fraction"] is None else f"{row['received_fraction']:.3f}" + print(f" {row['arm']:<14s} received {row['received_packets']}/{row['reference_pulses']} " + f"= {fraction} 95% CI [{row['wilson95'][0]:.3f}, {row['wilson95'][1]:.3f}]") + if row["focus_completed"]: + print(f" Focus {row['focus_completed']} completed, " + f"{row['focus_refused']} refused, away {row['focus_away_ms_total']} ms " + f"({row['away_fraction']:.1%} of the arm)") + print(f" restored-Watch gaps: median {row['watch_gap_s_median']}s, " + f"max {row['watch_gap_s_max']}s") + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--cardputer-port", required=True) + parser.add_argument("--heltec-port", required=True) + parser.add_argument("--log", required=True) + parser.add_argument("--results", required=True) + parser.add_argument("--duration-s", type=int, default=180, help="per arm") + parser.add_argument("--pulse-interval-s", type=float, default=2.0) + parser.add_argument("--bin", type=int, default=43, help="Focus bin for the interleaved arm") + parser.add_argument("--dwell-ms", type=int, default=2000) + parser.add_argument("--samples", type=int, default=8) + parser.add_argument("--allow-transmit", action="store_true") + args = parser.parse_args() + + if not args.allow_transmit: + parser.error("this comparison arms the transmitter; pass --allow-transmit") + if args.pulse_interval_s < 0.5: + parser.error("--pulse-interval-s below 0.5 risks overlapping airtime at SF8") + if not 2 <= args.dwell_ms <= 2000: + parser.error("--dwell-ms must be 2..2000") + + log_path = pathlib.Path(args.log) + log_path.parent.mkdir(parents=True, exist_ok=True) + results = ResultWriter(args.results) + + with log_path.open("a", encoding="utf-8") as log: + card = Endpoint("cardputer", args.cardputer_port, CARD_MARKER, log) + transmitter = None + try: + identity = require_ack(card, "HELLO", "-", timeout=45.0) + status = card_status(card) + home_khz = int(status.get("F", "0")) + if abs(home_khz - HOME_CANDIDATE_MHZ * 1000) > 1: + raise RuntimeError( + f"home channel is {home_khz} kHz but the reference train is " + f"{HOME_CANDIDATE} at {HOME_CANDIDATE_MHZ} MHz; Watch could not hear it " + "and both arms would measure nothing") + transmitter = Endpoint("heltec", args.heltec_port, TX_MARKER, log) + require_ack(transmitter, "HELLO", "-") + require_ack(transmitter, "QUIET", "-") + require_ack(transmitter, "CONFIG", HOME_CANDIDATE) + results.write({"event": "boot", "identity": identity, "status": status, + "duration_s": args.duration_s, + "pulse_interval_s": args.pulse_interval_s, + "focus": {"bin": args.bin, "dwell_ms": args.dwell_ms, + "samples": args.samples}}) + + print(f"reference train: {HOME_CANDIDATE} every {args.pulse_interval_s}s, " + f"{args.duration_s}s per arm") + with PulseTrain(transmitter, args.pulse_interval_s) as train: + baseline = run_arm(card, results, "watch-only", args.duration_s, train, None) + describe(baseline) + focus = run_arm(card, results, "watch+focus", args.duration_s, train, + (args.bin, args.dwell_ms, args.samples)) + describe(focus) + + if baseline["received_fraction"] is not None and focus["received_fraction"] is not None: + delta = focus["received_fraction"] - baseline["received_fraction"] + print(f"\ndelta: {delta:+.3f} received fraction with Focus interleaved") + print("Read this against away_fraction, not alone: a small loss at a small") + print("away fraction says nothing about a larger one. Overlapping intervals") + print("mean the arms are not distinguishable at this trial count.") + print("\nThis is the measurement §6.3 asks for. Accepting a radio-away budget " + "is a separate operator decision.") + finally: + if transmitter is not None: + try: + require_ack(transmitter, "QUIET", "-", timeout=5.0) + finally: + transmitter.close() + card.close() + results.close() + + +if __name__ == "__main__": + try: + main() + except KeyboardInterrupt: + print("interrupted; completed arms are already durable", file=sys.stderr) + sys.exit(130) + except (OSError, serial.SerialException, RuntimeError, TimeoutError, ValueError) as error: + print(f"phase12 Watch opportunity: {error}", file=sys.stderr) + sys.exit(2) diff --git a/scripts/ws17_sweep_floor_settle.py b/scripts/ws17_sweep_floor_settle.py new file mode 100644 index 0000000..9b8f342 --- /dev/null +++ b/scripts/ws17_sweep_floor_settle.py @@ -0,0 +1,143 @@ +#!/usr/bin/env python3 +"""Workstream 17: how far does Pass A's settle time move its noise floor? + +Needs no transmitter. That is the point: the floor is a property of the +receiver and its retune, so measuring it silently removes every source-timing +problem that invalidated the first attempts at the detection question. + +Pass A retunes each bin with standby/setFrequency/startReceive and then samples +immediately -- `ENERGY_SWEEP_SETTLE_DEFAULT_MS` is 0. `src/version.h` records +an 11 dB under-read on Cell from exactly that, and leaves Sweep's case open. +A first probe put Sweep's silent floor 14 dB below a settled reading; this +establishes it properly and finds where the reading stabilises. + +It also measures two things any fix has to trade against each other: + + lap time a settle is paid once per bin, so 85 bins multiply it. This is + the cost side of "make Pass A settle" versus "recalibrate the + margin against under-read values". + bin 0 bin 0 takes a full begin(), later bins take the light retune. + If the under-read is a settle artifact, bin 0 should be + unaffected while the rest are not -- which localises the cause + rather than inferring it. + +Reports floors per bin per settle value. It selects nothing and changes no +firmware; the shipped default is restored on exit. +""" + +import argparse +import pathlib +import statistics +import sys +import time + +import serial + +from bench_harness import (CARD_MARKER, Endpoint, ResultWriter, card_status, + parse_fields, require_ack) + +SWEEP_TERMINAL_TIMEOUT_S = 120.0 +DEFAULT_SETTLES_MS = (0, 1, 2, 3, 5, 10, 20, 40) +DEFAULT_BINS = (0, 20, 40, 66, 84) + + +def run_lap(card): + """One Pass A lap; returns its wall-clock duration in seconds.""" + started = time.monotonic() + require_ack(card, "SWEEP_START", "-", timeout=12.0) + deadline = time.monotonic() + SWEEP_TERMINAL_TIMEOUT_S + while time.monotonic() < deadline: + state = card_status(card).get("W") + if state in {"COMPLETE", "CANCELLED", "FAILED"}: + if state != "COMPLETE": + raise RuntimeError(f"sweep ended {state}") + return time.monotonic() - started + time.sleep(0.05) + raise TimeoutError("sweep did not reach a terminal state") + + +def read_bin(card, index): + opcode, payload = card.request("BENCH_SWEEP_FLOOR", str(index), timeout=8.0) + if opcode != "ACK": + raise RuntimeError(f"BENCH_SWEEP_FLOOR({index}) failed: {opcode} {payload}") + fields = parse_fields(payload) + return int(fields["FLOOR"]) / 10.0 + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument("--cardputer-port", required=True) + parser.add_argument("--log", required=True) + parser.add_argument("--results", required=True) + parser.add_argument("--laps", type=int, default=5, help="laps per settle value") + parser.add_argument("--settles-ms", type=int, action="append", + help=f"repeatable; default {' '.join(map(str, DEFAULT_SETTLES_MS))}") + parser.add_argument("--bins", type=int, action="append", + help=f"repeatable; default {' '.join(map(str, DEFAULT_BINS))}") + args = parser.parse_args() + settles = args.settles_ms or list(DEFAULT_SETTLES_MS) + bins = args.bins or list(DEFAULT_BINS) + + log_path = pathlib.Path(args.log) + log_path.parent.mkdir(parents=True, exist_ok=True) + results = ResultWriter(args.results) + with log_path.open("a", encoding="utf-8") as log: + card = Endpoint("cardputer", args.cardputer_port, CARD_MARKER, log) + try: + identity = require_ack(card, "HELLO", "-", timeout=45.0) + if "BENCH=1" not in identity: + raise RuntimeError(f"needs the bench image: {identity}") + results.write({"event": "boot", "identity": identity, + "settles_ms": settles, "bins": bins, "laps": args.laps}) + require_ack(card, "BENCH_SWEEP_RETUNE", "LIGHT", timeout=8.0) + + table = {} + print(f"{'settle':>7s} {'lap':>8s} " + " ".join(f"bin{b:<4d}" for b in bins)) + for settle in settles: + require_ack(card, "BENCH_SWEEP_SETTLE", str(settle), timeout=8.0) + per_bin = {b: [] for b in bins} + durations = [] + for _ in range(args.laps): + durations.append(run_lap(card)) + for b in bins: + per_bin[b].append(read_bin(card, b)) + med = {b: statistics.median(v) for b, v in per_bin.items()} + table[settle] = med + lap_s = statistics.median(durations) + results.write({"event": "settle", "settle_ms": settle, + "lap_seconds": round(lap_s, 3), + "floors": {str(b): med[b] for b in bins}, + "raw": {str(b): per_bin[b] for b in bins}}) + print(f"{settle:5d}ms {lap_s:7.2f}s " + + " ".join(f"{med[b]:7.1f}" for b in bins), flush=True) + + # bin 0 takes a full begin(); the rest take the light retune. If the + # under-read is a settle artifact it should show that asymmetry. + print() + base = table[max(settles)] + worst = table[min(settles)] + print(f"shift from {min(settles)}ms to {max(settles)}ms settle, per bin:") + for b in bins: + tag = " <- full begin() bin" if b == 0 else "" + print(f" bin {b:3d}: {worst[b]:7.1f} -> {base[b]:7.1f} " + f"({base[b] - worst[b]:+5.1f} dB){tag}") + finally: + # Never leave the bench image on a non-default sweep configuration. + try: + require_ack(card, "BENCH_SWEEP_SETTLE", "0", timeout=8.0) + require_ack(card, "BENCH_SWEEP_RETUNE", "LIGHT", timeout=8.0) + finally: + card.close() + results.close() + + +if __name__ == "__main__": + try: + main() + except KeyboardInterrupt: + print("interrupted; completed settle values are already durable", file=sys.stderr) + sys.exit(130) + except (OSError, serial.SerialException, RuntimeError, TimeoutError, ValueError) as error: + print(f"ws17 floor/settle: {error}", file=sys.stderr) + sys.exit(2) diff --git a/scripts/ws17_sweep_sensitivity_bench.py b/scripts/ws17_sweep_sensitivity_bench.py new file mode 100644 index 0000000..5fb8328 --- /dev/null +++ b/scripts/ws17_sweep_sensitivity_bench.py @@ -0,0 +1,238 @@ +#!/usr/bin/env python3 +"""Workstream 17: can Pass A flag a bin that is carrying traffic? + +v1.0.3 already settled that a full-band sweep cannot *decode* a packet -- that +needs the receiver parked for the whole airtime, which no 85-bin lap can offer, +and increasing samples per bin was tried and reverted on that reasoning. + +This asks a different question. Detecting a packet as *energy* needs only that +the bin's short sampling window overlaps the transmission, not that the whole +airtime is captured. Pass A visits each bin for about 3 ms (4 samples, 1 ms +apart) once per ~833 ms lap, so for a 148 ms packet on that bin the geometry +predicts an overlap chance near (148 + 3) / 833, roughly 18%. That is a +falsifiable number, and it is far from the "fraction of a percent" that 3 ms of +833 ms suggests at a glance. + +Three arms per run: + + persistent the transmitter radiates continuously. A control on the + instrument: if Pass A cannot flag the bin here, the sparse arm + measures nothing and the threshold margin is the story. + sparse one packet per interval, so the overlap is a lottery with known + odds. This is the measurement. + silent no transmission. Establishes the bin's own floor distribution, + which is what "flagged" has to be judged against. + +Reports the per-lap flag rate for each arm with Wilson intervals, against the +geometric prediction. It selects no threshold and changes no firmware. +""" + +import argparse +import json +import math +import pathlib +import statistics +import sys +import threading +import time + +import serial + +from bench_harness import (CARD_MARKER, TX_MARKER, Endpoint, ResultWriter, + card_status, parse_fields, require_ack) + + +BAND_HI_MHZ = 923.0 # fixed hardware ceiling regardless of region (energy_plan.h) +BIN_STEP_MHZ = 0.25 +SWEEP_TERMINAL_TIMEOUT_S = 90.0 +# Measured ARM-to-TX_STARTED latency over the WiFi control bridge. +PIPELINE_PRIME_S = 2.5 + + +def wilson(successes, total, z=1.959963984540054): + if total == 0: + return (0.0, 1.0) + p = successes / total + denom = 1.0 + z * z / total + centre = (p + z * z / (2 * total)) / denom + margin = z * math.sqrt(p * (1 - p) / total + z * z / (4 * total * total)) / denom + return (max(0.0, centre - margin), min(1.0, centre + margin)) + + +def bin_for_freq(freq_mhz, total_bins): + lo = BAND_HI_MHZ - (total_bins - 1) * BIN_STEP_MHZ + return max(0, min(total_bins - 1, round((freq_mhz - lo) / BIN_STEP_MHZ))) + + +class Firer: + """Fires the transmitter on its own schedule for the duration of a lap. + + `interval_s` of None means fire as fast as each transmission completes, + which is the persistent arm. Otherwise one pulse per interval, giving the + sparse arm its known duty. + """ + + def __init__(self, transmitter, interval_s): + self.transmitter = transmitter + self.interval_s = interval_s + self._stop = threading.Event() + self._thread = None + self.fired = 0 + + def _run(self): + while not self._stop.is_set(): + try: + require_ack(self.transmitter, "ARM", "0", timeout=2.0) + self.fired += 1 + except (RuntimeError, TimeoutError, OSError): + pass + self._stop.wait(self.interval_s if self.interval_s else 0.05) + + def __enter__(self): + self._thread = threading.Thread(target=self._run, daemon=True) + self._thread.start() + return self + + def __exit__(self, *exc): + self._stop.set() + if self._thread is not None: + self._thread.join(timeout=8.0) + return False + + +def run_lap(card, target_bin): + """One full Pass A lap; returns that bin's averaged RSSI in dBm.""" + require_ack(card, "SWEEP_START", "-", timeout=12.0) + deadline = time.monotonic() + SWEEP_TERMINAL_TIMEOUT_S + while time.monotonic() < deadline: + state = card_status(card).get("W") + if state in {"COMPLETE", "CANCELLED", "FAILED"}: + if state != "COMPLETE": + raise RuntimeError(f"sweep ended {state}") + break + time.sleep(0.1) + else: + raise TimeoutError("sweep did not reach a terminal state") + opcode, payload = card.request("BENCH_SWEEP_FLOOR", str(target_bin), timeout=8.0) + if opcode != "ACK": + raise RuntimeError(f"BENCH_SWEEP_FLOOR failed: {opcode} {payload}") + # Replies as "BIN=;FLOOR=". + fields = parse_fields(payload) + if "FLOOR" not in fields: + raise RuntimeError(f"BENCH_SWEEP_FLOOR gave no floor: {payload}") + return int(fields["FLOOR"]) / 10.0 + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument("--cardputer-port", required=True) + parser.add_argument("--heltec-port", required=True) + parser.add_argument("--log", required=True) + parser.add_argument("--results", required=True) + parser.add_argument("--candidate", default="MESH_OREGON") + parser.add_argument("--freq-mhz", type=float, default=918.5) + parser.add_argument("--airtime-ms", type=float, default=148.0, + help="measured airtime of the candidate, for the prediction") + parser.add_argument("--laps", type=int, default=30, help="laps per arm") + parser.add_argument("--sparse-interval-s", type=float, default=0.8) + parser.add_argument("--allow-transmit", action="store_true") + args = parser.parse_args() + if not args.allow_transmit: + parser.error("this bench arms the transmitter; pass --allow-transmit") + + log_path = pathlib.Path(args.log) + log_path.parent.mkdir(parents=True, exist_ok=True) + results = ResultWriter(args.results) + with log_path.open("a", encoding="utf-8") as log: + card = Endpoint("cardputer", args.cardputer_port, CARD_MARKER, log) + transmitter = None + try: + identity = require_ack(card, "HELLO", "-", timeout=45.0) + if "BENCH=1" not in identity: + raise RuntimeError(f"needs the bench image for BENCH_SWEEP_FLOOR: {identity}") + status = card_status(card) + total_bins = int(status.get("WN", "0")) or 85 + target = bin_for_freq(args.freq_mhz, total_bins) + transmitter = Endpoint("heltec", args.heltec_port, TX_MARKER, log) + require_ack(transmitter, "HELLO", "-", timeout=25.0) + require_ack(transmitter, "QUIET", "-", timeout=8.0) + require_ack(transmitter, "CONFIG", args.candidate, timeout=10.0) + + print(f"target {args.freq_mhz} MHz -> bin {target} of {total_bins}") + results.write({"event": "boot", "identity": identity, "target_bin": target, + "total_bins": total_bins, "candidate": args.candidate, + "freq_mhz": args.freq_mhz, "airtime_ms": args.airtime_ms, + "laps": args.laps, "sparse_interval_s": args.sparse_interval_s}) + + floors = {} + for arm, interval in (("silent", None), ("persistent", None), + ("sparse", args.sparse_interval_s)): + print(f"\n[{arm}] {args.laps} laps") + values = [] + for lap in range(args.laps): + if arm == "silent": + fired = 0 + floor = run_lap(card, target) + else: + with Firer(transmitter, interval) as firer: + # The control bridge adds roughly 1.5 s between ARM + # and the transmission actually starting. Firing + # only for the lap's duration therefore radiates + # entirely after the lap has ended, which reads as + # a silent band -- it did, identically across all + # three arms, before this wait was added. Prime the + # pipeline first so the source is genuinely on air + # while Pass A is sampling. + time.sleep(PIPELINE_PRIME_S) + floor = run_lap(card, target) + fired = firer.fired + values.append(floor) + results.write({"event": "lap", "arm": arm, "lap": lap, + "bin": target, "floor_dbm": floor, "pulses": fired}) + print(f" [{lap + 1:3d}/{args.laps}] bin {target} floor {floor:6.1f} dBm" + f" pulses {fired}", flush=True) + floors[arm] = values + require_ack(transmitter, "QUIET", "-", timeout=8.0) + + # "Flagged" is judged against the bin's own silent distribution, not + # an absolute number: Phase 12 rejected absolute thresholds at field + # levels, and the same objection applies here. + base = statistics.median(floors["silent"]) + spread = max(floors["silent"]) - base + threshold = base + max(3.0, spread + 1.0) + print(f"\nsilent floor median {base:.1f} dBm, max {max(floors['silent']):.1f} dBm") + print(f"flag threshold: {threshold:.1f} dBm (silent median + margin)\n") + predicted = (args.airtime_ms + 3.0) / 833.0 + for arm in ("silent", "persistent", "sparse"): + hits = sum(1 for v in floors[arm] if v >= threshold) + lo, hi = wilson(hits, len(floors[arm])) + extra = "" + if arm == "sparse": + extra = f" geometric prediction ~{predicted:.1%}" + print(f" {arm:11s} flagged {hits}/{len(floors[arm])} " + f"= {hits / len(floors[arm]):5.1%} 95% CI [{lo:.3f},{hi:.3f}]{extra}") + results.write({"event": "summary", "arm": arm, "hits": hits, + "laps": len(floors[arm]), "threshold_dbm": threshold, + "wilson95": [round(lo, 4), round(hi, 4)], + "floor_median": statistics.median(floors[arm]), + "floor_max": max(floors[arm])}) + finally: + if transmitter is not None: + try: + require_ack(transmitter, "QUIET", "-", timeout=8.0) + finally: + transmitter.close() + card.close() + results.close() + + +if __name__ == "__main__": + try: + main() + except KeyboardInterrupt: + print("interrupted; completed laps are already durable", file=sys.stderr) + sys.exit(130) + except (OSError, serial.SerialException, RuntimeError, TimeoutError, ValueError) as error: + print(f"ws17 sweep sensitivity: {error}", file=sys.stderr) + sys.exit(2) diff --git a/scripts/ws17_sweep_traffic_bench.py b/scripts/ws17_sweep_traffic_bench.py new file mode 100644 index 0000000..70ed297 --- /dev/null +++ b/scripts/ws17_sweep_traffic_bench.py @@ -0,0 +1,160 @@ +#!/usr/bin/env python3 +"""Workstream 17: does Pass A flag bins that are actually carrying traffic? + +Fully passive. The sources are real repeaters on known channels, so this +transmits nothing -- which is both correct etiquette on a live mesh and a more +realistic test than a fixture whose presence during a 3 ms bin visit cannot be +guaranteed. The bench transmitter is explicitly quieted and never armed. + +The design needs no external ground truth. Two bins are known to carry traffic +and the rest are not, so the quiet bins are the control *within the same lap*: + + bin 34 MeshCore default, 910.525 MHz (reported as the busier of the two) + bin 66 MeshOregon, 918.5 MHz + others controls, sampled across the band + +If Pass A sees traffic, 34 and 66 should be elevated relative to the controls +more often than the controls are relative to each other. If they are not +distinguishable, that is an answer too. + +Settle alternates 0 ms (shipped) and 3 ms (the value the floor measurement +found sufficient) lap by lap, so both configurations see the same traffic +rather than consecutive blocks that could differ in activity. That makes this +a direct test of whether the settle fix improves real detection, not just the +noise floor. +""" + +import argparse +import pathlib +import statistics +import sys +import time + +import serial + +from bench_harness import (CARD_MARKER, TX_MARKER, Endpoint, ResultWriter, + card_status, parse_fields, require_ack) + +SWEEP_TERMINAL_TIMEOUT_S = 120.0 +TRAFFIC_BINS = (34, 66) +# Immediate neighbours, to show whether energy lands only in the intended bin. +ADJACENT_BINS = (33, 35, 65, 67) +CONTROL_BINS = (5, 15, 25, 45, 55, 75, 80) + + +def run_lap(card): + require_ack(card, "SWEEP_START", "-", timeout=12.0) + deadline = time.monotonic() + SWEEP_TERMINAL_TIMEOUT_S + while time.monotonic() < deadline: + state = card_status(card).get("W") + if state in {"COMPLETE", "CANCELLED", "FAILED"}: + if state != "COMPLETE": + raise RuntimeError(f"sweep ended {state}") + return + time.sleep(0.05) + raise TimeoutError("sweep did not reach a terminal state") + + +def read_bin(card, index, attempts=3): + """Read one bin's floor, retrying transport failures. + + Native USB-CDC occasionally truncates a reply under sustained round trips; + an earlier run of this bench died at lap 68 of 200 for exactly that. The + query is idempotent -- it reads state the sweep already recorded -- so a + re-send cannot disturb the measurement. + """ + last = None + for attempt in range(attempts): + try: + opcode, payload = card.request("BENCH_SWEEP_FLOOR", str(index), timeout=12.0) + if opcode != "ACK": + raise RuntimeError(f"BENCH_SWEEP_FLOOR({index}): {opcode} {payload}") + return int(parse_fields(payload)["FLOOR"]) / 10.0 + except (RuntimeError, TimeoutError, KeyError, ValueError) as error: + last = error + time.sleep(0.3) + raise RuntimeError(f"BENCH_SWEEP_FLOOR({index}) failed {attempts}x: {last}") + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument("--cardputer-port", required=True) + parser.add_argument("--heltec-port", help="quieted if given; never armed") + parser.add_argument("--log", required=True) + parser.add_argument("--results", required=True) + parser.add_argument("--laps", type=int, default=120, help="total laps, alternating settle") + parser.add_argument("--settles-ms", type=int, action="append", + help="repeatable; default 0 and 3, alternated lap by lap") + args = parser.parse_args() + settles = args.settles_ms or [0, 3] + watched = sorted(set(TRAFFIC_BINS + ADJACENT_BINS + CONTROL_BINS)) + + log_path = pathlib.Path(args.log) + log_path.parent.mkdir(parents=True, exist_ok=True) + results = ResultWriter(args.results) + with log_path.open("a", encoding="utf-8") as log: + card = Endpoint("cardputer", args.cardputer_port, CARD_MARKER, log) + transmitter = None + try: + identity = require_ack(card, "HELLO", "-", timeout=45.0) + if "BENCH=1" not in identity: + raise RuntimeError(f"needs the bench image: {identity}") + if args.heltec_port: + # Not a source here. Quieted explicitly so a leftover beacon + # from earlier work cannot be mistaken for repeater traffic. + transmitter = Endpoint("heltec", args.heltec_port, TX_MARKER, log) + require_ack(transmitter, "HELLO", "-", timeout=25.0) + require_ack(transmitter, "QUIET", "-", timeout=8.0) + print("bench transmitter quieted; this run transmits nothing") + require_ack(card, "BENCH_SWEEP_RETUNE", "LIGHT", timeout=8.0) + results.write({"event": "boot", "identity": identity, "settles_ms": settles, + "traffic_bins": list(TRAFFIC_BINS), "adjacent": list(ADJACENT_BINS), + "controls": list(CONTROL_BINS), "laps": args.laps}) + + started = time.monotonic() + skipped = 0 + for lap in range(args.laps): + settle = settles[lap % len(settles)] + require_ack(card, "BENCH_SWEEP_SETTLE", str(settle), timeout=8.0) + # A lost lap is a lost sample, not a lost run: at a ~6% hit + # rate the value is in the count of laps, so aborting on one + # transport failure discards everything gathered so far. + try: + run_lap(card) + floors = {b: read_bin(card, b) for b in watched} + except (RuntimeError, TimeoutError) as error: + skipped += 1 + results.write({"event": "lap_error", "lap": lap, + "settle_ms": settle, "error": str(error)}) + print(f" [{lap + 1:4d}/{args.laps}] skipped: {error}", flush=True) + time.sleep(1.0) + continue + results.write({"event": "lap", "lap": lap, "settle_ms": settle, + "floors": {str(b): v for b, v in floors.items()}, + "elapsed_s": round(time.monotonic() - started, 1)}) + if lap % 10 == 0 or lap == args.laps - 1: + ctl = statistics.median([floors[b] for b in CONTROL_BINS]) + print(f" [{lap + 1:4d}/{args.laps}] settle {settle}ms " + f"bin34 {floors[34]:7.1f} bin66 {floors[66]:7.1f} " + f"controls {ctl:7.1f}", flush=True) + print(f"\n{args.laps} laps in {(time.monotonic() - started) / 60:.1f} min, {skipped} skipped") + finally: + try: + require_ack(card, "BENCH_SWEEP_SETTLE", "0", timeout=8.0) + finally: + if transmitter is not None: + transmitter.close() + card.close() + results.close() + + +if __name__ == "__main__": + try: + main() + except KeyboardInterrupt: + print("interrupted; completed laps are already durable", file=sys.stderr) + sys.exit(130) + except (OSError, serial.SerialException, RuntimeError, TimeoutError, ValueError) as error: + print(f"ws17 sweep traffic: {error}", file=sys.stderr) + sys.exit(2) diff --git a/src/bench_fault.cpp b/src/bench_fault.cpp index 9fe0ad2..fd1eb25 100644 --- a/src/bench_fault.cpp +++ b/src/bench_fault.cpp @@ -28,6 +28,11 @@ volatile uint16_t sweepSettleMs = ENERGY_SWEEP_SETTLE_DEFAULT_MS; volatile EnergyObservationResult passBCadLastResult = EnergyObservationResult::RADIO_ERROR; volatile bool passBCadHaveResult = false; +// Wide enough to overrun focusRequestTimeoutMs() for the longest legal +// request (2,000ms dwell + 1,000ms slack) and no wider. +constexpr uint32_t FOCUS_BENCH_STALL_MAX_MS = 4000; +volatile uint16_t focusStallMs = 0; + #if defined(LORATRACE_BENCH_FAULTS) // energy_plan.h's own ENERGY_BIN_RESERVED_COUNT -- the same hard ceiling // Pass A's bin loop is already bounded by, so this array is sized to hold @@ -271,6 +276,58 @@ bool benchRssiWindowTriggerAllowed() { #endif } +bool benchFocusSurveyTriggerAllowed() { +#if !defined(LORATRACE_BENCH_FAULTS) + return false; +#else + return true; +#endif +} + +bool benchFocusStallConfigure(uint32_t stall_ms) { +#if !defined(LORATRACE_BENCH_FAULTS) + (void)stall_ms; + return false; +#else + if (stall_ms > FOCUS_BENCH_STALL_MAX_MS) return false; + focusStallMs = (uint16_t)stall_ms; + return true; +#endif +} + +uint16_t benchFocusStallTakeMs() { +#if !defined(LORATRACE_BENCH_FAULTS) + return 0; +#else + const uint16_t stall = focusStallMs; + focusStallMs = 0; + return stall; +#endif +} + +bool benchHomeChannelOverride(ProfileOverrides &overrides) { +#if !defined(LORATRACE_BENCH_FAULTS) || !defined(LORATRACE_BENCH_HOME_FREQ_MHZ) + (void)overrides; + return false; +#else + overrides.meshtastic_set = true; + overrides.meshtastic.freq_mhz = LORATRACE_BENCH_HOME_FREQ_MHZ; + overrides.meshtastic.sf = LORATRACE_BENCH_HOME_SF; + overrides.meshtastic.bw_khz = LORATRACE_BENCH_HOME_BW_KHZ; + overrides.meshtastic.cr_denom = LORATRACE_BENCH_HOME_CR; + overrides.meshtastic.sync_word = LORATRACE_BENCH_HOME_SYNC; + return true; +#endif +} + +bool benchArbitrationTriggerAllowed() { +#if !defined(LORATRACE_BENCH_FAULTS) + return false; +#else + return true; +#endif +} + void benchPassBCadRecordResult(EnergyObservationResult result) { #if defined(LORATRACE_BENCH_FAULTS) passBCadLastResult = result; diff --git a/src/bench_fault.h b/src/bench_fault.h index 9b48a27..068df9e 100644 --- a/src/bench_fault.h +++ b/src/bench_fault.h @@ -2,6 +2,7 @@ #include +#include "channel_plans.h" #include "energy_observation.h" // Deterministic fault hooks used only by the dedicated cardputer-adv-bench @@ -113,6 +114,35 @@ bool benchSweepFloorQuery(uint16_t bin, int16_t &rssi_avg_dbm_x10); // holds still long enough to. Same production/bench split as the rest of // this file. bool benchRssiWindowTriggerAllowed(); +// Phase 12 bench-only Focus request gate. Production keeps Focus unreachable +// until its on-device control and claim gate are ready. +bool benchFocusSurveyTriggerAllowed(); + +// Bench-image-only one-shot stall injected into the Focus sample loop, in ms +// (0-4000). It is the only way to reach the timeout path deliberately: +// production Focus only times out when the SPI bus is genuinely contended, +// which a bench fixture cannot arrange on demand. Consumed once by the next +// accepted request; production stores nothing and always reports 0. +bool benchFocusStallConfigure(uint32_t stall_ms); +uint16_t benchFocusStallTakeMs(); + +// Bench-image-only home-channel override, applied after the SD config load. +// The bench's channel lives in config.txt on the bench SD card, so replacing +// a failed card silently moves `home` to the built-in default -- and Focus +// receives at the *home* bandwidth, so a bench measurement would quietly +// change bandwidth with nothing in its output to show it (2026-09-04: a card +// swap moved home from 918.5/SF8/BW125 to 906.875/SF11/BW250). +// +// Returns false, changing nothing, unless the bench image was built with +// -D LORATRACE_BENCH_HOME_* flags. Production always returns false. +bool benchHomeChannelOverride(ProfileOverrides &overrides); + +// Bench-image-only gate for entering Cell and Scope from framed control. +// Both are UI-only actions in production, so a fixture otherwise cannot prove +// Focus's mutual exclusion against them +// (docs/research/phase12-survey-truth-design.md §8's Device gate). It grants +// no new radio behavior -- it calls the same request functions the menu does. +bool benchArbitrationTriggerAllowed(); // Bench-image-only readback of the most recent BENCH_PASS_B_CAD attempt's // raw result (research/phase9-sweep-pass-b-design.md's standing diff --git a/src/focus_observation.h b/src/focus_observation.h new file mode 100644 index 0000000..54cd328 --- /dev/null +++ b/src/focus_observation.h @@ -0,0 +1,247 @@ +#pragma once +// LoRaTrace RX — Phase 12 Focus Survey bounded RSSI statistics and durable +// row contract. No coverage label is produced until its thresholds are earned +// by the controlled matrix in phase12-survey-truth-design.md. + +#include +#include +#include + +#include "detection.h" // missionProfileName() +#include "focus_plan.h" + +constexpr int16_t FOCUS_RSSI_NO_SAMPLE_DBM_X10 = 32767; +constexpr int16_t FOCUS_RSSI_HISTOGRAM_MIN_DBM = -140; +constexpr int16_t FOCUS_RSSI_HISTOGRAM_MAX_DBM = 0; +constexpr uint16_t FOCUS_RSSI_HISTOGRAM_BUCKET_COUNT = + (uint16_t)(FOCUS_RSSI_HISTOGRAM_MAX_DBM - FOCUS_RSSI_HISTOGRAM_MIN_DBM + 1); + +constexpr uint8_t FOCUS_RSSI_FLAG_SAMPLE_OVERFLOW = 0x01; +constexpr uint8_t FOCUS_RSSI_FLAG_BUCKET_OVERFLOW = 0x02; +constexpr uint8_t FOCUS_RSSI_FLAG_UNDERFLOW = 0x04; +constexpr uint8_t FOCUS_RSSI_FLAG_OVERFLOW = 0x08; + +// A fixed 1dB histogram is compact enough for one result while retaining no +// raw samples. Values beyond the documented receiver range are clamped to an +// endpoint and flagged, so summaries never pretend that the range is wider. +struct FocusRssiHistogram { + uint8_t bucket_counts[FOCUS_RSSI_HISTOGRAM_BUCKET_COUNT] = {}; + uint16_t sample_count = 0; + int16_t peak_dbm_x10 = FOCUS_RSSI_NO_SAMPLE_DBM_X10; + uint8_t flags = 0; +}; + +static_assert(sizeof(FocusRssiHistogram) <= 160, + "Focus histogram exceeds its one-result static SRAM budget"); + +inline int16_t focusRssiDbmX10ToNearestDbm(int16_t value_dbm_x10) { + // C++ integer division truncates toward zero, so handle negative values + // explicitly to retain half-away-from-zero 1dB binning. + return value_dbm_x10 >= 0 ? (int16_t)((value_dbm_x10 + 5) / 10) + : (int16_t)((value_dbm_x10 - 5) / 10); +} + +inline void focusHistogramAddSample(FocusRssiHistogram &histogram, + int16_t rssi_dbm_x10) { + if (histogram.sample_count == UINT16_MAX) { + histogram.flags |= FOCUS_RSSI_FLAG_SAMPLE_OVERFLOW; + return; + } + + if (histogram.sample_count == 0 || rssi_dbm_x10 > histogram.peak_dbm_x10) { + histogram.peak_dbm_x10 = rssi_dbm_x10; + } + histogram.sample_count++; + + int16_t dbm = focusRssiDbmX10ToNearestDbm(rssi_dbm_x10); + if (dbm < FOCUS_RSSI_HISTOGRAM_MIN_DBM) { + dbm = FOCUS_RSSI_HISTOGRAM_MIN_DBM; + histogram.flags |= FOCUS_RSSI_FLAG_UNDERFLOW; + } else if (dbm > FOCUS_RSSI_HISTOGRAM_MAX_DBM) { + dbm = FOCUS_RSSI_HISTOGRAM_MAX_DBM; + histogram.flags |= FOCUS_RSSI_FLAG_OVERFLOW; + } + + const uint16_t index = (uint16_t)(dbm - FOCUS_RSSI_HISTOGRAM_MIN_DBM); + if (histogram.bucket_counts[index] == UINT8_MAX) { + histogram.flags |= FOCUS_RSSI_FLAG_BUCKET_OVERFLOW; + } else { + histogram.bucket_counts[index]++; + } +} + +constexpr bool focusHistogramHasExactQuantiles(const FocusRssiHistogram &histogram) { + return histogram.sample_count > 0 && + (histogram.flags & (FOCUS_RSSI_FLAG_SAMPLE_OVERFLOW | + FOCUS_RSSI_FLAG_BUCKET_OVERFLOW)) == 0; +} + +// Returns a 1dB-quantized tenths-of-dBm value, or NO_SAMPLE when the retained +// buckets cannot faithfully represent the count (or no sample was accepted). +inline int16_t focusHistogramPercentileDbmX10(const FocusRssiHistogram &histogram, + uint8_t percentile) { + if (!focusHistogramHasExactQuantiles(histogram) || percentile == 0 || percentile > 100) { + return FOCUS_RSSI_NO_SAMPLE_DBM_X10; + } + const uint32_t rank = ((uint32_t)histogram.sample_count * percentile + 99u) / 100u; + uint32_t cumulative = 0; + for (uint16_t i = 0; i < FOCUS_RSSI_HISTOGRAM_BUCKET_COUNT; ++i) { + cumulative += histogram.bucket_counts[i]; + if (cumulative >= rank) { + return (int16_t)((FOCUS_RSSI_HISTOGRAM_MIN_DBM + (int16_t)i) * 10); + } + } + return FOCUS_RSSI_NO_SAMPLE_DBM_X10; +} + +inline int16_t focusHistogramMedianDbmX10(const FocusRssiHistogram &histogram) { + return focusHistogramPercentileDbmX10(histogram, 50); +} + +inline int16_t focusHistogramP90DbmX10(const FocusRssiHistogram &histogram) { + return focusHistogramPercentileDbmX10(histogram, 90); +} + +// How many accepted samples read at or above `threshold_dbm_x10`. +// +// A count is a different instrument from a summary statistic. `peak` is one +// sample and therefore noise-prone; a count integrates over the pass, which +// is what the 2026-09-05 field-level measurement suggested was missing when +// every summary statistic failed to separate a known source from ambient +// (docs/hardware-results/2026-09-04-phase12-focus-matrix.md). Pure, so the +// rule can be evaluated on the host before any of it is trusted on-device. +inline uint16_t focusHistogramCountAtLeast(const FocusRssiHistogram &histogram, + int16_t threshold_dbm_x10) { + const int16_t threshold_dbm = focusRssiDbmX10ToNearestDbm(threshold_dbm_x10); + if (threshold_dbm > FOCUS_RSSI_HISTOGRAM_MAX_DBM) return 0; + const int16_t lowest = threshold_dbm < FOCUS_RSSI_HISTOGRAM_MIN_DBM + ? FOCUS_RSSI_HISTOGRAM_MIN_DBM + : threshold_dbm; + uint16_t total = 0; + for (uint16_t i = (uint16_t)(lowest - FOCUS_RSSI_HISTOGRAM_MIN_DBM); + i < FOCUS_RSSI_HISTOGRAM_BUCKET_COUNT; ++i) { + total = (uint16_t)(total + histogram.bucket_counts[i]); + } + return total; +} + +// The same count expressed relative to the pass's own median, so the rule +// adapts to the ambient floor where it ran instead of assuming one. An +// absolute threshold was measured and rejected at field levels; this is the +// candidate that replaced it, and it is still a candidate. +inline uint16_t focusHistogramCountAboveMedian(const FocusRssiHistogram &histogram, + int16_t margin_dbm_x10) { + const int16_t median = focusHistogramMedianDbmX10(histogram); + if (median == FOCUS_RSSI_NO_SAMPLE_DBM_X10) return 0; + return focusHistogramCountAtLeast(histogram, (int16_t)(median + margin_dbm_x10)); +} + +enum class FocusRequestStatus : uint8_t { + COMPLETE = 0, + CANCELLED, + TIMEOUT, + FAILED, +}; + +inline const char *focusRequestStatusName(FocusRequestStatus status) { + switch (status) { + case FocusRequestStatus::COMPLETE: return "complete"; + case FocusRequestStatus::CANCELLED: return "cancelled"; + case FocusRequestStatus::TIMEOUT: return "timeout"; + case FocusRequestStatus::FAILED: return "failed"; + default: return "unknown"; + } +} + +// Queue/log record. GPS/run fields stay formatter parameters so radio +// ownership remains bounded. +struct FocusObservation { + uint32_t rx_millis = 0; + uint32_t observation_ms = 0; + float freq_mhz = 0.0f; + uint16_t focus_id = 0; + uint16_t selection_bin_index = 0; + uint16_t requested_dwell_ms = 0; + uint16_t requested_samples = 0; + uint16_t sample_count = 0; + uint16_t qualifying_count = 0; + int16_t rssi_median_dbm_x10 = FOCUS_RSSI_NO_SAMPLE_DBM_X10; + int16_t rssi_p90_dbm_x10 = FOCUS_RSSI_NO_SAMPLE_DBM_X10; + int16_t rssi_peak_dbm_x10 = FOCUS_RSSI_NO_SAMPLE_DBM_X10; + int16_t radio_status = 0; + uint8_t profile = 0; // MissionProfile + uint8_t requested_passes = 0; + uint8_t valid_passes = 0; + FocusSelectionSource selection_source = FocusSelectionSource::PRESET; + FocusRequestStatus request_status = FocusRequestStatus::FAILED; + bool home_restore = false; + bool wifi_on = false; +}; + +static_assert(sizeof(FocusObservation) <= 48, + "FocusObservation exceeds the Phase 12 one-result queue budget"); +static_assert(sizeof(FocusRssiHistogram) + sizeof(FocusObservation) <= 224, + "Focus working state must leave headroom below the 256B target"); + +// The logger's row buffer, defined here so the writer and its budget test +// cannot drift apart. An over-long row is dropped, not truncated +// (focusObservationFormatCsv() returns 0), so the margin is deliberate: +// measured worst case is 189 bytes with every field at its widest, including +// widths the bounded request cannot actually produce. +constexpr size_t FOCUS_CSV_ROW_MAX = 256; + +constexpr const char *FOCUS_CSV_HEADER = + "timestamp_utc,lat,lon,fix_quality,run,rx_uptime_ms,profile,focus_id," + "selection_source,selection_bin_index,freq_mhz,requested_passes,valid_passes," + "requested_dwell_ms,observation_ms,requested_samples,sample_count," + "rssi_median_dbm,rssi_p90_dbm,rssi_peak_dbm,qualifying_count,coverage," + "request_status,home_restore,wifi_on,radio_status"; + +inline void focusFormatRssiOrBlank(int16_t rssi_dbm_x10, char *out, size_t out_size) { + if (rssi_dbm_x10 == FOCUS_RSSI_NO_SAMPLE_DBM_X10) { + if (out_size > 0) out[0] = '\0'; + return; + } + snprintf(out, out_size, "%.1f", (double)rssi_dbm_x10 / 10.0); +} + +inline size_t focusObservationFormatCsv(const FocusObservation &observation, + char *out, size_t out_size, + const char *timestamp_utc, bool has_fix, + double lat, double lon, uint8_t fix_quality, + uint16_t run) { + if (out == nullptr || out_size == 0) return 0; + + char latbuf[16], lonbuf[16], medianbuf[12], p90buf[12], peakbuf[12]; + if (has_fix) { + snprintf(latbuf, sizeof(latbuf), "%.6f", lat); + snprintf(lonbuf, sizeof(lonbuf), "%.6f", lon); + } else { + latbuf[0] = '\0'; + lonbuf[0] = '\0'; + } + focusFormatRssiOrBlank(observation.rssi_median_dbm_x10, medianbuf, sizeof(medianbuf)); + focusFormatRssiOrBlank(observation.rssi_p90_dbm_x10, p90buf, sizeof(p90buf)); + focusFormatRssiOrBlank(observation.rssi_peak_dbm_x10, peakbuf, sizeof(peakbuf)); + + // The doubled comma before request_status is intentional: coverage is an + // empty persisted field until controlled measurements select its policy. + const int n = snprintf( + out, out_size, + "%s,%s,%s,%u,%u,%lu,%s,%u,%s,%u,%.3f,%u,%u,%u,%lu,%u,%u,%s,%s,%s,%u,,%s,%u,%u,%d", + timestamp_utc ? timestamp_utc : "", latbuf, lonbuf, + (unsigned)fix_quality, (unsigned)run, (unsigned long)observation.rx_millis, + missionProfileName(observation.profile), (unsigned)observation.focus_id, + focusSelectionSourceName(observation.selection_source), + (unsigned)observation.selection_bin_index, (double)observation.freq_mhz, + (unsigned)observation.requested_passes, (unsigned)observation.valid_passes, + (unsigned)observation.requested_dwell_ms, (unsigned long)observation.observation_ms, + (unsigned)observation.requested_samples, (unsigned)observation.sample_count, + medianbuf, p90buf, peakbuf, (unsigned)observation.qualifying_count, + focusRequestStatusName(observation.request_status), + (unsigned)observation.home_restore, (unsigned)observation.wifi_on, + (int)observation.radio_status); + + if (n < 0 || (size_t)n >= out_size) return 0; + return (size_t)n; +} diff --git a/src/focus_plan.h b/src/focus_plan.h new file mode 100644 index 0000000..754561b --- /dev/null +++ b/src/focus_plan.h @@ -0,0 +1,154 @@ +#pragma once +// LoRaTrace RX — Phase 12 Focus Survey request contract. This is pure +// selection math only: it does not own the radio or start an acquisition. + +#include + +#include "energy_plan.h" + +// Phase 12 starts with exactly one selected frequency. A wider request needs +// a new static-RAM and radio-away budget; it must not quietly grow here. +constexpr uint8_t FOCUS_SELECTED_BIN_COUNT = 1; +// The first bench prototype makes one measurement per request. The controlled +// matrix supplies its 30 trials by issuing 30 independently logged requests, +// not by monopolizing Watch for a hidden multi-pass loop. +constexpr uint8_t FOCUS_BENCH_REQUESTED_PASSES = 1; +constexpr uint16_t FOCUS_BENCH_DWELL_MIN_MS = 2; +constexpr uint16_t FOCUS_BENCH_DWELL_MAX_MS = 2000; +constexpr uint16_t FOCUS_BENCH_SAMPLES_MIN = 2; +// Raised from 64 to sweep sample spacing at a fixed dwell: the 2026-09-04 +// matrix showed detection tracks the source's airtime against +// dwell/(samples-1), not against dwell, so selecting a real sampling policy +// needs spacings down to ~10 ms across a 2,000 ms dwell (201 samples). +// +// The ceiling is set by the histogram, not by RAM: bucket_counts are uint8_t, +// so a pass whose samples all land in one 1 dB bucket saturates at 255 and +// focusHistogramHasExactQuantiles() then refuses to report a percentile at +// all. Staying below that keeps every quantile exact in the worst case. The +// histogram itself is a fixed 141 bytes regardless of sample count, so this +// costs no additional static RAM (docs/research/phase12-survey-truth-design.md +// §4.2). +constexpr uint16_t FOCUS_BENCH_SAMPLES_MAX = 208; +static_assert(FOCUS_BENCH_SAMPLES_MAX < 255, + "a pass must not be able to saturate a uint8 histogram bucket"); + +// How far apart a pass's RSSI samples may fall, in ms. This is the constant +// that decides what a pass can observe at all, and it was measured, not +// chosen: at a fixed 2,000 ms dwell, a 94 ms source was missed at 286 ms and +// 100 ms spacing (8/15 and 2/15 trials) and caught 15/15 at both 50 ms and +// 20 ms, with the worst-case reading improving from -97 dBm to -66 dBm +// between those two (docs/hardware-results/2026-09-04-phase12-focus-matrix.md). +// Detection collapses once spacing approaches the source's airtime, so 20 ms +// keeps roughly a 2x margin against the ~40-50 ms airtime of the fastest +// realistic mesh traffic. +// +// Finer sampling is free in the only currency that matters here: measured +// radio-away time was 2,073-2,075 ms across every arm, whether the pass took +// 8 samples or 101. The cost of a longer dwell is the dwell itself. +constexpr uint16_t FOCUS_SAMPLE_SPACING_MS = 20; + +// Samples a dwell should take under that policy. Fixing the sample count +// instead -- as the first bench slice did at 8 -- makes a longer dwell +// strictly worse at catching bursts, because the spacing grows with it: a +// 2,000 ms dwell then observes eight instants, not 2,000 ms, which is exactly +// what the coverage vocabulary must never imply (§3). +// Rounds the interval count up, so actual spacing is at most the policy +// rather than at least it -- truncating would let a 30 ms dwell take two +// samples 30 ms apart and quietly violate the constant it is derived from. +// Split in two because the device toolchain builds this as C++11 (pinned +// Arduino-ESP32 2.0.17), where a constexpr body must be a single return -- +// the native test environment's newer standard accepts locals and will not +// catch it. +constexpr uint16_t focusDerivedSampleCount(uint16_t dwell_ms) { + return (uint16_t)((dwell_ms + FOCUS_SAMPLE_SPACING_MS - 1) / FOCUS_SAMPLE_SPACING_MS + 1); +} + +constexpr uint16_t focusSamplesForDwell(uint16_t dwell_ms) { + return focusDerivedSampleCount(dwell_ms) < FOCUS_BENCH_SAMPLES_MIN + ? FOCUS_BENCH_SAMPLES_MIN + : focusDerivedSampleCount(dwell_ms); +} + +// A bounded request must be bounded in wall-clock time, not just in samples: +// each sample waits on the shared SPI bus (radio_task.cpp's 250ms BUS_WAIT), +// so contention can stretch a nominal dwell well past it while the radio is +// away from home. Past this deadline the request stops sampling and +// terminates as `timeout` instead of running long. The slack absorbs a few +// bus waits plus scheduling jitter; it is a bench-slice bound, and §6's +// matrix (docs/research/phase12-survey-truth-design.md) may revise it before +// any operator-facing control exists. +constexpr uint16_t FOCUS_REQUEST_TIMEOUT_SLACK_MS = 1000; + +enum class FocusSelectionSource : uint8_t { + SWEEP_BIN = 0, + WATERFALL_BIN, + PRESET, +}; + +inline const char *focusSelectionSourceName(FocusSelectionSource source) { + switch (source) { + case FocusSelectionSource::SWEEP_BIN: return "sweep"; + case FocusSelectionSource::WATERFALL_BIN: return "waterfall"; + case FocusSelectionSource::PRESET: return "preset"; + default: return "unknown"; + } +} + +constexpr bool focusSelectionSourceIsKnown(FocusSelectionSource source) { + return source == FocusSelectionSource::SWEEP_BIN || + source == FocusSelectionSource::WATERFALL_BIN || + source == FocusSelectionSource::PRESET; +} + +// The 100ms/500ms/2s bench arms are not production defaults, and coverage +// thresholds remain unselected. Bounds make this bench-only request finite. +struct FocusRequest { + Region region = Region::GLOBAL; + EnergyBinStep bin_step = ENERGY_SWEEP_DEFAULT_STEP; + uint16_t selection_bin_index = 0; + uint16_t requested_dwell_ms = 0; + uint16_t requested_samples = 0; + uint8_t requested_passes = 0; + FocusSelectionSource selection_source = FocusSelectionSource::PRESET; +}; + +static_assert(sizeof(FocusRequest) <= 16, + "FocusRequest must stay a small fixed radio-control payload"); + +constexpr bool focusRequestHasValidBin(const FocusRequest &request) { + return request.selection_bin_index < + energyBinCount(energySweepBandForRegion(request.region), request.bin_step); +} + +constexpr bool focusRequestIsValid(const FocusRequest &request) { + return FOCUS_SELECTED_BIN_COUNT == 1 && + focusSelectionSourceIsKnown(request.selection_source) && + focusRequestHasValidBin(request) && + request.requested_passes == FOCUS_BENCH_REQUESTED_PASSES && + request.requested_dwell_ms >= FOCUS_BENCH_DWELL_MIN_MS && + request.requested_dwell_ms <= FOCUS_BENCH_DWELL_MAX_MS && + request.requested_samples >= FOCUS_BENCH_SAMPLES_MIN && + request.requested_samples <= FOCUS_BENCH_SAMPLES_MAX; +} + +static_assert(focusSamplesForDwell(FOCUS_BENCH_DWELL_MAX_MS) <= FOCUS_BENCH_SAMPLES_MAX, + "the sampling policy must stay inside the request's sample bound"); + +// Deadline measured from the first sample-loop tick, not from the request's +// acceptance: queue latency is Core 1's scheduling, not the dwell's cost. +constexpr uint32_t focusRequestTimeoutMs(const FocusRequest &request) { + return (uint32_t)request.requested_dwell_ms + FOCUS_REQUEST_TIMEOUT_SLACK_MS; +} + +// Worst-case time a legal request may hold the radio before restore begins. +// Restore is bounded separately by restoreHomeListen()'s own path. +constexpr uint32_t FOCUS_MAX_SAMPLING_MS = + (uint32_t)FOCUS_BENCH_DWELL_MAX_MS + FOCUS_REQUEST_TIMEOUT_SLACK_MS; +static_assert(FOCUS_MAX_SAMPLING_MS <= 3000, + "Focus must not grow an unbudgeted radio-away window"); + +inline float focusRequestFrequencyMhz(const FocusRequest &request) { + return energyBinFrequencyMhz(request.selection_bin_index, + energySweepBandForRegion(request.region), + request.bin_step); +} diff --git a/src/focus_runtime.h b/src/focus_runtime.h new file mode 100644 index 0000000..fdeb242 --- /dev/null +++ b/src/focus_runtime.h @@ -0,0 +1,76 @@ +#pragma once +// Phase 12 radio-task lifecycle bookkeeping. Hardware calls remain outside +// this header; it makes the terminal row impossible to publish before home +// restore has been recorded. + +#include + +#include "focus_observation.h" // FocusRequestStatus + +enum class FocusRuntimeState : uint8_t { + IDLE, + SURVEYING, + RESTORING, + COMPLETE, + CANCELLED, + TIMEOUT, + FAILED, +}; + +struct FocusRuntime { + FocusRequest request; + FocusRuntimeState state = FocusRuntimeState::IDLE; + uint16_t valid_passes = 0; + uint32_t observation_ms = 0; + bool cancelled = false; + bool timed_out = false; + bool failed = false; +}; + +inline bool focusRuntimeBegin(FocusRuntime &runtime, const FocusRequest &request) { + if (runtime.state == FocusRuntimeState::SURVEYING || + runtime.state == FocusRuntimeState::RESTORING || !focusRequestIsValid(request)) { + return false; + } + runtime = FocusRuntime{}; + runtime.request = request; + runtime.state = FocusRuntimeState::SURVEYING; + return true; +} + +inline void focusRuntimeNoteValidPass(FocusRuntime &runtime, uint32_t observation_ms) { + if (runtime.state != FocusRuntimeState::SURVEYING) return; + runtime.valid_passes++; + runtime.observation_ms += observation_ms; +} + +inline void focusRuntimeCancel(FocusRuntime &runtime) { + if (runtime.state == FocusRuntimeState::SURVEYING) runtime.cancelled = true; +} + +inline void focusRuntimeTimeout(FocusRuntime &runtime) { + if (runtime.state == FocusRuntimeState::SURVEYING) runtime.timed_out = true; +} + +inline void focusRuntimeFail(FocusRuntime &runtime) { + if (runtime.state == FocusRuntimeState::SURVEYING) runtime.failed = true; +} + +inline void focusRuntimeBeginRestore(FocusRuntime &runtime) { + if (runtime.state == FocusRuntimeState::SURVEYING) runtime.state = FocusRuntimeState::RESTORING; +} + +inline FocusRequestStatus focusRuntimeFinishRestore(FocusRuntime &runtime, bool restored) { + if (!restored) runtime.failed = true; + if (runtime.failed) runtime.state = FocusRuntimeState::FAILED; + else if (runtime.cancelled) runtime.state = FocusRuntimeState::CANCELLED; + else if (runtime.timed_out) runtime.state = FocusRuntimeState::TIMEOUT; + else runtime.state = FocusRuntimeState::COMPLETE; + + switch (runtime.state) { + case FocusRuntimeState::COMPLETE: return FocusRequestStatus::COMPLETE; + case FocusRuntimeState::CANCELLED: return FocusRequestStatus::CANCELLED; + case FocusRuntimeState::TIMEOUT: return FocusRequestStatus::TIMEOUT; + default: return FocusRequestStatus::FAILED; + } +} diff --git a/src/logger_task.cpp b/src/logger_task.cpp index b0d8208..aded767 100644 --- a/src/logger_task.cpp +++ b/src/logger_task.cpp @@ -10,6 +10,7 @@ #include "cell_observation.h" #include "detection.h" #include "energy_observation.h" +#include "focus_observation.h" #include "gps_task.h" #include "memory_stats.h" #include "serial_control.h" @@ -30,6 +31,7 @@ constexpr const char *PROBE_LEAF = "probe.csv"; constexpr const char *ENERGY_LEAF = "energy.csv"; constexpr const char *NODES_LEAF = "nodes.csv"; constexpr const char *CELL_LEAF = "cell.csv"; +constexpr const char *FOCUS_LEAF = "focus.csv"; // Resolved once, on the first successful mount of this power-on. uint16_t runIndex = 0; @@ -39,6 +41,7 @@ char probePath[RUN_PATH_MAX]; char energyPath[RUN_PATH_MAX]; char nodesPath[RUN_PATH_MAX]; char cellPath[RUN_PATH_MAX]; +char focusPath[RUN_PATH_MAX]; // ~2KB holds a few complete raw-frame rows. Sized to keep a single flush // short (see the header): bigger buffers mean longer bus holds, which is @@ -67,6 +70,7 @@ QueueHandle_t scanObservationQueue = nullptr; QueueHandle_t energyObservationQueue = nullptr; QueueHandle_t identityQueue = nullptr; QueueHandle_t cellObservationQueue = nullptr; +QueueHandle_t focusObservationQueue = nullptr; bool sdReady = false; bool initialSdMounted = false; volatile bool sdRetryRequested = false; @@ -99,6 +103,9 @@ volatile uint32_t identityRowsDropped = 0; volatile uint32_t maxCellMs = 0; volatile uint32_t cellRowsWritten = 0; volatile uint32_t cellRowsDropped = 0; +volatile uint32_t maxFocusMs = 0; +volatile uint32_t focusRowsWritten = 0; +volatile uint32_t focusRowsDropped = 0; // Highest runNNNN index already on the card, or 0 if there are none. // Assumes the caller holds the bus and SD is mounted. @@ -185,6 +192,9 @@ bool openLogsLocked(bool remount) { if (runFilePath(cellPath, sizeof(cellPath), LOG_DIR, runIndex, CELL_LEAF) == 0) { return false; } + if (runFilePath(focusPath, sizeof(focusPath), LOG_DIR, runIndex, FOCUS_LEAF) == 0) { + return false; + } } if (!ensureCsvLocked(detectionsPath, LOG_CSV_HEADER)) return false; @@ -202,6 +212,7 @@ bool openLogsLocked(bool remount) { // Same durability tier as probePath/energyPath: Cell is mission // output too when an operator runs it. if (!ensureCsvLocked(cellPath, CELL_CSV_HEADER)) return false; + if (!ensureCsvLocked(focusPath, FOCUS_CSV_HEADER)) return false; return true; } @@ -523,6 +534,35 @@ void appendCellObservation(const CellObservation &observation) { } } +void appendFocusObservation(const FocusObservation &observation) { + if (!sdReady) { + focusRowsDropped++; + return; + } + GpsFix fix; + const bool haveFix = gpsGetFix(fix, pdMS_TO_TICKS(50)); + const uint32_t now = millis(); + const bool fresh = haveFix && gpsFixIsFresh(fix, now, FIX_MAX_AGE_MS); + char timestamp[24]; + detectionFormatTimestamp(timestamp, sizeof(timestamp), haveFix && fix.has_time, fix.year, + fix.month, fix.day, fix.hour, fix.minute, fix.second); + char row[FOCUS_CSV_ROW_MAX]; + const size_t n = focusObservationFormatCsv( + observation, row, sizeof(row), timestamp, fresh, fix.lat, fix.lon, + haveFix ? fix.fix_quality : 0, runIndex); + if (n == 0) { + focusRowsDropped++; + return; + } + row[n] = '\n'; + const WriteResult result = appendToFile(focusPath, row, n + 1, maxFocusMs); + if (result == WriteResult::OK) focusRowsWritten++; + else { + focusRowsDropped++; + if (result == WriteResult::FILE_ERROR) sdReady = false; + } +} + void appendNodeIdentity(const NodeIdentity &identity) { if (!sdReady) { identityRowsDropped++; @@ -615,6 +655,12 @@ void loggerTask(void *) { appendCellObservation(observation); } } + if (focusObservationQueue != nullptr) { + FocusObservation observation; + if (xQueueReceive(focusObservationQueue, &observation, 0) == pdTRUE) { + appendFocusObservation(observation); + } + } const uint32_t analyzerSweepRuns = radioEnergySweepCount(); if (analyzerSweepRuns != lastAnalyzerSweepSeen) { @@ -659,12 +705,14 @@ void loggerTask(void *) { } // namespace bool loggerTaskStart(QueueHandle_t queue, QueueHandle_t scanQueue, QueueHandle_t energyQueue, - QueueHandle_t nodesQueue, QueueHandle_t cellQueue, bool mountedAtBoot) { + QueueHandle_t nodesQueue, QueueHandle_t cellQueue, + QueueHandle_t focusQueue, bool mountedAtBoot) { detectionQueue = queue; scanObservationQueue = scanQueue; energyObservationQueue = energyQueue; identityQueue = nodesQueue; cellObservationQueue = cellQueue; + focusObservationQueue = focusQueue; initialSdMounted = mountedAtBoot; sdReady = false; sdRetryRequested = false; @@ -743,6 +791,8 @@ uint32_t loggerCellRowsWritten() { uint32_t loggerCellRowsDropped() { return cellRowsDropped; } +uint32_t loggerFocusRowsWritten() { return focusRowsWritten; } +uint32_t loggerFocusRowsDropped() { return focusRowsDropped; } void loggerDebugToggle() { loggerDebugSetEnabled(!debugVerbose); diff --git a/src/logger_task.h b/src/logger_task.h index fa473a2..0fe8861 100644 --- a/src/logger_task.h +++ b/src/logger_task.h @@ -42,6 +42,7 @@ #include "cell_observation.h" #include "energy_observation.h" +#include "focus_observation.h" #include "node_identity.h" #include "scan_observation.h" @@ -54,7 +55,8 @@ // it down and immediately remounting it. Returns false if the task could // not be created. bool loggerTaskStart(QueueHandle_t queue, QueueHandle_t scanQueue, QueueHandle_t energyQueue, - QueueHandle_t identityQueue, QueueHandle_t cellQueue, bool initialSdMounted); + QueueHandle_t identityQueue, QueueHandle_t cellQueue, + QueueHandle_t focusQueue, bool initialSdMounted); // True once SD is mounted and the log file is writable. When false the // task keeps draining the queue and discarding — a missing card must not @@ -89,6 +91,8 @@ uint32_t loggerIdentityRowsWritten(); uint32_t loggerIdentityRowsDropped(); uint32_t loggerCellRowsWritten(); uint32_t loggerCellRowsDropped(); +uint32_t loggerFocusRowsWritten(); +uint32_t loggerFocusRowsDropped(); // This power-on's run index, or 0 before SD has mounted. Matches the // runNNNN directory the logs are being written into. diff --git a/src/main.cpp b/src/main.cpp index 54ee1b4..391b8f3 100644 --- a/src/main.cpp +++ b/src/main.cpp @@ -30,9 +30,11 @@ #include "analyzer_state.h" #include "backlight.h" #include "battery.h" +#include "bench_fault.h" #include "board_pins.h" #include "capture_settings.h" #include "cell_observation.h" +#include "focus_observation.h" #include "channel_plans.h" #include "config.h" #include "detection.h" @@ -115,6 +117,7 @@ constexpr UBaseType_t ENERGY_OBSERVATION_QUEUE_DEPTH = 32; // covers a full sweep between logger drains — a starting choice, not a // measured one, same as the other queue depths here. constexpr UBaseType_t CELL_OBSERVATION_QUEUE_DEPTH = 16; +constexpr UBaseType_t FOCUS_OBSERVATION_QUEUE_DEPTH = 4; // How long the completed boot checklist stays on screen before uiTaskStart() // takes over the panel with the main status pages. constexpr uint32_t BOOT_CHECKLIST_HOLD_MS = 1000; @@ -123,6 +126,7 @@ QueueHandle_t scanObservationQueue = nullptr; QueueHandle_t energyObservationQueue = nullptr; QueueHandle_t identityQueue = nullptr; QueueHandle_t cellObservationQueue = nullptr; +QueueHandle_t focusObservationQueue = nullptr; void splashLine(const String &msg, uint16_t color = SPLASH_FG) { if (!displayReady) return; @@ -364,6 +368,16 @@ void setup() { // needed for this read. bool sdMounted = false; loadProfileOverridesFromSD(channelOverrides, PIN_SD_CS, sharedSpi(), &sdMounted); + // Bench images only, and only when built with the -D flags; production + // changes nothing here (bench_fault.h). Announced loudly so a bench + // channel can never be mistaken for the card's own configuration. + if (benchHomeChannelOverride(channelOverrides)) { + SerialLock lock(pdMS_TO_TICKS(200)); + if (lock.held()) { + Serial.println(F("[bench] home channel overridden by build flags, " + "ignoring config.txt's Meshtastic block.")); + } + } // Last profile an operator actually selected via the menu, not always // Meshtastic — see profile_state.h for why this used to be hardcoded. MissionProfile bootProfile = MissionProfile::MESHTASTIC; @@ -406,6 +420,10 @@ void setup() { if (cellObservationQueue == nullptr) { fatal(F("FATAL: could not allocate the cell queue."), F("FATAL: cell queue alloc")); } + focusObservationQueue = xQueueCreate(FOCUS_OBSERVATION_QUEUE_DEPTH, sizeof(FocusObservation)); + if (focusObservationQueue == nullptr) { + fatal(F("FATAL: could not allocate the Focus queue."), F("FATAL: Focus queue alloc")); + } // Consumers before producer: the logger must be draining before the // radio starts filling, or the first burst is dropped for no reason. @@ -427,7 +445,7 @@ void setup() { } if (!loggerTaskStart(detectionQueue, scanObservationQueue, energyObservationQueue, identityQueue, - cellObservationQueue, sdMounted)) { + cellObservationQueue, focusObservationQueue, sdMounted)) { fatal(F("FATAL: logger task failed to start."), F("FATAL: logger task")); } // No splash line on success: this is RTOS resource allocation, not a @@ -441,7 +459,7 @@ void setup() { // override the same way this boot resolved `bootProfile`'s. if (!radioTaskStart(activeChannel, bootProfile, channelOverrides, detectionQueue, scanObservationQueue, energyObservationQueue, identityQueue, - cellObservationQueue)) { + cellObservationQueue, focusObservationQueue)) { { SerialLock lock(pdMS_TO_TICKS(200)); if (lock.held()) { diff --git a/src/radio_task.cpp b/src/radio_task.cpp index 088c10d..0481800 100644 --- a/src/radio_task.cpp +++ b/src/radio_task.cpp @@ -48,6 +48,19 @@ portMUX_TYPE activeChannelMux = portMUX_INITIALIZER_UNLOCKED; ChannelParams activeChannel; MissionProfile activeProfile = MissionProfile::MESHTASTIC; +// Focus finishes on Core 1 but its bench-only compact result readback is +// served by Serial Control on Core 0. A 40-byte critical copy avoids a torn +// summary; it never carries GPS/run data and does not affect radio ownership. +portMUX_TYPE focusResultMux = portMUX_INITIALIZER_UNLOCKED; +FocusObservation lastFocusObservation; +bool haveLastFocusObservation = false; +#if defined(LORATRACE_BENCH_FAULTS) +// Bench images retain the pass's full histogram so a host can evaluate any +// candidate activity rule offline, from one run, instead of reflashing per +// guess. Production keeps only the fixed summary that reaches focus.csv. +FocusRssiHistogram lastFocusHistogram; +#endif + // Both writers and readers go through these so the locking cannot be // forgotten at a call site. Radio-task-internal code that already knows it // owns the radio still reads the raw fields directly; only the @@ -218,6 +231,20 @@ volatile uint32_t energyFailureCount = 0; volatile uint32_t energyRecoveryCount = 0; volatile uint32_t energyLastAwayMs = 0; +QueueHandle_t focusRequestQueue = nullptr; +QueueHandle_t focusObservationQueue = nullptr; +volatile bool focusActive = false; +// Reservation closes the cross-core window between accepting a Focus request +// and Core 1 taking it from its mailbox. Other bounded actions must refuse in +// this state, rather than queue behind Focus and run after it. +volatile bool focusPending = false; +volatile bool focusCancelRequested = false; +volatile FocusRuntimeState focusState = FocusRuntimeState::IDLE; +volatile uint32_t focusLastAwayMs = 0; +volatile uint32_t focusObservationCount = 0; +volatile uint32_t focusObservationDropCount = 0; +uint16_t focusNextId = 0; + // Phase 9 Pass B (research/phase9-sweep-pass-b-design.md). Cumulative // across sweeps, same convention as the counters above. volatile uint32_t passBAttemptCount = 0; @@ -551,6 +578,24 @@ bool energyAbortPending() { return false; } +bool focusAbortPending() { + return focusCancelRequested || + (profileSwitchQueue != nullptr && uxQueueMessagesWaiting(profileSwitchQueue) > 0) || + (pauseQueue != nullptr && uxQueueMessagesWaiting(pauseQueue) > 0); +} + +bool applyFocusBenchFault(BenchFaultPoint point, FocusRuntime &runtime) { + BenchFaultAction action; + if (!benchFaultTake(point, action)) return false; + if (action == BenchFaultAction::CANCEL) { + focusRuntimeCancel(runtime); + } else { + lastError = RADIOLIB_ERR_UNKNOWN; + focusRuntimeFail(runtime); + } + return true; +} + bool cellAbortPending() { if (cellCancelRequested) return true; if (profileSwitchQueue != nullptr && uxQueueMessagesWaiting(profileSwitchQueue) > 0) return true; @@ -650,6 +695,159 @@ bool restoreHomeListen(const ChannelParams &homeChannel, MissionProfile homeProf return true; } +void enqueueFocusObservation(const FocusRequest &request, const FocusRssiHistogram &histogram, + FocusRuntime &runtime, uint16_t focusId, bool restored, + int16_t operationStatus) { + FocusObservation observation; + observation.rx_millis = millis(); + observation.observation_ms = runtime.observation_ms; + observation.freq_mhz = focusRequestFrequencyMhz(request); + observation.focus_id = focusId; + observation.selection_bin_index = request.selection_bin_index; + observation.requested_dwell_ms = request.requested_dwell_ms; + observation.requested_samples = request.requested_samples; + observation.sample_count = histogram.sample_count; + observation.rssi_median_dbm_x10 = focusHistogramMedianDbmX10(histogram); + observation.rssi_p90_dbm_x10 = focusHistogramP90DbmX10(histogram); + observation.rssi_peak_dbm_x10 = histogram.peak_dbm_x10; + observation.profile = (uint8_t)activeProfile; + observation.requested_passes = request.requested_passes; + observation.valid_passes = (uint8_t)runtime.valid_passes; + observation.selection_source = request.selection_source; + observation.request_status = focusRuntimeFinishRestore(runtime, restored); + observation.home_restore = restored; + // Recovery must overwrite the live radio error to resume Watch, but the + // Focus row reports the request operation that led to this terminal state. + observation.radio_status = operationStatus; + portENTER_CRITICAL(&focusResultMux); + lastFocusObservation = observation; +#if defined(LORATRACE_BENCH_FAULTS) + lastFocusHistogram = histogram; +#endif + haveLastFocusObservation = true; + portEXIT_CRITICAL(&focusResultMux); + focusObservationCount++; + if (focusObservationQueue == nullptr || + xQueueSend(focusObservationQueue, &observation, 0) != pdTRUE) { + focusObservationDropCount++; + } +} + +void performFocusSurvey(const FocusRequest &request) { + if (focusActive || discoveryActive || energyActive || cellActive || scopeActive || + benchPassBCadActive || benchRssiWindowActive) { + focusPending = false; + return; + } + FocusRuntime runtime; + if (!focusRuntimeBegin(runtime, request)) { + focusPending = false; + return; + } + focusActive = true; + focusPending = false; + focusState = FocusRuntimeState::SURVEYING; + focusCancelRequested = false; + const uint32_t awayStarted = millis(); + const ChannelParams homeChannel = activeChannel; + const MissionProfile homeProfile = activeProfile; + const uint16_t focusId = ++focusNextId; + FocusRssiHistogram histogram; + const float frequency = focusRequestFrequencyMhz(request); + // Consumed here, not at the delay below, so an armed stall cannot survive + // a request that never reaches sampling and fire on a later one. Bench + // images only; production always returns 0 (bench_fault.h). + const uint16_t stallMs = benchFocusStallTakeMs(); + + const bool skipSampling = applyFocusBenchFault(BenchFaultPoint::BEFORE_RETUNE, runtime); + int beginState = RADIOLIB_ERR_NONE; + if (!skipSampling) { + SpiBusLock lock(BUS_WAIT); + if (!lock.held()) { + busMissCount++; + beginState = RADIOLIB_ERR_SPI_CMD_TIMEOUT; + } else { + beginState = radio.begin(frequency, homeChannel.bw_khz, homeChannel.sf, + homeChannel.cr_denom, homeChannel.sync_word); + if (beginState == RADIOLIB_ERR_NONE) beginState = radio.startReceive(); + } + lastError = beginState; + } + if (!skipSampling && beginState != RADIOLIB_ERR_NONE) { + lastError = beginState; + focusRuntimeFail(runtime); + } else if (!skipSampling && + !applyFocusBenchFault(BenchFaultPoint::AFTER_RETUNE, runtime)) { + const uint32_t passStarted = millis(); + // The dwell schedule alone does not bound radio-away time: every + // sample waits on the shared SPI bus, so a contended bus stretches + // the pass. Past this the request stops sampling and terminates as + // `timeout` rather than holding the radio (focus_plan.h). + const uint32_t deadline = passStarted + focusRequestTimeoutMs(request); + if (stallMs > 0) vTaskDelay(pdMS_TO_TICKS(stallMs)); + uint32_t firstSample = 0; + uint32_t lastSample = 0; + uint16_t passSamples = 0; + for (uint16_t sample = 0; sample < request.requested_samples; ++sample) { + if (focusAbortPending()) { + focusRuntimeCancel(runtime); + break; + } + if ((int32_t)(millis() - deadline) >= 0) { + focusRuntimeTimeout(runtime); + break; + } + const uint32_t target = passStarted + + (uint32_t)sample * request.requested_dwell_ms / (request.requested_samples - 1); + while ((int32_t)(millis() - target) < 0) { + if (focusAbortPending()) { + focusRuntimeCancel(runtime); + break; + } + if ((int32_t)(millis() - deadline) >= 0) { + focusRuntimeTimeout(runtime); + break; + } + vTaskDelay(1); + } + if (runtime.cancelled || runtime.timed_out) break; + float rssi = 0.0f; + bool gotSample = false; + { + SpiBusLock lock(BUS_WAIT); + if (lock.held()) { + rssi = radio.getRSSI(false); + gotSample = true; + } else { + busMissCount++; + } + } + if (!gotSample) { + focusRuntimeFail(runtime); + break; + } + const uint32_t sampledAt = millis(); + if (passSamples == 0) firstSample = sampledAt; + lastSample = sampledAt; + passSamples++; + focusHistogramAddSample(histogram, energyRssiDbmToFixed(rssi)); + } + if (!runtime.cancelled && !runtime.failed && passSamples == request.requested_samples) { + focusRuntimeNoteValidPass(runtime, lastSample - firstSample); + } + } + focusRuntimeBeginRestore(runtime); + int16_t operationStatus = (int16_t)lastError; + const bool restored = restoreHomeListen(homeChannel, homeProfile); + // A failed recovery is more actionable than the original request error. + if (!restored) operationStatus = (int16_t)lastError; + enqueueFocusObservation(request, histogram, runtime, focusId, restored, operationStatus); + focusState = runtime.state; + focusLastAwayMs = millis() - awayStarted; + focusCancelRequested = false; + focusActive = false; +} + void performDiscoverySweep() { if (discoveryActive || cellActive || scopeActive) return; @@ -1744,6 +1942,14 @@ void radioTask(void *) { continue; } + FocusRequest focusRequest; + if (focusRequestQueue != nullptr && + xQueueReceive(focusRequestQueue, &focusRequest, 0) == pdTRUE) { + if (!paused) performFocusSurvey(focusRequest); + else focusPending = false; // a concurrent pause must not reserve Focus forever + continue; + } + bool cellReq; if (cellSweepQueue != nullptr && xQueueReceive(cellSweepQueue, &cellReq, 0) == pdTRUE) { if (cellReq && !paused) { @@ -1825,7 +2031,7 @@ void radioTask(void *) { bool radioTaskStart(const ChannelParams &channel, MissionProfile profile, const ProfileOverrides &overrides, QueueHandle_t queue, QueueHandle_t scanQueue, QueueHandle_t energyQueue, - QueueHandle_t nodesQueue, QueueHandle_t cellQueue) { + QueueHandle_t nodesQueue, QueueHandle_t cellQueue, QueueHandle_t focusQueue) { // Boot-time and single-threaded (the task below does not exist yet), so // the lock is uncontended here -- taken anyway so every write to this // state goes through one path. @@ -1836,6 +2042,7 @@ bool radioTaskStart(const ChannelParams &channel, MissionProfile profile, energyObservationQueue = energyQueue; identityQueue = nodesQueue; cellObservationQueue = cellQueue; + focusObservationQueue = focusQueue; // Depth-1 mailbox for radioRequestProfileSwitch(). Created here, not // lazily, so a switch request right after boot can't race a @@ -1852,6 +2059,9 @@ bool radioTaskStart(const ChannelParams &channel, MissionProfile profile, energySweepQueue = xQueueCreate(1, sizeof(bool)); if (energySweepQueue == nullptr) return false; + focusRequestQueue = xQueueCreate(1, sizeof(FocusRequest)); + if (focusRequestQueue == nullptr) return false; + cellSweepQueue = xQueueCreate(1, sizeof(bool)); if (cellSweepQueue == nullptr) return false; @@ -2035,7 +2245,7 @@ bool radioRequestDiscoverySweep() { // none can preempt another; see radioRequestEnergySweep()'s/ // radioRequestCellSweep()'s/radioRequestScopeAcquire()'s matching // guards. - if (tracePaused || energyActive || cellActive || scopeActive) return false; + if (tracePaused || energyActive || cellActive || scopeActive || focusActive || focusPending) return false; const bool start = true; xQueueOverwrite(discoveryQueue, &start); xTaskNotifyGive(radioTaskHandle); @@ -2053,7 +2263,7 @@ bool radioRequestEnergySweep() { xTaskNotifyGive(radioTaskHandle); return true; } - if (tracePaused || discoveryActive || cellActive || scopeActive) return false; + if (tracePaused || discoveryActive || cellActive || scopeActive || focusActive || focusPending) return false; const bool start = true; xQueueOverwrite(energySweepQueue, &start); xTaskNotifyGive(radioTaskHandle); @@ -2082,7 +2292,8 @@ bool radioRequestEnergySweepRepeat() { } return true; } - if (energyActive || tracePaused || discoveryActive || cellActive || scopeActive) return false; + if (energyActive || tracePaused || discoveryActive || cellActive || scopeActive || focusActive || + focusPending) return false; energyRepeatActive = true; energyRepeatCount = 0; const bool start = true; @@ -2136,7 +2347,8 @@ bool radioRequestBenchPassBCadTrigger(uint8_t comboIndex) { if (!benchPassBCadTriggerAllowed()) return false; if (radioTaskHandle == nullptr || benchPassBCadQueue == nullptr) return false; if (comboIndex >= PASS_B_SF_BW_CANDIDATE_COUNT) return false; - if (benchPassBCadActive || energyActive || discoveryActive || tracePaused || scopeActive) { + if (benchPassBCadActive || energyActive || discoveryActive || tracePaused || scopeActive || + focusActive || focusPending) { return false; } xQueueOverwrite(benchPassBCadQueue, &comboIndex); @@ -2153,7 +2365,7 @@ bool radioRequestBenchRssiWindow(uint32_t freq_khz) { if (radioTaskHandle == nullptr || benchRssiWindowQueue == nullptr) return false; if (freq_khz < 860000UL || freq_khz > 930000UL) return false; if (benchRssiWindowActive || benchPassBCadActive || energyActive || discoveryActive || - cellActive || tracePaused || scopeActive) { + cellActive || tracePaused || scopeActive || focusActive || focusPending) { return false; } xQueueOverwrite(benchRssiWindowQueue, &freq_khz); @@ -2242,6 +2454,64 @@ uint32_t radioEnergyLastAwayMs() { return energyLastAwayMs; } +bool radioRequestFocusSurvey(const FocusRequest &request) { + if (!focusRequestIsValid(request) || radioTaskHandle == nullptr || focusRequestQueue == nullptr) { + return false; + } + if (focusActive) { + focusCancelRequested = true; + xTaskNotifyGive(radioTaskHandle); + return true; + } + if (focusPending || tracePaused || discoveryActive || energyActive || cellActive || scopeActive || + benchPassBCadActive || benchRssiWindowActive || + (discoveryQueue != nullptr && uxQueueMessagesWaiting(discoveryQueue) > 0) || + (energySweepQueue != nullptr && uxQueueMessagesWaiting(energySweepQueue) > 0) || + (cellSweepQueue != nullptr && uxQueueMessagesWaiting(cellSweepQueue) > 0) || + (scopeAcquireQueue != nullptr && uxQueueMessagesWaiting(scopeAcquireQueue) > 0) || + (benchPassBCadQueue != nullptr && uxQueueMessagesWaiting(benchPassBCadQueue) > 0) || + (benchRssiWindowQueue != nullptr && uxQueueMessagesWaiting(benchRssiWindowQueue) > 0)) { + return false; + } + if (xQueueOverwrite(focusRequestQueue, &request) != pdPASS) return false; + focusPending = true; + xTaskNotifyGive(radioTaskHandle); + return true; +} + +bool radioCancelFocusSurvey() { + if (!focusActive) return false; + focusCancelRequested = true; + xTaskNotifyGive(radioTaskHandle); + return true; +} + +bool radioFocusSurveyIsActive() { return focusActive; } +FocusRuntimeState radioFocusSurveyState() { return focusState; } +uint32_t radioFocusLastAwayMs() { return focusLastAwayMs; } +uint32_t radioFocusObservationCount() { return focusObservationCount; } +uint32_t radioFocusObservationDropCount() { return focusObservationDropCount; } +bool radioFocusLastHistogram(FocusRssiHistogram &out) { +#if !defined(LORATRACE_BENCH_FAULTS) + (void)out; + return false; +#else + portENTER_CRITICAL(&focusResultMux); + const bool available = haveLastFocusObservation; + if (available) out = lastFocusHistogram; + portEXIT_CRITICAL(&focusResultMux); + return available; +#endif +} + +bool radioFocusLastObservation(FocusObservation &out) { + portENTER_CRITICAL(&focusResultMux); + const bool available = haveLastFocusObservation; + if (available) out = lastFocusObservation; + portEXIT_CRITICAL(&focusResultMux); + return available; +} + uint32_t radioPassBAttemptCount() { return passBAttemptCount; } @@ -2259,7 +2529,7 @@ bool radioRequestCellSweep() { } // Mutually exclusive with Probe, Sweep, and Scope — same convention as // their own guards above. - if (tracePaused || discoveryActive || energyActive || scopeActive) return false; + if (tracePaused || discoveryActive || energyActive || scopeActive || focusActive || focusPending) return false; const bool start = true; xQueueOverwrite(cellSweepQueue, &start); xTaskNotifyGive(radioTaskHandle); @@ -2285,7 +2555,8 @@ bool radioRequestCellSweepRepeat() { } return true; } - if (cellActive || tracePaused || discoveryActive || energyActive || scopeActive) return false; + if (cellActive || tracePaused || discoveryActive || energyActive || scopeActive || focusActive || + focusPending) return false; cellRepeatActive = true; cellRepeatCount = 0; const bool start = true; @@ -2359,7 +2630,7 @@ bool radioRequestScopeAcquire(uint32_t freq_khz) { } // Mutually exclusive with Probe, Sweep, and Cell — same convention as // their own guards above. - if (tracePaused || discoveryActive || energyActive || cellActive) return false; + if (tracePaused || discoveryActive || energyActive || cellActive || focusActive || focusPending) return false; xQueueOverwrite(scopeAcquireQueue, &freq_khz); xTaskNotifyGive(radioTaskHandle); return true; diff --git a/src/radio_task.h b/src/radio_task.h index 88f8221..27abec4 100644 --- a/src/radio_task.h +++ b/src/radio_task.h @@ -23,6 +23,8 @@ #include "channel_plans.h" #include "detection.h" #include "energy_observation.h" +#include "focus_observation.h" +#include "focus_runtime.h" #include "node_identity.h" #include "region_plan.h" #include "scan_observation.h" @@ -45,7 +47,8 @@ bool radioTaskStart(const ChannelParams &channel, MissionProfile profile, const ProfileOverrides &overrides, QueueHandle_t queue, QueueHandle_t scanQueue, QueueHandle_t energyQueue, - QueueHandle_t identityQueue, QueueHandle_t cellQueue); + QueueHandle_t identityQueue, QueueHandle_t cellQueue, + QueueHandle_t focusQueue); void radioIdentityCaptureSetEnabled(bool enabled); bool radioIdentityCaptureIsEnabled(); @@ -239,6 +242,21 @@ uint32_t radioEnergyFailureCount(); uint32_t radioEnergyRecoveryCount(); uint32_t radioEnergyLastAwayMs(); +bool radioRequestFocusSurvey(const FocusRequest &request); +bool radioCancelFocusSurvey(); +bool radioFocusSurveyIsActive(); +FocusRuntimeState radioFocusSurveyState(); +uint32_t radioFocusLastAwayMs(); +uint32_t radioFocusObservationCount(); +uint32_t radioFocusObservationDropCount(); +// Copies the last completed Focus result under a short cross-core lock. +// It contains no GPS/run fields; Core 0 alone adds those while writing CSV. +bool radioFocusLastObservation(FocusObservation &out); +// Bench images only: the last pass's full RSSI histogram, so a host can test +// candidate activity rules offline rather than one reflash per candidate. +// Production always returns false. +bool radioFocusLastHistogram(FocusRssiHistogram &out); + // Phase 9 Pass B (research/phase9-sweep-pass-b-design.md): CAD attempts run // at the first PASS_B_MAX_PEAKS_PER_SWEEP Pass-A peaks this sweep, and how // many of those attempts promoted a real packet to Detection diff --git a/src/serial_control.cpp b/src/serial_control.cpp index 702485e..9265ddf 100644 --- a/src/serial_control.cpp +++ b/src/serial_control.cpp @@ -112,11 +112,14 @@ void sendStatus(uint16_t sequence) { // question that R= (Probe's recovery count, not a packet count // despite the letter) could not answer; distinct names on purpose so // a host script can't confuse them the way this session just did. - char argument[200] = {}; + // Sized from the protocol's own frame budget rather than by hand: this + // buffer was 240 against a 230-byte budget, which silently dropped the + // whole STATUS frame once the counters grew (serial_control_protocol.h). + char argument[SERIAL_CONTROL_ARGUMENT_MAX + 1] = {}; snprintf(argument, sizeof(argument), "P=%s;T=%u;B=%s;SD=%u;F=%lu;R=%lu;I=%u;N=%u;C=%u,%u,%u,%u;M=%04X;" "W=%s;WI=%u;WN=%u;WP=%u;PBA=%lu;PBD=%lu;BPC=%u;RW=%u;WIFI=%u;EA=%lu;" - "RXP=%lu;RXC=%lu", + "RXP=%lu;RXC=%lu;FS=%u;FA=%lu;FO=%lu;FD=%lu;FW=%lu;FL=%lu", profile, radioIsTracePaused() ? 0U : 1U, probe, loggerSdReady() ? 1U : 0U, (unsigned long)(channel.freq_mhz * 1000.0f), (unsigned long)radioDiscoveryRecoveryCount(), @@ -132,7 +135,12 @@ void sendStatus(uint16_t sequence) { (unsigned long)radioPassBAttemptCount(), (unsigned long)radioPassBDetectionCount(), radioBenchPassBCadIsActive() ? 1U : 0U, radioBenchRssiWindowIsActive() ? 1U : 0U, wifiIsEnabled() ? 1U : 0U, (unsigned long)radioEnergyLastAwayMs(), - (unsigned long)radioPacketCount(), (unsigned long)radioCrcErrorCount()); + (unsigned long)radioPacketCount(), (unsigned long)radioCrcErrorCount(), + (unsigned)radioFocusSurveyState(), (unsigned long)radioFocusLastAwayMs(), + (unsigned long)radioFocusObservationCount(), + (unsigned long)radioFocusObservationDropCount(), + (unsigned long)loggerFocusRowsWritten(), + (unsigned long)loggerFocusRowsDropped()); sendFrame(sequence, SerialControlOpcode::STATUS, argument); } @@ -320,6 +328,193 @@ void handleFrame(const SerialControlFrame &frame) { } break; } + case SerialControlOpcode::BENCH_FOCUS: { + if (!benchFocusSurveyTriggerAllowed()) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "UNSUPPORTED"); + break; + } + if (!loggerSdReady()) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "SD_REQUIRED"); + break; + } + char argument[40] = {}; + if (strlen(frame.argument) >= sizeof(argument)) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "BAD_ARGUMENT"); + break; + } + strcpy(argument, frame.argument); + char *a = argument; + char *b = strchr(a, ':'); + if (b == nullptr) { sendFrame(frame.sequence, SerialControlOpcode::ERROR, "BAD_ARGUMENT"); break; } + *b++ = '\0'; + char *c = strchr(b, ':'); + if (c == nullptr) { sendFrame(frame.sequence, SerialControlOpcode::ERROR, "BAD_ARGUMENT"); break; } + *c++ = '\0'; + // Optional 4th field: a one-shot sample-loop stall in ms, the only + // way to reach Focus's timeout on demand (bench_fault.h). Omitting + // it keeps the original bin:dwell:samples grammar working. + char *d = strchr(c, ':'); + if (d != nullptr) *d++ = '\0'; + uint32_t bin, dwell, samples, stall = 0; + if (!serialControlParseUint32(a, bin) || !serialControlParseUint32(b, dwell) || + !serialControlParseUint32(c, samples) || bin > UINT16_MAX || dwell > UINT16_MAX || + samples > UINT16_MAX || + (d != nullptr && !serialControlParseUint32(d, stall))) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "BAD_ARGUMENT"); + break; + } + if (d != nullptr && !benchFocusStallConfigure(stall)) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "BAD_ARGUMENT"); + break; + } + FocusRequest request; + request.region = radioEnergySweepRegion(); + request.selection_source = FocusSelectionSource::SWEEP_BIN; + request.selection_bin_index = (uint16_t)bin; + request.requested_dwell_ms = (uint16_t)dwell; + request.requested_samples = (uint16_t)samples; + request.requested_passes = FOCUS_BENCH_REQUESTED_PASSES; + const bool accepted = radioRequestFocusSurvey(request); + // A refused request never reaches Core 1's one-shot take, so the + // stall would otherwise fire on whichever request ran next. + if (!accepted) benchFocusStallConfigure(0); + sendFrame(frame.sequence, accepted ? SerialControlOpcode::ACK : SerialControlOpcode::ERROR, + accepted ? "QUEUED" : "UNAVAILABLE"); + break; + } + case SerialControlOpcode::BENCH_FOCUS_CANCEL: + if (!benchFocusSurveyTriggerAllowed()) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "UNSUPPORTED"); + } else if (radioCancelFocusSurvey()) { + sendFrame(frame.sequence, SerialControlOpcode::ACK, "CANCEL_QUEUED"); + } else { + sendFrame(frame.sequence, SerialControlOpcode::ACK, "IDLE"); + } + break; + case SerialControlOpcode::BENCH_FOCUS_RESULT: { + if (!benchFocusSurveyTriggerAllowed()) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "UNSUPPORTED"); + break; + } + FocusObservation observation; + if (!radioFocusLastObservation(observation)) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "NO_RESULT"); + break; + } + if (strcmp(frame.argument, "HEALTH") == 0) { + char argument[32] = {}; + snprintf(argument, sizeof(argument), "RS=%u;HR=%u;E=%d", + (unsigned)observation.request_status, + observation.home_restore ? 1U : 0U, + (int)observation.radio_status); + sendFrame(frame.sequence, SerialControlOpcode::ACK, argument); + break; + } + // Fixed-point dBm fields preserve exact device values without + // locale/float formatting or leaking CSV's GPS/run columns. + char argument[144] = {}; + snprintf(argument, sizeof(argument), + "ID=%u;BIN=%u;F=%lu;N=%u;MED=%d;P90=%d;MAX=%d;OBS=%lu;RS=%u;HR=%u;E=%d", + (unsigned)observation.focus_id, + (unsigned)observation.selection_bin_index, + (unsigned long)(observation.freq_mhz * 1000.0f), + (unsigned)observation.sample_count, + (int)observation.rssi_median_dbm_x10, + (int)observation.rssi_p90_dbm_x10, + (int)observation.rssi_peak_dbm_x10, + (unsigned long)observation.observation_ms, + (unsigned)observation.request_status, + observation.home_restore ? 1U : 0U, + (int)observation.radio_status); + sendFrame(frame.sequence, SerialControlOpcode::ACK, argument); + break; + } + case SerialControlOpcode::BENCH_FOCUS_COUNTS: { + // A ladder of counts above the pass's own median, so one run lets + // a host evaluate any adaptive activity rule offline instead of + // reflashing per candidate margin. Relative, not absolute: an + // absolute threshold was measured and rejected at field levels + // (docs/hardware-results/2026-09-04-phase12-focus-matrix.md). + if (!benchFocusSurveyTriggerAllowed()) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "UNSUPPORTED"); + break; + } + FocusRssiHistogram histogram; + if (!radioFocusLastHistogram(histogram)) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "NO_RESULT"); + break; + } + static const int16_t MARGINS_DBM_X10[] = {20, 40, 60, 80, 100, 150, 200}; + char argument[SERIAL_CONTROL_ARGUMENT_MAX + 1] = {}; + int written = snprintf(argument, sizeof(argument), "N=%u;MED=%d;P90=%d;MAX=%d", + (unsigned)histogram.sample_count, + (int)focusHistogramMedianDbmX10(histogram), + (int)focusHistogramP90DbmX10(histogram), + (int)histogram.peak_dbm_x10); + for (size_t i = 0; i < sizeof(MARGINS_DBM_X10) / sizeof(MARGINS_DBM_X10[0]); ++i) { + if (written < 0 || (size_t)written >= sizeof(argument)) break; + written += snprintf(argument + written, sizeof(argument) - (size_t)written, + ";C%d=%u", (int)(MARGINS_DBM_X10[i] / 10), + (unsigned)focusHistogramCountAboveMedian(histogram, + MARGINS_DBM_X10[i])); + } + sendFrame(frame.sequence, SerialControlOpcode::ACK, argument); + break; + } + case SerialControlOpcode::BENCH_ACTION: { + // Cell and Scope are menu-only actions in production, so a fixture + // has no other way to prove Focus refuses them and is refused by + // them (docs/research/phase12-survey-truth-design.md §8). This + // calls the same request functions the menu does -- both toggle to + // cancel while their own action is active -- and adds no radio + // behavior of its own. + if (!benchArbitrationTriggerAllowed()) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "UNSUPPORTED"); + break; + } + if (strcmp(frame.argument, "STATE") == 0) { + char argument[32] = {}; + snprintf(argument, sizeof(argument), "CELL=%u;SCOPE=%u;FOCUS=%u", + radioCellSweepIsActive() ? 1U : 0U, + radioScopeAcquireIsActive() ? 1U : 0U, + radioFocusSurveyIsActive() ? 1U : 0U); + sendFrame(frame.sequence, SerialControlOpcode::ACK, argument); + break; + } + const bool cell = strncmp(frame.argument, "CELL:", 5) == 0; + const bool scope = strncmp(frame.argument, "SCOPE:", 6) == 0; + if (!cell && !scope) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "BAD_ARGUMENT"); + break; + } + const char *verb = frame.argument + (cell ? 5 : 6); + const bool start = strcmp(verb, "START") == 0; + if (!start && strcmp(verb, "CANCEL") != 0) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "BAD_ARGUMENT"); + break; + } + const bool active = cell ? radioCellSweepIsActive() : radioScopeAcquireIsActive(); + if (start && active) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "ACTIVE"); + break; + } + if (!start && !active) { + sendFrame(frame.sequence, SerialControlOpcode::ACK, "IDLE"); + break; + } + // Scope parks at one frequency; the resolved home channel keeps the + // fixture from inventing one and keeps the hop short. + const uint32_t scopeFreqKhz = (uint32_t)(radioActiveChannel().freq_mhz * 1000.0f); + const bool accepted = cell ? radioRequestCellSweep() + : radioRequestScopeAcquire(scopeFreqKhz); + if (!accepted) { + sendFrame(frame.sequence, SerialControlOpcode::ERROR, "UNAVAILABLE"); + } else { + sendFrame(frame.sequence, SerialControlOpcode::ACK, + start ? "QUEUED" : "CANCEL_QUEUED"); + } + break; + } case SerialControlOpcode::BENCH_PASS_B_CAD_RESULT: { EnergyObservationResult result; if (!benchPassBCadLastResult(result)) { diff --git a/src/serial_control_protocol.h b/src/serial_control_protocol.h index 92036e6..a995f7b 100644 --- a/src/serial_control_protocol.h +++ b/src/serial_control_protocol.h @@ -9,15 +9,28 @@ #include #include -// 160 was tight even before WIFI_SET/EA (2026-09-02): STATUS's own -// argument alone can realistically approach 155-170 bytes once cumulative -// counters (PBA/PBD) grow past a few digits during a long-running session -// (the Phase 9 24-hour soak exit criterion, ROADMAP.md, being the exact -// case that would hit this) -- serialControlFormatFrame() silently drops -// the frame (returns 0, nothing sent) rather than truncating, so this -// would have failed quietly. 256 gives real headroom, not just enough for -// today's field set. -constexpr size_t SERIAL_CONTROL_FRAME_MAX = 256; +// STATUS's argument is what sizes this, and it only ever grows: 160 was tight +// before WIFI_SET/EA (2026-09-02) and 256 became tight again once Phase 12 +// added six FS/FA/FO/FD/FW/FL fields (2026-09-05). Counted against a +// long-running session's realistic digit counts -- not all-uint32-max -- +// STATUS reaches ~240 bytes of argument, past 256's ~230-byte budget. +// serialControlFormatFrame() drops an over-long frame silently (returns 0, +// nothing sent) rather than truncating, so overrunning this loses exactly +// the newest fields, with no error to notice. 384 restores real headroom. +constexpr size_t SERIAL_CONTROL_FRAME_MAX = 384; + +// Frames are "@LTRX/1 ", so an argument +// budget has to account for the rest. Derived, not eyeballed: the previous +// 240-byte STATUS buffer was set by hand and silently exceeded its frame. +constexpr size_t SERIAL_CONTROL_OPCODE_NAME_MAX = 23; // BENCH_PASS_B_CAD_RESULT +// A host test walks every opcode name against this, so adding a longer one +// fails there rather than silently overrunning buffers sized from it. +constexpr size_t SERIAL_CONTROL_FRAME_FIXED_BYTES = + 8 /* "@LTRX/1 " */ + 5 /* sequence */ + 1 + SERIAL_CONTROL_OPCODE_NAME_MAX + + 1 + 5 /* " " + 4 CRC digits */; +// Longest argument that still frames, for any opcode, with room for the NUL. +constexpr size_t SERIAL_CONTROL_ARGUMENT_MAX = + SERIAL_CONTROL_FRAME_MAX - SERIAL_CONTROL_FRAME_FIXED_BYTES - 1; // Enum value names below are the actual wire opcode strings (via // serialControlOpcodeName()/serialControlOpcodeFromName()) and stay stable @@ -44,6 +57,11 @@ enum class SerialControlOpcode : uint8_t { BENCH_RSSI_WINDOW, BENCH_RSSI_RESULT, BENCH_PASS_B_CAD_RESULT, + BENCH_FOCUS, + BENCH_FOCUS_CANCEL, + BENCH_FOCUS_RESULT, + BENCH_FOCUS_COUNTS, + BENCH_ACTION, KEY_DUMP, SD_RETRY, WIFI_SET, @@ -90,6 +108,11 @@ inline const char *serialControlOpcodeName(SerialControlOpcode opcode) { case SerialControlOpcode::BENCH_RSSI_WINDOW: return "BENCH_RSSI_WINDOW"; case SerialControlOpcode::BENCH_RSSI_RESULT: return "BENCH_RSSI_RESULT"; case SerialControlOpcode::BENCH_PASS_B_CAD_RESULT: return "BENCH_PASS_B_CAD_RESULT"; + case SerialControlOpcode::BENCH_FOCUS: return "BENCH_FOCUS"; + case SerialControlOpcode::BENCH_FOCUS_CANCEL: return "BENCH_FOCUS_CANCEL"; + case SerialControlOpcode::BENCH_FOCUS_RESULT: return "BENCH_FOCUS_RESULT"; + case SerialControlOpcode::BENCH_FOCUS_COUNTS: return "BENCH_FOCUS_COUNTS"; + case SerialControlOpcode::BENCH_ACTION: return "BENCH_ACTION"; case SerialControlOpcode::KEY_DUMP: return "KEY_DUMP"; case SerialControlOpcode::SD_RETRY: return "SD_RETRY"; case SerialControlOpcode::WIFI_SET: return "WIFI_SET"; @@ -120,6 +143,11 @@ inline SerialControlOpcode serialControlOpcodeFromName(const char *name) { if (strcmp(name, "BENCH_RSSI_WINDOW") == 0) return SerialControlOpcode::BENCH_RSSI_WINDOW; if (strcmp(name, "BENCH_RSSI_RESULT") == 0) return SerialControlOpcode::BENCH_RSSI_RESULT; if (strcmp(name, "BENCH_PASS_B_CAD_RESULT") == 0) return SerialControlOpcode::BENCH_PASS_B_CAD_RESULT; + if (strcmp(name, "BENCH_FOCUS") == 0) return SerialControlOpcode::BENCH_FOCUS; + if (strcmp(name, "BENCH_FOCUS_CANCEL") == 0) return SerialControlOpcode::BENCH_FOCUS_CANCEL; + if (strcmp(name, "BENCH_FOCUS_RESULT") == 0) return SerialControlOpcode::BENCH_FOCUS_RESULT; + if (strcmp(name, "BENCH_FOCUS_COUNTS") == 0) return SerialControlOpcode::BENCH_FOCUS_COUNTS; + if (strcmp(name, "BENCH_ACTION") == 0) return SerialControlOpcode::BENCH_ACTION; if (strcmp(name, "KEY_DUMP") == 0) return SerialControlOpcode::KEY_DUMP; if (strcmp(name, "SD_RETRY") == 0) return SerialControlOpcode::SD_RETRY; if (strcmp(name, "WIFI_SET") == 0) return SerialControlOpcode::WIFI_SET; @@ -208,11 +236,14 @@ inline size_t serialControlFormatFrame(char *out, size_t outSize, uint16_t seque argument == nullptr || strchr(argument, ' ') != nullptr) { return 0; } - char body[SERIAL_CONTROL_FRAME_MAX] = {}; - const int bodyLen = snprintf(body, sizeof(body), "@LTRX/1 %u %s %s", + // Built in place: a second SERIAL_CONTROL_FRAME_MAX staging buffer cost + // the 4KB UI task stack as much as the frame itself, and the CRC covers + // the body prefix either way. + const int bodyLen = snprintf(out, outSize, "@LTRX/1 %u %s %s", (unsigned)sequence, serialControlOpcodeName(opcode), argument); - if (bodyLen < 0 || (size_t)bodyLen >= sizeof(body)) return 0; - const int total = snprintf(out, outSize, "%s %04X", body, - (unsigned)serialControlCrc16(body, (size_t)bodyLen)); - return total >= 0 && (size_t)total < outSize ? (size_t)total : 0; + if (bodyLen < 0 || (size_t)bodyLen >= outSize) return 0; + const int crcLen = snprintf(out + bodyLen, outSize - (size_t)bodyLen, " %04X", + (unsigned)serialControlCrc16(out, (size_t)bodyLen)); + if (crcLen < 0 || (size_t)(bodyLen + crcLen) >= outSize) return 0; + return (size_t)(bodyLen + crcLen); } diff --git a/src/version.h b/src/version.h index 3ff43ae..858a756 100644 --- a/src/version.h +++ b/src/version.h @@ -4,11 +4,12 @@ // // TWO different things live here, deliberately kept separate: // -// 1. FIRMWARE_VERSION — the semantic version, bumped BY HAND. MAJOR.MINOR -// tracks the build-order phase reached (docs/ROADMAP.md Versioning); PATCH is -// for fixes that add no phase scope. This is a *statement* that a phase -// was reached, so it must not auto-increment — a number that changes on -// every build asserts nothing. +// 1. FIRMWARE_VERSION — the semantic version, bumped BY HAND. In completed +// v1, MAJOR.MINOR tracked the build-order phase reached and PATCH covered +// fixes adding no phase scope; V2 uses docs/ROADMAP.md's workstream/release +// policy. This is a *statement* that a release gate was reached, so it +// must not auto-increment — a number that changes on every build asserts +// nothing. // // **Bump this when a phase lands, and before pushing the matching // `vX.Y.Z` tag.** CI enforces the tag/version match in release.yml, so a diff --git a/test/test_focus_observation/test_main.cpp b/test/test_focus_observation/test_main.cpp new file mode 100644 index 0000000..3871766 --- /dev/null +++ b/test/test_focus_observation/test_main.cpp @@ -0,0 +1,189 @@ +#include + +#include + +#include "../../src/focus_observation.h" + +void test_focus_working_state_stays_below_one_result_budget() { + TEST_ASSERT_EQUAL_size_t(148, sizeof(FocusRssiHistogram)); + TEST_ASSERT_EQUAL_size_t(40, sizeof(FocusObservation)); + TEST_ASSERT_EQUAL_size_t(188, sizeof(FocusRssiHistogram) + sizeof(FocusObservation)); +} + +void test_focus_histogram_reports_median_p90_and_peak_without_raw_samples() { + FocusRssiHistogram histogram; + focusHistogramAddSample(histogram, -1000); + focusHistogramAddSample(histogram, -900); + focusHistogramAddSample(histogram, -800); + focusHistogramAddSample(histogram, -700); + focusHistogramAddSample(histogram, -600); + + TEST_ASSERT_EQUAL_UINT16(5, histogram.sample_count); + TEST_ASSERT_EQUAL_INT16(-800, focusHistogramMedianDbmX10(histogram)); + TEST_ASSERT_EQUAL_INT16(-600, focusHistogramP90DbmX10(histogram)); + TEST_ASSERT_EQUAL_INT16(-600, histogram.peak_dbm_x10); +} + +void test_focus_histogram_rounds_to_one_db_and_flags_clamped_input() { + FocusRssiHistogram histogram; + focusHistogramAddSample(histogram, -1005); // nearest 1dB bin: -101dBm + focusHistogramAddSample(histogram, -1600); // below documented range + focusHistogramAddSample(histogram, 50); // above documented range + + TEST_ASSERT_EQUAL_INT16(-1010, focusHistogramMedianDbmX10(histogram)); + TEST_ASSERT_TRUE(histogram.flags & FOCUS_RSSI_FLAG_UNDERFLOW); + TEST_ASSERT_TRUE(histogram.flags & FOCUS_RSSI_FLAG_OVERFLOW); +} + +void test_focus_histogram_refuses_quantiles_after_bucket_saturation() { + FocusRssiHistogram histogram; + for (uint16_t i = 0; i < 256; ++i) focusHistogramAddSample(histogram, -800); + TEST_ASSERT_TRUE(histogram.flags & FOCUS_RSSI_FLAG_BUCKET_OVERFLOW); + TEST_ASSERT_EQUAL_INT16(FOCUS_RSSI_NO_SAMPLE_DBM_X10, + focusHistogramMedianDbmX10(histogram)); +} + +void test_focus_request_status_names_are_explicit() { + TEST_ASSERT_EQUAL_STRING("complete", focusRequestStatusName(FocusRequestStatus::COMPLETE)); + TEST_ASSERT_EQUAL_STRING("cancelled", focusRequestStatusName(FocusRequestStatus::CANCELLED)); + TEST_ASSERT_EQUAL_STRING("timeout", focusRequestStatusName(FocusRequestStatus::TIMEOUT)); + TEST_ASSERT_EQUAL_STRING("failed", focusRequestStatusName(FocusRequestStatus::FAILED)); +} + +void test_focus_csv_with_fix_persists_blank_coverage_and_raw_counts() { + FocusObservation observation; + observation.rx_millis = 555000; + observation.observation_ms = 470; + observation.freq_mhz = 912.75f; + observation.focus_id = 7; + observation.selection_bin_index = 43; + observation.requested_dwell_ms = 500; + observation.requested_samples = 4; + observation.sample_count = 4; + observation.qualifying_count = 0; + observation.rssi_median_dbm_x10 = -920; + observation.rssi_p90_dbm_x10 = -850; + observation.rssi_peak_dbm_x10 = -850; + observation.profile = (uint8_t)MissionProfile::GENERAL_EXPLORATION; + observation.requested_passes = 1; + observation.valid_passes = 1; + observation.selection_source = FocusSelectionSource::SWEEP_BIN; + observation.request_status = FocusRequestStatus::COMPLETE; + observation.home_restore = true; + + char row[256]; + const size_t n = focusObservationFormatCsv(observation, row, sizeof(row), + "2026-09-04T10:00:00Z", true, + 45.5, -122.6, 1, 3); + TEST_ASSERT_TRUE(n > 0); + TEST_ASSERT_EQUAL_STRING( + "2026-09-04T10:00:00Z,45.500000,-122.600000,1,3,555000,general,7,sweep,43," + "912.750,1,1,500,470,4,4,-92.0,-85.0,-85.0,0,,complete,1,0,0", row); +} + +void test_focus_csv_without_fix_or_samples_keeps_unknown_values_blank() { + FocusObservation observation; + observation.rx_millis = 777; + observation.profile = (uint8_t)MissionProfile::RETICULUM; + observation.request_status = FocusRequestStatus::FAILED; + observation.radio_status = -2; + + char row[256]; + TEST_ASSERT_TRUE(focusObservationFormatCsv(observation, row, sizeof(row), "", false, + 0.0, 0.0, 0, 1) > 0); + TEST_ASSERT_NOT_NULL(strstr(row, ",,,0,1,777,reticulum,")); + TEST_ASSERT_NULL(strstr(row, "0.000000")); + TEST_ASSERT_NOT_NULL(strstr(row, ",0,0,,,,0,,failed,")); +} + +void test_focus_csv_truncation_is_reported() { + FocusObservation observation; + char row[8]; + TEST_ASSERT_EQUAL_size_t(0, focusObservationFormatCsv(observation, row, sizeof(row), + "timestamp", false, + 0.0, 0.0, 0, 1)); +} + +void test_worst_case_row_fits_the_logger_buffer_with_margin() { + // Every field at its widest, including widths the bounded request cannot + // actually produce. An over-long row is dropped rather than truncated, so + // this must hold at the maximum, not at today's typical bench values. + FocusObservation observation; + observation.rx_millis = 4294967295u; + observation.observation_ms = 4294967295u; + observation.freq_mhz = 928.1234f; + observation.focus_id = 65535; + observation.selection_bin_index = 65535; + observation.requested_dwell_ms = 65535; + observation.requested_samples = 65535; + observation.sample_count = 65535; + observation.qualifying_count = 65535; + observation.rssi_median_dbm_x10 = -1405; + observation.rssi_p90_dbm_x10 = -1405; + observation.rssi_peak_dbm_x10 = -1405; + observation.radio_status = -32768; + observation.profile = 200; // longest name path: "unknown" + observation.requested_passes = 255; + observation.valid_passes = 255; + observation.selection_source = FocusSelectionSource::WATERFALL_BIN; // "waterfall" + observation.request_status = FocusRequestStatus::CANCELLED; // "cancelled" + observation.home_restore = true; + observation.wifi_on = true; + + char row[FOCUS_CSV_ROW_MAX]; + const size_t n = focusObservationFormatCsv(observation, row, sizeof(row), + "2026-09-04T21:57:36Z", true, + -179.987654, -179.987654, 255, 65535); + TEST_ASSERT_TRUE(n > 0); + // The logger appends a newline at row[n], so n + 1 must still fit. + TEST_ASSERT_TRUE(n + 1 <= FOCUS_CSV_ROW_MAX); + // Measured at 189 bytes; fail loudly if a schema change eats the margin. + TEST_ASSERT_TRUE(n <= 200); +} + +void test_sample_counts_integrate_where_a_peak_would_not() { + FocusRssiHistogram histogram; + // A quiet pass with one stray strong sample: peak says -40, but only one + // sample in 50 is anywhere near it. A count is what tells those apart. + for (int i = 0; i < 49; ++i) focusHistogramAddSample(histogram, -1000); + focusHistogramAddSample(histogram, -400); + TEST_ASSERT_EQUAL_INT16(-400, histogram.peak_dbm_x10); + TEST_ASSERT_EQUAL_UINT16(50, focusHistogramCountAtLeast(histogram, -1000)); + TEST_ASSERT_EQUAL_UINT16(1, focusHistogramCountAtLeast(histogram, -900)); + TEST_ASSERT_EQUAL_UINT16(1, focusHistogramCountAboveMedian(histogram, 60)); + TEST_ASSERT_EQUAL_UINT16(0, focusHistogramCountAtLeast(histogram, -300)); + + // A genuinely busy pass reads the same peak but many elevated samples. + FocusRssiHistogram busy; + for (int i = 0; i < 25; ++i) focusHistogramAddSample(busy, -1000); + for (int i = 0; i < 25; ++i) focusHistogramAddSample(busy, -400); + TEST_ASSERT_EQUAL_INT16(-400, busy.peak_dbm_x10); + TEST_ASSERT_EQUAL_UINT16(25, focusHistogramCountAboveMedian(busy, 60)); +} + +void test_count_thresholds_clamp_outside_the_histogram_range() { + FocusRssiHistogram histogram; + for (int i = 0; i < 10; ++i) focusHistogramAddSample(histogram, -1000); + // Below the range counts everything; above it counts nothing. + TEST_ASSERT_EQUAL_UINT16(10, focusHistogramCountAtLeast(histogram, -2000)); + TEST_ASSERT_EQUAL_UINT16(0, focusHistogramCountAtLeast(histogram, 100)); + // No samples at all must not report a phantom count. + FocusRssiHistogram empty; + TEST_ASSERT_EQUAL_UINT16(0, focusHistogramCountAboveMedian(empty, 60)); +} + +int main(int argc, char **argv) { + UNITY_BEGIN(); + RUN_TEST(test_focus_working_state_stays_below_one_result_budget); + RUN_TEST(test_focus_histogram_reports_median_p90_and_peak_without_raw_samples); + RUN_TEST(test_focus_histogram_rounds_to_one_db_and_flags_clamped_input); + RUN_TEST(test_focus_histogram_refuses_quantiles_after_bucket_saturation); + RUN_TEST(test_focus_request_status_names_are_explicit); + RUN_TEST(test_focus_csv_with_fix_persists_blank_coverage_and_raw_counts); + RUN_TEST(test_focus_csv_without_fix_or_samples_keeps_unknown_values_blank); + RUN_TEST(test_focus_csv_truncation_is_reported); + RUN_TEST(test_worst_case_row_fits_the_logger_buffer_with_margin); + RUN_TEST(test_sample_counts_integrate_where_a_peak_would_not); + RUN_TEST(test_count_thresholds_clamp_outside_the_histogram_range); + return UNITY_END(); +} diff --git a/test/test_focus_plan/test_main.cpp b/test/test_focus_plan/test_main.cpp new file mode 100644 index 0000000..ee2750a --- /dev/null +++ b/test/test_focus_plan/test_main.cpp @@ -0,0 +1,114 @@ +#include + +#include "../../src/focus_observation.h" + +void test_focus_request_is_small_and_starts_with_one_selected_bin() { + TEST_ASSERT_EQUAL_UINT8(1, FOCUS_SELECTED_BIN_COUNT); + TEST_ASSERT_TRUE(sizeof(FocusRequest) <= 16); +} + +void test_focus_selection_sources_are_explicit_and_stable() { + TEST_ASSERT_EQUAL_STRING("sweep", focusSelectionSourceName(FocusSelectionSource::SWEEP_BIN)); + TEST_ASSERT_EQUAL_STRING("waterfall", focusSelectionSourceName(FocusSelectionSource::WATERFALL_BIN)); + TEST_ASSERT_EQUAL_STRING("preset", focusSelectionSourceName(FocusSelectionSource::PRESET)); +} + +void test_focus_request_resolves_a_sourced_energy_bin_frequency() { + FocusRequest request; + request.region = Region::US; + request.bin_step = EnergyBinStep::KHZ_250; + request.selection_bin_index = 13; + request.requested_dwell_ms = 100; + request.requested_samples = 4; + request.requested_passes = FOCUS_BENCH_REQUESTED_PASSES; + request.selection_source = FocusSelectionSource::SWEEP_BIN; + + TEST_ASSERT_TRUE(focusRequestIsValid(request)); + TEST_ASSERT_FLOAT_WITHIN(0.001f, 905.25f, focusRequestFrequencyMhz(request)); +} + +void test_focus_request_rejects_empty_or_out_of_band_work() { + FocusRequest request; + request.region = Region::US; + request.bin_step = EnergyBinStep::KHZ_250; + request.selection_bin_index = energyBinCount(ENERGY_SWEEP_BAND_US, request.bin_step); + request.requested_dwell_ms = 100; + request.requested_samples = FOCUS_BENCH_SAMPLES_MIN; + request.requested_passes = FOCUS_BENCH_REQUESTED_PASSES; + TEST_ASSERT_FALSE(focusRequestIsValid(request)); + + request.selection_bin_index = 0; + request.requested_dwell_ms = FOCUS_BENCH_DWELL_MIN_MS - 1; + TEST_ASSERT_FALSE(focusRequestIsValid(request)); + request.requested_dwell_ms = FOCUS_BENCH_DWELL_MAX_MS + 1; + TEST_ASSERT_FALSE(focusRequestIsValid(request)); + request.requested_dwell_ms = 100; + request.requested_samples = FOCUS_BENCH_SAMPLES_MIN - 1; + TEST_ASSERT_FALSE(focusRequestIsValid(request)); + request.requested_samples = FOCUS_BENCH_SAMPLES_MAX + 1; + TEST_ASSERT_FALSE(focusRequestIsValid(request)); + request.requested_samples = FOCUS_BENCH_SAMPLES_MIN; + request.requested_passes = 0; + TEST_ASSERT_FALSE(focusRequestIsValid(request)); +} + +void test_sample_ceiling_keeps_every_quantile_exact() { + // All samples landing in one 1 dB bucket is the worst case; above 255 the + // bucket saturates and the histogram refuses to report a percentile. + FocusRssiHistogram histogram; + for (uint16_t i = 0; i < FOCUS_BENCH_SAMPLES_MAX; ++i) { + focusHistogramAddSample(histogram, -1000); + } + TEST_ASSERT_TRUE(focusHistogramHasExactQuantiles(histogram)); + TEST_ASSERT_EQUAL_INT16(-1000, focusHistogramP90DbmX10(histogram)); +} + +void test_focus_request_is_bounded_in_time_not_only_in_samples() { + FocusRequest request; + request.region = Region::US; + request.bin_step = EnergyBinStep::KHZ_250; + request.selection_bin_index = 43; + request.requested_dwell_ms = 500; + request.requested_samples = 8; + request.requested_passes = FOCUS_BENCH_REQUESTED_PASSES; + request.selection_source = FocusSelectionSource::SWEEP_BIN; + + TEST_ASSERT_TRUE(focusRequestIsValid(request)); + // The deadline must exceed the dwell it bounds, or a healthy request + // would report `timeout` instead of what it actually observed. + TEST_ASSERT_TRUE(focusRequestTimeoutMs(request) > request.requested_dwell_ms); + TEST_ASSERT_EQUAL_UINT32(500u + FOCUS_REQUEST_TIMEOUT_SLACK_MS, + focusRequestTimeoutMs(request)); + + request.requested_dwell_ms = FOCUS_BENCH_DWELL_MAX_MS; + TEST_ASSERT_EQUAL_UINT32(FOCUS_MAX_SAMPLING_MS, focusRequestTimeoutMs(request)); + TEST_ASSERT_TRUE(FOCUS_MAX_SAMPLING_MS <= 3000); +} + +void test_sampling_policy_scales_with_dwell_and_stays_bounded() { + // The measured policy: spacing stays fixed, so the count follows the dwell. + TEST_ASSERT_EQUAL_UINT16(6, focusSamplesForDwell(100)); + TEST_ASSERT_EQUAL_UINT16(26, focusSamplesForDwell(500)); + TEST_ASSERT_EQUAL_UINT16(101, focusSamplesForDwell(2000)); + + // Actual spacing must never exceed the policy at any legal dwell -- that + // is the property the 2026-09-04 sweep bought, not the sample counts. + for (uint16_t dwell = FOCUS_BENCH_DWELL_MIN_MS; dwell <= FOCUS_BENCH_DWELL_MAX_MS; ++dwell) { + const uint16_t samples = focusSamplesForDwell(dwell); + TEST_ASSERT_TRUE(samples >= FOCUS_BENCH_SAMPLES_MIN); + TEST_ASSERT_TRUE(samples <= FOCUS_BENCH_SAMPLES_MAX); + TEST_ASSERT_TRUE(dwell / (samples - 1) <= FOCUS_SAMPLE_SPACING_MS); + } +} + +int main(int argc, char **argv) { + UNITY_BEGIN(); + RUN_TEST(test_focus_request_is_small_and_starts_with_one_selected_bin); + RUN_TEST(test_focus_selection_sources_are_explicit_and_stable); + RUN_TEST(test_focus_request_resolves_a_sourced_energy_bin_frequency); + RUN_TEST(test_focus_request_rejects_empty_or_out_of_band_work); + RUN_TEST(test_focus_request_is_bounded_in_time_not_only_in_samples); + RUN_TEST(test_sample_ceiling_keeps_every_quantile_exact); + RUN_TEST(test_sampling_policy_scales_with_dwell_and_stays_bounded); + return UNITY_END(); +} diff --git a/test/test_focus_runtime/test_main.cpp b/test/test_focus_runtime/test_main.cpp new file mode 100644 index 0000000..c0f5809 --- /dev/null +++ b/test/test_focus_runtime/test_main.cpp @@ -0,0 +1,74 @@ +#include + +#include "../../src/focus_runtime.h" + +FocusRequest validRequest() { + FocusRequest request; + request.region = Region::US; + request.selection_bin_index = 13; + request.requested_dwell_ms = 100; + request.requested_samples = 4; + request.requested_passes = FOCUS_BENCH_REQUESTED_PASSES; + return request; +} + +void test_focus_runtime_only_starts_a_bounded_valid_request() { + FocusRuntime runtime; + TEST_ASSERT_TRUE(focusRuntimeBegin(runtime, validRequest())); + TEST_ASSERT_EQUAL((int)FocusRuntimeState::SURVEYING, (int)runtime.state); + TEST_ASSERT_FALSE(focusRuntimeBegin(runtime, validRequest())); +} + +void test_focus_runtime_publishes_complete_only_after_restore() { + FocusRuntime runtime; + TEST_ASSERT_TRUE(focusRuntimeBegin(runtime, validRequest())); + focusRuntimeNoteValidPass(runtime, 93); + focusRuntimeBeginRestore(runtime); + TEST_ASSERT_EQUAL((int)FocusRequestStatus::COMPLETE, + (int)focusRuntimeFinishRestore(runtime, true)); + TEST_ASSERT_EQUAL_UINT16(1, runtime.valid_passes); + TEST_ASSERT_EQUAL_UINT32(93, runtime.observation_ms); +} + +void test_focus_runtime_restore_failure_overrides_cancel_and_timeout() { + FocusRuntime runtime; + TEST_ASSERT_TRUE(focusRuntimeBegin(runtime, validRequest())); + focusRuntimeCancel(runtime); + focusRuntimeTimeout(runtime); + focusRuntimeBeginRestore(runtime); + TEST_ASSERT_EQUAL((int)FocusRequestStatus::FAILED, + (int)focusRuntimeFinishRestore(runtime, false)); +} + +void test_focus_runtime_timeout_publishes_timeout_not_complete() { + FocusRuntime runtime; + TEST_ASSERT_TRUE(focusRuntimeBegin(runtime, validRequest())); + focusRuntimeTimeout(runtime); + focusRuntimeBeginRestore(runtime); + // A recovered timeout is still a timeout: it must not borrow `complete` + // from a restore that worked, and it recorded no valid pass. + TEST_ASSERT_EQUAL((int)FocusRequestStatus::TIMEOUT, + (int)focusRuntimeFinishRestore(runtime, true)); + TEST_ASSERT_EQUAL_UINT16(0, runtime.valid_passes); + TEST_ASSERT_EQUAL_UINT32(0, runtime.observation_ms); +} + +void test_focus_runtime_cancel_outranks_a_concurrent_timeout() { + FocusRuntime runtime; + TEST_ASSERT_TRUE(focusRuntimeBegin(runtime, validRequest())); + focusRuntimeCancel(runtime); + focusRuntimeTimeout(runtime); + focusRuntimeBeginRestore(runtime); + TEST_ASSERT_EQUAL((int)FocusRequestStatus::CANCELLED, + (int)focusRuntimeFinishRestore(runtime, true)); +} + +int main(int argc, char **argv) { + UNITY_BEGIN(); + RUN_TEST(test_focus_runtime_only_starts_a_bounded_valid_request); + RUN_TEST(test_focus_runtime_publishes_complete_only_after_restore); + RUN_TEST(test_focus_runtime_restore_failure_overrides_cancel_and_timeout); + RUN_TEST(test_focus_runtime_timeout_publishes_timeout_not_complete); + RUN_TEST(test_focus_runtime_cancel_outranks_a_concurrent_timeout); + return UNITY_END(); +} diff --git a/test/test_serial_control_protocol/test_main.cpp b/test/test_serial_control_protocol/test_main.cpp index 8726134..ab30dd3 100644 --- a/test/test_serial_control_protocol/test_main.cpp +++ b/test/test_serial_control_protocol/test_main.cpp @@ -98,6 +98,78 @@ void test_bench_pass_b_cad_opcode_round_trips() { TEST_ASSERT_EQUAL_STRING("7", frame.argument); } +void test_bench_focus_opcode_round_trips() { + char line[SERIAL_CONTROL_FRAME_MAX]; + TEST_ASSERT_TRUE(serialControlFormatFrame(line, sizeof(line), 24, + SerialControlOpcode::BENCH_FOCUS, + "43:500:8") > 0); + SerialControlFrame frame; + TEST_ASSERT_TRUE(serialControlParseFrame(line, frame)); + TEST_ASSERT_EQUAL_INT((int)SerialControlOpcode::BENCH_FOCUS, (int)frame.opcode); + TEST_ASSERT_EQUAL_STRING("43:500:8", frame.argument); +} + +void test_bench_focus_result_opcode_round_trips() { + char line[SERIAL_CONTROL_FRAME_MAX]; + TEST_ASSERT_TRUE(serialControlFormatFrame(line, sizeof(line), 25, + SerialControlOpcode::BENCH_FOCUS_RESULT, + "-") > 0); + SerialControlFrame frame; + TEST_ASSERT_TRUE(serialControlParseFrame(line, frame)); + TEST_ASSERT_EQUAL_INT((int)SerialControlOpcode::BENCH_FOCUS_RESULT, (int)frame.opcode); + TEST_ASSERT_EQUAL_STRING("-", frame.argument); +} + +void test_bench_focus_cancel_opcode_round_trips() { + char line[SERIAL_CONTROL_FRAME_MAX]; + TEST_ASSERT_TRUE(serialControlFormatFrame(line, sizeof(line), 26, + SerialControlOpcode::BENCH_FOCUS_CANCEL, + "-") > 0); + SerialControlFrame frame; + TEST_ASSERT_TRUE(serialControlParseFrame(line, frame)); + TEST_ASSERT_EQUAL_INT((int)SerialControlOpcode::BENCH_FOCUS_CANCEL, (int)frame.opcode); + TEST_ASSERT_EQUAL_STRING("-", frame.argument); +} + +void test_bench_action_opcode_round_trips() { + char line[SERIAL_CONTROL_FRAME_MAX]; + TEST_ASSERT_TRUE(serialControlFormatFrame(line, sizeof(line), 27, + SerialControlOpcode::BENCH_ACTION, + "CELL:START") > 0); + SerialControlFrame frame; + TEST_ASSERT_TRUE(serialControlParseFrame(line, frame)); + TEST_ASSERT_EQUAL_INT((int)SerialControlOpcode::BENCH_ACTION, (int)frame.opcode); + TEST_ASSERT_EQUAL_STRING("CELL:START", frame.argument); +} + +void test_argument_budget_covers_the_longest_opcode_name() { + // SERIAL_CONTROL_ARGUMENT_MAX is derived from this; a longer opcode name + // added later would quietly overrun every buffer sized from it. + for (int value = (int)SerialControlOpcode::HELLO; value <= (int)SerialControlOpcode::ERROR; + ++value) { + const char *name = serialControlOpcodeName((SerialControlOpcode)value); + TEST_ASSERT_TRUE(strlen(name) <= SERIAL_CONTROL_OPCODE_NAME_MAX); + } +} + +void test_formatter_frames_a_saturated_status_argument() { + // STATUS is the longest frame the device emits, and an over-long one is + // dropped silently rather than truncated -- so the budget has to hold at + // the maximum, not at today's typical field widths. + char argument[SERIAL_CONTROL_ARGUMENT_MAX + 2]; + memset(argument, 'x', sizeof(argument) - 1); + argument[sizeof(argument) - 1] = '\0'; + + char line[SERIAL_CONTROL_FRAME_MAX]; + TEST_ASSERT_EQUAL_UINT(0, serialControlFormatFrame(line, sizeof(line), 65535, + SerialControlOpcode::BENCH_PASS_B_CAD_RESULT, + argument)); + argument[SERIAL_CONTROL_ARGUMENT_MAX] = '\0'; + TEST_ASSERT_TRUE(serialControlFormatFrame(line, sizeof(line), 65535, + SerialControlOpcode::BENCH_PASS_B_CAD_RESULT, + argument) > 0); +} + int main(int argc, char **argv) { UNITY_BEGIN(); RUN_TEST(test_valid_frame_round_trips); @@ -109,5 +181,11 @@ int main(int argc, char **argv) { RUN_TEST(test_sweep_opcodes_round_trip); RUN_TEST(test_sd_retry_opcode_round_trips); RUN_TEST(test_bench_pass_b_cad_opcode_round_trips); + RUN_TEST(test_bench_focus_opcode_round_trips); + RUN_TEST(test_bench_focus_result_opcode_round_trips); + RUN_TEST(test_bench_focus_cancel_opcode_round_trips); + RUN_TEST(test_bench_action_opcode_round_trips); + RUN_TEST(test_argument_budget_covers_the_longest_opcode_name); + RUN_TEST(test_formatter_frames_a_saturated_status_argument); return UNITY_END(); }