v0.4.6 RC: web observability + heap hardening

README.md

@@ -189,6 +189,209 @@ Some hotspot implementations isolate connected clients from the hotspot host or
 
 The ESP32-C5 has one Wi-Fi radio. Active dual-band scouting temporarily visits channels away from the associated AP. BandiC5 explicitly uses Espressif's home-channel dwell between scan channels to help preserve the infrastructure connection, but brief latency during scans is expected. Uplink monitoring is intentionally low-bandwidth and read-only.
 
+## v0.4 web observability
+
+v0.4 turns the existing uplink page into a practical read-only RF and logging console.
+
+### Latest-scan AP table
+
+The browser shows a table from the most recently **completed** RF scan:
+
+- SSID
+- BSSID
+- 2.4 GHz / 5 GHz band
+- channel
+- RSSI
+- authentication mode
+
+Rows are sorted strongest-first.
+
+The device keeps a bounded cache of the strongest 64 APs. If a scan sees more than 64 APs, the JSON response marks the table as truncated rather than consuming unbounded RAM. The browser never reads a half-built scan; the working set is published only when a complete scan succeeds.
+
+The raw endpoint is:
+
+`/aps.json`
+
+SSID values are inserted into the browser through DOM `textContent`, not interpreted as HTML.
+
+### TF session browser and downloads
+
+When a TF card is mounted, the web page lists the newest 32 numbered BandiC5 sessions and provides a download link for each.
+
+Downloads use a numeric session parameter:
+
+`/download?session=17`
+
+The firmware reconstructs the expected `session-0017.csv` path internally. Arbitrary filesystem paths are never accepted from the browser.
+
+The current active session is marked separately and is intentionally not downloadable while it is still open for writing. Completed session files are stable and downloadable.
+
+Session metadata is available from:
+
+`/logs.json`
+
+CSV files are streamed in small chunks rather than loaded into RAM.
+
+
+### Mobile CSV download framing
+
+v0.4.1 changes completed-session downloads from HTTP chunked transfer to a fixed
+`Content-Length` response. The file is still streamed from TF in 2 KiB pieces,
+but mobile browsers/download managers can now display the real attachment size
+instead of `-1 byte` / unknown length.
+
+
+### Single-radio CSV transfer guard
+
+v0.4.2 serializes active RF scans and completed-session downloads with a shared
+radio mutex. A download waits for any current channel sweep to finish, then
+temporarily keeps the single ESP32-C5 Wi-Fi radio on the associated AP's home
+channel while streaming the CSV. RF scanning resumes automatically as soon as
+the transfer ends.
+
+This addresses mobile transfers that received the correct `Content-Length` but
+stalled at zero bytes when an active dual-band scan moved the STA off-channel.
+The web UI also retains the last known status/table during brief server-busy
+periods instead of replacing good data with a transient unavailable message.
+
+
+### Browser-managed CSV downloads
+
+v0.4.3 removes the hand-framed fixed-length HTTP response used in v0.4.1/v0.4.2.
+Completed-session CSVs are again sent with ESP-IDF's supported chunked response
+API, but the status page now fetches the file itself, tracks received bytes
+against the known session size, assembles a browser Blob, and triggers the local
+download only after the transfer completes.
+
+While a file transfer is active:
+- normal status/AP/session polling is paused
+- the shared radio mutex prevents new RF sweeps
+- the HTTP send timeout is extended to 20 seconds
+- polling and scanning resume automatically after success or failure
+
+This avoids mobile download-manager quirks around local chunked attachments and
+also avoids leaving the ESP-IDF HTTP server in an inconsistent state after a
+manually framed response.
+
+### v0.3.1 recovery landmark
+
+The branch `release/v0.3.1` points at the exact physically validated v0.3.1 mainline commit. The connected GitHub tooling used for this project can mutate branches/PRs but does not expose Git tag/release-asset creation, so this branch is the durable recovery pointer until a GitHub Release/tag is created separately.
+
+## v0.4.5 static RC
+
+This pass addresses the three merge blockers found by the adversarial re-review
+of v0.4.4, plus several small safety fixes that were cheap to close before
+hardware testing.
+
+### Uplink connection state
+
+- association/discovery now has a 15-second window instead of a 3-second total
+  connect deadline
+- `WIFI_EVENT_STA_CONNECTED` starts a separate 10-second DHCP window
+- the CONNECTING state is published before `esp_wifi_connect()`, so an
+  unusually fast GOT_IP event cannot be overwritten by the caller
+- connect attempts now own the same radio mutex as scans and downloads for the
+  asynchronous attempt lifetime
+- LOST_IP is treated as a DHCP-recovery state instead of immediately issuing a
+  second connect while still associated
+- a deferred scout scan gets priority before another connect or download starts
+
+### Completed-session downloads
+
+- the 30-second total transfer cap is removed
+- ESP-IDF's configured socket send timeout remains the server-side stalled-client
+  bound
+- the browser's 30-second timer is now inactivity-based and is re-armed whenever
+  a stream chunk arrives
+- completed sessions are validated/opened before attempting to reserve the radio
+- downloads back off when a scout scan is already waiting
+- the radio wait for a download is capped at one second
+- an in-flight download exits at the next chunk boundary when uplink shutdown
+  begins, so BOOT/OTA entry is not forced to wait for a healthy long transfer
+
+### OTA rollback interaction
+
+A pending OTA image is confirmed after the update AP and updater HTTP server
+successfully start. That recovery path is sufficient evidence to make update
+mode usable: a second firmware upload and a Wi-Fi-settings reboot can no longer
+be rejected or silently roll the new image back during its normal health window.
+Both POST handlers also confirm defensively, closing the tiny race between the
+HTTP server accepting its first request and the control task's post-start check.
+
+Normal scout-mode confirmation is still delayed until a successful scan/publish,
+the healthy-runtime window, and first uplink-attempt resolution.
+
+### Session creation safety
+
+- directory enumeration errors are distinguished from normal end-of-directory
+- new session files are created with exclusive-create semantics
+- a case-variant or otherwise previously missed filename cannot be silently
+  truncated by opening it with create-always behavior
+
+## v0.4.4 hardening RC
+
+This release-candidate pass follows an adversarial whole-repository review and
+focuses on concurrency, recovery, and long-run reliability rather than adding
+new user-facing features.
+
+### TF/FAT lifetime safety
+
+The storage module now owns a mutex and explicit completed-session reader leases.
+
+- storage state becomes unavailable before unmount begins
+- FAT/VFS unmount is deferred while any web reader is still open
+- no new completed-session reader can open after teardown starts
+- web downloads use storage-owned open/read/close wrappers instead of touching
+  FATFS directly
+- session listing is serialized with teardown and stats only the newest retained
+  entries
+- session allocation scans the directory once for the highest canonical session
+  number instead of probing every number with repeated `stat()`
+- session names must exactly match `session-NNNN.csv`
+- failed initial session creation removes the incomplete file
+- CSV cells beginning with spreadsheet formula trigger characters are prefixed
+  to prevent formula execution when logs are opened in spreadsheet software
+
+### Scout/uplink arbitration
+
+Radio ownership is now nonblocking from the scout control loop.
+
+- scans use a try-lock instead of waiting forever behind a browser transfer
+- the control loop continues servicing BOOT/OTA, storage recovery, and uplink
+  state while the radio is busy
+- uplink connection attempts have an explicit connecting state and a 3-second
+  bounded opportunity before the radio returns to scouting
+- scan-start `ESP_ERR_WIFI_STATE` is treated as a deferral rather than a hard
+  scan error
+- DHCP lost-IP events are handled
+- Wi-Fi configuration uses RAM storage; the BandiC5 NVS namespace remains the
+  credential source of truth
+
+### HTTP reliability
+
+- status server enables LRU socket purge and TCP keepalive
+- completed CSV transfers are bounded to 30 seconds server-side
+- browser-side fetches are also aborted after 30 seconds
+- normal page polling pauses during a CSV transfer
+- downloads retain the radio guard but can no longer park the main control loop
+
+### OTA and event-task hardening
+
+- HTTP and mDNS startup moved out of the ESP-IDF system event task
+- Wi-Fi/IP event callbacks now only update small state flags
+- update and Wi-Fi-config POST receives abort after six consecutive socket
+  timeouts instead of wedging update mode indefinitely
+- pending OTA images are no longer confirmed immediately after task creation
+- confirmation waits for a successful scan/publish path, a 30-second healthy
+  runtime window, and resolution of the first uplink attempt when configured
+- failed confirmation is retried rather than latched as successful
+
+### Scan failure cleanup
+
+Wi-Fi driver AP lists are explicitly cleared on count/allocation/retrieval
+failure paths so a dense-environment allocation failure does not retain the
+driver's scan result memory.
+
 ## Build
 
 The project targets **ESP-IDF v5.5.5** and **esp32c5**.

main/app_main.c

@@ -2,6 +2,8 @@
 #include "freertos/task.h"
 
 #include "esp_log.h"
+#include "esp_timer.h"
+#include "esp_wifi.h"
 #include "esp_system.h"
 
 #include "bandit_ota.h"
@@ -14,6 +16,7 @@
 #define SCAN_TASK_STACK_SIZE (8 * 1024)
 #define SCAN_INTERVAL_MS 3500
 #define OTA_REQUEST_POLL_MS 100
+#define OTA_CONFIRM_HEALTH_MS 30000
 
 static const char *TAG = "bandic5";
 
@@ -48,6 +51,16 @@ static void enter_ota_mode(void)
         esp_restart();
     }
 
+    // A live updater AP + HTTP server is the recovery path rollback is meant
+    // to preserve. Confirm a pending image here so update-mode uploads and a
+    // Wi-Fi-settings reboot cannot accidentally roll this working image back.
+    if (!bandit_ota_confirm_running_image()) {
+        ESP_LOGE(TAG, "unable to confirm running image after OTA recovery started");
+        bandit_ui_set_ota_progress(0, "OTA VERIFY FAILED");
+        vTaskDelay(pdMS_TO_TICKS(2500));
+        esp_restart();
+    }
+
     // The HTTP server owns the updater from here. This task is no longer needed.
     vTaskDelete(NULL);
 }
@@ -72,10 +85,21 @@ static void wait_for_next_scan_or_ota(void)
     }
 }
 
+static void handle_scan_record(const bandit_scan_record_t *record, void *ctx)
+{
+    (void)ctx;
+    bandit_storage_log_record(record, NULL);
+    bandit_uplink_observe_record(record);
+}
+
 static void scan_task(void *arg)
 {
     (void)arg;
 
+    bool first_scan_ok = false;
+    bool ota_confirmed = false;
+    int64_t healthy_since_ms = 0;
+
     while (1) {
         if (bandit_ota_requested()) {
             enter_ota_mode();
@@ -85,20 +109,33 @@ static void scan_task(void *arg)
         bandit_ui_set_storage_state(storage_ui_state());
 
         bandit_scan_snapshot_t snapshot;
+
+        if (!bandit_uplink_try_begin_scan()) {
+            bandit_ui_set_status("RADIO BUSY");
+            wait_for_next_scan_or_ota();
+            continue;
+        }
+
         bandit_ui_set_status("SCANNING");
 
         esp_err_t ret = bandit_scan_once(
             &snapshot,
-            bandit_storage_log_record,
+            handle_scan_record,
             NULL
         );
+        bandit_uplink_end_scan();
 
         if (ret == ESP_OK) {
             bandit_storage_finish_scan(snapshot.generation);
             bandit_ui_set_storage_state(storage_ui_state());
             bandit_ui_update(&snapshot);
             bandit_uplink_publish_snapshot(&snapshot);
 
+            if (!first_scan_ok) {
+                first_scan_ok = true;
+                healthy_since_ms = esp_timer_get_time() / 1000;
+            }
+
             ESP_LOGI(
                 TAG,
                 "scan=%lu aps=%u 2g=%u 5g=%u strongest=%s rssi=%d ch=%u",
@@ -110,12 +147,23 @@ static void scan_task(void *arg)
                 snapshot.strongest_rssi,
                 snapshot.strongest_channel
             );
+        } else if (ret == ESP_ERR_WIFI_STATE) {
+            ESP_LOGW(TAG, "scan deferred while Wi-Fi driver is busy");
+            bandit_ui_set_storage_state(storage_ui_state());
+            bandit_ui_set_status("SCAN DEFER");
         } else {
             ESP_LOGE(TAG, "scan failed: %s", esp_err_to_name(ret));
             bandit_ui_set_storage_state(storage_ui_state());
             bandit_ui_set_status("SCAN ERROR");
         }
 
+        if (!ota_confirmed &&
+            first_scan_ok &&
+            bandit_uplink_first_attempt_resolved() &&
+            (esp_timer_get_time() / 1000) - healthy_since_ms >= OTA_CONFIRM_HEALTH_MS) {
+            ota_confirmed = bandit_ota_confirm_running_image();
+        }
+
         if (bandit_ota_requested()) {
             enter_ota_mode();
         }
@@ -140,7 +188,13 @@ void app_main(void)
     bandit_ui_set_storage_state(storage_ui_state());
 
     if (ret == ESP_OK) {
-        ESP_LOGI(TAG, "TF logging enabled: %s", bandit_storage_get_session_path());
+        char session_path[96];
+        if (bandit_storage_get_session_path(
+                session_path,
+                sizeof(session_path)
+            ) == ESP_OK) {
+            ESP_LOGI(TAG, "TF logging enabled: %s", session_path);
+        }
     } else {
         ESP_LOGW(
             TAG,
@@ -188,7 +242,6 @@ void app_main(void)
         return;
     }
 
-    // A newly OTA-flashed image is accepted only after display, radio, OTA trigger,
-    // and the main scan task have all initialized successfully.
-    bandit_ota_confirm_running_image();
+    // Pending OTA images are confirmed by scan_task only after real runtime
+    // behavior has been exercised for the health window.
 }