Repository navigation
Memory optimisations (PR #227) with measured stack sizes and HTTP client fixes - #229
Merged
Merged
Conversation
build-check/ is a local out-of-tree build used to verify that each change compiles; it must never be committed.
A TLS session costs 36-49 KB of heap on this device. At "Got IP" the integration task immediately started the LibreView login while httpd, mDNS and LVGL asset loading were still allocating. Measured effect: free heap dropped to 40 bytes, the WiFi driver could no longer allocate RX buffers, httpd failed to start (8 KB stack allocation failed) and the boot aborted in ets_timer_setfn (ESP_ERR_NO_MEM), leading to a boot loop. acquire_ssl_semaphore() now waits on a startup gate that app_main() opens after "Startup complete". The wait is bounded (60 s) so a stalled boot can never block TLS forever.
Measured TLS sessions cost 24-49 KB of heap each (LibreView login 44-49 KB), while steady-state free heap is only 29-50 KB with a largest free block of 17-23 KB. With static buffers mbedtls_ssl_setup() needs a contiguous 16 KB IN buffer plus a 4.4 KB OUT buffer up front; on the fragmented heap the OUT allocation failed (-0x008D) and weather/LibreView fetches dropped. - MBEDTLS_DYNAMIC_BUFFER: record buffers are sized to the actual record and released when idle instead of 16 KB + 4 KB for the whole session. - MBEDTLS_DYNAMIC_FREE_CONFIG_DATA / DYNAMIC_FREE_CA_CERT: free the handshake-only config and CA data once the handshake is done. SSL_KEEP_PEER_CERTIFICATE is already disabled by default in IDF 6.
Stack high-water marks measured on device (memprobe build, ~12 min run including boot, weather, LibreView and web UI traffic): esp_timer 8192 B, 7252 B never used -> 4096 B (saves 4 KB) sys_evt 4096 B, 2944 B never used -> 3072 B (saves 1 KB) mdns 4096 B, 2372 B never used -> 3072 B (saves 1 KB) Tmr Svc 2048 B, 1604 B never used -> 1536 B (saves 0.5 KB) Every task keeps at least ~1 KB of margin. esp_timer callbacks only notify tasks now (the old 8 KB requirement dated from OTA work done in the timer callback); 4096 B still covers the 8-hourly POH file write. ~6.5 KB of heap is returned permanently.
Measured stack high-water marks (memprobe build): integration_update_task 8960 B, 3336 B never used -> 7680 B display_task 8960 B, 2348 B never used -> 8192 B The integration task peak was measured on the LibreView path (TLS + JSON); HA/stock/Dexcom/Nightscout/WU use the same HTTP client and cJSON code, so a ~2 KB margin is kept. display_task keeps ~1.6 KB margin for paths not hit during the measurement. ~2 KB of heap is returned permanently.
ota_update_task held a 10 KB stack for the whole uptime while blocked on a semaphore; measured high-water mark showed 8.7-9.7 KB of it never used between checks. The task is now spawned by f_ota_trigger_update() / f_ota_trigger_reinstall() and deletes itself when done, returning 10 KB of heap outside update checks. - A trigger while a run is in progress sets a pending flag; the running task loops once more instead of spawning a second task (a pending reinstall request is preserved). - If the 10 KB stack cannot be allocated the check is skipped with a log (free/largest block) and retried at the next timer interval. - The stack size stays 10240 B for the manifest ECDSA verification. - f_ota_start/stop_update_thread() now only enable/disable triggering; the unused ota_update_semaphore global is removed.
graph_canvas_buf was realloc()ed to width*height*2 whenever the configured graph size changed (e.g. day/night layout profiles). On the fragmented heap the realloc(4320) failed 196 times in 12 minutes of measurement, each failure skipping the render and each successful resize leaving another hole. The buffer is now allocated once at GRAPH_MAX_W * GRAPH_MAX_H * 2 (5760 B) the first time a graph is shown and reused for any size; it is still released when the graph is disabled in the layout.
Measured heap drop per static file request was 9-16 KB, and opening the web UI fetches ~20 files (css/js/i18n/images) back to back. - generic_file_handler() now disables the stdio buffer (setvbuf _IONBF): the handler already reads in 1 KB chunks, so the FILE buffer only added a per-request heap allocation. - .css, .js, .ico and i18n/ files are sent with Cache-Control: max-age=600 so repeat visits do not re-download the whole UI. These only change with an OTA asset update (worst case 10 min stale). User-editable files (layouts, fonts, images) remain uncached.
The LibreView login is the most expensive request measured on the device (44-49 KB heap drop during the TLS session). During a 10 minute run it was performed 9 times because: - any transport error in fetch_freestyle_glucose() (typically a TLS handshake that failed for lack of heap) cleared the auth token and patient id, forcing a full login + patient lookup on the next try; - a failed fetch was retried on the very next 1 s integration tick. Now the token is only discarded on HTTP 401, and failed Freestyle updates back off exponentially (30 s, 60 s, 120 s ...) capped at the configured glucose refresh interval; the backoff resets on the first success.
The shared HTTP buffer pool is a static array (MAX_BUFFER_POOL_SIZE x HTTP_BUFFER_SIZE). When no free buffer was large enough, get_shared_buffer() called heap_caps_free() on the oldest idle one - a pointer into .bss - and replaced it with a heap allocation. That corrupts the heap as soon as a caller asks for more than HTTP_BUFFER_SIZE (current callers stay <= 3912 B, so it was latent). Oversized requests are now rejected with an explicit log; the "pool exhausted" path is unchanged.
wifi_lookup_iana_from_coords() (GET /api/timezone with lat/lon, httpd task) opened an HTTPS session to timeapi.io without acquire_ssl_semaphore(). Every other TLS client is serialized because one session costs 24-49 KB of heap on this device; a second concurrent session leaves nothing for the WiFi driver. The request now takes the SSL semaphore like the other clients and fails cleanly if it cannot be obtained within the usual timeout.
fetch_freestyle_glucose() streams the /graph response and publishes each graphData point to the shared CGM history as it arrives. When the transfer failed mid-stream (e.g. a TLS record buffer could not be allocated on a fragmented heap), only the oldest points had been captured. The graph backfill then seeded the 60-slot ring from that partial history, and live sampling filled the newest slots, leaving a gap in the middle of the curve (only the first and last ~10 pixels drawn) until the ring rolled over. The CGM history is now cleared when an attempt fails, so the backfill sees no data and re-arms until a complete response has been captured.
… stack sizes.
Conflicts resolved against the recursive SSL lock and the Freestyle hardening
that landed after the PR's base: the TLS startup gate now waits before the
recursive take (the PR's xSemaphoreGive on a fresh mutex is dropped), and the
Freestyle transport-error log keeps master's esp_err/errno detail. The timeapi.io
lock commit was already on master.
Stack sizes stay at master's values (esp_timer 8192, sys_evt 4096, mDNS default,
timer service 2048, display/integration 8960) until they are measured on current
code. Of the PR's sdkconfig changes only DYNAMIC_FREE_CONFIG_DATA and
DYNAMIC_FREE_CA_CERT remain; DYNAMIC_BUFFER was already on master.
Review fixes on top of the PR:
- Open the TLS startup gate when the provisioning portal starts; app_main never
reaches "Startup complete" there, so a portal lookup waited 60 s.
- Clear the OTA pending/reinstall flags under the lock when the task cannot be
created, so a stale pending flag does not cause an extra run later.
- Match Cache-Control extensions at the end of the name (".js" also hit ".json").
- Reset the Freestyle backoff on a settings change or forced update, so fixed
credentials are tried at once instead of after the backoff.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Opt-in, so the normal status poll is unchanged. Covers the named FreeRTOS tasks and the on-demand OTA task (its minimum is kept across runs). Used to size task stacks from measurements on current code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured on the test device (HA + LibreView, met.no, web UI, timezone/locate lookups, settings and font changes) over ~40 min with /api/status?stacks=1. Rule: peak used + 25% + 1 KB, rounded up to 256 B, with a floor where a known path did not run during the measurement. display_task 8960 -> 6656 (4452 B peak) integration_update_task 8960 -> 8448 (5736 B peak) esp_timer 8192 -> 4096 (1396 B peak; floor for the 8-hourly POH file save) sys_evt 4096 -> 3072 (1084 B peak; floor for Wi-Fi drop/reconnect events) Tmr Svc 2048 -> 1792 (436 B peak) mDNS stays at 4096 (2308 B peak, too close for the PR's 3072). OTA keeps 10240 for the manifest ECDSA path, which a no-update check does not reach. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…port error. On a read timeout esp_http_client_perform() returns ESP_ERR_HTTP_EAGAIN and leaves the client in REQ_COMPLETE_DATA. The next perform on the same handle skips sending and only waits for headers again, so one slow /llu/connections reply made every later attempt time out until reboot (seen on the test device: "Freestyle connections HTTP ESP_ERR_HTTP_EAGAIN" every retry, gray drop). The graph fetch already reset the connection on error; login (other than ESP_FAIL) and fetch_patient_id() did not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A read timeout leaves esp_http_client mid-request (EAGAIN), and every later perform on the persistent client only waits for headers again, so one slow reply stopped Nightscout updates until reboot. Same fix as Freestyle. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merges #227 by @benoitpoirier onto current master, with review fixes, stack sizes taken from high-water marks measured on the test device, and two integration fixes.
What's in it
DYNAMIC_FREE_CONFIG_DATAandDYNAMIC_FREE_CA_CERTare new; the 16 KB IN buffer met.no needs is unchanged (dynamic buffers allocate at record length)..json); Freestyle backoff resets when settings change./api/status?stacks=1reports each task's minimum free stack.esp_http_clientstays mid-request (ESP_ERR_HTTP_EAGAIN) and every later request on the persistent client fails until reboot ("gray drop"). Both now close the connection on transport errors.Tested on hardware (HA + Freestyle device)
Closes #227.
🤖 Generated with Claude Code