Repository navigation
ADSB background fetch with http/1.1 - #90
Merged
Merged
Conversation
Aircraft positions only moved on each ~3 s ADS-B fetch, so motion was steppy. Now extrapolate each aircraft from its last fetched fix along its ground track (track_deg) at its ground speed (gs_knots) using elapsed time, and redraw at 4 Hz between fetches. - adsb_client: record lastUpdateMs() at each successful fetch (the time the stored positions are valid). - radar_display: extrapolatedLatLon() dead-reckons position using the same flat 111 km/deg projection as the drawing path (round-trips exactly); drawAircraft uses it for symbols, tags, and beyond-ring dots. - main loop: redraw every kRadarRedrawIntervalMs (250 ms) between fetches. The fetch interval goes 3 s -> 5 s, since smooth motion no longer depends on how often the data is refreshed. Extrapolation also accounts for the age of the fix itself. adsb.fi reports seen_pos: seconds since each aircraft's position was measured (often 10-30 s old). Starting from the fetch time drew a stale fix lagging its true position, so seen_pos is stored per aircraft (pos_age_ms, capped at 30 s) and added to the elapsed time. Positions snap to real data on each fetch. Motion pauses briefly during the blocking fetch itself. Applies to both targets. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The fetch blocked the main loop for ~1-2 s every 3 s, pausing the 2 Hz dead-reckoned redraw. Move it to a FreeRTOS task pinned to core 0 so the render loop (core 1) keeps refreshing smoothly throughout the request. - adsb_client: init() creates a mutex; fetchUpdate() parses into a local buffer and publish()es atomically under the lock; snapshotAircraft() copies the list + base timestamp under the lock for readers on another thread. - radar_display: drawAircraft() renders from a locked snapshot; extrapolation takes the base time as a parameter. - main: adsbFetchTask loops fetchUpdate every kAdsbFetchIntervalMs; the loop just redraws at 2 Hz. No more inline fetch / poll-fn. Applies to both targets (on the single-core C3 the scheduler interleaves the task with the loop). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The fetch buffered the whole response body into a String and then handed that to ArduinoJson, so peak heap held the payload and the document at the same time -- and the document stored all ~40 fields of every aircraft record when the radar reads only 15 of them. Replace readResponseBodyWithPoll with PollingBodyReader, which ArduinoJson pulls from directly. It drains the socket in 512-byte blocks and serves single bytes, since the JSON deserializer never calls readBytes, and it runs the network poll callback on every refill -- the reason HTTPClients
e2e94c6 forced HTTP/1.0 because the response body was arriving chunked and the JSON parser choked on the chunk-size lines. The cause is that getStreamPtr() hands back the raw socket -- HTTPClient de-chunks only inside its own body readers, which this code can't use because they block without running the network poll callback. Decode the framing here instead. BodyFramer unwraps an identity or chunked body from a byte source, stripping chunk sizes, extensions, the inter-chunk CRLFs and any trailer section. It pulls a byte at a time, so a chunk boundary landing anywhere in the socket's own buffering -- mid-hex-digit, or between the CR and the LF -- needs no special handling. PollingBodyReader loses its length bookkeeping to the framer and becomes PollingSocketSource, a plain pump that keeps the poll-callback refill loop. Transfer-Encoding has to be requested via collectHeaders() before the GET; HTTPClient's own _transferEncoding is private. Dropping useHTTP10 also re-enables keep-alive on the request, but nothing is reused yet -- the client and HTTPClient are still torn down every fetch. The post-parse drain() is groundwork for that: a parser stops at the closing brace, and a connection reused without consuming the terminating chunk would read it as the head of the next response. The header is free of Arduino includes so the framing can be tested on a host. Compiles clean on both arduino-esp32 2.x and 3.x. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
MatixYo
approved these changes
Sep 24, 2026
Owner
|
Looks good, could you resolve conflicts? @pvanbaren |
Contributor
Author
|
I've resolved the conflicts. I kept the timeout (which fixed the retry storm) and filter changes from the main branch, otherwise merged in my changes to fetch on a background thread. |
lmoiseichuk
added a commit
to lmoiseichuk/ESP32-Plane-Radar
that referenced
this pull request
Oct 8, 2026
MatixYo#94 (e23b82c) stopped a TLS retry storm by replacing performGetWithPoll with a single GET and a 5000 ms connect timeout. When main was merged into MatixYo#90's branch (dc969f6, "Merge branch 'main' into feature/adsb-pipeline"), the conflict was resolved by keeping performGetWithPoll, and MatixYo#90 took it back to main (78958a5). It calls GET() again every 5 ms until a 6000 ms deadline whenever the result is CONNECTION_REFUSED or NOT_CONNECTED. A TLS setup that cannot allocate its record buffers comes back that way. mbedtls_ssl_setup fails with -32512 (SSL - Memory allocation failed), WiFiClientSecure::connect returns 0, and HTTPClient reports a refused connection. Nothing frees heap between attempts, so the same allocation fails every 5 ms until the deadline. The storm fires whenever the TLS setup cannot get its heap, or a connect is refused. On the 240x240 board the heap is there and it stays latent; it was seen on a fork driving a 360x360 panel, whose larger frame buffer leaves less heap. There each fetch repeated the failed allocation about 170 times, gave up after 6 s with "adsb: HTTP -11", and logged about 900 allocation failures a minute. Go back to MatixYo#94's behaviour: one GET with the 5000 ms connect timeout. A failed connect is retried by the next fetch, kAdsbFetchIntervalMs later. The background task, streaming parse and chunked framing from MatixYo#90 are unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DL3fMjXoiVk9VGMHmQkfyW
lmoiseichuk
added a commit
to lmoiseichuk/ESP32-Plane-Radar
that referenced
this pull request
Oct 8, 2026
MatixYo#94 (e23b82c) stopped a TLS retry storm by replacing performGetWithPoll with a single GET and a 5000 ms connect timeout. When main was merged into MatixYo#90's branch (dc969f6, "Merge branch 'main' into feature/adsb-pipeline"), the conflict was resolved by keeping performGetWithPoll, and MatixYo#90 took it back to main (78958a5). It calls GET() again every 5 ms until a 6000 ms deadline whenever the result is CONNECTION_REFUSED or NOT_CONNECTED. A TLS setup that cannot allocate its record buffers comes back that way. mbedtls_ssl_setup fails with -32512 (SSL - Memory allocation failed), WiFiClientSecure::connect returns 0, and HTTPClient reports a refused connection. Nothing frees heap between attempts, so the same allocation fails every 5 ms until the deadline. The storm fires whenever the TLS setup cannot get its heap, or a connect is refused. On the 240x240 board the heap is there and it stays latent; it was seen on a fork driving a 360x360 panel, whose larger frame buffer leaves less heap. There each fetch repeated the failed allocation about 170 times, gave up after 6 s with "adsb: HTTP -11", and logged about 900 allocation failures a minute. Go back to MatixYo#94's behaviour: one GET with the 5000 ms connect timeout. A failed connect is retried by the next fetch, kAdsbFetchIntervalMs later. The background task, streaming parse and chunked framing from MatixYo#90 are unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DL3fMjXoiVk9VGMHmQkfyW
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request moves the ADSB fetch over to a background thread using http/1.1 chunked transfer, streaming+filtering the JSON, and updating the display at 4Hz using dead-reckoning between ADSB updates.