diff --git a/FORK_PRS.md b/FORK_PRS.md new file mode 100644 index 0000000..60e6dd9 --- /dev/null +++ b/FORK_PRS.md @@ -0,0 +1,68 @@ +# Fork mining record + +Tracks GitHub forks of [`CamM2325/microlink`](https://github.com/CamM2325/microlink) that were +investigated for fixes/features worth pulling into this fork — distinct from +[`UPSTREAM_PRS.md`](UPSTREAM_PRS.md), which tracks upstream's own six open PRs. Exists so this +research (51 forks surveyed, 13 with real divergence, 4 parallel investigations) doesn't have to be +redone, and so a future contributor knows which fork commits map to which tracking issue. + +**Survey status as of 2026-08-18:** 51 forks total. 38 are pure stale mirrors (identical `pushed_at` +to upstream's last commit, `216da33`, 2026-03-17) or otherwise trivially behind — not investigated +individually. 13 forks have real commits ahead of upstream; all 13 were inspected directly (commit +diffs, not just messages). + +## Worth pulling — tracked as issues + +Each row is one independently-shippable fix/feature, tracked as its own GitHub issue so it can +become its own scoped PR. Issue numbers filled in once created. + +| # | Issue | Source fork | Commit(s) | Summary | Tier | +|---|-------|-------------|-----------|---------|------| +| 1 | ✅ [#14](https://github.com/fugo101/microlink/issues/14) (done) | [`cplewes/microlink`](https://github.com/cplewes/microlink) | `38602ab0`, `b25b1eee` | DERP TLS context leak on every failed `ml_derp_connect()` (~3-8KB/attempt, verified on hardware). Pre-existing issue filed 2026-08-17 from `UPSTREAM_PRS.md`'s "mineable from #22" note; enriched here with the cplewes source instead of building `derp_tls_abort()` from scratch. Adapted (not literal cherry-pick, our fork lacks entropy/ctr_drbg fields) into `ml_derp.c`'s `derp_free_tls_state()` + `fail_tls` goto path. | 1 | +| 2 | [#21](https://github.com/fugo101/microlink/issues/21) | [`cplewes/microlink`](https://github.com/cplewes/microlink) | `a415d646` | Teardown UAF: `microlink_stop()`/`destroy()` didn't join worker tasks before freeing context; replaces "sleep 3s and hope" with a real per-task liveness bitmask + reap-orphans path | 1 | +| 3 | [#22](https://github.com/fugo101/microlink/issues/22) | [`cplewes/microlink`](https://github.com/cplewes/microlink) | `7120dfa4` | Three crash sites triggered by captive-portal DNS failures: DERP double-init dangling pointer, `ml_coord_task` touching a freed event group post-destroy, corrupt-state races during parallel teardown | 1 | +| 4 | [#23](https://github.com/fugo101/microlink/issues/23) | [`cplewes/microlink`](https://github.com/cplewes/microlink) | `8f2ff39b`, `5c7303b4` | Captive-portal detection (HTTP 302 / TCP close) in `ml_coord.c`, backs off 5 min instead of retrying every 16s | 1 | +| 5 | [#24](https://github.com/fugo101/microlink/issues/24) | [`cplewes/microlink`](https://github.com/cplewes/microlink) | `9f6af750` | Yield 1 tick per peer in `disco_periodic_probes` — prevents starving other same-core tasks on large tailnets | 1 | +| 6 | [#25](https://github.com/fugo101/microlink/issues/25) | [`cplewes/microlink`](https://github.com/cplewes/microlink) | `b9636816` | DERP TLS session resumption (skip full ECDHE handshake on reconnect, ~7.5s→sub-second), connect timeout 10s→25s, per-peer exponential backoff on direct-path upgrade probes | 1 | +| 7 | [#26](https://github.com/fugo101/microlink/issues/26) | [`antmanler/microlink`](https://github.com/antmanler/microlink) (branch `duoduo-edge`) | `2a7ba328` | `wg_udp_output_cb` thread-safety bug (raw UDP PCB called from two task contexts, corrupts heap) — same bug class as absorbed upstream PR #20, opposite direction; bundled with WG/DISCO RX queue depth increase (8→32/16) and a handshake-gating bug in `process_disco_pong` that permanently blocked sessions | 1 | +| 8 | [#27](https://github.com/fugo101/microlink/issues/27) | [`antmanler/microlink`](https://github.com/antmanler/microlink) | `6ef9f5a0` | New `CONFIG_ML_CONFIG_HTTPD` Kconfig toggle to skip the port-80 httpd task (~7-8KB RAM saved); peer-update queue depth fix 400→32 (was allowing an unbounded ~80KB internal-RAM burst) | 1 | +| 9 | [#28](https://github.com/fugo101/microlink/issues/28) | [`antmanler/microlink`](https://github.com/antmanler/microlink) | `2a7ba328` | WG netif MTU 1420→1280 (Tailscale-standard) — needs verification against our existing `CONFIG_LWIP_IP4_FRAG`/`IP4_REASSEMBLY=y` before deciding it's still needed | 1 | +| 10 | [#29](https://github.com/fugo101/microlink/issues/29) | [`liestrela/microlink`](https://github.com/liestrela/microlink) | `1649d987` | Gate ESP32 temp-sensor feature behind `CONFIG_SOC_TEMP_SENSOR_SUPPORTED` — `esp_driver_tsens` was an unconditional `REQUIRES`, breaks build on SoC variants without a temp sensor | 1 | +| 11 | [#30](https://github.com/fugo101/microlink/issues/30) | `cplewes` vs `Csontikka` (conflict) | `cplewes` `83a102be`/`01d51697` vs [`Csontikka/microlink`](https://github.com/Csontikka/microlink) `dd5714c4`→`2e68e546` | `ip_input()` vs `netif->input`/`tcpip_input` threading fix for decrypted WG RX pbufs — cplewes fixes a UAF this way, Csontikka's own branch later reverted the equivalent fix citing a throughput regression (~30pps cap). Needs research, not a blind cherry-pick | 2 | +| 12 | [#31](https://github.com/fugo101/microlink/issues/31) | `Csontikka/microlink` | `647c2ab0` | `LOCK_TCPIP_CORE` around `netif_set_link_up/down` — compare against our documented `netif->state` NULL-and-restore workaround in `ESP_IDF_6X_COMPAT.md` before adopting | 2 | +| 13 | [#32](https://github.com/fugo101/microlink/issues/32) | `Csontikka/microlink` | `5bda1783` | Peer direct-handshake init was one-shot; peers with dropped/unanswered handshakes stayed permanently blackholed — retry every 30s via timestamp instead of boolean latch | 2 | +| 14 | [#33](https://github.com/fugo101/microlink/issues/33) | `Csontikka/microlink` | `2e68e546`, `f1de3143` | DISCO trust-expiry gate (15s→60s, checks `last_rx` first), NAT-rebind handshake-skip when data path still alive, SPIRAM pbuf headroom fix for `wg_udp_output_cb` | 2 | +| 15 | [#34](https://github.com/fugo101/microlink/issues/34) | `Csontikka/microlink` | `b7442d0f` | Handshake retries throttled to 1/tick round-robin instead of firing to every eligible peer in one tick (was blocking the caller ~280ms with 7 peers) | 2 | +| 16 | [#35](https://github.com/fugo101/microlink/issues/35) | `Csontikka/microlink` | `7a24a9b3`, `4d3b3add` | DERP client backpressure/reconnect-storm fixes: don't tear down on TLS write backpressure, TCP_NODELAY + coalesced writes, fix a hot-spinning I/O loop, split into concurrent reader/writer tasks — check overlap against cplewes's DERP session-resumption fix (#6) before scoping | 2 | +| 17 | [#36](https://github.com/fugo101/microlink/issues/36) | `Csontikka/microlink` | `27806be3` | `microlink_stop()` never closed `derp.sockfd`/`coord_sock`, letting `derp_tx` outlive the teardown wait → UAF — reconcile against cplewes's teardown UAF fix (#2), may be the same root cause | 2 | +| 18 | [#37](https://github.com/fugo101/microlink/issues/37) | `Csontikka/microlink` | `cbdf1603`, `aad403af`, `533f1f88`, `46e34917`, `017b3588`, `372ca277` | H2 frame reassembly across `noise_recv()` read boundaries — large MapResponses spanning reads deterministically corrupt the stream ("implausible message size"); genuine protocol-correctness bug independent of control-plane backend | 2 | +| 19 | [#38](https://github.com/fugo101/microlink/issues/38), [#39](https://github.com/fugo101/microlink/issues/39) | [`djorr5/microlink`](https://github.com/djorr5/microlink) | `67b230b2` | Two independent changes, split into two issues: (a) dynamic H2 RX window sizing based on free heap for RAM-constrained boards, roughly halving MapResponse-parsing PSRAM footprint; (b) `ip4_route_src_hook` to force tailnet-range traffic onto the WG netif directly | 2 | +| 20 | [#40](https://github.com/fugo101/microlink/issues/40) | [`AELovelace/LAIN-MicrolinkRouter`](https://github.com/AELovelace/LAIN-MicrolinkRouter) | `e1239460` | `microlink_set_exit_node()` — default-route exit-node support while preserving per-peer `/32` routes, with routing-loop and liveness fixes. New capability, not a bugfix — needs a product decision (does any current/future downstream want SoftAP/NAPT exit-node routing?) before it's worth the review cost | 2 | +| 21 | [#41](https://github.com/fugo101/microlink/issues/41) | [`caslavskola/microlink`](https://github.com/caslavskola/microlink) | `0265718e` | PSA crypto init migration for mbedTLS 4.x — verify overlap with existing `ESP_IDF_6X_COMPAT.md` PSA work; may be partially/fully redundant | 2 | +| 22 | [#42](https://github.com/fugo101/microlink/issues/42) | `caslavskola/microlink` | `9ac49212` | Peer endpoint tracking dropped the "packet arrived via DERP" signal after the first packet, causing replies to attempt bad direct routing — genuine bug, but the commit bundles debug cruft and a divergent netif-flag rewrite; extract just the DERP-routing-flag fix | 2 | +| 23 | [#43](https://github.com/fugo101/microlink/issues/43) | [`dj-oyu/microlink`](https://github.com/dj-oyu/microlink) | multiple (different/pre-refactor file naming) | Bundle of fixes needing adaptation to our `ml_*.c` layout: ~~TAI64N via `gettimeofday()` (fixes replay-rejection after reboot)~~ **superseded — see `UPSTREAM_PRS.md`'s upstream PR #22 (partial) absorption, commit `6a4447c5`: cleaner source, no pre-refactor adaptation needed, don't redo this from dj-oyu**; DERP→direct re-handshake trilogy (`a30160a8`, `eb3dce06`, `c505539d`), `LOCK_TCPIP_CORE` no-op bug fixed via `tcpip_try_callback` (check `ml_net_io.c`/`ml_zerocopy.c` exposure), DISCO PONG rate-limiting (1/5s per peer once direct path established) | 2 | + +## Investigated, not worth pursuing + +| Fork | Why skipped | +|---|---| +| `lixy123/microlink`, `szf2020/microlink`, `CryptoKylan/microlink` | Pure stale mirrors of upstream, zero meaningful divergence. | +| `flowjob1/microlink` (branch `combined`) | Board-specific Waveshare 4G-USB-modem hardware enablement (out of scope) plus a `json`→`cjson` fix we already have (`ESP_IDF_6X_COMPAT.md`). Messy WIP history. | +| `chat-l18l/microlink` | Exploratory/unfinished streaming-MapResponse rewrite (commit trail reads "checkpoint", no verification) — Csontikka's own equivalent work in the same area is more mature and battle-tested; prefer that lineage. A WiFi-optional/ESP32-P4 portability commit is mildly interesting but not for a target we support. | +| `sumagnadas/microlink`, `LuccaMS/microlink` | Personal app-layer product work (homelab dashboard, Wake-on-LAN example) on top of `ml_config_httpd`, not core-library fixes. | +| `cadl/libts3ds` | Full Nintendo 3DS rebrand/port with its own backend abstraction — not general-purpose fixes. | +| `youngthuggayslxo-code/microlink` | Single commit adds a nonsensical MSBuild CI workflow to a CMake/ESP-IDF project. Junk. | +| `GrieferPig/microlink` (1 star) | Real fix (peer-lookup keypair preference + src/dest IP validation per WireGuard spec) — but verified already present and correct in our `components/wireguard_lwip` submodule. No action needed. | + +## Notes + +- None of the 13 diverged forks are owned by the three already-absorbed upstream PR authors + (snowpaper, letalvoj, pepabo/kentaro) — confirmed independent work, no double-absorption risk. +- `cplewes/microlink` and `Csontikka/microlink` both use our post-refactor `ml_*.c` file naming, + making their commits directly comparable/cherry-pickable. `dj-oyu/microlink` still uses upstream's + pre-refactor naming (`microlink_wireguard.c` etc.) — every fix from it needs re-implementation + against our layout, not a cherry-pick. +- Two explicit conflicts to resolve before scoping PRs: (1) the `ip_input()`/`netif->input` fix + (#11) where `cplewes` and `Csontikka`'s own history disagree; (2) potential overlap between + `cplewes`'s teardown UAF fix (#2) and `Csontikka`'s socket-shutdown-on-stop fix (#17) — likely the + same root cause approached from different angles, verify before duplicating work. diff --git a/UPSTREAM_PRS.md b/UPSTREAM_PRS.md index 5470e9b..7c7adc0 100644 --- a/UPSTREAM_PRS.md +++ b/UPSTREAM_PRS.md @@ -16,18 +16,20 @@ commit our own compat work (`fce0875` onward) also sits on top of. | [#21](https://github.com/CamM2325/microlink/pull/21) | snowpaper | `da55beb7` | `5d8369c` (PR [#4](https://github.com/fudio101/microlink/pull/4)) | 2026-08-14 | Verbatim cherry-pick. Fixes [#18](https://github.com/CamM2325/microlink/issues/18) — CGNAT peers got a one-way tunnel because an unvalidated advertised endpoint was installed at peer-add. | | [#20](https://github.com/CamM2325/microlink/pull/20) | snowpaper | `e74be464` | `1314bab` (PR [#5](https://github.com/fudio101/microlink/pull/5)) | 2026-08-14 | Verbatim cherry-pick + a docs commit (`be5c0e3`) updating `components/wireguard_lwip/README.md`'s divergence record. Fixes [#17](https://github.com/CamM2325/microlink/issues/17) — RX path called `ip_input()` directly instead of via `netif->input`, racing `tcpip_thread`. | | [#23](https://github.com/CamM2325/microlink/pull/23) | letalvoj | `4cc3dded` | `82e5518` (PR [#6](https://github.com/fudio101/microlink/pull/6)) | 2026-08-14 | Verbatim cherry-pick. Logs `RegisterResponse.Error` instead of failing silently into a misleading downstream "node not found". | +| [#22](https://github.com/CamM2325/microlink/pull/22) (partial) | multiple | `125b5294`, `fcdc8d95`, `ae3d4386`, `8367c1ed`+`9ef10bbb` | PR [#44](https://github.com/fugo101/microlink/pull/44) | in review | **Bugfix/hardening subset only** — adapted, not cherry-picked (base diverged too far). DERP: mbedTLS-context leak on failed connect (already landed as the first commit on this PR, sourced from `cplewes/microlink` — see `FORK_PRS.md` #1/issue #14), region fallback when HomeDERP isn't in the DERPMap, TLS 1.2 pin (Let's Encrypt ECDSA cert / TLS 1.3 OID issue), return-code checking on mbedTLS setup calls, blocking-recv BIO rework. Coord: Hostinfo `OS` field corrected to `"esp32"` (was `"linux"`), new `ML_STATE_AUTH_FAILED` state (registration `Error`/`MachineAuthorized` now actually fails registration instead of silently "succeeding" into a broken state), `state_cb` now fires on the `RECONNECTING` transition and doesn't get clobbered by it while `AUTH_FAILED`. The 3 headscale/custom-control-plane feature commits (`CONFIG_ML_CTRL_HOST`+TLS transport, `ctrl_host`/`ctrl_noise_pubkey`, `streaming_map_fetch`, new `ml_coord_tls.c`) and the already-duplicate `netif->input` fix (`5c8d60c3`, functionally identical to already-absorbed PR #20) are **not** included — see the "Deliberately skipped" row below, now scoped to just those. | -All three cherry-picked via `git fetch upstream refs/pull/N/head` + `git cherry-pick -x`, which -preserves the original author; we appear only as committer. Each carries an `Upstream-PR:` / -`Upstream-Issue:` trailer. +All three verbatim cherry-picks were done via `git fetch upstream refs/pull/N/head` + `git +cherry-pick -x`, which preserves the original author; we appear only as committer. Each carries an +`Upstream-PR:` / `Upstream-Issue:` trailer. PR #22's absorption above is different — the base has +diverged too far for a literal cherry-pick, so it's a hand-adapted equivalent with an `Adapted +from` trailer instead. ## Deliberately skipped (for now) | Upstream PR | Why skipped | Revisit when | |---|---|---| | [#24](https://github.com/CamM2325/microlink/pull/24) — peer `online` from real `Node.Online` | The only one of the four small PRs that conflicts against this fork (one line at `ml_wg_mgr.c` where our `strncpy`→`memcpy` compat change sits on the same hunk). Its entire payload is the `microlink_peer_info_t.online` field delivered via the peer callback — the downstream consumer we checked (ZenClock) never registers a peer callback and never calls `microlink_get_peer_info()`/`get_peer_count()`, so the fix is currently unobservable. Correct fix, just no consumer yet. | The day any downstream project registers a `peer_cb` and reads `.online`. | -| [#22](https://github.com/CamM2325/microlink/pull/22) — custom (non-Tailscale) control planes: TLS control, streaming map-poll, DERP | +3424/−2671, effectively a rewrite of `ml_coord.c`. Built for headscale/Ionscale-style deployments; we run Tailscale's own SaaS control plane, so the TLS-control-transport, streaming-map-poll, and DERP-region-fallback pieces don't apply. Merging it whole against our diverged fork is a rewrite-on-rewrite conflict, not a cherry-pick. | If a downstream project needs a non-Tailscale control plane. | -| ↳ mineable from #22: `derp_tls_abort()` | `ml_derp_connect()`'s failure paths only close the socket, never free the mbedTLS context — leaks ~20 KB of TLS I/O buffers (internal RAM under `CONFIG_MBEDTLS_INTERNAL_MEM_ALLOC=y`) on every failed DERP connect attempt, and `microlink_rebind()` reconnects DERP on every WiFi reconnect. This leak fix is independent of the headscale-specific parts of #22 and is worth taking on its own — **not done yet**, needs adapting: our fork already deleted the `entropy`/`ctr_drbg` init+free pair from `ml_derp.c` for mbedTLS 4.x compat (see `ESP_IDF_6X_COMPAT.md`), so `derp_tls_abort()` would need to free only `ssl`/`ssl_conf`, not the fields we removed. | Next microlink maintenance pass — tracked here explicitly so it isn't lost. | +| [#22](https://github.com/CamM2325/microlink/pull/22) — custom (non-Tailscale) control planes: TLS control, streaming map-poll | The 3 headscale/Ionscale-specific feature commits only: build-time `CONFIG_ML_CTRL_HOST`+TLS control transport (new `ml_coord_tls.c`), runtime `ctrl_host`/`ctrl_noise_pubkey`, and `streaming_map_fetch` for controllers that ignore `Stream=false`. We run Tailscale's own SaaS control plane, so none of this applies. Everything else in #22 (DERP hardening, Hostinfo OS string, `ML_STATE_AUTH_FAILED`, the reconnect-state-callback fixes) **was absorbed** — see the "Absorbed" table above (PR #22, partial). | If a downstream project needs a non-Tailscale control plane. | | [#25](https://github.com/CamM2325/microlink/pull/25) — optional TLS for the control-plane transport | Overlaps/superseded in scope by #22 (same non-Tailscale-SaaS use case), and its diff includes a stray committed clangd index binary (`examples/eth_connect/.cache/clangd/index/...`) that shouldn't be merged as-is regardless. | Same trigger as #22. | | [#14](https://github.com/CamM2325/microlink/pull/14) (closed, not merged) — cross-network web access / thread safety | Closed by upstream without merging; not re-evaluated here. | — | | [#3](https://github.com/CamM2325/microlink/pull/3) (closed, not merged) — DISCO PONG rate-limit | Author closed it themselves: "Created against wrong repo." | — | diff --git a/components/microlink/include/microlink.h b/components/microlink/include/microlink.h index 114b740..72feacb 100644 --- a/components/microlink/include/microlink.h +++ b/components/microlink/include/microlink.h @@ -65,6 +65,12 @@ typedef enum { ML_STATE_CONNECTED, ML_STATE_RECONNECTING, ML_STATE_ERROR, + /* Registration was rejected by the control plane (expired/revoked auth + * key, or the node awaits authorization). The stack keeps retrying at + * maximum backoff, but recovery normally needs a new auth key — hosts + * should surface this to the user as their equivalent of a "re-login" + * prompt (headless devices have no UI to notice it otherwise). */ + ML_STATE_AUTH_FAILED, } microlink_state_t; /* Callback types */ diff --git a/components/microlink/src/ml_coord.c b/components/microlink/src/ml_coord.c index 7a02812..d5c5865 100644 --- a/components/microlink/src/ml_coord.c +++ b/components/microlink/src/ml_coord.c @@ -687,6 +687,9 @@ static int do_h2_preface(microlink_t *ml, ml_noise_state_t *noise) { * State: REGISTER - Send RegisterRequest, parse RegisterResponse * ========================================================================== */ +/* do_register: the control plane rejected our auth (vs. -1 = I/O error) */ +#define ML_REG_AUTH_FAILED (-2) + static int do_register(microlink_t *ml, ml_noise_state_t *noise) { int64_t t_reg_start = esp_timer_get_time(); @@ -714,7 +717,7 @@ static int do_register(microlink_t *ml, ml_noise_state_t *noise) { cJSON *hostinfo = cJSON_CreateObject(); const char *dev_name = (ml->config.device_name && ml->config.device_name[0]) ? ml->config.device_name : microlink_default_device_name(); cJSON_AddStringToObject(hostinfo, "Hostname", dev_name); - cJSON_AddStringToObject(hostinfo, "OS", "linux"); + cJSON_AddStringToObject(hostinfo, "OS", "esp32"); cJSON_AddStringToObject(hostinfo, "OSVersion", "ESP-IDF"); cJSON_AddStringToObject(hostinfo, "GoArch", "arm"); @@ -946,10 +949,24 @@ static int do_register(microlink_t *ml, ml_noise_state_t *noise) { } /* The server reports a refused registration only in Error — a rejected auth * key gives "invalid key: unable to validate API key". Node is absent either - * way, so without this the first symptom is MapResponse "node not found". */ + * way, so without this check a rejected registration looked like success + * and the client spun in the reconnect loop with a confusing downstream + * MapResponse "node not found" instead of a clear auth-failure signal. */ const cJSON *reg_error = cJSON_GetObjectItem(resp_json, "Error"); if (cJSON_IsString(reg_error) && reg_error->valuestring[0] != '\0') { ESP_LOGE(TAG, "RegisterResponse error: %s", reg_error->valuestring); + cJSON_Delete(resp_json); + parse_start[parse_len] = saved; + free(resp_buf); + return ML_REG_AUTH_FAILED; + } + const cJSON *machine_authorized = cJSON_GetObjectItem(resp_json, "MachineAuthorized"); + if (cJSON_IsBool(machine_authorized) && !cJSON_IsTrue(machine_authorized)) { + ESP_LOGE(TAG, "Registration not authorized by control plane"); + cJSON_Delete(resp_json); + parse_start[parse_len] = saved; + free(resp_buf); + return ML_REG_AUTH_FAILED; } parse_start[parse_len] = saved; @@ -1337,7 +1354,7 @@ static int do_fetch_peers(microlink_t *ml, ml_noise_state_t *noise) { cJSON *hostinfo = cJSON_CreateObject(); const char *dev_name = (ml->config.device_name && ml->config.device_name[0]) ? ml->config.device_name : microlink_default_device_name(); cJSON_AddStringToObject(hostinfo, "Hostname", dev_name); - cJSON_AddStringToObject(hostinfo, "OS", "linux"); + cJSON_AddStringToObject(hostinfo, "OS", "esp32"); cJSON_AddStringToObject(hostinfo, "OSVersion", "ESP-IDF"); cJSON_AddStringToObject(hostinfo, "GoArch", "arm"); cJSON_AddItemToObject(root, "Hostinfo", hostinfo); @@ -1822,7 +1839,7 @@ static int do_start_long_poll(microlink_t *ml, ml_noise_state_t *noise) { if (hostinfo) { const char *dev_name = (ml->config.device_name && ml->config.device_name[0]) ? ml->config.device_name : microlink_default_device_name(); cJSON_AddStringToObject(hostinfo, "Hostname", dev_name); - cJSON_AddStringToObject(hostinfo, "OS", "linux"); + cJSON_AddStringToObject(hostinfo, "OS", "esp32"); cJSON_AddStringToObject(hostinfo, "OSVersion", "ESP-IDF"); cJSON_AddStringToObject(hostinfo, "GoArch", "arm"); cJSON_AddItemToObject(root, "Hostinfo", hostinfo); @@ -1927,7 +1944,7 @@ static int do_send_endpoint_update(microlink_t *ml, ml_noise_state_t *noise) { if (hostinfo) { const char *dev_name = (ml->config.device_name && ml->config.device_name[0]) ? ml->config.device_name : microlink_default_device_name(); cJSON_AddStringToObject(hostinfo, "Hostname", dev_name); - cJSON_AddStringToObject(hostinfo, "OS", "linux"); + cJSON_AddStringToObject(hostinfo, "OS", "esp32"); cJSON_AddStringToObject(hostinfo, "OSVersion", "ESP-IDF"); cJSON_AddStringToObject(hostinfo, "GoArch", "arm"); cJSON_AddItemToObject(root, "Hostinfo", hostinfo); @@ -2266,12 +2283,26 @@ void ml_coord_task(void *arg) { case COORD_REGISTER: ESP_LOGI(TAG, "Registering..."); - if (do_register(ml, &noise) < 0) { - ESP_LOGE(TAG, "Registration failed"); - ml_close_sock(ml->coord_sock); - ml->coord_sock = -1; - state = COORD_RECONNECTING; - break; + { + int reg_rc = do_register(ml, &noise); + if (reg_rc < 0) { + ESP_LOGE(TAG, "Registration failed"); + if (reg_rc == ML_REG_AUTH_FAILED) { + /* Not transient: retrying with the same key will keep + * failing. Notify the host (its "re-login" prompt) and + * jump to max backoff — the loop keeps running in case + * the key is re-enabled server-side. */ + ml->state = ML_STATE_AUTH_FAILED; + if (ml->state_cb) { + ml->state_cb(ml, ML_STATE_AUTH_FAILED, ml->state_cb_data); + } + reconnect_attempts = 5; + } + ml_close_sock(ml->coord_sock); + ml->coord_sock = -1; + state = COORD_RECONNECTING; + break; + } } state = COORD_FETCH_PEERS; break; @@ -2503,6 +2534,10 @@ void ml_coord_task(void *arg) { break; } else { ESP_LOGE(TAG, "Key expired but no auth_key — manual re-provisioning needed!"); + ml->state = ML_STATE_AUTH_FAILED; + if (ml->state_cb) { + ml->state_cb(ml, ML_STATE_AUTH_FAILED, ml->state_cb_data); + } } } /* Warn 1 hour before expiry (rough uptime-based check) */ @@ -2560,7 +2595,18 @@ void ml_coord_task(void *arg) { ESP_LOGI(TAG, "Reconnecting in %lu ms (attempt %d)", (unsigned long)backoff_ms, reconnect_attempts + 1); - ml->state = ML_STATE_RECONNECTING; + /* Notify the host on the first entry into this state (a + * backoff wait re-enters this case repeatedly, so only fire + * once per reconnect cycle). Don't overwrite AUTH_FAILED — + * that's a "needs a new key" state the host should keep + * showing across retries, not something a plain reconnect + * attempt should paper over. */ + if (ml->state != ML_STATE_RECONNECTING && ml->state != ML_STATE_AUTH_FAILED) { + ml->state = ML_STATE_RECONNECTING; + if (ml->state_cb) { + ml->state_cb(ml, ML_STATE_RECONNECTING, ml->state_cb_data); + } + } /* Wait on command queue with backoff timeout (interruptible!) */ ml_coord_cmd_t wake_cmd; diff --git a/components/microlink/src/ml_derp.c b/components/microlink/src/ml_derp.c index 53402a1..37a27ac 100644 --- a/components/microlink/src/ml_derp.c +++ b/components/microlink/src/ml_derp.c @@ -38,36 +38,32 @@ static const char *TAG = "ml_derp"; #define DERP_CONNECT_TIMEOUT_MS 10000 /* ============================================================================ - * Custom BIO callbacks for non-blocking TLS I/O + * Custom BIO callbacks for TLS I/O * - * These wrap lwIP recv/send with guaranteed timeout behavior. - * We don't trust mbedtls_net_recv_timeout on lwIP because lwIP's select() - * can sometimes block indefinitely on ESP32. + * These wrap the socket layer (ml_read_sock/ml_write_sock) so both the + * lwIP and AT socket backends work transparently. recv is a plain blocking + * f_recv bounded by SO_RCVTIMEO on the socket (no f_recv_timeout / + * mbedtls_ssl_conf_read_timeout) — combining recv_timeout with a TLS 1.3 + * handshake fails with MBEDTLS_ERR_SSL_BAD_INPUT_DATA on some DERP relays. * ========================================================================== */ /** - * Custom recv with timeout for mbedtls BIO. - * Uses SO_RCVTIMEO on the socket as the timeout mechanism (simpler than select). - * Returns bytes read, or MBEDTLS_ERR_SSL_TIMEOUT, MBEDTLS_ERR_SSL_WANT_READ. + * Custom blocking recv for mbedtls BIO (f_recv signature, no timeout arg). + * SO_RCVTIMEO on the socket bounds the wait (set at both the TCP-connect + * phase and the post-handshake data phase); a timeout surfaces as + * EAGAIN/EWOULDBLOCK, mapped to MBEDTLS_ERR_SSL_WANT_READ. Using a plain + * f_recv (rather than f_recv_timeout + mbedtls_ssl_conf_read_timeout) avoids + * a TLS 1.3 MBEDTLS_ERR_SSL_BAD_INPUT_DATA failure on some DERP relays — + * matches the scheme ml_coord_tls.c-style control-plane TLS already uses. */ -static int ml_derp_bio_recv_timeout(void *ctx, unsigned char *buf, size_t len, - uint32_t timeout) { +static int ml_derp_bio_recv(void *ctx, unsigned char *buf, size_t len) { int fd = *(int *)ctx; if (fd < 0) return MBEDTLS_ERR_NET_INVALID_CONTEXT; - /* Set SO_RCVTIMEO to the requested timeout. - * If timeout is 0 (mbedTLS default = "no timeout"), use 10s as a sane - * default to avoid indefinite blocking on AT sockets. */ - uint32_t effective_timeout = (timeout > 0) ? timeout : DERP_CONNECT_TIMEOUT_MS; - struct timeval tv; - tv.tv_sec = effective_timeout / 1000; - tv.tv_usec = (effective_timeout % 1000) * 1000; - ml_setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv)); - int ret = (int)ml_read_sock(fd, buf, len); if (ret < 0) { if (errno == EAGAIN || errno == EWOULDBLOCK) { - return MBEDTLS_ERR_SSL_TIMEOUT; + return MBEDTLS_ERR_SSL_WANT_READ; } if (errno == EPIPE || errno == ECONNRESET) { return MBEDTLS_ERR_NET_CONN_RESET; @@ -77,6 +73,9 @@ static int ml_derp_bio_recv_timeout(void *ctx, unsigned char *buf, size_t len, } return MBEDTLS_ERR_NET_RECV_FAILED; } + if (ret == 0) { + return MBEDTLS_ERR_NET_CONN_RESET; + } return ret; } @@ -343,7 +342,8 @@ static void dispatch_derp_frame(microlink_t *ml, uint8_t frame_type, /** * Try to read one DERP frame. - * Uses mbedtls recv_timeout (100ms) so ssl_read never blocks indefinitely. + * SO_RCVTIMEO on the socket bounds each read so ssl_read never blocks + * indefinitely (timeout surfaces as WANT_READ from the blocking-recv BIO). * Returns: 1 = frame read and dispatched, 0 = timeout (no data), <0 = error */ static int poll_derp_read(microlink_t *ml) { @@ -638,6 +638,22 @@ void ml_derp_tx_task(void *arg) { * DERP Connection Management (called from coord task) * ========================================================================== */ +/* Free the mbedtls state initialized in ml_derp_connect's TLS phase and close + * the underlying socket. Used by both ml_derp_disconnect (graceful teardown, + * after close_notify) and the fail_tls cleanup path in ml_derp_connect + * (handshake-failure unwind), so the two call sites can't drift apart. + * Adapted from cplewes/microlink@b25b1eee: our ml_derp_conn_t has no + * entropy/ctr_drbg fields (RNG is PSA-owned post mbedTLS 4.x migration, see + * ESP_IDF_6X_COMPAT.md), so this frees only ssl/ssl_conf. */ +static void derp_free_tls_state(microlink_t *ml) { + mbedtls_ssl_free(&ml->derp.ssl); + mbedtls_ssl_config_free(&ml->derp.ssl_conf); + if (ml->derp.sockfd >= 0) { + ml_close_sock(ml->derp.sockfd); + ml->derp.sockfd = -1; + } +} + esp_err_t ml_derp_connect(microlink_t *ml) { /* Determine DERP host/port from DERPMap with node failover. * Always start from node 0 (the first/preferred node in the DERPMap). @@ -645,6 +661,9 @@ esp_err_t ml_derp_connect(microlink_t *ml) { * NOT on connection failure (to avoid bouncing between nodes). */ const char *derp_host = ML_DERP_HOST; int derp_port = ML_DERP_PORT; + /* Track the region actually used, for logging (differs from HomeDERP on fallback) */ + uint16_t derp_region_used = ml->derp_home_region ? ml->derp_home_region : ML_DERP_REGION; + bool node_selected = false; if (ml->derp_region_count > 0 && ml->derp_home_region > 0) { for (int i = 0; i < ml->derp_region_count; i++) { @@ -658,6 +677,7 @@ esp_err_t ml_derp_connect(microlink_t *ml) { if (ml->derp_regions[i].nodes[attempt].derp_port > 0) { derp_port = ml->derp_regions[i].nodes[attempt].derp_port; } + node_selected = true; break; } } @@ -666,10 +686,37 @@ esp_err_t ml_derp_connect(microlink_t *ml) { } } + /* Fallback for when HomeDERP is not present in the DERPMap (e.g. a + * control plane that advertises its own region IDs outside Tailscale's + * official numbering). Falling back to the default ML_DERP_HOST in that + * case is a dead end — auth/TLS won't succeed against a mismatched + * region — so pick the first usable node (not avoid, not stun-only, has + * a hostname) from the DERPMap actually handed to us instead. In the + * normal case HomeDERP is in the DERPMap so node_selected is already + * true and this loop never runs; behavior is unchanged. */ + if (!node_selected && ml->derp_region_count > 0) { + for (int i = 0; i < ml->derp_region_count && !node_selected; i++) { + ml_derp_region_t *r = &ml->derp_regions[i]; + if (r->avoid) continue; + for (int j = 0; j < r->node_count; j++) { + if (!r->nodes[j].stun_only && r->nodes[j].hostname[0]) { + derp_host = r->nodes[j].hostname; + derp_port = (r->nodes[j].derp_port > 0) ? r->nodes[j].derp_port + : ML_DERP_PORT; + derp_region_used = r->region_id; + node_selected = true; + ESP_LOGW(TAG, "Home DERP region %d not in DERPMap, falling back to region %d (%s)", + ml->derp_home_region, derp_region_used, derp_host); + break; + } + } + } + } + int64_t t_derp_start = esp_timer_get_time(); ESP_LOGI(TAG, "Connecting to DERP %s:%d (region %d)", - derp_host, derp_port, ml->derp_home_region ? ml->derp_home_region : ML_DERP_REGION); + derp_host, derp_port, derp_region_used); /* DNS resolve — accept IPv4 or IPv6 (carrier may be IPv6-only) */ struct addrinfo hints = { .ai_family = AF_UNSPEC, .ai_socktype = SOCK_STREAM }; @@ -712,22 +759,45 @@ esp_err_t ml_derp_connect(microlink_t *ml) { mbedtls_ssl_init(&ml->derp.ssl); mbedtls_ssl_config_init(&ml->derp.ssl_conf); - mbedtls_ssl_config_defaults(&ml->derp.ssl_conf, + /* Check every mbedTLS setup return and abort before mbedtls_ssl_handshake. + * A failing mbedtls_ssl_setup (typically ALLOC_FAILED under internal-RAM + * pressure) NULLs ssl->conf, so proceeding to handshake anyway returns + * BAD_INPUT_DATA and the real cause (OOM) shows up as a misleading + * "Bad input parameters" instead. */ + int cfg_ret; + cfg_ret = mbedtls_ssl_config_defaults(&ml->derp.ssl_conf, MBEDTLS_SSL_IS_CLIENT, MBEDTLS_SSL_TRANSPORT_STREAM, MBEDTLS_SSL_PRESET_DEFAULT); + if (cfg_ret != 0) { + ESP_LOGE(TAG, "ssl_config_defaults failed: -0x%04x", -cfg_ret); + goto fail_tls; + } mbedtls_ssl_conf_authmode(&ml->derp.ssl_conf, MBEDTLS_SSL_VERIFY_NONE); - mbedtls_ssl_conf_read_timeout(&ml->derp.ssl_conf, DERP_CONNECT_TIMEOUT_MS); - - mbedtls_ssl_setup(&ml->derp.ssl, &ml->derp.ssl_conf); - mbedtls_ssl_set_hostname(&ml->derp.ssl, derp_host); + /* Pin the DERP connection to TLS 1.2. When the DERP relay uses a + * Let's Encrypt ECDSA certificate, the ESP-IDF mbedTLS TLS 1.3 path + * can't parse the certificate's signature-algorithm OID and fails even + * under VERIFY_NONE (TLS 1.3 can't skip certificate processing). DERP + * also accepts TLS 1.2, so pin to 1.2 to bypass the 1.3 cert path. */ + mbedtls_ssl_conf_max_tls_version(&ml->derp.ssl_conf, MBEDTLS_SSL_VERSION_TLS1_2); + + cfg_ret = mbedtls_ssl_setup(&ml->derp.ssl, &ml->derp.ssl_conf); + if (cfg_ret != 0) { + ESP_LOGE(TAG, "mbedtls_ssl_setup failed: -0x%04x", -cfg_ret); + goto fail_tls; + } + cfg_ret = mbedtls_ssl_set_hostname(&ml->derp.ssl, derp_host); + if (cfg_ret != 0) { + ESP_LOGE(TAG, "ssl_set_hostname failed: -0x%04x", -cfg_ret); + goto fail_tls; + } /* Store socket fd BEFORE setting bio. * Use custom BIO callbacks that route through ml_read_sock/ml_write_sock, * which transparently support both lwIP and AT socket backends. - * Timeout is handled via SO_RCVTIMEO. */ + * Timeout is handled via SO_RCVTIMEO, not mbedtls_ssl_conf_read_timeout. */ ml->derp.sockfd = sock; mbedtls_ssl_set_bio(&ml->derp.ssl, &ml->derp.sockfd, - ml_derp_bio_send, NULL, ml_derp_bio_recv_timeout); + ml_derp_bio_send, ml_derp_bio_recv, NULL); /* TLS handshake - socket has 10s SO_RCVTIMEO from connect phase. */ int ret; @@ -738,9 +808,7 @@ esp_err_t ml_derp_connect(microlink_t *ml) { char err_buf[128]; mbedtls_strerror(ret, err_buf, sizeof(err_buf)); ESP_LOGE(TAG, "TLS handshake failed: %s", err_buf); - ml_close_sock(sock); - ml->derp.sockfd = -1; - return ESP_FAIL; + goto fail_tls; } int64_t t_derp_tls = esp_timer_get_time(); @@ -760,9 +828,7 @@ esp_err_t ml_derp_connect(microlink_t *ml) { ret = mbedtls_ssl_write(&ml->derp.ssl, (const uint8_t *)upgrade_req, strlen(upgrade_req)); if (ret < 0) { ESP_LOGE(TAG, "Failed to send HTTP upgrade"); - ml_close_sock(sock); - ml->derp.sockfd = -1; - return ESP_FAIL; + goto fail_tls; } /* Read HTTP response byte-by-byte until \r\n\r\n to avoid over-reading @@ -776,9 +842,7 @@ esp_err_t ml_derp_connect(microlink_t *ml) { while (resp_len < (int)sizeof(resp_buf) - 1) { if (ml_get_time_ms() - http_start > DERP_CONNECT_TIMEOUT_MS) { ESP_LOGE(TAG, "HTTP upgrade response timeout"); - ml_close_sock(sock); - ml->derp.sockfd = -1; - return ESP_FAIL; + goto fail_tls; } ret = mbedtls_ssl_read(&ml->derp.ssl, resp_buf + resp_len, 1); @@ -789,15 +853,11 @@ esp_err_t ml_derp_connect(microlink_t *ml) { continue; } ESP_LOGE(TAG, "HTTP upgrade read failed: -0x%04x", -ret); - ml_close_sock(sock); - ml->derp.sockfd = -1; - return ESP_FAIL; + goto fail_tls; } if (ret == 0) { ESP_LOGE(TAG, "Connection closed during HTTP upgrade"); - ml_close_sock(sock); - ml->derp.sockfd = -1; - return ESP_FAIL; + goto fail_tls; } resp_len++; @@ -814,9 +874,7 @@ esp_err_t ml_derp_connect(microlink_t *ml) { if (!found_end || strstr((char *)resp_buf, "101") == NULL) { ESP_LOGE(TAG, "DERP upgrade rejected: %.100s", resp_buf); - ml_close_sock(sock); - ml->derp.sockfd = -1; - return ESP_FAIL; + goto fail_tls; } ESP_LOGI(TAG, "HTTP 101 Switching Protocols received"); } @@ -845,17 +903,13 @@ esp_err_t ml_derp_connect(microlink_t *ml) { esp_err_t err = derp_recv_frame_header(ml, &frame_type, &frame_len, DERP_CONNECT_TIMEOUT_MS); if (err != ESP_OK) { ESP_LOGE(TAG, "Failed to read ServerKey frame header (err=%d)", err); - ml_close_sock(sock); - ml->derp.sockfd = -1; - return ESP_FAIL; + goto fail_tls; } if (frame_type != DERP_FRAME_SERVER_KEY || frame_len < 40) { ESP_LOGE(TAG, "Expected ServerKey frame (0x01), got 0x%02x len=%lu", frame_type, (unsigned long)frame_len); - ml_close_sock(sock); - ml->derp.sockfd = -1; - return ESP_FAIL; + goto fail_tls; } /* Read and verify 8-byte magic */ @@ -863,18 +917,14 @@ esp_err_t ml_derp_connect(microlink_t *ml) { static const uint8_t DERP_MAGIC[8] = {0x44, 0x45, 0x52, 0x50, 0xf0, 0x9f, 0x94, 0x91}; if (derp_tls_read_all(ml, magic, 8, DERP_CONNECT_TIMEOUT_MS) < 0) { ESP_LOGE(TAG, "Failed to read ServerKey magic"); - ml_close_sock(sock); - ml->derp.sockfd = -1; - return ESP_FAIL; + goto fail_tls; } if (memcmp(magic, DERP_MAGIC, 8) != 0) { ESP_LOGE(TAG, "Invalid DERP magic: %02x%02x%02x%02x%02x%02x%02x%02x", magic[0], magic[1], magic[2], magic[3], magic[4], magic[5], magic[6], magic[7]); - ml_close_sock(sock); - ml->derp.sockfd = -1; - return ESP_FAIL; + goto fail_tls; } ESP_LOGI(TAG, "DERP magic verified"); @@ -882,9 +932,7 @@ esp_err_t ml_derp_connect(microlink_t *ml) { uint8_t derp_server_key[32]; if (derp_tls_read_all(ml, derp_server_key, 32, DERP_CONNECT_TIMEOUT_MS) < 0) { ESP_LOGE(TAG, "Failed to read server key"); - ml_close_sock(sock); - ml->derp.sockfd = -1; - return ESP_FAIL; + goto fail_tls; } ESP_LOGI(TAG, "DERP server key received (first 8): %02x%02x%02x%02x%02x%02x%02x%02x", @@ -916,9 +964,7 @@ esp_err_t ml_derp_connect(microlink_t *ml) { size_t ciphertext_len = json_len + NACL_BOX_MACBYTES; uint8_t *ciphertext = malloc(ciphertext_len); if (!ciphertext) { - ml_close_sock(sock); - ml->derp.sockfd = -1; - return ESP_FAIL; + goto fail_tls; } if (nacl_box(ciphertext, @@ -929,9 +975,7 @@ esp_err_t ml_derp_connect(microlink_t *ml) { ) != 0) { ESP_LOGE(TAG, "NaCl box encrypt failed"); free(ciphertext); - ml_close_sock(sock); - ml->derp.sockfd = -1; - return ESP_FAIL; + goto fail_tls; } /* Build ClientInfo frame payload: nodekey(32) + nonce(24) + ciphertext */ @@ -939,9 +983,7 @@ esp_err_t ml_derp_connect(microlink_t *ml) { uint8_t *ci_payload = malloc(ci_payload_len); if (!ci_payload) { free(ciphertext); - ml_close_sock(sock); - ml->derp.sockfd = -1; - return ESP_FAIL; + goto fail_tls; } memcpy(ci_payload, ml->wg_public_key, 32); @@ -959,9 +1001,7 @@ esp_err_t ml_derp_connect(microlink_t *ml) { if (derp_write_frame(ml, DERP_FRAME_CLIENT_INFO, ci_payload, ci_payload_len) < 0) { ESP_LOGE(TAG, "Failed to send ClientInfo"); free(ci_payload); - ml_close_sock(sock); - ml->derp.sockfd = -1; - return ESP_FAIL; + goto fail_tls; } free(ci_payload); @@ -993,11 +1033,13 @@ esp_err_t ml_derp_connect(microlink_t *ml) { } /* Switch socket to short timeout for data phase. - * Long timeout was needed for TLS handshake, but polling must be fast. */ + * Long timeout was needed for TLS handshake, but polling must be fast. + * conf_read_timeout is deliberately not called: recv uses the blocking + * f_recv scheme (f_recv_timeout is NULL), so only SO_RCVTIMEO bounds + * reads — same policy as the TLS 1.3 Bad Input Data workaround above. */ { struct timeval tv = { .tv_sec = 0, .tv_usec = 200000 }; /* 200ms */ ml_setsockopt(ml->derp.sockfd, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv)); - mbedtls_ssl_conf_read_timeout(&ml->derp.ssl_conf, 200); } ml->derp.connected = true; @@ -1013,6 +1055,16 @@ esp_err_t ml_derp_connect(microlink_t *ml) { (t_derp_done - t_derp_tls) / 1000); ESP_LOGI(TAG, "DERP handshake complete, connected"); return ESP_OK; + +fail_tls: + /* Reached on any error after mbedtls_ssl_init/mbedtls_ssl_config_init. + * Without this teardown each failed handshake leaks the mbedtls state + * (~3-8 KB internal heap) until even a fresh handshake can't allocate + * buffers — microlink_rebind() reconnects DERP on every WiFi + * reconnect, so this compounds fast under captive-portal/bad-network + * conditions. Adapted from cplewes/microlink@38602ab0/@b25b1eee. */ + derp_free_tls_state(ml); + return ESP_FAIL; } void ml_derp_disconnect(microlink_t *ml) { @@ -1021,10 +1073,7 @@ void ml_derp_disconnect(microlink_t *ml) { if (ml->derp.sockfd >= 0) { mbedtls_ssl_close_notify(&ml->derp.ssl); - mbedtls_ssl_free(&ml->derp.ssl); - mbedtls_ssl_config_free(&ml->derp.ssl_conf); - ml_close_sock(ml->derp.sockfd); - ml->derp.sockfd = -1; + derp_free_tls_state(ml); } /* Drain TX queue */