3.1.0: vpn_ip stays 0.0.0.0 while state=CONNECTED, control-plane last-seen never advances

[fudio101]

Symptom

After upgrading a device (LilyGo T-Display-S3, zen-clock firmware) from microlink 3.0.0 to
3.1.0, microlink_get_vpn_ip() returns 0 (displayed as 0.0.0.0) indefinitely, even though
microlink_get_state() reports ML_STATE_CONNECTED and the device is clearly functional at
the relay/peer layer:

• DERP connects successfully (DERP handshake complete, connected, DERP status: connected=1)
• DISCO probing and PONG exchanges succeed with ~10 known peers (RTT 30-900ms range)
• The control-plane long-poll stream is alive and receiving updates
  (ml_coord: Long-poll MapResponse update received)
• No RegisterResponse error, no Registration not authorized by control plane
  (MachineAuthorized check), no ML_STATE_AUTH_FAILED anywhere in the log

Despite this, the device's "Last seen" on the Tailscale admin console stays stuck at a fixed
timestamp and does not advance, even after several minutes of confirmed DERP/control-plane
activity — consistent with the control plane not treating this session as a fresh, fully
registered check-in.

What's ruled out

Diffed components/microlink/src/ml_coord.c between v3.0.0 and v3.1.0: the
Node.Addresses → ml->vpn_ip parsing block in do_register() is byte-for-byte unchanged
between the two versions. The only new registration-path logic in 3.1.0 is the added
MachineAuthorized/Error check (#51's PR, closes #27) — and that would produce a distinct
ML_STATE_AUTH_FAILED + "Registration not authorized by control plane" log line, which does
not appear. So this isn't (as far as the log evidence shows) that new check misfiring.

Known confound

ml_peer_nvs.c caches the peer table across reboots ("Loaded %d cached peers from NVS") and
that cache is loaded unconditionally at microlink_start(), independent of whether the current
session's registration actually completed with a valid Node.Addresses in the response. So
DERP connecting and DISCO-probing cached peers successfully does not by itself prove this
session's registration returned a usable self IP — it's necessary but not sufficient evidence
of a fully successful register.

What's still unconfirmed

Could not yet capture the actual RegisterResponse body / the "Our VPN IP: %s" log line for
this session — it fires within the first ~9s of boot, and reset-triggered serial captures on
this board (ESP32-S3 native USB-CDC re-enumerates on reset) keep missing that exact window.

Repro environment

• Board: LilyGo T-Display-S3 (ESP32-S3)
• microlink 3.1.0 (component_hash 9fb3633d7b0104b804722af15246579ef4c42531d22b00ae2271454d07e478e4),
  wireguard_lwip 1.0.3 transitively
• CONFIG_ML_ENABLE_CONFIG_HTTPD off, CONFIG_ML_MAX_PEERS=16
• Same device/auth key/tailnet worked correctly on 3.0.0 prior to this upgrade (no other change
  besides the dependency bump — consuming project's manifest, Kconfig-relevant settings, and
  WiFi/network environment are otherwise unchanged)

Next step

Plan to add temporary ESP_LOGI instrumentation around the Node.Addresses parse in
do_register() to dump the raw RegisterResponse JSON for one boot and see exactly what's
missing or malformed, then report back here with findings.

[fudio101]

Update: caught the actual RegisterResponse/MapResponse exchange

Managed a clean serial capture from true boot (t=0), not just the steady-state I had before.

I (7161) ml_coord: Registering...
I (7182) ml_coord: RegisterRequest: 319 bytes
I (7183) ml_coord: RegisterRequest sent (404 H2 bytes) [TIMING] send: 21 ms
I (7186) ml_coord: RegisterResponse Noise frame 0: 9 bytes
I (7428) ml_coord: RegisterResponse: 477 bytes total data
I (7432) ml_coord: RegisterResponse first 32 bytes (hex): 7b 22 55 73 65 72 22 3a 7b 22 49 44 22 3a
                                                            36 33 31 36 34 37 30 36 36 31 33 36 32 36 36 36 2c 22
                                                            = {"User":{"ID":631647066136266,"...
I (7446) ml_coord: [TIMING] Register total: 284 ms
I (7450) ml_coord: Fetching peers...

No "Our VPN IP: %s" line ever fires. do_register()'s Node→Addresses extraction
(ml_coord.c ~line 1000) silently no-ops — the top-level key visible in the first 32 bytes is
"User", not "Node", and whatever's later in the 477-byte body doesn't satisfy
if (node) { if (addresses && cJSON_GetArraySize(addresses) > 0) { ... } }. No error is logged
either way, since that block has no else branch.

Right after, the separate MapResponse parse (parse_peers_from_map_response) shows the same
shape of gap:

I (8070) ml_coord: MapResponse JSON: 532 bytes
I (8074) ml_coord: MapResponse first 32 bytes (hex): 10 02 00 00 7b 22 4e 6f 64 65 22 3a 7b 22 49 44 22 3a
                                                       = ...{"Node":{"ID":8315213622489822...
I (8105) ml_coord: Home DERP region: 9 (default)   <- falls back to the Kconfig/hardcoded
                                                        default, not a value read from the response

Hypothesis

This device's node key is persistent (load_or_generate_keys()), so every reboot re-registers
an already-known node rather than creating a fresh one. Tailscale's control plane does
delta-style responses: fields that haven't changed since the server believes the client already
knows them (like Node.Addresses, which is assigned once and never changes for a given node)
get omitted on repeat registration/poll rather than resent every time.

ml->vpn_ip lives only in the in-RAM microlink_t struct and is set exactly once, only from
that Node.Addresses parse — there's no persistence and no fallback, unlike the peer table
(ml_peer_nvs.c explicitly caches peers across reboots for this exact class of problem). So:

• First-ever registration of a brand-new node → server sends the full Node object
  including Addresses → vpn_ip gets set correctly, "Our VPN IP: ..." logs.
• Every reboot after that → server treats it as a known node, omits Addresses (and
  apparently HomeDERP too, per the "(default)" fallback above) → vpn_ip silently stays 0
  forever, even though DERP/WireGuard/peer traffic all work fine, because none of that
  actually depends on the node knowing its own address.

This also lines up with the reporter's separate observation that the Tailscale admin console's
"last seen" for this device wasn't advancing during testing — consistent with the client
under-declaring/never confirming its own registered state back to the operator-visible UI, even
while the data plane is healthy.

Where a fix likely belongs

components/microlink/src/ml_coord.c, near the Node.Addresses parse in do_register()
(~line 1000) and/or parse_peers_from_map_response():

• Persist vpn_ip in NVS the same way ml_peer_nvs.c persists peers, and load it as a
  fallback at init so a reboot that gets an elided Addresses field doesn't regress to 0.
• And/or fall back to reading the self address out of MapResponse's own Node object when
  RegisterResponse didn't include it (it's present there per the second hex dump above, just
  not currently read for this purpose).

Not yet confirmed whether this pre-dates the 3.0.0→3.1.0 bump (couldn't test 3.0.0 head-to-head
here since the device's node key is already established on the tailnet either way) — but given
the parsing code itself is unchanged between the two versions, it's a pre-existing latent gap
now surfaced by directly testing this upgrade rather than something the 3.1.0 diff introduced.