Skip to content

Add MQTT / Home Assistant integration with auto-discovery - #101

Open
vfranchi wants to merge 2 commits into
MatixYo:mainfrom
vfranchi:pr/mqtt-homeassistant
Open

vfranchi wants to merge 2 commits into
MatixYo:mainfrom
vfranchi:pr/mqtt-homeassistant

Conversation

@vfranchi

@vfranchi vfranchi commented Sep 30, 2026 •

Copy link
Copy Markdown

The radar publishes itself to Home Assistant over MQTT with auto-discovery:
ten entities (range select, miles/runways switches, latitude/longitude
numbers, and sensors for aircraft count, nearest aircraft, Wi-Fi RSSI, free
heap and device info), so there is no YAML on the HA side. Commands arrive on

/cmd/# and are applied to NVS and re-published as state, so HA always shows the value the radar actually accepted.

Configuration is eight fields in the existing config portal, stored in NVS.
Every field starts empty on purpose: the firmware ships no broker address,
user or password, and an empty host keeps MQTT completely inert.

Details that were measured, not guessed, on an ESP32-C3 Super Mini:

  • The broker session is kept (cleanSession=false, commands subscribed at
    QoS 1). With PubSubClient's default clean session the broker discards the
    subscription and any command published while the connection is down, which
    showed up as switch taps in Home Assistant being silently ignored.
  • Discovery publishes one entity at a time with a 250 ms gap, and the number
    entities declare step=0.001: Home Assistant drops a discovery message whose
    step is below that, without logging anything on the broker.
  • Memory. There is no room for a second TLS stack on this chip, so MQTT runs
    plain TCP on 1883 only. The ADS-B poll now keeps its TLS session alive
    between polls (static WiFiClientSecure/HTTPClient + setReuse(true)): the
    handshake costs ~1 s of CPU here, and paying it every poll is what made
    mbedtls_ssl_setup() -- two 16 KB record buffers, no
    CONFIG_MBEDTLS_SSL_VARIABLE_BUFFER_LENGTH on the C3 -- fail once the MQTT
    client was connected. The MQTT client also stays connected across the
    fetch (fetchInProgress() keeps the two off the same socket), and the fetch
    task computes the nearest aircraft while it holds the parsed list instead
    of the MQTT task copying 3.3 KB of aircraft on every publish.

Measured over 120 s with MQTT enabled: 0 TLS allocation failures, 22 ADS-B
polls, one MQTT connect, TLS session reused on 21 of 22 polls. The same
window before the connection reuse: 799 failures, 5 polls.

This relies on the RGB332 frame buffer (#100) for headroom: at 16bpp the
frame sprite left no contiguous block for those 16 KB buffers.

Build flags: -DMQTT_MAX_PACKET_SIZE=768, because PubSubClient's default 256 B
truncates discovery payloads silently (the ceiling covers topic + payload +
header together), and -DMQTT_KEEPALIVE=60.

Captura de tela 2026-09-30 140524

The radar publishes itself to Home Assistant over MQTT with auto-discovery:
ten entities (range select, miles/runways switches, latitude/longitude
numbers, and sensors for aircraft count, nearest aircraft, Wi-Fi RSSI, free
heap and device info), so there is no YAML on the HA side. Commands arrive on
<base>/cmd/# and are applied to NVS and re-published as state, so HA always
shows the value the radar actually accepted.

Configuration is eight fields in the existing config portal, stored in NVS.
Every field starts empty on purpose: the firmware ships no broker address,
user or password, and an empty host keeps MQTT completely inert.

Details that were measured, not guessed, on an ESP32-C3 Super Mini:

- The broker session is kept (cleanSession=false, commands subscribed at
  QoS 1). With PubSubClient's default clean session the broker discards the
  subscription and any command published while the connection is down, which
  showed up as switch taps in Home Assistant being silently ignored.
- Discovery publishes one entity at a time with a 250 ms gap, and the number
  entities declare step=0.001: Home Assistant drops a discovery message whose
  step is below that, without logging anything on the broker.
- Memory. There is no room for a second TLS stack on this chip, so MQTT runs
  plain TCP on 1883 only. The ADS-B poll now keeps its TLS session alive
  between polls (static WiFiClientSecure/HTTPClient + setReuse(true)): the
  handshake costs ~1 s of CPU here, and paying it every poll is what made
  mbedtls_ssl_setup() -- two 16 KB record buffers, no
  CONFIG_MBEDTLS_SSL_VARIABLE_BUFFER_LENGTH on the C3 -- fail once the MQTT
  client was connected. The MQTT client also stays connected across the
  fetch (fetchInProgress() keeps the two off the same socket), and the fetch
  task computes the nearest aircraft while it holds the parsed list instead
  of the MQTT task copying 3.3 KB of aircraft on every publish.

Measured over 120 s with MQTT enabled: 0 TLS allocation failures, 22 ADS-B
polls, one MQTT connect, TLS session reused on 21 of 22 polls. The same
window before the connection reuse: 799 failures, 5 polls.

This relies on the RGB332 frame buffer (MatixYo#100) for headroom: at 16bpp the
frame sprite left no contiguous block for those 16 KB buffers.

Build flags: -DMQTT_MAX_PACKET_SIZE=768, because PubSubClient's default 256 B
truncates discovery payloads silently (the ceiling covers topic + payload +
header together), and -DMQTT_KEEPALIVE=60.
@vfranchi
vfranchi force-pushed the pr/mqtt-homeassistant branch from 371228b to e48812a Compare September 30, 2026 19:48
@nanamitm

nanamitm commented Oct 2, 2026

Copy link
Copy Markdown

MQTT settings from the portal are never persisted

In onPortalParamsSaved() (src/services/wifi_setup.cpp, line 158 on this branch), refreshPortalParamValues() runs before services::mqtt::saveFromPortal(). The refresh (added in #99) calls loadConfig() and setValue()s the stored MQTT values back into the WiFiManagerParameters, so by the time saveFromPortal() reads s_param_mqtt_*.getValue() the submitted form values have already been replaced by the old ones. With a fresh device the host stays empty forever, and the log keeps printing MQTT: enabled but no broker host configured.

Reproduced on an ESP32-C3 SuperMini: entered a broker host on /param, pressed Save, and the field came back empty and nothing was stored in NVS. Moving the refresh to the end fixes it:

void onPortalParamsSaved() {
  if (!services::location::saveFromStrings(s_param_lat.getValue(),
                                           s_param_lon.getValue())) {
    Serial.println("Invalid lat/lon in portal — keeping previous location");
  }
  ui::radar::saveMilesFromPortal(s_param_miles.getValue());
  ui::radar::saveRunwaysFromPortal(s_param_runways.getValue());
  services::mqtt::saveFromPortal(
      s_param_mqtt_host.getValue(), s_param_mqtt_port.getValue(),
      s_param_mqtt_user.getValue(), s_param_mqtt_pass.getValue(),
      s_param_mqtt_topic.getValue(), s_param_mqtt_prefix.getValue(),
      s_param_mqtt_name.getValue(), s_param_mqtt_on.getValue());

  // Re-render only after every setting is persisted.
  refreshPortalParamValues();
}

With that change it connects to a Mosquitto broker (Home Assistant add-on) with auth and all 10 discovery entities are published.

Minor: MQTT settings only apply after a reboot

services::mqtt::init() reads the config once at boot, and nothing re-initializes the client after a portal save. So after entering the broker details the device stays in State::Disabled until it is restarted. Either re-running init() (with a disconnect) from the save callback, or at least a note in the README / portal ("restart to apply MQTT changes"), would avoid confusion.

Bugs in the config portal path, all in the MQTT fields:

- read the parameters BEFORE refreshPortalParamValues(): it reloads the saved
  config into every parameter, so running it first replaced what the user had
  just typed with the old values -- and saveFromPortal() then stored those
  back. The portal could not change the broker at all.
- call services::mqtt::init() after the save so a new host or port takes effect
  without a reboot. For that to be safe, init() closes the session it replaces:
  it publishes "offline" (retained) on the old base first, because a clean
  DISCONNECT suppresses the will and an undriven session never loses its
  socket.
- reset the discovery burst in init(): a save can rename the base topic, the
  prefix or the device, and waiting out kMqttDiscoveryRefreshMs (10 min) looks
  exactly like a broken entity.
- "<br/>" in front of the MQTT broker host label: WiFiManager emits a text
  field's label glued to the checkbox label above it (WFM_LABEL_AFTER).

wifi_setup.cpp now includes services/mqtt_client.h for init().
@vfranchi

vfranchi commented Oct 2, 2026

Copy link
Copy Markdown
Author

MQTT settings from the portal are never persisted

In onPortalParamsSaved() (src/services/wifi_setup.cpp, line 158 on this branch), refreshPortalParamValues() runs before services::mqtt::saveFromPortal(). The refresh (added in #99) calls loadConfig() and setValue()s the stored MQTT values back into the WiFiManagerParameters, so by the time saveFromPortal() reads s_param_mqtt_*.getValue() the submitted form values have already been replaced by the old ones. With a fresh device the host stays empty forever, and the log keeps printing MQTT: enabled but no broker host configured.

Reproduced on an ESP32-C3 SuperMini: entered a broker host on /param, pressed Save, and the field came back empty and nothing was stored in NVS. Moving the refresh to the end fixes it:

void onPortalParamsSaved() {
  if (!services::location::saveFromStrings(s_param_lat.getValue(),
                                           s_param_lon.getValue())) {
    Serial.println("Invalid lat/lon in portal — keeping previous location");
  }
  ui::radar::saveMilesFromPortal(s_param_miles.getValue());
  ui::radar::saveRunwaysFromPortal(s_param_runways.getValue());
  services::mqtt::saveFromPortal(
      s_param_mqtt_host.getValue(), s_param_mqtt_port.getValue(),
      s_param_mqtt_user.getValue(), s_param_mqtt_pass.getValue(),
      s_param_mqtt_topic.getValue(), s_param_mqtt_prefix.getValue(),
      s_param_mqtt_name.getValue(), s_param_mqtt_on.getValue());

  // Re-render only after every setting is persisted.
  refreshPortalParamValues();
}

With that change it connects to a Mosquitto broker (Home Assistant add-on) with auth and all 10 discovery entities are published.

Minor: MQTT settings only apply after a reboot

services::mqtt::init() reads the config once at boot, and nothing re-initializes the client after a portal save. So after entering the broker details the device stays in State::Disabled until it is restarted. Either re-running init() (with a disconnect) from the save callback, or at least a note in the README / portal ("restart to apply MQTT changes"), would avoid confusion.

Thanks, patches applied. I tested on a board that already had all the settings in place so I never tested on a clean board to notice the portal issue. My bad.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants