Skip to content

Give every node its own mDNS name, and share owl.local between them - #53

Merged
Purple10101 merged 2 commits into
mainfrom
20260821-mdns-fleet-discovery
Aug 31, 2026
Merged

Purple10101 merged 2 commits into
mainfrom
20260821-mdns-fleet-discovery

Conversation

@Purple10101

Copy link
Copy Markdown
Contributor

Why

Every node ships with /etc/hostname set to owl, so on a LAN with more than one of them Avahi decides by boot race who gets owl.local and who gets owl-2.local, and the answer reshuffles on every reboot. No node has an address worth writing down, and when the holder of owl.local is unplugged the name dies with it, because Avahi never reclaims a name it lost.

avahi-alias-retina.service was worse. It had every node publish retina.local as a unique record, which cannot work with more than one node. It was failing continuously on all four nodes checked: 826, 1438, 339 and 466 restarts. It does not even fail cleanly, it aborts on an assertion.

What this does

Two names, published two different ways:

Name Mode Published by
ret<node_id>.local unique, probed, Avahi's own host-name that node only
owl.local shared, unprobed every node, always

Because owl.local is a shared record, all nodes answer it at once and the client picks one. No election, no reclaim, no failover, no split brain. /etc/hostname stays owl, so Docker, log tags and Mender are untouched.

Nodes no longer contend for a name, so owl-2.local stops occurring at all. Discovery for the GUI is DNS-SD (_owl-node._tcp), which is multi-instance by design.

Two things worth a reviewer's attention

avahi-publish -a cannot make a shared record, and neither can the AddAddress call under it: avahi_server_add_address() ORs in AVAHI_PUBLISH_UNIQUE regardless of caller flags (avahi-core/entry.c, 0.8). EntryGroup.AddRecord passes flags through untouched, which is why the alias talks DBus. Adds python3-dbus.

Also fixes a pre-existing bug. Avahi advertises on every interface by default, and a node running the radar stack has half a dozen Docker bridges. Their 172.x addresses were going out under the node's name: avahi-resolve -n owl.local on a live node answered 172.18.0.1. Avahi is now restricted to interfaces with a backing device in /sys.

Verified on hardware

Two nodes locally and two remote. Both answer owl.local simultaneously (confirmed with a RecordBrowser enumerating every A record on the wire). Taking one fully off the network left owl.local resolving with no gap. A cold boot puts the name in place before avahi-daemon starts. Mixed-fleet contention resolves safely in both boot orders. owl-2.local and retina.local are gone.

Ansible syntax check passes and ansible-lint reports only fqcn style findings, which match the rest of the repo (46 short module names against 1 FQCN across existing roles).

Not yet verified

The role has never run in a real EDI build. Scripts were tested against fixtures and installed by hand. Worth a workflow_dispatch build on this branch before merging, which publishes nothing.

No client device has resolved these names. All testing was node to node over Linux and avahi.

MUST LAND WITH

offworldlabs/retina-gui#PENDING, which ships the fleet banner. Merged without it, nodes get the naming layer with no UI that uses it. Merged the other way round, they get a banner with nothing to discover.

Behaviour change for users

retina.local stops resolving. It works today on every single-node site.

Purple10101 and others added 2 commits August 21, 2026 17:51
…ween them

All nodes ship with /etc/hostname set to "owl", so on a LAN with more than one
of them Avahi arbitrated by boot race: first to boot got owl.local, the rest got
owl-2.local, and it reshuffled on every reboot. Nothing had an address worth
writing down, and when the holder of owl.local was unplugged the name died with
it, because Avahi never reclaims a name it lost.

New mdns_identity role, two names published two different ways:

  ret<node_id>.local  unique, probed, Avahi's own host-name. Derived from the
                      Mender node_id, so it is the same for the life of the
                      board. Nodes no longer contend, so owl-2.local is gone.

  owl.local           shared, unprobed, published by every node at once. All of
                      them answer, the client picks one, and a node dying is a
                      non-event because there was never a single holder to fail
                      over from. No election, no reclaim, no split brain.

/etc/hostname stays "owl". Docker, log tags, the shell prompt and Mender all
want the generic name; only what Avahi puts on the wire changes.

Nothing here depends on how many nodes exist: every node does the same thing on
every boot. The node count is consulted in exactly one place, retina-gui,
deciding whether to show the fleet page or redirect to the only node it can see.

## Why the alias talks DBus

avahi-publish -a cannot make a shared record and neither can the AddAddress call
under it: avahi_server_add_address() ORs in AVAHI_PUBLISH_UNIQUE regardless of
what the caller passed (avahi-core/entry.c, 0.8), so the record is always probed
and always collides. ALLOW_MULTIPLE is not the way out either. AddAddress
rejects it as invalid and forces it on internally, and it means "multiple local
records" for a multi-homed host, not "let other hosts answer this name".

EntryGroup.AddRecord passes flags through untouched, so omitting UNIQUE gives a
genuinely shared record, and it takes an explicit TTL. Adds python3-dbus.

## Retires avahi-alias-retina

That published retina.local the broken way, from every node, and had been
failing on both test nodes for a long time: 826 and 1438 restarts. It does not
even fail cleanly, it aborts on an assertion after trying to pick an alternative
name for an address record and getting NULL. owl.local covers what it was for.

## Also fixes a pre-existing bug

Avahi advertises on every interface by default, and a node running the radar
stack has half a dozen Docker bridges. Their 172.x addresses went out under the
node's name, so `avahi-resolve -n owl.local` on a live node answered 172.18.0.1,
which is reachable from nowhere. Avahi is now restricted to interfaces with a
backing device in /sys, which keeps end0 and wlan0 and drops every bridge and
veth. Names are not used for the test: the br-<hash> ones are generated per
compose network.

## Verified on two nodes

Both answer owl.local at once (confirmed with a RecordBrowser enumerating every
A record on the wire, from each node independently). Taking one fully off the
network left owl.local resolving with no gap and the survivor back to
redirecting; restoring it brought both back. owl-2.local and retina.local are
gone. A cold boot puts the name in place before avahi-daemon starts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Renaming and the node's own address moved off the home page onto the config
page, so three references here were stale.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Purple10101
Purple10101 merged commit 7e2b228 into main Aug 31, 2026
1 check passed
@Purple10101
Purple10101 deleted the 20260821-mdns-fleet-discovery branch September 6, 2026 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant