v1.6.0.7 — the Web UI could start unable to read its own config

[fduflyer]

Two defects found during field testing of v1.6.0.6 on a touchscreen node.

The Web UI could reach first boot unable to read its own config

A fresh install left the settings panel refusing to load, with nothing in the
journal to say why. It later repaired itself, but only because droneaware refresh happens to re-apply the same permission grant as a side effect — so
the cause was never established. The evidence was in a volatile journal and had
been overwritten by the time anyone looked.

Three candidates were eliminated with evidence rather than reasoning:

• sed -i on the first-boot token write — tested against a real 660 root:droneaware file in a setgid directory; ownership and mode survive intact
• install ordering — useradd → _grant_webui_access → enable is correct,
  and the missing-group case already warns
• the config-file cache — it retries on every call and does not trap failures

Rather than offer a fourth guess, droneaware-web.service now enforces the
access it needs on every start, through an ExecStartPre that runs as root
before the unit drops to User=droneaware. Idempotent, and it closes the class
rather than an instance: a node in this state now repairs itself at the next
boot regardless of which step lost the grant.

The silence was the more expensive half of this bug. web_ui now reports an
unreadable config.env once per distinct fault — naming the uid it runs as,
the mode and owner it found, and the command that fixes it — and reports again
on recovery, so a repair is visible too. Latched, so a two-second status poll
cannot turn it into a log flood.

A divergence found while tracing it

The CLI's install-webui created the droneaware account without
--user-group — the flag install.sh adds deliberately so that the group
does not depend on USERGROUPS_ENAB in login.defs. The fix had been applied
to one copy of the logic and not the other.

This is latent, not the bug above: stock Pi OS defaults that setting to
yes, so the group is created anyway. On a node where it is no the group is
absent, _grant_webui_access returns quietly, and the symptom is identical.

The backlog count read high after a restart

A node that rebooted holding 2,589 undelivered events reported 2,953 of
them, converging only as the spool drained.

backlog_events is derived rather than counted — pending bytes over the average
event size — and that average comes from per-process counters that start at
zero. A feeder that has just started has nothing to measure, so it fell back to
a hardcoded 280 against a real average nearer 320. The estimate was least
accurate at exactly the moment the backlog was largest.

The average is now seeded from the segments already on disk: one bounded read of
the newest segment measures what this node's events actually cost. A failed
sample returns None rather than the fallback, so it is retried instead of
cached for the life of the process.

	reported	error
before	2,783	7.5%
after	2,598	0.3%

Within 0.5% throughout the drain, and zero stays exact — which is the reading
operators actually act on.

Validation

Unit validated with systemd-analyze verify (exit 0). Spool suite 38/38,
forwarder side-by-side harness against the shipping implementation 37/37, plus
12 new tests for the backlog estimate and 14 for the diagnostic logging.

Worth knowing

Deploying this via droneaware refresh will repair permissions as a side
effect and mask the very thing being tested. To exercise the guard, break it
deliberately first — chown root:root and chmod 600 on config.env, then
restart the service and confirm it comes back as root:droneaware 660.

The node now says when its own clock is wrong

ASTM F3411's System message carries an absolute timestamp — uint32 at bytes
20–23, seconds since 2019-01-01, GPS-derived. Three decoders (both feeders and
the server) were written to agree on this message's layout and all three
stopped at byte 14, so the field arrived on every System message and was
thrown away. "Kept in sync with that file" is a comment, not a check — the
cross-check that should have caught this was comparing against a copy of the
same blind spot.

It matters because it is the only absolute time in the whole broadcast, and
being GPS-derived it is right when the node's clock is not. A Pi has no RTC; one
that boots with no network restores a stale time and stamps everything it hears
with it. A node did exactly that on 2026-09-12 and backdated a flight by
9h 48m. Nothing on screen said so, because every age was computed from the
same wrong clock and therefore read correctly — the error was invisible
because it was consistent.

Location/Vector cannot substitute: its timestamp is seconds-past-the-hour, so it
is ambiguous modulo an hour and a ten-hour error shows up there as a
twelve-minute discrepancy.

Each feeder now measures the median offset between broadcast time and its own
clock and publishes it to its tmpfs state file; the Web UI shows "out of sync
by 9h 48m" once past a minute. This is the only such check an offline node
has — clock_synced answers "did NTP ever reach us", which on a node that
booted without a network is false in a way an operator cannot act on.

Measured, never acted on. A System message is an unauthenticated broadcast,
so setting the clock from one would let anything able to transmit RID walk a
node's clock — and the node would then mis-stamp its own spool and rate-limiter
windows too. The median across several aircraft means one hostile or broken
transmitter cannot move the estimate.

drone_time is written to the spool — the node's own history, which feeds
the replay — and stripped before the POST. The server decodes the same value
from the raw hex it already receives, so the field buys it nothing, and an
unrecognised key is not something to introduce into a working ingest path. The
strip handles sub-messages inside a message pack, which is where most of these
timestamps actually arrive.

Tests assert the two feeder copies produce identical output for identical bytes
including malformed input, and the forwarder harness asserts end to end that the
field reaches the spool and never the wire.