Skip to content

Set up the dead-man's heartbeat on the Pi #55

Description

@baz8080

sudo esb test-alert on the Pi delivers the webhook but reports that ESB_HEARTBEAT_URL is not set. grep -c '^ESB_HEARTBEAT_URL=.' /etc/esb-outages.env prints 0, so the heartbeat described in notes/alerting.md § The heartbeat has never been active there.

Without it, the collector not running at all (Pi off, timer disabled, dead SD card, lost uplink) reaches nobody. The webhook can only report failures the collector itself sees. The only other signal is the site's stale banner, which trips 10 hours after the last push.

To do

  1. Create a check at healthchecks.io (the free tier is enough) with a 30-minute period and a 90-minute grace, and copy its ping URL.
  2. Add ESB_HEARTBEAT_URL=https://hc-ping.com/<uuid> to /etc/esb-outages.env on the Pi (sudo nano). Use no quotes and no spaces around =. The URL is a secret, like the ntfy topic.
  3. Run sudo esb test-alert. It should end with heartbeat delivered, and the check should turn green.
  4. Leave it an hour and confirm the timer's runs keep the check green. Each run reads the file fresh, so nothing needs restarting.

Done when the check has been green through at least one six-hourly backup slot, and test-alert reports both channels delivered.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions