Review the collector: the raw line no longer waits on the database

[baz8080]

lift_status/ and scripts/ were written before PRs here went through a review agent, and have barely changed since (parse.py, client.py and __main__.py not at all). They are also the only code whose mistakes a rebuild cannot undo: a response that never reached the raw log is gone. So they got one whole-file review, and the rest of the repository did not: lift_access/ and most of lift_site/ have been reviewed diff by diff since 2026-08-26, and the real-corpus and golden tests pin what they publish.

The review made ten findings. Eight are fixed here, one turned out not to be a real path, and one is left for now. All ten are in notes/collector-review.md. A second review, of this PR itself, is addressed in the last commit.

Fixed

• A corrupt database cost the response. Store() opened SQLite before the raw line was written, so a database broken by a power cut stopped every poll from reaching the log, with a traceback and no alert. append_raw now writes the line before the database is opened, and it is the only way to write one. A database failure is exit 7, and the banner says what to do for a lock, a full card or corruption. A fetch that failed keeps its own alert even when the database is broken too.
• A full SD card never reached the storage alert. check_writable touches an empty file, which passes on a full disk. An OSError from the append now goes to the storage banner.
• A line cut off by a power cut swallowed the next one. The append now adds the missing newline first, so only the fragment is lost.
• A truncated or corrupt gzip body escaped the client. EOFError and zlib.error are neither an OSError nor an HTTPException. They are now a TransientError, so they get retried and logged.
• A second outage within a day of the first was silent. The alert dedup marker outlived a recovery. It now clears after four consecutive clean runs (two hours), and a suppressed failure resets the count, so a flapping API is still one alert a day.
• systemd could kill a poll before it logged anything. TimeoutStartSec=60 was below the client's own worst case, so it is now 300.
• The backup could hang forever. A oneshot unit has no start timeout by default, and ssh had no keepalive. ssh now has ConnectTimeout and ServerAliveInterval, and the unit has a 15-minute cap. The script traps TERM, so hitting the cap still alerts.
• A sort -u merge replayed out of order. sort -u orders lines by body, but replay followed line order, so two collectors' merged logs would have rebuilt the wrong history. Replay now sorts each file by fetched_at_utc, stably. The cost, written into the note: after a power cut, a catch-up poll stamped up to an hour early is applied before runs it followed. On the real data this moves nothing.

Left for now

• time-sync.target does not wait for NTP unless systemd-time-wait-sync.service is enabled. Enabling it risks polls that silently never run if NTP is unreachable, so it is left as it is, by the owner's call.

Deploying

The unit files and the backup script only take effect on the Pi after scripts/install-native.sh and a daemon-reload.

Verification

• ruff check and scripts/no-em-dash.sh are clean
• Full suite OK with LIFT_STATUS_DATA_DIR set. The 12 skips were already there: no survey directory, and running as root
• Collector suites pass on Python 3.11
• rebuild then stats against lifts-data gives identical output before and after, on every commit
• The backup script, run with a git fetch that hangs and its process group sent TERM, sends its alert with exit 143

🤖 Generated with Claude Code

https://claude.ai/code/session_01AnrcmsrTHYCgnVGyqBqji4