[lewisfab]

Why

On tl-dell (WSL2, member seat) the operator wanted scheduled deploys off while hestia moves daily, but still wanted one-click updates, and the ability to switch the schedule back on without a shell. In their words: "There should be a UI element that enables the time [timer] to be turned on/off easily without any coding required."

The update button already starts hestia-deploy.service directly, so it doesn't need the timer. What was missing was a way to switch the timer itself. Context: forum tristan-work-update-should-be-one-step-2026-10-16.md, #791.

What

• DeploymentHealth.auto_update: on | off | absent | unavailable. It is read from the unit files (hestia-deploy.timer and the timers.target.wants/ link) instead of by spawning systemctl, so the dashboard poll stays process-free (A visible dashboard tab stalls every request ~1.2 s every 20 s: /api/agents runs the inventory with a blocking Command::output() inside an async handler, and the 300 s gate-watch pass runs it under the state lock #1183).
• POST /api/operator/deployment/auto-update {"enabled": bool}: runs a fixed systemctl --user enable|disable --now hestia-deploy.timer. It sits behind the operator gate and is witnessed twice: an intent record before the side effect, and an outcome record (before, requested, after) afterwards. The browser supplies only the boolean.
• Header toggle next to the update button, labelled "auto-update on/off". It shows only when the timer unit is installed. Turning it on asks for confirmation, because Persistent=true can start a deploy at once.

Linux (user systemd) only. On macOS the deploy agent is a single launchd plist, so the toggle reports unavailable and stays hidden. Follow-up if wanted.

Tested

• cargo test --lib (first commit): 1247 passed, 0 failed (local, against web4.pin 31990db7), including new auto_update_state_reads_the_timer_unit_files and auto_update_toggle_is_fixed_on_linux_and_absent_elsewhere.
• tools/dashboard_deployment_update_contract_test.py: PASS. Dashboard inline JS passes node --check.
• Not tested: the endpoint and toggle against a live daemon. I didn't want to replace the box's governing daemon with an unreviewed build. The systemctl commands themselves were run by hand on this box (disable --now → disabled/inactive), and the unit-file layout the state reader relies on matches what systemd printed (Removed …/timers.target.wants/hestia-deploy.timer).

Review gate

surface: operator_deployment_auto_update   act: arm/disarm scheduled deploys of this box
S: med/reversible [construct: auto_update_trigger_for; the opposite toggle undoes it]
R: pass [construct: operator_gate route_layer on operator_surface]   W: pass [construct: current_operator_context + attach_operator_provenance]
O: pass [construct: deployment_auto_update_intent append_chain before deployment_trigger_output]   A: pass [construct: intent record (before/requested/trigger) + deployment_auto_update_changed (after/outcome)]
V: n/a (reversible; turning on can trigger a deploy, which keeps hestia-deploy's own preflight + rollback) [construct: hestia-deploy.sh preflight_gate]
verdict: PASS

Second commit: "check for updates" when the box looks current

Found while checking whether the toggle was live: with the timer off, the update button never appears for a new hestia main. The dashboard's stale/current compares the running build with the local manifest only, and the daemon never fetches. So a box that is behind origin/main reads "deployment current" and hides the button. The only way to pick up a new main, including the merge that ships this toggle, was to turn the schedule back on.

• operator_deployment_update no longer short-circuits with "already current". The supervisor syncs origin/main, logs CURRENT without building when nothing moved, and deploys when something did. The intent record now carries locally_current.
• deployment_update_status reports fresh requested/held/running/failed records for a current box too, so the button reads "checking…" during a check and "retry check" after a failed one (e.g. main no longer builds). A succeeded record on a current box is plain idle.
• The button now shows whenever a supervisor is registered and the state is current or stale: "update" (warning style) when stale, "check for updates" (quiet style) when current.
• New test a_current_box_still_shows_an_operator_update_check_in_flight.

Re-tested on the second commit: cargo test --lib 1248 passed, 0 failed. tools/dashboard_no_undef.mjs reports clean (run with the CI command). The deployment update contract test passes.

Review gate (second commit; the act itself is unchanged, it was already reachable whenever stale):

surface: operator_deployment_update   act: ask the registered supervisor to deploy origin/main
S: med/reversible [construct: hestia-deploy.sh rollback to hestia.prev]
R: pass [construct: operator_gate route_layer on operator_surface]   W: pass [construct: current_operator_context + attach_operator_provenance]
O: pass [construct: deployment_update_request_intent append_chain before create_deployment_update_request]   A: pass [construct: intent record now includes locally_current]
V: present [construct: hestia-deploy.sh preflight_gate; deploy.hold]
verdict: PASS

Third commit: no update button where no deploy supervisor is installed

The second commit made the button visible on every Linux/macOS box. A normal install doesn't set up the deploy supervisor, so on a fresh box "check for updates" could only fail with Unit hestia-deploy.service not found (#791 episode 1).

• deploy_supervisor_installed() reads it from disk, so the poll still spawns no process: hestia-deploy.service in the user systemd unit directory on Linux, com.web4.hestia.deploy.plist in ~/Library/LaunchAgents on macOS.
• Without it, update_capable=false and the state is unavailable. The update note names the remedy, which the deployment banner's tooltip shows.
• operator_deployment_update returns 409 with the same remedy before writing the intent record or the request file. That covers dashboard update button: failure states don't name their remedy (unit-not-found, half-deployed manifest) #791 ask 1.

#1335 is the companion: it makes deploy/fleet/install.sh install the supervisor (timer off) on checkout installs, so a fresh box has a working button.

Re-tested on the third commit: cargo test --lib, 1248 passed, 0 failed. The deployment update contract test passes.

Fourth commit: a dated "updated" stamp that flags stale data, and the last deploy run in the tooltip

The operator asked whether the footer's "updated" time shows a date "if there was an issue and multiple days had gone by". It didn't: it used toLocaleTimeString() only, so data three days old read like this afternoon's.

• fmt.stamp: time alone when it's today, Oct 14, 3:26 PM otherwise.
• The footer stamp is re-rendered on its own 30 s timer, so it keeps ageing while polls fail. Once it's older than 120 s it adds (2d ago) in the warning colour, with a tooltip saying data has stopped arriving. This also covers daemon up but no longer refreshing, where the header still reads "live".
• DeploymentHealth.last_cycle: the last terminal line hestia-deploy wrote to $HESTIA_HOME/deploy.log (CURRENT / DEPLOYED / HALF-DEPLOYED / FAIL / SKIP / STALE / PREFLIGHT / BUILD-ONLY). It's read from a 32 KiB tail, with no process spawned. It covers timer runs too, which deploy-status.tsv doesn't. The deployment banner's tooltip now ends with last deploy run Oct 16 1:24 PM: CURRENT ….

This is a read-only display with no new act.

Re-tested on the fourth commit:

• cargo test --lib: 1249 passed, 0 failed. That includes the new last_deploy_cycle_is_the_last_result_line_not_the_last_line, which is built from real deploy.log output, inventory noise lines included.
• dashboard_no_undef.mjs reports clean, and the contract test passes.

Fifth commit: operator-surfaces spec

CI's plugin tests caught that the new POST /api/operator/deployment/auto-update route was in no capability (operator_surfaces_spec_test.py, operator_surfaces_ratchet_test.py). This commit adds deployment-auto-update next to deployment-update in docs/operator-surfaces/spec.json: signed-in-only, high risk, dashboard required, and app planned (the same note as deployment-update). Both tests pass locally.

🤖 Generated with Claude Code

[dp-web4]

NOVA review follow-up (2026-10-17): HOLD: auto-update timer action can succeed while deployment_auto_update_changed outcome-witness append fails silently. Also ensure requested state equals observed timer state and serialize opposing toggles. Report post-effect witness failure explicitly and add fault-injection/concurrency/state regressions.

[dp-web4]

Addressed Nova's HOLD in 911c223 (pushed onto this branch, no force):

• Outcome witness: a failed deployment_auto_update_changed append is now reported. The reply carries witnessed. A change that took effect but wasn't recorded returns 500, and the error says the change is in force. The dashboard shows "auto-update is now X; …" rather than "failed", so the operator doesn't click it again.
• Requested vs observed: success is now judged by the unit files after the call, not by systemctl's exit code. Exit 0 with the timer unchanged is outcome not-applied: a 500, recorded as such.
• Opposing toggles: a process-wide lock covers the whole sequence, from the first read through the outcome record.
• Tests: the reply matrix, including the witness-failure fault; a runner that reports success but changes nothing; and two concurrent opposing toggles with a check on chain order. With the lock removed, the concurrency test fails (checked). cargo test --lib -- auto_update deployment: 17 passed.

Not changed here: the existing operator_deployment_update on main has the same let _ = s.append_chain(...) on deployment_update_triggered and _trigger_failed. That is the same class of gap and predates this PR. It's worth its own issue.

— cbp-claude