diff --git a/CHANGELOG.md b/CHANGELOG.md index 6016e38..92fcbbd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,44 @@ All notable changes to VesselStack are documented here. The project follows Semantic Versioning. +## Unreleased + +Planned for 0.6.2 (`VERSION` is not bumped here). + +### Highlights + +- The Control Panel's **Back up & install** and **update from folder** no longer fail when the backup + drive is not mounted. A dialog lets you, for that one request, mount the drive from `/etc/fstab`, + mount a connected USB drive (ext4 recommended, exFAT allowed; chosen by UUID from a list built as + root), back up to the system disk once, use a different backup folder, or install without a backup + (needs an explicit acknowledgement). Nothing is remembered, and neither `/etc/fstab` nor + `vesselstack.env` is written; `backup_mode` / `backup_skipped` appear in the operation record for + display only. `VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false` (in `vesselstack.env` or the process + environment, re-checked by the worker) turns off the system-disk and no-backup choices. A drive + mounted by UUID is verified after mounting; the `/etc/fstab` mount uses `nodev,nosuid,noexec`; + `--system-disk-once` is refused when the data size cannot be measured. `backup` is accepted only for + update actions. Without a choice the upgrade still requires a mounted backup. +- New controller commands: `vesselstackctl backup --system-disk-once`, `vesselstackctl backup-drive list + --json` and `vesselstackctl backup-drive mount --fstab | --uuid UUID`; `vesselstackctl update DIR + --backup-choice=system-disk-once|skip` is accepted for the Control Panel and the pinned installer. +- New read-only `GET /api/backup-preflight` for the dialog. + +### Fixes + +- `tests/test-lifecycle.sh` no longer fails intermittently on "Backup is missing the Prometheus data directory": the archive listing is read into a variable instead of `tar -t | grep -q` under `pipefail` (SIGPIPE when the match came early). Test-only; product behaviour is unchanged. +- "Back up to the system disk this once" (terminal menu, pinned installer with a current controller) + now writes to `/var/backups/vesselstack` and leaves the unmounted `/media` or `/mnt` folder empty, + so the drive can still be mounted there later. An older installed controller still writes into that + folder. +- Tests: the VesselTwin preload listing assertion no longer depends on filesystem directory order. + +### Upgrade notes + +This cannot help the upgrade that delivers it: the Control Panel worker uses the controller and panel +that are already installed. For the upgrade to 0.6.2, mount the drive by hand or run the update from a +terminal with `get-vesselstack.sh`. Rollback: unmount with `umount`; there is nothing to undo in +`/etc/fstab` or `vesselstack.env`. + ## 0.6.1 - 2026-11-02 ### Highlights diff --git a/README.md b/README.md index a6bba0a..f4ca3ad 100644 --- a/README.md +++ b/README.md @@ -454,6 +454,15 @@ than the fix); use `get-vesselstack.sh`, or mount the drive first, or run `sudo env VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=true vesselstackctl update ` to back up to the system disk once. +**Backup drive not mounted, from the Control Panel (0.6.2).** The Control Panel can now resolve this +itself, for that one request: **Back up & install** and **Back up & update from folder** check the +backup folder first and, when the drive is not mounted, show a dialog to mount it from `/etc/fstab`, +mount a listed USB drive, back up to the system disk once, use a different backup folder, or install +without a backup. See [Backups](#backups). This cannot help the upgrade that delivers it: the +worker uses the controller that is already installed, so for the upgrade to 0.6.2 mount the drive by +hand or run the update from a terminal. Rollback: unmount with `umount`; nothing is written to +`/etc/fstab` or `vesselstack.env`. + Rollback: set `BOAT_UNITS` back to `us_customary` (or `metric`) in the same file. 0.6.1 needs no other manual step. @@ -1160,7 +1169,10 @@ Nothing has been changed yet. - **[3]** checks the mount again after you plug the drive in. - **[4]** writes this one backup to the system disk after a warning. It is not offered when the free space is smaller than the data (`du -skx`, 60 s limit) or when `vesselstack.env` sets - `VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false`. Nothing is saved: the next backup asks again. + `VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false`. Since 0.6.2 the archive goes to + `/var/backups/vesselstack` on the system disk, not into the unmounted mount folder, which stays + empty (so the mount point is still free to mount later). Nothing is saved: the next backup asks + again. - **[c]**, end of input or no answer within 5 minutes cancels without changing anything (in the pinned installer it prints `Cancelled. The update was not started; your installed VesselStack was not changed.`; the packages and the release folder may already be in place). An answer must @@ -1174,6 +1186,45 @@ menu; use `get-vesselstack.sh` (it asks before calling the controller) or `sudo env VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=true vesselstackctl update `. Rollback: unmount the drive with `umount`; remove the added `/etc/fstab` line if you do not want it. +**Control Panel (0.6.2).** The panel cannot answer prompts, so before **Back up & install** and +**Back up & update from folder** it asks `GET /api/backup-preflight`. If the backup folder is on a +mounted drive the usual confirmation appears, unchanged. Otherwise a dialog offers, for this request +only (nothing is saved and `vesselstack.env` is never edited): + +- mount the drive from `/etc/fstab`, when there is an entry for the backup folder; +- mount one of the listed USB drives (chosen by its UUID from a list the root side produced; the + browser never sends a device path or a folder). The same rules as the terminal apply: ext4 + (recommended) or exFAT, removable, not part of the system disk, `nodev,nosuid,noexec` (exFAT also + `uid=0,gid=0,umask=0077`), an empty mount folder, and the post-mount ownership and symbolic-link + check. `/etc/fstab` is never written; refused and already-mounted drives are listed with the reason; +- back up to the system disk this once: `/var/backups/vesselstack` when the configured folder is on an + unmounted `/media` or `/mnt` mount point (otherwise the configured folder). It is refused when the + free space is smaller than the data; +- use a different backup folder: saves `VESSELSTACK_BACKUP` through the normal settings call, then + checks again (not available while Settings has unsaved edits); +- install without a backup: needs the box **I understand there is no backup to restore if this update + fails**. The result banner then says no backup was made. + +Setting `VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false` in `vesselstack.env`, or in the environment of the +panel or the controller, turns off the system-disk and no-backup choices (mounting remains available). +The Update worker checks this again right before it skips or redirects the backup, and the controller +refuses `--system-disk-once` while it is set. After a drive is mounted by UUID the controller checks +that the filesystem now at the mount folder has that UUID, and unmounts it again if not; the +`/etc/fstab` mount adds `nodev,nosuid,noexec`. The choice applies to `update` and `update-release` +only (`/api/action` answers 400 for any other action that carries `backup`), and `--system-disk-once` +is refused when the size of the data could not be measured. The choice is recorded in the operation +record as `backup_mode` / `backup_skipped` for display only; it is never written to `vesselstack.env` +or reused by a later request. The mount runs as root outside the panel's sandbox: +in the update worker for **Back up & install** (after the installer's `--dry-run`, before anything is +stopped), and in a transient `systemd-run --wait --pipe --collect --property=RuntimeMaxSec=300` unit for **update from folder** +(before the update starts). The controller gained `vesselstackctl backup --system-disk-once`, +`vesselstackctl backup-drive list --json` and `vesselstackctl backup-drive mount --fstab | --uuid UUID`; +`vesselstackctl update DIR --backup-choice=system-disk-once|skip` is accepted for the Control Panel and +the pinned installer and is not meant for terminal use (a terminal run shows the menu above). The +panel refuses these choices clearly when the installed controller is older. This cannot help the upgrade that delivers it: the Control Panel worker uses the controller and panel that are already installed. For the upgrade to 0.6.2 itself, mount the drive by hand (or run the update from a terminal with `get-vesselstack.sh`) as before. +Rollback: unmount with `umount`; to undo an update installed without a backup, restore an earlier +backup, if you have one (`vesselstackctl restore`, see [Updating](#updating)). + Besides the application and data trees the archive includes `/etc/systemd/system/vesselstack-*`, `/usr/local/sbin/vesselstack{ctl,-firewall}` and `/etc/vesselstack/update.env` @@ -1236,8 +1287,9 @@ live database files is insufficient. This is the one place in the README that describes updating and rolling back. Updates and backups started from the Control Panel never prompt; when one stops -for something you can fix, the result banner shows a **To fix:** line with the -exact step. +for something you can fix, the result banner shows a **To fix:** line with the exact step. +**Back up & install** and **update from folder** instead show a dialog when the backup drive is +not mounted (see [Backups](#backups)). 1. **Back up first** to a mounted external drive (see [Backups](#backups)): @@ -1272,7 +1324,8 @@ exact step. **Check for updates**. **Back up & install** asks for confirmation in a dialog, verifies the downloaded SHA-256 manifest, safely extracts the release, runs preflight, then makes and verifies a backup on a mounted external drive (`/media` or `/mnt`; the browser cannot change the destination). A failed or unmounted - backup stops the update. A detached updater restarts the Control Panel and verifies its + backup stops the update unless you choose, in the dialog the panel shows, to mount the drive, back + up to the system disk once or install without a backup. A detached updater restarts the Control Panel and verifies its new version before the browser reconnects. The panel uses ordinary HTTP and serializes its own update, configuration and component operations; do not run CLI maintenance commands at the same time. Transient updater files under `/var/lib/vesselstack` are not @@ -1352,7 +1405,8 @@ The backup folder is under `/media` or `/mnt` and the drive is not mounted. From USB drive, check again, or back up to the system disk once (see [Backups](#backups)). Without a terminal, or from an installed 0.6.0 or earlier controller, mount the drive and run again, or use `sudo env VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=true vesselstackctl update ` for one -backup on the system disk. A FAT32 (vfat) drive is refused; use ext4 or exFAT. +backup on the system disk. In the Control Panel (0.6.2 or later) **Back up & install** shows a dialog +for this instead of failing. A FAT32 (vfat) drive is refused; use ext4 or exFAT. ### The installer stops on `BOAT_UNITS` or "cannot be written safely" @@ -1448,6 +1502,10 @@ Run `python3 -m unittest discover -s tests -p 'test_*.py'` and the shell suites (protected temporary roots, CI jobs, and the ESPHome config test) as described in [Development: Developer validation](docs/DEVELOPMENT.md#developer-validation). +Test assertions on directory listings sort the result first, so they do not depend on filesystem order. +Archive-listing assertions read `tar -t` into a variable instead of piping it into `grep -q`: under `pipefail` an early match made `tar` die of SIGPIPE, so `tests/test-lifecycle.sh` failed on some filesystems, depending on directory order. +To reproduce CI on a Mac, run `sudo tests/test-lifecycle.sh` in a privileged `ubuntu:24.04` container with `jq`, `shellcheck`, `python3` and `util-linux` installed. + ## Documentation policy Every repository change must update this README in the same pull request. CI diff --git a/control-panel/app.py b/control-panel/app.py index 9dca9b7..1999037 100644 --- a/control-panel/app.py +++ b/control-panel/app.py @@ -24,6 +24,7 @@ try: from browser_updates import installed_version, release_catalog, read_operation, public_operation, release_snapshot, running_identity, acquire_slot, release_slot, start_detached + from browser_updates import normalize_backup_choice, effective_backup_choice, backup_locked, ctl_supports except ImportError: import importlib.util _spec = importlib.util.spec_from_file_location("browser_updates", Path(__file__).with_name("browser_updates.py")) @@ -35,6 +36,8 @@ acquire_slot = _updates.acquire_slot; release_slot = _updates.release_slot start_detached = _updates.start_detached release_snapshot = _updates.release_snapshot + normalize_backup_choice = _updates.normalize_backup_choice; effective_backup_choice = _updates.effective_backup_choice + backup_locked = _updates.backup_locked; ctl_supports = _updates.ctl_supports ROOT = Path(os.environ.get("VESSELSTACK_ROOT", "/opt/vesselstack")) @@ -700,7 +703,7 @@ def verify_release_tree(release: Path) -> None: verify_release_path(path) -def action_command(action: str, parameters: dict[str, Any]) -> list[str]: +def action_command(action: str, parameters: dict[str, Any], backup_choice: dict[str, str] | None = None) -> list[str]: if action in {"preflight", "apply", "install"}: if not INSTALLER.is_file(): raise ValueError(f"Bundled installer not found at {INSTALLER}") @@ -729,7 +732,12 @@ def action_command(action: str, parameters: dict[str, Any]) -> list[str]: for candidate in (reviewed_root, resolved): verify_release_path(candidate) verify_release_tree(resolved) - return [str(CTL), "update", str(resolved)] + command = [str(CTL), "update", str(resolved)] + # backup_choice only ever comes from request_backup_choice(), never straight from the request body. + if backup_choice and backup_choice.get("mode") in {"system_disk_once", "skip"}: + # The reviewed folder comes first (older controllers read only the first argument). + command.append("--backup-choice=" + backup_choice["mode"].replace("_", "-")) + return command raise ValueError("Unsupported action") @@ -741,6 +749,15 @@ def command_output(command: list[str], timeout: int = 8) -> tuple[int, str]: return 1, str(exc) +def command_stdout(command: list[str], timeout: int = 20) -> tuple[int, str]: + """Like command_output, but stderr is dropped so a warning cannot corrupt JSON.""" + try: + result = subprocess.run(command, text=True, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, timeout=timeout, check=False) + return result.returncode, result.stdout.strip() + except (OSError, subprocess.TimeoutExpired): + return 1, "" + + def describe_component(name: str, kind: str, enabled: bool, state: str) -> dict[str, Any]: """One component as the UI shows it: real name, plain status, no contradictions.""" label, description, section = COMPONENT_INFO.get(name, (name.replace("-", " ").title(), "", "")) @@ -881,6 +898,92 @@ def backup_status(vessel: dict[str, str] | None = None) -> dict[str, Any]: return result +DRIVE_FIELDS = ("device", "label", "size", "fstype") + + +def _drive_rows(value: Any, extra: tuple[str, ...]) -> list[dict[str, str]]: + rows: list[dict[str, str]] = [] + for item in value if isinstance(value, list) else []: + if not isinstance(item, dict): + continue + row = {key: str(item.get(key) or "")[:120] for key in (*DRIVE_FIELDS, *extra)} + if "uuid" in extra and not re.fullmatch(r"[A-Za-z0-9-]{1,64}", row["uuid"]): + continue + rows.append(row) + return rows[:20] + + +def backup_is_locked() -> bool: + """Locked by vesselstack.env or by the panel's own process environment.""" + return bool(backup_locked(VESSEL_ENV) or os.environ.get("VESSELSTACK_BACKUP_ALLOW_UNMOUNTED") == "false") + + +def backup_preflight() -> dict[str, Any]: + """What the owner may choose before a browser upgrade or folder update. Read-only. + + default_ok means the normal path (a mounted backup folder) will work. The drive list comes from + the installed vesselstackctl (root side); the browser can only pick an entry by its UUID, and the + root side lists again before it mounts anything. Any problem there yields an empty drive list. + """ + status = backup_status() + mount = status.get("mount", "unknown") + locked = backup_is_locked() + listing: dict[str, Any] = {} + if mount == "not_mounted": + code, text = command_stdout([str(CTL), "backup-drive", "list", "--json"]) + if code == 0: + try: + parsed = json.loads(text) + listing = parsed if isinstance(parsed, dict) else {} + except ValueError: + listing = {} + offered = (not locked) and mount in {"not_mounted", "system_disk"} + if locked: + reason = "Turned off: VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false is set in the configuration." + elif not offered: + reason = "The backup folder is on a mounted drive or could not be checked." + else: + reason = "" + return { + "path": status.get("path", ""), + "mount": mount, + "default_ok": mount == "mounted", + "fstab_mount": bool(listing.get("fstab_mount")) and mount == "not_mounted", + "drives": _drive_rows(listing.get("drives"), ("uuid",)), + "refused": _drive_rows(listing.get("refused"), ("reason",)), + "mounted_elsewhere": _drive_rows(listing.get("mounted_elsewhere"), ("mountpoint",)), + "system_disk_once": {"offered": offered, "reason": reason, "target": DEFAULT_BACKUP_DIR if mount == "not_mounted" else status.get("path", "")}, + "skip": {"offered": not locked}, + } + + +def request_backup_choice(payload: dict[str, Any], folder: bool = False) -> dict[str, str] | None: + """The validated one-shot backup choice of this request (None means the normal mounted backup). + + The release worker checks the installed controller itself; a folder update runs the installed + controller directly, so it is checked here. + """ + choice = normalize_backup_choice(payload.get("backup"), backup_is_locked()) + choice, _ = effective_backup_choice(choice, VESSEL_ENV) + if folder and choice is not None and not ctl_supports("backup-drive", str(CTL)): + raise ValueError("The installed vesselstackctl is older than this Control Panel and does not support that choice. Mount the drive by hand, or run the update from a terminal.") + return choice + + +def folder_update_steps(choice: dict[str, str] | None) -> list[list[str]]: + """Commands that run before `vesselstackctl update DIR`: the mount, outside the panel's sandbox. + + The panel runs with a private mount namespace, so a mount made here would not be visible to the + rest of the system. A transient unit is started by systemd instead and runs as plain root. + """ + if not choice or choice["mode"] not in {"mount_fstab", "mount_drive"}: + return [] + unit = ["systemd-run", "--wait", "--pipe", "--collect", "--quiet", "--property=RuntimeMaxSec=300", + f"--setenv=VESSELSTACK_ROOT={ROOT}", f"--setenv=VESSELSTACK_ENV={VESSEL_ENV}", "--setenv=VESSELSTACK_NONINTERACTIVE=true"] + mount = [str(CTL), "backup-drive", "mount"] + (["--fstab"] if choice["mode"] == "mount_fstab" else ["--uuid", choice["uuid"]]) + return [unit + mount] + + ERROR_LINE = re.compile(r"^(?:\[[^\]]*\]\s*)?(?:ERROR|Error|error|FATAL|Refusing[^:]*|Backup[^:]*failed)[:\s]+(.*)$") @@ -938,11 +1041,12 @@ def __init__(self) -> None: self.current: dict[str, Any] | None = None self.counter = 0 - def start(self, label: str, command_or_factory: list[str] | Any) -> dict[str, Any]: + def start(self, label: str, command_or_factory: list[str] | Any, before: Any = None, extra: dict[str, Any] | None = None) -> dict[str, Any]: slot_fd = take_lifecycle_slot() operation = None try: command = command_or_factory() if callable(command_or_factory) else command_or_factory + steps = (before() if callable(before) else before) or [] with self.lock: if self.current and self.current["state"] == "running": raise ValueError("Another operation is already running") @@ -952,9 +1056,10 @@ def start(self, label: str, command_or_factory: list[str] | Any) -> dict[str, An "command": command[0:1] + ["[arguments hidden]"], "state": "running", "started_at": int(time.time()), "finished_at": None, "exit_code": None, "output": deque(maxlen=400), + **(extra or {}), } self.current = operation - threading.Thread(target=self._run, args=(operation, command, slot_fd), daemon=True).start() + threading.Thread(target=self._run, args=(operation, command, slot_fd, steps), daemon=True).start() return self.public() except Exception: if operation is not None: @@ -966,19 +1071,24 @@ def start(self, label: str, command_or_factory: list[str] | Any) -> dict[str, An release_lifecycle_slot(slot_fd) raise - def _run(self, operation: dict[str, Any], command: list[str], slot_fd) -> None: + def _run(self, operation: dict[str, Any], command: list[str], slot_fd, steps: list[list[str]] | None = None) -> None: environment = os.environ.copy() environment["VESSELSTACK_SKIP_PANEL_RESTART"] = "true" environment["VESSELSTACK_NONINTERACTIVE"] = "true" # the panel cannot answer prompts; scripts print "To fix:" guidance instead exit_code = 1 succeeded = False try: - process = subprocess.Popen(command, text=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, env=environment) - assert process.stdout is not None - for line in process.stdout: - with self.lock: - operation["output"].append(line.rstrip()) - exit_code = process.wait() + # Steps (for example mounting the backup drive) run first; the first failure stops everything. + exit_code = 0 + for step in [*(steps or []), command]: + process = subprocess.Popen(step, text=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, env=environment) + assert process.stdout is not None + for line in process.stdout: + with self.lock: + operation["output"].append(line.rstrip()) + exit_code = process.wait() + if exit_code != 0: + break succeeded = exit_code == 0 with self.lock: if succeeded and operation["label"] in {"apply", "install", "update"} and restart_needed(): @@ -1108,6 +1218,8 @@ def do_GET(self) -> None: operation = memory if memory.get("state") == "running" else max((persisted, memory), key=lambda item: int(item.get("finished_at") or item.get("started_at") or 0)) vessel = parse_env(VESSEL_ENV) self.send_json(200, {"components": component_status(), "operation": with_summary(operation), "installed_version": installed_version(ROOT), "boat_name": vessel.get("BOAT_NAME", ""), "backup": backup_status(vessel), "restart_needed": restart_needed(), **RUNNING_IDENTITY}) + elif path == "/api/backup-preflight": + self.send_json(200, backup_preflight()) elif path == "/api/operation": persisted = public_operation(ROOT) memory = OPERATIONS.public() @@ -1153,6 +1265,8 @@ def do_POST(self) -> None: return if path == "/api/action": action = str(payload.get("action", "")) + if payload.get("backup") is not None and action not in {"update", "update-release"}: + raise ValueError("backup choices apply only to updates") if action == "restart-panel": slot = take_lifecycle_slot() try: @@ -1166,13 +1280,23 @@ def do_POST(self) -> None: raise ValueError("release_id and channel are required") if OPERATIONS.public().get("state") == "running": raise ValueError("Another operation is already running") + choice = request_backup_choice(payload) slot = take_lifecycle_slot() try: snapshot = release_snapshot(release_id, channel, ROOT) - result = start_detached("update-release", ROOT, RUNNING_IDENTITY, release_id, channel, target_version=snapshot["version"], snapshot=snapshot) + extra_args = {"backup": choice} if choice else {} + result = start_detached("update-release", ROOT, RUNNING_IDENTITY, release_id, channel, target_version=snapshot["version"], snapshot=snapshot, **extra_args) finally: release_lifecycle_slot(slot) self.send_json(202, result); return + if action == "update" and payload.get("backup") is not None: + choice = request_backup_choice(payload, folder=True) + if choice is not None: + info: dict[str, Any] = {"backup_mode": choice["mode"]} + if choice["mode"] == "skip": + info["backup_skipped"] = True + self.send_json(202, OPERATIONS.start(action, lambda: action_command(action, payload, choice), before=lambda: folder_update_steps(choice), extra=info)) + return self.send_json(202, OPERATIONS.start(action, lambda: action_command(action, payload))) return if path == "/api/restart-panel": diff --git a/control-panel/browser_updates.py b/control-panel/browser_updates.py index 1fbb331..b2323e1 100644 --- a/control-panel/browser_updates.py +++ b/control-panel/browser_updates.py @@ -43,9 +43,100 @@ MAX_ARCHIVE_EXPANDED = 1024 * 1024 * 1024 FIXED_CTL = "/usr/local/sbin/vesselstackctl" OPERATION_FILE_NAME = "browser-update.json" +DEFAULT_BACKUP_DIR = "/var/backups/vesselstack" + +# One-shot choices for an upgrade whose backup drive is not mounted. They live only in the per-job +# request and the operation record; nothing is ever written to vesselstack.env. The same rules run +# in the panel (admission) and again in the detached worker, which is a copy of this file. +BACKUP_UUID = re.compile(r"^[A-Za-z0-9-]{1,64}$") +BACKUP_CONFIRM = {"system_disk_once": "system-disk-once", "skip": "install-without-backup"} +BACKUP_MODES = ("require_mounted", "mount_fstab", "mount_drive", "system_disk_once", "skip") +BACKUP_LOCK = re.compile(r"^(export[ \t]+)?VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=[\"']?false", re.MULTILINE) +BACKUP_SKIP_WARNING = "WARNING: backup skipped at the owner's request for this update; there is no archive to restore from." PROCESS_BOOT_ID = uuid.uuid4().hex +def _env_value(config: Path, key: str) -> str: + """Last KEY=value in an env file, without quotes. Empty when absent or unreadable.""" + try: + text = Path(config).read_text(errors="replace") + except OSError: + return "" + value = "" + for line in text.splitlines(): + match = re.match(rf"^(?:export[ \t]+)?{re.escape(key)}=(.*)$", line.strip()) + if match: + value = match.group(1).strip() + if len(value) >= 2 and value[0] == value[-1] and value[0] in "'\"": + value = value[1:-1] + return value + + +def backup_locked(config: Path) -> bool: + """True when vesselstack.env pins VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false (absent means not locked).""" + try: + return bool(BACKUP_LOCK.search(Path(config).read_text(errors="replace"))) + except OSError: + return False + + +def backup_destination(config: Path) -> tuple[str, str]: + """(resolved backup folder, mounted | not_mounted | system_disk), derived like vesselstackctl backup.""" + value = _env_value(config, "VESSELSTACK_BACKUP") or os.environ.get("VESSELSTACK_BACKUP") or DEFAULT_BACKUP_DIR + destination = os.path.realpath(value) + parts = Path(destination).parts + if len(parts) >= 3 and parts[1] in {"media", "mnt"}: + return destination, "mounted" if os.path.ismount(os.path.join("/", parts[1], parts[2])) else "not_mounted" + return destination, "system_disk" + + +def normalize_backup_choice(raw: Any, locked: bool) -> dict[str, str] | None: + """Validate the optional backup object. None means the default (require a mounted backup). + + Unknown keys or modes are refused. system_disk_once and skip need their exact confirmation word + and are refused while the environment file pins VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false. + """ + if raw is None: + return None + if not isinstance(raw, dict): + raise ValueError("backup must be an object") + mode = raw.get("mode") + if not isinstance(mode, str) or mode not in BACKUP_MODES: + raise ValueError("unknown backup mode") + allowed = {"mode"} | ({"uuid"} if mode == "mount_drive" else set()) | ({"confirm"} if mode in BACKUP_CONFIRM else set()) + if set(raw) - allowed: + raise ValueError("unexpected backup field") + if mode == "require_mounted": + return None + if mode == "mount_drive": + uuid_value = raw.get("uuid") + if not isinstance(uuid_value, str) or not BACKUP_UUID.fullmatch(uuid_value): + raise ValueError("backup drive id is invalid") + return {"mode": mode, "uuid": uuid_value} + if mode in BACKUP_CONFIRM: + if locked: + raise ValueError("This option is turned off: VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false is set") + if raw.get("confirm") != BACKUP_CONFIRM[mode]: + raise ValueError("backup confirmation is missing") + return {"mode": mode, "confirm": BACKUP_CONFIRM[mode]} + return {"mode": mode} + + +def effective_backup_choice(choice: dict[str, str] | None, config: Path) -> tuple[dict[str, str] | None, str]: + """Falls back to the default when the drive got mounted after the choice was made. + + Returns (choice or None, a note for the log; empty when nothing changed). + """ + if choice is None: + return None, "" + _, mount = backup_destination(config) + if mount == "mounted": + return None, "The backup drive is mounted now, so the normal backup is used." + if mount == "system_disk" and choice["mode"] in {"mount_fstab", "mount_drive"}: + return None, "The backup folder is not on a removable drive, so the normal backup is used." + return choice, "" + + def _require_trusted_path(path: Path) -> Path: """Require an absolute path whose existing ancestors are root-owned (or us).""" raw = os.fspath(path) @@ -323,7 +414,7 @@ def release_slot(fd) -> None: os.close(fd) -def start_detached(action: str, root: Path, old_identity: dict[str, str], release_id: int | None = None, channel: str | None = None, ctl: str = FIXED_CTL, target_version: str | None = None, snapshot: dict[str, Any] | None = None) -> dict[str, Any]: +def start_detached(action: str, root: Path, old_identity: dict[str, str], release_id: int | None = None, channel: str | None = None, ctl: str = FIXED_CTL, target_version: str | None = None, snapshot: dict[str, Any] | None = None, backup: dict[str, str] | None = None) -> dict[str, Any]: state = read_operation(root) if state.get("state") == "running": raise ValueError("Another operation is already running") @@ -344,9 +435,13 @@ def start_detached(action: str, root: Path, old_identity: dict[str, str], releas request = {"id": int(time.time() * 1000), "action": action, "root": str(root), "config": str(root / "config" / "vesselstack.env"), "ctl": ctl, "old_boot_id": old_identity.get("boot_id", ""), "target_version": target_version, "release_id": release_id, "channel": channel} if action == "update-release": request["release_snapshot"] = snapshot + if backup is not None: + request["backup"] = backup (job / "request.json").write_text(json.dumps(request, sort_keys=True)); os.chmod(job / "request.json", 0o600) shutil.copy2(Path(__file__), job / "runner.py"); os.chmod(job / "runner.py", 0o700) operation = {"id": request["id"], "label": action, "state": "running", "phase": "queued", "target_version": request["target_version"], "backup_path": None, "started_at": int(time.time()), "finished_at": None, "exit_code": None, "output": [], "boot_id": old_identity.get("boot_id"), "running_version": old_identity.get("running_version")} + if action == "update-release" and backup is not None: + operation["backup_mode"] = backup["mode"] write_operation(root, operation) command = ["systemd-run", "--unit=vesselstack-browser-update", "--collect", "--property=Type=exec", "/usr/bin/python3", "-I", str(job / "runner.py"), "--worker", str(job / "request.json")] try: @@ -601,6 +696,15 @@ def safe_extract(archive: Path, destination: Path, expected_version: str | None raise ValueError("archive is missing required release structure") +def ctl_supports(token: str, ctl: str = FIXED_CTL) -> bool: + """True when the installed vesselstackctl lists the command or option (older ones cannot).""" + try: + result = subprocess.run([ctl, "--help"], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, text=True, timeout=15, check=False) + except (OSError, subprocess.TimeoutExpired): + return False + return token in (result.stdout or "") + + def run_update(request: dict[str, Any], operation: dict[str, Any], root: Path) -> None: """Execute the reviewed update sequence for a detached worker request.""" release_id = int(request["release_id"]) @@ -609,6 +713,8 @@ def run_update(request: dict[str, Any], operation: dict[str, Any], root: Path) - saved = request.get("release_snapshot") if not isinstance(saved, dict): raise ValueError("release snapshot is required") + # The choice was validated at admission; it is checked again here, against the env file as it is now. + choice = normalize_backup_choice(request.get("backup"), backup_locked(Path(config))) fresh = release_snapshot(release_id, channel, root) assert_snapshot_unchanged(saved, fresh) target_version = fresh["version"] @@ -654,27 +760,62 @@ def run_update(request: dict[str, Any], operation: dict[str, Any], root: Path) - _validate_fixed_ctl() operation.update(phase="preflight"); write_operation(root, operation) run_command([str(installer), "--config", config, "--dry-run"], root, operation, 1800, child_env) - operation.update(phase="backing-up"); write_operation(root, operation) - backup_result = run_command([FIXED_CTL, "backup", "--require-mounted"], root, operation, 1800, child_env) - output = (backup_result.stdout or "").splitlines() - if not output or not output[-1].strip(): - raise ValueError("backup did not report an archive path") - operation["backup_path"] = output[-1].strip() - write_operation(root, operation) + choice, note = effective_backup_choice(choice, Path(config)) + mode = choice["mode"] if choice else "require_mounted" + # The record names the handling that is really used (a fallback to the normal backup clears it). + if choice: + operation["backup_mode"] = mode + else: + operation.pop("backup_mode", None) + if mode in {"mount_fstab", "mount_drive"}: + if not ctl_supports("backup-drive"): + raise ValueError("The installed vesselstackctl is older than this Control Panel and cannot mount the backup drive. Mount the drive by hand, or run the update from a terminal.") + operation.update(phase="mounting-backup-drive"); write_operation(root, operation) + mount_argv = [FIXED_CTL, "backup-drive", "mount"] + (["--fstab"] if mode == "mount_fstab" else ["--uuid", choice["uuid"]]) + run_command(mount_argv, root, operation, 300, child_env) + elif mode == "system_disk_once" and not ctl_supports("--system-disk-once"): + raise ValueError("The installed vesselstackctl is older than this Control Panel and cannot back up to the system disk this once. Mount the drive by hand, or run the update from a terminal.") + if mode in {"skip", "system_disk_once"} and (backup_locked(Path(config)) or os.environ.get("VESSELSTACK_BACKUP_ALLOW_UNMOUNTED") == "false"): + raise ValueError("This backup option is turned off: VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false is set. Nothing was changed.") + if mode == "skip": + # No backup is made and the backup command is never called. + operation.update(phase="backing-up", backup_skipped=True, backup_path=None, + output=[BACKUP_SKIP_WARNING] + [line for line in operation.get("output") or [] if line != BACKUP_SKIP_WARNING]) + write_operation(root, operation) + else: + operation.update(phase="backing-up"); write_operation(root, operation) + backup_argv = [FIXED_CTL, "backup", "--system-disk-once"] if mode == "system_disk_once" else [FIXED_CTL, "backup", "--require-mounted"] + backup_result = run_command(backup_argv, root, operation, 1800, child_env) + output = (backup_result.stdout or "").splitlines() + if not output or not output[-1].strip(): + raise ValueError("backup did not report an archive path") + operation["backup_path"] = output[-1].strip() + if note: + operation["output"] = [note] + list(operation.get("output") or []) + write_operation(root, operation) operation.update(phase="installing"); write_operation(root, operation) install_env = dict(child_env) install_env["VESSELSTACK_SKIP_PANEL_RESTART"] = "true" install_env["VESSELSTACK_ROOT"] = str(root) install_env["VESSELSTACK_ENV"] = config run_command([str(installer), "--config", config, "--start"], root, operation, 1800, install_env) + keep_skip_warning(operation) operation.update(phase="restarting"); write_operation(root, operation) run_command(["systemctl", "restart", "vesselstack-control-panel.service"], root, operation, 60) + keep_skip_warning(operation) operation.update(phase="verifying-restart"); write_operation(root, operation) identity = verify_restart(root, str(request["old_boot_id"]), target_version) + keep_skip_warning(operation) operation.update(state="succeeded", phase="complete", finished_at=int(time.time()), exit_code=0, boot_id=identity["boot_id"], running_version=identity["running_version"]) +def keep_skip_warning(operation: dict[str, Any]) -> None: + """Every command replaces the logged output, so the skipped-backup warning is put back on top.""" + if operation.get("backup_skipped"): + operation["output"] = [BACKUP_SKIP_WARNING] + [line for line in operation.get("output") or [] if line != BACKUP_SKIP_WARNING] + + def run_worker(request_path: Path) -> None: request = _read_json_private(request_path, 64 * 1024) root = Path(str(request.get("root", ""))) @@ -702,6 +843,7 @@ def run_worker(request_path: Path) -> None: prior = list(operation.get("output") or []) prior.append(str(exc)[:MAX_OUTPUT]) operation.update(state="failed", phase="failed", finished_at=int(time.time()), exit_code=1, output=prior[-400:]) + keep_skip_warning(operation) finally: write_operation(root, operation); release_slot(lock) diff --git a/control-panel/static/app.js b/control-panel/static/app.js index 26cfb08..c17eb99 100644 --- a/control-panel/static/app.js +++ b/control-panel/static/app.js @@ -164,6 +164,166 @@ function ask(spec) { }); } +// ---------------------------------------------------------------- backup choice (drive not mounted) + +const SKIP_ACK_TEXT = "I understand there is no backup to restore if this update fails"; +const mountFolder = (path) => { + const parts = String(path || "").split("/").filter(Boolean); + return ["media", "mnt"].includes(parts[0]) && parts[1] ? `/${parts[0]}/${parts[1]}` : String(path || ""); +}; + +function backupOptions(pre) { + const folder = mountFolder(pre.path); + const options = []; + if (pre.fstab_mount) options.push({ id: "fstab", label: "Mount the backup drive now", detail: `It is listed in /etc/fstab for ${folder}.`, ok: "Mount and continue", backup: { mode: "mount_fstab" } }); + (pre.drives || []).forEach((drive) => options.push({ + id: `drive:${drive.uuid}`, + label: `Mount ${drive.label ? `"${drive.label}"` : "the drive with no name"} (${drive.size}, ${drive.fstype})`, + detail: `${drive.device} is mounted at ${folder} until the next restart. /etc/fstab is not changed.`, + ok: "Mount and continue", + backup: { mode: "mount_drive", uuid: drive.uuid }, + })); + options.push({ id: "recheck", label: "I plugged the drive in; check again", ok: "Check again", action: "recheck" }); + if (pre.system_disk_once?.offered) options.push({ + id: "system", label: "Back up to the system disk this once (not recommended)", + detail: `Saved in ${pre.system_disk_once.target || "the system backup folder"} on this computer's own disk. It will not survive a failed disk. Nothing is remembered; the next update asks again.`, + ok: "Back up to the system disk and continue", backup: { mode: "system_disk_once", confirm: "system-disk-once" }, + }); + options.push({ id: "folder", label: "Use a different backup folder", detail: "Saves the new folder as your backup folder setting, then checks again.", ok: "Save folder and check again", action: "folder" }); + if (pre.skip?.offered) options.push({ + id: "skip", label: "Install without a backup", danger: true, + detail: "Nothing is saved first. If the update fails there is no archive to restore from.", + ok: "Install without a backup", backup: { mode: "skip", confirm: "install-without-backup" }, + }); + return options; +} + +// Resolves to the chosen backup object, { mode: "require_mounted" } when the drive turned up while the dialog +// was open (the normal backup then works), or null when cancelled. Nothing chosen here is saved anywhere. +function askBackupChoice(spec, first) { + const dialog = $("#backup-choice-dialog"); + if (dialog.open) return Promise.resolve(null); + const opener = document.activeElement; + let current = first; + let selected = ""; + let working = false; + let outcome = null; + let options = []; + const chosen = () => options.find((option) => option.id === selected); + const refresh = () => { + const option = chosen(); + $("#backup-choice-folder").hidden = option?.id !== "folder"; + $("#backup-choice-skip-ack").hidden = option?.id !== "skip"; + const ok = $("#backup-choice-ok"); + ok.textContent = working ? "Working…" : option?.ok || "Continue"; + ok.className = option?.danger ? "danger solid" : "primary"; + ok.disabled = working || !option + || (option.id === "skip" && !$("#backup-choice-ack").checked) + || (option.id === "folder" && !$("#backup-choice-path").value.trim()); + $("#backup-choice-detail").textContent = option?.detail || ""; + $("#backup-choice-detail").hidden = !option?.detail; + }; + const render = (error = "") => { + options = backupOptions(current); + if (!chosen()) selected = (options.find((option) => !option.danger) || options[0]).id; + const dirty = isConfigDirty(); + $("#backup-choice-options").innerHTML = options.map((option) => { + const blocked = option.id === "folder" && dirty; + return ``; + }).join(""); + const notes = [ + ...(current.refused || []).map((item) => `${item.device} ${item.size} ${item.label ? `"${item.label}" ` : ""}${item.fstype}: ${item.reason}`), + ...(current.mounted_elsewhere || []).map((item) => `${item.device} ${item.size} ${item.label ? `"${item.label}" ` : ""}${item.fstype}: already mounted at ${item.mountpoint}. Unmount it first to use it here.`), + ...(current.system_disk_once && !current.system_disk_once.offered && current.system_disk_once.reason ? [`System disk: ${current.system_disk_once.reason}`] : []), + ]; + $("#backup-choice-notes").innerHTML = notes.map((note) => `
  • ${escapeHtml(note)}
    • `).join(""); + $("#backup-choice-notes").hidden = !notes.length; + $("#backup-choice-error").textContent = error; + $("#backup-choice-error").hidden = !error; + refresh(); + }; + const folder = mountFolder(current.path); + $("#backup-choice-title").textContent = spec.title; + $("#backup-choice-intro").textContent = current.mount === "not_mounted" + ? `The backup drive is not mounted (${folder}). ${spec.intro || ""} Choose how to handle the backup for this update only; nothing here is remembered.`.replace(/\s+/g, " ") + : `The backup folder ${current.path || ""} is not on an external drive, so the normal backup cannot be required. ${spec.intro || ""} Choose how to handle the backup for this update only.`.replace(/\s+/g, " "); + $("#backup-choice-items").innerHTML = (spec.items || []).map((item) => `
  • ${escapeHtml(item)}
    • `).join(""); + $("#backup-choice-items").hidden = !(spec.items || []).length; + $("#backup-choice-ack-text").textContent = SKIP_ACK_TEXT; + $("#backup-choice-ack").checked = false; + $("#backup-choice-path").value = ""; + $("#backup-choice-form").onsubmit = (event) => event.preventDefault(); + $("#backup-choice-options").onchange = (event) => { selected = String(event.target?.value || ""); refresh(); }; + $("#backup-choice-ack").onchange = refresh; + $("#backup-choice-path").oninput = refresh; + $("#backup-choice-cancel").onclick = () => dialog.close("cancel"); + $("#backup-choice-ok").onclick = async () => { + const option = chosen(); + if (!option || working) return; + if (option.id === "skip" && !$("#backup-choice-ack").checked) return; + if (option.backup) { + outcome = option.backup; + dialog.close("ok"); + return; + } + const session = state.session; + working = true; + refresh(); + try { + if (option.action === "folder") { + const saved = await api("/api/config", { method: "POST", body: JSON.stringify({ settings: { VESSELSTACK_BACKUP: $("#backup-choice-path").value.trim() } }) }); + $("#backup-choice-path").value = ""; + // This option is unavailable while Settings has unsaved edits, so showing the saved values loses nothing. + if (saved?.configuration && !isConfigDirty()) { state.config = saved.configuration; renderConfiguration(state.config); } + } + const next = await api("/api/backup-preflight"); + if (session !== state.session || !state.token) return; + current = next; + working = false; + if (current.default_ok === true) { + outcome = { mode: "require_mounted" }; + dialog.close("ok"); + return; + } + selected = ""; + render(); + } catch (error) { + if (session !== state.session || !state.token) return; + if (error.status === 401) { dialog.close("cancel"); requireReauth("Your session ended."); return; } + working = false; + render(error.message); + } + }; + return new Promise((resolve) => { + dialog.onclose = () => { + dialog.onclose = null; + if (opener && opener.isConnected !== false && typeof opener.focus === "function") opener.focus(); + resolve(dialog.returnValue === "ok" ? outcome : null); + }; + dialog.returnValue = ""; + render(); + dialog.showModal(); + }); +} + +// Asks first. When the normal mounted backup will not work, the owner picks what to do for this one +// request; otherwise the usual confirmation is shown unchanged. A failed lookup also falls back to the +// usual confirmation: the server still refuses to install without the normal backup. +async function confirmWithBackup(spec) { + const session = state.session; + let pre = null; + try { + pre = await api("/api/backup-preflight"); + } catch (error) { + if (error.status === 401) { requireReauth("Your session ended."); return { proceed: false }; } + } + if (session !== state.session || !state.token || state.busy) return { proceed: false }; + if (!pre || pre.default_ok !== false) return { proceed: await ask(spec) }; + const choice = await askBackupChoice(spec, pre); + if (!choice) return { proceed: false }; + return choice.mode === "require_mounted" ? { proceed: true } : { proceed: true, backup: choice }; +} + // ---------------------------------------------------------------- session function lock(force = false) { @@ -433,8 +593,12 @@ function describeOperation(operation) { } if (label === "update-release") { const target = operation.target_version || "the update"; + if (operation.backup_skipped || operation.backup_mode === "skip") return { running: `Installing ${target} without a backup…`, done: `Installed ${target} (no backup was made)`, failed: `Installing ${target} failed`, next: "No backup was made for this update, so there is no archive to restore from. Check the details before trying again." }; return { running: `Backing up, then installing ${target}…`, done: `Installed ${target}`, failed: `Installing ${target} failed`, next: "Your boat was backed up first if the backup step finished. Check the details before trying again." }; } + if (label === "update" && (operation.backup_skipped || operation.backup_mode === "skip")) { + return { running: "Updating from the folder without a backup…", done: "Update finished (no backup was made)", failed: "Update failed", next: "No backup was made for this update, so there is no archive to restore from. Check the details before trying again." }; + } const entry = OPERATION_TEXT[label]; if (entry) return { running: entry[0], done: entry[1], failed: entry[2], next: entry[3] }; return { running: "Working…", done: "Finished", failed: "Failed", next: "Check the details, then try again." }; @@ -556,6 +720,12 @@ async function startOperation(payload) { async function runAction(action, extra = {}) { const spec = action === "update" ? updateConfirm(extra) : CONFIRMS[action]; + if (action === "update") { + const decision = await confirmWithBackup(spec); + if (!decision.proceed) return; + startOperation({ action, ...extra, ...(decision.backup ? { backup: decision.backup } : {}) }); + return; + } if (spec && !(await ask(spec))) return; startOperation({ action, ...extra }); } @@ -711,8 +881,9 @@ async function checkReleases() { async function installRelease(releaseId) { const release = state.releases.find((item) => item.id === Number(releaseId) && releaseEligible(item)); if (!release || state.busy) return; - if (!(await ask(releaseConfirm(release)))) return; - startOperation({ action: "update-release", release_id: release.id, channel: state.releaseChannel }); + const decision = await confirmWithBackup(releaseConfirm(release)); + if (!decision.proceed) return; + startOperation({ action: "update-release", release_id: release.id, channel: state.releaseChannel, ...(decision.backup ? { backup: decision.backup } : {}) }); } // ---------------------------------------------------------------- settings diff --git a/control-panel/static/index.html b/control-panel/static/index.html index f6f062d..e13a7ee 100644 --- a/control-panel/static/index.html +++ b/control-panel/static/index.html @@ -158,6 +158,22 @@

    +
    +

    +

    + +
    + + + + + +
    +
    +
    diff --git a/control-panel/static/styles.css b/control-panel/static/styles.css index 4232778..281239c 100644 --- a/control-panel/static/styles.css +++ b/control-panel/static/styles.css @@ -310,3 +310,14 @@ dialog .keep{margin:.2rem 0 .5rem} .linklike{min-height:44px;padding:0;color:var(--sea);background:transparent;border:0;text-decoration:underline} .linklike:hover{background:transparent} .action .b button{white-space:nowrap} + +/* backup choice dialog */ +.choices{display:grid;gap:.4rem;margin:.6rem 0} +.choice{display:flex;align-items:flex-start;gap:.6rem;min-height:44px;margin:0;padding:.6rem .75rem;font-weight:600;border:1px solid var(--strong);border-radius:8px;cursor:pointer} +.choice input{width:auto;min-height:0;margin-top:.2rem;flex:none} +.danger-choice{border-color:var(--red)} +.ack{display:flex;align-items:flex-start;gap:.6rem;margin:.6rem 0;font-weight:600} +.ack input{width:auto;min-height:0;margin-top:.2rem;flex:none} +.notes{margin:.4rem 0;font-size:.85rem} +.choice-error{margin:.4rem 0;color:var(--red)} +#backup-choice-dialog{width:min(36rem,calc(100% - 2rem));max-height:calc(100% - 2rem);overflow:auto} diff --git a/docs/CONTROL_PANEL.md b/docs/CONTROL_PANEL.md index 936ce15..115b59c 100644 --- a/docs/CONTROL_PANEL.md +++ b/docs/CONTROL_PANEL.md @@ -154,14 +154,17 @@ notes on GitHub and a **Back up & install** button. for testing; leave it off on a boat in regular use. The 0.5.x panels called the same switch "Include lab prereleases"; an owner updating from 0.5.x uses that older name. -**Back up & install** asks in a dialog, then runs the update in a detached +**Back up & install** asks in a dialog (when the backup drive is not mounted, a longer one, see +[When the backup drive is not mounted](#when-the-backup-drive-is-not-mounted)), then runs the update in a detached worker so it survives the panel restarting. It: 1. downloads the release archive and its checksum file from GitHub and checks the size, the SHA-256 checksum and the digest GitHub reports; 2. unpacks it and runs the installer's checks (`--dry-run`); 3. takes a backup with `vesselstackctl backup --require-mounted`, so the update - stops here if the backup drive is not mounted; + stops here if the backup drive is not mounted (unless you chose another way in the dialog + described below, in which case it mounts the drive or backs up to the system disk first; + with **Install without a backup** it makes no backup at all); 4. runs the new installer with `--config … --start`; 5. restarts the Control Panel and waits for it to report the new version. @@ -174,7 +177,34 @@ a separate step you can run yourself (see **Advanced: install a reviewed release folder** is for a release you already downloaded, checked and unpacked on the Pi under `/var/lib/vesselstack/releases`. **Back up & update from folder** runs `vesselstackctl update FOLDER`, which -takes its own backup first. +takes its own backup first. It shows the same dialog when the backup drive is not mounted. + +### When the backup drive is not mounted + +Before **Back up & install** or **Back up & update from folder** the panel checks the backup folder. +If it is on a mounted drive, the usual confirmation appears. If the folder is under `/media` or +`/mnt` and that drive is not mounted (or the folder is not on an external drive at all), a dialog +lets you choose, for this one request only. Nothing you pick here is remembered, and +`vesselstack.env` and `/etc/fstab` are not changed: + +| Choice | What happens | +|---|---| +| Mount the backup drive now | Only when `/etc/fstab` has an entry for the folder. | +| Mount a listed drive | Connected USB drives (ext4 recommended, exFAT allowed) that are not part of the system disk; refused and already-mounted drives are listed with the reason. The panel only sends the drive's UUID; the root side lists again and mounts nothing else. Mounted with `nodev,nosuid,noexec`; the mount folder must be empty. | +| I plugged the drive in; check again | Reads the list again. | +| Back up to the system disk this once | Saves to `/var/backups/vesselstack` on the SD card, not into the unmounted folder. Refused when the data would not fit. Not recommended. | +| Use a different backup folder | Saves the folder as the **Backup folder** setting, then checks again. Unavailable while Settings has unsaved edits. | +| Install without a backup | Needs the box **I understand there is no backup to restore if this update fails**. No backup is made; the result banner says so. | + +The system-disk and no-backup choices are not offered, and the server refuses them, when +`vesselstack.env` or the panel's own environment contains `VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false`; +the update worker checks again right before it skips or redirects the backup. The `backup` field is +accepted only for the `update` and `update-release` actions (other actions get a 400). The choice is +recorded in the operation record as `backup_mode` / `backup_skipped` for display only; it is never +written to `vesselstack.env` or reused. A drive mounted by UUID is checked after the mount +(its filesystem UUID must match) and unmounted again if not. The mount runs as root +outside the panel's own sandbox (in the update worker, or a transient `systemd-run` unit limited to 300 seconds +for the folder update). This cannot help the upgrade that delivers it: the Control Panel worker uses the controller and panel that are already installed, so for the upgrade to 0.6.2 itself mount the drive by hand or run the update from a terminal. Rollback: `sudo umount` the drive. ![Control Panel Updates tab with a newer release listed and a Back up and install button](images/control-panel-update-available.png) @@ -240,7 +270,7 @@ what is affected and has a plain **Cancel**. | **Back up now** | It is the backup | No | No | Services pause during the copy | | **Apply** (Settings) | No | No | Renders and installs them; containers are not recreated | The displays bridge, if running, is restarted; other services keep running until restarted | | **Reinstall & start** | No | No | Yes | Every service restarts, about a minute | -| **Back up & install** (Updates) | Yes, first, drive must be mounted | Yes, to the chosen release | Yes | Panel and services, a few minutes | +| **Back up & install** (Updates) | Yes, first, drive must be mounted (or you choose otherwise in the dialog) | Yes, to the chosen release | Yes | Panel and services, a few minutes | **Apply** runs `install.sh --config …` against your saved settings without `--start`. That is the installer's render-only mode: it writes the rendered @@ -310,7 +340,7 @@ require the node-exporter image only when `NODE_EXPORTER_ENABLE=true`. | Symptom | Likely cause and fix | |---|---| -| Back up now fails with a mount message | The backup drive is not mounted. Mount it, then try again; the Home tab shows the drive state. | +| Back up now fails with a mount message | The backup drive is not mounted. Mount it, then try again; the Home tab shows the drive state. (**Back up & install** offers to mount it for you; see [above](#when-the-backup-drive-is-not-mounted).) | | Home says backups are on the computer's own disk | The backup folder is not on an external drive. Set **Backup folder** under Settings → Advanced → Storage & history to a path under `/media` or `/mnt`. | | Save is rejected with field messages | Fix the marked fields (unknown timezone, number out of range) and save again. | | Check for updates fails | The Pi has no route to `github.com`. Check its internet connection; offline boats use [the offline path](UPDATING.md). | diff --git a/docs/CONTROL_PANEL_TESTING.md b/docs/CONTROL_PANEL_TESTING.md index f321024..993e621 100644 --- a/docs/CONTROL_PANEL_TESTING.md +++ b/docs/CONTROL_PANEL_TESTING.md @@ -45,9 +45,28 @@ verification. Acceptance cases: -- With the backup path unmounted, `--require-mounted` must refuse the update +- With the backup path unmounted and no choice made, `--require-mounted` must refuse the update even if `ALLOW_UNMOUNTED=true`; no install may start. The failure keeps the - path and recovery help visible. + path and recovery help visible. From the browser the same press shows the backup dialog instead + of starting; verify each choice on a disposable Pi (an upgrade started from 0.6.2 or later; the + upgrade that delivers 0.6.2 uses the old worker and cannot show it): + - the dialog lists a spare ext4 USB drive by label, size and filesystem, lists a vfat drive and a + desktop-mounted drive as refused with the reason, and never shows a device path as a choice; + - mounting the listed drive makes `findmnt /media/` show `nodev,nosuid,noexec`, the + backup lands there, and `/etc/fstab` and `vesselstack.env` are byte-identical afterwards; + - **system disk this once** puts the archive in `/var/backups/vesselstack`, leaves the + `/media/` mount point empty, and the next upgrade asks again; + - **Install without a backup** stays disabled until the box is ticked, makes no archive, and the + banner says no backup was made; + - with `VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false` in `vesselstack.env`, the system-disk and + no-backup choices are absent and a hand-made request for them is refused (HTTP 400); + - the same with the variable only in the panel's process environment, and with it added after the + request was accepted but before the worker reaches the backup step (the worker refuses); + - a hand-made `/api/action` request with a `backup` field for any action other than `update` or + `update-release` is refused (HTTP 400); `backup_mode` / `backup_skipped` appear in the operation + record for display only and `vesselstack.env` stays unchanged; + - mount propagation: after the mount, `findmnt` in a normal root shell (not the panel's + namespace) shows the drive for both **Back up & install** and **update from folder**. - Use a checksum-mismatch fixture and an unsafe-archive fixture; both must stop before preflight/install. Do not weaken the live release checksum check or hack the production endpoint. @@ -87,7 +106,8 @@ target (or use a path that is not a mountpoint), then choose **Back up now** on Home and, separately, run an update from the Maintenance tab. Each stops with a red result banner that gives the reason, then a **To fix:** line naming the mount point (the update version also shows the one-off -`VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=true` command in monospace). Confirm +`VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=true` command in monospace; a Control Panel upgrade shows the +backup dialog described above instead). Confirm nothing was stopped or changed, that no prompt waited in the Details log, mount the drive, and run the operation again to see it succeed. Setting an unsupported value such as `BOAT_UNITS=imperial` in the settings file and starting an update diff --git a/docs/UPDATING.md b/docs/UPDATING.md index 385e5bd..8b6301c 100644 --- a/docs/UPDATING.md +++ b/docs/UPDATING.md @@ -42,7 +42,8 @@ prereleases appear only after you turn on **Include early test releases** (0.5.x call the same switch "Include lab prereleases"). **Back up & install** downloads the archive, checks its size and SHA-256 checksum, runs the installer's `--dry-run` check, takes a backup with `--require-mounted` (so the update stops if the external backup drive -under `/media` or `/mnt` is not mounted), installs, restarts the Control Panel and waits +under `/media` or `/mnt` is not mounted, unless you choose in the dialog to mount it, back up to the +system disk once or install without a backup), installs, restarts the Control Panel and waits for it to report the new version. The page reconnects by itself. See [Control Panel](CONTROL_PANEL.md#updates). @@ -63,6 +64,13 @@ an installed 0.6.0 or earlier controller has no menu: use the pinned installer, first, or run `sudo env VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=true vesselstackctl update `. See [Backups](../README.md#backups). +From 0.6.2 the Control Panel's **Back up & install** and **update from folder** show a dialog for an +unmounted backup drive (mount from `/etc/fstab`, mount a listed USB drive, back up to the system disk +once, use a different folder, or install without a backup). This cannot help the upgrade that delivers it: the Control Panel worker uses the controller and panel that are already installed, so for the upgrade to 0.6.2 itself mount the drive by hand or run the update from a terminal. Setting +`VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false` (in `vesselstack.env` or the environment) turns off the system-disk and no-backup choices; the choice is never saved. In the terminal +menu, "back up to the system disk this once" also now writes to `/var/backups/vesselstack`, leaving the +unmounted mount folder empty. + ## From a source checkout Use a reviewed tag from a git clone, with the dry run first: diff --git a/scripts/get-vesselstack.sh b/scripts/get-vesselstack.sh index 12d7bd8..91e555d 100755 --- a/scripts/get-vesselstack.sh +++ b/scripts/get-vesselstack.sh @@ -1245,6 +1245,16 @@ ensure_service_account() { # library (trusted only after the archive was verified and its folder is root-controlled). # Sets BACKUP_ALLOW_ONCE=1 when the owner chose to back up to the system disk this once. BACKUP_ALLOW_ONCE=0 +# 1 when the installed vesselstackctl is this release or newer (it lists backup-drive and takes +# --backup-choice). The help text is captured first so a closed pipe cannot flip the answer. +CTL_TAKES_CHOICE=0 +detect_ctl_choice() { + local ctl help='' + CTL_TAKES_CHOICE=0 + ctl=$(find_ctl) || return 0 + help=$("$ctl" --help 2>/dev/null &2 @@ -237,8 +247,9 @@ vs_mib() { # Sets VS_BACKUP_OFFER_SYSTEM=1 when a backup to the system disk is allowed and fits, and prints why not. vs_backup_system_disk_check() { - local dest=$1 ancestor free_kib used_kib=0 size_kib du_status=0 du_out + local dest=$1 ancestor free_kib used_kib=0 size_kib du_status=0 du_out label=${VS_BACKUP_SPACE_LABEL:-system disk} VS_BACKUP_OFFER_SYSTEM=0 + VS_BACKUP_SIZE_UNKNOWN=0 if [ -n "${VS_BACKUP_ENV_FILE:-}" ] && [ -r "$VS_BACKUP_ENV_FILE" ] \ && grep -Eq '^(export[[:space:]]+)?VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=["'"'"']?false' "$VS_BACKUP_ENV_FILE"; then return 0 @@ -251,6 +262,8 @@ vs_backup_system_disk_check() { if [ "$du_status" -eq 124 ]; then echo " The size of the VesselStack data could not be measured in 60 seconds (size unknown)." >&2 VS_BACKUP_OFFER_SYSTEM=1 + # shellcheck disable=SC2034 # read by vesselstackctl after the call + VS_BACKUP_SIZE_UNKNOWN=1 return 0 fi while read -r size_kib _; do @@ -258,8 +271,8 @@ vs_backup_system_disk_check() { used_kib=$((used_kib + size_kib)) done <<<"$du_out" if [ "$free_kib" -lt "$used_kib" ]; then - echo " The system disk has $(vs_mib "$free_kib") MiB free but the backup needs about $(vs_mib "$used_kib") MiB," >&2 - echo " so a backup to the system disk is not offered." >&2 + echo " The $label has $(vs_mib "$free_kib") MiB free but the backup needs about $(vs_mib "$used_kib") MiB," >&2 + echo " so a backup to the $label is not offered." >&2 return 0 fi VS_BACKUP_OFFER_SYSTEM=1 @@ -308,12 +321,11 @@ vs_backup_mount_device() { return 0 } -# vs_backup_pick_drive MOUNT_DIR: lists connected USB drives and mounts the chosen one. -# 0 mounted, 1 nothing mounted (the caller shows the menu again). -vs_backup_pick_drive() { - local mount_dir=$1 json='' sources='' target src listing kind path size label fs uuid reason shown label_text pick - local fstab_file=${VS_FSTAB_FILE:-/etc/fstab} fstab_src fstab_mp fstab_lines='' elsewhere='' - local -a paths=() fss=() uuids=() +# vs_backup_list_drives MOUNT_DIR: the rows of VS_LSBLK_PY for the connected USB drives (stdout). +# 0 when lsblk answered (the list may be empty), 1 with a message on stderr when it could not be asked. +vs_backup_list_drives() { + local mount_dir=$1 json='' sources='' target src + local fstab_file=${VS_FSTAB_FILE:-/etc/fstab} fstab_src fstab_mp fstab_lines='' if ! command -v lsblk >/dev/null 2>&1 || ! command -v python3 >/dev/null 2>&1; then echo "Listing drives needs lsblk and python3." >&2 return 1 @@ -337,7 +349,44 @@ vs_backup_pick_drive() { fstab_lines+="$fstab_src"$'\t'"$fstab_mp"$'\n' done <"$fstab_file" fi - listing=$(printf '%s' "$json" | python3 -I -c "$VS_LSBLK_PY" "$sources" "$fstab_lines") || listing= + local rows + if ! rows=$(printf '%s' "$json" | python3 -I -c "$VS_LSBLK_PY" "$sources" "$fstab_lines"); then + echo "Could not read the list of connected drives." >&2 + return 1 + fi + [ -z "$rows" ] || printf '%s\n' "$rows" +} + +# Reads the rows of vs_backup_list_drives on stdin and prints one JSON object. Nothing here is +# trusted by a later mount: the root side lists again and requires exactly one match. +VS_ROWS_JSON_PY=' +import json, sys +out = {"drives": [], "refused": [], "mounted_elsewhere": []} +for line in sys.stdin.read().split("\n"): + parts = line.split("\x1f") + if len(parts) < 6: + continue + kind, device, size, label, fstype, last = parts[:6] + base = {"device": device, "size": size, "label": label, "fstype": fstype} + if kind == "OK": + out["drives"].append(dict(base, uuid=last)) + elif kind == "NO": + out["refused"].append(dict(base, reason=last)) + elif kind == "MOUNTED": + out["mounted_elsewhere"].append(dict(base, mountpoint=last)) +print(json.dumps(out, sort_keys=True)) +' + +vs_backup_drives_json() { + python3 -I -c "$VS_ROWS_JSON_PY" +} + +# vs_backup_pick_drive MOUNT_DIR: lists connected USB drives and mounts the chosen one. +# 0 mounted, 1 nothing mounted (the caller shows the menu again). +vs_backup_pick_drive() { + local mount_dir=$1 listing kind path size label fs uuid reason shown label_text pick elsewhere='' + local -a paths=() fss=() uuids=() + listing=$(vs_backup_list_drives "$mount_dir") || return 1 echo "Connected USB drives that are not mounted:" >&2 shown=0 while IFS=$'\x1f' read -r kind path size label fs uuid; do @@ -383,7 +432,16 @@ vs_backup_menu() { has_fstab=1 fi VS_BACKUP_OFFER_SYSTEM=0 - [ -n "$strict" ] || vs_backup_system_disk_check "$dest" + # The archive of choice [4] goes to the system-disk folder, so that is where the space is checked. + # With the older controller (VS_BACKUP_ONCE_REDIRECTS=0) the archive lands under the unmounted folder. + # Choice [4] is not offered when the configuration locks the system disk, so nothing is measured then. + if [ -z "$strict" ] && [ "${VS_BACKUP_LOCKED:-0}" != 1 ]; then + if [ "${VS_BACKUP_ONCE_REDIRECTS:-1}" = 1 ]; then + vs_backup_system_disk_check "$VS_BACKUP_SYSTEM_DIR" + else + vs_backup_system_disk_check "$dest" + fi + fi while :; do { echo @@ -425,7 +483,13 @@ vs_backup_menu() { ;; 4) echo "WARNING: the backup will be written to the system disk (SD card) this once. It will not survive loss of that disk, and the next backup asks again." >&2 + if [ "${VS_BACKUP_ONCE_REDIRECTS:-1}" = 1 ]; then + echo "It goes to $(vs_printable "$VS_BACKUP_SYSTEM_DIR"); $(vs_printable "$mount_dir") stays empty." >&2 + else + echo "This installed controller is older, so it writes the archive under $(vs_printable "$mount_dir") on the system disk." >&2 + fi VS_BACKUP_ALLOW_ONCE=1 + VS_BACKUP_ONCE_TARGET=$VS_BACKUP_SYSTEM_DIR return 0 ;; esac @@ -436,6 +500,7 @@ vs_backup_destination_fix() { local dest=$1 context=${2:-update} strict=${3:-} mount_dir rc=0 update_dir=${VS_BACKUP_UPDATE_DIR:-} mount_dir=$(printf '%s' "$dest" | cut -d/ -f1-3) VS_BACKUP_ALLOW_ONCE=0 + VS_BACKUP_ONCE_TARGET= if ! vs_prompt_open; then { echo "Refusing to back up: $(vs_printable "$mount_dir") is not mounted, so the archive would land on the system disk (SD card). Nothing was changed." diff --git a/scripts/vesselstackctl b/scripts/vesselstackctl index 1604cbb..21c387f 100755 --- a/scripts/vesselstackctl +++ b/scripts/vesselstackctl @@ -10,6 +10,8 @@ CORE_COMPONENTS=(homeassistant influxdb grafana prometheus mosquitto heimdall) SYSTEM_ROOT=${VESSELSTACK_SYSTEM_ROOT:-} SYSTEMD_DIR="$SYSTEM_ROOT/etc/systemd/system" SBIN_DIR="$SYSTEM_ROOT/usr/local/sbin" +# Where "back up to the system disk this once" writes, so an unmounted /media or /mnt folder stays empty. +SYSTEM_BACKUP_DIR="$SYSTEM_ROOT/var/backups/vesselstack" LIBEXEC_DIR="$SYSTEM_ROOT/usr/local/libexec/vesselstack" usage() { @@ -51,6 +53,13 @@ Commands: whole stack briefly). Refuses /media and /mnt destinations that are not mounted unless --allow-unmounted is given + backup --system-disk-once Back up to the system disk (not the unmounted drive's + folder) this once; refused when the system disk is too + small or VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false + backup-drive list --json List the connected USB drives that could hold backups + backup-drive mount --fstab | --uuid UUID + Mount the backup folder's drive (from /etc/fstab, or the + listed drive with that UUID); never edits /etc/fstab verify-backup ARCHIVE Verify an archive and its checksum restore ARCHIVE --yes [--skip-installer] Stage and verify a backup, then swap it in; the @@ -1505,8 +1514,10 @@ load_remediation_lib() { fix_unmounted_backup() { local destination=$1 mount_dir=$2 strict=${3:-} context=${VS_BACKUP_CONTEXT:-backup} update_dir=${VS_BACKUP_UPDATE_DIR:-} if load_remediation_lib; then - VS_BACKUP_ROOT=$VESSELSTACK_ROOT VS_BACKUP_DATA=$VESSELSTACK_DATA VS_BACKUP_ENV_FILE=$ENV_FILE \ - vs_backup_destination_fix "$destination" "$context" "$strict" + local locked=0 + if backup_system_disk_locked; then locked=1; fi + VS_BACKUP_ROOT=$VESSELSTACK_ROOT VS_BACKUP_DATA=$VESSELSTACK_DATA VS_BACKUP_ENV_FILE=$ENV_FILE VS_BACKUP_LOCKED=$locked \ + VS_BACKUP_SYSTEM_DIR=$SYSTEM_BACKUP_DIR vs_backup_destination_fix "$destination" "$context" "$strict" return fi echo "Refusing to back up: $mount_dir is not mounted, so the archive would land on the system disk (SD card). Nothing was changed." >&2 @@ -1554,8 +1565,11 @@ verify_mounted_destination() { done } +# Sets BACKUP_REDIRECT to the system-disk folder when the owner chose "this once" at the menu. +BACKUP_REDIRECT= check_backup_destination() { local destination=$1 allow_unmounted=$2 mount_dir ancestor + BACKUP_REDIRECT= case "$destination" in /media/*|/mnt/*) mount_dir=$(printf '%s' "$destination" | cut -d/ -f1-3) @@ -1564,8 +1578,17 @@ check_backup_destination() { echo "WARNING: $mount_dir is not a mounted filesystem; the backup will be written to the system disk" >&2 else VS_BACKUP_ALLOW_ONCE=0 + VS_BACKUP_ONCE_TARGET= fix_unmounted_backup "$destination" "$mount_dir" || return 1 - if [ "${VS_BACKUP_ALLOW_ONCE:-0}" != 1 ] && mountpoint -q "$mount_dir"; then + if [ "${VS_BACKUP_ALLOW_ONCE:-0}" = 1 ]; then + if backup_system_disk_locked; then + echo "Refusing to back up to the system disk: VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false is set. Nothing was changed." >&2 + return 1 + fi + # The unmounted folder stays empty: the archive goes to the system-disk folder. + BACKUP_REDIRECT=${VS_BACKUP_ONCE_TARGET:-$SYSTEM_BACKUP_DIR} + destination=$BACKUP_REDIRECT + elif mountpoint -q "$mount_dir"; then verify_mounted_destination "$destination" "$mount_dir" || return 1 fi fi @@ -1579,18 +1602,94 @@ check_backup_destination() { fi } +# True (0) when vesselstack.env, or the environment, pins VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false. +backup_system_disk_locked() { + [ "${VESSELSTACK_BACKUP_ALLOW_UNMOUNTED:-}" = false ] && return 0 + [ -r "$ENV_FILE" ] && grep -Eq '^(export[[:space:]]+)?VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=["'"'"']?false' "$ENV_FILE" +} + +# Prints the folder for "back up to the system disk this once": the system-disk folder when the +# configured destination sits on an unmounted /media or /mnt folder, otherwise the configured one. +# Refuses (exit 1) when the owner pinned VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false or the archive +# will not fit on the system disk. Nothing has been stopped yet. +system_disk_once_target() { + local configured mount_dir='' target redirected=0 + if backup_system_disk_locked; then + echo "Refusing to back up to the system disk: VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false is set. Nothing was changed." >&2 + return 1 + fi + configured=$(realpath -m -- "${VESSELSTACK_BACKUP:-/var/backups/vesselstack}") || return 1 + target=$configured + case "$configured" in + /media/*|/mnt/*) + mount_dir=$(printf '%s' "$configured" | cut -d/ -f1-3) + if mountpoint -q "$mount_dir"; then + echo "$mount_dir is mounted, so the backup goes to the drive as usual." >&2 + else + target=$SYSTEM_BACKUP_DIR + redirected=1 + fi + ;; + esac + if ! load_remediation_lib; then + echo "Refusing to back up to the system disk: the installed prompt library could not be trusted, so the free space cannot be checked. Nothing was changed." >&2 + return 1 + fi + VS_BACKUP_ROOT=$VESSELSTACK_ROOT VS_BACKUP_DATA=$VESSELSTACK_DATA VS_BACKUP_ENV_FILE=$ENV_FILE \ + VS_BACKUP_SPACE_LABEL=$([ "$redirected" -eq 1 ] && echo "system disk" || echo "backup drive") \ + vs_backup_system_disk_check "$target" + if [ "${VS_BACKUP_OFFER_SYSTEM:-0}" != 1 ]; then + if [ "$redirected" -eq 1 ]; then + echo "Refusing to back up to the system disk. Nothing was changed." >&2 + else + echo "Refusing to back up: the backup drive has too little free space. Nothing was changed." >&2 + fi + return 1 + fi + if [ "${VS_BACKUP_SIZE_UNKNOWN:-0}" = 1 ]; then + echo "Refusing to back up: the size of the VesselStack data could not be measured, so the free space cannot be checked. Nothing was changed." >&2 + return 1 + fi + if [ "$redirected" -eq 1 ]; then + echo "WARNING: the backup will be written to the system disk (SD card) this once. It will not survive loss of that disk." >&2 + echo "It goes to $target; $mount_dir stays empty." >&2 + fi + printf '%s\n' "$target" +} + +check_backup_tree_overlap() { + python3 - "$VESSELSTACK_ROOT" "$VESSELSTACK_DATA" "$1" <<'PY' +import pathlib +import sys + +root, data, backup = (pathlib.Path(value).resolve(strict=False) for value in sys.argv[1:]) +if backup == pathlib.Path("/"): + raise SystemExit("Backup destination must be a specific directory, not /") +if backup == root or root in backup.parents or backup == data or data in backup.parents: + raise SystemExit("Backup destination must be outside the VesselStack application and data trees") +PY +} + backup_stack() { - local allow_unmounted=0 require_mounted=0 destination='' + local allow_unmounted=0 require_mounted=0 system_once=0 destination='' mount_dir while [ $# -gt 0 ]; do case "$1" in --require-mounted) require_mounted=1 ;; --allow-unmounted) allow_unmounted=1 ;; + --system-disk-once) system_once=1 ;; -*) echo "Unknown backup option: $1" >&2; exit 2 ;; *) [ -z "$destination" ] || { echo "backup takes at most one directory" >&2; exit 2; } destination=$1 ;; esac shift done + if [ "$system_once" -eq 1 ]; then + if [ "$require_mounted" -eq 1 ] || [ "$allow_unmounted" -eq 1 ] || [ -n "$destination" ]; then + echo "--system-disk-once cannot be combined with --require-mounted, --allow-unmounted or a directory" >&2 + exit 2 + fi + destination=$(system_disk_once_target) || exit 1 + fi destination=${destination:-${VESSELSTACK_BACKUP:-/var/backups/vesselstack}} destination=$(python3 - "$destination" <<'PY' import os, sys @@ -1612,17 +1711,12 @@ PY fi local stamp archive manifest sample_timer_active=0 indexer_timer_active=0 chat_active=0 telegram_active=0 displays_active=0 signalk_active=0 socketcan_active=0 local -a extras was_compose_services=() - python3 - "$VESSELSTACK_ROOT" "$VESSELSTACK_DATA" "$destination" <<'PY' -import pathlib -import sys - -root, data, backup = (pathlib.Path(value).resolve(strict=False) for value in sys.argv[1:]) -if backup == pathlib.Path("/"): - raise SystemExit("Backup destination must be a specific directory, not /") -if backup == root or root in backup.parents or backup == data or data in backup.parents: - raise SystemExit("Backup destination must be outside the VesselStack application and data trees") -PY + check_backup_tree_overlap "$destination" check_backup_destination "$destination" "$allow_unmounted" + if [ -n "$BACKUP_REDIRECT" ]; then + destination=$BACKUP_REDIRECT + check_backup_tree_overlap "$destination" + fi mapfile -t extras < <(extra_backup_paths) stamp=$(date -u +%Y%m%dT%H%M%SZ) archive="$destination/vesselstack-$stamp.tar.gz" @@ -1878,11 +1972,152 @@ PY echo "Verify the system, then remove them to free disk space: sudo rm -rf -- '$root.pre-restore-$stamp' '$data.pre-restore-$stamp'" } +# ---------------------------------------------------------------- backup drive (control panel helpers) +# The Control Panel never names a device or a folder: it passes a drive UUID that this side found +# in its own list, and the folder always derives from the configured VESSELSTACK_BACKUP. + +# Sets BD_DEST (resolved backup folder) and BD_MOUNT_DIR (its /media or /mnt mount folder). +backup_drive_target() { + BD_DEST=$(python3 -I - "${VESSELSTACK_BACKUP:-/var/backups/vesselstack}" <<'PY' +import os, sys +print(os.path.realpath(sys.argv[1])) +PY +) || return 1 + case "$BD_DEST" in + /media/*|/mnt/*) BD_MOUNT_DIR=$(printf '%s' "$BD_DEST" | cut -d/ -f1-3) ;; + *) + echo "Refusing: the backup folder ${BD_DEST//[![:print:]]/?} is not under /media or /mnt, so there is no drive to mount. Nothing was changed." >&2 + return 1 + ;; + esac +} + +backup_drive_list_json() { + local mounted=false fstab=false listing='' listing_error='' errfile removable=true + BD_DEST=$(python3 -I - "${VESSELSTACK_BACKUP:-/var/backups/vesselstack}" <<'PY' +import os, sys +print(os.path.realpath(sys.argv[1])) +PY +) + BD_MOUNT_DIR= + case "$BD_DEST" in + /media/*|/mnt/*) BD_MOUNT_DIR=$(printf '%s' "$BD_DEST" | cut -d/ -f1-3) ;; + *) removable=false ;; + esac + if [ "$removable" = true ]; then + if mountpoint -q "$BD_MOUNT_DIR"; then mounted=true; fi + if command -v findmnt >/dev/null 2>&1 && findmnt --fstab -n --mountpoint "$BD_MOUNT_DIR" >/dev/null 2>&1; then fstab=true; fi + if [ "$mounted" = false ]; then + if load_remediation_lib; then + errfile=$(mktemp) + listing=$(VS_BACKUP_DATA=$VESSELSTACK_DATA vs_backup_list_drives "$BD_MOUNT_DIR" 2>"$errfile") || listing_error=$(head -c 300 "$errfile" | tr -c '[:print:]' '?') + rm -f -- "$errfile" + else + listing_error="The installed drive library could not be trusted." + fi + fi + fi + if load_remediation_lib; then + printf '%s' "$listing" | vs_backup_drives_json | python3 -I -c ' +import json, sys +data = json.load(sys.stdin) +data.update(path=sys.argv[1], mount_dir=sys.argv[2], removable=sys.argv[3] == "true", + mounted=sys.argv[4] == "true", fstab_mount=sys.argv[5] == "true", listing_error=sys.argv[6]) +print(json.dumps(data, sort_keys=True)) +' "$BD_DEST" "$BD_MOUNT_DIR" "$removable" "$mounted" "$fstab" "$listing_error" + else + echo "The installed drive library could not be trusted." >&2 + exit 1 + fi +} + +backup_drive_mount() { + local mode='' uuid='' kind path size label fs id listing count=0 dev='' devfs='' actual='' + while [ $# -gt 0 ]; do + case "$1" in + --fstab) [ -z "$mode" ] || { usage >&2; exit 2; }; mode=fstab ;; + --uuid) + [ -z "$mode" ] && [ $# -ge 2 ] || { usage >&2; exit 2; } + mode=uuid; uuid=$2; shift ;; + *) usage >&2; exit 2 ;; + esac + shift + done + [ -n "$mode" ] || { usage >&2; exit 2; } + if [ "$mode" = uuid ] && [[ ! $uuid =~ ^[A-Za-z0-9-]{1,64}$ ]]; then + echo "Refusing to mount: that is not a drive id. Nothing was changed." >&2 + exit 2 + fi + if ! load_remediation_lib; then + echo "Refusing to mount: the installed drive library could not be trusted. Nothing was changed." >&2 + exit 1 + fi + backup_drive_target || exit 1 + if mountpoint -q "$BD_MOUNT_DIR"; then + echo "Refusing to mount: $(vs_printable "$BD_MOUNT_DIR") is already mounted. Nothing was changed." >&2 + exit 1 + fi + if [ -L "$BD_MOUNT_DIR" ] || { [ -e "$BD_MOUNT_DIR" ] && [ ! -d "$BD_MOUNT_DIR" ]; }; then + echo "Refusing to mount: $(vs_printable "$BD_MOUNT_DIR") exists but is not a plain folder. Nothing was changed." >&2 + exit 1 + fi + if [ -d "$BD_MOUNT_DIR" ] && [ -n "$(ls -A -- "$BD_MOUNT_DIR" 2>/dev/null)" ]; then + echo "Refusing to mount: $(vs_printable "$BD_MOUNT_DIR") is not empty (files hidden under a mount would be unreachable). Nothing was changed." >&2 + exit 1 + fi + if [ "$mode" = fstab ]; then + if ! command -v findmnt >/dev/null 2>&1 || ! findmnt --fstab -n --mountpoint "$BD_MOUNT_DIR" >/dev/null 2>&1; then + echo "Refusing to mount: $(vs_printable "$BD_MOUNT_DIR") is not listed in /etc/fstab. Nothing was changed." >&2 + exit 1 + fi + if ! mount -o nodev,nosuid,noexec -- "$BD_MOUNT_DIR" || ! mountpoint -q "$BD_MOUNT_DIR"; then + echo "Could not mount $(vs_printable "$BD_MOUNT_DIR") from /etc/fstab." >&2 + exit 1 + fi + echo "Mounted $(vs_printable "$BD_MOUNT_DIR")." >&2 + else + listing=$(VS_BACKUP_DATA=$VESSELSTACK_DATA vs_backup_list_drives "$BD_MOUNT_DIR") || exit 1 + while IFS=$'\x1f' read -r kind path size label fs id; do + [ "$kind" = OK ] && [ "$id" = "$uuid" ] || continue + count=$((count + 1)); dev=$path; devfs=$fs + done <<<"$listing" + if [ "$count" -ne 1 ]; then + echo "Refusing to mount: the drive is not (or no longer) listed exactly once as a usable backup drive. Nothing was changed." >&2 + exit 1 + fi + vs_backup_mount_device "$BD_MOUNT_DIR" "$dev" "$devfs" "$uuid" || exit 1 + actual=$(findmnt -n -o UUID --mountpoint "$BD_MOUNT_DIR" 2>/dev/null | head -n 1) + if [ "${actual,,}" != "${uuid,,}" ]; then + umount -- "$BD_MOUNT_DIR" || echo "WARNING: could not unmount $(vs_printable "$BD_MOUNT_DIR") again; unmount it by hand: sudo umount $(printf '%q' "$BD_MOUNT_DIR")" >&2 + echo "Refusing to use the drive: what is mounted at $(vs_printable "$BD_MOUNT_DIR") is not the drive that was chosen. It was unmounted again." >&2 + exit 1 + fi + fi + if ! verify_mounted_destination "$BD_DEST" "$BD_MOUNT_DIR"; then + umount -- "$BD_MOUNT_DIR" || echo "WARNING: could not unmount $BD_MOUNT_DIR again; unmount it by hand: sudo umount $(printf '%q' "$BD_MOUNT_DIR")" >&2 + exit 1 + fi + echo "Backup drive mounted at $(vs_printable "$BD_MOUNT_DIR")" +} + +backup_drive_command() { + [ $# -ge 1 ] || { usage >&2; exit 2; } + local sub=$1 + shift + case "$sub" in + list) [ $# -eq 1 ] && [ "$1" = --json ] || { usage >&2; exit 2; }; backup_drive_list_json ;; + mount) backup_drive_mount "$@" ;; + *) usage >&2; exit 2 ;; + esac +} + update_stack() { - local source='' offline=${VESSELSTACK_OFFLINE:-false} argument + local source='' offline=${VESSELSTACK_OFFLINE:-false} argument choice='' for argument in "$@"; do case "$argument" in --offline) offline=true ;; + # Used by the Control Panel only (see README); not advertised for terminal use. + --backup-choice=system-disk-once|--backup-choice=skip) choice=${argument#--backup-choice=} ;; -*) usage >&2; exit 2 ;; *) [ -z "$source" ] || { usage >&2; exit 2; }; source=$argument ;; esac @@ -1895,9 +2130,17 @@ update_stack() { local install_args=(--config "$ENV_FILE") [ "$offline" != true ] || install_args+=(--offline) [ -x "$source/install.sh" ] || { echo "Not a VesselStack release directory: $source" >&2; exit 1; } + if [ -n "$choice" ] && backup_system_disk_locked; then + echo "Refusing to update without the configured backup: VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false is set. Nothing was changed." >&2 + exit 1 + fi "$source/install.sh" "${install_args[@]}" --dry-run - # The backup check runs before anything is stopped; it may ask where to back up. - VS_BACKUP_CONTEXT=update VS_BACKUP_UPDATE_DIR=$source backup_stack "${VESSELSTACK_BACKUP:-/var/backups/vesselstack}" + case "$choice" in + system-disk-once) VS_BACKUP_CONTEXT=update VS_BACKUP_UPDATE_DIR=$source backup_stack --system-disk-once ;; + skip) echo "WARNING: backup skipped at the owner's request for this update; there is no archive to restore from." >&2 ;; + # The backup check runs before anything is stopped; it may ask where to back up. + *) VS_BACKUP_CONTEXT=update VS_BACKUP_UPDATE_DIR=$source backup_stack "${VESSELSTACK_BACKUP:-/var/backups/vesselstack}" ;; + esac "$source/install.sh" "${install_args[@]}" --start } @@ -1986,6 +2229,7 @@ case "$command" in restart) [ $# -eq 0 ] || { usage; exit 2; }; stop_stack; start_stack ;; component) [ "$#" -eq 2 ] || { usage; exit 2; }; case "$1" in add|remove) component_change "$1" "$2" ;; *) echo "Unknown component action: $1" >&2; exit 2 ;; esac ;; backup) backup_stack "$@" ;; + backup-drive) backup_drive_command "$@" ;; verify-backup) [ $# -eq 1 ] || { usage; exit 2; }; verify_backup "$1" ;; restore) [ $# -ge 2 ] || { usage; exit 2; }; restore_stack "$@" ;; update) [ $# -ge 1 ] || { usage; exit 2; }; update_stack "$@" ;; diff --git a/tests/test-get-vesselstack.sh b/tests/test-get-vesselstack.sh index 967a1eb..a41b832 100755 --- a/tests/test-get-vesselstack.sh +++ b/tests/test-get-vesselstack.sh @@ -166,9 +166,16 @@ echo "ctl $* WHEELHOUSE=${VESSELSTACK_WHEELHOUSE:-} OFFLINE=${VESSELSTACK_OFFLIN echo "ctlenv $1 NONINTERACTIVE=${VESSELSTACK_NONINTERACTIVE:-} ALLOW=${VESSELSTACK_BACKUP_ALLOW_UNMOUNTED:-}" >> "$LOG" # Behave like the released v0.2.0/v0.3.0 controllers: "update" reads ONLY $1, as the # release directory, and fails on anything else (for example a leading --offline). +# FAKE_CTL_NEW=1 behaves like this release's controller: --help lists backup-drive and update takes +# --backup-choice=system-disk-once after the directory. +if [ "$1" = --help ] && [ -n "${FAKE_CTL_NEW:-}" ]; then + echo " backup-drive list --json" + exit 0 +fi if [ "$1" = update ]; then [ -x "$2/install.sh" ] || { echo "Not a VesselStack release directory: $2" >&2; exit 1; } - [ "$#" -eq 2 ] || { echo "old ctl: unexpected extra arguments" >&2; exit 1; } + if [ -n "${FAKE_CTL_NEW:-}" ] && [ "$#" -eq 3 ] && [ "$3" = --backup-choice=system-disk-once ]; then : + else [ "$#" -eq 2 ] || { echo "old ctl: unexpected extra arguments" >&2; exit 1; }; fi fi case "$1" in urls) echo "Boat Chat http://0.0.0.0:8765" ;; esac [ -z "${FAKE_CTL_FAIL:-}" ] || [ "$1" != "${FAKE_CTL_FAIL}" ] @@ -1569,6 +1576,23 @@ expect_out "\\[4\\] Back up to the system disk this once" expect_out "WARNING: the backup will be written to the system disk" expect_log "^ctl update $RELEASE_BASE/$VER " expect_log "^ctlenv update NONINTERACTIVE=true ALLOW=true$" +refuse_log "backup-choice" +expect_out "This installed controller is older, so it writes the archive under $BACKUP_MNT on the system disk" +refuse_out "stays empty" +test ! -e "$BACKUP_MNT" + +# A controller of this release takes the choice as an option (the archive goes to the system-disk +# folder and the unmounted folder stays empty); the environment variable is not set. +backup_upgrade +printf 'y\ny\n4\n' > "$TEST_ROOT/answers" +EXTRA_ENV=VESSELSTACK_BOOTSTRAP_ANSWERS="$TEST_ROOT/answers" EXTRA_ENV2=FAKE_CTL_NEW=1 run +EXTRA_ENV= +EXTRA_ENV2= +expect_rc 0 "backup drive: system disk once with a current controller" +expect_out "It goes to /var/backups/vesselstack; $BACKUP_MNT stays empty." +expect_log "^ctl update $RELEASE_BASE/$VER --backup-choice=system-disk-once " +expect_out "Running: vesselstackctl update $RELEASE_BASE/$VER --backup-choice=system-disk-once" +expect_log "^ctlenv update NONINTERACTIVE=true ALLOW=$" test ! -e "$BACKUP_MNT" backup_upgrade diff --git a/tests/test-lifecycle.sh b/tests/test-lifecycle.sh index 14a8bbf..33aa800 100755 --- a/tests/test-lifecycle.sh +++ b/tests/test-lifecycle.sh @@ -190,11 +190,20 @@ echo "\${*: -1}" >> "\$FAKE_MOUNTS" [ -z "\${FAKE_MOUNT_CHOWN:-}" ] || chown "\$FAKE_MOUNT_CHOWN" "\${*: -1}" if [ -n "\${FAKE_MOUNT_LINK:-}" ]; then ln -s /tmp "\${*: -1}/backups"; fi EOF +# Fake umount logs its arguments and drops the folder from the mounted list. +cat > "$FAKE_BIN/umount" <> "$TEST_ROOT/mount.log" +grep -Fvx -- "\${*: -1}" "\$FAKE_MOUNTS" > "\$FAKE_MOUNTS.new" || true +mv "\$FAKE_MOUNTS.new" "\$FAKE_MOUNTS" +EOF cat > "$FAKE_BIN/findmnt" <> "$TEST_ROOT/findmnt.log" case " \$* " in *" --fstab "*) grep -Fxq -- "\${*: -1}" "\${FAKE_FSTAB:-/dev/null}" ;; + # The UUID of the filesystem mounted at a folder: the drive that backup-drive mount chose, unless the test says otherwise. + *" -o UUID "*) printf '%s\n' "\${FAKE_MOUNT_UUID:-1111-aaaa}" ;; *" --target "*) awk -v t="\${*: -1}" '\$1 == t {print \$2}' "\${FAKE_FINDMNT_SOURCES:-/dev/null}" ;; esac EOF @@ -206,6 +215,7 @@ EOF # Fake df reports FAKE_DF_FREE KiB free when set. cat > "$FAKE_BIN/df" <> "$TEST_ROOT/df.log" if [ -n "\${FAKE_DF_FREE:-}" ]; then printf 'Filesystem 1024-blocks Used Available Capacity Mounted on\nfake 1000 0 %s 0%% /\n' "\$FAKE_DF_FREE" exit 0 @@ -213,7 +223,15 @@ fi PATH=\$(printf '%s' "\$PATH" | tr ':' '\n' | grep -vxF "$FAKE_BIN" | paste -sd:) exec df "\$@" EOF -chmod +x "$FAKE_BIN/mountpoint" "$FAKE_BIN/mount" "$FAKE_BIN/findmnt" "$FAKE_BIN/lsblk" "$FAKE_BIN/df" +# Fake du fails like a timeout (exit 124) when FAKE_DU_TIMEOUT is set. +cat > "$FAKE_BIN/du" < "$DATA_ROOT/prometheus/lifecycle-marker" archive=$(run_ctl backup "$BACKUP_ROOT" | tail -n 1) -tar -tzf "$archive" | grep -q '/prometheus/lifecycle-marker$' || { +# The listing goes to a variable first: `tar -t | grep -q` under pipefail fails with SIGPIPE whenever grep +# matches early in a listing longer than one pipe write (which entry comes first depends on the filesystem). +prometheus_listing=$(tar -tzf "$archive") +grep -q '/prometheus/lifecycle-marker$' <<<"$prometheus_listing" || { echo 'Backup is missing the Prometheus data directory' >&2 exit 1 } @@ -394,13 +415,13 @@ if ! grep -q '/displays/bridge.py' <<<"$archive_listing"; then echo 'Displays sources are missing from the backup' >&2 exit 1 fi -if tar -tzf "$archive" | grep -q '/boat-chat/.venv'; then +if grep -q '/boat-chat/.venv' <<<"$archive_listing"; then echo 'Rebuildable virtual environment was included in backup' >&2 exit 1 fi for extra in etc/systemd/system/vesselstack-chat.service usr/local/sbin/vesselstackctl \ usr/local/sbin/vesselstack-firewall etc/vesselstack/update.env; do - tar -tzf "$archive" | grep -q "${SYSTEM_ROOT#/}/$extra\$" || { + grep -q "${SYSTEM_ROOT#/}/$extra\$" <<<"$archive_listing" || { echo "Backup is missing $extra" >&2 exit 1 } @@ -690,7 +711,14 @@ EOF menu_run $'4\n' backup "$unmounted/backups" test "$MENU_RC" -eq 0 want "WARNING: the backup will be written to the system disk (SD card) this once." + want "It goes to $SYSTEM_ROOT/var/backups/vesselstack; $unmounted stays empty." test -s "$MENU_OUT" + # The archive lands in the system-disk folder; the unmounted mount folder is never created. + case "$(tail -n 1 "$TEST_ROOT/menu.out")" in + "$SYSTEM_ROOT/var/backups/vesselstack/vesselstack-"*.tar.gz) ;; + *) echo "menu choice [4] wrote the archive somewhere else: $(tail -n 1 "$TEST_ROOT/menu.out")" >&2; exit 1 ;; + esac + test ! -e "$unmounted" test "$(sha256sum "$APP_ROOT/config/vesselstack.env")" = "$env_hash" rm -rf -- "$unmounted" # Hidden when the system disk is too small for the backup ... @@ -705,6 +733,17 @@ EOF refuse_text "[4] Back up" want "Please enter one of: 2, 3, or c" cp -p "$env_backup" "$APP_ROOT/config/vesselstack.env" + # ... and when the lock is only in the process environment. + VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false menu_run $'4\nc\n' backup "$unmounted/backups" + test "$MENU_RC" -eq 1 + refuse_text "[4] Back up" + want "Please enter one of: 2, 3, or c" + # The system-disk folder is where the free space is measured (the archive goes there), not the unmounted folder. + ancestor_of() { local d=$1; while [ ! -e "$d" ] && [ "$d" != / ]; do d=$(dirname -- "$d"); done; printf '%s' "$d"; } + : > "$TEST_ROOT/df.log" + menu_run $'c\n' backup "$unmounted/backups" + grep -qxF -- "df -Pk -- $(ancestor_of "$SYSTEM_ROOT/var/backups/vesselstack")" "$TEST_ROOT/df.log" + rm -rf -- "$unmounted" # --require-mounted offers [1]-[3] only, even with the override in the file. printf 'VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=true\n' >> "$APP_ROOT/config/vesselstack.env" @@ -715,6 +754,196 @@ EOF cp -p "$env_backup" "$APP_ROOT/config/vesselstack.env" rm -rf -- "$unmounted" + # ---- Control Panel helpers: backup --system-disk-once, update --backup-choice, backup-drive. + sed -i "s|^VESSELSTACK_BACKUP=.*|VESSELSTACK_BACKUP=$unmounted/backups|" "$APP_ROOT/config/vesselstack.env" + sdo_env="$TEST_ROOT/vesselstack.env.sdo" + cp -p "$APP_ROOT/config/vesselstack.env" "$sdo_env" + sdo_hash=$(sha256sum "$APP_ROOT/config/vesselstack.env") + sdo_run() { # args...: the controller without a terminal; sets MENU_RC, MENU_OUT, MENU_ERR + : > "$TEST_ROOT/mount.log" + : > "$TEST_ROOT/mounts" + : > "$TEST_ROOT/update-args" + : > "$menu_calls" + MENU_RC=0 + VESSELSTACK_NONINTERACTIVE=true run_ctl "$@" >"$TEST_ROOT/menu.out" 2>"$TEST_ROOT/menu.err" &2; exit 1 ;; + esac + test ! -e "$unmounted" + test "$(sha256sum "$APP_ROOT/config/vesselstack.env")" = "$sdo_hash" + # It cannot be combined with the strict or override flags, or with a folder. + for combo in "--require-mounted" "--allow-unmounted" "$BACKUP_ROOT"; do + sdo_run backup --system-disk-once "$combo" + test "$MENU_RC" -eq 2 + test ! -s "$menu_calls" + done + # The lock pinned in the environment file refuses it before anything is stopped. + printf 'VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false\n' >> "$APP_ROOT/config/vesselstack.env" + sdo_run backup --system-disk-once + test "$MENU_RC" -eq 1 + want "VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false is set" + test ! -s "$menu_calls" + sdo_run update --backup-choice=system-disk-once "$UPDATE_SOURCE" + test "$MENU_RC" -eq 1 + test ! -s "$TEST_ROOT/update-args" + sdo_run update --backup-choice=skip "$UPDATE_SOURCE" + test "$MENU_RC" -eq 1 + want "Refusing to update without the configured backup" + test ! -s "$TEST_ROOT/update-args" + test ! -s "$menu_calls" + cp -p "$sdo_env" "$APP_ROOT/config/vesselstack.env" + # A system disk that is too small refuses it before anything is stopped. + FAKE_DF_FREE=1 sdo_run backup --system-disk-once + test "$MENU_RC" -eq 1 + want "so a backup to the system disk is not offered." + test ! -s "$menu_calls" + test ! -e "$unmounted" + # A data size that cannot be measured in time refuses the non-interactive choice (the menu only says "size unknown"). + FAKE_DU_TIMEOUT=1 sdo_run backup --system-disk-once + test "$MENU_RC" -eq 1 + want "could not be measured" + test ! -s "$menu_calls" + test ! -e "$unmounted" + # update: the chosen handling follows the dry run, and nothing is written to the unmounted folder. + sdo_run update --backup-choice=system-disk-once "$UPDATE_SOURCE" + test "$MENU_RC" -eq 0 + test "$(wc -l < "$TEST_ROOT/update-args")" -eq 2 + tail -n 1 "$TEST_ROOT/update-args" | grep -q -- '--start' + test ! -e "$unmounted" + sdo_run update --backup-choice=skip "$UPDATE_SOURCE" + test "$MENU_RC" -eq 0 + want "WARNING: backup skipped at the owner's request for this update; there is no archive to restore from." + test "$(wc -l < "$TEST_ROOT/update-args")" -eq 2 + test ! -s "$menu_calls" + test ! -e "$unmounted" + sdo_run update --backup-choice=bogus "$UPDATE_SOURCE" + test "$MENU_RC" -eq 2 + test ! -s "$TEST_ROOT/update-args" + + # backup-drive list --json: only usable drives are listed; refused and already-mounted ones carry reasons. + json_check() { # python-expression-on-d + python3 -I - "$TEST_ROOT/menu.out" "$1" <<'PY' +import json, sys +d = json.load(open(sys.argv[1])) +# The expressions are literals written in this test file, never external input. +if not eval(sys.argv[2]): + sys.exit("json check failed: %s\n%s" % (sys.argv[2], json.dumps(d, indent=1))) +PY + } + sdo_run backup-drive list --json + test "$MENU_RC" -eq 0 + json_check '[x["uuid"] for x in d["drives"]] == ["1111-aaaa"] and d["drives"][0]["device"] == "/dev/sda1"' + json_check 'any(x["device"] == "/dev/sdb1" and "not supported" in x["reason"] for x in d["refused"])' + json_check 'd["mounted"] is False and d["fstab_mount"] is False and d["mount_dir"] == "'"$unmounted"'"' + printf '%s\n' "$unmounted" > "$TEST_ROOT/fstab" + sdo_run backup-drive list --json + json_check 'd["fstab_mount"] is True' + : > "$TEST_ROOT/fstab" + FAKE_LSBLK="$TEST_ROOT/lsblk-mounted.json" sdo_run backup-drive list --json + json_check 'd["drives"] == [] and d["mounted_elsewhere"][0]["mountpoint"] == "/media/pi/USBSTICK"' + sdo_run backup-drive list --json + printf '%s\n' "$unmounted" > "$TEST_ROOT/mounts" + ( VESSELSTACK_NONINTERACTIVE=true run_ctl backup-drive list --json ) > "$TEST_ROOT/menu.out" 2>/dev/null "$TEST_ROOT/mounts" + sdo_run backup-drive list + test "$MENU_RC" -eq 2 + + # backup-drive mount --uuid: re-lists, mounts exactly that drive, never edits /etc/fstab or the env file. + sdo_run backup-drive mount --uuid 1111-aaaa + test "$MENU_RC" -eq 0 + grep -qx -- "mount -t ext4 -o nodev,nosuid,noexec /dev/sda1 $unmounted" "$TEST_ROOT/mount.log" + [ "$(wc -l < "$TEST_ROOT/mount.log")" -eq 1 ] + want "UUID=1111-aaaa $unmounted ext4 nofail,noatime,nodev,nosuid,noexec 0 2" + test "$(stat -c %a "$unmounted")" = 700 + test "$(sha256sum "$APP_ROOT/config/vesselstack.env")" = "$sdo_hash" + test ! -s "$TEST_ROOT/fstab" + rm -rf -- "$unmounted" + # A drive that is not in the list, a refused filesystem, a malformed id and a duplicate id mount nothing. + for bad in 9999-zzzz 2222-BBBB 3333-cccc; do + sdo_run backup-drive mount --uuid "$bad" + test "$MENU_RC" -eq 1 + test ! -s "$TEST_ROOT/mount.log" + done + for bad in '/dev/sda1' 'a b' "\$(id)" ''; do + sdo_run backup-drive mount --uuid "$bad" + test "$MENU_RC" -eq 2 + test ! -s "$TEST_ROOT/mount.log" + done + sdo_run backup-drive mount --uuid + test "$MENU_RC" -eq 2 + sdo_run backup-drive mount /dev/sda1 + test "$MENU_RC" -eq 2 + cat > "$TEST_ROOT/lsblk-dup.json" < "$TEST_ROOT/mount.log" + printf '%s\n' "$unmounted" > "$TEST_ROOT/mounts" + ( VESSELSTACK_NONINTERACTIVE=true run_ctl backup-drive mount --uuid 1111-aaaa ) >/dev/null 2>"$TEST_ROOT/menu.err" &2; exit 1; fi + rm -rf -- "$unmounted" + install -d "$unmounted" + printf 'keep\n' > "$unmounted/precious.txt" + sdo_run backup-drive mount --uuid 1111-aaaa + test "$MENU_RC" -eq 1 + want "is not empty" + test ! -s "$TEST_ROOT/mount.log" + rm -rf -- "$unmounted" + FAKE_MOUNT_FAIL=1 sdo_run backup-drive mount --uuid 1111-aaaa + test "$MENU_RC" -eq 1 + rm -rf -- "$unmounted" + # A mount whose root another user owns is unmounted again. + FAKE_MOUNT_CHOWN=1234 sdo_run backup-drive mount --uuid 1111-aaaa + test "$MENU_RC" -eq 1 + want "not owned by root" + grep -qx -- "umount -- $unmounted" "$TEST_ROOT/mount.log" + if grep -qx -- "$unmounted" "$TEST_ROOT/mounts"; then echo 'the folder stayed mounted' >&2; exit 1; fi + rm -rf -- "$unmounted" + # A different filesystem at the folder than the chosen UUID is unmounted again and refused. + FAKE_MOUNT_UUID=ffff-ffff sdo_run backup-drive mount --uuid 1111-aaaa + test "$MENU_RC" -eq 1 + want "is not the drive that was chosen" + grep -qx -- "umount -- $unmounted" "$TEST_ROOT/mount.log" + if grep -qx -- "$unmounted" "$TEST_ROOT/mounts"; then echo 'the folder stayed mounted' >&2; exit 1; fi + rm -rf -- "$unmounted" + # --fstab needs an entry for the folder; then it is a plain "mount -- DIR". + sdo_run backup-drive mount --fstab + test "$MENU_RC" -eq 1 + want "is not listed in /etc/fstab" + test ! -s "$TEST_ROOT/mount.log" + printf '%s\n' "$unmounted" > "$TEST_ROOT/fstab" + sdo_run backup-drive mount --fstab + test "$MENU_RC" -eq 0 + grep -qx -- "mount -o nodev,nosuid,noexec -- $unmounted" "$TEST_ROOT/mount.log" + : > "$TEST_ROOT/fstab" + rm -rf -- "$unmounted" + # A backup folder outside /media and /mnt has no drive to mount. + sed -i "s|^VESSELSTACK_BACKUP=.*|VESSELSTACK_BACKUP=$BACKUP_ROOT|" "$APP_ROOT/config/vesselstack.env" + sdo_run backup-drive mount --uuid 1111-aaaa + test "$MENU_RC" -eq 1 + test ! -s "$TEST_ROOT/mount.log" + cp -p "$env_backup" "$APP_ROOT/config/vesselstack.env" + rm -rf -- "$unmounted" + # Update: the question comes after the dry run and before anything is stopped. sed -i "s|^VESSELSTACK_BACKUP=.*|VESSELSTACK_BACKUP=$unmounted/backups|" "$APP_ROOT/config/vesselstack.env" : > "$TEST_ROOT/update-args" @@ -726,6 +955,7 @@ EOF test ! -s "$menu_calls" menu_run $'4\n' update "$UPDATE_SOURCE" test "$MENU_RC" -eq 0 + test ! -e "$unmounted" test "$(wc -l < "$TEST_ROOT/update-args")" -eq 3 tail -n 1 "$TEST_ROOT/update-args" | grep -q -- '--start' : > "$TEST_ROOT/update-args" diff --git a/tests/test_browser_update_admission.py b/tests/test_browser_update_admission.py index bf46b75..f488990 100644 --- a/tests/test_browser_update_admission.py +++ b/tests/test_browser_update_admission.py @@ -137,5 +137,133 @@ def test_authenticated_http_admission_rejects_config_component_and_update(self) self.assertEqual((self.root / "config" / "vesselstack.env").read_bytes(), before) + # ---- one-shot backup choices over HTTP + + def serve(self): + server = self.panel.ThreadingHTTPServer(("127.0.0.1", 0), self.panel.ControlPanelHandler) + threading.Thread(target=server.serve_forever, daemon=True).start() + self.addCleanup(server.server_close) + self.addCleanup(server.shutdown) + return server + + def get(self, server, path: str, token: str | None = "test-token") -> tuple[int, dict[str, object]]: + headers = {"X-VesselStack-Token": token} if token else {} + request = urllib.request.Request(f"http://127.0.0.1:{server.server_port}{path}", headers=headers) + try: + with urllib.request.urlopen(request, timeout=3) as response: + return response.status, json.loads(response.read()) + except urllib.error.HTTPError as error: + return error.code, json.loads(error.read()) + + def post_update(self, server, backup: object = "absent") -> tuple[int, dict[str, object], mock.Mock]: + payload: dict[str, object] = {"action": "update-release", "release_id": 42, "channel": "stable"} + if backup != "absent": + payload["backup"] = backup + with mock.patch.object(self.panel, "release_snapshot", return_value={"version": "0.6.2"}), \ + mock.patch.object(self.panel, "start_detached", return_value={"state": "running", "id": 1}) as start: + status, body = self.request(server, "/api/action", payload) + return status, body, start + + def test_release_upgrade_default_call_is_unchanged_without_a_backup_choice(self) -> None: + server = self.serve() + for backup in ("absent", None, {"mode": "require_mounted"}): + with self.subTest(backup=backup): + status, _, start = self.post_update(server, backup) + self.assertEqual(status, 202) + start.assert_called_once_with("update-release", self.root, self.panel.RUNNING_IDENTITY, 42, "stable", target_version="0.6.2", snapshot={"version": "0.6.2"}) + + def test_release_upgrade_passes_each_valid_choice_to_the_worker_request(self) -> None: + server = self.serve() + (self.root / "config" / "vesselstack.env").write_text("VESSELSTACK_BACKUP=/media/backup-drive/backups\n") + valid = ({"mode": "mount_fstab"}, {"mode": "mount_drive", "uuid": "AB12-CD34"}, + {"mode": "system_disk_once", "confirm": "system-disk-once"}, {"mode": "skip", "confirm": "install-without-backup"}) + with mock.patch.object(self.panel.os.path, "ismount", return_value=False), mock.patch.object(self.updates.os.path, "ismount", return_value=False), \ + mock.patch.object(self.panel, "effective_backup_choice", side_effect=lambda c, _: (c, "")): + for backup in valid: + with self.subTest(backup=backup): + status, _, start = self.post_update(server, backup) + self.assertEqual(status, 202) + self.assertEqual(start.call_args.kwargs["backup"], backup) + + def test_release_upgrade_rejects_bad_choices_before_starting(self) -> None: + server = self.serve() + bad = ({"mode": "format"}, {"mode": "mount_drive"}, {"mode": "mount_drive", "uuid": "/dev/sda1"}, {"mode": "mount_drive", "uuid": "a b"}, + {"mode": "mount_fstab", "device": "/dev/sda1"}, {"mode": "mount_fstab", "path": "/media/x"}, {"mode": "skip"}, + {"mode": "skip", "confirm": "system-disk-once"}, {"mode": "system_disk_once", "confirm": "install-without-backup"}, + {"mode": "system_disk_once"}, {"mode": "require_mounted", "confirm": "x"}, "skip", ["skip"], 7) + for backup in bad: + with self.subTest(backup=backup): + status, body, start = self.post_update(server, backup) + self.assertEqual(status, 400, body) + start.assert_not_called() + + def test_lock_in_the_environment_file_refuses_skip_and_system_disk(self) -> None: + server = self.serve() + (self.root / "config" / "vesselstack.env").write_text("VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false\n") + for backup in ({"mode": "skip", "confirm": "install-without-backup"}, {"mode": "system_disk_once", "confirm": "system-disk-once"}): + with self.subTest(backup=backup): + status, body, start = self.post_update(server, backup) + self.assertEqual(status, 400) + self.assertIn("turned off", str(body)) + start.assert_not_called() + with mock.patch.object(self.panel.os.path, "ismount", return_value=False), mock.patch.object(self.panel, "effective_backup_choice", side_effect=lambda c, _: (c, "")): + status, _, start = self.post_update(server, {"mode": "mount_fstab"}) + self.assertEqual(status, 202) + + def test_backup_choice_is_refused_for_other_actions(self) -> None: + server = self.serve() + for action in ("backup", "restart-panel", "restart", "unknown"): + with self.subTest(action=action), mock.patch.object(self.panel.OPERATIONS, "start", return_value={"state": "running"}) as start, \ + mock.patch.object(self.panel, "start_detached", return_value={"state": "running"}) as detached: + status, body = self.request(server, "/api/action", {"action": action, "backup": {"mode": "skip", "confirm": "install-without-backup"}}) + self.assertEqual(status, 400, body) + start.assert_not_called(); detached.assert_not_called() + + def test_process_environment_lock_refuses_skip_over_http(self) -> None: + server = self.serve() + with mock.patch.dict(self.panel.os.environ, {"VESSELSTACK_BACKUP_ALLOW_UNMOUNTED": "false"}): + status, body, start = self.post_update(server, {"mode": "skip", "confirm": "install-without-backup"}) + self.assertEqual(status, 400) + start.assert_not_called() + + def test_folder_update_choices_are_validated_and_run_the_mount_as_a_separate_step(self) -> None: + server = self.serve() + (self.root / "config" / "vesselstack.env").write_text("VESSELSTACK_BACKUP=/media/backup-drive/backups\n") + def post(backup): + with mock.patch.object(self.panel.OPERATIONS, "start", return_value={"state": "running"}) as start, \ + mock.patch.object(self.panel, "effective_backup_choice", side_effect=lambda c, _: (c, "")), \ + mock.patch.object(self.panel, "ctl_supports", return_value=True): + payload = {"action": "update", "source_directory": "/var/lib/vesselstack/releases/x"} + if backup != "absent": + payload["backup"] = backup + status, body = self.request(server, "/api/action", payload) + return status, body, start + status, _, start = post("absent") + self.assertEqual(status, 202) + self.assertEqual(start.call_args.args[0], "update"); self.assertEqual(start.call_args.kwargs, {}) + status, _, start = post({"mode": "mount_drive", "uuid": "AB12-CD34"}) + self.assertEqual(status, 202) + self.assertEqual(start.call_args.kwargs["extra"], {"backup_mode": "mount_drive"}) + self.assertEqual(start.call_args.kwargs["before"]()[0][-2:], ["--uuid", "AB12-CD34"]) + status, _, start = post({"mode": "skip", "confirm": "install-without-backup"}) + self.assertEqual(start.call_args.kwargs["extra"], {"backup_mode": "skip", "backup_skipped": True}) + self.assertEqual(start.call_args.kwargs["before"](), []) + for bad in ({"mode": "skip"}, {"mode": "mount_drive", "uuid": "../x"}, {"mode": "nope"}): + status, _, start = post(bad) + self.assertEqual(status, 400) + start.assert_not_called() + + def test_preflight_requires_the_token_and_is_read_only(self) -> None: + server = self.serve() + self.assertEqual(self.get(server, "/api/backup-preflight", token=None)[0], 401) + self.assertEqual(self.get(server, "/api/backup-preflight", token="wrong")[0], 401) + before = (self.root / "config" / "vesselstack.env").read_bytes() + with mock.patch.object(self.panel, "command_stdout", return_value=(1, "")): + status, body = self.get(server, "/api/backup-preflight") + self.assertEqual(status, 200) + self.assertIn("default_ok", body) + self.assertEqual((self.root / "config" / "vesselstack.env").read_bytes(), before) + + if __name__ == "__main__": unittest.main() diff --git a/tests/test_browser_updates.py b/tests/test_browser_updates.py index 73da761..8216729 100644 --- a/tests/test_browser_updates.py +++ b/tests/test_browser_updates.py @@ -2,6 +2,7 @@ import importlib.util import hashlib import io +import json import shutil import tarfile import tempfile @@ -214,5 +215,197 @@ def test_safe_extract_rejects_traversal_and_links(self): with self.assertRaises(ValueError): updates.safe_extract(archive, target) + # ---- one-shot backup choices (backup drive not mounted) + + def _run_choice(self, backup, mount="not_mounted", env_text="", supports=True, fail_on=None): + """Runs run_update with a fake control binary; returns (operation, calls, error, mocks).""" + directory = tempfile.mkdtemp(); self.addCleanup(shutil.rmtree, directory, True) + root = Path(directory); archive, checksum, digest = self._release_fixture(directory) + config = root / "vesselstack.env"; config.write_text(env_text) + snapshot = {"id": 9, "tag": "v0.6.0", "version": "0.6.0", "prerelease": False, "updated_at": "u", + "assets": {"archive": {"id": 1, "name": archive.name, "size": archive.stat().st_size, "updated_at": "a", "digest": f"sha256:{digest}", "url": "https://github.com/a"}, + "checksum": {"id": 2, "name": checksum.name, "size": checksum.stat().st_size, "updated_at": "a", "digest": None, "url": "https://github.com/c"}}} + request = {"release_id": 9, "channel": "lab", "config": str(config), "old_boot_id": "old", "release_snapshot": snapshot} + if backup is not None: + request["backup"] = backup + operation = {"id": 1, "state": "running", "phase": "queued", "backup_path": None} + if isinstance(backup, dict): + operation["backup_mode"] = backup.get("mode") # as start_detached records it + calls = [] + def download(url, dest, **kwargs): shutil.copy2(archive if dest.suffix == ".gz" else checksum, dest) + def run(argv, root_arg=None, op=None, *args, **kw): + calls.append(list(argv)) + if isinstance(op, dict): + op["output"] = ["output of " + Path(argv[0]).name] # like the real runner, each command replaces the log + if fail_on and argv[:len(fail_on)] == fail_on: + raise RuntimeError("step failed") + return mock.Mock(stdout="/backup/archive.tar.gz\n") + error = None + with mock.patch.object(updates, "release_snapshot", side_effect=[snapshot, snapshot]), mock.patch.object(updates, "bounded_download", side_effect=download) as downloaded, \ + mock.patch.object(updates, "verify_restart", return_value={"boot_id": "new", "running_version": "0.6.0"}), mock.patch.object(updates, "_validate_fixed_ctl"), \ + mock.patch.object(updates, "backup_destination", return_value=("/media/backup/backups", mount)), \ + mock.patch.object(updates, "ctl_supports", return_value=supports) as supported, mock.patch.object(updates, "run_command", side_effect=run): + try: + updates.run_update(request, operation, root) + except Exception as exc: # the caller inspects it + error = exc + return operation, calls, error, {"downloaded": downloaded, "supported": supported, "root": root} + + def test_default_run_never_probes_or_mounts(self): + operation, calls, error, mocks = self._run_choice(None) + self.assertIsNone(error) + self.assertEqual([c for c in calls if "backup" in c[1:2] or "backup-drive" in c], [[updates.FIXED_CTL, "backup", "--require-mounted"]]) + mocks["supported"].assert_not_called() + self.assertNotIn("backup_skipped", operation) + self.assertEqual(operation["backup_path"], "/backup/archive.tar.gz") + _, calls, error, _ = self._run_choice({"mode": "require_mounted"}) + self.assertIsNone(error) + self.assertIn([updates.FIXED_CTL, "backup", "--require-mounted"], calls) + + def test_mount_modes_mount_after_preflight_and_before_backup(self): + for backup, mount_argv in (({"mode": "mount_fstab"}, [updates.FIXED_CTL, "backup-drive", "mount", "--fstab"]), + ({"mode": "mount_drive", "uuid": "AB12-CD34"}, [updates.FIXED_CTL, "backup-drive", "mount", "--uuid", "AB12-CD34"])): + with self.subTest(backup=backup): + operation, calls, error, mocks = self._run_choice(backup) + self.assertIsNone(error) + backup_call = [updates.FIXED_CTL, "backup", "--require-mounted"] + dry = next(i for i, c in enumerate(calls) if c[-1] == "--dry-run") + self.assertLess(dry, calls.index(mount_argv)) + self.assertLess(calls.index(mount_argv), calls.index(backup_call)) + self.assertLess(calls.index(backup_call), next(i for i, c in enumerate(calls) if c[-1] == "--start")) + mocks["supported"].assert_called_with("backup-drive") + self.assertEqual(operation["state"], "succeeded") + + def test_mount_failure_stops_before_backup_and_install(self): + operation, calls, error, _ = self._run_choice({"mode": "mount_drive", "uuid": "AB12-CD34"}, fail_on=[updates.FIXED_CTL, "backup-drive"]) + self.assertIsInstance(error, RuntimeError) + self.assertFalse(any(c[:2] == [updates.FIXED_CTL, "backup"] for c in calls)) + self.assertFalse(any(c[-1] == "--start" for c in calls)) + self.assertFalse(any(c[:2] == ["systemctl", "restart"] for c in calls)) + + def test_system_disk_once_uses_the_one_shot_backup_flag(self): + operation, calls, error, mocks = self._run_choice({"mode": "system_disk_once", "confirm": "system-disk-once"}) + self.assertIsNone(error) + self.assertIn([updates.FIXED_CTL, "backup", "--system-disk-once"], calls) + self.assertNotIn([updates.FIXED_CTL, "backup", "--require-mounted"], calls) + mocks["supported"].assert_called_with("--system-disk-once") + self.assertEqual(updates.read_operation(mocks["root"]).get("backup_path"), "/backup/archive.tar.gz") + + def test_skip_installs_without_calling_backup(self): + operation, calls, error, mocks = self._run_choice({"mode": "skip", "confirm": "install-without-backup"}) + self.assertIsNone(error) + self.assertFalse(any(c[:2] == [updates.FIXED_CTL, "backup"] for c in calls)) + self.assertTrue(any(c[-1] == "--start" for c in calls)) + self.assertIs(operation["backup_skipped"], True) + self.assertIsNone(operation["backup_path"]) + self.assertEqual(operation["state"], "succeeded") + self.assertEqual(operation["backup_mode"], "skip") + self.assertEqual(operation["output"][0], updates.BACKUP_SKIP_WARNING, "the warning survives the commands that replace the log") + self.assertEqual(operation["output"].count(updates.BACKUP_SKIP_WARNING), 1) + self.assertIn("backup skipped at the owner's request", updates.BACKUP_SKIP_WARNING) + + def test_skip_keeps_earlier_log_below_the_warning(self) -> None: + snapshots = [] + real_write = updates.write_operation + def spy(root_arg, op): + if op.get("phase") == "backing-up": + snapshots.append(list(op.get("output") or [])) + return real_write(root_arg, op) + with mock.patch.object(updates, "write_operation", side_effect=spy): + operation, calls, error, _ = self._run_choice({"mode": "skip", "confirm": "install-without-backup"}) + self.assertIsNone(error) + self.assertEqual(snapshots[-1][0], updates.BACKUP_SKIP_WARNING) + self.assertIn("output of install.sh", snapshots[-1][1:], "the dry-run log is kept below the warning") + + def test_worker_rechecks_the_lock_before_skip_or_system_disk(self) -> None: + for backup in ({"mode": "skip", "confirm": "install-without-backup"}, {"mode": "system_disk_once", "confirm": "system-disk-once"}): + with self.subTest(backup=backup): + original = updates.run_command + directory_holder = {} + def locking_run(argv, root_arg=None, op=None, *args, **kw): + if argv[-1] == "--dry-run": + (directory_holder["config"]).write_text("VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false\n") # locked after admission + return mock.Mock(stdout="/backup/archive.tar.gz\n") + directory = tempfile.mkdtemp(); self.addCleanup(shutil.rmtree, directory, True) + root = Path(directory); archive, checksum, digest = self._release_fixture(directory) + config = root / "vesselstack.env"; config.write_text(""); directory_holder["config"] = config + snapshot = {"id": 9, "tag": "v0.6.0", "version": "0.6.0", "prerelease": False, "updated_at": "u", + "assets": {"archive": {"id": 1, "name": archive.name, "size": archive.stat().st_size, "updated_at": "a", "digest": f"sha256:{digest}", "url": "https://github.com/a"}, + "checksum": {"id": 2, "name": checksum.name, "size": checksum.stat().st_size, "updated_at": "a", "digest": None, "url": "https://github.com/c"}}} + request = {"release_id": 9, "channel": "lab", "config": str(config), "old_boot_id": "old", "release_snapshot": snapshot, "backup": backup} + operation = {"id": 1, "state": "running", "phase": "queued", "backup_path": None} + seen = [] + def download(url, dest, **kwargs): shutil.copy2(archive if dest.suffix == ".gz" else checksum, dest) + def run(argv, *args, **kw): + seen.append(list(argv)); return locking_run(argv, *args, **kw) + with mock.patch.object(updates, "release_snapshot", side_effect=[snapshot, snapshot]), mock.patch.object(updates, "bounded_download", side_effect=download), \ + mock.patch.object(updates, "_validate_fixed_ctl"), mock.patch.object(updates, "backup_destination", return_value=("/media/backup/backups", "not_mounted")), \ + mock.patch.object(updates, "ctl_supports", return_value=True), mock.patch.object(updates, "run_command", side_effect=run): + with self.assertRaisesRegex(ValueError, "turned off"): + updates.run_update(request, operation, root) + self.assertFalse(any(c[-1] == "--start" or c[:2] == [updates.FIXED_CTL, "backup"] for c in seen)) + + def test_worker_refuses_invalid_or_locked_choices_before_downloading(self): + for backup, env_text in (({"mode": "format_disk"}, ""), ({"mode": "mount_drive", "uuid": "../../dev/sda1"}, ""), ({"mode": "mount_drive"}, ""), + ({"mode": "skip"}, ""), ({"mode": "skip", "confirm": "yes"}, ""), ({"mode": "mount_fstab", "device": "/dev/sda1"}, ""), + ({"mode": "skip", "confirm": "install-without-backup"}, "VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false\n"), + ({"mode": "system_disk_once", "confirm": "system-disk-once"}, "export VESSELSTACK_BACKUP_ALLOW_UNMOUNTED='false'\n"), + ("skip", ""), ([], "")): + with self.subTest(backup=backup, env=env_text): + operation, calls, error, mocks = self._run_choice(backup, env_text=env_text) + self.assertIsInstance(error, ValueError) + self.assertEqual(calls, []) + mocks["downloaded"].assert_not_called() + + def test_choice_falls_back_to_the_default_when_the_drive_is_mounted_now(self): + for backup in ({"mode": "mount_fstab"}, {"mode": "system_disk_once", "confirm": "system-disk-once"}, {"mode": "skip", "confirm": "install-without-backup"}): + with self.subTest(backup=backup): + operation, calls, error, _ = self._run_choice(backup, mount="mounted") + self.assertIsNone(error) + self.assertIn([updates.FIXED_CTL, "backup", "--require-mounted"], calls) + self.assertFalse(any("backup-drive" in c or "--system-disk-once" in c for c in calls)) + self.assertNotIn("backup_skipped", operation) + self.assertNotIn("backup_mode", operation, "the record never claims a handling that was not used") + + def test_older_control_binary_is_reported_before_anything_changes(self): + for backup in ({"mode": "mount_fstab"}, {"mode": "system_disk_once", "confirm": "system-disk-once"}): + with self.subTest(backup=backup): + operation, calls, error, _ = self._run_choice(backup, supports=False) + self.assertIsInstance(error, ValueError) + self.assertIn("older than this Control Panel", str(error)) + self.assertFalse(any(c[:2] == [updates.FIXED_CTL, "backup"] or c[-1] == "--start" for c in calls)) + + def test_normalize_backup_choice(self): + normalize = updates.normalize_backup_choice + self.assertIsNone(normalize(None, False)) + self.assertIsNone(normalize({"mode": "require_mounted"}, True)) + self.assertEqual(normalize({"mode": "mount_drive", "uuid": "AB12-CD34"}, True), {"mode": "mount_drive", "uuid": "AB12-CD34"}) + self.assertEqual(normalize({"mode": "skip", "confirm": "install-without-backup"}, False)["mode"], "skip") + for bad in ({"mode": "require_mounted", "uuid": "x"}, {"mode": "mount_fstab", "confirm": "x"}, {"mode": "mount_drive", "uuid": "a" * 65}, {"mode": "mount_drive", "uuid": 5}, {}, {"mode": None}): + with self.subTest(bad=bad), self.assertRaises(ValueError): + normalize(bad, False) + + def test_backup_lock_matches_the_shell_rule(self): + with tempfile.TemporaryDirectory() as directory: + config = Path(directory) / "vesselstack.env" + for text, expected in (("", False), ("VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=true\n", False), ("VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false\n", True), + ("export VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=\"false\"\n", True), ("# VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false\n", False)): + config.write_text(text) + self.assertIs(updates.backup_locked(config), expected, text) + self.assertIs(updates.backup_locked(Path(directory) / "missing.env"), False) + + def test_start_detached_stores_the_choice_only_when_given(self): + for backup in (None, {"mode": "mount_fstab"}): + with self.subTest(backup=backup), tempfile.TemporaryDirectory() as directory: + root = Path(directory); snapshot = {"version": "0.6.0"} + kwargs = {"backup": backup} if backup else {} + with mock.patch.object(updates, "release_snapshot", return_value=snapshot), mock.patch.object(updates, "assert_snapshot_unchanged"), mock.patch.object(updates.subprocess, "run"): + operation = updates.start_detached("update-release", root, {"boot_id": "old", "running_version": "0.5.3"}, 9, "lab", snapshot=snapshot, **kwargs) + request = json.loads(next((root / "var/lib/vesselstack").glob("browser-update-*/request.json")).read_text()) + self.assertEqual(request.get("backup"), backup) + self.assertEqual(operation.get("backup_mode"), backup["mode"] if backup else None) + self.assertEqual("backup" in request, backup is not None) + + if __name__ == "__main__": unittest.main() diff --git a/tests/test_control_panel.py b/tests/test_control_panel.py index 14f85ca..f917c05 100644 --- a/tests/test_control_panel.py +++ b/tests/test_control_panel.py @@ -3,6 +3,7 @@ from __future__ import annotations import importlib.util +import json import os import tempfile import time @@ -727,5 +728,152 @@ def test_restart_note_only_when_a_newer_panel_is_installed(self) -> None: self.assertFalse(panel.restart_needed()) + # ---- backup preflight and one-shot backup choices + + def _preflight(self, mount, listing=None, env_text="", code=0): + with tempfile.TemporaryDirectory() as directory: + env = Path(directory) / "vesselstack.env" + env.write_text(env_text) + status = {"path": "/media/backup-drive/backups", "mount": mount} + output = (listing if isinstance(listing, str) else json.dumps(listing)) if listing is not None else "" + with mock.patch.object(panel, "VESSEL_ENV", env), mock.patch.object(panel, "backup_status", return_value=status), \ + mock.patch.object(panel, "command_stdout", return_value=(code, output)) as listed: + return panel.backup_preflight(), listed + + def test_preflight_mounted_backup_is_the_default_and_lists_nothing(self) -> None: + result, listed = self._preflight("mounted") + self.assertTrue(result["default_ok"]) + self.assertEqual((result["drives"], result["refused"], result["mounted_elsewhere"]), ([], [], [])) + self.assertFalse(result["system_disk_once"]["offered"]) + listed.assert_not_called() + + def test_preflight_unmounted_backup_lists_root_side_drives(self) -> None: + listing = {"fstab_mount": True, + "drives": [{"uuid": "AB12-CD34", "label": "BACKUP", "size": "58G", "fstype": "ext4", "device": "/dev/sda1"}, + {"uuid": "../bad", "label": "X", "size": "1G", "fstype": "ext4", "device": "/dev/sdz1"}], + "refused": [{"device": "/dev/sdb1", "label": "CAMERA", "size": "29G", "fstype": "vfat", "reason": "vfat is not supported for backups"}], + "mounted_elsewhere": [{"device": "/dev/sde1", "label": "", "size": "16G", "fstype": "ext4", "mountpoint": "/media/pi/USBSTICK"}]} + result, listed = self._preflight("not_mounted", listing) + self.assertEqual(listed.call_args.args[0][1:], ["backup-drive", "list", "--json"]) + self.assertFalse(result["default_ok"]) + self.assertTrue(result["fstab_mount"]) + self.assertEqual([d["uuid"] for d in result["drives"]], ["AB12-CD34"]) + self.assertEqual(set(result["drives"][0]), {"uuid", "device", "label", "size", "fstype"}) + self.assertIn("not supported", result["refused"][0]["reason"]) + self.assertEqual(result["mounted_elsewhere"][0]["mountpoint"], "/media/pi/USBSTICK") + self.assertEqual(result["system_disk_once"], {"offered": True, "reason": "", "target": panel.DEFAULT_BACKUP_DIR}) + self.assertEqual(result["skip"], {"offered": True}) + self.assertEqual(set(result), {"path", "mount", "default_ok", "fstab_mount", "drives", "refused", "mounted_elsewhere", "system_disk_once", "skip"}) + + def test_preflight_degrades_to_no_drives_when_listing_fails(self) -> None: + for kwargs in ({"code": 1, "listing": ""}, {"listing": "not json"}, {"listing": "[1, 2]"}): + with self.subTest(kwargs=kwargs): + result, _ = self._preflight("not_mounted", **kwargs) + self.assertEqual((result["drives"], result["refused"], result["fstab_mount"]), ([], [], False)) + self.assertTrue(result["system_disk_once"]["offered"]) + + def test_preflight_lock_turns_off_system_disk_and_skip(self) -> None: + result, _ = self._preflight("not_mounted", {"drives": []}, env_text="VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false\n") + self.assertFalse(result["system_disk_once"]["offered"]) + self.assertIn("VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false", result["system_disk_once"]["reason"]) + self.assertEqual(result["skip"], {"offered": False}) + + def test_command_stdout_drops_stderr(self) -> None: + with mock.patch.object(panel.subprocess, "run", return_value=mock.Mock(returncode=0, stdout=" {} \n")) as run: + self.assertEqual(panel.command_stdout(["ctl"]), (0, "{}")) + self.assertEqual(run.call_args.kwargs["stderr"], panel.subprocess.DEVNULL) + + def _folder_fixture(self, directory): + reviewed = Path(directory) / "releases" + release = reviewed / "vesselstack-0.9.0" + release.mkdir(parents=True) + (release / "install.sh").write_text("#!/bin/sh\n"); (release / "VERSION").write_text("0.9.0\n") + return reviewed, release + + def test_update_from_folder_default_argv_is_unchanged_and_choices_append_after_the_folder(self) -> None: + with tempfile.TemporaryDirectory() as directory: + reviewed, release = self._folder_fixture(directory) + with mock.patch.object(panel, "REVIEWED_RELEASE_ROOT", reviewed), mock.patch.object(panel, "verify_release_path"), mock.patch.object(panel, "verify_release_tree"): + default = panel.action_command("update", {"source_directory": str(release)}) + self.assertEqual(default, [str(panel.CTL), "update", str(release.resolve())]) + for mode, flag in (("system_disk_once", "--backup-choice=system-disk-once"), ("skip", "--backup-choice=skip")): + command = panel.action_command("update", {"source_directory": str(release)}, {"mode": mode}) + self.assertEqual(command, [str(panel.CTL), "update", str(release.resolve()), flag]) + # Mount choices change nothing in the update command; the mount is a separate step. + self.assertEqual(panel.action_command("update", {"source_directory": str(release)}, {"mode": "mount_fstab"}), default) + # A browser-supplied field can never select a choice. + forged = panel.action_command("update", {"source_directory": str(release), "_backup_choice": {"mode": "skip"}, "backup_choice": {"mode": "skip"}}) + self.assertEqual(forged, default) + + def test_folder_update_mounts_in_a_transient_unit_outside_the_panel_sandbox(self) -> None: + self.assertEqual(panel.folder_update_steps(None), []) + self.assertEqual(panel.folder_update_steps({"mode": "skip", "confirm": "install-without-backup"}), []) + fstab = panel.folder_update_steps({"mode": "mount_fstab"})[0] + self.assertEqual(fstab[:5], ["systemd-run", "--wait", "--pipe", "--collect", "--quiet"]) + self.assertIn("--property=RuntimeMaxSec=300", fstab) + self.assertEqual(fstab[-4:], [str(panel.CTL), "backup-drive", "mount", "--fstab"]) + drive = panel.folder_update_steps({"mode": "mount_drive", "uuid": "AB12-CD34"})[0] + self.assertEqual(drive[-5:], [str(panel.CTL), "backup-drive", "mount", "--uuid", "AB12-CD34"]) + + def test_process_environment_lock_applies_to_choice_and_preflight(self) -> None: + with tempfile.TemporaryDirectory() as directory: + env = Path(directory) / "vesselstack.env" + env.write_text("") + skip = {"backup": {"mode": "skip", "confirm": "install-without-backup"}} + with mock.patch.object(panel, "VESSEL_ENV", env), mock.patch.dict(os.environ, {"VESSELSTACK_BACKUP_ALLOW_UNMOUNTED": "false"}): + with self.assertRaisesRegex(ValueError, "turned off"): + panel.request_backup_choice(skip) + result, _ = self._preflight("not_mounted", {"drives": []}) + self.assertFalse(result["skip"]["offered"]) + self.assertFalse(result["system_disk_once"]["offered"]) + + def test_request_backup_choice_validates_locks_and_falls_back(self) -> None: + with tempfile.TemporaryDirectory() as directory: + env = Path(directory) / "vesselstack.env" + env.write_text("") + skip = {"backup": {"mode": "skip", "confirm": "install-without-backup"}} + with mock.patch.object(panel, "VESSEL_ENV", env), mock.patch.object(panel, "effective_backup_choice", side_effect=lambda c, _: (c, "")): + self.assertIsNone(panel.request_backup_choice({})) + self.assertIsNone(panel.request_backup_choice({"backup": {"mode": "require_mounted"}})) + self.assertEqual(panel.request_backup_choice(skip)["mode"], "skip") + with self.assertRaises(ValueError): + panel.request_backup_choice({"backup": {"mode": "skip"}}) + with self.assertRaises(ValueError): + panel.request_backup_choice({"backup": {"mode": "mount_drive", "uuid": "/dev/sda1"}}) + env.write_text("VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false\n") + with self.assertRaisesRegex(ValueError, "turned off"): + panel.request_backup_choice(skip) + env.write_text("") + with mock.patch.object(panel, "ctl_supports", return_value=False): + with self.assertRaisesRegex(ValueError, "older than this Control Panel"): + panel.request_backup_choice({"backup": {"mode": "mount_fstab"}}, folder=True) + self.assertEqual(panel.request_backup_choice(skip)["mode"], "skip") # the release worker checks for itself + with mock.patch.object(panel, "VESSEL_ENV", env), mock.patch.object(panel.os.path, "ismount", return_value=True): + env.write_text("VESSELSTACK_BACKUP=/media/backup-drive/backups\n") + self.assertIsNone(panel.request_backup_choice({"backup": {"mode": "mount_fstab"}})) + + def test_operation_store_runs_steps_first_and_stops_on_failure(self) -> None: + ran: list = [] + + class FakeProcess: + def __init__(self, code): + self.stdout = iter(["line\n"]); self.code = code + def wait(self): + return self.code + + def popen(command, **kwargs): + ran.append(command) + return FakeProcess(1 if command[0] == "mount-step" else 0) + + for failing, expected in ((False, [["ok-step"], ["main"]]), (True, [["mount-step"]])): + ran.clear() + store = panel.OperationStore() + operation = {"label": "update", "state": "running", "output": panel.deque(maxlen=10), "exit_code": None, "finished_at": None} + with mock.patch.object(panel.subprocess, "Popen", side_effect=popen), mock.patch.object(panel, "release_lifecycle_slot"): + store._run(operation, ["main"], None, [["mount-step"] if failing else ["ok-step"]]) + self.assertEqual(ran, expected) + self.assertEqual(operation["state"], "failed" if failing else "succeeded") + + if __name__ == "__main__": unittest.main() diff --git a/tests/test_control_panel_ui.cjs b/tests/test_control_panel_ui.cjs index c405408..94eb3e8 100644 --- a/tests/test_control_panel_ui.cjs +++ b/tests/test_control_panel_ui.cjs @@ -5,7 +5,7 @@ const vm = require('node:vm'); const appSource = fs.readFileSync('control-panel/static/app.js', 'utf8'); const html = fs.readFileSync('control-panel/static/index.html', 'utf8'); const css = fs.readFileSync('control-panel/static/styles.css', 'utf8'); -const source = appSource + '\nthis.__ui = { state, normalizeToken, isConfigDirty, dirtyKeys, currentSettings, lock, setBusy, renderComponents, checkReleases, startOperation, renderOperation, operationSucceeded, refreshStatus, renderReleases, runComponent, runAction, installRelease, discardEdits, toast, fieldMarkup, optionLabel, updateConfigStatus, renderBand, renderUpdates, requireReauth, CONFIRMS, describeOperation, setNotice, showFieldErrors, renderConfiguration, selectTab };'; +const source = appSource + '\nthis.__ui = { state, normalizeToken, isConfigDirty, dirtyKeys, currentSettings, lock, setBusy, renderComponents, checkReleases, startOperation, renderOperation, operationSucceeded, refreshStatus, renderReleases, runComponent, runAction, installRelease, discardEdits, toast, fieldMarkup, optionLabel, updateConfigStatus, renderBand, renderUpdates, requireReauth, CONFIRMS, describeOperation, askBackupChoice, confirmWithBackup, backupOptions, setNotice, showFieldErrors, renderConfiguration, selectTab };'; const nodes = new Map(); const makeNode = () => ({ @@ -29,6 +29,15 @@ dialog.showModal = function showModal() { if (this.onclose) this.onclose(); }; dialog.close = function close(value) { this.returnValue = value; }; +const backupDialog = node('#backup-choice-dialog'); +const backupShown = []; +let backupDriver = null; +backupDialog.showModal = function showModal() { + this.open = true; this.returnValue = ''; + backupShown.push({ title: node('#backup-choice-title').textContent, intro: node('#backup-choice-intro').textContent, options: node('#backup-choice-options').innerHTML, notes: node('#backup-choice-notes').innerHTML, ok: node('#backup-choice-ok').textContent, okClass: node('#backup-choice-ok').className }); + setImmediate(async () => { if (backupDriver) await backupDriver(); else backupDialog.close('cancel'); }); +}; +backupDialog.close = function close(value) { this.returnValue = value; this.open = false; if (this.onclose) this.onclose(); }; let fetchImpl = async () => ({ ok: true, json: async () => ({ configuration: context.__ui.state.config }) }); const context = { console, setTimeout: () => 0, clearTimeout() {}, @@ -59,6 +68,9 @@ assert.match(html, /Reinstall & start/); assert.match(html, /Check setup/); ['Home', 'Settings', 'Updates', 'Maintenance'].forEach((name) => assert.match(html, new RegExp(`role="tab"[^>]*>${name}`))); assert.equal((html.match(/role="tab"/g) || []).length, 4); assert.match(html, / ui.installRelease(9)); assert.equal(install[0].title, 'Back up & install 0.6.1?'); assert.equal(posts.length, 0); + // ---- backup choice: the normal dialog when the backup works, a choice dialog when it does not ---------- + { + const pre = (extra = {}) => ({ path: '/media/backup-drive/backups', mount: 'not_mounted', default_ok: false, fstab_mount: false, drives: [], refused: [], mounted_elsewhere: [], + system_disk_once: { offered: true, reason: '', target: '/var/backups/vesselstack' }, skip: { offered: true }, ...extra }); + const drive = { uuid: 'AB12-CD34', label: 'BACKUP ', size: '58G', fstype: 'ext4', device: '/dev/sda1' }; + const originalFetch = fetchImpl; + let preflights = []; + const configPosts = []; + const useBackupFetch = () => { + fetchImpl = async (path, options = {}) => { + if (path === '/api/backup-preflight') { if (preflights.length === 0) throw new Error('offline'); const next = preflights.length > 1 ? preflights.shift() : preflights[0]; if (next instanceof Error) throw next; return ok(next); } + if (path === '/api/config' && options.method === 'POST') { configPosts.push(JSON.parse(options.body)); return ok({ configuration: ui.state.config }); } + if (options.method === 'POST') { posts.push({ path, body: JSON.parse(options.body) }); return ok({ id: 1, label: 'x', state: 'running', output: [] }); } + return ok({ components: [], operation: { state: 'idle' }, installed_version: '0.6.0', running_version: '0.6.0' }); + }; + }; + const reset = () => { ui.state.busy = false; ui.state.pendingOperation = null; posts.length = 0; backupShown.length = 0; dialogTitles.length = 0; configPosts.length = 0; node('#backup-choice-ack').checked = false; node('#backup-choice-path').value = ''; }; + const pick = (value) => node('#backup-choice-options').onchange({ target: { value } }); + useBackupFetch(); + + // The backup works: today's confirmation, nothing else, and no backup field in the request. + reset(); preflights = [pre({ default_ok: true, mount: 'mounted' })]; dialogAnswer = 'ok'; + await ui.installRelease(9); await tick(); + assert.equal(backupShown.length, 0, 'a working backup never shows the choice dialog'); + assert.equal(dialogTitles[0].title, 'Back up & install 0.6.1?'); + assert.deepEqual(posts[0].body, { action: 'update-release', release_id: 9, channel: ui.state.releaseChannel }, 'the default request is unchanged'); + // A failed lookup falls back to the same confirmation (the server still requires a mounted backup). + reset(); preflights = []; + await ui.installRelease(9); await tick(); + assert.equal(backupShown.length, 0); assert.equal(dialogTitles[0].title, 'Back up & install 0.6.1?'); + assert.equal(Object.hasOwn(posts[0].body, 'backup'), false); + + // Not mounted: the choice dialog lists drives, refused and already-mounted ones, and escapes names. + const listing = pre({ fstab_mount: true, drives: [drive], + refused: [{ device: '/dev/sdb1', label: 'CAMERA', size: '29G', fstype: 'vfat', reason: 'vfat is not supported for backups' }], + mounted_elsewhere: [{ device: '/dev/sde1', label: '', size: '16G', fstype: 'ext4', mountpoint: '/media/pi/USBSTICK' }] }); + reset(); preflights = [listing]; backupDriver = async () => { backupDialog.close('cancel'); }; + await ui.installRelease(9); await tick(); + assert.equal(dialogTitles.length, 0, 'the usual confirmation is replaced by the choice dialog'); assert.equal(backupShown.length, 1); assert.equal(posts.length, 0, 'cancel starts nothing'); + const shown = backupShown[0]; + assert.equal(shown.title, 'Back up & install 0.6.1?'); assert.match(shown.intro, /not mounted \(\/media\/backup-drive\)/); + assert.match(shown.options, /Mount the backup drive now/); assert.match(shown.options, /Mount "BACKUP <b>" \(58G, ext4\)/); assert.doesNotMatch(shown.options, //, 'drive names are escaped'); + assert.match(shown.options, /check again/); assert.match(shown.options, /system disk this once \(not recommended\)/); assert.match(shown.options, /different backup folder/); assert.match(shown.options, /Install without a backup/); + assert.match(shown.options, /value="fstab" checked/, 'the first safe choice is preselected, never the skip'); + assert.match(shown.notes, /\/dev\/sdb1 29G "CAMERA" vfat: vfat is not supported/); assert.match(shown.notes, /already mounted at \/media\/pi\/USBSTICK/); + + // Each choice posts exactly its backup object. + const choose = async (value, action = 'update-release') => { + reset(); preflights = [listing]; + backupDriver = async () => { pick(value); await node('#backup-choice-ok').onclick(); }; + if (action === 'update') await ui.runAction('update', { source_directory: '/var/lib/vesselstack/releases/x' }); else await ui.installRelease(9); + await tick(); + return posts.at(-1)?.body; + }; + assert.deepEqual((await choose('fstab')).backup, { mode: 'mount_fstab' }); + assert.deepEqual((await choose('drive:AB12-CD34')).backup, { mode: 'mount_drive', uuid: 'AB12-CD34' }); + assert.deepEqual((await choose('system')).backup, { mode: 'system_disk_once', confirm: 'system-disk-once' }); + const folderUpdate = await choose('drive:AB12-CD34', 'update'); + assert.deepEqual(folderUpdate, { action: 'update', source_directory: '/var/lib/vesselstack/releases/x', backup: { mode: 'mount_drive', uuid: 'AB12-CD34' } }, 'update from a folder offers the same choices'); + + // Skipping needs the checkbox and uses the danger button. + reset(); preflights = [listing]; let blocked; + backupDriver = async () => { + pick('skip'); + assert.equal(node('#backup-choice-skip-ack').hidden, false); assert.equal(node('#backup-choice-ok').disabled, true, 'the button is off until the box is ticked'); + assert.equal(node('#backup-choice-ok').className, 'danger solid'); assert.equal(node('#backup-choice-ok').textContent, 'Install without a backup'); + assert.equal(node('#backup-choice-ack-text').textContent, 'I understand there is no backup to restore if this update fails'); + await node('#backup-choice-ok').onclick(); blocked = backupDialog.open; + node('#backup-choice-ack').checked = true; node('#backup-choice-ack').onchange(); + assert.equal(node('#backup-choice-ok').disabled, false); + await node('#backup-choice-ok').onclick(); + }; + await ui.installRelease(9); await tick(); + assert.equal(blocked, true, 'pressing the disabled button does nothing'); + assert.deepEqual(posts.at(-1).body.backup, { mode: 'skip', confirm: 'install-without-backup' }); + pick('fstab'); assert.equal(node('#backup-choice-skip-ack').hidden, true, 'the checkbox only shows for the skip choice'); + + // Turned off by the lock: neither the system disk nor the skip is offered, and the reason is shown. + reset(); preflights = [pre({ drives: [drive], system_disk_once: { offered: false, reason: 'Turned off: VESSELSTACK_BACKUP_ALLOW_UNMOUNTED=false is set in the configuration.', target: '' }, skip: { offered: false } })]; + backupDriver = async () => { backupDialog.close('cancel'); }; + await ui.installRelease(9); await tick(); + assert.doesNotMatch(backupShown[0].options, /system disk this once|Install without a backup/); assert.match(backupShown[0].notes, /ALLOW_UNMOUNTED=false/); + + // "Check again": a drive that turned up means the normal backup, so no backup field is sent. + reset(); preflights = [pre(), pre({ default_ok: true, mount: 'mounted' })]; + backupDriver = async () => { pick('recheck'); await node('#backup-choice-ok').onclick(); }; + await ui.installRelease(9); await tick(); + assert.equal(Object.hasOwn(posts.at(-1).body, 'backup'), false); + // ... and a drive that is still missing keeps the dialog open with a refreshed list. + reset(); preflights = [pre(), pre({ drives: [drive] })]; + backupDriver = async () => { pick('recheck'); await node('#backup-choice-ok').onclick(); assert.match(node('#backup-choice-options').innerHTML, /AB12-CD34/); backupDialog.close('cancel'); }; + await ui.installRelease(9); await tick(); + assert.equal(posts.length, 0); + + // A different folder is saved through the settings call, then the check runs again. + reset(); preflights = [pre(), pre({ default_ok: true, mount: 'mounted', path: '/media/other/backups' })]; + backupDriver = async () => { pick('folder'); assert.equal(node('#backup-choice-ok').disabled, true, 'an empty folder cannot be saved'); node('#backup-choice-path').value = '/media/other/backups'; node('#backup-choice-path').oninput(); await node('#backup-choice-ok').onclick(); }; + await ui.installRelease(9); await tick(); + assert.deepEqual(configPosts, [{ settings: { VESSELSTACK_BACKUP: '/media/other/backups' } }]); + assert.equal(Object.hasOwn(posts.at(-1).body, 'backup'), false); + backupDriver = null; dialogAnswer = 'cancel'; preflights = []; + fetchImpl = originalFetch; + reset(); + } // Edits are never discarded and the panel never locks without asking. formControls[0].value = 'Edited'; ui.state.config = { fields: [{ key: 'BOAT_NAME', kind: 'text', read_only: false }], values: { BOAT_NAME: 'Original' } }; assert.equal((await askedFor(() => ui.discardEdits()))[0].title, 'Discard your edits?'); @@ -236,6 +352,12 @@ assert.equal(node('#sb-apply').disabled, false, 'Apply is available once nothing assert.equal(ui.operationSucceeded({ label: 'backup', state: 'succeeded', output: [] }), true); assert.equal(ui.operationSucceeded({ label: 'restart grafana', state: 'succeeded' }), true); + assert.match(ui.describeOperation({ label: 'update-release', target_version: '0.6.2', backup_skipped: true }).running, /without a backup/); + assert.match(ui.describeOperation({ label: 'update-release', target_version: '0.6.2', backup_mode: 'skip' }).done, /no backup was made/); + assert.doesNotMatch(JSON.stringify(ui.describeOperation({ label: 'update-release', target_version: '0.6.2', backup_skipped: true })), /backed up first/, 'a skipped backup is never described as made'); + assert.match(ui.describeOperation({ label: 'update', backup_skipped: true }).running, /without a backup/); + assert.match(ui.describeOperation({ label: 'update-release', target_version: '0.6.2' }).running, /Backing up, then installing 0\.6\.2/); + // ---- field markup ---------------------------------------------------------- assert.equal(ui.optionLabel('BOAT_UNITS', 'us_customary'), 'US customary'); assert.equal(ui.optionLabel('BOAT_CHAT_FALLBACK_PROVIDER', ''), 'None'); diff --git a/tests/test_signalk_vesseltwin.py b/tests/test_signalk_vesseltwin.py index aff1ca8..4840b9b 100644 --- a/tests/test_signalk_vesseltwin.py +++ b/tests/test_signalk_vesseltwin.py @@ -191,7 +191,7 @@ def test_publish_creates_package_json_dependency(self): self.publish(raw, root) data = json.loads((root / "signalk/package.json").read_text()) self.assertEqual(data["dependencies"], {"signalk-vesseltwin": "0.2.3"}) - self.assertEqual(os.listdir(root / "signalk"), ["node_modules", "package.json"]) + self.assertEqual(sorted(os.listdir(root / "signalk")), ["node_modules", "package.json"]) def test_publish_preserves_package_json_keys_and_is_idempotent(self): raw = valid_archive()