Restructure documentation and add a first-backup quick start

docs/README.md

@@ -0,0 +1,28 @@
+# BackupDock Documentation
+
+[Main README](../README.md)
+
+Start with the quick-start guide, then choose the pages that match the deployment.
+
+## Getting started
+
+- [Quick Start: create and verify the first Docker backup](quick-start.md)
+- [Installation and updates](installation.md)
+- [Configuration reference](configuration.md)
+
+## Backup operation
+
+- [Docker Compose backup behavior and discovery](backup-behavior.md)
+- [Remote Docker backups through a reverse SSH tunnel](remote-backup.md)
+- [Command-line reference](cli.md)
+- [Scheduling, logs, retention, and checks](scheduling.md)
+
+## Recovery and maintenance
+
+- [Restore and test Docker Compose backups](restore.md)
+- [Upgrade notes](upgrading.md)
+- [Security model](security.md)
+
+## Project development
+
+- [Development and tests](development.md)

docs/backup-behavior.md

@@ -0,0 +1,98 @@
+# Docker Compose Backup Behavior
+
+[Documentation index](README.md) · [Main README](../README.md)
+
+
+## Design goals
+
+- No assumptions about host directory layouts.
+- No mandatory Docker labels.
+- Automatic bind-mount and Docker-volume discovery.
+- Docker Compose projects are consistency groups.
+- Only containers in the project currently being backed up are stopped.
+- Within a Compose project, running services are stopped in reverse dependency order and restarted in dependency order.
+- Containers that were already stopped stay stopped.
+- Running state is restored even when Restic or a stop operation fails.
+- Docker's own stop timeout and stop signal configuration are respected; BackupDock does not override the stop timeout.
+- Running `--rm`/AutoRemove containers are detected before any stop operation and fail safely unless explicitly ignored.
+- Explicitly ignored containers cannot silently defeat consistency checks when they have writable access to selected project backup data.
+- Standalone containers are supported as one-container backup groups.
+- Shared/overlapping persistent storage across different groups fails safely instead of silently producing an inconsistent backup.
+- An optional preseed pass can warm a repository while containers are still running before the final consistent stopped-container snapshot.
+- Restic remains the backup engine and repository format.
+- Local backups remain the default; backup-server-initiated remote backups are optional.
+- Scheduling is left to systemd, cron, or another external scheduler.
+
+## How grouping works
+
+For containers created by Docker Compose, BackupDock uses Docker's Compose metadata, primarily `com.docker.compose.project`, to form a consistency group.
+
+```text
+Compose project A
+  stop currently-running containers in reverse dependency order
+  back up A's bind mounts, volumes and Compose metadata
+  restart exactly the previously-running containers in dependency order
+
+Compose project B
+  stop currently-running containers in reverse dependency order
+  back up B's data
+  restart exactly the previously-running containers in dependency order
+```
+
+Other Compose projects remain online while a project is being backed up.
+
+A container without Compose metadata becomes its own standalone backup group.
+
+For Compose projects, BackupDock reconstructs service dependencies from the `com.docker.compose.depends_on` metadata written by Docker Compose. It also adds resolvable same-project runtime relationships from Docker inspect data for `links`, `volumes_from`, and container network sharing. Services without dependency relationships are ordered deterministically by name. A dependency cycle aborts discovery instead of guessing an unsafe order. Dependency targets that have no container in the currently deployed project are ignored for ordering with a warning.
+
+The dependency graph only changes ordering. A service that was stopped before the backup is never started merely because another service depends on it.
+
+If writable persistent paths overlap across two different groups, BackupDock aborts discovery. It will never decide on its own to stop an unrelated Compose project. Shared read-only bind mounts are allowed.
+
+## What is discovered
+
+BackupDock reads the Docker daemon and discovers:
+
+- bind mounts (`Type=bind`)
+- Docker volumes (`Type=volume`)
+- running/stopped container state
+- container AutoRemove (`--rm`) state
+- Docker Compose project and service names
+- Docker Compose service dependency metadata
+- Compose config-file metadata exposed by Docker Compose
+- the Compose working-directory `.env` file when present
+- service-level `env_file` inputs declared in the Compose file set for known, non-ignored services
+
+`tmpfs` and non-file bind sources such as sockets or FIFOs are ignored because they are not persistent backup data.
+
+There are **no built-in paths such as `/srv/files`, `/srv/docker-config`, `/opt/docker`, or `/var/lib/docker/volumes`**. Volume source paths come from Docker itself.
+
+BackupDock also inspects each named or anonymous volume's driver and options. Ordinary local volumes use their persistent host directory. Local bind-backed volumes (`o=bind` or `rbind`, `type` empty or `none`, and an absolute `device`) use the actual device directory, because Docker's volume mountpoint may be unmounted when the last container stops. The manifest preserves Docker's original mount source and records the resolved backup source. Mounted subdirectories are preserved. Exclusions by volume name remain unchanged; path exclusions recognize both the original and resolved path. Storage overlap and ignored-container writer checks use the resolved data path.
+
+Included volumes with uninspectable metadata, plugin drivers, or unsupported local mount options such as NFS, CIFS, or block-device filesystems abort discovery before any container stop. Explicitly excluded volumes are skipped; an ignored running container with an unresolved writable volume also fails because its storage overlap cannot be verified. These volume types need a separate supported backup strategy rather than reading a potentially empty Docker mountpoint.
+
+Docker images and the container writable filesystem layer are not included. Compose configuration and backed-up persistent data support recreating containers, provided their images can be pulled or rebuilt. Files stored only in the container layer need a separate backup strategy.
+
+With `backup.include_compose_metadata: true`, service environment files are included alongside the Compose configuration. Relative paths are resolved from the Compose working directory. String/list declarations and long-form `path`/`required` entries are supported; a missing required file aborts discovery before containers are stopped, while missing optional files are skipped during backup. Explicit path exclusions still apply. Symlink metadata files retain their original link and also include the target file; the manifest records the mapping so test restore reads archived contents rather than following the link into a live host path.
+
+Variable-based `env_file` paths require Docker Compose V2 with JSON config output. BackupDock resolves those paths using the project's `.env` or the interpolation files recorded by Docker, without inheriting unrelated shell variables. Docker Compose is not required for discovering plain literal environment-file paths. Compose `include`/`extends` workflows still require separate consideration and are unsupported by the current test restore.
+
+## Safety behavior
+
+Before any container is stopped, BackupDock:
+
+1. discovers every container and persistent mount,
+2. groups containers into Compose/standalone consistency groups,
+3. validates Compose dependency ordering,
+4. checks for unsafe writable storage overlap between groups,
+5. rejects running AutoRemove containers that would have to be stopped,
+6. rejects writable overlap from running ignored containers into selected backup data,
+7. verifies the Restic repository is reachable.
+
+These checks happen before BackupDock starts changing container state. A safety failure therefore does not leave earlier projects stopped while a later unsafe project is discovered.
+
+For each group it records which containers are running. Running containers are stopped in reverse dependency order and backed up inside a guarded transaction. BackupDock does not override Docker's stop timing; the stop request is sent without a timeout argument so Docker applies the container's configured stop behavior. Restart is attempted in dependency order even after a backup failure, interruption, or partial stop failure. Containers that were already stopped are never added to the restart set.
+
+Dry-run deliberately follows this same orchestration code path. `DockerBackend` and `ResticRunner` switch only their execution behavior: Docker mutations and Restic subprocess calls are rendered as `DRY-RUN ...` output. Manifest generation follows the same routine but prints the target manifest path instead of writing it. The normal process lock is still acquired so the plan is not produced concurrently with another BackupDock run.
+
+A manifest describing the group, containers, dependency metadata, AutoRemove state, mount destinations, volume names, and host source paths is stored under BackupDock's state directory and included in the Restic snapshot during a real backup. Test restore uses this archived metadata to map each service's mounts to restored data.

docs/cli.md

@@ -0,0 +1,172 @@
+# BackupDock Command-Line Reference
+
+[Documentation index](README.md) · [Main README](../README.md)
+
+
+Inspect the discovered groups, sources, dependencies, AutoRemove state, ignored containers, and excluded Docker volumes:
+
+```bash
+backupdock inventory
+```
+
+Run the normal backup workflow in dry-run mode:
+
+```bash
+backupdock backup --dry-run
+```
+
+Dry-run uses the same discovery, source validation, dependency ordering, Restic preflight routine, group loop, stop/backup/restart transaction, host-path backup routine, and retention path as a real backup. The difference is at the execution boundary: mutating Docker actions, Restic commands, and manifest writes are printed instead of executed.
+
+This makes dry-run useful for checking the actual command and action sequence without stopping containers or writing backup data.
+
+Preview only one Compose project:
+
+```bash
+backupdock backup --project paperless --dry-run
+```
+
+Back up all groups sequentially:
+
+```bash
+backupdock backup
+```
+
+Back up only one Compose project:
+
+```bash
+backupdock backup --project paperless
+```
+
+Start a configured remote backup from a backup server:
+
+```bash
+backupdock remote-backup docker-prod
+```
+
+## Preseed large first backups
+
+For a large first backup, `--preseed` can reduce the period during which containers must remain stopped:
+
+```bash
+backupdock backup --preseed
+```
+
+or in controller mode:
+
+```bash
+backupdock remote-backup docker-prod --preseed
+```
+
+For each stateful group BackupDock first runs a Restic backup while the group's containers are still running. This snapshot is tagged `backupdock-preseed` and must **not** be treated as the consistency-guaranteed backup because files may change while it is being created. If the preseed fails, BackupDock aborts before stopping that group's containers.
+
+After a successful preseed, BackupDock stops the group normally and creates the final snapshot tagged `backupdock`. Restic can then reuse already stored data through its normal incremental/deduplication mechanisms, often reducing the amount of new data that must be transferred during downtime. Changed large files may still need to be read again during the final pass, so preseed reduces expected downtime but cannot guarantee a specific maximum.
+
+On append-only remote repositories the source cannot delete the preseed snapshot. It remains clearly separated by its tag and can be removed later during controller-side repository maintenance.
+
+## Live backup progress
+
+When BackupDock's output is attached to a TTY, real Restic backup operations automatically use Restic's JSON status stream and render a single updating progress line, for example:
+
+```text
+[compose:nextcloud] backup   34.8%  18.2 GiB / 52.3 GiB  12,481 / 31,205 files  ETA 04:17
+```
+
+The same applies to a preseed pass, where the phase is shown as `preseed`.
+
+Remote/controller mode deliberately keeps SSH in non-PTY mode (`ssh -T`) because the controller payload and credentials are sent over standard input. If the controller's stdout is a TTY, it asks the source-side BackupDock process to enable the same JSON progress renderer. The resulting carriage-return status line is forwarded through the existing SSH connection without allocating a remote terminal.
+
+When stdout is not a TTY, for example under systemd or when redirected to a file, BackupDock keeps Restic's normal non-interactive output path and does not emit a continuously rewritten progress line.
+
+The byte counters in the live line represent source data processed by Restic, not bytes transferred over the network or final repository growth. Deduplication and compression can make the amount actually stored much smaller.
+
+Initialize the configured local Restic repository:
+
+```bash
+backupdock init
+```
+
+Initialize the repository belonging to one configured remote on the controller:
+
+```bash
+backupdock init --remote docker-prod
+```
+
+List the latest normal BackupDock snapshot per host and group in a compact table, without source paths or raw tags:
+
+```bash
+backupdock snapshots
+```
+
+The table shows group, host, retained snapshot count (`SNAPS`), container count (`CONTAINERS`), snapshot time, source-data size (`SIZE`), stored group data (`GROUP STORED`), and snapshot ID. A summary counts how many known host/group pairs have their latest snapshot dated today. Times and the meaning of `today` use the local timezone of the machine running the command. Older groups remain visible in the default listing, so a missing recent snapshot is easier to spot. Groups that have never produced a normal snapshot cannot be inferred from repository history alone.
+
+`SNAPS` counts all retained normal snapshots for that host/group. `GROUP STORED` uses `restic stats --mode raw-data` over their explicit snapshot IDs, counting stored blobs once across that group's history after deduplication and compression. Both columns cover the whole retained group history even with `--today` or `--all`; unrelated hosts, other groups, and preseeds are excluded. Shared blobs can be referenced by several groups, so group sizes cannot be added to obtain total repository usage and do not indicate how much space deleting a group would reclaim. Pack overhead, keys, indexes, and unreferenced repository data are not included.
+
+`CONTAINERS` comes from the manifest archived in the displayed snapshot, including containers that were stopped or explicitly ignored. It describes the backed-up group, not current Docker state. In the default listing it is the latest snapshot's count; with `--all`, each historical row uses its own manifest. Host-path snapshots show zero. Missing, unreadable, or unrecognized manifests show `unknown`. The command locates the original manifest path from snapshot metadata; if only a containing directory was backed up, it uses `backup.state_dir/manifests` from the current configuration.
+
+The compact view reads group statistics and archived manifests in addition to the snapshot list. Large histories may take longer to inspect; an immediate status message announces the additional reads. No source-host SSH or Docker access is used. `--details` and `--json` retain their original full-listing behavior and skip these additional reads.
+
+```bash
+backupdock snapshots --today         # Latest snapshots whose timestamps are dated today
+backupdock snapshots --all           # Compact history of all normal snapshots
+backupdock snapshots --all --today   # All normal snapshots dated today
+backupdock snapshots --details       # Original detailed Restic listing, including paths
+backupdock snapshots --json          # Raw Restic JSON for all normal snapshots
+backupdock snapshots --remote docker-prod
+```
+
+`--details` and `--json` are alternative full-listing formats and cannot be combined with `--all` or `--today`. Preseed snapshots are excluded from the compact listing. Sizes describe processed source data, not repository space or network transfer; snapshots without size statistics show `unknown`.
+
+The snapshot timestamp records the start of that backup. An existing normal snapshot does not prove that Restic read every file without errors, that container recovery succeeded, or that the complete scheduled backup and maintenance sequence finished. Use the run log to confirm those outcomes; listing snapshots does not perform `restic check`.
+
+Check repository integrity:
+
+```bash
+backupdock check
+```
+
+Apply the configured retention policy:
+
+```bash
+backupdock forget
+```
+
+Or forget and prune:
+
+```bash
+backupdock forget --prune
+```
+
+## Retention
+
+Retention is optional. Example:
+
+```yaml
+retention:
+  after_backup: false
+  prune: false
+  keep_daily: 14
+  keep_weekly: 8
+  keep_monthly: 12
+```
+
+Keeping `after_backup: false` avoids making every normal backup run perform repository maintenance. `backupdock forget` can be scheduled independently.
+
+`backupdock forget` first applies the configured retention policy to normal `backupdock` snapshots. It then removes a `backupdock-preseed` snapshot only if a remaining normal snapshot has the same hostname, BackupDock group tag, and source paths, and an equal or later timestamp. Preseeds without such a replacement remain available, including after an unsuccessful first consistent backup. Unrecognized or incomplete snapshot metadata is never used to authorize preseed removal. Preseeds do not participate in the normal retention selection.
+
+`backupdock forget --prune` runs pruning once, after both retention and preseed cleanup succeed. Without pruning, snapshot removal alone does not reclaim all unreferenced repository data.
+
+For `snapshots`, `check`, and `forget`, an explicit top-level `restic.repository` (including `RESTIC_REPOSITORY`) retains the existing local access behavior. Otherwise, a controller with exactly one remote automatically selects it. BackupDock reads `BACKUP_DIR` from `/etc/default/restic-rest-server`, appends that remote's `repository_path`, and uses the remote's repository `password_file`. The storage root and password do not need to be duplicated in the BackupDock configuration. Reads happen only when invoking these commands, so a missing service configuration does not affect `remote-backup` or inventory.
+
+With several remotes, select the repository explicitly:
+
+```bash
+backupdock snapshots --remote docker-prod
+backupdock check --remote docker-prod
+backupdock forget --remote docker-prod --prune
+```
+
+`--remote` explicitly selects the remote even if a top-level repository is configured. Selection applies to one repository per command; BackupDock does not silently clean every remote.
+
+If the packaged service uses another configuration file, set `restic.rest_server_config_file` to that file's path. Supported `BACKUP_DIR` assignments are literal absolute paths, optionally quoted and with whitespace or inline comments. The file is never executed or sourced. Missing, duplicate, relative, or shell-expanded values, a `--path` override, path traversal, a symlink escaping the root, or a missing repository `config` file cause an error before invoking Restic. Automatic discovery requires a loopback rest-server and does not inspect custom systemd command overrides or Docker mount mappings; configure an explicit local repository for those setups.
+
+For append-only remote backups, run retention locally on the backup server against the repository path rather than through `remote-backup`.