diff --git a/README.md b/README.md index 90e21c3..7631904 100644 --- a/README.md +++ b/README.md @@ -105,11 +105,27 @@ Branch [main]: Use SSH deploy key? [Y/n]: ``` +Before building, seploy checks free space on the Docker data root (from +`docker info`, falling back to `/var/lib/docker`); if less than 2 GB is +free, the deploy aborts before touching any container, with a hint to run +`docker system prune`. (If the check itself fails, the deploy proceeds.) + Then seploy clones the repo, builds the Docker image, detects the app port from the image's `EXPOSE`, starts the container on a random localhost port, writes the Caddy config and reloads Caddy. Done — Caddy fetches a TLS certificate and your app is live at `https://stats.example.com`. +As a last step, seploy fetches `https://` (or the app's healthcheck +path, if set) from the public internet to confirm it is actually reachable +— not just that the container passed its local health check on +127.0.0.1. It retries for up to 30 seconds, since Caddy may still be +fetching a TLS certificate on a first deploy. Success prints +`edge: https://stats.example.com reachable (HTTP )`; a failure prints +a warning with a hint (e.g. "the domain does not resolve yet", or "check: +systemctl status cloudflared" for a tunneled app whose cloudflared is down) +without failing the deploy, since the release itself is already live and +healthy. + ### Private repos (deploy keys) If you answer yes to the deploy key prompt (the default), seploy: @@ -141,6 +157,25 @@ prints a URL to open in your browser), creates a tunnel named `seploy`, and installs it as a service. All tunnel traffic goes to Caddy, which routes per app — so one tunnel serves every app on the server. +After setup, the Cloudflare credentials (`cert.pem` and the tunnel's +credentials file) live in `/etc/cloudflared` — the service and all seploy +tunnel operations use that copy, so they don't depend on the home directory +of whoever ran the login. + +`cloudflared` installs are pinned to a specific release and checksum-verified: +seploy downloads the exact version it's tested against and checks its SHA256 +against the digest Cloudflare published for that release before the binary +is ever installed — an unverified download is never installed. Get the +newest pinned version with: + +```bash +sudo seploy tunnel upgrade +``` + +This downloads and verifies the new version, atomically replaces the +installed binary, restarts the `cloudflared` service if the tunnel is +already configured, and prints the resulting `cloudflared --version` output. + After that, app setup gains one question: ```text @@ -148,7 +183,12 @@ Expose via Cloudflare Tunnel (DNS record created automatically)? [Y/n]: ``` Answer yes and seploy creates the Cloudflare DNS record itself. No A -records, no waiting for propagation. Check things with: +records, no waiting for propagation. If the domain already has a +conflicting DNS record — an A record, or a CNAME to a different tunnel — +seploy fails loudly and tells you what the domain currently resolves to +instead of silently leaving the app unreachable; a pre-existing CNAME that +already points at this server's tunnel (`.cfargotunnel.com`) is +left alone and treated as success. Check things with: ```bash seploy tunnel status @@ -157,6 +197,55 @@ seploy tunnel status Tunneled apps are served by Caddy as plain HTTP on localhost — TLS is terminated at Cloudflare's edge, and nothing is exposed publicly. +### Tunnel setup without a browser (token) + +`seploy tunnel setup` above needs an interactive login (it prints a URL +and waits for you to open it), which doesn't work well for scripted +provisioning or servers with no one watching the terminal. For those cases, +create the tunnel in the Cloudflare dashboard instead and hand seploy its +token: + +1. In the Cloudflare dashboard, go to **Zero Trust > Networks > Tunnels** + and create a tunnel (choose "Cloudflared" as the connector). +2. Copy the token shown on the install step — it's the long string after + `--token` in the install command Cloudflare suggests, not the whole + command. +3. Run, on the VPS: + + ```bash + sudo seploy tunnel setup --token + ``` + + (or set `SEPLOY_TUNNEL_TOKEN` in the environment instead of passing + `--token`). This downloads `cloudflared` and installs it as a service + running that token — no login, no `tunnel create`, no `config.yml`. + +This is a **remotely-managed** tunnel: unlike `seploy tunnel setup`, which +owns a local `config.yml` and lets seploy create DNS records and ingress +rules for you, a token-mode tunnel's routing lives entirely in the +dashboard. For every app you expose via the tunnel, add a **Public +Hostname** in **Zero Trust > Networks > Tunnels > \ > Public +Hostname**: + +```text +Domain: +Service: http://localhost:80 +``` + +That single step both creates the DNS record and routes the domain to +Caddy — no separate DNS entry is needed. seploy prints this same reminder +when you run `seploy ` for a tunneled app while in token mode, +since it cannot create this DNS record automatically (token-mode tunnels +have no local `cert.pem` for seploy to authenticate DNS API calls with). +`seploy tunnel status` and `seploy doctor` both report when the tunnel is +running in token mode. + +On OpenRC hosts, note that the token has to be embedded directly in +`/etc/init.d/cloudflared`'s `command_args` (OpenRC has no equivalent of +systemd's token-file handling); seploy writes that file with mode `0700` +so it's readable only by root, but be aware the token is there in plain +text. + ## Redeploy Push to your branch, then: @@ -176,6 +265,20 @@ Container logs are size-capped (10 MB × 3 files per container) and Docker's build cache is trimmed after each deploy, so long-running servers don't slowly fill their disk. +Add `--no-cache` to force a fresh build without Docker's build cache: + +```bash +sudo seploy deploy lets-talk-statistics --no-cache +``` + +Use `--all` to redeploy every configured app, one at a time, in sorted +order. It's mutually exclusive with a positional app name, continues past +per-app failures, and prints a summary at the end: + +```bash +sudo seploy deploy --all +``` + ## Deploy automatically on push (webhooks) ```bash @@ -210,6 +313,30 @@ This restarts the previous release, health checks it, switches Caddy traffic to it, then stops the broken one. Running rollback again switches back. +Older releases are kept too (see `keep_releases` under Configuration), so +you can roll back further than one step by targeting a specific commit: + +```bash +sudo seploy rollback lets-talk-statistics --to a1b2c3d +``` + +`--to` accepts a full commit SHA or any unambiguous prefix. It searches the +current, previous and history releases; if the SHA doesn't match anything, +or matches more than one release, the command lists the available commits +and when they were deployed. After a `--to` rollback, the release you left +becomes "previous" (so a plain `seploy rollback` switches straight back to +it), and history is left untouched. + +To see what's available before rolling back: + +```bash +seploy releases lets-talk-statistics +``` + +This lists the current, previous and history releases with their commit, +deploy time, and whether the release's image is still on disk (an image +that's missing can no longer be targeted with `rollback --to`). + ## Restart (apply env changes without rebuilding) ```bash @@ -218,7 +345,11 @@ sudo seploy restart lets-talk-statistics ``` `restart` recreates the current container from the already-built image -with the fresh env file — seconds instead of a full clone + build. +with the fresh env file — seconds instead of a full clone + build. It is +zero-downtime: the replacement container starts on a new port, is health +checked, and Caddy is switched over to it before the old container is +stopped. If the replacement fails its health check, the old container is +left running untouched and `restart` reports an error. ## Doctor @@ -229,8 +360,19 @@ sudo seploy doctor Checks the server (docker daemon, caddy, ports 80/443, disk space, public IP) and every app (container running, domain DNS actually pointing at this server, tunnel and webhook services healthy), and prints a fix hint for -anything that fails. New-app setup also runs a DNS preflight and warns if -the domain doesn't point at the server yet. +anything that fails. For tunneled apps it verifies the domain's CNAME +target is this server's tunnel (`.cfargotunnel.com`), not just +that the domain resolves — so it catches a stray A record or a CNAME to +someone else's tunnel; if the CNAME lookup itself fails it falls back to a +plain resolve check and notes that the CNAME target wasn't verified. +New-app setup also runs a DNS preflight and warns if the domain doesn't +point at the server yet. + +For every deployed app it also does the same public-edge reachability check +as `deploy` — fetching `https://` (or its healthcheck path) from +the internet — and reports it as a pass/fail line with a hint (DNS not +resolving, TLS handshake failing, connection refused, or an HTTP >= 500 +from Caddy/cloudflared/the app itself). ## Logs @@ -298,6 +440,12 @@ config, reloads Caddy, and removes `/var/lib/seploy/apps/`. The SSH deploy key in `/etc/seploy/keys/` is intentionally **kept** — delete it manually (and remove it from GitHub) if you no longer need it. +If the app was exposed through a Cloudflare Tunnel (`expose = "tunnel"`), +its DNS record is **not** removed — the `cloudflared` CLI has no command to +delete DNS records. `seploy remove` prints a reminder; delete the CNAME +record for the app's domain yourself in the Cloudflare dashboard +(DNS → Records) if it's no longer used. + ## App requirements - The repo must contain a `Dockerfile` at its root (or at the path set in @@ -308,7 +456,14 @@ manually (and remove it from GitHub) if you no longer need it. - The app must listen on `0.0.0.0` *inside* the container (containers are published only on the host's loopback, so this is safe). - Optional: set `healthcheck = "/healthz"` in `app.toml` to make deploys - wait (up to 30s) for a 2xx/3xx response before switching traffic. + wait (up to 30s by default, or `health_timeout` if set) for a 2xx/3xx + response before switching traffic. +- Optional: set `health_timeout = "90s"` (a Go duration like `"90s"` or + `"3m"`) in `app.toml` to change how long deploys wait for the health + check (default 30s) before failing the deploy. +- Optional: set `build_timeout = "10m"` (a Go duration like `"10m"` or + `"1h"`) in `app.toml` to change how long `docker build` is allowed to run + (default 30m) before the deploy fails. ## File layout @@ -337,7 +492,7 @@ config_dir = "/etc/seploy" caddy_conf_dir = "/etc/caddy/conf.d" caddyfile = "/etc/caddy/Caddyfile" default_branch = "main" -keep_releases = 3 +keep_releases = 3 # images of the last N releases; containers only for the current and previous release webhook_port = 8316 ``` @@ -352,6 +507,8 @@ domain = "stats.example.com" dockerfile = "Dockerfile" port = 3000 healthcheck = "" +health_timeout = "" # optional, Go duration like "90s" or "3m"; default 30s +build_timeout = "" # optional, Go duration like "10m" or "1h"; default 30m ssh_key = "/etc/seploy/keys/lets-talk-statistics_ed25519" expose = "" # "" = Caddy + DNS, "tunnel" = Cloudflare Tunnel memory = "" # optional, e.g. "512m" diff --git a/cmd/seploy/deploy.go b/cmd/seploy/deploy.go index da8d388..426215b 100644 --- a/cmd/seploy/deploy.go +++ b/cmd/seploy/deploy.go @@ -1,7 +1,9 @@ package main import ( + "errors" "fmt" + "net" "net/http" "os" "path/filepath" @@ -13,16 +15,32 @@ import ( "github.com/Telep-IO/seploy/internal/config" "github.com/Telep-IO/seploy/internal/docker" "github.com/Telep-IO/seploy/internal/git" + "github.com/Telep-IO/seploy/internal/netcheck" "github.com/Telep-IO/seploy/internal/system" "github.com/Telep-IO/seploy/internal/webhook" "github.com/spf13/cobra" ) +var deployWaitLock time.Duration +var deployNoCache bool +var deployAll bool + var deployCmd = &cobra.Command{ Use: "deploy ", Short: "Build and deploy the latest commit of an app", - Args: cobra.ExactArgs(1), + Args: func(cmd *cobra.Command, args []string) error { + if deployAll { + if len(args) != 0 { + return fmt.Errorf("cannot use --all together with an app name.") + } + return nil + } + return cobra.ExactArgs(1)(cmd, args) + }, RunE: func(cmd *cobra.Command, args []string) error { + if deployAll { + return runDeployAll() + } if err := system.RequireRoot("seploy deploy " + args[0]); err != nil { return err } @@ -39,13 +57,73 @@ var deployCmd = &cobra.Command{ } func init() { + deployCmd.Flags().DurationVar(&deployWaitLock, "wait-lock", 0, "wait up to this long for another deployment's lock instead of failing immediately") + deployCmd.Flags().MarkHidden("wait-lock") + deployCmd.Flags().BoolVar(&deployNoCache, "no-cache", false, "build the image without using the Docker build cache") + deployCmd.Flags().BoolVar(&deployAll, "all", false, "deploy every configured app, one at a time") rootCmd.AddCommand(deployCmd) } +// runDeployAll deploys every configured app sequentially in sorted order, +// continuing past per-app failures, and reports a summary at the end. +func runDeployAll() error { + if err := system.RequireRoot("seploy deploy --all"); err != nil { + return err + } + cfg, err := config.Load() + if err != nil { + return err + } + names, err := app.List(cfg) + if err != nil { + return err + } + if len(names) == 0 { + fmt.Println("No apps configured. Nothing to deploy.") + return nil + } + + var deployed []string + var failed []string + for _, name := range names { + fmt.Printf("=== %s ===\n", name) + a, err := app.Load(cfg, name) + if err != nil { + fmt.Println("Error:", err) + failed = append(failed, name) + continue + } + if err := runDeploy(cfg, a); err != nil { + fmt.Println("Error:", err) + failed = append(failed, name) + continue + } + deployed = append(deployed, name) + fmt.Println() + } + + fmt.Printf("deployed: %d, failed: %d", len(deployed), len(failed)) + if len(failed) > 0 { + fmt.Printf(" (%s)", strings.Join(failed, ", ")) + } + fmt.Println() + + if len(failed) > 0 { + return fmt.Errorf("%d app(s) failed to deploy: %s", len(failed), strings.Join(failed, ", ")) + } + return nil +} + // runDeploy is the full deployment pipeline: pull, build, start, health // check, switch Caddy, update state, retire the old release. func runDeploy(cfg config.Config, a *app.App) error { - lock, err := app.AcquireLock(cfg, a.Name) + var lock *app.Lock + var err error + if deployWaitLock > 0 { + lock, err = app.AcquireLockWait(cfg, a.Name, deployWaitLock) + } else { + lock, err = app.AcquireLock(cfg, a.Name) + } if err != nil { return err } @@ -81,9 +159,19 @@ func runDeploy(cfg config.Config, a *app.App) error { return fmt.Errorf("This repo must be containerized. Add a Dockerfile and try again.") } + if err := checkBuildDiskSpace(); err != nil { + return err + } + image := fmt.Sprintf("seploy/%s:%s", a.Name, sha) fmt.Printf("Building %s...\n", image) - if err := docker.Build(repoDir, dockerfile, image); err != nil { + if err := docker.Build(docker.BuildOpts{ + Dir: repoDir, + Dockerfile: dockerfile, + Tag: image, + Timeout: a.BuildTimeoutOrDefault(), + NoCache: deployNoCache, + }); err != nil { return err } @@ -108,6 +196,11 @@ func runDeploy(cfg config.Config, a *app.App) error { if err != nil { return err } + // Captured before st.Current is reassigned below, to distinguish a + // first-ever deploy (no release, and for a non-tunneled app no TLS + // certificate yet either) from a redeploy when the post-deploy edge + // check fails. + firstDeploy := st.Current == nil mounts, err := a.VolumeMounts() if err != nil { @@ -120,10 +213,27 @@ func runDeploy(cfg config.Config, a *app.App) error { // Redeploying the commit that is already live: keep it serving // while the replacement starts under a unique name, then switch. container = fmt.Sprintf("%s-%d", container, time.Now().Unix()) - } - // Clear any leftover container with this name from a failed deploy. - if err := docker.Remove(container); err != nil { - return err + } else if docker.Exists(container) && docker.IsRunning(container) { + // A container with this name is live even though state.json does + // not point at it (e.g. state was lost or reset). Do not kill a + // running container as "leftover from a failed deploy" -- start + // the replacement under a unique name instead, the same as the + // same-release case above. + conflicting := container + container = fmt.Sprintf("%s-%d", container, time.Now().Unix()) + // This container's name encodes the app and commit sha, so if + // state.json knew about it as the current release, sameRelease + // above would already be true. Reaching here means state.json + // does not know about it, so it will not be stopped by the + // old/pruneReleases path below -- warn so the operator can + // remove it manually. + fmt.Printf("Warning: container %s is running but not tracked in state.json; leaving it running.\n", conflicting) + fmt.Printf("Warning: once you've confirmed %s is serving correctly, remove it with: docker rm -f %s\n", container, conflicting) + } else { + // Clear any leftover container with this name from a failed deploy. + if err := docker.Remove(container); err != nil { + return err + } } fmt.Printf("Starting %s...\n", container) @@ -152,12 +262,11 @@ func runDeploy(cfg config.Config, a *app.App) error { return err } - if err := caddy.WriteSite(cfg.CaddyConfDir, a.Name, siteFor(cfg, a, hostPort)); err != nil { - return err - } - if err := caddy.Reload(cfg.Caddyfile); err != nil { - return err - } + // Snapshot the pre-switch state so it can be restored verbatim if the + // Caddy switch below fails. Current/Previous are about to be + // reassigned to new pointers and History to a new slice, so the + // fields saved here are never mutated in place. + preSwitch := &app.State{Current: st.Current, Previous: st.Previous, History: st.History} // On a same-commit redeploy the old container is still stopped below // and pruned, but Previous keeps pointing at the genuinely previous @@ -174,10 +283,45 @@ func runDeploy(cfg config.Config, a *app.App) error { ContainerPort: a.Port, DeployedAt: time.Now().UTC(), } + st.History = append([]*app.Release{st.Current}, st.History...) + if len(st.History) > cfg.KeepReleases { + st.History = st.History[:cfg.KeepReleases] + } + + // Persist the intended end state before switching traffic: the new + // container is already running and healthy, so if we crash before the + // Caddy switch below, a re-run of `seploy deploy` converges via the + // same-commit redeploy path, and `seploy status`/`doctor` point at a + // live container either way. if err := app.SaveState(cfg, a.Name, st); err != nil { + docker.Remove(container) return err } + if err := caddy.WriteSite(cfg.CaddyConfDir, a.Name, siteFor(cfg, a, hostPort)); err != nil { + if serr := app.SaveState(cfg, a.Name, preSwitch); serr != nil { + fmt.Println("Warning: failed to restore state.json after a failed Caddy site write:", serr) + } + if old != nil { + restoreSite(cfg, a, old.HostPort) + } + docker.Remove(container) + return trafficNotSwitchedErr(old, err) + } + if err := caddy.Reload(cfg.Caddyfile); err != nil { + if serr := app.SaveState(cfg, a.Name, preSwitch); serr != nil { + fmt.Println("Warning: failed to restore state.json after a failed Caddy reload:", serr) + } + // The on-disk site file now points at the replacement container, + // which is about to be removed. Best-effort restore it so a later + // reload does not pick up a site pointing at a dead port. + if old != nil { + restoreSite(cfg, a, old.HostPort) + } + docker.Remove(container) + return trafficNotSwitchedErr(old, err) + } + if old != nil { fmt.Printf("Stopping old container %s...\n", old.Container) if err := docker.Stop(old.Container); err != nil { @@ -185,7 +329,7 @@ func runDeploy(cfg config.Config, a *app.App) error { } } - pruneReleases(a.Name, st) + pruneReleases(a.Name, st, cfg) docker.PruneBuildCache() fmt.Println() @@ -199,9 +343,86 @@ func runDeploy(cfg config.Config, a *app.App) error { } else { fmt.Printf(" domain: https://%s\n", a.Domain) } + printEdgeCheck(a, firstDeploy) return nil } +// minBuildFreeSpace is the minimum free space required on the Docker data +// root before a build is attempted. +const minBuildFreeSpace = 2 << 30 // 2 GiB + +// checkBuildDiskSpace aborts the deploy before docker build runs if the +// Docker data root has less than minBuildFreeSpace free. This runs before +// any container is touched, so it is always safe to abort here. The check +// itself is best-effort: if the Docker root directory or its free space +// cannot be determined, the deploy proceeds silently. +func checkBuildDiskSpace() error { + root, err := docker.RootDir() + if err != nil { + return nil + } + free, err := system.DiskFree(root) + if err != nil { + return nil + } + if free < minBuildFreeSpace { + return fmt.Errorf("only %.1f GB free on the Docker data root (%s); need at least 2 GB to build. Try: docker system prune", float64(free)/(1<<30), root) + } + return nil +} + +// edgeCheckRetryFor and edgeCheckInterval bound how long printEdgeCheck +// retries a failing edge check before giving up: Caddy may still be +// fetching a TLS certificate for a first-time, non-tunneled deploy. +const ( + edgeCheckRetryFor = 30 * time.Second + edgeCheckInterval = 3 * time.Second +) + +// printEdgeCheck verifies the app's public URL is actually reachable after +// the deploy, and prints the result. It never fails the deploy: the release +// is already live and healthy locally, so an edge problem (DNS, TLS, +// cloudflared down, ...) is environmental, not a bad deploy. +func printEdgeCheck(a *app.App, firstDeploy bool) { + deadline := time.Now().Add(edgeCheckRetryFor) + var code int + var err error + for { + code, err = netcheck.CheckEdge(a.Domain, a.Healthcheck) + if err == nil || !time.Now().Before(deadline) { + break + } + time.Sleep(edgeCheckInterval) + } + + if err == nil { + fmt.Printf(" edge: https://%s reachable (HTTP %d)\n", a.Domain, code) + return + } + + hint := err.Error() + var edgeErr *netcheck.EdgeError + if errors.As(err, &edgeErr) { + hint = edgeErr.Hint(a.Tunneled()) + if edgeErr.Class == netcheck.EdgeClassTLS && !a.Tunneled() && firstDeploy { + hint = "Caddy may still be provisioning the TLS certificate; check again shortly." + } + } + fmt.Println() + fmt.Printf("WARNING: https://%s does not look reachable from the public internet.\n", a.Domain) + fmt.Printf(" %s\n", hint) +} + +// trafficNotSwitchedErr wraps a Caddy write/reload failure with the +// reminder that the old release, recorded back as Current in state.json, +// is the one still serving traffic. +func trafficNotSwitchedErr(old *app.Release, err error) error { + if old == nil { + return fmt.Errorf("traffic was not switched: %w", err) + } + return fmt.Errorf("traffic was not switched, the old release (commit %s, container %s) is still serving: %w", old.Commit, old.Container, err) +} + // releaseLabels are the docker labels that mark a container as managed by // seploy and tie it to its app and commit. func releaseLabels(appName, sha string) map[string]string { @@ -241,12 +462,28 @@ func healthCheck(a *app.App, hostPort int, container string) error { } func waitHealthy(a *app.App, hostPort int, container string) error { + timeout := a.HealthTimeoutOrDefault() + if a.Healthcheck == "" { time.Sleep(2 * time.Second) if !docker.IsRunning(container) { return fmt.Errorf("container %s exited right after starting", container) } - return nil + + addr := fmt.Sprintf("127.0.0.1:%d", hostPort) + deadline := time.Now().Add(timeout) + for { + conn, err := net.DialTimeout("tcp", addr, 3*time.Second) + if err == nil { + conn.Close() + return nil + } + if !time.Now().Before(deadline) { + break + } + time.Sleep(time.Second) + } + return fmt.Errorf("nothing accepted a connection on %s within %s. The app may be binding 127.0.0.1 inside the container instead of 0.0.0.0. Consider also setting healthcheck in app.toml to check an HTTP endpoint", addr, timeout) } path := a.Healthcheck @@ -261,7 +498,7 @@ func waitHealthy(a *app.App, hostPort int, container string) error { }, } fmt.Printf("Waiting for health check %s...\n", url) - deadline := time.Now().Add(30 * time.Second) + deadline := time.Now().Add(timeout) for time.Now().Before(deadline) { resp, err := client.Get(url) if err == nil { @@ -272,20 +509,41 @@ func waitHealthy(a *app.App, hostPort int, container string) error { } time.Sleep(time.Second) } - return fmt.Errorf("health check %s did not pass within 30 seconds", url) + return fmt.Errorf("health check %s did not pass within %s", url, timeout) } -// pruneReleases removes app containers and images other than the current -// and previous release. Best-effort: failures only warn. -func pruneReleases(appName string, st *app.State) { - keepContainers := map[string]bool{} - keepImages := map[string]bool{} +// releaseKeepSets computes which container names and image names must be +// kept when pruning an app's releases. Containers are kept only for the +// current and previous release (the two that can be running or rolled back +// to instantly); images are kept for every release in History (which is +// truncated to keep, plus Current and Previous defensively in case History +// predates this field or is otherwise out of sync). +func releaseKeepSets(st *app.State, keep int) (keepContainers, keepImages map[string]bool) { + keepContainers = map[string]bool{} + keepImages = map[string]bool{} for _, rel := range []*app.Release{st.Current, st.Previous} { if rel != nil { keepContainers[rel.Container] = true keepImages[rel.Image] = true } } + history := st.History + if keep >= 0 && len(history) > keep { + history = history[:keep] + } + for _, rel := range history { + if rel != nil { + keepImages[rel.Image] = true + } + } + return keepContainers, keepImages +} + +// pruneReleases removes app containers other than the current and previous +// release, and images other than those of releases still in History (plus +// current and previous defensively). Best-effort: failures only warn. +func pruneReleases(appName string, st *app.State, cfg config.Config) { + keepContainers, keepImages := releaseKeepSets(st, cfg.KeepReleases) containers, err := docker.ContainersForApp(appName) if err == nil { for _, c := range containers { diff --git a/cmd/seploy/deploy_test.go b/cmd/seploy/deploy_test.go new file mode 100644 index 0000000..a197c7a --- /dev/null +++ b/cmd/seploy/deploy_test.go @@ -0,0 +1,128 @@ +package main + +import ( + "sort" + "testing" + + "github.com/Telep-IO/seploy/internal/app" +) + +func keys(m map[string]bool) []string { + var ks []string + for k := range m { + ks = append(ks, k) + } + sort.Strings(ks) + return ks +} + +func TestReleaseKeepSets(t *testing.T) { + rel := func(commit string) *app.Release { + return &app.Release{ + Commit: commit, + Image: "img-" + commit, + Container: "container-" + commit, + } + } + + tests := []struct { + name string + st *app.State + keep int + wantContainer []string + wantImages []string + }{ + { + name: "history longer than keep", + st: &app.State{ + Current: rel("c"), + Previous: rel("b"), + History: []*app.Release{rel("c"), rel("b"), rel("a")}, + }, + keep: 2, + wantContainer: []string{"container-b", "container-c"}, + wantImages: []string{"img-b", "img-c"}, + }, + { + name: "empty history falls back to current and previous images", + st: &app.State{ + Current: rel("c"), + Previous: rel("b"), + History: nil, + }, + keep: 3, + wantContainer: []string{"container-b", "container-c"}, + wantImages: []string{"img-b", "img-c"}, + }, + { + name: "previous nil", + st: &app.State{ + Current: rel("c"), + Previous: nil, + History: []*app.Release{rel("c")}, + }, + keep: 3, + wantContainer: []string{"container-c"}, + wantImages: []string{"img-c"}, + }, + { + name: "current nil", + st: &app.State{ + Current: nil, + Previous: rel("b"), + History: []*app.Release{rel("b")}, + }, + keep: 3, + wantContainer: []string{"container-b"}, + wantImages: []string{"img-b"}, + }, + { + name: "everything nil", + st: &app.State{}, + keep: 3, + wantContainer: nil, + wantImages: nil, + }, + { + name: "history shorter than keep keeps all of it", + st: &app.State{ + Current: rel("c"), + Previous: rel("b"), + History: []*app.Release{rel("c"), rel("b")}, + }, + keep: 5, + wantContainer: []string{"container-b", "container-c"}, + wantImages: []string{"img-b", "img-c"}, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + gotContainers, gotImages := releaseKeepSets(tt.st, tt.keep) + + gotC := keys(gotContainers) + if len(gotC) != len(tt.wantContainer) { + t.Errorf("containers = %v, want %v", gotC, tt.wantContainer) + } else { + for i, c := range gotC { + if c != tt.wantContainer[i] { + t.Errorf("containers = %v, want %v", gotC, tt.wantContainer) + break + } + } + } + + gotI := keys(gotImages) + if len(gotI) != len(tt.wantImages) { + t.Errorf("images = %v, want %v", gotI, tt.wantImages) + } else { + for i, im := range gotI { + if im != tt.wantImages[i] { + t.Errorf("images = %v, want %v", gotI, tt.wantImages) + break + } + } + } + }) + } +} diff --git a/cmd/seploy/doctor.go b/cmd/seploy/doctor.go index bfb8f3d..94b08ef 100644 --- a/cmd/seploy/doctor.go +++ b/cmd/seploy/doctor.go @@ -1,11 +1,11 @@ package main import ( + "errors" "fmt" "net" "os" "strings" - "syscall" "time" "github.com/Telep-IO/seploy/internal/app" @@ -94,7 +94,7 @@ func runDoctor(cfg config.Config) error { } } - if free, ok := diskFree(cfg.DataDir); ok { + if free, err := system.DiskFree(cfg.DataDir); err == nil { d.check(free >= 2<<30, fmt.Sprintf("disk space: %.1f GB free", float64(free)/(1<<30)), "Less than 2 GB free. Old images may pile up; try: docker system prune") } @@ -102,9 +102,19 @@ func runDoctor(cfg config.Config) error { publicIP, ipErr := netcheck.PublicIP() d.check(ipErr == nil, "public IP: "+publicIP, "Could not reach api.ipify.org — does this server have internet access?") + var tunnelID string + tunnelTokenMode := false if anyTunnel { d.check(tunnel.Configured(), "Cloudflare Tunnel configured", "Run: sudo seploy tunnel setup") d.check(system.ServiceActive("cloudflared"), "cloudflared running", "Start it: sudo systemctl start cloudflared (or rc-service cloudflared start)") + if tunnel.Mode() == tunnel.ModeToken { + // tunnel.Ensure() runs `cloudflared tunnel list`, which needs + // cert.pem — remotely-managed (token mode) tunnels don't have + // one. Fall back to the resolve-only CNAME check below. + tunnelTokenMode = true + } else if id, err := tunnel.Ensure(); err == nil { + tunnelID = id + } } if anyWebhook { d.check(system.ServiceActive("seploy-webhook"), "webhook listener running", "Re-run: sudo seploy webhook enable ") @@ -127,9 +137,24 @@ func runDoctor(cfg config.Config) error { "Check: seploy logs "+a.Name) if a.Tunneled() { + hint := "Create the record: cloudflared tunnel route dns " + tunnel.Name + " " + a.Domain + if tunnelTokenMode { + hint = "Add a Public Hostname for " + a.Domain + " → http://localhost:80 in Zero Trust > Networks > Tunnels > > Public Hostname" + } + if tunnelID != "" { + want := tunnel.CNAMETarget(tunnelID) + cname, cnameErr := netcheck.LookupCNAME(a.Domain) + if cnameErr == nil { + ok := strings.EqualFold(strings.TrimSuffix(cname, "."), want) + hint = a.Domain + " points at " + strings.TrimSuffix(cname, ".") + " — expected " + want + ". Delete the conflicting record in the Cloudflare dashboard and re-run: sudo seploy new (or cloudflared tunnel route dns " + tunnel.Name + " " + a.Domain + ")" + d.check(ok, a.Domain+" CNAME → this tunnel", hint) + continue + } + // CNAME lookup failed; fall back to a resolve-only check. + hint += " (note: could not verify the CNAME target, only that the domain resolves)" + } _, err := netcheck.LookupHost(a.Domain) - d.check(err == nil, a.Domain+" resolves (Cloudflare)", - "Create the record: cloudflared tunnel route dns "+tunnel.Name+" "+a.Domain) + d.check(err == nil, a.Domain+" resolves (Cloudflare)", hint) } else if ipErr == nil { ok, addrs := netcheck.PointsTo(a.Domain, publicIP) hint := "Add an A record for " + a.Domain + " pointing at " + publicIP @@ -138,6 +163,18 @@ func runDoctor(cfg config.Config) error { } d.check(ok, a.Domain+" → "+publicIP, hint) } + + code, edgeErr := netcheck.CheckEdge(a.Domain, a.Healthcheck) + if edgeErr == nil { + d.check(true, fmt.Sprintf("https://%s reachable (HTTP %d)", a.Domain, code), "") + } else { + hint := edgeErr.Error() + var ee *netcheck.EdgeError + if errors.As(edgeErr, &ee) { + hint = ee.Hint(a.Tunneled()) + } + d.check(false, "https://"+a.Domain+" reachable", hint) + } } fmt.Println() @@ -156,11 +193,3 @@ func listening(port int) bool { conn.Close() return true } - -func diskFree(path string) (uint64, bool) { - var st syscall.Statfs_t - if err := syscall.Statfs(path, &st); err != nil { - return 0, false - } - return st.Bavail * uint64(st.Bsize), true -} diff --git a/cmd/seploy/new.go b/cmd/seploy/new.go index bd38094..5111a58 100644 --- a/cmd/seploy/new.go +++ b/cmd/seploy/new.go @@ -52,7 +52,11 @@ func runNew(rawURL string) error { Branch: cfg.DefaultBranch, Dockerfile: "Dockerfile", } - a.Domain = ui.PromptRequired(fmt.Sprintf("Domain for %s", name)) + domain, err := promptDomain(cfg, name) + if err != nil { + return err + } + a.Domain = domain a.Branch = ui.Prompt("Branch", cfg.DefaultBranch) if tunnel.Configured() { @@ -92,15 +96,68 @@ func runNew(rawURL string) error { } if a.Tunneled() { - fmt.Printf("Creating the Cloudflare DNS record for %s...\n", a.Domain) - if err := tunnel.RouteDNS(a.Domain); err != nil { - fmt.Println("Warning:", err) - fmt.Printf("Create it manually with: cloudflared tunnel route dns %s %s\n", tunnel.Name, a.Domain) + if tunnel.Mode() == tunnel.ModeToken { + printTokenModeRoutingInstructions(a.Domain) + } else { + fmt.Printf("Creating the Cloudflare DNS record for %s...\n", a.Domain) + id, err := tunnel.Ensure() + if err != nil { + fmt.Println("Warning:", err) + fmt.Printf("Create it manually with: cloudflared tunnel route dns %s %s\n", tunnel.Name, a.Domain) + } else if err := tunnel.RouteDNS(id, a.Domain); err != nil { + fmt.Println("Warning:", err) + fmt.Printf("Create it manually with: cloudflared tunnel route dns %s %s\n", tunnel.Name, a.Domain) + } } } return runDeploy(cfg, a) } +// printTokenModeRoutingInstructions tells the operator how to route domain +// through the tunnel when this server's tunnel is remotely managed +// (`seploy tunnel setup --token`). Token-mode tunnels have no local cert.pem, +// so tunnel.RouteDNS (which needs one to call `cloudflared tunnel route dns`) +// cannot run; instead, both the DNS record and the ingress rule are created +// together by adding a Public Hostname in the dashboard, so that's the only +// step needed here. +func printTokenModeRoutingInstructions(domain string) { + fmt.Println() + fmt.Println("This server's tunnel is remotely managed (token mode), so seploy cannot") + fmt.Println("create the DNS record automatically. Add a Public Hostname instead:") + fmt.Println() + fmt.Println(" 1. Cloudflare dashboard > Zero Trust > Networks > Tunnels") + fmt.Println(" 2. Open this server's tunnel, then the \"Public Hostname\" tab") + fmt.Println(" 3. Add a public hostname:") + fmt.Println(" Domain: " + domain) + fmt.Println(" Service: http://localhost:80") + fmt.Println() + fmt.Println("That single step creates both the DNS record and the routing rule; no") + fmt.Println("separate DNS entry is needed.") + fmt.Println() +} + +// promptDomain prompts for a domain, normalizing and re-prompting until it +// is both plausible and not already used by another app. +func promptDomain(cfg config.Config, name string) (string, error) { + for { + raw := ui.PromptRequired(fmt.Sprintf("Domain for %s", name)) + domain, err := app.NormalizeDomain(raw) + if err != nil { + fmt.Println(err) + continue + } + existing, err := app.FindByDomain(cfg, domain) + if err != nil { + return "", err + } + if existing != nil { + fmt.Printf("That domain is already used by app %s.\n", existing.Name) + continue + } + return domain, nil + } +} + // warnIfDNSMismatch is a best-effort preflight. Deploys still proceed, but // wrong DNS is the most common cause of TLS failures, so say so up front. func warnIfDNSMismatch(domain string) { diff --git a/cmd/seploy/releases.go b/cmd/seploy/releases.go new file mode 100644 index 0000000..3ba5f4c --- /dev/null +++ b/cmd/seploy/releases.go @@ -0,0 +1,57 @@ +package main + +import ( + "fmt" + + "github.com/Telep-IO/seploy/internal/app" + "github.com/Telep-IO/seploy/internal/config" + "github.com/Telep-IO/seploy/internal/docker" + "github.com/spf13/cobra" +) + +var releasesCmd = &cobra.Command{ + Use: "releases ", + Short: "List an app's releases (current, previous and history)", + Args: cobra.ExactArgs(1), + RunE: func(cmd *cobra.Command, args []string) error { + cfg, err := config.Load() + if err != nil { + return err + } + a, err := app.Load(cfg, args[0]) + if err != nil { + return err + } + st, err := app.LoadState(cfg, a.Name) + if err != nil { + return err + } + + if st.Current == nil { + fmt.Println("Status: never deployed") + return nil + } + + printRelease("current", st.Current) + if st.Previous != nil { + printRelease("previous", st.Previous) + } + for _, rel := range st.History { + printRelease("history", rel) + } + return nil + }, +} + +func init() { + rootCmd.AddCommand(releasesCmd) +} + +func printRelease(role string, rel *app.Release) { + image := "missing" + if docker.ImageExists(rel.Image) { + image = "present" + } + fmt.Printf("%-8s commit %s deployed %s image %s\n", + role, rel.Commit, rel.DeployedAt.Format("2006-01-02 15:04:05 MST"), image) +} diff --git a/cmd/seploy/remove.go b/cmd/seploy/remove.go index 1ddf55e..b0a1f36 100644 --- a/cmd/seploy/remove.go +++ b/cmd/seploy/remove.go @@ -83,6 +83,9 @@ var removeCmd = &cobra.Command{ } fmt.Println("Delete them with: docker volume rm ") } + if a.Tunneled() { + fmt.Printf("DNS record NOT removed: the cloudflared CLI cannot delete DNS records. If %s is no longer used, delete its CNAME record in the Cloudflare dashboard (DNS -> Records).\n", a.Domain) + } fmt.Printf("Removed %s.\n", name) return nil }, diff --git a/cmd/seploy/restart.go b/cmd/seploy/restart.go index 7f33149..a8920c0 100644 --- a/cmd/seploy/restart.go +++ b/cmd/seploy/restart.go @@ -5,6 +5,7 @@ import ( "time" "github.com/Telep-IO/seploy/internal/app" + "github.com/Telep-IO/seploy/internal/caddy" "github.com/Telep-IO/seploy/internal/config" "github.com/Telep-IO/seploy/internal/docker" "github.com/Telep-IO/seploy/internal/system" @@ -35,8 +36,12 @@ func init() { rootCmd.AddCommand(restartCmd) } -// runRestart recreates the current container without pulling or building: -// same image, same host port (so Caddy needs no change), fresh env file. +// runRestart recreates the current container without pulling or building +// (same image, fresh env file), using the same start-then-switch pattern as +// runDeploy: the replacement starts under a unique name on a Docker-chosen +// port and is health checked before Caddy is switched over and the old +// container is retired. If the replacement fails, the old container is left +// running untouched. func runRestart(cfg config.Config, a *app.App) error { lock, err := app.AcquireLock(cfg, a.Name) if err != nil { @@ -62,17 +67,18 @@ func runRestart(cfg config.Config, a *app.App) error { return err } - fmt.Printf("Recreating %s from %s...\n", cur.Container, cur.Image) - if err := docker.Remove(cur.Container); err != nil { + container := fmt.Sprintf("seploy-%s-%s-%d", a.Name, cur.Commit, time.Now().Unix()) + // Clear any leftover container with this name from a failed restart. + if err := docker.Remove(container); err != nil { return err } - // Reuse the same host port so the Caddy config stays valid. + + fmt.Printf("Starting %s from %s...\n", container, cur.Image) err = docker.Run(docker.RunOpts{ - Name: cur.Container, + Name: container, Image: cur.Image, EnvFile: envPath, - HostPort: cur.HostPort, - ContainerPort: cur.ContainerPort, + ContainerPort: cur.ContainerPort, // HostPort 0: Docker binds a free localhost port Labels: releaseLabels(a.Name, cur.Commit), Memory: a.Memory, CPUs: a.CPUs, @@ -81,16 +87,56 @@ func runRestart(cfg config.Config, a *app.App) error { if err != nil { return err } + hostPort, err := docker.HostPort(container, cur.ContainerPort) + if err != nil { + docker.Remove(container) + return fmt.Errorf("the current release of %q was left running and untouched: %w", a.Name, err) + } + fmt.Printf("Published on 127.0.0.1:%d\n", hostPort) - if err := healthCheck(a, cur.HostPort, cur.Container); err != nil { - return err + if err := healthCheck(a, hostPort, container); err != nil { + docker.Remove(container) + return fmt.Errorf("the current release of %q was left running and untouched: %w", a.Name, err) + } + + if err := caddy.WriteSite(cfg.CaddyConfDir, a.Name, siteFor(cfg, a, hostPort)); err != nil { + // The failed write may have left a partial site file on disk; + // best-effort restore the site for the old container's port. + restoreSite(cfg, a, cur.HostPort) + docker.Remove(container) + return fmt.Errorf("the current release of %q was left running and untouched: %w", a.Name, err) + } + if err := caddy.Reload(cfg.Caddyfile); err != nil { + // The on-disk site file now points at the replacement container, + // which is about to be removed. Best-effort restore it so a later + // reload does not pick up a site pointing at a dead port. + restoreSite(cfg, a, cur.HostPort) + docker.Remove(container) + return fmt.Errorf("the current release of %q was left running and untouched: %w", a.Name, err) } + oldContainer := cur.Container + cur.Container = container + cur.HostPort = hostPort cur.DeployedAt = time.Now().UTC() if err := app.SaveState(cfg, a.Name, st); err != nil { return err } + fmt.Printf("Stopping old container %s...\n", oldContainer) + if err := docker.Remove(oldContainer); err != nil { + fmt.Println("Warning:", err) + } + fmt.Printf("Restarted %s (env changes are now live).\n", a.Name) return nil } + +// restoreSite best-effort rewrites the app's Caddy site file to point back +// at the given host port after a failed switch, warning instead of failing +// if the restore itself errors. +func restoreSite(cfg config.Config, a *app.App, hostPort int) { + if err := caddy.WriteSite(cfg.CaddyConfDir, a.Name, siteFor(cfg, a, hostPort)); err != nil { + fmt.Println("Warning:", err) + } +} diff --git a/cmd/seploy/rollback.go b/cmd/seploy/rollback.go index f18c947..d072f06 100644 --- a/cmd/seploy/rollback.go +++ b/cmd/seploy/rollback.go @@ -2,6 +2,8 @@ package main import ( "fmt" + "sort" + "strings" "time" "github.com/Telep-IO/seploy/internal/app" @@ -12,6 +14,8 @@ import ( "github.com/spf13/cobra" ) +var rollbackTo string + var rollbackCmd = &cobra.Command{ Use: "rollback ", Short: "Switch an app back to its previous release", @@ -28,11 +32,15 @@ var rollbackCmd = &cobra.Command{ if err != nil { return err } + if rollbackTo != "" { + return runRollbackTo(cfg, a, rollbackTo) + } return runRollback(cfg, a) }, } func init() { + rollbackCmd.Flags().StringVar(&rollbackTo, "to", "", "roll back to the release with this commit SHA (or an unambiguous prefix) instead of the previous release") rootCmd.AddCommand(rollbackCmd) } @@ -52,77 +60,193 @@ func runRollback(cfg config.Config, a *app.App) error { } prev, cur := st.Previous, st.Current + hostPort, err := switchToRelease(cfg, a, prev, st, "previous release", "Previous container") + if err != nil { + return err + } + + st.Current, st.Previous = prev, cur + if err := app.SaveState(cfg, a.Name, st); err != nil { + return err + } + + fmt.Println() + fmt.Printf("Rolled back %s to commit %s\n", a.Name, prev.Commit) + fmt.Printf(" container: %s\n", prev.Container) + fmt.Printf(" upstream: 127.0.0.1:%d\n", hostPort) + fmt.Println() + fmt.Println("Rolling back again switches to the release you just left.") + return nil +} + +func runRollbackTo(cfg config.Config, a *app.App, sha string) error { + lock, err := app.AcquireLock(cfg, a.Name) + if err != nil { + return err + } + defer lock.Release() + + st, err := app.LoadState(cfg, a.Name) + if err != nil { + return err + } + target, err := findRelease(st, sha) + if err != nil { + return err + } + if err := checkNotCurrent(st, target); err != nil { + return err + } + cur := st.Current + + hostPort, err := switchToRelease(cfg, a, target, st, "release", "Target container") + if err != nil { + return err + } + + st.Previous = cur + st.Current = target + if err := app.SaveState(cfg, a.Name, st); err != nil { + return err + } + + fmt.Println() + fmt.Printf("Rolled back %s to commit %s\n", a.Name, target.Commit) + fmt.Printf(" container: %s\n", target.Container) + fmt.Printf(" upstream: 127.0.0.1:%d\n", hostPort) + return nil +} + +// switchToRelease starts rel (or reruns its kept image if the container is +// gone), health checks it, switches Caddy traffic to it and stops the +// currently running container. It returns the host port rel ends up +// published on. On success rel.HostPort and rel.DeployedAt are updated; +// st.Current is left untouched so the caller decides how to update state. +func switchToRelease(cfg config.Config, a *app.App, rel *app.Release, st *app.State, releaseLabel, goneLabel string) (int, error) { envPath := app.EnvPath(cfg, a.Name) if err := app.EnsureEnvFile(envPath); err != nil { - return err + return 0, err } mounts, err := a.VolumeMounts() if err != nil { - return err + return 0, err } - // The previous container is kept stopped after each deploy, so usually - // it just needs starting. If it was removed, rerun its image (which is - // also kept by release pruning). - hostPort := prev.HostPort - fmt.Printf("Starting previous release %s (commit %s)...\n", prev.Container, prev.Commit) - if err := docker.Start(prev.Container); err != nil { - if err := docker.Remove(prev.Container); err != nil { - return err + // The target container is kept stopped after each deploy, so usually it + // just needs starting. If it was removed, rerun its image (which is also + // kept by release pruning). + hostPort := rel.HostPort + fmt.Printf("Starting %s %s (commit %s)...\n", releaseLabel, rel.Container, rel.Commit) + if err := docker.Start(rel.Container); err != nil { + if err := docker.Remove(rel.Container); err != nil { + return 0, err } - fmt.Printf("Previous container is gone; starting it again from %s...\n", prev.Image) + fmt.Printf("%s is gone; starting it again from %s...\n", goneLabel, rel.Image) err = docker.Run(docker.RunOpts{ - Name: prev.Container, - Image: prev.Image, + Name: rel.Container, + Image: rel.Image, EnvFile: envPath, - ContainerPort: prev.ContainerPort, // HostPort 0: Docker picks - Labels: releaseLabels(a.Name, prev.Commit), + ContainerPort: rel.ContainerPort, // HostPort 0: Docker picks + Labels: releaseLabels(a.Name, rel.Commit), Memory: a.Memory, CPUs: a.CPUs, Volumes: mounts, }) if err != nil { - return err + return 0, err } - hostPort, err = docker.HostPort(prev.Container, prev.ContainerPort) + hostPort, err = docker.HostPort(rel.Container, rel.ContainerPort) if err != nil { - docker.Remove(prev.Container) - return err + docker.Remove(rel.Container) + return 0, err } } - if err := healthCheck(a, hostPort, prev.Container); err != nil { + if err := healthCheck(a, hostPort, rel.Container); err != nil { // Leave the current release serving traffic. - docker.Stop(prev.Container) - return fmt.Errorf("rollback aborted, the current release keeps serving traffic.\n%v", err) + docker.Stop(rel.Container) + return 0, fmt.Errorf("rollback aborted, the current release keeps serving traffic.\n%v", err) } if err := caddy.WriteSite(cfg.CaddyConfDir, a.Name, siteFor(cfg, a, hostPort)); err != nil { - return err + return 0, err } if err := caddy.Reload(cfg.Caddyfile); err != nil { - return err + return 0, err } - if cur != nil { - fmt.Printf("Stopping %s...\n", cur.Container) - if err := docker.Stop(cur.Container); err != nil { + if st.Current != nil { + fmt.Printf("Stopping %s...\n", st.Current.Container) + if err := docker.Stop(st.Current.Container); err != nil { fmt.Println("Warning:", err) } } - prev.HostPort = hostPort - prev.DeployedAt = time.Now().UTC() - st.Current, st.Previous = prev, cur - if err := app.SaveState(cfg, a.Name, st); err != nil { - return err - } + rel.HostPort = hostPort + rel.DeployedAt = time.Now().UTC() + return hostPort, nil +} - fmt.Println() - fmt.Printf("Rolled back %s to commit %s\n", a.Name, prev.Commit) - fmt.Printf(" container: %s\n", prev.Container) - fmt.Printf(" upstream: 127.0.0.1:%d\n", hostPort) - fmt.Println() - fmt.Println("Rolling back again switches to the release you just left.") +// checkNotCurrent rejects a rollback target that is already the current +// release: switching to it would end with stopping the very container that +// was just started, taking the app down. +func checkNotCurrent(st *app.State, target *app.Release) error { + if st.Current != nil && st.Current.Commit == target.Commit { + return fmt.Errorf("commit %q is already the current release; nothing to roll back to.", target.Commit) + } return nil } + +// findRelease finds the release in st (checking Current, Previous and +// History) whose Commit equals sha exactly, or, failing that, whose Commit +// has sha as an unambiguous prefix. It returns an error listing the +// available commits and their deploy times when sha matches nothing or +// matches more than one release as a prefix. +func findRelease(st *app.State, sha string) (*app.Release, error) { + var all []*app.Release + if st.Current != nil { + all = append(all, st.Current) + } + if st.Previous != nil { + all = append(all, st.Previous) + } + all = append(all, st.History...) + + for _, r := range all { + if r.Commit == sha { + return r, nil + } + } + + var matches []*app.Release + seen := map[string]bool{} + for _, r := range all { + if seen[r.Commit] { + continue + } + if strings.HasPrefix(r.Commit, sha) { + seen[r.Commit] = true + matches = append(matches, r) + } + } + if len(matches) == 1 { + return matches[0], nil + } + + available := map[string]time.Time{} + for _, r := range all { + if _, ok := available[r.Commit]; !ok { + available[r.Commit] = r.DeployedAt + } + } + var lines []string + for commit, at := range available { + lines = append(lines, fmt.Sprintf(" %s %s", commit, at.Format("2006-01-02 15:04:05 MST"))) + } + sort.Slice(lines, func(i, j int) bool { return lines[i] < lines[j] }) + + if len(matches) > 1 { + return nil, fmt.Errorf("commit %q is ambiguous, it matches multiple releases. Available releases:\n%s", sha, strings.Join(lines, "\n")) + } + return nil, fmt.Errorf("no release found for commit %q. Available releases:\n%s", sha, strings.Join(lines, "\n")) +} diff --git a/cmd/seploy/rollback_test.go b/cmd/seploy/rollback_test.go new file mode 100644 index 0000000..13fca7b --- /dev/null +++ b/cmd/seploy/rollback_test.go @@ -0,0 +1,99 @@ +package main + +import ( + "strings" + "testing" + "time" + + "github.com/Telep-IO/seploy/internal/app" +) + +func TestFindRelease(t *testing.T) { + rel := func(commit string, at time.Time) *app.Release { + return &app.Release{ + Commit: commit, + Image: "img-" + commit, + Container: "container-" + commit, + DeployedAt: at, + } + } + + t0 := time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC) + t1 := t0.Add(time.Hour) + t2 := t0.Add(2 * time.Hour) + t3 := t0.Add(3 * time.Hour) + + st := &app.State{ + Current: rel("deadbeef", t3), + Previous: rel("cafef00d", t2), + History: []*app.Release{ + rel("cafef00d", t2), + rel("ab00c001", t1), + rel("ab112233", t0), + }, + } + + tests := []struct { + name string + sha string + wantCommit string + wantErr string + }{ + { + name: "exact match", + sha: "deadbeef", + wantCommit: "deadbeef", + }, + { + name: "unambiguous prefix", + sha: "cafe", + wantCommit: "cafef00d", + }, + { + name: "ambiguous prefix", + sha: "ab", + wantErr: "ambiguous", + }, + { + name: "not found", + sha: "0000000", + wantErr: "no release found", + }, + } + + t.Run("current release is rejected by the guard", func(t *testing.T) { + got, err := findRelease(st, "dead") + if err != nil { + t.Fatalf("findRelease(%q) unexpected error: %v", "dead", err) + } + if err := checkNotCurrent(st, got); err == nil { + t.Fatal("checkNotCurrent allowed rolling back to the current release") + } else if !strings.Contains(err.Error(), "already the current release") { + t.Fatalf("checkNotCurrent error = %q; want it to mention the release is already current", err.Error()) + } + if err := checkNotCurrent(st, st.Previous); err != nil { + t.Fatalf("checkNotCurrent rejected a non-current release: %v", err) + } + }) + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := findRelease(st, tt.sha) + if tt.wantErr != "" { + if err == nil { + t.Fatalf("findRelease(%q) = %v, %v; want error containing %q", tt.sha, got, err, tt.wantErr) + } + if !strings.Contains(err.Error(), tt.wantErr) { + t.Fatalf("findRelease(%q) error = %q; want it to contain %q", tt.sha, err.Error(), tt.wantErr) + } + return + } + if err != nil { + t.Fatalf("findRelease(%q) unexpected error: %v", tt.sha, err) + } + if got.Commit != tt.wantCommit { + t.Fatalf("findRelease(%q).Commit = %q; want %q", tt.sha, got.Commit, tt.wantCommit) + } + }) + } +} diff --git a/cmd/seploy/tunnel.go b/cmd/seploy/tunnel.go index 2d8a99c..7222a4f 100644 --- a/cmd/seploy/tunnel.go +++ b/cmd/seploy/tunnel.go @@ -2,6 +2,9 @@ package main import ( "fmt" + "os" + "os/exec" + "strings" "github.com/Telep-IO/seploy/internal/app" "github.com/Telep-IO/seploy/internal/config" @@ -10,16 +13,31 @@ import ( "github.com/spf13/cobra" ) +// tunnelTokenEnvVar is the fallback source for the tunnel token when +// --token is not passed, so it does not have to be typed on the command +// line (and show up in shell history / process listings via cobra's flag +// parsing). +const tunnelTokenEnvVar = "SEPLOY_TUNNEL_TOKEN" + var tunnelCmd = &cobra.Command{ Use: "tunnel", Short: "Expose apps through a Cloudflare Tunnel (no DNS records, no open ports)", } +var tunnelSetupToken string + var tunnelSetupCmd = &cobra.Command{ Use: "setup", Short: "Install cloudflared and create this server's tunnel", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { + token := tunnelSetupToken + if token == "" { + token = os.Getenv(tunnelTokenEnvVar) + } + if token != "" { + return runTunnelSetupToken(token) + } return runTunnelSetup() }, } @@ -33,8 +51,18 @@ var tunnelStatusCmd = &cobra.Command{ }, } +var tunnelUpgradeCmd = &cobra.Command{ + Use: "upgrade", + Short: "Upgrade cloudflared to the version seploy is pinned to", + Args: cobra.NoArgs, + RunE: func(cmd *cobra.Command, args []string) error { + return runTunnelUpgrade() + }, +} + func init() { - tunnelCmd.AddCommand(tunnelSetupCmd, tunnelStatusCmd) + tunnelSetupCmd.Flags().StringVar(&tunnelSetupToken, "token", "", "Dashboard-issued tunnel token for a remotely-managed tunnel (skips interactive login). Falls back to the "+tunnelTokenEnvVar+" env var.") + tunnelCmd.AddCommand(tunnelSetupCmd, tunnelStatusCmd, tunnelUpgradeCmd) rootCmd.AddCommand(tunnelCmd) } @@ -68,6 +96,58 @@ func runTunnelSetup() error { return nil } +// runTunnelSetupToken configures this server's tunnel non-interactively +// from a dashboard-issued token: no `cloudflared tunnel login`, no +// `tunnel create`, no local config.yml. The tunnel and its ingress rules +// (which domain routes where) are entirely managed in the Cloudflare +// dashboard; this server only has to run cloudflared pointed at the token. +func runTunnelSetupToken(token string) error { + if err := system.RequireRoot("seploy tunnel setup --token "); err != nil { + return err + } + if err := tunnel.Install(); err != nil { + return err + } + if err := tunnel.InstallServiceToken(token); err != nil { + return err + } + if err := tunnel.RecordMode(tunnel.ModeToken); err != nil { + return err + } + + fmt.Println() + fmt.Println("Cloudflare Tunnel is ready (token mode: remotely managed).") + fmt.Println() + fmt.Println("Ingress rules live in the dashboard, not on this server. For each app") + fmt.Println("exposed via the tunnel, add a Public Hostname in Zero Trust > Networks >") + fmt.Println("Tunnels > > Public Hostname, pointing the domain at") + fmt.Println("service http://localhost:80. seploy will print this reminder again") + fmt.Println("when you create a tunneled app with 'seploy '.") + return nil +} + +func runTunnelUpgrade() error { + if err := system.RequireRoot("seploy tunnel upgrade"); err != nil { + return err + } + fmt.Printf("Upgrading cloudflared to %s...\n", tunnel.Version) + if err := tunnel.Upgrade(); err != nil { + return err + } + if tunnel.Configured() { + if err := system.RestartService("cloudflared"); err != nil { + return err + } + } + + out, err := exec.Command(tunnel.Bin(), "--version").Output() + if err != nil { + return fmt.Errorf("cloudflared upgraded, but running '%s --version' failed", tunnel.Bin()) + } + fmt.Println(strings.TrimSpace(string(out))) + return nil +} + func runTunnelStatus() error { cfg, err := config.Load() if err != nil { @@ -82,9 +162,16 @@ func runTunnelStatus() error { if !tunnel.Configured() { fmt.Println("tunnel: not configured") fmt.Println("Set it up with: sudo seploy tunnel setup") + fmt.Println(" or: sudo seploy tunnel setup --token (no browser needed)") return nil } - fmt.Println("tunnel: configured (" + tunnel.ConfigPath + ")") + switch mode := tunnel.Mode(); mode { + case tunnel.ModeToken: + fmt.Println("tunnel: configured (token mode: remotely managed)") + fmt.Println(" ingress rules live in the Cloudflare dashboard, not " + tunnel.ConfigPath) + default: + fmt.Println("tunnel: configured (" + tunnel.ConfigPath + ")") + } if system.ServiceActive("cloudflared") { fmt.Println("service: running") } else { diff --git a/cmd/seploy/webhook.go b/cmd/seploy/webhook.go index 72be872..129bda7 100644 --- a/cmd/seploy/webhook.go +++ b/cmd/seploy/webhook.go @@ -11,6 +11,7 @@ import ( "os/exec" "strings" "sync" + "time" "github.com/Telep-IO/seploy/internal/app" "github.com/Telep-IO/seploy/internal/caddy" @@ -245,7 +246,7 @@ func (r *deployRunner) loop(name string) { log.Printf("deploy %s: %v", name, err) } else { log.Printf("deploying %s", name) - cmd := exec.Command(exe, "deploy", name) + cmd := exec.Command(exe, "deploy", name, "--wait-lock=10m") cmd.Stdout = os.Stdout cmd.Stderr = os.Stderr if err := cmd.Run(); err != nil { @@ -280,7 +281,14 @@ func runWebhookServe() error { addr := fmt.Sprintf("127.0.0.1:%d", cfg.WebhookPort) log.Printf("seploy webhook listener on %s%s", addr, webhook.Path) - return http.ListenAndServe(addr, mux) + srv := &http.Server{ + Addr: addr, + Handler: mux, + ReadHeaderTimeout: 10 * time.Second, + ReadTimeout: 30 * time.Second, + WriteTimeout: 30 * time.Second, + } + return srv.ListenAndServe() } func handleHook(cfg config.Config, runner *deployRunner, w http.ResponseWriter, r *http.Request) { @@ -293,8 +301,8 @@ func handleHook(cfg config.Config, runner *deployRunner, w http.ResponseWriter, if h, _, err := net.SplitHostPort(host); err == nil { host = h } - a := findAppByDomain(cfg, host) - if a == nil { + a, err := app.FindByDomain(cfg, host) + if err != nil || a == nil { http.Error(w, "unknown app", http.StatusNotFound) return } @@ -336,17 +344,3 @@ func handleHook(cfg config.Config, runner *deployRunner, w http.ResponseWriter, fmt.Fprintln(w, "ignored event") } } - -func findAppByDomain(cfg config.Config, domain string) *app.App { - names, err := app.List(cfg) - if err != nil { - return nil - } - for _, n := range names { - a, err := app.Load(cfg, n) - if err == nil && strings.EqualFold(a.Domain, domain) { - return a - } - } - return nil -} diff --git a/docs/HARDENING-PLAN-1.md b/docs/HARDENING-PLAN-1.md new file mode 100644 index 0000000..5017d82 --- /dev/null +++ b/docs/HARDENING-PLAN-1.md @@ -0,0 +1,937 @@ +# Hardening Plan 1 + +Goal: make seploy a super-reliable simple-deploy utility installable on any +VPS, deploying services from a root-level Dockerfile, with Cloudflare Tunnel +DNS via cloudflared. + +This plan expands the findings of a full code review (2026-07-07) into +independent tasks. Each task ends with a **Prompt** block that is +self-contained — it can be handed to an agent with no other context beyond a +checkout of this repository. + +Conventions every task must follow (repeated inside each prompt so tasks stay +independent): + +- Go 1.21+, standard library preferred; external tools are invoked with + `exec.Command` and argument arrays, never shell strings. +- Error messages are user-facing: full sentences that say what to do next, + matching the existing style (see `internal/git/git.go`). +- Run `make fmt` and `make test` before finishing. Add table-driven tests for + any new pure function (see `internal/caddy/caddy_test.go` for style). +- Do not add new third-party dependencies without strong justification. +- Update `README.md` whenever behavior or config documented there changes. + +Suggested order: Phase 1 tasks are correctness fixes and should land first. +Tasks within a phase are independent of each other unless noted. + +--- + +## Phase 1 — Correctness + +### Task 1.1 — Wire up `keep_releases` (currently dead config) + +`Config.KeepReleases` (`internal/config/config.go:23`, default 3) is defined +and documented in `README.md`, but `pruneReleases` in +`cmd/seploy/deploy.go:280-309` ignores it and always keeps only the Current +and Previous releases' containers and images. The config field is dead and +the docs lie. + +Design decision: containers stay limited to Current + Previous (a stopped +container is only useful for instant rollback one step back), but **images** +are kept for the `keep_releases` most recent releases so older commits can be +re-run. This also lays groundwork for Task 3.3 (`rollback --to`). + +**Prompt:** + +> You are working in the `seploy` Go repository (a Git-to-VPS deploy tool; +> module `github.com/Telep-IO/seploy`). Conventions: Go stdlib only, shell +> out via `exec.Command` with argument arrays, user-facing error messages as +> full sentences, run `make fmt` and `make test` when done, add table-driven +> tests for new pure functions, update README.md if documented behavior +> changes. +> +> Problem: the global config field `keep_releases` +> (`internal/config/config.go`, `Config.KeepReleases`, default 3, documented +> in README.md) is never read. `pruneReleases` in `cmd/seploy/deploy.go` +> always keeps only the Current and Previous releases. +> +> Task: make `keep_releases` real. +> +> 1. Add a `History []*Release` field to `app.State` +> (`internal/app/app.go`) recording past releases, most recent first. +> On every successful deploy (in `runDeploy`, `cmd/seploy/deploy.go`), +> prepend the new Current release to History and truncate History to +> `cfg.KeepReleases` entries. Loading old `state.json` files without the +> field must keep working (it will — the field is additive JSON). +> 2. Change `pruneReleases` to take `cfg config.Config` and keep: +> containers for Current and Previous only (unchanged), but **images** +> for every release in History (plus Current and Previous defensively). +> 3. Validate `KeepReleases` in `config.Load`: values < 1 are clamped to 1 +> with no error (be liberal in what you accept). +> 4. Update the `keep_releases` description in README.md to state precisely +> what is kept: "images of the last N releases; containers only for the +> current and previous release". +> 5. Add tests: a pure helper that computes the keep-sets from a State and +> KeepReleases value is easiest to test — extract one and table-test it +> (History longer than N, History empty, Previous nil, Current nil). +> +> Acceptance: `make test` passes; deploying does not remove images listed in +> History; `state.json` from before this change still loads. + +--- + +### Task 1.2 — `tunnel.RouteDNS` reports success on conflicting DNS records + +`RouteDNS` (`internal/tunnel/tunnel.go:228-238`) runs +`cloudflared tunnel route dns seploy ` and treats any error whose +output contains the substring "already" as success. If the domain already has +an A record, or a CNAME pointing at a *different* tunnel, cloudflared fails +with a message containing "already", seploy swallows it, and the app is +silently unreachable while seploy reports success. Also `seploy doctor` only +checks that a tunneled domain *resolves*, not that it resolves to this +server's tunnel. + +**Prompt:** + +> You are working in the `seploy` Go repository (a Git-to-VPS deploy tool; +> module `github.com/Telep-IO/seploy`). Conventions: Go stdlib only, shell +> out via `exec.Command` with argument arrays, user-facing error messages as +> full sentences, run `make fmt` and `make test` when done, add table-driven +> tests for new pure functions, update README.md if documented behavior +> changes. +> +> Background: seploy exposes apps through a single Cloudflare Tunnel named +> `seploy`. `internal/tunnel/tunnel.go` wraps the `cloudflared` CLI. +> `RouteDNS(domain)` runs `cloudflared tunnel route dns seploy ` and +> currently treats any error output containing the substring "already" as +> success. That is wrong when the existing record is an A record or a CNAME +> to a *different* tunnel — the command fails, seploy reports success, and +> the app is unreachable. +> +> Task: +> +> 1. In `internal/tunnel/tunnel.go`, make the success/conflict decision +> precise. A pre-existing record is only OK if it already points at +> *this* tunnel. The correct CNAME target is `.cfargotunnel.com`. +> Implement `RouteDNS(id, domain string) error` (it now needs the tunnel +> id; update the caller in `cmd/seploy/new.go`): +> - Run the route command. On success, return nil. +> - On failure, resolve the domain's CNAME with +> `net.DefaultResolver.LookupCNAME` (5s timeout via context — see +> `internal/netcheck/netcheck.go` for the pattern). If the CNAME target +> (trim trailing dot, case-insensitive) equals +> `.cfargotunnel.com`, return nil — the record already exists and +> is correct. +> - Otherwise return an error that states what the domain currently +> resolves to and tells the user to delete the conflicting record in +> the Cloudflare dashboard and re-run. +> - Extract the pure comparison (given a looked-up CNAME and a tunnel id, +> is it correct?) into a testable function and table-test it (matching, +> trailing dot, different tunnel id, empty). +> - Note: the caller `runNew` in `cmd/seploy/new.go` currently calls +> `tunnel.RouteDNS(a.Domain)` without an id; obtain the id there via +> the existing `tunnel.Ensure()`. +> 2. In `cmd/seploy/doctor.go`, strengthen the tunneled-app check: instead +> of only "domain resolves", also verify via `LookupCNAME` that the +> domain's CNAME target is `.cfargotunnel.com` for this +> server's tunnel (get the id with `tunnel.Ensure()` — but only when at +> least one app is tunneled, and tolerate lookup failure by falling back +> to the resolve-only check with a note). The failure hint should name +> the expected CNAME target. +> 3. Update README's Cloudflare Tunnel and Doctor sections accordingly. +> +> Acceptance: `make test` passes. A domain with a conflicting A record now +> produces a hard error naming the conflict instead of a false success. + +--- + +### Task 1.3 — `seploy restart` can leave the app down with no recovery + +`runRestart` (`cmd/seploy/restart.go`) force-removes the live container, then +starts a replacement on the *same fixed host port*. If `docker run` fails +(port stolen in the gap, bad env var) or the health check fails, nothing is +serving and seploy just exits with an error. Deploy already does this right +(start new on a fresh port → health check → rewrite Caddy → stop old); +restart should use the same start-then-switch pattern. + +**Prompt:** + +> You are working in the `seploy` Go repository (a Git-to-VPS deploy tool; +> module `github.com/Telep-IO/seploy`). Conventions: Go stdlib only, shell +> out via `exec.Command` with argument arrays, user-facing error messages as +> full sentences, run `make fmt` and `make test` when done, update README.md +> if documented behavior changes. +> +> Background: `seploy restart ` (`cmd/seploy/restart.go`) recreates the +> current container from its existing image to apply env-file changes. It +> currently does: `docker rm -f ` then `docker run` with the same +> fixed host port (`RunOpts.HostPort = cur.HostPort`). If the run or the +> health check fails, the app is down and there is no recovery path. By +> contrast, `runDeploy` (`cmd/seploy/deploy.go`) starts the new container +> under a different name on a Docker-chosen free localhost port +> (`RunOpts.HostPort = 0`, read back with `docker.HostPort`), health checks +> it (`healthCheck`), rewrites the Caddy site (`caddy.WriteSite` + +> `caddy.Reload`, see `siteFor` in deploy.go), and only then stops the old +> container. +> +> Task: rewrite `runRestart` to use the same start-then-switch pattern: +> +> 1. Start the replacement under a unique name — reuse the same-commit +> naming trick from deploy.go: `seploy---` — +> with `HostPort: 0` so Docker picks a free port. Remove any leftover +> container with that name first (`docker.Remove`). +> 2. Read the published port with `docker.HostPort`, run `healthCheck` +> (already in package main, deploy.go). On any failure: remove the new +> container and return an error saying the current release was left +> running and untouched. +> 3. On success: write the Caddy site for the new port, reload Caddy, update +> `st.Current.Container`, `st.Current.HostPort`, `st.Current.DeployedAt`, +> save state (`app.SaveState`), then stop and remove the old container +> (warn on failure, don't abort). +> 4. Keep the existing flock (`app.AcquireLock`), env-file, volume-mount and +> resource-limit handling exactly as-is. +> 5. Update the README "Restart" section: restarts are now zero-downtime and +> a failed restart leaves the old container serving. +> +> Acceptance: `make test` passes; `go build ./...` passes; a restart whose +> health check fails leaves the original container running (verify by +> reading the code path — the old container must not be touched before the +> new one passes its health check). + +--- + +### Task 1.4 — Atomic writes for state and config files + +`app.SaveState`, `App.Save`, `Config.Save`, and the env-file writer use plain +truncate-and-write (`os.WriteFile` / `O_TRUNC`). A crash, OOM-kill, or full +disk mid-write corrupts `state.json` or `app.toml`, after which seploy cannot +load the app at all. For a tool whose value is "state survives anything", +these must be atomic: write temp file in the same directory → fsync → rename. + +**Prompt:** + +> You are working in the `seploy` Go repository (a Git-to-VPS deploy tool; +> module `github.com/Telep-IO/seploy`). Conventions: Go stdlib only, +> user-facing error messages as full sentences, run `make fmt` and +> `make test` when done, add tests for new functions. +> +> Problem: seploy's on-disk state is written non-atomically. A crash or full +> disk mid-write corrupts the file and bricks the app config. Affected +> writers: +> +> - `SaveState` in `internal/app/app.go` (`state.json`, mode 0644) +> - `(*App).Save` in `internal/app/app.go` (`app.toml`, mode 0644, via +> `os.OpenFile` + toml encoder) +> - `(Config).Save` in `internal/config/config.go` (`config.toml`, 0644) +> - `SetEnv` in `internal/app/env.go` (env file, mode 0600) +> +> Task: +> +> 1. Create `internal/fsutil/fsutil.go` with +> `WriteFileAtomic(path string, data []byte, perm os.FileMode) error`: +> - `os.CreateTemp(filepath.Dir(path), ".-*")` in the same +> directory (rename must not cross filesystems), +> - write data, `f.Sync()`, `f.Close()`, `os.Chmod` to perm (CreateTemp +> uses 0600), +> - `os.Rename` onto path; on any error remove the temp file. +> 2. Convert all four writers to use it. For the TOML encoders, encode to a +> `bytes.Buffer` first, then write the buffer atomically. +> 3. Add a test in `internal/fsutil/fsutil_test.go`: writes a file, verifies +> content and permissions, overwrites an existing file, and verifies no +> stray temp files remain in the directory afterwards. +> +> Acceptance: `make test` passes; `grep -rn "os.WriteFile" internal/app +> internal/config` shows no remaining non-atomic writes of state.json, +> app.toml, config.toml, or the env file. + +--- + +### Task 1.5 — Reject duplicate domains at app creation + +`runNew` (`cmd/seploy/new.go`) never checks whether another app already uses +the entered domain. Two apps with the same domain produce two Caddy site +blocks for the same address, and `caddy reload` then fails **for every app on +the server** — one typo takes down config reloads globally. + +**Prompt:** + +> You are working in the `seploy` Go repository (a Git-to-VPS deploy tool; +> module `github.com/Telep-IO/seploy`). Conventions: Go stdlib only, +> user-facing error messages as full sentences, run `make fmt` and +> `make test` when done, add table-driven tests for new pure functions. +> +> Problem: `runNew` in `cmd/seploy/new.go` prompts for a domain +> (`ui.PromptRequired`) but never checks it against existing apps. Duplicate +> domains produce duplicate Caddy site blocks, which makes `caddy reload` +> fail server-wide. +> +> Task: +> +> 1. Add `FindByDomain(cfg config.Config, domain string) (*App, error)` to +> `internal/app` that iterates `List(cfg)` + `Load` and returns the app +> whose Domain matches case-insensitively (`strings.EqualFold`), or nil. +> Note `cmd/seploy/webhook.go` has a private `findAppByDomain` doing the +> same thing — move the logic into this new function and make the webhook +> code use it. +> 2. In `runNew`, after the domain prompt, loop: if `FindByDomain` returns +> an existing app, print "That domain is already used by app ." and +> re-prompt. Also validate the entered domain is plausible: lowercase it, +> strip any scheme or trailing slash or leading `*.` the user pasted, and +> reject values with spaces, `/`, or no dot, re-prompting with a short +> explanation. Put this normalization/validation in a pure function +> `NormalizeDomain(string) (string, error)` (a new `internal/app` or +> `internal/netcheck` helper — your choice) and table-test it (plain +> domain, `https://` prefix, trailing slash, uppercase, "no dot", +> embedded space). +> 3. Acceptance: `make test` passes; creating a second app with an existing +> app's domain is impossible via the interactive flow. + +--- + +### Task 1.6 — Webhook pushes are dropped when the deploy lock is held + +The webhook listener's `deployRunner` (`cmd/seploy/webhook.go:220-267`) +serializes deploys it starts itself, but the actual deploy is a subprocess +(`seploy deploy `) that takes a non-blocking flock +(`app.AcquireLock`). If a manual `sudo seploy deploy` is running when a push +arrives, the subprocess exits immediately with "another deployment … already +in progress", the runner logs the failure, and the push is lost forever. + +**Prompt:** + +> You are working in the `seploy` Go repository (a Git-to-VPS deploy tool; +> module `github.com/Telep-IO/seploy`). Conventions: Go stdlib only, +> user-facing error messages as full sentences, run `make fmt` and +> `make test` when done. +> +> Background: `seploy webhook serve` (`cmd/seploy/webhook.go`) receives +> GitHub push events and triggers deploys through `deployRunner`, which +> spawns `seploy deploy ` as a subprocess. Deploys take a non-blocking +> flock in `internal/app/lock.go` (`AcquireLock`, +> `syscall.Flock(..., LOCK_EX|LOCK_NB)`) and fail fast when it is held. So a +> push arriving during a *manual* deploy fails instantly and is dropped — +> the runner's pending/coalescing map only covers deploys it launched +> itself. +> +> Task: +> +> 1. Add `AcquireLockWait(cfg config.Config, name string, timeout time.Duration) (*Lock, error)` +> to `internal/app/lock.go`: retry the non-blocking flock every 2 seconds +> until it succeeds or the timeout elapses (keep it simple — a retry loop +> is fine and keeps the "stale lock dies with process" flock semantics). +> On timeout return the existing "another deployment of %q is already in +> progress" error. +> 2. Add a hidden `--wait-lock` duration flag to the `deploy` command +> (`cmd/seploy/deploy.go`), default 0 (current fail-fast behavior). When +> set, `runDeploy` uses `AcquireLockWait` with that timeout. +> 3. In `deployRunner.loop` (`cmd/seploy/webhook.go`), invoke the subprocess +> as `seploy deploy --wait-lock=10m` so webhook-triggered deploys +> queue behind manual ones instead of being dropped. +> 4. Add a test for the retry behavior in `internal/app/lock_test.go`: +> acquire a lock, spawn `AcquireLockWait` with a short timeout in a +> goroutine, verify it fails on timeout; then verify it succeeds when the +> first lock is released before the timeout. +> +> Acceptance: `make test` passes; a push received during a manual deploy +> results in a deploy that starts as soon as the manual one finishes. + +--- + +### Task 1.7 — Same-commit redeploy can kill a live container after state loss + +In `runDeploy` (`cmd/seploy/deploy.go:117-127`), when the computed container +name `seploy--` does not match `st.Current.Container`, seploy +assumes any container with that name is a leftover from a failed deploy and +force-removes it (`docker.Remove`) **before** the replacement has been built, +started, or health-checked. If `state.json` was ever lost or reset while that +container is live and serving traffic, this removes the live release and +causes an outage on what should be a safe redeploy. + +**Prompt:** + +> You are working in the `seploy` Go repository (a Git-to-VPS deploy tool; +> module `github.com/Telep-IO/seploy`). Conventions: Go stdlib only, +> user-facing error messages as full sentences, run `make fmt` and +> `make test` when done. +> +> Background: in `runDeploy` (`cmd/seploy/deploy.go`) the new container is +> named `seploy--`. If the current release (per `state.json`) has +> the same commit AND container name, a timestamp suffix is appended so the +> live container keeps serving while the replacement starts. Then the code +> runs `docker.Remove(container)` to clear "any leftover container with this +> name from a failed deploy". Flaw: if `state.json` was lost or reset +> (`st.Current == nil` or pointing elsewhere) while a container named +> `seploy--` is live, that `docker rm -f` kills the live release +> before the replacement is health-checked. +> +> Task: make the leftover cleanup safe. +> +> 1. `internal/docker/docker.go` already has `IsRunning(name string) bool`. +> Add `Exists(name string) bool` (via `docker inspect`, any state). +> 2. In `runDeploy`, replace the unconditional cleanup: if a container with +> the target name exists and `IsRunning` reports true — regardless of +> what state.json says — treat it like the same-release case: append the +> timestamp suffix to the new container's name and leave the running one +> alone (it will be stopped and pruned after the switch, via the existing +> `old`/`pruneReleases` path only if state knows about it; if state does +> NOT know about it, print a warning naming the container and telling the +> user it was left running and can be removed with `docker rm -f` once +> they confirm the new release is serving). If it exists but is not +> running, remove it as today. +> 3. Keep the change minimal and local to this block; do not restructure the +> rest of the deploy pipeline. +> +> Acceptance: `make test` and `go build ./...` pass; with a live container +> named `seploy--` and empty state, a deploy of the same commit +> never removes the running container before the new one passes health +> checks. + +--- + +## Phase 2 — Hardening + +### Task 2.1 — Pin and checksum-verify the cloudflared download; add `seploy tunnel upgrade` + +`tunnel.Install` (`internal/tunnel/tunnel.go:63-100`) downloads +`releases/latest/download/cloudflared-linux-` from GitHub with no +version pinning and no checksum verification, then installs it to +`/usr/local/bin` as root. A compromised or corrupted download runs as root +forever. Cloudflared releases publish per-release checksums. + +**Prompt:** + +> You are working in the `seploy` Go repository (a Git-to-VPS deploy tool; +> module `github.com/Telep-IO/seploy`). Conventions: Go stdlib only, +> user-facing error messages as full sentences, run `make fmt` and +> `make test` when done, add table-driven tests for new pure functions, +> update README.md for new commands. +> +> Background: `Install` in `internal/tunnel/tunnel.go` downloads the latest +> cloudflared binary from +> `https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-` +> with no pinned version and no checksum verification, and installs it to +> `/usr/local/bin/cloudflared` (root). +> +> Task: +> +> 1. Pin a version: add `const Version = "2026.6.0"` (check +> https://github.com/cloudflare/cloudflared/releases for the actual +> latest stable tag and use that) and download from +> `https://github.com/cloudflare/cloudflared/releases/download//cloudflared-linux-`. +> 2. Verify integrity: each release publishes SHA256 sums per asset. Embed a +> map of `arch -> expected sha256` for the pinned version (fetch the real +> values from the release's checksum file — for cloudflared each binary +> asset has a `.sha256` sidecar file, e.g. +> `cloudflared-linux-amd64.sha256`; if you cannot reach the network, +> leave clearly-marked `TODO-FILL` placeholders and instead download the +> `.sha256` sidecar at install time over HTTPS and verify against it — +> the sidecar approach is acceptable as the primary mechanism too, and +> avoids hardcoding). Compute sha256 of the downloaded bytes +> (`crypto/sha256`) before the rename into place; on mismatch delete the +> temp file and return an error stating expected vs got. +> 3. Extract the pure verification (given bytes/hash + expected hex, ok?) +> into a testable function with a table test. +> 4. Add `seploy tunnel upgrade` in `cmd/seploy/tunnel.go`: requires root +> (`system.RequireRoot`), downloads+verifies the pinned version to a temp +> file, atomically renames over the binary path, then restarts the +> cloudflared service if configured (`system.RestartService`, only when +> `tunnel.Configured()`); print the installed version by running +> `cloudflared --version` after. +> 5. README: document `seploy tunnel upgrade` and that installs are pinned +> and checksum-verified. +> +> Acceptance: `make test` passes; `Install` never renames an unverified +> binary into `/usr/local/bin`. + +--- + +### Task 2.2 — Move tunnel credentials out of `$HOME` into `/etc/cloudflared` + +`certPath()` (`internal/tunnel/tunnel.go:102-108`) resolves cert.pem under +`os.UserHomeDir()`. Under sudo, `$HOME` may be the invoking user's home, so +`cloudflared tunnel login` writes credentials to `/home//.cloudflared`, +and `WriteConfig` then points the root-run cloudflared **service** at files +inside a user's home directory — which breaks if that user is deleted, the +home is encrypted/unmounted, or perms change. + +**Prompt:** + +> You are working in the `seploy` Go repository (a Git-to-VPS deploy tool; +> module `github.com/Telep-IO/seploy`). Conventions: Go stdlib only, +> user-facing error messages as full sentences, run `make fmt` and +> `make test` when done, update README.md if behavior changes. +> +> Background: `internal/tunnel/tunnel.go` manages a Cloudflare Tunnel. +> `cloudflared tunnel login` writes `cert.pem` to `$HOME/.cloudflared/`; +> tunnel creation writes `.json` credentials next to it. +> `certPath()` uses `os.UserHomeDir()`, and `WriteConfig(id)` writes +> `/etc/cloudflared/config.yml` whose `credentials-file:` points into that +> home directory. Under sudo this can be a regular user's home — fragile for +> a root-run system service. +> +> Task: make `/etc/cloudflared` the canonical location. +> +> 1. In `WriteConfig(id)`: after locating `.json` (keep the current +> home-dir search as the source), copy it to +> `/etc/cloudflared/.json` with mode 0600 (directory 0755, owner +> root), and point `credentials-file:` at the copy. Also copy `cert.pem` +> to `/etc/cloudflared/cert.pem` (0600) so future cloudflared commands +> can find it. +> 2. Teach the credential *lookup* to prefer `/etc/cloudflared`: `LoggedIn` +> should check `/etc/cloudflared/cert.pem` first, then the home path. +> Commands that shell out to cloudflared for account-level operations +> (`lookupID`/`Ensure` via `tunnel list`, `tunnel create`, and +> `RouteDNS`) must set `TUNNEL_ORIGIN_CERT=/etc/cloudflared/cert.pem` in +> the child's environment when that file exists (cloudflared honors this +> env var), so they keep working after the original home directory +> disappears. +> 3. Handle re-runs: `seploy tunnel setup` is idempotent today; keep it so. +> If `/etc/cloudflared/cert.pem` already exists, `Login()` should treat +> the server as logged in. +> 4. README: note that credentials live in `/etc/cloudflared` after setup. +> +> Acceptance: `make test` and `go build ./...` pass; after setup, the +> cloudflared service and all seploy tunnel operations depend only on files +> under `/etc/cloudflared`. + +--- + +### Task 2.3 — HTTP timeouts: Caddy apt-key download and webhook server + +Two missing-timeout issues: (a) `SetupCaddyAptRepo` +(`internal/system/system.go:86-107`) uses bare `http.Get` — a hung connection +stalls `seploy init` forever; (b) `runWebhookServe` +(`cmd/seploy/webhook.go:283`) uses `http.ListenAndServe` with no server +timeouts. + +**Prompt:** + +> You are working in the `seploy` Go repository (a Git-to-VPS deploy tool; +> module `github.com/Telep-IO/seploy`). Conventions: Go stdlib only, run +> `make fmt` and `make test` when done. +> +> Task — two small hardening fixes: +> +> 1. `SetupCaddyAptRepo` in `internal/system/system.go` downloads Caddy's +> apt signing key with `http.Get` (no timeout). Use an `http.Client` with +> a 30-second timeout, matching the pattern used in +> `internal/netcheck/netcheck.go`. Also wrap the body read with +> `io.LimitReader` (1 MB is plenty for a GPG key) so a malicious/broken +> server can't balloon memory. +> 2. `runWebhookServe` in `cmd/seploy/webhook.go` uses +> `http.ListenAndServe`. Replace with an explicit `&http.Server{Addr: +> addr, Handler: mux, ReadHeaderTimeout: 10 * time.Second, ReadTimeout: +> 30 * time.Second, WriteTimeout: 30 * time.Second}` and call +> `srv.ListenAndServe()`. The listener is localhost-only behind Caddy, +> but Caddy proxies external connections to it, so slow-client protection +> still matters. +> +> Acceptance: `make test` and `go build ./...` pass; no bare `http.Get` or +> bare `http.ListenAndServe` remains in the two files. + +--- + +### Task 2.4 — Configurable health-check timeout + TCP liveness fallback + +`waitHealthy` (`cmd/seploy/deploy.go:243-276`) hardcodes a 30-second window; +apps that run DB migrations on boot always fail their first deploy. And when +no `healthcheck` path is configured, the check is just "sleep 2s, is the +process still running?" — a hung-but-alive app passes. + +**Prompt:** + +> You are working in the `seploy` Go repository (a Git-to-VPS deploy tool; +> module `github.com/Telep-IO/seploy`). Conventions: Go stdlib only, +> user-facing error messages as full sentences, run `make fmt` and +> `make test` when done, update README.md (it documents `app.toml` fields +> and the 30s health-check behavior in several places — update them all). +> +> Background: `waitHealthy` in `cmd/seploy/deploy.go` health-checks a newly +> started container before traffic switches to it. With +> `app.Healthcheck` set it polls `http://127.0.0.1:` for up to +> a hardcoded 30 seconds; without it, it sleeps 2 seconds and checks +> `docker.IsRunning`. +> +> Task: +> +> 1. Add `HealthTimeout string \`toml:"health_timeout"\`` to `App` +> (`internal/app/app.go`), a Go duration string (e.g. "90s", "3m"), +> empty = default. Add a method `HealthTimeoutOrDefault() time.Duration` +> returning 30s when empty and parsing with `time.ParseDuration` +> otherwise; invalid values should surface as an error at app-load time +> (`Load`) with a message naming the field and an example. Table-test the +> parsing (empty, "90s", "2m", garbage). +> 2. Use it in `waitHealthy` instead of the hardcoded 30s deadline. +> 3. Improve the no-healthcheck path: after the 2-second sleep and the +> IsRunning check, also attempt a TCP dial to `127.0.0.1:` +> (`net.DialTimeout`, 3s), retrying once per second until the same +> deadline. If nothing ever accepts a connection, fail with a message +> suggesting the app may be binding 127.0.0.1 inside the container +> instead of 0.0.0.0, and suggesting the `healthcheck` app.toml option. +> 4. README: document `health_timeout` in the app.toml example and the "App +> requirements" section; update the two places that say "up to 30s". +> +> Acceptance: `make test` passes; a deploy with `health_timeout = "2m"` +> polls for 2 minutes; the no-path check now requires an accepted TCP +> connection. + +--- + +### Task 2.5 — `seploy remove` should mention leftover Cloudflare DNS records + +Removing a tunneled app (`cmd/seploy/remove.go`) leaves the domain's CNAME +record pointing at the tunnel, with no mention. cloudflared's CLI cannot +delete DNS records, so seploy can't clean it up itself — but it must tell the +user. + +**Prompt:** + +> You are working in the `seploy` Go repository (a Git-to-VPS deploy tool; +> module `github.com/Telep-IO/seploy`). Conventions: Go stdlib only, run +> `make fmt` and `make test` when done, update README.md. +> +> Task: in `cmd/seploy/remove.go`, after an app is removed, if the app was +> exposed via Cloudflare Tunnel (`a.Tunneled()`, see `internal/app/app.go`), +> print a note that the DNS record was NOT removed (the cloudflared CLI +> cannot delete DNS records) and that the user should delete the CNAME +> record for `` in the Cloudflare dashboard (DNS → Records) if the +> domain is no longer used. Follow the existing style of the "Deploy key +> kept at …" and "Data volumes kept:" messages in the same file. Update the +> README "Remove an app" section to mention this. +> +> Acceptance: `go build ./...` and `make test` pass; removing a tunneled app +> prints the cleanup hint; removing a non-tunneled app does not. + +--- + +### Task 2.6 — Save state before stopping the old container (crash-safe deploy ordering) + +In `runDeploy` (`cmd/seploy/deploy.go:155-190`), Caddy is switched to the new +container, then state is saved, then the old container is stopped. If the +process dies between the Caddy reload and `SaveState`, live traffic is on a +container that `state.json` doesn't know about, and the next deploy's cleanup +logic reasons from stale state. (Complements Task 1.4's atomic writes and +Task 1.7's safe cleanup.) + +**Prompt:** + +> You are working in the `seploy` Go repository (a Git-to-VPS deploy tool; +> module `github.com/Telep-IO/seploy`). Conventions: Go stdlib only, run +> `make fmt` and `make test` when done. +> +> Background: `runDeploy` in `cmd/seploy/deploy.go` currently orders the +> deploy tail as: write Caddy site → reload Caddy → update in-memory state → +> `app.SaveState` → stop old container → prune. A crash between the Caddy +> reload and SaveState leaves traffic on a container state.json doesn't +> record. +> +> Task: narrow the inconsistency window by saving state *before* switching +> traffic, in two steps: +> +> 1. After the new container passes its health check and before +> `caddy.WriteSite`, build the new state (Current = new release, +> Previous per the existing sameRelease logic) and `app.SaveState` it. +> From this point state describes the *intended* end state; the new +> container is already running and healthy, so if we crash before the +> Caddy switch, `seploy deploy` re-run converges (same-commit redeploy +> path) and `seploy status`/`doctor` point at a live container either +> way. +> 2. If `caddy.WriteSite` or `caddy.Reload` then fails, roll the state file +> back: restore the previous State (keep a copy before mutating) and +> SaveState again, remove the new container, and return the existing +> "traffic was not switched" error semantics — the old release must +> remain both serving AND recorded as Current. Preserve the current +> behavior that the old container is only stopped after a successful +> switch, and keep all existing log output. +> +> Be careful with the `old := st.Current` / `sameRelease` bookkeeping — read +> the whole function first. Do not change `pruneReleases`. +> +> Acceptance: `make test` and `go build ./...` pass; in the failure path of +> `caddy.Reload`, `state.json` on disk records the OLD release as Current. + +--- + +## Phase 3 — Features + +### Task 3.1 — Post-deploy end-to-end verification through the real edge + +A deploy can "succeed" while the site is actually unreachable (dead +cloudflared service, DNS pointing elsewhere, Caddy misroute). After the +traffic switch, seploy should verify the app end-to-end by requesting +`https://` the way a user would, and report clearly if the edge path +is broken — without failing the deploy (the container itself is healthy). + +**Prompt:** + +> You are working in the `seploy` Go repository (a Git-to-VPS deploy tool; +> module `github.com/Telep-IO/seploy`). Conventions: Go stdlib only, +> user-facing messages as full sentences, run `make fmt` and `make test` +> when done, add table-driven tests for pure logic, update README.md. +> +> Background: `runDeploy` in `cmd/seploy/deploy.go` health-checks the new +> container on 127.0.0.1 before switching Caddy, but never verifies the +> public URL afterwards. Apps are served either directly by Caddy on +> 80/443, or through a Cloudflare Tunnel (`a.Tunneled()`, TLS at +> Cloudflare's edge). A dead cloudflared service or wrong DNS makes the +> deploy "succeed" while the site is down. +> +> Task: +> +> 1. Create `internal/netcheck/edge.go` with +> `CheckEdge(domain, healthPath string) error`. It GETs +> `https://` (path = healthcheck path if set, else `/`) +> with a 10-second-timeout client that does NOT follow redirects +> (`CheckRedirect: http.ErrUseLastResponse`). Success = any status < +> 500 (auth walls, redirects, 404s on `/` are all fine — we're testing +> reachability, not app semantics). Distinguish failure classes in the +> returned error: DNS resolution failure, TLS failure, connection +> refused/timeout, HTTP >= 500 — each with a one-line hint (e.g. 502/503 +> on a tunneled app → "check: systemctl status cloudflared"; DNS failure +> → "the domain does not resolve yet"). Extract the status-code +> classification into a pure function and table-test it. +> 2. In `runDeploy`, after the final summary block, call it and print either +> ` edge: https:// reachable (HTTP )` or a WARNING +> paragraph with the hint. Never return an error from this check — the +> release is healthy and serving locally; the edge problem is +> environmental. Retry for up to 30 seconds (once per 3s) before +> declaring failure, since Caddy may still be fetching a certificate for +> first-time non-tunneled deploys — in that specific case (TLS failure, +> non-tunneled app, first deploy) soften the message to "Caddy may still +> be provisioning the TLS certificate; check again shortly." +> 3. Also call `CheckEdge` from `seploy doctor` (`cmd/seploy/doctor.go`) for +> each deployed app as an additional per-app check line. +> 4. README: mention the post-deploy edge check in "Deploy an app" and +> "Doctor". +> +> Acceptance: `make test` passes; deploy output includes an edge line; +> doctor gains a per-app edge check. + +--- + +### Task 3.2 — Token-based (non-interactive) tunnel setup + +`seploy tunnel setup` requires an interactive browser login, which blocks +scripted VPS provisioning — exactly the "install on any VPS" use case. +Cloudflare supports remotely-managed tunnels: the dashboard (Zero Trust → +Networks → Tunnels) issues a connector token, and +`cloudflared service install ` runs the tunnel with no cert.pem on the +machine. DNS records for token tunnels are managed in the dashboard, not via +`cloudflared tunnel route dns`. + +**Prompt:** + +> You are working in the `seploy` Go repository (a Git-to-VPS deploy tool; +> module `github.com/Telep-IO/seploy`). Conventions: Go stdlib only, +> user-facing error messages as full sentences, run `make fmt` and +> `make test` when done, add table-driven tests for pure functions, update +> README.md. +> +> Background: seploy exposes apps through a Cloudflare Tunnel. Current flow +> (`cmd/seploy/tunnel.go`, `internal/tunnel/tunnel.go`): interactive +> `cloudflared tunnel login` → `tunnel create seploy` → write +> `/etc/cloudflared/config.yml` with a catch-all `http://localhost:80` +> ingress → install service. Apps marked `expose = "tunnel"` +> (`internal/app/app.go`, `Tunneled()`) get plain-HTTP Caddy sites and a DNS +> record via `tunnel.RouteDNS`. +> +> Task: add a non-interactive alternative using a remotely-managed tunnel +> token. +> +> 1. `seploy tunnel setup --token ` (and env var `SEPLOY_TUNNEL_TOKEN` +> as fallback): skip login/create/WriteConfig entirely; run +> `cloudflared service install ` (systemd; for OpenRC write the +> existing openrcScript but with `command_args="tunnel run --token +> "` — beware: the token then appears in the init script, chmod it +> 0700 and say so). Still call `tunnel.Install()` first to fetch the +> binary. +> 2. Record the mode: write `/etc/cloudflared/seploy-tunnel.toml` containing +> `mode = "token"` or `mode = "local"` (written in both setup paths). Add +> `tunnel.Mode() string` reading it, defaulting to "local" when the +> legacy `ConfigPath` config exists and "" when nothing is set up. Update +> `Configured()` to return true in token mode too (config.yml does not +> exist in token mode — use the mode file and/or service presence). +> 3. Token-mode consequences (guard these code paths): `RouteDNS` cannot +> work without cert.pem. In `cmd/seploy/new.go`, when the app is tunneled +> and `tunnel.Mode() == "token"`, instead of calling RouteDNS print +> instructions: add a public hostname for `` → service +> `http://localhost:80` in Zero Trust → Networks → Tunnels → +> → Public Hostname. Similarly `seploy tunnel status` +> should print the mode. +> 4. Important: for remotely-managed tunnels the ingress rules come from the +> dashboard, so the "Public Hostname" instruction above IS the routing — +> make the printed instructions exact and copy-pasteable. +> 5. README: new subsection "Tunnel setup without a browser (token)" +> explaining where to create the tunnel in the dashboard, what to point +> the public hostname at (`http://localhost:80`), and the one-liner +> `sudo seploy tunnel setup --token `. +> +> Acceptance: `make test` and `go build ./...` pass; token setup never +> attempts interactive login; app creation in token mode prints dashboard +> instructions instead of calling RouteDNS. + +--- + +### Task 3.3 — `seploy rollback --to ` using release history + +Rollback today only toggles between Current and Previous. With Task 1.1's +`History` (images kept for the last `keep_releases` releases), rollback can +target any recorded release. **Depends on Task 1.1.** + +**Prompt:** + +> You are working in the `seploy` Go repository (a Git-to-VPS deploy tool; +> module `github.com/Telep-IO/seploy`). Conventions: Go stdlib only, +> user-facing error messages as full sentences, run `make fmt` and +> `make test` when done, update README.md. +> +> Precondition: `app.State` (`internal/app/app.go`) has a `History +> []*Release` field (most recent first) maintained by deploys, and image +> pruning keeps images for every release in History. If that field does not +> exist yet, STOP and report that Task 1.1 of docs/HARDENING-PLAN-1.md must +> land first. +> +> Background: `seploy rollback ` (`cmd/seploy/rollback.go`) swaps +> Current and Previous: start previous container (or re-run its kept image), +> health check, switch Caddy, stop current. Read `runRollback` fully first. +> +> Task: +> +> 1. Add a `--to ` flag to the rollback command. Without it, behavior +> is unchanged. With it, find the release in History (also check Current +> and Previous) whose `Commit` equals the given short SHA (also accept +> any unambiguous prefix); error listing available SHAs+dates when not +> found or ambiguous. +> 2. Deploy that release through the same start → health check → switch → +> stop pipeline `runRollback` already uses, factoring the common logic +> into a helper `switchToRelease(cfg, a, rel, st)` shared by both paths. +> The target's container may be long gone — the re-run-from-image branch +> already handles that; the image is guaranteed by History pruning. After +> a `--to` rollback set `st.Previous` to what was Current, `st.Current` +> to the target, and leave History untouched. +> 3. Add `seploy releases ` (new small command file, model it on +> `cmd/seploy/status.go`): lists Current/Previous/History releases with +> commit, deployed-at, and whether the image still exists +> (`docker image inspect` — add a tiny `docker.ImageExists` helper). +> 4. README: document both under the Rollback section. +> +> Acceptance: `make test` passes; `rollback --to` with an unknown SHA prints +> the available ones; plain rollback behavior is byte-for-byte unchanged. + +--- + +### Task 3.4 — Disk-space preflight and build timeout before `docker build` + +A runaway build on a small VPS is the most common way deploy tools brick a +box. Doctor already checks free disk (`diskFree`, `cmd/seploy/doctor.go`); +deploy should do the same *before* building, and the build should have a +generous timeout so a wedged build can't hold the deploy lock forever. + +**Prompt:** + +> You are working in the `seploy` Go repository (a Git-to-VPS deploy tool; +> module `github.com/Telep-IO/seploy`). Conventions: Go stdlib only, +> user-facing error messages as full sentences, run `make fmt` and +> `make test` when done, update README.md if documented behavior changes. +> +> Task — two guards around `docker build` in the deploy pipeline: +> +> 1. Disk preflight: `cmd/seploy/doctor.go` has a private `diskFree(path) +> (uint64, bool)` using `syscall.Statfs`. Move it to +> `internal/system/system.go` as `DiskFree(path string) (uint64, error)` +> and update doctor. In `runDeploy` (`cmd/seploy/deploy.go`), before +> `docker.Build`, check free space on the Docker data root — use `docker +> info --format '{{.DockerRootDir}}'` (add a small +> `docker.RootDir() (string, error)` helper in +> `internal/docker/docker.go`, falling back to `/var/lib/docker`). If +> less than 2 GiB free, abort with an error stating the free space and +> suggesting `docker system prune` — before any container is touched, so +> this is completely safe. If the check itself errors, proceed silently +> (best-effort). +> 2. Build timeout: change `docker.Build` to accept a +> `timeout time.Duration` and run the build with +> `exec.CommandContext(context.WithTimeout(...))`. On timeout, return an +> error saying the build exceeded the limit. Add a `build_timeout` field +> to `app.toml` (`App` struct in `internal/app/app.go`, Go duration +> string, default "30m", validated at load like other duration fields if +> any exist — otherwise validate in a `BuildTimeoutOrDefault()` method +> mirroring how you'd parse `time.ParseDuration`, with a table test). +> Wire it through the single `docker.Build` call site. +> 3. README: document `build_timeout` in the app.toml example, and the disk +> preflight in the Deploy section. +> +> Acceptance: `make test` passes; a deploy on a nearly-full disk fails +> before building; `docker.Build` is context-bounded. + +--- + +### Task 3.5 — `seploy deploy --all` and `--no-cache` + +Two small quality-of-life flags: redeploy every app (after a server +migration, base-image security update, or seploy upgrade), and force a build +without Docker's cache. + +**Prompt:** + +> You are working in the `seploy` Go repository (a Git-to-VPS deploy tool; +> module `github.com/Telep-IO/seploy`). Conventions: Go stdlib only, +> user-facing error messages as full sentences, run `make fmt` and +> `make test` when done, update README.md. +> +> Background: `cmd/seploy/deploy.go` defines `deployCmd` +> (`seploy deploy `, `cobra.ExactArgs(1)`) calling `runDeploy(cfg, a)`, +> which builds via `docker.Build` in `internal/docker/docker.go`. Apps are +> enumerated with `app.List(cfg)` + `app.Load` (see `cmd/seploy/list.go` +> for the pattern). +> +> Task: +> +> 1. `--no-cache` flag: plumb a boolean from the deploy command through +> `runDeploy` into `docker.Build`, which appends `--no-cache` to the +> build args when set. Prefer adding a `BuildOpts` struct or an explicit +> parameter over a package-level variable — check `docker.Build`'s +> current signature and all call sites (deploy, and possibly others) and +> keep them compiling. +> 2. `--all` flag: `seploy deploy --all` (mutually exclusive with a +> positional arg — enforce with a clear error; change `Args` validation +> accordingly). It loads every app from `app.List`, deploys each +> sequentially in sorted order, prints a `=== ===` header per app, +> continues past per-app failures, and at the end prints a summary +> (`deployed: N, failed: M (names)`); exit non-zero if any failed. Each +> app's deploy still takes its own per-app lock via the existing +> `runDeploy`. +> 3. Webhook note: `cmd/seploy/webhook.go` spawns `seploy deploy ` as +> a subprocess — do not break that invocation. +> 4. README: document both flags in the Redeploy section. +> +> Acceptance: `make test` passes; `seploy deploy --all` with zero apps says +> so and exits 0; `seploy deploy app --all` errors clearly. + +--- + +## Execution checklist + +| # | Task | Phase | Depends on | +|---|------|-------|------------| +| 1.1 | Wire up `keep_releases` | 1 | — | +| 1.2 | RouteDNS conflict detection + doctor CNAME check | 1 | — | +| 1.3 | Zero-downtime, fail-safe restart | 1 | — | +| 1.4 | Atomic state/config writes | 1 | — | +| 1.5 | Duplicate-domain guard | 1 | — | +| 1.6 | Webhook lock-wait (no dropped pushes) | 1 | — | +| 1.7 | Safe same-name container cleanup | 1 | — | +| 2.1 | Pinned + verified cloudflared, `tunnel upgrade` | 2 | — | +| 2.2 | Tunnel credentials in /etc/cloudflared | 2 | — | +| 2.3 | HTTP timeouts (apt key, webhook server) | 2 | — | +| 2.4 | Configurable health timeout + TCP check | 2 | — | +| 2.5 | Remove: tunnel DNS cleanup hint | 2 | — | +| 2.6 | Crash-safe deploy state ordering | 2 | 1.4 recommended first | +| 3.1 | Post-deploy edge verification | 3 | — | +| 3.2 | Token-based tunnel setup | 3 | 2.2 recommended first | +| 3.3 | `rollback --to ` + `seploy releases` | 3 | 1.1 | +| 3.4 | Disk preflight + build timeout | 3 | — | +| 3.5 | `deploy --all`, `--no-cache` | 3 | — | + +Tasks touching `cmd/seploy/deploy.go` (1.1, 1.6, 1.7, 2.4, 2.6, 3.1, 3.4, +3.5) will conflict textually if run fully in parallel — land them +sequentially or rebase between merges. Everything else is disjoint. diff --git a/docs/SIMILAR-UTILITIES.md b/docs/SIMILAR-UTILITIES.md new file mode 100644 index 0000000..7776f4b --- /dev/null +++ b/docs/SIMILAR-UTILITIES.md @@ -0,0 +1,331 @@ +# Similar Utilities — the seploy family + +seploy exists because deploying a Dockerized app to a VPS used to mean +stitching together git, Docker, a reverse proxy, TLS, health checks and +rollbacks by hand. seploy collapses that into one static binary you run +once. That same philosophy — **take a fiddly multi-tool ops workflow that +every self-hoster re-derives from scratch, and collapse it into one +reliable binary** — applies to a lot more than deployment. + +This document collects the candidate utilities, why each one is needed, +and what it would solve. Roughly ordered by how much daily pain each +removes. + +--- + +## 1. `sebak` — backups with automatic restore drills + +### The problem + +Backups are the most common silent failure in self-hosting. Tools like +restic and borg are excellent at *storing* backups, but they leave the +hard parts to you: + +- Figuring out **what** to back up (Docker volumes, databases running in + containers, env files, app state) — every app is a special snowflake. +- Getting **consistent** database backups (you can't just copy Postgres's + data directory while it's running; you need `pg_dump` or a snapshot). +- Actually **verifying** the backups. Almost nobody restores a backup + until the day they desperately need to — and that's the day they + discover the cron job silently died eight months ago, or the dump is + of an empty database, or the encryption key is on the disk that just + failed. + +### What it would do + +- `sebak init` inspects the box and auto-detects what matters: Docker + volumes, Postgres/MySQL/SQLite inside containers, and — because seploy + standardized the layout — everything under + `/var/lib/seploy/apps/` (checkout config, env files, runtime + state). +- Snapshots to cheap object storage (Backblaze B2, Cloudflare R2, any + S3-compatible endpoint), encrypted client-side, with sane retention + (daily/weekly/monthly) by default. +- Databases are dumped properly (`pg_dump` via `docker exec`), not + file-copied. +- **The killer feature: automated restore drills.** On a schedule, sebak + pulls the latest backup, restores it into a throwaway container, + runs a basic sanity check (database accepts connections, row counts + are nonzero, files match manifests), and reports the result. A backup + that has never been restored is a hope, not a backup. + +### What it solves + +- Eliminates the "I thought I had backups" catastrophe — the single most + expensive failure mode for a self-hoster. +- Removes the per-app research project of "how do I back *this* one up." +- Pairs directly with seploy: apps deployed by seploy are known + quantities, so backup configuration can be near zero. + +--- + +## 2. `sewatch` — uptime monitoring that diagnoses, not just alerts + +### The problem + +Every monitoring tool tells you **that** your site is down. Almost none +tell you **why**. You get "HTTP 503" at 2am, and then you do the same +ritual every time: SSH in, `docker ps`, check Caddy, check disk space, +check whether the TLS cert expired, check whether cloudflared is still +connected. That ritual is completely mechanical — which means a tool +should do it. + +Hosted monitors (UptimeRobot, etc.) also can't see inside your box, and +self-hosted dashboards (Uptime Kuma) monitor from the same machine +that's failing, which defeats the point. + +### What it would do + +- External HTTP checks against your domains (from outside the box, so a + dead server is actually detected). +- When a check fails, sewatch runs the diagnosis you'd do by hand — + locally or over SSH: + - Is the container running? Did it OOM or crash-loop? + - Is Caddy up and is the site config present? + - Did the disk fill up? (This one causes a shocking fraction of + outages.) + - Is the TLS cert valid / about to expire? + - Is the cloudflared tunnel connected? + - Recent container logs, last N lines. +- Sends **one** notification with the actual cause: not "site down" but + "site down: container exited (OOM-killed), disk at 97%." Delivery via + ntfy, Telegram, email, or a webhook. +- Because seploy owns the app layout, sewatch knows exactly which + container, port, and Caddy config belong to each domain — zero + configuration. + +### What it solves + +- Cuts the 2am debugging ritual from 20 minutes of groggy SSHing to + reading one message. +- Catches slow-burn failures (disk filling, cert expiry, memory creep) + *before* they take the site down. +- Closes the loop on seploy deployments: deploy, then know it stays up. + +--- + +## 3. `sedb` — managed-database feel on a $5 VPS + +### The problem + +Managed databases (RDS, Neon, PlanetScale) are great but cost more than +the VPS running your entire app. Running your own Postgres in Docker is +easy in theory, but in practice it's a 20-minute checklist you redo for +every project: pick an image and version, bind it to localhost only, +generate a password, pick a port, wire up the connection string, set up +dumps, remember tuning flags, and hope you didn't leave 5432 open to the +internet (a genuinely common breach vector). + +### What it would do + +- `sedb create postgres myapp` → a Postgres container bound to + **127.0.0.1** on a free port, strong generated credentials, and a + connection string printed once (and written to the app's env file if + it's a seploy app). +- Automated dumps on a schedule, feeding into sebak's storage and + restore drills. +- `sedb branch myapp dev` → a copy-on-write (or dump/restore) clone of + production data for local development or testing a migration, then + `sedb drop` when done. +- `sedb upgrade myapp 16` → handles the dump → new-version container → + restore → switchover dance that everyone dreads. +- Support for Postgres first, then MySQL/MariaDB and managed SQLite + (Litestream-style replication). + +### What it solves + +- The recurring 20-minute setup tax on every new project. +- The "exposed database port" security foot-gun, structurally. +- Major-version upgrades, the thing people defer for years out of fear. +- Realistic dev/test data without hand-rolled dump scripts. + +--- + +## 4. `sesec` — secrets and env distribution + +### The problem + +Every seploy app has an env file at `/var/lib/seploy/apps//env`. +How did the secrets get there? For most self-hosters the honest answer +is "I SSHed in and pasted them," which means secrets exist only on the +server, aren't versioned, can't be recovered if the box dies, and are a +manual step in every deployment. The existing tools (Vault, sops) +either require running a service or demand nontrivial key-management +setup. + +### What it would do + +- Secrets live **encrypted in your git repo** (age-encrypted, sops-like, + but with zero configuration — `sesec init` generates the keypair, the + private key lives on machines you authorize). +- `sesec edit myapp` decrypts to a temp file, opens `$EDITOR`, + re-encrypts on save. Diffs are meaningful at the key level. +- `sesec push` (or a seploy deploy hook) decrypts and syncs to + `/var/lib/seploy/apps//env` on the target box. +- Audit trail for free: it's git history. + +### What it solves + +- "How do I get secrets onto the box without pasting them over SSH." +- Secrets become versioned, recoverable, and reviewable instead of + living only on one machine's disk. +- Removes the last manual step in the repo → running app pipeline. + +--- + +## 5. `sevps` — one-shot VPS hardening and baseline + +### The problem + +Every fresh VPS starts insecure and unconfigured: root SSH login +enabled, password auth on, no firewall, no fail2ban, no automatic +security updates, no swap, default sysctls. Everyone knows the checklist; +everyone re-derives it from a half-remembered gist every time they +provision a server, and everyone forgets a step. This is the natural +step *before* `seploy init`, and today it's the least reliable one. + +### What it would do + +- `sevps init` on a fresh box: + - Creates a non-root user with your SSH key; disables root login and + password auth in sshd. + - Configures a firewall (nftables/ufw): 22/80/443 only by default. + - Installs fail2ban (or nftables rate limiting) for SSH. + - Enables unattended security upgrades (apt/apk equivalents). + - Sets up swap sized to RAM, sane sysctls, correct timezone/NTP. +- `sevps audit` re-checks the baseline anytime and prints a report — + drift detection for the config you set six months ago. +- Same portability targets as seploy: Ubuntu, Debian, Alpine; systemd + and OpenRC. + +### What it solves + +- Turns 45 error-prone minutes of checklist archaeology into one + command. +- Removes the window where a fresh box sits on the internet with root + password auth enabled (bots start brute-forcing within minutes of + boot). +- Gives you a provable, re-checkable baseline instead of "I think I + hardened this one." + +--- + +## 6. `sedns` — DNS as code for Cloudflare + +### The problem + +DNS is the last manual, click-ops step in "repo → live URL." You deploy +with seploy, then tab over to the Cloudflare dashboard to add an A +record or CNAME by hand. There's no record of what you did, no way to +diff, and no way to rebuild your DNS if you fat-finger a zone. +Terraform can do this but is a sledgehammer for a personal zone. + +### What it would do + +- A small declarative file (in the repo, or one per zone) listing + records. +- `sedns plan` shows a diff against the live zone; `sedns apply` makes + it so via the Cloudflare API. `sedns import` bootstraps the file from + an existing zone. +- seploy integration: deploying an app can create/verify its DNS record + automatically, making "git repo → live HTTPS URL" genuinely one + command with zero dashboard visits. + +### What it solves + +- DNS becomes versioned, diffable, and rebuildable like everything else. +- Eliminates the "wait, why is this record pointing there?" archaeology. +- Closes the final manual gap in the deployment pipeline. + +--- + +## 7. `sestatus` — static status page generator + +### The problem + +Once you run services other people (or future-you) depend on, "is it +down for everyone or just me?" needs an answer that doesn't require SSH. +Hosted status pages are absurdly expensive for what they are, and most +self-hosted ones are full web apps — another service to deploy, secure, +and keep up, which can itself go down with the thing it's reporting on. + +### What it would do + +- Reads check history from sewatch and renders a **static** HTML status + page: current state, uptime percentages, incident history. +- Static output means it can be hosted anywhere that can't go down with + your VPS — Cloudflare Pages, GitHub Pages, an object-storage bucket — + or, pleasingly recursively, deployed as a tiny app with seploy. +- Optional manual incident notes ("maintenance tonight 10pm"). + +### What it solves + +- A trustworthy answer to "is it up?" that survives the outage it's + reporting. +- Zero additional attack surface or runtime — it's just files. + +--- + +## 8. `semail` — reliable transactional email for side projects + +### The problem + +Every side project eventually needs to send email — signup +confirmations, password resets, alerts. And every project re-implements +the same fragile version: an SDK call to SES/Resend/Mailgun inline in a +request handler, no queue, no retries, no visibility. When the provider +hiccups, the email is just gone, and you find out when a user says they +never got their reset link. + +### What it would do + +- One binary running on the VPS exposing a dead-simple localhost HTTP + API (or SMTP shim): apps POST `{to, subject, body}` and forget. +- semail persists the message in a local queue (SQLite), delivers via + your configured provider (SES, Resend, Mailgun, or raw SMTP), and + retries with backoff on failure. +- A log of every message: sent, retrying, bounced — one place to answer + "did the email actually go out?" +- Shared by every app on the box: configure the provider and DKIM/SPF + once, not per project. + +### What it solves + +- Emails stop silently vanishing when a provider blips. +- Every app drops its bespoke email code for one HTTP call. +- Provider configuration (keys, domains, DNS records for + DKIM/SPF/DMARC — which sedns could manage) happens once per server. + +--- + +## How they compose + +The real value is that these aren't eight independent tools — they +compound, because seploy standardized where apps live and how they run: + +``` +sevps init # harden the fresh box +seploy init # install the deploy stack +sesec push # secrets arrive encrypted, land as env files +seploy # app is live behind Caddy/cloudflared +sedns apply # ...and its DNS record exists (or automatic) +sedb create ... # its database, localhost-only, backed up +sebak init # everything snapshotted, restore-drilled +sewatch init # watched from outside, diagnosed from inside +sestatus # and publicly accounted for +``` + +Deploy, secure, watch, back up: together this is a self-hosted PaaS +without the platform — each piece a single static Go binary, useful +alone, better together. + +## Suggested build order + +1. **sebak** or **sewatch** first — both address failure modes you only + discover when it's too late, and both get smarter for free because + seploy standardized the app layout. +2. **sevps** — small scope, immediately useful, and the natural step + before `seploy init` in every new-server story. +3. **sedb** and **sesec** — remove the remaining per-project setup tax. +4. **sedns**, **sestatus**, **semail** — quality-of-life multipliers + once the core trio (deploy / watch / back up) exists. diff --git a/internal/app/app.go b/internal/app/app.go index 3f80198..5a46322 100644 --- a/internal/app/app.go +++ b/internal/app/app.go @@ -3,6 +3,7 @@ package app import ( + "bytes" "encoding/json" "fmt" "os" @@ -14,6 +15,7 @@ import ( "github.com/BurntSushi/toml" "github.com/Telep-IO/seploy/internal/config" + "github.com/Telep-IO/seploy/internal/fsutil" ) // ExposeTunnel marks an app as exposed through the server's Cloudflare @@ -30,10 +32,18 @@ type App struct { Dockerfile string `toml:"dockerfile"` Port int `toml:"port"` Healthcheck string `toml:"healthcheck"` - SSHKey string `toml:"ssh_key"` - Expose string `toml:"expose"` // "" = Caddy + DNS, "tunnel" = Cloudflare Tunnel - Memory string `toml:"memory"` // optional docker --memory limit, e.g. "512m" - CPUs string `toml:"cpus"` // optional docker --cpus limit, e.g. "1.5" + // HealthTimeout is how long a deploy waits for the new container to + // become healthy, as a Go duration string (e.g. "90s", "3m"). Empty + // uses the default of 30s. + HealthTimeout string `toml:"health_timeout"` + // BuildTimeout is how long a deploy waits for `docker build` to finish, + // as a Go duration string (e.g. "10m", "1h"). Empty uses the default of + // 30m. + BuildTimeout string `toml:"build_timeout"` + SSHKey string `toml:"ssh_key"` + Expose string `toml:"expose"` // "" = Caddy + DNS, "tunnel" = Cloudflare Tunnel + Memory string `toml:"memory"` // optional docker --memory limit, e.g. "512m" + CPUs string `toml:"cpus"` // optional docker --cpus limit, e.g. "1.5" // Volumes are persistent named volumes, "name:/path/in/container". // Data in them survives redeploys (e.g. a SQLite database). Volumes []string `toml:"volumes"` @@ -42,6 +52,44 @@ type App struct { // Tunneled reports whether the app is exposed through a Cloudflare Tunnel. func (a *App) Tunneled() bool { return a.Expose == ExposeTunnel } +// defaultHealthTimeout is how long a deploy waits for the new container to +// become healthy when health_timeout is unset in app.toml. +const defaultHealthTimeout = 30 * time.Second + +// HealthTimeoutOrDefault returns the configured health check timeout, +// defaulting to 30s when HealthTimeout is empty. HealthTimeout is validated +// at Load time, so a parse failure here (which should not happen for an app +// loaded via Load) also falls back to the default. +func (a *App) HealthTimeoutOrDefault() time.Duration { + if a.HealthTimeout == "" { + return defaultHealthTimeout + } + d, err := time.ParseDuration(a.HealthTimeout) + if err != nil { + return defaultHealthTimeout + } + return d +} + +// defaultBuildTimeout is how long a deploy waits for `docker build` to +// finish when build_timeout is unset in app.toml. +const defaultBuildTimeout = 30 * time.Minute + +// BuildTimeoutOrDefault returns the configured build timeout, defaulting to +// 30m when BuildTimeout is empty. BuildTimeout is validated at Load time, so +// a parse failure here (which should not happen for an app loaded via Load) +// also falls back to the default. +func (a *App) BuildTimeoutOrDefault() time.Duration { + if a.BuildTimeout == "" { + return defaultBuildTimeout + } + d, err := time.ParseDuration(a.BuildTimeout) + if err != nil { + return defaultBuildTimeout + } + return d +} + var volumeNameRe = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*$`) // VolumeMounts resolves the app's volumes config into docker -v values: @@ -75,6 +123,10 @@ type Release struct { type State struct { Current *Release `json:"current"` Previous *Release `json:"previous"` + // History holds past releases, most recent first, truncated to the + // global keep_releases setting on every successful deploy. Older + // state.json files predate this field and simply load with it empty. + History []*Release `json:"history,omitempty"` } // Dir returns the app's directory under the data dir. @@ -130,6 +182,16 @@ func Load(cfg config.Config, name string) (*App, error) { if err := toml.Unmarshal(data, &a); err != nil { return nil, fmt.Errorf("parse %s: %w", ConfigPath(cfg, name), err) } + if a.HealthTimeout != "" { + if _, err := time.ParseDuration(a.HealthTimeout); err != nil { + return nil, fmt.Errorf("invalid health_timeout %q in app config: want a Go duration like \"90s\" or \"3m\"", a.HealthTimeout) + } + } + if a.BuildTimeout != "" { + if _, err := time.ParseDuration(a.BuildTimeout); err != nil { + return nil, fmt.Errorf("invalid build_timeout %q in app config: want a Go duration like \"10m\" or \"1h\"", a.BuildTimeout) + } + } return &a, nil } @@ -141,12 +203,11 @@ func (a *App) Save(cfg config.Config) error { if err := os.MkdirAll(Dir(cfg, a.Name), 0o755); err != nil { return err } - f, err := os.OpenFile(ConfigPath(cfg, a.Name), os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o644) - if err != nil { + var buf bytes.Buffer + if err := toml.NewEncoder(&buf).Encode(a); err != nil { return err } - defer f.Close() - return toml.NewEncoder(f).Encode(a) + return fsutil.WriteFileAtomic(ConfigPath(cfg, a.Name), buf.Bytes(), 0o644) } // LoadState reads an app's runtime state, returning an empty state when @@ -172,7 +233,26 @@ func SaveState(cfg config.Config, name string, st *State) error { if err != nil { return err } - return os.WriteFile(StatePath(cfg, name), append(data, '\n'), 0o644) + return fsutil.WriteFileAtomic(StatePath(cfg, name), append(data, '\n'), 0o644) +} + +// FindByDomain returns the app whose Domain matches (case-insensitively), +// or nil if none does. +func FindByDomain(cfg config.Config, domain string) (*App, error) { + names, err := List(cfg) + if err != nil { + return nil, err + } + for _, n := range names { + a, err := Load(cfg, n) + if err != nil { + continue + } + if strings.EqualFold(a.Domain, domain) { + return a, nil + } + } + return nil, nil } // List returns the names of all configured apps, sorted. diff --git a/internal/app/build_timeout_test.go b/internal/app/build_timeout_test.go new file mode 100644 index 0000000..78637db --- /dev/null +++ b/internal/app/build_timeout_test.go @@ -0,0 +1,65 @@ +package app + +import ( + "testing" + "time" + + "github.com/Telep-IO/seploy/internal/config" +) + +func TestBuildTimeoutOrDefault(t *testing.T) { + cases := []struct { + in string + want time.Duration + wantErr bool + }{ + {"", 30 * time.Minute, false}, + {"10m", 10 * time.Minute, false}, + {"1h", time.Hour, false}, + {"garbage", 0, true}, + } + for _, c := range cases { + a := &App{BuildTimeout: c.in} + got := a.BuildTimeoutOrDefault() + if c.wantErr { + // Invalid durations should never reach BuildTimeoutOrDefault via + // Load (validated there), but the method falls back to the + // default defensively rather than panicking. + if got != 30*time.Minute { + t.Errorf("BuildTimeoutOrDefault(%q) = %v, want fallback %v", c.in, got, 30*time.Minute) + } + continue + } + if got != c.want { + t.Errorf("BuildTimeoutOrDefault(%q) = %v, want %v", c.in, got, c.want) + } + } +} + +func TestLoadRejectsInvalidBuildTimeout(t *testing.T) { + dir := t.TempDir() + cfg := config.Config{DataDir: dir} + a := &App{Name: "my-app", BuildTimeout: "not-a-duration"} + if err := a.Save(cfg); err != nil { + t.Fatal(err) + } + if _, err := Load(cfg, "my-app"); err == nil { + t.Error("Load() with invalid build_timeout should return an error") + } +} + +func TestLoadAcceptsValidBuildTimeout(t *testing.T) { + dir := t.TempDir() + cfg := config.Config{DataDir: dir} + a := &App{Name: "my-app", BuildTimeout: "10m"} + if err := a.Save(cfg); err != nil { + t.Fatal(err) + } + loaded, err := Load(cfg, "my-app") + if err != nil { + t.Fatal(err) + } + if loaded.BuildTimeoutOrDefault() != 10*time.Minute { + t.Errorf("BuildTimeoutOrDefault() = %v, want 10m", loaded.BuildTimeoutOrDefault()) + } +} diff --git a/internal/app/domain.go b/internal/app/domain.go new file mode 100644 index 0000000..1481204 --- /dev/null +++ b/internal/app/domain.go @@ -0,0 +1,33 @@ +package app + +import ( + "fmt" + "strings" +) + +// NormalizeDomain cleans up a user-entered domain: lowercases it, strips +// any "http://"/"https://" scheme, a trailing slash, and a leading "*." +// wildcard prefix. It rejects values that still contain whitespace, a +// slash, or no dot after normalization, since those are not usable as a +// Caddy site address. +func NormalizeDomain(raw string) (string, error) { + d := strings.ToLower(strings.TrimSpace(raw)) + d = strings.TrimPrefix(d, "https://") + d = strings.TrimPrefix(d, "http://") + d = strings.TrimSuffix(d, "/") + d = strings.TrimPrefix(d, "*.") + + if d == "" { + return "", fmt.Errorf("domain cannot be empty") + } + if strings.ContainsAny(d, " \t") { + return "", fmt.Errorf("domain %q cannot contain spaces", raw) + } + if strings.Contains(d, "/") { + return "", fmt.Errorf("domain %q cannot contain a path", raw) + } + if !strings.Contains(d, ".") { + return "", fmt.Errorf("domain %q does not look like a valid domain (missing a dot)", raw) + } + return d, nil +} diff --git a/internal/app/domain_test.go b/internal/app/domain_test.go new file mode 100644 index 0000000..02e7483 --- /dev/null +++ b/internal/app/domain_test.go @@ -0,0 +1,36 @@ +package app + +import "testing" + +func TestNormalizeDomain(t *testing.T) { + cases := []struct { + in string + want string + }{ + {"example.com", "example.com"}, + {"https://example.com", "example.com"}, + {"http://example.com", "example.com"}, + {"example.com/", "example.com"}, + {"EXAMPLE.com", "example.com"}, + {"*.example.com", "example.com"}, + {" example.com ", "example.com"}, + } + for _, c := range cases { + got, err := NormalizeDomain(c.in) + if err != nil { + t.Errorf("NormalizeDomain(%q) returned error: %v", c.in, err) + continue + } + if got != c.want { + t.Errorf("NormalizeDomain(%q) = %q, want %q", c.in, got, c.want) + } + } +} + +func TestNormalizeDomainInvalid(t *testing.T) { + for _, in := range []string{"", "no dot here", "example.com/path", "exa mple.com", "nodothere"} { + if got, err := NormalizeDomain(in); err == nil { + t.Errorf("NormalizeDomain(%q) = %q, want error", in, got) + } + } +} diff --git a/internal/app/env.go b/internal/app/env.go index 2e359a0..8a92c85 100644 --- a/internal/app/env.go +++ b/internal/app/env.go @@ -5,6 +5,8 @@ import ( "os" "regexp" "strings" + + "github.com/Telep-IO/seploy/internal/fsutil" ) var envKeyRe = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`) @@ -49,7 +51,7 @@ func SetEnv(path, key, value string) error { if !replaced { lines = append(lines, entry) } - return os.WriteFile(path, []byte(strings.Join(lines, "\n")+"\n"), 0o600) + return fsutil.WriteFileAtomic(path, []byte(strings.Join(lines, "\n")+"\n"), 0o600) } // EnvKeys returns the keys defined in the env file, in file order. diff --git a/internal/app/health_timeout_test.go b/internal/app/health_timeout_test.go new file mode 100644 index 0000000..0394c15 --- /dev/null +++ b/internal/app/health_timeout_test.go @@ -0,0 +1,65 @@ +package app + +import ( + "testing" + "time" + + "github.com/Telep-IO/seploy/internal/config" +) + +func TestHealthTimeoutOrDefault(t *testing.T) { + cases := []struct { + in string + want time.Duration + wantErr bool + }{ + {"", 30 * time.Second, false}, + {"90s", 90 * time.Second, false}, + {"2m", 2 * time.Minute, false}, + {"garbage", 0, true}, + } + for _, c := range cases { + a := &App{HealthTimeout: c.in} + got := a.HealthTimeoutOrDefault() + if c.wantErr { + // Invalid durations should never reach HealthTimeoutOrDefault via + // Load (validated there), but the method falls back to the + // default defensively rather than panicking. + if got != 30*time.Second { + t.Errorf("HealthTimeoutOrDefault(%q) = %v, want fallback %v", c.in, got, 30*time.Second) + } + continue + } + if got != c.want { + t.Errorf("HealthTimeoutOrDefault(%q) = %v, want %v", c.in, got, c.want) + } + } +} + +func TestLoadRejectsInvalidHealthTimeout(t *testing.T) { + dir := t.TempDir() + cfg := config.Config{DataDir: dir} + a := &App{Name: "my-app", HealthTimeout: "not-a-duration"} + if err := a.Save(cfg); err != nil { + t.Fatal(err) + } + if _, err := Load(cfg, "my-app"); err == nil { + t.Error("Load() with invalid health_timeout should return an error") + } +} + +func TestLoadAcceptsValidHealthTimeout(t *testing.T) { + dir := t.TempDir() + cfg := config.Config{DataDir: dir} + a := &App{Name: "my-app", HealthTimeout: "90s"} + if err := a.Save(cfg); err != nil { + t.Fatal(err) + } + loaded, err := Load(cfg, "my-app") + if err != nil { + t.Fatal(err) + } + if loaded.HealthTimeoutOrDefault() != 90*time.Second { + t.Errorf("HealthTimeoutOrDefault() = %v, want 90s", loaded.HealthTimeoutOrDefault()) + } +} diff --git a/internal/app/lock.go b/internal/app/lock.go index 21e2ac1..4bc18a1 100644 --- a/internal/app/lock.go +++ b/internal/app/lock.go @@ -5,10 +5,15 @@ import ( "os" "path/filepath" "syscall" + "time" "github.com/Telep-IO/seploy/internal/config" ) +// lockRetryInterval is how often AcquireLockWait retries the flock. It is a +// variable so tests can shorten it. +var lockRetryInterval = 2 * time.Second + // Lock is a per-app deployment lock backed by flock(2). type Lock struct { f *os.File @@ -32,6 +37,24 @@ func AcquireLock(cfg config.Config, name string) (*Lock, error) { return &Lock{f: f}, nil } +// AcquireLockWait takes the app's deploy lock, retrying every +// lockRetryInterval until it succeeds or timeout elapses. It is used by +// webhook-triggered deploys so a push arriving during a manual deploy queues +// behind it instead of failing immediately. +func AcquireLockWait(cfg config.Config, name string, timeout time.Duration) (*Lock, error) { + deadline := time.Now().Add(timeout) + for { + lock, err := AcquireLock(cfg, name) + if err == nil { + return lock, nil + } + if time.Now().After(deadline) { + return nil, fmt.Errorf("another deployment of %q is already in progress", name) + } + time.Sleep(lockRetryInterval) + } +} + // Release drops the lock. func (l *Lock) Release() { syscall.Flock(int(l.f.Fd()), syscall.LOCK_UN) diff --git a/internal/app/lock_test.go b/internal/app/lock_test.go new file mode 100644 index 0000000..9f783ee --- /dev/null +++ b/internal/app/lock_test.go @@ -0,0 +1,77 @@ +package app + +import ( + "testing" + "time" + + "github.com/Telep-IO/seploy/internal/config" +) + +func testConfig(t *testing.T) config.Config { + t.Helper() + cfg := config.Default() + cfg.DataDir = t.TempDir() + return cfg +} + +func TestAcquireLockWaitTimesOutWhileHeld(t *testing.T) { + // Speed up the retry loop for the test. + orig := lockRetryInterval + lockRetryInterval = 10 * time.Millisecond + defer func() { lockRetryInterval = orig }() + + cfg := testConfig(t) + + held, err := AcquireLock(cfg, "app1") + if err != nil { + t.Fatalf("AcquireLock: %v", err) + } + defer held.Release() + + start := time.Now() + _, err = AcquireLockWait(cfg, "app1", 50*time.Millisecond) + if err == nil { + t.Fatal("expected AcquireLockWait to time out while lock is held") + } + if elapsed := time.Since(start); elapsed < 50*time.Millisecond { + t.Fatalf("AcquireLockWait returned too early: %v", elapsed) + } +} + +func TestAcquireLockWaitSucceedsAfterRelease(t *testing.T) { + orig := lockRetryInterval + lockRetryInterval = 10 * time.Millisecond + defer func() { lockRetryInterval = orig }() + + cfg := testConfig(t) + + held, err := AcquireLock(cfg, "app1") + if err != nil { + t.Fatalf("AcquireLock: %v", err) + } + + type result struct { + lock *Lock + err error + } + done := make(chan result, 1) + go func() { + lock, err := AcquireLockWait(cfg, "app1", time.Second) + done <- result{lock, err} + }() + + // Give AcquireLockWait a moment to start retrying, then release the lock + // well before the 1s timeout. + time.Sleep(30 * time.Millisecond) + held.Release() + + select { + case r := <-done: + if r.err != nil { + t.Fatalf("AcquireLockWait: %v", r.err) + } + r.lock.Release() + case <-time.After(time.Second): + t.Fatal("AcquireLockWait did not return after lock was released") + } +} diff --git a/internal/config/config.go b/internal/config/config.go index 41d532d..6a3c6fb 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -3,11 +3,13 @@ package config import ( + "bytes" "fmt" "os" "path/filepath" "github.com/BurntSushi/toml" + "github.com/Telep-IO/seploy/internal/fsutil" ) // Path is the location of the global config file. @@ -51,6 +53,9 @@ func Load() (Config, error) { if err := toml.Unmarshal(data, &cfg); err != nil { return cfg, fmt.Errorf("parse %s: %w", Path, err) } + if cfg.KeepReleases < 1 { + cfg.KeepReleases = 1 + } return cfg, nil } @@ -59,12 +64,11 @@ func (c Config) Save() error { if err := os.MkdirAll(filepath.Dir(Path), 0o755); err != nil { return err } - f, err := os.OpenFile(Path, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o644) - if err != nil { + var buf bytes.Buffer + if err := toml.NewEncoder(&buf).Encode(c); err != nil { return err } - defer f.Close() - return toml.NewEncoder(f).Encode(c) + return fsutil.WriteFileAtomic(Path, buf.Bytes(), 0o644) } // AppsDir is where per-app directories live. diff --git a/internal/docker/docker.go b/internal/docker/docker.go index 97fc13a..5dd33d2 100644 --- a/internal/docker/docker.go +++ b/internal/docker/docker.go @@ -3,11 +3,14 @@ package docker import ( + "context" + "errors" "fmt" "os" "os/exec" "strconv" "strings" + "time" ) // RunOpts describes a container to start. @@ -23,18 +26,55 @@ type RunOpts struct { Volumes []string // resolved -v values, e.g. "seploy-app-data:/app/data" } -// Build builds the image from dockerfile inside dir, streaming output. -func Build(dir, dockerfile, tag string) error { - cmd := exec.Command("docker", "build", "-f", dockerfile, "-t", tag, ".") - cmd.Dir = dir +// BuildOpts describes an image build. +type BuildOpts struct { + Dir string + Dockerfile string + Tag string + Timeout time.Duration + NoCache bool // append --no-cache to the build args +} + +// Build builds the image from o.Dockerfile inside o.Dir, streaming output. +// The build is aborted if it runs longer than o.Timeout. +func Build(o BuildOpts) error { + ctx, cancel := context.WithTimeout(context.Background(), o.Timeout) + defer cancel() + args := []string{"build", "-f", o.Dockerfile, "-t", o.Tag} + if o.NoCache { + args = append(args, "--no-cache") + } + args = append(args, ".") + cmd := exec.CommandContext(ctx, "docker", args...) + cmd.Dir = o.Dir cmd.Stdout = os.Stdout cmd.Stderr = os.Stderr - if err := cmd.Run(); err != nil { - return fmt.Errorf("docker build failed for %s", tag) + err := cmd.Run() + if errors.Is(ctx.Err(), context.DeadlineExceeded) { + return fmt.Errorf("docker build for %s exceeded the build_timeout of %s", o.Tag, o.Timeout) + } + if err != nil { + return fmt.Errorf("docker build failed for %s", o.Tag) } return nil } +// RootDir returns the Docker daemon's data root directory (where images, +// containers and the build cache live), falling back to the default +// "/var/lib/docker" if it cannot be determined. +func RootDir() (string, error) { + const fallback = "/var/lib/docker" + out, err := exec.Command("docker", "info", "--format", "{{.DockerRootDir}}").Output() + if err != nil { + return fallback, nil + } + dir := strings.TrimSpace(string(out)) + if dir == "" { + return fallback, nil + } + return dir, nil +} + // Run starts a detached container bound to 127.0.0.1 only. Logs are // size-capped so a chatty app cannot slowly fill the VPS disk. func Run(o RunOpts) error { @@ -90,6 +130,20 @@ func IsRunning(name string) bool { return err == nil && strings.TrimSpace(string(out)) == "true" } +// Exists reports whether a container with the given name exists, regardless +// of its state (running, stopped, etc). +func Exists(name string) bool { + err := exec.Command("docker", "container", "inspect", name).Run() + return err == nil +} + +// ImageExists reports whether an image with the given name/tag exists +// locally. +func ImageExists(image string) bool { + err := exec.Command("docker", "image", "inspect", image).Run() + return err == nil +} + // HostPort returns the host port Docker published for the container's // port (used with RunOpts.HostPort == 0). func HostPort(name string, containerPort int) (int, error) { diff --git a/internal/fsutil/fsutil.go b/internal/fsutil/fsutil.go new file mode 100644 index 0000000..76f50b9 --- /dev/null +++ b/internal/fsutil/fsutil.go @@ -0,0 +1,50 @@ +// Package fsutil provides small filesystem helpers shared across seploy. +package fsutil + +import ( + "os" + "path/filepath" +) + +// WriteFileAtomic writes data to path atomically: it writes to a temporary +// file in the same directory as path (so the final rename cannot cross a +// filesystem boundary), fsyncs and closes it, sets the requested +// permissions, then renames it onto path. This ensures a crash or full disk +// mid-write cannot leave path truncated or corrupted; readers always see +// either the old contents or the fully-written new contents. +func WriteFileAtomic(path string, data []byte, perm os.FileMode) error { + dir := filepath.Dir(path) + base := filepath.Base(path) + + f, err := os.CreateTemp(dir, "."+base+"-*") + if err != nil { + return err + } + tmpPath := f.Name() + + cleanup := func(err error) error { + f.Close() + os.Remove(tmpPath) + return err + } + + if _, err := f.Write(data); err != nil { + return cleanup(err) + } + if err := f.Sync(); err != nil { + return cleanup(err) + } + if err := f.Close(); err != nil { + os.Remove(tmpPath) + return err + } + if err := os.Chmod(tmpPath, perm); err != nil { + os.Remove(tmpPath) + return err + } + if err := os.Rename(tmpPath, path); err != nil { + os.Remove(tmpPath) + return err + } + return nil +} diff --git a/internal/fsutil/fsutil_test.go b/internal/fsutil/fsutil_test.go new file mode 100644 index 0000000..2190ea1 --- /dev/null +++ b/internal/fsutil/fsutil_test.go @@ -0,0 +1,69 @@ +package fsutil + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +func TestWriteFileAtomic(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "state.json") + + if err := WriteFileAtomic(path, []byte("hello"), 0o644); err != nil { + t.Fatalf("WriteFileAtomic: %v", err) + } + + data, err := os.ReadFile(path) + if err != nil { + t.Fatalf("ReadFile: %v", err) + } + if string(data) != "hello" { + t.Fatalf("content = %q, want %q", data, "hello") + } + + info, err := os.Stat(path) + if err != nil { + t.Fatalf("Stat: %v", err) + } + if info.Mode().Perm() != 0o644 { + t.Fatalf("perm = %v, want %v", info.Mode().Perm(), os.FileMode(0o644)) + } + + // Overwrite an existing file with different content and permissions. + if err := WriteFileAtomic(path, []byte("world!"), 0o600); err != nil { + t.Fatalf("WriteFileAtomic (overwrite): %v", err) + } + data, err = os.ReadFile(path) + if err != nil { + t.Fatalf("ReadFile after overwrite: %v", err) + } + if string(data) != "world!" { + t.Fatalf("content after overwrite = %q, want %q", data, "world!") + } + info, err = os.Stat(path) + if err != nil { + t.Fatalf("Stat after overwrite: %v", err) + } + if info.Mode().Perm() != 0o600 { + t.Fatalf("perm after overwrite = %v, want %v", info.Mode().Perm(), os.FileMode(0o600)) + } + + // No stray temp files should remain in the directory. + entries, err := os.ReadDir(dir) + if err != nil { + t.Fatalf("ReadDir: %v", err) + } + for _, e := range entries { + if e.Name() != "state.json" { + t.Errorf("stray file left behind: %s", e.Name()) + } + if strings.HasPrefix(e.Name(), ".state.json-") { + t.Errorf("stray temp file left behind: %s", e.Name()) + } + } + if len(entries) != 1 { + t.Fatalf("directory has %d entries, want 1", len(entries)) + } +} diff --git a/internal/netcheck/edge.go b/internal/netcheck/edge.go new file mode 100644 index 0000000..6772cfa --- /dev/null +++ b/internal/netcheck/edge.go @@ -0,0 +1,152 @@ +package netcheck + +import ( + "crypto/x509" + "errors" + "fmt" + "net" + "net/http" + "strings" + "time" +) + +// EdgeClass categorizes why a public edge check failed, so callers can pick +// a relevant hint without re-parsing an error string. +type EdgeClass int + +const ( + // EdgeClassDNS means the domain did not resolve. + EdgeClassDNS EdgeClass = iota + // EdgeClassTLS means the TLS handshake failed (bad/missing cert, wrong + // hostname, etc). + EdgeClassTLS + // EdgeClassConnection means DNS and TLS were never reached: connection + // refused, connection timeout, or any other network-level failure. + EdgeClassConnection + // EdgeClassHTTPStatus means the server answered with HTTP >= 500. + EdgeClassHTTPStatus +) + +// EdgeError describes a failed CheckEdge call. +type EdgeError struct { + Domain string + Class EdgeClass + Code int // valid when Class == EdgeClassHTTPStatus + Err error +} + +func (e *EdgeError) Error() string { + switch e.Class { + case EdgeClassDNS: + return fmt.Sprintf("%s: DNS lookup failed: %v", e.Domain, e.Err) + case EdgeClassTLS: + return fmt.Sprintf("%s: TLS handshake failed: %v", e.Domain, e.Err) + case EdgeClassHTTPStatus: + return fmt.Sprintf("%s: server error (HTTP %d)", e.Domain, e.Code) + default: + return fmt.Sprintf("%s: connection failed: %v", e.Domain, e.Err) + } +} + +func (e *EdgeError) Unwrap() error { return e.Err } + +// Hint returns a one-line, actionable hint for the failure, tailored to +// whether the app is served through a Cloudflare Tunnel (TLS terminated at +// Cloudflare's edge, traffic reaching the origin over cloudflared) versus +// directly by Caddy on 80/443. +func (e *EdgeError) Hint(tunneled bool) string { + return edgeHint(e.Class, e.Code, tunneled) +} + +// edgeHint is the pure classification of a failure class (plus HTTP status +// code, where relevant) into a hint message. It is kept free of network I/O +// so it can be table tested directly. +func edgeHint(class EdgeClass, code int, tunneled bool) string { + switch class { + case EdgeClassDNS: + return "the domain does not resolve yet" + case EdgeClassTLS: + if tunneled { + return "TLS is terminated at Cloudflare's edge; check the tunnel is connected and the domain's DNS in the Cloudflare dashboard" + } + return "check: systemctl status caddy, and confirm the domain's DNS points at this server" + case EdgeClassHTTPStatus: + if code >= 500 { + if tunneled { + return "check: systemctl status cloudflared" + } + return "check: journalctl -u caddy, and confirm the app container is healthy" + } + return "" + default: // EdgeClassConnection + if tunneled { + return "check: systemctl status cloudflared" + } + return "check: systemctl status caddy" + } +} + +// edgeOK reports whether an HTTP status code from the public edge counts as +// "reachable". Auth walls, redirects and 404s on "/" are all fine -- we are +// testing reachability, not app semantics. Only a server error (HTTP >= +// 500) counts as a failure. +func edgeOK(code int) bool { + return code > 0 && code < 500 +} + +// edgeClient is a short-timeout HTTP client that never follows redirects, +// since a redirect response is itself evidence the edge is reachable. +func edgeClient() *http.Client { + return &http.Client{ + Timeout: 10 * time.Second, + CheckRedirect: func(req *http.Request, via []*http.Request) error { + return http.ErrUseLastResponse + }, + } +} + +// CheckEdge checks that domain is reachable over HTTPS from the public +// internet: DNS resolves, TLS completes, and the app answers with +// anything short of a server error. path is healthPath if set, else "/". +// On success it returns the HTTP status code observed and a nil error. On +// failure it returns a *EdgeError classifying why (DNS, TLS, connection, or +// HTTP >= 500), each carrying a one-line hint via EdgeError.Hint. +func CheckEdge(domain, healthPath string) (int, error) { + path := healthPath + if path == "" { + path = "/" + } else if !strings.HasPrefix(path, "/") { + path = "/" + path + } + url := fmt.Sprintf("https://%s%s", domain, path) + + resp, err := edgeClient().Get(url) + if err != nil { + return 0, classifyEdgeErr(domain, err) + } + defer resp.Body.Close() + + if !edgeOK(resp.StatusCode) { + return resp.StatusCode, &EdgeError{Domain: domain, Class: EdgeClassHTTPStatus, Code: resp.StatusCode} + } + return resp.StatusCode, nil +} + +// classifyEdgeErr inspects a client.Get error to distinguish DNS, TLS and +// plain connection failures (refused, timed out, reset, etc). +func classifyEdgeErr(domain string, err error) *EdgeError { + var dnsErr *net.DNSError + if errors.As(err, &dnsErr) { + return &EdgeError{Domain: domain, Class: EdgeClassDNS, Err: err} + } + + var certInvalid x509.CertificateInvalidError + var unknownAuth x509.UnknownAuthorityError + var hostnameErr x509.HostnameError + if errors.As(err, &certInvalid) || errors.As(err, &unknownAuth) || errors.As(err, &hostnameErr) || + strings.Contains(err.Error(), "tls:") || strings.Contains(err.Error(), "x509") { + return &EdgeError{Domain: domain, Class: EdgeClassTLS, Err: err} + } + + return &EdgeError{Domain: domain, Class: EdgeClassConnection, Err: err} +} diff --git a/internal/netcheck/edge_test.go b/internal/netcheck/edge_test.go new file mode 100644 index 0000000..05fe1d5 --- /dev/null +++ b/internal/netcheck/edge_test.go @@ -0,0 +1,55 @@ +package netcheck + +import "testing" + +func TestEdgeOK(t *testing.T) { + cases := []struct { + name string + code int + want bool + }{ + {"zero (no response)", 0, false}, + {"ok", 200, true}, + {"redirect", 302, true}, + {"auth wall", 401, true}, + {"not found", 404, true}, + {"bad gateway", 502, false}, + {"service unavailable", 503, false}, + {"server error boundary", 500, false}, + {"just below server error boundary", 499, true}, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + if got := edgeOK(c.code); got != c.want { + t.Errorf("edgeOK(%d) = %v, want %v", c.code, got, c.want) + } + }) + } +} + +func TestEdgeHint(t *testing.T) { + cases := []struct { + name string + class EdgeClass + code int + tunneled bool + want string + }{ + {"dns not tunneled", EdgeClassDNS, 0, false, "the domain does not resolve yet"}, + {"dns tunneled", EdgeClassDNS, 0, true, "the domain does not resolve yet"}, + {"tls not tunneled", EdgeClassTLS, 0, false, "check: systemctl status caddy, and confirm the domain's DNS points at this server"}, + {"tls tunneled", EdgeClassTLS, 0, true, "TLS is terminated at Cloudflare's edge; check the tunnel is connected and the domain's DNS in the Cloudflare dashboard"}, + {"connection not tunneled", EdgeClassConnection, 0, false, "check: systemctl status caddy"}, + {"connection tunneled", EdgeClassConnection, 0, true, "check: systemctl status cloudflared"}, + {"http 502 tunneled", EdgeClassHTTPStatus, 502, true, "check: systemctl status cloudflared"}, + {"http 503 not tunneled", EdgeClassHTTPStatus, 503, false, "check: journalctl -u caddy, and confirm the app container is healthy"}, + {"http below 500 has no hint", EdgeClassHTTPStatus, 404, false, ""}, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + if got := edgeHint(c.class, c.code, c.tunneled); got != c.want { + t.Errorf("edgeHint(%v, %d, %v) = %q, want %q", c.class, c.code, c.tunneled, got, c.want) + } + }) + } +} diff --git a/internal/netcheck/netcheck.go b/internal/netcheck/netcheck.go index 3280c7f..676544d 100644 --- a/internal/netcheck/netcheck.go +++ b/internal/netcheck/netcheck.go @@ -44,6 +44,17 @@ func LookupHost(domain string) ([]string, error) { return addrs, nil } +// LookupCNAME resolves a domain's CNAME target with a short timeout. +func LookupCNAME(domain string) (string, error) { + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + cname, err := net.DefaultResolver.LookupCNAME(ctx, domain) + if err != nil { + return "", fmt.Errorf("could not resolve CNAME for %s", domain) + } + return cname, nil +} + // PointsTo reports whether domain resolves to ip, returning whatever // addresses it does resolve to. func PointsTo(domain, ip string) (bool, []string) { diff --git a/internal/system/system.go b/internal/system/system.go index d926c18..8b0be03 100644 --- a/internal/system/system.go +++ b/internal/system/system.go @@ -8,6 +8,8 @@ import ( "net/http" "os" "os/exec" + "syscall" + "time" ) // PackageManager identifies the host's package manager. @@ -33,6 +35,16 @@ func RequireRoot(command string) error { return fmt.Errorf("this command needs root to manage Docker, Caddy and system paths.\nRe-run it with: sudo %s", command) } +// DiskFree returns the number of free bytes available on the filesystem +// containing path. +func DiskFree(path string) (uint64, error) { + var st syscall.Statfs_t + if err := syscall.Statfs(path, &st); err != nil { + return 0, fmt.Errorf("statfs %s: %w", path, err) + } + return st.Bavail * uint64(st.Bsize), nil +} + // HasCommand reports whether a binary is on PATH. func HasCommand(name string) bool { _, err := exec.LookPath(name) @@ -84,7 +96,8 @@ const ( // SetupCaddyAptRepo adds Caddy's official apt repository, for // Debian/Ubuntu releases that do not ship a caddy package. func SetupCaddyAptRepo() error { - resp, err := http.Get(caddyKeyURL) + client := &http.Client{Timeout: 30 * time.Second} + resp, err := client.Get(caddyKeyURL) if err != nil { return fmt.Errorf("download Caddy signing key: %w", err) } @@ -92,7 +105,7 @@ func SetupCaddyAptRepo() error { if resp.StatusCode != http.StatusOK { return fmt.Errorf("download Caddy signing key: HTTP %d", resp.StatusCode) } - key, err := io.ReadAll(resp.Body) + key, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20)) if err != nil { return fmt.Errorf("download Caddy signing key: %w", err) } diff --git a/internal/tunnel/tunnel.go b/internal/tunnel/tunnel.go index 04377fb..28435fc 100644 --- a/internal/tunnel/tunnel.go +++ b/internal/tunnel/tunnel.go @@ -4,6 +4,8 @@ package tunnel import ( + "crypto/sha256" + "encoding/hex" "encoding/json" "errors" "fmt" @@ -12,10 +14,13 @@ import ( "os" "os/exec" "path/filepath" + "regexp" "runtime" "strings" "time" + "github.com/Telep-IO/seploy/internal/fsutil" + "github.com/Telep-IO/seploy/internal/netcheck" "github.com/Telep-IO/seploy/internal/system" ) @@ -23,14 +28,56 @@ const ( // Name is the Cloudflare tunnel name used for this server. Name = "seploy" + // Version is the pinned cloudflared release seploy installs and + // upgrades to. Bump this deliberately after checking + // https://github.com/cloudflare/cloudflared/releases. + Version = "2026.6.1" + binPath = "/usr/local/bin/cloudflared" // ConfigPath is the cloudflared config written by `seploy tunnel setup`. ConfigPath = "/etc/cloudflared/config.yml" systemdUnitPath = "/etc/systemd/system/cloudflared.service" openrcPath = "/etc/init.d/cloudflared" + + // etcCloudflaredDir is the canonical, root-owned location for + // cloudflared credentials once `seploy tunnel setup` has run. cloudflared + // itself runs as a system service (usually root), and seploy is commonly + // invoked via sudo, so the invoking user's home directory is not a + // reliable place to keep credentials the service depends on long-term. + etcCloudflaredDir = "/etc/cloudflared" + // etcCertPath is the copy of cert.pem seploy maintains under + // etcCloudflaredDir. + etcCertPath = etcCloudflaredDir + "/cert.pem" + + // modeFilePath records which setup path this server's tunnel was + // configured with ("local", the interactive `tunnel login`/`tunnel + // create` flow that owns config.yml, or "token", a remotely-managed + // tunnel run from a dashboard-issued token). Written by both setup + // paths so Mode() does not have to guess from side effects alone. + modeFilePath = etcCloudflaredDir + "/seploy-tunnel.toml" + + // ModeLocal is a locally-managed tunnel: `cloudflared tunnel login` + + // `tunnel create` + a config.yml this server owns and can edit. + ModeLocal = "local" + // ModeToken is a remotely-managed tunnel: a dashboard-issued token runs + // via `cloudflared service install `, and ingress rules (public + // hostnames) live in the Cloudflare dashboard, not in a local config.yml. + ModeToken = "token" + + // releaseAPIURL, given a tag, returns cloudflared's GitHub release + // metadata as JSON. cloudflared does not publish a `.sha256` sidecar + // file per binary asset; instead every release's notes body contains a + // "SHA256 Checksums" block listing ": " for every asset. + // That block is this project's only source of published checksums, so + // it is what Install/Upgrade verify downloads against. + releaseAPIURL = "https://api.github.com/repos/cloudflare/cloudflared/releases/tags/" ) +// checksumLineRE matches one ": <64 hex chars>" line from a +// cloudflared release's "SHA256 Checksums" notes block. +var checksumLineRE = regexp.MustCompile(`(?m)^([\w.+-]+):\s*([0-9a-fA-F]{64})\s*$`) + var errNotFound = errors.New("tunnel not found") // Installed reports whether cloudflared is available. @@ -41,13 +88,69 @@ func Installed() bool { return system.HasCommand("cloudflared") } -// Configured reports whether this server's tunnel has been set up. +// Configured reports whether this server's tunnel has been set up, under +// either setup path: a local config.yml (interactive login) or a recorded +// token mode (remotely-managed tunnel, which has no local config.yml). func Configured() bool { if !Installed() { return false } - _, err := os.Stat(ConfigPath) - return err == nil + if _, err := os.Stat(ConfigPath); err == nil { + return true + } + if Mode() == ModeToken { + return true + } + return false +} + +// Mode reports which setup path configured this server's tunnel: +// ModeLocal, ModeToken, or "" if `seploy tunnel setup` has not been run. +// +// It reads the mode file both setup paths write. When that file is absent +// (upgrades from before token-mode support existed), it falls back to +// ModeLocal if the legacy config.yml is present, since that file only ever +// existed under the local setup path. +func Mode() string { + if data, err := os.ReadFile(modeFilePath); err == nil { + if mode := parseModeFile(string(data)); mode != "" { + return mode + } + } + if _, err := os.Stat(ConfigPath); err == nil { + return ModeLocal + } + return "" +} + +// parseModeFile extracts the `mode = "..."` value from modeFilePath's +// contents. It returns "" if no such line is present. +func parseModeFile(data string) string { + for _, line := range strings.Split(data, "\n") { + line = strings.TrimSpace(line) + if !strings.HasPrefix(line, "mode") { + continue + } + parts := strings.SplitN(line, "=", 2) + if len(parts) != 2 { + continue + } + if mode := strings.Trim(strings.TrimSpace(parts[1]), `"`); mode != "" { + return mode + } + } + return "" +} + +// RecordMode records mode in modeFilePath, creating etcCloudflaredDir if +// needed. Both setup paths (local, via WriteConfig, and token, via callers +// of InstallServiceToken) call this so Mode() never has to guess. +func RecordMode(mode string) error { + if err := os.MkdirAll(etcCloudflaredDir, 0o755); err != nil { + return err + } + content := fmt.Sprintf("# Managed by seploy. Records which `seploy tunnel setup` path configured\n# this server's tunnel.\nmode = %q\n", mode) + return fsutil.WriteFileAtomic(modeFilePath, []byte(content), 0o644) } func bin() string { @@ -57,46 +160,183 @@ func bin() string { return "cloudflared" } +// Bin returns the path (or PATH-resolved name) of the cloudflared binary +// seploy manages, for callers that need to run it directly. +func Bin() string { + return bin() +} + // Install downloads the cloudflared binary if it is missing. The static // binary from Cloudflare's releases works on every distro (it's a Go // program), which avoids per-distro package repositories entirely. +// +// The download is for the pinned Version and is checksum-verified before it +// is ever renamed into place: an unverified binary never reaches binPath. func Install() error { if Installed() { return nil } - arch := map[string]string{"amd64": "amd64", "arm64": "arm64", "arm": "arm", "386": "386"}[runtime.GOARCH] + data, err := downloadVerified() + if err != nil { + return err + } + return installBinary(data) +} + +// Upgrade downloads and verifies the pinned Version of cloudflared +// regardless of whether it is already installed, then atomically replaces +// the installed binary. +func Upgrade() error { + data, err := downloadVerified() + if err != nil { + return err + } + return installBinary(data) +} + +// cloudflaredArch maps Go's GOARCH to the arch suffix cloudflared uses in +// its release asset names. +func cloudflaredArch(goarch string) string { + return map[string]string{"amd64": "amd64", "arm64": "arm64", "arm": "arm", "386": "386"}[goarch] +} + +// assetName is the release asset filename for the Linux cloudflared binary +// built for arch (a cloudflaredArch result, e.g. "amd64"). +func assetName(arch string) string { + return "cloudflared-linux-" + arch +} + +func downloadURL(arch string) string { + return fmt.Sprintf("https://github.com/cloudflare/cloudflared/releases/download/%s/%s", Version, assetName(arch)) +} + +// downloadVerified downloads the pinned Version of cloudflared for the +// current host architecture and verifies it against the checksum Cloudflare +// published for this release before returning its bytes. It never writes +// the binary to binPath itself, so a caller can only install bytes that +// have already passed verification. +func downloadVerified() ([]byte, error) { + arch := cloudflaredArch(runtime.GOARCH) if arch == "" { - return fmt.Errorf("no cloudflared build available for %s", runtime.GOARCH) + return nil, fmt.Errorf("no cloudflared build available for %s", runtime.GOARCH) + } + name := assetName(arch) + + checksums, err := fetchChecksums(Version) + if err != nil { + return nil, err + } + expected, ok := checksums[name] + if !ok { + return nil, fmt.Errorf("cloudflared %s did not publish a checksum for %s", Version, name) } - url := "https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-" + arch - fmt.Println("Downloading cloudflared...") + url := downloadURL(arch) + fmt.Printf("Downloading cloudflared %s...\n", Version) client := &http.Client{Timeout: 5 * time.Minute} resp, err := client.Get(url) if err != nil { - return fmt.Errorf("download cloudflared: %w", err) + return nil, fmt.Errorf("download cloudflared: %w", err) } defer resp.Body.Close() if resp.StatusCode != http.StatusOK { - return fmt.Errorf("download cloudflared: HTTP %d from %s", resp.StatusCode, url) + return nil, fmt.Errorf("download cloudflared: HTTP %d from %s", resp.StatusCode, url) + } + data, err := io.ReadAll(resp.Body) + if err != nil { + return nil, fmt.Errorf("download cloudflared: %w", err) } + if !verifySHA256(data, expected) { + return nil, fmt.Errorf("cloudflared download failed checksum verification: expected sha256 %s, got %s", + strings.ToLower(strings.TrimSpace(expected)), sha256Hex(data)) + } + return data, nil +} + +// installBinary atomically installs already-verified binary bytes at +// binPath: it writes to a temp file in the same directory, makes it +// executable, then renames it into place. The rename is atomic, so +// concurrent readers (or a running cloudflared process) never observe a +// partially-written binary. +func installBinary(data []byte) error { tmp, err := os.CreateTemp(filepath.Dir(binPath), ".cloudflared-*") if err != nil { return err } - defer os.Remove(tmp.Name()) - if _, err := io.Copy(tmp, resp.Body); err != nil { + tmpName := tmp.Name() + if _, err := tmp.Write(data); err != nil { tmp.Close() - return fmt.Errorf("download cloudflared: %w", err) + os.Remove(tmpName) + return fmt.Errorf("write cloudflared binary: %w", err) } if err := tmp.Close(); err != nil { + os.Remove(tmpName) return err } - if err := os.Chmod(tmp.Name(), 0o755); err != nil { + if err := os.Chmod(tmpName, 0o755); err != nil { + os.Remove(tmpName) return err } - return os.Rename(tmp.Name(), binPath) + if err := os.Rename(tmpName, binPath); err != nil { + os.Remove(tmpName) + return err + } + return nil +} + +// sha256Hex is the lowercase hex-encoded SHA256 digest of data. +func sha256Hex(data []byte) string { + sum := sha256.Sum256(data) + return hex.EncodeToString(sum[:]) +} + +// verifySHA256 reports whether data's SHA256 digest matches expectedHex. +// The comparison ignores case and surrounding whitespace, matching how +// cloudflared and sha256sum format digests. +func verifySHA256(data []byte, expectedHex string) bool { + return strings.EqualFold(sha256Hex(data), strings.TrimSpace(expectedHex)) +} + +// fetchChecksums downloads cloudflared's GitHub release metadata for tag +// and returns the asset-name -> hex-digest checksums published in its +// release notes. +func fetchChecksums(tag string) (map[string]string, error) { + url := releaseAPIURL + tag + client := &http.Client{Timeout: 30 * time.Second} + resp, err := client.Get(url) + if err != nil { + return nil, fmt.Errorf("download cloudflared checksums: %w", err) + } + defer resp.Body.Close() + if resp.StatusCode != http.StatusOK { + return nil, fmt.Errorf("download cloudflared checksums: HTTP %d from %s", resp.StatusCode, url) + } + body, err := io.ReadAll(resp.Body) + if err != nil { + return nil, fmt.Errorf("download cloudflared checksums: %w", err) + } + var release struct { + Body string `json:"body"` + } + if err := json.Unmarshal(body, &release); err != nil { + return nil, fmt.Errorf("parse cloudflared release metadata: %w", err) + } + return parseChecksums(release.Body) +} + +// parseChecksums extracts the ": " lines from a +// cloudflared release notes body into an asset-name -> digest map. +func parseChecksums(body string) (map[string]string, error) { + matches := checksumLineRE.FindAllStringSubmatch(body, -1) + if len(matches) == 0 { + return nil, errors.New("no SHA256 checksums found in cloudflared release notes") + } + out := make(map[string]string, len(matches)) + for _, m := range matches { + out[m[1]] = strings.ToLower(m[2]) + } + return out, nil } func certPath() string { @@ -107,12 +347,53 @@ func certPath() string { return filepath.Join(home, ".cloudflared", "cert.pem") } -// LoggedIn reports whether this server is authorized with Cloudflare. +// LoggedIn reports whether this server is authorized with Cloudflare. It +// checks the canonical /etc/cloudflared location first, then falls back to +// the invoking user's home directory (where `cloudflared tunnel login` +// writes cert.pem before seploy has had a chance to copy it into place). func LoggedIn() bool { + if _, err := os.Stat(etcCertPath); err == nil { + return true + } _, err := os.Stat(certPath()) return err == nil } +// certEnv returns the environment for cloudflared child processes that +// perform account-level operations (listing/creating tunnels, routing DNS). +// When the canonical cert copy exists under /etc/cloudflared, it is passed +// via TUNNEL_ORIGIN_CERT so cloudflared uses it even if the original home +// directory it was copied from no longer exists. +func certEnv() []string { + env := os.Environ() + if _, err := os.Stat(etcCertPath); err == nil { + env = append(env, "TUNNEL_ORIGIN_CERT="+etcCertPath) + } + return env +} + +// runVisibleWithCertEnv runs a command with its output streamed to the +// terminal, like system.RunVisible, but with certEnv() applied so cloudflared +// picks up the canonical /etc/cloudflared cert when present. +func runVisibleWithCertEnv(name string, args ...string) error { + cmd := exec.Command(name, args...) + cmd.Env = certEnv() + cmd.Stdin = os.Stdin + cmd.Stdout = os.Stdout + cmd.Stderr = os.Stderr + return cmd.Run() +} + +// copyFile copies src to dst with the given permissions, overwriting dst if +// it already exists. +func copyFile(src, dst string, mode os.FileMode) error { + data, err := os.ReadFile(src) + if err != nil { + return err + } + return os.WriteFile(dst, data, mode) +} + // Login runs the interactive Cloudflare authorization. cloudflared prints a // URL the user opens in any browser, then writes cert.pem locally. func Login() error { @@ -150,7 +431,9 @@ func ParseTunnelID(out []byte, name string) (string, error) { } func lookupID() (string, error) { - out, err := exec.Command(bin(), "tunnel", "list", "--output", "json").Output() + cmd := exec.Command(bin(), "tunnel", "list", "--output", "json") + cmd.Env = certEnv() + out, err := cmd.Output() if err != nil { return "", fmt.Errorf("cloudflared tunnel list failed. Run 'cloudflared tunnel login' first if this server is not authorized") } @@ -167,26 +450,62 @@ func Ensure() (string, error) { return "", err } fmt.Printf("Creating tunnel %q...\n", Name) - if err := system.RunVisible(bin(), "tunnel", "create", Name); err != nil { + if err := runVisibleWithCertEnv(bin(), "tunnel", "create", Name); err != nil { return "", fmt.Errorf("could not create tunnel %q", Name) } return lookupID() } // WriteConfig writes the cloudflared config: a single catch-all ingress -// pointing at Caddy, which routes per-app by Host header. +// pointing at Caddy, which routes per-app by Host header. It also copies the +// tunnel credentials and cert.pem into /etc/cloudflared, the canonical, +// root-owned location the cloudflared service and future seploy tunnel +// operations rely on, so they no longer depend on the invoking user's home +// directory. func WriteConfig(id string) error { - creds := filepath.Join(filepath.Dir(certPath()), id+".json") + homeCreds := filepath.Join(filepath.Dir(certPath()), id+".json") + etcCreds := filepath.Join(etcCloudflaredDir, id+".json") + + creds := homeCreds if _, err := os.Stat(creds); err != nil { - return fmt.Errorf("tunnel credentials not found at %s", creds) + // Fall back to a credentials copy from a previous run, so re-running + // `seploy tunnel setup` stays idempotent even after the original + // home directory is gone. + if _, etcErr := os.Stat(etcCreds); etcErr == nil { + creds = etcCreds + } else { + return fmt.Errorf("tunnel credentials not found at %s", homeCreds) + } + } + + if err := os.MkdirAll(etcCloudflaredDir, 0o755); err != nil { + return err + } + if creds != etcCreds { + if err := copyFile(creds, etcCreds, 0o600); err != nil { + return fmt.Errorf("copy tunnel credentials to %s: %w", etcCreds, err) + } + } + if homeCert := certPath(); homeCert != etcCertPath { + if _, err := os.Stat(homeCert); err == nil { + if err := copyFile(homeCert, etcCertPath, 0o600); err != nil { + return fmt.Errorf("copy cloudflared cert to %s: %w", etcCertPath, err) + } + } else if _, err := os.Stat(etcCertPath); err != nil { + return fmt.Errorf("cloudflared cert not found at %s", homeCert) + } } + if err := os.MkdirAll(filepath.Dir(ConfigPath), 0o755); err != nil { return err } content := fmt.Sprintf( "# Managed by seploy. All traffic goes to Caddy, which routes by Host.\ntunnel: %s\ncredentials-file: %s\n\ningress:\n - service: http://localhost:80\n", - id, creds) - return os.WriteFile(ConfigPath, []byte(content), 0o600) + id, etcCreds) + if err := os.WriteFile(ConfigPath, []byte(content), 0o600); err != nil { + return err + } + return RecordMode(ModeLocal) } const openrcScript = `#!/sbin/openrc-run @@ -223,16 +542,93 @@ func InstallService() error { return errors.New("no supported service manager found (systemd or OpenRC). Run cloudflared manually") } -// RouteDNS creates the Cloudflare DNS record pointing domain at this -// server's tunnel. Existing records for the same tunnel are fine. -func RouteDNS(domain string) error { - out, err := exec.Command(bin(), "tunnel", "route", "dns", Name, domain).CombinedOutput() - if err != nil { - msg := strings.TrimSpace(string(out)) - if strings.Contains(msg, "already") { - return nil +// openrcTokenScript is the OpenRC init script cloudflared runs from in +// token mode. Unlike the local-mode script (which reads credentials from +// config.yml's credentials-file), the token itself has to be embedded in +// command_args, so this script is chmod 0700 by InstallServiceToken: +// keeping it root-readable-only is the only thing that keeps the token out +// of reach of other local users on this server. +const openrcTokenScriptTmpl = `#!/sbin/openrc-run +description="Cloudflare Tunnel (managed by seploy, token mode)" +command="` + binPath + `" +command_args="tunnel run --token %s" +command_background=true +pidfile="/run/cloudflared.pid" +output_log="/var/log/cloudflared.log" +error_log="/var/log/cloudflared.log" + +depend() { + need net +} +` + +// InstallServiceToken installs and starts cloudflared as a system service +// running a remotely-managed tunnel identified by token, rather than the +// local config.yml InstallService uses. +// +// On systemd, cloudflared's own `service install ` writes and +// enables the unit. On OpenRC, seploy has to embed the token directly in +// the init script's command_args (OpenRC has no equivalent of systemd's +// token-file support), so that file is written with mode 0700 — readable +// only by root — since the token otherwise sits in plain text on disk. +func InstallServiceToken(token string) error { + switch { + case system.HasCommand("systemctl"): + if err := system.RunVisible(bin(), "service", "install", token); err != nil { + return fmt.Errorf("cloudflared service install failed") + } + return system.RestartService("cloudflared") + case system.HasCommand("rc-service"): + script := fmt.Sprintf(openrcTokenScriptTmpl, token) + if err := os.WriteFile(openrcPath, []byte(script), 0o700); err != nil { + return err } - return fmt.Errorf("could not create the DNS record for %s:\n%s", domain, msg) + system.RunVisible("rc-update", "add", "cloudflared", "default") + return system.RestartService("cloudflared") } - return nil + return errors.New("no supported service manager found (systemd or OpenRC). Run cloudflared manually") +} + +// CNAMETarget is the CNAME target every domain routed at tunnel id must +// have. +func CNAMETarget(id string) string { + return id + ".cfargotunnel.com" +} + +// cnameMatchesTunnel reports whether a resolved CNAME (as returned by +// net.LookupCNAME, which includes a trailing dot) already points at the +// given tunnel id. +func cnameMatchesTunnel(cname, id string) bool { + if cname == "" || id == "" { + return false + } + got := strings.ToLower(strings.TrimSuffix(cname, ".")) + want := strings.ToLower(CNAMETarget(id)) + return got == want +} + +// RouteDNS creates the Cloudflare DNS record pointing domain at this +// server's tunnel (identified by id). If the route command fails because a +// record already exists, that's only fine when the existing record already +// points at this tunnel; anything else (an A record, or a CNAME to a +// different tunnel) is a real conflict and is reported as an error. +func RouteDNS(id, domain string) error { + cmd := exec.Command(bin(), "tunnel", "route", "dns", Name, domain) + cmd.Env = certEnv() + out, err := cmd.CombinedOutput() + if err == nil { + return nil + } + + cname, lookupErr := netcheck.LookupCNAME(domain) + if lookupErr == nil && cnameMatchesTunnel(cname, id) { + return nil + } + + msg := strings.TrimSpace(string(out)) + if lookupErr == nil { + return fmt.Errorf("could not create the DNS record for %s: it currently resolves to %s, not this server's tunnel (%s). Delete the conflicting record in the Cloudflare dashboard and re-run.", + domain, strings.TrimSuffix(cname, "."), CNAMETarget(id)) + } + return fmt.Errorf("could not create the DNS record for %s:\n%s", domain, msg) } diff --git a/internal/tunnel/tunnel_test.go b/internal/tunnel/tunnel_test.go index 29c8c58..f39a4b3 100644 --- a/internal/tunnel/tunnel_test.go +++ b/internal/tunnel/tunnel_test.go @@ -2,6 +2,7 @@ package tunnel import ( "errors" + "strings" "testing" ) @@ -31,3 +32,163 @@ func TestParseTunnelIDBadJSON(t *testing.T) { t.Error("want error for invalid JSON") } } + +func TestVerifySHA256(t *testing.T) { + const helloDigest = "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824" + cases := []struct { + name string + data []byte + expectedHex string + want bool + }{ + {"matching digest", []byte("hello"), helloDigest, true}, + {"matching digest uppercase", []byte("hello"), strings.ToUpper(helloDigest), true}, + {"matching digest with whitespace", []byte("hello"), " " + helloDigest + "\n", true}, + {"wrong digest", []byte("hello"), strings.Repeat("0", 64), false}, + {"digest for different data", []byte("goodbye"), helloDigest, false}, + {"empty data wrong digest", []byte{}, helloDigest, false}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + if got := verifySHA256(tc.data, tc.expectedHex); got != tc.want { + t.Errorf("verifySHA256(%q, %q) = %v, want %v", tc.data, tc.expectedHex, got, tc.want) + } + }) + } +} + +func TestParseChecksums(t *testing.T) { + const amd64Digest = "5861a10a438fe8ddcfebb3b830f83966cbf193edafce0fe2eeb198fbae1f7a22" + const arm64Digest = "59816ce9b16db71f5bc2a86d59b3632a96c8c3ee934bde2bc8641ee83a6070eb" + + cases := []struct { + name string + body string + want map[string]string + wantErr bool + }{ + { + name: "typical release notes", + body: "### SHA256 Checksums:\n```\n" + + "cloudflared-linux-amd64: " + amd64Digest + "\n" + + "cloudflared-linux-arm64: " + arm64Digest + "\n" + + "```\n", + want: map[string]string{ + "cloudflared-linux-amd64": amd64Digest, + "cloudflared-linux-arm64": arm64Digest, + }, + }, + { + name: "mixed case digest is lowercased", + body: "cloudflared-linux-amd64: " + strings.ToUpper(amd64Digest) + "\n", + want: map[string]string{"cloudflared-linux-amd64": amd64Digest}, + }, + { + name: "no checksums present", + body: "This release fixes a bug.\n", + wantErr: true, + }, + { + name: "empty body", + body: "", + wantErr: true, + }, + { + name: "digest too short is ignored", + body: "cloudflared-linux-amd64: deadbeef\n", + wantErr: true, + }, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got, err := parseChecksums(tc.body) + if tc.wantErr { + if err == nil { + t.Fatalf("parseChecksums() = %v, want error", got) + } + return + } + if err != nil { + t.Fatalf("parseChecksums() unexpected error: %v", err) + } + if len(got) != len(tc.want) { + t.Fatalf("parseChecksums() = %v, want %v", got, tc.want) + } + for k, v := range tc.want { + if got[k] != v { + t.Errorf("parseChecksums()[%q] = %q, want %q", k, got[k], v) + } + } + }) + } +} + +func TestCloudflaredArch(t *testing.T) { + cases := []struct { + goarch string + want string + }{ + {"amd64", "amd64"}, + {"arm64", "arm64"}, + {"arm", "arm"}, + {"386", "386"}, + {"riscv64", ""}, + {"", ""}, + } + for _, tc := range cases { + t.Run(tc.goarch, func(t *testing.T) { + if got := cloudflaredArch(tc.goarch); got != tc.want { + t.Errorf("cloudflaredArch(%q) = %q, want %q", tc.goarch, got, tc.want) + } + }) + } +} + +func TestParseModeFile(t *testing.T) { + cases := []struct { + name string + data string + want string + }{ + {"local mode", "mode = \"local\"\n", "local"}, + {"token mode", "mode = \"token\"\n", "token"}, + {"with leading comment", "# Managed by seploy.\nmode = \"token\"\n", "token"}, + {"no spaces around equals", `mode="token"`, "token"}, + {"empty file", "", ""}, + {"no mode line", "# just a comment\n", ""}, + {"empty mode value", `mode = ""`, ""}, + {"unrelated key", "other = \"token\"\n", ""}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + if got := parseModeFile(tc.data); got != tc.want { + t.Errorf("parseModeFile(%q) = %q, want %q", tc.data, got, tc.want) + } + }) + } +} + +func TestCnameMatchesTunnel(t *testing.T) { + const id = "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee" + cases := []struct { + name string + cname string + id string + want bool + }{ + {"matching", id + ".cfargotunnel.com", id, true}, + {"matching with trailing dot", id + ".cfargotunnel.com.", id, true}, + {"matching mixed case", strings.ToUpper(id) + ".cfargotunnel.com.", id, true}, + {"different tunnel id", "11111111-2222-3333-4444-555555555555.cfargotunnel.com.", id, false}, + {"empty cname", "", id, false}, + {"empty id", id + ".cfargotunnel.com.", "", false}, + {"unrelated cname", "example.com.", id, false}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + if got := cnameMatchesTunnel(tc.cname, tc.id); got != tc.want { + t.Errorf("cnameMatchesTunnel(%q, %q) = %v, want %v", tc.cname, tc.id, got, tc.want) + } + }) + } +}