Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 28 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,32 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

## [0.14.0] - 2026-09-10

### Added

- A test asserting every CLI flag appears in `README.md`. Four had been shipping undocumented and nothing said so (#642); this walks the commands' own flag sets, so a flag cannot be added to one and missed. `--check` is exempt by name, with its reason, and a second hidden flag fails the build rather than joining a skip list (#646).

### Fixed

- Three dropped failures. A reconnecting control host no longer leaks the connection it replaces — one descriptor per run, for the agent's whole life. An agent that cannot open its channel says so at the first ask instead of timing out on `no control host attached`. A local workdir that cannot be created fails the run, naming itself (#638).

- `shellf status` takes the inputs `run` takes: `--vars`, `--set`, `-v` and `--agent-ttl`. The command that answers "what would this plan see?" could not be handed what the plan sees — a plan using `${k}` from `--set` failed to resolve under `status` while applying cleanly under `run`. Both now register one shared flag definition (#640).

- The README's install snippet fetches the latest release instead of pinning `v0.10.0`, three versions behind. It downloads one architecture rather than both, and verifies with `--ignore-missing` — required once the download is narrowed, and checked not to weaken the verification: a binary that never arrived or whose bytes changed still exits non-zero (#647).

- `archive.extract-member` passes the member name to `tar` after `--`. A member named `-rf.txt` was read as options, so the observe saw a mismatch and the apply failed on every run. The name comes from whoever built the archive, not from the plan (#639).

- A def written with the retired `pre-check` phase now gets the message telling it to rename, instead of the generic "expected a phase". The entry was keyed `check` — a valid phase, matched earlier — so it was unreachable, and its test asserted a substring the generic message already contained, so nothing said so (#637).

- `internal/report` no longer calls `os.Exit`. `JSON` returns its marshal error and `Render` passes it up; the two commands decide. The package's own comment said nothing there decides an exit code, twelve lines above the line that did — and a second comment stated the rule as the reason the package exists (#641).

- The README no longer teaches the observe that cost #486: its `def` example asked `dpkg -s`, which exits 0 for a package removed without `--purge`. It now carries `apt.install`'s real question and says why. `--json`, `-v`, `--parallel` and `--limit` are documented, and the secrets note states the limit that is left rather than one ADR-0025 closed (#642).

- `htpasswd.entry` and `system.timezone` observe everything their apply sets. The first left a credentials file world-readable whenever its hash already verified; the second left `/etc/timezone` naming another zone whenever the symlink was right. Both reported `already`. Two adverse cases, each seen red on a real machine first (#635, #636).

- A def calling another instruction with too few arguments is refused instead of binding the missing one to the empty string. `file.write(path)` inside a def overwrote the file with nothing and reported `ok.done` — a file destroyed under a success verdict. A plan-level call was always checked on both bounds; only the def side was not (#633).

## [0.13.0] - 2026-09-09

### Changed
Expand Down Expand Up @@ -383,7 +409,8 @@ agent that evaluates on the host — "raw shell, but idempotent, previewable, fa
per-user agent/workdir scoping.
- Commands: `run`, `status`, `clean`, and `version`.

[Unreleased]: https://github.com/haribo/shellf/compare/v0.13.0...HEAD
[Unreleased]: https://github.com/haribo/shellf/compare/v0.14.0...HEAD
[0.14.0]: https://github.com/haribo/shellf/compare/v0.13.0...v0.14.0
[0.13.0]: https://github.com/haribo/shellf/compare/v0.12.0...v0.13.0
[0.12.0]: https://github.com/haribo/shellf/compare/v0.11.0...v0.12.0
[0.11.0]: https://github.com/haribo/shellf/compare/v0.10.0...v0.11.0
Expand Down
38 changes: 32 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,17 @@ workdir and its own binary and exits, leaving nothing behind.
Download a release and **check what you got**:

```sh
gh release download v0.10.0 --repo haribo/shellf
sha256sum -c SHA256SUMS # shellf-linux-amd64: OK
gh release download --repo haribo/shellf --pattern 'shellf-linux-amd64' --pattern 'SHA256SUMS'
sha256sum -c --ignore-missing SHA256SUMS # shellf-linux-amd64: OK
chmod +x shellf-linux-amd64
```

No tag, so this is the latest release — `gh` requires `--pattern` in that form, which is also
what keeps the download to the one architecture you want (swap in `shellf-linux-arm64`).
`--ignore-missing` is needed for the same reason: `SHA256SUMS` lists both binaries, and
without it the absent one fails the check. It still fails loudly on what matters — a binary
that did not arrive, or one whose bytes do not match, both exit non-zero.

`SHA256SUMS` covers both `shellf-linux-amd64` and `shellf-linux-arm64`, and is published
from v0.10.0 onward. The verification is the point rather than a formality: shellf refuses a
download it cannot verify — `file.download(url, dst, sha256)` takes the hash as a required
Expand Down Expand Up @@ -155,9 +161,12 @@ shellf run plan.shellf --secret-file rclone_pass=./secret --secret-env db=DB_PW
```

A secret is a variable like any other (`${rclone_pass}`), but shellf **redacts
its value** (`***`) from every report, `--dry-run`, and `status`. Honest limit: the
secret still reaches the target (in the request file, `0600`, and the process
env) — root there can read it; at-rest secrecy is not yet solved.
its value** (`***`) from every report, `--dry-run`, and `status`. At rest it is kept off
persistent disk: the agent's job files live on tmpfs (`/dev/shm`), so a secret is outside
backups, snapshots and undelete — falling back to `/tmp`, owner-only and deleted after, on a
host with no writable `/dev/shm` (ADR-0025). Honest limit: the secret still reaches the target,
in that job file and in the process environment, and **root there reads both in real time**.
No on-target scheme changes that, encryption included — the key would have to travel too.

**Control flow.** `if` takes an instruction (or a captured result); the branch is
taken on its outcome. `dir.exists` is a read-only *question*, so it stays honest
Expand Down Expand Up @@ -211,7 +220,10 @@ its success:
```
def install(pkg: str) as root {
observe {
return state(installed: shell { dpkg -s "$pkg" >/dev/null 2>&1 }.exit == 0)
# `install ok installed` is the one status meaning the files are on the host
return state(installed: shell {
dpkg-query -W -f='${Status}' "$pkg" 2>/dev/null | grep -q '^install ok installed'
}.exit == 0)
}
apply {
r = shell { apt-get install -y "$pkg" }
Expand All @@ -221,6 +233,12 @@ def install(pkg: str) as root {
}
```

That is `apt.install`'s real observe, and the comment is the whole lesson: **the question an
`observe` asks has to be the one its `apply` answers**. This def first asked `dpkg -s`, which
exits 0 for a package removed without `--purge` — its config files survive, its binaries do
not — so it reported `already` on a host where the package was gone. The wrong answer was
*stable*, so re-running agreed with itself and only the machine said otherwise (#486).

A field with no same-named argument (like `installed`) must simply hold;
fields that match a parameter (`service.ensure` → `running`, `git.clone` → `url`) are
compared to it. See [ADR-0013](docs/adr/0013-observe-state-contract.md).
Expand Down Expand Up @@ -352,6 +370,14 @@ agents and wipes shellf's files from the targets.
| `--known-hosts <file>` | host-key file (default `~/.ssh/known_hosts`) |
| `--insecure` | skip host-key verification (dev only) |
| `--agent-ttl <dur>` | resident agent inactivity TTL before it self-erases (default 2h) |
| `--parallel <n>` | hosts dialled at once (default 16); `1` serialises the fan-out |
| `--limit <host\|group>` | restrict the run to a host or group (repeatable); narrows the plan, never extends it |
| `--json` | report as JSON on stdout; diagnostics stay on stderr |
| `-v` | trace the control host's decisions, and every command run on the target |

`run` and `status` take the same flags, `--dry-run` excepted — it is a mode, and reading the
state without acting is what `status` already is. `clean` reads no plan and takes
`--inventory`, `--insecure` and `--known-hosts`.

## How it works

Expand Down
75 changes: 75 additions & 0 deletions cmd/shellf/flagdoc_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
package main

import (
"flag"
"os"
"strings"
"testing"
)

// Every flag the CLI accepts is documented in `README.md` (#646).
//
// #642 found four — `--json`, `-v`, `--parallel`, `--limit` — accepted by `run` and named in
// no document. They were not new: they had been shipping undocumented, and nothing would ever
// have said so. The check that found them was a throwaway written during that fix; this is it,
// kept, which is what should have shipped then.
//
// It walks the commands' own flag sets rather than parsing `main.go`: `runFlags`,
// `statusFlags` and `cleanFlags` are what the commands themselves call, so a flag cannot be
// added to one and missed by the other.
func TestEveryFlagIsDocumented(t *testing.T) {
readme, err := os.ReadFile("../../README.md")
if err != nil {
t.Fatal(err)
}
doc := string(readme)

// `--check` is the one exception, by name and with its reason: it is registered with an
// empty usage string so it stays out of `-h`, and exists only to tell an operator the
// flag is now `--dry-run` (ADR-0035). A documented `--check` would be the defect.
const undocumentedOnPurpose = "check"

runFS, _, _, _ := runFlags()
statusFS, _ := statusFlags()
cleanFS, _, _, _ := cleanFlags()

for _, c := range []struct {
cmd string
fs *flag.FlagSet
}{{"run", runFS}, {"status", statusFS}, {"clean", cleanFS}} {
c.fs.VisitAll(func(f *flag.Flag) {
if f.Name == undocumentedOnPurpose {
if f.Usage != "" {
t.Errorf("--%s carries a usage string now, so it shows in -h: "+
"either document it or keep it hidden", f.Name)
}
return
}
// The README writes a flag in backticks, either `--name` (with or without an
// argument after it) or `-v` for the single-letter one.
if strings.Contains(doc, "`--"+f.Name) || strings.Contains(doc, "`-"+f.Name+"`") {
return
}
t.Errorf("shellf %s accepts -%s and README.md does not mention it", c.cmd, f.Name)
})
}
}

// The exemption is one name, not a habit: if a second hidden flag appears, this fails and the
// decision gets made deliberately rather than by adding a line to a skip list.
func TestOnlyOneFlagIsHidden(t *testing.T) {
hidden := []string{}
runFS, _, _, _ := runFlags()
statusFS, _ := statusFlags()
cleanFS, _, _, _ := cleanFlags()
for _, fs := range []*flag.FlagSet{runFS, statusFS, cleanFS} {
fs.VisitAll(func(f *flag.Flag) {
if f.Usage == "" {
hidden = append(hidden, f.Name)
}
})
}
if len(hidden) != 1 || hidden[0] != "check" {
t.Fatalf("hidden flags: got %v, want [check] — a new one needs a decision, not a skip", hidden)
}
}
Loading
Loading