Raw shell, but idempotent, previewable, and fast.

shellf runs configuration as plain shell — but structured so every action is idempotent (skip when already in the desired state), previewable (a dry-run shows what would change, without touching anything), and fast. It needs nothing installed on the target: a single static binary pushes itself over SSH and evaluates the plan on the target. The pushed agent stays resident between jobs, so a second run costs no round trip; after an inactivity TTL (default 2h) it removes its workdir and its own binary and exits, leaving nothing behind.

Status: experimental. Ships a stdlib of instructions (apt.install, service.ensure, dir.ensure, file.*, ufw.*, docker.*, …) written as shellf defs, plus a raw shell form; you write the plan and inventory. Instruction libraries can be shared: a local package by path, or a remote module pinned by tag in shellf.lock. Targets linux/amd64 and linux/arm64 — a release binary carries both agents and pushes the one the target runs (ADR-0048). Cross-distro agents are not there yet: Debian/systemd targets.

Install

Download a release and check what you got:

gh release download v0.10.0 --repo haribo/shellf
sha256sum -c SHA256SUMS      # shellf-linux-amd64: OK
chmod +x shellf-linux-amd64

SHA256SUMS covers both shellf-linux-amd64 and shellf-linux-arm64, and is published from v0.10.0 onward. The verification is the point rather than a formality: shellf refuses a download it cannot verify — file.download(url, dst, sha256) takes the hash as a required argument — so installing it any other way would be the one unchecked step of a shellf-managed setup.

Or build it:

CGO_ENABLED=0 go build -o shellf ./cmd/shellf

A static binary either way. The same binary is what gets pushed to targets as the agent.

Quickstart

Describe your hosts in an inventory file (hosts.shellf):

defaults = { user: "root", port: "22" }

host web1 = { address: "10.0.0.1" }
host web2 = { address: "10.0.0.2" }

group web = [web1, web2]

Authentication uses your ssh-agent (SSH_AUTH_SOCK) by default, so an encrypted key never leaves the agent. To pin a specific key instead, add key: "~/.ssh/id_…" to defaults or a host (it is an optional override).

Describe what to do in a plan file (plan.shellf):

on web {
  apt.install("nginx")
  file.copy("/tmp/nginx.conf", "/etc/nginx/nginx.conf")
  service.ensure("nginx", "true", "true")       # running now, enabled at boot
}

Preview first — this touches nothing, and reports what would change:

shellf run --inventory hosts.shellf --dry-run plan.shellf

Then apply:

shellf run --inventory hosts.shellf plan.shellf

Hosts run in parallel; steps run in order per host. A re-run is idempotent: an instruction that finds the state it wants reports ok.already and does nothing.

Inventory

Construct	Form
Defaults	defaults = { user: "…", port: "…" }
Host	host <alias> = { address: "…", user: "…", port: "…" }
Group	group <name> = [<alias>, <alias>]

Omitted host fields fall back to defaults, then to 22 for the port. Only address is required. A host may belong to several groups. key: "…" is an optional field (a pinned ssh key); without it, authentication uses the ssh-agent.

A host with local: "true" is provisioned on the control host itself, with no SSH — host self = { local: "true" } (no address needed). Same agent, plan, and reports as a remote target; as root still goes through sudo.

Plan

Construct	Form
Block	on <group-or-host> { <steps> } — blocks run sequentially
Parallel	parallel { <steps> } — branches run concurrently on one host

Blocks target a group (or a single host). A host that errors is dropped from later blocks.

Writing shellf

A worked tour. Runnable examples live under examples/ — one project with two plans, start with webserver.shellf, then the containerized blog.shellf for user defs, secrets, templates, and docker/ufw; see examples/README.md.

A project is laid out by type (ADR-0038): plans/ holds what a run invokes, defs/<pkg>/ the reusable instructions (called pkg.name, no import), assets/ the content a plan delivers, inventories/ the hosts.

Variables come from the inventory (per-host) or --set. Use them as bare identifiers, or ${name} inside strings:

host web = { address: "10.0.0.1", pkg: "nginx", webroot: "/var/www/app" }

on web {
  apt.install("${inventory.pkg}")           # a host's own field, named at the call site
  dir.ensure("${inventory.webroot}")
}

Secrets (ADR-0018) come from a file or an env var — never the command line (so not in ps/history) and never the plan (so not committed):

shellf run plan.shellf --secret-file rclone_pass=./secret --secret-env db=DB_PW

A secret is a variable like any other (${rclone_pass}), but shellf redacts its value (***) from every report, --dry-run, and status. Honest limit: the secret still reaches the target (in the request file, 0600, and the process env) — root there can read it; at-rest secrecy is not yet solved.

Control flow. if takes an instruction (or a captured result); the branch is taken on its outcome. dir.exists is a read-only question, so it stays honest in --dry-run:

if dir.exists("/opt/app") { service.ensure("app", "true", "true") }
if !dir.exists("/opt/app") { dir.ensure("/opt/app") }   # act only when absent

Capture and match outcomes. An instruction returns a Result: a tagged outcome (ok/err + a tag), plus a changed flag.

x = file.write("/etc/app.conf", cfg)
if x { restart() }               # `if x` = it succeeded; `if !x` = it failed
if x == ok.written { reload() }  # match a specific tag
if x.changed { … }               # did it actually act (not a converged skip)?

Error handling. By default the first err halts the host. To handle a specific error, mark the instruction ? and test it — testing == err without a ? is a compile error (the branch would be unreachable):

x = apt.install("nginx")?
if x == err.dbLocked { retry() } else { report() }

Privilege escalation. shellf runs as the SSH user. as <user> escalates a block (via sudo/doas); many stdlib defs (apt.install, service.ensure, …) declare as root themselves and escalate on their own:

on web {
  apt.install("nginx")           # escalates itself (intrinsic `as root`)
  as root {                      # escalate a block of generic instructions
    dir.ensure("/opt/app")
    file.write("/etc/app.conf", cfg)
  }
}

Custom instructions are defs written in shellf. A def has phases (observe read-only, apply effectful) and returns an outcome. observe reports the current state as a state(...) record; shellf compares each field to the arguments and runs apply only on a mismatch — no hand-written skip. Inside a def, a shell {} is a struct — read .exit/.stdout, and if r / if !r test its success:

def install(pkg: str) as root {
  observe {
    return state(installed: shell { dpkg -s "$pkg" >/dev/null 2>&1 }.exit == 0)
  }
  apply {
    r = shell { apt-get install -y "$pkg" }
    if !r { return err.runtime(r) }
    return ok.installed
  }
}

A field with no same-named argument (like installed) must simply hold; fields that match a parameter (service.ensure → running, git.clone → url) are compared to it. See ADR-0013.

Preview, then apply. --dry-run runs only the read-only phases, never mutates, and prints would.<tag> for what would change. A second real run is idempotent (everything reads ok.already). shellf status reports the same observed state as current → desired, without acting.

Instructions

Most instructions are defs written in shellf and embedded in the binary; only shell and seven primitives — ~file.read, ~file.write, ~file.render, ~dir.list, ~dir.sync, ~text.matches, ~text.replace — are built in. Most are idempotent: a def that declares observe skips its apply when the desired state already holds. Some are action-shaped and always act — service.restart, docker.compose-up — because restarting a service has no "already restarted" to observe; --dry-run says would for those rather than pretending otherwise.

• Packages & services — apt.install(pkg) · apt.update() · service.ensure(name, running, enabled) (running/enabled are "true"/"false"; a .timer unit works as the name) · service.restart(name) · service.reload(name) · systemd.daemon-reload() · systemd.unit(name, content) (installs a unit file, refusing one systemd-analyze verify rejects before it reaches /etc; a timer is a unit) · user.group(user, group) · user.ensure(name, shell)
• Files & directories — file.copy(%"src", dst) (deliver a file from the control host, binary-safe) · file.template(%"src", dst) (render a control-host file's ~{var} and deliver it — src must be marked %"…", an unmarked path is refused) · dir.copy(%"src", dst, compare) (deliver a control-host tree verbatim, binary-safe; sends only what differs, so a converged tree transfers nothing — compare defaults to size+mtime, pass "sha256" when a change may preserve both) · file.write(path, content) · file.mode(path, mode) · file.replace(path, key, value) (a key=value line) · file.line(path, line) · file.delete(path) · file.download(url, dst, sha256) · dir.sync(%"src", dst, compare) (same transfer, and it removes what the source does not have — --dry-run names every file it would delete) · dir.ensure(path) · dir.owner(path, owner) · dir.mode(path, mode) · file.owner(path, owner) · file.ensure(path, mode) (create with an exact mode if absent, never touch the content of one that exists) · archive.extract(src, dst) · archive.extract-member(src, dst, member) (one file out of a tarball) · git.clone(url, dst) · git.sync(url, dst, ref) (update to a pinned ref)
• Questions (read-only, deterministic in --dry-run) — dir.exists(path) · file.exists(path) · http.check(url, status) · http.wait-for(url, timeout) (retries until ready)
• Validated configs — sudo.write(name, content) (checked with visudo -cf, set 0440) · sshd.config(name, content) (checked with sshd -t -f). The check runs before anything is written, and in --dry-run too, so an invalid file is caught before it can lock you out
• System — sysctl.set(key, value) (live and persisted — a value written to /etc/sysctl.d and never applied does nothing until a reboot) · system.timezone(zone)
• PostgreSQL — postgres.role(name, password) (verifies the password by connecting with it) · postgres.database(name, owner) · postgres.config(key, value) · postgres.hba(rule) (both ask postgres where its files are, so no major version is written into a path)
• Credentials — htpasswd.entry(path, user, password) (one file holds several accounts; the file is left at 0600)
• Firewall — ufw.enable() · ufw.default(incoming, outgoing) · ufw.open(port, proto)
• Docker — docker.install() · docker.network(name) · docker.compose-up(dir, build) (build "true" rebuilds local images; always re-applies — up -d is idempotent) · docker.prune(until) (reclaims disk from unused images; docker system prune is deliberately not covered — it removes volumes) · docker.compose-restart(dir, service) (handler — omit service.ensure for the whole stack; gate it on .changed, e.g. after a mounted config is edited)

Primitives (ADR-0036) — ~ marks an engine primitive (no phases, no override), and % marks a path on your machine: ~file.read(path) reads — on your machine if the path is marked, on the target otherwise — ~file.write(path, bytes) writes on the target, ~file.render(%"path") reads a template on your machine and substitutes its ~{var} there, and ~dir.list(path) lists a directory, and ~dir.sync(src, dst, delete, compare) transfers a tree. Two more read and rewrite a value rather than a file, with the agent's own RE2 and a literal replacement (ADR-0055): ~text.matches(s, pattern) answers true or false, and ~text.replace(s, pattern, repl) rewrites every match — which is how a def refuses an argument it cannot honour without shelling out to the target. Only those seven names may carry a ~; anything else is a parse error, and the control host serves only the paths the plan marked — a render included, which is why it names a file rather than carrying one.

def deliver(src: str, dst: str) {
  apply { file.write(dst, ~file.render(src)) }
}

on web { deliver(%"app.conf.j2", "/etc/app.conf") }

Write your own — see Writing shellf.

Raw shell

The first-class citizen: run anything the builtins don't cover. shell is a special form, not a name(args) call.

on server {
  shell docker compose up -d              # one-line: ends at the newline

  shell {                                 # block: raw, verbatim (heredocs work)
    curl -fsSL https://get.docker.com | sh
  }

  # idempotent + previewable: gate the effect on a read-only test
  if !shell { docker network inspect web } {
    shell { docker network create web }
  }
}

• A bare shell always runs (raw, like bash; not previewable).
• To make it idempotent and previewable, gate it: if !shell { <test> } { shell { <cmd> } } — the test is read-only, so --dry-run runs only the test, never the command.
• Escalate with shell as root { … } (see Writing shellf).
• A block ends at its balanced }. A lone unbalanced } in a string ends it early — use a heredoc or the one-line form.

The escape hatch is explicit, and countable

Shell that an instruction already does does not parse:

shell { mkdir -p /opt/app }
→ mkdir here — dir.ensure(path) is idempotent and previewable.
  Write `unsafe shell { … }` to keep the shell.

unsafe shell { … } keeps it, and runs exactly like shell. unsafe does not mean dangerous — it means shellf cannot vouch for what the block does. An atomic lock (mkdir /var/lock/x || exit) is irreproachable shell that no instruction replaces, and it is marked, correctly.

That is what makes the hatch countable: grep -r 'unsafe shell' lists every place shellf's guarantees stop, imported modules included. Configuration tools lose on the moment you give up on the module and write shell; here that moment is visible.

And it is a signal, not a verdict. An unsafe shell doing something every deployment would want is a missing instruction — worth an issue carrying the block it would replace. That is how the standard library grows: examples/plans/hosting.shellf is a real deployment written this way, and the six blocks it needed became six instructions. It now carries none.

See docs/language.md for the detector's rules and its limits, and docs/dogfood.md for the method.

CLI

shellf run    --inventory <hosts.shellf> [flags] <plan.shellf>
shellf status --inventory <hosts.shellf> [flags] <plan.shellf>
shellf clean  --inventory <hosts.shellf> [flags] [target...]
shellf version

run applies, status reports observed state without acting, clean kills the resident agents and wipes shellf's files from the targets.

Flag	Meaning
--inventory <file>	inventory file (required)
--dry-run	decide and preview, never mutate
--vars <file>	global name = value bindings
--set k=v	override one variable (repeatable); wins over --vars
--secret-file n=path	secret read from a file (repeatable); redacted in all output
--secret-env n=VAR	secret read from an environment variable (repeatable)
--known-hosts <file>	host-key file (default ~/.ssh/known_hosts)
--insecure	skip host-key verification (dev only)
--agent-ttl <dur>	resident agent inactivity TTL before it self-erases (default 2h)

How it works

One static binary. The control host orchestrates; pushed over SSH, it re-runs as a resident agent that executes the plan locally on each target, and exits after its inactivity TTL.

Two planes, kept separate. The orchestration plane (control host) decides which hosts run what, in what order. The execution plane is the agent: the binary is pushed over SSH (cached by hash, skipped on re-runs), evaluates the plan on the target, and stays resident between jobs — then self-erases after an idle TTL, leaving nothing after a reboot. Guards are read-only, which is what makes --dry-run honest across a whole fleet.

Shell values are passed to the target via the environment, never concatenated into commands — so a value like nginx; rm -rf / cannot inject a command.