Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 15 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,12 +195,13 @@ unreachable endpoint. A minimal image that ships none of them is a case for

Warning: a docker node's image must run a long-lived foreground process.
DART creates the container from the image's own `CMD`/`ENTRYPOINT` with no
TTY and no attached stdin, and there is no `command:` option, so an image
whose default command is an interactive shell — `ubuntu`, `debian`,
`alpine` — exits the moment it starts. Node setup then polls for up to two
minutes waiting for the container to report running and fails with
`timeout waiting for container ... to become ready`. Purpose-built service
images work; bare distribution images belong on an `lxd` (or `lxd-vm`) or `ssh`
TTY and no attached stdin, so an image whose default command is an
interactive shell — `ubuntu`, `debian`, `alpine` — exits the moment it
starts. Node setup then polls for up to two minutes waiting for the
container to report running and fails with
`timeout waiting for container ... to become ready`. Give such an image a
`command:` that stays up (`command: ["sleep", "infinity"]`), use a
purpose-built service image, or put it on an `lxd` (or `lxd-vm`) or `ssh`
node instead.

### Does my config deploy correctly?
Expand Down Expand Up @@ -242,13 +243,13 @@ Both `create_dir` and `overwrite` default to false: without them the step
fails when the parent directory is missing, and fails again when the
destination already exists.

Note: local paths in file steps — `source` on `file_push` and
`file_template`, and `dest` on `file_fetch` — resolve against the directory
DART is invoked from, not the directory holding the suite file. Absolute
paths, or paths written relative to the runner's working directory, are the
safe form. Platform paths do not share this rule: `docker.images[].dockerfile`
and the `!!load_from` directive resolve relative to the suite file, so a
suite that mixes both cannot use one convention throughout.
Note: every local path a suite writes follows one rule — absolute paths are
used as-is, `~` is the invoking user's home directory, and anything else is
relative to the directory holding the suite file. That covers file-step
sources and destinations, docker `volumes`, LXD disk `source`s, SSH keys and
`known_hosts`, LXD certificates, `compose_file`, `docker.images[].dockerfile`,
and `!!load_from`. A suite is therefore portable: it behaves the same run from
the repository root, from its own directory, or from a CI checkout elsewhere.

### Is the package installable on a clean machine?

Expand All @@ -266,7 +267,7 @@ setup:
node: clean
step:
type: file_push
# source is read on the machine running DART, relative to its working directory
# source is read on the machine running DART, relative to the suite file
options: { source: dist/myservice.deb, dest: /tmp/myservice.deb }
- name: install it
node: clean
Expand Down
12 changes: 12 additions & 0 deletions cmd/dart/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -430,6 +430,18 @@ func runCheck(cfgPath, reportValue, varsValue, onlyValue, skipValue string) int
return 1
}

// Constraints across the whole node list — duplicate names, more than
// one local node — are the same ones a real run enforces
if err := nodetypes.ValidateNodeSet(cfg.Nodes); err != nil {
var cfgErr *config.ConfigError
if errors.As(err, &cfgErr) {
fmt.Fprint(os.Stderr, config.RenderConfigError(cfgErr))
} else {
fmt.Fprintf(os.Stderr, "\n%s %s\n\n", errorStyle.Sprint("Error:"), err)
}
return 1
}

mocks := make(map[string]ifaces.Node, len(cfg.Nodes))
for _, node := range cfg.Nodes {
// Unknown node types must fail --check exactly as they fail a run
Expand Down
148 changes: 90 additions & 58 deletions docs/node-types.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,16 @@ DART supports several types of nodes that can be used as test targets:
- **Local Node (`local`)**
Execute tests on the local machine where DART is running.
Invariant: at most one `local` node per suite. A second one fails configuration
with `only one local node allowed; "<name>" is a duplicate`, reported against
that node's line in the YAML. The limit applies only to `local`; other types may
with `only one local node allowed; "<name>" duplicates "<first>"`, reported
against that node's line in the YAML and caught by `--check`. The limit applies only to `local`; other types may
appear any number of times. Several roles on one machine are modelled with a
single local node, distinguished by test and step naming rather than by separate
node entries.

- **Docker Node (`docker`)**
Run tests inside Docker containers, with volume, environment, port, capability,
and privileged-mode options. Supports both local and remote Docker hosts.
privileged-mode, and command/entrypoint options. Supports both local and remote
Docker hosts.

- **Docker Compose Node (`docker-compose`)**
Manage and test services defined in Docker Compose files. Multiple nodes can target different services in the same compose stack.
Expand Down Expand Up @@ -135,10 +136,16 @@ on the target platform.
- **Docker Compose nodes:** the node name is used as the Compose project name when
`project_name` is omitted.

`container_name` (docker) and `instance_name` (lxd, lxd-vm) decouple the platform
identifier from the node identity. Both default to the node name, which is what
makes a suite's containers and instances findable by the name the YAML uses;
setting one is for suites that must match an externally fixed name. The node name
remains what `node:` references, what reports and console output show, and — for
docker — what the container's hostname is set to, so node-side commands still see
the name the suite uses.

Note: name syntax is not validated by DART. A name the platform rejects surfaces as
the daemon's or LXD server's own error during node setup. There is no
`container_name` or `instance_name` option that decouples the platform identifier
from the node identity.
the daemon's or LXD server's own error during node setup.

### Node Security Defaults

Expand All @@ -161,43 +168,49 @@ for an ephemeral target need not relax it for the long-lived jump host;
reconnects after `reboot` route through the bastion too, and chained
bastions are rejected rather than silently dropped.

`--check` validates the node options that need no connection: SSH
authentication (a key file that exists and parses, or a password),
`known_hosts` readability under the configured host-key policy, and — when a
`bastion:` block is present — that it names a host, carries usable
credentials, and is not chained. For docker nodes it checks `volumes` and
`ports` specification syntax, resolving relative volume host paths
(`./fixtures:/fixtures`) to absolute paths, since the Engine API would
otherwise treat them as *named volumes* and mount an empty one. These
breaking changes therefore surface before a run rather than during one.

Note: `--check` does not verify that required fields are present. An `ssh`
node missing `host` passes the check and then fails the run dialling `:22`; a
`docker` node missing `image` fails at container creation; a `docker-compose`
node missing `compose_file` fails at node construction. Only the *bastion's*
host is required at check time.
`--check` validates everything about a node that needs no connection:

- **Required fields** — `host` on `ssh`, `image` on `docker`, `compose_file` on
`docker-compose`, and a bastion's `host` when a `bastion:` block is present.
- **Option names** — any key the node type does not accept is an error naming
the accepted set (see [Unrecognised Options](#unrecognised-options)).
- **Credentials and host keys** — SSH authentication (a key file that exists and
parses, or a password), `known_hosts` readability under the configured
host-key policy, and that a bastion carries usable credentials and is not
chained.
- **Specification syntax** — docker `volumes` and `ports`, resolving relative
volume host paths (`./fixtures:/fixtures`) to absolute paths, since the Engine
API would otherwise treat them as *named volumes* and mount an empty one.
- **Cross-node constraints** — duplicate node names, and more than one `local`
node.

What remains outside its reach is anything that needs the platform to answer:
whether an image exists, whether a host is reachable, whether a bind source
exists on the daemon.

### Unrecognised Options

Option names must match exactly. On `docker`, `docker-compose`, `ssh`, and `lxd`
nodes, `options:` is decoded by a JSON round-trip into a typed struct, so any key
that is not a recognised option is discarded without an error or a warning. A
misspelling such as `priviliged` instead of `privileged`, `hostname` instead of
`host`, or `known_host` instead of `known_hosts` leaves the option at its default
and the suite runs on.

Only `local` nodes warn. A local node prints
`Warning: node "<name>": option "<key>" is not recognized and was ignored (known options: env, shell, sudo, exec_opts)`
to stderr. That warning is specific to local nodes and is not a general guarantee.

Note: `--check` does not detect option typos on any node type. It validates the
*semantics* of recognised options that need no connection, but it decodes options
through the same round-trip that drops unknown keys, and it substitutes mock nodes
for real ones — so even the local node's warning appears only in a real run.

A dropped SSH security key fails safe: a mistyped `insecure_skip_host_key` leaves
it `false`, and a mistyped `known_hosts` falls back to `~/.ssh/known_hosts`, so
host-key verification stays on and the symptom is a confusing connection error
Option names must match exactly. A key the node type does not accept is a
configuration error naming the offending key and the full accepted set:

```text
Error: node "web": unknown option "privilaged" for a docker node (accepted:
capabilities, command, container_name, entrypoint, env, exec_opts, image,
networks, ports, privileged, volumes)
```

Rationale: `options:` is decoded by a JSON round-trip into a typed struct, which
discards anything it does not recognise. Without this check a misspelling such as
`priviliged` for `privileged` left the option at its default while the suite read
as though it were set — the assertion looked configured and tested nothing.

`--check` reports these, so a typo surfaces before any infrastructure is created.
Local nodes additionally warn about keys misplaced inside `exec_opts`.

Historically a dropped SSH security key failed safe: a mistyped
`insecure_skip_host_key` left it `false`, and a mistyped `known_hosts` fell back
to `~/.ssh/known_hosts`, so host-key verification stayed on and the symptom was a
confusing connection error
rather than a silent downgrade. The real cost is a silently ineffective option — a
`privileged` or `capabilities` typo, for example, surfaces later as an unexplained
permission failure inside the container.
Expand Down Expand Up @@ -413,6 +426,9 @@ setup, and before any setup step runs. Consequences worth knowing:
| `ports` | list of `host:container[/proto]` | Published ports. |
| `privileged` | bool | Opt-in full host capabilities; defaults to `false`. |
| `capabilities` | list of strings | Individual Linux capabilities, for example `[NET_ADMIN]`. |
| `command` | list of strings | Overrides the image's `CMD`. Use it to give an image that would otherwise exit a process that stays in the foreground. |
| `entrypoint` | list of strings | Overrides the image's `ENTRYPOINT`. |
| `container_name` | string | The container's name on the daemon; defaults to the node name. |

Note: DART does not pull Docker images. The `image:` a docker node references must
already exist in the local daemon — pulled beforehand (`docker pull nginx:alpine`)
Expand All @@ -422,19 +438,33 @@ node setup with `could not create container: ...` followed by the daemon's
`No such image`. This applies to `type: docker` nodes only: `docker-compose` nodes
pull through Compose, and LXD/Incus nodes fetch images through the LXD client.

Note: the container is created from the image's own `CMD`/`ENTRYPOINT`. DART sets
the image, hostname, environment, published ports, bind mounts, and the privilege
options from the table above, and offers no `command`,
`entrypoint`, or `tty` option, so the image must run a process that stays in the
foreground. After starting the container, node setup polls every second for up to
two minutes until the container reports `Running` and a trivial `exec` of `true`
succeeds. An image whose `CMD` exits immediately — such as bare `ubuntu:latest`,
whose `CMD` is `/bin/bash` and which exits at once because DART allocates no TTY
and attaches no stdin — never becomes ready, and setup fails after two minutes with
The container is created from the image's own `CMD`/`ENTRYPOINT` unless
`command:` or `entrypoint:` overrides them. DART allocates no TTY and attaches no
stdin, so the process it runs must stay in the foreground. After starting the
container, node setup polls every second for up to two minutes until the container
reports `Running` and a trivial `exec` of `true` succeeds. An image whose `CMD`
exits immediately — bare `ubuntu:latest`, whose `CMD` is `/bin/bash` — never
becomes ready, and setup fails after two minutes with
`container <name> not ready: timeout waiting for container ... context deadline exceeded`.
A service image (`nginx:alpine`, `postgres:16`) or a purpose-built image whose
`CMD` is a supervisor satisfies the check; `examples/docker/docker.yaml` builds
exactly such an image through the `docker.images` block.

Three ways to satisfy the readiness check:

- a service image whose `CMD` already stays up (`nginx:alpine`, `postgres:16`);
- a bare distribution image plus a `command:` that stays up:

```yaml
nodes:
- name: shellbox
type: docker
options:
image: ubuntu:24.04
command: ["sleep", "infinity"]
```

- a purpose-built image whose `CMD` is a supervisor, as
`examples/docker/docker.yaml` builds through the `docker.images` block.

Note: there is no `tty` option. A command that requires a terminal still fails.

Warning: `networks` on a `docker` node is not implemented. The option parses but is
never applied — `DockerNode.Setup` does not read it, and containers are created
Expand Down Expand Up @@ -524,8 +554,8 @@ dart -c config.yaml

Warning: `volumes` host paths are resolved on the machine running DART but
interpreted by the daemon. DART expands a leading `~` from the local `$HOME` and
makes any relative path absolute against DART's working directory; the result is
handed to the daemon as-is. With a remote `DOCKER_HOST`, `./fixtures:/fixtures`
makes any relative path absolute against the suite file's directory; the result
is handed to the daemon as-is. With a remote `DOCKER_HOST`, `./fixtures:/fixtures`
becomes a local absolute path the daemon host probably does not have — and a bind
source that does not exist is created as an empty directory rather than failing, so
a test can read nothing and still pass. `--check` validates the
Expand Down Expand Up @@ -573,6 +603,7 @@ docker compose -f <compose_file> -p <project_name> down
| `boot_wait` | map | — | Replaces the default readiness check; see [Empty VMs and ISO Boot](#empty-vms-and-iso-boot). |
| `exec_opts` | map | — | Currently one key, `shell`, defaulting to `/bin/bash`. |
| `project` | string | `default` | LXD project the instance is created in. Not inherited from `lxd.project`. |
| `instance_name` | string | the node name | The instance's name on the LXD/Incus server. |
| `socket` | string | auto-detected | Unix socket path; used only when the suite has no top-level `lxd:` block. |
| `server`, `protocol` | string | `local`, `lxd` | Image server URL and protocol; used only with a bare image alias. |
| `remote_addr`, `trust_token`, `client_cert`, `client_key`, `server_cert`, `skip_verify` | — | — | Remote connection settings; used only when the suite has no top-level `lxd:` block. See [Remote LXD Support](#remote-lxd-support). |
Expand Down Expand Up @@ -776,10 +807,11 @@ Notes:
setting both `empty: true` and `image` is rejected.
- `devices` accepts any LXD device configuration and is merged over the NICs generated from
[`networks`](#networks), so a node can override a generated device if it needs to.
- Relative `source` paths on pool-less disk devices are made absolute against DART's working
directory — not the suite file's directory, unlike `docker.images[].dockerfile`. A disk
device that names a `pool` refers to a storage volume and is passed through untouched, as
are all sources on remote nodes, which are paths on the remote server.
- Relative `source` paths on pool-less disk devices are made absolute against the suite
file's directory, the same rule `docker.images[].dockerfile` and every other local path
follows. A disk device that names a `pool` refers to a storage volume and is passed
through untouched, as are all sources on remote nodes, which are paths on the remote
server.
- `boot_wait` replaces the default readiness check: DART polls `ready_command` through the
node's shell (`exec_opts.shell`, default `/bin/bash`) until it exits zero or the timeout
expires. Without `ready_command`, being able to run any command at all counts as ready. An
Expand Down
24 changes: 11 additions & 13 deletions docs/steps.md
Original file line number Diff line number Diff line change
Expand Up @@ -507,20 +507,18 @@ bit, and an existing destination overwritten with `overwrite: true` keeps its
current mode. A following `execute` step with `chmod` covers the cases where that
is not what is wanted.

Warning: local paths in these steps — `source` for `file_push` and
`file_template`, `dest` for `file_fetch` — resolve against the working directory
DART is invoked from, not against the directory holding the suite file. This
differs from `docker.images[].dockerfile` and `load_from`, which the loader
rewrites relative to the config file's directory. Running
`dart -c examples/foo/suite.yaml` from the repository root therefore looks for
`fixtures/app.conf.tmpl` at `./fixtures/app.conf.tmpl`, not at
`examples/foo/fixtures/app.conf.tmpl`. The two failures surface at different
times: a missing `file_template` source fails at step construction with
Local paths in these steps — `source` for `file_push` and `file_template`,
`dest` for `file_fetch` — follow the same rule as every other local path a
suite writes: absolute paths are used as-is, `~` expands to the invoking user's
home directory, and anything else is relative to the directory holding the
suite file. Running `dart -c examples/foo/suite.yaml` from the repository root
therefore reads `fixtures/app.conf.tmpl` at `examples/foo/fixtures/app.conf.tmpl`,
and the same command works unchanged from any directory.

Note: a missing source still surfaces at different times by step type. A
missing `file_template` source fails at step construction with
`cannot read template <path> in step "<name>"`, before any step runs, while a
missing `file_push` source fails mid-run with `failed to read source <path>`, and
a `file_fetch` `dest` is simply created relative to the working directory.
Absolute paths, `{{env.*}}`/`{{var.*}}` substitution used to build them, or
always invoking DART from a fixed directory all avoid the ambiguity.
missing `file_push` source fails mid-run with `failed to read source <path>`.

Content to container and SSH nodes is written in 32 KiB base64 chunks, so files
are not limited by the shell's per-argument size cap. That write is not atomic:
Expand Down
Loading
Loading