diff --git a/CLAUDE.md b/CLAUDE.md index 1ee0dc3..bd3535b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -150,7 +150,17 @@ These are deliberate design decisions, not oversights. Do not "improve" them wit `late_command` or a chroot script, where one `curl` is writable and composing Proxmox's JSON is not. - `src/config.rs` — environment configuration. `Config::from_lookup` takes a lookup closure so - tests never touch the process environment. + tests never touch the process environment — which is also the seam every configuration + file hangs off: a file is just another source behind that closure. +- `src/tomlconfig.rs` — the optional **TOML** file `RESCRIPTUM_CONFIG` names, for the + platform where a person edits configuration by hand. It **maps a document onto the same + `RESCRIPTUM_*` names and does nothing else**, so one place still decides what a setting + means and the file cannot grow behaviour the environment lacks. `MAPPING` is that table, + and a unit test asserts it covers `envfile::KNOWN_KEYS` exactly — a setting missing from + it is one the file silently cannot configure. Writes go through `toml_edit`, which edits + the document in place: **replace the value, never the entry**, because a setting's + explanation lives in the *key's* decor and inserting over the key throws the paragraph + away. - `src/envfile.rs` — the optional file of defaults `RESCRIPTUM_ENV_FILE` names, and the writer behind `config set`: `rewrite()` edits lines where they stand, **uncommenting** a commented setting rather than appending a duplicate, because on a packaged install those @@ -401,9 +411,16 @@ spend into an apparent 293% overrun.** | Build | Bytes | |---|---| -| `sqlite` + `boot` (default) | 2,741,360 | -| `sqlite` only | 2,482,000 | -| neither | 1,316,648 | +| `sqlite` + `boot` (default) | 2,813,712 | +| `sqlite` only | 2,557,592 | +| `boot` only | 1,649,048 | +| neither | 1,392,544 | + +Re-measured 2026-08-29 on armv7-gnueabihf (floor 2.17), all four in one sitting. **Both +tables that held these numbers were stale by roughly 200 KB** — this one and +`docs/guide/reference/configuration`, which disagreed with each other as well. +`boot` costs **1,164,120** against `sqlite` alone by this measurement; the budget question +below is written against the older figure and needs re-deciding against this one. **`boot` costs 259,360 bytes, against a ≤170 KB budget the plan set before any of it was written** — the image-source catalogue added 31,520 of that. That is recorded in `plans/boot-media.md` with a per-phase breakdown rather @@ -745,11 +762,14 @@ file or failure). When a PXE install won't start, this is the only diagnostic av ## Configuration -Environment variables only — plus an optional file to read some of them from: +Environment variables — plus an optional file to read them from, in either of two shapes. +Both files set the same variables under the same rules; **the environment wins over both, +and the TOML file wins over the env file**: | Variable | Default | Role | |---|---|---| -| `RESCRIPTUM_ENV_FILE` | unset | A file of the same variables. See below | +| `RESCRIPTUM_CONFIG` | unset | A **TOML** file of the same settings, under readable names. See below | +| `RESCRIPTUM_ENV_FILE` | unset | A `KEY=value` file of the same variables. See below | | `RESCRIPTUM_STORE` | `files` | `files` or `sqlite` | | `RESCRIPTUM_ANSWERS_DIR` | `/srv/answers` | Directory of answer documents | | `RESCRIPTUM_DB_PATH` | `/srv/answers.db` | SQLite database, when `RESCRIPTUM_STORE=sqlite` | @@ -803,6 +823,23 @@ It is not a shell: no `${}` expansion, no inline comments (a `#` in a value is p value — truncating a token silently is worse than a comment landing in a value, where it is loud), `export` accepted so one file can also be sourced, a duplicate key is an error. +**`RESCRIPTUM_CONFIG` (`src/tomlconfig.rs`)** is the same job in the shape a person reads: +the prefix goes away and tables do the grouping (`store.kind`, `admin.token`, +`server.workers`). It exists because on DSM there is no environment — there is a file — and +`RESCRIPTUM_ANSWERS_DIR=…` on every line is a poor thing to hand somebody editing in File +Station. Every rule above carries over unchanged, and three things are specific to it: + +- **It costs a mapping, not a dependency.** `toml_edit` already parses every answer + document. Measured on armv7: **+14,544 bytes** (2,799,168 → 2,813,712), 0.5%. +- **`""` is unset**, the same rule an exported-but-empty variable has — which is what lets + `config unset` empty a line instead of deleting the paragraph documenting it. A list or a + table where a value belongs is a startup *error*, unlike a misspelled key, which warns: + it was aimed at a real setting, so serving the default would be the silent failure. +- **A configuration file must not live in the answers directory.** Every servable `.toml` + at the top of that directory is an answer document, and this format shares the extension + — `check` reports one dropped there as a misplaced answer and `migrate` offers to move + it. A test pins that rather than leaving it to be discovered. + ## Commands Local development (once the crate exists): @@ -880,7 +917,7 @@ is the procedure*), which `AGENTS.md` also points at. ## Testing expectations -582 tests, plus the package's own harnesses (see *The DSM package*, and note that +613 tests, plus the package's own harnesses (see *The DSM package*, and note that `cargo test` does not run those). `docs/development/testing.md` has the per-suite table; the rules that decide where a test goes: diff --git a/docs/guide/reference/cli.fr.md b/docs/guide/reference/cli.fr.md index 65053f5..21c84a4 100644 --- a/docs/guide/reference/cli.fr.md +++ b/docs/guide/reference/cli.fr.md @@ -24,12 +24,13 @@ Sans argument, `rescriptum` lance le serveur. Tout le reste est une sous-command | `rescriptum config` | afficher la configuration, et d'où vient chaque valeur | | `rescriptum config --json` | la même chose, pour un panneau de réglages | | `rescriptum config --value CLÉ` | une valeur, pour un script — jamais un identifiant | -| `rescriptum config set C=V …` | éditer le fichier que `RESCRIPTUM_ENV_FILE` nomme | -| `rescriptum config unset CLÉ …` | recommenter un réglage dedans | +| `rescriptum config set C=V …` | éditer le fichier que `RESCRIPTUM_CONFIG` ou `RESCRIPTUM_ENV_FILE` nomme | +| `rescriptum config unset CLÉ …` | y retirer un réglage | | `rescriptum --help` | usage et variables d'environnement | Toutes lisent les mêmes [variables d'environnement](./configuration.md), dont -[`RESCRIPTUM_ENV_FILE`](./configuration.md#le-fichier-denvironnement) — résolu en premier, +[`RESCRIPTUM_CONFIG`](./configuration.md#le-fichier-toml) et +[`RESCRIPTUM_ENV_FILE`](./configuration.md#le-fichier-denvironnement) — résolus en premier, donc un fichier illisible arrête toute commande ayant besoin de la configuration. `--help` et `--version` répondent avant sa lecture, parce que ce sont les commandes qu'on lance quand quelque chose ne va pas. Il n'y a pas d'options globales. @@ -124,10 +125,16 @@ env file: /var/packages/rescriptum/etc/rescriptum.env RESCRIPTUM_ADMIN_TOKEN (set) file ``` -La troisième colonne est l'essentiel. Le fichier fournit des **valeurs par défaut** et -l'environnement réel l'emporte : une valeur marquée `environment` ne peut donc pas être -changée en éditant le fichier — et `config set` le dit, plutôt que de vous laisser écrire -quelque chose que le serveur en cours continuera d'ignorer. +La troisième colonne est l'essentiel. Les fichiers fournissent des **valeurs par défaut** +et l'environnement réel l'emporte : une valeur marquée `environment` ne peut donc pas être +changée en éditant un fichier — et `config set` le dit, plutôt que de vous laisser écrire +quelque chose que le serveur en cours continuera d'ignorer. Avec un fichier TOML la colonne +affiche `toml file`, et nommer les deux fichiers affiche les deux chemins ainsi que l'ordre +dans lequel ils l'emportent. + +**`config set` écrit dans le fichier TOML quand les deux sont nommés**, parce que c'est +celui que le serveur lit en premier : écrire l'autre serait une modification qui ne change +rien en silence. **Un identifiant n'est jamais affiché**, sous aucune forme de cette commande. Un jeton apparaît comme `(set)` ou `(not set)` ; `--value` refuse tout net. @@ -140,7 +147,10 @@ wrote /var/packages/rescriptum/etc/rescriptum.env L'écriture laisse le fichier tel qu'il est par ailleurs : les commentaires restent, un réglage est remplacé là où il se trouve, et un réglage commenté est **décommenté sur place** plutôt qu'ajouté en dessous — ce qui compte quand le commentaire au-dessus est la seule -documentation qu'a le fichier. +documentation qu'a le fichier. Dans un fichier TOML, le même soin s'applique au document : +la valeur est remplacée là où elle est, son commentaire de fin de ligne survit, et `config +unset` **vide la valeur au lieu de supprimer la ligne**, pour que le paragraphe qui +explique le réglage reste en place. Deux refus sont délibérés : diff --git a/docs/guide/reference/cli.md b/docs/guide/reference/cli.md index 31d5057..e09a92b 100644 --- a/docs/guide/reference/cli.md +++ b/docs/guide/reference/cli.md @@ -24,12 +24,13 @@ With no arguments, `rescriptum` runs the server. Everything else is a subcommand | `rescriptum config` | show the configuration, and where each value comes from | | `rescriptum config --json` | the same, for a settings panel | | `rescriptum config --value KEY` | one value, for a script — never a credential | -| `rescriptum config set K=V …` | edit the file `RESCRIPTUM_ENV_FILE` names | -| `rescriptum config unset KEY …` | comment a setting back out of it | +| `rescriptum config set K=V …` | edit the file `RESCRIPTUM_CONFIG` or `RESCRIPTUM_ENV_FILE` names | +| `rescriptum config unset KEY …` | take a setting back out of it | | `rescriptum --help` | usage and the environment variables | All of them read the same [environment variables](./configuration.md), including -[`RESCRIPTUM_ENV_FILE`](./configuration.md#the-env-file) — which is resolved first, so a +[`RESCRIPTUM_CONFIG`](./configuration.md#the-toml-file) and +[`RESCRIPTUM_ENV_FILE`](./configuration.md#the-env-file) — which are resolved first, so a file that cannot be read stops any command that needs configuration. `--help` and `--version` are answered before it is read, because they are what you reach for when something is wrong. There are no global flags. @@ -123,9 +124,14 @@ env file: /var/packages/rescriptum/etc/rescriptum.env RESCRIPTUM_ADMIN_TOKEN (set) file ``` -The third column is the point. The file supplies **defaults** and the real environment -wins, so a value marked `environment` cannot be changed by editing the file — and `config -set` says so rather than letting you write something the running server will ignore. +The third column is the point. The files supply **defaults** and the real environment +wins, so a value marked `environment` cannot be changed by editing a file — and `config +set` says so rather than letting you write something the running server will ignore. With +a TOML file the column reads `toml file`, and naming both files prints both paths plus the +order they win in. + +**`config set` writes the TOML file when both are named**, because it is the one the server +reads first: writing the other would be a change that silently does nothing. **A credential is never printed**, by any form of this command. A token shows as `(set)` or `(not set)`; `--value` refuses outright. @@ -137,7 +143,10 @@ wrote /var/packages/rescriptum/etc/rescriptum.env Writing keeps the file as it is otherwise: comments stay, a setting is replaced where it stands, and one that is commented out is **uncommented in place** rather than appended -below — which matters when the comment above it is the only documentation the file has. +below — which matters when the comment above it is the only documentation the file has. In +a TOML file the same care applies to the document: the value is replaced where it stands, +its trailing comment survives, and `config unset` **empties the value rather than deleting +the line**, so the paragraph explaining the setting stays where it was. Two refusals are deliberate: diff --git a/docs/guide/reference/configuration.fr.md b/docs/guide/reference/configuration.fr.md index 2a6410e..07bcfa3 100644 --- a/docs/guide/reference/configuration.fr.md +++ b/docs/guide/reference/configuration.fr.md @@ -1,6 +1,6 @@ --- title: Configuration -description: Chaque variable d'environnement, sa valeur par défaut, et ce qui arrive quand on se trompe. +description: Chaque variable d'environnement, sa valeur par défaut, les deux formats de fichier qui peuvent les fournir, et ce qui arrive quand on se trompe. sidebar: label: Configuration order: 1 @@ -8,14 +8,18 @@ sidebar: # Configuration -Des variables d'environnement — et, en option, un fichier d'où en lire une partie. Il n'y a -pas de *format* de configuration à apprendre ni de ligne de commande à se tromper. +Des variables d'environnement — et, en option, un fichier d'où les lire, dans l'une de deux +formes. Il n'y a pas de ligne de commande à se tromper, et les variables *sont* toute la +configuration : les deux formats de fichier règlent exactement les mêmes choses sous +exactement les mêmes règles, donc rien de ce qu'on écrit dans un fichier ne peut signifier +quelque chose que l'environnement ne dirait pas. ## Les variables | Variable | Défaut | Signification | |---|---|---| -| `RESCRIPTUM_ENV_FILE` | non défini | Lire aussi les valeurs par défaut depuis ce fichier — voir [plus bas](#le-fichier-denvironnement) | +| `RESCRIPTUM_CONFIG` | non défini | Lire les valeurs par défaut depuis ce fichier **TOML** — voir [plus bas](#le-fichier-toml) | +| `RESCRIPTUM_ENV_FILE` | non défini | Lire les valeurs par défaut depuis ce fichier `CLÉ=valeur` — voir [plus bas](#le-fichier-denvironnement) | | `RESCRIPTUM_STORE` | `files` | `files` (un répertoire) ou `sqlite` (une base) | | `RESCRIPTUM_ANSWERS_DIR` | `/srv/answers` | Répertoire des documents de réponse | | `RESCRIPTUM_DB_PATH` | `/srv/answers.db` | Chemin de la base, quand `RESCRIPTUM_STORE=sqlite` | @@ -93,6 +97,99 @@ qu'il ne peut pas signaler quelque chose. La rotation vous incombe. Sous systemd il n'y a rien à faire, le log part dans le journal ; avec un fichier, pointez `logrotate` dessus avec `copytruncate`. +## Le fichier TOML + +`RESCRIPTUM_CONFIG` nomme un fichier en TOML qui règle les mêmes variables, dans une forme +faite pour être lue. C'est celui vers lequel se tourner quand **une personne édite le +fichier à la main** — sur un NAS, dans File Station ou via SMB — c'est-à-dire exactement là +où `RESCRIPTUM_ANSWERS_DIR=…` sur chaque ligne se lit mal, et où le mot « environnement » +envoie chercher un shell qui n'existe pas. + +```toml +# /etc/rescriptum.toml (chmod 600, appartenant à root) +answers_dir = "/srv/answers" +listen_addr = "0.0.0.0:8000" +log = "problems" # all | problems | off + +[store] +kind = "sqlite" +db_path = "/srv/answers.db" + +[server] +workers = 2 +max_connections = 2048 +timeout_secs = 10 + +[admin] +addr = "127.0.0.1:8001" +token = "…" + +[answer] +token = "…" +capture_dir = "/var/lib/rescriptum/captures" +``` + +```console +$ RESCRIPTUM_CONFIG=/etc/rescriptum.toml rescriptum +2026-08-29T12:42:02Z - reading configuration defaults from /etc/rescriptum.toml (8 set) +``` + +Toutes les règles du fichier d'environnement valent ici aussi : **jamais découvert, +seulement nommé** (il n'y a pas de `./rescriptum.toml`) ; **l'environnement réel gagne** ; +et **un fichier demandé et illisible est une erreur de démarrage**, jamais un +avertissement. + +**Placez-le hors du répertoire de réponses.** Tout `.toml` servable à la racine de ce +répertoire est un document réponse, et ce format partage l'extension — un fichier de +configuration déposé là est signalé par `check` comme une réponse mal placée, et `migrate` +propose de le déplacer. `/etc` est le foyer évident ; sur une installation empaquetée, le +paquet en choisit un. + +### Les noms + +Le préfixe disparaît et les tables font le regroupement. Rien d'autre ne change : chaque +ligne ci-dessous est la variable du même nom, et `rescriptum config` affiche les deux +orthographes. + +| Dans le fichier | Variable | +|---|---| +| `answers_dir` | `RESCRIPTUM_ANSWERS_DIR` | +| `listen_addr` | `RESCRIPTUM_LISTEN_ADDR` | +| `log`, `log_file` | `RESCRIPTUM_LOG`, `RESCRIPTUM_LOG_FILE` | +| `public_host` | `RESCRIPTUM_PUBLIC_HOST` | +| `user`, `group` | `RESCRIPTUM_USER`, `RESCRIPTUM_GROUP` | +| `store.kind`, `store.db_path` | `RESCRIPTUM_STORE`, `RESCRIPTUM_DB_PATH` | +| `server.workers`, `server.max_connections`, `server.timeout_secs` | `RESCRIPTUM_WORKERS`, `RESCRIPTUM_MAX_CONNECTIONS`, `RESCRIPTUM_TIMEOUT_SECS` | +| `admin.addr`, `admin.token` | `RESCRIPTUM_ADMIN_ADDR`, `RESCRIPTUM_ADMIN_TOKEN` | +| `answer.token`, `answer.capture_dir` | `RESCRIPTUM_ANSWER_TOKEN`, `RESCRIPTUM_CAPTURE_DIR` | +| `media.dir`, `media.addr`, `media.timeout_secs`, `media.max_connections` | les quatre `RESCRIPTUM_MEDIA_*` | +| `boot.dir`, `boot.allow`, `boot.unclaimed`, `boot.timeout_secs`, `boot.logo`, `boot.title` | les six `RESCRIPTUM_BOOT_*` | +| `tftp.addr`, `tftp.port_range`, `tftp.blksize` | les trois `RESCRIPTUM_TFTP_*` | +| `installed.token` | `RESCRIPTUM_INSTALLED_TOKEN` | + +### Le format + +| | | +|---|---| +| N'importe quel scalaire TOML | un nombre peut s'écrire en nombre (`workers = 2`) ou en chaîne ; les deux arrivent au serveur comme le même réglage | +| Un commentaire `#` | n'importe où, y compris en fin de ligne — contrairement au fichier d'environnement, qui n'a pas d'échappements et ne peut donc pas en avoir | +| Une valeur avec un `#`, un guillemet ou une espace | sans problème, échappée comme TOML échappe, et relue à l'identique | +| `""` | **non défini** — la même règle qu'une variable exportée mais vide, et c'est ce qui permet à `config unset` de vider une ligne au lieu de supprimer le paragraphe qui la documente | +| La même clé deux fois | refusée par TOML lui-même, donc le fichier ne se charge pas | +| Une clé que ce programme ne lit pas | un avertissement la nommant : `admin.tokenn` est attrapé au lieu d'être ignoré | +| Une liste ou une table là où une valeur est attendue | une **erreur de démarrage** : contrairement à une faute de frappe, elle visait un réglage réel, et servir la valeur par défaut alors que le fichier dit le contraire serait silencieux | +| Un fichier lisible par d'autres | un avertissement avec son mode, parce qu'il peut contenir `admin.token` | + +Les avertissements nomment les clés et les chemins, jamais les valeurs. + +### Les deux fichiers à la fois + +Nommer les deux est une transition plutôt qu'un état stable : rien n'est refusé et l'ordre +est annoncé au démarrage — **l'environnement bat le fichier TOML, qui bat le fichier +d'environnement.** `rescriptum config` montre lequel des trois a mis chaque valeur en +vigueur, et `config set` écrit dans le fichier TOML — celui que le serveur lit en premier, +pour qu'une écriture ne puisse pas être une modification qui ne change rien en silence. + ## Le fichier d'environnement `RESCRIPTUM_ENV_FILE` nomme un fichier contenant les mêmes variables. Il existe pour les @@ -143,11 +240,13 @@ Les avertissements nomment les clés et les chemins, jamais les valeurs. ## Le lire et le modifier -`rescriptum config` affiche chaque variable, sa valeur, et **qui du fichier ou de -l'environnement l'y a mise** — la distinction qui compte, puisque le fichier fournit des -valeurs par défaut et que l'environnement réel l'emporte. `config set` modifie le fichier -comme on voudrait qu'il le soit : commentaires conservés, réglage commenté décommenté sur -place plutôt que dupliqué, et refus avant toute écriture d'une modification qui laisserait +`rescriptum config` affiche chaque variable, sa valeur, et **qui des fichiers ou de +l'environnement l'y a mise** — la distinction qui compte, puisque les fichiers fournissent +des valeurs par défaut et que l'environnement réel l'emporte. `config set` modifie le +fichier comme on voudrait qu'il le soit, dans l'un ou l'autre format : commentaires +conservés, réglage commenté décommenté sur place plutôt que dupliqué (fichier +d'environnement) ou valeur remplacée là où elle est (TOML), et refus avant toute écriture +d'une modification qui laisserait un serveur incapable de démarrer. C'est documenté dans la [référence de la ligne de commande](./cli.md#config), et c'est ce que l'[application DSM](../operations/synology.md#lapplication-de-bureau) pilote dessous. @@ -160,7 +259,8 @@ l'[application DSM](../operations/synology.md#lapplication-de-bureau) pilote des | Uniquement des espaces | pareil, et les valeurs sont trimées | | Un nombre nul ou impossible à parser | retombe sur la **valeur par défaut**, plutôt que de démarrer un serveur qui accepte des connexions sans jamais répondre | | `RESCRIPTUM_STORE` avec toute autre valeur | un avertissement, et `files` est utilisé | -| `RESCRIPTUM_ENV_FILE` nommant un fichier absent, illisible ou malformé | une **erreur** de démarrage | +| `RESCRIPTUM_ENV_FILE` ou `RESCRIPTUM_CONFIG` nommant un fichier absent, illisible ou malformé | une **erreur** de démarrage | +| Un réglage TOML recevant une liste ou une table | une **erreur** de démarrage, contrairement à une clé mal orthographiée, qui avertit | | `RESCRIPTUM_STORE=sqlite` sur un binaire construit sans la feature | une **erreur** au démarrage | ## Erreurs de démarrage @@ -204,15 +304,17 @@ Ceux-ci sont affichés et le serveur continue : | `sqlite` | activée | Le store SQLite et l'API d'administration | | `boot` | activée | Le catalogue de médias, le lecteur ISO et le listener média | -Mesuré sur ARMv7 (gnueabihf, plancher glibc 2.17). Remesurez plutôt que de citer ces -chiffres : ils ont bougé d'environ 375 Ko quand cette cible est passée de musl à glibc. +Mesuré sur ARMv7 (gnueabihf, plancher glibc 2.17), les quatre d'affilée le 2026-08-29. +Remesurez plutôt que de citer ces chiffres : ils ont bougé d'environ 375 Ko quand cette +cible est passée de musl à glibc, et le jeu qu'ils remplacent ici avait dérivé d'environ +200 Ko. | Build | Octets | |---|---| -| les deux (défaut) | 2 602 056 | -| `sqlite` seule | 2 482 000 | -| `boot` seule | 1 436 704 | -| aucune | 1 316 648 | +| les deux (défaut) | 2 813 712 | +| `sqlite` seule | 2 557 592 | +| `boot` seule | 1 649 048 | +| aucune | 1 392 544 | ## Limites fixes diff --git a/docs/guide/reference/configuration.md b/docs/guide/reference/configuration.md index e7b40e2..6702790 100644 --- a/docs/guide/reference/configuration.md +++ b/docs/guide/reference/configuration.md @@ -1,6 +1,6 @@ --- title: Configuration -description: Every environment variable, its default, and what happens when you get one wrong. +description: Every environment variable, its default, the two file formats that can supply them, and what happens when you get one wrong. sidebar: label: Configuration order: 1 @@ -8,14 +8,17 @@ sidebar: # Configuration -Environment variables only — and, optionally, a file to read some of them from. There is -no configuration *format* to learn and no command line to get wrong. +Environment variables — and, optionally, a file to read them from, in either of two +shapes. There is no command line to get wrong, and the variables are the whole +configuration: both file formats set exactly the same things under exactly the same +rules, so nothing you can write in a file means anything the environment could not. ## The variables | Variable | Default | Meaning | |---|---|---| -| `RESCRIPTUM_ENV_FILE` | unset | Read defaults from this file too — see [below](#the-env-file) | +| `RESCRIPTUM_CONFIG` | unset | Read defaults from this **TOML** file — see [below](#the-toml-file) | +| `RESCRIPTUM_ENV_FILE` | unset | Read defaults from this `KEY=value` file — see [below](#the-env-file) | | `RESCRIPTUM_STORE` | `files` | `files` (a directory) or `sqlite` (a database) | | `RESCRIPTUM_ANSWERS_DIR` | `/srv/answers` | Directory of answer documents | | `RESCRIPTUM_DB_PATH` | `/srv/answers.db` | Database path, when `RESCRIPTUM_STORE=sqlite` | @@ -89,6 +92,96 @@ fail every install in flight in order to report that it could not report somethi Rotation is yours. Under systemd there is nothing to do, since the log goes to the journal; with a file, point `logrotate` at it with `copytruncate`. +## The TOML file + +`RESCRIPTUM_CONFIG` names a file in TOML that sets the same variables in a shape meant to +be read. Reach for it when a **person edits the file by hand** — on a NAS, in File Station +or over SMB — which is exactly where `RESCRIPTUM_ANSWERS_DIR=…` on every line reads +poorly and where the word "environment" sends people looking for a shell that is not +there. + +```toml +# /etc/rescriptum.toml (chmod 600, owned by root) +answers_dir = "/srv/answers" +listen_addr = "0.0.0.0:8000" +log = "problems" # all | problems | off + +[store] +kind = "sqlite" +db_path = "/srv/answers.db" + +[server] +workers = 2 +max_connections = 2048 +timeout_secs = 10 + +[admin] +addr = "127.0.0.1:8001" +token = "…" + +[answer] +token = "…" +capture_dir = "/var/lib/rescriptum/captures" +``` + +```console +$ RESCRIPTUM_CONFIG=/etc/rescriptum.toml rescriptum +2026-08-29T12:42:02Z - reading configuration defaults from /etc/rescriptum.toml (8 set) +``` + +Every rule the env file has, this one has too: **never discovered, only named** (there is +no `./rescriptum.toml`); **the real environment wins**; and **a file that was asked for and +cannot be read is a startup error**, never a warning. + +**Put it outside the answers directory.** Every servable `.toml` at the top of that +directory is an answer document, and this format shares the extension — a configuration +file dropped in there is reported by `check` as a misplaced answer, and `migrate` offers +to move it. `/etc` is the obvious home; on a packaged install the package chooses one. + +### The names + +The prefix goes away and tables do the grouping. Nothing else changes: each line below is +the variable of the same name, and `rescriptum config` prints both spellings. + +| In the file | Variable | +|---|---| +| `answers_dir` | `RESCRIPTUM_ANSWERS_DIR` | +| `listen_addr` | `RESCRIPTUM_LISTEN_ADDR` | +| `log`, `log_file` | `RESCRIPTUM_LOG`, `RESCRIPTUM_LOG_FILE` | +| `public_host` | `RESCRIPTUM_PUBLIC_HOST` | +| `user`, `group` | `RESCRIPTUM_USER`, `RESCRIPTUM_GROUP` | +| `store.kind`, `store.db_path` | `RESCRIPTUM_STORE`, `RESCRIPTUM_DB_PATH` | +| `server.workers`, `server.max_connections`, `server.timeout_secs` | `RESCRIPTUM_WORKERS`, `RESCRIPTUM_MAX_CONNECTIONS`, `RESCRIPTUM_TIMEOUT_SECS` | +| `admin.addr`, `admin.token` | `RESCRIPTUM_ADMIN_ADDR`, `RESCRIPTUM_ADMIN_TOKEN` | +| `answer.token`, `answer.capture_dir` | `RESCRIPTUM_ANSWER_TOKEN`, `RESCRIPTUM_CAPTURE_DIR` | +| `media.dir`, `media.addr`, `media.timeout_secs`, `media.max_connections` | the `RESCRIPTUM_MEDIA_*` four | +| `boot.dir`, `boot.allow`, `boot.unclaimed`, `boot.timeout_secs`, `boot.logo`, `boot.title` | the `RESCRIPTUM_BOOT_*` six | +| `tftp.addr`, `tftp.port_range`, `tftp.blksize` | the `RESCRIPTUM_TFTP_*` three | +| `installed.token` | `RESCRIPTUM_INSTALLED_TOKEN` | + +### The format + +| | | +|---|---| +| Any TOML scalar | a number may be written as a number (`workers = 2`) or as a string; both reach the server as the same setting | +| A `#` comment | anywhere, including at the end of a line — unlike the env file, which has no escapes and so cannot have inline comments | +| A value with a `#`, a quote or a space | fine, quoted the way TOML quotes things, and read back unchanged | +| `""` | **unset** — the same rule as an exported-but-empty variable, which is what lets `config unset` empty a line instead of deleting the paragraph that documents it | +| The same key twice | refused by TOML itself, so the file does not load | +| A key this program does not read | a warning naming it, so `admin.tokenn` is caught rather than ignored | +| A list or a table where a value belongs | a **startup error**: unlike a misspelling it was aimed at a real setting, and serving the default while the file says otherwise would be silent | +| A file others can read | a warning with its mode, because it may hold `admin.token` | + +Warnings name keys and paths, never values. + +### Both files at once + +Naming both is a transition rather than a steady state, so nothing is refused and the +order is stated at startup: **the environment beats the TOML file, which beats the env +file.** `rescriptum config` shows which of the three put every value in force, and +`config set` writes to the TOML file — the one the server reads first, so that a write +cannot be a change that silently does nothing. + ## The env file `RESCRIPTUM_ENV_FILE` names a file of the same variables. It exists for deployments with @@ -138,12 +231,13 @@ Warnings name keys and paths, never values. ## Reading and editing it -`rescriptum config` prints every variable, its value, and **which of the file and the -environment put it there** — the distinction that matters, because the file supplies +`rescriptum config` prints every variable, its value, and **which of the files and the +environment put it there** — the distinction that matters, because the files supply defaults and the real environment wins. `config set` edits the file the way you would want -it edited: comments kept, a commented-out setting uncommented in place rather than -duplicated, and a change that would leave a server unable to start refused before anything -is written. It is documented in the [command line reference](./cli.md#config), and it is +it edited, in either format: comments kept, a commented-out setting uncommented in place +rather than duplicated (in the env file) or the value replaced where it stands (in TOML), +and a change that would leave a server unable to start refused before anything is +written. It is documented in the [command line reference](./cli.md#config), and it is what the [DSM application](../operations/synology.md#the-desktop-application) drives underneath. @@ -155,7 +249,8 @@ underneath. | Whitespace-only | same, and values are trimmed | | A zero or unparseable number | falls back to the **default**, rather than starting a server that accepts connections and never answers | | `RESCRIPTUM_STORE` set to anything else | a warning, and `files` is used | -| `RESCRIPTUM_ENV_FILE` naming a missing, unreadable or malformed file | a startup **error** | +| `RESCRIPTUM_ENV_FILE` or `RESCRIPTUM_CONFIG` naming a missing, unreadable or malformed file | a startup **error** | +| A TOML setting given a list or a table | a startup **error**, unlike a misspelled key, which warns | | `RESCRIPTUM_STORE=sqlite` on a binary built without the feature | a startup **error** | ## Startup errors @@ -199,15 +294,16 @@ These are printed and the server carries on: | `sqlite` | on | The SQLite store and the admin API | | `boot` | on | The media catalogue, the ISO reader and the media listener | -Measured on ARMv7 (gnueabihf, glibc floor 2.17). Re-measure rather than quoting these: -they moved by about 375 KB when that target changed from musl. +Measured on ARMv7 (gnueabihf, glibc floor 2.17), all four in one sitting on 2026-08-29. +Re-measure rather than quoting these: they moved by about 375 KB when that target changed +from musl, and the set they replace here had drifted about 200 KB out of date. | Build | Bytes | |---|---| -| both (default) | 2,602,056 | -| `sqlite` only | 2,482,000 | -| `boot` only | 1,436,704 | -| neither | 1,316,648 | +| both (default) | 2,813,712 | +| `sqlite` only | 2,557,592 | +| `boot` only | 1,649,048 | +| neither | 1,392,544 | ## Fixed limits diff --git a/src/cli.rs b/src/cli.rs index ef0ddf0..2fcdb7e 100644 --- a/src/cli.rs +++ b/src/cli.rs @@ -28,8 +28,8 @@ USAGE: rescriptum config show the configuration, and where each value comes from rescriptum config --json the same, for a settings panel rescriptum config --value K one value, for a script (never a credential) - rescriptum config set K=V edit the file RESCRIPTUM_ENV_FILE names - rescriptum config unset K comment a setting back out of it + rescriptum config set K=V edit the configuration file, whichever one is named + rescriptum config unset K take a setting back out of it rescriptum media list the installer images this server holds rescriptum media add FILE register one already in the media directory rescriptum media add URL fetch one into it, then register it @@ -42,7 +42,8 @@ USAGE: rescriptum --help ENVIRONMENT: - RESCRIPTUM_ENV_FILE read these from a file too (the real environment wins) + RESCRIPTUM_CONFIG read these from a TOML file (the real environment wins) + RESCRIPTUM_ENV_FILE read these from a KEY=value file, same rules RESCRIPTUM_STORE files | sqlite (default files) RESCRIPTUM_ANSWERS_DIR directory of answer files (default /srv/answers) RESCRIPTUM_DB_PATH sqlite database (default /srv/answers.db) @@ -1641,21 +1642,14 @@ fn boot_check(cfg: &Config) -> ExitCode { /// The exit code is a contract, like `check`'s: **zero when the configuration would /// start**, one when it would not, or when a write was refused. pub fn config(args: &[String]) -> ExitCode { - let path = std::env::var(crate::envfile::ENV_FILE) - .ok() - .map(|v| v.trim().to_string()) - .filter(|v| !v.is_empty()); + let named = Named::from_environment(); match args.split_first() { - None => show(path.as_deref(), false), - Some((flag, rest)) if flag == "--json" && rest.is_empty() => show(path.as_deref(), true), - Some((flag, rest)) if flag == "--value" && rest.len() == 1 => { - value(path.as_deref(), &rest[0]) - } - Some((cmd, rest)) if cmd == "set" && !rest.is_empty() => edit(path.as_deref(), rest, true), - Some((cmd, rest)) if cmd == "unset" && !rest.is_empty() => { - edit(path.as_deref(), rest, false) - } + None => show(&named, false), + Some((flag, rest)) if flag == "--json" && rest.is_empty() => show(&named, true), + Some((flag, rest)) if flag == "--value" && rest.len() == 1 => value(&named, &rest[0]), + Some((cmd, rest)) if cmd == "set" && !rest.is_empty() => edit(&named, rest, true), + Some((cmd, rest)) if cmd == "unset" && !rest.is_empty() => edit(&named, rest, false), _ => { eprintln!( "usage: rescriptum config\n\ @@ -1669,16 +1663,89 @@ pub fn config(args: &[String]) -> ExitCode { } } +/// The configuration files, **as named** — never discovered. Either, both or neither may +/// be set; a container configures the environment directly and names none. +struct Named { + toml: Option, + env: Option, +} + +impl Named { + fn from_environment() -> Named { + let named = |var: &str| { + std::env::var(var) + .ok() + .map(|v| v.trim().to_string()) + .filter(|v| !v.is_empty()) + }; + Named { + toml: named(crate::tomlconfig::CONFIG_FILE), + env: named(crate::envfile::ENV_FILE), + } + } + + /// Where a write lands. **The TOML file when both are named**, because that is the + /// one the server reads first — writing the other would produce a change that + /// silently does nothing, which is the failure this whole area exists to remove. + fn target(&self) -> Option<&str> { + self.toml.as_deref().or(self.env.as_deref()) + } +} + +/// Both files, loaded: what they set, what is worth saying about them, and — separately — +/// the reason one of them could not be read at all. +/// +/// A file that will not parse still **prints**, rather than being a single error line: +/// seeing the other settings next to the reason the file is broken is what makes this +/// usable. The failure is returned apart from the warnings because it is not one — an +/// unreadable file is a startup *error*, so it has to reach the exit code. +struct Loaded { + env: Option, + toml: Option, + warnings: Vec, + unreadable: Option, +} + +fn load_files(named: &Named) -> Loaded { + let mut loaded = Loaded { + env: None, + toml: None, + warnings: Vec::new(), + unreadable: None, + }; + if let Some(path) = &named.toml { + match crate::tomlconfig::TomlFile::load(path.as_str()) { + Ok(file) => { + loaded.warnings.extend(file.warnings.clone()); + loaded.toml = Some(file); + } + Err(e) => loaded.unreadable = Some(e), + } + } + if let Some(path) = &named.env { + match crate::envfile::EnvFile::load(path.as_str()) { + Ok(file) => { + loaded.warnings.extend(file.warnings.clone()); + loaded.env = Some(file); + } + // The first failure is the one reported: two broken files is one problem to + // fix at a time, and the TOML file is the one the server reads first. + Err(e) => loaded.unreadable = loaded.unreadable.or(Some(e)), + } + } + loaded +} + /// One value, on stdout, for a script that wants it. /// -/// The alternative is a shell reading the env file with `sed`, which gets the *defaults* +/// The alternative is a shell reading the file with `sed`, which gets the *defaults* /// wrong: a variable absent from the file is not unset, it is whatever this program falls -/// back to. Precedence goes the same way — the environment beats the file — and neither is -/// visible to something grepping a file. +/// back to. Precedence goes the same way — the environment beats both files — and none of +/// it is visible to something grepping a file. /// /// **A secret is never printed**, whatever is asked. Exit code one means "no such value", /// so `if v=$(rescriptum config --value KEY)` reads correctly. -fn value(path: Option<&str>, key: &str) -> ExitCode { +fn value(named: &Named, key: &str) -> ExitCode { let Some(known) = crate::config::KNOWN.iter().find(|k| k.key == key) else { eprintln!("{key} is not a variable this program reads"); return ExitCode::FAILURE; @@ -1688,12 +1755,12 @@ fn value(path: Option<&str>, key: &str) -> ExitCode { return ExitCode::FAILURE; } - let (file, _, unreadable) = load_file(path); - if let Some(reason) = unreadable { + let loaded = load_files(named); + if let Some(reason) = loaded.unreadable { eprintln!("{reason}"); return ExitCode::FAILURE; } - match crate::config::settings(file.as_ref(), from_environment) + match crate::config::settings(loaded.env.as_ref(), loaded.toml.as_ref(), from_environment) .into_iter() .find(|s| s.key == key) .and_then(|s| s.value) @@ -1706,63 +1773,63 @@ fn value(path: Option<&str>, key: &str) -> ExitCode { } } -/// Load the named file, if one is named at all. -/// -/// A file that will not parse still **prints**, rather than being a single error line: -/// seeing the other twelve variables next to the reason the file is broken is what makes -/// this usable. It is returned separately from the warnings because it is not one — an -/// unreadable env file is a startup *error*, so it has to reach the exit code. -fn load_file(path: Option<&str>) -> (Option, Vec, Option) { - match path { - None => (None, Vec::new(), None), - Some(p) => match crate::envfile::EnvFile::load(p) { - Ok(file) => { - let warnings = file.warnings.clone(); - (Some(file), warnings, None) - } - Err(e) => (None, Vec::new(), Some(e)), - }, - } -} - /// The environment as `settings` and `Config` both want to read it. fn from_environment(key: &str) -> Option { std::env::var(key).ok() } -/// Rebuild the configuration exactly as the server would, from a file plus the real +/// Rebuild the configuration exactly as the server would, from both files plus the real /// environment, so that what this command reports is what would actually happen. -fn effective(file: Option<&crate::envfile::EnvFile>) -> Config { +fn effective(loaded: &Loaded) -> Config { Config::from_lookup(|key| { std::env::var(key) .ok() .filter(|v| !v.trim().is_empty()) - .or_else(|| file.and_then(|f| f.get(key))) + .or_else(|| loaded.toml.as_ref().and_then(|f| f.get(key))) + .or_else(|| loaded.env.as_ref().and_then(|f| f.get(key))) }) } -fn show(path: Option<&str>, as_json: bool) -> ExitCode { - let (file, problems, unreadable) = load_file(path); - let settings = crate::config::settings(file.as_ref(), from_environment); +fn show(named: &Named, as_json: bool) -> ExitCode { + let loaded = load_files(named); + let settings = + crate::config::settings(loaded.env.as_ref(), loaded.toml.as_ref(), from_environment); // Either of these stops a server starting, so either of them is the answer here. // A file that cannot be read comes first: it is the more basic failure, and the // configuration `validate` would inspect is not the one the operator wrote. - let refusal = unreadable.or_else(|| effective(file.as_ref()).validate().err()); + let refusal = loaded + .unreadable + .clone() + .or_else(|| effective(&loaded).validate().err()); if as_json { println!( "{}", - as_json_text(path, &settings, &problems, refusal.as_deref()) + as_json_text(named, &settings, &loaded.warnings, refusal.as_deref()) ); } else { - match path { - Some(p) => println!("env file: {p}"), + match (&named.toml, &named.env) { // Not an error. Plenty of deployments configure a container or a unit file // and have nothing for this to edit; saying so beats an empty line. - None => println!( - "env file: none — {} names one, and nothing does", + (None, None) => println!( + "config file: none — {} and {} name one, and nothing does", + crate::tomlconfig::CONFIG_FILE, crate::envfile::ENV_FILE ), + (toml, env) => { + if let Some(path) = toml { + println!("toml file: {path}"); + } + if let Some(path) = env { + println!("env file: {path}"); + } + // Only worth a line when it could actually change an answer above. + if toml.is_some() && env.is_some() { + println!( + "(the toml file wins where they disagree; the environment wins over both)" + ); + } + } } println!(); @@ -1776,7 +1843,7 @@ fn show(path: Option<&str>, as_json: bool) -> ExitCode { println!(" {:, as_json: bool) -> ExitCode { } fn as_json_text( - path: Option<&str>, + named: &Named, settings: &[crate::config::Setting], problems: &[String], refusal: Option<&str>, @@ -1809,13 +1876,21 @@ fn as_json_text( "default": s.default, "secret": s.secret, "help": s.help, + // The name this setting has in a TOML file, so a panel can show the line + // somebody would edit by hand rather than only the environment name. + "path": crate::tomlconfig::path_for(s.key), }) }) .collect(); serde_json::json!({ - "env_file": path, - "writable": path.is_some_and(writable), + // Kept under its old name: the DSM panel reads it, and the env file is still the + // file a packaged install writes. + "env_file": named.env, + "toml_file": named.toml, + // Which of them a write would land in — the only one `writable` can be about. + "target": named.target(), + "writable": named.target().is_some_and(writable), "settings": rows, "warnings": problems, "starts": refusal.is_none(), @@ -1849,14 +1924,19 @@ fn writable(path: &str) -> bool { } } -fn edit(path: Option<&str>, args: &[String], setting: bool) -> ExitCode { - let Some(path) = path else { +fn edit(named: &Named, args: &[String], setting: bool) -> ExitCode { + let Some(path) = named.target() else { eprintln!( - "there is no file to edit: {} names one, and nothing does", + "there is no file to edit: {} and {} name one, and nothing does", + crate::tomlconfig::CONFIG_FILE, crate::envfile::ENV_FILE ); return ExitCode::FAILURE; }; + // The format follows the variable that named the file, not the extension: a + // deployment that calls its TOML file `rescriptum.conf` is still writing TOML, and + // guessing from a suffix is how a file gets rewritten in the wrong language. + let toml = named.toml.is_some(); let mut changes: std::collections::BTreeMap> = Default::default(); for arg in args { @@ -1875,7 +1955,17 @@ fn edit(path: Option<&str>, args: &[String], setting: bool) -> ExitCode { // A misspelled name would otherwise be written, read back as a stranger, and // warned about only at the next start — by which time nobody connects the two. if !crate::envfile::KNOWN_KEYS.contains(&key.as_str()) { - eprintln!("{key} is not a variable this program reads — check the spelling"); + // Somebody reading the TOML file types the name they see there. Answer with + // the one that works rather than with "no such setting". + match crate::tomlconfig::key_for(&key) { + Some(env_key) => eprintln!( + "{key} is what this setting is called inside the file; on the command \ + line it is {env_key}" + ), + None => { + eprintln!("{key} is not a variable this program reads — check the spelling") + } + } return ExitCode::FAILURE; } if changes.insert(key.clone(), value).is_some() { @@ -1896,12 +1986,22 @@ fn edit(path: Option<&str>, args: &[String], setting: bool) -> ExitCode { return ExitCode::FAILURE; } }; - if let Err(e) = crate::envfile::parse(&text) { + let sound = if toml { + crate::tomlconfig::parse(&text).map(|_| ()) + } else { + crate::envfile::parse(&text).map(|_| ()) + }; + if let Err(e) = sound { eprintln!("{path} does not parse, so it will not be edited: {e}"); return ExitCode::FAILURE; } - let rewritten = match crate::envfile::rewrite(&text, &changes) { + let rewritten = if toml { + crate::tomlconfig::rewrite(&text, &changes) + } else { + crate::envfile::rewrite(&text, &changes) + }; + let rewritten = match rewritten { Ok(text) => text, Err(e) => { eprintln!("{e}"); @@ -1912,18 +2012,30 @@ fn edit(path: Option<&str>, args: &[String], setting: bool) -> ExitCode { // **A write may never leave a server that cannot start.** The same reasoning as the // admin API's rollback: the panel doing the writing is reached over the very service // this would stop, so getting it wrong costs somebody an SSH session at best. - let parsed = match crate::envfile::parse(&rewritten) { + let parsed = if toml { + crate::tomlconfig::parse(&rewritten).map(|(vars, _)| vars) + } else { + crate::envfile::parse(&rewritten) + }; + let parsed = match parsed { Ok(vars) => vars, Err(e) => { eprintln!("refusing to write a file this program could not read back: {e}"); return ExitCode::FAILURE; } }; + // The *other* file still applies underneath this one, so the question is what the + // server would make of both together — not of this file alone. + let other = load_files(&Named { + toml: None, + env: named.env.clone().filter(|_| toml), + }); let would = Config::from_lookup(|key| { std::env::var(key) .ok() .filter(|v| !v.trim().is_empty()) .or_else(|| parsed.get(key).cloned()) + .or_else(|| other.env.as_ref().and_then(|f| f.get(key))) }); if let Err(reason) = would.validate() { eprintln!("refused: this would leave a server that cannot start — {reason}"); diff --git a/src/config.rs b/src/config.rs index 553f842..d9bd2ca 100644 --- a/src/config.rs +++ b/src/config.rs @@ -146,12 +146,33 @@ impl Config { /// silent failure the file exists to remove. Warnings about it are logged here, the /// way `Capture::new` reports its own. pub fn from_env() -> Result { - let named = std::env::var(crate::envfile::ENV_FILE) - .ok() - .map(|v| v.trim().to_string()) - .filter(|v| !v.is_empty()); + let named = |var: &str| { + std::env::var(var) + .ok() + .map(|v| v.trim().to_string()) + .filter(|v| !v.is_empty()) + }; - let file = match named { + // Loaded before the env file because it wins over it, and reported the same way: + // one line saying where the defaults came from and how many there are, then + // anything the file itself deserves to be told about. + let toml = match named(crate::tomlconfig::CONFIG_FILE) { + Some(path) => { + let file = crate::tomlconfig::TomlFile::load(path)?; + crate::log::server(&format!( + "reading configuration defaults from {} ({} set)", + file.path.display(), + file.len() + )); + for warning in &file.warnings { + crate::log::server(&format!("warning: {warning}")); + } + Some(file) + } + None => None, + }; + + let file = match named(crate::envfile::ENV_FILE) { Some(path) => { let file = crate::envfile::EnvFile::load(path)?; crate::log::server(&format!( @@ -167,12 +188,26 @@ impl Config { None => None, }; + // Two configuration files is a transition, not a steady state — a deployment + // moving to TOML, most likely mid-upgrade. Saying which one wins costs one line + // and removes the only question an operator could not answer by reading either + // file. + if toml.is_some() && file.is_some() { + crate::log::server(&format!( + "note: {} and {} are both named — the TOML file wins where they set the \ + same thing, and the environment wins over both", + crate::tomlconfig::CONFIG_FILE, + crate::envfile::ENV_FILE + )); + } + Ok(Config::from_lookup(|key| { std::env::var(key) .ok() // An exported-but-empty variable is a mistake, not an instruction — so it - // does not count as "set in the environment" and the file still applies. + // does not count as "set in the environment" and the files still apply. .filter(|v| !v.trim().is_empty()) + .or_else(|| toml.as_ref().and_then(|f| f.get(key))) .or_else(|| file.as_ref().and_then(|f| f.get(key))) })) } @@ -911,22 +946,28 @@ pub const KNOWN: [Known; 30] = [ }, ]; -/// Which of the three places a value came from. +/// Which of the four places a value came from, in the order they win. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum Source { - /// The process environment, which **wins over the file**. + /// The process environment, which **wins over both files**. Environment, + /// The file `RESCRIPTUM_CONFIG` names, which wins over the env file. + TomlFile, /// The file `RESCRIPTUM_ENV_FILE` names. - File, + EnvFile, /// Nothing set it. Default, } impl Source { + /// **These strings are a contract**: `config --json` emits them, and the DSM panel + /// decides whether a field is editable by comparing against `environment` — a value + /// the environment sets cannot be changed by writing a file. pub fn label(self) -> &'static str { match self { Source::Environment => "environment", - Source::File => "file", + Source::TomlFile => "toml file", + Source::EnvFile => "env file", Source::Default => "default", } } @@ -946,8 +987,8 @@ pub struct Setting { pub help: &'static str, } -/// Describe every variable: what is in force, and **which of the file and the environment -/// put it there**. +/// Describe every variable: what is in force, and **which of the two files and the +/// environment put it there**. /// /// That distinction is the entire reason this returns a source rather than a map. The /// file supplies defaults and the real environment wins, so anything offering to edit the @@ -958,6 +999,7 @@ pub struct Setting { /// exported-but-empty variable is a mistake, not an instruction. pub fn settings( file: Option<&crate::envfile::EnvFile>, + toml: Option<&crate::tomlconfig::TomlFile>, env: impl Fn(&str) -> Option, ) -> Vec { let useful = |v: String| -> Option { @@ -969,12 +1011,15 @@ pub fn settings( .iter() .map(|known| { let from_env = env(known.key).and_then(useful); + let from_toml = toml.and_then(|f| f.get(known.key)).and_then(useful); let from_file = file.and_then(|f| f.get(known.key)).and_then(useful); let source = if from_env.is_some() { Source::Environment + } else if from_toml.is_some() { + Source::TomlFile } else if from_file.is_some() { - Source::File + Source::EnvFile } else { Source::Default }; @@ -988,7 +1033,10 @@ pub fn settings( "RESCRIPTUM_PUBLIC_HOST" => derive_public_host(), _ => known.default.map(str::to_string), }; - let value = from_env.or(from_file).or_else(|| default.clone()); + let value = from_env + .or(from_toml) + .or(from_file) + .or_else(|| default.clone()); Setting { key: known.key, @@ -1364,7 +1412,7 @@ mod tests { // **A panel with an empty field here is showing something other than what the // server does.** The value is derived at startup, so the table has to derive it // too — the same treatment the CPU count already gets, and for the same reason. - let s = settings(None, |_| None); + let s = settings(None, None, |_| None); let host = setting(&s, "RESCRIPTUM_PUBLIC_HOST"); assert!(host.set, "a derived value is still a value in force"); assert_eq!( @@ -1514,7 +1562,7 @@ mod tests { "override", "RESCRIPTUM_LISTEN_ADDR=0.0.0.0:8000\nRESCRIPTUM_LOG=problems\n", ); - let s = settings(Some(&file), |k| { + let s = settings(Some(&file), None, |k| { (k == "RESCRIPTUM_LISTEN_ADDR").then(|| "127.0.0.1:9999".to_string()) }); @@ -1523,7 +1571,7 @@ mod tests { assert_eq!(addr.value.as_deref(), Some("127.0.0.1:9999")); let log = setting(&s, "RESCRIPTUM_LOG"); - assert_eq!(log.source, Source::File); + assert_eq!(log.source, Source::EnvFile); assert_eq!(log.value.as_deref(), Some("problems")); let store = setting(&s, "RESCRIPTUM_STORE"); @@ -1538,13 +1586,13 @@ mod tests { // This is the whole reason `value` is separate from `set`: a settings panel has // to show that a token exists without ever being handed one. let (dir, file) = env_file("secret", "RESCRIPTUM_ADMIN_TOKEN=0123456789abcdef0\n"); - let s = settings(Some(&file), |_| None); + let s = settings(Some(&file), None, |_| None); let token = setting(&s, "RESCRIPTUM_ADMIN_TOKEN"); assert!(token.secret); assert!(token.set, "it is set"); assert_eq!(token.value, None, "a secret's value must never be carried"); - assert_eq!(token.source, Source::File); + assert_eq!(token.source, Source::EnvFile); let unset = setting(&s, "RESCRIPTUM_ANSWER_TOKEN"); assert!(!unset.set); @@ -1559,7 +1607,7 @@ mod tests { // rather than an instruction — otherwise the panel and the server would disagree // about what is in force. let (dir, file) = env_file("empty", "RESCRIPTUM_LOG=\n"); - let s = settings(Some(&file), |k| { + let s = settings(Some(&file), None, |k| { (k == "RESCRIPTUM_STORE").then(|| " ".to_string()) }); @@ -1574,7 +1622,7 @@ mod tests { fn the_one_default_that_is_not_a_constant_is_filled_in() { // The CPU count is this machine's, so the table cannot hold it and `settings` // has to. A panel showing "default: (none)" for workers would be wrong. - let s = settings(None, |_| None); + let s = settings(None, None, |_| None); let workers = setting(&s, "RESCRIPTUM_WORKERS"); assert_eq!(workers.default, Some(default_workers().to_string())); assert_eq!(workers.value, Some(default_workers().to_string())); diff --git a/src/envfile.rs b/src/envfile.rs index 7ac9b18..8a9a0d8 100644 --- a/src/envfile.rs +++ b/src/envfile.rs @@ -132,14 +132,14 @@ impl EnvFile { /// `0o600` and friends return `None`; anything a group or the world can read returns the /// mode, so it can be named in the warning. #[cfg(unix)] -fn readable_by_others(path: &Path) -> Option { +pub(crate) fn readable_by_others(path: &Path) -> Option { use std::os::unix::fs::PermissionsExt; let mode = std::fs::metadata(path).ok()?.permissions().mode() & 0o777; (mode & 0o077 != 0).then_some(mode) } #[cfg(not(unix))] -fn readable_by_others(_path: &Path) -> Option { +pub(crate) fn readable_by_others(_path: &Path) -> Option { None } diff --git a/src/lib.rs b/src/lib.rs index baae7c6..87cd853 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -19,3 +19,4 @@ pub mod log; pub mod merge; pub mod select; pub mod store; +pub mod tomlconfig; diff --git a/src/tomlconfig.rs b/src/tomlconfig.rs new file mode 100644 index 0000000..bbbfd22 --- /dev/null +++ b/src/tomlconfig.rs @@ -0,0 +1,699 @@ +//! An optional configuration file in TOML, named by `RESCRIPTUM_CONFIG`. +//! +//! The same job as [`crate::envfile`], in a shape meant to be read. It exists for one +//! platform and one act: somebody editing the file by hand on a NAS, in File Station or +//! over SMB, where `RESCRIPTUM_ANSWERS_DIR=/volume1/…` is a poor thing to hand a person. +//! The name misleads too — people read "environment variable" and go looking for a shell +//! to export it in, when on DSM it has been a file all along. +//! +//! **The configuration is still the same settings.** This module maps a document onto +//! their `RESCRIPTUM_*` names and does nothing else: every value reaches +//! `Config::from_lookup` under the key it would have had in the environment, so there is +//! exactly one place that decides what a setting *means*, and this format cannot grow +//! behaviour the environment does not have. That is what keeps two configuration files +//! from becoming two configurations. +//! +//! The rules are `envfile`'s, deliberately unchanged: +//! +//! * **Named, never discovered.** There is no `./rescriptum.toml`. This binary runs as +//! root; a file picked up from whatever directory it was launched in would hand +//! `admin.token` — and therefore the root password of every machine installed +//! afterwards — to anyone who could write there. +//! * **The real environment wins.** Both files supply defaults. +//! * **A file that was asked for and cannot be read is a startup error**, never a +//! warning. The silent path is what these files exist to remove. +//! +//! And one rule the second format adds: **where both files set the same thing, this one +//! wins**, because a deployment that names both is moving *to* TOML rather than sitting +//! between the two. `Config::from_env` says so out loud at startup rather than quietly +//! picking one. +//! +//! Unlike the env file this one has escapes, so no value has to be refused for what it +//! contains — a token with a `#` in it, a title with a quote, and a path with a space are +//! all writable and all read back unchanged. + +use std::collections::BTreeMap; +use std::path::PathBuf; +use toml_edit::{DocumentMut, Item, Table, Value}; + +/// The variable that points at the file. Deliberately the only way in. +pub const CONFIG_FILE: &str = "RESCRIPTUM_CONFIG"; + +/// One setting, under both of its names. +pub struct Mapped { + /// What it is called everywhere else in this program. + pub key: &'static str, + /// Where it lives in the document, dotted. One or two segments; the table is the + /// grouping the environment cannot express. + pub path: &'static str, + /// Written as a bare integer rather than a quoted string, when the value is one. + /// `workers = 2` is the point of the format; `workers = "2"` reads like a mistake. + pub numeric: bool, +} + +/// Every setting, in the document's own order. +/// +/// **The names shed the `RESCRIPTUM_` prefix and gain tables**, which was the open +/// question in the plan and is settled here by what the file is for: it exists to be +/// read, `answers_dir` reads better than `RESCRIPTUM_ANSWERS_DIR`, and a table is the +/// only thing that says `store.kind` and `store.db_path` belong together. The +/// one-to-one mapping with the environment lives in this table instead of in the +/// spelling, which is the right place for it — `config --value` and the panel both go +/// through it, so nobody has to hold two names in their head. +pub const MAPPING: [Mapped; 30] = [ + Mapped { + key: "RESCRIPTUM_ANSWERS_DIR", + path: "answers_dir", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_LISTEN_ADDR", + path: "listen_addr", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_LOG", + path: "log", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_LOG_FILE", + path: "log_file", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_PUBLIC_HOST", + path: "public_host", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_USER", + path: "user", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_GROUP", + path: "group", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_STORE", + path: "store.kind", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_DB_PATH", + path: "store.db_path", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_WORKERS", + path: "server.workers", + numeric: true, + }, + Mapped { + key: "RESCRIPTUM_MAX_CONNECTIONS", + path: "server.max_connections", + numeric: true, + }, + Mapped { + key: "RESCRIPTUM_TIMEOUT_SECS", + path: "server.timeout_secs", + numeric: true, + }, + Mapped { + key: "RESCRIPTUM_ADMIN_ADDR", + path: "admin.addr", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_ADMIN_TOKEN", + path: "admin.token", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_ANSWER_TOKEN", + path: "answer.token", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_CAPTURE_DIR", + path: "answer.capture_dir", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_MEDIA_DIR", + path: "media.dir", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_MEDIA_ADDR", + path: "media.addr", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_MEDIA_TIMEOUT_SECS", + path: "media.timeout_secs", + numeric: true, + }, + Mapped { + key: "RESCRIPTUM_MEDIA_MAX_CONNECTIONS", + path: "media.max_connections", + numeric: true, + }, + Mapped { + key: "RESCRIPTUM_BOOT_DIR", + path: "boot.dir", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_BOOT_ALLOW", + path: "boot.allow", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_BOOT_UNCLAIMED", + path: "boot.unclaimed", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_BOOT_TIMEOUT_SECS", + path: "boot.timeout_secs", + numeric: true, + }, + Mapped { + key: "RESCRIPTUM_BOOT_LOGO", + path: "boot.logo", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_BOOT_TITLE", + path: "boot.title", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_TFTP_ADDR", + path: "tftp.addr", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_TFTP_PORT_RANGE", + path: "tftp.port_range", + numeric: false, + }, + Mapped { + key: "RESCRIPTUM_TFTP_BLKSIZE", + path: "tftp.blksize", + numeric: true, + }, + // Its own table rather than a line in `[boot]` or `[answer]`, because it is neither: + // the token is Proxmox's, it arrives on the answer listener, and what it changes is a + // boot claim. `POST /installed` is the thing being configured, and the section is + // named after it. + Mapped { + key: "RESCRIPTUM_INSTALLED_TOKEN", + path: "installed.token", + numeric: false, + }, +]; + +/// The document's name for a setting, or `None` if this program does not read it. +pub fn path_for(key: &str) -> Option<&'static str> { + MAPPING.iter().find(|m| m.key == key).map(|m| m.path) +} + +fn mapped_path(path: &str) -> Option<&'static Mapped> { + MAPPING.iter().find(|m| m.path == path) +} + +/// The environment name for a setting written the way the document writes it. Used to +/// answer somebody who typed `config set answers_dir=…` with the name that works, rather +/// than with "not a setting this program reads" — they are looking at a file where that +/// *is* the name. +pub fn key_for(path: &str) -> Option<&'static str> { + mapped_path(path).map(|m| m.key) +} + +fn mapped_key(key: &str) -> Option<&'static Mapped> { + MAPPING.iter().find(|m| m.key == key) +} + +/// A loaded file: the settings it makes, under their environment names, and anything +/// worth saying about it out loud. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct TomlFile { + pub path: PathBuf, + vars: BTreeMap, + /// Reported at startup. Never contains a value — this file holds the admin token. + pub warnings: Vec, +} + +impl TomlFile { + /// Read and parse the file, or say why not. + pub fn load(path: impl Into) -> Result { + let path = path.into(); + let text = std::fs::read_to_string(&path) + .map_err(|e| format!("{CONFIG_FILE}={} cannot be read: {e}", path.display()))?; + + let (vars, mut warnings) = parse(&text).map_err(|e| format!("{}: {e}", path.display()))?; + for warning in &mut warnings { + *warning = format!("{}: {warning}", path.display()); + } + if let Some(mode) = crate::envfile::readable_by_others(&path) { + warnings.push(format!( + "{} is mode {mode:04o} — it may hold admin.token, so chmod 600 it", + path.display() + )); + } + + Ok(TomlFile { + path, + vars, + warnings, + }) + } + + /// By environment name, which is the only name the rest of the program knows. + pub fn get(&self, key: &str) -> Option { + self.vars.get(key).cloned() + } + + pub fn len(&self) -> usize { + self.vars.len() + } + + pub fn is_empty(&self) -> bool { + self.vars.is_empty() + } +} + +/// Parse a document into settings under their environment names, plus what to say about +/// it. +/// +/// A key this program does not read is a **warning naming it**, not an error: a typo has +/// to be visible — silently ignoring `answer_dir` is how somebody spends an afternoon +/// wondering why their answers directory moved back — but refusing to start over one +/// would take a fleet down for a spelling mistake. A *duplicate* key needs no rule here: +/// TOML forbids it, so the parser refuses the file before this function sees it, which is +/// the same answer `envfile::parse` gives by hand. +pub fn parse(text: &str) -> Result<(BTreeMap, Vec), String> { + let doc = text + .parse::() + .map_err(|e| e.to_string().replace('\n', " "))?; + + let mut vars = BTreeMap::new(); + let mut warnings = Vec::new(); + walk(doc.as_table(), "", &mut vars, &mut warnings)?; + Ok((vars, warnings)) +} + +fn walk( + table: &Table, + prefix: &str, + vars: &mut BTreeMap, + warnings: &mut Vec, +) -> Result<(), String> { + for (key, item) in table.iter() { + let path = if prefix.is_empty() { + key.to_string() + } else { + format!("{prefix}.{key}") + }; + match item { + Item::Table(inner) => walk(inner, &path, vars, warnings)?, + Item::Value(Value::InlineTable(inner)) => { + // `store = { kind = "sqlite" }` is the same configuration written on one + // line. Reading it costs nothing and refusing it would be pedantry. + for (key, value) in inner.iter() { + leaf(&format!("{path}.{key}"), value, vars, warnings)?; + } + } + Item::Value(value) => leaf(&path, value, vars, warnings)?, + Item::ArrayOfTables(_) => { + warnings.push(format!("{path} is not a setting this program reads")); + } + Item::None => {} + } + } + Ok(()) +} + +fn leaf( + path: &str, + value: &Value, + vars: &mut BTreeMap, + warnings: &mut Vec, +) -> Result<(), String> { + let Some(mapped) = mapped_path(path) else { + warnings.push(format!( + "{path} is not a setting this program reads — check the spelling" + )); + return Ok(()); + }; + // A setting that exists but was given a list or a table is a mistake worth stopping + // for: unlike a misspelled name it *was* aimed at something real, and carrying on + // would serve the default while the file plainly says otherwise. + let Some(rendered) = scalar(value) else { + return Err(format!( + "{path} takes one value, not {}", + match value { + Value::Array(_) => "a list", + _ => "a table", + } + )); + }; + // Empty is unset, decided here so that every reader agrees rather than each one + // trimming for itself — and so that `unset`, which empties a line rather than + // deleting it, means what it says all the way down to `len()`. + if !rendered.trim().is_empty() { + vars.insert(mapped.key.to_string(), rendered); + } + Ok(()) +} + +/// Every scalar becomes the text the environment would have carried, so a number written +/// as a number and a number written as a string mean the same thing — the file is for +/// people, and both are what a person writes. +fn scalar(value: &Value) -> Option { + match value { + Value::String(s) => Some(s.value().clone()), + Value::Integer(i) => Some(i.value().to_string()), + Value::Float(f) => Some(f.value().to_string()), + Value::Boolean(b) => Some(b.value().to_string()), + Value::Datetime(d) => Some(d.value().to_string()), + Value::Array(_) | Value::InlineTable(_) => None, + } +} + +/// Apply changes to the **text** of a configuration file, leaving everything else exactly +/// as it is. +/// +/// Keyed by environment name, like [`crate::envfile::rewrite`], so one caller can drive +/// either format. What differs is what the format allows: `toml_edit` edits the document +/// in place, so comments, ordering and spacing survive on their own rather than by +/// hand — and there is no value that cannot be written, because TOML has escapes. +/// +/// `None` **empties a setting rather than deleting its line**, and that is deliberate: +/// removing the key would take the comment above it with it, and on a packaged install +/// those comments are the only documentation the configuration has. An empty value +/// already counts as unset everywhere else in this program — an exported-but-empty +/// variable is a mistake, not an instruction — so the file keeps saying what the setting +/// is while saying that nobody set it. A key that is not in the file at all stays absent. +pub fn rewrite(text: &str, changes: &BTreeMap>) -> Result { + let mut doc = text + .parse::() + .map_err(|e| e.to_string().replace('\n', " "))?; + + for (key, change) in changes { + let Some(mapped) = mapped_key(key) else { + return Err(format!("{key} is not a setting this program reads")); + }; + let segments: Vec<&str> = mapped.path.split('.').collect(); + let (last, tables) = segments + .split_last() + .expect("a path has at least one segment"); + + match change { + Some(value) => { + let table = make_table(doc.as_table_mut(), tables, mapped.path)?; + let mut new = render(value, mapped.numeric); + // **Replace the value, never the entry.** A setting's explanation sits in + // the *key's* decor, so inserting over an existing key throws away the + // paragraph above it — which on a packaged install is the only + // documentation the configuration has. Editing the value in place keeps + // that, and keeps the spacing and any trailing comment with it. + if let Some(Item::Value(existing)) = table.get_mut(last) { + *new.decor_mut() = existing.decor().clone(); + *existing = new; + } else { + table.insert(last, Item::Value(new)); + } + } + // Only a setting that is actually there means anything. Writing an empty + // value for one nobody set would add noise rather than remove a setting. + None => { + if let Some(table) = find_table(doc.as_table_mut(), tables) + && let Some(Item::Value(existing)) = table.get_mut(last) + { + let mut empty = Value::from(""); + *empty.decor_mut() = existing.decor().clone(); + *existing = empty; + } + } + } + } + + Ok(doc.to_string()) +} + +/// Walk to the table a setting lives in, creating what is missing. A new table is an +/// ordinary `[section]` rather than a dotted key, because this file is read by people. +fn make_table<'a>( + mut table: &'a mut Table, + segments: &[&str], + path: &str, +) -> Result<&'a mut Table, String> { + for segment in segments { + let entry = table + .entry(segment) + .or_insert_with(|| Item::Table(Table::new())); + table = entry.as_table_mut().ok_or_else(|| { + format!("{segment} is not a table in this file, so {path} cannot be written") + })?; + } + Ok(table) +} + +/// The same walk, creating nothing: `None` as soon as a segment is missing or is not a +/// table. +fn find_table<'a>(mut table: &'a mut Table, segments: &[&str]) -> Option<&'a mut Table> { + for segment in segments { + table = table.get_mut(segment)?.as_table_mut()?; + } + Some(table) +} + +/// A number written as a number, when the setting is one and the value is one. Anything +/// else is a string — including a numeric setting given something that is not a number, +/// which the file is entitled to hold and `from_lookup` is entitled to ignore, exactly as +/// it ignores `RESCRIPTUM_WORKERS=lots`. +fn render(value: &str, numeric: bool) -> Value { + if numeric && let Ok(n) = value.trim().parse::() { + return Value::from(n); + } + Value::from(value) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn parsed(text: &str) -> BTreeMap { + parse(text).expect("parses").0 + } + + #[test] + fn every_setting_is_mapped_exactly_once() { + // The two tables are the same set under two spellings, and nothing else in the + // program checks that. A setting missing here is one the file silently cannot + // configure — the failure this format would otherwise ship with. + for key in crate::envfile::KNOWN_KEYS { + assert!( + path_for(key).is_some(), + "{key} has no place in the document" + ); + } + assert_eq!(MAPPING.len(), crate::envfile::KNOWN_KEYS.len()); + + let mut paths: Vec<&str> = MAPPING.iter().map(|m| m.path).collect(); + paths.sort_unstable(); + let before = paths.len(); + paths.dedup(); + assert_eq!(before, paths.len(), "two settings share one path"); + } + + #[test] + fn tables_become_environment_names() { + let vars = parsed( + r#" + answers_dir = "/srv/answers" + + [store] + kind = "sqlite" + db_path = "/srv/answers.db" + + [server] + workers = 2 + "#, + ); + assert_eq!( + vars.get("RESCRIPTUM_ANSWERS_DIR").map(String::as_str), + Some("/srv/answers") + ); + assert_eq!( + vars.get("RESCRIPTUM_STORE").map(String::as_str), + Some("sqlite") + ); + assert_eq!( + vars.get("RESCRIPTUM_DB_PATH").map(String::as_str), + Some("/srv/answers.db") + ); + // A number reaches `from_lookup` as the text the environment would have carried. + assert_eq!( + vars.get("RESCRIPTUM_WORKERS").map(String::as_str), + Some("2") + ); + } + + #[test] + fn an_inline_table_is_the_same_configuration() { + let vars = parsed(r#"store = { kind = "sqlite" }"#); + assert_eq!( + vars.get("RESCRIPTUM_STORE").map(String::as_str), + Some("sqlite") + ); + } + + #[test] + fn a_dotted_key_is_the_same_configuration() { + let vars = parsed(r#"store.kind = "sqlite""#); + assert_eq!( + vars.get("RESCRIPTUM_STORE").map(String::as_str), + Some("sqlite") + ); + } + + #[test] + fn an_unknown_key_is_named_rather_than_ignored() { + let (vars, warnings) = parse( + r#" + answer_dir = "/srv/answers" + + [store] + knid = "sqlite" + "#, + ) + .expect("an unknown key does not stop the file being read"); + assert!(vars.is_empty(), "{vars:?}"); + assert_eq!(warnings.len(), 2, "{warnings:?}"); + assert!( + warnings.iter().any(|w| w.contains("answer_dir")), + "{warnings:?}" + ); + assert!( + warnings.iter().any(|w| w.contains("store.knid")), + "{warnings:?}" + ); + } + + #[test] + fn a_real_setting_given_a_list_is_an_error() { + // Not a warning: unlike a misspelling this was aimed at something real, and + // serving the default while the file plainly says otherwise is the silent + // failure both file formats exist to remove. + let e = parse("answers_dir = [\"/srv/answers\"]").expect_err("refused"); + assert!(e.contains("answers_dir"), "{e}"); + assert!(e.contains("not a list"), "{e}"); + } + + #[test] + fn a_duplicate_key_is_refused_by_the_format_itself() { + let e = parse("answers_dir = \"/a\"\nanswers_dir = \"/b\"\n").expect_err("refused"); + assert!(e.contains("answers_dir"), "{e}"); + } + + #[test] + fn a_value_keeps_its_hash_and_its_quotes() { + // The env file has to refuse both of these, because it has no escapes. This one + // does, which is most of why it is nicer to edit by hand. + let vars = parsed("[admin]\ntoken = \"a#b'c\\\"d\"\n"); + assert_eq!( + vars.get("RESCRIPTUM_ADMIN_TOKEN").map(String::as_str), + Some("a#b'c\"d") + ); + } + + #[test] + fn writing_keeps_the_comments_that_document_the_file() { + let text = "# Where answers come from.\nanswers_dir = \"/srv/answers\" # the share\n"; + let changes = BTreeMap::from([( + "RESCRIPTUM_ANSWERS_DIR".to_string(), + Some("/volume1/rescriptum/answers".to_string()), + )]); + let out = rewrite(text, &changes).expect("rewrites"); + assert!(out.contains("# Where answers come from."), "{out}"); + assert!(out.contains("/volume1/rescriptum/answers"), "{out}"); + assert!(out.contains("# the share"), "{out}"); + assert_eq!( + parsed(&out) + .get("RESCRIPTUM_ANSWERS_DIR") + .map(String::as_str), + Some("/volume1/rescriptum/answers") + ); + } + + #[test] + fn writing_creates_the_table_a_setting_lives_in() { + let out = rewrite( + "", + &BTreeMap::from([("RESCRIPTUM_STORE".to_string(), Some("sqlite".to_string()))]), + ) + .expect("rewrites"); + assert!(out.contains("[store]"), "{out}"); + assert_eq!( + parsed(&out).get("RESCRIPTUM_STORE").map(String::as_str), + Some("sqlite") + ); + } + + #[test] + fn a_numeric_setting_is_written_as_a_number() { + let out = rewrite( + "", + &BTreeMap::from([("RESCRIPTUM_WORKERS".to_string(), Some("2".to_string()))]), + ) + .expect("rewrites"); + assert!(out.contains("workers = 2"), "{out}"); + // And one that is not a number is still writable, and still ignored later — + // exactly as `RESCRIPTUM_WORKERS=lots` is. + let out = rewrite( + "", + &BTreeMap::from([("RESCRIPTUM_WORKERS".to_string(), Some("lots".to_string()))]), + ) + .expect("rewrites"); + assert!(out.contains("workers = \"lots\""), "{out}"); + } + + #[test] + fn unsetting_empties_the_line_and_keeps_its_paragraph() { + let text = "# The admin API's bearer token.\n[admin]\ntoken = \"secret\"\n"; + let out = rewrite( + text, + &BTreeMap::from([("RESCRIPTUM_ADMIN_TOKEN".to_string(), None)]), + ) + .expect("rewrites"); + assert!(out.contains("# The admin API's bearer token."), "{out}"); + assert!(out.contains("token = \"\""), "{out}"); + assert!(!out.contains("secret"), "{out}"); + // Empty is unset, the same way an exported-but-empty variable is. + assert!(!parsed(&out).contains_key("RESCRIPTUM_ADMIN_TOKEN")); + } + + #[test] + fn unsetting_something_nobody_set_writes_nothing() { + let out = rewrite( + "answers_dir = \"/srv/answers\"\n", + &BTreeMap::from([("RESCRIPTUM_ADMIN_TOKEN".to_string(), None)]), + ) + .expect("rewrites"); + assert_eq!(out, "answers_dir = \"/srv/answers\"\n"); + } + + #[test] + fn an_empty_value_counts_as_unset() { + let vars = parsed("answers_dir = \"\"\n[admin]\ntoken = \" \"\n"); + // Kept out at this level so every reader agrees, rather than each one trimming. + assert!(!vars.contains_key("RESCRIPTUM_ANSWERS_DIR"), "{vars:?}"); + assert!(!vars.contains_key("RESCRIPTUM_ADMIN_TOKEN"), "{vars:?}"); + } +} diff --git a/tests/cli.rs b/tests/cli.rs index 219416e..f00c3c6 100644 --- a/tests/cli.rs +++ b/tests/cli.rs @@ -21,6 +21,7 @@ struct Case { impl Drop for Case { fn drop(&mut self) { let _ = fs::remove_dir_all(&self.dir); + let _ = fs::remove_dir_all(self.etc()); } } @@ -52,6 +53,27 @@ impl Case { } } + /// A scratch directory **beside** the answers directory, for the files that + /// configure the server rather than being served by it. + /// + /// This is not tidiness. Every servable `.toml` at the top of the answers directory + /// is a misplaced answer document, and a configuration file is not exempt — see + /// `a_configuration_file_inside_the_answers_directory_is_reported_as_a_stray_answer`, + /// which pins that rather than leaving it to be discovered. + fn etc(&self) -> PathBuf { + let mut name = self.dir.file_name().expect("a name").to_os_string(); + name.push("-etc"); + self.dir.with_file_name(name) + } + + fn conf(&self, name: &str, body: &str) -> PathBuf { + let dir = self.etc(); + fs::create_dir_all(&dir).expect("scratch etc"); + let path = dir.join(name); + fs::write(&path, body).expect("configuration file"); + path + } + fn run(&self, args: &[&str]) -> Run { self.run_env(&[("RESCRIPTUM_ANSWERS_DIR", self.dir.as_path())], args) } @@ -699,6 +721,188 @@ fn naming_the_env_file_inside_itself_says_why_it_does_nothing() { ); } +// ---- the toml file -------------------------------------------------------- +// +// The same job as the env file, in the shape a person edits by hand on a NAS. What these +// pin is that it is the *same* configuration: one set of settings, one precedence rule, +// and a document that cannot mean anything the environment could not. + +#[test] +fn a_toml_file_supplies_configuration() { + let c = Case::new(&[("groups/rack-a.toml", RACK)]); + let toml = c.conf( + "rescriptum.toml", + &format!( + "answers_dir = \"{}\"\n\n[server]\ntimeout_secs = 7\n", + c.dir.display() + ), + ); + + let r = c.run_env(&[("RESCRIPTUM_CONFIG", &toml)], &["check"]); + assert!(r.ok, "{r}"); + assert!(r.stdout.contains("1 group(s)"), "{r}"); + assert!( + r.stderr.contains("reading configuration defaults from"), + "the file it read must be named in the log\n{r}" + ); +} + +#[test] +fn the_real_environment_wins_over_the_toml_file() { + // The rule the env file already has, and the reason both files are only defaults. + let c = Case::new(&[("98fa9b50d810.toml", "marker = \"from-the-flag\"\n")]); + let elsewhere = c.dir.join("unused"); + fs::create_dir_all(&elsewhere).unwrap(); + let toml = c.conf( + "rescriptum.toml", + &format!("answers_dir = \"{}\"\n", elsewhere.display()), + ); + + let r = c.run_env( + &[ + ("RESCRIPTUM_CONFIG", &toml), + ("RESCRIPTUM_ANSWERS_DIR", c.dir.as_path()), + ], + &["render", "98:fa:9b:50:d8:10"], + ); + assert!(r.ok, "{r}"); + assert!(r.stdout.contains("from-the-flag"), "{r}"); +} + +#[test] +fn the_toml_file_wins_over_the_env_file_and_says_that_both_are_named() { + // Naming both is a deployment mid-migration. Which one wins is the single question + // neither file can answer by itself, so the server answers it out loud. + let c = Case::new(&[("98fa9b50d810.toml", "marker = \"from-the-toml\"\n")]); + let elsewhere = c.dir.join("unused"); + fs::create_dir_all(&elsewhere).unwrap(); + let env = c.conf( + "rescriptum.env", + &format!("RESCRIPTUM_ANSWERS_DIR={}\n", elsewhere.display()), + ); + let toml = c.conf( + "rescriptum.toml", + &format!("answers_dir = \"{}\"\n", c.dir.display()), + ); + + let r = c.run_env( + &[("RESCRIPTUM_CONFIG", &toml), ("RESCRIPTUM_ENV_FILE", &env)], + &["render", "98:fa:9b:50:d8:10"], + ); + assert!(r.ok, "{r}"); + assert!(r.stdout.contains("from-the-toml"), "{r}"); + assert!( + r.stderr.contains("are both named"), + "an operator must be told which file is winning\n{r}" + ); +} + +#[test] +fn a_toml_file_that_cannot_be_read_refuses_to_start() { + // Carrying on with defaults is what a named file exists to prevent, whichever format + // it is written in. + let c = Case::new(&[]); + let r = c.run_env( + &[("RESCRIPTUM_CONFIG", &c.dir.join("absent.toml"))], + &["check"], + ); + assert!(!r.ok, "{r}"); + assert!(r.stderr.contains("RESCRIPTUM_CONFIG"), "{r}"); + assert!(r.stderr.contains("cannot be read"), "{r}"); +} + +#[test] +fn a_malformed_toml_file_refuses_to_start_and_says_where() { + let c = Case::new(&[]); + let toml = c.conf("bad.toml", "answers_dir = \n"); + + let r = c.run_env(&[("RESCRIPTUM_CONFIG", &toml)], &["check"]); + assert!(!r.ok, "{r}"); + assert!(r.stderr.contains("bad.toml"), "{r}"); + // The parser's own message carries the line and column; what matters here is that it + // arrives on one line, since this is a startup error in a log. + assert_eq!( + r.stderr.lines().filter(|l| l.contains("bad.toml")).count(), + 1, + "{r}" + ); +} + +#[test] +fn a_setting_given_a_list_is_refused_rather_than_quietly_defaulted() { + // A misspelling is a warning; this was aimed at something real, so serving the + // default while the file plainly says otherwise would be the silent failure. + let c = Case::new(&[]); + let toml = c.conf("list.toml", "answers_dir = [\"/srv/answers\"]\n"); + + let r = c.run_env(&[("RESCRIPTUM_CONFIG", &toml)], &["check"]); + assert!(!r.ok, "{r}"); + assert!(r.stderr.contains("answers_dir"), "{r}"); +} + +#[test] +fn a_misspelled_setting_in_the_toml_file_is_warned_about() { + let c = Case::new(&[("groups/rack-a.toml", RACK)]); + let toml = c.conf( + "typo.toml", + &format!( + "answers_dir = \"{}\"\n\n[admin]\ntoken = \"0123456789abcdef0\"\ntokenn = \"hunter2\"\n", + c.dir.display() + ), + ); + + let r = c.run_env(&[("RESCRIPTUM_CONFIG", &toml)], &["check"]); + assert!(r.ok, "a typo is a warning, not a refusal\n{r}"); + assert!(r.stderr.contains("admin.tokenn"), "{r}"); + assert!( + !r.stderr.contains("hunter2"), + "a warning must never print what the file holds\n{r}" + ); +} + +#[test] +fn no_toml_file_is_ever_discovered_on_its_own() { + // Same reasoning as the env file, and it is not negotiable: this binary runs as root, + // and the file holds admin.token. + let c = Case::new(&[("98fa9b50d810.toml", "marker = \"real\"\n")]); + for name in ["rescriptum.toml", "config.toml", ".rescriptum.toml"] { + fs::write(c.dir.join(name), "answers_dir = \"/nonexistent/planted\"\n").unwrap(); + } + + let mut cmd = Command::new(env!("CARGO_BIN_EXE_rescriptum")); + cmd.env("RESCRIPTUM_ANSWERS_DIR", c.dir.as_path()) + .current_dir(&c.dir); + let r = Run::from( + cmd.args(["render", "98:fa:9b:50:d8:10"]) + .output() + .expect("run"), + ); + assert!(r.ok, "{r}"); + assert!(r.stdout.contains("real"), "{r}"); +} + +#[test] +fn a_configuration_file_inside_the_answers_directory_is_reported_as_a_stray_answer() { + // Found by writing the tests above: a `.toml` at the top of the answers directory is + // a misplaced answer document, and this format shares that extension. Nothing here + // makes an exception for it — a configuration file belongs beside the store, not in + // it — so `check` says the same thing it says about any stray document. Better said + // than discovered when `migrate --apply` offers to move somebody's configuration. + let c = Case::new(&[("groups/rack-a.toml", RACK)]); + let inside = c.dir.join("rescriptum.toml"); + fs::write(&inside, format!("answers_dir = \"{}\"\n", c.dir.display())).unwrap(); + + let r = c.run_env(&[("RESCRIPTUM_CONFIG", &inside)], &["check"]); + assert!( + !r.ok, + "a stray document is a problem, and problems fail check\n{r}" + ); + assert!(r.stdout.contains("rescriptum.toml"), "{r}"); + assert!(r.stdout.contains("an answer is a directory now"), "{r}"); + // And it is still read as configuration, because the variable named it. + assert!(r.stdout.contains("1 group(s)"), "{r}"); +} + // ---- config --------------------------------------------------------------- // // The command a settings panel drives, and the one people reach for when the server will @@ -725,6 +929,13 @@ impl Case { fs::write(&path, body).expect("env file"); path.to_string_lossy().into_owned() } + + /// Beside the answers directory, where a configuration file belongs — see `etc`. + fn toml_file(&self, body: &str) -> String { + self.conf("rescriptum.toml", body) + .to_string_lossy() + .into_owned() + } } #[test] @@ -927,7 +1138,7 @@ fn config_json_carries_the_source_and_the_help_a_panel_needs() { let r = c.run_config(&[("RESCRIPTUM_ENV_FILE", &env)], &["config", "--json"]); assert!(r.ok, "{r}"); assert!(r.stdout.contains("\"key\":\"RESCRIPTUM_STORE\""), "{r}"); - assert!(r.stdout.contains("\"source\":\"file\""), "{r}"); + assert!(r.stdout.contains("\"source\":\"env file\""), "{r}"); assert!( r.stdout.contains("\"secret\":true"), "the tokens are marked\n{r}" @@ -952,6 +1163,145 @@ fn config_with_no_file_named_says_so_rather_than_failing() { assert!(w.stderr.contains("RESCRIPTUM_ENV_FILE"), "{w}"); } +#[test] +fn config_writes_the_toml_file_and_leaves_its_documentation_standing() { + // On a packaged install the comments *are* the configuration's documentation. A + // writer that regenerated the file would throw them away the first time anyone + // changed a setting, which is why this edits the document rather than rendering one. + let c = Case::new(&[]); + let before = "# How much to log.\nlog = \"all\" # all | problems | off\n\n[store]\n# Where answers come from.\nkind = \"files\"\n"; + let toml = c.toml_file(before); + + let r = c.run_config( + &[("RESCRIPTUM_CONFIG", &toml)], + &["config", "set", "RESCRIPTUM_LOG=problems"], + ); + assert!(r.ok, "{r}"); + + let after = fs::read_to_string(&toml).expect("still there"); + assert!(after.contains("# How much to log."), "{after}"); + assert!(after.contains("# all | problems | off"), "{after}"); + assert!(after.contains("# Where answers come from."), "{after}"); + assert!(after.contains("log = \"problems\""), "{after}"); + assert_eq!(after.matches("log =").count(), 1, "duplicated\n{after}"); + + // And the server reads back what was written, from the file that was written. + let r = c.run_config(&[("RESCRIPTUM_CONFIG", &toml)], &["config"]); + let log = line_for(&r.stdout, "RESCRIPTUM_LOG "); + assert!( + log.contains("problems") && log.ends_with("toml file"), + "{log:?}\n{r}" + ); +} + +#[test] +fn config_unset_empties_a_setting_rather_than_deleting_its_paragraph() { + // Deleting the key would take the comment above it with it. Empty already counts as + // unset everywhere else in this program, so the line stays and says nothing is set. + let c = Case::new(&[]); + let toml = + c.toml_file("# The token every installer must present.\n[answer]\ntoken = \"hunter2\"\n"); + + let r = c.run_config( + &[("RESCRIPTUM_CONFIG", &toml)], + &["config", "unset", "RESCRIPTUM_ANSWER_TOKEN"], + ); + assert!(r.ok, "{r}"); + + let after = fs::read_to_string(&toml).expect("still there"); + assert!( + after.contains("# The token every installer must present."), + "{after}" + ); + assert!(after.contains("token = \"\""), "{after}"); + assert!(!after.contains("hunter2"), "{after}"); + + let r = c.run_config(&[("RESCRIPTUM_CONFIG", &toml)], &["config"]); + let token = line_for(&r.stdout, "RESCRIPTUM_ANSWER_TOKEN"); + assert!(token.contains("(not set)"), "{token:?}\n{r}"); +} + +#[test] +fn config_set_refuses_to_leave_a_server_that_cannot_start_in_toml_too() { + // The same guard, over the other format. A write that reaches the disk and stops the + // server is the failure; the format it was written in is not the interesting part. + let c = Case::new(&[]); + let before = "[store]\nkind = \"sqlite\"\n"; + let toml = c.toml_file(before); + + let r = c.run_config( + &[("RESCRIPTUM_CONFIG", &toml)], + &["config", "set", "RESCRIPTUM_ADMIN_ADDR=127.0.0.1:8001"], + ); + assert!(!r.ok, "an unauthenticated admin API must be refused\n{r}"); + assert!(r.stderr.contains("refused"), "{r}"); + assert_eq!( + fs::read_to_string(&toml).expect("still there"), + before, + "the file must be untouched when the write is refused" + ); +} + +#[test] +fn config_set_writes_the_toml_file_when_both_files_are_named() { + // It is the one the server reads first, so writing the other would be a change that + // silently does nothing — which is the whole failure mode this area exists to remove. + let c = Case::new(&[]); + let env = c.env_file("RESCRIPTUM_STORE=files\n"); + let toml = c.toml_file("log = \"all\"\n"); + + let r = c.run_config( + &[("RESCRIPTUM_CONFIG", &toml), ("RESCRIPTUM_ENV_FILE", &env)], + &["config", "set", "RESCRIPTUM_LOG=off"], + ); + assert!(r.ok, "{r}"); + assert!( + r.stderr.contains(&toml), + "it must say which file it wrote\n{r}" + ); + assert!(fs::read_to_string(&toml).unwrap().contains("log = \"off\"")); + assert_eq!( + fs::read_to_string(&env).unwrap(), + "RESCRIPTUM_STORE=files\n", + "the env file must be left alone" + ); +} + +#[test] +fn config_set_answers_the_documents_name_with_the_one_that_works() { + // Somebody reading the file types the name they see in it. "Not a setting this + // program reads" would be true of the command line and useless to them. + let c = Case::new(&[]); + let toml = c.toml_file("log = \"all\"\n"); + + let r = c.run_config( + &[("RESCRIPTUM_CONFIG", &toml)], + &["config", "set", "answers_dir=/srv/answers"], + ); + assert!(!r.ok, "{r}"); + assert!(r.stderr.contains("RESCRIPTUM_ANSWERS_DIR"), "{r}"); +} + +#[test] +fn config_json_names_both_files_and_the_name_each_setting_has_in_one() { + // The panel renders this. It has to be able to say which file a save would land in, + // and to show the line somebody would edit by hand. + let c = Case::new(&[]); + let toml = c.toml_file("[store]\nkind = \"sqlite\"\n"); + + let r = c.run_config(&[("RESCRIPTUM_CONFIG", &toml)], &["config", "--json"]); + assert!(r.ok, "{r}"); + assert!(r.stdout.contains("\"source\":\"toml file\""), "{r}"); + assert!(r.stdout.contains("\"path\":\"store.kind\""), "{r}"); + assert!( + r.stdout.contains(&format!("\"toml_file\":\"{toml}\"")), + "{r}" + ); + assert!(r.stdout.contains(&format!("\"target\":\"{toml}\"")), "{r}"); + assert!(r.stdout.contains("\"writable\":true"), "{r}"); + assert_eq!(r.stdout.lines().count(), 1, "{r}"); +} + #[test] fn config_value_prints_one_setting_but_never_a_credential() { // The DSM panel's backend reads a value this way rather than grepping the file, diff --git a/tests/integration.rs b/tests/integration.rs index 9952bdb..2580b34 100644 --- a/tests/integration.rs +++ b/tests/integration.rs @@ -878,6 +878,75 @@ fn a_running_server_takes_its_configuration_from_the_env_file() { let _ = fs::remove_file(&env_path); } +#[test] +fn a_running_server_takes_its_configuration_from_the_toml_file() { + // The same proof as the env file's, over the format a person edits by hand. What it + // pins is that the document reaches the *running server* and not only the CLI — the + // token below is in force on a socket, or it is not. + static N: AtomicUsize = AtomicUsize::new(0); + let path = std::env::temp_dir().join(format!( + "rescriptum-tomlconfig-{}-{}.toml", + std::process::id(), + N.fetch_add(1, Ordering::Relaxed) + )); + fs::write( + &path, + "# guarded by a token nothing else knows about\n[answer]\ntoken = \"from-the-toml-file-0123\"\n", + ) + .expect("write toml file"); + + let s = Server::start_env( + &[("default.toml", "marker = \"served\"\n")], + "5", + &[("RESCRIPTUM_CONFIG", path.to_str().unwrap())], + ); + + let body = installer_body("98:fa:9b:50:d8:10"); + let unauthenticated = s.post(&body); + assert!( + status_line(&unauthenticated).starts_with("HTTP/1.1 401"), + "the file's token must be in force\n{unauthenticated}" + ); + + let authenticated = s.raw( + format!( + "POST /answer HTTP/1.1\r\nHost: nas\r\nAuthorization: Bearer from-the-toml-file-0123\r\n\ + Content-Length: {}\r\n\r\n{body}", + body.len() + ) + .as_bytes(), + ); + assert!( + status_line(&authenticated).starts_with("HTTP/1.1 200"), + "{authenticated}" + ); + assert!( + body_of(&authenticated).contains("served"), + "{authenticated}" + ); + + let _ = fs::remove_file(&path); +} + +#[test] +fn a_server_told_to_read_a_toml_file_that_is_not_there_does_not_come_up() { + // Fatal for the same reason the env file's absence is: coming up on defaults would + // mean the wrong answers directory and no token, silently. + let out = Command::new(env!("CARGO_BIN_EXE_rescriptum")) + .env("RESCRIPTUM_LISTEN_ADDR", "127.0.0.1:0") + .env("RESCRIPTUM_CONFIG", "/nonexistent/rescriptum.toml") + .output() + .expect("run server"); + assert!(!out.status.success(), "it must refuse to start"); + let stderr = String::from_utf8_lossy(&out.stderr); + assert!(stderr.contains("RESCRIPTUM_CONFIG"), "{stderr}"); + assert!(stderr.contains("cannot be read"), "{stderr}"); + assert!( + !stderr.contains("listening on"), + "it must not bind before giving up\n{stderr}" + ); +} + #[test] fn a_server_told_to_read_a_file_that_is_not_there_does_not_come_up() { // Starting on defaults instead — wrong answers directory, no admin token, no word in