diff --git a/docs/architecture.md b/docs/architecture.md index 90d8b99..9126323 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -54,7 +54,7 @@ Nothing in `src/cli/` reads the process it runs in. A command receives a `CliRun What a run remembers (`--verbose`, whether work has started, the resolved configuration, the controls whose version was already checked, the running progress display) is `rt.state`, created with the runtime. No module keeps state between runs, so one process can run any number of commands, one after another or at the same time, and a run never has to be reset. -`src/cli.ts` builds the runtime of the real process (`processRuntime`) and calls `runCli(argv, rt)` from `src/cli/program.ts`, which builds the command tree for that run, loads only the module of the command the arguments name and returns the exit code. A test builds a runtime from plain objects (`tests/cli-runtime.ts`) and calls the same function, so it exercises the real parser and handlers without a child process and without patching a global. ESLint rejects `process`, `console` and `os.homedir` in `src/cli/` outside `runtime.ts`, so the boundary holds as the CLI grows. Shared domain modules (`images/`, `compose/`, `transport.ts`) take an explicit `env` or version where the CLI passes one and otherwise default to the process, because the control plane and the agent use them too. Time (`Date.now`) and the temporary directory are not part of the boundary. +`src/cli.ts` builds the runtime of the real process (`processRuntime`) and calls `runCli(argv, rt)` from `src/cli/program.ts`, which builds the command tree for that run, loads only the module of the command the arguments name and returns the exit code. A test builds a runtime from plain objects (`tests/cli-runtime.ts`) and calls the same function, so it exercises the real parser and handlers without a child process and without patching a global. ESLint rejects `process`, `console` and `os.homedir` in `src/cli/` outside `runtime.ts`, so the boundary holds as the CLI grows. The same configuration rejects `child_process` imports in `src/cli/` outside `process.ts` and `commands/host.ts` (and `config.ts`, which runs git), so every ssh goes through `runStep`, `runCapture`, `runLocal` or the host probes, which turn a host key refusal into the `ssh-keygen -R` remedy; `src/cli/runtime-boundary.test.ts` checks the rule. Shared domain modules (`images/`, `compose/`, `transport.ts`) take an explicit `env` or version where the CLI passes one and otherwise default to the process, because the control plane and the agent use them too. Time (`Date.now`) and the temporary directory are not part of the boundary. ## Desired state and reconciliation diff --git a/docs/testing.md b/docs/testing.md index 498c006..7f04c32 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -36,7 +36,7 @@ Managed-ingress convergence tests exercise current and outdated configuration ha - **Shell scripts** are tested by running the generated text against fake executables placed first on `PATH`, instead of asserting on strings (`src/cli/install.test.ts`). - **Image archives** are built in the test, so malformed and hostile layouts can be checked without Docker (`src/images/artifacts.test.ts`). - **The CLI runs in-process.** The CLI reaches its process only through a `CliRuntime` ([architecture](architecture.md#the-cli-runtime)), so `execCli` in `tests/cli-inprocess.ts` builds one from plain objects (`testRuntime` in `tests/cli-runtime.ts`: the environment, working directory, stdin, streams and signals of that one call), runs the real `runCli` and resolves or rejects like a promisified `execFile` (`code`, `stdout`, `stderr`). Nothing global is patched, so calls may overlap and a test cannot leak its environment into the next. Every call has a 60 second deadline, as a child process had, so a hang fails the test. A node process costs about half a second to start, so parsing, help, `--json`, config and API tests use `execCli`; a test of one helper (`runStep`, `createProgress`, `importCredential`) builds a `testRuntime` and passes `rt` to it. Start a real process only for what in-process cannot show: a terminal (`--import` with a fake TTY), signals, a command that must exit while stdin stays open, and the built `dist` entry. CI sets `NODE_COMPILE_CACHE` for `npm test` so the processes that remain compile the module graph once; export it locally for the same effect. -- **The CLI** is run against a stub `ssh` that records its arguments (`src/cli/templates.process.test.ts`). Tests that run the install commands also put a stub `docker` first on the PATH (`writeStubDocker` in `tests/helpers.ts`) that answers `manifest inspect` as published and passes everything else to the real Docker: the commit that cuts a release runs CI before its image exists, and a test must not depend on the registry. Subprocess exit tests leave stdin open after secret entry, repeated visible/hidden prompts, prompted command failure, HTTP responses, and interactive exec success, nonzero exit, remote error, disconnect and SIGINT cancellation (`src/cli/commands/secret.process.test.ts`, `src/cli/prompt.process.test.ts`, `src/cli/stdin.process.test.ts`, `src/cli/live-transport.process.test.ts`). Each subprocess has a 60 second deadline so a leaked handle fails the test. The deadline catches a hang and is not a speed check: a 5 or 10 second one failed `secret set`, `var set` and `ps` runs on CI runners that start a dozen Node processes at once, and 60 seconds still fails a hang within the job's budget. A timer-based test that needs a state to last or to end (a rolling `drain`) states both sides with a margin of hundreds of milliseconds, not tens, since a runner can pause for longer than a short drain between two calls. Whole-document readers (`init -`, `node register --csr -`, backup destination credentials and runtime `agent install-cert`) receive EOF and must then exit; they intentionally wait for the complete document before producing a result. +- **The CLI** is run against a stub `ssh` that records its arguments (`src/cli/templates.process.test.ts`). The host key remedy is guarded in layers instead of by running every command: unit tests cover the hint, the wrappers and the host probes, an ESLint rule keeps ssh behind those wrappers, and `src/cli/ssh-hostkey.test.ts` runs only the commands that catch an ssh failure. The install commands ask the registry whether the default image exists, which is false on the commit that cuts a release (CI runs before its image is built), so no test may depend on the registry. An in-process test gets that for free: `testRuntime` sets `inspectImage` on the runtime to answer "published" (pass your own to test the unpublished path). A test that starts a real process puts a stub `docker` first on the PATH (`writeStubDocker` in `tests/helpers.ts`) that answers `manifest inspect` as published and passes everything else to the real Docker. Subprocess exit tests leave stdin open after secret entry, repeated visible/hidden prompts, prompted command failure, HTTP responses, and interactive exec success, nonzero exit, remote error, disconnect and SIGINT cancellation (`src/cli/commands/secret.process.test.ts`, `src/cli/prompt.process.test.ts`, `src/cli/stdin.process.test.ts`, `src/cli/live-transport.process.test.ts`). Each subprocess has a 60 second deadline so a leaked handle fails the test. The deadline catches a hang and is not a speed check: a 5 or 10 second one failed `secret set`, `var set` and `ps` runs on CI runners that start a dozen Node processes at once, and 60 seconds still fails a hang within the job's budget. A timer-based test that needs a state to last or to end (a rolling `drain`) states both sides with a margin of hundreds of milliseconds, not tens, since a runner can pause for longer than a short drain between two calls. Whole-document readers (`init -`, `node register --csr -`, backup destination credentials and runtime `agent install-cert`) receive EOF and must then exit; they intentionally wait for the complete document before producing a result. - **Waiting** for something asynchronous uses `until(condition, what)` from `tests/wait.ts`, never a fixed sleep followed by an assertion. A sleep is a guess about machine speed, and CI runners that run test files in parallel are slower than a laptop at unpredictable moments, so a guess that passes locally fails there now and then (the flakes in the job runner, the stuck control backup and the TUI poller were all this). `until` polls with a generous deadline that a passing test never pays for and fails naming what it waited for. Do not assert how many times a timer fired in a fixed window; wait for the count you need, or compare two rates over the same elapsed time. A fixed sleep is still right to show that something does *not* happen, after waiting for the last positive signal. ### Fast and process test files diff --git a/eslint.config.js b/eslint.config.js index feb15b6..80f63ef 100644 --- a/eslint.config.js +++ b/eslint.config.js @@ -95,6 +95,29 @@ export default [ ], }, }, + { + // Every ssh the CLI runs goes through runStep, runCapture or runLocal (src/cli/process.ts) or the host probes + // (src/cli/commands/host.ts), which turn a host key refusal into the ssh-keygen -R remedy and leave known_hosts + // alone. A command that spawned ssh itself would skip that, so child processes are started only in those modules + // (and by config.ts, which runs git). + files: ['src/cli/**/*.ts'], + ignores: [ + 'src/cli/**/*.test.ts', + 'src/cli/process.ts', + 'src/cli/commands/host.ts', + 'src/cli/config.ts', + ], + rules: { + 'no-restricted-syntax': [ + 'error', + { + selector: 'ImportDeclaration[source.value=/^(node:)?child_process$/]', + message: + 'Run ssh through runStep, runCapture or runLocal (src/cli/process.ts) so a host key refusal prints the ssh-keygen -R remedy; do not start child processes here.', + }, + ], + }, + }, { files: ['tests/**/*.ts', 'src/**/*.test.ts'], languageOptions: { diff --git a/src/cli/commands/host.test.ts b/src/cli/commands/host.test.ts index 806483f..f1a84ca 100644 --- a/src/cli/commands/host.test.ts +++ b/src/cli/commands/host.test.ts @@ -5,7 +5,14 @@ import os from 'node:os'; import path from 'node:path'; import { createHash } from 'node:crypto'; import * as tar from 'tar'; -import { bootstrapCommand, dockerBootstrap, immutableLocalTag, transferTugImage } from './host.ts'; +import { + assertPublished, + bootstrapCommand, + dockerBootstrap, + immutableLocalTag, + transferTugImage, +} from './host.ts'; +import { defaultImage } from '../../images/image.ts'; import { STAGE_MARKER } from '../process.ts'; import { testRuntime } from '../../../tests/cli-runtime.ts'; @@ -131,3 +138,19 @@ test("the Docker bootstrap command hands the script this run's stage marker", as 'the script names its stage with the marker the command provides, and works without one (cloud-init)', ); }); + +test('the registry lookup of a test runtime says published, and an unpublished answer stops the command', () => { + const image = defaultImage('999.0.0'); + assert.doesNotThrow( + () => assertPublished(testRuntime().rt, image), + 'a test runtime must never reach the registry, so an unreleased version reads as published', + ); + const unpublished = testRuntime({ + inspectImage: () => ({ status: 1, stderr: 'no such manifest' }), + }).rt; + assert.throws( + () => assertPublished(unpublished, image), + /999\.0\.0 is not published yet/, + 'the runtime lookup should be the one consulted, and a missing manifest should name the image', + ); +}); diff --git a/src/cli/commands/host.ts b/src/cli/commands/host.ts index fec1a7b..54892fc 100644 --- a/src/cli/commands/host.ts +++ b/src/cli/commands/host.ts @@ -152,15 +152,19 @@ function probe(rt: CliRuntime, target: string, command: string) { * manifest is looked up with this run's environment, so a Docker on its PATH is the one that answers. */ export function assertPublished(rt: CliRuntime, image: string): void { - assertImagePublished(image, (ref) => { - const result = spawnSync('docker', ['manifest', 'inspect', ref], { - encoding: 'utf8', - timeout: 20_000, - env: rt.env, - cwd: rt.cwd, - }); - return { status: result.status, stderr: String(result.stderr ?? '') }; - }); + assertImagePublished( + image, + rt.inspectImage ?? + ((ref) => { + const result = spawnSync('docker', ['manifest', 'inspect', ref], { + encoding: 'utf8', + timeout: 20_000, + env: rt.env, + cwd: rt.cwd, + }); + return { status: result.status, stderr: String(result.stderr ?? '') }; + }), + ); } /** Whether the host already has a working Docker Engine. */ diff --git a/src/cli/runtime-boundary.test.ts b/src/cli/runtime-boundary.test.ts index b362d3e..27c397a 100644 --- a/src/cli/runtime-boundary.test.ts +++ b/src/cli/runtime-boundary.test.ts @@ -58,3 +58,15 @@ test('code that takes everything from the runtime is accepted', async () => { ].join('\n'); assert.deepEqual(await errors(code, 'src/cli/example.ts'), [], 'the runtime forms are the way'); }); + +test('only the process and host modules may start child processes, so every ssh gets the host key remedy', async () => { + const code = "import { spawnSync } from 'node:child_process';\nexport const a = spawnSync;"; + for (const spelling of [code, code.replace('node:child_process', 'child_process')]) + assert.ok( + (await errors(spelling, 'src/cli/commands/node.ts')).length > 0, + 'a command importing child_process would run ssh without the host key remedy', + ); + for (const file of ['src/cli/process.ts', 'src/cli/commands/host.ts', 'src/cli/config.ts']) + assert.deepEqual(await errors(code, file), [], `${file} is allowed to start child processes`); + assert.deepEqual(await errors(code, 'src/control/example.ts'), [], 'other roles are not covered'); +}); diff --git a/src/cli/runtime.ts b/src/cli/runtime.ts index 410d295..2268462 100644 --- a/src/cli/runtime.ts +++ b/src/cli/runtime.ts @@ -57,6 +57,11 @@ export interface CliRuntime { signals: EventEmitter; /** Sends the signal to this process, for a handler that restored the terminal and now lets the signal act. */ raise(signal: NodeJS.Signals): void; + /** + * Answers whether a registry image exists, as `docker manifest inspect` would. Unset runs that command with this + * run's environment; a test runtime sets it so no test depends on the registry or on which release is published. + */ + inspectImage?: (image: string) => { status: number | null; stderr: string }; /** The exit code a command asked for; the entry point applies it. Unset means success. */ exitCode?: number; state: CliState; diff --git a/src/cli/ssh-hostkey.test.ts b/src/cli/ssh-hostkey.test.ts index b923d36..0df1429 100644 --- a/src/cli/ssh-hostkey.test.ts +++ b/src/cli/ssh-hostkey.test.ts @@ -123,7 +123,7 @@ test('host probes stop on a host key refusal instead of reading it as no Docker }); }); -test('every command that runs ssh prints the ssh-keygen -R remedy and never touches known_hosts', async () => { +test('the commands that catch an ssh failure still print the ssh-keygen -R remedy and never touch known_hosts', async () => { const dir = await fs.mkdtemp(path.join(os.tmpdir(), 'tug-hostkey-cli-')); const control = await startMtlsControl(path.join(dir, 'data')); const bin = path.join(dir, 'bin'); @@ -147,12 +147,13 @@ test('every command that runs ssh prints the ssh-keygen -R remedy and never touc const cli = (...args: string[]) => execCli([...args], { env, cwd: dir }); try { for (const args of [ + // One command that runs ssh directly; the rest reach it through the wrappers, which src/cli's lint rule enforces + // (runtime-boundary.test.ts) and the tests above cover. ['node', 'add', target, '--name', 'n1', '--no-wait'], - ['node', 'replace', 'n1', target, '--no-wait'], - ['node', 'repair', 'n1', target, '--no-wait'], + // The commands that catch an ssh failure: each must let the host key refusal through instead of rewording it. ['node', 'upgrade', 'n1', target, '--no-wait'], - ['node', 'migrate', 'n1', target, '--no-wait'], - ['node', 'remove', 'n1', target, '--teardown', '-y'], + ['control', 'upgrade', target, '-y'], + // Reads the installation first and treats any failure there as "nothing installed yet". [ 'control', 'setup', @@ -163,10 +164,6 @@ test('every command that runs ssh prints the ssh-keygen -R remedy and never touc '--credential-out', path.join(dir, 'c'), ], - ['control', 'upgrade', target, '-y'], - ['control', 'migrate', target], - ['control', 'stop', target, '-y'], - ['control', 'restart', target, '-y'], ]) { const label = `tug ${args.slice(0, 2).join(' ')}`; const failure = await cli(...args).then( diff --git a/tests/cli-runtime.ts b/tests/cli-runtime.ts index f01a7b6..586c9ff 100644 --- a/tests/cli-runtime.ts +++ b/tests/cli-runtime.ts @@ -13,6 +13,11 @@ export interface TestRuntimeOptions { input?: string; /** The version the CLI reports; defaults to this package's. */ version?: string; + /** + * Answers the registry lookup of the install commands; defaults to "published". The real lookup is false on the + * commit that cuts a release, whose image builds minutes later, so a test must never reach it unless it says so. + */ + inspectImage?: CliRuntime['inspectImage']; /** Present stdin and stdout as terminals. */ tty?: boolean; columns?: number; @@ -51,6 +56,7 @@ export function testRuntime(options: TestRuntimeOptions = {}) { stderr: sink(err, false, 100), inheritStdio: false, version: options.version ?? packageVersion(), + inspectImage: options.inspectImage ?? (() => ({ status: 0, stderr: '' })), signals: new EventEmitter(), raise: () => {}, state: freshState(env),