Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ Nothing in `src/cli/` reads the process it runs in. A command receives a `CliRun

What a run remembers (`--verbose`, whether work has started, the resolved configuration, the controls whose version was already checked, the running progress display) is `rt.state`, created with the runtime. No module keeps state between runs, so one process can run any number of commands, one after another or at the same time, and a run never has to be reset.

`src/cli.ts` builds the runtime of the real process (`processRuntime`) and calls `runCli(argv, rt)` from `src/cli/program.ts`, which builds the command tree for that run, loads only the module of the command the arguments name and returns the exit code. A test builds a runtime from plain objects (`tests/cli-runtime.ts`) and calls the same function, so it exercises the real parser and handlers without a child process and without patching a global. ESLint rejects `process`, `console` and `os.homedir` in `src/cli/` outside `runtime.ts`, so the boundary holds as the CLI grows. Shared domain modules (`images/`, `compose/`, `transport.ts`) take an explicit `env` or version where the CLI passes one and otherwise default to the process, because the control plane and the agent use them too. Time (`Date.now`) and the temporary directory are not part of the boundary.
`src/cli.ts` builds the runtime of the real process (`processRuntime`) and calls `runCli(argv, rt)` from `src/cli/program.ts`, which builds the command tree for that run, loads only the module of the command the arguments name and returns the exit code. A test builds a runtime from plain objects (`tests/cli-runtime.ts`) and calls the same function, so it exercises the real parser and handlers without a child process and without patching a global. ESLint rejects `process`, `console` and `os.homedir` in `src/cli/` outside `runtime.ts`, so the boundary holds as the CLI grows. The same configuration rejects `child_process` imports in `src/cli/` outside `process.ts` and `commands/host.ts` (and `config.ts`, which runs git), so every ssh goes through `runStep`, `runCapture`, `runLocal` or the host probes, which turn a host key refusal into the `ssh-keygen -R` remedy; `src/cli/runtime-boundary.test.ts` checks the rule. Shared domain modules (`images/`, `compose/`, `transport.ts`) take an explicit `env` or version where the CLI passes one and otherwise default to the process, because the control plane and the agent use them too. Time (`Date.now`) and the temporary directory are not part of the boundary.

## Desired state and reconciliation

Expand Down
2 changes: 1 addition & 1 deletion docs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Managed-ingress convergence tests exercise current and outdated configuration ha
- **Shell scripts** are tested by running the generated text against fake executables placed first on `PATH`, instead of asserting on strings (`src/cli/install.test.ts`).
- **Image archives** are built in the test, so malformed and hostile layouts can be checked without Docker (`src/images/artifacts.test.ts`).
- **The CLI runs in-process.** The CLI reaches its process only through a `CliRuntime` ([architecture](architecture.md#the-cli-runtime)), so `execCli` in `tests/cli-inprocess.ts` builds one from plain objects (`testRuntime` in `tests/cli-runtime.ts`: the environment, working directory, stdin, streams and signals of that one call), runs the real `runCli` and resolves or rejects like a promisified `execFile` (`code`, `stdout`, `stderr`). Nothing global is patched, so calls may overlap and a test cannot leak its environment into the next. Every call has a 60 second deadline, as a child process had, so a hang fails the test. A node process costs about half a second to start, so parsing, help, `--json`, config and API tests use `execCli`; a test of one helper (`runStep`, `createProgress`, `importCredential`) builds a `testRuntime` and passes `rt` to it. Start a real process only for what in-process cannot show: a terminal (`--import` with a fake TTY), signals, a command that must exit while stdin stays open, and the built `dist` entry. CI sets `NODE_COMPILE_CACHE` for `npm test` so the processes that remain compile the module graph once; export it locally for the same effect.
- **The CLI** is run against a stub `ssh` that records its arguments (`src/cli/templates.process.test.ts`). Tests that run the install commands also put a stub `docker` first on the PATH (`writeStubDocker` in `tests/helpers.ts`) that answers `manifest inspect` as published and passes everything else to the real Docker: the commit that cuts a release runs CI before its image exists, and a test must not depend on the registry. Subprocess exit tests leave stdin open after secret entry, repeated visible/hidden prompts, prompted command failure, HTTP responses, and interactive exec success, nonzero exit, remote error, disconnect and SIGINT cancellation (`src/cli/commands/secret.process.test.ts`, `src/cli/prompt.process.test.ts`, `src/cli/stdin.process.test.ts`, `src/cli/live-transport.process.test.ts`). Each subprocess has a 60 second deadline so a leaked handle fails the test. The deadline catches a hang and is not a speed check: a 5 or 10 second one failed `secret set`, `var set` and `ps` runs on CI runners that start a dozen Node processes at once, and 60 seconds still fails a hang within the job's budget. A timer-based test that needs a state to last or to end (a rolling `drain`) states both sides with a margin of hundreds of milliseconds, not tens, since a runner can pause for longer than a short drain between two calls. Whole-document readers (`init -`, `node register --csr -`, backup destination credentials and runtime `agent install-cert`) receive EOF and must then exit; they intentionally wait for the complete document before producing a result.
- **The CLI** is run against a stub `ssh` that records its arguments (`src/cli/templates.process.test.ts`). The host key remedy is guarded in layers instead of by running every command: unit tests cover the hint, the wrappers and the host probes, an ESLint rule keeps ssh behind those wrappers, and `src/cli/ssh-hostkey.test.ts` runs only the commands that catch an ssh failure. The install commands ask the registry whether the default image exists, which is false on the commit that cuts a release (CI runs before its image is built), so no test may depend on the registry. An in-process test gets that for free: `testRuntime` sets `inspectImage` on the runtime to answer "published" (pass your own to test the unpublished path). A test that starts a real process puts a stub `docker` first on the PATH (`writeStubDocker` in `tests/helpers.ts`) that answers `manifest inspect` as published and passes everything else to the real Docker. Subprocess exit tests leave stdin open after secret entry, repeated visible/hidden prompts, prompted command failure, HTTP responses, and interactive exec success, nonzero exit, remote error, disconnect and SIGINT cancellation (`src/cli/commands/secret.process.test.ts`, `src/cli/prompt.process.test.ts`, `src/cli/stdin.process.test.ts`, `src/cli/live-transport.process.test.ts`). Each subprocess has a 60 second deadline so a leaked handle fails the test. The deadline catches a hang and is not a speed check: a 5 or 10 second one failed `secret set`, `var set` and `ps` runs on CI runners that start a dozen Node processes at once, and 60 seconds still fails a hang within the job's budget. A timer-based test that needs a state to last or to end (a rolling `drain`) states both sides with a margin of hundreds of milliseconds, not tens, since a runner can pause for longer than a short drain between two calls. Whole-document readers (`init -`, `node register --csr -`, backup destination credentials and runtime `agent install-cert`) receive EOF and must then exit; they intentionally wait for the complete document before producing a result.
- **Waiting** for something asynchronous uses `until(condition, what)` from `tests/wait.ts`, never a fixed sleep followed by an assertion. A sleep is a guess about machine speed, and CI runners that run test files in parallel are slower than a laptop at unpredictable moments, so a guess that passes locally fails there now and then (the flakes in the job runner, the stuck control backup and the TUI poller were all this). `until` polls with a generous deadline that a passing test never pays for and fails naming what it waited for. Do not assert how many times a timer fired in a fixed window; wait for the count you need, or compare two rates over the same elapsed time. A fixed sleep is still right to show that something does *not* happen, after waiting for the last positive signal.

### Fast and process test files
Expand Down
23 changes: 23 additions & 0 deletions eslint.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,29 @@ export default [
],
},
},
{
// Every ssh the CLI runs goes through runStep, runCapture or runLocal (src/cli/process.ts) or the host probes
// (src/cli/commands/host.ts), which turn a host key refusal into the ssh-keygen -R remedy and leave known_hosts
// alone. A command that spawned ssh itself would skip that, so child processes are started only in those modules
// (and by config.ts, which runs git).
files: ['src/cli/**/*.ts'],
ignores: [
'src/cli/**/*.test.ts',
'src/cli/process.ts',
'src/cli/commands/host.ts',
'src/cli/config.ts',
],
rules: {
'no-restricted-syntax': [
'error',
{
selector: 'ImportDeclaration[source.value=/^(node:)?child_process$/]',
message:
'Run ssh through runStep, runCapture or runLocal (src/cli/process.ts) so a host key refusal prints the ssh-keygen -R remedy; do not start child processes here.',
},
],
},
},
{
files: ['tests/**/*.ts', 'src/**/*.test.ts'],
languageOptions: {
Expand Down
25 changes: 24 additions & 1 deletion src/cli/commands/host.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,14 @@ import os from 'node:os';
import path from 'node:path';
import { createHash } from 'node:crypto';
import * as tar from 'tar';
import { bootstrapCommand, dockerBootstrap, immutableLocalTag, transferTugImage } from './host.ts';
import {
assertPublished,
bootstrapCommand,
dockerBootstrap,
immutableLocalTag,
transferTugImage,
} from './host.ts';
import { defaultImage } from '../../images/image.ts';
import { STAGE_MARKER } from '../process.ts';
import { testRuntime } from '../../../tests/cli-runtime.ts';

Expand Down Expand Up @@ -131,3 +138,19 @@ test("the Docker bootstrap command hands the script this run's stage marker", as
'the script names its stage with the marker the command provides, and works without one (cloud-init)',
);
});

test('the registry lookup of a test runtime says published, and an unpublished answer stops the command', () => {
const image = defaultImage('999.0.0');
assert.doesNotThrow(
() => assertPublished(testRuntime().rt, image),
'a test runtime must never reach the registry, so an unreleased version reads as published',
);
const unpublished = testRuntime({
inspectImage: () => ({ status: 1, stderr: 'no such manifest' }),
}).rt;
assert.throws(
() => assertPublished(unpublished, image),
/999\.0\.0 is not published yet/,
'the runtime lookup should be the one consulted, and a missing manifest should name the image',
);
});
22 changes: 13 additions & 9 deletions src/cli/commands/host.ts
Original file line number Diff line number Diff line change
Expand Up @@ -152,15 +152,19 @@ function probe(rt: CliRuntime, target: string, command: string) {
* manifest is looked up with this run's environment, so a Docker on its PATH is the one that answers.
*/
export function assertPublished(rt: CliRuntime, image: string): void {
assertImagePublished(image, (ref) => {
const result = spawnSync('docker', ['manifest', 'inspect', ref], {
encoding: 'utf8',
timeout: 20_000,
env: rt.env,
cwd: rt.cwd,
});
return { status: result.status, stderr: String(result.stderr ?? '') };
});
assertImagePublished(
image,
rt.inspectImage ??
((ref) => {
const result = spawnSync('docker', ['manifest', 'inspect', ref], {
encoding: 'utf8',
timeout: 20_000,
env: rt.env,
cwd: rt.cwd,
});
return { status: result.status, stderr: String(result.stderr ?? '') };
}),
);
}

/** Whether the host already has a working Docker Engine. */
Expand Down
12 changes: 12 additions & 0 deletions src/cli/runtime-boundary.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,15 @@ test('code that takes everything from the runtime is accepted', async () => {
].join('\n');
assert.deepEqual(await errors(code, 'src/cli/example.ts'), [], 'the runtime forms are the way');
});

test('only the process and host modules may start child processes, so every ssh gets the host key remedy', async () => {
const code = "import { spawnSync } from 'node:child_process';\nexport const a = spawnSync;";
for (const spelling of [code, code.replace('node:child_process', 'child_process')])
assert.ok(
(await errors(spelling, 'src/cli/commands/node.ts')).length > 0,
'a command importing child_process would run ssh without the host key remedy',
);
for (const file of ['src/cli/process.ts', 'src/cli/commands/host.ts', 'src/cli/config.ts'])
assert.deepEqual(await errors(code, file), [], `${file} is allowed to start child processes`);
assert.deepEqual(await errors(code, 'src/control/example.ts'), [], 'other roles are not covered');
});
5 changes: 5 additions & 0 deletions src/cli/runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,11 @@ export interface CliRuntime {
signals: EventEmitter;
/** Sends the signal to this process, for a handler that restored the terminal and now lets the signal act. */
raise(signal: NodeJS.Signals): void;
/**
* Answers whether a registry image exists, as `docker manifest inspect` would. Unset runs that command with this
* run's environment; a test runtime sets it so no test depends on the registry or on which release is published.
*/
inspectImage?: (image: string) => { status: number | null; stderr: string };
/** The exit code a command asked for; the entry point applies it. Unset means success. */
exitCode?: number;
state: CliState;
Expand Down
15 changes: 6 additions & 9 deletions src/cli/ssh-hostkey.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ test('host probes stop on a host key refusal instead of reading it as no Docker
});
});

test('every command that runs ssh prints the ssh-keygen -R remedy and never touches known_hosts', async () => {
test('the commands that catch an ssh failure still print the ssh-keygen -R remedy and never touch known_hosts', async () => {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), 'tug-hostkey-cli-'));
const control = await startMtlsControl(path.join(dir, 'data'));
const bin = path.join(dir, 'bin');
Expand All @@ -147,12 +147,13 @@ test('every command that runs ssh prints the ssh-keygen -R remedy and never touc
const cli = (...args: string[]) => execCli([...args], { env, cwd: dir });
try {
for (const args of [
// One command that runs ssh directly; the rest reach it through the wrappers, which src/cli's lint rule enforces
// (runtime-boundary.test.ts) and the tests above cover.
['node', 'add', target, '--name', 'n1', '--no-wait'],
['node', 'replace', 'n1', target, '--no-wait'],
['node', 'repair', 'n1', target, '--no-wait'],
// The commands that catch an ssh failure: each must let the host key refusal through instead of rewording it.
['node', 'upgrade', 'n1', target, '--no-wait'],
['node', 'migrate', 'n1', target, '--no-wait'],
['node', 'remove', 'n1', target, '--teardown', '-y'],
['control', 'upgrade', target, '-y'],
// Reads the installation first and treats any failure there as "nothing installed yet".
[
'control',
'setup',
Expand All @@ -163,10 +164,6 @@ test('every command that runs ssh prints the ssh-keygen -R remedy and never touc
'--credential-out',
path.join(dir, 'c'),
],
['control', 'upgrade', target, '-y'],
['control', 'migrate', target],
['control', 'stop', target, '-y'],
['control', 'restart', target, '-y'],
]) {
const label = `tug ${args.slice(0, 2).join(' ')}`;
const failure = await cli(...args).then(
Expand Down
6 changes: 6 additions & 0 deletions tests/cli-runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ export interface TestRuntimeOptions {
input?: string;
/** The version the CLI reports; defaults to this package's. */
version?: string;
/**
* Answers the registry lookup of the install commands; defaults to "published". The real lookup is false on the
* commit that cuts a release, whose image builds minutes later, so a test must never reach it unless it says so.
*/
inspectImage?: CliRuntime['inspectImage'];
/** Present stdin and stdout as terminals. */
tty?: boolean;
columns?: number;
Expand Down Expand Up @@ -51,6 +56,7 @@ export function testRuntime(options: TestRuntimeOptions = {}) {
stderr: sink(err, false, 100),
inheritStdio: false,
version: options.version ?? packageVersion(),
inspectImage: options.inspectImage ?? (() => ({ status: 0, stderr: '' })),
signals: new EventEmitter(),
raise: () => {},
state: freshState(env),
Expand Down
Loading