Skip to content

Give the CLI a runtime instead of the process, and cut npm test from 2m49s to about 77s - #151

Merged
kasparsd merged 8 commits into
mainfrom
claude/sweet-davinci-ovjmax
Nov 17, 2026
Merged

kasparsd merged 8 commits into
mainfrom
claude/sweet-davinci-ovjmax

Conversation

@kasparsd

@kasparsd kasparsd commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

npm test took 2m49s (8m35s of CPU) because tests started the CLI as a child process 467 times, at about half a second each. A first version of this PR made the CLI callable in-process by patching process.env, cwd, stdout, stderr, stdin and process.exit from a test helper. Review found that this automated around the real problem: the CLI read ambient process state in about 25 files and kept per-run state in module singletons. This version removes the cause.

The CLI runtime

  • src/cli/runtime.ts defines CliRuntime: env, cwd, stdin, stdout, stderr, signals, version, exitCode, and state. Commands, helpers, prompts, pickers, progress and child-process runners take rt explicitly; output goes through out(rt, …) and err(rt, …), command-line paths through resolvePath(rt, …), the home directory through homeDir(rt). Child processes start with rt.env and rt.cwd, and their output is forwarded to the runtime's streams unless those are the real terminal.
  • What a run remembers (--verbose, work started, resolved config, version checks, the running progress display) is rt.state, created with the runtime. The four module singletons are gone, along with resetUsage, setVerbose, setResolvedConfig, resetVersionChecks and the pinPackageVersion seam, so nothing needs resetting between runs.
  • src/cli/program.ts exports runCli(argv, rt), which builds the command tree per run, loads only the module of the command the arguments name (root options come from the program itself, one REGISTRARS table), and returns the exit code. src/cli.ts is a three-line entry.
  • ESLint rejects process, console and os.homedir in src/cli/ outside runtime.ts, including node:process and process imports, globalThis.process and globalThis.console, and a named homedir import. src/cli/runtime-boundary.test.ts runs the repository's ESLint config over snippets so the rule cannot be loosened or bypassed silently. Shared domain modules (images/, compose/, transport.ts) take an explicit env or version from the CLI and default to the process for the control plane and agent. Time and the temp directory are not part of the boundary. See docs/architecture.md#the-cli-runtime.

Tests

  • tests/cli-runtime.ts builds a runtime from plain objects; execCli (tests/cli-inprocess.ts) runs the real runCli with one. Nothing global is patched, calls overlap safely, each has a 60 s deadline, and the TUG_LOG_LEVEL workaround and the global TUG_DATA_DIR leak in startMtlsControl are gone.
  • Tests that patched console, process.env.PATH, cwd, stdin or process.exitCode (wait-progress, ssh-hostkey, host, init.import, context.skew, progress, process) now use a runtime.
  • Files that must start a real process (terminal, piped stdin, signals, exit with stdin open, the built entry) are *.process.test.ts, import cliEntry/mainEntry from tests/cli-process.ts, and are guarded by src/test-layout.test.ts, which also rejects a fast file that spawns a process and names the CLI or runtime entry in any spelling. npm run test:process runs them.
  • CI sets NODE_COMPILE_CACHE on the test step. No runner script: npm test is plain node --test.
Measure Before After
Wall time, npm test (4 cores) 2 min 49 s about 77 s (70 s with --test-concurrency=4)
CPU time (user) 8 min 35 s about 3 min 22 s
CLI processes started 467 106
Test cases 1,175 1,192

docs/test-performance.md has the measurements (serial per-file timings, concurrency sweep) and docs/testing.md the conventions.

Review

Two independent read-only reviews of this branch found no regression in the real CLI: help, usage, errors and exit codes matched main byte for byte for 32 invocations, including multi-command modules, root options before the command, and a 60-column terminal. Their findings were fixed: a vacuous exit-code test now lints a failing file, the ESLint gaps above, --credential-out and relative TUG_CREDENTIALS_DIR/XDG_CONFIG_HOME now resolve against rt.cwd, tug lint still names an unreadable file as typed, the layout guard hole, and doc inconsistencies.

Test plan

  • npm test: 1,192 pass, 0 fail (with NODE_COMPILE_CACHE set)
  • npm run lint (3 pre-existing warnings), npm run typecheck, npm run build, npm run test:smoke:dist
  • The built dist/cli.js run by hand: exit codes and output for lint (ok, findings, missing file), an unknown command, --version, a missing option and a missing credential
  • CI run on this branch (Node 22.18.0, 22 and 24); only 22.23.4 was available locally, so top-level await in src/cli.ts under type stripping on 22.18.0 is untested here

Notes

  • Behavior is unchanged for operators: same commands, output and exit codes. tug credential list starts in about 0.34 s instead of 0.45 s because only that command's module loads.
  • Changeset: patch. The CLI contract, x-tug, the wire protocol and persisted formats are unchanged.

🤖 Generated with Claude Code

https://claude.ai/code/session_01B8Ng6auNzySjPMPZLrnpsh

runCli(argv) returns the exit code, loads one command module per call, and
backs execCli in tests, so parsing and API tests no longer start node.
npm test goes through tools/scripts/test.mjs with NODE_COMPILE_CACHE.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8Ng6auNzySjPMPZLrnpsh
…orter

The layout guard lands with the *.process.test.ts renames.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8Ng6auNzySjPMPZLrnpsh
Files that start real tug processes are *.process.test.ts and import their
entry points from tests/cli-process.ts. src/test-layout.test.ts keeps the
tier honest, and npm run test:fast / test:process run one tier.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8Ng6auNzySjPMPZLrnpsh
npm test is plain node --test again. The layout guard carries its own tier
helper, and test:process runs the *.process.test.ts files.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8Ng6auNzySjPMPZLrnpsh
…ements

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8Ng6auNzySjPMPZLrnpsh
Everything in src/cli that read process.env, cwd, stdio, signals, console or
the exit code now takes a CliRuntime (src/cli/runtime.ts). Per-run state that
lived in module singletons (verbose, usage, resolved config, version checks,
progress) lives on the runtime, so nothing resets between runs. runCli(argv,
rt) builds the command tree per run and loads only the named command.

execCli builds a test runtime from plain objects, so no global is patched and
calls overlap safely; each call has a 60 s deadline. ESLint rejects process,
console and os.homedir in src/cli outside runtime.ts. Shared image, compose and
transport modules take an explicit env or version from the CLI.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8Ng6auNzySjPMPZLrnpsh
@kasparsd kasparsd changed the title Cut the npm test run from 2m49s to about 70s: in-process CLI tests, lazy command loading, process test files Give the CLI a runtime instead of the process, and cut npm test from 2m49s to about 77s Nov 17, 2026
…ng paths, doc fixes

- the exit-code test lints a failing file, so it proves rt.exitCode reaches runCli's result
- ESLint also rejects node:process and process imports, globalThis.process/console and a named homedir import; src/cli/runtime-boundary.test.ts runs the repository config over snippets
- --credential-out and a relative TUG_CREDENTIALS_DIR / XDG_CONFIG_HOME resolve against rt.cwd
- lint names an unreadable file as typed instead of as an absolute path
- the layout guard also catches a fast file that spawns a process and names the CLI or runtime entry in any spelling
- docs: one headline time, correct file counts, wait-progress is in-process, no first person

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8Ng6auNzySjPMPZLrnpsh
@kasparsd
kasparsd merged commit 06e68cc into main Nov 17, 2026
9 checks passed
@kasparsd
kasparsd deleted the claude/sweet-davinci-ovjmax branch November 17, 2026 07:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants