Skip to content

Make CLI tests independent of the published image - #152

Merged
kasparsd merged 2 commits into
mainfrom
fix-registry-free-cli-tests
Nov 17, 2026
Merged

kasparsd merged 2 commits into
mainfrom
fix-registry-free-cli-tests

Conversation

@kasparsd

@kasparsd kasparsd commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Why main went red

CI failed on the `chore: version packages (#141)` commit in all three Check jobs. `every command that runs ssh prints the ssh-keygen -R remedy` ran `tug node add` with the package's default image (`ghcr.io/wpelevator/tug:3.1.0`). The CLI looks that image up in the registry first, and on the commit that cuts a release it is not built yet. The command stopped with "is not published yet" before it reached ssh, so the assertion never saw the remedy. Every release commit would have failed the same way.

Fix

  • `CliRuntime` gets an optional `inspectImage`; `assertPublished` uses it when set and otherwise runs `docker manifest inspect` as before.
  • `testRuntime` sets it to "published" by default, so no in-process CLI test can reach the registry, whichever release is current. A test can pass its own to exercise the unpublished path.
  • New test in `host.test.ts` covers both answers.
  • `docs/testing.md` now says this instead of asking each test to remember `writeStubDocker`, which stays for real-process tests.

No changeset: test-only, no change to the CLI, wire protocol or persisted formats.

Verification

Typecheck, lint and prettier pass locally, as do `ssh-hostkey.test.ts` and `host.test.ts`. The full `npm test` was still running when I opened this; CI is the check for the rest.

🤖 Generated with Claude Code

Follow-up in this PR: a cheaper guard for the ssh claim

The failing test claimed every command that runs ssh prints the host key remedy. Unit tests already cover the hint, the wrappers and the probes, so the end-to-end run only had to show that every ssh goes through those wrappers. An ESLint rule now enforces that (child processes only in process.ts, commands/host.ts and config.ts), tested in runtime-boundary.test.ts. The end-to-end test runs 4 commands instead of 11: the ones that catch an ssh failure, plus node add.

kasparsd and others added 2 commits November 17, 2026 09:38
The ssh host key test ran `tug node add` with the package's default image, which the CLI looks up in the registry. On the commit that cuts a release the image is not built yet, so the lookup failed and CI went red on every version-packages commit.

The runtime now carries an optional `inspectImage`, and the test runtime answers "published" by default, so no in-process test reaches the registry.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
An ESLint rule keeps child processes, and so ssh, inside process.ts and host.ts, where the host key remedy is applied. The end-to-end test shrinks from eleven commands to the ones that catch an ssh failure.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@kasparsd
kasparsd merged commit 99540dd into main Nov 17, 2026
9 checks passed
@kasparsd
kasparsd deleted the fix-registry-free-cli-tests branch November 17, 2026 08:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant