Skip to content

Bound the VAX image test's wait for sshd on the clock - #13

Merged
jacob-carlborg merged 1 commit into
masterfrom
claude/vax-ssh-poll-deadline-wsrfrg
Aug 15, 2026
Merged

Bound the VAX image test's wait for sshd on the clock#13
jacob-carlborg merged 1 commit into
masterfrom
claude/vax-ssh-poll-deadline-wsrfrg

Conversation

@jacob-carlborg

Copy link
Copy Markdown
Contributor

The wait counted attempts — 180 of them, spaced by sleep 20 — and the comment claimed that came to about an hour:

ok=
for _ in $(seq 1 180); do
  if ssh_vax true 2>/dev/null; then ok=1; break; fi
  sleep 20
done

It only comes to an hour when every attempt fails instantly, which is what happens while the guest's ssh port is still closed: slirp answers with an RST. An attempt against a guest that accepts the connection and then stalls costs the whole ConnectTimeout instead — 60s here — making the same loop nearly four hours.

That is not hypothetical. Two VAX jobs on the same commit failed the same nominal way and took wildly different amounts of time:

job outcome Test Image (vax)
10.1, vax failed 60m21s — every attempt RST, so the intended bound held
11.0, vax still running when noticed 2h+, and would have run to ~4h

So the bound depended on how the guest was broken, which is the one thing a timeout should not depend on.

The fix

Take the deadline off the clock, so it holds regardless of what an individual attempt costs:

deadline=$(($(date +%s) + 3600))
ok=
while [ "$(date +%s)" -lt "$deadline" ]; do
  if ssh_vax true 2>/dev/null; then ok=1; break; fi
  sleep 20
done

The overshoot is now just the attempt in flight when the hour passes, instead of a multiple of the whole budget. Verified against a stubbed ssh_vax at 1/1000th scale — exits immediately on success, and honours the budget whether attempts fail instantly or burn time (overshooting by one attempt, ~80s at real scale).

The comment also loses its "~15 minutes" figure. A healthy guest reaches sshd in about two minutes; that number described first-boot host key generation, which the image stopped doing when the keys started being baked in.

Scope

Branched from master, and touches only the VAX test step, so it is independent of both #11 and the ntpdate fix in #12. This is purely about the harness bounding itself — #12 addressed one cause of a stalled guest, whereas this makes any future cause cost an hour instead of four.


Generated by Claude Code

The wait counted attempts -- 180 of them, spaced by sleep 20 -- and the
comment claimed that came to about an hour. It only does when every
attempt fails instantly, which is what happens while the guest's ssh
port is still closed: slirp answers with an RST.

An attempt against a guest that accepts the connection and then stalls
costs the whole ConnectTimeout instead, which is 60s here, making the
same loop nearly four hours. That is not hypothetical: a guest that
stalled mid-boot sat in this step for over two hours before anyone
noticed, and would have kept the runner for four.

Take the deadline off the clock so the bound holds regardless of what an
individual attempt costs. The overshoot is now just the attempt in
flight when the hour passes, rather than a multiple of the whole budget.

Also drop the "~15 minutes" figure from the comment while rewriting it.
A healthy guest reaches sshd in about two minutes; the host keys stopped
being generated on first boot a while ago, which is what that number
described.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M9LHkNMNNfDf2Fa1ieGT8g
@jacob-carlborg
jacob-carlborg merged commit 6513d02 into master Aug 15, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants