hack: repoint the kind kubeconfig when it cannot reach the apiserver - #1292
Open
Yuan Gao (ygao-g) wants to merge 3 commits into
Open
hack: repoint the kind kubeconfig when it cannot reach the apiserver#1292Yuan Gao (ygao-g) wants to merge 3 commits into
Yuan Gao (ygao-g) wants to merge 3 commits into
Conversation
On an IPv6-only cluster the script rewrote kind's `https://[::1]:PORT` kubeconfig entry to `https://localhost:PORT` unconditionally. That breaks any host whose `/etc/hosts` leaves `localhost` off the `::1` line, including the Ubuntu cloud image Lima runs: `localhost` resolves v4-only and every later kubectl fails at connect. The rewrite existed for one case, a macOS client reaching kind inside a Lima VM, where limactl re-forwards the published port to the host's v4 loopback only. Running the loop inside the guest reaches `[::1]` directly and avoids that path entirely.
On a fresh IPv6-only kind cluster nothing resolves from inside a pod, so the install never completes. CoreDNS inherits the node's IPv4 resolver, which a v6-only pod cannot reach, and the in-cluster registry has no name a pod can look up. This gives the cluster its own Corefile, gated on ipv6: a forward to an IPv6 upstream, overridable with IPV6_DNS_UPSTREAM, and a kind-registry:53 server block so atelet can pull from its own network namespace. The Corefile patch runs kubectl straight after `kind create`, which returns before the apiserver answers, so the script now waits for the control plane from inside the node first.
For macOS + Lima setup, where the daemon lives in a VM, kind publishes the port inside the VM and limactl re-forwards it to the host's v4 loopback only, so an IPv6-only cluster leaves an unreachable [::1] entry and the next kubectl dies at connect. The fix is to probe that entry once and, when it is refused, repoint at whichever loopback spelling answers.
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1099
Stacked on #958 — merge that first. The first two commits here are #958's; only
the last one is new. #958 removes the unconditional repoint; this restores it in
a checked form, which is what #1099 tracks.
Where the Docker daemon lives in a VM (Lima on macOS, for instance), kind
publishes the port inside the VM at
[::1]and limactl re-forwards it to thehost's v4 loopback only, so an IPv6-only cluster is left with an unreachable
[::1]kubeconfig entry and the next kubectl dies at connect. Seelima-vm/lima#1540.
The script now probes that entry once and, if it is refused, repoints at
https://localhost:<port>.localhostis the only spelling that reaches it:[::1]is refused because limactl does not forward v6, and127.0.0.1connectsbut fails verification, because the cert an IPv6-only cluster issues carries
localhostas a DNS SAN and no v4 IP SAN. One probe is enough because #958'sin-node wait has already established readiness: a refusal here means the address
is wrong, not that the apiserver is slow, so there is no retry to mask a real
outage. #877 rewrote the entry unconditionally, which is what broke hosts where
localhostis v4-only. The probe runs for everyIP_FAMILY: on ipv4 and dual itanswers first try and nothing is rewritten.
Verified by hand. CI cannot cover this — a GitHub runner reaches its own
published port, so the new code is a no-op there.
IP_FAMILY=ipv6https://[::1]:41985dial tcp [::1]:49276: connect: connection refusedat the CoreDNS patch, exit 1[::1]:52600refused, repointed tohttps://localhost:52600, exit 0The first row is the regression guard for the behaviour #958 fixes; the second is
the case it leaves broken. An
IP_FAMILY=ipv4run on the same host is a no-op:nothing is rewritten and the kubeconfig is left at
https://127.0.0.1:<port>.IP_FAMILY=ipv6andIP_FAMILY=ipv4runs on a macOS/Lima host)🤖 Generated with Claude Code