[WIP] hack: add opt-in NAT64 support to IPv6-only kind clusters - #1275
Draft
Yuan Gao (ygao-g) wants to merge 4 commits into
Draft
[WIP] hack: add opt-in NAT64 support to IPv6-only kind clusters#1275Yuan Gao (ygao-g) wants to merge 4 commits into
Yuan Gao (ygao-g) wants to merge 4 commits into
Conversation
This was referenced Aug 27, 2026
Collaborator
|
why do we need egress off of the host machine from the test cluster at all? |
Collaborator
|
we push images into the registry / cluster |
Collaborator
Author
Benjamin Elder (@BenTheElder) It's the actor-egress e2e tests, which fetch real external origins:
|
Yuan Gao (ygao-g)
force-pushed
the
kind-ipv6-nat64
branch
from
August 28, 2026 00:21
a22c540 to
de34199
Compare
Yuan Gao (ygao-g)
force-pushed
the
kind-ipv6-nat64
branch
from
August 28, 2026 16:23
de34199 to
39d6148
Compare
Yuan Gao (ygao-g)
marked this pull request as draft
August 28, 2026 22:50
Yuan Gao (ygao-g)
force-pushed
the
kind-ipv6-nat64
branch
11 times, most recently
from
August 31, 2026 22:48
30a4ba9 to
09a9de5
Compare
Yuan Gao (ygao-g)
assigned
Yuan Gao (ygao-g) and
Bowei Du (bowei) and unassigned
Yuan Gao (ygao-g)
Yuan Gao (ygao-g)
force-pushed
the
kind-ipv6-nat64
branch
from
August 31, 2026 23:07
09a9de5 to
4e5b685
Compare
Yuan Gao (ygao-g)
force-pushed
the
kind-ipv6-nat64
branch
from
September 1, 2026 22:05
4e5b685 to
a71b19d
Compare
Yuan Gao (ygao-g)
force-pushed
the
kind-ipv6-nat64
branch
3 times, most recently
from
September 1, 2026 23:42
16dfaea to
d4aa49b
Compare
Before, on an IPv6-only cluster the script rewrote kind's `https://[::1]:PORT` kubeconfig entry to `https://localhost:PORT` unconditionally. That breaks any host whose `/etc/hosts` leaves `localhost` off the `::1` line, including the Ubuntu cloud image Lima runs. Remove the repoint logic, but note that limactl re-forwards the published port to the host's v4 loopback only, so a macOS client cannot directly access kind clusters run inside a Lima VM via `[::1]`. Instead, operations should run inside the guest. Tested: manual tests creating IPv6-only kind clusters on a local env with IPv6 egress, namely macOS + Lima VM.
Yuan Gao (ygao-g)
force-pushed
the
kind-ipv6-nat64
branch
from
September 3, 2026 20:22
d4aa49b to
91951ab
Compare
CoreDNS inherits the node's IPv4 resolver, which a v6-only pod cannot reach. Change CoreDNS's Corefile to forward to an overridable IPv6 upstream. Also add a kind-registry:53 server block to CoreDNS. In addition, `kind create` returns before the apiserver answers, so add logic to wait for the control plane from inside the node first. Tested: manual tests of creation and installation on a local env with IPv6 egress, namely macOS + Lima VM. CI cannot verify this — the GitHub runner is IPv4-only, so an IPv6-only cluster there additionally needs DNS64 and NAT64.
IP_FAMILY=ipv6 has no documentation, so the two things that reliably go wrong get rediscovered every time: a Docker daemon with IPv6 off, and a host that cannot route IPv6 at all. The second is usually measured with `curl -6 <host>`, which cannot tell a resolver returning no AAAA apart from a host with no route, and reports neither. Also covers the kubeconfig rewrite kubectl needs when the cluster runs in a Lima VM on macOS, where kind publishes the apiserver on [::1] and Lima forwards that port to the host's 127.0.0.1.
Yuan Gao (ygao-g)
force-pushed
the
kind-ipv6-nat64
branch
from
September 3, 2026 20:26
91951ab to
18cc6f3
Compare
Collaborator
Author
Updates on this:
Benjamin Elder (@BenTheElder) Does this answer your questions? |
Yuan Gao (ygao-g)
changed the title
An IPv6-only kind cluster on a host without IPv6 egress comes up but reaches nothing: external names resolve to unroutable addresses, and the tree had no way to set up the translation such a host needs. IPV6_DNS64_PREFIX now deploys the kubernetes-sigs/nat64 agent, points cluster DNS through it, and probes from a pod that both halves meet. Off by default, since translate_all routes every external name through a translator a host with working IPv6 egress does not need. The agent's IPv4 pool caps a node at a /120 Pod CIDR, so the prefix narrows podSubnet too.
Yuan Gao (ygao-g)
force-pushed
the
kind-ipv6-nat64
branch
from
September 4, 2026 23:26
18cc6f3 to
872a8fb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Do not merge until #958 lands. Its two commits are in the diff below
because a fork branch cannot be based on another pull request; only the last
commit
is under review here.
Part of #246.
An IPv6-only kind cluster is usable on a host that has IPv6 egress of its own,
and not on one that does not: external names resolve to addresses nothing can
reach, so anything that pulls an image or calls out times out. Every GitHub
Actions runner is the second kind of host, which is what stops an IPv6-only CI
lane from getting as far as the install.
IPV6_DNS64_PREFIXonhack/create-kind-cluster.shturns on both halvestogether: it deploys the kubernetes-sigs/nat64 agent from
hack/third_party/nat64/and points cluster DNS through the prefix that agenttranslates. Neither half is any use alone — the translator sits unused without
the DNS half, and the synthesized addresses go nowhere without the translator.
It stays opt-in and off by default, because it routes every external name
through the prefix, including names with AAAA records that already work.
The agent runs in the cluster rather than on the host, so nothing here enables
forwarding on the developer's machine or installs a userspace translator. It
translates Pod source addresses only; a hostNetwork pod and containerd use the
node's own dual-stack interfaces and reach IPv4 without it. Its
--nat-v4-cidrdefault is why the script narrows each node's pod CIDR to a
/120.The manifest is vendored rather than fetched at run time, because upstream ships
no release asset and its
install.yamlat tag v0.4.1 still names the v0.2.1image — the only copy matching the release is on a moving branch. So
hack/third_party/nat64/VERSIONpins the upstream commit and the image digest,and
hack/update/nat64.shregenerates the manifest from that pin. The copy istherefore reproducible rather than hand-made: re-running the script and
diffing is a drift check. Upstream's LICENSE is vendored alongside it.
It is amd64 only in practice — every published tag ships an x86-64 binary in the
arm64 slot of its image index — so
NAT64_IMAGEoverrides it for arm64developers. That covers CI, which is amd64.
#939 is the IPv6-only CI job this exists for. It asks for this setup from
hack/and reads back what it got, rather than configuring the translatorinline: a job that supplied the very thing it exists to gate would prove
nothing by going green. It is also how this is verified — a dispatch of
958 753 1080 979 1116 1057 1275ontomaincomes up green(example run).
b4e5c9db3d8fa87f4e5b6859🤖 This PR was developed with AI assistance. I have reviewed and tested all changes.