Firewalld blocking K3s interfaces by default

[brittyazel]

After installing CIQ Ascender on a VM of ours running Rocky 9 using K3s, I was immediately faced with issues after re-enabling Firewalld. I followed the two guides listed here and here, and I opened all of the ports listed, but even still there were ports needing to be opened. The ports listed in the guide were working for incoming access to server, generally speaking, but from reading about how K3s orchestrates communications amongst itself, there are interfaces internal to K3s that also need to be accounted for in Firewalld to allow the different pods and services to communicate with each other.

Following the upstream K3s installation guide, they have this section:

It is recommended to turn off firewalld:

systemctl disable firewalld --now

If you wish to keep firewalld enabled, by default, the following rules are required:

firewall-cmd --permanent --add-port=6443/tcp #apiserver
firewall-cmd --permanent --zone=trusted --add-source=10.42.0.0/16 #pods
firewall-cmd --permanent --zone=trusted --add-source=10.43.0.0/16 #services
firewall-cmd --reload

While it was unnecessary for me to open port 6443/tcp as I am hosting everything locally, adding the 10.42.0.0/16 and 10.43.0.0/16 to the trusted zone resolved my issues outright and got everything working as it should.

I'm not sure if the solution here is to add something to the installer to add these ports if you enabled kube-install, or perhaps the solution is to just update the documentation to make a note of adding these two IP ranges to the trusted zone. Either way, the instructions make it seem like it's adequate to keep Firewalld enabled as long as you set the specifc https/http/ssh rules, so it seems like those sections can and should be clarified accordingly with this information.

[cigamit]

Ya, we typically just disable the firewall for k3s because it is recommended by that same "requirements" page.

I will get with the team and decide whether to add something into the installer or just update the docs to give some points.

[linuxcube]

After installing CIQ Ascender on a VM of ours running Rocky 9 using K3s, I was immediately faced with issues after re-enabling Firewalld. I followed the two guides listed here and here, and I opened all of the ports listed, but even still there were ports needing to be opened. The ports listed in the guide were working for incoming access to server, generally speaking, but from reading about how K3s orchestrates communications amongst itself, there are interfaces internal to K3s that also need to be accounted for in Firewalld to allow the different pods and services to communicate with each other.

Following the upstream K3s installation guide, they have this section:

It is recommended to turn off firewalld:
systemctl disable firewalld --now
If you wish to keep firewalld enabled, by default, the following rules are required:
firewall-cmd --permanent --add-port=6443/tcp #apiserver
firewall-cmd --permanent --zone=trusted --add-source=10.42.0.0/16 #pods
firewall-cmd --permanent --zone=trusted --add-source=10.43.0.0/16 #services
firewall-cmd --reload

While it was unnecessary for me to open port 6443/tcp as I am hosting everything locally, adding the 10.42.0.0/16 and 10.43.0.0/16 to the trusted zone resolved my issues outright and got everything working as it should.

I'm not sure if the solution here is to add something to the installer to add these ports if you enabled kube-install, or perhaps the solution is to just update the documentation to make a note of adding these two IP ranges to the trusted zone. Either way, the instructions make it seem like it's adequate to keep Firewalld enabled as long as you set the specifc https/http/ssh rules, so it seems like those sections can and should be clarified accordingly with this information.

I normally disable firewalld and selinux to avoid issues with k3s, but do you need to have firewalld and selinux to operate in your environment?

[brittyazel]

@linuxcube We have very strict security requirements placed on us by my org, and while I could probably go through the security exception process to keep firewalld disabled, given that I was able to get K3s and firewalld working together with the rules above, I'm not sure if it's worth the paperwork. That said, that's not necessarily anyone at CIQs issue to solve, that's unique to my circumstance, but at the very least it would have been helpful if the documentation and guides could have mentioned this potential issue and workaround.