During upgrade fron 1.6 to 1.7 namespace-node-affinity-pod-webhook haven't been recreated however namespace-node-affinity-webhook-certs secret was refreshed.
Which leading to old secrets are still mounted to the pod. And it can't work with the following error message in the log:
"failed calling webhook "namespace-node-affinity-pod-webhook.default.svc": failed to call webhook: Post "https://namespace-node-affinity-pod-webhook.kubeflow.svc:443/mutate?timeout=5s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "127.0.0.1")"
To fix the issue just delete webhook pod was enough.
During upgrade fron 1.6 to 1.7 namespace-node-affinity-pod-webhook haven't been recreated however namespace-node-affinity-webhook-certs secret was refreshed.
Which leading to old secrets are still mounted to the pod. And it can't work with the following error message in the log:
To fix the issue just delete webhook pod was enough.