Setting kind inference server

increasing inotify max values

# append this to /ect/sysctl.conf
fs.inotify.max_user_watches = 524288
fs.inotify.max_user_instances = 512

Kind needs to watch a lot of files in order to keep things running, which with default values from ubuntu (8192) will not be enough.

Create the cluster using kind-config.yaml

kind create cluster --config kind-config.yaml --name inference-cluster

As we set disableDefaultCNI: true it wont have communication between the cluster resources, so we need to install our custom CNI. My choice was Calico as I am already familiar with it's features and installation.

Installing Calico CNI

Refer to https://docs.tigera.io/calico/latest/getting-started/kubernetes/kind

1. Install Calico's CRDs and Operator

# Set your desired Calico Version
CALICO_VERSION=v3.31.4

kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/$CALICO_VERSION/manifests/operator-crds.yaml
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/$CALICO_VERSION/manifests/tigera-operator.yaml

2. Install Calico resources

Make sure that the cidr under spec.calicoNetwork.ipPools[0].cidr matches the podSubnet configured in our kind cluster (in this case: 192.168.0.0/16)

# Set your wished Calico version
CALICO_VERSION=v3.31.4 

curl -O kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/$CALICO_VERSION/manifests/custom-resources.yaml