Skip to content

Feat E2E CI and an exectuable#10

Open
leoarpin wants to merge 3 commits into
ca-gip:masterfrom
leoarpin:e2eCI
Open

Feat E2E CI and an exectuable#10
leoarpin wants to merge 3 commits into
ca-gip:masterfrom
leoarpin:e2eCI

Conversation

@leoarpin

Copy link
Copy Markdown
Contributor

Created a e2e CI to test the basic functionment of the Kotary operator.

This CI runs when you push a commit. It builds an image from the current files of the repository and run it in a Kind cluster. Then it uses a bash script to make an basic e2e test to verify that everything still works as intended.

The CI test the operator for versions 1.21, 23, 24, and 26 of kubernetes.

The e2e.sh script can also be drectly used if you are already connected to a kubernetes cluster. Be careful, the script run the tests ,using the version of kotary referenced in the deployment.yaml manifest. If you want to test your modifications you will have to build your own image, push it to dockerhub and modify the manifest accordingly.

@evrardjp evrardjp left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfinished review, sorry for sending it too quickly.

Comment thread .github/workflows/e2eCI.yaml Outdated
Comment thread e2e/KindConfig/badpod.yml Outdated
Comment thread e2e/KindConfig/pod1.yml Outdated
Comment thread e2e/KindConfig/pod2.yml Outdated
Comment thread e2e/KindConfig/pod3.yml Outdated
Comment thread e2e/KindConfig/pod4.yml Outdated
Comment thread e2e/KindConfig/pod5.yml Outdated

@evrardjp evrardjp left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see comments above.

Comment thread .github/workflows/e2eCI.yaml Outdated
Comment thread e2e/e2e.sh Outdated
ROOT=$(git rev-parse --show-toplevel) #get root of git repo
QUOTACLAIM='resourcequotaclaims.cagip.github.com'

#clean up function. If you want to totaly remove kotary after test, uncomment the lines

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably be a trap.

Comment thread e2e/e2e.sh
kubectl apply -f $ROOT/e2e/KotaryService/ConfigMap.yaml -n kube-system

kubectl create ns $NS
while ! kubectl get pods -n kube-system | grep kotary | grep Running > /dev/null ; do echo -e "${BLUE}.... Waiting for Kotary pod to be Running ....${NC}" ; sleep 2; done

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: You could use a different command to ensure 1/1 are running, but this is fine for me.

Comment thread e2e/e2e.sh Outdated
kubectl apply -f $ROOT/e2e/KotaryService/QuotaClaim.yaml -n $NS && sleep 3
echo -e "${GREEN} -- OK --${NC}\\n"

#echo -e "\\n ${PURPLE}-- Adding a pod with bad image --> should not impact the ressources used --${NC}" && sleep 3

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you remove this?

Comment thread e2e/e2e.sh
@@ -0,0 +1,94 @@
#!/bin/bash

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the tests should stop on first error, and throw a recap.

you should probably:

  • set -o errexit,pipefail
  • use a trap (see comment below)
  • add the "recap" of the tests who succeeded/failed in the trap.

After that (and the indentation issues), I think we can merge up this script. In the future, I think it might be more robust to use a different test framework than a self made bash script. (Either a test framework in bash, or a golang test).

@leoarpin leoarpin force-pushed the e2eCI branch 2 times, most recently from ba224df to ffe1829 Compare January 15, 2024 15:50
Features a new e2e CI based on a bash script. the script makes some
tests to verify that the operator is working as intended. It does not go
into really deep tests, this is a way to simulate how a classic user
would use kotary. Mainly used to verify that the operator is working
fine.

@evrardjp evrardjp left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few last minute things to clean up.

Comment thread e2e/KindConfig/kind-cluster-1.24.yaml Outdated
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
image: "kindest/node:v1.25.11"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

highly confusing.

Comment thread .github/workflows/e2eCI.yaml Outdated

- name: testing cluster kinD
run: |
kubectl cluster-info --context kind-chart-testing echo " current-context:" $(kubectl config current-context)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you seme to be using kind-chart-testing or chart-testing... yet:
there is no helm chart testing, and there is no chart-testing name in the config of the cluster.

Comment thread .github/workflows/e2eCI.yaml Outdated
- name: testing cluster kinD
run: |
kubectl cluster-info --context kind-chart-testing echo " current-context:" $(kubectl config current-context)
kubectl get all --all-namespaces

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not necessary.

Comment thread .github/workflows/e2eCI.yaml Outdated

- name: Docker Build image
run: |
docker build -t "ci/kotary:${{ matrix.kubernetes }}" .

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is confusing. When multiple jobs will run together, they will override each other. Let's include a sha.

Without this, 1.24 is deploying 1.25, and the CI would actually
fail.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants