@@ -3,7 +3,6 @@ Documentation Check delete master
33Library Collections
44Library OperatingSystem
55Library String
6- Library Process
76Resource ../Lib/lib.robot
87
98
@@ -14,55 +13,25 @@ Check Delete Master
1413
1514 ${MASTER } = Get Master Pod
1615 ${OLD_MASTER_NAME } = Set Variable ${MASTER.metadata.name }
17- ${OLD_MASTER_STS } = Get Statefulset Name From Pod Name ${OLD_MASTER_NAME }
1816
19- # insert test records before failover
17+ # insert test records before deleting master
2018 ${RID } ${EXPECTED } = Insert Test Record ${MASTER.status.pod_ip }
2119
22- Log To Console Scaling down old master StatefulSet "${OLD_MASTER_STS } "
23- Scale Statefulset ${OLD_MASTER_STS } 0
24-
2520 Log To Console Deleting Master Pod "${OLD_MASTER_NAME } "
26- Delete Pod ${OLD_MASTER_NAME } 30
27-
28- Log To Console Wait old master pod deletion
29- Wait Until Keyword Succeeds 120 sec 2 sec Pod Should Not Exist ${OLD_MASTER_NAME }
21+ Run Keyword Delete Pod ${OLD_MASTER_NAME } 30
3022
31- Log To Console Wait new master election
32- Wait Until Keyword Succeeds 180 sec 2 sec Check If New Master Elected ${OLD_MASTER_NAME }
23+ # wait until cluster is healthy again
24+ Log To Console Wait until cluster recovers after master deletion
25+ Wait Until Keyword Succeeds 300 sec 2 sec Wait Replica Pods In Up State
26+ Wait Until Keyword Succeeds 300 sec 2 sec Check Replica Count
3327
3428 ${NEW_MASTER } = Get Master Pod
35- Log To Console New Master ${NEW_MASTER.metadata.name }
36-
37- Log To Console Scaling old master StatefulSet "${OLD_MASTER_STS } " back to 1
38- Scale Statefulset ${OLD_MASTER_STS } 1
39-
40- # wait while all replicas are back
41- Wait Until Keyword Succeeds 180 sec 2 sec Check Replica Count
42- Wait Until Keyword Succeeds 180 sec 2 sec Wait Replica Pods In Up State
29+ Log To Console Current Master ${NEW_MASTER.metadata.name }
4330
44- # check new master is writable
45- Log To Console Test New Master Works
31+ # check current master is writable
32+ Log To Console Test Current Master Works
4633 Wait Until Keyword Succeeds 120 sec 1 sec Insert Test Record ${NEW_MASTER.status.pod_ip }
4734
35+ # check replication after recovery
4836 Run Keyword Check Replica Count
49- Run Keyword Replication Works
50-
51-
52- *** Keywords ***
53- Get Statefulset Name From Pod Name
54- [Arguments] ${pod_name }
55- ${sts_name } = Evaluate "${pod_name } ".rsplit("-", 1)[0]
56- RETURN ${sts_name }
57-
58- Scale Statefulset
59- [Arguments] ${sts_name } ${replicas }
60- ${result } = Run Process kubectl -n %{POD_NAMESPACE } scale statefulset ${sts_name } --replicas ${replicas } stdout=PIPE stderr=PIPE
61- Log ${result.stdout }
62- Log ${result.stderr }
63- Should Be Equal As Integers ${result.rc } 0
64-
65- Pod Should Not Exist
66- [Arguments] ${pod_name }
67- ${result } = Run Process kubectl -n %{POD_NAMESPACE } get pod ${pod_name } stdout=PIPE stderr=PIPE
68- Should Not Be Equal As Integers ${result.rc } 0
37+ Run Keyword Replication Works
0 commit comments