New master/slave nodes are not joining the cluster #80
Description
I bought up the redis cluster with 3 masters and 3 slaves initially, everything went fine and cluster is formed.
root@rkota-k8s-1:~/redis-operator/helm-charts# kubectl get pods -A -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
...
redis-operator redis-master-0 2/2 Running 0 69m 10.244.1.9 rkota-k8s-2
redis-operator redis-master-1 2/2 Running 0 75m 10.244.1.5 rkota-k8s-2
redis-operator redis-master-2 2/2 Running 0 74m 10.244.1.7 rkota-k8s-2
redis-operator redis-operator-78c558545b-ftfl7 1/1 Running 0 81m 10.244.1.2 rkota-k8s-2
redis-operator redis-slave-0 2/2 Running 0 75m 10.244.1.3 rkota-k8s-2
redis-operator redis-slave-1 2/2 Running 0 75m 10.244.1.6 rkota-k8s-2
redis-operator redis-slave-2 2/2 Running 0 74m 10.244.1.8 rkota-k8s-2
root@rkota-k8s-1:~/redis-operator/helm-charts# kubectl exec -it redis-slave-0 -n redis-operator -- /bin/bash
Defaulted container "redis-slave" out of: redis-slave, redis-exporter
bash-4.4# redis-cli -a Opstree@1234
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> CLUSTER nodes
5d140c693a4c4bf3ba850348e6375c8815052688 10.244.1.8:6379@16379 slave 5bafacf844a073ddd83a9cff4291980374416964 0 1621930603798 3 connected
6f395a21e04a7e2a70effd0f78590547b2c20cc7 10.244.1.3:6379@16379 myself,master - 0 1621930602000 4 connected 0-5460
5bafacf844a073ddd83a9cff4291980374416964 10.244.1.7:6379@16379 master - 0 1621930603000 3 connected 10923-16383
71985a3577721d4b3c3487d65b778de99c61322d 10.244.1.5:6379@16379 master - 0 1621930603297 2 connected 5461-10922
217b0f6b58e45f0504e4ace91fd25aa73532da40 10.244.1.6:6379@16379 slave 71985a3577721d4b3c3487d65b778de99c61322d 0 1621930602000 2 connected
57b093a18faa5c4241deb7a8bcf1121e59aff3b9 10.244.1.9:6379@16379 slave 6f395a21e04a7e2a70effd0f78590547b2c20cc7 0 1621930602295 4 connected
Now i tried to upgrade the cluster size to 4 as below.
root@rkota-k8s-1:~/redis-operator/helm-charts# helm upgrade redis-cluster ./charts/redis-setup/ --set setupMode="cluster" --set cluster.size=4 --set "persistence.size=1Gi" --set "persistence.storageClass=local-storage" --install --namespace redis-operator
Release "redis-cluster" has been upgraded. Happy Helming!
NAME: redis-cluster
LAST DEPLOYED: Tue May 25 08:25:15 2021
NAMESPACE: redis-operator
STATUS: deployed
REVISION: 2
TEST SUITE: None
But i see that new master and slave has joined the cluster.
root@rkota-k8s-1:~/redis-operator/helm-charts# kubectl get pods -A -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
...
redis-operator redis-master-0 2/2 Running 0 80m 10.244.1.9 rkota-k8s-2
redis-operator redis-master-1 2/2 Running 0 2m58s 10.244.1.10 rkota-k8s-2
redis-operator redis-master-2 2/2 Running 0 86m 10.244.1.7 rkota-k8s-2
redis-operator redis-master-3 2/2 Running 0 38s 10.244.1.12 rkota-k8s-2
redis-operator redis-operator-78c558545b-ftfl7 1/1 Running 0 92m 10.244.1.2 rkota-k8s-2
redis-operator redis-slave-0 2/2 Running 0 87m 10.244.1.3 rkota-k8s-2
redis-operator redis-slave-1 2/2 Running 0 86m 10.244.1.6 rkota-k8s-2
redis-operator redis-slave-2 2/2 Running 0 86m 10.244.1.8 rkota-k8s-2
redis-operator redis-slave-3 2/2 Running 0 38s 10.244.1.11 rkota-k8s-2
root@rkota-k8s-1:~# kubectl exec -it redis-master-3 -n redis-operator -- /bin/bash
Defaulted container "redis-master" out of: redis-master, redis-exporter
bash-4.4# redis-cli -a Opstree@1234 -c
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> CLUSTER nodes
beb625801bdf1d27e2a004edbe36c1f90392dd75 10.244.1.12:6379@16379 myself,master - 0 0 0 connected