"replication-master-service" label is not appear at redis master role pod

[takara9]

Describe your question
Could you tell me how to appropriate HA configuration of Redis operator ?

redis-operator version: 0.15.1

Additional context

We are very appreciate your project to provide good Redis operator!
We seeking Redis HA configuration to use productucion. and then, I read this #453

I am evaluating redis-replication plus redis-sentinel with your operator as HA-Redis.

our configuration in below.

mini:redis takara$ helm list -n ot-operators
NAME             	NAMESPACE   	REVISION	UPDATED                             	STATUS  	CHART                    	APP VERSION
redis-operator   	ot-operators	1       	2024-01-31 13:24:43.28988 +0900 JST 	deployed	redis-operator-0.15.9    	0.15.1     
redis-replication	ot-operators	1       	2024-01-31 16:46:10.916312 +0900 JST	deployed	redis-replication-0.15.11	0.15.1     
redis-sentinel   	ot-operators	1       	2024-01-31 16:46:24.753886 +0900 JST	deployed	redis-sentinel-0.15.12   	0.15.1   

mini:redis takara$ kubectl get po -n ot-operators -o wide
NAME                             READY   STATUS    RESTARTS   AGE     IP            NODE       NOMINATED NODE   READINESS GATES
redis-operator-c7d844dd4-lqdnq   1/1     Running   0          3h47m   10.244.0.3    minikube   <none>           <none>
redis-replication-0              1/1     Running   0          25m     10.244.0.19   minikube   <none>           <none>
redis-replication-1              1/1     Running   0          25m     10.244.0.20   minikube   <none>           <none>
redis-replication-2              1/1     Running   0          25m     10.244.0.21   minikube   <none>           <none>
redis-sentinel-sentinel-0        1/1     Running   0          23m     10.244.0.27   minikube   <none>           <none>
redis-sentinel-sentinel-1        1/1     Running   0          23m     10.244.0.26   minikube   <none>           <none>
redis-sentinel-sentinel-2        1/1     Running   0          24m     10.244.0.25   minikube   <none>           <none>

Master is redis-replication-0, Slave are redis-replication-1/2

mini:redis takara$ kubectl exec -it redis-replication-0 -n ot-operators -- redis-cli info replication |head -n 3
# Replication
role:master
connected_slaves:2

mini:redis takara$ kubectl exec -it redis-replication-1 -n ot-operators -- redis-cli info replication |head -n 3
# Replication
role:slave
master_host:10.244.0.19

mini:redis takara$ kubectl exec -it redis-replication-2 -n ot-operators -- redis-cli info replication |head -n 3
# Replication
role:slave
master_host:10.244.0.19

In my expectation to appear "replication-master-service" label at master role pod, but I can not find it at redis-replication-* pods.

mini:redis takara$ kubectl get po -n ot-operators redis-replication-0 -o=jsonpath='{.metadata.labels}'|jq -r .
{
  "app": "redis-replication",
  "app.kubernetes.io/component": "middleware",
  "app.kubernetes.io/instance": "redis-replication",
  "app.kubernetes.io/managed-by": "Helm",
  "app.kubernetes.io/name": "redis-replication",
  "app.kubernetes.io/version": "0.15.1",
  "apps.kubernetes.io/pod-index": "0",
  "controller-revision-hash": "redis-replication-b69b49756",
  "helm.sh/chart": "redis-replication-0.15.11",
  "redis_setup_type": "replication",
  "role": "replication",
  "statefulset.kubernetes.io/pod-name": "redis-replication-0"
}
mini:redis takara$ kubectl get po -n ot-operators redis-replication-1 -o=jsonpath='{.metadata.labels}'|jq -r .
{
  "app": "redis-replication",
  "app.kubernetes.io/component": "middleware",
  "app.kubernetes.io/instance": "redis-replication",
  "app.kubernetes.io/managed-by": "Helm",
  "app.kubernetes.io/name": "redis-replication",
  "app.kubernetes.io/version": "0.15.1",
  "apps.kubernetes.io/pod-index": "1",
  "controller-revision-hash": "redis-replication-b69b49756",
  "helm.sh/chart": "redis-replication-0.15.11",
  "redis_setup_type": "replication",
  "role": "replication",
  "statefulset.kubernetes.io/pod-name": "redis-replication-1"
}
mini:redis takara$ kubectl get po -n ot-operators redis-replication-2 -o=jsonpath='{.metadata.labels}'|jq -r .
{
  "app": "redis-replication",
  "app.kubernetes.io/component": "middleware",
  "app.kubernetes.io/instance": "redis-replication",
  "app.kubernetes.io/managed-by": "Helm",
  "app.kubernetes.io/name": "redis-replication",
  "app.kubernetes.io/version": "0.15.1",
  "apps.kubernetes.io/pod-index": "2",
  "controller-revision-hash": "redis-replication-b69b49756",
  "helm.sh/chart": "redis-replication-0.15.11",
  "redis_setup_type": "replication",
  "role": "replication",
  "statefulset.kubernetes.io/pod-name": "redis-replication-2"
}

sentinels seems to be working well.

mini:redis takara$ kubectl logs redis-sentinel-sentinel-0  -n ot-operators
Sentinel is running without password which is not recommended
Running sentinel without TLS mode
ACL_MODE is not true, skipping ACL file modification
Starting  sentinel service .....
7:X 31 Jan 2024 07:48:50.713 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
7:X 31 Jan 2024 07:48:50.713 * Redis version=7.2.1, bits=64, commit=00000000, modified=0, pid=7, just started
7:X 31 Jan 2024 07:48:50.713 * Configuration loaded
7:X 31 Jan 2024 07:48:50.714 * monotonic clock: POSIX clock_gettime
7:X 31 Jan 2024 07:48:50.714 # Failed to write PID file: Permission denied
7:X 31 Jan 2024 07:48:50.714 * Running mode=sentinel, port=26379.
7:X 31 Jan 2024 07:48:50.716 * Sentinel new configuration saved on disk
7:X 31 Jan 2024 07:48:50.716 * Sentinel ID is a67e793fefdc47dadc251a4cb97d24ab481f9cc6
7:X 31 Jan 2024 07:48:50.716 # +monitor master myMaster 10.244.0.19 6379 quorum 2
7:X 31 Jan 2024 07:48:50.717 * +slave slave 10.244.0.20:6379 10.244.0.20 6379 @ myMaster 10.244.0.19 6379
7:X 31 Jan 2024 07:48:50.719 * Sentinel new configuration saved on disk
7:X 31 Jan 2024 07:48:50.719 * +slave slave 10.244.0.21:6379 10.244.0.21 6379 @ myMaster 10.244.0.19 6379
7:X 31 Jan 2024 07:48:50.720 * Sentinel new configuration saved on disk
7:X 31 Jan 2024 07:48:51.020 * +sentinel sentinel 8072bc1a7b56afd0107f8d0c3d11f0fa6f5f7ace 10.244.0.26 26379 @ myMaster 10.244.0.19 6379
7:X 31 Jan 2024 07:48:51.023 * Sentinel new configuration saved on disk
7:X 31 Jan 2024 07:48:51.505 * +sentinel sentinel 81dd07d332e271c5162576b6247e9089c20148bf 10.244.0.25 26379 @ myMaster 10.244.0.19 6379
7:X 31 Jan 2024 07:48:51.507 * Sentinel new configuration saved on disk

[drivebyer]

In my expectation to appear "replication-master-service" label at master role pod, but I can not find it at redis-replication-* pods.

If you want to connect to master redis, you can seek help from sentinel. plz see: https://redis.io/docs/management/sentinel/#obtaining-the-address-of-the-current-master

[takara9]

Thank you for your answer.
Unbfortunally, Our the existing redis client does not adapt sentinel client.
Such as AWS elasticsearch and GCP redis cache service provide HA service without sentinel or replication client. If the primary redis server failed then the replica server starts using same IP address. the Redis clients no need any change. We need such kind of configuration for Redis.

[drivebyer]

Such as AWS elasticsearch and GCP redis cache service provide HA service without sentinel or replication client. If the primary redis server failed then the replica server starts using same IP address. the Redis clients no need any change. We need such kind of configuration for Redis.

Understood. Certainly, if we were to have such a feature, it would be much better.

[takara9]

Hello,
I am thinking to add new feature of redis-replication in this operator.

• Adding the label to redis-replication-* pods that indicates role "master" or "slave/replica"
• Adding new service that has new selector for write master role pod.

In this way, the redis client pod on kubernetes cluster can update to the master pod without sentinel client function. If the master pod fails down, slave/replica pod will takes over master pod. the redis-operator inspects redis role such as master or slave continously and write current role to the label.

How do you think this idea ?
If you agree to this idea, I will make PR for redis-operator.