fix: should get the really leader count when scale in

[drivebyer]

Description

close #860

The operator should rely on the actual status queried from the redis cluster, not just the status it assumes. Otherwise, we might end up with outdated information, leading to downsizing even when the desired leader count is met.

when i scale in cluster from 6 shards to 3 shards, the log Redis cluster is downscaling show more than 3 times.
see:

{"level":"info","ts":"2024-04-21T14:23:18+08:00","logger":"controllers.RedisCluster","msg":"Redis cluster is downscaling...","Request.Namespace":"default","Request.Name":"redis-cluster-v1beta2","Ready.ReadyLeaderReplicas":6,"Expected.ReadyLeaderReplicas":3}
{"level":"info","ts":"2024-04-21T14:23:18+08:00","logger":"controllers.RedisCluster","msg":"Redis cluster is downscaling","Request.Namespace":"default","Request.Name":"redis-cluster-v1beta2","The times of loop":0}
{"level":"info","ts":"2024-04-21T14:24:58+08:00","logger":"controllers.RedisCluster","msg":"Redis cluster is downscaling","Request.Namespace":"default","Request.Name":"redis-cluster-v1beta2","The times of loop":1}
{"level":"info","ts":"2024-04-21T14:26:31+08:00","logger":"controllers.RedisCluster","msg":"Redis cluster is downscaling","Request.Namespace":"default","Request.Name":"redis-cluster-v1beta2","The times of loop":2}
{"level":"info","ts":"2024-04-21T14:27:51+08:00","logger":"controllers.RedisCluster","msg":"Redis cluster is downscaled... Rebalancing the cluster","Request.Namespace":"default","Request.Name":"redis-cluster-v1beta2"}
{"level":"info","ts":"2024-04-21T14:29:57+08:00","logger":"controllers.RedisCluster","msg":"Redis cluster is downscaled... Rebalancing the cluster is done","Request.Namespace":"default","Request.Name":"redis-cluster-v1beta2"}
{"level":"info","ts":"2024-04-21T14:29:58+08:00","logger":"controllers.RedisCluster","msg":"Reconciling opstree redis Cluster controller","Request.Namespace":"default","Request.Name":"redis-cluster-v1beta2"}
{"level":"info","ts":"2024-04-21T14:29:58+08:00","logger":"controllers.RedisCluster","msg":"Redis cluster is downscaling...","Request.Namespace":"default","Request.Name":"redis-cluster-v1beta2","Ready.ReadyLeaderReplicas":6,"Expected.ReadyLeaderReplicas":3}
{"level":"info","ts":"2024-04-21T14:29:58+08:00","logger":"controllers.RedisCluster","msg":"Redis cluster is downscaling","Request.Namespace":"default","Request.Name":"redis-cluster-v1beta2","The times of loop":0}

Type of change

• Bug fix (non-breaking change which fixes an issue)

Checklist

• Tests have been added/modified and all tests pass.
• Functionality/bugs have been confirmed to be unchanged or fixed.
• I have performed a self-review of my own code.

Additional Context

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 5 lines in your changes are missing coverage. Please review.

Project coverage is 39.57%. Comparing base (d121d86) to head (59ad503).
Report is 29 commits behind head on master.

Files	Patch %	Lines
controllers/rediscluster_controller.go	0.00%	4 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #885      +/-   ##
==========================================
+ Coverage   35.20%   39.57%   +4.37%     
==========================================
  Files          19       19              
  Lines        3213     2656     -557     
==========================================
- Hits         1131     1051      -80     
+ Misses       2015     1536     -479     
- Partials       67       69       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.