ShardingController - Supports pluggable shard policies #4944
Description
What is the problem you're trying to solve
NodeShard, a newly introduced API, allows for cluster sharding, distributing different shards to multiple schedulers (Volcano agent fast scheduler, Volcano batch scheduler, etc.) for scheduling. The Sharding Controller slices the cluster according to the shard policy, creates NodeShards, and assigns them to different schedulers.
However, #4777 only completes part of the work. Currently, shard policy only supports sharding based on allocation rate (pod request / node allocatable), and the code is tightly coupled, making it difficult to extend: https://github.com/volcano-sh/volcano/blob/master/pkg/controllers/sharding/sharding_manager.go
Describe the solution you'd like
We need to do the following:
- The sharding strategy based on allocation rate will be separated into plugins, transforming the existing framework into a plugin-based architecture.
- Supports more sharding strategies (such as capability, which allows the agent scheduler to use only 30% of the node resources, and warmup, which allows the agent scheduler to prioritize scheduling nodes in the warm pool).
Additional context
For the original design documentation of the Volcano agent, please refer to: https://docs.google.com/document/d/1NDTEGulhQ__sdZ_c2kN3ojtJIZie5O7_GyZyKIOLz-g/edit?usp=sharing
Original issue: #4722
Documentation Updates
- This feature requires design or user documentation changes.
- If documentation changes are required, I will ensure the relevant documents are updated and published to the Volcano official website (https://volcano.sh) via the volcano-sh/website repository.