Kubernetes Not Distributing Pods Evenly Across the Nodes

Problem

Kubernetes is not distributing pods evenly among the nodes within the cluster – even after adding additional nodes.

Environment

• Platform9 Managed Kubernetes - All Versions
• Kubernetes Scheduler

Cause

The Kubernetes Scheduler (kube-scheduler) is a component deployed among the master nodes (acting as the Kubernetes control plane) responsible for scheduling and distributing pods to nodes based on certain attributes and resource allocation(s).

Resolution

1. Use Node Affinity/Anti-Affinity in order to co-locate or spread out pods based on certain requirements (e.g. SSD vs HDD).
2. For more precise scheduling, consider implementing Pod Topology Spread Constraints in order to dictate how the pods are spread among the cluster across different failure or topology domains.
3. Re-schedule/re-scale existing workloads which may still be scoped to older, less-newly added nodes. Existing workloads will not be re-scheduled upon adding additional nodes/capacity to the cluster, unless there is an issue which prompts it.