Scheduling

Anti-Affinity

By default, Kubernetes will schedule a pod onto a random node as long as the node has capacity to execute the pod based on the resource constraints. However, when you scale a deployment to 2, there is a chance where both of the Pods are scheduled onto the same Kuberntes Node. This can cause issues if the Node goes down, causing both available Pods to shutdown and need to reschedule onto another node.
One solution is to avoid scheduling the pods onto the same node and this is called ant-affinity.

Required Anti-Affinity

This example will enforce anti-affinity and not schedule any pods if the anti-affinity requirement cannot be met. For the Hello World container, add additional configuration to make sure the pods are scheduled onto different nodes.
k8s/deployment.yaml
1
apiVersion: apps/v1
2
kind: Deployment
3
metadata:
4
name: helloworld
5
labels:
6
app: helloworld
7
spec:
8
replicas: 1
9
selector:
10
matchLabels:
11
app: helloworld
12
template:
13
metadata:
14
labels:
15
app: helloworld
16
spec:
17
containers:
18
- name: helloworld
19
image: gcr.io/.../helloworld
20
# Add configuration for anti-affinity
21
affinity:
22
podAntiAffinity:
23
requiredDuringSchedulingIgnoredDuringExecution:
24
- labelSelector:
25
matchExpressions:
26
- key: app
27
operator: In
28
values:
29
- helloworld
30
# Prefer spreading the Pods across multiple Nodes.
31
# There are other keys you can use, e.g., anti-affinity
32
# across zones.
33
topologyKey: "kubernetes.io/hostname"
Copied!
Scale out the deployment to 4 pods:
1
kubectl scale deployment helloworld --replicas=4
Copied!
List all the pods and show which node it is running on:
1
kubectl get pods -lapp=helloworld -owide
Copied!
Observe in the NODE column, that each pod is running on a different node.
But since the demo cluster only has 4 nodes, if you scale out to 5 pods, it can no longer satisfy the anti-affinity requirement, and the 5th pod will remain in the unschedulable (Pending) state:
1
kubectl scale deployment helloworld --replicas=5
Copied!
Find the pending pod:
1
kubectl get pods -lapp=helloworld --field-selector='status.phase=Pending'
Copied!
Describe it's detail:
1
POD_NAME=$(kubectl get pods -lapp=helloworld \
2
--field-selector='status.phase=Pending' \
3
-o jsonpath='{.items[0].metadata.name}')
4
5
kubectl describe pod $POD_NAME
Copied!
See Kubernetes Affinity / Anti-Affinity documentation for more information

Preferred Anti-Affinity

In the Required case, if a pod cannot ensure anti-affinity, it'll simply not run. This may not be desirable for most workload. Instead, tell Kubernetes that anti-affinity is Preferred, but if the condition cannot be met, schedule it onto a host with another pod anyways. Use preferredDuringSchedulingIgnoredDuringExecution block instead.
k8s/deployment.yaml
1
apiVersion: apps/v1
2
kind: Deployment
3
metadata:
4
name: helloworld
5
labels:
6
app: helloworld
7
spec:
8
replicas: 1
9
selector:
10
matchLabels:
11
app: helloworld
12
template:
13
metadata:
14
labels:
15
app: helloworld
16
spec:
17
containers:
18
- name: helloworld
19
image: gcr.io/.../helloworld
20
affinity:
21
podAntiAffinity:
22
# Use Preferred rather than Required
23
preferredDuringSchedulingIgnoredDuringExecution:
24
- weight: 100
25
podAffinityTerm:
26
labelSelector:
27
matchExpressions:
28
- key: app
29
operator: In
30
values:
31
- helloworld
32
topologyKey: "kubernetes.io/hostname"
Copied!
Scale out to 5 pods, it can no longer satisfy the anti-affinity requirement, and the 5th pod will still be scheduled, but onto a node that already has another Hello World pod running.
1
kubectl scale deployment helloworld --replicas=5
Copied!
See that all the pods are running:
1
kubectl get pods -lapp=helloworld
Copied!

Affinity

Sometimes you may want to pin a Pod to run together, or only on certain nodes, or only in certain zones/regions. To do this, you can specify Affinity rather than Anti-Affinity.
See Kubernetes Affinity / Anti-Affinity documentation for more information

Disruption Budget

See Kubernetes Disruption Budget documentation for more information.