Scheduling

Anti-Affinity
By default, Kubernetes will schedule a pod onto a random node as long as the node has capacity to execute the pod based on the resource constraints. However, when you scale a deployment to 2, there is a chance where both of the Pods are scheduled onto the same Kuberntes Node. This can cause issues if the Node goes down, causing both available Pods to shutdown and need to reschedule onto another node.
One solution is to avoid scheduling the pods onto the same node and this is called ant-affinity.
Required Anti-Affinity
This example will enforce anti-affinity and not schedule any pods if the anti-affinity requirement cannot be met. For the Hello World container, add additional configuration to make sure the pods are scheduled onto different nodes.
k8s/deployment.yaml
1
apiVersion: apps/v1
2
kind: Deployment
3
metadata:
4
  name: helloworld
5
  labels:
6
    app: helloworld
7
spec:
8
  replicas: 1
9
  selector:
10
    matchLabels:
11
      app: helloworld
12
  template:
13
    metadata:
14
      labels:
15
        app: helloworld
16
    spec:
17
      containers:
18
      - name: helloworld
19
        image: gcr.io/.../helloworld
20
      # Add configuration for anti-affinity
21
      affinity:
22
        podAntiAffinity:
23
          requiredDuringSchedulingIgnoredDuringExecution:
24
          - labelSelector:
25
              matchExpressions:
26
              - key: app
27
                operator: In
28
                values:
29
                - helloworld
30
            # Prefer spreading the Pods across multiple Nodes.
31
            # There are other keys you can use, e.g., anti-affinity
32
            # across zones.
33
            topologyKey: "kubernetes.io/hostname"
Copied!
Scale out the deployment to 4 pods:
1
kubectl scale deployment helloworld --replicas=4
Copied!
List all the pods and show which node it is running on:
1
kubectl get pods -lapp=helloworld -owide
Copied!
Observe in the NODE column, that each pod is running on a different node.
But since the demo cluster only has 4 nodes, if you scale out to 5 pods, it can no longer satisfy the anti-affinity requirement, and the 5th pod will remain in the unschedulable (Pending) state:
1
kubectl scale deployment helloworld --replicas=5
Copied!
Find the pending pod:
1
kubectl get pods -lapp=helloworld --field-selector='status.phase=Pending'
Copied!
Describe it's detail:
1
POD_NAME=$(kubectl get pods -lapp=helloworld \
2
  --field-selector='status.phase=Pending' \
3
  -o jsonpath='{.items[0].metadata.name}')
4
​
5
kubectl describe pod $POD_NAME
Copied!
See Kubernetes Affinity / Anti-Affinity documentation for more information
Preferred Anti-Affinity
In the Required case, if a pod cannot ensure anti-affinity, it'll simply not run. This may not be desirable for most workload. Instead, tell Kubernetes that anti-affinity is Preferred, but if the condition cannot be met, schedule it onto a host with another pod anyways. Use preferredDuringSchedulingIgnoredDuringExecution block instead.
k8s/deployment.yaml
1
apiVersion: apps/v1
2
kind: Deployment
3
metadata:
4
  name: helloworld
5
  labels:
6
    app: helloworld
7
spec:
8
  replicas: 1
9
  selector:
10
    matchLabels:
11
      app: helloworld
12
  template:
13
    metadata:
14
      labels:
15
        app: helloworld
16
    spec:
17
      containers:
18
      - name: helloworld
19
        image: gcr.io/.../helloworld
20
      affinity:
21
        podAntiAffinity:
22
          # Use Preferred rather than Required
23
          preferredDuringSchedulingIgnoredDuringExecution:
24
          - weight: 100
25
            podAffinityTerm:
26
              labelSelector:
27
                matchExpressions:
28
                - key: app
29
                  operator: In
30
                  values:
31
                  - helloworld
32
              topologyKey: "kubernetes.io/hostname"
Copied!
Scale out to 5 pods, it can no longer satisfy the anti-affinity requirement, and the 5th pod will still be scheduled, but onto a node that already has another Hello World pod running.
1
kubectl scale deployment helloworld --replicas=5
Copied!
See that all the pods are running:
1
kubectl get pods -lapp=helloworld
Copied!
Affinity
Sometimes you may want to pin a Pod to run together, or only on certain nodes, or only in certain zones/regions. To do this, you can specify Affinity rather than Anti-Affinity.
See Kubernetes Affinity / Anti-Affinity documentation for more information
Disruption Budget
See Kubernetes Disruption Budget documentation for more information.

Internal Load Balancing

Workload Identity

Last modified 1yr ago

Copy link

Contents

Anti-Affinity

Required Anti-Affinity

Preferred Anti-Affinity

Affinity

Disruption Budget