External Load Balancing

Search…
This section continues from the previous section - make sure you do the tutorial in sequence.
Service
There are primarily 2 ways to expose a Kubernetes Service on the public internet:
Type
Protocol
Locality
When to use?
​External Network Load Balancer​
TCP/UDP
Regional
Non-HTTP requests, or no need for a global load balancer. Connection to the Load Balancer is routed by public Internet to region of the load balancer.
​External HTTP(s) Load Balancer​
HTTP(s)
Global
HTTP requests. GCP's L7 Load Balancer is a global load balancer - a single IP address can automatically route traffic to the nearest region within the GCP network.
External Network Load Balancer
Service YAML
To create an external network load balancer, simply change Kubernetes Service's type from clusterip to loadbalancer. Modify the k8s/service.yaml:
k8s/service.yaml
1
apiVersion: v1
2
kind: Service
3
metadata:
4
  name: helloworld
5
  labels:
6
    app: helloworld
7
  annotations:
8
    cloud.google.com/neg: '{"exposed_ports": {"8080":{}}}'
9
spec:
10
  ports:
11
  - name: 8080-8080
12
    port: 8080
13
    protocol: TCP
14
    targetPort: 8080
15
  selector:
16
    app: helloworld
17
  # Use LoadBalancer type instead of ClusterIP
18
  type: LoadBalancer
Copied!
Deploy
Use kubectl command line to deploy the YAML file:
1
kubectl apply -f k8s/service.yaml
Copied!
To verify the application is deployed, run :
1
kubectl get svc helloworld
Copied!
You should see that the Service has a Cluster IP address, but also the External IP address with the initial value of <pending>. This is because, behind the scenes, Kubernetes Engine is provisioning a Google Cloud Network Load Balancer.
1
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
2
helloworld   ClusterIP   ...          <pending>     8080/TCP   7s
Copied!
Connect
Continuously check the External IP address, until an IP address is assigned. Once the IP Address is assigned, you can connect to the External IP address, and it'll be load balanced to the helloworld service backend pods.
1
EXTERNAL_IP=$(kubectl get svc helloworld -ojsonpath="{.status.loadBalancer.ingress[0].ip}")
2
curl $EXTERNAL_IP:8080
Copied!
Static IP Address
You can assign a static IP address to the Network Load Balancer.
Reserve a regional static IP address:
1
REGION=$(gcloud config get-value compute/region)
2
gcloud compute addresses create helloworld-service-ip --region=${REGION}
Copied!
See the reserved IP address:
1
REGION=$(gcloud config get-value compute/region)
2
gcloud compute addresses describe helloworld-service-ip --region=${REGION} --format='value(address)'
Copied!
Update the k8s/service.yaml to pin the Load Balancer IP address:
k8s/service.yaml
1
apiVersion: v1
2
kind: Service
3
metadata:
4
  name: helloworld
5
  labels:
6
    app: helloworld
7
spec:
8
  ports:
9
  - name: 8080-8080
10
    port: 8080
11
    protocol: TCP
12
    targetPort: 8080
13
  selector:
14
    app: helloworld
15
  type: LoadBalancer
16
  # Replace the value with the IP address you reserved
17
  loadBalancerIP: RESERVED_IP_ADDRESS
Copied!
External HTTP Load Balancer
You can configure an external HTTP load balancer using Kubernetes Ingress. In order for the HTTP Load Balancer to find the backends, it's recommended to use container-native load balancing on Google Cloud.
Service YAML
In the k8s/service.yaml, use the cloud.google.com/neg annotation to enable Network Endpoint Group (NEG) in order to use container-native load balancing:
k8s/service.yaml
1
apiVersion: v1
2
kind: Service
3
metadata:
4
  name: helloworld
5
  labels:
6
    app: helloworld
7
  # Add the NEG annotation to enable Network Endpoint Group
8
  # in order to use container-native load balancing
9
  annotations:
10
    cloud.google.com/neg: '{"ingress": true}'
11
spec:
12
  ports:
13
  - name: 8080-8080
14
    port: 8080
15
    protocol: TCP
16
    targetPort: 8080
17
  selector:
18
    app: helloworld
19
  type: ClusterIP
Copied!
Ingress YAML
Create a Kubernetes Ingress configuration that will create the HTTP Load Balancer. Create a k8s/ingress.yaml:
k8s/ingress.yaml
1
apiVersion: networking.k8s.io/v1
2
kind: Ingress
3
metadata:
4
  name: helloworld
5
spec:
6
  rules:
7
  - http:
8
      paths:
9
      - path: /*
10
        backend:
11
          service:
12
            name: helloworld
13
            post:
14
              number: 8080
Copied!
Deploy
Use kubectl command line to deploy the YAML files:
1
# Delete the existing service because it may contain a node port:
2
kubectl delete -f k8s/service.yaml
3
​
4
# Redeploy the service
5
kubectl apply -f k8s/service.yaml
6
​
7
# Deploy the ingress
8
kubectl apply -f k8s/ingress.yaml
Copied!
To verify the Ingress is deployed:
1
kubectl get ingress helloworld
Copied!
You should see that the Ingress has an IP address provisioned:
1
NAME         HOSTS   ADDRESS         PORTS   AGE
2
helloworld   *       ...             80      81s
Copied!
Many Google Cloud components are being configured behind the scenes to enable global load balancing. It'll take a few minutes before the address is accessible. Use kubectl describe to see the current status:
1
kubectl describe ingress helloworld
Copied!
Initially, you may see:
1
Name:             helloworld
2
Namespace:        default
3
Address:          ...
4
Default backend:  helloworld:8080 (...)
5
Rules:
6
  Host  Path  Backends
7
  ----  ----  --------
8
  *     *     helloworld:8080 (...)
9
Annotations:
10
  ...
11
  ingress.kubernetes.io/backends:  {"...":"Unknown"}
Copied!
When the annotation value of ingress.kubernetes.io/backends is Unknown, it means that the backend is not yet accessible.
Re-check the status until the backend becomes HEALTHY.
1
Name:             helloworld
2
Namespace:        default
3
Address:          ...
4
Default backend:  helloworld:8080 (...)
5
Rules:
6
  Host  Path  Backends
7
  ----  ----  --------
8
  *     *     helloworld:8080 (...)
9
Annotations:
10
  ...
11
  ingress.kubernetes.io/backends:  {"...":"HEALTHY"}
Copied!
Connect
You can then use the IP address to connect:
1
EXTERNAL_IP=$(kubectl get ingress helloworld -ojsonpath="{.status.loadBalancer.ingress[0].ip}")
2
curl $EXTERNAL_IP
Copied!
Static IP Address
By default, the Ingress IP address is ephemeral - it'll change if you ever delete and recreate the Ingress. You can associate the Ingress with a static IP address instead.
Global Static IP Address
Reserve a global static IP address:
1
gcloud compute addresses create helloworld-ingress-ip --global
Copied!
See the static IP address you reserved:
1
gcloud compute addresses describe helloworld-ingress-ip --global \
2
  --format='value(address)'
Copied!
Configurations
In k8s/ingress.yaml, use the kubernetes.io/ingress.global-static-ip-name annotation to specify the IP name:
1
apiVersion: networking.k8s.io/v1
2
kind: Ingress
3
metadata:
4
  name: helloworld
5
  annotations:
6
    kubernetes.io/ingress.global-static-ip-name: "helloworld-ingress-ip"
7
spec:
8
  rules:
9
  - http:
10
      paths:
11
      - path: /*
12
        backend:
13
          service:
14
            name: helloworld
15
            post:
16
              number: 8080
Copied!
Deploy
Deploy the Ingress:
1
kubectl apply -f k8s/ingress.yaml
Copied!
Continuously check the IP address to be updated. It'll take several minutes for the IP address to update:
1
kubectl get ingress helloworld
Copied!
SSL Certificate
In order to use a SSL certificate to serve HTTPs traffic, you must use a real fully qualified domain name and configure it to point to the IP address. If you don't have a real domain, then you can use xip.io.
1
EXTERNAL_IP=$(kubectl get ingress helloworld -ojsonpath="{.status.loadBalancer.ingress[0].ip}")
2
DOMAIN="${EXTERNAL_IP}.xip.io"
3
curl $DOMAIN
4
​
5
echo $DOMAIN
Copied!
You can provision the External HTTP(s) Load Balancer using Ingress with a Managed Certificate, or you can provide your own Self-Managed Certificate.
Managed Certificate
Google Cloud can automatically provision a certificate for your domain name when using the External HTTP(s) Load Balancer.
Create a new k8s/certificate.yaml:
With xip.io
With Custom Domain
1
EXTERNAL_IP=$(kubectl get ingress helloworld -ojsonpath="{.status.loadBalancer.ingress[0].ip}")
2
DOMAIN="${EXTERNAL_IP}.xip.io"
3
​
4
cat << EOF > k8s/certificate.yaml
5
apiVersion: networking.gke.io/v1
6
kind: ManagedCertificate
7
metadata:
8
  name: helloworld-certificate
9
spec:
10
  domains:
11
  # Replace the value with your domain name
12
  - ${DOMAIN}
13
EOF
Copied!
k8s/certificate.yaml
1
apiVersion: networking.gke.io/v1
2
kind: ManagedCertificate
3
metadata:
4
  name: helloworld-certificate
5
spec:
6
  domains:
7
  # Replace the value with your domain name
8
  - YOUR_DOMAIN_NAME
Copied!
In k8s/ingress.yaml, use the networking.gke.io/managed-certificates annotation to associate the certificate:
k8s/ingress.yaml
1
apiVersion: networking.k8s.io/v1
2
kind: Ingress
3
metadata:
4
  name: helloworld
5
  annotations:
6
    kubernetes.io/ingress.global-static-ip-name: "helloworld-ingress-ip"
7
    # Associate the ingress with the certificate name
8
    networking.gke.io/managed-certificates: "helloworld-certificate"
9
spec:
10
  rules:
11
  - http:
12
      paths:
13
      - path: /*
14
        backend:
15
          service:
16
            name: helloworld
17
            post:
18
              number: 8080
Copied!
Deploy both files:
1
kubectl apply -f k8s/certificate.yaml
2
kubectl apply -f k8s/ingress.yaml
Copied!
It may take several minutes to provision the certificate. Check the Managed Certificate status:
1
kubectl describe managedcertificate helloworld-certificate
Copied!
Wait until the Certificate Status becomes ACTIVE:
1
Name:         helloworld
2
Namespace:    default
3
...
4
Status:
5
  Certificate Name:    ...
6
  Certificate Status:  Active
7
...
Copied!
You can then use HTTPs to connect:
Use xip.io
With Custom Domain
1
EXTERNAL_IP=$(kubectl get ingress helloworld -ojsonpath="{.status.loadBalancer.ingress[0].ip}")
2
DOMAIN="${EXTERNAL_IP}.xip.io"
3
​
4
curl "https://${DOMAIN}"
Copied!
1
curl https://YOUR_DOMAIN_NAME
Copied!
See Using Google-managed SSL certificates for more details.
Self-Managed Certificate
You can configure the Ingress to serve with your own SSL certificate. Usually you would already have a certificate/key pair.
If you don't already have one, you can provision a self-signed certificate for non-production use.
1
EXTERNAL_IP=$(kubectl get ingress helloworld -ojsonpath="{.status.loadBalancer.ingress[0].ip}")
2
DOMAIN="${EXTERNAL_IP}.xip.io"
3
​
4
mkdir -p cert/
5
​
6
# Generate a key
7
openssl genrsa -out cert/helloworld-tls.key 2048
8
​
9
# Generate a certificate signing request
10
openssl req -new -key cert/helloworld-tls.key \
11
  -out cert/helloworld-tls.csr \
12
  -subj "/CN=${DOMAIN}"
13
​
14
# 
15
openssl x509 -req -days 365 -in cert/helloworld-tls.csr \
16
  -signkey cert/helloworld-tls.key \
17
  -out cert/helloworld-tls.crt
Copied!
Create a Kubernetes Secret to hold the certificate/key pair:
1
kubectl create secret tls helloworld-tls \
2
  --cert cert/helloworld-tls.crt --key cert/helloworld-tls.key \
3
  --dry-run -oyaml > k8s/tls-secret.yaml
Copied!
Update the Ingress to refer to the secret for TLS certificate/key pair:
k8s/ingress.yaml
1
apiVersion: networking.k8s.io/v1
2
kind: Ingress
3
metadata:
4
  name: helloworld
5
  annotations:
6
    kubernetes.io/ingress.global-static-ip-name: "helloworld-ingress-ip"
7
spec:
8
  # Associate with the TLS certificate/key pair by the secret name
9
  tls:
10
  - secretName: helloworld-tls
11
  rules:
12
  - http:
13
      paths:
14
      - path: /*
15
        backend:
16
          service:
17
            name: helloworld
18
            post:
19
              number: 8080
Copied!
Deploy the configurations:
1
kubectl apply -f k8s/tls-secret.yaml
2
kubectl apply -f k8s/ingress.yaml
Copied!
It will take several minutes for the new configuration to take effect.
You can then use HTTPs to connect. However, if you used a self-signed certificate, you will need to ignore certificate validation errors:
1
EXTERNAL_IP=$(kubectl get ingress helloworld -ojsonpath="{.status.loadBalancer.ingress[0].ip}")
2
DOMAIN="${EXTERNAL_IP}.xip.io"
3
​
4
curl -k "https://${DOMAIN}"
Copied!
See Using multiple SSL certificates in HTTP(s) load balancing with Ingress for more details.
Load Balancing
Internal Load Balancing
Last modified 10mo ago
Copy link
Contents
External Network Load Balancer
External HTTP Load Balancer