External Load Balancing
This section continues from the previous section - make sure you do the tutorial in sequence.
There are primarily 2 ways to expose a Kubernetes Service on the public internet:
Type | Protocol | Locality | When to use? |
TCP/UDP | Regional | Non-HTTP requests, or no need for a global load balancer. Connection to the Load Balancer is routed by public Internet to region of the load balancer. | |
HTTP(s) | Global | HTTP requests. GCP's L7 Load Balancer is a global load balancer - a single IP address can automatically route traffic to the nearest region within the GCP network. |
To create an external network load balancer, simply change Kubernetes Service's type from
clusterip
to loadbalancer
. Modify the k8s/service.yaml
:k8s/service.yaml
apiVersion: v1
kind: Service
metadata:
name: helloworld
labels:
app: helloworld
annotations:
cloud.google.com/neg: '{"exposed_ports": {"8080":{}}}'
spec:
ports:
- name: 8080-8080
port: 8080
protocol: TCP
targetPort: 8080
selector:
app: helloworld
# Use LoadBalancer type instead of ClusterIP
type: LoadBalancer
Use
kubectl
command line to deploy the YAML file:kubectl apply -f k8s/service.yaml
To verify the application is deployed, run :
kubectl get svc helloworld
You should see that the Service has a Cluster IP address, but also the External IP address with the initial value of
<pending>
. This is because, behind the scenes, Kubernetes Engine is provisioning a Google Cloud Network Load Balancer.NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
helloworld ClusterIP ... <pending> 8080/TCP 7s
Continuously check the External IP address, until an IP address is assigned. Once the IP Address is assigned, you can connect to the External IP address, and it'll be load balanced to the
helloworld
service backend pods.EXTERNAL_IP=$(kubectl get svc helloworld -ojsonpath="{.status.loadBalancer.ingress[0].ip}")
curl $EXTERNAL_IP:8080
You can assign a static IP address to the Network Load Balancer.
Reserve a regional static IP address:
REGION=$(gcloud config get-value compute/region)
gcloud compute addresses create helloworld-service-ip --region=${REGION}
See the reserved IP address:
REGION=$(gcloud config get-value compute/region)
gcloud compute addresses describe helloworld-service-ip --region=${REGION} --format='value(address)'
Update the
k8s/service.yaml
to pin the Load Balancer IP address:k8s/service.yaml
apiVersion: v1
kind: Service
metadata:
name: helloworld
labels:
app: helloworld
spec:
ports:
- name: 8080-8080
port: 8080
protocol: TCP
targetPort: 8080
selector:
app: helloworld
type: LoadBalancer
# Replace the value with the IP address you reserved
loadBalancerIP: RESERVED_IP_ADDRESS
You can configure an external HTTP load balancer using Kubernetes Ingress. In order for the HTTP Load Balancer to find the backends, it's recommended to use container-native load balancing on Google Cloud.
In the
k8s/service.yaml
, use the cloud.google.com/neg
annotation to enable Network Endpoint Group (NEG) in order to use container-native load balancing:k8s/service.yaml
apiVersion: v1
kind: Service
metadata:
name: helloworld
labels:
app: helloworld
# Add the NEG annotation to enable Network Endpoint Group
# in order to use container-native load balancing
annotations:
cloud.google.com/neg: '{"ingress": true}'
spec:
ports:
- name: 8080-8080
port: 8080
protocol: TCP
targetPort: 8080
selector:
app: helloworld
type: ClusterIP
Create a Kubernetes Ingress configuration that will create the HTTP Load Balancer. Create a
k8s/ingress.yaml
:k8s/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: helloworld
spec:
rules:
- http:
paths:
- path: /*
backend:
service:
name: helloworld
post:
number: 8080
Use
kubectl
command line to deploy the YAML files:# Delete the existing service because it may contain a node port:
kubectl delete -f k8s/service.yaml
# Redeploy the service
kubectl apply -f k8s/service.yaml
# Deploy the ingress
kubectl apply -f k8s/ingress.yaml
To verify the Ingress is deployed:
kubectl get ingress helloworld
You should see that the Ingress has an IP address provisioned:
NAME HOSTS ADDRESS PORTS AGE
helloworld * ... 80 81s
Many Google Cloud components are being configured behind the scenes to enable global load balancing. It'll take a few minutes before the address is accessible. Use
kubectl describe
to see the current status:kubectl describe ingress helloworld
Initially, you may see:
Name: helloworld
Namespace: default
Address: ...
Default backend: helloworld:8080 (...)
Rules:
Host Path Backends
---- ---- --------
* * helloworld:8080 (...)
Annotations:
...
ingress.kubernetes.io/backends: {"...":"Unknown"}
When the annotation value of
ingress.kubernetes.io/backends
is Unknown
, it means that the backend is not yet accessible.Re-check the status until the backend becomes
HEALTHY
.Name: helloworld
Namespace: default
Address: ...
Default backend: helloworld:8080 (...)
Rules:
Host Path Backends
---- ---- --------
* * helloworld:8080 (...)
Annotations:
...
ingress.kubernetes.io/backends: {"...":"HEALTHY"}
You can then use the IP address to connect:
EXTERNAL_IP=$(kubectl get ingress helloworld -ojsonpath="{.status.loadBalancer.ingress[0].ip}")
curl $EXTERNAL_IP
By default, the Ingress IP address is ephemeral - it'll change if you ever delete and recreate the Ingress. You can associate the Ingress with a static IP address instead.
Reserve a global static IP address:
gcloud compute addresses create helloworld-ingress-ip --global
See the static IP address you reserved:
gcloud compute addresses describe helloworld-ingress-ip --global \
--format='value(address)'
In
k8s/ingress.yaml
, use the kubernetes.io/ingress.global-static-ip-name
annotation to specify the IP name:apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: helloworld
annotations:
kubernetes.io/ingress.global-static-ip-name: "helloworld-ingress-ip"
spec:
rules:
- http:
paths:
- path: /*
backend:
service:
name: helloworld
post:
number: 8080
Deploy the Ingress:
kubectl apply -f k8s/ingress.yaml
Continuously check the IP address to be updated. It'll take several minutes for the IP address to update:
kubectl get ingress helloworld
In order to use a SSL certificate to serve HTTPs traffic, you must use a real fully qualified domain name and configure it to point to the IP address. If you don't have a real domain, then you can use xip.io.
EXTERNAL_IP=$(kubectl get ingress helloworld -ojsonpath="{.status.loadBalancer.ingress[0].ip}")
DOMAIN="${EXTERNAL_IP}.xip.io"
curl $DOMAIN
echo $DOMAIN
You can provision the External HTTP(s) Load Balancer using Ingress with a Managed Certificate, or you can provide your own Self-Managed Certificate.
Google Cloud can automatically provision a certificate for your domain name when using the External HTTP(s) Load Balancer.
Create a new
k8s/certificate.yaml
:With xip.io
With Custom Domain
EXTERNAL_IP=$(kubectl get ingress helloworld -ojsonpath="{.status.loadBalancer.ingress[0].ip}")
DOMAIN="${EXTERNAL_IP}.xip.io"
cat << EOF > k8s/certificate.yaml
apiVersion: networking.gke.io/v1
kind: ManagedCertificate
metadata:
name: helloworld-certificate
spec:
domains:
# Replace the value with your domain name
- ${DOMAIN}
EOF
k8s/certificate.yaml
apiVersion: networking.gke.io/v1
kind: ManagedCertificate
metadata:
name: helloworld-certificate
spec:
domains:
# Replace the value with your domain name
- YOUR_DOMAIN_NAME
In
k8s/ingress.yaml
, use the networking.gke.io/managed-certificates
annotation to associate the certificate:k8s/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: helloworld
annotations:
kubernetes.io/ingress.global-static-ip-name: "helloworld-ingress-ip"
# Associate the ingress with the certificate name
networking.gke.io/managed-certificates: "helloworld-certificate"
spec:
rules:
- http:
paths:
- path: /*
backend:
service:
name: helloworld
post:
number: 8080
Deploy both files:
kubectl apply -f k8s/certificate.yaml
kubectl apply -f k8s/ingress.yaml
It may take several minutes to provision the certificate. Check the Managed Certificate status:
kubectl describe managedcertificate helloworld-certificate
Wait until the Certificate Status becomes
ACTIVE
:Name: helloworld
Namespace: default
...
Status:
Certificate Name: ...
Certificate Status: Active
...
You can then use HTTPs to connect:
Use xip.io
With Custom Domain
EXTERNAL_IP=$(kubectl get ingress helloworld -ojsonpath="{.status.loadBalancer.ingress[0].ip}")
DOMAIN="${EXTERNAL_IP}.xip.io"
curl "https://${DOMAIN}"
curl https://YOUR_DOMAIN_NAME
You can configure the Ingress to serve with your own SSL certificate. Usually you would already have a certificate/key pair.
If you don't already have one, you can provision a self-signed certificate for non-production use.
EXTERNAL_IP=$(kubectl get ingress helloworld -ojsonpath="{.status.loadBalancer.ingress[0].ip}")
DOMAIN="${EXTERNAL_IP}.xip.io"
mkdir -p cert/
# Generate a key
openssl genrsa -out cert/helloworld-tls.key 2048
# Generate a certificate signing request
openssl req -new -key cert/helloworld-tls.key \
-out cert/helloworld-tls.csr \
-subj "/CN=${DOMAIN}"
#
openssl x509 -req -days 365 -in cert/helloworld-tls.csr \
-signkey cert/helloworld-tls.key \
-out cert/helloworld-tls.crt
Create a Kubernetes Secret to hold the certificate/key pair:
kubectl create secret tls helloworld-tls \
--cert cert/helloworld-tls.crt --key cert/helloworld-tls.key \
--dry-run -oyaml > k8s/tls-secret.yaml
Update the Ingress to refer to the secret for TLS certificate/key pair:
k8s/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: helloworld
annotations:
kubernetes.io/ingress.global-static-ip-name: "helloworld-ingress-ip"
spec:
# Associate with the TLS certificate/key pair by the secret name
tls:
- secretName: helloworld-tls
rules:
- http:
paths:
- path: /*
backend:
service:
name: helloworld
post:
number: 8080
Deploy the configurations:
kubectl apply -f k8s/tls-secret.yaml
kubectl apply -f k8s/ingress.yaml
It will take several minutes for the new configuration to take effect.
You can then use HTTPs to connect. However, if you used a self-signed certificate, you will need to ignore certificate validation errors:
EXTERNAL_IP=$(kubectl get ingress helloworld -ojsonpath="{.status.loadBalancer.ingress[0].ip}")
DOMAIN="${EXTERNAL_IP}.xip.io"
curl -k "https://${DOMAIN}"
Last modified 1yr ago