Profiling

Cloud Profiler

Cloud Profiler allows you to continuously profile CPU and heap usages to help identify performance bottlenecks and critical paths in your application. It'll be able to produce flame graph on which parts of your application uses the most CPU and/or Heap.

Enable API

1
gcloud services enable cloudprofiler.googleapis.com
Copied!

CPU Time

The CPU time for a function tells you how long the CPU was busy executing instructions. It doesn't include the time the CPU was waiting or processing instructions for something else.

Wall Time

The wall-clock time for a function measures the time elapsed between entering and exiting a function. Wall-clock time includes all wait time, including that for locks and thread synchronization. If the wall-clock time is significantly longer than the CPU time, then that is an indication the code spends a lot of time waiting. This might be an indication of a resource bottleneck.

Heap

The heap consumption is the amount of memory allocated in the Java program's heap - this can help you find potential inefficiencies and memory leaks in your application.

Java Agent

Cloud Profiler works by adding a Java agent to your JVM startup argument, and the agent can communicate with the Cloud Profiler service in the Cloud. Through the Cloud Console, you can then see the collected profile data.

Agent Files

A Cloud Profiler agent can work both within Google Cloud environments using the Machine Credentials, and outside of Google Cloud environments (e.g., on-premises, and in another cloud) using a Service Account key file.
Unfortunately, the list of versions are not available on GitHub. The only way to see the list of available versions is listing the Google Cloud Storage bucket that contains all the binaries:
1
gsutil ls gs://cloud-profiler/java/cloud-profiler-*
Copied!
See Profiling Java Applications for more information.

Agent Configurations

Agent Path

To use the agent, you'll need to configure the JVM command line using the standard -agentpath , e.g.:
1
java -agentpath:/opt/cprof/profiler_java_agent.so \
2
-jar ...
Copied!
Rather than hard coding the startup command line, you can also configure it with the JAVA_TOOL_OPTIONS environmental variable:
1
JAVA_TOOL_OPTIONS="-agentpath:/opt/cprof/profiler_java_agent.so"
2
java -jar ...
Copied!

Agent Configurations

You can specify additional Agent configurations within the same agentpath argument, in the form of java -agentpath:/opt/cprof/profiler_java_agent.so=FLAG1=VALUE1,FLAG2=VALUE2.
Heap sampling is only available in Java 11 and higher. To turn on both CPU and Heap profiling for a Java 11 application:
1
java -agentpath:/opt/cprof/profiler_java_agent.so=-cprof_enable_heap_sampling=true \
2
-jar ...
Copied!
See Profiling Java applications Agent Configuration document for all the possible agent configuration flags.

Logging

By default the Cloud Profiler does not output any logs. You can turn on logging by using -logtostderrflag, and configure the log level using ‑minloglevelflag.
1
java -agentpath:/opt/cprof/profiler_java_agent.so=-logtostderr,-minloglevel=2 \
2
-jar ...
Copied!
See Profiling Java applications Agent Logging document for all the possible log levels.

Runtime Configuration

App Engine
Cloud Run
Kubernetes Engine
Compute Engine
Non-Google Cloud Environment
Follow App Engine Hello World! instructions to deploy an application to App Engine.
Cloud Profiler agent is already present in your App Engine application. However, it is not on by default. You can turn it on by using the JAVA_TOOL_OPTIONS environmental variable in an app.yaml file:
app.yaml
1
runtime: java11
2
env_variables:
3
JAVA_TOOL_OPTIONS: "-agentpath:/opt/cprof/profiler_java_agent.so=-logtostderr,-cprof_enable_heap_sampling=true"
Copied!
Redeploy the application with the app.yaml file:
1
gcloud app deploy target/helloworld.jar \
2
--appyaml app.yaml
Copied!
It'll take a couple of minutes before Cloud Profiler can display the information. In Cloud Profiler console, you can find the Default service in the drop down:
Add the Cloud Profiler Java agent to the container, and configure the agent in the startup command line.

Clone

1
# Clone the sample repository manually
2
git clone https://github.com/saturnism/jvm-helloworld-by-example
3
cd jvm-helloworld-by-example/helloworld-springboot-tomcat
Copied!

Containerize with a Dockerfile

In the Dockerfile, download the Cloud Debugger and build it as part of the container image:
Dockerfile
1
FROM openjdk:11
2
3
# Create a directory for the Profiler. Add and unzip the agent in the directory.
4
RUN mkdir -p /opt/cprof && \
5
wget -q -O- https://storage.googleapis.com/cloud-profiler/java/latest/profiler_java_agent.tar.gz \
6
| tar xzv -C /opt/cprof
7
8
COPY target/helloworld.jar /app.jar
9
10
ENTRYPOINT ["java", "-jar", "/app.jar"]
Copied!
Then build and push the container:
1
mvn package
2
3
PROJECT_ID=$(gcloud config get-value project)
4
docker build -t gcr.io/${PROJECT_ID}/helloworld .
5
docker push gcr.io/${PROJECT_ID}/helloworld
Copied!

Containerize with Jib

Download the Cloud Debugger Java agent into src/main/jib directory so that Jib can include the agent files as part of the container image:
1
# Make a directory to store the Java agent
2
mkdir -p src/main/jib/opt/cprof
3
4
# Download and extract the Java agent to the directory
5
wget -qO- https://storage.googleapis.com/cloud-profiler/java/latest/profiler_java_agent.tar.gz | \
6
tar xvz -C src/main/jib/opt/cprof
Copied!
Create the container image with Jib:
1
PROJECT_ID=$(gcloud config get-value project)
2
mvn compile com.google.cloud.tools:jib-maven-plugin:2.4.0:build \
3
-Dimage=gcr.io/${PROJECT_ID}/helloworld
Copied!

Deploy

Deploy to Cloud Run with Debugger Enabled using the environmental variable:
1
gcloud run deploy helloworld \
2
--region=us-central1 \
3
--platform=managed \
4
--allow-unauthenticated \
5
--set-env-vars="JAVA_TOOL_OPTIONS=-agentpath:/opt/cprof/profiler_java_agent.so=-logtostderr,-cprof_enable_heap_sampling=true" \
6
--image=gcr.io/${PROJECT_ID}/helloworld
Copied!
In Cloud Profiler console, you can see the helloworld service in the drop down:
Add the Cloud Profiler Java agent to the container, and configure the agent in the startup command line.

Clone

1
# Clone the sample repository manually
2
git clone https://github.com/saturnism/jvm-helloworld-by-example
3
cd jvm-helloworld-by-example/helloworld-springboot-tomcat
Copied!

Containerize with a Dockerfile

In the Dockerfile, download the Cloud Debugger and build it as part of the container image:
Dockerfile
1
FROM openjdk:11
2
3
# Create a directory for the Profiler. Add and unzip the agent in the directory.
4
RUN mkdir -p /opt/cprof && \
5
wget -q -O- https://storage.googleapis.com/cloud-profiler/java/latest/profiler_java_agent.tar.gz \
6
| tar xzv -C /opt/cprof
7
8
COPY target/helloworld.jar /app.jar
9
10
ENTRYPOINT ["java", "-jar", "/app.jar"]
Copied!
Then build and push the container:
1
mvn package
2
3
PROJECT_ID=$(gcloud config get-value project)
4
docker build -t gcr.io/${PROJECT_ID}/helloworld .
5
docker push gcr.io/${PROJECT_ID}/helloworld
Copied!

Containerize with Jib

Download the Cloud Debugger Java agent into src/main/jib directory so that Jib can include the agent files as part of the container image:
1
# Make a directory to store the Java agent
2
mkdir -p src/main/jib/opt/cprof
3
4
# Download and extract the Java agent to the directory
5
wget -qO- https://storage.googleapis.com/cloud-profiler/java/latest/profiler_java_agent.tar.gz | \
6
tar xvz -C src/main/jib/opt/cprof
Copied!
Create the container image with Jib:
1
PROJECT_ID=$(gcloud config get-value project)
2
mvn compile com.google.cloud.tools:jib-maven-plugin:2.4.0:build \
3
-Dimage=gcr.io/${PROJECT_ID}/helloworld
Copied!

Deploy

Deploy to Kubernetes Engine with Debugger Enabled using the environmental variable using a Deployment YAML:
1
# Make a directory to store Kubernetes YAMLs
2
mkdir k8s/
Copied!
Create a Deployment YAML file and configure the environmental variable:
k8s/deployment.yaml
1
apiVersion: apps/v1
2
kind: Deployment
3
metadata:
4
labels:
5
app: helloworld
6
name: helloworld
7
spec:
8
replicas: 1
9
selector:
10
matchLabels:
11
app: helloworld
12
template:
13
metadata:
14
labels:
15
app: helloworld
16
spec:
17
containers:
18
- image: gcr.io/YOUR_PROJECT_ID/helloworld
19
name: helloworld
20
env:
21
- name: JAVA_TOOL_OPTIONS
22
value: "-agentpath:/opt/cprof/profiler_java_agent.so=-logtostderr,-cprof_enable_heap_sampling=true,-cprof_service=helloworld-gke,-cprof_service_version=1.0"
Copied!
Deploy the YAML file:
1
kubectl apply -f k8s/deployment.yaml
Copied!
In Cloud Debugger console, you can see the helloworld-gke service in the drop down:
Follow the Compute Engine Hello World! to deploy an application in Compute Engine.
SSH into the Compute Engine instance:
1
gcloud compute ssh helloworld
Copied!
From the Compute Engine instance, download the Java agent:
1
sudo mkdir -p /opt/cprof
2
curl -s -o- https://storage.googleapis.com/cloud-profiler/java/latest/profiler_java_agent.tar.gz \
3
| sudo tar xvz -C /opt/cprof
Copied!
Run the Java application with the Cloud Debugger agent:
1
java -agentpath:/opt/cprof/profiler_java_agent.so=-logtostderr,-cprof_enable_heap_sampling=true,-cprof_service=helloworld-gce,-cprof_service_version=1.0 \
2
-jar helloworld.jar
Copied!
In Cloud Debugger console, you can see the helloworld-gce service in the drop down:
You can attach the Cloud Debugger agent to any Java application even if it runs outside of the Google Cloud environment (whether it's in a container, or on your local laptop, or in another cloud). Authentication has to be done using Service Account key file rather than using the Machine Credentials.
This only works on a Linux x86 based system.

Clone

1
# Clone the sample repository manually
2
git clone https://github.com/saturnism/jvm-helloworld-by-example
3
cd jvm-helloworld-by-example/helloworld-springboot-tomcat
Copied!

Build

1
mvn package
Copied!

Download Agent

1
sudo mkdir -p /opt/cprof
2
curl -s -o- https://storage.googleapis.com/cloud-profiler/java/latest/profiler_java_agent.tar.gz \
3
| sudo tar xvz -C /opt/cprof
Copied!

Create a Service Account

1
PROJECT_ID=$(gcloud config get-value project)
2
gcloud iam service-accounts create helloworld-app
Copied!

Add IAM Permission

1
PROJECT_ID=$(gcloud config get-value project)
2
gcloud projects add-iam-policy-binding ${PROJECT_ID} \
3
--member serviceAccount:[email protected]${PROJECT_ID}.iam.gserviceaccount.com \
4
--role roles/cloudprofiler.agent
Copied!

Create a Service Account Key File

1
PROJECT_ID=$(gcloud config get-value project)
2
gcloud iam service-accounts keys create \
3
$HOME/helloworld-app-sa.json \
4
--iam-account [email protected]${PROJECT_ID}.iam.gserviceaccount.com
Copied!

Use Service Account Cloud Debugger Agent

1
PROJECT_ID=$(gcloud config get-value project)
2
GOOGLE_APPLICATION_CREDENTAILS=$HOME/helloworld-app-sa.json
3
java -agentpath:/opt/cprof/profiler_java_agent.so=-logtostderr,-cprof_enable_heap_sampling=true,-cprof_service=helloworld-local,-cprof_service_version=1.0,-cprof_project_id=${PROJECT_ID} \
4
-jar target/helloworld.jar
Copied!
Last modified 1yr ago