Integrate Pyroscope with OSM
What is Pyroscope
Pyroscope is a continuous profiling tool which profiles the performance of a program written in variety of languages, GoLang included. It provides a great Web UI for reading the profiling results.
When a GoLang program exposes the web API endpoint provided by the built-in net/http/pprof
library, Pyroscope can directly pull profiling results from the given endpoint. OSM exposes the required endpoint when debug server feature is enabled. This approach allows the integration of Pyroscope with no change to the OSM codebase.
Enable OSM debug server
To enable OSM debug server at installation time, simply add --set=osm.enableDebugServer=true
argument to the osm install
command, or run kubectl patch meshconfig osm-mesh-config -n "$OSM_NAMESPACE" -p '{"spec":{"observability":{"enableDebugServer":true}}}' --type=merge
if OSM is already installed in the cluster.
Install Pyroscope
In this section, we will install Pyroscope to profile OSM control plane. Currently only osm-controller exposes pprof web endpoints.
First step is to add annotation to the osm-controller
service so that Pyroscope monitors it.
Replace $OSM_NAMESPACE
with the OSM namespace on your Kubernetes cluster. This command informs Pyroscope to profile both CPU and memory usages of the service. The pprof web endpoint is exposed on port 9092.
Next we can install Pyroscope with Helm.
This command will install a service named prof-pyroscope
in the OSM_NAMESPACE. Some configurations of Pyroscope can be found in the pyroscope-values.yaml file. You can make a copy of it on your local machine and edit the values if you want to customize the Pyroscope installation.
Check the installation by running:
By default, Pyroscope service exposes port 4040. You can visit the web UI after forwarding the port to your localhost:
This command will keep the forwarded port open. Open http://localhost:4040 in your web browser to use Pyroscope. You should be able to see osm-controller
from the Application dropdown list like what is shown below:
Uninstall Pyroscope
To uninstall Pyroscope, run the command below:
References
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.