This guide covers common troubleshooting procedures for the Dash0 operator.

Create Heap Profiles

The instructions in this section are mainly meant to be used in a shared troubleshooting session with Dash0 support.

Operator Manager Heap Profile

To get a heap profile from the operator manager container:

Deploy the operator manager with the additional Helm value operator.pprofPort=1777.
Take note of the namespace the operator is deployed in (default: dash0-system).
Run kubectl get pod -n <operator-namespace> -l app.kubernetes.io/component=controller to get the name of the operator manager pod (usually something like dash0-operator-controller-xxxxxxxxx-xxxxx, but the name depends on the Helm release name).
Using the information from the previous two steps, run kubectl port-forward -n <operator-namespace> <operator-manager-pod-name> 1777.
In a separate shell, while the kubectl port-forward command from the previous step is still running, run curl http://localhost:1777/debug/pprof/heap > dash0-operator-manager-heap.out.
Terminate the kubectl port-forward command.
Redeploy the operator without the Helm setting operator.pprofPort=1777.

Collector Daemonset Heap Profile

To get a heap profile from a OpenTelemetry collector daemonset container:

Deploy the operator manager with the additional Helm value operator.collectors.enablePprof=true. Note that this will restart the collector pods.
Take note of the namespace the operator is deployed in (default: dash0-system).
Run kubectl top pod -n <operator-namespace> -l app.kubernetes.io/component=agent-collector to get the name of a collector pod that has high memory usage (usually something like dash0-operator-opentelemetry-collector-agent-daemonset-xxxxx, but the name depends on the Helm release name).
Using the information from the previous two steps, run kubectl port-forward -n <operator-namespace> <collector-daemonset-pod-name> 1777.
In a separate shell, while the kubectl port-forward command from the previous step is still running, run curl http://localhost:1777/debug/pprof/heap > dash0-daemonset-collector.out.
Terminate the kubectl port-forward command.
Redeploy the operator without the Helm setting operator.collectors.enablePprof=true.

Collector Deployment Heap Profile

To get a heap profile from a OpenTelemetry collector deployment container:

Follow the same steps as for the collector daemonset, but use -l app.kubernetes.io/component=cluster-metrics-collector in step (3).

SignalControl Collector Heap Profile

To get a heap profile from the SignalControl collector (only present when SignalControl Edge is enabled):

Follow the same steps as for the collector daemonset, but use -l app.kubernetes.io/component=signal-control-collector in step (3).

Auxiliary Container Heap Profiles

The Helm value operator.collectors.enablePprof=true also enables pprof in the auxiliary containers of the collector pods. The pprof ports are as follows:

Container	Port	Present in
`opentelemetry-collector`	`1777`	daemonset, cluster metrics deployment, SignalControl collector
`configuration-reloader`	`1778`	daemonset, cluster metrics deployment, SignalControl collector
`filelog-offset-sync`	`1779`	daemonset

To get a heap profile from one of the auxiliary containers, follow the same steps as for the collector daemonset, but use the port of the container in question in steps (4) and (5), for example kubectl port-forward -n <operator-namespace> <collector-daemonset-pod-name> 1778 and curl http://localhost:1778/debug/pprof/heap > dash0-configuration-reloader.out.

Getting Help

For additional help:

Review the operator logs: kubectl logs -n dash0-system -l app.kubernetes.io/component=controller
Contact Dash0 support at support@dash0.com

Platform Specific - Platform-specific notes that may help troubleshoot issues
Configuration - Verify your configuration settings

Troubleshooting

Table of Contents