Diagnose runtime issues
This page explains how to diagnose runtime issues. It can also be helpful to provide these details to the development team.
Squidex is built on .NET Core, which provides a tool to create dumps.
A dump is a file that contains a snapshot of the process at the time it was created and can be useful for examining the state of your application. Dumps can be used to debug your .NET application when it is difficult to attach a debugger to it such as production or CI environments. Using dumps allows you to capture the state of the problematic process and examine it without having to stop the application.
Dumps can be analyzed with tools like
  • Visual Studio
  • Visual Studio Code
  • Memory Analyzers, e.g. dotmemory
If you want to create a dump you have to execute the following steps.
  1. 1.
    Connect to your production machine, container (docker) or pod (kubernetes).
  2. 2.
    Install the .NET SDK and the necessary tools.
  3. 3.
    Install the .NET tools
  4. 4.
    Create the dump.
  5. 5.
    Download the dump to your local machine.
  6. 6.
    Optional: Upload the dump file to a network drive to make it available it others.

1. Connect to your production machine

It depends on your hosting environment how to connect:

Connect to Linux machine

Use SSH or putty for Windows.

Connect to Windows machine

Use remote desktop connection.

Connect to your docker container

If you use the official Squidex container you can use the following command to run the shell inside your container. If you have built a custom image, bash might not be installed.
1
command docker exec -it <container name> /bin/bash
Copied!

Connect to your kubernetes pod

If you use the official Squidex container you can use the following command to run the shell inside your container. If you have built a custom image, bash might not be installed.
1
kubectl exec -it <pod_name> -- /bin/bash
Copied!

2. Install .NET SDK

Usually the .NET SDK is not installed on your server and only the runtime. This is also true for the official docker image. You can find detailed installation instructions for your environment from the official documentation:
Install .NET on Windows, Linux, and macOS
docsmsft
How to install .NET SDK
If you use the official docker image, you can execute the following steps:
1
apk add wget
2
3
# Download DotnetSDK Installer
4
wget -O sdk_install.sh https://dot.net/v1/dotnet-install.sh
5
6
# Add permissions to file
7
chmod 777 sdk_install.sh
8
9
# Install sdk.
10
./sdk_install.sh -c 5.0
Copied!

3. Install the .NET tools

There are a wide range of tools that are helpful:
dotnet-counters diagnostic tool - .NET CLI
docsmsft
.NET tools
We focus on the following tools:
  1. 1.
    dotnet-dump: Creates a full dump of the process. You can analyze the memory usage, stack traces and threads using this tool or third party solutions. The resulting file is really big, usually it has around the same size as the process.
  2. 2.
    dotnet-gcdump: This global tool collects GC (Garbage Collector) dumps of live .NET processes. GC dumps are created by triggering a GC in the target process, turning on special events, and regenerating the graph of object roots from the event stream. This process allows for GC dumps to be collected while the process is running and with minimal overhead. These dumps are useful for several scenarios:
    • Comparing the number of objects on the heap at several points in time.
    • Analyzing roots of objects and memory leaks.
    • Collecting general statistics about the counts of objects on the heap.
In the following step we focus on dotnet-gcdump.
1
Go to the dotnet folder. In order to use SDK the process needs to be run directly
2
cd /root/.dotnet
3
4
# install dotnet gcdump or install dotnet-dump to get a full dump
5
./dotnet tool install --global dotnet-gcdump
6
7
# Go to the tools folder
8
cd tools
Copied!

4. Create the dump

In this following step we focus on dotnet-gcdump.
1
# Go to the tools folde
2
cd /root/.dotnet/tools
3
4
# dotnet-gcdump collect -p 1
Copied!
In docker and kubernetes there is only one .NET process running with the Process ID (pid) 1. In another environment you can use the following command to list the dotnet processes that GC dumps can be collected for:
1
dotnet-gcdump ps
Copied!

5. Download the dump to your local machine

It depends on your environment how to copy the dump file to your local machine.

How to copy with docker

1
docker cp <container_name>:<container_path> <local_path>
Copied!
for example in my case it was like this:
1
docker cp squidex:/root/.dotnet/tools/core_20211101_160516 full.dump
Copied!

How to copy with kubernetes

1
docker cp <pd_name>:<pod_path> <local_path>
Copied!
For example in my case it was like this:
1
kubectl cp squidex-123:/root/.dotnet/tools/core_20211101_160516 full.dump
Copied!