Integrating Prometheus

Connecting Prometheus to your Altinity.Cloud environment

Prometheus is a popular open-source library used for event monitoring and alerting. We’ll look at how to set up an external Prometheus server and how to configure your Altintiy.Cloud environment to send data to it.

Setting up an external Prometheus server with Grafana Cloud

As an example, we’ll look at setting up an external Prometheus server with Grafana Cloud. What we’ll do here can be done with a free Grafana Cloud account, although you can upgrade to a paid account easily.

Log in to your Grafana Cloud account (if you don’t have one already, create a free one). Click the Grafana menu in the upper left corner, then select the Connections menu:

The Connections menu

Figure 1 - The Connections menu

On the Add new connection page, enter hosted prometheus as the connection type. You’ll see the tile for Hosted Prometheus metrics:

Prometheus hosted metrics

Figure 2 - Select Hosted Prometheus metrics

Click the Hosted Prometheus metrics tile. On the Hosted Prometheus metrics page, select From my local Prometheus server in section 1:

Hosted Prometheus metrics - section 1

Figure 3 - Choose a method for forwading metrics

In section 2, select Send metrics from a single Prometheus instance:

Hosted Prometheus metrics - section 2

Figure 4 - Send metrics from a single Prometheus instance

Next, create a token for the Prometheus server in section 3. First give your token a name; this example uses ch-to-prometheus-at-grafana:

Hosted Prometheus metrics - section 3

Figure 5 - Defining a token name for the hosted Prometheus server

With the token name defined, click the Create token button. You’ll see something like this:

Hosted Prometheus metrics - section 3

Figure 6 - Creating a token for the hosted Prometheus server

Grafana Cloud generates YAML that should go into the Prometheus configuration. Fortunately for us, the Altinity Cloud Manager (ACM) makes it easy to configure the connection between our Altinity.Cloud environment and the external Prometheus server. As you probably guessed, we need three values from the YAML snippet above: The url field and the username and password fields in the basic_auth section.

Before we go on, make a note of the url and username; we’ll look at those when we check our external Prometheus server. Now let’s go to the ACM and configure our environment.

Configuring an external Prometheus server

To connect your environment to an external Prometheus server, go to the list of environments in the ACM and click the vertical dots icon and select Edit from the menu:

The Edit Environment menu

Figure 7 - The Edit Environment menu

In the Environment Configuration dialog, go to the Metrics tab and fill in the details of your external Prometheus server in the External Prometheus section:

Configuring an external Prometheus server

Figure 8 - Configuring an external Prometheus server in the Metrics tab of the Environment Configuration dialog

Click OK to save the new configuration. Now the ACM will send metrics to your Prometheus server.

Our example here uses the Grafana Cloud Prometheus service, but the same procedure applies if you’re using any other Prometheus service provider.

Checking your external Prometheus server

Now that we have the ClickHouse clusters i nour Altinity.Cloud environment sending metrics to our external Prometheus server, we’ll take a look at that Prometheus server as a Grafana data source. Switch back to your Grafana Cloud account and click the Grafana menu in the upper left corner. Click on Data sources:

The Data sources menu

Figure 9 - The Data Sources menu

When the list of data sources appears, type prom in the search box. You’ll see at least one Prometheus data source:

Prometheus data sources

Figure 10 - Prometheus data sources

Let’s make sure we’ve got the Prometheus data source that our Altinity.Cloud environment is using. The one labeled default has the same URL as the server we created earlier, so click on that one. You’ll see the details of the data source:

The Prometheus data source can't be modified

Figure 11 - The Prometheus data source can’t be modified

This data source was created by Grafana automatically and can’t be edited. That’s a good sign, but let’s make completely sure this is the right data source. Scroll down to the Authentication section and check the username:

The username for the Prometheus data source

Figure 12 - The username for the Prometheus data source

As you can see, the username is 1120750, which matches the username generated when we created our Prometheus data source. So we know this is the Prometheus server connected to our Altinity.Cloud environment. Scroll down to the bottom of the panel to make sure the data source is working. Click the button. You should see a green success message:

A successful connection to the Prometheus data source

Figure 13 - A successful connection to the Prometheus data source