Working with Environments

An Environment is a way of grouping ClickHouse® clusters together in your account. There are several things you’ll want to do with your environments; we’ve broken the details into several sections:

• The environments view
• Setting up an environment
• Configuring an environment
  There’s a lot of material in this section, so we suggest you go directly to the subsection that’s right for you:
  • General configuration
  • Kubernetes configuration
  • Node defaults
  • Resource limits
  • Backups
  • Metrics
  • Logs
• Other configuration tasks
• Deleting an environment

The environments view

In working with Environments, you’ll always start by selecting the Environments tab to load the list of Environments:

Figure 1 - The Environments tab

You’ll see the Environment list view:

Figure 2 - The list of Altinity.Cloud Environments

The columns in the table are:

• Select button - Lets you select an Environment for operations and includes the vertical dots icon, which includes options to Edit, Export, or Delete the environment. (More on those tasks later.)
• Environment number - Each Environment has a unique number. If you’re using the Altinity Cloud Manager API, you will need this number when making certain API calls.
• Environment Name - Um, the name of the environment.
• Status - The status of the cluster. is what we’re looking for here.
• Deployment Type - The kind of deployment, either Kubernetes (Altinity.Cloud) or Anywhere (Altinity.Cloud Anywhere).
• Cloud Provider - Either AWS, GCP, or Azure.
• Created - The time the Environment was created.
• Owner - The ID of the Environment’s administrator.
• Organization - The Organization that owns this Environment.

• The button - Clicking this button takes you to the Clusters (plural) view of the ClickHouse clusters in this environment.

Setting up an Environment

The first step in setting up an environment is clicking the button. In the Environment Setup dialog, click a radio button to select the type of environment you need:

Figure 3 - The list of environment types

The environment types are:

Bring Your Own Kubernetes (BYOK) - Altinity.Cloud runs in your Kubernetes environment. See the Bring Your Own Kubernetes documentation for complete details.

Bring Your Own Cloud (BYOC) - Altinity.Cloud provisions Kubernetes clusters in your cloud account, then provision ClickHouse clusters inside those Kubernetes clusters. When you select this option, you’ll see radio buttons that let you choose your cloud provider:

Figure 4 - BYOC environment - Choose your cloud provider

See the Bring Your Own Cloud documentation for complete details.

SaaS Provisioned by Altinity - ClickHouse as a service, hosted and managed by Altinity. When you select this option, you’ll see radio buttons and a dropdown list that let you choose a cloud provider and region for your new environment:

Figure 5 - SaaS environment - Choose a cloud provider and region

Manual Setup - This option is only available to Altinity administrators.

Choose an environment type and give your new environment a name. Depending on the environment type you choose, you may need to specify a cloud provider or region.

Click OK to create the environment.

Configuring an Environment

To configure an environment, click the vertical dots icon and select Edit from the menu:

Figure 6 - The Edit Environment menu

This takes you to the Environment Configuration dialog. The dialog has seven tabs:

• General
• Kubernetes
• Node Defaults
• Resource Limits
• Backups
• Metrics
• Logs

We’ll cover each of these next.

The General tab

The most high-level settings are on the General tab:

Figure 7 - The General tab

Field details

Name

The name of the Environment.

Deployment Type

This field is read-only for most users. Under normal circumstances, the only values that will appear here are Kubernetes or Anywhere.

Availability Zones

Enter the availability zones for your cloud provider.

Use External DNS

The names of the endpoints of ClickHouse resources used by ACM are based on a naming convention that includes the cluster name and altinity.cloud. Checking this box means the ACM will access those resources through endpoints resolved through an external DNS. Contact Altinity support for help in setting this up.

Domain

You can define a custom domain; contact Altinity support to set this up. The default domain uses a combination of your ACM environment name and altinity.cloud.

Database Login
Database Password

The username and password for the ClickHouse cluster. The ACM uses these values to retrieve information about schemas, users, and other high-level information from the ClickHouse cluster. These values are set for you by default.

If you change the username to default and leave the password blank, the ACM will attempt to retrieve the password from the ClickHouse operator’s configuration.

Automatic Publish

If selected, any changes made to the Environment’s configuration are automatically published when you make them. Turning this off lets you decide when a change is published, giving you more control over the configuration. For example, you might want to make a number of configuration changes and publish them at one time.

Monitoring

Monitoring is turned on and the button is disabled. This control is a legacy item that will disappear in future versions of the ACM.

The Kubernetes tab

The Kubernetes tab, as you would expect, lets you configure Kubernetes settings:

Figure 8 - The Kubernetes tab

Field details

Master API URL

Under normal circumstances this value should not be changed. It can be useful in a Bring Your Own Kubernetes environment when defining connections between the ACM and Kubernetes. Contact Altinity support for help setting this up.

Auth Options - TLS Handshake + Token

This authentication scheme is the most common:

Figure 9 - Using the TLS Handshake and Token authentication scheme

The Client Key and Client Cert are defined for you. Under normal circumstances, you won’t need to change these values at all.

Auth Options - No Auth (Proxy)

This approach is used for development purposes only. It should never be used in production.

Figure 10 - Using the No Auth authentication scheme

Auth Options - Access Token

This is a legacy technique that is no longer used. It will be removed in a future version of the ACM.

Figure 11 - Using the Access Token authentication scheme

Namespace

The namespace used for the ClickHouse clusters deployed by Altinity. The default value is altinity-cloud-managed-clickhouse. Once the ClickHouse cluster is created, the namespace cannot be changed.

Manage Namespace

If this switch is turned on, namespaces will be managed by the ACM.

Certificate ARN

The ARN (Amazon Resource Name) for the certificate used by this environment. This is managed for you in Altinity.Cloud environments, and is typically not needed in an Altinity.Cloud Anywhere environment.

Load Balancer Type

The default value (and likely the only value available) is Altinity Edge Ingress. If your cloud provider is AWS, AWS NLB may be an option as well. In some cases, you’ll see checkboxes that let you enable the load balancer for public and/or private traffic. If you need to use a different load balancer, contact Altinity support.

Node Scope Label

Defines a label that will be added to nodes created by the ACM. This is useful for Kubernetes clusters that have user-created nodes. With a node scope label, the ACM will only deploy pods to nodes with that label.

Subnets CIDR

This is used in Bring Your Own Cloud (BYOC) environments. You define subnets in your cloud account, then reference them when you use the ACM to create a Kubernetes cluster in your cloud account. The subnets you provided when you defined the environment appear here.

K8S Dashboard URL

You have the option of installing a Kubernetes monitoring tool inside the Kubernetes cluster that hosts your ClickHouse clusters. If you do, enter the URL of the monitoring tool here. Most customers use tools that run outside the Kubernetes cluster (k9s, for example), so this option is rarely used.

Server Startup Mode

The startup modes affect how Altinity Cloud Manager starts a ClickHouse cluster when you change its configuration. The three modes are:

• Default mode
• Direct mode - If something goes wrong during startup, the ACM will retry the startup several times before giving up.
• Troubleshooting mode - If something goes wrong during startup, the ACM will not try to restart it if it fails. This is useful for debugging any problems with your cluster’s updated configuration.

Server Startup Time

Depending on the number of tables and the amount of data in your ClickHouse cluster, it may take longer than normal to start. That means it’s possible that the Kubernetes cluster hosting your ClickHouse cluster will delete and restart the pods needed to run ClickHouse before ClickHouse can start. For that reason, you can define a startup time, which is the number of seconds the Kubernetes cluster should wait for your ClickHouse cluster to start.

SNI Proxy

In an Altinity.Cloud Anywhere environment, the endpoints the ACM needs to access may not be publicly accessible. This field lets you define an SNI proxy to route requests from the ACM to the correct endpoint inside your Altinity.Cloud Anywhere environment. (Server Name Indication is an extension to TLS that allows multiple hostnames to be served over a single HTTP endpoint.)

Use SNI Proxy to access ClickHouse

Turns the SNI proxy on or off.

Use Operator Managed PVs

If selected, the persistent volumes used by your ClickHouse clusters will be managed by the Altinity Kubernetes Operator. This is the default; only legacy Altinity.Cloud environments should disable this option.

The Node Defaults tab

This tab lets you define the default ClickHouse version, default node types for ClickHouse and Zookeeper nodes, and storage settings. In addition, you can define the URL of a private image registry if you want your Kubernetes cluster to use a custom registry.

Figure 12 - The Node Defaults tab

Field details

ClickHouse Version

Click the down arrow icon to see a list of available ClickHouse versions.

ClickHouse Node Type

Click the down arrow icon to see a list of available node types.

ClickHouse Storage Multiplier

Defines the amount of storage that will be allocated to your ClickHouse cluster. The ACM takes the amount of RAM allocated to the cluster (in GB), multiplies it by the storage multiplier, then rounds up to the nearest 100GB. For the mathematically inclined, the formula’s notation is , where x is the RAM in GB and y is the storage multiplier.

Here are a couple of examples:

• With 8GB of RAM and a storage multiplier of 10, the amount of storage is 100GB (8 * 10 is 80, rounded up to 100).
• With 16GB of RAM and a storage multiplier of 10, the amount of storage is 200GB (16 * 10 is 160, rounded up to 200).

All that being said, it’s unlikely you’ll ever need to change the default value of 10.

Zookeeper Node Type

Click the down arrow icon to see a list of available node types.

Zookeeper Storage (GB)

Defines the storage that should be allocated for each Zookeeper node.

Private Image Registry

Enter the URL of your private image registry in the format https://username:password@imageurl.com. If a private registry is defined, the ACM will pull container images for the ClickHouse server, Zookeeper, and the ClickHouse Backup tool from the private registry.

The Resource Limits tab

Figure 13 - The Resource Limits tab

Field details

Shards per Cluster

Enter the maximum number of shards per cluster.

Replicas per Cluster

Enter the maximum number of replicas per cluster.

Total Storage, GiB

Enter, in gigabytes, the total amount of storage used by the ClickHouse cluster.

Total CPU Cores

Enter the maximum number of cores across all CPUs.

Volume Rescale Percent

Enter the minimum percentage of space that must be used on a given disk before it can be rescaled.

Minimum Volume Size, GiB

Enter, in gigabytes, the minimum volume size for a multi-volume configuration.

Minimum st1 Volume Size, GiB

Enter, in gigabytes, the minimum volume size for EBS st1 volumes. If your cloud provider is not AWS, this field is disabled.

Total Nodes

Enter the maximum number of nodes for your ClickHouse cluster. A value of 0 means there is no limit.

The Backups tab

Figure 14 - The Backups tab

Field details

Turn On Backups

Select this to, um, turn on backups. If not selected, no backups of your data will be made.

Backup Schedule

Allows you to define a schedule of backups. See the Configuring backups documentation in the User Guide for complete details.

Backup Tool Image

The name and tag of the container image used to create and manage backups. If left blank, the default is currently altinity/clickhouse-backup:2.4.34.

Compression Format

The default is tar; other options are gzip and zstd. Be aware that creating a tar file has the lowest impact on the CPU, but it creates the largest file because a tar file isn’t compressed. On the other hand, the other compression formats take more CPU cycles to create but have smaller file sizes. Choose accordingly.

Enable Objects Labeling

If selected, everything in a backup is labeled with the name of the cluster. This can be useful if you’re working directly with the bucket where the backups are stored.

Backup Storage Provider - AWS

If your backup storage provider is AWS, you’ll see these fields:

Figure 15 - Using AWS as the backup storage provider

Bucket

The name of the S3 bucket.

Region

The region where the bucket is hosted.

Access Key

The access key credential for your AWS account. This field is ignored if you have a value in the Assume ARN field below.

Secret Key

The secret key credential for your AWS account. This field is ignored if you have a value in the Assume ARN field below.

Assume ARN

The ARN (Amazon Resource Name) for the bucket. If you have a value in this field, ACM ignores any values in the Access Key and Secret Key fields above. NOTE - Configuring an ARN that works with your service accounts and the Altinity Cloud Manager is complicated. Contact Altinity support for help in setting up your ARN.

Path (optional)

The path inside the bucket where the backups should be stored. The default value is altinity-cloud-managed-clickhouse.

TEST CONNECTION

If you change the Secret Key, the TEST CONNECTION button becomes active. If the connection works, you’ll see this message:

Figure 16 - A successful connection test

Otherwise you’ll see a message that tells you what went wrong:

Figure 17 - An unsuccessful connection test

Backup Storage Provider - GCP

If your backup storage provider is GCP, you’ll see these fields:

Figure 18 - Using GCP as the backup storage provider

Bucket

The name of the bucket.

Credentials JSON

JSON data that contains credentials associated with a GCP service account. That service account can have access to your entire GCP project, or it may be restricted to a single bucket or even a single folder within a single bucket. See the Google Cloud documentation for details on Creating a service account and Creating a credentials.json file with the gcloud iam service-accounts keys create command

Path (optional)

The path to the directory inside the bucket where your data is stored. The default value is altinity-cloud-managed-clickhouse.

TEST CONNECTION

When you’ve defined a complete set of credentials, the TEST CONNECTION button at the bottom of the dialog will become active. Clicking the button will return one of these messages:

Figure 19 - A successful connection test

Figure 20 - An unsuccessful connection test

You’ll of course need to correct any errors before you can continue.

Note that when you return to this panel, the value of the Credentials JSON field will be hidden and the TEST CONNECTION button will be disabled. You’ll need to enter your Credentials JSON again if you want to re-test the connection.

Backup Storage Provider - Azure

If your backup storage provider is Azure, you’ll see these fields:

Figure 21 - Using Azure as the backup storage provider

Container

The name of the container.

Account Name

The organization name for your Azure account.

Account Key

The key for your Azure account.

Path (optional)

The path inside the container where the backups should be stored. The default value is altinity-cloud-managed-clickhouse.

TEST CONNECTION

When you’ve defined a complete set of credentials, the TEST CONNECTION button at the bottom of the dialog will become active. Clicking the button will return one of these messages:

Figure 22 - Successful and unsuccessful connection test messages

You’ll of course need to correct any errors before you can continue.

Note that when you return to this panel, the value of the Account Key field will be hidden and the TEST CONNECTION button will be disabled. You’ll need to enter your Account Key again if you want to re-test the connection.

The Metrics tab

Figure 23 - The Metrics tab

Field details

Datadog options

Turn on Datadog

Enables Datadog for clusters in this environment. See Datadog's ClickHouse integration documentation for the details on configuring Datadog and the ClickHouse metrics that Datadog can monitor. Complete details on configuring Datadog access in the Altinity Cloud Monitor are in the section Enabling Datadog at the environment level.

API Key

Your Datadog API key.

Region

The region where your Datadog server is located.

Send Logs
Send Metrics
Send Table Metrics

These buttons allow you to send logs, metrics on the ClickHouse cluster, and/or table-level metrics in your ClickHouse cluster.

External Prometheus configuration options

Remote URL

The URL of your remote Prometheus server. In Figure 22 above, the remote write endpoint is https://logs-prod-006.grafana.net.

Auth User
Auth Password

The username and password for your remote Prometheus server. In the example above, the User ID is 4739826. The password in this example is a token generated by Grafana Cloud.

Metric Storage configuration option

Retention Period, Days

The number of days backups should be kept. The default value for Altinity.Cloud Anywhere environments is 30. For all other environments, this value is 180 and cannot be changed.

The Logs tab

Figure 24 - The Logs tab

Field details

Loki Logs Storage Configuration

Logs Bucket
Logs Region

The name of the bucket where your Loki logs should be stored and the region where the bucket is hosted. The bucket is used internally by the ACM; it’s not accessible by users. In addition, this setting is only used for Bring Your Own Kubernetes (BYOK) environments. For Bring Your Own Cloud (BYOC) environments, the bucket is configured automatically and these fields are not editable.

Send Logs to an External Loki service

External Loki URL

The URL of your external Loki server in the format https://username:password@lokiserver.com/api/prom/push. For complete details on integrating your Altinity.Cloud environment with an external Loki server, see the Administrator Guide section on Integrating Loki.

Other configuration tasks

Defining custom node types

When you create a ClickHouse cluster in your environment, the ACM automatically includes a number of node types. The names and capabilities of those node types are based on the cloud provider associated with your environment.

To see the list of node types defined for your environment, go to the summary view of the environment and click the NODE TYPES menu at the top:

Figure 25 - The Node Types menu

You’ll see the list of node types defined in your environment:

Figure 26 - The list of node types

How (or if) you define custom node types depends on your environment:

• If you’re running Altinity.Cloud as a Service, the node types are managed for you by Altinity.Cloud. If you need to add or edit node types, contact Altinity support.
• If you’re running Altinity.Cloud in a Bring Your Own Cloud (BYOC) environment, Altinity defines a number of node types for you. It’s highly unlikely you’ll need to edit or create node types, but if you do, you can.
• If you’re running Altinity.Cloud in a Bring Your Own Kubernetes (BYOK) environment, the ACM has no way of knowing what kinds of nodes you have in your environment. If you configure autoscaling with Karpenter (AWS), the AKS cluster autoscaler (Azure), or the GKE Autopilot (GCP), you can use the ACM to refer to those node types.

To work with node types, click the button to add a new node type.

It’s not common, but you can also click the vertical dots icon next to a node type in the list and select Edit to edit an existing node type:

Figure 27 - The edit node type menu

You’ll see this dialog:

Figure 28 - The Node Type Details dialog

Field details

Name - The name of the node. The node name and the instance type are typically the same, although that’s not a requirement.

Scope - Either Clickhouse, Zookeeper, or System. The scope allows you to define which node types you want to use with each component of the system.

Instance Type - Clicking the down arrow displays a list of dozens of cloud-provider-specific instance types. The example here is from a ClickHouse environment hosted on Azure, so Standard_D2s_v5 is an option. Similarly, n2-standard-4 would be an option for GCP and t4g.large would be an option for AWS.

Memory, MB - The amount of memory assigned to this node type. A value of at least 4096 is recommended.

CPU - The number of CPUs assigned to this node type.

Kubernetes options

These options are typically used in BYOK environments only. In many cases, the underlying BYOK infrastructure hosts workloads other than ClickHouse. That means you’ll want to make sure the right nodes are used for the ClickHouse, Zookeeper, and System nodes, as well as all of the other nodes in your environment. Defining Kubernetes tolerations or nodeSelectors gives you control over how nodes are allocated and managed in your environment. (In some cases BYOC environments are shared with other workloads; if that’s true, BYOC users may need Tolerations or NodeSelectors as well.)

Tolerations - One or more Kubernetes tolerations associated with this node type. Multiple tolerations should be separated by semicolons (;).

Node Selector - One or more Kubernetes nodeSelectors associated with this node type. Multiple nodeSelectors should be separated by commas (,).

Storage Class - Leave this field as is. It will be removed in a future version of the ACM.

Deleting an Environment

Before you delete an environment, it’s recommended that you delete all of the ClickHouse clusters in that environment.

When you’re ready to delete the environment, go to the list of environments, click the vertical dots icon next to your environment name and select Delete from the menu:

Figure 29 - The Delete Environment menu

If your environment doesn’t have any ClickHouse clusters, the dialog is straightforward:

Figure 30 - Deleting an environment without any ClickHouse clusters

On the other hand, if there are clusters in the environment, you can use the Delete Clusters slider to delete all of the ClickHouse clusters associated with your environment. If you’re deleting clusters, the ACM requires you to type the name of the environment before deleting it. Safety first!

Figure 31 - Deleting an environment that does have ClickHouse clusters

You also have the option of deleting the Altinity.Cloud environment without deleting the ClickHouse clusters associated with them. In other words, you can disconnect your ClickHouse clusters from Altinity Cloud Manager without losing data or the ClickHouse instance that manages it. That’s just one of the ways we avoid vendor lock-in.