Creating a new Cluster

Many times you’ll just want to set up a quick cluster. That’s why we’ve created a quick setup panel that lets you launch a new ClickHouse cluster as quickly as possible. When you click the button, you’ll see this simple panel:

Figure 2 – The quick version of the Launch Cluster wizard

You can simply give your cluster a name, take the defaults for everything else, and click the button. You can change any configuration values later except the cluster name and the availability zones your cluster uses.

Quick Path field details

Here are the details of the options in the panel.

• Purpose - Lets you classify this as a Development cluster or a Production cluster. You can change this setting later if you need to.

• Advanced Setup - Takes you to the Advanced Setup dialog described in the section below. The advanced version of the wizard gives you all of the configuration options. Again, if you click Advanced Setup you won’t be able to come back to the quick version of the wizard.

• Name - The name of your ClickHouse cluster. It can only contain lowercase letters, digits, and hyphens. As mentioned above, this can’t be changed later.

• ClickHouse Version - Lets you choose the version of ClickHouse you want your new cluster to run. There are two options: Altinity Builds, which are stable builds supported for three years by Altinity. The other option, Upstream Builds, lets you choose newer versions from the ClickHouse community. You can change the version later, but it may degrade performance while the Altinity Cloud Manager migrates your cluster from one version to another. The wizard also contains a link to the release notes for the version you select.

• Node Type - Lets you choose the node type for your new cluster. Make a choice from the drop-down list. We recommend you choose the smallest node type that can probably handle the workload of your cluster. You can change this later if you need to.

• Node Storage (GiB) - The storage, in gigabytes, that each ClickHouse node will have.

As you make choices, the cost estimate at the bottom of the panel will change to reflect the resources you’re requesting When you’re ready, click the button to provision your ClickHouse cluster. That’s all you need to do!

While your cluster is being provisioned, you’ll see a dialog that gives you the password for the admin user:

Figure 3 – Getting the admin password

Be sure to click the icon to copy the password; you won’t be able to see it again. (You can change it later, but you can’t change it now.)

See the section Working with your new cluster below for details on how to work with your new cluster.

Obviously there are times when you want to configure your new cluster beyond the basics. If you click Advanced Setup in the panel in Figure 2, you’ll be taken to the full Launch Cluster Wizard. It takes you through seven pages that let you set advanced options like backup schedules, shards, replicas, custom endpoints, and other useful things.

NOTE: After your ClickHouse cluster has been created, you can change any settings except the cluster name and the availability zones your cluster will use.

We’ll cover all the details here, explaining every option along the way and discussing why you might need or want to use them. If you’re looking for help on a particular section of the wizard, you can skip ahead to any of the tabs:

• ClickHouse setup
• Resources configuration
• High Availability configuration
• Connection configuration
• Activity schedule
• Annotations
• Review and Launch

1. ClickHouse Setup tab

The first tab in the wizard is the ClickHouse Setup tab.

Figure 4 – The ClickHouse Setup tab

Field details

Name

Enter a name for your new cluster. The name will become part of a URL once the cluster is deployed, so it must follow these rules:

• It must begin with a lowercase letter [a-z]
• It can only contain lowercase letters [a-z], numbers [0-9], and hyphens
• It must end with either a letter or a number
• It can’t be more than 15 characters long.

NOTE: Once you create your ClickHouse cluster, its name cannot be changed.

Cluster Role

This can be Not set, Development, or Production. These are simply labels applied to your ClickHouse clusters; they can make it easier to manage your clusters. It’s perfectly fine to leave this as Not set. And you can change it later if you want.

ClickHouse Version

Select the version of ClickHouse you want your cluster to use. Click the down arrow to see a list of available versions. ALTINITY BUILDS is selected by default; that lets you choose which Altinity Stable Build you want to use. You can also click UPSTREAM BUILDS to see other versions of ClickHouse.

Beneath the ClickHouse version is a link to the release notes for the build you’ve selected. The release notes have extensive details of what is new and changed and fixed in each release. Click the link to open the release notes in a new browser tab.

ClickHouse User Name

Currently this is admin and can’t be changed.

ClickHouse User Password

Enter and confirm a password at least 12 characters long. The NEXT button is disabled until the password is at least 12 characters long and the two passwords match.

Click NEXT to continue.

2. Resources Configuration tab

With the basics of your ClickHouse cluster defined, it’s time to give it some CPUs and storage.

Figure 5 – The Resources Configuration tab

Field details

Node Type

Click the down arrow to see what machine types are available. Each item in the list will tell you how many CPUs and how much RAM that machine type has.

Node Storage

The amount of storage in GB that each ClickHouse host will have. If the environment doesn’t have enough resources to meet your requirements, the graphs of CPU and Storage usage will turn red. You can reduce the amount of storage each ClickHouse node will have. If that’s not an option, contact Altinity Support.

Number of Volumes

The number of volumes used by your cluster. Increasing the number of volumes can improve performance and increase storage throughput.

Volume Type

Click the down arrow to see what classes of storage are available for your ClickHouse clusters.

Number of Shards

Enter the number of shards you want for your ClickHouse cluster. The graphs of CPU and Storage usage will be updated as you change the number of shards.

NOTE: sharding is only recommended if you have more than 5 TB of data.

Estimated maximum node throughput

This displays the disk throughput performance you’re likely to see. This is based on disk size, number of vCPUs, and other factors.

If you’re running on AWS, you’ll be able to select the throughput you need:

Figure 6 – Selecting throughput on AWS

Mousing over the graphs or values for CPU, Storage, and Shards displays more specifics:

Figure 7 – More details on CPU, Storage, and Shards

Click NEXT to continue.

3. High Availability Configuration tab

This tab covers server redundancy and failover.

Figure 8 – The High Availability Configuration tab

Field details

Number of Replicas

Select 1, 2, or 3. The CPU and Storage graphs will update as you change the number of replicas.

As with the values for CPU, Storage, and Shards, mousing over Replicas displays more specifics:

Figure 9 – More details on Replicas

Availability Zones

Select which availability zones you want to use. It is highly recommended that you use at least two availability zones.

NOTE: Once your ClickHouse cluster is created, the availability zones cannot be changed.

Zookeeper Node Type

The type of node that Zookeeper should run on.

Backup Schedule

You can create a schedule to create backups automatically at certain times of the day, week, or month. You define a backup schedule with these controls:

Figure 10 - Backup Schedule details

There are five options to define the Period when backups should occur:

• Monthly - Define the day of the month
• Weekly - Define the day of the week
• Daily - Define the time of day
• Every 6 hours - Backups occur every six hours
• Every hour - Backups occur every hour.

NOTE: All times are expressed in GMT and are displayed in 12- or 24-hour format depending on your machine’s settings.

In addition to defining the period, you can also define the number of Backups to Keep. The default is six.

The button lets you define multiple schedules. For example, if you only want backups to occur on Friday and Saturday, create two Weekly schedules, one for Friday and one for Saturday. You can define up to three schedules.

Preferred Backup Replica

The replica you prefer to use for backups. If that replica is available, it will be used.

Click NEXT to continue.

4. Connection Configuration tab

Next you need to define your ClickHouse cluster’s connections to the outside world.

Figure 11 – The Connection Configuration tab

Field details

Endpoint

This is the endpoint of your cluster. Notice that the value here is the cluster name you defined in the first tab combined with an Altinity domain.

Protocols

Port 9440 enables the ClickHouse binary protocol, port 8443 enables HTTP connections, and port 9004 enables MySQL connections. Note: MySQL connections are for BYOC environments only.

Alternate Endpoints

You can define alternate endpoints for your cluster. The name of the alternate endpoint can contain lowercase letters, numbers, and hyphens. It must start with a letter, and it cannot end with a hyphen.

Figure 12 – Defining an Alternate Endpoint

For example, you might want an endpoint that uses your organization’s domain, such as cluster.environment.example.com instead ofcluster.environment.altinity.cloud. You might also want to create an alternate endpoint and use it as the ClickHouse access point in your applications. With that approach, pointing the alternate endpoint to another cluster lets you switch the cluster your applications are using without changing the applications at all.

Clicking the yellow triangle icon displays the Create DNS records panel, which lists the required and optional DNS records you’ll need to create:

Figure 13 – Creating DNS records

When your DNS records are created, contact Altinity support to set up the alternate endpoint.

Load Balancer Type

The default value is Altinity Edge Ingress. Depending on how your Environment is configured, other options may be available. Contact Altinity support if the load balancer you need to use is not available.

IP restrictions

This is enabled by default. It is initially filled in with your current IP address. That means only ClickHouse applications or clients coming from your current IP address are allowed to connect to the cluster. You can add other addresses to the list, including ranges of IP addresses in CIDR format.

Disabling IP restrictions means that any ClickHouse application or client can connect to your ClickHouse cluster from any IP address.

NOTE: This restriction only applies to ClickHouse applications or clients. Anyone with the proper credentials can access the Altinity Cloud Manager from any IP address.

Datadog Integration

In Figure 9 above, Datadog integration is disabled. To use Datadog to monitor a ClickHouse cluster, you first need to enable Datadog at the Environment level. Once Datadog is enabled for your environment, the Datadog integration buttons are enabled as well:

Figure 14 – Datadog integration controls for the cluster are enabled when the environment is enabled for Datadog

The three options above allow you to send logs, metrics on the ClickHouse cluster, and/or table-level metrics in your ClickHouse cluster. See the section Enabling Datadog at the environment level for the details. Be aware that you must have the appropriate privileges to edit an environment’s settings, so you may need to contact your administrator.

Click NEXT to continue.

5. Activity Schedule tab

The next configuration option is to determine when your ClickHouse cluster should run.

Figure 15 – The Activity Schedule tab

There are four options:

• ALWAYS ON - Self-explanatory; your cluster is always on.
• STOP WHEN INACTIVE - Lets you define how many hours your ClickHouse cluster can be idle before it is stopped.
• STOP ON SCHEDULE - Lets you define specific times when your cluster is on.
• RESCALE ON SCHEDULE - Your cluster is always on, but this option also lets you define times when your cluster should run on larger, more powerful nodes and when it should scale back to run on smaller, cheaper nodes.

See Setting an activity schedule for complete details about how to set an activity schedule.

Click NEXT to continue.

6. Annotations tab

This tab lets you define any Kubernetes Annotations you may need:

Figure 16 – The Annotations tab

As you would expect, clicking the button creates a new annotation, and clicking the trash can icon deletes one. NOTE: Annotations are only available in Bring Your Own Kubernetes (BYOK) deployments. The Annotations are added to the chi instance that hosts your ClickHouse clusters. (A chi is a ClickHouse Installation, a Kubernetes custom resource.)

Click NEXT to continue.

7. Review & Launch tab

Finally, you have a chance to review your options before you provision and deploy the ClickHouse cluster. You can’t change anything on this screen, but you can use the BACK button to make changes on earlier tabs.

Figure 17 – The Review & Launch tab

If everything looks good, click LAUNCH to launch your new ClickHouse cluster. The cluster should be active in a few minutes.