Configuring Storage

Clicking the Storage menu item displays the list of volumes for your ClickHouse® cluster:

Figure 1 - The Volumes view

There are several buttons at the top of the display:

• MODIFY VOLUME - Lets you make changes to the selected volume.
• ADD VOLUME - Lets you add another volume to your cluster.
• FREE VOLUME - Moves all data from the selected volume to non-cordoned volumes in the cluster.
• CORDON VOLUME - Changes the selected volume’s status to cordoned. A cordoned volume will not receive any new data; cordoning a volume is the first step towards removing it.
• REMOVE VOLUME - Removes the selected volume from the cluster.
• STORAGE POLICY - Lets you change the storage policy for the cluster. Available policies are JBOD (just a bunch of disks) or Tiered.

We’ll cover these options next.

Modifying a volume

Selecting a volume and clicking the button lets you change the properties of the selected volume. At a minimum, this allows you to change the type of disk and its size:

Figure 2 - The Modify Volume dialog

Clicking the down arrow icon displays a menu of available disk types based on the cloud provider hosting your ClickHouse cluster. You can also change the size of the volume.

The dialog may have other options based on your cloud provider. For example, if your ClickHouse cluster is hosted on AWS, you can change the throughput of the volume:

Figure 3 - Setting throughput for an AWS volume

Finally, if your storage policy is JBOD, you’ll get a warning message if the volume type you’ve selected is different from the other volumes in your cluster:

Figure 4 - The JBOD warning message for mixed volume types

Using different types of volumes with a JBOD policy will give inconsistent performance, as the different volumes may not have the same capabilities.

Adding a volume

Clicking the button lets you add another volume to your cluster:

Figure 5 - The Add New Volume dialog

Be aware that the size of each volume must be at least 350 GB to use multiple volumes. As with modifying a volume, if your storage policy is JBOD, you’ll get a warning message if the new volume is of a different type from the other volumes in your cluster:

Figure 6 - The JBOD warning message for mixed volume types

Click SAVE to add the new volume. It will appear in the list of volumes.

Freeing a volume

Selecting a volume and clicking the button moves all data from the selected volume to non-cordoned volumes in the cluster. You must first cordon the volume for the FREE VOLUME button to be enabled. You’ll be asked to confirm that you want to free the volume:

Figure 7 - The Free Volume dialog

When all data is moved off of this volume, the REMOVE VOLUME button will become active.

Cordoning a volume

Selecting a volume and clicking the button cordons the volume, which means no new data will be written to that volume. Clicking the button changes its text to UNCORDON VOLUME, which reverses the operation. A cordoned volume can be freed, which moves all data from the volume to non-cordoned volumes in the cluster.

Removing a volume

If the selected volume has no data, the button will be active. To remove a volume, you must cordon it, which means no new data will be written to it, then free the volume, which moves any data on the volume to non-cordoned volumes. As you would expect, clicking the button gives you a confirmation message:

Figure 8 - The Remove Volume dialog

Click OK to remove the volume.

Setting the storage policy

Clicking the button lets you set the storage policy. There are two options: JBOD (just a bunch of disks) and Tiered (hot and cold data are stored separately). If the storage policy is JBOD, the dialog is simple:

Figure 9 - Modifying the storage policy - JBOD

On the other hand, Tiered storage is more complicated. Tiered storage is useful, for example, in a hot/cold architecture. You could keep your newest data in faster, more expensive storage, while moving older data to a slower, cheaper volume.

In ClickHouse you can define a Time-To-Live (TTL) period for data. The most common use of TTL is to delete old data that is no longer needed, but ClickHouse also lets you use TTL to move data from one volume to another when it reaches a certain age. To make the most of Tiered storage, you’ll need to define your own TTL properties.

See Manage Data with TTL in the ClickHouse documentation for all the details on TTL.

The dialog for Tiered storage has an additional field:

Figure 10 - Modifying the storage policy - Tiered

The Move factor lets you define when the cluster should move data to alternate storage. The default value is 0.1, which means data is moved when the amount of available space on a volume is less than 10%. In the example here, the move factor is 0.2, which sets the threshold to 20%.

Changing the storage policy may result in data being moved between volumes. That can impact performance of running applications. For that reason, you’re asked to confirm any changes to the policy:

Figure 11 - Confirming a change to the storage policy

Click OK to change the storage policy.