This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Clusters

How to launch clusters and manage clusters.

ClickHouse databases are managed through clusters, which harness the power of distributed processing to quickly deliver results on even the most complex and data intensive queries.

Altinity.Cloud users can create their own ClickHouse Clusters tailored to their organization’s needs.

1 - View Cluster Details

How to view details of a running cluster and its nodes.

Cluster Details Page

Once a cluster has been launched, its current operating details can be viewed by selecting the cluster from the Clusters View. This displays the Cluster Details page. From the Cluster Dashboard page, select Nodes to view the Nodes Summary page.

From the Cluster Details page, users can perform the following:

Cluster Details Page
  • A: Manage the cluster’s:
    • Actions
    • Configuration
    • Tables and structure with Explore
    • Alerts
    • Logs
  • B: Check Cluster Health.
  • C: View the cluster’s Access Point.
  • D: Monitor the Cluster and its Queries.
  • E: View summary details for the Cluster or Node. Select Nodes to view details on the cluster’s Nodes.

Nodes Summary

The Nodes Summary Page displays all nodes that are part of the selected cluster. From this page the following options and information is available:

  1. The Node Summary that lists:
    1. Endpoint: The connection settings for this node. See Node Connection.
    2. Details and Node View: Links to the Node Dashboard and Node Metrics.
    3. Version: The ClickHouse version running on this node.
    4. Type: The processor setting for the node.
    5. Node Storage: Storage space in GB available.
    6. Memory: RAM memory allocated for the node.
    7. Availability Zone: Which AWS Availability Zone the node is hosted on.
Node Summary Page

Node Connection

The Node Connection Details shows how to connect from various clients, including the clickhouse-client, JDBC drivers, HTTPS, and Python. Unlike the Cluster Access Point, this allows a connection directly to the specific node.

Node Connection

Node Dashboard

From the Node Dashboard Page users can:

Node Dashboard
  • A: Manage the node’s:
    • Tables and structure with Explore
    • Logs
  • B: Check the node’s health.
  • C: View summary details node, it’s Metrics and its Schema.
  • D: Perform Node Actions.

Node Metrics

Node Metrics provides a breakdown of the node’s performance, such as CPU data, active threads, etc.

Node Schema

The Node Schema provides a view of the databases’ schema and tables. For more information on how to interact with a Node by submitting queries, viewing the schema of its databases and tables, and viewing process, see the Cluster Explore Guide.

2 - Cluster Actions

Actions that can be taken on launched clusters.

Launched clusters can be have different actions applied to them based on your needs.

2.1 - Upgrade Cluster

How to upgrade an existing cluster.

Clusters can be upgraded to versions of ClickHouse other than the one your cluster is running.

When upgrading to a ClickHouse Altinity Stable Build, review the release notes for the version that you are upgrading to.

How to Upgrade an Altinity Cloud Cluster

To upgrade a launched cluster:

  1. Select Actions for the cluster to upgrade.

  2. Select Upgrade.

  3. Select the ClickHouse version to upgrade to.

  4. Select Upgrade to start the process.

    Cluster Upgrade

The upgrade process completion time varies with the size of the cluster, as each server is upgraded individually. This may cause downtime while the cluster is upgraded.

2.2 - Rescale Cluster

How to rescale an existing cluster.

The size and structure of the cluster may need to be altered after launching based on your organization’s needs. The following settings can be rescaled:

  • Number of Shards
  • Number of Replicas
  • Node Type
  • Node Storage
  • Number of Volumes
  • Apply to new nodes only: This setting will only effect nodes created from this point forward.

See Cluster Settings for more information.

How to Rescale a Cluster

To rescale a cluster:

  1. Select Actions for the cluster to rescale.

  2. Select Rescale.

  3. Set the new values of the cluster.

  4. Click OK to begin rescaling.

    Cluster Rescale

Depending on the size of the cluster, this may take several minutes.

2.3 - Stop and Start a Cluster

How to stop or start an existing cluster.

To stop an launched cluster, or start a stopped cluster:

  1. From either the Clusters View or the Cluster Details Page, select Actions.
    1. If the cluster is currently running, select Stop to halt its operations.
    2. If the cluster has been stopped, select Start to restart it.

Depending on the size of your cluster, it may take a few minutes until it is fully stopped or is restarted. To access the health and availability of the cluster, see Cluster Health or the Cluster Availability.

2.4 - Export Cluster Settings

How to export a cluster’s settings.

The structure of an Altinity Cloud cluster can be exported as JSON. For details on the cluster’s settings that are exported, see Cluster Settings.

To export a cluster’s settings to JSON:

  1. From either the Clusters View or the Cluster Details Page, select Actions, then select Export.
  2. A new browser window will open with the settings for the cluster in JSON.

2.5 - Replicate a Cluster

How to replicate an existing cluster.

Clusters can be replicated with the same or different settings. These can include the same database schema as the replicated cluster, or launched without the schema. This may be useful to create a test cluster, then launch the production cluster with different settings ready for production data.

For complete details on Altinity.Cloud clusters settings, see Cluster Settings.

To create a replica of an existing cluster:

  1. From either the Clusters View or the Cluster Details Page, select Actions, then select Launch a Replica Cluster.
  2. Enter the desired values for Resources Configuration.
    1. To replicate the schema of the source directory, select Replicate Schema.

      Replicate Schema
    2. Click Next to continue.

  3. High Availability Configuration, and Connection Configuration.
    1. Each section must be completed in its entirety before moving on to the next one.
  4. In the module Review & Launch, verify the settings are correct. When finished, select Launch.

Depending on the size of the new cluster it will be available within a few minutes. To verify the health and availability of the new cluster, see Cluster Health or the Cluster Availability.

2.6 - Destroy Cluster

How to destroy an existing cluster.

When a cluster is no longer required, the entire cluster and all of its data can be destroyed.

  • IMPORTANT NOTE: Once destroyed, a cluster can not be recovered. It must be manually recreated.

To destroy a cluster:

  1. From either the Clusters View or the Cluster Details Page, select Actions, then select Destroy.

  2. Enter the cluster name, then select OK to confirm its deletion.

    Destroy Cluster

3 - Cluster Settings

Settings and values used for Altinity.Cloud ClickHouse Clusters.

ClickHouse Clusters hosted on Altinity.Cloud have the following structural attributes. These determine options such as the version of ClickHouse installed on them, how many replicas, and other important features.

Name Description Values
Cluster Name The name for this cluster. It will be used for the hostname of the cluster. Cluster names must be DNS compliant. This includes:
  • Alphanumeric characters and underscores only
  • No special characters such as periods, ?, #, etc.
    Example:
    • Good: mycluster
    • Bad: my.cluster?
  • Can not start with a number.
Node Type Determines the number of CPUs and the amount of RAM used per node. The following Node Types are sample values, and may be updated at any time:
  • m5.large: CPU x2, RAM 6.5 GB
  • m5.xlarge: CPU x4, RAM 14 GB
  • M5.2xlarge: (CPU x8, RAM 29 GB)
  • m5.4xlarge: (CPU x16, RAM 58 GB)
  • m5.8xlarge: (CPU x32, RAM 120 GB)
Node Storage The amount of storage space available to each node, in GB.  
Number of Volumes Storage can be split across multiple volumes. The amount of data stored per node is the same as set in Node Storage, but it split into multiple volumes.
Separating storage into multiple volumes can increase query performance.
 
Volume Type Defines the Amazon Web Services volume class. Typically used to determine whether or not to encrypt the columns. Values:
  • gp2 (Not Encrypted)
  • gp2-encrypted (encrypted)
Number of Shards Shards represent a set of nodes. Shards can be replicated to provide increased availability and computational power.  
ClickHouse Version The version of the ClickHouse database that will be used on each node.
To run a custom ClickHouse container version, specify the Docker image to use.
  • IMPORTANT NOTE: The nodes in the cluster will all be running the same version of ClickHouse. If you want to run multiple versions of ClickHouse, they will have to be set on different clusters.
Currently available options:
  • Altinity Stable:
    • 19.11.12.69
    • 19.16.19.85
    • 20.3.21.2
    • 20.8.7.15
  • Standard Release
    • 20.10.5.10
    • 20.11.4.13
  • Custom Identifier
ClickHouse Admin Name The name of the ClickHouse administrative user. Set to admin by default. Can not be changed.
ClickHouse Admin Password The password for the ClickHouse administrative user.  
Data Replication Toggles whether shards will be replicated. When enabled, Zookeeper is required to manage the shard replication process. Values:
  • Enabled (Default): Each Cluster Shard will be replicated to the value set in Number of Replicas.
  • Disabled: Shards will not be replicated.
Number of Replicas Sets the number of replicas per shard. Only enabled if Data Replication is enabled.  
Zookeeper Configuration When Data Replication is set to Enabled, Zookeeper is required. This setting determines how Zookeeper will run and manage shard replication.
The Zookeeper Configuration mainly sets how many Zookeeper nodes are used to manage the shards. More Zookeeper nodes increases the availability of the cluster.
Values:
  • Single Node (Default): Replication is managed by one Zookeeper node.
  • Three Nodes: Increases the Zookeeper nodes to an ensemble of 3.
Zookeeper Node Type Determines the type of Zookeeper node. Defaults to default and can not be changed.
Node Placement Sets how nodes are distributed via Kubernetes. Depending on your situation and how robust you want your replicas and clusters. Values:
  • Separate Nodes (Default): ClickHouse containers are distributed across separate cluster nodes.
  • Separate Shards: ClickHouse containers for different shards are distributed across separate cluster nodes.
  • Separate Replicas: ClickHouse containers for different replicas are distributed across separate cluster nodes.
Enable Backups Backs up the cluster. These can be restored in the event data loss or to roll back to previous versions. Values:
  • Enabled (Default): The cluster will be backed up automatically.
  • Disabled: Automatic Backups are disabled.
Backup Schedule Determines how often the cluster will be backed up. Defaults to Daily
Number of Backups to keep Sets how many backups will be stored before deleting the oldest one Defaults to 5.
Endpoint The Access point Domain Name. This is hard set by the name of your cluster and your organization.
Use TLS Sets whether or not to encrypt external communications with the cluster to TLS. Default to Enabled and can not be changed.
Load Balancer Type The load balancer manages communications between the various nodes to ensure that nodes are not overwhelmed. Defaults to Altinity Edge Ingress
Protocols Sets the TCP ports used in external communications with the cluster. Defaults to ClickHouse TCP port 9440 and HTTP port 8443.

4 - Configure Cluster

How to configure launched clusters.

Once a cluster has been launched, it’s configuration can be updated to best match your needs.

4.1 - How to Configure Cluster Settings

How to update the cluster’s settings.

Cluster settings can be updated from the Clusters View or from the Cluster Details by selecting Configure > Settings.

  • IMPORTANT NOTE: Changing a cluster’s settings will require a restart of the entire cluster.

Note that some settings are locked - their values can not be changed from this screen.

Cluster Settings

How to Set Troubleshooting Mode

Troubleshooting mode prevents your cluster from auto-starting after a crash. To update this setting:

  1. Toggle Troubleshooting Mode either On or Off.

How to Edit an Existing Setting

To edit an existing setting:

  1. Select the menu on the left side of the setting to update.
  2. Select Edit.
  3. Set the following:
    1. Setting Type.
    2. Name
    3. Value
  4. Select OK to save the setting.
Edit Cluster Setting

How to Add a New Setting

To add a new setting to your cluster:

  1. Select Add Setting.
  2. Set the following:
    1. Setting Type.
    2. Name
    3. Value
  3. Select OK to save the setting.
Add a New Cluster Setting

How to Delete an Existing Setting

To delete an existing setting:

  1. Select the menu on the left side of the setting to update.
  2. Select OK.
  3. Select Remove to confirm removing the setting.
Delete a Cluster Setting

4.2 - How to Configure Cluster Profiles

How to update the cluster’s profiles.

Cluster profiles allow you to set the user permissions and settings based on their assigned profile.

The Cluster Profiles can be accessed from the Clusters View or from the Cluster Details by selecting Configure > Settings.

Cluster Profile Settings

Add a New Profile

To add a new cluster profile:

  1. From the Cluster Profile View page, select Add Profile.
  2. Provide profile’s Name and Description, then click OK.

Edit an Existing Profile

To edit an existing profile:

  1. Select the menu to the left of the profile to update and select Edit, or select Edit Settings.
  2. To add a profile setting, select Add Setting and enter the Name and Value, then click OK to store your setting value.
  3. To edit an existing setting, select the menu to the left of the setting to update. Update the Name and Value, then click OK to store the new value.

Delete an Existing Profile

To delete an existing profile:

  1. Select the menu to the left of the profile to update and select Delete.
  2. Select OK to confirm the profile deletion.

4.3 - How to Configure Cluster Users

How to update the cluster’s users.

The cluster’s Users allow you to set one or more entities who can access your cluster, based on their Cluster Profile.

Cluster users can be updated from the Clusters View or from the Cluster Details by selecting Configure > Users.

Cluster Users

How to Add a New User

To add a new user to your cluster:

  1. Select Add User

  2. Enter the following:

    Add New User
    1. Login: the name of the new user.
    2. Password and Confirm Password: the authenticating credentials for the user. Th
    3. Networks: The networks that the user can connect from. Leave as 0.0.0.0/0 to allow access from all networks.
    4. Databases: Which databases the user can connect to. Leave empty to allow access all databases.
    5. Profile: Which profile settings to apply to this user.
  3. Select OK to save the new user.

How to Edit a User

To edit an existing user:

  1. Select the menu to the left of the user to edit, then select Edit.
  2. Enter the following:
    1. Login: the new name of the user.
    2. Password and Confirm Password: the authenticating credentials for the user. Th
    3. Networks: The networks that the user can connect from. Leave as 0.0.0.0/0 to allow access from all networks.
    4. Databases: Which databases the user can connect to. Leave empty to allow access all databases.
    5. Profile: Which profile settings to apply to this user.
  3. Select OK to save the updated user user.

How to Delete a User

  1. Select the menu to the left of the user to edit, then select Delete.
  2. Select OK to verify the user deletion.

5 - Launch New Cluster

How to launch a new ClickHouse Cluster from Altinity.Cloud.

Launching a new ClickHouse Cluster is incredibly easy, and only takes a few minutes. For those looking to create their first ClickHouse cluster with the minimal steps, see the Quick Start Guide. For complete details on Altinity.Cloud clusters settings, see Cluster Settings.

To launch a new ClickHouse cluster:

  1. From the Clusters View page, select Launch Cluster. This starts the Cluster Launch Wizard.

    Launch New Cluster
  2. Enter the desired values for Resources Configuration, High Availability Configuration, and Connection Configuration.

    1. Each section must be completed in its entirety before moving on to the next one.
  3. In the module Review & Launch, verify the settings are correct. When finished, select Launch.

Within a few minutes, the new cluster will be ready for your use and display that all health checks have been passed.

6 - Cluster Alerts

How to be notified about cluster issues

The Cluster Alerts module allows users to set up when they are notified for a set fo events. Alerts can either be a popup, displaying the alert when the user is logged into Altinity.Cloud, or email so they can receive an alert even when they are not logged into Altinty.Cloud.

To set which alerts you receive:

  1. From the Clusters view, select the cluster to for alerts.

  2. Select Alerts.

    Cluster Alerts
  3. Add the Email address to send alerts to.

  4. Select whether to receive a Popup or Email alert for the following events:

    1. ClickHouse Version Upgrade: Alert triggered when the version of ClickHouse that is installed in the cluster has a new update.
    2. Cluster Rescale: Alert triggered when the cluster is rescaled, such as new shards added.
    3. Cluster Stop: Alert triggered when some event has caused the cluster to stop running.
    4. Cluster Resume: Alert triggered when a cluster that was stopped has resumed operations.

7 - Cluster Health Check

How to quickly check your cluster’s health.

From the Clusters View, you can see the health status of your cluster and its nodes at a glance.

How to Check Node Health

The quick health check of your cluster’s nodes is displayed from the Clusters View. Next to the cluster name is a summary of your nodes’ statuses, indicating the total number of nodes and how many nodes are available.

View the Access Point

How to Check Cluster Health

The overall health of the cluster is shown in the Health row of the cluster summary, showing the number of health checks passed.

View the Access Point

Click checked passed to view a detailed view of the cluster’s health.

How to View a Cluster’s Health Checks

The cluster’s Health Check module displays the status of the following health checks:

  • Access point availability check
  • Distributed query check
  • Zookeeper availability check
  • Zookeeper contents check
  • Readonly replica check
  • Delayed inserts check

To view details on what queries are used to verify the health check, select the caret for each health check.

Cluster Health Details

8 - Cluster Monitoring

How to monitor your clusters performance.

Altinity.Cloud integrates Grafana into its monitoring tools. From a cluster, you can quickly access the following monitoring views:

  • Cluster Metrics
  • Queries
  • Logs

How to Access Cluster Metrics

To access the metrics views for your cluster:

  1. From the Clusters view, select the cluster to monitor.
  2. From Monitoring, select the drop down View in Grafana and select from one of the following options:
    1. Cluster Metrics
    2. Queries
    3. Logs
  3. Each metric view opens in a separate tab.

Cluster Metrics

Cluster Metrics displays how the cluster is performing from a hardware and connection standpoint.

Cluster Monitoring View

Some of the metrics displayed here include:

  • DNS and Distributed Connection Errors: Displays the rate of any connection issues.
  • Select Queries: The number of select queries submitted to the cluster.
  • Zookeeper Transactions: The communications between the zookeeper nodes.
  • ClickHouse Data Size on Disk: The total amount of data the ClickHouse database is using.

Queries

The Queries monitoring page displays the performance of clusters, including the top requests, queries that require the most memory, and other benchmarks. This can be useful in identifying queries that can cause performance issues and refactoring them to be more efficient.

Query Monitoring View

Log Metrics

The Log monitoring page displays the logs for your clusters, and allows you to make queries directly on them. If there’s a specific detail you’re trying to iron out, the logs are the most granular way of tracking down those issues.

Log Monitoring View

9 - Cluster Logs

How to access your cluster’s logs

Altinity.Cloud provides the cluster log details so users can track down specific issues or performance bottlenecks.

To access a cluster’s logs:

  1. From the Clusters view, select the cluster to for alerts.
  2. Select Logs.
  3. From the Log Page, you can display the number of rows to view, or filter logs by specific text.
  4. To download the logs, select the download icon in the upper right corner (A).
  5. To refresh the logs page, select the refresh icon (B).
Cluster Logs Page

The following logs are available:

  • ACM Logs: These logs are specific to Altinity.Cloud issues and include the following:
    • System Log: Details the system actions such as starting a cluster, updating endpoints, and other details.
    • API Log: Displays updates to the API and activities.
  • ClickHouse Logs: Displays the Common Log that stores ClickHouse related events. From this view a specific host can be selected form the dropdown box.
  • Backup Logs: Displays backup events from the clickhouse-backup service. Log details per cluster host can be selected from the dropdown box.
  • Operator Logs: Displays logs from the Altinity Kubernetes Operator service, which is used to manage cluster replication cluster and communications in the Kubernetes environment.