This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Altinity.Cloud

Manuals, quick start guides, code samples and tutorials on how to use Altinity.Cloud to launch and get the most out of your ClickHouse clusters.

Altinity.Cloud provides the best experience in managing ClickHouse. Create new clusters with the version of ClickHouse, set your node configurations, and get right to work.

1 - Altinity.Cloud 101

What is Altinity.Cloud?

Welcome to Altinity.Cloud. In this guide, we will be answering a few simple questions:

  • What is Altinity.Cloud?
  • Why should I use it?
  • How does it work?

What is Altinity.Cloud?

Altinity.Cloud is a fully managed ClickHouse services provider. Altinity.Cloud is the easiest way to set up a ClickHouse cluster with different configurations of shards and replicas, with the version of ClickHouse or Altinity Stable for ClickHouse you want. From one spot you can monitor performance, run queries, upload data from S3 or other cloud stores, and other essential operations.

For more details on Altinity.Cloud abilities, see the Administrator Guide. For a crash course on how to create your own ClickHouse clusters with Altinity.Cloud, we have the Altinity.Cloud Quick Start Guide.

What Can I Do with Altinity.Cloud?

Altinity.Cloud lets you create, manage, and monitor ClickHouse clusters with a few simple clicks. Here’s a brief look at the user interface:

Clusters View
  • A: Cluster Creation: Clusters can be created from scratch with Launch Cluster.
  • B: Clusters: Each cluster associated with your Altinity.Cloud account is listed in either tile format, or as a short list. They’ll display a short summary of their health and performance. By selecting a cluster, you can view the full details.
  • C: User and Environment Management:
    • Change to another environment.
    • Manage environments and zookeepers.
    • Update account settings.

Clusters can be spun up and set with the number of replicas and shards, the specific version of ClickHouse that you want to run on them, and what kind of virtual machines to power the nodes.

When your clusters are running you can connect to them with the ClickHouse client, or your favorite applications like Grafana, Kafka, Tableau, and more. See the Altinity.Cloud connectivity guide for more details.

Monitoring

Cluster performance can be monitored in real time through the Cluster Monitor system.

Cluster Monitoring View

Some of the metrics displayed here include:

  • DNS and Distributed Connection Errors: Displays the rate of any connection issues.
  • Select Queries: The number of select queries submitted to the cluster.
  • Zookeeper Transactions: The communications between the zookeeper nodes.
  • ClickHouse Data Size on Disk: The total amount of data the ClickHouse database is using.

How is Altinity.Cloud organized?

Security Tiers

Altinity.Cloud starts at the Organization level - that’s your company. When you and members of your team log into Altinity.Cloud, they’ll start here. Depending on their access level, they can then access the different systems within the organization.

The next level down from there are the Environments. Each organization has at least one Environment, and these are used to allow users access to one or more Clusters.

Clusters consist of one or more Nodes - individual containers that run the ClickHouse databases. These nodes are grouped into shards, which are sets of nodes that all work together to improve performance and reliability. Shards can then be set as replicas, where groups of nodes are copied. If one replica goes down, the other replicas can keep running and copy their synced data when the replica is restored or when a new replica is added.

To recap in reverse order:

  • Nodes are individual virtual machines or containers that run ClickHouse.
  • Shards are groups of nodes work together to improve performance and share data.
  • Replicas are groups of shards that mirror data and performance so when one replica goes down, they can keep going.
  • Clusters are sets of replicas that work together to replicate data and improve performance.
  • Environments contain different clusters into a set to control access and resources.
  • Organizations have one or more environments that service your company.

Altinity.Cloud Access

Altinity.Cloud keeps your users organized in the following roles:

Role Environment Cluster
orgadmin These users can create environments and clusters, and assign users in their organization to them.
envadmin These users have control over environments they are assigned to by the orgadmin. They can create clusters and control clusters within these environments.
envuser These users can access the clusters they are specifically assigned to within specific environments.

More details are available in the Account Administration guide.

Where can I find out more?

Altinity provides the following resources to our customers an the Open Source community:

2 - Quick Start Guide

The minimal steps to get Altinity.Cloud running with your first cluster.

Welcome to Altinity.Cloud! Altinity.Cloud is the fastest, easiest way to set up, administer and use ClickHouse. Your ClickHouse is fully managed so you can focus on your work.

If this is your first time using Altinity.Cloud, this quick start guide will give you the minimum steps to become familiar with the system. When you’re ready to dig deeper and use the full power of ClickHouse in your Altinity.Cloud environment, check out our Administrator and Developer Guides for in depth knowledge and best practices.

A full PDF version of this document is available through this link: Quick Start Guide PDF

2.1 - Account Creation and Login

How to set up your Altinity.Cloud account, and login to the service.

Create an Account

To start your Altinity.Cloud journey, the first thing you need is an account. New users can sign up for a test account on the Altinity.Cloud Test Drive page. Enter your contact information, and an Altinity.Cloud rep will get right with you.

Once finished, the Altinity.Cloud team will reach out to you with your login credentials. Altinity.Cloud uses your email address as your username, and you’ll be assigned an initial password.

Login to Altinity.Cloud

There are two methods to login:

Login with Username and Password

If you’ve used any web site, you’re likely familiar with this process.

To login to Altinity.Cloud with your username and password:

  1. Enter your username - in this case your email address - in the field marked Login.
  2. Enter your Password, then click Sign In.

Login with Auth0

Auth0 allows you to authenticate to Altinity.Cloud through a trusted authentication provider, in this case Google. Once set up, you can click Auth0 authenticate through your Google account.

  • Requirements: In order to use Auth0, you must have a Google account with the same email address that you use for Altinity.Cloud.

To setup authentication with Auth0 for the first time:

  1. Access the Altinity.Cloud page.
  2. Select Auth0.
  3. Select the Google account to use for authentication.
    IMPORTANT NOTE: The Google account must have the same email address as your Altinity.Cloud account.
  4. Select Continue with Google, and you’ll be in Altinity.Cloud.

After you’ve completed the Auth0 setup process, you can login to Altinity.Cloud by selecting Auth0 from the login page.

2.2 - Explore Clusters View

An overview of managing your ClickHouse clusters with Altinity.Cloud.

Explore Clusters View

Once you’ve logged in to Altinity.Cloud, let’s take a moment and familiarize ourselves with the environment. The default page is the Clusters View.

The Clusters View page is separated into the following sections:

Clusters View
  • A: Cluster Creation: Clusters can be created from scratch with Launch Cluster.
  • B: Clusters: Each cluster associated with your Altinity.Cloud account is listed in either tile format, or as a short list. They’ll display a short summary of their health and performance. By selecting a cluster, you can view the full details.
  • C: User and Environment Management:
    • Change to another environment.
    • Manage environments and zookeepers.
    • Update account settings.

2.3 - Create Your First Cluster

How to create your first ClickHouse cluster with Altinity.Cloud.

Time to make your first cluster! For this example, we’re creating a minimally sized cluster, but you can rescale your cluster later to make it the exact size you need for your ClickHouse needs.

As of October 21, 2021, Altinity.Cloud supports Google Compute Platform (GCP) and Amazon Web Services (AWS). For more information, see the Altinity.Cloud Administrator Guide.

To create your first cluster:

  1. From the Clusters View page, select Launch Cluster. This starts the Cluster Launch Wizard.

  2. The first page is Resources Configuration, where we set the name, size and authentication for the new cluster. When finished, click Next. Use the following settings:

    Setting Value
    Name Cluster names will be used to create the DNS name of the cluster. Therefore, cluster names must follow DNS name restrictions (letters, numbers, and dashes allowed, periods and special characters are not).
    Cluster names must start with a letter, and should be 15 characters at most.
    Node Type Select m5.large
    This is the size of the node. This selection gives us a cluster with 2 CPUs and around 7 GB RAM. Recall that we can rescale this cluster later. For more information, see the Administrator Guide.
    Node Storage Set to 30 GB.
    The size of each Cluster node in GB (gigabytes). Each node will have the same storage area.
    Number of Volumes Set to 1.
    Network storage can be split into separate volumes. Use more volumes to increase query performance.
    Volume Type Select gp2 (Not Encrypted).
    Volumes can be either encrypted or unencrypted, depending on your security requirements.
    Number of Shards Set to 1.
    The shard represents a set of nodes. Shards can then be replicated to provide increased availability and recovery.
    ClickHouse Version Select the most recent Altinity Stable Build.
    Your ClickHouse cluster can use the version that best meets your needs. Note that all nodes will run the same ClickHouse version.
    ClickHouse User Name Auto-set to admin.
    The default administrative user.
    ClickHouse User Password and Confirm Password Set to your security requirements. Both the ClickHouse User Password and Confirm Password must match.
  3. The next page is High Availability Configuration. This is where you can set your replication, Zookeeper, and backup options. Use the following values for your first cluster, then click Next to continue:

    Setting Value
    Data Replication Set to Enabled.
    Data replication duplicates data across replica clusters for increased performance and availability.
    Number of Replicas Set to 2.
    Only required if Data Replication is Enabled.
    Sets the number of replicas for each cluster shard.
    Zookeeper Configuration The only option at this time is Dedicated
    Apache Zookeeper manages synchronization between the clusters.
    Zookeeper Node Type Default is selected by default.
    Enable Backups Set to Enabled by default and cannot be disabled as of this time.
    Backup Schedule and Number of Backups to keep Is set to Daily and 5, and can not be changed as of this time.
  4. The Connection Configuration page determines how to communicate with your new cluster. Set the following values, then select Next to continue:

    Setting Value
    Endpoint This is automatically set based on your cluster name. It will display the final DNS name for your cluster end point.
    Use TLS Set to Enabled.
    When enabled, communications with your cluster are encrypted with TLS.
    Load Balancer Type Select Altinity Edge Ingress.
    IMPORTANT NOTE: This setting requires clients to support SNI (Server Name Indication). This will require the most current ClickHouse client and Python clickhouse-driver.
    This setting cannot be changed after the cluster is created.
    1. Protocols can restrict communications to the Altinity.Cloud cluster based on your organizations needs. By default **Binary Protocol (port:9440) and HTTP Protocol (port: 8443) are enabled.
    2. Datadog integration: Not enabled at this time. Stay tuned for future developments.
    3. IP restrictions: Restrict IP communications to the cluster to specific IP addresses. For more information, see the Administrator Guide. Leave blank for now.
  5. Last page! Review & Launch lets you double check your settings and see the estimated cluster cost. When you’re ready, click Launch.

It will take a little while before your new cluster is ready, so grab your coffee or tea or other hydrating beverage. When it’s complete, you’ll see your new cluster with all nodes online and all health checks passed.

2.4 - Your First Queries

How to use Cluster Explore to run queries, view table structures and available processes.

Once your cluster is created, time to create some tables and do some queries.

For those experienced with ClickHouse, this will be very familiar. For people who are new to ClickHouse, creating tables is very similar to other SQL based databases, with some extra syntax that defines the type of table we’re making. This is part of what gives ClickHouse its speed. For complete details on ClickHouse commands, see the ClickHouse SQL Reference Guide.

Cluster Explore

The Cluster Explore page allows you to run queries, view the schema, and check on processes for your cluster. It’s a quick way of getting into your ClickHouse database, run commands and view your schema straight from your web browser. We’ll be using this to generate our first tables and input some data.

To access Cluster Explore for your cluster, just click Explore for the specific cluster to manage.

For our example, we’re going to create two tables:

  • events_local: This table will use the ReplicatedMergeTree table engine. If you don’t know about table engines, don’t worry about that for now. See the ClickHouse Engines page for complete information.
  • events: This table will be distributed on your cluster with the Distributed table engine.

In our examples, we’ll be using macro variables - these are placed between curly brackets and let us use the same SQL commands on different clusters and environments without having to fill in every detail. Any time you see an entry like {cluster} or {shard} you should recognize those as a macro variable.

The commands below will create these tables into the default database on your cluster.

Create Tables

To create your first tables:

  1. From the Clusters View select Explore for the cluster to manage.

  2. The Query tab is selected by default. (This may change in future releases.)

  3. For our first table, copy and paste the following into the Query window, then select Execute.

    CREATE TABLE IF NOT EXISTS events_local ON CLUSTER '{cluster}' (
        event_date  Date,
        event_type  Int32,
        article_id  Int32,
        title       String
    ) ENGINE = ReplicatedMergeTree('/clickhouse/{cluster}/tables/{shard}/{database}/{table}', '{replica}')
        PARTITION BY toYYYYMM(event_date)
        ORDER BY (event_type, article_id);
    

    You should see the following under Execute confirming the command ran, just replacing docdemo with your cluster:

    docdemo.demo.beta.altinity.cloud:8443 (query time: 0.342s)
    chi-docdemo-docdemo-0-0	9000 0	 	1	0
    chi-docdemo-docdemo-0-1	9000 0	 	0	0
    
  4. Now let’s create our second table. Back in the Query window, enter the following and select Execute:

    CREATE TABLE events ON CLUSTER '{cluster}' AS events_local
        ENGINE = Distributed('{cluster}', default, events_local, rand())
    

    Once again, you should see confirmation under Execute:

    docdemo.demo.beta.altinity.cloud:8443 (query time: 0.162s)
    chi-docdemo-docdemo-0-0	9000 0	 	1	0
    chi-docdemo-docdemo-0-1	9000 0	 	0	0
    
  5. Now that we have some tables, let’s not leave them empty. Inserting data into a ClickHouse table is very similar to most SQL systems. Let’s Insert our data, then do a quick Select on it. Enter the following Insert command into Query, then select Execute:

    INSERT INTO events VALUES(today(), 1, 13, 'Example');
    

    You’ll see the results confirmed under Execute, just like before.

    OK.
    

    Then enter the following Select command and select Execute again:

    SELECT * FROM events;
    

    The results will look like the following:

    docdemo.demo.beta.altinity.cloud:8443 (query time: 0.018s)
    ┌─event_date─┬─event_type─┬─article_id─┬─title───┐
    │ 2020-11-19 │          113 │ Example │
    └────────────┴────────────┴────────────┴─────────┘
    

View Schema

The Database Schema shows a graphical view of your cluster’s database, the tables in it, and their structure.

To view your Schema:

  1. From the Clusters View select Explore for the cluster to manage.
  2. Select Schema.

You can expand the databases to display the tables in each database, or select the table to view its details, schema, and some sample rows.

View Processes

To view current actions running on your cluster select Processes. This displays what processes are currently running, what user account they are running under, and allows you to view more details regarding the process.

2.5 - Connect Remote Clients

How to install the command line and Python clients, and connect to your Altinity.Cloud ClickHouse Cluster.

Now that we’ve shown how to create a cluster and use ClickHouse SQL queries into your new cluster, let’s connect to it remotely.

For the following, we’re going to be using the clickhouse-client program, but the same process will help you gain access from your favorite client.

Full instructions for installing ClickHouse can be found on the ClickHouse Installation page. We’ll keep this simple and assume you’re using a Linux environment like Ubuntu. For this example, we set up a virtual machine running Ubuntu 20.04.

First, we need to know our connection details for our Altinity.Cloud ClickHouse cluster. To view your connection details:

  1. From the Clusters View, select Connection Details for the cluster to manage.

  2. From here, you can copy and paste the settings for the ClickHouse client from your cluster’s Connection Details. For example:

    clickhouse-client -h yourdataset.yourcluster.altinity.cloud --port 9440 -s --user=admin --password
    

ClickHouse Client

The ClickHouse Client is a command line based program that will be familiar to SQL based users.

Setting Up ClickHouse Client in Linux

If you’ve already set up ClickHouse client, then you can skip this step. These instructions are modified from the Altinity Stable Builds Quick Start Guides to quickly get a ClickHouse client running on your system.

Deb Linux Based Installs

For those who need quick instructions on how to install ClickHouse client in their deb based Linux environment (like Ubuntu), use the following:

  1. Update the apt-get local repository:

    sudo apt-get update
    
  2. Install the Altinity package signing keys:

    sudo sh -c 'mkdir -p /usr/share/keyrings && curl -s https://builds.altinity.cloud/apt-repo/pubkey.gpg | gpg --dearmor > /usr/share/keyrings/altinity-dev-archive-keyring.gpg'
    
  3. Update the apt-get repository to include the Altinity Stable build repository with the following commands:

    sudo sh -c 'echo "deb [signed-by=/usr/share/keyrings/altinity-dev-archive-keyring.gpg] https://builds.altinity.cloud/apt-repo stable main" > /etc/apt/sources.list.d/altinity-dev.list'
    
    sudo apt-get update
    
  4. Install the most current version of the Altinity Stable Build for ClickHouse client with the following:

    sudo apt-get install clickhouse-client
    

macOS Based Installs

For macOS users, the Altinity Stable for ClickHouse client can be installed through Homebrew with the through the Altinity Homebrew Tap for ClickHouse with the following quick command:

brew install altinity/clickhouse/clickhouse

Connect With ClickHouse Client

If your ClickHouse client is ready, then you can copy and paste your connection settings into your favorite terminal program, and you’ll be connected.

clickhouse-client to Altinity.Cloud demo

ClickHouse Python Driver

Users who prefer Python can use the clickhouse-driver to connect through Python. These instructions are very minimal, and are intended just to get you working in Python with your new Altinity.Cloud cluster.

These instructions are in the bash shell and require the following be installed in your environment:

  • Python 3.7 and above

  • The Python module venv

  • git

  • IMPORTANT NOTE: Install the clickhouse-driver 0.2.0 or above which has support for Server Name Indication (SNI) when using TLS communications.

To connect with the Python clickhouse-driver:

  1. (Optional) Setup your local environment. For example:

    python3 -m venv test 
    . test/bin/activate
    
  2. Install the driver with pip3:

    pip3 install clickhouse-driver
    
  3. Start Python.

  4. Add the client and connection details. The Access Point provides the necessary information to link directly to your cluster.

    AltinityCloud Cluster Connection Details

    Import the clickhouse_driver client and enter the connection settings:

    from clickhouse_driver import Client
    client = Client('<HOSTNAME>', user='admin', password=<PASSWORD>, port=9440, secure='y', verify=False)
    
  5. Run client.execute and submit your query. Let’s just look at the tables from within Python:

    >>> client.execute('SHOW TABLES in default')
    [('events',), ('events_local',)]
    
  6. You can perform selects and inserts as you need. For example, continuing from our Your First Queries using Cluster Explore.

    >>> result = client.execute('SELECT * FROM default.events')
    >>> print(result)
    [(datetime.date(2020, 11, 23), 1, 13, 'Example')]
    

For more information see the article ClickHouse And Python: Getting To Know The ClickHouse-driver Client.

2.6 - Conclusion

Closing instructions for the Altinity.Cloud Quick Start Guide.

There are several ways of running ClickHouse to take advantage of the robust features and speed in your big data applications. Altinity.Cloud makes it easy to start up a cluster the way you need, manage it, and connect to it so you can go from concept to execution in the fastest way possible.

If you have any questions, please feel free to Contact Us at any time.

2.7 - FAQ

Frequently Asked Questions

When using Launch Cluster, I can’t Click Next or complete the process

Make sure that all of your settings are filled in. Some common gotchas are:

  • Make sure that the ClickHouse User Password field has been entered and confirmed.
  • Cluster names must follow DNS name restrictions (letters, numbers, and dashes allowed, periods and special characters are not).
  • Cluster names must start with a letter, and should be 15 characters at most.

3 - Altinity.Cloud Demo Cluster

A demo cluster to for testing ClickHouse connections and play with publicly available data.

To help new users to experience ClickHouse, and assist experience ClickHouse administrators test connections between applications and ClickHouse, Altinity has created several guides in both the Integrations section of the Altinity Documentation site, and various blog posts posted in the Altinity Resources Page.

As part of these guides, Altinity has created a public ClickHouse database hosted through Altinity.Cloud. This allows users to test business intelligence applications, understand how to construct SQL queries in a way that maximizes on the ClickHouse column based architecture, and use the understand in developing their own ClickHouse based applications and services.

The following details the connection details for the Altinity.Cloud demo ClickHouse database, and some examples of connecting to it.

Altinity.Cloud Demo Database Details

The Altinity.Cloud Demo database has the following connection information:

  • URL: github.demo.altinity.cloud
  • HTTP Protocol: HTTPS
  • HTTPS Port: 8443
  • ClickHouse Client Port: 9440
  • Database: default
  • Username: demo
  • Password: demo

Altinity.Cloud Demo Database Connection Examples

The following examples provide a quick demonstration on how to connect specific interfaces and applications to the Altinity.Cloud Demo database. For more information on creating a trial Altinity.Cloud environment for your organizations use, see the Altinity.Cloud trial registration page.

ClickHouse Client

To connect to the Altinity.Cloud Demo database with the native ClickHouse client, use the following command (note that for operating systems such as OS X, use clickhouse client instead of clickhouse-client):

clickhouse-client --host github.demo.altinity.cloud --user=demo --password=demo --port=9440 --secure

DBeaver and JDBC Connectors

For applications like Dbeaver that use JDBC drivers to connect to database, use the following settings:

  1. From the main menu, select Database->New Database Connection.
  2. From the Select your database screen, select ClickHouse.
  3. Set the following options:
    1. Tab *Main:
      1. JDBC URL: jdbc:clickhouse://github.demo.altinity.cloud:8443/default (DBeaver populates this by default based on the other settings).
      2. Host: github.demo.altinity.cloud
      3. Port: 8443
      4. Database/Schema: default
      5. Username: demo
      6. Password: demo
    2. Tab Driver Properties:
      1. ssl: true
  4. Select Finish.

Grafana

For Grafana, see the Integrating Grafana with ClickHouse guide.

Altinity.Cloud Tables

The following tables are part of the Altinity.Cloud Demo database.

ontime

The table ontime is provided from the Bureau of Transportation Statistics.

The ontime table is collected from Airline On-Time Performance Data. This information is based on flight data collected by US certified air carriers regarding departure and arrival times, flights cancelled, flights with delayed departure or arrival times, and other relevant data.

airports

The table airports is information on airports including the name of the airport, city, latitude and longitude, and other relevant information. For the Altinity.Cloud Demo database, this is often joined with the ontime table.

The information in this table is provided by OpenFlights.org, from their Airport database.

tripdata

The tripdata table is based on the New York City Taxi &amp; Limousine Commission (TLC) Trip Record Data Yellow Cab ride data. This table contains information about rides starting and ending in New York City, the data and time of pickups and dropoffs, the total amount of the fair, and other data. This was used as part of the article Tableau Tricks with ClickHouse.

github_events

The table github_events is a collection of all Github events between 2011 to 2020, comprising 3.1 billion records and 200 GB of data. This is provided from GH Archive.

References

datatype_test

The datatype_test table is a small table for testing datatypes, and does not current play into current demonstrations.

4 - General User Guide

Instructions on general use of Altinity.Cloud

Altinity.Cloud is made to be both convenient and powerful for ClickHouse users. Whether you’re a ClickHouse administrator or a developer, these are the concepts and procedures common to both.

4.1 - How to Create an Account

Creating your Altinity.Cloud account.

To create an Altinity.Cloud account, visit the Altinity.Cloud info page and select Free Trial. Fill in your contact information, and our staff will reach out to you to create a test account.

If you’re ready to upgrade to a full production account, talk to one of our consultants by filling out your contact information on our Consultation Request page.

4.2 - How to Login

Login to Altinity.Cloud

Altinity.Cloud provides the following methods to login to your account:

  • Username and Password
  • Auth0

Login with Username and Password

To login to Altinity.Cloud with your Username and Password:

  1. Open the Altinity.Cloud website.
  2. Enter your Email Address registered to your Altinity.Cloud account.
  3. Enter your Password.
  4. Click Sign In.

Once authenticated, you will be logged into Altinity.Cloud.

Login with Auth0

Auth0 allows you to use your existing Altinity account using trust authentication platforms such as Google to verify your identity.

  • IMPORTANT NOTE: This requires that your Altinity.Cloud account matches the authentication platform you are using. For example, if your email address in Altinity.Cloud is listed as Nancy.Doe@gmail.com, your Gmail address must also be Nancy.Doe@gmail.com.

To login using Auth0:

  1. Open the Altinity.Cloud website.
  2. Select Auth0.
  3. Select which authentication platform to use from the list (for example: Google).
    1. If this is your first time using Auth0, select which account to use. You must be already logged into the authentication platform
  4. You will be automatically logged into Altinity.Cloud.

4.3 - How to Logout

Logout of Altinity.Cloud

To logout:

  1. Select your profile icon in the upper right hand corner.
  2. Select Log out.

Your session will be ended, and you will have to authenticate again to log back into Altinity.Cloud.

4.4 - Account Settings

Account and profile settings.

Access My Account

To access your account profile:

  1. Select your user profile in the upper right hand corner.

  2. Select My Account.

    Access user account

My Account Settings

From the My Account page the following settings can be viewed:

  • Common Information. From here you can update or view the following:
    • Email Address View Only: Your email address or login
    • Password settings.
    • Dark Mode: Set the user interface to either the usual or darker interface.
  • API Access: The security access rights assigned to this account.
  • Access Rights: What security related actions this account can perform.

Update Password

To update your account password:

  1. Click your profile icon in the upper right hand corner.

  2. Select My Account.

  3. In the Common Information tab, enter the new password in the Password field.

  4. Select Save.

    Altinity Cloud user common settings

API Access Settings

Accounts can make calls to Altinity.Cloud through the API address at https://acm.altinity.cloud/api, and the Swagger API definition file is available at https://acm.altinity.cloud/api/reference.json.

Access is controlled through API access keys and API Allow Domains.

API Access Keys

Accounts can use this page to generate one or more API keys that can be used without exposing the accounts username and password. They allow API calls made to Altinity.Cloud to be made by the same Account as the keys were generated for.

When an Altinity.Cloud API key is generated, an expiration date is set for the key. By default, the expiration date is set 24 hours after the key is generated, with the date and time set to GMT. This date can be manually adjusted to allow the expiration date to make the API key invalid at the date of your choosing.

Create Altinity.Cloud API Key

To generate a new API key:

  1. Click your profile icon in the upper right hand corner.
  2. Select My Account.
  3. In the API Access tab, select + Add Key. The key will be available for use with the Altinity.Cloud API.

To change the expiration date of an API key:

  1. Click your profile icon in the upper right hand corner.
  2. Select My Account.
  3. In the API Access tab, update the date and time for the API key being modified. Note that the date and time are in GMT (Greenwich Mean Time).

To remove an API key:

  1. Click your profile icon in the upper right hand corner.
  2. Select My Account.
  3. In the API Access tab, select the trashcan icon next to the API key to delete. The key will no longer be allowed to connect to the Altinity.Cloud API for this account.

API Allow Domains

API submissions can be restricted by the source domain address. This provides enhanced security by keeping API communications only between authorized sources.

To update the list of domains this account can submit API commands from:

  1. Click your profile icon in the upper right hand corner.
  2. Select My Account.
  3. In the API Access tab, list each URL this account can submit API commands from. Each URl is a separate line.
  4. Click Save to update the account settings.
Altinity Cloud user common settings

Access Rights

The Access Rights page displays which permissions your account has. These are listed in three columns:

  • Section: The area of access within Altinity.Cloud, such as Accounts, Environments, and Console.
  • Action: What actions the access right rule allows within the section. Actions marked as * include all actions within the section.
  • Rule: Whether the Action in the Section is Allow (marked with a check mark), or Deny (marked with an X).

4.5 - Notifications

Notifications critical to your Altinity.Cloud account.

Notifications allow you to see any messages related to your Altinity.Cloud account. For example: billing, service issues, etc.

To access your notifications:

  1. From the upper right corner of the top navigation bar, select your user ID, then Notifications.

    Access notifications

Notifications History

The Notifications History page shows the notifications for your account, including the following:

  • Message: The notifications message.
  • Level: The priority level which can be:
    • Danger: Critical notifications that can effect your clusters or account.
    • Warning: Notifications of possible issues that are less than critical.
    • News: Notifications of general news and updates in Altinity.Cloud.
    • Info: Updates for general information.

4.6 - Billing

Managing billing for Altinity.Cloud.

Accounts with the role orgadmin are able to access the Billing page for their organizations.

To access the Billing page:

  1. Login to your Altinity.Cloud with an account with the orgadmin role.
  2. From the upper right hand corner, select the Account icon, and select Billing.
Access Billing

From the billing page, the following Usage Summary and the Billing Summary are available for the environments connected to the account.

Billing page

Usage Summary

The Usage Summary displays the following:

  • Current Period: The current billing month displaying the following:
    • Current Spend: The current total value of charges for Altinity.Cloud services.
    • Avg. Daily Spend: The average cost of Altinity.Cloud services per day.
    • Est. Monthly BIll: The total estimated value for the current period based on Current Spend and if usage continues at the current rate.
  • Usage for Period: Select the billing period to display.
  • Environment: Select the environment or All environments to display billing costs for. Each environment, its usage, and cost will be displayed with the total cost.

Billing Summary

The Billing Summary section displays the payment method, service address, and email address used for billing purposes. Each of these settings can be changed as required.

4.7 - System Status

View the status of Altinity.Cloud services.

The System Status page provides a quick view of whether the Altinity.Cloud services are currently up or down. This provides a quick glance to help devops staff determine where any issues may be when communicating with their Altinity.Cloud clusters.

To access tne System Status page:

  1. Login to your Altinity.Cloud account.

  2. From the upper right hand corner, select the Account icon, and select System Status.

    Access user account

System Status Page

The System Status page displays the status of the Altinity.Cloud services. To send a message to Altinity.Cloud support representatives, select Get in touch.

From the page the following information is displayed:

Altinity.Cloud system statut page

This sample is from a staging environment and cluster that was stopped and started to demonstrate how the uptime tracking system works.

  • Whether all Altinity.Cloud services are online or if there are any issues.
  • The status of services by product, with the uptime of the last 60 days shown as either green (the service was fully available that day), or red (the service suffered an issue). Hovering over a red bar will display how long the service was unavailable for the following services:
    • ClickHouse clusters
    • Ingress
    • Management Console

Enter your email at the bottom of the page in the section marked Subscribe to status updates to receive notifications via email regarding any issues with Altinity.Cloud services.

4.8 - Clusters View

Overview of the Clusters View

The Clusters View page allows you to view available clusters and access your profile settings.

To access the Clusters View page while logged in to Altinity.Cloud, click Altinity Cloud Manager.

The Clusters View page is separated into the following sections:

  • A: Cluster Creation: For more information on how to create new clusters, see the Administrator Guide.
  • B: Clusters: Each cluster associated with your Altinity.Cloud account is listed in either tile format, or as a short list.
  • C: User Management:
    • Change which environment clusters are on display.
    • Access your Account Settings.
Clusters View

Organizational Admins have additional options in the left navigation panel that allows them to select the Accounts, Environments, and Clusters connected to the organization’s Altinity.Cloud account.

Change Environment

Accounts who are assigned to multiple Altinity.Cloud environments can select which environment’s clusters they are viewing. To change your current environment:

  1. Click the environment dropdown in the upper right hand corner, next to your user profile icon.
  2. Select the environment to use. You will automatically view that environment’s clusters.
Change Environment

Manage Environments

Accounts that have permission to manage environments access them through the following process:

  1. Select the Settings icon in the upper right hand corner.
  2. Select Environments.
Manage Environments

For more information on managing environments, see the Administrator Guide.

Access Settings

For information about access your account profile and settings, see Account Settings.

Cluster Access

For details on how to launch and manage clusters, see the Administrator Guide for Clusters.

4.9 - Uptime Schedules

Uptime Schedule Overview

Uptime schedules allow users to shut off ClickHouse servers automatically on a schedule or after a period of inactivity. It’s an easy way to reduce the bill for servers that are used intermittently. When the server is shut down, Altinity.Cloud does not bill you for server compute resources or support. Storage (including backups) is still billed.

All users who have access to the cluster dashboard may access uptime schedules.

Important note! Uptime schedules are intended for ClickHouse clusters that only operate intermittently. Do not enable them for production clusters that operate continuously. Doing so may cause the ClickHouse server to stop unexpectedly, creating a potential service outage.

Accessing Uptime Schedules

To set up an uptime schedule open the cluster dashboard and select Uptime Schedule from the CONFIGURE menu. The Clusters View page allows you to view available clusters and access your profile settings.

Uptime Menu

Clusters with an active uptime schedule will also have a stopwatch icon beside the cluster name on the dashboard panel. You can access the uptime schedule by clicking on the icon.

Uptime Schedule Icon

If there is no icon, it means that no uptime schedule is currently active.

Regardless of how you access the uptime schedule, you will see a popup window that shows the current status. The default setting is ALWAYS ON, which means the cluster operates continuously.

Default Uptime Schedule

Stopping Servers after a Period of Inactivity

Select STOP WHEN INACTIVE to configure the cluster to stop after a period of inactivity. For example, you may stop clusters after two hours of inactivity. This is a useful setting for development servers. Choose your desired inactivity period. Press CONFIRM to save the setting.

Inactive Uptime Schedule

Once the server becomes inactive, you must restart it manually using the ACTIONS->RESUME on the cluster dashboard.

Stopping Clusters on a Schedule

You may also configure clusters to run on a schedule using the ON SCHEDULE tab. The following example shows how to configure a server to run 24 hours on Monday, Wednesday, Friday and 8am to 5pm on Tuesday, Thursday. Note that the time zone is GMT, so you’ll need to convert your local time zone to GMT values. Press CONFIRM to save the schedule.

Calendar Uptime Schedule

4.10 - Cluster Explore Guide

How to explore a Cluster through queries, schema and processes

Altinity.Cloud users a range of options they can take on existing clusters.

For a quick view on how to create a cluster, see the Altinity.Cloud Quick Start Guide. For more details on interacting with clusters, see the Administrative Clusters Guide.

4.10.1 - Query Tool

How to submit ClickHouse queries to a cluster or nodes of the cluster

The Query Tool page allows users to submit ClickHouse SQL queries directly to the cluster or a specific cluster node.

To use the Query Tool:

  1. Select Explore from either the Clusters View or the Clusters Detail Page.

  2. Select Query from the top tab. This is the default view for the Explore page.

  3. Select from the following:

    Query Page
    1. Select which cluster to run a query against.

    2. Select Run DDLs ON CLUSTER to run Distributed DDL Queries.

    3. Select the following node options:

      Select node for query.
      1. Any: Any node selected from the Zookeeper parameters.
      2. All: Run the query against all nodes in the cluster.
      3. Node: Select a specific node to run the query against.
    4. The Query History allows you to scroll through queries that have been executed.

    5. Enter the query in the Query Textbox. For more information on ClickHouse SQL queries, see the SQL Reference page on ClickHouse.tech.

    6. Select Execute to submit the query from the Query Textbox.

    7. The results of the query will be displayed below the Execute button.

Additional tips and examples are listed on the Query page.

4.10.2 - Schema View

Viewing the database schema for clusters and nodes.

The Schema page allows you to view the databases, tables, and other details.

To access the Schema page:

  1. Select Explore from either the Clusters View or the Clusters Detail Page.

  2. Select Schema from the top tab.

  3. Select the following node options:

    Select node for query.
    1. Any: Any node selected from the Zookeeper parameters.
    2. All: Run the query against all nodes in the cluster.
    3. Node: Select a specific node to run the query against.

To view details on a table, select the table name. The following details are displayed:

  • Table Description: Details on the table’s database, engine, and other details.
  • Table Schema: The CREATE TABLE command used to generate the table.
  • Sample Rows: A display of 5 selected rows from the table to give an example of the data contents.

4.10.3 - Processes

How to view the processes for a cluster or node.

The Processes page displays the currently running processes on a cluster or node.

To view the processes page:

  1. Select Explore from either the Clusters View or the Clusters Detail Page.

  2. Select Processes from the top tab.

  3. Select the following node options:

    Select node for query.
    1. Any: Any node selected from the Zookeeper parameters.
    2. All: Run the query against all nodes in the cluster.
    3. Node: Select a specific node to run the query against.

The following information is displayed:

  • Query ID: The ClickHouse ID of the query.
  • Query: The ClickHouse query that the process is running.
  • Time: The elapsed time of the process.
  • User: The ClickHouse user running the process.
  • Client Address: The address of the client submitting the process.
  • Action: Stop or restart a process.

5 - Administrator Guide

How to manage Altinity.Cloud.

Altinity.Cloud allows administrators to manage clusters, users, and keep control of their ClickHouse environments with a few clicks. Monitoring tools are provided so you can keep track of everything in your environment to keep on top of your business.

5.1 - Clusters

How to launch clusters and manage clusters.

ClickHouse databases are managed through clusters, which harness the power of distributed processing to quickly deliver results on even the most complex and data intensive queries.

Altinity.Cloud users can create their own ClickHouse Clusters tailored to their organization’s needs.

5.1.1 - View Cluster Details

How to view details of a running cluster and its nodes.

Cluster Details Page

Once a cluster has been launched, its current operating details can be viewed by selecting the cluster from the Clusters View. This displays the Cluster Details page. From the Cluster Dashboard page, select Nodes to view the Nodes Summary page.

From the Cluster Details page, users can perform the following:

Cluster Details Page
  • A: Manage the cluster’s:
    • Actions
    • Configuration
    • Tables and structure with Explore
    • Alerts
    • Logs
  • B: Check Cluster Health.
  • C: View the cluster’s Access Point.
  • D: Monitor the Cluster and its Queries.
  • E: View summary details for the Cluster or Node. Select Nodes to view details on the cluster’s Nodes.

Nodes Summary

The Nodes Summary Page displays all nodes that are part of the selected cluster. From this page the following options and information is available:

  1. The Node Summary that lists:
    1. Endpoint: The connection settings for this node. See Node Connection.
    2. Details and Node View: Links to the Node Dashboard and Node Metrics.
    3. Version: The ClickHouse version running on this node.
    4. Type: The processor setting for the node.
    5. Node Storage: Storage space in GB available.
    6. Memory: RAM memory allocated for the node.
    7. Availability Zone: Which AWS Availability Zone the node is hosted on.
Node Summary Page

Node Connection

The Node Connection Details shows how to connect from various clients, including the clickhouse-client, JDBC drivers, HTTPS, and Python. Unlike the Cluster Access Point, this allows a connection directly to the specific node.

Node Connection

Node Dashboard

From the Node Dashboard Page users can:

Node Dashboard
  • A: Manage the node’s:
    • Tables and structure with Explore
    • Logs
  • B: Check the node’s health.
  • C: View summary details node, it’s Metrics and its Schema.
  • D: Perform Node Actions.

Node Metrics

Node Metrics provides a breakdown of the node’s performance, such as CPU data, active threads, etc.

Node Schema

The Node Schema provides a view of the databases’ schema and tables. For more information on how to interact with a Node by submitting queries, viewing the schema of its databases and tables, and viewing process, see the Cluster Explore Guide.

5.1.2 - Cluster Actions

Actions that can be taken on launched clusters.

Launched clusters can be have different actions applied to them based on your needs.

5.1.2.1 - Upgrade Cluster

How to upgrade an existing cluster.

Clusters can be upgraded to versions of ClickHouse other than the one your cluster is running.

When upgrading to a ClickHouse Altinity Stable Build, review the release notes for the version that you are upgrading to.

How to Upgrade an Altinity Cloud Cluster

To upgrade a launched cluster:

  1. Select Actions for the cluster to upgrade.

  2. Select Upgrade.

  3. Select the ClickHouse version to upgrade to.

  4. Select Upgrade to start the process.

    Cluster Upgrade

The upgrade process completion time varies with the size of the cluster, as each server is upgraded individually. This may cause downtime while the cluster is upgraded.

5.1.2.2 - Rescale Cluster

How to rescale an existing cluster.

The size and structure of the cluster may need to be altered after launching based on your organization’s needs. The following settings can be rescaled:

  • Number of Shards
  • Number of Replicas
  • Node Type
  • Node Storage
  • Number of Volumes
  • Apply to new nodes only: This setting will only effect nodes created from this point forward.

See Cluster Settings for more information.

How to Rescale a Cluster

To rescale a cluster:

  1. Select Actions for the cluster to rescale.

  2. Select Rescale.

  3. Set the new values of the cluster.

  4. Click OK to begin rescaling.

    Cluster Rescale

Depending on the size of the cluster, this may take several minutes.

5.1.2.3 - Stop and Start a Cluster

How to stop or start an existing cluster.

To stop an launched cluster, or start a stopped cluster:

  1. From either the Clusters View or the Cluster Details Page, select Actions.
    1. If the cluster is currently running, select Stop to halt its operations.
    2. If the cluster has been stopped, select Start to restart it.

Depending on the size of your cluster, it may take a few minutes until it is fully stopped or is restarted. To access the health and availability of the cluster, see Cluster Health or the Cluster Availability.

5.1.2.4 - Export Cluster Settings

How to export a cluster’s settings.

The structure of an Altinity Cloud cluster can be exported as JSON. For details on the cluster’s settings that are exported, see Cluster Settings.

To export a cluster’s settings to JSON:

  1. From either the Clusters View or the Cluster Details Page, select Actions, then select Export.
  2. A new browser window will open with the settings for the cluster in JSON.

5.1.2.5 - Replicate a Cluster

How to replicate an existing cluster.

Clusters can be replicated with the same or different settings. These can include the same database schema as the replicated cluster, or launched without the schema. This may be useful to create a test cluster, then launch the production cluster with different settings ready for production data.

For complete details on Altinity.Cloud clusters settings, see Cluster Settings.

To create a replica of an existing cluster:

  1. From either the Clusters View or the Cluster Details Page, select Actions, then select Launch a Replica Cluster.
  2. Enter the desired values for Resources Configuration.
    1. To replicate the schema of the source directory, select Replicate Schema.

      Replicate Schema
    2. Click Next to continue.

  3. High Availability Configuration, and Connection Configuration.
    1. Each section must be completed in its entirety before moving on to the next one.
  4. In the module Review & Launch, verify the settings are correct. When finished, select Launch.

Depending on the size of the new cluster it will be available within a few minutes. To verify the health and availability of the new cluster, see Cluster Health or the Cluster Availability.

5.1.2.6 - Destroy Cluster

How to destroy an existing cluster.

When a cluster is no longer required, the entire cluster and all of its data can be destroyed.

  • IMPORTANT NOTE: Once destroyed, a cluster can not be recovered. It must be manually recreated.

To destroy a cluster:

  1. From either the Clusters View or the Cluster Details Page, select Actions, then select Destroy.

  2. Enter the cluster name, then select OK to confirm its deletion.

    Destroy Cluster

5.1.3 - Cluster Settings

Settings and values used for Altinity.Cloud ClickHouse Clusters.

ClickHouse Clusters hosted on Altinity.Cloud have the following structural attributes. These determine options such as the version of ClickHouse installed on them, how many replicas, and other important features.

Name Description Values
Cluster Name The name for this cluster. It will be used for the hostname of the cluster. Cluster names must be DNS compliant. This includes:
  • Alphanumeric characters and underscores only
  • No special characters such as periods, ?, #, etc.
    Example:
    • Good: mycluster
    • Bad: my.cluster?
  • Can not start with a number.
Node Type Determines the number of CPUs and the amount of RAM used per node. The following Node Types are sample values, and may be updated at any time:
  • m5.large: CPU x2, RAM 6.5 GB
  • m5.xlarge: CPU x4, RAM 14 GB
  • M5.2xlarge: (CPU x8, RAM 29 GB)
  • m5.4xlarge: (CPU x16, RAM 58 GB)
  • m5.8xlarge: (CPU x32, RAM 120 GB)
Node Storage The amount of storage space available to each node, in GB.  
Number of Volumes Storage can be split across multiple volumes. The amount of data stored per node is the same as set in Node Storage, but it split into multiple volumes.
Separating storage into multiple volumes can increase query performance.
 
Volume Type Defines the Amazon Web Services volume class. Typically used to determine whether or not to encrypt the columns. Values:
  • gp2 (Not Encrypted)
  • gp2-encrypted (encrypted)
Number of Shards Shards represent a set of nodes. Shards can be replicated to provide increased availability and computational power.  
ClickHouse Version The version of the ClickHouse database that will be used on each node.
To run a custom ClickHouse container version, specify the Docker image to use.
  • IMPORTANT NOTE: The nodes in the cluster will all be running the same version of ClickHouse. If you want to run multiple versions of ClickHouse, they will have to be set on different clusters.
Currently available options:
  • Altinity Stable:
    • 19.11.12.69
    • 19.16.19.85
    • 20.3.21.2
    • 20.8.7.15
  • Standard Release
    • 20.10.5.10
    • 20.11.4.13
  • Custom Identifier
ClickHouse Admin Name The name of the ClickHouse administrative user. Set to admin by default. Can not be changed.
ClickHouse Admin Password The password for the ClickHouse administrative user.  
Data Replication Toggles whether shards will be replicated. When enabled, Zookeeper is required to manage the shard replication process. Values:
  • Enabled (Default): Each Cluster Shard will be replicated to the value set in Number of Replicas.
  • Disabled: Shards will not be replicated.
Number of Replicas Sets the number of replicas per shard. Only enabled if Data Replication is enabled.  
Zookeeper Configuration When Data Replication is set to Enabled, Zookeeper is required. This setting determines how Zookeeper will run and manage shard replication.
The Zookeeper Configuration mainly sets how many Zookeeper nodes are used to manage the shards. More Zookeeper nodes increases the availability of the cluster.
Values:
  • Single Node (Default): Replication is managed by one Zookeeper node.
  • Three Nodes: Increases the Zookeeper nodes to an ensemble of 3.
Zookeeper Node Type Determines the type of Zookeeper node. Defaults to default and can not be changed.
Node Placement Sets how nodes are distributed via Kubernetes. Depending on your situation and how robust you want your replicas and clusters. Values:
  • Separate Nodes (Default): ClickHouse containers are distributed across separate cluster nodes.
  • Separate Shards: ClickHouse containers for different shards are distributed across separate cluster nodes.
  • Separate Replicas: ClickHouse containers for different replicas are distributed across separate cluster nodes.
Enable Backups Backs up the cluster. These can be restored in the event data loss or to roll back to previous versions. Values:
  • Enabled (Default): The cluster will be backed up automatically.
  • Disabled: Automatic Backups are disabled.
Backup Schedule Determines how often the cluster will be backed up. Defaults to Daily
Number of Backups to keep Sets how many backups will be stored before deleting the oldest one Defaults to 5.
Endpoint The Access point Domain Name. This is hard set by the name of your cluster and your organization.
Use TLS Sets whether or not to encrypt external communications with the cluster to TLS. Default to Enabled and can not be changed.
Load Balancer Type The load balancer manages communications between the various nodes to ensure that nodes are not overwhelmed. Defaults to Altinity Edge Ingress
Protocols Sets the TCP ports used in external communications with the cluster. Defaults to ClickHouse TCP port 9440 and HTTP port 8443.

5.1.4 - Configure Cluster

How to configure launched clusters.

Once a cluster has been launched, it’s configuration can be updated to best match your needs.

5.1.4.1 - How to Configure Cluster Settings

How to update the cluster’s settings.

Cluster settings can be updated from the Clusters View or from the Cluster Details by selecting Configure > Settings.

  • IMPORTANT NOTE: Changing a cluster’s settings will require a restart of the entire cluster.

Note that some settings are locked - their values can not be changed from this screen.

Cluster Settings

How to Set Troubleshooting Mode

Troubleshooting mode prevents your cluster from auto-starting after a crash. To update this setting:

  1. Toggle Troubleshooting Mode either On or Off.

How to Edit an Existing Setting

To edit an existing setting:

  1. Select the menu on the left side of the setting to update.
  2. Select Edit.
  3. Set the following:
    1. Setting Type.
    2. Name
    3. Value
  4. Select OK to save the setting.
Edit Cluster Setting

How to Add a New Setting

To add a new setting to your cluster:

  1. Select Add Setting.
  2. Set the following:
    1. Setting Type.
    2. Name
    3. Value
  3. Select OK to save the setting.
Add a New Cluster Setting

How to Delete an Existing Setting

To delete an existing setting:

  1. Select the menu on the left side of the setting to update.
  2. Select OK.
  3. Select Remove to confirm removing the setting.
Delete a Cluster Setting

5.1.4.2 - How to Configure Cluster Profiles

How to update the cluster’s profiles.

Cluster profiles allow you to set the user permissions and settings based on their assigned profile.

The Cluster Profiles can be accessed from the Clusters View or from the Cluster Details by selecting Configure > Settings.

Cluster Profile Settings

Add a New Profile

To add a new cluster profile:

  1. From the Cluster Profile View page, select Add Profile.
  2. Provide profile’s Name and Description, then click OK.

Edit an Existing Profile

To edit an existing profile:

  1. Select the menu to the left of the profile to update and select Edit, or select Edit Settings.
  2. To add a profile setting, select Add Setting and enter the Name and Value, then click OK to store your setting value.
  3. To edit an existing setting, select the menu to the left of the setting to update. Update the Name and Value, then click OK to store the new value.

Delete an Existing Profile

To delete an existing profile:

  1. Select the menu to the left of the profile to update and select Delete.
  2. Select OK to confirm the profile deletion.

5.1.4.3 - How to Configure Cluster Users

How to update the cluster’s users.

The cluster’s Users allow you to set one or more entities who can access your cluster, based on their Cluster Profile.

Cluster users can be updated from the Clusters View or from the Cluster Details by selecting Configure > Users.

Cluster Users

How to Add a New User

To add a new user to your cluster:

  1. Select Add User

  2. Enter the following:

    Add New User
    1. Login: the name of the new user.
    2. Password and Confirm Password: the authenticating credentials for the user. Th
    3. Networks: The networks that the user can connect from. Leave as 0.0.0.0/0 to allow access from all networks.
    4. Databases: Which databases the user can connect to. Leave empty to allow access all databases.
    5. Profile: Which profile settings to apply to this user.
  3. Select OK to save the new user.

How to Edit a User

To edit an existing user:

  1. Select the menu to the left of the user to edit, then select Edit.
  2. Enter the following:
    1. Login: the new name of the user.
    2. Password and Confirm Password: the authenticating credentials for the user. Th
    3. Networks: The networks that the user can connect from. Leave as 0.0.0.0/0 to allow access from all networks.
    4. Databases: Which databases the user can connect to. Leave empty to allow access all databases.
    5. Profile: Which profile settings to apply to this user.
  3. Select OK to save the updated user user.

How to Delete a User

  1. Select the menu to the left of the user to edit, then select Delete.
  2. Select OK to verify the user deletion.

5.1.5 - Launch New Cluster

How to launch a new ClickHouse Cluster from Altinity.Cloud.

Launching a new ClickHouse Cluster is incredibly easy, and only takes a few minutes. For those looking to create their first ClickHouse cluster with the minimal steps, see the Quick Start Guide. For complete details on Altinity.Cloud clusters settings, see Cluster Settings.

To launch a new ClickHouse cluster:

  1. From the Clusters View page, select Launch Cluster. This starts the Cluster Launch Wizard.

    Launch New Cluster
  2. Enter the desired values for Resources Configuration, High Availability Configuration, and Connection Configuration.

    1. Each section must be completed in its entirety before moving on to the next one.
  3. In the module Review & Launch, verify the settings are correct. When finished, select Launch.

Within a few minutes, the new cluster will be ready for your use and display that all health checks have been passed.

5.1.6 - Cluster Alerts

How to be notified about cluster issues

The Cluster Alerts module allows users to set up when they are notified for a set fo events. Alerts can either be a popup, displaying the alert when the user is logged into Altinity.Cloud, or email so they can receive an alert even when they are not logged into Altinty.Cloud.

To set which alerts you receive:

  1. From the Clusters view, select the cluster to for alerts.

  2. Select Alerts.

    Cluster Alerts
  3. Add the Email address to send alerts to.

  4. Select whether to receive a Popup or Email alert for the following events:

    1. ClickHouse Version Upgrade: Alert triggered when the version of ClickHouse that is installed in the cluster has a new update.
    2. Cluster Rescale: Alert triggered when the cluster is rescaled, such as new shards added.
    3. Cluster Stop: Alert triggered when some event has caused the cluster to stop running.
    4. Cluster Resume: Alert triggered when a cluster that was stopped has resumed operations.

5.1.7 - Cluster Health Check

How to quickly check your cluster’s health.

From the Clusters View, you can see the health status of your cluster and its nodes at a glance.

How to Check Node Health

The quick health check of your cluster’s nodes is displayed from the Clusters View. Next to the cluster name is a summary of your nodes’ statuses, indicating the total number of nodes and how many nodes are available.

View the Access Point

How to Check Cluster Health

The overall health of the cluster is shown in the Health row of the cluster summary, showing the number of health checks passed.

View the Access Point

Click checked passed to view a detailed view of the cluster’s health.

How to View a Cluster’s Health Checks

The cluster’s Health Check module displays the status of the following health checks:

  • Access point availability check
  • Distributed query check
  • Zookeeper availability check
  • Zookeeper contents check
  • Readonly replica check
  • Delayed inserts check

To view details on what queries are used to verify the health check, select the caret for each health check.

Cluster Health Details

5.1.8 - Cluster Monitoring

How to monitor your clusters performance.

Altinity.Cloud integrates Grafana into its monitoring tools. From a cluster, you can quickly access the following monitoring views:

  • Cluster Metrics
  • Queries
  • Logs

How to Access Cluster Metrics

To access the metrics views for your cluster:

  1. From the Clusters view, select the cluster to monitor.
  2. From Monitoring, select the drop down View in Grafana and select from one of the following options:
    1. Cluster Metrics
    2. Queries
    3. Logs
  3. Each metric view opens in a separate tab.

Cluster Metrics

Cluster Metrics displays how the cluster is performing from a hardware and connection standpoint.

Cluster Monitoring View

Some of the metrics displayed here include:

  • DNS and Distributed Connection Errors: Displays the rate of any connection issues.
  • Select Queries: The number of select queries submitted to the cluster.
  • Zookeeper Transactions: The communications between the zookeeper nodes.
  • ClickHouse Data Size on Disk: The total amount of data the ClickHouse database is using.

Queries

The Queries monitoring page displays the performance of clusters, including the top requests, queries that require the most memory, and other benchmarks. This can be useful in identifying queries that can cause performance issues and refactoring them to be more efficient.

Query Monitoring View

Log Metrics

The Log monitoring page displays the logs for your clusters, and allows you to make queries directly on them. If there’s a specific detail you’re trying to iron out, the logs are the most granular way of tracking down those issues.

Log Monitoring View

5.1.9 - Cluster Logs

How to access your cluster’s logs

Altinity.Cloud provides the cluster log details so users can track down specific issues or performance bottlenecks.

To access a cluster’s logs:

  1. From the Clusters view, select the cluster to for alerts.
  2. Select Logs.
  3. From the Log Page, you can display the number of rows to view, or filter logs by specific text.
  4. To download the logs, select the download icon in the upper right corner (A).
  5. To refresh the logs page, select the refresh icon (B).
Cluster Logs Page

The following logs are available:

  • ACM Logs: These logs are specific to Altinity.Cloud issues and include the following:
    • System Log: Details the system actions such as starting a cluster, updating endpoints, and other details.
    • API Log: Displays updates to the API and activities.
  • ClickHouse Logs: Displays the Common Log that stores ClickHouse related events. From this view a specific host can be selected form the dropdown box.
  • Backup Logs: Displays backup events from the clickhouse-backup service. Log details per cluster host can be selected from the dropdown box.
  • Operator Logs: Displays logs from the Altinity Kubernetes Operator service, which is used to manage cluster replication cluster and communications in the Kubernetes environment.

5.2 - Access Control

How to control access to your organizations, environments, and clusters.

Altinity.Cloud provides role based access control. Depending the role granted to an Altinity.Cloud Account, they can assign other Altinity.Cloud accounts roles and grant permissions to access organizations, environments, or clusters.

5.2.1 - Role Based Access and Security Tiers

Altinity.Cloud hierarchy and role based access.

Access to ClickHouse data hosted in Altinity.Cloud is controlled through a combination of security tiers and account roles. This allows companies to tailor access to data in a way that maximizes security while still allowing ease of access.

Security Tiers

Altinity.Cloud groups sets of clusters together in ways that allows companies to provide Accounts access only to the clusters or groups of clusters that they need to.

Altinity.Cloud groups clusters into the following security related tiers:

Security Tiers
  • Nodes: The most basic level - an individual ClickHouse database and tables.
  • Clusters: These contain one or more nodes provide ClickHouse database access.
  • Environments: Environments contain one or more clusters.
  • Organizations: Organizations contain one or more environments.

Account access is controlled by assigning an account a single role and a security tier depending on their role. A single account can be assigned to multiple organizations, environments, multiple clusters in an environment, or a single cluster depending on their account role.

Account Roles

The actions that can be taken by Altinity.Cloud accounts is based on the role they are assigned. The following roles and their actions based on the security tier is detailed in the table below:

Role Environment Cluster
orgadmin Create, Edit, and Delete environments that they create, or are assigned to, within the assigned organizations.
Administrate Accounts associated with environments they are assign to.
Create, Edit, and Delete clusters within environments they create or assigned to in the organization.
envadmin Access assigned environments. Create, Edit, and Delete clusters within environments they are assigned to in the organization.
envuser Access assigned environments. Access one or more clusters the account is specifically assigned to.

The account roles are tied into the security tiers, and allow an account to access multiple environment and clusters depending on what type of tier they are assigned to.

For example, we may have the following situation:

  • Accounts peter, paul, and mary and jessica are all members of the organization HappyDragon.
  • HappyDragon has the following environments: HappyDragon_Dev and HappyDragon_Prod, each with the clusters marketing, sales, and ops.

The accounts are assigned the following roles and security tiers:

Account Role Organization Environments Clusters
mary orgadmin HappyDragon HappyDragon_Prod *
peter envadmin HappyDragon HappyDragon_Dev *
jessica envadmin HappyDragon HappyDragon_Prod, HappyDragon_Dev *
paul envuser HappyDragon HappyDragon_Prod marketing

In this scenario, mary has the ability to access the environment HappyDragon_Prod, or can create new environments and manage them and any clusters within them. However, she is not able to edit or access HappyDragon_Dev or any of its clusters.

  • Both peter and jessica have the ability to create and remove clusters within their assigned environments.
    • peter is able to modify the clusters in the environment HappyDragon_Dev.
    • jessica can modify clusters in both environments.
  • paul can only access the cluster marketing in the environment HappyDragon_Prod.

5.2.2 - Account Management

How to manage Altinity.Cloud accounts.

Altinity.Cloud accounts with the role orgadmin are able to create new Altinity.Cloud accounts and associate them with organizations, environments, and one or more clusters depending on their role. For more information on roles, see Role Based Access and Security Tiers.

Account Page

The Account Page displays all accounts assigned to the same Organization and Environments as the logged in account.

For example: the accounts mario, luigi, and peach and todd are members of the organizations MushroomFactory and BeanFactory as follows:

Account Role Organization: MushroomFactory Organization: BeanFactory
peach orgadmin *  
mario orgadmin   *
luigi envuser   *
todd envuser *  
  • peach will be able to see their account and todd in the Account Page, while accounts mario and luigi will be hidden from them.
  • mario will be able to see their account and luigi.

Access Accounts

To access the accounts that are assigned to the same Organizations and Environments as the logged in user with the account role orgadmin:

  1. Login to Altinity.Cloud with an account granted the orgadmin role.
  2. From the left navigation panel, select Accounts.
  3. All accounts that are in the same Organizations and Environments as the logged in account will be displayed.

Account Details

Accounts have the following details that can be set by an account with the orgadmin role:

  1. Common Information:
    1. Name: The name of the account.
    2. Email (Required): The email address of the account. This will be used to login, reset passwords, notifications, and other uses. This must be a working email for these functions to work.
    3. Password: The password for the account. Once a user has authenticated to the account, they can change their password.
    4. Confirm Password: Confirm the password for the account.
    5. Role (Required): The role assigned to the account. For more information on roles, see Role Based Access and Security Tiers.
    6. Organization: The organization assigned to the account. Note that the orgadmin can only assign accounts the same organizations that the orgadmin account also belongs to.
    7. Suspended: When enabled, this prevents the account from logging into Altinity.Cloud.
  2. Environment Access:
    1. Select the environments that the account will require access to. Note that the orgadmin can only assign accounts the same environments that the orgadmin account also belongs to.
  3. Cluster Access:
    1. This is only visible if the Role is set to envuser. This allows one or more clusters in the environments the new account was assigned to in Environmental Access to be accessed by them.
  4. API Access:
    1. Allows the new account to make API calls from the listed domain names.

Account Actions

Create Account

orgadmin accounts can create new accounts and assign them to the same organization and environments they are assigned to. For example, continuing the scenario from above, if account peach is assigned to the organization MushroomFactory and the environments MushroomProd and MushroomDev, they can assign new accounts to the organization MushroomFactory, and to the environments MushroomProd or MushroomDev or both.

To create a new account:

  1. Login to Altinity.Cloud with an account granted the orgadmin role.

  2. From the left navigation panel, select Accounts.

  3. Select Add Account.

  4. Set the fields as listed in the Account Details section.

    New User Settings
  5. Once all settings are completed, select Save. The account will be able to login with the username and password, or if their email address is registered through Google, Auth0.

Edit Account

  1. Login to Altinity.Cloud with an account granted the orgadmin role.
  2. From the left navigation panel, select Accounts.
  3. From the left hand side of the Accounts table, select the menu icon for the account to update and select Edit.
  4. Update the fields as listed in the Account Details section.
  5. When finished, select Save.

Suspend Account

Instead of deleting an account, setting an account to Suspended may be more efficient to preserve the accounts name and other settings. A suspended account is unable to login to Altinity.Cloud. This includes directly logging through a browser and API calls made under the account.

To suspend or activate an account:

  1. Login to Altinity.Cloud with an account granted the orgadmin role.
  2. From the left navigation panel, select Accounts.
  3. From the left hand side of the Accounts table, select the menu icon for the account to update and select Edit.
    1. To suspend an account, toggle Suspended to on.
    2. To activate a suspended account, toggle Suspended to off.
  4. When finished, select Save.

Delete Account

Accounts can be deleted which removes all information on the account. Clusters and environments created by account will remain.

To delete an existing account:

  1. Login to Altinity.Cloud with an account granted the orgadmin role.
  2. From the left navigation panel, select Accounts.
  3. From the left hand side of the Accounts table, select the menu icon for the account to update and select Delete.
  4. Verify the account is to be deleted by selecting OK.

6 - Connectivity

Connecting Altinity.Cloud with other services.

The following guides are designed to help organizations connect their existing services to Altinity.Cloud.

6.1 - Cluster Access Point

How to view your Cluster’s access information.

ClickHouse clusters created in Altinity.Cloud can be accessed through the Access Point. The Access Point is configured by the name of your cluster and environment it is hosted in.

Information on the Access Point is displayed from the Clusters View. Clusters with TLS Enabled will display a green shield icon.

View Cluster Access Point

To view your cluster’s access point:

  1. From the Clusters View, select Access Point.
  2. The Access Point details will be displayed.
View the Access Point

Access Point Details

The Access Point module displays the following details:

  • Host: The dns host name of the cluster, based on the name of the cluster and environment the cluster is hosted in.

  • TCP Port: The ClickHouse TCP port for the cluster.

  • HTTP Point: The HTTP port used for the cluster.

  • Client Connections: The client connections are quick commands you can copy and paste into your terminal or use in your code. This make it a snap to have your code connecting to your cluster by copying the details right from your cluster’s Access Point. Provided client includes are:

    • clickhouse-client
    • jdbc
    • https
    • python

6.2 - Configure Cluster Connections

Configure the connection protocols to your Altinity.Cloud cluster

Altinity.Cloud provides accounts the ability to customize their connections to their clusters. This allows organizations the ability to enable or disable:

  • The Binary Protocol: The native ClickHouse client secure port on port 9440.
  • The HTTP Protocol: The HTTPS protocol on port 8443.
  • IP Restrictions: Restricts ClickHouse client connections to the provided whitelist. The IP addresses must be listed in CIDR format. For example, ip_address1,ip_address2,etc.

As of this time, accounts can update the IP Restrictions section. Binary Protocol and HTTP Protocol are enabled by default and can not be disabled.

Update Connection Configuration

To update the cluster’s Connection Configuration:

  1. Log into Altinity.Cloud with an account.

  2. Select the cluster to update.

  3. From the top menu, select Configure->Connections.

    Select Configure->Connections for the cluster.
  4. To restrict IP communication only to a set of whitelisted IP addresses:

    1. Under IP Restrictions, select Enabled.

    2. Enter a list of IP addresses. These can be separated by comma, spaces, or a new line. The following examples are all equivalent:

      192.168.1.1,192.168.1.2
      
      192.168.1.1
      192.168.1.2
      
      192.168.1.1 192.168.1.2
      
  5. When finished, select Confirm to save the Connection Configuration settings.

    Cluster Connection Configuration Settings

6.3 - Connecting with DBeaver

Creating a connection to Altinity.Cloud from DBeaver.

Connecting to Altinity.Cloud from DBeaver is a quick, secure process thanks to the available JDBC driver plugin.

Required Settings

The following settings are required for the driver connection:

  • hostname: The DNS name of the Altinity.Cloud cluster. This is typically based on the name of your cluster, environment, and organization. For example, if the organization name is CameraCo and the environment is prod with the cluster sales, then the URL may be https://sales.prod.cameraco.altinity.cloud. Check the cluster’s Access Point to verify the DNS name of the cluster.
  • port: The port to connect to. For Altinity.Cloud, it will be HTTPS on port 8443.
  • Username: The ClickHouse user to authenticate to the ClickHouse server.
  • Password: The ClickHouse user password used to authenticate to the ClickHouse server.

Example

The following example is based on connecting to the Altinity.Cloud public demo database, with the following settings:

  • Server: github.demo.trial.altinity.cloud
  • Port: 8443
  • Database: default
  • Username: demo
  • Password: demo
  • Secure: yes

DBeaver Example

  1. Start DBeaver and select Database->New Database Connection.

    Create Database Connection
  2. Select All, then in the search bar enter ClickHouse.

  3. Select the ClickHouse icon in the “Connect to a database” screen.

    Select ClickHouse JDBC Driver
  4. Enter the following settings:

    1. Host: github.demo.trial.altinity.cloud
    2. Port: 8443
    3. Database: default
    4. User: demo
    5. Password: demo
    Connection details.
  5. Select the Driver Properties tab. If prompted, download the ClickHouse JDBC driver.

  6. Scroll down to the ssl property. Change the value to true.

    Set secure.
  7. Press the Test Connection button. You should see a successful connection message.

    Successful Test.

6.4 - clickhouse-client

How to install and connect to an Altinity.Cloud cluster with clickhouse-client.

The ClickHouse Client is a command line based program that will be familiar to SQL based users. For more information on clickhouse-client, see the ClickHouse Documentation Command-Line Client page.

The access points for your Altinity.Cloud ClickHouse cluster can be viewed through the Cluster Access Point.

How to Setup clickhouse-client for Altinity.Cloud in Linux

As of this document’s publication, version 20.13 and above of the ClickHouse client is required to connect with the SNI enabled clusters. These instructions use the testing version of that client. An updated official stable build is expected to be released soon.

sudo apt-get install apt-transport-https ca-certificates dirmngr
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv E0C56BD4

echo "deb https://repo.clickhouse.tech/deb/testing/ main/" | sudo tee \
    /etc/apt/sources.list.d/clickhouse.list
sudo apt-get update

sudo apt-get install -y clickhouse-client

Connect With clickhouse-client to a Altinity.Cloud Cluster

If your ClickHouse client is ready, then you can copy and paste your connection settings into your favorite terminal program, and you’ll be connected.

clickhouse-client to Altinity.Cloud demo

6.5 - Amazon VPC Endpoint

How to create an Amazon VPC Endpoint for Altinity.Cloud Services

Altinity.Cloud users can connect a VPC (Virtual Private Cloud) Endpoint from existing AWS environments to their Altinity.Cloud environment. The VPC Endpoint becomes a private connection between their existing Amazon services and Altinity.Cloud, without exposing the connection to the Internet.

The following instructions are based on using the AWS console. Examples of the Terraform equivalent settings are included.

Requirements

Altinity.Cloud requires the AWS ID for the account that will be linked to the Altinity.Cloud environment. This can be found when you login to your AWS Console, and select your username from the upper right hand corner:

Create Endpoint Details

Instructions

To create an VPC Endpoint, the following general steps are required:

  • Retrieve Your Altinity.Cloud Environment URL.
  • Request an Endpoint Service Name from Altinity.Cloud.
  • Create a VPC Endpoint. This must be in the same region as the service to be connected to.
  • Create a private Route 53 Hosted Zone to internal.{Altinity.Cloud environment name}.altinity.cloud.
  • Create a CNAME that points to the VPC Endpoint.

Retrieve Your Altinity.Cloud Environment URL

Your AWS service will be connected to the URL for your Altinity.Cloud environment. Typically this will be the name of your environment, followed by internal.{Altinity.Cloud environment name}.altinity.cloud. For example: if your environment is named trafficanalysis, then your environment URL will be internal.trafficanalysis.altinity.cloud.

This may differ depending on your type of service. If you have any questions, please contact your Altinity Support representative.

Request an Endpoint Service Name

Before creating a VPC Endpoint, Altinity.Cloud will need to provide you a AWS Service Name that will be used for your Endpoint. To request your AWS Service Name to use in later steps of creating the VPC Endpoint to Altinity.Cloud:

  1. Login to your AWS console and retrieve your AWS ID.

    Create Endpoint Details
  2. Contact your Altinity.Cloud support representative and inform them that you want to set up a VPC Endpoint to your Altinity.Cloud environment. They will require your AWS ID.

  3. Your Altinity.Cloud support representative will process your request, and return your AWS Service Name to you. Store this in a secure location for your records.

Create a VPC Endpoint

The next step in connecting Altinity.Cloud to the existing AWS Service is to create an Endpoint.

  1. From the AWS Virtual Private Cloud console, select Endpoints > Create Endpoint.

    Select Create Endpoint
  2. Set the following:

    1. Service Category: Set to Find service by name. (1)
    2. Service Name: Enter the Service Name (2) provided in the step Create Service Name, then select Verify. (3)
    Create Endpoint Details
  3. Select the VPC from the dropdown.

  4. Select Create Endpoint.

Terraform VPC Endpoint Configuration

resource "aws_vpc_endpoint" "this" {
    service_name = local.service_name,
    vpc_endpoint_type = "Interface",
    vpc_id = aws_vpc.this.id,
    subnet_ids = [aws_subnet.this.id],
    security_group_ids  = [aws_vpc.this.default_security_group_id],
    private_dns_enabled = false,
    tags = local.tags
}

Create Route 53 Hosted Zone

To create the Route 53 Hosted Zone for the newly created endpoint:

  1. From the AWS Console, select Endpoints.

  2. Select the Endpoint to connect to Altinity.Cloud, then the tab Details. In the section marked DNS names, select the DNS entry created and copy it. Store this in a separate location until ready.

    Copy Endpoint DNS Entry
  3. Enter the Route 53 console, and select Hosted zones.

    Select Create hosted zone
  4. Select Create hosted zone.

  5. On the Hosted zone configuration page, update the following:

    1. Domain name: Enter the URL of the Altinity.Cloud environment. Recall this will be internal.{Altinity.Cloud environment name}.altinity.cloud, where {your environment name} was determined in the step Retrieve Your Altinity.Cloud Environment URL.
    2. Description (optional): Enter a description of the hosted zone.
    3. Type: Set to Private hosted zone.
    Create hosted zone details
  6. In VPCs to associated with the hosted zone, set the following:

    1. Region: Select the region for the VPC to use.
    2. VPC ID: Enter the ID of the VPC that is being used.
  7. Verify the information is correct, then select Create hosted zone.

    Create hosted zone

Terraform Route 53 Configuration

resource "aws_route53_zone" "this" {
    name  = "$internal.{environment_name}.altinity.cloud.",
    vpc {
        vpc_id = aws_vpc.this.id
    }
    tags = local.tags
}

Create CNAME for VPC Endpoint

Once the Hosted Zone that will be used to connect the VPC to Altinity.Cloud has been created, the CNAME for the VPC Endpoint can be configured through the following process:

  1. From the AWS Console, select Route 53 > Hosted Zones, then select Create record.

    Create hosted zone
  2. Select the Hosted Zone that will be used for the VPC connection. This will be the internal.{Altinity.Cloud environment name}.altinity.cloud.

  3. Select Create record.

  4. From Choose routing policy select Simple routing, then select Next.

    Choose routing policy
  5. From Configure records, select Define simple record.

    Select Define simple record
  6. From Define simple record, update the following:

    1. Record name: set to *. (1)
    2. Value/Route traffic to:
      1. Select Ip address or another value depending on the record type. (3)
      2. Enter the DNS name for the Endpoint created in Create Route 53 Hosted Zone.
    3. Record type
      1. Select CNAME (2).
    Define simple record
  7. Verify the information is correct, then select Define simple record.

Terraform CNAME Configuration

resource "aws_route53_record" "this" {
    zone_id = aws_route53_zone.this.zone_id,
    name = "*.${aws_route53_zone.this.name}",
    type = "CNAME",
    ttl = 300,
    records = [aws_vpc_endpoint.this.dns_entry[0]["dns_name"]]
}

Test

To verify the VPC Endpoint works, launch a EC2 instance and execute the following curl command, and will return OK if successful. Use the name of your Altinity.Cloud environment’s host name in place of {your environment name here}:

curl -sS https://statuscheck.{your environment name here}
OK

For example, if your environment is internal.trafficanalysis.altinity.cloud, then use:

curl -sS https://statuscheck.internal.trafficanalysis.altinity.cloud
OK

References

6.6 - Amazon VPC Endpoint for Amazon MSK

How to create Amazon VPC Endpoint Services to connect Altinity.Cloud to Amazon MSK within your VPC

Altinity.Cloud users can connect a VPC (Virtual Private Cloud) Endpoint service from their existing AWS (Amazon Web Services) MSK (Amazon Managed Streaming for Apache Kafka) environments to their Altinity.Cloud environment. The VPC Endpoint services become a private connection between their existing Amazon services and Altinity.Cloud, without exposing Amazon MSK to the Internet.

The following instructions are based on using the AWS console. Examples of the Terraform equivalent settings are included.

Requirements

  • Amazon MSK
  • Provision Broker mapping.

Instructions

To create an VPC Endpoint Service, the following general steps are required:

  1. Contact your Altinity Support representative to retrieve the Altinity.Cloud AWS Account ID.
  2. Create VPC Endpoint Services: For each broker in the Amazon MSK cluster, provision a VPC endpoint service in the same region your Amazon MSK cluster. For more information, see the Amazon AWS service endpoints documentation.
  3. Configure each endpoint service to a Kafka broker. For example:
    1. Endpoint Service: com.amazonaws.vpce.us-east-1.vpce-svc-aaa
    2. Kafka broker: b-0.xxx.yyy.zzz.kafka.us-east-1.amazonaws.com
    3. Endpoint service provision settings: Set com.amazonaws.vpce.us-east-1.vpce-svc-aaa = b-0.xxx.yyy.zzz.kafka.us-east-1.amazonaws.com
  4. Provide Endpoint Services and MSK Broker mappings to your Altinity Support representative.

Create VPC Endpoint Services

To create the VPC Endpoint Service that connects your Altinity.Cloud environment to your Amazon MSK service:

  1. From the AWS Virtual Private Cloud console, select Endpoints Services > Create Endpoint Service.

    Select Create Endpoint
  2. Set the following:

    1. Name: Enter a Name of you own choice (A).
    2. Load balancer type: Set to Network. (B)
    3. Available load balancers: Set to the load balancer you provisioned for this broker. (C)
    4. Additional settings:
      1. If you are required to manually accept the endpoint, set Acceptence Required to Enabled (D).
      2. Otherwise, leave Acceptance Required unchecked.
        Create Endpoint Details
  3. Select Create.

Test

To verify the VPC Endpoint Service works, please contact your Altinity Support representative.

References