GCP remote provisioning

Altinity.Cloud Anywhere operates inside your GCP account. You need two things before you can create ClickHouse® clusters in a Kubernetes (GKE) cluster in your account:

• A new Project in your GCP account
• Permissions that let the Altinity service account do very specific things in the new Project

Once those things are created, you need to create a new Altinity.Cloud environment, connect that environment to your GCP project, then create the GKE cluster.

There are two ways to do this:

• Method 1. Use our Terraform module to do everything. It’s completely automated. Set a couple of parameters and run the script.
• Method 2. Do everything by hand. You create all the resources yourself and connect them together. Perfect for those who enjoy typing.

We strongly recommend Method 1.

Method 1. Using our Terraform module

This approach is the easiest way to set everything up. You fill in a few details (name a couple of things, tell us where things should run, etc.), and Terraform does the rest.

Prerequisites

• Terraform version >= 1.5
• kubectl
• An Anywhere API access token from your Altinity.Cloud account
• gcloud, the GCP command-line tool

Before Terraform can do anything on your behalf, you’ll need to log in to your GCP account:

gcloud auth login

Your GCP account must have permissions to create projects, GKE clusters, and the other things you’ll need. (If it doesn’t, that’ll be obvious pretty quickly.)

Getting an Anywhere API access token

The GCP remote provisioning process needs an Anywhere API Access token. To get the token, log in to your Altinity.Cloud account and click your account name in the upper right. Click the My Account menu item:

Click the Anywhere API Access tab to generate a new token:

It’s recommended that you click the copy icon to copy the text of the token; the token is too wide to appear in the dialog. You won’t be able to see the token after you leave this panel, so be sure to store it for later in a secure place. See the Altinity API guide for complete details on creating and managing API tokens.

Now create the environment variable ALTINITYCLOUD_API_TOKEN:

export ALTINITYCLOUD_API_TOKEN=eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJkZXYuYWx0aW5pdHkuY2xvdWQiLCJpYXQiOjE3MTYyMTE0NjgsInN1YiI6ImR0aWR3ZWxsQGFsdGluaXR5LmNvbSJ9.LPb56KQjF_SLn8dGAzUP1fAlTmYItfdH-SCMTsngGB4EUPVZHOM9axW3B7KvupgNxsBuRfIOkmyKPmJUG1dsAw

Using the Terraform script to do everything else

To get started, create a new directory for your Terraform project and create a file named main.tf inside it. Copy and paste the following text into the main.tf file:

terraform {
  required_providers {
    altinitycloud = {
      source = "Altinity/altinitycloud"
      version = "0.2.7"
    }
  }
}

provider "google" {
}

resource "google_project" "this" {
  folder_id       = "406594290810"
  billing_account = "ABCDEF-ABCDEF-ABCDEF"

  project_id          = "maddie-tf"
  name                = "maddie-tf"
  auto_create_network = false
}

resource "google_project_iam_member" "this" {
  for_each = toset([
    # https://cloud.google.com/iam/docs/understanding-roles
    "roles/compute.admin",
    "roles/container.admin",
    "roles/dns.admin",
    "roles/storage.admin",
    "roles/storage.hmacKeyAdmin",
    "roles/iam.serviceAccountAdmin",
    "roles/iam.serviceAccountKeyAdmin",
    "roles/iam.serviceAccountTokenCreator",
    "roles/iam.serviceAccountUser",
    "roles/iam.workloadIdentityPoolAdmin",
    "roles/iam.roleAdmin",
    "roles/serviceusage.serviceUsageAdmin",
    "roles/resourcemanager.projectIamAdmin",
    "roles/iap.tunnelResourceAccessor"
  ])
  project = google_project.this.id
  role    = each.key
  member  = "group:anywhere-admin@altinity.com"
}

resource "altinitycloud_env_gcp" "this" {
  name           = "maddie-byoc-gcp-tf"
  gcp_project_id = google_project.this.project_id
  region         = "us-east1"
  zones          = ["us-east1-b", "us-east1-d"]
  cidr           = "10.67.0.0/21"

  load_balancers = {
    public = {
      enabled          = true
      source_ip_ranges = ["0.0.0.0/0"]
    }
  }

  node_groups = [
    {
      node_type         = "e2-standard-2"
      capacity_per_zone = 10
      reservations      = ["SYSTEM", "ZOOKEEPER"]
    },
    {
      node_type         = "n2d-standard-2"
      capacity_per_zone = 10
      reservations      = ["CLICKHOUSE"]
    }
  ]
}

Field details

You need to set some things in the script:

• folder_id is the ID of the folder in which your new GCP project will be created. This has to be a numeric value. You probably use the folder’s name most of the time, but you can see the ID in the project list in the GCP console. Here the ID of the dev folder is 406594290810:
  
  

  

  
  Figure 3 - Finding the ID of a GCP project
• billing_account is your GCP billing account.
• name (in resource google_project) is the name of your GCP project.
• project_id (in resource google_project) is the ID of your GCP project.
• name (in resource altinitycloud_env_gcp) is the name of your Altinity.Cloud environment. Your Altinity.Cloud environment name must start with your account name. If your account name is maddie, you could name your environment maddie-byoc-gcp-tf.
• region is the GCP region where your project and its resources will run. The default is us-east1.
• zones are the availability zones your project will use. These must be zones within the region you selected.

In the example above, Terraform will create a new GCP project named maddie-tf (its ID will be maddie-tf as well). Once a GKE cluster is provisioned inside the new project, Terraform will create a new Altinity.Cloud environment named altinity-byoc-gcp-tf. All ClickHouse clusters defined in this environment will run in the us-east1 region.

Applying the configuration

Open the terminal and navigate to the directory you created. Run the following commands to initialize the Terraform project and apply it, specifying the names for your Altinity.Cloud environment and GCP project:

# initialize the terraform project
terraform init

# apply module changes, passing along names for your
# Altinity.Cloud environment and your GCP project
tf apply 

This operation will take a while to complete. When Terraform, is done you’ll a reassuring message like this:

Apply complete! Resources: 15 added, 0 changed, 0 destroyed.

But things are still happening in GKE. Terraform has created the GCP project and created the GKE cluster, but it almost certainly won’t be done yet. Take a look in the GCP console, open your new project, and you’ll see the progress of provisioning:

Depending on various factors, this may take a while. When GKE provisioning is done, log in to your Altinity.Cloud account and click the Environments menu in the upper right. Your new environment will be in the list:

If things still aren’t finished in the background, you’ll see this panel:

If you see this message, wait a few minutes and try again. (We really wish we had a better answer here. We’re working on it.)

Once everything is connected, you’ll see the Environment View for your new environment:

Click the button to see your ClickHouse clusters. You won’t have any, of course, so use the Launch Cluster Wizard to create new ClickHouse clusters.

Deleting the configuration

To delete the GCP project and everything in it, first delete the Altinity.Cloud environment in your account. See Deleting an Environment for details on deleting an Altinity.Cloud environment. Once the environment is deleted, run the following command to delete the resources created by Terraform:

# Delete your Altinity.Cloud environment in the Altinity
# Cloud Manager before running this command!
terraform destroy 

When this command finishes, the GCP project will be deleted.

Method 2. Doing everything by hand

Creating a project

Creating a separate project makes it easy to isolate resources and do cost management, not to mention security. You can create a project from the command line or in the GCP web UI.

You can use the gcloud projects create command to create a new project:

# Create project 'maddie-byoc-gcp'
gcloud projects create maddie-byoc-gcp

You also need to assign a billing account to the project. Currently the gcloud command looks like this:

# Assign a billing account to 'maddie-byoc-gcp'
gcloud beta billing projects link maddie-byoc-gcp \
  --billing-account ABCDEF-ABCDEF-ABCDEF

You can also create a project from the GCP web UI:

Granting permissions

For Altinity to be able to create Kubernetes and ClickHouse clusters in your cloud account, you need to grant the following permissions to anywhere-admin@altinity.com inside the project you just created:

• roles/compute.admin
• roles/container.admin
• roles/dns.admin
• roles/storage.admin
• roles/storage.hmacKeyAdmin
• roles/iam.serviceAccountAdmin
• roles/iam.serviceAccountKeyAdmin
• roles/iam.serviceAccountTokenCreator
• roles/iam.serviceAccountUser
• roles/iam.workloadIdentityPoolAdmin
• roles/iam.roleAdmin
• roles/serviceusage.serviceUsageAdmin
• roles/resourcemanager.projectIamAdmin
• roles/iap.tunnelResourceAccessor

You can use the gcloud command for each role:

# Add a role for a member of a group associated with 
# project 'maddie-byoc-gcp'
gcloud projects add-iam-policy-binding maddie-byoc-gcp \
  --member='group:anywhere-admin@altinity.com' \
  --role='roles/compute.admin'

Alternately, you can use the GCP web UI:

However you grant permissions, you can view them in the web UI:

Creating the Kubernetes environment

With the project created and the appropriate permissions granted to the Altinity.Cloud Anywhere admin account, Altinity can create Kubernetes clusters and ClickHouse clusters inside them. The following sections demonstrate how to create the Kubernetes environment.

Set up the environment

In Altinity Cloud Manager, go to the Environments tab. Click the button at the top of the screen.

In the Environment Setup dialog, select Bring Your Own Cloud (BYOC), give your environment a name, and select GCP as your cloud provider:

Figure 11 - Choosing a BYOC / GCP environment

Click OK to continue.

Connecting your GCP project to Altinity.Cloud

Next, you need to connect your GCP project to Altinity.Cloud. You’ve already created the project, so click PROCEED to continue.

Define your Kubernetes cluster’s resources

The Resources Configuration tab looks like this:

Field details

• Cloud Provider - GCP is selected automatically.
• Region - Click the down arrow to see a list of available regions. Be sure your project is authorized to create resources in the region you select.
• Number of AZs - The number of availability zones for your cluster. NOTE: It is highly recommended that you use at least two availability zones.
• Project ID - Enter the name of your GCP project. This is how Altinity.Cloud Anywhere ties everything together; you’ve created this project and given Altinity the permissions it needs to deploy and manage ClickHouse. (In this example the names of the Altinity.Cloud environment and the GCP project are similar, but they can be completely different.)
• CIDR Block - The address range allocated to your cluster. NOTE: Be sure you define enough addresses. We recommend x.x.x.x/21 at a minimum. If you run out of addresses, this setting is difficult to change.
• Storage Classes - Enter the storage classes your cluster will use. You can delete the entries that appear; you can also click the ADD STORAGE CLASS button to add other storage classes.
• Node Pools - Define the node pools that your cluster will use. At least one node pool must be defined for ClickHouse and Zookeeper and System. In this example, one node pool will host Zookeeper and the System utilities Altinity.Cloud uses; four other node pools will host ClickHouse itself. When a user creates a new ClickHouse cluster, they can choose which node size they want to use. You can click the ADD NODE POOL button to add more node pools as needed now, and you can also add more later.

Click PROCEED to continue. The Altinity Cloud Manager connects to your GCP project and creates resources inside it. You can click VIEW LOG to see the system’s progress:

It will take a few minutes for all the resources to be provisioned. While you wait, you can also go to the GCP web UI to see the resources being created.

Connection completed

Once the connection is fully set up, the ACM Environments dashboard will display your new environment:

Click the button at the top of the page to go to the Clusters page. You don’t have any ClickHouse clusters yet, of course, but the ACM makes it easy to create one. See the documentation for the Launch Cluster Wizard for all the details.