Bring your own cloud (BYOC)

• 1: AWS remote provisioning
• 2: GCP remote provisioning

Altinity.Cloud Anywhere can use your cloud account to provision cloud resources to host your ClickHouse clusters. Your cloud account needs to have certain permissions and other properties; in this section we’ll go over those requirements.

Introduction

Altinity.Cloud Anywhere can operate inside a user’s cloud account.

Altinity technical support can remotely provision Kubernetes clusters for an Altinity.Cloud Anywhere environment inside your AWS or GCP account. Once the Kubernetes environment is set up, Altinity can provision ClickHouse clustes inside Kubernetes. This section describes how to configure your AWS or GCP accounts to authorize Altinity.Cloud to provision Kubernetes and ClickHouse in your account.

Figure 1 is a high-level view of the Altinity.Cloud Kubernetes infrastructure.

Figure 1 - The basic Altinity.Cloud Anywhere service architecture

Platform-specific requirements are discussed in the following sections.

1 - AWS remote provisioning

Provisioning process overview

This section summarizes the bootstrap process that lets you use Altinity.Cloud Anywhere to create an EKS environment and deploy ClickHouse clusters to it. You’ll go through these steps:

1. Get an Altinity.Cloud Anywhere account.
2. Get an Altinity.Cloud Anywhere environment record.
3. Get a connection token from the Altinity Cloud Manager.
4. Provision an AWS EKS cluster using EC2 instance running with a user account.
   The EC2 instance is required in order to deploy altinitycloud-connect, which will establish an outbound connection to Altinity.Cloud and start the EKS provisioning process.
   The EC2 instance can be set up in two ways:
   • Automatically by using the AWS Cloud Formation Template to automate the process.
   • Manually set up by a user following Altinity documentation.
5. In the Altinity Cloud Manager, complete the configuration of EKS resources.

Automatically provisioning EKS using an EC2 instance created from the AWS Cloud Formation Template

An Amazon AWS EC2 instance is required to deploy altinitycloud-connect, which establishes an outbound connection to Altinity.Cloud Anywhere and starts the EKS provisioning process.

In Altinity.Cloud Anywhere

1. Get an Altinity.Cloud Anywhere account.

2. Get an Altinity.Cloud Anywhere environment record.

3. Get a connection token from the Altinity Cloud Manager. The connection token is everything after token= in the “Connect to Altinity.Cloud” text box. In Figure 1, the token is all of the text in red.

   

   Figure 1 - The Altinity.Cloud Anywhere connection token

4. Login to your AWS account and navigate to Cloud Formation and click the Create Stack button.

5. Go to the altinitycloud-connect releases page and download the latest Cloud Formation YAML file, altinitycloud-connect-x.xx.x.aws-cloudformation.yaml. (That’s currently version 0.20.0.) The link to the YAML file is at the bottom of the page.

6. In the Create Stack panel, choose Upload a template file and select the Altinity Cloud Formation Template YAML file as shown in Figure 2. Click Next to continue.

7. Fill in any missing fields on the Specify Stack Details page (Figure 3):

   • Set Stack Name to altinitycloud-connect-$ENV_NAME (if your environment name is maddie-byok, the stack name should be altinity-connect-maddie-byok.)

   • Select the Subnets where the altinitycloud-connect EC2 instance(s) should be launched. At least one subnet is required.
     NOTE: The subnet(s) must have the “Auto-assign public IPv4 address” property set to yes.

   • Set the Token presented by https://acm.altinity.cloud/ to the token value from Step 3.

   • Click Next to continue.

   

   Figure 3 - AWS CloudFormation Stack details panel

8. Take the defaults on the Configure stack options panel (Figure 4). Click Next to continue.

   

   Figure 4 - AWS CloudFormation Configure stock options panel

9. On the Review panel, check the box to allow CloudFormation to create resources as shown in Figure 5. The process will not work unless you click this button. Click Submit to start provisioning the new stack.

   

   Figure 5 - Granting permissions on the AWS CloudFormation Review panel

10. You’ll see the main CloudFormation panel (Figure 6), which will include your new stack. You can follow its progress in the Events list.

    

    Figure 6 - AWS CloudFormation stack list

EC2 background processing explained

The EC2 instance is processed in the background as follows:

• EC2 instance gets started from the cloud formation template
• EC2 gets connected to Altinity.Cloud using altinitycloud-connect
• EKS cluster gets provisioned
• EKS cluster gets connected to Altinity.Cloud using altinitycloud-connect

In Altinity Cloud Manager

11. Once your new stack is created, click the ‘Proceed’ button in the connection wizard. Altinity Cloud Manager will connect to the stack named altinitycloud-connect-$ENV_NAME where $ENV_NAME is the name of your environment. NOTE: It is ok to select Proceed more than once, since provisioning takes some time. Once the EKS cluster is provisioned, the wizard switches to the Resources Configuration page.

12. Finish configuring your cluster as described in the Configuring resources section.

Manual provisioning of the EC2 instance

The AWS EC2 instance should meet the following requirements:

EC2 Instance Requirements

• CPU: t2.micro minimum
• OS: Ubuntu Server v20.04

Creating a Role with AIM policies

Set up a role with IAM policies to access IAM, EC2, VPC, EKS, S3 & Lambda as follows:

  arn:aws:iam::aws:policy/IAMFullAccess
  arn:aws:iam::aws:policy/AmazonEC2FullAccess
  arn:aws:iam::aws:policy/AmazonVPCFullAccess
  arn:aws:iam::aws:policy/AmazonS3FullAccess
  arn:aws:iam::aws:policy/AWSLambda_FullAccess
  arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore

NOTE:

• AmazonSSMManagedInstanceCore is used for the Break Glass Procedure over AWS SSM.

Creating a policy for EKS full access

1. Create a standard policy for EKS full access as follows:

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Effect":"Allow",
         "Action":[
            "eks  ":"*"
         ],
         "Resource":"*"
      },
      {
         "Effect":"Allow",
         "Action":"iam:PassRole",
         "Resource":"*",
         "Condition":{
            "StringEquals":{
               "iam:PassedToService":"eks.amazonaws.com"
            }
         }
      }
   ]
}

2. To set this instance to have access to the EC2 metadata and Internet, set the Security group to:

   • deny all inbound traffic
   • allow all outbound traffic

Installing Altinity.Cloud Connect

1. Download altinitycloud-connect. NOTE: The following example is for an Intel Linux installation. Change the filename to match your machine’s architecture.

   curl -sSL https://github.com/altinity/altinitycloud-connect/releases/download/v0.20.0/altinitycloud-connect-0.20.0-linux-amd64 -o altinitycloud-connect \
   

2. With the file downloaded, run these commands to make the file executable and put it in your path.

   chmod a+x altinitycloud-connect \
   && sudo mv altinitycloud-connect /usr/local/bin/
   

3. Login to Altinity.Cloud using a connection token as shown in Figure 1 above.

   altinitycloud-connect login --token=<registration token>
   

NOTE: altinitycloud-connect creates a cloud-connect.pem file in the current working directory.

4. Connect to Altinity.Cloud:

   altinitycloud-connect --capability aws
   

Start EKS provisioning

Provide additional configuration data to Altinity Support

5. The following data is required in order to create the VPC and EKS cluster properly:

• The CIDR for the Kubernetes VPC (at least /21 recommended, such as 10.1.0.0/21) that does not overlap with existing VPCs
• The Number of Availability Zones (3 are recommended)

Please send this information to your Altinity support representative to start the EKS provisioning process. When completed, the Altinity Cloud Manager (ACM) will be updated. At that point, you can create your ClickHouse clusters.

The remainder of the provisioning process is handled by Altinity.Cloud. Users may switch back to ACM and wait for connection to be established in order to finish configuration.

In Altinity.Cloud

6. Select the Proceed button in the connection wizard. You may repeat this step more than once to see if the connection has completed, since provisioning takes some time. Once the EKS cluster is provisioned, the connection wizard will switch to the Resources Configuration page.

7. Finish configuration of node pools as described in the Configuring resources section.

Break Glass procedure

The “Break Glass” procedure allows Altinity access to EC2 instance with SSH, using AWS SSM in order to troubleshoot altinitycloud-connect that is running on this instance.

1. Create an AnywhereAdmin IAM role with trust policy set:

    {
       "Version":"2012-10-17",
       "Statement":[
          {
             "Effect":"Allow",
             "Principal":{
                "AWS":"arn:aws:iam::313342380333:role/AnywhereAdmin"
             },
             "Action":"sts:AssumeRole"
          }
       ]
    }

2. Add a permission policy set:

    {
       "Version":"2012-10-17",
       "Statement":[
          {
             "Effect":"Allow",
             "Action":"ssm:StartSession",
             "Resource":[
                "arn:aws:ec2:$REGION:$ACCOUNT_ID:instance/$INSTANCE_ID",
                "arn:aws:ssm:*:*:document/AWS-StartSSHSession"
             ]
          }
       ]
    }

3. Send the following ARN string to Altinity: NOTE: This is used to revoke the Break Glass Procedure access change, or remove the permission policy.

    arn:aws:ec2:$REGION:$ACCOUNT_ID:instance/$INSTANCE_ID

2 - GCP remote provisioning

Introduction

Altinity.Cloud Anywhere operates inside a user’s cloud account.

Altinity technical support can remotely provision GCP GKE clusters with an Altinity.Cloud Anywhere environment in your GCP account. Instructions on this page describe how to configure your GCP account to provide permission to Altinity to provision ClickHouse to your GCP Kubernetes environment.

Overview - Google GKE Installation

This section covers how to authorize Altinity.Cloud Anywhere to create Kubernetes and ClickHouse resources inside your cloud account.

Creating a project

Creating a separate project makes it easy to isolate resources and do cost management, not to mention security. You can create a project from the command line or in the GCP web UI.

You can use the gcloud projects create command to create a new project:

# Create project 'maddie'
gcloud projects create maddie

You also need to assign a billing account to the project. Currently the gcloud command looks like this:

# Assign a billing account to 'maddie'
gcloud beta billing projects link maddie \
  --billing-account 0X0X0X-0X0X0X-0X0X0X

You can also create a project from the GCP web UI:

Granting permissions

For Altinity to be able to create Kubernetes and ClickHouse clusters in your cloud account, you need to grant the following permissions to anywhere-admin@altinity.com inside the project you just created:

• roles/compute.admin
• roles/container.admin
• roles/dns.admin
• roles/storage.admin
• roles/storage.hmacKeyAdmin
• roles/iam.serviceAccountAdmin
• roles/iam.serviceAccountKeyAdmin
• roles/iam.serviceAccountTokenCreator
• roles/iam.serviceAccountUser
• roles/iam.workloadIdentityPoolAdmin
• roles/serviceusage.serviceUsageAdmin
• roles/resourcemanager.projectIamAdmin
• roles/iap.tunnelResourceAccessor

You can use the gcloud command for each role:

# Add a role for a member of a group associated with project 'maddie'
gcloud projects add-iam-policy-binding maddie \
  --member='group:anywhere-admin@altinity.com' \
  --role='roles/compute.admin'

Alternately, you can use the GCP web UI:

Creating the Kubernetes environment

With the project created and the appropriate permissions granted to the Altinity.Cloud Anywhere admin account, Altinity can create Kubernetes clusters and ClickHouse clusters inside them. The following sections demonstrate how to create the Kubernetes environment. With the Kubernetes environment created, Altinity.Cloud Anywhere can create ClickHouse clusters inside it.

Define the environment type

In Altinity Cloud Manager, go to the Environments tab. Click the “+ SETUP ENVIRONMENT” button at the top of the screen. Select “Altinity.Cloud Anywhere Bring Your Own Cloud (BYOC)” as the environment type:

Name your environment

With the environment type selected, give your new environment a name:

Click OK to continue.

Choose the correct Kubernetes setup

To set up your connection, Altinity.Cloud needs to know which Kubernetes setup to use. “GKE Provisioned by Altinity” is the right choice here:

Click PROCEED to continue.

Define your Kubernetes cluster’s resources

The Resources Configuration tab looks like this:

Fields

• Cloud Provider - GCP should be selected automatically; select it if it isn’t.
• Region - Click the arrow next to the field to see a list of available regions.
• Number of AZs - The number of availability zones for your cluster. NOTE: It is highly recommended that you use at least two availability zones.
• Project ID - Give your project a name.
• CIDR Block - The address range allocated to your cluster. NOTE: Be sure you define enough addresses. We recommend x.x.x.x/21 at a minimum. If you run out of addresses, this setting is difficult to change.
• Storage Classes - Enter the storage classes your cluster will use. You can delete the entries that appear; you can also click the ADD STORAGE CLASS button to add other storage classes.
• Node Pools - Define the node pools that your cluster will use. At least one node pool must be defined for ClickHouse and Zookeeper and System. In this example, one node pool will host Zookeeper and the System utilities Altinity.Cloud uses; four other node pools will host ClickHouse itself. You can click the ADD NODE POOL button to add more node pools as needed. You can also add more node pools later.

Click PROCEED to continue.

Confirm your settings

The Confirmation screen in Figure 7 displays a JSON representation of the settings you just made. Review these settings; you can edit the JSON directly if needed. When the JSON is correct, select FINISH.

It will take a few minutes for all the resources to be provisioned.

Connection completed

Once the connection is fully set up, the ACM Environments dashboard will display your new environment as shown in Figure 8.