Kubernetes requirements

When running Altinity.Cloud inside your Kubernetes environment, that environment has to be configured a certain way. In this section we’ll cover the general requirements for your Kubernetes environment.

That being said, here are the requirements for your Kubernetes environment:

• Kubernetes version 1.26 or higher in EKS (AWS), GKE (GCP), or AKS (Azure)
• Every Node should have the following labels:
  • node.kubernetes.io/instance-type
  • kubernetes.io/arch
  • topology.kubernetes.io/zone
  • altinity.cloud/use=anywhere
• It is recommended to taint node
  • with altinity.cloud/use=anywhere:NoSchedule only for nodes that are dedicated exclusively to Altinity pods
  • and dedicated=clickhouse:NoSchedule additionally for nodes dedicated to ClickHouse
• A StorageClass with dynamic provisioning is required
• LoadBalancer services must be supported

To get the most from Altinity.Cloud in your Kubernetes environment:

• Each StorageClass should preferably allow volume expansion
• Multiple zones are preferable for high availability
• Autoscaling is preferable for easier vertical scaling

For platform-specific requirements, see the following sections:

• AWS requirements
• GCP requirements
• Azure requirements

AWS requirements

We recommend setting up karpenter or cluster-autoscaler to launch instances in at least 3 Availability Zones. If you’re using the AWS command line interface, version 2.0 or higher is required.

If you plan on sharing your Kubernetes cluster with other workloads, it’s recommended you label Kubernetes Nodes dedicated exclusively to Altinity.Cloud with altinity.cloud/use=anywhere and taint them with altinity.cloud/use=anywhere:NoSchedule. Please make sure there are still nodes available to run kube-system pods after applying the Altinity.Cloud taints.

Instance types

For Zookeeper infrastructure nodes

• t3.large or t4g.large*

t4g instances are AWS Graviton2-based (ARM).

For ClickHouse nodes

ClickHouse works the best in AWS when using nodes from ’m’ instance type families:

• m5 (deprecated)
• m6i
• m6a
• m6g*
• m7i
• m7a
• m7g*

m6g and m7g instances are AWS Graviton-based (ARM).

For RAM-intensive scenarios, r6 and r7 instance type families may be used as well.

Instance sizes from large (2 vCPUs) to 8xlarge (32 vCPUs) are typical.

Storage classes

• gp2*
• gp3-encrypted*

We recommend using gp3 storage classes that provide more flexibility and performance over gp2. The gp3 storage classes require the Amazon EBS CSI driver; that driver is not automatically installed. See the AWS CSI driver documentation for details on how to install the driver.

Storage class can be installed with the following manifest:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: gp3-encrypted
  annotations:
    storageclass.kubernetes.io/is-default-class: 'true'
provisioner: ebs.csi.aws.com
parameters:
  encrypted: 'true'
  fsType: ext4
  type: gp3
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true

The default throughput for gp3 is 125MB/s for any volume size. It can be increased in AWS console or using storage class parameters. Here is an example:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: gp3-encrypted-500
provisioner: ebs.csi.aws.com
parameters:
  encrypted: 'true'
  fsType: ext4
  throughput: '500'
  type: gp3
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true

Alternatively, you recommend installing the Altinity EBS parameters controller. That allows you to manage EBS volume throughput dynamically through annotations. This is also integrated to Altinity.Cloud UI (ACM).

Azure requirements

Instance types

For Zookeeper and infrastructure nodes

• Standard_B2s_v2

For ClickHouse nodes

Use nodes from the Standard Dsv5 series

It’s recommended to set node pool taints with dedicated=clickhouse:NoSchedule.

Storage classes

• managed-csi-premium
• premium2-disk-sc - See the Microsoft Premium SSD v2 documentation for more details.

Be aware that you can define custom throughput for premium2 disks:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: premium2-disk-sc-500
parameters:
  DiskIOPSReadWrite: "12000"
  DiskMBpsReadWrite: "500"
  cachingMode: None
  skuName: PremiumV2_LRS
provisioner: disk.csi.azure.com
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true

GCP requirements

Instance types

For Zookeeper and infrastructure nodes

• e2-standard-2

For ClickHouse nodes

• n2d-standard-2
• n2d-standard-4
• n2d-standard-8
• n2d-standard-16
• n2d-standard-32

If GCP is out of n2d-standard-* instances in the region of your choice, we recommend substituting them with n2-standard-*.

It’s recommended to taint node pools with dedicated=clickhouse:NoSchedule (in addition to altinity.cloud/use=anywhere:NoSchedule).

Storage classes

• standard-rwo
• premium-rwo

GKE comes pre-configured with both.