Kubernetes requirements
Altinity.Cloud Anywhere operates inside your Kubernetes environment. The general requirements for your Kubernetes environment are:
- Kubernetes version 1.26 or higher in EKS (AWS), GKE (GCP), or AKS (Azure)
- Every
Node
should have the following labels:node.kubernetes.io/instance-type
kubernetes.io/arch
topology.kubernetes.io/zone
altinity.cloud/node-group: infra
- It is recommended to taint node
- with
altinity.cloud/use=anywhere:NoSchedule
for all nodes that are used for Altinity pods - and
dedicated=clickhouse:NoSchedule
additionally for nodes dedicated to ClickHouse
- with
- A
StorageClass
with dynamic provisioning is required LoadBalancer
services must be supported
To get the most from Altinity.Cloud Anywhere features:
- Each
StorageClass
should preferably allow volume expansion - Multiple zones are preferable for high availability
- Autoscaling is preferable for easier vertical scaling
For platform-specific requirements, see the following sections:
AWS requirements
We recommend setting up karpenter or cluster-autoscaler to launch instances in at least 3 Availability Zones.
If you plan on sharing Kubernetes cluster with other workloads, it’s
recommended you label Kubernetes Nodes intended for Altinity.Cloud Anywhere
with altinity.cloud/use=anywhere
and taint them with dedicated=anywhere:NoSchedule
.
Instance types
For Zookeeper infrastructure nodes
t3.large
ort4g.large
*
t4g
instances are AWS Graviton2-based (ARM).
For ClickHouse nodes
ClickHouse works the best in AWS when using nodes from ’m’ instance type families:
m5
(deprecated)m6i
m6a
m6g
*m7i
(new in 2023, may be not available in all regions)m7a
(new in 2023, may be not available in all regions)m7g
* (new in 2023, may be not available in all regions)
m6g
and m7g
instances are AWS Graviton-based (ARM).
For RAM intesive scenarios, r6
and r7
instance type families may be used as well.
Instance sizes from large
(2 vCPUs) to 8xlarge
(32 vCPUs) are typical.
Storage classes
gp2
*gp3-encrypted
*
We recommend using gp3
storage classes that provide more flexibility and performance over gp2
. The gp3
storage classes require the Amazon EBS CSI driver;
that driver is not automatically installed. See
the AWS CSI driver documentation
for details on how to install the driver.
Storage class can be installed with the following manifest:
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: gp3-encrypted
annotations:
storageclass.kubernetes.io/is-default-class: 'true'
provisioner: ebs.csi.aws.com
parameters:
encrypted: 'true'
fsType: ext4
type: gp3
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
The default throughput for gp3
is 125MB/s for any volume size. It can be increased in AWS console or using storage class parameters. Here is an example:
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: gp3-encrypted-500
provisioner: ebs.csi.aws.com
parameters:
encrypted: 'true'
fsType: ext4
throughput: '500'
type: gp3
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
Alternatively, you recommend installing the Altinity EBS parameters controller. That allows you to manage EBS volume throughput dynamically through annotations. This is also integrated to Altinity.Cloud UI (ACM).
GCP requirements
Instance types
For Zookeeper and infrastructure nodes
e2-standard-2
For ClickHouse nodes
n2d-standard-2
n2d-standard-4
n2d-standard-8
n2d-standard-16
n2d-standard-32
If GCP is out of n2d-standard-*
instances in the
region of your choice, we recommend
substituting them with n2-standard-*
.
It’s recommended to taint node pools with dedicated=clickhouse:NoSchedule
(in addition to altinity.cloud/use=anywhere:NoSchedule
).
Storage classes
standard-rwo
premium-rwo
GKE comes pre-configured with both.
Azure requirements
Instance types
For Zookeeper and infrastructure nodes
Standard_B2s_v2
For ClickHouse nodes
Use nodes from the Standard Dsv5 series
It’s recommended to set node pool taints with dedicated=clickhouse:NoSchedule
.
Storage classes
managed-csi-premium
premium2-disk-sc
- See the Microsoft Premium SSD v2 documentation for more details.
Be aware that you can define custom throughput for premium2
disks:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: premium2-disk-sc-500
parameters:
DiskIOPSReadWrite: "12000"
DiskMBpsReadWrite: "500"
cachingMode: None
skuName: PremiumV2_LRS
provisioner: disk.csi.azure.com
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true