Installation Guide

• 1: Basic Installation Guide
• 2: Custom Installation Guide
• 3: Source Build Guide - 0.18 and Up
• 4: Specific Version Installation Guide
• 5: Upgrade Guide

1 - Basic Installation Guide

Requirements

The Altinity Kubernetes Operator for Kubernetes has the following requirements:

• Kubernetes 1.15.11+. For instructions on how to install Kubernetes for your particular environment, see the Kubernetes Install Tools page.
• Access to the clickhouse-operator-install-bundle.yaml file.

Instructions

To install the Altinity Kubernetes Operator for Kubernetes:

1. Deploy the Altinity Kubernetes Operator from the manifest directly from GitHub. It is recommended that the version be specified during installation - this insures maximum compatibility and that all replicated environments are working from the same version. For more information on installing other versions of the Altinity Kubernetes Operator, see the specific Version Installation Guide.

   The most current version is 0.18.1:

kubectl apply -f https://github.com/Altinity/clickhouse-operator/raw/0.18.1/deploy/operator/clickhouse-operator-install-bundle.yaml

2. The following will be displayed on a successful installation. For more information on the resources created in the installation, see [Altinity Kubernetes Operator Resources]({<ref “operatorresources” >})

customresourcedefinition.apiextensions.k8s.io/clickhouseinstallations.clickhouse.altinity.com created
customresourcedefinition.apiextensions.k8s.io/clickhouseinstallationtemplates.clickhouse.altinity.com created
customresourcedefinition.apiextensions.k8s.io/clickhouseoperatorconfigurations.clickhouse.altinity.com created
serviceaccount/clickhouse-operator created
clusterrole.rbac.authorization.k8s.io/clickhouse-operator-kube-system created
clusterrolebinding.rbac.authorization.k8s.io/clickhouse-operator-kube-system created
configmap/etc-clickhouse-operator-files created
configmap/etc-clickhouse-operator-confd-files created
configmap/etc-clickhouse-operator-configd-files created
configmap/etc-clickhouse-operator-templatesd-files created
configmap/etc-clickhouse-operator-usersd-files created
deployment.apps/clickhouse-operator created
service/clickhouse-operator-metrics created

3. Verify the installation by running:

kubectl get pods --namespace kube-system

The following will be displayed on a successful installation, with your particular image:

NAME                                   READY   STATUS    RESTARTS      AGE
clickhouse-operator-77b54889b4-cnvqf   2/2     Running   0             4s
coredns-78fcd69978-nthp2               1/1     Running   2 (92m ago)   23d
etcd-minikube                          1/1     Running   2 (92m ago)   23d
kube-apiserver-minikube                1/1     Running   2 (92m ago)   23d
kube-controller-manager-minikube       1/1     Running   2 (92m ago)   23d
kube-proxy-lsggn                       1/1     Running   2 (92m ago)   23d
kube-scheduler-minikube                1/1     Running   2 (92m ago)   23d
storage-provisioner                    1/1     Running   5 (91m ago)   23d

2 - Custom Installation Guide

Users who need to customize their Altinity Kubernetes Operator namespace or can not directly connect to Github from the installation environment can perform a custom install.

Requirements

The Altinity Kubernetes Operator for Kubernetes has the following requirements:

• Kubernetes 1.15.11+. For instructions on how to install Kubernetes for your particular environment, see the Kubernetes Install Tools page.
• Access to the clickhouse-operator-install-bundle.yaml file.

Instructions

Script Install into Namespace

By default, the Altinity Kubernetes Operator installed into the kube-system namespace when using the Basic Installation instructions. To install into a different namespace use the following command replacing {custom namespace here} with the namespace to use:

curl -s https://raw.githubusercontent.com/Altinity/clickhouse-operator/master/deploy/operator-web-installer/clickhouse-operator-install.sh | OPERATOR_NAMESPACE={custom_namespace_here} bash

For example, to install into the namespace test-clickhouse-operator namespace, use:

curl -s https://raw.githubusercontent.com/Altinity/clickhouse-operator/master/deploy/operator-web-installer/clickhouse-operator-install.sh | OPERATOR_NAMESPACE=test-clickhouse-operator bash

Setup ClickHouse Operator into 'test-clickhouse-operator' namespace
No 'test-clickhouse-operator' namespace found. Going to create
namespace/test-clickhouse-operator created
customresourcedefinition.apiextensions.k8s.io/clickhouseinstallations.clickhouse.altinity.com created
customresourcedefinition.apiextensions.k8s.io/clickhouseinstallationtemplates.clickhouse.altinity.com created
customresourcedefinition.apiextensions.k8s.io/clickhouseoperatorconfigurations.clickhouse.altinity.com created
serviceaccount/clickhouse-operator created
clusterrole.rbac.authorization.k8s.io/clickhouse-operator-test-clickhouse-operator created
clusterrolebinding.rbac.authorization.k8s.io/clickhouse-operator-test-clickhouse-operator created
configmap/etc-clickhouse-operator-files created
configmap/etc-clickhouse-operator-confd-files created
configmap/etc-clickhouse-operator-configd-files created
configmap/etc-clickhouse-operator-templatesd-files created
configmap/etc-clickhouse-operator-usersd-files created
deployment.apps/clickhouse-operator created
service/clickhouse-operator-metrics created

If no OPERATOR_NAMESPACE value is set, then the Altinity Kubernetes Operator will be installed into kube-system.

Manual Install into Namespace

For organizations that can not access GitHub directly from the environment they are installing the Altinity Kubernetes Operator in, they can perform a manual install through the following steps:

1. Download the install template file: clickhouse-operator-install-template.yaml.

2. Edit the file and set OPERATOR_NAMESPACE value.

3. Use the following commands, replacing {your file name} with the name of your YAML file:

   namespace = "custom-clickhouse-operator"
   bash("sed -i s/'${OPERATOR_NAMESPACE}'/test-clickhouse-operator/ clickhouse-operator-install-template.yaml", add_to_text=False)
   bash(f"kubectl apply -f clickhouse-operator-install-template.yaml", add_to_text=False)
   
   try:
   
       retry(bash, timeout=60, delay=1)("kubectl get pods --namespace test-clickhouse-operator "
           "-o=custom-columns=NAME:.metadata.name,STATUS:.status.phase",
           exitcode=0, message="Running", lines=slice(1, None),
           fail_message="not all pods in Running state", add_to_text=true)
   
   finally:
       bash(f"kubectl delete namespace test-clickhouse-operator', add_to_text=False)
   

   kubectl apply -f {your file name}
   

   For example:

   kubectl apply -f customtemplate.yaml
   

Alternatively, instead of using the install template, enter the following into your console (bash is used below, modify depending on your particular shell). Change the OPERATOR_NAMESPACE value to match your namespace.

# Namespace to install operator into
OPERATOR_NAMESPACE="${OPERATOR_NAMESPACE:-clickhouse-operator}"
# Namespace to install metrics-exporter into
METRICS_EXPORTER_NAMESPACE="${OPERATOR_NAMESPACE}"

# Operator's docker image
OPERATOR_IMAGE="${OPERATOR_IMAGE:-altinity/clickhouse-operator:latest}"
# Metrics exporter's docker image
METRICS_EXPORTER_IMAGE="${METRICS_EXPORTER_IMAGE:-altinity/metrics-exporter:latest}"

# Setup Altinity Kubernetes Operator into specified namespace
kubectl apply --namespace="${OPERATOR_NAMESPACE}" -f <( \
    curl -s https://raw.githubusercontent.com/Altinity/clickhouse-operator/master/deploy/operator/clickhouse-operator-install-template.yaml | \
        OPERATOR_IMAGE="${OPERATOR_IMAGE}" \
        OPERATOR_NAMESPACE="${OPERATOR_NAMESPACE}" \
        METRICS_EXPORTER_IMAGE="${METRICS_EXPORTER_IMAGE}" \
        METRICS_EXPORTER_NAMESPACE="${METRICS_EXPORTER_NAMESPACE}" \
        envsubst \
)

Verify Installation

To verify the Altinity Kubernetes Operator is running in your namespace, use the following command:

kubectl get pods -n clickhouse-operator

NAME                                   READY   STATUS    RESTARTS   AGE
clickhouse-operator-5d9496dd48-w6rcb   2/2     Running   0          16s

3 - Source Build Guide - 0.18 and Up

For organizations who prefer to build the software directly from source code, they can compile the Altinity Kubernetes Operator and install it into a Docker container through the following process. The following procedure is available for versions of the Altinity Kubernetes Operator 0.18.0 and up.

Binary Build

Binary Build Requirements

• go-lang compiler: Go.
• Go mod Package Manager.
• The source code from the Altinity Kubernetes Operator repository. This can be downloaded using git clone https://github.com/altinity/clickhouse-operator.

Binary Build Instructions

1. Switch working dir to clickhouse-operator.

2. Link all packages with the command: echo {root_password} | sudo -S -k apt install -y golang.

3. Build the sources with go build -o ./clickhouse-operator cmd/operator/main.go.

This creates the Altinity Kubernetes Operator binary. This binary is only used within a kubernetes environment.

Docker Image Build and Usage

Docker Build Requirements

• kubernetes: https://kubernetes.io/
• docker: https://www.docker.com/
• Complete the Binary Build Instructions

Install Docker Buildx CLI plugin

1. Download Docker Buildx binary file releases page on GitHub

2. Create folder structure for plugin

   mkdir -p ~/.docker/cli-plugins/
   

3. Rename the relevant binary and copy it to the destination matching your OS

   mv buildx-v0.7.1.linux-amd64  ~/.docker/cli-plugins/docker-buildx
   

4. On Unix environments, it may also be necessary to make it executable with chmod +x:

   chmod +x ~/.docker/cli-plugins/docker-buildx
   

5. Set buildx as the default builder

   docker buildx install
   

6. Create config.json file to enable the plugin

   touch ~/.docker/config.json
   

7. Create config.json file to enable the plugin

   echo "{"experimental": "enabled"}" >> ~/.docker/config.json
   

Docker Build Instructions

1. Switch working dir to clickhouse-operator

2. Build docker image with docker: docker build -f dockerfile/operator/Dockerfile -t altinity/clickhouse-operator:dev .

3. Register freshly build docker image inside kubernetes environment with the following:

   docker save altinity/clickhouse-operator | (eval $(minikube docker-env) && docker load)
   

4. Install the Altinity Kubernetes Operator as described in either the Basic Build or Custom Build.

4 - Specific Version Installation Guide

Users may want to install a specific version of the Altinity Kubernetes Operator for a variety of reasons: to maintain parity between different environments, to preserve the version between replicas, or other reasons.

The following procedures detail how to install a specific version of the Altinity Kubernetes Operator in the default Kubernetes namespace kube-system. For instructions on performing custom installations based on the namespace and other settings, see the Custom Installation Guide.

Requirements

The Altinity Kubernetes Operator for Kubernetes has the following requirements:

• Kubernetes 1.15.11+. For instructions on how to install Kubernetes for your particular environment, see the Kubernetes Install Tools page.
• Access to the Altinity Kubernetes Operator GitHub repository.

Instructions

Altinity Kubernetes Operator Versions After 0.17.0

To install a specific version of the Altinity Kubernetes Operator after version 0.17.0`:

1. Run kubectl and apply the manifest directly from the GitHub Altinity Kubernetes Operator repository, or by downloading the manifest and applying it directly. The format for the URL is:

   https://github.com/Altinity/clickhouse-operator/raw/{OPERATOR_VERSION}/deploy/operator/clickhouse-operator-install-bundle.yaml
   

   Replace the {OPERATOR_VERSION} with the version to install. For example, for the Altinity Kubernetes Operator version 0.18.1, the URL would be:

   https://github.com/Altinity/clickhouse-operator/raw/0.18.1/deploy/operator/clickhouse-operator-install-bundle.yaml

   The command to apply the Docker manifest through kubectl is:

   kubectl apply -f https://github.com/Altinity/clickhouse-operator/raw/0.18.1/deploy/operator/clickhouse-operator-install-bundle.yaml
   

   customresourcedefinition.apiextensions.k8s.io/clickhouseinstallations.clickhouse.altinity.com configured
   customresourcedefinition.apiextensions.k8s.io/clickhouseinstallationtemplates.clickhouse.altinity.com configured
   customresourcedefinition.apiextensions.k8s.io/clickhouseoperatorconfigurations.clickhouse.altinity.com configured
   serviceaccount/clickhouse-operator created
   clusterrole.rbac.authorization.k8s.io/clickhouse-operator-kube-system created
   clusterrolebinding.rbac.authorization.k8s.io/clickhouse-operator-kube-system created
   configmap/etc-clickhouse-operator-files created
   configmap/etc-clickhouse-operator-confd-files created
   configmap/etc-clickhouse-operator-configd-files created
   configmap/etc-clickhouse-operator-templatesd-files created
   configmap/etc-clickhouse-operator-usersd-files created
   deployment.apps/clickhouse-operator created
   service/clickhouse-operator-metrics created
   

2. Verify the installation is complete and the clickhouse-operator pod is running:

   kubectl get pods --namespace kube-system
   

   A similar result to the following will be displayed on a successful installation:

   NAME                                   READY   STATUS    RESTARTS      AGE
   clickhouse-operator-77b54889b4-vqgcs   2/2     Running   0             5s
   coredns-78fcd69978-nthp2               1/1     Running   2 (95m ago)   23d
   etcd-minikube                          1/1     Running   2 (95m ago)   23d
   kube-apiserver-minikube                1/1     Running   2 (95m ago)   23d
   kube-controller-manager-minikube       1/1     Running   2 (95m ago)   23d
   kube-proxy-lsggn                       1/1     Running   2 (95m ago)   23d
   kube-scheduler-minikube                1/1     Running   2 (95m ago)   23d
   storage-provisioner                    1/1     Running   5 (94m ago)   23d
   

3. To verify the version of the Altinity Kubernetes Operator, use the following command:

   kubectl get pods -l app=clickhouse-operator --all-namespaces -o jsonpath="{.items[*].spec.containers[*].image}" | tr -s "[[:space:]]" | sort | uniq -c
   

   1 altinity/clickhouse-operator:0.18.1 altinity/metrics-exporter:0.18.1
   

5 - Upgrade Guide

The Altinity Kubernetes Operator can be upgraded at any time by applying the new manifest from the Altinity Kubernetes Operator GitHub repository.

The following procedures detail how to install a specific version of the Altinity Kubernetes Operator in the default Kubernetes namespace kube-system. For instructions on performing custom installations based on the namespace and other settings, see the Custom Installation Guide.

Requirements

The Altinity Kubernetes Operator for Kubernetes has the following requirements:

• Kubernetes 1.15.11+. For instructions on how to install Kubernetes for your particular environment, see the Kubernetes Install Tools page.
• Access to the Altinity Kubernetes Operator GitHub repository.

Instructions

The following instructions are based on installations of the Altinity Kubernetes Operator greater than version 0.16.0. In the following examples, Altinity Kubernetes Operator version 0.16.0 has been installed and will be upgraded to 0.18.1.

For instructions on installing specific versions of the Altinity Kubernetes Operator, see the Specific Version Installation Guide.

1. Deploy the Altinity Kubernetes Operator from the manifest directly from GitHub. It is recommended that the version be specified during the installation for maximum compatibilty. In this example, the version being upgraded to is :

   kubectl apply -f https://github.com/Altinity/clickhouse-operator/raw/0.18.1/deploy/operator/clickhouse-operator-install-bundle.yaml
   

2. The following will be displayed on a successful installation. For more information on the resources created in the installation, see [Altinity Kubernetes Operator Resources]({<ref “operatorresources” >})

   customresourcedefinition.apiextensions.k8s.io/clickhouseinstallations.clickhouse.altinity.com configured
   customresourcedefinition.apiextensions.k8s.io/clickhouseinstallationtemplates.clickhouse.altinity.com configured
   customresourcedefinition.apiextensions.k8s.io/clickhouseoperatorconfigurations.clickhouse.altinity.com configured
   serviceaccount/clickhouse-operator configured
   clusterrole.rbac.authorization.k8s.io/clickhouse-operator-kube-system configured
   clusterrolebinding.rbac.authorization.k8s.io/clickhouse-operator-kube-system configured
   configmap/etc-clickhouse-operator-files configured
   configmap/etc-clickhouse-operator-confd-files configured
   configmap/etc-clickhouse-operator-configd-files configured
   configmap/etc-clickhouse-operator-templatesd-files configured
   configmap/etc-clickhouse-operator-usersd-files configured
   deployment.apps/clickhouse-operator configured
   service/clickhouse-operator-metrics configured
   

3. Verify the installation by running:

   The following will be displayed on a successful installation, with your particular image:

   kubectl get pods --namespace kube-system
   

   NAME                                   READY   STATUS    RESTARTS         AGE
   chi-simple-02-cl-0-0-0                 1/1     Running   6 (27d ago)      46d
   clickhouse-operator-77b54889b4-w849x   2/2     Running   0                4s
   coredns-78fcd69978-ftd6f               1/1     Running   15 (27d ago)     85d
   etcd-minikube                          1/1     Running   15 (27d ago)     85d
   kube-apiserver-minikube                1/1     Running   16 (27d ago)     85d
   kube-controller-manager-minikube       1/1     Running   15 (27d ago)     85d
   kube-proxy-lhbnt                       1/1     Running   15 (27d ago)     85d
   kube-scheduler-minikube                1/1     Running   15 (27d ago)     85d
   storage-provisioner                    1/1     Running   56 (6h42m ago)   85d
   

4. To verify the version of the Altinity Kubernetes Operator, use the following command:

   kubectl get pods -l app=clickhouse-operator -n kube-system -o jsonpath="{.items[*].spec.containers[*].image}" | tr -s "[[:space:]]" | sort | uniq -c
   

   1 altinity/clickhouse-operator:0.18.1 altinity/metrics-exporter:0.18.1