Minikube Installation (for test or development only)

How to install Altinity.Cloud Anywhere on Minikube. For testing and development use only.

24 April 2023 · Read time 30 min

Overview - Minikube Installation (for testing and development use only)

This guide covers the installation of Minikube in your own Kubernetes environment by using Altinity.Cloud Anywhere to do the provisioning. Any computer or cloud instance that can run Kubernetes and Minikube will work. Note that while Minicube is ok to use for development purposes, it should not be used for production.

These instructions have been tested on:

  • Ubuntu 22.04 server
  • Windows 10 with WS2 Ubuntu 20.04
  • VMWare running Ubuntu on Intel & M1 ARM
  • M1 Silicon Mac running Monteray (v12.6.3) and Ventura (v13.3.1)
  • Intel Mac running Big Sur (v11.7.4)


The following Altinity.Cloud service subscriptions are needed:

Server requirements Minikube needs a minimum of 2 processors. Allocate RAM and disk space to accommodate your clusters. Check the values by running the terminal commands (Example: lscpu).

  • Minimum of 2 CPU (lscpu, or sysctl -a [for Mac])
  • Minimum 8 GB RAM (grep MemTotal /proc/meminfo )
  • 30 GB disk space ( df -h)

The following software must first be installed on your Minikube installation:

The following software is installed as part of the provisioning:


From a terminal, check the versions of all the installed software by running each command in turn.

Checking versions To make sure you have the required software installed, check the versions for each using the following commands:

docker --version
docker-machine --version
docker-compose --version
minikube version
kubectl version -o json
watch --version
k9s version 
altinitycloud-connect version

Starting Minikube

From the terminal, run the command:

minikube start

Linux ARM Ubuntu 22.04 This is Minikube’s response from an Ubuntu 22.04 server running on ARM:

# minikube start

😄  minikube v1.30.1 on Ubuntu 22.04 (arm64)
✨  Using the qemu2 driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🔄  Restarting existing qemu2 VM for "minikube" ...
🐳  Preparing Kubernetes v1.26.3 on Docker 20.10.23 ...
🔗  Configuring bridge CNI (Container Networking Interface) ...
    ▪ Using image
🔎  Verifying Kubernetes components...
🌟  Enabled addons: default-storageclass, storage-provisioner
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

Linux ARM Apple Macintosh M1 This is Minikube’s response from a Mac running Ventura:

# minikube start

😄  minikube v1.29.0 on Darwin 13.2.1 (arm64)
✨  Using the docker driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
🏃  Updating the running docker "minikube" container ...
🐳  Preparing Kubernetes v1.26.1 on Docker 20.10.23 ...
🔗  Configuring bridge CNI (Container Networking Interface) ...
🔎  Verifying Kubernetes components...
    ▪ Using image
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

Linux Windows Intel This is Minikube’s response from a Microsoft Windows system running Ubuntu:

# minikube start

😄  minikube v1.30.1 on Ubuntu 20.04
✨  Using the docker driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
🔄  Restarting existing docker container for "minikube" ...
🐳  Preparing Kubernetes v1.26.3 on Docker 23.0.2 ...
🔗  Configuring bridge CNI (Container Networking Interface) ...
🔎  Verifying Kubernetes components...
    ▪ Using image
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" 

Checking Minikube’s status If you are not sure if Minikube is already running, run a status check as follows:

minikube status
# minikube
# type: Control Plane
# host: Running
# kubelet: Running
# apiserver: Running
# kubeconfig: Configured

Checking the Kubernetes kubectl command This step checks that the kubectl command works on your Minikube host. Running the kubectl get ns command lists the namespaces that are currently running on your Minikube server.

Run the kubectl namespace list command:

kubectl get ns

# Example response:
# -------------------
# NAME              STATUS   AGE
# default           Active   15d
# kube-node-lease   Active   15d
# kube-public       Active   15d
# kube-system       Active   15d

Altinity Connection Setup

To start the Connection Setup:

  1. From the Altinity Cloud Manager, select the Environments section, then make sure you are in the correct environment by selecting it from the menu located at the top right of the screen.

  2. In Figure 1 the Connection Setup step 2, Connect to Altinity.Cloud text box, select all the text.

  3. In your Minikube terminal, copy and paste the text and press the return key. A command prompt appears immediately.

altinitycloud-connect login --token=
eyJhbGciOiJSUzI1Ni        808 characters           Rpbml0eS5jbG91ZCIsImV4cCI6MTY3
OTMzNzMwMywic3ViIjoicm1rYzIzdGVzdC1kNDgxIn0.tODyYF8WnTSx6mbAZA5uwW176... cont.

Example altinitycloud-connect login token string from the Altinity Cloud Manager Connection Setup wizard step 2, Connect to Altinity.Cloud.

Starting the Provisioning

From Figure 1, in the Connection Setup screen step 3, Deploy connector to your Kubernetes cluster, copy the string and paste it into your terminal. This begins the provisioning process inside your Minikube Kubernetes environment.

altinitycloud-connect kubernetes | kubectl apply -f -

Response The response appears similar to the following:

namespace/altinity-cloud-system created
namespace/altinity-cloud-managed-clickhouse created unchanged unchanged unchanged unchanged unchanged
serviceaccount/cloud-connect created unchanged unchanged unchanged unchanged unchanged created created
secret/cloud-connect created
deployment.apps/cloud-connect created

1 of 3 Connection Setup

From the Altinity Cloud Manager Connection Setup page, select the green PROCEED button.

Figure 1 - The Environments > Connection Setup screen.

2 of 3 Resources Configuration

Confirm the following settings, then select the green PROCEED button:

Figure 2 - The Resources Configuration screen.

NOTE: In Figure 2, if the table for the Node Pools section does not include a row for your Minikube server, select the ADD NODE POOL button and add the Zone name and Instance Type name and Capacity, and check each of the Used For checkboxes as shown.

  • Cloud Provider = Not Specified
  • Storage Classes = Standard
  • Node Pools:
    • Zone = minikube-zone-a
    • Instance Type = minikube-node
    • Capacity = 10 GB (this is an example setting)
    • Used for: (checkmark each of these items)
      • ClickHouse (checked on)
      • Zookeeper (checked on)
      • System (checked on)
    • Tolerations = dedicated=clickhouse:NoSchedule

3 of 3 Confirmation

In Figure 3, the Confirmation tab displays the Resources Specifications text box. Review these values and correct them if necessary by selecting the Resources Configuration tab to make changes.

To complete the Connection Setup wizard:

  1. Select the green Finish button.

    • A progress bar and message appear: “Connection is not ready yet.
  2. Select “Continue waiting…” until the next screen appears.

Figure 3 - The Confirmation screen showing the Resources Specification JSON and the Connection is not ready yet message that appears until the connection to your Minikube is established.

In the Confirmation screen shown in Figure 3, an example Resources Specification JSON string appears with the names of the storageClasses, nodePools and instanceType, zone and capacity value.

    "storageClasses": [
        "name": "standard"
    "nodePools": [
        "for": [
        "instanceType" : "minikube-node",
        "zone"         : "minikube-zone-a",
        "capacity"     : 10

Node Registration

The following step registers your Minikube label so that the ACM can find your ClickHouse Kubernetes server that Altinity.Cloud just provisioned for you. Refer to the Resource Specification JSON for the values for instanceType minikube-node and zone name minikube-zone-a where they are set.

Run the following string from your Kubernetes host terminal.

kubectl --context=minikube label nodes minikube \ \

Optional Watch Commands

To monitor in real time the progress of a provisioning installation, run the watch commands on the two altinity-cloud prefixed namespaces.

Running Watch command 1 of 2 To monitor the process of the provisioning, use the watch or k9s command utility to monitor altinity-cloud-system. The display updates every 2 seconds.

watch kubectl -n altinity-cloud-system get all

Response The result appears similar to the following display:

Every 2.0s: kubectl -n altinity-cloud-system get all                     john.doe-yourcomputer.local: Sun Mar 19 23:03:18 2023

NAME                                      READY   STATUS    RESTARTS   AGE
pod/cloud-connect-d6ff8499f-bkc5k         1/1     Running   0          10h
pod/crtd-665fd5cb85-wqkkk                 1/1     Running   0          10h
pod/edge-proxy-66d44f7465-t9446           2/2     Running   0          10h
pod/grafana-5b466574d-vvt9p               1/1     Running   0          10h
pod/kube-state-metrics-58d86c747c-7hj79   1/1     Running   0          10h
pod/node-exporter-762b5                   1/1     Running   0          10h
pod/prometheus-0                          1/1     Running   0          10h
pod/statuscheck-f7c9b4d98-2jlt6           1/1     Running   0          10h

NAME                          TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                                       AGE
service/edge-proxy            ClusterIP      <none>        443/TCP,8443/TCP,9440/TCP                     10h
service/edge-proxy-lb         LoadBalancer   <pending>     443:31873/TCP,8443:32612/TCP,9440:31596/TCP   10h
service/grafana               ClusterIP     <none>        3000/TCP                                      10h
service/prometheus            ClusterIP   <none>        9090/TCP                                      10h
service/prometheus-headless   ClusterIP      None             <none>        9090/TCP                                      10h
service/statuscheck           ClusterIP   <none>        80/TCP                                        10h

daemonset.apps/node-exporter   1         1         1       1            1           <none>          10h

NAME                                 READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/cloud-connect        1/1     1            1           10h
deployment.apps/crtd                 1/1     1            1           10h
deployment.apps/edge-proxy           1/1     1            1           10h
deployment.apps/grafana              1/1     1            1           10h
deployment.apps/kube-state-metrics   1/1     1            1           10h
deployment.apps/statuscheck          1/1     1            1           10h

NAME                                            DESIRED   CURRENT   READY   AGE
replicaset.apps/cloud-connect-d6ff8499f         1         1         1       10h
replicaset.apps/crtd-665fd5cb85                 1         1         1       10h
replicaset.apps/edge-proxy-66d44f7465           1         1         1       10h
replicaset.apps/grafana-5b466574d               1         1         1       10h
replicaset.apps/grafana-6478f89b7c              0         0         0       10h
replicaset.apps/kube-state-metrics-58d86c747c   1         1         1       10h
replicaset.apps/statuscheck-f7c9b4d98           1         1         1       10h

NAME                          READY   AGE
statefulset.apps/prometheus   1/1     10h

Running Watch command 2 of 2 Open a second terminal window to monitor altinity-cloud-managed-clickhouse.

watch kubectl -n altinity-cloud-system get all

Response The result appears similar to the following display:

Every 2.0s: kubectl -n altinity-cloud-managed-clickhouse get all        john.doe-yourcomputer.local: Mon Mar 20 00:14:44 2023

NAME                                            READY   STATUS    RESTARTS   AGE
pod/chi-test-anywhere-6-test-anywhere-6-0-0-0   2/2     Running   0          11h
pod/clickhouse-operator-996785fc-rgfvl          2/2     Running   0          11h
pod/zookeeper-5244-0                            1/1     Running   0          11h

NAME                                              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
service/chi-test-anywhere-6-test-anywhere-6-0-0   ClusterIP    <none>        8123/TCP,9000/TCP,9009/TCP   11h
service/clickhouse-operator-metrics               ClusterIP   <none>        8888/TCP                     11h
service/clickhouse-test-anywhere-6                ClusterIP    <none>        8443/TCP,9440/TCP            11h
service/zookeeper-5244                            ClusterIP    <none>        2181/TCP,7000/TCP            11h
service/zookeepers-5244                           ClusterIP   None            <none>        2888/TCP,3888/TCP            11h

NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/clickhouse-operator   1/1     1            1           11h

NAME                                           DESIRED   CURRENT   READY   AGE
replicaset.apps/clickhouse-operator-996785fc   1         1         1       11h

NAME                                                       READY   AGE
statefulset.apps/chi-test-anywhere-6-test-anywhere-6-0-0   1/1     11h
statefulset.apps/zookeeper-5244                            1/1     11h

Optional K9S Commands

Similar to Watch, but in color and in a smaller interactive window, K9S is a free utility that lets you monitor in real time the progress of a provisioning installation.

To open monitoring windows for each altinity-cloud namespaces, open a new terminal instance and run the k9s command:

k9s -n altinity-cloud-system
k9s -n altinity-cloud-managed-clickhouse

KS9 monitoring terminal windows

Figure 4 - The k9s monitoring windows for the two namespaces altinity-cloud-system and altinity-cloud-managed-clickhouse listing each node name, IP address, and the run status.

Environment Dashboard

When provisioning is complete and the connection is established, the ACM displays the dashboard page showing the green connected icon. Since there is no cluster yet, the dashboard shows zeros for the number of Nodes and Clusters.

Figure 5 - The Environments dashboard screen shows you a snapshot of your Minikube server configuration, including the green connected status.

Listing Namespaces

To verify the presence of the new namespaces on your Minikube server, open a third terminal window and list the namespaces to show the two altinity-cloud additions:

kubectl get ns

Response Note the two new altinity-cloud namespaces at the top:

NAME                                STATUS   AGE
altinity-cloud-managed-clickhouse   Active   8h
altinity-cloud-system               Active   8h
default                             Active   16d
kube-node-lease                     Active   16d
kube-public                         Active   16d
kube-system                         Active   16d

Creating a ClickHouse Cluster

These instructions run through the use of the Altinity.Cloud Manager (ACM) Clusters > LAUNCH CLUSTER wizard to create a ClickHouse cluster running in a Minikube Kubernetes environment. The Cluster dashboard in Figure 6 shows the finished result.

Figure 6 - The Clusters dashboard screen showing your new cluster on your Minikube server created by the Altinity Cloud Manager.

To create a new ClickHouse Cluster using the Launch Cluster wizard:

NOTE: The Cluster Launch Wizard lets you navigate back and forth between the previously filled-in screens by selecting the title links on the left, or using the BACK and NEXT buttons.

  1. In the Altinity Cloud Manager, select Clusters.
  2. Select the LAUNCH CLUSTER blue button.
  3. In step 1 ClickHouse Setup screen, fill in the following, and select the blue NEXT button:
    • Name = test-anywhere (15-character limit, lower-case letters only)
    • ClickHouse Version = ALTINITY BUILDS: 22.8.13 Stable Build
    • ClickHouse User Name = admin
    • ClickHouse User Password = admin-password
  4. In step 2 Resources Configuration screen, fill in the following then select the NEXT button:
    • Node Type = minikube-node (CPU xnull, RAM pending)
    • Node Storage = 10 GB
    • Number of Volumes = 1
    • Volume Type = standard
    • Number of Shards = 1
  5. In step 3 High Availability Configuration screen, fill in the following then select NEXT:
    • Number of Replicas = 1
    • Zookeeper Configuration = Dedicated
    • Zookeeper Node Type = default
    • Enable Backups = OFF (unchecked)
    • Number of Backups to keep = 0 (leave blank)
  6. In step 4 Connection Configuration screen, fill in the following then select NEXT:
    • Endpoint =
    • Use TLS = Checked
    • Load Balancer Type = Altinity Edge Ingress
    • Protocols: Binary Protocol (port:9440) - is checked ON
    • Protocols: HTTP Protocol (port:8443) - is checked ON
    • Datadog integration = disabled
    • IP restrictions = OFF (Enabled is unchecked)
  7. In step 5 Uptime Schedule screen, select ALWAYS ON then NEXT:
  8. In the final screen step 6 Review & Launch, select the green LAUNCH button.

Your new ClickHouse Cluster will start building inside your Minikube. When the cluster is finished building and running, the cluster dashboard appears, similar to the screenshot shown in Figure 6. Beside your cluster name, two green status boxes nodes online, and checks passed appear.

Creating a Database and Running Queries

In this section, you will create tables on your cluster using the ACM and run queries from both the ACM and then from your local terminal.

Testing your database on ACM

To create a new database on your Altinity.Cloud Anywhere cluster from the ACM:

  1. Login to the ACM and select Clusters, then select EXPLORE on your cluster.
  2. In the Query text box, enter the following create table SQL query:
CREATE TABLE IF NOT EXISTS events_local ON CLUSTER '{cluster}' (
    event_date  Date,
    event_type  Int32,
    article_id  Int32,
    title       String
) ENGINE = ReplicatedMergeTree('/clickhouse/{cluster}/tables/{shard}/{database}/{table}', '{replica}')
    PARTITION BY toYYYYMM(event_date)
    ORDER BY (event_type, article_id);
  1. Create a second table:
CREATE TABLE events ON CLUSTER '{cluster}' AS events_local
   ENGINE = Distributed('{cluster}', default, events_local, rand())
  1. Add some data with this query:
INSERT INTO events VALUES(today(), 1, 13, 'Example');
  1. List the data you just entered:
SELECT * FROM events;

# Response (query time: 0.196s)
│ 2023-03-24 │          113 │ Example │
  1. Show all the tables:
show tables

# Response (query time: 0.275s)
│ events       │
│ events_local │

Testing ClickHouse on your local terminal

This section shows you how to use your local Minikube computer terminal to log into your Clickhouse Cluster that ACM created. NOTE: With Minikube, you cannot use your cluster Connection Details strings to directly run clickhouse-client commands, you must first log into the ClickHouse pod as described in the following steps.

  1. Find your pod name:
kubectl -n altinity-cloud-managed-clickhouse get all

# Response
NAME                                               READY   STATUS    RESTARTS        AGE
pod/chi-test-anywhere-6-johndoe-anywhere-6-0-0-0   2/2     Running   8 (3h25m ago)   2d17h
  1. On your Minikube computer terminal, log into that pod using the name you got from step 1:
kubectl -n altinity-cloud-managed-clickhouse exec -it pod/chi-test-anywhere-6-johndoe-anywhere-6-0-0-0 -- bash

# Response
Defaulted container "clickhouse-pod" out of: clickhouse-pod, clickhouse-backup
  1. Log into your ClickHouse database using the clickhouse-client command to get the :) happy face prompt:
clickhouse@chi-test-anywhere-6-johndoe-anywhere-6-0-0-0:/$ clickhouse-client

# Response
<jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
<jemalloc>: (This is the expected behavior if you are running under QEMU)
ClickHouse client version (altinity build).
Connecting to localhost:9000 as user default.
Connected to ClickHouse server version 22.8.13 revision 54460.

test-anywhere-6 :) 
  1. Run a show tables SQL command:
test-anywhere-6 :) show tables

# Response


Query id: da01133d-0130-4b98-9090-4ebc6fa4b568

│ events       │
│ events_local │

2 rows in set. Elapsed: 0.013 sec.  
  1. Run the following SQL query to show data in the events table:
test-anywhere-6 :) SELECT * FROM events;

# Response

FROM events

Query id: 00fef876-e9b0-44b1-b768-9e662eda0483

│ 2023-03-24 │          113 │ Example │

1 row in set. Elapsed: 0.023 sec. 

test-anywhere-6 :) 

Exiting from ClickHouse client and your pod

  1. To leave the ClickHouse client, enter the exit command.
  2. To leave the pod and return to the Linux prompt enter the exit command again.
  3. Verify you are at the command prompt by entering a linux command such as pwd (print working directory) to see what directory you are currently in.
aws-anyw-test :) exit

clickhouse@chi-was-anyw-test-was-anyw-test-0-0-0:/$ exit

ubuntu@ip-172-31-16-238:~$ pwd

Review the following database creation and query instructions:


This section provides a few commonly used Minikube maintenance operations.

Rescaling a cluster

Use the Altinity Cloud Manager Actions 》Rescale to change the CPU, Node Storage, Volumes, and Number of Shards and Replicas.

  1. From the list of Clusters, select a running cluster.
  2. Select the menu ACTIONS > Rescale item.
  3. In the Rescale Cluster window, adjust the following settings as needed:
  • Desired Cluster Size > Number of Shards
  • Desired Cluster Size > Number of Replicas
  • Desired Node Size > Node Type
  • Desired Node Storage (GB) > (integer: example 50)
  • Number of Volumes > (integer: example 2)
  1. Select OK, then CONFIRM at the Rescale Confirmation window.
  2. Confirm that the new values appear in your cluster dashboard panel.

Note that cluster Node Storage size may not be decreased, only increased by at least 10%.

Resetting Altinity.Cloud Anywhere

Reset your Altinity.Cloud Anywhere cluster from the ACM and your Minikube installation to create a new Altinity.Cloud Anywhere connection.

To use the Reset Anywhere function.

  1. In the ACM, select Environments from the left-hand navigation pane.
  2. From the environment menu located beside your login name at the top right of the ACM, select your environment name.
  3. In the ACTION menu, select Reset Anywhere.

The result is that you will see the Anywhere Connection Setup screen and provisioning wizard that shows you the connection string to copy and paste to deploy a new Anywhere environment.

Deleting a cluster

Deletion steps involve the ACM and the server hosting your cluster. If necessary, first Reset Anywhere.

From the Altinity Cloud Manager:

  1. In the Clusters section, select from your cluster menu ACTIONS > Destroy.
  2. At the Delete Cluster confirmation dialog box, type in the name of your cluster (example-cluster) and select OK.
  3. From the Environments section, select your Environment Name link.
  4. Select the menu ACTIONS > Reset Anywhere.

To list the ClickHouse namespaces, delete your Kubernetes managed environments from your server and run the following commands: (NOTE: Make sure you have run the minikube start command first. )

# List the namespaces
kubectl get ns

# Delete the following in this order
kubectl -n altinity-cloud-managed-clickhouse delete chi --all
kubectl delete ns altinity-cloud-managed-clickhouse
kubectl delete ns altinity-cloud-system

Deleting a pod

Deleting a pod may be necessary if it is not starting up.


One of the pods won’t start. (Example: see line 3 edge-proxy-66d44f7465-lxjjn)

    ┌──────────────── Pods(altinity-cloud-system)[8] ──────────────────────────┐
    │ NAME↑                                PF READY RESTARTS STATUS            │
 1  │ cloud-connect-d6ff8499f-bkc5k        ●  1/1       3    Running           │
 2  │ crtd-665fd5cb85-wqkkk                ●  1/1       3    Running           │
 3  │ edge-proxy-66d44f7465-lxjjn          ●  1/2       7    CrashLoopBackOff  │
 4  │ grafana-5b466574d-4scjc              ●  1/1       1    Running           │
 5  │ kube-state-metrics-58d86c747c-7hj79  ●  1/1       6    Running           │
 6  │ node-exporter-762b5                  ●  1/1       3    Running           │
 7  │ prometheus-0                         ●  1/1       3    Running           │
 8  │ statuscheck-f7c9b4d98-2jlt6          ●  1/1       3    Running           │

Terminal listing 1 - The pod in Line 3 edge-proxy-66d44f7465-lxjjn won’t start.


Delete the pod using the kubectl delete pod command and it will regenerate.
(Example: see line 3 edge-proxy-66d44f7465-lxjjn)

kubectl -n altinity-cloud-system delete pod edge-proxy-66d44f7465-lxjjn

Stopping minikube

To stop the Minikube service, run the following command:

minikube stop
✋  Stopping node "minikube"  ...
🛑  1 node stopped.

Last modified 2023.06.06: Fix broken Quickstart link