Altinity.Cloud Anywhere 101

17 May 2023 · Read time 4 min

Introduction and Benefits

Altinity.Cloud Anywhere provides the convenient cloud management of Altinity.Cloud but allows users to keep data within their own cloud VPCs and private data centers, and run managed ClickHouse in their own Kubernetes clusters. We call these clusters Altinity.Cloud Anywhere environments.

Altinity.Cloud Anywhere offers several important benefits for users.

• Compliance - Retain full control of data (including backups) as well as the operating environment and impose your policies for security and privacy.
• Cost - Optimize infrastructure costs by running in your accounts.
• Location - Place ClickHouse clusters close to data sources and applications.
• Vendor Unlocking - Disconnect at any time and continue to operate ClickHouse using open-source components.

The rest of this document explains concepts that help users understand Altinity.Cloud Anywhere and maximize benefits.

The Altinity.Cloud Manager UI manages Altinity.Cloud Anywhere environments are just like fully hosted Altinity.Cloud environments. Users can control multiple environments from the same Altinity.Cloud account and can mix/match environment types. ClickHouse management operations are identical in all environments.

Service Architecture

The Altinity.Cloud service architecture consists of a shared management plane that serves as a single point of management for all tenants and a data plane that consists of isolated environments for each tenant. The following diagram shows the service architecture and data plane relationships.

Figure 1 - Service Architecture.

Each environment is a dedicated Kubernetes cluster. In the case of Altinity.Cloud environments, Kubernetes clusters run on Altinity’s cloud accounts and are completely hidden from users. In the Altinity.Cloud Anywhere case, Kubernetes clusters run in the user’s cloud account or data center.

For example, the user may run an EKS cluster within a VPC belonging to the user’s AWS cloud account.

Altinity.Cloud Anywhere environments can also use on-prem Kubernetes clusters. They can even use development versions of Kubernetes running on a user’s PC or laptop.

Open Source Analytic Stack

Altinity.Cloud Anywhere uses open-source software for the analytic stack and selected management services–the Altinity Operator for ClickHouse, Prometheus, and Grafana. The following diagram shows how the principal components map to resources in AWS. (GCP is essentially identical.) Open-source components are marked in orange.

Figure 2 - Management and observability.

Users can terminate the service, and disconnect the Altinity.Cloud Anywhere environment from Altinity.Cloud, and run ClickHouse services themselves. There is no migration, since all data, software, and support services are already in the user Kubernetes cluster.

Altinity.Cloud Anywhere Connectivity Model

Altinity.Cloud Anywhere environments use the Altinity Connector to establish a management connection from the user Kubernetes cluster to Altinity.Cloud. The Altinity Connector establishes an outbound HTTPS connection to a management endpoint secured by certificates. This allows management commands and monitoring data to move securely between locations.

Users connect an Altinity.Cloud Anywhere environment to Altinity.Cloud in three simple steps.

1. Download the Altinity Connector executable program (altinitycloud-connect).
2. Run and register Altinity Connector with Altinity.Cloud Manager.
   • If Altinity Connector is installed on a separate VM, it may run provisioning of the Kubernetes cluster (EKS, GKE, AKS). This process deploys a new instance of Altinity Connector into the provisioned Kubernetes cluster as well.
   • When Altinity Connector is installed directly in Kubernetes, it runs the provisioning of Kubernetes resources.
3. Complete registration in the Altinity.Cloud Manager.

Altinity.Cloud Anywhere environments run all services in two namespaces.

• The altinity-cloud-system namespace contains system services including the Altinity Connector.
• The altinity-cloud-managed-clickhouse namespace contains ClickHouse and ZooKeeper. Users can run services in other namespaces provided they do not make changes to the Altinity-managed namespaces.

See the Quickstart page for steps to register an Altinity.Cloud Anywhere environment.

Kubernetes Cluster Preparation for Use

Kubernetes clusters must meet a small number of requirements to serve as an Altinity.Cloud Anywhere environment for production use.

• Configure storage classes that can allocate block storage on-demand, for example using the AWS EBS CSI driver.
• Enable auto-provisioning, e.g., node groups or Karpenter. This allows Altinity.Cloud to expand or contract clusters as well as rescale server pods efficiently.
• Kubernetes pods must be able to connect to S3-compatible object storage or GCS (Google Cloud Storage). Object storage is used for backups.

These requirements can be relaxed for non-production environments, such as Minikube. Check the Kubernetes Requirements page for more recommendations on specific Kubernetes distributions.

Shared Administration between Altinity.Cloud and User

In Altinity.Cloud Anywhere environments the responsibility for administration is shared between Altinity and users. The following table shows major system components.

Table 1 - Altinity.Cloud Anywhere Environment - Administrative Responsibility.

Altinity is developing a new model called Altinity.Cloud Anywhere Plus. It will shift responsibility for Kubernetes and VPC management to Altinity. Contact Altinity Support for more information on this model.