Altinity Kubernetes Operator for ClickHouse® Release Notes

The latest release of the Altinity Kubernetes Operator for ClickHouse® is always available at https://github.com/Altinity/clickhouse-operator/releases/latest. Links to release notes for older releases follow the notes for the current release.

Release 0.27.4

Released 2026-09-21

Security

  • BACKWARD INCOMPATIBLE. Removed the k8s_secret_ and k8s_secret_env_ user-settings syntax. A CHI that still uses the syntax is now rejected with RemovedSecretRefSyntax. Migrate to valueFrom/secretKeyRef syntax unstead (see docs/security_hardening.md):

    users:
  user1/password:
    valueFrom:
      secretKeyRef:
        name: clickhouse-secret
        key: pwduser1

    by @sunsingerus

  • Bumped the Go toolchain 1.26.5 → 1.26.8 to address stdlib CVEs in the operator and metrics-exporter images (GO-2026-6090, GO-2026-6089, GO-2026-6218, GO-2026-5972, GO-2026-5026). No API or behavior changes. The FIPS module (GOFIPS140=v1.0.0) is unchanged. by @sunsingerus

  • Bumped the OpenTelemetry SDK v1.43.0 → v1.45.0 (otel, otel/metric, otel/sdk, otel/sdk/metric, otel/trace). Clears CVE-2026-41178, an uncapped baggage header parse in otel/baggage and otel/propagation, both of which the operator links. Also clears CVE-2026-81870; it is filed against otel/sdk, so module-keyed scanners flagged us, but it logs trace exporter endpoints and the operator links no trace SDK or trace exporter. by @sunsingerus

Behavior Changes

  • A pod that will not terminate is force-deleted so StatefulSet Recreate finishes in one pass. A wedged shutdown previously aborted Recreate with the host left at Replicas=0. The operator now deletes that pod after a full wait timeout, emits HostPodForceDeleted, and completes Delete+Create in the same pass. A terminating pod is no longer treated as a healthy shard peer. by @alex-zaitsev in #2080. Fixes #2078

  • ZooKeeper endpoint changes no longer restart ClickHouse. configurationRestartPolicy has advertised zookeeper/*: "no" since 0.25.6; the policy is now honored, so endpoint edits reload via conf.d with no restarts. This also removes the one-time CHI→CHK restart noted in 0.27.2. by @identw in #2074

  • Keeper rolling updates no longer take an ensemble below Raft quorum. A Ready Keeper replica is deferred ([RaftQuorumUnsafe]) when disrupting it would lose majority; not-Ready replicas are recovered first. Affects ClickHouseKeeperInstallation rolls. by @alex-zaitsev in #2070. Fixes #2069

  • Helm CRD-install hook uses registry.k8s.io/kubectl:v1.36.3 instead of bitnami/kubectl:latest. Air-gapped and mirrored installs must add that image before upgrade, or set crdHook.image / crdHook.enabled: false. Helm installs only. by @fhoekstra in #2065

  • A failed schema migration no longer reports the CHI as Completed. HostCreateTables errors now abort the pass instead of being recorded as success. A CHI that already had an un-creatable object (for example a Dictionary with an unreachable source) will turn Aborted on first reconcile after upgrade. by @Tyagiquamar in #2077. Fixes #2021

  • Operator informers and caches no longer hold every object in the Kubernetes cluster. They are narrowed at the API server to clickhouse.altinity.com/app=chop, and clickhouse-keeper.altinity.com/app=chop respectively, so memory scales with the number of managed installations rather than cluster size. by @sunsingerus. Approach proposed by @gregakinman in #2058

  • ActionPlan is rendered once on status update. A status update invoked String() many times per retry, and each call reflection-dumped every diffed object, so on a large CHI that dominated operator CPU. by @erikasb in #2067

  • A failed Keeper start no longer disables Keeper reconciliation until operator restart. Start-up now contacts the API server (required by the narrowed cache), so a transient error can fail the first attempt. The operator retries indefinitely without exiting; ClickHouse reconciliation is not blocked. by @sunsingerus

  • OTEL_GO_X_CARDINALITY_LIMIT no longer caps operator metrics. The SDK bump brings a default of 2000 data points per instrument per scrape, which silently folds everything past it into one otel.metric.overflow series; the operator now pins the limit to unlimited, so series keep scaling with the installations watched. Anyone who set the variable to protect Prometheus must limit at the scrape instead. by @sunsingerus

Fixed

  • Host deletion is no longer reported as completed when the operator cannot tell whether the host still exists. Only NotFound means the host is gone; other API errors now raise DeleteFailed. Present since 0.10.0. Residual PVC-orphan on finalizer removal is tracked in #2056. by @somanchi004-code in #2057

  • Operator no longer keeps probing a ZooKeeper ensemble it can no longer reach. A disconnected ZK session leaked the client and re-dialed stale addresses until operator restart. Every terminal state now closes the connection. Root-path ensure is cancellable, skipped on empty recovery passes, and raises CreateFailed after retries (still non-fatal). by @alex-zaitsev in #2073

  • Oversized CRD ConfigMap broke GitOps installs of the Helm chart. The CRD-install hook packed three CRDs into one ConfigMap past the 256Ki annotation ceiling, so ArgoCD client-side apply and helm template | kubectl apply failed. Each CRD now gets its own ConfigMap. helm install / helm upgrade were unaffected. by @sunsingerus

  • Stale ClickHouse HTTP connection pools are reset after connection failures. by @norrs in #2068

  • A failed read is no longer mistaken for a missing object. Only NotFound means absent; any other error — RBAC still propagating, an overloaded API server — escalated to recovery, deleting a StatefulSet and its pods once a later read succeeded, or reporting phantom data loss on a Keeper PersistentVolumeClaim. Such reads now raise a host reconcile error and retry on the next pass. Affects ClickHouse and Keeper alike. by @sunsingerus

  • Lost-volume recovery no longer ignores shutdown. The poll that waits up to an hour for a PersistentVolumeClaim to disappear did not observe the operator context, so a stop issued during it could be held up until the poll ran out. by @sunsingerus

New Contributors

Full Changelog: https://github.com/Altinity/clickhouse-operator/compare/release-0.27.3...release-0.27.4

Release 0.27.3

Released 2026-08-12

Added

  • Helm: scrape native ClickHouse Keeper Prometheus metrics when serviceMonitor.enabled is set, by @Slach in #2048. Closes #2038. Opt out with serviceMonitor.keeperMetrics.enabled: false. Keeper must expose a metrics port; clusters without Keeper are unaffected.
serviceMonitor:
  enabled: true
  keeperMetrics:
    enabled: true
    port: metrics

Changed

  • Protect the last healthy replica in a shard during an interrupted roll. Closes #1704. Replaces #1967. Unhealthy replicas in a shard are reconciled before healthy ones; a disruptive restart or image roll is deferred when it would take down the last healthy replica. Sibling shards keep converging — a deferred host no longer aborts the rest of the pass. The CHI still reports Aborted with event reason HostReconcileDeferredShardSafety until the deferred host can be rolled.

Fixed

  • Image upgrades again exclude the host first (LB drain, remote_servers de-prioritisation, in-flight query drain). Closes #2055. That was a regression since 0.26.3.
  • Honor onLostVolume: no so the operator does not SYSTEM DROP REPLICA when a volume is lost. The setting existed but was ignored on the ForceDrop path.
reconcile:
  host:
    drop:
      replicas:
        onLostVolume: no
  • Retry transient Kubernetes API errors on CHI kube Get calls, by @aaron276h in #2025. Closes #2026. Bounded exponential backoff (5 attempts, ~7.5s) for connection refused/reset, timeouts, and apiserver 5xx — a brief control-plane blip no longer aborts a large reconcile. Terminal errors (NotFound, Forbidden, …) still fail immediately.
  • Fix informer delete handling for DeletedFinalStateUnknown tombstones, by @27rohan in #2039. Closes #1882. Missed-watch relist payloads no longer panic the operator; malformed delete events are logged instead.
  • Skip cluster/host reconcile hooks on unreachable hosts (e.g. a scale-up replica that has no pod yet, or a dying host that cannot serve SQL). Closes #2052. instead of failing the hook and blocking reconcile. Emits HookSkippedUnreachableHost. Reachability overrides failurePolicy: Fail. The hook still runs on hosts that can be reached.

Security

  • Bumped dependencies to address CVEs in the operator and metrics-exporter images. An image scan flagged four CVEs; all are addressed by patch/minor bumps with no API or behavior changes:

    • CVE-2026-39822 (HIGH) and CVE-2026-42505 (MEDIUM) — Go standard library. Fixed by building on Go 1.26.5 (go.mod 1.26.4 → 1.26.5).
    • CVE-2026-46600 — golang.org/x/net bumped v0.55.0 → v0.56.0.
    • CVE-2026-56852 — golang.org/x/text bumped v0.37.0 → v0.39.0.

    Reachability note: govulncheck confirms the stdlib crypto/tls and x/text issues are reachable from operator/exporter code; the x/net and stdlib os issues are present-in-image but not reachable. All four are bumped for image hygiene regardless. The FIPS module (GOFIPS140=v1.0.0) is pinned independently of the Go toolchain and is unaffected.

New Contributors

Full Changelog: https://github.com/Altinity/clickhouse-operator/compare/release-0.27.2...release-0.27.3

Release 0.27.2

Released 2026-07-23

Added

reconcile:
  runtime:
    reconcileCHKsThreadsNumber: 1

reconcile:
  recovery:
    onStatus:
      # Recovery for a CHI whose .status.status is Aborted (reconcile did not complete)
      # when one of its host pods transitions to Ready — auto-resumes the reconcile.
      aborted:
      completed:
        # Action when a Completed CHI's pod flips Ready=True -> Ready=False and
        # stays NotReady for at least onPodNotReadyThreshold:
        #   none (default) — do nothing
        #   retry          — re-enqueue the CHI so the stuck host is force-restarted
        # OFF by default: force-recreating a Completed CHI's pod is destructive — it can
        # interrupt a replica's in-progress recovery and means hard downtime for a
        # single-replica shard. Opt in with `retry` only where that trade-off is acceptable.
        onPodNotReady: none
        # Minimum duration a pod must stay Ready=False before recovery fires, once enabled
        # (Go duration string; default 5m). Raise it for slow-recovering replicas.
        onPodNotReadyThreshold: 5m
  • Configure operator rootCA with a secret. Closes #1925.
 clickhouse:
    access:
      scheme: https
      port: 8443
      rootCASecretRef:
        name: root-ca-secret
        key: ca.crt

Changed

  • Split Keeper services to internal and external ones. Closes #1982.
  • Apply filter earlier when scraping metrics in order reduce memory usage by @dentiny in #2028. Closes 2019.
  • Backward-incompatible config rename in ClickHouseOperatorConfiguration: reconcile.recovery.from.{aborted,completed} → reconcile.recovery.onStatus.{aborted,completed} (the from grouping level is removed; per-status action keys are unchanged, and the obsolete from key is silently ignored on load). If you set reconcile.recovery.from.aborted.onPodReady: none on 0.27.0/0.27.1 to disable Aborted auto-recovery, re-apply it as reconcile.recovery.onStatus.aborted.onPodReady: none — otherwise the default (retry) silently re-enables it on upgrade.

Security

  • Operator metrics port no longer exposes /debug/pprof. The operator’s metrics HTTP listener (default :9999) served Go’s shared http.DefaultServeMux, onto which net/http/pprof is transitively registered through a controller-runtime dependency — unintentionally exposing the /debug/pprof/* endpoints (including the CPU profile and trace handlers) to anything able to reach the operator’s metrics Service, enabling unauthenticated profiling/DoS and heap/goroutine dumps. The listener now uses a dedicated mux that serves only the metrics endpoint; /metrics is unchanged.
  • Bumped versions of dependent libraries to address CVEs

Fixed

  • Add replica restart fix for operator when remote_servers configmap is propagated by @oo007 in #2024. Closes #2013.
  • Escape reserved characters in element text by @AruneshDwivedi in #2034.
  • Helm: Wire watchNamespaces value into operator ConfigMap by @jtomaszon in #2007. Closes #1919.
  • Fixed incorrect watched namespace override #2008.

FIPS Harness

  • Align FIPS test plan, requirements, and e2e FIPS coverage, fixed failing operator tests by @Elmo33 in #2011.
  • Wire security.kubernetes.tls.minVersion into the Kubernetes API client transport by @Elmo33 in #2020.
  • Expand FIPS TLS e2e coverage with host-run cipher probes and broader listener rejection checks by @Elmo33 in #2031.

New Contributors

Full Changelog: https://github.com/Altinity/clickhouse-operator/compare/release-0.27.1...release-0.27.2

Release 0.27.1

Released 2026-06-04

Full Changelog: release-0.27.0...release-0.27.1

NOTE: This is a major update to 0.27.0. Starting from 0.27.1, clickhouse-operator is FIPS-enabled

Added

Fixed

  • Fix permanent ArgoCD diff on resourceFieldRef.divisor by @pkieszcz in #1989
  • Fix inconsistent secret rendering in autogenerated remote_servers by @realyota in #1992. Closes #1991.
  • Fixed skipped Delete when scale-to-0 Update fails by @dashashutosh80 in #1993. Closes #1990.
  • Fixed .status.usedTemplates that were not properly cleaned when templates were removed
  • Bump dependent libraries to address CVEs

New Contributors

Release 0.27.0

Released 2026-05-08

Full Changelog: release-0.26.3...release-0.27.0

Added

  • Reference CHK from CHI by a name rather than a service:
spec:
  configuration:
    zookeeper:
      keeper:
        name: my-keeper
  • EXPERIMENTAL Pre/post hooks. Allows to inject different SQL commands on reconcile events.
    • supported events: Any, HostCreate, HostUpdate, HostStart, HostStop, HostConfigRestart, HostRollout, HostShutdown, HostDelete
    • target controls where to run SQL: FirstHost (default), AllHosts, AllShards
    • failurePolicy specifies what to do if hook execution fails with an error: Fail (default) or Ignore. See CRD for detailed description. Example:
  reconcile:
    host:
      hooks:
        pre:
          - sql:
              queries:
                - "SYSTEM STOP REPLICATION QUEUES"
            events:
              - HostShutdown
  • Now operator automatically restarts aborted reconcile if failed pod goes online after abort
  • Restart operator on operator configuration change #1960. Closes #1930.
  • Allow to exclude certain metrics from exporting. Noisy OS/CPU metrics are now excluded by default #1975. Closes #1876. Exclusion rules are applied by default and can be changed in operator configuration:
    excludeMetricsRegexp:
      - "^metric\\.(OS.*CPU[0-9]+|CPUFrequencyMHz_[0-9]+)$"
  • Helm: Allow readiness and liveness probes for operator containers by @janeklb in #1976
  • Helm: Add security context for crdHook to enable security policy compliance by @qlevasseur-genetec in #1950.

Changed

  • Enabled async_replication and use_xid_64 in Keeper default configuration. Requires Keeper 25.3 or above.
  • Whitelist Keeper four letter commands by default
  • Switched Keeper probes to ruok
  • Kubernetes client library has been upgraded to 0.30.14 by @dcoppa in #1970.

Fixed

  • Fix watch ClickHouseOperatorConfiguration in operator namespace. #1959.
  • Fix chk probe crashloop by @hananbs in #1961. Closes #1962.
  • Fix secret-backed env upgrade rollout #1966. Closes #1963.
  • Helm: fix configmap too long by @madrisan in #1949. Closes #1911.
  • build(deps): bump go.opentelemetry.io/otel/sdk from 1.42.0 to 1.43.0 by @dependabot[bot] in #1953
  • Fix: format string issues by @destinyoooo in #1955.
  • Fix: Prevent CrashLoopBackOff during image upgrade with RollingUpdate #1926 by @dashashutosh80 in #1956. Closes #1926.
  • Fixed a bug when host has not been removed from status fields hostsWithReplicaCaughtUp and hostsWithTablesCreated when removing from a cluster.
  • Prevent nil pointer panic in poll when CR Get fails by @wucm667 in #1974. Closes #1972.
  • Scope schema discovery to target host’s cluster in multi-cluster CHI by @lukas-pfannschmidt-tr in #1965. Closes #1964.
  • Fix: use plain errors for clickhouse model messages in #1980 by @immanuwell.

New Contributors

Release 0.26.3

Released 2026-04-17

Full changelog: release-0.26.2...release-0.26.3

Fixed

  • Prevent CrashLoopBackOff during image upgrade with RollingUpdate #1956 by @dashashutosh80. Closes #1926
  • Address CVEs in dependent libraries

Release 0.26.2

Released 2026-03-24

Full changelog: release-0.26.1...release-0.26.2

Fixed

  • Fixed a race condition when updating ClickHouse configuration and version altogether with a configuration setting that did not exist in the old version. Closes #1926
  • Fixed a bug when some Keeper nodes could be left offline after configuration changes
  • Fixed a bug when operator did not respect watched namespaces for Keeper. Closes #1923
  • Fixed potential races in configuration hash calculation. May close #1907
  • Updated dependent libraries to address CVE-2026-24051

Changed

  • Deleting multi-node CHI and CHK is now much faster
  • Added asynchronous_metrics_keeper_metrics_only to default Keeper configuration

Release 0.26.1

Released 2026-03-13

Full changelog: release-0.26.0...release-0.26.1

Fixed

  • Fixed Keeper startup that was slow due to missing quorum. Closes #1931 and #1856
  • Fixed Keeper deletion logic that could previously leave PVCs undeleted
  • Fixed hostName generation in status that might result in excessive schema propagation cycles
  • Fixed FQDN normalization to prevent trailing-dot inconsistencies between internal hostname representations
  • Bump stdlib version to address CVE
  • Bump base image version by @Slach in #1941. Closes #1940
  • Document custom service template behaviour for CHK by @realyota in #1939

Release 0.26.0

Released 2026-02-24

Full changelog: release-0.25.6...release-0.26.0

IMPORTANT: Due to ClickHouse upstream regression ClickHouse/ClickHouse#89693 DDL queries may not work on newly created ClickHouse pods. It affects Kubernetes deployments only in some new ClickHouse versions (25.8.10+ and above). The workaround is to restart ClickHouse pods. The problem is fixed by ClickHouse/ClickHouse#92339, see backports for different release branches. The fix is backported to Altinity Stable 25.8.16.10001 as well.
Closes #1883 and #1913

Added

  • Added an option to abort reconcile if STS needs to be recreated. It can be configured in operator configuration or CHI.
# Reconcile StatefulSet scenario  
reconcile:  
statefulSet:  
recreate:  
# What to do in case operator is in need to recreate StatefulSet?  
# Possible options:  
# 1. abort - abort the process, do nothing with the problematic StatefulSet, leave it as it is,  
#    do not try to fix or delete or update it, just abort reconcile cycle.  
#    Do not proceed to the next StatefulSet(s) and wait for an admin to assist.  
# 2. recreate - proceed and recreate StatefulSet.  
# Triggered when StatefulSet update fails or StatefulSet is not ready  
onUpdateFailure: recreate
  • Added an option to configure system tables for metrics scrapping. The default is system.metrics and system.custom_metrics tables, but those can be changed with a regular expression if needed:
  tablesRegexp: "^(metrics|custom_metrics)$"

Changed

  • The suspend flag now immediately aborts a running reconcile. Previously, it did not affect the one that was running
  • When suspend flag is set, any reconcile attempt automatically sets CHI/CHK status to aborted.
  • Add optional registry prefix for operator and metrics images in Helm chart by @lesandie in #1928
  • Improve ClickHouse Keeper Grafana Dashboard by @discostur in #1872
  • Add CRDHook annotations by @eyyu in #1914
  • Hotfix crdhook, add imagePullSecrets by @Slach in #1917
  • Fix installer to default template URL to OPERATOR_VERSION by @realyota in #1910
  • sort keys in Settings.Keys() method for consistent order (fix manifest reconcile issue) by @mastercactapus in #1900
  • Multiple documentation fixes

Fixed

  • Fixed Keeper rolling update logic. Closes #1796 #1915
  • Fixed a bug when replica was not added to monitoring until it catches up the replication lag
  • Fixed version parsing for FIPS compatible builds of ClickHouse. Closes #1850
  • Fixed stop and suspend attributes for CHK that were previously ignored
  • Fix distributed_ddl.replicas_path mismatch that could prevent sharing (Zoo)Keeper between multiple clusters @Elmo33 in #1922
  • Fixed a bug when defaults.storageManagement.reclaimPolicy was not respected
  • Fixed slow initial connectivity to newly created pods caused by DNS search list exhaustion (ndots:5). Added trailing dot to FQDN and increased connect timeout
  • Fixed a bug where reconcile settings specified at CHI level (e.g. spec.reconcile.statefulSet.recreate.onUpdateFailure) were not inherited by cluster-level reconcile configuration

Other

  • stdlib has been upgraded to 1.25.6 to address CVEs
  • Operator has been certified for 25.8.16.10001 Altinity.Stable.

New Contributors

Release 0.25.6

Released 2025-12-11

Full changelog: release-0.25.5...release-0.25.6

Changed

  • Hosts are not excluded from remote_servers anymore if restart is not needed. Previously, replicas in replicated cluster might be removed for a short time even if restart was not needed.
  • configuration.zookeeper section changes do not require restart anymore
  • Last reconciliation error and list of errors are now stored in CHI status

Fixed

  • actionPlan is now optional in status. That fixes operator upgrade problems that might happen in some environments.
  • Fixed excessive reconciles triggered by endpoint slices. Closes #1873
  • Fixed crash in CHK that might happen sometimes. Closes #1863
  • Fixed an issue with handling fractional requests/limits that would result in excessive reconcile. Closes #1849 #1821
  • Fixed a bug with operator crash in Terminating namespace. Closes #1871
  • stdlib was upgraded in order to address CVEs in dependent libraries

NOTE: Due to regression in upstream ClickHouse ClickHouse/ClickHouse#89693 schema propagation and DDL statements do not work with ClickHouse versions 25.8.10+ and newer until it is resolved.

Release 0.25.5

Released 2025-10-24

Full changelog: release-0.25.4...release-0.25.5

Changed

  • Hosts are not excluded from remote_servers anymore if restart is not needed. Previously, replicas in replicated cluster might be removed for a short time even if restart was not needed.
  • configuration.zookeeper section changes do not require restart anymore
  • Last reconciliation error and list of errors are now stored in CHI status

Fixed

  • actionPlan is now optional in status. That fixes operator upgrade problems that might happen in some environments.
  • Fixed excessive reconciles triggered by endpoint slices. Closes #1873
  • Fixed crash in CHK that might happen sometimes. Closes #1863
  • Fixed an issue with handling fractional requests/limits that would result in excessive reconcile. Closes #1849 #1821
  • Fixed a bug with operator crash in Terminating namespace. Closes #1871
  • stdlib was upgraded in order to address CVEs in dependent libraries

NOTE: Due to regression in upstream ClickHouse ClickHouse/ClickHouse#89693 schema propagation and DDL statements do not work with ClickHouse versions 25.8.10+ and newer until it is resolved.

Added

  • The latest applied ActionPlan is now stored in chi-storage ConfigMap
  • volumeClaimTemplate.volumeAttributeClass attribute. Closes #1818
  • Configuration for DROP REPLICA behavior:
reconcile:
  host:
    drop:
      replicas:
        # Whether the operator during reconcile procedure should drop replicas when replica is deleted
        onDelete: yes
        # Whether the operator during reconcile procedure should drop replicas when replica volume is lost
        onLostVolume: yes
        # Whether the operator during reconcile procedure should drop active replicas when replica is deleted or recreated
        active: no

Now active replicas are never dropped. That solves a potential bug when a replica could be dropped on a multi-volume node if a newly added volume is not yet available.

Changed

  • Enabled ReadinessProbe for Keeper. Closes #1846
  • Enabled query_log for all DDL statements performed by operator
  • minor logging improvements by @janeklb in #1829

Fixed

  • Removed excessive logging isUpdatedEndpointSlice():unknown
  • Changed metrics collection query that could be broken in 0.25.4 for old ClickHouse versions

Helm updates

New Contributors

Release 0.25.4

Released 2025-09-26

Full changelog: release-0.25.3...release-0.25.4

Added

  • Operator configuration ‘reconcile’ section is now fully supported at CHI level under both ‘reconcile’ and old ‘reconciling’ name. Previously, only selected settings were available at CHI level.
  • Allow to exclude namespaces that operator watches. by @AdheipSingh in #1770
spec:  
  watch:  
    namespaces:  
      include: [ ]  
      exclude: [ ] # new
  • Option to choose which probe should operator wait for during reconcile. Previously, it always waited for pod to be ready. This can now be configured in ‘reconcile’ section of operator or CHI:
spec:  
  reconcile:  
    host:  
      wait:  
        probes:  
          startup: "yes"  
          readiness: "no"

Changed

  • system.custom_metrics table is currently scrapped for monitoring in addition to metrics and asynchronous_metrics. That allows to inject custom monitoring data from ClickHouse side.
  • Deprecated Endpoints API has been replaced with EndpointSlice. Closes #1801

Fixed

  • Fixed a bug with long environment variables used for secrets being truncated. Closes #1804
  • Fixed a bug that operator did not respect watched namespaces for CHK

Helm updates

  • Define values.schema.json by @Slach in #1815. Closes #1814
  • Added clickhouse-operator deployment strategy parameters to Helm chart by @Slach in #1789
  • Publish operator helm chart to helm.altinity.com in addition to artifacthub.io

Release 0.25.3

Released 2025-08-08

Full changelog: release-0.25.2...release-0.25.3

Added

  • Added support for pdbMaxUnavailable in CHK
  • Added .spec.configuration.clusters[].pdbManaged for CHI and CHK that allow to set external PDBs by @zrudzionis in #1768

Fixed

  • Add ZK error handling and logging by @wilkermichael in #1762
  • Fixed collision between PDBs for CHI and CHK with the same name. As a side effect PDB for CHI will be re-created with this upgrade
  • Fixed rare panic in buildCRFromObj() when it can not find CR
  • Fixed update of storage configmap that might block in rare cases. Closes #1781
  • Fixed the check for host to be included in remote_servers, that did not work correctly in some network configurations. Closes #1782

Helm updates

  • feat(helm): add priority class to helm chart by @nobletooth in #1774
  • feat(helm): publish as an OCI helm package as well by @ogirardot in #1779
  • There is no helm repo upgrade, should be update by @CaptTofu in #1780
  • Hotfix port names, to avoid warning during helm install by @Slach in #1784

New Contributors

Release 0.25.2

Released 2025-07-07

Full changelog: release-0.25.1...release-0.25.2

Fixed

  • Refactored concurrent reconcile code to fix #1749
  • Changed ClickHouse version detection code to reduce number of calls
  • Force-skipped replica wait code for the first host in the cluster
  • Fixed clickhouse-keeper manifests for old Kubernetes versions by @Slach in #1767

Release 0.25.1

Released 2025-07-01

Full changelog: release-0.25.0...release-0.25.1

Added

  • Added an option to substitute macros in settings, can be enabled in spec.reconciling CHI section.

Fixed

  • Fix panic by @alex-zaitsev in #1750. Closes #1749
  • Fixed rootCA parsing. Closes #1666
  • Added ownerReference in chi-storage config map. Closes #1714
  • Fixed a bug when CHI was not included in the monitoring on operator restart until the end of reconcile
  • Fixed a bug with podTemplate env variables being incorrectly merged from CHI template

Helm/misc changes

New Contributors

Release 0.25.0

Released 2025-05-26

Full changelog: release-0.24.5...release-0.25.0

Added

  • Multiple service templates are now supported for a single purpose. Example of syntax:
defaults:
  templates:
    serviceTemplates:
     - service-template
     - service-template-internal
  • Added availabilityZone attribute for ZooKeeper spec:
configuration:
  zookeeper: # Add Zookeeper
    nodes:
      - host: zookeeper
        port: 2181
        availabilityZone: "my-azone"
  • Operator can now inject version-specific ClickHouse configuration, it is configured in the addons section of the operator configuration. This closes #1603 but can be used for other use cases as well
  • Added an option to specify reconciling.runtime.reconcileShardsThreadsNumber and reconcilingShardsMaxConcurrencyPercent on the CHI level
  • [Helm] ServiceMonitor additional configuration by @nlamirault in #1624
  • [Helm] Add namespace override capability for multi-namespace deployments by @bwdmr in #1640
  • [Helm] Add common labels and annotations to all resources by @dashashutosh80 in #1692
  • [Helm] Add support for rbac at namespace scope in @dashashutosh80 #1698

Changed

  • When a new cluster is started with reconcilingShardsMaxConcurrencyPercent=100, all shards are created in parallel. Previously, the first shard was always created alone.
  • The new option to wait for replication to catch up when adding a new replica is enabled by default. It can be configured in operator reconcile settings:
reconcile:
  host:
    wait:
      replicas:
        all: no
        new: yes
        delay: 10

Fixed

  • Fixed a bug with underlying replicated tables were not properly created when adding new nodes sometimes. Closes #1689
  • Fixed a bug when service was not updated if clusterIP is changed from missing to None in service template
  • Fixed a bug when annotations were not removed from service if removed from service template

New Contributors

Release 0.24.5

Released 2025-03-14

Full changelog: release-0.24.4...release-0.24.5

What’s Changed

  • Follow-up to the previous release. CLICKHOUSE_SKIP_USER_SETUP is not added if default entrypoint is not used.
  • Fixed a bug when ClickHouseOperatorConfiguration was not correctly merged sometimes
  • Adjusted default configuration so configuration files with ’no_restart’ in the file name would not cause ClickHouse restarts
  • Changed how operator handles remote_servers.xml when doing rolling updates. Now instead of removing/adding a replica, it sets low priority to make sure replica does not get any traffic from distributed queries
  • Bumped dependencies to address CVE-2025-22868 and CVE-2025-22870
  • Fixed a typo in Keeper Prometheus alert by @morkalfon in #1657
  • Fixed Keeper dashboard - Ephemeral Node count by @Slach in #1656

New Contributors

Release 0.24.4

Released 2025-02-06

Full changelog: release-0.24.3...release-0.24.4

Fixed

Release 0.24.3

Released 2025-01-27

Full changelog: release-0.24.2...release-0.24.3

Added

  • Ability to suspend ClickHouseInstallation and ClickHouseKeeper reconciliation by @a7i in #1580. Controlled by .spec.suspend attribute. Closes #1433.
  • New clickhouse_operator_chi metric in metrics-exporter
  • Add kafka-related alerts by @Slach in #1596

Changed

  • NormalizedCHI that is used to plan reconciliation was moved from CHI status to a separate configmap object. Closes #1444.
  • Changed a way ClickHouse is restarted in order to pickup server configuration change. Instead of pod re-creation it tries SYSTEM SHUTDOWN first. That speeds things up on nodes with big volumes, since those should not be re-attached.
  • Unused fields ‘HostsUnchanged’ and ‘HostsDelete’ were removed from extended status output
  • Upgrade prometheus-operator to 0.79.0 (alerts tested with prometheus 3.0.1) by @Slach in #1599
  • Interserver secret is now used for ‘all-sharded’ cluster if it is defined for the main cluster

Fixed

  • Custom ZK path creation is not retried anymore for stopped CHI
  • Fixed CVE-2024-45338 in golang.org/x/net 0.23.0

New Contributors

  • @a7i made their first contribution in #1580

Release 0.24.2

Released 2024-12-06

Full changelog: https://github.com/Altinity/clickhouse-operator/compare/release-0.24.1...release-0.24.2

Changed

  • Inherit kubectl annotations when managing statefulsets by @ondrej-smola in #1582. That allows to avoid unexpected restarts
  • Remove openSSL from /etc/clickhouse-keeper/config.d/01-keeper-01-default-config.xml by @Slach in #1574
  • Faster startup of big clusters

Fixed

Release 0.24.1

Released 2024-12-06

Full changelog: release-0.24.0...release-0.24.1

Added

  • Option to exclude certain labels/annotations from exposed prometheus metrics. Closes #1525

Changed

  • Removed SYSTEM DROP DNS cache that is not needed since ClickHouse 19.11
  • Removed redundant synchronization between threads when multiple shards were reconciled in parallel. That speeds up reconcile of big clusters
  • Action log in CHI/CHK status is disabled by default
  • When custom (Zoo)Keeper path is used, operator will re-try path creation until (Zoo)Keeper is available
  • Improved Keeper scale out/in logic
  • Respect PVC storage value if current value is greater than value in spec by @ondrej-smola in #1573
  • Update zookeeper deployment examples to 3.8.4 by @Slach in #1553

Fixed

Helm and docs

  • Doc : Add OpenEBS example by @lorellalou in #1527
  • Hotfix volumeClaimTemplate linking in chk-examples by @Slach in #1543
  • fix helm chart’s README by @janeklb in #1555
  • Hotfix chk crd, add {chk} and {chkID} to .spec.templates.serviceTemplates.generatedName description by @Slach in #1562
  • fix: add missing CRD link to helm chart readme by @janeklb in #1563
  • Docs: fix version to match manifests in chi-examples/08-* by @xavierleune in #1566
  • Add chk image pull secrets example by @Slach in #1576

New contributors

Release 0.24.0

Released 2024-10-14

Full changelog: https://github.com/Altinity/clickhouse-operator/compare/release-0.23.7...release-0.24.0

Backward Incompatible Changes

  • ClickHouseKeeper support has been fully re-written. It is now fully compatible with CHI in syntax and supported functionality that closes multiple issues discovered by early adopters. The downside is that CHK resources created by operator 0.23.x will loose the storage and change service name due to different naming conventions. See CHK upgrade instructions from 0.23 to 0.24 for the details. This closes #1504, #1497, #1469, #1465, #1459, #1408, #1362, #1296

Added

Changed

Fixed

New Contributors

Older releases

Details of all releases, including older releases, are available through the project’s Releases page.