FIPS-Compatible Altinity Stable® Builds

Altinity Stable® Builds with FIPS 140-3 support

What are Altinity FIPS-compatible builds?

FIPS 140-3 is a United States standard for cryptography used in high-security government environments. FIPS 140-3 specifies a number of properties for encryption including handling of keys, permitted versions of TLS, allowed cipher suites, and protections against tampering of builds.

ClickHouse uses AWS-LC (Amazon’s fork of BoringSSL) libraries for encryption of most application and inter-server traffic. Altinity FIPS-compatible builds use AWS-LC FIPS 2.0.0, which has been validated under FIPS 140-3 (CMVP Certificate #4816). Combined with documented procedures this enables ClickHouse to function in a manner that is compatible with the FIPS standard.

FIPS-compatible Altinity Stable Builds are built, tested, and released in the same way as regular Altinity Stable Builds for ClickHouse. FIPS-compatible builds have altinityfips embedded in the release name. They use separate channels for distribution on builds.altinity.cloud/#altinityfips and have separate release notes. FIPS-compatible builds are also available as Docker images.

Other than support for FIPS 140-3, FIPS-compatible builds are essentially identical to Altinity Stable Builds. Any differences are highlighted in the release notes.

Release notes

For more technical details, see the release notes for the FIPS releases:

FIPS-Compatible Network Interfaces

The following network connections of ClickHouse can operate in FIPS-compatible mode in Altinity FIPS builds.

Name Type Description Default Server Port
HTTPS Port Server Accepts HTTPS API connections from clients 8443
Secure Native TCP Port Server Accepts native TCP protocol connections from clients (e.g., clickhouse-client) 9440
Interserver HTTPS Port Server & Client Used for communication between ClickHouse replicas 9010
ZooKeeper Client Connection Client Connection from ClickHouse to ZooKeeper or ClickHouse Keeper
Keeper Server Port Server Accepts ZooKeeper protocol connections from clients 9281
Raft Server Port Server & Client Used for synchronization between ClickHouse Keeper servers 9444

Prerequisites for FIPS-Compatible Operation

The minimal requirements for FIPS-compatible operation are:

  • Install FIPS-compatible Altinity Stable Build.
  • Apply FIPS-compatible configuration settings to set allowed ports, TLS version, and ciphers.

Installation

Yum and Apt Packages

FIPS-compatible Altinity Stable releases are distributed from a separate repo from standard Altinity Stable Builds. Follow the directions to set the repo for FIPS-compatible builds at builds.altinity.cloud/#altinityfips.

Important note! FIPS builds use a different repo from standard Altinity Stable Builds. Be sure you’re in the FIPS-compatible build section.

Once the repo is set correctly, you can download and install packages using the same commands as for regular Altinity Stable Builds.

Docker

FIPS-compatible Altinity Stable containers have fips in the container tag. For example:

docker pull altinity/clickhouse-server:24.3.5.8.altinityfips

Configuration of Altinity Stable Builds for FIPS-Compatible Operation

To operate in a FIPS-compatible mode, users must update the ClickHouse configuration from the usual defaults. The specific changes include the following.

ClickHouse Server Configuration

Required server configuration changes include the following. These settings are by convention stored in /etc/clickhouse-server/config.xml and /etc/clickhouse-server/config.d/.

Disable all ports not in the FIPS-Compatible Network Protocol list, including the following. Comment them out and ensure they are not present in preprocessed_config.xml.

  • ClickHouse HTTP port
  • ClickHouse TCP port
  • Additional non-FIPS client protocols: MySQL, PostgreSQL, gRPC, etc.
  • Enable HTTPS and TCP secure protocols
  • Configure openSSL sections to restrict TLS to TLSv1.2/TLSv1.3 and FIPS 140-3-approved ciphers

Place configuration changes in file /etc/clickhouse-server/config.d/fips.xml. This will prevent your settings from being overridden when installing new Altinity Stable Builds. Here is an example of file contents.

<clickhouse>
  <https_port>8443</https_port>
  <tcp_port_secure>9440</tcp_port_secure>
  <openSSL>
    <server>
      <certificateFile>${CERT_PATH}/server.crt</certificateFile>
      <privateKeyFile>${CERT_PATH}/server.key</privateKeyFile>
      <dhParamsFile>${CERT_PATH}/dh_params.pem</dhParamsFile>
      <cipherList>ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-GCM-SHA384</cipherList>
      <cipherSuites>TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384</cipherSuites>
      <loadDefaultCAFile>true</loadDefaultCAFile>
      <cacheSessions>true</cacheSessions>
      <preferServerCiphers>true</preferServerCiphers>
      <disableProtocols>sslv2,sslv3,tlsv1,tlsv1_1</disableProtocols>
      <!-- set to `none` for dev-tests with self-signed certificates -->
      <verificationMode>relaxed</verificationMode>
    </server>
    <client>
   	 <!-- in case of private CA, otherwise set `loadDefaultCAFile` to `true` and remove `caConfig` -->
   	 <loadDefaultCAFile>false</loadDefaultCAFile>
   	 <caConfig>${CA_PATH}/CA.crt</caConfig>
   	 <cacheSessions>true</cacheSessions>
   	 <preferServerCiphers>true</preferServerCiphers>
   	 <disableProtocols>sslv2,sslv3,tlsv1,tlsv1_1</disableProtocols>
   	 <!-- set to 'none' for dev-tests with self-signed certificates -->
   	 <verificationMode>relaxed</verificationMode>
   	 <invalidCertificateHandler>
   		 <!-- AcceptCertificateHandler may not be used here -->
   		 <name>RejectCertificateHandler</name>
   	 </invalidCertificateHandler>
    </client>
  </openSSL>
</clickhouse>

Please note the following:

  • There is no need to set an openSSL/fips value to true in the XML above as it is ON by default in the FIPS build and can’t be turned off.
  • ${CERT_PATH} and ${CA_PATH} are placeholders. Replace them with appropriate values for your configuration.
  • disableProtocols must disable versions from the bottom up (e.g. sslv2,sslv3,tlsv1,tlsv1_1) to maintain a contiguous version range. Selectively disabling a middle version (e.g. only tlsv1_1) will cause a NO_SUPPORTED_VERSIONS_ENABLED error due to AWS-LC/BoringSSL’s contiguous version range requirement.
  • TLSv1.3 is now enabled (no longer in disableProtocols). The <cipherSuites> setting controls which TLSv1.3 cipher suites are permitted; <cipherList> continues to control TLSv1.2 ciphers.
  • <requireTLSv1_2> has been removed since both TLSv1.2 and TLSv1.3 are now permitted.

ClickHouse Client Configuration

clickhouse-client configuration settings are by convention stored in /etc/clickhouse-client/config.xml and /etc/clickhouse-client/config.d/.

To enable FIPS 140-3 operation for clickhouse-client, configure the openSSL section to restrict TLS to TLSv1.2/TLSv1.3 and FIPS 140-3-approved ciphers. Place values for the changes in /etc/clickhouse-client/config.d/fips.xml.

Running AWS-LC SSL and ACVP Validation Tests

The cryptographic module in Altinity FIPS builds can be verified by running the AWS-LC SSL conformance tests (8,037 tests) and ACVP known-answer tests (31 algorithm suites).

Prerequisites

  • An installed Altinity FIPS build of ClickHouse (.deb package or Docker image)
  • Go >= 1.13 installed on the test machine
  • The AWS-LC FIPS 2.0.0 source (used as the test harness):
    curl -L -o /tmp/aws-lc.zip https://github.com/aws/aws-lc/archive/refs/tags/AWS-LC-FIPS-2.0.0.zip
    unzip /tmp/aws-lc.zip -d /tmp
    export AWSLC_SRC=/tmp/aws-lc-AWS-LC-FIPS-2.0.0
    

The ClickHouse multi-tool binary exposes the test harness via symlinks:

ln -sf /usr/bin/clickhouse /usr/bin/clickhouse-ssl-shim
ln -sf /usr/bin/clickhouse /usr/bin/clickhouse-ssl-handshaker
ln -sf /usr/bin/clickhouse /usr/bin/clickhouse-acvp-server

SSL Tests (8,037 tests)

These exercise the full TLS stack: handshake flows, cipher negotiation, session resumption, certificate handling, and more.

cd "$AWSLC_SRC/ssl/test/runner"
go test -v . \
  -shim-path /usr/bin/clickhouse-ssl-shim \
  -handshaker-path /usr/bin/clickhouse-ssl-handshaker \
  -num-workers 4

A passing run ends with:

PASS
ok  	boringssl.googlesource.com/boringssl/ssl/test/runner	142.538s

ACVP Tests (31 algorithm suites)

These validate that the FIPS cryptographic module produces correct outputs for known-answer test vectors covering AES, ECDSA, RSA, HMAC, KDF, DRBG, and more.

# Build the test tools
cd "$AWSLC_SRC/util/fipstools/acvp/acvptool"
go build -o /tmp/acvptool .
cd "$AWSLC_SRC/util/fipstools/acvp/acvptool/testmodulewrapper"
go build -o /tmp/testmodulewrapper .

# Run the tests
cd "$AWSLC_SRC/util/fipstools/acvp/acvptool/test"
go run check_expected.go \
  -tool /tmp/acvptool \
  -module-wrappers "modulewrapper:/usr/bin/clickhouse-acvp-server,testmodulewrapper:/tmp/testmodulewrapper" \
  -tests tests.json

A passing run ends with:

31 ACVP tests matched expectations

Verification of FIPS-Compatible Altinity Stable Operation

Verify FIPS library Startup

FIPS-compatible Altinity.Cloud servers will print the following message after a successful start-up test. This ensures that FIPS AWS-LC libraries are present and free from tampering.

$ grep 'FIPS mode' /var/log/clickhouse-server/clickhouse-server.log
2023.05.28 18:19:03.064038 [ 1 ] {} <Information> Application: Starting in FIPS mode, KAT test result: 1

Verify FIPS-Compatible Altinity Stable Version

To verify the software version, run select version() on the running server with any client program. This example confirms the version for both clickhouse-client as well as clickhouse-server.

$ clickhouse-client <options>
ClickHouse client version 23.3.19.34.altinityfips (altinity build).

5f1b329b5fdf :) select version()

SELECT version()

┌─version()───────────────┐
│ 23.3.19.34.altinityfips │
└─────────────────────────┘