1. Custom Resource元素
apiVersion: "clickhouse.altinity.com/v1"
kind: "ClickHouseInstallation"
metadata:name: "clickhouse-installation-test"这是clickhouse operator自定义的资源ClickHouseInstallation
1.1. .spec.defaults
spec:defaults: - section represents default values for sections below.replicasUseFQDN: "no" - should replicas be specified by FQDN in <host></host>distributedDDL: - reference to <yandex><distributed_ddl></distributed_ddl></yandex>profile: defaultstorageManagement:# Specify PVC provisioner.# 1. StatefulSet. PVC would be provisioned by the StatefulSet# 2. Operator. PVC would be provisioned by the operatorprovisioner: StatefulSet# Specify PVC reclaim policy.# 1. Retain. Keep PVC from being deleted# Retaining PVC will also keep backing PV from deletion. This is useful in case we need to keep data intact.# 2. DeletereclaimPolicy: Retain1.2. .spec.configuration
表示ClickHouse配置文件的来源。无论是用户，远程服务器等配置文件。
1.2.1. .spec.configuration.zookeeper
表示标签 <yandex><zookeeper></zookeeper></yandex> 配置
zookeeper:nodes:- host: zookeeper-0.zookeepers.zoo3ns.svc.cluster.localport: 2181- host: zookeeper-1.zookeepers.zoo3ns.svc.cluster.localport: 2181- host: zookeeper-2.zookeepers.zoo3ns.svc.cluster.localport: 2181session_timeout_ms: 30000operation_timeout_ms: 10000root: /path/to/zookeeper/nodeidentity: user:password1.2.2. .spec.configuration.profiles
profiles:readonly/readonly: 1 表示<yandex><profiles></profiles></yandex> settings1.2.3. .spec.configuration.users
users:readonly/profile: readonly# <users># <readonly># <profile>readonly</profile># </readonly># </users>test/networks/ip:- "127.0.0.1"- "::/0"# <users># <test># <networks># <ip>127.0.0.1</ip># <ip>::/0</ip># </networks># </test># </users>test/profile: defaulttest/quotas: default1.2.4. .spec.configuration.settings
profiles:readonly/readonly: "1"# <profiles># <readonly># <readonly>1</readonly># </readonly># </profiles>default/max_memory_usage: "1000000000"1.2.5. .spec.configuration.files
file允许通过YAML自定义ClickHouse文件。这可以用于创建复杂的自定义配置。
spec:configuration:settings:dictionaries_config: config.d/*.dictfiles:dict_one.dict: |<yandex><dictionary><name>one</name><source><clickhouse><host>localhost</host><port>9000</port><user>default</user><password/><db>system</db><table>one</table></clickhouse></source><lifetime>60</lifetime><layout><flat/></layout><structure><id><name>dummy</name></id><attribute><name>one</name><expression>dummy</expression><type>UInt8</type><null_value>0</null_value></attribute></structure></dictionary></yandex>1.2.6. .spec.configuration.clusters
集群配置
clusters:- name: all-counts # 基础配置 使用layouttemplates:podTemplate: clickhouse-v23.8dataVolumeClaimTemplate: default-volume-claimlogVolumeClaimTemplate: default-volume-claimschemaPolicy:replica: Allshard: Alllayout:shardsCount: 3 分片数replicasCount: 2 副本数- name: shards-onlytemplates:podTemplate: clickhouse-v23.8dataVolumeClaimTemplate: default-volume-claimlogVolumeClaimTemplate: default-volume-claimlayout:shardsCount: 3# replicasCount not specified, assumed = 1, by default- name: replicas-onlytemplates:podTemplate: clickhouse-v23.8dataVolumeClaimTemplate: default-volume-claimlogVolumeClaimTemplate: default-volume-claimlayout:# shardsCount not specified, assumed = 1, by defaultreplicasCount: 3- name: customized #自定义分片和副本 使用shards and replicas标签.templates:podTemplate: clickhouse-v23.8dataVolumeClaimTemplate: default-volume-claimlogVolumeClaimTemplate: default-volume-claimschemaPolicy:replica: Noneshard: Nonelayout:shards:- name: shard0replicasCount: 3weight: 1internalReplication: Disabledtemplates:podTemplate: clickhouse-v23.8dataVolumeClaimTemplate: default-volume-claimlogVolumeClaimTemplate: default-volume-claim- name: shard1templates:podTemplate: clickhouse-v23.8dataVolumeClaimTemplate: default-volume-claimlogVolumeClaimTemplate: default-volume-claimreplicas:- name: replica0- name: replica1- name: replica2- name: shard2replicasCount: 3templates:podTemplate: clickhouse-v23.8dataVolumeClaimTemplate: default-volume-claimlogVolumeClaimTemplate: default-volume-claimreplicaServiceTemplate: replica-service-templatereplicas:- name: replica0tcpPort: 9000httpPort: 8123interserverHTTPPort: 9009templates:podTemplate: clickhouse-v23.8dataVolumeClaimTemplate: default-volume-claimlogVolumeClaimTemplate: default-volume-claimreplicaServiceTemplate: replica-service-template- name: with-secret# Insecure communication.# Opens/Closes insecure portsinsecure: "yes"# Secure communication.# Opens/Closes secure ports# Translates into <secure>1</secure> ClickHouse setting for remote replicassecure: "yes"# Shared secret value to secure cluster communicationssecret:# Auto-generate shared secret value to secure cluster communicationsauto: "True"# Cluster shared secret value in plain textvalue: "plaintext secret"# Cluster shared secret sourcevalueFrom:secretKeyRef:name: "SecretName"key: "Key"layout:shardsCount: 21.3. .spec.templates
1.3.1. .serviceTemplates
templates:serviceTemplates:- name: chi-service-template# generateName understands different sets of macroses,# depending on the level of the object, for which Service is being created:## For CHI-level Service:# 1. {chi} - ClickHouseInstallation name# 2. {chiID} - short hashed ClickHouseInstallation name (BEWARE, this is an experimental feature)## For Cluster-level Service:# 1. {chi} - ClickHouseInstallation name# 2. {chiID} - short hashed ClickHouseInstallation name (BEWARE, this is an experimental feature)# 3. {cluster} - cluster name# 4. {clusterID} - short hashed cluster name (BEWARE, this is an experimental feature)# 5. {clusterIndex} - 0-based index of the cluster in the CHI (BEWARE, this is an experimental feature)## For Shard-level Service:# 1. {chi} - ClickHouseInstallation name# 2. {chiID} - short hashed ClickHouseInstallation name (BEWARE, this is an experimental feature)# 3. {cluster} - cluster name# 4. {clusterID} - short hashed cluster name (BEWARE, this is an experimental feature)# 5. {clusterIndex} - 0-based index of the cluster in the CHI (BEWARE, this is an experimental feature)# 6. {shard} - shard name# 7. {shardID} - short hashed shard name (BEWARE, this is an experimental feature)# 8. {shardIndex} - 0-based index of the shard in the cluster (BEWARE, this is an experimental feature)## For Replica-level Service:# 1. {chi} - ClickHouseInstallation name# 2. {chiID} - short hashed ClickHouseInstallation name (BEWARE, this is an experimental feature)# 3. {cluster} - cluster name# 4. {clusterID} - short hashed cluster name (BEWARE, this is an experimental feature)# 5. {clusterIndex} - 0-based index of the cluster in the CHI (BEWARE, this is an experimental feature)# 6. {shard} - shard name# 7. {shardID} - short hashed shard name (BEWARE, this is an experimental feature)# 8. {shardIndex} - 0-based index of the shard in the cluster (BEWARE, this is an experimental feature)# 9. {replica} - replica name# 10. {replicaID} - short hashed replica name (BEWARE, this is an experimental feature)# 11. {replicaIndex} - 0-based index of the replica in the shard (BEWARE, this is an experimental feature)generateName: "service-{chi}"# type ObjectMeta struct from k8s.io/meta/v1metadata:labels:custom.label: "custom.value"annotations:cloud.google.com/load-balancer-type: "Internal"service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0service.beta.kubernetes.io/azure-load-balancer-internal: "true"service.beta.kubernetes.io/openstack-internal-load-balancer: "true"service.beta.kubernetes.io/cce-load-balancer-internal-vpc: "true"# type ServiceSpec struct from k8s.io/core/v1spec:ports:- name: httpport: 8123- name: clientport: 9000type: LoadBalancer1.3.2. podTemplates
zone和distribution一起定义节点上ClickHouse实例的分区布局。
podTemplates:- name: clickhouse-v23.8# We may need to label nodes with clickhouse=allow label for this example to run# See ./label_nodes.sh for this purposezone:key: "clickhouse"values:- "allow"# Shortcut version for AWS installations#zone:# values:# - "us-east-1a"# Possible values for podDistribution are:# Unspecified - empty value# ClickHouseAntiAffinity - AntiAffinity by ClickHouse instances.# Pod pushes away other ClickHouse pods, which allows one ClickHouse instance per topologyKey-specified unit# CH - (push away) - CH - (push away) - CH# ShardAntiAffinity - AntiAffinity by shard name.# Pod pushes away other pods of the same shard (replicas of this shard),# which allows one replica of a shard instance per topologyKey-specified unit.# Other shards are allowed - it does not push all shards away, but CH-instances of the same shard only.# Used for data loss avoidance - keeps all copies of the shard on different topologyKey-specified units.# shard1,replica1 - (push away) - shard1,replica2 - (push away) - shard1,replica3# ReplicaAntiAffinity - AntiAffinity by replica name.# Pod pushes away other pods of the same replica (shards of this replica),# which allows one shard of a replica per topologyKey-specified unit.# Other replicas are allowed - it does not push all replicas away, but CH-instances of the same replica only.# Used to evenly distribute load from "full cluster scan" queries.# shard1,replica1 - (push away) - shard2,replica1 - (push away) - shard3,replica1# AnotherNamespaceAntiAffinity - AntiAffinity by "another" namespace.# Pod pushes away pods from another namespace, which allows same-namespace pods per topologyKey-specified unit.# ns1 - (push away) - ns2 - (push away) - ns3# AnotherClickHouseInstallationAntiAffinity - AntiAffinity by "another" ClickHouseInstallation name.# Pod pushes away pods from another ClickHouseInstallation,# which allows same-ClickHouseInstallation pods per topologyKey-specified unit.# CHI1 - (push away) - CHI2 - (push away) - CHI3# AnotherClusterAntiAffinity - AntiAffinity by "another" cluster name.# Pod pushes away pods from another Cluster,# which allows same-cluster pods per topologyKey-specified unit.# cluster1 - (push away) - cluster2 - (push away) - cluster3# MaxNumberPerNode - AntiAffinity by cycle index.# Pod pushes away pods from the same cycle,# which allows to specify maximum number of ClickHouse instances per topologyKey-specified unit.# Used to setup circular replication.# NamespaceAffinity - Affinity by namespace.# Pod attracts pods from the same namespace,# which allows pods from same namespace per topologyKey-specified unit.# ns1 + (attracts) + ns1# ClickHouseInstallationAffinity - Affinity by ClickHouseInstallation name.# Pod attracts pods from the same ClickHouseInstallation,# which allows pods from the same CHI per topologyKey-specified unit.# CHI1 + (attracts) + CHI1# ClusterAffinity - Affinity by cluster name.# Pod attracts pods from the same cluster,# which allows pods from the same Cluster per topologyKey-specified unit.# cluster1 + (attracts) + cluster1# ShardAffinity - Affinity by shard name.# Pod attracts pods from the same shard,# which allows pods from the same Shard per topologyKey-specified unit.# shard1 + (attracts) + shard1# ReplicaAffinity - Affinity by replica name.# Pod attracts pods from the same replica,# which allows pods from the same Replica per topologyKey-specified unit.# replica1 + (attracts) + replica1# PreviousTailAffinity - Affinity to overlap cycles. Used to make cycle pod distribution# cycle head + (attracts to) + previous cycle tailpodDistribution:- type: ShardAntiAffinity- type: MaxNumberPerNodenumber: 2# Apply podDistribution on per-host basistopologyKey: "kubernetes.io/hostname"# Apply podDistribution on per-zone basis#topologyKey: "kubernetes.io/zone"# type ObjectMeta struct {} from k8s.io/meta/v1metadata:labels:a: "b"# type PodSpec struct {} from k8s.io/core/v1spec:containers:- name: clickhouseimage: clickhouse/clickhouse-server:23.8volumeMounts:- name: default-volume-claimmountPath: /var/lib/clickhouseresources:requests:memory: "64Mi"cpu: "100m"limits:memory: "64Mi"cpu: "100m"- name: clickhouse-logimage: clickhouse/clickhouse-server:23.8command:- "/bin/sh"- "-c"- "--"args:- "while true; do sleep 30; done;"# pod template for ClickHouse v23.8- name: clickhouse-v23.8# type ObjectMeta struct {} from k8s.io/meta/v1metadata:labels:a: "b"# type PodSpec struct {} from k8s.io/core/v1spec:containers:- name: clickhouseimage: clickhouse/clickhouse-server:23.8volumeMounts:- name: default-volume-claimmountPath: /var/lib/clickhouseresources:requests:memory: "64Mi"cpu: "100m"limits:memory: "64Mi"cpu: "100m"- name: clickhouse-logimage: clickhouse/clickhouse-server:23.8command:- "/bin/sh"- "-c"- "--"args:- "while true; do sleep 30; done;"2. 建表
2.1. Manifest
创建一个双副本双飞分片的clickhouse集群
apiVersion: "clickhouse.altinity.com/v1"
kind: "ClickHouseInstallation"metadata:name: "repl-05"spec:defaults:templates: dataVolumeClaimTemplate: defaultpodTemplate: clickhouse:19.6configuration:zookeeper:nodes:- host: zookeeper.zoo1nsclusters:- name: replicatedlayout:shardsCount: 2replicasCount: 2templates:volumeClaimTemplates:- name: defaultspec:accessModes:- ReadWriteOnceresources:requests:storage: 500MipodTemplates:- name: clickhouse:19.6spec:containers:- name: clickhouse-podimage: clickhouse/clickhouse-server:23.82.2. Replicated table setup
2.2.1. 宏
Operator provides set of macros, which are:
{installation}-- ClickHouse Installation name{cluster}-- primary cluster name{replica}-- replica name in the cluster, maps to pod service name{shard}-- shard id
ClickHouse also supports internal macros {database} and {table} that maps to current database and table respectively.
2.2.2. Create replicated table
Now we can create replicated table, using specified macros
CREATE TABLE events_local on cluster '{cluster}' (event_date Date,event_type Int32,article_id Int32,title String
) engine=ReplicatedMergeTree('/clickhouse/{installation}/{cluster}/tables/{shard}/{database}/{table}', '{replica}')
PARTITION BY toYYYYMM(event_date)
ORDER BY (event_type, article_id);
第一个参数是分片的 zk_path 一般按照：/clickhouse/table/{shard}/{table_name} 的格式
第二个参数是副本名称，相同的分片副本名称不能相同。CREATE TABLE events on cluster '{cluster}' AS events_local
ENGINE = Distributed('{cluster}', default, events_local, rand());
'{cluster}': 代表集群的名称，用于指定数据将如何在集群中分布。default: 指定数据库的名称。在这个例子中，使用 default 数据库，这意味着表会在 default 数据库中创建。events_local: 指定实际存储数据的本地表的名称。在查询 events 表时，数据将从集群中的 events_local 表中检索。events_local 表是在每个节点上本地存储数据的实际表。rand(): 指定分布的方式。在这里，rand() 函数表示使用随机方式将数据分配到不同的分片上。这有助于实现数据的均匀分布。We can generate some data:
INSERT INTO events SELECT today(), rand()%3, number, 'my title' FROM numbers(100);And check how these data are distributed over the cluster
SELECT count() FROM events;
SELECT count() FROM events_local;3. 用户
3.1. The 'default' user
“默认”用户用于从运行的pod连接到ClickHouse实例，也用于分布式查询。它使用空密码部署，这是ClickHouse开箱即用安装的长期默认密码。
为了确保它的安全，operator应用网络安全规则，限制连接到运行ClickHouse集群的pod，而不是其他。
下面的users.xml是由operator为具有两个节点的集群设置的。在此配置中，“默认”用户不能从集群外部连接。
3.2. 修改operate配置
clickhouse:access:secret:# Empty `namespace` means that k8s secret would be looked# in the same namespace where the operator's pod is running.namespace: ""name: "clickhouse-operator"The following example shows a secret:
apiVersion: v1
kind: Secret
metadata:name: clickhouse-operator
type: Opaque
stringData:username: clickhouse_operatorpassword: chpassword3.3. Using hashed passwords
ClickHouseInstallation中的用户密码可以明文指定为sha256和双sha1哈希值。
当以明文形式指定密码时，操作员在部署到ClickHouse时将其散列，但在ClickHouse安装中仍以不安全的明文格式保留。
Altinity建议显式提供哈希值如下:
spec:useTemplates:- name: clickhouse-versionconfiguration:users:user1/password: pwduser1 # This will be hashed in ClickHouse config files, but this NOT RECOMMENDEDuser2/password_sha256_hex: 716b36073a90c6fe1d445ac1af85f4777c5b7a155cea359961826a030513e448user3/password_double_sha1_hex: cbe205a7351dd15397bf423957559512bd4be3953.4. Using secrets
spec:configuration:users:user1/password:valueFrom:secretKeyRef:name: clickhouse_secretkey: pwduser1user2/password_sha256_hex:valueFrom:secretKeyRef:name: clickhouse_secretkey: pwduser2 user3/password_double_sha1_hex:valueFrom:secretKeyRef:name: clickhouse_secretkey: pwduser3 apiVersion: v1
kind: Secret
metadata:name: clickhouse-secret
type: Opaque
stringData:pwduser1: pwduser1pwduser2: e106728c3541ec3694822a29d4b8b5f1f473508adc148fcb58a60c42bcf3234cpwduser3: cbe205a7351dd15397bf423957559512bd4be3953.5. Securing the 'default' user
虽然“默认”用户受到网络规则的保护，但信息安全团队通常不允许无密码操作。可以像更改其他用户一样更改“默认”用户的密码。然而，ClickHouse也使用“默认”用户来运行分布式查询。如果密码更改，分布式查询可能会停止工作。
为了在不暴露密码的情况下保持分布式查询的运行，配置ClickHouse为集群间通信使用秘密令牌，而不是“默认”用户凭据。
3.5.1. 'auto' token
下面的示例展示了如何让ClickHouse自动生成秘密令牌。这是最简单和推荐的方法。
spec:configuration:users:default/password_sha256_hex: 716b36073a90c6fe1d445ac1af85f4777c5b7a155cea359961826a030513e448clusters:- name: defaultsecret:auto: "true"3.5.2. Custom token
The following example shows how to define a token.
spec:configuration:users:default/password_sha256_hex: 716b36073a90c6fe1d445ac1af85f4777c5b7a155cea359961826a030513e448clusters:- name: "default"secret:value: "my_secret"3.5.3. Custom token from Kubernetes secret
The following example shows how to define a token within a secret.
spec:configuration:users:default/password_sha256_hex: 716b36073a90c6fe1d445ac1af85f4777c5b7a155cea359961826a030513e448clusters:- name: "default"secret:valueFrom:secretKeyRef:name: "secure-inter-cluster-communications"key: "secret"4. zookeeper
This document describes how to setup ZooKeeper in k8s environment.
Zookeeper installation is available in two options:
- Quick start - just run it quickly and ask no questions
- Advanced setup - setup internal details, such as storage class, replicas number, etc
During ZooKeeper installation the following items are created/configured:
- [OPTIONAL] Create separate namespace to run Zookeeper in
- Create k8s resources (optionally, within namespace):
- Service - used to provide central access point to Zookeeper
- Headless Service - used to provide DNS namings
- Disruption Balance - used to specify max number of offline pods
- [OPTIONAL] Storage Class - used to specify storage class to be used by Zookeeper for data storage
- Stateful Set - used to manage and scale sets of pods
4.1. Quick start
Quick start is represented in two flavors:
- With persistent volume - good for AWS. File are located in deploy/zookeeper/quick-start-persistent-volume
- With local emptyDir storage - good for standalone local run, however has to true persistence.
Files are located in deploy/zookeeper/quick-start-volume-emptyDir
Each quick start flavor provides the following installation options:
- 1-node Zookeeper cluster (zookeeper-1- files). No failover provided.
- 3-node Zookeeper cluster (zookeeper-3- files). Failover provided.
In case you'd like to test with AWS or any other cloud provider, we recommend to go with deploy/zookeeper/quick-start-persistent-volume persistent storage. In case of local test, you'd may prefer to go with deploy/zookeeper/quick-start-volume-emptyDir emptyDir.
4.1.1. Script-based Installation
In this example we'll go with simple 1-node Zookeeper cluster on AWS and pick deploy/zookeeper/quick-start-persistent-volume. Both create and delete shell scripts are available for simplification.
4.1.2. Manual Installation
In case you'd like to deploy Zookeeper manually, the following steps should be performed:
4.1.3. Namespace
Create namespace
kubectl create namespace zoo1ns4.1.4. Zookeeper
Deploy Zookeeper into this namespace
kubectl apply -f zookeeper-1-node.yaml -n zoo1nsNow Zookeeper should be up and running. Let's explore Zookeeper cluster.
IMPORTANT quick-start zookeeper installation are for test purposes mainly.
For fine-tuned Zookeeper setup please refer to advanced setup options.
4.2. Advanced setup
Advanced files are are located in deploy/zookeeper/advanced folder. All resources are separated into different files so it is easy to modify them and setup required options.
Advanced setup is available in two options:
- With persistent volume
- With emptyDir volume
Each of these options have both create and delete scripts provided
- Persistent volume create and delete scripts
- EmptyDir volume create and delete scripts
Step-by-step explanations:
4.2.1. Namespace
Create namespace in which all the rest resources would be created
kubectl create namespace zoons4.2.2. Zookeeper Service
Create service. This service provides DNS name for client access to all Zookeeper nodes.
kubectl apply -f 01-service-client-access.yaml -n zoonsShould have as a result
service/zookeeper created4.2.3. Zookeeper Headless Service
Create headless service. This service provides DNS names for all Zookeeper nodes
kubectl apply -f 02-headless-service.yaml -n zoonsShould have as a result
service/zookeeper-nodes created4.2.4. Disruption Budget
Create budget. Disruption Budget instructs k8s on how many offline Zookeeper nodes can be at any time
kubectl apply -f 03-pod-disruption-budget.yaml -n zoonsShould have as a result
poddisruptionbudget.policy/zookeeper-pod-distribution-budget created4.2.5. Storage Class
This part is not that straightforward and may require communication with k8s instance administrator.
First of all, we need to decide, whether Zookeeper would use Persistent Volume as a storage or just stick to more simple Volume (In doc emptyDir type is used)
In case we'd prefer to stick with simpler solution and go with Volume of type emptyDir, we need to go with emptyDir StatefulSet config 05-stateful-set-volume-emptyDir.yaml as described in next Stateful Set unit. Just move to it.
In case we'd prefer to go with Persistent Volume storage, we need to go with Persistent Volume StatefulSet config 05-stateful-set-persistent-volume.yaml
Shortly, Storage Class is used to bind together Persistent Volumes, which are created either by k8s admin manually or automatically by Provisioner. In any case, Persistent Volumes are provided externally to an application to be deployed into k8s. So, this application has to know Storage Class Name to ask for from the k8s in application's claim for new persistent volume - Persistent Volume Claim. This Storage Class Name should be asked from k8s admin and written as application's Persistent Volume Claim .spec.volumeClaimTemplates.storageClassName parameter in StatefulSet configuration. StatefulSet manifest with emptyDir 05-stateful-set-volume-emptyDir.yaml and/or StatefulSet manifest with Persistent Volume 05-stateful-set-persistent-volume.yaml.
4.2.6. Stateful Set
Edit StatefulSet manifest with emptyDir 05-stateful-set-volume-emptyDir.yaml and/or StatefulSet manifest with Persistent Volume 05-stateful-set-persistent-volume.yaml according to your Storage Preferences.
In case we'd go with Volume of type emptyDir, ensure .spec.template.spec.containers.volumes is in place and look like the following:
volumes:- name: datadir-volumeemptyDir:medium: "" #accepted values: empty str (means node's default medium) or MemorysizeLimit: 1Giand ensure .spec.volumeClaimTemplates is commented.
In case we'd go with Persistent Volume storage, ensure .spec.template.spec.containers.volumes is commented and ensure .spec.volumeClaimTemplates is uncommented.
volumeClaimTemplates:- metadata:name: datadir-volumespec:accessModes:- ReadWriteOnceresources:requests:storage: 1Gi
## storageClassName has to be coordinated with k8s admin and has to be created as a `kind: StorageClass` resourcestorageClassName: storageclass-zookeeperand ensure storageClassName (storageclass-zookeeper in this example) is specified correctly, as described in Storage Class section
As .yaml file is ready, just apply it with kubectl
kubectl apply -f 05-stateful-set.yaml -n zoonsShould have as a result
statefulset.apps/zookeeper-node createdNow we can take a look into Zookeeper cluster deployed in k8s:
4.3. Explore Zookeeper cluster
4.3.1. DNS names
We are expecting to have ZooKeeper cluster of 3 pods inside zoons namespace, named as:
zookeeper-0
zookeeper-1
zookeeper-2Those pods are expected to have short DNS names as:
zookeeper-0.zookeepers.zoons
zookeeper-1.zookeepers.zoons
zookeeper-2.zookeepers.zoonswhere zookeepers is name of Zookeeper headless service and zoons is name of Zookeeper namespace.
and full DNS names (FQDN) as:
zookeeper-0.zookeepers.zoons.svc.cluster.local
zookeeper-1.zookeepers.zoons.svc.cluster.local
zookeeper-2.zookeepers.zoons.svc.cluster.local4.3.2. Resources
List pods in Zookeeper's namespace
kubectl get pod -n zoonsExpected output is like the following
NAME READY STATUS RESTARTS AGE
zookeeper-0 1/1 Running 0 9m2s
zookeeper-1 1/1 Running 0 9m2s
zookeeper-2 1/1 Running 0 9m2sList services
kubectl get service -n zoonsExpected output is like the following
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
zookeeper ClusterIP 10.108.36.44 <none> 2181/TCP 168m
zookeepers ClusterIP None <none> 2888/TCP,3888/TCP 31mList statefulsets
kubectl get statefulset -n zoonsExpected output is like the following
NAME READY AGE
zookeepers 3/3 10mIn case all looks fine Zookeeper cluster is up and running
：虚幻引擎编辑器界面详解)