Kubernetes的有状态应用示例：ZooKeeper

文章目录

  • 环境
  • ZooKeeper简介
  • 准备
  • 部署
    • 尝试1
    • 尝试2
  • 验证
  • 深入了解ZooKeeper
    • leader和follower
    • 测试
    • 容忍node故障
    • 持久化存储
  • 参考

环境

  • RHEL 9.3
  • Docker Community 24.0.7
  • minikube v1.32.0

ZooKeeper简介

Apache ZooKeeper是一个分布式的开源协调服务，用于分布式系统。ZooKeeper允许你读、写数据以及发现数据更新。数据按层次结构组织在文件系统中，并复制到ensemble（ZooKeeper服务器集合）中所有的ZooKeeper服务器。对数据的所有操作都是原子的和顺序一致的。ZooKeeper通过Zab共识协议在ensemble的所有服务器之间复制状态机，来确保这个特性。

Ensemble使用Zab协议选举一个leader，在选举出leader前不能写入数据。选举出leader后，ensemble使用Zab来确保所有写操作被复制到一个quorum，然后这些写操作才会被确认并对客户端可见。如果没有遵照加权quorums，quorum是包含当前leader的ensemble的主要组件。 例如，如果ensemble有3个服务器，一个包含leader和另一个服务器的组件构成一个quorum。如果ensemble不能达成一个quorum，数据将不能被写入。

ZooKeeper在内存中保存它们的整个状态机，并把每个改变都写入一个持久的WAL（Write Ahead Log）。当一个服务器宕机时，它能够通过回放WAL恢复之前的状态。为了防止WAL无限制的增长，ZooKeeper服务器会定期的将内存状态快照保存到存储介质。这些快照能够直接加载到内存，快照之前的所有WAL条目都可以被丢弃。

准备

清理环境：

minikube delete --all

重启电脑。

启动minikube：

minikube start

确认环境干净：

$ kubectl get all
NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   11m

$ kubectl get pvc
No resources found in default namespace.

$ kubectl get pv
No resources found

创建文件 zookeeper.yaml 如下：

apiVersion: v1
kind: Service
metadata:name: zk-hslabels:app: zk
spec:ports:- port: 2888name: server- port: 3888name: leader-electionclusterIP: Noneselector:app: zk
---
apiVersion: v1
kind: Service
metadata:name: zk-cslabels:app: zk
spec:ports:- port: 2181name: clientselector:app: zk
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:name: zk-pdb
spec:selector:matchLabels:app: zkmaxUnavailable: 1
---
apiVersion: apps/v1
kind: StatefulSet
metadata:name: zk
spec:selector:matchLabels:app: zkserviceName: zk-hsreplicas: 3updateStrategy:type: RollingUpdatepodManagementPolicy: OrderedReadytemplate:metadata:labels:app: zkspec:affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:- labelSelector:matchExpressions:- key: "app"operator: Invalues:- zktopologyKey: "kubernetes.io/hostname"containers:- name: kubernetes-zookeeperimagePullPolicy: Always# image: "registry.k8s.io/kubernetes-zookeeper:1.0-3.4.10"image: "docker.io/kaiding1/kubernetes-zookeeper:1.0-3.4.10"resources:requests:memory: "1Gi"cpu: "0.5"ports:- containerPort: 2181name: client- containerPort: 2888name: server- containerPort: 3888name: leader-electioncommand:- sh- -c- "start-zookeeper \--servers=3 \--data_dir=/var/lib/zookeeper/data \--data_log_dir=/var/lib/zookeeper/data/log \--conf_dir=/opt/zookeeper/conf \--client_port=2181 \--election_port=3888 \--server_port=2888 \--tick_time=2000 \--init_limit=10 \--sync_limit=5 \--heap=512M \--max_client_cnxns=60 \--snap_retain_count=3 \--purge_interval=12 \--max_session_timeout=40000 \--min_session_timeout=4000 \--log_level=INFO"readinessProbe:exec:command:- sh- -c- "zookeeper-ready 2181"initialDelaySeconds: 10timeoutSeconds: 5livenessProbe:exec:command:- sh- -c- "zookeeper-ready 2181"initialDelaySeconds: 10timeoutSeconds: 5volumeMounts:- name: datadirmountPath: /var/lib/zookeepersecurityContext:runAsUser: 1000fsGroup: 1000volumeClaimTemplates:- metadata:name: datadirspec:accessModes: [ "ReadWriteOnce" ]resources:requests:storage: 10Gi

注：因为访问不了 registry.k8s.io ，所以事先把image pull下来，并push到了可访问的位置。

部署

尝试1

$ kubectl apply -f zookeeper.yaml
service/zk-hs created
service/zk-cs created
poddisruptionbudget.policy/zk-pdb created
statefulset.apps/zk created

检查pod：

$ kubectl get pod
NAME   READY   STATUS    RESTARTS   AGE
zk-0   1/1     Running   0          57s
zk-1   0/1     Pending   0          35s

进入 zk-0 pod：

kubectl exec -it zk-0 -- bash

查看当前身份以及 /var/lib/zookeeper 目录：

zookeeper@zk-0:/$ whoami
zookeeper

zookeeper@zk-0:/$ ls -l /var/lib
total 0
drwxr-xr-x. 1 root root  42 Jun 13  2017 apt
......
drwxrwxrwx. 3 root root  18 Feb  6 07:13 zookeeper

zookeeper@zk-0:/$ ls -l /var/lib/zookeeper/
total 0
drwxr-xr-x. 4 zookeeper zookeeper 46 Feb  6 07:13 data

可见， /var/lib/zookeeper 目录是 777 ，所以 zookeeper 能在此创建目录。

退出容器，再次查看pod：

$ kubectl get pod
NAME   READY   STATUS    RESTARTS   AGE
zk-0   1/1     Running   0          7m37s
zk-1   0/1     Pending   0          7m15s

可见， zk-1 始终处于 Pending 状态。

$ kubectl describe pod zk-1
......
Events:Type     Reason            Age                   From               Message----     ------            ----                  ----               -------Warning  FailedScheduling  8m4s                  default-scheduler  0/1 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..Warning  FailedScheduling  2m39s (x2 over 8m3s)  default-scheduler  0/1 nodes are available: 1 node(s) didn't match pod anti-affinity rules. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod..

可见，在affinity规则下，没有符合条件的node。

尝试2

清理环境：

$ kubectl delete -f zookeeper.yaml 
service "zk-hs" deleted
service "zk-cs" deleted
poddisruptionbudget.policy "zk-pdb" deleted
statefulset.apps "zk" deleted

$ kubectl get pvc
NAME           STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
datadir-zk-0   Bound    pvc-f4a42476-b3e7-48ca-ad8d-b8f6ec278f1e   10Gi       RWO            standard       15m
datadir-zk-1   Bound    pvc-924378b6-5810-4ebc-86a7-90037221381f   10Gi       RWO            standard       14m

$ kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                  STORAGECLASS   REASON   AGE
pvc-924378b6-5810-4ebc-86a7-90037221381f   10Gi       RWO            Delete           Bound    default/datadir-zk-1   standard                14m
pvc-f4a42476-b3e7-48ca-ad8d-b8f6ec278f1e   10Gi       RWO            Delete           Bound    default/datadir-zk-0   standard                15m

$ kubectl delete pvc datadir-zk-0 datadir-zk-1
persistentvolumeclaim "datadir-zk-0" deleted
persistentvolumeclaim "datadir-zk-1" deleted

修改 zookeeper.yaml ，把 requiredDuringSchedulingIgnoredDuringExecution 改为 preferredDuringSchedulingIgnoredDuringExecution 。

......spec:affinity:podAntiAffinity:preferredDuringSchedulingIgnoredDuringExecution:- weight: 100podAffinityTerm:labelSelector:matchExpressions:- key: "app"operator: Invalues:- zookeepertopologyKey: "kubernetes.io/hostname"
......

$ kubectl apply -f zookeeper2.yaml
service/zk-hs created
service/zk-cs created
poddisruptionbudget.policy/zk-pdb created
statefulset.apps/zk created

$ kubectl get pod
NAME   READY   STATUS    RESTARTS   AGE
zk-0   1/1     Running   0          2m1s
zk-1   1/1     Running   0          99s
zk-2   1/1     Running   0          77s

$ kubectl get sts
NAME   READY   AGE
zk     3/3     6m31s

注：如果又遇到pod Pending 问题，有以下错误：

Events:Type     Reason            Age        From               Message----     ------            ----       ----               -------Warning  FailedScheduling  <unknown>  default-scheduler  0/1 nodes are available: 1 Insufficient memory.Warning  FailedScheduling  <unknown>  default-scheduler  0/1 nodes are available: 1 Insufficient memory.

则需要先 minikube delete ，然后在启动minikube时，增大内存，比如：

minikube start --memory='7500mb'

查看存储：

$ kubectl get pvc
NAME           STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
datadir-zk-0   Bound    pvc-7b501343-085f-48d8-b88a-ade5686b26c9   10Gi       RWO            standard       7m18s
datadir-zk-1   Bound    pvc-4af3ab29-a3c8-4b26-addf-40656e998021   10Gi       RWO            standard       6m56s
datadir-zk-2   Bound    pvc-60869d4a-389d-4f86-8446-4fda9858fe40   10Gi       RWO            standard       6m34s

$ kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                  STORAGECLASS   REASON   AGE
pvc-4af3ab29-a3c8-4b26-addf-40656e998021   10Gi       RWO            Delete           Bound    default/datadir-zk-1   standard                6m58s
pvc-60869d4a-389d-4f86-8446-4fda9858fe40   10Gi       RWO            Delete           Bound    default/datadir-zk-2   standard                6m36s
pvc-7b501343-085f-48d8-b88a-ade5686b26c9   10Gi       RWO            Delete           Bound    default/datadir-zk-0   standard                7m20s

验证

进入容器：

kubectl exec -it zk-0 -- bash

检查连通性：

zookeeper@zk-0:/$ echo "Are you ok? $(echo ruok | nc 127.0.0.1 2181)"
Are you ok? imok

检查模式：

zookeeper@zk-0:/$ echo srvr | nc localhost 2181
Zookeeper version: 3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
Latency min/avg/max: 0/0/0
Received: 123
Sent: 122
Connections: 1
Outstanding: 0
Zxid: 0x0
Mode: follower
Node count: 4

可见，pod zk-0 是一个follower。

退出容器，用同样的方法检查另外两个pod：

zookeeper@zk-1:/$ echo srvr | nc localhost 2181 | grep -i mode
Mode: leader

zookeeper@zk-2:/$ echo srvr | nc localhost 2181 | grep -i mode
Mode: follower

在leader pod里：

zookeeper@zk-1:/$ echo dump | nc localhost 2181
SessionTracker dump:
Session Sets (0):
ephemeral nodes dump:
Sessions with Ephemerals (0):

zookeeper@zk-1:/$ echo stat | nc localhost 2181
Zookeeper version: 3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
Clients:/127.0.0.1:60780[0](queued=0,recved=1,sent=0)Latency min/avg/max: 0/0/0
Received: 221
Sent: 220
Connections: 1
Outstanding: 0
Zxid: 0x100000000
Mode: leader
Node count: 4

zookeeper@zk-1:/$ echo envi | nc localhost 2181
Environment:
zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
host.name=zk-1.zk-hs.default.svc.cluster.local
java.version=1.8.0_131
java.vendor=Oracle Corporation
java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
java.io.tmpdir=/tmp
java.compiler=<NA>
os.name=Linux
os.arch=amd64
os.version=5.14.0-362.18.1.el9_3.x86_64
user.name=zookeeper
user.home=/home/zookeeper
user.dir=/

zookeeper@zk-1:/$ echo conf | nc localhost 2181
clientPort=2181
dataDir=/var/lib/zookeeper/data/version-2
dataLogDir=/var/lib/zookeeper/data/log/version-2
tickTime=2000
maxClientCnxns=60
minSessionTimeout=4000
maxSessionTimeout=40000
serverId=2
initLimit=10
syncLimit=5
electionAlg=3
electionPort=3888
quorumPort=2888
peerType=0

zookeeper@zk-1:/$ echo mntr | nc localhost 2181
zk_version	3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
zk_avg_latency	0
zk_max_latency	0
zk_min_latency	0
zk_packets_received	242
zk_packets_sent	241
zk_num_alive_connections	1
zk_outstanding_requests	0
zk_server_state	leader
zk_znode_count	4
zk_watch_count	0
zk_ephemerals_count	0
zk_approximate_data_size	27
zk_open_file_descriptor_count	41
zk_max_file_descriptor_count	1048576
zk_followers	2
zk_synced_followers	2
zk_pending_syncs	0

事实上，在spec中，使用了 zookeeper-ready 2181 来作为 readinessProbe 和 livenessProbe ：

......readinessProbe:exec:command:- sh- -c- "zookeeper-ready 2181"initialDelaySeconds: 10timeoutSeconds: 5livenessProbe:exec:command:- sh- -c- "zookeeper-ready 2181"initialDelaySeconds: 10timeoutSeconds: 5
......

查看 zookeeper-ready ：

zookeeper@zk-1:/$ which zookeeper-ready
/usr/bin/zookeeper-ready

zookeeper@zk-1:/$ cat /usr/bin/zookeeper-ready
#!/usr/bin/env bash
# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.# zkOk.sh uses the ruok ZooKeeper four letter work to determine if the instance
# is health. The $? variable will be set to 0 if server responds that it is 
# healthy, or 1 if the server fails to respond.OK=$(echo ruok | nc 127.0.0.1 $1)
if [ "$OK" == "imok" ]; thenexit 0
elseexit 1
fi

可见，它就是用上面的方法来检测连通性的。

深入了解ZooKeeper

leader和follower

每个ZooKeeper ensemble里的服务器节点都有一个唯一的ID与其网络地址关联。每个node都知道该ID。

hostname：

$ for i in 0 1 2; do kubectl exec zk-$i -- hostname; done
zk-0
zk-1
zk-2

ZooKeeper node在数据目录的 myid 文件中存储其服务器ID（本例中数据目录是 /var/lib/zookeeper ）。

$ for i in 0 1 2; do echo "myid zk-$i"; kubectl exec zk-$i -- cat /var/lib/zookeeper/data/myid; done
myid zk-0
1
myid zk-1
2
myid zk-2
3

注意：Kubernetes从0开始计数，而ZooKeeper是从1开始计数。

查看FQDN（Fully Qualified Domain Name）：

$ for i in 0 1 2; do kubectl exec zk-$i -- hostname -f; done
zk-0.zk-hs.default.svc.cluster.local
zk-1.zk-hs.default.svc.cluster.local
zk-2.zk-hs.default.svc.cluster.local

ZooKeeper headless service为StatefulSet中的每个pod创建一个域名。

Kubernetes DNS中的A记录将FQDN解析为pod的IP地址。将来如果pod被重新调度或者升级，则A记录会放置新的IP地址，但是名字保持不变。

ZooKeeper使用一个 zoo.cfg 的配置文件（ /opt/zookeeper/conf/zoo.cfg ）。我们来查看该文件：

$ kubectl exec zk-0 -- cat /opt/zookeeper/conf/zoo.cfg
#This file was autogenerated DO NOT EDIT
clientPort=2181
dataDir=/var/lib/zookeeper/data
dataLogDir=/var/lib/zookeeper/data/log
tickTime=2000
initLimit=10
syncLimit=5
maxClientCnxns=60
minSessionTimeout=4000
maxSessionTimeout=40000
autopurge.snapRetainCount=3
autopurge.purgeInteval=12
server.1=zk-0.zk-hs.default.svc.cluster.local:2888:3888
server.2=zk-1.zk-hs.default.svc.cluster.local:2888:3888
server.3=zk-2.zk-hs.default.svc.cluster.local:2888:3888

该文件是由 start-zookeeper 创建的：

......- sh- -c- "start-zookeeper \--servers=3 \--data_dir=/var/lib/zookeeper/data \--data_log_dir=/var/lib/zookeeper/data/log \--conf_dir=/opt/zookeeper/conf \
......

测试

使用ZooKeeper的命令行工具 zkCli.sh 来测试。它可以：

  • 创建znode
  • 获取数据
  • 监视znode变化
  • 把数据放置于znode
  • 为znode创建子节点
  • 列出子节点
  • 检查状态
  • 删除znode

ZooKeeper里的znode既像文件（可以有内容），也像目录（可以有子znode）。

我们来把 world 写入 zk-0 的 /hello 里。

$ kubectl exec zk-0 zkCli.sh create /hello world
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
Connecting to localhost:2181
2024-02-06 08:35:34,023 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:35:34,027 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=zk-0.zk-hs.default.svc.cluster.local
2024-02-06 08:35:34,027 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_131
2024-02-06 08:35:34,028 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2024-02-06 08:35:34,028 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:35:34,029 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:35:34,029 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:35:34,029 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2024-02-06 08:35:34,029 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2024-02-06 08:35:34,029 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
2024-02-06 08:35:34,029 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
2024-02-06 08:35:34,029 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:35:34,030 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=zookeeper
2024-02-06 08:35:34,030 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/home/zookeeper
2024-02-06 08:35:34,030 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/
2024-02-06 08:35:34,031 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:35:34,051 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:35:34,096 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@876] - Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:35:34,114 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x18d7d581b330000, negotiated timeout = 30000WATCHER::WatchedEvent state:SyncConnected type:None path:null
Created /hello

然后从另一个pod读取该值：

$ kubectl exec zk-1 -- zkCli.sh get /hello
Connecting to localhost:2181
2024-02-06 08:36:34,736 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:36:34,738 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=zk-1.zk-hs.default.svc.cluster.local
2024-02-06 08:36:34,738 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_131
2024-02-06 08:36:34,740 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2024-02-06 08:36:34,740 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:36:34,740 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:36:34,740 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:36:34,740 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2024-02-06 08:36:34,740 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2024-02-06 08:36:34,740 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
2024-02-06 08:36:34,740 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
2024-02-06 08:36:34,741 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:36:34,741 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=zookeeper
2024-02-06 08:36:34,741 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/home/zookeeper
2024-02-06 08:36:34,741 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/
2024-02-06 08:36:34,742 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:36:34,755 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:36:34,820 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@876] - Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:36:34,833 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x28d7d581b360000, negotiated timeout = 30000WATCHER::WatchedEvent state:SyncConnected type:None path:null
world
cZxid = 0x100000002
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000002
mtime = Tue Feb 06 08:35:34 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0

事实上，ZooKeeper ensemble里的每个服务器都可以读取 /hello ：

$ for i in 0 1 2; do kubectl exec zk-$i -- zkCli.sh get /hello | grep world; done
world
cZxid = 0x100000002
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000002
mtime = Tue Feb 06 08:35:34 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0
world
cZxid = 0x100000002
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000002
mtime = Tue Feb 06 08:35:34 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0
cZxid = 0x100000002
world
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000002
mtime = Tue Feb 06 08:35:34 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0

容忍node故障

我们来看一下共识算法。删除一个服务器，然后尝试向ZooKeeper ensemble中写入数据，由于还有两个ZooKeeper服务器，所以写入操作还能继续工作。

$ kubectl delete --force=true --grace-period=0 pod zk-2  &
sleep 1; kubectl delete --force=true --grace-period=0 pod zk-2  &
sleep 1
kubectl exec zk-0 zkCli.sh set /hello world_should_work
sleep 1
kubectl exec zk-1 zkCli.sh get /hello
[1] 148438
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "zk-2" force deleted
[1]+  Done                    kubectl delete --force=true --grace-period=0 pod zk-2
[1] 148686
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "zk-2" force deleted
[1]+  Done                    kubectl delete --force=true --grace-period=0 pod zk-2
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
Connecting to localhost:2181
2024-02-06 08:45:21,133 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:45:21,135 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=zk-0.zk-hs.default.svc.cluster.local
2024-02-06 08:45:21,135 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_131
2024-02-06 08:45:21,137 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2024-02-06 08:45:21,137 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:45:21,137 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:45:21,137 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:45:21,138 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2024-02-06 08:45:21,138 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2024-02-06 08:45:21,138 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
2024-02-06 08:45:21,140 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
2024-02-06 08:45:21,140 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:45:21,140 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=zookeeper
2024-02-06 08:45:21,140 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/home/zookeeper
2024-02-06 08:45:21,140 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/
2024-02-06 08:45:21,141 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:45:21,157 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:45:21,201 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@876] - Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:45:21,206 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x18d7d581b330003, negotiated timeout = 30000WATCHER::WatchedEvent state:SyncConnected type:None path:null
cZxid = 0x100000002
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000017
mtime = Tue Feb 06 08:45:21 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 17
numChildren = 0
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
Connecting to localhost:2181
2024-02-06 08:45:22,821 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:45:22,824 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=zk-1.zk-hs.default.svc.cluster.local
2024-02-06 08:45:22,824 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_131
2024-02-06 08:45:22,827 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2024-02-06 08:45:22,827 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:45:22,827 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:45:22,827 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:45:22,827 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2024-02-06 08:45:22,827 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2024-02-06 08:45:22,828 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
2024-02-06 08:45:22,828 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
2024-02-06 08:45:22,828 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:45:22,828 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=zookeeper
2024-02-06 08:45:22,828 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/home/zookeeper
2024-02-06 08:45:22,828 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/
2024-02-06 08:45:22,830 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:45:22,845 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:45:22,897 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@876] - Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:45:22,905 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x28d7d581b360005, negotiated timeout = 30000WATCHER::WatchedEvent state:SyncConnected type:None path:null
cZxid = 0x100000002
world_should_work
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000017
mtime = Tue Feb 06 08:45:21 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 17
numChildren = 0

验证：

$ for i in 0 1 2; do kubectl exec zk-$i -- zkCli.sh get /hello | grep world; done
cZxid = 0x100000002
world_should_work
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000017
mtime = Tue Feb 06 08:45:21 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 17
numChildren = 0
cZxid = 0x100000002
world_should_work
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000017
mtime = Tue Feb 06 08:45:21 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 17
numChildren = 0
cZxid = 0x100000002
world_should_work
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000017
mtime = Tue Feb 06 08:45:21 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 17
numChildren = 0

写入操作还能工作，是因为ZooKeeper ensemble的quorum（法定人数），如果删除两个服务器，就没有quorum了。

$ kubectl delete --force=true --grace-period=0 pod zk-2  &
kubectl delete --force=true --grace-period=0 pod zk-1  &
sleep 1; kubectl delete --force=true --grace-period=0 pod zk-2  &
sleep 1; kubectl delete --force=true --grace-period=0 pod zk-1  &
sleep 1
kubectl exec zk-0 zkCli.sh set /hello world_should_not_work
sleep 1
kubectl exec zk-0 zkCli.sh get /hello
sleep 20 # If you are running manually use kubectl get pods to see status of pods restarting
kubectl exec zk-0 zkCli.sh get /hello
[1] 159326
[2] 159327
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "zk-2" force deleted
pod "zk-1" force deleted
[1]-  Done                    kubectl delete --force=true --grace-period=0 pod zk-2
[2]+  Done                    kubectl delete --force=true --grace-period=0 pod zk-1
[1] 159519
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
Error from server (NotFound): pods "zk-2" not found
[1]+  Exit 1                  kubectl delete --force=true --grace-period=0 pod zk-2
[1] 159526
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "zk-1" force deleted
[1]+  Done                    kubectl delete --force=true --grace-period=0 pod zk-1
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
Connecting to localhost:2181
2024-02-06 08:51:54,920 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:51:54,923 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=zk-0.zk-hs.default.svc.cluster.local
2024-02-06 08:51:54,923 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_131
2024-02-06 08:51:54,925 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2024-02-06 08:51:54,925 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:51:54,925 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:51:54,925 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:51:54,925 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2024-02-06 08:51:54,925 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2024-02-06 08:51:54,926 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
2024-02-06 08:51:54,926 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
2024-02-06 08:51:54,926 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:51:54,926 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=zookeeper
2024-02-06 08:51:54,926 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/home/zookeeper
2024-02-06 08:51:54,926 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/
2024-02-06 08:51:54,929 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:51:54,949 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:51:55,025 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@876] - Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:51:55,036 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x18d7d581b330005, negotiated timeout = 30000WATCHER::WatchedEvent state:SyncConnected type:None path:null
cZxid = 0x100000002
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000022
mtime = Tue Feb 06 08:51:55 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 21
numChildren = 0
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
Connecting to localhost:2181
2024-02-06 08:51:56,664 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:51:56,666 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=zk-0.zk-hs.default.svc.cluster.local
2024-02-06 08:51:56,666 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_131
2024-02-06 08:51:56,668 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2024-02-06 08:51:56,668 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:51:56,668 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:51:56,668 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:51:56,668 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2024-02-06 08:51:56,668 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2024-02-06 08:51:56,668 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
2024-02-06 08:51:56,668 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
2024-02-06 08:51:56,668 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:51:56,668 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=zookeeper
2024-02-06 08:51:56,669 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/home/zookeeper
2024-02-06 08:51:56,669 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/
2024-02-06 08:51:56,670 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:51:56,690 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:51:56,748 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@876] - Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:51:56,755 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x18d7d581b330006, negotiated timeout = 30000WATCHER::WatchedEvent state:SyncConnected type:None path:null
world_should_not_work
cZxid = 0x100000002
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000022
mtime = Tue Feb 06 08:51:55 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 21
numChildren = 0
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
Connecting to localhost:2181
2024-02-06 08:52:17,354 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2024-02-06 08:52:17,355 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=zk-0.zk-hs.default.svc.cluster.local
2024-02-06 08:52:17,356 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_131
2024-02-06 08:52:17,357 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2024-02-06 08:52:17,357 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2024-02-06 08:52:17,357 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/usr/bin/../build/classes:/usr/bin/../build/lib/*.jar:/usr/bin/../share/zookeeper/zookeeper-3.4.10.jar:/usr/bin/../share/zookeeper/slf4j-log4j12-1.6.1.jar:/usr/bin/../share/zookeeper/slf4j-api-1.6.1.jar:/usr/bin/../share/zookeeper/netty-3.10.5.Final.jar:/usr/bin/../share/zookeeper/log4j-1.2.16.jar:/usr/bin/../share/zookeeper/jline-0.9.94.jar:/usr/bin/../src/java/lib/*.jar:/usr/bin/../etc/zookeeper:
2024-02-06 08:52:17,357 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2024-02-06 08:52:17,358 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2024-02-06 08:52:17,358 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2024-02-06 08:52:17,358 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
2024-02-06 08:52:17,358 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
2024-02-06 08:52:17,358 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=5.14.0-362.18.1.el9_3.x86_64
2024-02-06 08:52:17,358 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=zookeeper
2024-02-06 08:52:17,358 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/home/zookeeper
2024-02-06 08:52:17,358 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/
2024-02-06 08:52:17,359 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2024-02-06 08:52:17,374 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2024-02-06 08:52:17,436 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@876] - Socket connection established to localhost/127.0.0.1:2181, initiating session
2024-02-06 08:52:17,444 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x18d7d581b330007, negotiated timeout = 30000WATCHER::WatchedEvent state:SyncConnected type:None path:null
cZxid = 0x100000002
world_should_not_work
ctime = Tue Feb 06 08:35:34 UTC 2024
mZxid = 0x100000022
mtime = Tue Feb 06 08:51:55 UTC 2024
pZxid = 0x100000002
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 21
numChildren = 0

可见，还是写入了。我测试了几次，都是这样。难道是因为pod恢复的太快了？

持久化存储

删除StatefulSet：

kubectl delete statefulset zk

验证：

$ kubectl get pod
No resources found in default namespace.

重建StatefulSet：

$ kubectl apply -f zookeeper2.yaml
service/zk-hs unchanged
service/zk-cs unchanged
poddisruptionbudget.policy/zk-pdb configured
statefulset.apps/zk created

$ kubectl exec zk-0 -- zkCli.sh get /hello
......
WATCHER::WatchedEvent state:SyncConnected type:None path:null
ccccc
......

注： ccccc 是最近一次写入的数据。

可见，重建StatefulSet和其pod之后，原来的数据还在。

当ZooKeeper的StatefulSet的pod被重新调度或者升级时，会将PV mount到ZooKeeper服务器的数据目录。持久化数据仍然保存在那里。对于Cassandra或者Consul或者etcd或其它任何数据库也都一样。

参考

  • https://github.com/cloudurable/kube-zookeeper-statefulsets/wiki/Tutorial-Part-1:--Managing-Kubernetes-StatefulSets-using-ZooKeeper-and-Minikube
  • https://kubernetes.io/docs/tutorials/stateful-application/zookeeper

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.mzph.cn/news/674129.shtml

如若内容造成侵权/违法违规/事实不符，请联系多彩编程网进行投诉反馈email:809451989@qq.com，一经查实，立即删除！

相关文章

服务器安装Docker （centOS）

服务器安装Docker （centOS）

1. 卸载旧版本的Docker&#xff08;如果有&#xff09; 首先&#xff0c;如果您的系统上安装了旧版本的Docker&#xff0c;需要将其卸载。Docker的旧版本称为docker或docker-engine。使用以下命令来卸载旧版本&#xff1a; sudo yum remove docker \ docker-client \ docker-…
阅读更多...
2024牛客寒假算法基础集训营2部分题解

2024牛客寒假算法基础集训营2部分题解

Tokitsukaze and Bracelet 链接&#xff1a;登录—专业IT笔试面试备考平台_牛客网 来源&#xff1a;牛客网 题目描述 《绯染天空》是一款由 key 社与飞机社共同开发的角色扮演游戏&#xff0c;剧情内容由著名的剧本作家麻枝准编写。它是一款氪金手游&#xff0c;但也有 st…
阅读更多...
成为CSDN博客优质创作者或者博客专家吧

成为CSDN博客优质创作者或者博客专家吧

成为CSDN博客优质创作者或者博客专家吧 文章目录 成为CSDN博客优质创作者或者博客专家吧一、前言二、如何成为CSDN的博客专家1、2009年的要求和申请方式2、最新的CSDN博客专家要求和申请方式3、创作者身份认证4、CSDN所有认证的介绍 三、写博客的好处1、比较官方的说法&#xf…
阅读更多...
Nacos1.X源码解读（待完善）

Nacos1.X源码解读（待完善）

目录 下载源码 注册服务 客户端注册流程 注册接口API 服务端处理注册请求 设计亮点 服务端流程图 下载源码 1. 克隆git地址到本地 # 下载nacos源码 git clone https://github.com/alibaba/nacos.git 2. 切换分支到1.4.7, maven编译(3.5.1) 3. 找到启动类com.alibaba.na…
阅读更多...
波卡 2023 四季度报告：开发者数量位列加密生态前三，五项新技术将于今年发布

波卡 2023 四季度报告：开发者数量位列加密生态前三，五项新技术将于今年发布

作者&#xff1a;Nicholas Garcia&#xff5c;Messari 研究分析师 编译&#xff1a;OneBlock 原文&#xff1a;https://messari.io/report/state-of-polkadot-q4-2023?utm_mediumorganic_social&utm_sourcetwitter_messari&utm_campaignstate_of_polkadot_q4_2023 …
阅读更多...
RabbitMQ的延迟队列实现[死信队列](笔记二)

RabbitMQ的延迟队列实现[死信队列](笔记二)

上一篇已经讲述了实现死信队列的rabbitMQ服务配置&#xff0c;可以点击: RabbitMQ的延迟队列实现(笔记一) 目录 搭建一个新的springboot项目模仿订单延迟支付过期操作启动项目进行测试 搭建一个新的springboot项目 1.相关核心依赖如下 <dependency><groupId>org.…
阅读更多...
13. UE5 RPG限制Attribute的值的范围以及生成结构体

13. UE5 RPG限制Attribute的值的范围以及生成结构体

前面几章&#xff0c;我们实现了通过GameplayEffect对Attribute值的修改&#xff0c;比如血量和蓝量&#xff0c;我们都是有一个最大血量和最大蓝量去限制它的最大值&#xff0c;而且血量和蓝量最小值不会小于零。之前我们是没有实现相关限制的&#xff0c;接下来&#xff0c;我…
阅读更多...
小白水平理解面试经典题目LeetCode 71. Simplify Path【Stack类】

小白水平理解面试经典题目LeetCode 71. Simplify Path【Stack类】

71. 简化路径 小白渣翻译 给定一个字符串 path &#xff0c;它是 Unix 风格文件系统中文件或目录的绝对路径&#xff08;以斜杠 ‘/’ 开头&#xff09;&#xff0c;将其转换为简化的规范路径。 在 Unix 风格的文件系统中&#xff0c;句点 ‘.’ 指的是当前目录&#xff0c;…
阅读更多...
flutter监听app进入前后台状态的实现

flutter监听app进入前后台状态的实现

在开发app的过程中&#xff0c;我们经常需要根据app的前后台的状态&#xff0c;做一些事情&#xff0c;那么我们在flutter中是如何实现这一监听的&#xff1f; flutter给我们提供了WidgetsBindingObserver来进行一些状态的判断&#xff0c;但是判断前后台的状态只是该API种其中…
阅读更多...
微软.NET6开发的C#特性——接口和属性

微软.NET6开发的C#特性——接口和属性

我是荔园微风&#xff0c;作为一名在IT界整整25年的老兵&#xff0c;看到不少初学者在学习编程语言的过程中如此的痛苦&#xff0c;我决定做点什么&#xff0c;下面我就重点讲讲微软.NET6开发人员需要知道的C#特性。 C#经历了多年发展&#xff0c; 进行了多次重大创新&#xf…
阅读更多...
为什么要设置止损

为什么要设置止损

2024年1月至2月7日&#xff0c;A股最令人瞩目的事件就是代表小微盘的中证500和中证1000雪球连续敲入&#xff0c;以及万得微盘指数的崩塌&#xff08;1个月下跌50%&#xff09;。 这次的这个过程中&#xff0c;止损很重要。一般情况下&#xff0c;如果设置了20%回撤止损的话&am…
阅读更多...
Vue-58、Vue技术命名路由

Vue-58、Vue技术命名路由

命名路由 1、作用&#xff1a;可以简化路由的跳转 2、如何使用 1、给路由命名&#xff1a;routes:[{name:guanyu,//给路由命名path:/about,component:About},{path:/home,component:Home,children:[{path:news,component:News},{path:message,component:Message,children:[{…
阅读更多...
JDK17新特性

JDK17新特性

JDK 17 是 Java Development Kit 的一个版本&#xff0c;它引入了许多新的特性和改进。以下是 JDK 17 中一些主要的新特性&#xff1a; Sealed classes&#xff1a;此特性允许开发人员限制某个类的子类数量&#xff0c;并且只有在同一个模块中定义的类才能作为其子类。 Patter…
阅读更多...
#Js篇：数组的方法es5和es6

#Js篇：数组的方法es5和es6

数组方法学习 判断一个数组的方法 Array.isArray() javascript内置的方法&#xff0c;用于检测给定的对象是否为数组类型。 早期版本兼容性es5引入&#xff0c;对于不支持es5的老旧浏览器&#xff08;如ie8及更早版本&#xff09;&#xff0c;该方法不可用。 es5实例方法 …
阅读更多...
跨品牌智能家居控制_从原理到实现_HomeAssistant

跨品牌智能家居控制_从原理到实现_HomeAssistant

项目地址&#xff1a;https://github.com/home-assistant/core Star&#xff1a;67 K 1 引言 最近去南方玩&#xff0c;住了一些智能酒店&#xff0c;自动开关电视、窗帘、灯、空调&#xff0c;还挺好用的&#xff0c;尤其喜欢关灯这功能。先不说它的理解能力&#xff08;对同…
阅读更多...
鸿蒙学习-app.json5配置文件

鸿蒙学习-app.json5配置文件

官网文档参考&#xff1a;https://developer.harmonyos.com/cn/docs/documentation/doc-guides-V3/app-configuration-file-0000001427584584-V3 位于AppScope下的app.json5配置文件 一、基础属性 {"app": {/*包名*/"bundleName": "com.example.dem…
阅读更多...
豪掷770亿！华为员工集体“分红大狂欢”：至少14万人受益

豪掷770亿！华为员工集体“分红大狂欢”：至少14万人受益

豪掷770亿&#xff01;华为员工集体“分红大狂欢”&#xff1a;至少14万人受益 近日&#xff0c;华为宣布了其2023年度分红计划&#xff0c;总金额高达770.85亿元&#xff0c;预计至少将惠及14万员工。这一消息引发了广泛关注和热议&#xff0c;成为业界的一大亮点。作为中国领…
阅读更多...
Go 语言中如何大小端字节序？int 转 byte 是如何进行的？

Go 语言中如何大小端字节序？int 转 byte 是如何进行的？

嗨&#xff0c;大家好&#xff01;我是波罗学。 本文是系列文章 Go 技巧第十五篇&#xff0c;系列文章查看&#xff1a;Go 语言技巧。 我们先看这样一个问题&#xff1a;“Go 语言中&#xff0c;将 byte 转换为 int 时是否涉及字节序&#xff08;endianness&#xff09;&#x…
阅读更多...
代码随想录算法训练营第42天 | 01背包理论基础 416.分割等和子集

代码随想录算法训练营第42天 | 01背包理论基础 416.分割等和子集

01背包理论基础 问题定义&#xff1a;有n件物品和一个能装重量为w的背包&#xff0c;第i件物品的重量是weight[i]&#xff0c;得到的价值是value[i]。每件物品只能用一次&#xff0c;求解将哪些物品装入背包获得的总价值最大。dp数组含义&#xff1a;dp[i][j] 表示从下标为 [0…
阅读更多...
C++服务器端开发（2）:确定服务器框架

C++服务器端开发（2）:确定服务器框架

选择C服务器框架时&#xff0c;可以考虑&#xff1a; 并发性能&#xff1a;C的强项之一是其并发性能。选择一个具有高并发处理能力的服务器框架&#xff0c;可以更好地满足大量并发请求的需求。例如&#xff0c;libevent、Boost.Asio和CppServer都是具有良好并发性能的C服务器框…
阅读更多...
最新文章

Copyright @ 2022~2023