etcd是CoreOS团队于2013年6月发起的开源项目,它的目标是构建一个高可用的分布式键值(key-value)数据库,用于配置共享和服务发现
etcd内部采用raft
协议作为一致性算法,etcd基于Go语言实现。
etcd作为服务发现系统,有以下的特点:
- 简单:安装配置简单,HTTP+JSON的 API进行交互,使用curl命令可以轻松使用
- 安全:支持SSL证书验证机制
- 快速:根据官方提供的benchmark数据,单实例支持每秒1k+写操作
- 可靠:采用raft算法,实现分布式系统数据的可用性和一致性
- 分布式系统中数据分为控制数据和应用数据,etcd处理的数据为控制数据
同一个分布式集群中的进程或服务,互相感知并建立链接,这就是服务发现。解决服务发现问题的三大支柱:
- 一个强一致性、高可用的服务存储目录
- 基于Raft算法
- 一种注册服务和监控服务健康状态的机制
- 一种查找和链接服务的机制
- 微服务协同工作架构中,服务动态添加
etcd基本命令
集群搭建参考k8s集群搭建里的etcd
更新一个节点
#首先查看所有节点 [root@slave2 ~]# etcdctl member list 646ec025a72fc8a: name=etcd3 peerURLs=http://192.168.132.133:2380 clientURLs=http://192.168.132.133:2379 isLeader=true dde447f371b55f50: name=etcd1 peerURLs=http://192.168.132.131:2380 clientURLs=http://192.168.132.131:2379 isLeader=false e96057c7f8ba5f4b: name=etcd2 peerURLs=http://192.168.132.132:2380 clientURLs=http://192.168.132.132:2379 isLeader=false#更新一个节点 [root@slave2 ~]# etcdctl member update 646ec025a72fc8a http://192.168.132.133:2380 Updated member with ID 646ec025a72fc8a in cluster
删除一个节点
#查看所有节点
[root@slave2 ~]# etcdctl member list 646ec025a72fc8a: name=etcd3 peerURLs=http://192.168.132.133:2380 clientURLs=http://192.168.132.133:2379 isLeader=false dde447f371b55f50: name=etcd1 peerURLs=http://192.168.132.131:2380 clientURLs=http://192.168.132.131:2379 isLeader=true e96057c7f8ba5f4b: name=etcd2 peerURLs=http://192.168.132.132:2380 clientURLs=http://192.168.132.132:2379 isLeader=false
#删除一个节点
[root@slave2 ~]# etcdctl member remove 646ec025a72fc8a Removed member 646ec025a72fc8a from cluster
#再次查看节点,使用当前命令报错
[root@slave2 ~]# etcdctl member list Error: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 127.0.0.1:2379: getsockopt: connection refused ; error #1: dial tcp 127.0.0.1:4001: getsockopt: connection refused error #0: dial tcp 127.0.0.1:2379: getsockopt: connection refused error #1: dial tcp 127.0.0.1:4001: getsockopt: connection refused
#换一个命令
[root@slave2 ~]# etcdctl --endpoints "http://192.168.132.131:2379" member list dde447f371b55f50: name=etcd1 peerURLs=http://192.168.132.131:2380 clientURLs=http://192.168.132.131:2379 isLeader=true e96057c7f8ba5f4b: name=etcd2 peerURLs=http://192.168.132.132:2380 clientURLs=http://192.168.132.132:2379 isLeader=false
添加节点
#将删除的节点添加回来
[root@slave2 ~]# etcdctl --endpoints "http://192.168.132.132:2379" member add etcd3 http://192.168.132.133:2380 Added member named etcd3 with ID 7a6809311b4a3579 to clusterETCD_NAME="etcd3" ETCD_INITIAL_CLUSTER="etcd3=http://192.168.132.133:2380,etcd1=http://192.168.132.131:2380,etcd2=http://192.168.132.132:2380" ETCD_INITIAL_CLUSTER_STATE="existing"
#再次查看,状态为unstarted [root@slave2 ~]# etcdctl --endpoints "http://192.168.132.132:2379" member list 7a6809311b4a3579[unstarted]: peerURLs=http://192.168.132.133:2380 dde447f371b55f50: name=etcd1 peerURLs=http://192.168.132.131:2380 clientURLs=http://192.168.132.131:2379 isLeader=true e96057c7f8ba5f4b: name=etcd2 peerURLs=http://192.168.132.132:2380 clientURLs=http://192.168.132.132:2379 isLeader=false
清空目标节点数据
目标节点从集群中删除后,成员信息会更新。新节点是作为一个全新的节点加入集群,如果data-dir有数据,etcd启动时会读取己经存在的数据,仍然用旧的memberID会造成无法加入集群,所以一定要清空新节点的data-dir。
#删除的节点的主机
$ rm -rf /var/lib/etcd/etcd3
在目标节点上启动新增加的成员
修改配置文件中ETCD_INITIAL_CLUSTER_STATE
标记为existing
,如果为new,则会自动生成一个新的memberID,这和前面添加节点时生成的ID不一致,故日志中会报节点ID不匹配的错。
[root@slave2 ~]# cat /etc/etcd/etcd.conf |grep -v "^#" [Member] ETCD_DATA_DIR="/var/lib/etcd/etcd3" ETCD_LISTEN_PEER_URLS="http://192.168.132.133:2380" ETCD_LISTEN_CLIENT_URLS="http://192.168.132.133:2379,http://127.0.0.1:2379" ETCD_NAME="etcd3" [Clustering] ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.132.133:2380" ETCD_ADVERTISE_CLIENT_URLS="http://192.168.132.133:2379" ETCD_INITIAL_CLUSTER_STATE="existing" ETCD_INITIAL_CLUSTER="etcd1=http://192.168.132.131:2380,etcd2=http://192.168.132.132:2380,etcd3=http://192.168.132.133:2380"
重启服务,再次查看
[root@slave2 ~]# systemctl restart etcd [root@slave2 ~]# etcdctl --endpoints "http://192.168.132.132:2379" member list 7a6809311b4a3579: name=etcd3 peerURLs=http://192.168.132.133:2380 clientURLs=http://192.168.132.133:2379 isLeader=false dde447f371b55f50: name=etcd1 peerURLs=http://192.168.132.131:2380 clientURLs=http://192.168.132.131:2379 isLeader=true e96057c7f8ba5f4b: name=etcd2 peerURLs=http://192.168.132.132:2380 clientURLs=http://192.168.132.132:2379 isLeader=false [root@slave2 ~]# etcdctl member list 7a6809311b4a3579: name=etcd3 peerURLs=http://192.168.132.133:2380 clientURLs=http://192.168.132.133:2379 isLeader=false dde447f371b55f50: name=etcd1 peerURLs=http://192.168.132.131:2380 clientURLs=http://192.168.132.131:2379 isLeader=true e96057c7f8ba5f4b: name=etcd2 peerURLs=http://192.168.132.132:2380 clientURLs=http://192.168.132.132:2379 isLeader=false