ClouderaManager agent 报错，无法连接到结群 Error, CM server guid updated, expected xxx , received xxx

场景

集群版本：CDH 6.0.1

服务器版本：Centos 7.3

JAVA版本：1.8

异常出现原因

搭建集群中遇到异常，删除agent节点，重新部署CM后发现异常，CDH处于无法使用状态

异常

查看agent日志会出现,说无法连接到服务端7182端口，排查防火墙等问题，均正常

Heartbeating to xxxx:7182 failed.

继续往下看日志，提示我们获取的guid不同

[15/Oct/2019 14:46:11 +0000] 4516 MainThread agent        ERROR    Error, CM server guid updated, expected 8115403a-7f9c-463f-b6d4-45fed899215b, received 87bf3025-6565-4a83-851b-81cdcaa936b6
[15/Oct/2019 14:46:16 +0000] 4516 MainThread agent        ERROR    Error, CM server guid updated, expected 8115403a-7f9c-463f-b6d4-45fed899215b, received 87bf3025-6565-4a83-851b-81cdcaa936b6
[15/Oct/2019 14:46:21 +0000] 4516 MainThread agent        ERROR    Error, CM server guid updated, expected 8115403a-7f9c-463f-b6d4-45fed899215b, received 87bf3025-6565-4a83-851b-81cdcaa936b6
[15/Oct/2019 14:46:26 +0000] 4516 MainThread agent        ERROR    Error, CM server guid updated, expected 8115403a-7f9c-463f-b6d4-45fed899215b, received 87bf3025-6565-4a83-851b-81cdcaa936b6
[15/Oct/2019 14:46:31 +0000] 4516 MainThread agent        ERROR    Error, CM server guid updated, expected 8115403a-7f9c-463f-b6d4-45fed899215b, received 87bf3025-6565-4a83-851b-81cdcaa936b6
[15/Oct/2019 14:46:36 +0000] 4516 MainThread agent        ERROR    Error, CM server guid updated, expected 8115403a-7f9c-463f-b6d4-45fed899215b, received 87bf3025-6565-4a83-851b-81cdcaa936b6
[15/Oct/2019 14:46:41 +0000] 4516 MainThread agent        ERROR    Error, CM server guid updated, expected 8115403a-7f9c-463f-b6d4-45fed899215b, received 87bf3025-6565-4a83-851b-81cdcaa936b6
[15/Oct/2019 14:46:46 +0000] 4516 MainThread agent        ERROR    Error, CM server guid updated, expected 8115403a-7f9c-463f-b6d4-45fed899215b, received 87bf3025-6565-4a83-851b-81cdcaa936b6
[15/Oct/2019 14:46:51 +0000] 4516 MainThread agent        ERROR    Error, CM server guid updated, expected 8115403a-7f9c-463f-b6d4-45fed899215b, received 87bf3025-6565-4a83-851b-81cdcaa936b6
[15/Oct/2019 14:46:56 +0000] 4516 MainThread agent        ERROR    Error, CM server guid updated, expected 8115403a-7f9c-463f-b6d4-45fed899215b, received 87bf3025-6565-4a83-851b-81cdcaa936b6
[15/Oct/2019 14:47:01 +0000] 4516 MainThread agent        ERROR    Error, CM server guid updated, expected 8115403a-7f9c-463f-b6d4-45fed899215b, received 87bf3025-6565-4a83-851b-81cdcaa936b6
[15/Oct/2019 14:47:06 +0000] 4516 MainThread agent        ERROR    Error, CM server guid updated, expected 8115403a-7f9c-463f-b6d4-45fed899215b, received 87bf3025-6565-4a83-851b-81cdcaa936b6

解决方法

1.查找cm的guid存放目录，发现在这个位置 /var/lib/cloudera-scm-agent/cm_guid

[root@xxx cloudera-scm-server]# find / -name *cm_guid*
/var/lib/cloudera-scm-agent/cm_guid

2.查看内容，发现就是上面提示的811xxx，（查看文件时间，是之前搭建的时间）

cat /var/lib/cloudera-scm-agent/cm_guid
8115403a-7f9c-463f-b6d4-45fed899215b

3.停止agent节点后，删除cm_guid，并启动agent节点

systemctl stop cloudera-scm-agent
#这里等几秒，服务停止后再删除
systemctl status cloudera-scm-agent
rm -f /var/lib/cloudera-scm-agent/cm_guid
systemctl start cloudera-scm-agent

4.查看agent日志，一切正常

5.登录CM，分配CDH，一切正常。