# MySQL高可用架构集群环境搭建手册
## 环境准备
### 机器规划
| 节点 | IP | 配置 | 角色 |
| -------- | -------------- | ---- | ---------------- |
| master-1 | 192.168.73.129 | 1c2g | master、HMA node |
| slave-1 | 192.168.73.130 | 1c1g | salve、HMA node |
| slave-2 | 192.168.73.131 | 1c1g | salve、HMA node |
| slave-3 | 192.168.73.132 | 1c1g | HMA manager |
### 安装包
mysql-5.7.28-1.el7.x86_64.rpm-bundle.tar
拷贝到master和salve节点的/opt/software/mysql/目录下并解压
```shell
tar -xvf mysql-5.7.28-1.el7.x86_64.rpm-bundle.tar
```
### 安装Mysql
在192.168.73.129、192.168.73.130、192.168.73.131分别按如下步骤命令安装Mysql。
1)卸载自带的mariadb
```shell
rpm -qa|grep mariadb
rpm -e mariadb-libs-5.5.64-1.el7.x86_64 --nodeps
```
2)安装Mysql
```shell
rpm -ivh mysql-community-common-5.7.28-1.el7.x86_64.rpm
rpm -ivh mysql-community-libs-5.7.28-1.el7.x86_64.rpm
rpm -ivh mysql-community-libs-compat-5.7.28-1.el7.x86_64.rpm
rpm -ivh mysql-community-client-5.7.28-1.el7.x86_64.rpm
rpm -ivh mysql-community-server-5.7.28-1.el7.x86_64.rpm
rpm -ivh mysql-community-devel-5.7.28-1.el7.x86_64.rpm
```
3)初始化mysql
```shell
mysqld --initialize --user=mysql
```
4)查看临时密码
```shell
[root@localhost mysql]# cat /var/log/mysqld.log
2020-07-27T11:35:17.156086Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2020-07-27T11:35:17.291065Z 0 [Warning] InnoDB: New log files created, LSN=45790
2020-07-27T11:35:17.321303Z 0 [Warning] InnoDB: Creating foreign key constraint system tables.
2020-07-27T11:35:17.389391Z 0 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 3eb43224-cffd-11ea-b6e3-000c2998582e.
2020-07-27T11:35:17.390659Z 0 [Warning] Gtid table is not ready to be used. Table 'mysql.gtid_executed' cannot be opened.
2020-07-27T11:35:17.797306Z 0 [Warning] CA certificate ca.pem is self signed.
2020-07-27T11:35:17.916216Z 1 [Note] A temporary password is generated for root@localhost: =L=9m4yg29kA
```
5)启动mysql，并设置开机启动
```shell
[root@localhost mysql]# systemctl start mysqld.service
[root@localhost mysql]# systemctl status mysqld.service
● mysqld.service - MySQL Server
Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
Active: active (running) since 一 2020-07-27 19:37:41 CST; 5s ago
Docs: man:mysqld(8)
http://dev.mysql.com/doc/refman/en/using-systemd.html
Process: 2328 ExecStart=/usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid $MYSQLD_OPTS (code=exited, status=0/SUCCESS)
Process: 2311 ExecStartPre=/usr/bin/mysqld_pre_systemd (code=exited, status=0/SUCCESS)
Main PID: 2331 (mysqld)
CGroup: /system.slice/mysqld.service
└─2331 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid
7月 27 19:37:41 localhost.localdomain systemd[1]: Starting MySQL Server...
7月 27 19:37:41 localhost.localdomain systemd[1]: Started MySQL Server.
```
6)登录mysql，修改密码
```shell
[root@localhost mysql]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.28
Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> set password=password('root');
Query OK, 0 rows affected, 1 warning (0.00 sec)
```
7)关闭防火墙
```shell
[root@localhost mysql]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: active (running) since 日 2020-07-26 23:36:30 CST; 20h ago
Docs: man:firewalld(1)
Main PID: 775 (firewalld)
CGroup: /system.slice/firewalld.service
└─775 /usr/bin/python2 -Es /usr/sbin/firewalld --nofork --nopid
7月 26 23:36:28 localhost.localdomain systemd[1]: Starting firewalld - dynamic firewall daemon...
7月 26 23:36:30 localhost.localdomain systemd[1]: Started firewalld - dynamic firewall daemon.
[root@localhost mysql]# systemctl stop firewalld
[root@localhost mysql]# systemctl disable firewalld.service
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@localhost mysql]# systemctl status iptables
Unit iptables.service could not be found.
```
没有iptables就不用禁用了。
## 主从配置
### Master-1配置
1)修改mysql配置文件，默认在/etc/my.cnf
在[mysqld]下添加如下配置
```ini
# 开启binlog并指定日志文件的文件名，主从复制必须开启
log_bin=mysql-master-bin
# 为master节点设置id，集群之间各个节点保持唯一
server-id=1
# 每次数据写入都同步到磁盘
sync-binlog=1
# 设置mysql自带的数据库不进行同步
binlog-ignore-db=information_schema
binlog-ignore-db=performance_schema
binlog-ignore-db=sys
# 指定哪些同步哪些数据库
# binlog-do-db=xxx
# 双主模式下开启relay_log，设置中继日志名称
relay_log=mysql-master-relay
# 当前master执行更新操作时，写入日志文件
log_slave_updates=1
```
2)重启mysql
```shell
systemctl restart mysqld
```
3)登录mysql进行授权
```shell
# 授权任意节点都可以进行数据复制
mysql> grant replication slave on *.* to 'root'@'%' identified by 'root';
Query OK, 0 rows affected, 1 warning (0.00 sec)
# 授权任意节点可以连接mysql
mysql> grant all privileges on *.* to 'root'@'%' identified by 'root';
Query OK, 0 rows affected, 1 warning (0.00 sec)
# 刷新权限
mysql> flush privileges;
Query OK, 0 rows affected (0.01 sec)
```
4)验证，查看主库状态
```shell
mysql> show master status;
+-------------------------+----------+--------------+-------------------------------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+-------------------------+----------+--------------+-------------------------------------------+-------------------+
| mysql-master-bin.000001 | 869 | | information_schema,performance_schema,sys | |
+-------------------------+----------+--------------+-------------------------------------------+-------------------+
1 row in set (0.00 sec)
```
从库进行同步时，需要指定File、Position。
### Slave-1配置
1)修改配置文件
slave-1作为备选master，在[mysqld]下添加如下配置
```ini
# 设置slave-1的id
server-id=2
# 开启binlog并指定日志文件的文件名，主从复制必须开启
log_bin=mysql-master-bin
# 每次数据写入都同步到磁盘
sync-binlog=1
# 设置mysql自带的数据库不进行同步
binlog-ignore-db=information_schema
binlog-ignore-db=performance_schema
binlog-ignore-db=sys
# 设置中继日志的文件名
relay_log=mysql-slave-1-relay
# 当前master执行更新操作时，写入日志文件
log_slave_updates=1
# 设置当前节点为只读，即双主单写模式
read_only=1
```
2)重启mysql
```shell
systemctl restart mysqld
```
3)登录mysql，查看从库状态
```shell
mysql> show ma status;
Empty set (0.00 sec)
```
初始状态下`show slave status`会显示`Empty set`，如果能显示信息说明已经进行过主从配置。
4)对备选主进行初始化
```shell
mysql> change master to master_host='192.168.73.129',master_port=3306,master_user='root',master_password='root',master_log_file='mysql-master-bin.000001',master_log_pos=869;
Query OK, 0 rows affected, 2 warnings (0.01 sec)
```
5)因为salve-1是备选主，需要授权master-1进行数据同步
```shell
grant replication slave on *.* to 'root'@'%' identified by 'root';
grant all privileges on *.* to 'root'@'%' identified by 'root';
flush privileges;
```
6)查看备选主状态
```shell
mysql> show master status;
+--------------------------+----------+--------------+-------------------------------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+--------------------------+----------+--------------+-------------------------------------------+-------------------+
| mysql-slave-1-bin.000001 | 589 | | information_schema,performance_schema,sys | |
+--------------------------+----------+--------------+-------------------------------------------+-------------------+
```
然后将master1的主节点配置成备选主信息，**登录master1然后执行如下命令**：
```shell
change master to master_host='192.168.73.130',master_port=3306,master_user='root',master_password='root',master_log_file='mysql-slave-1-bin.000001',master_log_pos=589;
```
### Slave-2配置
1)修改配置文件
```shell
# 设置slave-2的id
server-id=3
# 设置中继日志的文件名
relay_log=mysql-slave-2-relay
```
2)剩下步骤参照slave-1。
## 集群半同步配置
### master配置
1)查看是否支持插件动态加载
```shell
mysql> select @@have_dynamic_loading;
+------------------------+
| @@have_dynamic_loading |
+------------------------+
| YES |
+------------------------+
1 row in set (0.00 sec)
```
> show plugins;可以显示mysql已加载的插件
2)安装semi插件
```shell
mysql> install plugin rpl_semi_sync_master soname 'semisync_master.so';
Query OK, 0 rows affected (0.03 sec)
```
rpl_semi_sync_master：插件名
soname：指定插件别名，参照mariadb命名
3)semi配置
```shell
mysql> show variables like '%semi%';
+-------------------------------------------+------------+
| Variable_name | Value |
+-------------------------------------------+------------+
| rpl_semi_sync_master_enabled | OFF |
| rpl_semi_sync_master_timeout | 10000 |
| rpl_semi_sync_master_trace_level | 32 |
| rpl_semi_sync_master_wait_for_slave_count | 1 |
| rpl_semi_sync_master_wait_no_slave | ON |
| rpl_semi_sync_master_wait_point | AFTER_SYNC |
+-------------------------------------------+------------+
6 rows in set (0.02 sec)
```
通过set命令修改以下配置：
```shell
# 开启semi
set global rpl_semi_sync_master_enabled=1;
# 设置超时时间为1s
set global rpl_semi_sync_master_timeout=1000;
```
### slave配置
1)安装semi
```shell
install plugin rpl_semi_sync_slave soname 'semisync_slave.so';
```
2)semi配置
```shell
mysql> show variables like '%semi%';
+---------------------------------+-------+
| Variable_name | Value |
+---------------------------------+-------+
| rpl_semi_sync_slave_enabled | OFF |
| rpl_semi_sync_slave_trace_level | 32 |
+---------------------------------+-------+
2 rows in set (0.02 sec)
```
通过set命令修改以下配置：
```shell
set global rpl_semi_sync_slave_enabled=1;
```
3)重启slave
```shell
stop slave;
start slave;
```
## MHA配置
1)准备
mha-node安装包：mha4mysql-node-0.58-0.el7.centos.noarch.rpm
mha-manager安装包：mha4mysql-manager-0.58-0.el7.centos.noarch.rpm
2)在192.168.73.129、192.168.73.130、192.168.73.131三台机器上安装mha-node
```shell
rpm -ivh mha4mysql-node-0.58-0.el7.centos.noarch.rpm
```
3)在192.168.73.132上安装mha-manager和mha-node
```shell
rpm -ivh mha4mysql-node-0.58-0.el7.centos.noarch.rpm
rpm -ivh mha4mysql-manager-0.58-0.el7.centos.noarch.rpm
```
4)配置manager节点和node节点之间的ssh通信
```shell
# 在所有节点都执行以下命令
ssh-keygen -t rsa
ssh-copy-id -i /root/.ssh/id_rsa.pub root@192.168.73.129
ssh-copy-id -i /root/.ssh/id_rsa.pub root@192.168.73.130
ssh-copy-id -i /root/.ssh/id_rsa.pub root@192.168.73.131
ssh-copy-id -i /root/.ssh/id_rsa.pub root@192.168.73.132
```
5)配置manager
分别创建以下目录：
```shell
mkdir /opt/software/mysql/hma
mkdir /opt/software/mysql/hma/app
mkdir /etc/mysql/mha
```
编写manager主配置文件
vi /etc/mysql/mha/app.cnf
```ini
[server default]
#mysql用户
user=root
#mysql密码
password=root
# manager工作目录
manager_workdir=/opt/software/mysql/hma/app
# manager日志文件存放位置
manager_log=/opt/software/mysql/hma/app/manager.log
# ssh通信用户名
ssh_user=root
# ？
repl_user=root
repl_password=root
# 心跳检测
ping_interval=1
[server1]
hostname=192.168.73.129
port=3306
candidate_master=1
[server2]
hostname=192.168.73.130
port=3306
candidate_master=1
[server3]
hostname=192.168.73.131
port=3306
no_master=1
```
6)运行manager
- 检查SSH配置
```shell
[root@localhost hma]# masterha_check_ssh --conf=/etc/mysql/hma/app.cnf
```
- 检查MHA当前配置
```shell
masterha_check_repl --conf=/etc/mysql/hma/app.cnf
```
- 启动mha
```shell
nohup masterha_manager --conf=/etc/mysql/hma/app.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null > /opt/software/mysql/hma/app/manager.log 2>&1 &
```
- 查看状态
```shell
masterha_check_status --conf=/etc/mysql/hma/app.cnf
```
- 查看日志
```shell
tail -f /opt/software/mysql/hma/app/manager.log
```
## 问题记录
1.安装mysql-server时失败
```shell
[root@localhost mysql]# rpm -ivh mysql-community-server-5.7.28-1.el7.x86_64.rpm
警告：mysql-community-server-5.7.28-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
错误：依赖检测失败：
/usr/bin/perl 被 mysql-community-server-5.7.28-1.el7.x86_64 需要
net-tools 被 mysql-community-server-5.7.28-1.el7.x86_64 需要
perl(Getopt::Long) 被 mysql-community-server-5.7.28-1.el7.x86_64 需要
perl(strict) 被 mysql-community-server-5.7.28-1.el7.x86_64 需要
```
解决：
```shell
yum -y install perl
yum -y install net-tools
```
2.安装mha-node失败
```shell
[root@localhost mysql]# rpm -ivh mha4mysql-node-0.58-0.el7.centos.noarch.rpm
错误：依赖检测失败：
perl(DBD::mysql) 被 mha4mysql-node-0.58-0.el7.centos.noarch 需要
perl(DBI) 被 mha4mysql-node-0.58-0.el7.centos.noarch 需要
```
解决：
```shell
yum install perl-DBD-MySQL
```
3.安装mha-manager失败
```shell
[root@localhost mysql]# rpm -ivh mha4mysql-manager-0.58-0.el7.centos.noarch.rpm
错误：依赖检测失败：
mha4mysql-node >= 0.54 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(Carp) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(Config::Tiny) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(DBD::mysql) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(DBI) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(Data::Dumper) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(English) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(Errno) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(File::Basename) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(File::Copy) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(Getopt::Long) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(IO::File) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(Log::Dispatch) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(Log::Dispatch::File) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(Log::Dispatch::Screen) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(MHA::BinlogManager) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(MHA::NodeConst) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(MHA::NodeUtil) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(MHA::SlaveUtil) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(POSIX) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(Parallel::ForkManager) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(Pod::Usage) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(Sys::Hostname) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(Time::HiRes) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(constant) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(strict) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
perl(warnings) 被 mha4mysql-manager-0.58-0.el7.centos.noarch 需要
```
解决：
```shell
yum -y install wget
wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
rpm -ivh epel-release-latest-7.noarch.rpm
yum -y install perl-DBD-MySQL perl-Config-Tiny perl-Log-Dispatch perl-Parallel-ForkManager
```
4.检测配置失败
```shell
[root@localhost hma]# masterha_check_ssh --conf=/etc/mysql/hma/app.cnf
# .....
Mon Jul 27 21:52:38 2020 - [error][/usr/share/perl5/vendor_perl/MHA/Server.pm, ln180] Got MySQL error when connecting 192.168.73.130(192.168.73.130:3306) :1130:Host '192.168.73.132' is not allowed to connect to this MySQL server, but this is not a MySQL crash. Check MySQL server settings.
Mon Jul 27 21:52:38 2020 - [error][/usr/share/perl5/vendor_perl/MHA/ServerManager.pm, ln301] at /usr/share/perl5/vendor_perl/MHA/ServerManager.pm line 297.
Mon Jul 27 21:52:38 2020 - [error][/usr/share/perl5/vendor_perl/MHA/Server.pm, ln180] Got MySQL error when connecting 192.168.73.131(192.168.73.131:3306) :1130:Host '192.168.73.132' is not allowed to connect to this MySQL server, but this is not a MySQL crash. Check MySQL server settings.
Mon Jul 27 21:52:38 2020 - [error][/usr/share/perl5/vendor_perl/MHA/ServerManager.pm, ln301] at /usr/share/perl5/vendor_perl/MHA/ServerManager.pm line 297.
Mon Jul 27 21:52:38 2020 - [error][/usr/share/perl5/vendor_perl/MHA/ServerManager.pm, ln309] Got fatal error, stopping operations
Mon Jul 27 21:52:38 2020 - [error][/usr/share/perl5/vendor_perl/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. at /usr/share/perl5/vendor_perl/MHA/MasterMonitor.pm line 329.
Mon Jul 27 21:52:38 2020 - [error][/usr/share/perl5/vendor_perl/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Mon Jul 27 21:52:38 2020 - [info] Got exit code 1 (Not master dead).
```
修改slave-1和slave-2的权限
```shell
grant all privileges on *.* to 'root'@'%' identified by 'root';
flush privileges;
```
再次检测仍然报错：
```shell
Mon Jul 27 21:54:12 2020 - [info] Checking slave configurations..
Mon Jul 27 21:54:12 2020 - [info] read_only=1 is not set on slave 192.168.73.130(192.168.73.130:3306).
Mon Jul 27 21:54:12 2020 - [warning] relay_log_purge=0 is not set on slave 192.168.73.130(192.168.73.130:3306).
Mon Jul 27 21:54:12 2020 - [warning] log-bin is not set on slave 192.168.73.130(192.168.73.130:3306). This host cannot be a master.
Mon Jul 27 21:54:12 2020 - [info] read_only=1 is not set on slave 192.168.73.131(192.168.73.131:3306).
Mon Jul 27 21:54:12 2020 - [warning] relay_log_purge=0 is not set on slave 192.168.73.131(192.168.73.131:3306).
Mon Jul 27 21:54:12 2020 - [warning] log-bin is not set on slave 192.168.73.131(192.168.73.131:3306). This host cannot be a master.
Mon Jul 27 21:54:12 2020 - [info] Checking replication filtering settings..
Mon Jul 27 21:54:12 2020 - [info] binlog_do_db= , binlog_ignore_db= information_schema,performance_schema,sys
Mon Jul 27 21:54:12 2020 - [info] Replication filtering check ok.
Mon Jul 27 21:54:12 2020 - [error][/usr/share/perl5/vendor_perl/MHA/MasterMonitor.pm, ln364] None of slaves can be master. Check failover configuration file or log-bin settings in my.cnf
Mon Jul 27 21:54:12 2020 - [error][/usr/share/perl5/vendor_perl/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. at /usr/bin/masterha_check_repl line 48.
Mon Jul 27 21:54:12 2020 - [error][/usr/share/perl5/vendor_perl/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Mon Jul 27 21:54:12 2020 - [info] Got exit code 1 (Not master dead).
```
报错信息显示，当前环境时一台master和两台slave，没有备选的master，需要讲其中一个slave改造成备选master。参照课程Mysql集群双主实战进行修改。
一键复制
编辑
Web IDE
原始数据
按行查看
历史
