BCLinux8U6系统部署oceanbase分布式数据库社区版之二、数据库服务器准备

本文是在完成步骤一、准备 OBD 中控机后的第二步，准备3台oceanbase分布式数据库服务器。

前序步骤：BCLinux8U6系统部署oceanbase分布式数据库社区版之一、准备 OBD 中控机

一、服务器配置

1、服务器硬件配置

本例采用vmware虚拟机来构建测试平台，共3台虚拟机，16核CPU、32GB内存,1T数据盘

2、服务器操作系统

采用BCLinux8U6

[root@localhost ~]# cat /etc/os-release 
NAME="BigCloud Enterprise Linux"
VERSION="8.6 (Core)"
ID="bclinux"
ID_LIKE="rhel fedora"
VERSION_ID="8.6"
PLATFORM_ID="platform:an8"
PRETTY_NAME="BigCloud Enterprise Linux 8.6 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:bclinux:bclinux:8"
HOME_URL="https://mirrors.bclinux.org/"
BUG_REPORT_URL="https://bugs.bclinux.org/"BCLINUX_BUGZILLA_PRODUCT="BigCloud Enterprise Linux 8 (Core)"
BCLINUX_BUGZILLA_PRODUCT_VERSION=8.6
BCLINUX_SUPPORT_PRODUCT="BigCloud Enterprise Linux 8 (Core)"
BCLINUX_SUPPORT_PRODUCT_VERSION=8.6

3、系统内核，操作系统内核为5.10

[root@localhost ~]# uname -r
5.10.134-12.2.el8.bclinux.x86_64
[root@localhost ~]# free -gtotal        used        free      shared  buff/cache   available
Mem:             31           0          30           0           0          30
Swap:             1           0           1

二、服务器系统环境初始化（每台服务器需完成相同设置）

1、准备数据盘分区

[root@localhost ~]# fdisk -l |grep /dev/sdb
Disk /dev/sdb：1000 GiB，1073741824000 字节，2097152000 个扇区
[root@localhost ~]# fdisk /dev/sdb欢迎使用 fdisk (util-linux 2.32.1)。
更改将停留在内存中，直到您决定将更改写入磁盘。
使用写入命令前请三思。设备不包含可识别的分区表。
创建了一个磁盘标识符为 0x9febb181 的新 DOS 磁盘标签。命令(输入 m 获取帮助)：n
分区类型p   主分区 (0个主分区，0个扩展分区，4空闲)e   扩展分区 (逻辑分区容器)
选择 (默认 p)：p
分区号 (1-4, 默认  1): 
第一个扇区 (2048-2097151999, 默认 2048): 
上个扇区，+sectors 或 +size{K,M,G,T,P} (2048-2097151999, 默认 2097151999): 创建了一个新分区 1，类型为“Linux”，大小为 1000 GiB。命令(输入 m 获取帮助)：w
分区表已调整。
将调用 ioctl() 来重新读分区表。
正在同步磁盘。[root@localhost ~]# fdisk -l |grep /dev/sdb
Disk /dev/sdb：1000 GiB，1073741824000 字节，2097152000 个扇区
/dev/sdb1        2048 2097151999 2097149952 1000G 83 Linux
[root@localhost ~]# pvcreate /dev/sdb1Physical volume "/dev/sdb1" successfully created.
[root@localhost ~]# vgcreate appvg /dev/sdb1Volume group "appvg" successfully created
[root@localhost ~]# lvcreate -n applv -l 100%vg /dev/appvgLogical volume "applv" created.
[root@localhost ~]# lvs |grep applvapplv appvg -wi-a----- <1000.00g                                                    
[root@localhost ~]# mkfs.xfs -m bigtime=1 /dev/appvg/applv
meta-data=/dev/appvg/applv       isize=512    agcount=4, agsize=65535744 blks=                       sectsz=512   attr=2, projid32bit=1=                       crc=1        finobt=1, sparse=1, rmapbt=0=                       reflink=1    bigtime=1 inobtcount=0
data     =                       bsize=4096   blocks=262142976, imaxpct=25=                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=127999, version=2=                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
[root@localhost ~]# vi /etc/fstab
[root@localhost ~]# grep applv /etc/fstab
/dev/appvg/applv        /app                    xfs     defaults        0 0
[root@localhost ~]# mkdir /app
[root@localhost ~]# mount -a
[root@localhost ~]# df -hT |grep applv
/dev/mapper/appvg-applv xfs      1000G  7.1G  993G    1% /app

2、检查时间同步

因是分布式环境，时间同步非常重要，否则可能会出现各种意想不到的问题

[root@localhost ~]# chronyc sources -v.-- Source mode  '^' = server, '=' = peer, '#' = local clock./ .- Source state '*' = current best, '+' = combined, '-' = not combined,
| /             'x' = may be in error, '~' = too variable, '?' = unusable.
||                                                 .- xxxx [ yyyy ] +/- zzzz
||      Reachability register (octal) -.           |  xxxx = adjusted offset,
||      Log2(Polling interval) --.      |          |  yyyy = measured offset,
||                                \     |          |  zzzz = estimated error.
||                                 |    |           \
MS Name/IP address         Stratum Poll Reach LastRx Last sample               
===============================================================================
^* 192.168.2.33                  2   6   377    51  -1808us[ -325us] +/-   27ms
^+ 192.168.2.34                  2   6   377    53  +1443us[+2925us] +/-   42ms
^+ 192.168.2.35                  2   6   377    50    +15ms[  +15ms] +/-   50ms

3、系统限制参数设置

[root@localhost ~]# vi /etc/security/limits.conf 
[root@localhost ~]# egrep -v "^#|^$" /etc/security/limits.conf 
root soft nofile 655350
root hard nofile 655350
* soft nofile 655350
* hard nofile 655350
* soft stack unlimited
* hard stack unlimited
* soft nproc 655360
* hard nproc 655360
* soft core unlimited
* hard core unlimited

4、系统控制参数配置

[root@localhost ~]# vi /etc/sysctl.conf
[root@localhost ~]# cat /etc/sysctl.conf
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1# for oceanbase
## 修改内核异步 I/O 限制
fs.aio-max-nr=1048576## 网络优化
net.core.somaxconn = 2048
net.core.netdev_max_backlog = 10000 
net.core.rmem_default = 16777216 
net.core.wmem_default = 16777216 
net.core.rmem_max = 16777216 
net.core.wmem_max = 16777216net.ipv4.ip_local_port_range = 3500 65535 
net.ipv4.ip_forward = 0 
net.ipv4.conf.default.rp_filter = 1 
net.ipv4.conf.default.accept_source_route = 0 
net.ipv4.tcp_syncookies = 1 
net.ipv4.tcp_rmem = 4096 87380 16777216 
net.ipv4.tcp_wmem = 4096 65536 16777216 
net.ipv4.tcp_max_syn_backlog = 16384 
net.ipv4.tcp_fin_timeout = 15 
net.ipv4.tcp_max_syn_backlog = 16384 
net.ipv4.tcp_tw_reuse = 1 
net.ipv4.tcp_tw_recycle = 1 
net.ipv4.tcp_slow_start_after_idle=0vm.swappiness = 0
vm.min_free_kbytes = 2097152
fs.file-max = 6573688# 修改进程可以拥有的虚拟内存区域数量
vm.max_map_count = 655360# 此处为 OceanBase 数据库的 data 目录
kernel.core_pattern = /app/core-%e-%p-%t
[root@localhost ~]# sysctl -p
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
fs.aio-max-nr = 1048576
net.core.somaxconn = 2048
net.core.netdev_max_backlog = 10000
net.core.rmem_default = 16777216
net.core.wmem_default = 16777216
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.ip_local_port_range = 3500 65535
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_tw_reuse = 1
sysctl: cannot stat /proc/sys/net/ipv4/tcp_tw_recycle: 没有那个文件或目录
net.ipv4.tcp_slow_start_after_idle = 0
vm.swappiness = 0
vm.min_free_kbytes = 2097152
fs.file-max = 6573688
vm.max_map_count = 655360
kernel.core_pattern = /app/core-%e-%p-%t
[root@localhost ~]#

5、停用selinux服务

[root@localhost ~]# getenforce
Disabled
[root@localhost ~]# cat /etc/selinux/config # This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

6、创建admin管理用户并初始化密码

[root@localhost ~]# useradd -U admin -d /home/admin -s /bin/bash -m
[root@localhost ~]# ll /home
总用量 8
drwx------. 2 AAAA  AAAA  4096 4月  10 09:40 AAAA
drwx------  2 admin admin 4096 4月  15 16:08 admin
[root@localhost ~]# ll /home |grep admin
drwx------  2 admin admin 4096 4月  15 16:08 admin
[root@localhost ~]# passwd admin
更改用户 admin 的密码 。
新的 密码：
重新输入新的 密码：
passwd：所有的身份验证令牌已经成功更新。

7、admin用户sudo免密设置及加入wheel组

配置admin用户的sudo免密设置，为避免其它安全策略影响，建议加到/etc/sudoers文件的最后面；同时将admin用户加入wheel组。

[root@localhost ~]# vim /etc/sudoers
[root@localhost ~]# grep ^admin /etc/sudoers
admin   ALL=(ALL)       NOPASSWD:ALL
[root@localhost ~]# vi /etc/group
[root@localhost ~]# grep wheel /etc/group
wheel:x:10:admin

8、准备数据库的数据目录和日志目录，有条件的话，可以按部署建议进行分盘部署

[root@localhost ~]# mkdir -p /app/oceanbase/data /app/oceanbase/redo
[root@localhost ~]# chown -R admin:admin /app
[root@localhost ~]# tree /app          
/app
└── oceanbase├── data└── redo3 directories, 0 files
[root@localhost ~]#

三、中控机ssh免密操作

以下皆在中控机上操作

1、生成admin用户的公钥文件

[root@localhost ~]# su - admin
上一次登录：一 4月 15 16:45:26 CST 2024pts/1 上
[admin@localhost ~]$ ls ~/.ssh/id_rsa.pub
ls: 无法访问'/home/admin/.ssh/id_rsa.pub': 没有那个文件或目录
[admin@localhost ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/admin/.ssh/id_rsa): 
Created directory '/home/admin/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/admin/.ssh/id_rsa
Your public key has been saved in /home/admin/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:dioymcUobYwt9nUpt5OWuqhCiS+KQ7sVpzUIKHhBEuQ admin@localhost.localdomain
The key's randomart image is:
+---[RSA 3072]----+
|++o              |
|+. .             |
|+E.              |
|...=.o   .       |
|. *o*+= S .      |
|.= ===.= *       |
|o..o* . B        |
|=.o  + + .       |
|+=o.. o.         |
+----[SHA256]-----+

2、分发公钥文件到各数据库服务器

[admin@localhost ~]$ ssh-copy-id -i ~/.ssh/id_rsa.pub -p4422 admin@192.168.2.161 
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/admin/.ssh/id_rsa.pub"
The authenticity of host '[192.168.2.161]:4422 ([192.168.2.161]:4422)' can't be established.
ED25519 key fingerprint is SHA256:wySYs++V/r0GwbmVUSZpvBNFej8nrE2ptlPWLPzyVEk.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
Authorized users only. All activities may be monitored and reported.
admin@192.168.2.161's password: Number of key(s) added: 1Now try logging into the machine, with:   "ssh -p 4422 'admin@192.168.2.161'"
and check to make sure that only the key(s) you wanted were added.[admin@localhost ~]$

3、免密登录各数据库服务器测试

[admin@localhost ~]$ ssh -p 4422 'admin@192.168.2.161'
Authorized users only. All activities may be monitored and reported.
Authorized users only. All activities may be monitored and reported.
[admin@localhost ~]$ sudo su____  _        ____ _                 _
| __ )(_) __ _ / ___| | ___  _   _  __| |
|  _ \| |/ _` | |   | |/ _ \| | | |/ _` |
| |_) | | (_| | |___| | (_) | |_| | (_| |
|____/|_|\__, |\____|_|\___/ \__,_|\__,_||___/_____       _                       _            _     _
| ____|_ __ | |_ ___ _ __ _ __  _ __(_)___  ___  | |   (_)_ __  _   ___  __
|  _| | '_ \| __/ _ \ '__| '_ \| '__| / __|/ _ \ | |   | | '_ \| | | \ \/ /
| |___| | | | ||  __/ |  | |_) | |  | \__ \  __/ | |___| | | | | |_| |>  <
|_____|_| |_|\__\___|_|  | .__/|_|  |_|___/\___| |_____|_|_| |_|\__,_/_/\_\|_|___( _ )/ _ \
| (_) |\___/Welcome to BigCloud Enterprise Linux 8 (GNU/Linux 5.10.134-12.2.el8.bclinux.x86_64 x86_64)System information as of 2024年 04月 15日 星期一 16:55:05 CST* System CPU load:    0.00 0.00 0.00        * System   uptime:         16:55:05 up  1:18  * Active sessions:    2                     * Memory    usage:         32185 / 32819 MB* Processes count:    306* Contact US   :MAIL : support@bclinux.org / TEL : 4001-10086-5[root@localhost admin]# exit
exit
[admin@localhost ~]$ exit
注销
Connection to 192.168.2.161 closed.
[admin@localhost ~]$