使用rke2安装k8s,master节点有三台，agent节点一台，三台master通过etcd存储保证master节点的高可用，使用nginx对master进行负载均衡。

• 主机清单如下

ip	主机名称	用途
192.168.16.72	node72	server节点
192.168.16.73	node73	master节点
192.168.16.74	node74	master节点
192.168.16.75	node75	master节点，nginx

• 规划好主机并修改主机名称
  修改对应主机的名称，如何示例如：
  hostnamectl set-hostname node72
• 所有主机修改hosts

cat > /etc/hosts << EOF
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.16.71 node71
192.168.16.72 node72
192.168.16.73 node73
192.168.16.74 node74
192.168.16.75 node75
EOF

• 所有主机时间同步
  一小时同步一次，大家可以视情况频率高一些

yum install ntpdate -y
​​​​​​​crontab -e
0 */1 * * * root ntpdate -s ntp.aliyun.com
crontab -l

• 所有主机防火墙关闭

systemctl stop firewalld
systemctl disable firewalld
sudo systemctl status firewalld

• 所有主机关闭swap

#永久关闭，需要重启服务器
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
#昨时关闭
swapoff -a && sysctl -w vm.swappiness=0 

• 关闭selinux

#永久关闭
sed -ri '/^[^#]*SELINUX=/s#=.+$#=disabled#' /etc/selinux/config
#临时关闭
setenforce 0
#注意：不关闭会产生文件读取权限等问题出现

• 添加网桥过滤

#添加网桥过滤
cat >>  /etc/sysctl.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-arptables = 1
net.ipv4.ip_forward = 1
EOF
#加载 br_netfilter 模块
modprobe br_netfilter
#是否加载
lsmod | grep br_netfilter
#加载网桥过滤配置文件
sysctl -p&&sysctl -p /etc/sysctl.conf
查看是否添加成功
sysctl -a|grep net.bridge.bridge-nf-call-ip6tables

• ipvs安装

#安装相关软件包
yum -y install ipset ipvsadm 
cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
EOF
chmod +x /etc/sysconfig/modules/ipvs.modules 
bash /etc/sysconfig/modules/ipvs.modules 
lsmod | grep -e ip_vs -e nf_conntrack_ipv4

• rke2下载

下载地址：Releases · rancher/rke2 · GitHub

下载的文件：rke2.linux-amd64.tar.gz、rke2-images.linux-amd64.tar.zst、sha256sum-amd64.txt从上面的地址
下载：wget https://rancher-mirror.rancher.cn/rke2/install.sh

将这些文件下载到服务器 ~/rke2-artifacts 目录下面

• rke2 master config配置 

mkdir -p /etc/rancher/rke2
cat > /etc/rancher/rke2/config.yaml  <<EOF
token:  0fcef8600c960e74d639f08e9abd8a72
system-default-registry: "registry.cn-hangzhou.aliyuncs.com"
data-dir: /data/lib/rke2
#node-name: 不设置取主机名称
write-kubeconfig-mode: 644
cni: "canal" 
kube-proxy-arg:- proxy-mode=ipvs- ipvs-strict-arp=true
EOF

• rke2 master环境变量配置

cat > /etc/profile.d/rke2.sh <<EOF
export PATH=/data/lib/rke2/bin:$PATH
export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
export CRI_CONFIG_FILE=/data/lib/rke2/agent/etc/crictl.yaml
export CONTAINERD_ADDRESS=/run/k3s/containerd/containerd.sock
export CONTAINERD_NAMESPACE=k8s.io
EOF
source /etc/profile

rke2 master registry 配置

mkdir -p /etc/rancher/rke2
cat > /etc/rancher/rke2/registries.yaml<<EOF
mirrors:docker.io:endpoint:- "https://registry.cn-hangzhou.aliyuncs.com"- "https://docker.mirrors.ustc.edu.cn"swr.cn-north-1.*****.com:endpoint:- "https://swr.cn-north-1.****.com"pregistry.bshcn.com.cn:endpoint:- "https://**pregistry.****.com.cn"
configs:"swr.cn-north-1.****.com":auth:username: cn-north-1@****password: ****"pregistry.****.com.cn":tls:insecure_skip_verify: true
EOF

• rke2 master安装

cd /root/rke2-artifacts/
INSTALL_RKE2_ARTIFACT_PATH=/root/rke2-artifacts INSTALL_RKE2_AGENT_IMAGES_DIR=/data/lib/rke2/agent/images sh install.sh

• rke2  master- 设置开机启动

systemctl enable rke2-server
systemctl start rke2-server

• rke2 master2、maseter3安装

除了config.yaml中server地址为master地址外，master2、maseter3其它步骤和master安装方式方法一样

mkdir -p /etc/rancher/rke2
cat > /etc/rancher/rke2/config.yaml  <<EOF
#为master节点地址
server: https://node75:9345   
token:  0fcef8600c960e74d639f08e9abd8a72
system-default-registry: "registry.cn-hangzhou.aliyuncs.com"
data-dir: /data/lib/rke2
#node-name: 不设置取主机名称
write-kubeconfig-mode: 644
cni: "canal" 
kube-proxy-arg:- proxy-mode=ipvs- ipvs-strict-arp=true
EOF

• master节点负载均衡配置
  master有三台，这里通过nginx进行负载均衡配置具体如下：

  stream {upstream rke2_servers {server node73:9345;server node74:9345;server node75:9345;}server {listen 80;proxy_pass rke2_servers;}
  }

• rke2 agent安装说明
  所有agent安装方式一样
   
• rke2 agent config配置

mkdir -p /etc/rancher/rke2
cat > /etc/rancher/rke2/config.yaml  <<EOF
server: https://rke2_servers:9345
token:  0fcef8600c960e74d639f08e9abd8a72
system-default-registry: "registry.cn-hangzhou.aliyuncs.com"
data-dir: /data/lib/rke2
node-name: agent41
write-kubeconfig-mode: 644
cni: "canal" 
kube-proxy-arg:- proxy-mode=ipvs- ipvs-strict-arp=true
EOF

• rke2 agent registry 配置

  mkdir -p /etc/rancher/rke2
  cat > /etc/rancher/rke2/registries.yaml<<EOF
  mirrors:docker.io:endpoint:- "https://registry.cn-hangzhou.aliyuncs.com"- "https://docker.mirrors.ustc.edu.cn"swr.cn-north-1.*****.com:endpoint:- "https://swr.cn-north-1.****.com"pregistry.bshcn.com.cn:endpoint:- "https://**pregistry.****.com.cn"
  configs:"swr.cn-north-1.****.com":auth:username: cn-north-1@****password: ****"pregistry.****.com.cn":tls:insecure_skip_verify: true
  EOF

• rke2 agent 安装

cd /root/rke2-artifacts
INSTALL_RKE2_ARTIFACT_PATH=/root/rke2-artifacts INSTALL_RKE2_AGENT_IMAGES_DIR=/data/lib/rke2/agent/images INSTALL_RKE2_TYPE="agent"  sh install.sh

• rke2 agent启动

systemctl enable rke2-agent.service
systemctl start rke2-agent.service