一、主机准备
主机硬件配置说明
作用 | IP地址 | 操作系统 | 配置 |
---|---|---|---|
k8s-master01 | 192.168.136.55 | openEuler-22.03-LTS-SP1 | 2颗CPU 4G内存 50G硬盘 |
k8s-node01 | 192.168.136.56 | openEuler-22.03-LTS-SP1 | 2颗CPU 4G内存 50G硬盘 |
k8s-node02 | 192.168.136.57 | openEuler-22.03-LTS-SP1 | 2颗CPU 4G内存 50G硬盘 |
1、配置主机名和IP
hostnamectl set-hostname k8s-master01
hostnamectl set-hostname k8s-node01
hostnamectl set-hostname k8s-node02
2、配置hosts解析
# cat >> /etc/hosts << EOF
192.168.136.55 k8s-master01
192.168.136.56 k8s-node01
192.168.136.57 k8s-node02
EOF
3、配置免密登录,只在k8s-master01上操作
[root@k8s-master01 ~]# ssh-keygen -f ~/.ssh/id_rsa -N '' -q# 点拷贝秘钥到其他 2 台节点
[root@k8s-master01 ~]# ssh-copy-id k8s-node01
[root@k8s-master01 ~]# ssh-copy-id k8s-node02
4、防火墙和SELinux
systemctl disable firewalld
systemctl stop firewalld
sed -ri 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
setenforce 0
5、时间同步配置
dnf install -y chrony
systemctl restart chronyd
systemctl enable chronyd
6、配置内核转发及网桥过滤
添加网桥过滤及内核转发配置文件
# cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
vm.swappiness = 0
EOF加载br_netfilter模块
# modprobe br_netfilter查看是否加载
# lsmod | grep br_netfilter
br_netfilter 22256 0
bridge 151336 1 br_netfilter使用新添加配置文件生效
# sysctl -p /etc/sysctl.d/k8s.conf
7、启用ipvs
三台机器
cat > /etc/sysconfig/modules/ipvs.modules <<END
#!/bin/bash
ipvs_modules="ip_vs ip_vs_lc ip_vs_wlc ip_vs_rr ip_vs_wrr ip_vs_lblc ip_vs_lblcr ip_vs_dh ip_vs_sh ip_vs_nq ip_vs_sed ip_vs_ftp nf_conntrack"
for kernel_module in ${ipvs_modules}
do/sbin/modinfo -F filename ${kernel_module} > /dev/null 2>&1if [ 0 -eq 0 ]then/sbin/modprobe ${kernel_module}fi
done
END
chmod 755 /etc/sysconfig/modules/ipvs.modules
bash /etc/sysconfig/modules/ipvs.modules
8、关闭swap
临时关闭
# swapoff -a永远关闭swap分区
sed -i 's/.*swap.*/#&/' /etc/fstab
二、容器运行时工具安装及运行
1、安装docker
1、下载官方repo
cd /etc/yum.repos.d/
curl -O https://download.docker.com/linux/centos/docker-ce.repo
sed -i 's/$releasever/8/g' docker-ce.repo
2. 安装docker
yum install -y docker-ce# 设置国内镜像加速
mkdir -p /etc/docker
tee /etc/docker/daemon.json <<-'EOF'
{"registry-mirrors": ["https://hub-mirror.c.163.com","https://mirror.baidubce.com","https://ccr.ccs.tencentyun.com"]
}
EOF设置docker开机启动并启动
# systemctl enable --now docker查看docker版本
# docker version# 所有节点执行以下命令,将 docker 的 CgroupDriver 改成 systemd,在/etc/docker/daemon.json 中添加配置:{"exec-opts": ["native.cgroupdriver=systemd"],"registry-mirrors": ["https://hub-mirror.c.163.com","https://mirror.baidubce.com","https://ccr.ccs.tencentyun.com"]
}# 所有节点执行以下命令,重启 docker:
systemctl daemon-reload
systemctl restart docker
因为k8s 1.25以后版本默认装的是containerd,所以还要安装cri-dockerd
2、安装cri-dockerd
1. 下载最新版cri-dockerd rpm包
网络条件好的话直接使用wget下载,网络条件一般的话可以在github上面先下载再上传到虚拟机
下载地址:Releases · Mirantis/cri-dockerd (github.com)。
https://github.com/Mirantis/cri-dockerd/releases/download/v0.3.4/cri-dockerd-0.3.4-3.el8.x86_64.rpm
2、安装cri-docker
# wget -c http://github.com/Mirantis/cri-dockerd/releases/download/v0.3.4/cri-dockerd-0.3.4-3.el8.x86_64.rpm
# yum install cri-dockerd-0.3.4-3.el8.x86_64.rpm
3、启动cri-docker服务
systemctl enable cri-docker
4、cri-dockerd设置国内镜像加速
$ vi /usr/lib/systemd/system/cri-docker.service # 找到第10行ExecStart=
# 修改为ExecStart=/usr/bin/cri-dockerd --network-plugin=cni --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.9
# 重启Docker组件
$ systemctl daemon-reload && systemctl restart docker cri-docker.socket cri-docker
# 检查Docker组件状态
$ systemctl status docker cir-docker.socket cri-docker
三、K8S软件安装
1、配置kubernetes源
# 此操作会覆盖 /etc/yum.repos.d/kubernetes.repo 中现存的所有配置
cat <<EOF | tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.28/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.28/rpm/repodata/repomd.xml.key
#exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni
EOF2、安装kubelet、kubeadm、kubectl、kubernetes-cni
yum install -y kubelet kubeadm kubectl kubernetes-cni
systemctl enable kubelet.service
四、K8S集群初始化
只在master01节点上操作
创建初始化文件 kubeadm-init.yaml
[root@k8s-master01 ~]# kubeadm config print init-defaults > kubeadm-init.yaml
修改如下配置:
- advertiseAddress:为控制平面地址,( Master 主机 IP )advertiseAddress: 1.2.3.4
修改为 advertiseAddress: 192.168.136.55
- criSocket:为 containerd 的 socket 文件地址criSocket: unix:///var/run/containerd/containerd.sock
修改为 criSocket: unix:///var/run/cri-dockerd.sock
- name: node 修改node为 k8s-master01name: node
修改为 name: k8s-master01
高可用才添加下面的
clusterName下面添加 VIP和端口
controlPlaneEndpoint: 192.168.136.50:16443- imageRepository:阿里云镜像代理地址,否则拉取镜像会失败imageRepository: registry.k8s.io
修改为:imageRepository: registry.aliyuncs.com/google_containers
- kubernetesVersion:为 k8s 版本kubernetesVersion: 1.28.0
修改为:kubernetesVersion: 1.28.9
注意:一定要配置镜像代理,否则会由于防火墙问题导致集群安装失败
文件末尾增加启用ipvs功能
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: ipvs# 根据配置文件启动 kubeadm 初始化 k8s
$ kubeadm init --config=kubeadm-init.yaml --upload-certs --v=6
注意:这里可能会报错
[ERROR FileContent--proc-sys-net-ipv4-ip_forward]: /proc/sys/net/ipv4/ip_forward contents are not set to 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
error execution phase preflight
解决:
[root@k8s-master01 ~]# sysctl -w net.ipv4.ip_forward=1
此页面 成功
[root@k8s-master01 ~]# mkdir -p $HOME/.kube
[root@k8s-master01 ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@k8s-master01 ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config
[root@k8s-master01 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master01 NotReady control-plane 3m48s v1.28.9
五、K8S集群工作节点加入
一定要加上 --cri-socket unix:///var/run/cri-dockerd.sock
[root@k8s-node01 ~]# kubeadm join 192.168.136.55:6443 --token abcdef.0123456789abcdef --discovery-token-ca-cert-hash sha256:6f28419e1b194870cdae7a25803752d968eac71a96dcda865b1e5fbe267396df --cri-socket unix:///var/run/cri-dockerd.sock
六、K8S集群网络插件使用
# 只在master01上操作
[root@k8s-master01 ~]# curl -O https://docs.projectcalico.org/archive/v3.23/manifests/calico.yaml
[root@k8s-master01 ~]# kubectl create -f calico.yaml
[root@k8s-master01 ~]# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-7cb4fd5784-7l5cs 1/1 Running 0 8m3s
calico-node-k5m7x 1/1 Running 0 8m3s
calico-node-mjjzs 1/1 Running 0 8m3s
calico-node-pzskl 1/1 Running 0 8m3s
coredns-66f779496c-767wf 1/1 Running 0 27m
coredns-66f779496c-ltw5p 1/1 Running 0 27m
etcd-k8s-master01 1/1 Running 0 27m
kube-apiserver-k8s-master01 1/1 Running 0 27m
kube-controller-manager-k8s-master01 1/1 Running 0 27m
kube-proxy-hmxvm 1/1 Running 0 24m
kube-proxy-htgbt 1/1 Running 0 24m
kube-proxy-tbm7d 1/1 Running 0 27m
kube-scheduler-k8s-master01 1/1 Running 0 27m
部署成功