K8S安装和部署(kubeadmin安装1主2从)

这里用kubeadmin方式进行安装部署

1. 准备三台服务器

服务器地址	节点名称
192.168.190.200	master	主
192.168.190.201	node1	从
192.168.190.202	node2	从

2. 主机初始化（所有主机）

2.1根据规划设置主机名

#切换到192.168.190.200
hostnamectl set-hostname  master
#切换到192.168.190.201
hostnamectl set-hostname  node1
#切换到192.168.190.202
hostnamectl set-hostname  node2

并三台主机设置主机名和IP地址的映射关系

vim /etc/hosts

172.19.3.240  k8s-master
172.19.3.241  k8s-node1
172.19.3.242  k8s-node2

2.2 时间同步（如果服务器时间同步忽略此步骤）

#启动chronyd服务
systemctl start chronyd
systemctl enable chronyd
date

💡 Tips：执行命令前可以使用 rpm -qa |grep chrony 查看系统是否已安装chrony，没有安装环境可使用 yum

install chrony 命令安装

2.3 安装docker(三台服务器均需安装)

1 安装命令工具 yum install -y yum-utils

2 切换源 yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

3 查看版本号 3.10以上可安装推荐centos7以上系统

4 查看可安装docker软件包

yum list docker-ce --showduplicates | sort -r

5 安装并启动docker

yum install docker-ce-20.10.21-3.el7 -y
systemctl start docker
systemctl enable docker

这里在执行 yum install docker-ce-20.10.21-3.el7 -y 命令是报错，报错内容如下

---> 软件包 docker-compose-plugin.x86_64.0.2.16.0-1.el7 将被安装

---> 软件包 docker-scan-plugin.x86_64.0.0.23.0-3.el7 将被安装

--> 解决依赖关系完成

错误：软件包：docker-ce-rootless-extras-23.0.1-1.el7.x86_64 (docker-ce-stable)

需要：fuse-overlayfs >= 0.7

错误：软件包：docker-ce-rootless-extras-23.0.1-1.el7.x86_64 (docker-ce-stable)

需要：slirp4netns >= 0.4

错误：软件包：containerd.io-1.6.18-3.1.el7.x86_64 (docker-ce-stable)

需要：container-selinux >= 2:2.74

错误：软件包：3:docker-ce-20.10.21-3.el7.x86_64 (docker-ce-stable)

需要：container-selinux >= 2:2.74

您可以尝试添加 --skip-broken 选项来解决该问题

您可以尝试执行：rpm -Va --nofiles --nodigest

解决错误，执行以下命令参考RedHat7使用阿里云镜像建立元数据缓存时404解决_yum makecache aliyun 404-CSDN博客

wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo

yum clean all

yum makecache //这里执行会报错，按下面命令继续执行

sed -i 's/\$releasever/7/' /etc/yum.repos.d/CentOS-Base.repo

sed -i 's#$basearch#x86_64#g' /etc/yum.repos.d/CentOS-Base.repo

yum clean all

yum makecache

最后执行安装和启动命令即可成功安装

yum install docker-ce-20.10.21-3.el7 -y
systemctl start docker
systemctl enable docker

查看安装版本 docker version

2.4 关闭防火墙和SELinux,并且禁用Swap（三台服务器均需操作）

### 关闭防火墙
systemctl disable --now firewalld### 关闭selinux, 让容器可以读取主机文件系统
getenforce      # 查看 
setenforce 0    # 临时关闭
sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config    # 永久关闭### 关闭swap
swapoff -a      # 临时关闭swap
sed -i 's/.*swap.*/#&/' /etc/fstab    # 永久关闭swap

3. 添加网桥过滤和地址转发功能（三台服务器均需操作）

cat > /etc/sysctl.d/kubernetes.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF#然后执行
sysctl --system   //生效命令

4. kubernetes镜像切换成国内源（三台服务器均需操作）

cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg 
https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

5. 安装指定版本的 kubeadm，kubelet和kubectl（三台服务器均需操作）

yum install -y kubelet-1.23.5 kubeadm-1.23.5 kubectl-1.23.5
#设置kubelet开机自启
systemctl enable kubelet
systemctl start kubelet

查看安装版本

kubelet --version
kubeadm version
kubectl version

6. 部署Kubernetes

💡 Tips：下面的操作只需要在master节点上执行即可（初始化完成后，最后会输出一个join命令，可以先保存后边会用）

kubeadm init \--apiserver-advertise-address=172.16.2.240 \--image-repository registry.aliyuncs.com/google_containers \--kubernetes-version v1.23.5 \--service-cidr=10.1.0.0/12 \--pod-network-cidr=10.244.0.0/16 \--ignore-preflight-errors=all#172.16.2.240为主机IP

–apiserver-advertise-address #集群通告地址(master 机器IP)

–image-repository #由于默认拉取镜像地址k8s.gcr.io国内无法访问，这里指定阿里云镜像仓库地址

–kubernetes-version #K8s版本，与上面安装的一致

–service-cidr #集群内部虚拟网络，Pod统一访问入口

–pod-network-cidr #Pod网络，与下面部署的CNI网络组件yaml中保持一致

执行以上命令可能会报错，

报错解决：The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz"

#解决上述报错,如果没有配置的所有服务器都需要配置，这里也要配置docker的国内镜像，不然镜像拉取会比较慢
# 添加以下内容
vim /etc/docker/daemon.json
{"exec-opts": ["native.cgroupdriver=systemd"]
}# 重启docker
systemctl restart docker
# 重新初始化
kubeadm reset # 先重置
# 重新执行上面的 kubeadm init

初始化成功后会输出以下信息（第11步会用到）

master上执行命令：kubeadm token create --print-join-command 重新查看join命令

7. 在master上配置kubectl命令行工具

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

8.在master节点上下载flannel网络配置文件

 wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

下边是下载的文件内容kube-flannel.yml,下载不了可以保存下边这份文件，注意修改文件中地址跟第6步中的配置的一样

--pod-network-cidr=10.244.0.0/16

---
kind: Namespace
apiVersion: v1
metadata:name: kube-flannellabels:pod-security.kubernetes.io/enforce: privileged
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: flannel
rules:
- apiGroups:- ""resources:- podsverbs:- get
- apiGroups:- ""resources:- nodesverbs:- get- list- watch
- apiGroups:- ""resources:- nodes/statusverbs:- patch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: flannel
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: flannel
subjects:
- kind: ServiceAccountname: flannelnamespace: kube-flannel
---
apiVersion: v1
kind: ServiceAccount
metadata:name: flannelnamespace: kube-flannel
---
kind: ConfigMap
apiVersion: v1
metadata:name: kube-flannel-cfgnamespace: kube-flannellabels:tier: nodeapp: flannel
data:cni-conf.json: |{"name": "cbr0","cniVersion": "0.3.1","plugins": [{"type": "flannel","delegate": {"hairpinMode": true,"isDefaultGateway": true}},{"type": "portmap","capabilities": {"portMappings": true}}]}net-conf.json: |{"Network": "10.244.0.0/16","Backend": {"Type": "vxlan"}}
---
apiVersion: apps/v1
kind: DaemonSet
metadata:name: kube-flannel-dsnamespace: kube-flannellabels:tier: nodeapp: flannel
spec:selector:matchLabels:app: flanneltemplate:metadata:labels:tier: nodeapp: flannelspec:affinity:nodeAffinity:requiredDuringSchedulingIgnoredDuringExecution:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/osoperator: Invalues:- linuxhostNetwork: truepriorityClassName: system-node-criticaltolerations:- operator: Existseffect: NoScheduleserviceAccountName: flannelinitContainers:- name: install-cni-plugin#image: flannelcni/flannel-cni-plugin:v1.1.0 for ppc64le and mips64le (dockerhub limitations may apply)image: docker.io/rancher/mirrored-flannelcni-flannel-cni-plugin:v1.1.0command:- cpargs:- -f- /flannel- /opt/cni/bin/flannelvolumeMounts:- name: cni-pluginmountPath: /opt/cni/bin- name: install-cni#image: flannelcni/flannel:v0.20.2 for ppc64le and mips64le (dockerhub limitations may apply)image: docker.io/rancher/mirrored-flannelcni-flannel:v0.20.2command:- cpargs:- -f- /etc/kube-flannel/cni-conf.json- /etc/cni/net.d/10-flannel.conflistvolumeMounts:- name: cnimountPath: /etc/cni/net.d- name: flannel-cfgmountPath: /etc/kube-flannel/containers:- name: kube-flannel#image: flannelcni/flannel:v0.20.2 for ppc64le and mips64le (dockerhub limitations may apply)image: docker.io/rancher/mirrored-flannelcni-flannel:v0.20.2command:- /opt/bin/flanneldargs:- --ip-masq- --kube-subnet-mgrresources:requests:cpu: "100m"memory: "50Mi"limits:cpu: "100m"memory: "50Mi"securityContext:privileged: falsecapabilities:add: ["NET_ADMIN", "NET_RAW"]env:- name: POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: EVENT_QUEUE_DEPTHvalue: "5000"volumeMounts:- name: runmountPath: /run/flannel- name: flannel-cfgmountPath: /etc/kube-flannel/- name: xtables-lockmountPath: /run/xtables.lockvolumes:- name: runhostPath:path: /run/flannel- name: cni-pluginhostPath:path: /opt/cni/bin- name: cnihostPath:path: /etc/cni/net.d- name: flannel-cfgconfigMap:name: kube-flannel-cfg- name: xtables-lockhostPath:path: /run/xtables.locktype: FileOrCreate

9.在master节点上应用flannel网络的配置文件

kubectl apply -f kube-flannel.yml

命令执行成功输出以下信息

10. 在node1和node2节点上拉取 quay.io/coreos/flannel:v0.11.0-amd64 的镜像

 docker pull quay.io/coreos/flannel:v0.11.0-amd64

11. 在node1和node2节点上分别执行以下命令，将这两个节点加入Kubernet

这里根据实际的来，在master上执行命令可获取 kubeadm token create --print-join-command

kubeadm join 192.168.190.200:6443 --token 7cmfxu.smm6jqlx0llsym4p --discovery-token-ca-cert-hash sha256:4af0d23c767f2a23605747963d8d0de78082bbd8147e261469b6fb75fe136a63

如果在单节点上执行join操作时出现错误，可以加上参数 --ignore-preflight-errors=all

加入成功

这里也可能会遇到跟2.3步骤中一样的错误，操作一样，修改docker配置文件即可

It seems like the kubelet isn‘t running or healthy 报错解决

配置docker文件，在/etc/docker/daemon.json 添加以下内容

"exec-opts": [

"native.cgroupdriver=systemd"

],

重启docker

systemctl restart docker

重启kubelet （optional）

systemctl restart kubelet

再次执行命令

kubeadm join 192.168.190.200:6443 --token 7cmfxu.smm6jqlx0llsym4p --discovery-token-ca-cert-hash sha256:4af0d23c767f2a23605747963d8d0de78082bbd8147e261469b6fb75fe136a63