前奏
搭建一主二从的k8s集群,如图所示,准备3台虚拟机。
不会创建的同学,可以看我上上篇博客:https://blog.csdn.net/dawnto/article/details/135086252
和上篇博客:https://blog.csdn.net/dawnto/article/details/135096182
由于测试机器资源有限,将3台虚拟机内存均设置为2048MB,CPU核心均设置为2核。
宿主机cpu和内存均在可控范围内,不影响其他程序运行
这三台虚拟机都可正常访问互联网,且配置了静态ip,分别是:
192.168.66.1
192.168.66.2
192.168.66.3
网关是:192.168.66.254
现在修改一下三台虚拟机的主机名
sudo vim /etc/hostname
分别修改为
ubuntu1
ubuntu2
ubuntu3
分别reboot
即可
看一下,非常的整齐
为了加快我们控制三台机器的速度:
选择一个好用的,开源免费的,国人维护的终端工具:https://github.com/electerm/electerm/releases
点书签可以创建连接
这里创建3个连接
将下边的开关打开,这样命令会同步在多个终端中执行
我们同步一下hosts文件,让虚拟机内部可以通过hostname访问其他虚拟机(若DNS支持主机域名解析,则无需配置)
cat >> /etc/hosts <<EOF
192.168.66.1 ubuntu1
192.168.66.2 ubuntu2
192.168.66.3 ubuntu3
EOF
此时验证一下,在ubuntu2 中 ping ubuntu3
,可见没有问题
关闭防火墙,两步走
停止:service ufw stop
永久关闭:systemctl disable ufw
关闭交换分区,两步走
swapoff -a
禁用交换分区
vim /etc/fstab
注释掉下图最后一行 /swap …,永久禁用交换分区
禁用 selinux(可选的)
ubuntu 默认不安装 selinux
使用:getenforce
,查看一下,是否有安装selinux,若有该命令则安装,安装则执行下述操作
有的话就,两步走
setenforce 0
,临时关闭
vim /etc/selinux/config
永久关闭selinux
将SELINUX
设置成disabled
未安装就不用管了
同步时间,使用ntpdate
工具
sudo apt-get install ntpdate
若没有,此时更新一下索引和源
sudo apt-get update
sudo apt-get upgrade
再次安装
安装以后执行ntpdate time.windows.com
,同步时间
发现现在时区还没设置,图片中时区明显不对
使用date -R
再看一番,雀食不对鸭
那就设置一下东八区
使用命令:timedatectl set-timezone Asia/Shanghai
设置时区,亚洲,上海
使用命令再次同步:ntpdate time.windows.com
,ok啦~
允许 iptables 检查桥接流量
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sudo sysctl --system
后续操作可能不生效,reboot
一下再切回root用户
安装containerd,容器运行时,(不直接用docker)
# 使用 docker 源安装最新版本 containerd / docker
sudo apt install \ca-certificates \curl \gnupg \lsb-releasesudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpgecho \"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/nullsudo apt update
sudo apt install containerd.io# sudo apt install docker-ce docker-ce-cli containerd.io docker-compose-plugin
配置containerd
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml
sudo vim /etc/containerd/config.toml
将 [plugins.“io.containerd.grpc.v1.cri”]
中 sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.6"
,换成红色区域内的
将 [plugins.“io.containerd.grpc.v1.cri”.containerd.runtimes.runc.options]
中的 SystemdCgroup = true
,替换为true
安装kubernetes
sudo apt-get update
# apt-transport-https 可能是一个虚拟包(dummy package);如果是的话,你可以跳过安装这个包
sudo apt-get install -y apt-transport-https ca-certificates curl gpg
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.29/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
# 此操作会覆盖 /etc/apt/sources.list.d/kubernetes.list 中现存的所有配置。
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
使用kubeadm创建一主二从集群
此命令在主节点中执行
kubeadm init \--image-repository registry.aliyuncs.com/google_containers \--pod-network-cidr=10.244.0.0/16 \--cri-socket unix:///run/containerd/containerd.sock \--skip-phases=addon/kube-proxy \--apiserver-advertise-address=192.168.66.1
–apiserver-advertise-address=192.168.66.1
,即主节点的ip
–pod-network-cidr=10.244.0.0/16 是容器内网络,随意设置,合规即可
然后。。。它报错了
root@ubuntu1:~# kubeadm init \--image-repository registry.aliyuncs.com/google_containers \--pod-network-cidr=10.244.0.0/16 \--cri-socket unix:///run/containerd/containerd.sock \--skip-phases=addon/kube-proxy \--apiserver-advertise-address=192.168.66.1[init] Using Kubernetes version: v1.29.0
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:[ERROR FileContent--proc-sys-net-ipv4-ip_forward]: /proc/sys/net/ipv4/ip_forward contents are not set to 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
root@ubuntu1:~#
使用 echo "1" > /proc/sys/net/ipv4/ip_forward
,修改值为1
然后再次执行初始化集群命令,成了
拷贝最后一行kubeadm join …
kubeadm join 192.168.66.1:6443 --token fwdwt9.bkqic5biggzkflfi --discovery-token-ca-cert-hash sha256:5802abd8404665ad92c3b92c0da34a16fdcf9768d54e7a38f234721af3025bdc
到另外两个机器中执行,让从节点加入集群
复制一下配置文件
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown (id -u):(id -g) $HOME/.kube/config
在主节点执行kubectl get pods -A
,发现有pod正处于Pending状态
执行kubectl get nodes
,发现节点都没有准备好
NotReady的主要原因是集群缺少DNS插件
在主节点上创建一个文件:kube-flannel.yml,内容如下
---
kind: Namespace
apiVersion: v1
metadata:name: kube-flannellabels:k8s-app: flannelpod-security.kubernetes.io/enforce: privileged
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:labels:k8s-app: flannelname: flannel
rules:
- apiGroups:- ""resources:- podsverbs:- get
- apiGroups:- ""resources:- nodesverbs:- get- list- watch
- apiGroups:- ""resources:- nodes/statusverbs:- patch
- apiGroups:- networking.k8s.ioresources:- clustercidrsverbs:- list- watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:labels:k8s-app: flannelname: flannel
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: flannel
subjects:
- kind: ServiceAccountname: flannelnamespace: kube-flannel
---
apiVersion: v1
kind: ServiceAccount
metadata:labels:k8s-app: flannelname: flannelnamespace: kube-flannel
---
kind: ConfigMap
apiVersion: v1
metadata:name: kube-flannel-cfgnamespace: kube-flannellabels:tier: nodek8s-app: flannelapp: flannel
data:cni-conf.json: |{"name": "cbr0","cniVersion": "0.3.1","plugins": [{"type": "flannel","delegate": {"hairpinMode": true,"isDefaultGateway": true}},{"type": "portmap","capabilities": {"portMappings": true}}]}net-conf.json: |{"Network": "10.244.0.0/16","Backend": {"Type": "vxlan"}}
---
apiVersion: apps/v1
kind: DaemonSet
metadata:name: kube-flannel-dsnamespace: kube-flannellabels:tier: nodeapp: flannelk8s-app: flannel
spec:selector:matchLabels:app: flanneltemplate:metadata:labels:tier: nodeapp: flannelspec:affinity:nodeAffinity:requiredDuringSchedulingIgnoredDuringExecution:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/osoperator: Invalues:- linuxhostNetwork: truepriorityClassName: system-node-criticaltolerations:- operator: Existseffect: NoScheduleserviceAccountName: flannelinitContainers:- name: install-cni-pluginimage: docker.io/flannel/flannel-cni-plugin:v1.2.0command:- cpargs:- -f- /flannel- /opt/cni/bin/flannelvolumeMounts:- name: cni-pluginmountPath: /opt/cni/bin- name: install-cniimage: docker.io/flannel/flannel:v0.24.0command:- cpargs:- -f- /etc/kube-flannel/cni-conf.json- /etc/cni/net.d/10-flannel.conflistvolumeMounts:- name: cnimountPath: /etc/cni/net.d- name: flannel-cfgmountPath: /etc/kube-flannel/containers:- name: kube-flannelimage: docker.io/flannel/flannel:v0.24.0command:- /opt/bin/flanneldargs:- --ip-masq- --kube-subnet-mgrresources:requests:cpu: "100m"memory: "50Mi"securityContext:privileged: falsecapabilities:add: ["NET_ADMIN", "NET_RAW"]env:- name: POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: EVENT_QUEUE_DEPTHvalue: "5000"volumeMounts:- name: runmountPath: /run/flannel- name: flannel-cfgmountPath: /etc/kube-flannel/- name: xtables-lockmountPath: /run/xtables.lockvolumes:- name: runhostPath:path: /run/flannel- name: cni-pluginhostPath:path: /opt/cni/bin- name: cnihostPath:path: /etc/cni/net.d- name: flannel-cfgconfigMap:name: kube-flannel-cfg- name: xtables-lockhostPath:path: /run/xtables.locktype: FileOrCreate
执行:kubectl apply -f kube-flannel.yml
,创建kube-flannel pod
再次执行:kubectl get nodes
,发现已经Ready
,撒花~
若出现:
lugin type="flannel" failed (add): loadFlannelSubnetEnv failed: open /run/flannel/subnet.env: no such file or directory
看引用3
在每个节点创建文件/run/flannel/subnet.env写入以下内容
FLANNEL_NETWORK=10.244.0.0/16
FLANNEL_SUBNET=10.244.0.1/24
FLANNEL_MTU=1450
FLANNEL_IPMASQ=true
Reference
1.https://zhuanlan.zhihu.com/p/627310856?utm_id=0
2.https://juejin.cn/post/7208088676853252156
3.https://blog.csdn.net/YangCheney/article/details/127046204