【Kubernetes资源篇】ingress-nginx最佳实践详解

文章目录

- 一、Ingress Controller理论知识
- - 1、Ingress Controller、Ingress简介
  - 2、四层代理与七层代理的区别
  - 3、Ingress Controller中封装Nginx，为什么不直接用Nginx呢？
  - 4、Ingress Controller代理K8S内部Pod流程
- 二、实践：部署Ingress Controller高可用架构
- - 1、部署Ingress Controller
  - 2、在Node节点上安装并配置Nginx、keepalived
  - 3、测试主备切换
- 三、实践：创建Ingress规则进行七层转发
- - 1、基于HTTP七层代理转发后端Pod
  - 2、基于HTTPS七层代理转发后端

一、Ingress Controller理论知识

Ingress官方中文参考文档：

1、Ingress Controller、Ingress简介

Ingress Controller是一个七层负载调度器，常见的七层负载均衡器有nginx、traefik，以我们熟悉的nginx为例，客户端的请求首先会到Ingress Controller七层负载调度器，由七层负载调度器将请求代理到后端的Pod。

以Nginx举例，客户端请求首先会到Nginx中，由Nginx中的upstream模块将请求代理到后端的服务上，但是K8s场景下，后端Pod的IP地址不是固定的，因此在Pod前面需要添加一个service资源，请求到达Service，由Service代理到后端的Pod。

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-bjf2hcl2-1690102684594)(D:\MD归档文档\IMG\image-20230722173017074.png)]$

Ingress是K8S中的资源，简单理解就是Ingress Controller的配置文件，创建ingress规则在管理Ingress Controller。

2、四层代理与七层代理的区别

四层代理：

工作在传输层，可以解析传输层协议，TCP、UDP等。
四层代理基于传IP+端口方式进行转发。

七层代理：

工作在应有层，可以解析应用层协议，如HTTP、FTP等。
七层负载工作在四层的基础之上，基于虚拟主机的URL或主机的IP进行转发。

总体而言，四层代理更关注于网络层面的流量控制和安全，主要基于传输层的信息进行处理；而七层代理更加智能，能够理解和处理应用层协议的内容，提供更加精细的控制和调度。选择使用哪种类型的代理取决于具体需求和使用场景。

OSI七层模型：

在这里插入图片描述
722190254323.png)]

3、Ingress Controller中封装Nginx，为什么不直接用Nginx呢？

在宿主机安装Nginx，只要配置文件有改动，就必须手动reload加载才可以生效，但是如果使用Ingress Controller封装的Nginx，你ingress维护配置，ingress创建好了之后，会自动把配置文件传到Ingress Controller这个Pod中，自动进行reload加载。

4、Ingress Controller代理K8S内部Pod流程

第一步：部署Ingress Controller

第二步：创建Pod，可以使用控制器进行创建

第三步：创建Service，管理Pod

第四步：创建Ingress http或https规则

第五步：测试，客户端通过七层访问

二、实践：部署Ingress Controller高可用架构

高可用架构请求转发图：

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-cUwOluxz-1690102684596)(D:\MD归档文档\IMG\image-20230723165049687.png)]$

ingress-nginx GitHub地址：

ingress-nginx YAML GitHub地址：

1、部署Ingress Controller

1、编写YAML文件，基于官方下载，根基自己需求进行对应修改。

cat ingress-controller-nginx.yaml
---
apiVersion: v1
kind: Namespace
metadata:name: ingress-nginxlabels:app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginx---
# Source: ingress-nginx/templates/controller-serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:labels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: controllername: ingress-nginxnamespace: ingress-nginx
automountServiceAccountToken: true
---
# Source: ingress-nginx/templates/controller-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:labels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: controllername: ingress-nginx-controllernamespace: ingress-nginx
data:allow-snippet-annotations: "true"
---
# Source: ingress-nginx/templates/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmname: ingress-nginx
rules:- apiGroups:- ""resources:- configmaps- endpoints- nodes- pods- secrets- namespacesverbs:- list- watch- apiGroups:- ""resources:- nodesverbs:- get- apiGroups:- ""resources:- servicesverbs:- get- list- watch- apiGroups:- networking.k8s.ioresources:- ingressesverbs:- get- list- watch- apiGroups:- ""resources:- eventsverbs:- create- patch- apiGroups:- networking.k8s.ioresources:- ingresses/statusverbs:- update- apiGroups:- networking.k8s.ioresources:- ingressclassesverbs:- get- list- watch
---
# Source: ingress-nginx/templates/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:labels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmname: ingress-nginx
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: ingress-nginx
subjects:- kind: ServiceAccountname: ingress-nginxnamespace: ingress-nginx
---
# Source: ingress-nginx/templates/controller-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:labels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: controllername: ingress-nginxnamespace: ingress-nginx
rules:- apiGroups:- ""resources:- namespacesverbs:- get- apiGroups:- ""resources:- configmaps- pods- secrets- endpointsverbs:- get- list- watch- apiGroups:- ""resources:- servicesverbs:- get- list- watch- apiGroups:- networking.k8s.ioresources:- ingressesverbs:- get- list- watch- apiGroups:- networking.k8s.ioresources:- ingresses/statusverbs:- update- apiGroups:- networking.k8s.ioresources:- ingressclassesverbs:- get- list- watch- apiGroups:- ""resources:- configmapsresourceNames:- ingress-controller-leaderverbs:- get- update- apiGroups:- ""resources:- configmapsverbs:- create- apiGroups:- ""resources:- eventsverbs:- create- patch
---
# Source: ingress-nginx/templates/controller-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:labels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: controllername: ingress-nginxnamespace: ingress-nginx
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: ingress-nginx
subjects:- kind: ServiceAccountname: ingress-nginxnamespace: ingress-nginx
---
# Source: ingress-nginx/templates/controller-service-webhook.yaml
apiVersion: v1
kind: Service
metadata:labels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: controllername: ingress-nginx-controller-admissionnamespace: ingress-nginx
spec:type: ClusterIPports:- name: https-webhookport: 443targetPort: webhookappProtocol: httpsselector:app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/component: controller
---
# Source: ingress-nginx/templates/controller-service.yaml
apiVersion: v1
kind: Service
metadata:annotations:labels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: controllername: ingress-nginx-controllernamespace: ingress-nginx
spec:type: NodePortipFamilyPolicy: SingleStackipFamilies:- IPv4ports:- name: httpport: 80protocol: TCPtargetPort: httpappProtocol: http- name: httpsport: 443protocol: TCPtargetPort: httpsappProtocol: httpsselector:app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/component: controller
---
# Source: ingress-nginx/templates/controller-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:labels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: controllername: ingress-nginx-controllernamespace: ingress-nginx
spec:replicas: 2selector:matchLabels:app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/component: controllerrevisionHistoryLimit: 10minReadySeconds: 0template:metadata:labels:app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/component: controllerspec:hostNetwork: trueaffinity:podAntiAffinity:preferredDuringSchedulingIgnoredDuringExecution:- weight: 100podAffinityTerm:labelSelector:matchLabels:app.kubernetes.io/name: ingress-nginxtopologyKey: kubernetes.io/hostnamednsPolicy: ClusterFirstWithHostNetcontainers:- name: controllerimage: registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.1.0imagePullPolicy: IfNotPresentlifecycle:preStop:exec:command:- /wait-shutdownargs:- /nginx-ingress-controller- --election-id=ingress-controller-leader- --controller-class=k8s.io/ingress-nginx- --configmap=$(POD_NAMESPACE)/ingress-nginx-controller- --validating-webhook=:8443- --validating-webhook-certificate=/usr/local/certificates/cert- --validating-webhook-key=/usr/local/certificates/keysecurityContext:capabilities:drop:- ALLadd:- NET_BIND_SERVICErunAsUser: 101allowPrivilegeEscalation: trueenv:- name: POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: LD_PRELOADvalue: /usr/local/lib/libmimalloc.solivenessProbe:failureThreshold: 5httpGet:path: /healthzport: 10254scheme: HTTPinitialDelaySeconds: 10periodSeconds: 10successThreshold: 1timeoutSeconds: 1readinessProbe:failureThreshold: 3httpGet:path: /healthzport: 10254scheme: HTTPinitialDelaySeconds: 10periodSeconds: 10successThreshold: 1timeoutSeconds: 1ports:- name: httpcontainerPort: 80protocol: TCP- name: httpscontainerPort: 443protocol: TCP- name: webhookcontainerPort: 8443protocol: TCPvolumeMounts:- name: webhook-certmountPath: /usr/local/certificates/readOnly: trueresources:requests:cpu: 100mmemory: 90MinodeSelector:kubernetes.io/os: linuxserviceAccountName: ingress-nginxterminationGracePeriodSeconds: 300volumes:- name: webhook-certsecret:secretName: ingress-nginx-admission
---
# Source: ingress-nginx/templates/controller-ingressclass.yaml
# We don't support namespaced ingressClass yet
# So a ClusterRole and a ClusterRoleBinding is required
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:labels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: controllername: nginxnamespace: ingress-nginx
spec:controller: k8s.io/ingress-nginx
---
# Source: ingress-nginx/templates/admission-webhooks/validating-webhook.yaml
# before changing this value, check the required kubernetes version
# https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#prerequisites
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:labels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhookname: ingress-nginx-admission
webhooks:- name: validate.nginx.ingress.kubernetes.iomatchPolicy: Equivalentrules:- apiGroups:- networking.k8s.ioapiVersions:- v1operations:- CREATE- UPDATEresources:- ingressesfailurePolicy: FailsideEffects: NoneadmissionReviewVersions:- v1clientConfig:service:namespace: ingress-nginxname: ingress-nginx-controller-admissionpath: /networking/v1/ingresses
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:name: ingress-nginx-admissionnamespace: ingress-nginxannotations:helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgradehelm.sh/hook-delete-policy: before-hook-creation,hook-succeededlabels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhook
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:name: ingress-nginx-admissionannotations:helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgradehelm.sh/hook-delete-policy: before-hook-creation,hook-succeededlabels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhook
rules:- apiGroups:- admissionregistration.k8s.ioresources:- validatingwebhookconfigurationsverbs:- get- update
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: ingress-nginx-admissionannotations:helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgradehelm.sh/hook-delete-policy: before-hook-creation,hook-succeededlabels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhook
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: ingress-nginx-admission
subjects:- kind: ServiceAccountname: ingress-nginx-admissionnamespace: ingress-nginx
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:name: ingress-nginx-admissionnamespace: ingress-nginxannotations:helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgradehelm.sh/hook-delete-policy: before-hook-creation,hook-succeededlabels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhook
rules:- apiGroups:- ""resources:- secretsverbs:- get- create
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:name: ingress-nginx-admissionnamespace: ingress-nginxannotations:helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgradehelm.sh/hook-delete-policy: before-hook-creation,hook-succeededlabels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhook
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: ingress-nginx-admission
subjects:- kind: ServiceAccountname: ingress-nginx-admissionnamespace: ingress-nginx
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/job-createSecret.yaml
apiVersion: batch/v1
kind: Job
metadata:name: ingress-nginx-admission-createnamespace: ingress-nginxannotations:helm.sh/hook: pre-install,pre-upgradehelm.sh/hook-delete-policy: before-hook-creation,hook-succeededlabels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhook
spec:template:metadata:name: ingress-nginx-admission-createlabels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhookspec:containers:- name: createimage: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1imagePullPolicy: IfNotPresentargs:- create- --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc- --namespace=$(POD_NAMESPACE)- --secret-name=ingress-nginx-admissionenv:- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespacesecurityContext:allowPrivilegeEscalation: falserestartPolicy: OnFailureserviceAccountName: ingress-nginx-admissionnodeSelector:kubernetes.io/os: linuxsecurityContext:runAsNonRoot: truerunAsUser: 2000
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/job-patchWebhook.yaml
apiVersion: batch/v1
kind: Job
metadata:name: ingress-nginx-admission-patchnamespace: ingress-nginxannotations:helm.sh/hook: post-install,post-upgradehelm.sh/hook-delete-policy: before-hook-creation,hook-succeededlabels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhook
spec:template:metadata:name: ingress-nginx-admission-patchlabels:helm.sh/chart: ingress-nginx-4.0.10app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 1.1.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhookspec:containers:- name: patchimage: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1imagePullPolicy: IfNotPresentargs:- patch- --webhook-name=ingress-nginx-admission- --namespace=$(POD_NAMESPACE)- --patch-mutating=false- --secret-name=ingress-nginx-admission- --patch-failure-policy=Failenv:- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespacesecurityContext:allowPrivilegeEscalation: falserestartPolicy: OnFailureserviceAccountName: ingress-nginx-admissionnodeSelector:kubernetes.io/os: linuxsecurityContext:runAsNonRoot: truerunAsUser: 2000

2、执行YAML文件

kubectl apply -f ingress-controller-nginx.yaml

如果执行YAML文件有报错，如下：

报错内容：Error from server (InternalError): error when creating “ingress.yaml“: Internal error occurred: fail

报错解决方法：

kubectl delete -A ValidatingWebhookConfiguration ingress-nginx-admission

3、查看创建的Pod资源状态是否已运行

kubectl get pod -n ingress-nginx

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-bODP7iYg-1690102684603)(D:\MD归档文档\IMG\image-20230723131517381.png)]$

2、在Node节点上安装并配置Nginx、keepalived

1、上面部署ingress controller分配在不同的两台Node节点(两台Node节点同步操作)

yum install  epel-release  nginx keepalived nginx-mod-stream nc -y

2、修改 nginx.conf 配置文件(两台Node节点同步操作)

mv /etc/nginx/nginx.conf{,.$(date +%F)}
vim /etc/nginx/nginx.confuser nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;include /usr/share/nginx/modules/*.conf;events {worker_connections 1024;
}# 四层负载
stream {log_format  main  '$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';access_log  /var/log/nginx/k8s-access.log  main;# 定义后端负载节点upstream k8s-ingress-controller {server 16.32.15.201:80 weight=5 max_fails=3 fail_timeout=30s;server 16.32.15.202:80 weight=5 max_fails=3 fail_timeout=30s;}# 访问30080代理到后端节点server {listen 30080; proxy_pass k8s-ingress-controller;}
}http {log_format  main  '$remote_addr - $remote_user [$time_local] "$request" ''$status $body_bytes_sent "$http_referer" ''"$http_user_agent" "$http_x_forwarded_for"';access_log  /var/log/nginx/access.log  main;sendfile            on;tcp_nopush          on;tcp_nodelay         on;keepalive_timeout   65;types_hash_max_size 2048;include             /etc/nginx/mime.types;default_type        application/octet-stream;}

检查配置 && 启动并加入开机自启动

nginx -t 
systemctl enable nginx --now
systemctl status nginx

3、修改Keepalived Master节点配置文件(Keepalived Master操作，我这里将16.32.15.201定义为主)

mv /etc/keepalived/keepalived.conf{,.$(date +%F)}
vim /etc/keepalived/keepalived.conf
vrrp_script check_nginx {script "/etc/keepalived/check_nginx.sh"
}vrrp_instance VI_1 { state MASTER interface ens33  # 网卡名称virtual_router_id 51 priority 100 advert_int 1   authentication { auth_type PASS      auth_pass 1111 }  # 虚拟IPvirtual_ipaddress { 16.32.15.100/24} track_script {check_nginx} 
}

添加判断Nginx是否运行脚本

vim /etc/keepalived/check_nginx.sh
#!/bin/bash
nc -z localhost 30080
if [[ $? -ne 0 ]];thensystemctl stop keepalived.service
fichmod +x /etc/keepalived/check_nginx.sh

启动主节点keepalived

systemctl enable keepalived --now

4、修改Keepalived Backup节点配置文件(Keepalived Master操作，我这里将16.32.15.202定义为备)

mv /etc/keepalived/keepalived.conf{,.$(date +%F)}
vim /etc/keepalived/keepalived.conf
vrrp_script check_nginx {script "/etc/keepalived/check_nginx.sh"
}vrrp_instance VI_1 { state BACKUP interface ens33  # 网卡名称virtual_router_id 51 priority 90advert_int 1   authentication { auth_type PASS      auth_pass 1111 }  # 虚拟IPvirtual_ipaddress { 16.32.15.100/24} track_script {check_nginx} 
}

添加判断Nginx是否运行脚本

vim /etc/keepalived/check_nginx.sh
#!/bin/bash
nc -z localhost 30080
if [[ $? -ne 0 ]];thensystemctl stop keepalived.service
fichmod +x /etc/keepalived/check_nginx.sh

启动备节点keepalived

systemctl enable keepalived --now

3、测试主备切换

1、在主机停止nginx服务

systemctl stop nginx

2、在备机，查看VIP是否漂移过去

ip a|grep 100

如果漂移过去表示无问题，如下图：

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-TUQuELl8-1690102684604)(D:\MD归档文档\IMG\image-20230723143033758.png)]$

3、在主机启动，VIP会自动表漂移到主机

systemctl start nginx keepalived
ip a|grep 100

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-ghq73yAv-1690102684604)(D:\MD归档文档\IMG\image-20230723143203306.png)]$

三、实践：创建Ingress规则进行七层转发

Ingress规则官方参考文档

1、基于HTTP七层代理转发后端Pod

1、创建后端Pod、Server资源

cat ingress-demo.yaml
---
apiVersion: v1
kind: Service
metadata:name: ingress-tomcat-servicenamespace: default
spec:selector:app: tomcatports:- name: httptargetPort: 8080port: 8080- name: ajptargetPort: 8009port: 8009
---
apiVersion: apps/v1
kind: Deployment
metadata:name: ingress-tomcat-deploymentnamespace: default
spec:replicas: 2selector:matchLabels:app: tomcattemplate:metadata:labels:app: tomcatspec:containers:- name: tomcatimage: tomcat:8.5.34-jre8-alpine imagePullPolicy: IfNotPresent  ports:- name: httpcontainerPort: 8080name: ajpcontainerPort: 8009

执行YAML文件：

kubectl apply -f ingress-demo.yaml

查看创建的Pod、Service

kubectl get pods,svc

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-ewDW110C-1690102684604)(D:\MD归档文档\IMG\image-20230723150303523.png)]$

2、创建Ingress转发规则

cat ingress-tomcat.yamlapiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: ingress-tomcatnamespace: default
spec: ingressClassName: nginx     # 指定ingress类名称,这里是Nginxrules:- host: tomcat.ingress.com  # 客户端访问的域名http:paths:- backend:service:name: ingress-tomcat-service  # 转发到SVC名称port:number: 8080                # 转发到SVC端口path: /                           # 转发到/pathType: Prefix

执行YAML

kubectl apply -f ingress-tomcat.yaml

3、添加域名解析：

打开 C:\Windows\System32\drivers\etc\hosts 文件，添加解析，如下图：

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-904hIlOE-1690102684605)(D:\MD归档文档\IMG\image-20230723151540821.png)]$

浏览器访问tomcat.ingress.com:30080进行测试

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-ZIvwYI4w-1690102684605)(D:\MD归档文档\IMG\image-20230723151720613.png)]$

2、基于HTTPS七层代理转发后端

基于上面 HTTP七层代理转发的 Pod、Service做实验，不在创建新的资源。

1、创建证书

生成一个私钥

openssl genrsa -out tls.key 2048

基于私钥生成根证书，并签发qinzt.ingress.com 域名

openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=Beijing/L=Beijing/O=DevOps/CN=qinzt.ingress.com

2、创建secret，对证书进行加密

kubectl create secret tls ingress-tomcat-secret --cert=tls.crt --key=tls.key

查看secret

kubectl describe secret ingress-tomcat-secret

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-tY1SB7IC-1690102684605)(D:\MD归档文档\IMG\image-20230723155504076.png)]$

3、创建ingress规则

cat ingress-tomcat-tls.yamlapiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: ingress-tomcat-tlsnamespace: default
spec:ingressClassName: nginxtls:- hosts:-  qinzt.ingress.comsecretName: ingress-tomcat-secret   # secret名称rules:- host: qinzt.ingress.comhttp:paths:- path: /pathType:  Prefixbackend:service:name: tomcatport:number: 8080

执行YAML文件：

kubectl apply -f ingress-tomcat-tls.yaml

4、添加域名解析：

打开 C:\Windows\System32\drivers\etc\hosts 文件，添加解析，如下图：

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-QTgim04r-1690102684605)(D:\MD归档文档\IMG\image-20230723160113701.png)]$

5、浏览器访问域名测试

由于证书是自签名，所有浏览器会提示不安全，点击确认继续访问即可

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-wCHoPi0f-1690102684606)(D:\MD归档文档\IMG\image-20230723160301173.png)]$

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-JSoIXDiW-1690102684606)(D:\MD归档文档\IMG\image-20230723161554949.png)]$