6、Ingress ★

Ingress：Service 的统一网关入口，底层就是 nginx。（服务）

官网地址：https://kubernetes.github.io/ingress-nginx/ （都是从这里看的）

所有的请求都先通过 Ingress，由 Ingress 来 打理这些请求。类似微服务中的 网关。

6.1 安装 Ingress

wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.47.0/deploy/static/provider/baremetal/deploy.yaml## 这里我喜欢把depoly.yaml名字修改为ingress.yaml# 修改镜像
vi ingress.yaml
# 将 image 的值改为如下值
registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/ingress-nginx-controller:v0.46.0# 安装资源
kubectl apply -f ingress.yaml# 检查安装的结果
kubectl get pod,svc -n ingress-nginx# 最后别忘记把 svc 暴露的端口 在安全组放行

ingress.yaml整个文件内容：

apiVersion: v1
kind: Namespace
metadata:name: ingress-nginxlabels:app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginx---
# Source: ingress-nginx/templates/controller-serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:labels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: controllername: ingress-nginxnamespace: ingress-nginx
automountServiceAccountToken: true
---
# Source: ingress-nginx/templates/controller-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:labels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: controllername: ingress-nginx-controllernamespace: ingress-nginx
data:
---
# Source: ingress-nginx/templates/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmname: ingress-nginx
rules:- apiGroups:- ''resources:- configmaps- endpoints- nodes- pods- secretsverbs:- list- watch- apiGroups:- ''resources:- nodesverbs:- get- apiGroups:- ''resources:- servicesverbs:- get- list- watch- apiGroups:- extensions- networking.k8s.io   # k8s 1.14+resources:- ingressesverbs:- get- list- watch- apiGroups:- ''resources:- eventsverbs:- create- patch- apiGroups:- extensions- networking.k8s.io   # k8s 1.14+resources:- ingresses/statusverbs:- update- apiGroups:- networking.k8s.io   # k8s 1.14+resources:- ingressclassesverbs:- get- list- watch
---
# Source: ingress-nginx/templates/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:labels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmname: ingress-nginx
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: ingress-nginx
subjects:- kind: ServiceAccountname: ingress-nginxnamespace: ingress-nginx
---
# Source: ingress-nginx/templates/controller-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:labels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: controllername: ingress-nginxnamespace: ingress-nginx
rules:- apiGroups:- ''resources:- namespacesverbs:- get- apiGroups:- ''resources:- configmaps- pods- secrets- endpointsverbs:- get- list- watch- apiGroups:- ''resources:- servicesverbs:- get- list- watch- apiGroups:- extensions- networking.k8s.io   # k8s 1.14+resources:- ingressesverbs:- get- list- watch- apiGroups:- extensions- networking.k8s.io   # k8s 1.14+resources:- ingresses/statusverbs:- update- apiGroups:- networking.k8s.io   # k8s 1.14+resources:- ingressclassesverbs:- get- list- watch- apiGroups:- ''resources:- configmapsresourceNames:- ingress-controller-leader-nginxverbs:- get- update- apiGroups:- ''resources:- configmapsverbs:- create- apiGroups:- ''resources:- eventsverbs:- create- patch
---
# Source: ingress-nginx/templates/controller-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:labels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: controllername: ingress-nginxnamespace: ingress-nginx
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: ingress-nginx
subjects:- kind: ServiceAccountname: ingress-nginxnamespace: ingress-nginx
---
# Source: ingress-nginx/templates/controller-service-webhook.yaml
apiVersion: v1
kind: Service
metadata:labels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: controllername: ingress-nginx-controller-admissionnamespace: ingress-nginx
spec:type: ClusterIPports:- name: https-webhookport: 443targetPort: webhookselector:app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/component: controller
---
# Source: ingress-nginx/templates/controller-service.yaml
apiVersion: v1
kind: Service
metadata:annotations:labels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: controllername: ingress-nginx-controllernamespace: ingress-nginx
spec:type: NodePortports:- name: httpport: 80protocol: TCPtargetPort: http- name: httpsport: 443protocol: TCPtargetPort: httpsselector:app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/component: controller
---
# Source: ingress-nginx/templates/controller-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:labels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: controllername: ingress-nginx-controllernamespace: ingress-nginx
spec:selector:matchLabels:app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/component: controllerrevisionHistoryLimit: 10minReadySeconds: 0template:metadata:labels:app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/component: controllerspec:dnsPolicy: ClusterFirstcontainers:- name: controllerimage: registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/ingress-nginx-controller:v0.46.0imagePullPolicy: IfNotPresentlifecycle:preStop:exec:command:- /wait-shutdownargs:- /nginx-ingress-controller- --election-id=ingress-controller-leader- --ingress-class=nginx- --configmap=$(POD_NAMESPACE)/ingress-nginx-controller- --validating-webhook=:8443- --validating-webhook-certificate=/usr/local/certificates/cert- --validating-webhook-key=/usr/local/certificates/keysecurityContext:capabilities:drop:- ALLadd:- NET_BIND_SERVICErunAsUser: 101allowPrivilegeEscalation: trueenv:- name: POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: LD_PRELOADvalue: /usr/local/lib/libmimalloc.solivenessProbe:failureThreshold: 5httpGet:path: /healthzport: 10254scheme: HTTPinitialDelaySeconds: 10periodSeconds: 10successThreshold: 1timeoutSeconds: 1readinessProbe:failureThreshold: 3httpGet:path: /healthzport: 10254scheme: HTTPinitialDelaySeconds: 10periodSeconds: 10successThreshold: 1timeoutSeconds: 1ports:- name: httpcontainerPort: 80protocol: TCP- name: httpscontainerPort: 443protocol: TCP- name: webhookcontainerPort: 8443protocol: TCPvolumeMounts:- name: webhook-certmountPath: /usr/local/certificates/readOnly: trueresources:requests:cpu: 100mmemory: 90MinodeSelector:kubernetes.io/os: linuxserviceAccountName: ingress-nginxterminationGracePeriodSeconds: 300volumes:- name: webhook-certsecret:secretName: ingress-nginx-admission
---
# Source: ingress-nginx/templates/admission-webhooks/validating-webhook.yaml
# before changing this value, check the required kubernetes version
# https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#prerequisites
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:labels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhookname: ingress-nginx-admission
webhooks:- name: validate.nginx.ingress.kubernetes.iomatchPolicy: Equivalentrules:- apiGroups:- networking.k8s.ioapiVersions:- v1beta1operations:- CREATE- UPDATEresources:- ingressesfailurePolicy: FailsideEffects: NoneadmissionReviewVersions:- v1- v1beta1clientConfig:service:namespace: ingress-nginxname: ingress-nginx-controller-admissionpath: /networking/v1beta1/ingresses
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:name: ingress-nginx-admissionannotations:helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgradehelm.sh/hook-delete-policy: before-hook-creation,hook-succeededlabels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhooknamespace: ingress-nginx
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:name: ingress-nginx-admissionannotations:helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgradehelm.sh/hook-delete-policy: before-hook-creation,hook-succeededlabels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhook
rules:- apiGroups:- admissionregistration.k8s.ioresources:- validatingwebhookconfigurationsverbs:- get- update
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: ingress-nginx-admissionannotations:helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgradehelm.sh/hook-delete-policy: before-hook-creation,hook-succeededlabels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhook
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: ingress-nginx-admission
subjects:- kind: ServiceAccountname: ingress-nginx-admissionnamespace: ingress-nginx
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:name: ingress-nginx-admissionannotations:helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgradehelm.sh/hook-delete-policy: before-hook-creation,hook-succeededlabels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhooknamespace: ingress-nginx
rules:- apiGroups:- ''resources:- secretsverbs:- get- create
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:name: ingress-nginx-admissionannotations:helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgradehelm.sh/hook-delete-policy: before-hook-creation,hook-succeededlabels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhooknamespace: ingress-nginx
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: ingress-nginx-admission
subjects:- kind: ServiceAccountname: ingress-nginx-admissionnamespace: ingress-nginx
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/job-createSecret.yaml
apiVersion: batch/v1
kind: Job
metadata:name: ingress-nginx-admission-createannotations:helm.sh/hook: pre-install,pre-upgradehelm.sh/hook-delete-policy: before-hook-creation,hook-succeededlabels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhooknamespace: ingress-nginx
spec:template:metadata:name: ingress-nginx-admission-createlabels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhookspec:containers:- name: createimage: docker.io/jettech/kube-webhook-certgen:v1.5.1imagePullPolicy: IfNotPresentargs:- create- --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc- --namespace=$(POD_NAMESPACE)- --secret-name=ingress-nginx-admissionenv:- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespacerestartPolicy: OnFailureserviceAccountName: ingress-nginx-admissionsecurityContext:runAsNonRoot: truerunAsUser: 2000
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/job-patchWebhook.yaml
apiVersion: batch/v1
kind: Job
metadata:name: ingress-nginx-admission-patchannotations:helm.sh/hook: post-install,post-upgradehelm.sh/hook-delete-policy: before-hook-creation,hook-succeededlabels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhooknamespace: ingress-nginx
spec:template:metadata:name: ingress-nginx-admission-patchlabels:helm.sh/chart: ingress-nginx-3.33.0app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/version: 0.47.0app.kubernetes.io/managed-by: Helmapp.kubernetes.io/component: admission-webhookspec:containers:- name: patchimage: docker.io/jettech/kube-webhook-certgen:v1.5.1imagePullPolicy: IfNotPresentargs:- patch- --webhook-name=ingress-nginx-admission- --namespace=$(POD_NAMESPACE)- --patch-mutating=false- --secret-name=ingress-nginx-admission- --patch-failure-policy=Failenv:- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespacerestartPolicy: OnFailureserviceAccountName: ingress-nginx-admissionsecurityContext:runAsNonRoot: truerunAsUser: 2000

耐心等待pod创建完成，所有状态变成running。

开放青云服务器端口 30000-32767

查看 映射后的 IP

kubectl get pod,svc -n ingress-nginx

6.2 访问

每台服务器 都开放 映射后的 端口：
31737、32641

https://xxxxxxxx:32641
http://xxxxxxxxx:31737

6.3 安装不成功的bug解决

查看pod，一直在创建中。

使用describe，查看详细问题日志：

kubectl describe pod ingress-nginx-controller-65bf56f7fc-nwgff -n ingress-nginx

解决办法：



成功启动：

重新查看端口并访问

6.4 测试使用

官网地址：https://kubernetes.github.io/ingress-nginx/
ingress底层其实就是nginx。

6.4.1 搭建测试环境

创建两个 Service和deployment

vi test.yaml# 复制下面kubectl apply -f test.yaml

test.yaml整个文件内容：

apiVersion: apps/v1
kind: Deployment
metadata:name: hello-server
spec:replicas: 2selector:matchLabels:app: hello-servertemplate:metadata:labels:app: hello-serverspec:containers:- name: hello-serverimage: registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/hello-serverports:- containerPort: 9000
---
apiVersion: apps/v1
kind: Deployment
metadata:labels:app: nginx-demoname: nginx-demo
spec:replicas: 2selector:matchLabels:app: nginx-demotemplate:metadata:labels:app: nginx-demospec:containers:- image: nginxname: nginx
---
apiVersion: v1
kind: Service
metadata:labels:app: nginx-demoname: nginx-demo
spec:selector:app: nginx-demoports:- port: 8000protocol: TCPtargetPort: 80
---
apiVersion: v1
kind: Service
metadata:labels:app: hello-servername: hello-server
spec:selector:app: hello-serverports:- port: 8000protocol: TCPtargetPort: 9000

6.4.2 配置 Ingress的规则

vi ingress-rule.yaml# 复制下面配置kubectl apply -f ingress-rule.yaml# 查看 集群中的 Ingress
kubectl get ingress

ingress-rule.yaml完整内容：

apiVersion: networking.k8s.io/v1
kind: Ingress  
metadata:name: ingress-host-bar
spec:ingressClassName: nginxrules:- host: "hello.tigerhhzz.com"http:paths:- pathType: Prefixpath: "/"backend:service:name: hello-serverport:number: 8000 # hello-server （service） 的端口是 8000- host: "demo.tigerhhzz.com"http:paths:- pathType: Prefixpath: "/"  # 把请求会转给下面的服务，下面的服务一定要能处理这个路径，不能处理就是404backend:service:name: nginx-demo  #java，比如使用路径重写，去掉前缀nginxport:number: 8000apiVersion: networking.k8s.io/v1
kind: Ingress  
metadata:name: ingress-host-bar
spec:ingressClassName: nginxrules:- host: "hello.tigerhhzz.com"http:paths:- pathType: Prefixpath: "/"backend:service:name: hello-serverport:number: 8000 # hello-server 的端口是 8000- host: "demo.tigerhhzz.com"http:paths:- pathType: Prefixpath: "/"backend:service:name: nginx-demoport:number: 8000

解决上面错误的方法：

kubectl get ValidatingWebhookConfiguration
# 把该死的 admission 删掉
kubectl delete -A ValidatingWebhookConfiguration ingress-nginx-admission

6.4.3 测试I

在 自己电脑（不是虚拟机） hosts 中增加映射：
master的公网IP hello.tigerhhzz.com
master的公网IP demo.tigerhhzz.com

6.4.4 测试II

# kubectl get ingress
kubectl get ingkubectl edit ing ingress的NAME -n# 改变匹配的 path- host: "demo.tigerhhzz.com"http:paths:- pathType: Prefixpath: "/nginx" # 匹配请求 /nginx 的，并且查找 nginx 文件.backend:service:name: nginx-demoport:number: 8000

随便写 /xxx 不匹配 nginx的，都返回 Ingress的 404的nginx

下面这个是 通过了 Ingress，Service 里的 Pod 没匹配到，才返回的 404（下面打印的 nginx 版本不一样的）

页面 进入 Pod 的那个nginx

cd /usr/share/nginx/html
ls
echo "hello tigerhhzz" > nginx

6.4.5 路径重写

修改ingress配置规则，下面这个这样子 就和 SpringCloud Gateway 网管转发一样的效果了。
ingress-rule.yaml的完整内容如下：

apiVersion: networking.k8s.io/v1
kind: Ingress  
metadata:annotations:nginx.ingress.kubernetes.io/rewrite-target: /$2name: ingress-host-bar
spec:ingressClassName: nginxrules:- host: "hello.tigerhhzz.com"http:paths:- pathType: Prefixpath: "/"backend:service:name: hello-serverport:number: 8000- host: "demo.tigerhhzz.com"http:paths:- pathType: Prefixpath: "/nginx(/|$)(.*)" backend:service:name: nginx-demo  port:number: 8000

在可视化界面 创建

6.4.6 限流

官网文档：https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: ingress-limit-rateannotations:# 限流nginx.ingress.kubernetes.io/limit-rps: "1"
spec:ingressClassName: nginxrules:- host: "haha.tigerhhzz.com"http:paths:- pathType: Exactpath: "/"backend:service:name: nginx-demoport:number: 8000

vim ingress-rule-2.yaml# 复制上面配置kubectl apply -f ingress-rule-2.yamlkubect get ing

在可视化界面 创建ingress


在 自己电脑（不是虚拟机） hosts 中增加映射：
公网IP haha.tigerhhzz.com

访问测试：

http://haha.tigerhhzz.com:30808/

刷新过快 返回 503，官网文档也写了。


速率限制
这些注释定义了连接和传输速率的限制。这些可用于缓解DDoS 攻击

• nginx.ingress.kubernetes.io/limit-connections:允许来自单个IP 地址的并发连接数超过此限制时返回 503 错误
• nginx.ingress.kubernetes.io/limit-rps :每秒从给定IP 接受的请求数。突发限制设置为此限制乘以突发倍数，默认倍数为 5。当客户端超过此限制时，返回limit-reg-status-codedefault: 503
• nginx.ingress,kubernetes.io/limit-rpm:每分钟从给定IP 接受的请求数。突发限制设署为此限制乘以突发倍数，默认倍数为 5。当客户端超过此限制时，返回limit-reg-status-codedefault: 503
• nginx.ingress.kubernetes.io/limit-burst-multiplier:突发大小限制率的乘数。默认突发乘数为5，此注释覆盖默认乘数。当客户端超过此限制时，返回limit-req-status-codedefault: 503
• nginx.ingress.kubernetes.io/limit-rate-after :初始千字节数，之后对给定连接的响应的进一步传输将受到速率限制。此功能必须在启用代理缓冲的情况下使用
• nginx.ingress.kubernetes.io/limit-rate :每秒允许发送到给定连接的千字节数。零值禁用速率限制。此功能必须在启用代理缓冲的情况下使用。
• nginx.ingress.kubernetes.io/limit-whitelist :要从速率限制中排除的客户端IP源范围。该值是一个逗号分隔的 CIDR 列表

如果您在单个Ingress 规则中指定多个注释，则会按顺序应用限制
limit-connections，limit-rpm，limit-rps。

要为所有Ingress 规则全局配置设置，可以在NGINX ConfigMap limit-rate-after
中设置和值.Ingress注解中设置的值将覆盖全局设置。limit-rate

客户端IP 地址将根据PROXY 协议的使用或启用use-forwarded-headers X-Forwarded-For
时的标头值设置。

7. Kubernetes 存储抽象

类似于 Docker 中的 挂载。但要考虑 自愈、故障转移 时的情况

7.1 NFS 搭建

网络文件系统

1、所有节点
安装nfs-utils

# 所有机器执行
yum install -y nfs-utils

2、主节点

# 只在 mster 机器执行：nfs主节点，rw 读写
echo "/nfs/data/ *(insecure,rw,sync,no_root_squash)" > /etc/exportsmkdir -p /nfs/data
systemctl enable rpcbind --now
systemctl enable nfs-server --now# 配置生效
exportfs -r

3、从节点

# 检查，下面的 IP 是master IP
showmount -e xxx.xxx.xxx.xxx# 在 2 个从服务器 执行，执行以下命令挂载 nfs 服务器上的共享目录到本机路径 /root/nfsmount
mkdir -p /nfs/data# 在 2 个从服务器执行，将远程 和本地的 文件夹 挂载
mount -t nfs 139.198.36.162:/nfs/data /nfs/data# 在 master 服务器，写入一个测试文件
echo "hello nfs server" > /nfs/data/test.txt# 在 2 个从服务器查看
cd /nfs/data
ls# 在 从服务器 修改，然后去 其他 服务器 查看，也能 同步

7.2 原生方式 数据挂载

在 /nfs/data/nginx-pv 挂载，然后 修改， 里面 两个 Pod 也会 同步修改。
问题：删掉之后，文件还在，内容也在，是没法管理大小的。

apiVersion: apps/v1
kind: Deployment
metadata:labels:app: nginx-pv-demoname: nginx-pv-demo
spec:replicas: 2selector:matchLabels:app: nginx-pv-demotemplate:metadata:labels:app: nginx-pv-demospec:containers:- image: nginxname: nginxvolumeMounts:- name: htmlmountPath: /usr/share/nginx/html # 挂载目录volumes:# 和 volumeMounts.name 一样- name: htmlnfs:# master IPserver: 192.168.27.251  #master节点ippath: /nfs/data/nginx-pv # 要提前创建好文件夹，否则挂载失败

cd /nfs/data
mkdir -p nginx-pv
lsvi deploy.yaml# 复制上面配置kubectl apply -f deploy.yamlkubectl get pod -owide

cd /nfs/data/
ls
cd nginx-pv/
echo "cgxin" > index.html# 进入 pod 里面查看

问题：占用空间，删掉之后，文件还在，内容也在，是没法管理大小的。

7.3 PV 和 PVC ★

PV：持久卷（Persistent Volume），将应用需要持久化的数据保存到指定位置
PVC：持久卷申明（Persistent Volume Claim），申明需要使用的持久卷规格

挂载目录。ConfigMap 挂载配置文件。
这里是 是 静态的， 就是自己创建好了 容量，然后 PVC 去挑。 还有 动态供应的，不用手动去创建 PV池子。

7.3.1 创建 PV 池

静态供应

# 在 nfs主节点（master服务器） 执行
mkdir -p /nfs/data/01
mkdir -p /nfs/data/02
mkdir -p /nfs/data/03

使用pv.yaml 创建 3个 PV

apiVersion: v1
kind: PersistentVolume
metadata:name: pv01-10m
spec:# 限制容量capacity:storage: 10M# 读写模式：可读可写accessModes:- ReadWriteManystorageClassName: nfsnfs:# 挂载 上面创建过的文件夹path: /nfs/data/01# nfs 主节点服务器的 IPserver: 139.198.36.162
---
apiVersion: v1
kind: PersistentVolume
metadata:# 这个name 要小写，如 Gi 大写就不行name: pv02-1gi
spec:capacity:storage: 1giaccessModes:- ReadWriteManystorageClassName: nfsnfs:path: /nfs/data/02# nfs 主节点服务器的 IPserver: 139.198.36.162
---
apiVersion: v1
kind: PersistentVolume
metadata:name: pv03-3gi
spec:capacity:storage: 3giaccessModes:- ReadWriteManystorageClassName: nfsnfs:path: /nfs/data/03# nfs 主节点服务器的 IPserver: 139.198.36.162

vi pv.yaml# 复制上面文件kubectl apply -f pv.yaml# 查看 pv， kubectl get pv
kubectl get persistentvolume

7.3.2 创建、绑定 PCV

相当于创建设用pv的申请书

创建pvc

pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:name: nginx-pvc
spec:accessModes:- ReadWriteManyresources:requests:# 需要 200M的 PVstorage: 200Mi# 上面 PV 写的什么 这里就写什么    storageClassName: nfs

vi pvc.yaml# 复制上面配置kubectl get pvkubectl apply -f pvc.yamlkubectl get pvkubectl get pvc

绑定了， 绑定了1G的，10M 不够，3G太大，就选择了 1G

7.3.3 创建 Pod 绑定 PVC

创建 Pod，绑定 PVC

apiVersion: apps/v1
kind: Deployment
metadata:labels:app: nginx-deploy-pvcname: nginx-deploy-pvc
spec:replicas: 2selector:matchLabels:app: nginx-deploy-pvctemplate:metadata:labels:app: nginx-deploy-pvcspec:containers:- image: nginxname: nginxvolumeMounts:- name: htmlmountPath: /usr/share/nginx/htmlvolumes:- name: html# 之前是 nfs，这里用 pvcpersistentVolumeClaim:claimName: nginx-pvc

vi dep02.yaml# 复制上面 yamlkubectl apply -f dep02.yamlkubectl get podkubectl get pvkubectl get pvc

挂载后，测试


进入 Pod 内部查看 同步的文件

7.4 ConfigMap ★

ConfigMap：抽取应用配置，并且可以自动更新。挂载配置文件， PV 和 PVC 是挂载目录的。

7.4.1 redis示例

1. 创建 ConfigMap
   创建 / 删除 cm

vi redis.conf
# 写
appendonly yes# 创建配置，redis保存到k8s的etcd；
kubectl create cm redis-conf --from-file=redis.conf# 查看
kubectl get cmrm -rf redis.conf

# 查看 ConfigMap 的 yaml 配置咋写的
kubectl get cm redis-conf -oyaml

apiVersion: v1
data:    # data是所有真正的数据，key：默认是文件名   value：配置文件的内容（appendonly yes 是随便写的）redis.conf: |appendonly yes
kind: ConfigMap
metadata:name: redis-confnamespace: default

2. 创建 Pod
   redis.yaml

apiVersion: v1
kind: Pod
metadata:name: redis
spec:containers:- name: redisimage: rediscommand:# 启动命令- redis-server# 指的是redis容器内部的位置- "/redis-master/redis.conf"  ports:- containerPort: 6379volumeMounts:- mountPath: /dataname: data- mountPath: /redis-mastername: configvolumes:- name: dataemptyDir: {}- name: configconfigMap:name: redis-confitems:- key: redis.confpath: redis.conf

redis.conf 会放在 /redis-master 下

vi redis.yaml# 复制上面配置kubectl apply -f redis.yamlkubectl get pod

页面中 进入刚才创建的 pod redis 内部

查看 redis.conf 配置文件 内容

kubectl get cm# 修改配置 里 redis.conf 的内容
kubectl edit cm redis-conf

修改 redis-conf 的 redis.conf 内容

修改 redis-conf 的 redis.conf 内容

过了一会， 就同步了

3. 检查默认配置

kubectl exec -it redis -- redis-cli127.0.0.1:6379> CONFIG GET appendonly
127.0.0.1:6379> CONFIG GET requirepass

和 命令行一样的

删除，重新创建 Pod，更新 配置文件的 配置值

查看 更新的 配置值

总结：
● 修改了 ConfigMap，Pod里面的配置文件会跟着同步。
● 但配置值 未更改，需要重新启动 Pod 才能从关联的ConfigMap 中获取 更新的值。 Pod 部署的中间件 自己本身没有热更新能力。

7.5 Secret

Secret ：是对象类型，用来保存敏感信息，例如密码、OAuth 令牌和 SSH 密钥。 将这些信息放在 secret 中比放在 Pod 的定义或者 容器镜像 中来说更加安全和灵活。

7.5.1 拉取失败

Docker hub 仓库中，自己的仓库设置成私有的。 然后去 下载私有的。下载不了（未登录）。

自己的仓库设置成私有的
查看 拉取命令

拒绝拉取

mypod.yaml

apiVersion: v1
kind: Pod
metadata:name: private-cgxin-docker
spec:containers:- name: private-cgxin-dockerimage: cgxin/cgxin_docker:1.0

vi mypod.yaml# 复制上面配置kubectl apply -f mypod.yamlkubectl get pod

提示 镜像拉取失败

可视化界面 查看错误描述：也是没有权限。

删除配置文件 创建的错误 Pod

7.5.2 创建 Secret

创建 Secret

kubectl create secret docker-registry cgxin-docker-secret \
--docker-username=leifengyang \
--docker-password=Lfy123456 \
--docker-email=534096094@qq.com##命令格式
kubectl create secret docker-registry regcred \--docker-server=<你的镜像仓库服务器> \--docker-username=<你的用户名> \--docker-password=<你的密码> \--docker-email=<你的邮箱地址>

# 查看
kubectl get secretkubectl get secret cgxin-docker-secret -oyaml

重新修改 配置文件，加入 Secret

apiVersion: v1
kind: Pod
metadata:name: private-cgxin-docker
spec:containers:- name: private-cgxin-dockerimage: cgxin/cgxin_docker:1.0# 加上 Secret  imagePullSecrets:- name: cgxin-docker-secret

vi mypod.yaml# 复制上面配置kubectl apply -f mypod.yamlkubectl get pod

使用 Secret 后，可以成功 拉取下来了。

Docker Hub 镜像 复原成 public

总结：
可视化界面 操作很方便