K8s: Ingress对象, 创建Ingress控制器, 创建Ingress资源并暴露服务

Ingress对象

1 ）概述

Ingress 是对集群中服务的外部访问进行管理的 API 对象，典型的访问方式是 HTTP
Ingress-nginx 本质是网关，当你请求 abc.com/service/a, Ingress 就把对应的地址转发给你，底层运行了一个 nginx
但 K8s 为什么不直接使用 nginx 呢，是因为 K8s 也需要把转发的路由规则纳入它的配置管理
变成 ingress 对象，所有才有 ingress 这个资源对象, Ingress 公开了从集群外部到集群内服务的 HTTP 和 HTTPS 路由
流量路由由 Ingress 资源上定义的规则控制
所以，它的功能类似 Nginx，可以根据域名、路径把请求转发到不同的 Service
Ingress 为外部访问集群提供了一个统一入口，避免了对外暴露集群端口，也可以配置 https

2 ）示例图

下面是一个将所有流量都发送到同一 Service 的简单 Ingress 示例

在 Service 层已经可以对外提供服务了，但是
在后端 Service 安全权限非常高的情况下，直连 Service 层风险非常大
从客户端里，通过Ingress的controller调度到Ingress服务，Ingress 可以理解为一个反向代理服务
这样，避免了直连Service层的风险，所以，Ingress 也类似于网关层，调度到Service之后
再由底层调度到相关的 Pod 中访问对应的服务
Ingress 有两种实践方法
- 一种是, Ingress Nginx 实现，在Nginx官方中有相关说明
- 另一种就是在 K8s 中的实践
对于典型生产环境来说，有上图这样一套调用链
可以将 Ingress 配置为服务提供外部可访问的 URL、负载均衡流量、终止 SSL/TLS，以及提供基于名称的虚拟主机等能力
Ingress控制器通常负责通过负载均衡器来实现 Ingress

3 ）最小 Ingress 资源示例

定义 ing-min.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: minimal-ingressannotations:nginx.ingress.kubernetes.io/rewrite-target: /
spec:rules:- http:   # 除了 http 还可以定义其他路由规则paths:  # 这个名称意味着可以定义多个 path- path: /testpathpathType: Prefixbackend:service:name: testport:number: 80

基于以上的配置定义，客户端可以通过比如 xxx.com/testpath 请求
通过这个请求，会被 Ingress 捕获，根据这个请求规则，会匹配后端的 backend service
这个 service 名称就是 k8s 中的 service 名称，下面是对应的端口号
通过这个转发，类似于 nginx，实现路由规则的http转发
关于 Ingress 规则，每个 HTTP 规则都包含以下信息
- 1 ）可选的 host
  - 在此示例中，未指定 host，因此该规则适用于通过指定 IP 地址的所有入站 HTTP 通信
  - 如果提供了 host（例如 foo.bar.com），则 rules 适用于该 host
- 2 ）路径列表 paths（例如，/testpath）
  - 每个路径都有一个由 serviceName 和 servicePort 定义的关联后端
  - 在负载均衡器将流量定向到引用的服务之前，主机和路径都必须匹配传入请求的内容
- 3 ）backend（后端）
  - 是 Service 文档中所述的服务和端口名称的组合
  - 与规则的 host 和 path 匹配的对 Ingress 的 HTTP（和 HTTPS ）请求将发送到列出的 backend

4 ）Ingress 控制器

关于 Ingress 控制器
- 为了让 Ingress 资源工作，集群必须有一个正在运行的 Ingress 控制器
- 与其他类型的控制器不同，Ingress 控制器不是随集群自动启动的
版本对应
- 介于之前试错的经验，在各个版本的K8s上部署不同的yaml配置，会导致各种不一样的报错，
- 我在官方github上找到这个对应的版本信息，如下
  - https://github.com/kubernetes/ingress-nginx
  - 目前我的K8s的版本是1.22.4，所以这个控制器最高可以选择版本 v1.4.0
  - https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.4.0/deploy/static/provider/cloud/deploy.yaml
  - 这个文件下载下来后，需要做一些修改
  - 注意：如果上述github无法访问，可以找gitee中对应的镜像里的对应的版本

安装 Ingress 控制器

这里创建一个 ing-nginx-ctrl.yaml 文件

和上面官方不同的几点是:

在第一个Service中找到 spec 下
- externalTrafficPolicy: Local 修改为 externalTrafficPolicy: Cluster
- 并在这个配置的上面添加一行: clusterIP: 10.1.211.240
- 在 name: http 下添加一行 nodePort: 31686
- 在 name: https 下添加一行 ``
- 找到 type: LoadBalancer 修改为 type: NodePort
替换通用镜像
- 先找到 image: registry.k8s.io/ingress-nginx/controller:v1.4.0@sha256:34ee929b111ffc7aa426ffd409af44da48e5a0eea1eb2207994d9e0c0882d143
- 修改为: image: registry.aliyuncs.com/google_containers/nginx-ingress-controller:v1.4.0
- 再找到 image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20220916-gd32f8c343@sha256:39c5b2e3310dc4264d638ad28d9d1d96c4cbb2b2dcfb52368fe4e3c63f61e10f
- 修改为: image: registry.aliyuncs.com/google_containers/kube-webhook-certgen:v1.4.0
- 注意，这些镜像可以先拉到本地
  - $ sudo docker pull registry.aliyuncs.com/google_containers/nginx-ingress-controller:v1.4.0
  - $ sudo docker pull registry.aliyuncs.com/google_containers/kube-webhook-certgen:v1.4.0

修改后的 ing-nginx-ctrl.yaml 文件内容如下

apiVersion: v1
kind: Namespace
metadata:labels:app.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxname: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginxnamespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginx-admissionnamespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginxnamespace: ingress-nginx
rules:
- apiGroups:- ""resources:- namespacesverbs:- get
- apiGroups:- ""resources:- configmaps- pods- secrets- endpointsverbs:- get- list- watch
- apiGroups:- ""resources:- servicesverbs:- get- list- watch
- apiGroups:- networking.k8s.ioresources:- ingressesverbs:- get- list- watch
- apiGroups:- networking.k8s.ioresources:- ingresses/statusverbs:- update
- apiGroups:- networking.k8s.ioresources:- ingressclassesverbs:- get- list- watch
- apiGroups:- ""resourceNames:- ingress-controller-leaderresources:- configmapsverbs:- get- update
- apiGroups:- ""resources:- configmapsverbs:- create
- apiGroups:- coordination.k8s.ioresourceNames:- ingress-controller-leaderresources:- leasesverbs:- get- update
- apiGroups:- coordination.k8s.ioresources:- leasesverbs:- create
- apiGroups:- ""resources:- eventsverbs:- create- patch
- apiGroups:- discovery.k8s.ioresources:- endpointslicesverbs:- list- watch- get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginx-admissionnamespace: ingress-nginx
rules:
- apiGroups:- ""resources:- secretsverbs:- get- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:app.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginx
rules:
- apiGroups:- ""resources:- configmaps- endpoints- nodes- pods- secrets- namespacesverbs:- list- watch
- apiGroups:- coordination.k8s.ioresources:- leasesverbs:- list- watch
- apiGroups:- ""resources:- nodesverbs:- get
- apiGroups:- ""resources:- servicesverbs:- get- list- watch
- apiGroups:- networking.k8s.ioresources:- ingressesverbs:- get- list- watch
- apiGroups:- ""resources:- eventsverbs:- create- patch
- apiGroups:- networking.k8s.ioresources:- ingresses/statusverbs:- update
- apiGroups:- networking.k8s.ioresources:- ingressclassesverbs:- get- list- watch
- apiGroups:- discovery.k8s.ioresources:- endpointslicesverbs:- list- watch- get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginx-admission
rules:
- apiGroups:- admissionregistration.k8s.ioresources:- validatingwebhookconfigurationsverbs:- get- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginxnamespace: ingress-nginx
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: ingress-nginx
subjects:
- kind: ServiceAccountname: ingress-nginxnamespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginx-admissionnamespace: ingress-nginx
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: ingress-nginx-admission
subjects:
- kind: ServiceAccountname: ingress-nginx-admissionnamespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:labels:app.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginx
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: ingress-nginx
subjects:
- kind: ServiceAccountname: ingress-nginxnamespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginx-admission
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: ingress-nginx-admission
subjects:
- kind: ServiceAccountname: ingress-nginx-admissionnamespace: ingress-nginx
---
apiVersion: v1
data:allow-snippet-annotations: "true"
kind: ConfigMap
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginx-controllernamespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginx-controllernamespace: ingress-nginx
spec:clusterIP: 10.1.211.240externalTrafficPolicy: ClusteripFamilies:- IPv4ipFamilyPolicy: SingleStackports:- appProtocol: httpname: httpnodePort: 31686port: 80protocol: TCPtargetPort: http- appProtocol: httpsname: httpsnodePort: 30036port: 443protocol: TCPtargetPort: httpsselector:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxtype: NodePort
---
apiVersion: v1
kind: Service
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginx-controller-admissionnamespace: ingress-nginx
spec:ports:- appProtocol: httpsname: https-webhookport: 443targetPort: webhookselector:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxtype: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginx-controllernamespace: ingress-nginx
spec:minReadySeconds: 0revisionHistoryLimit: 10selector:matchLabels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxtemplate:metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxspec:containers:- args:- /nginx-ingress-controller- --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller- --election-id=ingress-controller-leader- --controller-class=k8s.io/ingress-nginx- --ingress-class=nginx- --configmap=$(POD_NAMESPACE)/ingress-nginx-controller- --validating-webhook=:8443- --validating-webhook-certificate=/usr/local/certificates/cert- --validating-webhook-key=/usr/local/certificates/keyenv:- name: POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: LD_PRELOADvalue: /usr/local/lib/libmimalloc.soimage: registry.aliyuncs.com/google_containers/nginx-ingress-controller:v1.4.0imagePullPolicy: IfNotPresentlifecycle:preStop:exec:command:- /wait-shutdownlivenessProbe:failureThreshold: 5httpGet:path: /healthzport: 10254scheme: HTTPinitialDelaySeconds: 10periodSeconds: 10successThreshold: 1timeoutSeconds: 1name: controllerports:- containerPort: 80name: httpprotocol: TCP- containerPort: 443name: httpsprotocol: TCP- containerPort: 8443name: webhookprotocol: TCPreadinessProbe:failureThreshold: 3httpGet:path: /healthzport: 10254scheme: HTTPinitialDelaySeconds: 10periodSeconds: 10successThreshold: 1timeoutSeconds: 1resources:requests:cpu: 100mmemory: 90MisecurityContext:allowPrivilegeEscalation: truecapabilities:add:- NET_BIND_SERVICEdrop:- ALLrunAsUser: 101volumeMounts:- mountPath: /usr/local/certificates/name: webhook-certreadOnly: truednsPolicy: ClusterFirstnodeSelector:kubernetes.io/os: linuxserviceAccountName: ingress-nginxterminationGracePeriodSeconds: 300volumes:- name: webhook-certsecret:secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginx-admission-createnamespace: ingress-nginx
spec:template:metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginx-admission-createspec:containers:- args:- create- --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc- --namespace=$(POD_NAMESPACE)- --secret-name=ingress-nginx-admissionenv:- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespaceimage: registry.aliyuncs.com/google_containers/kube-webhook-certgen:v1.4.0imagePullPolicy: IfNotPresentname: createsecurityContext:allowPrivilegeEscalation: falsenodeSelector:kubernetes.io/os: linuxrestartPolicy: OnFailuresecurityContext:fsGroup: 2000runAsNonRoot: truerunAsUser: 2000serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginx-admission-patchnamespace: ingress-nginx
spec:template:metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginx-admission-patchspec:containers:- args:- patch- --webhook-name=ingress-nginx-admission- --namespace=$(POD_NAMESPACE)- --patch-mutating=false- --secret-name=ingress-nginx-admission- --patch-failure-policy=Failenv:- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespaceimage: registry.aliyuncs.com/google_containers/kube-webhook-certgen:v1.4.0imagePullPolicy: IfNotPresentname: patchsecurityContext:allowPrivilegeEscalation: falsenodeSelector:kubernetes.io/os: linuxrestartPolicy: OnFailuresecurityContext:fsGroup: 2000runAsNonRoot: truerunAsUser: 2000serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: nginx
spec:controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:- v1clientConfig:service:name: ingress-nginx-controller-admissionnamespace: ingress-nginxpath: /networking/v1/ingressesfailurePolicy: FailmatchPolicy: Equivalentname: validate.nginx.ingress.kubernetes.iorules:- apiGroups:- networking.k8s.ioapiVersions:- v1operations:- CREATE- UPDATEresources:- ingressessideEffects: None

简单来说 ingress controller 实际在系统里面创建一系列的pod
本质上就是运行在 K8s服务器上的一系列的 pod, 通过 pod 来接管
外部到 K8s work node 上的请求，所以，它就是类似于 nginx 的组件

$ kubectl apply -f ing-nginx-ctrl.yaml

namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created

$ kubectl get all -n ingress-nginx 查看命名空间下的所有信息

  NAME                                            READY   STATUS      RESTARTS   AGE
pod/ingress-nginx-admission-create--1-8nbrv     0/1     Completed   0          65s
pod/ingress-nginx-admission-patch--1-2q9x9      0/1     Completed   3          65s
pod/ingress-nginx-controller-6747799754-v2vhq   1/1     Running     0          65sNAME                                         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
service/ingress-nginx-controller             NodePort    10.1.211.240   <none>        80:31686/TCP,443:30036/TCP   65s
service/ingress-nginx-controller-admission   ClusterIP   10.1.195.73    <none>        443/TCP                      65sNAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/ingress-nginx-controller   1/1     1            1           65sNAME                                                  DESIRED   CURRENT   READY   AGE
replicaset.apps/ingress-nginx-controller-6747799754   1         1         1       65sNAME                                       COMPLETIONS   DURATION   AGE
job.batch/ingress-nginx-admission-create   1/1           21s        65s
job.batch/ingress-nginx-admission-patch    1/1           44s        65s

这里，发现namespace为ingress-nginx的三个pod已经成功完成
status为Completed的两个pod为job类型资源，Completed表示job已经成功执行
status为Running的pod就是控制器

有了这样的一个组件在K8s平台运行起来之后，可以检查部署版本，粘贴如下
- $ POD_NAMESPACE=ingress-nginx
- $ POD_NAME=$(kubectl get pods -n $POD_NAMESPACE -l app.kubernetes.io/name=ingress-nginx --field-selector=status.phase=Running -o jsonpath='{.items[0].metadata.name}')
- $ kubectl exec -it $POD_NAME -n $POD_NAMESPACE -- /nginx-ingress-controller --version
```
-------------------------------------------------------------------------------
NGINX Ingress controllerRelease:       v1.4.0Build:         50be2bf95fd1ef480420e2aa1d6c5c7c138c95eaRepository:    https://github.com/kubernetes/ingress-nginxnginx version: nginx/1.19.10-------------------------------------------------------------------------------
```

$ kubectl get svc -n ingress-nginx 查看可用Services

NAME                                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.1.211.240   <none>        80:31686/TCP,443:30036/TCP   9m49s
ingress-nginx-controller-admission   ClusterIP   10.1.195.73    <none>        443/TCP                      9m49s

到现在为止，服务已经搭建起来了，我们来验证一下
- $ curl node1.k8s:31686 或 curl node2.k8s:31686
- 说明: node1.k8s 或 node2.k8s 是可用的work node, 本地配置了 hosts，才可这样访问
- 如果结果显示如下，则表示服务已经通了
```
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
```
综上，ingress 的控制器已经搭建完毕

5 ）基于 ingress 控制器创建 ingress 资源，并对外暴露服务

在创建 ingress 资源之前，先部署我们的后端应用服务，这里做最简单的示例
- $ kubectl create deployment web --image=registry.cn-beijing.aliyuncs.com/qingfeng666/hello-app:1.0 基于 development 维护一个pod
```
deployment.apps/web created
```
- $ kubectl get po -w 监控pod的状态，等待 Running
```
NAME                   READY   STATUS    RESTARTS   AGE
web-6db77f5fdb-qkk6n   1/1     Running   0          7s
```
- $ kubectl expose deployment web --type=NodePort --port=8080 将 development 服务暴露出来
```
service/web exposed
```
- $ kubectl get svc 获取目前的服务
```
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)          AGE
kubernetes   ClusterIP   10.1.0.1     <none>        443/TCP          5d8h
web          NodePort    10.1.47.34   <none>        8080:32041/TCP   8s
```
- $ curl node1.k8s:32041 或 curl node2.k8s:32041
```
 Hello, world!Version: 1.0.0Hostname: web-6db77f5fdb-65wfv
```
  - 可见，在集群内部，我们的服务已经启动起来了
- 现在内部pod和Service已经就绪，现在可以进行创建 ingress 资源了
- $ vi ing-demo.yaml
```
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: ingress-nginxannotations:nginx.ingress.kubernetes.io/rewrite-target: /
spec:ingressClassName: nginxrules:- host: hello-world.infohttp:paths:- path: /pathType: Prefixbackend:service:name: webport:number: 8080
```
- $ kubectl apply -f ing-demo.yaml 创建 ingress 资源
```
ingress.networking.k8s.io/ingress-nginx created
```
- $ kubectl get ing 查看 ingress 资源
```
NAME            CLASS   HOSTS              ADDRESS        PORTS   AGE
ingress-nginx   nginx   hello-world.info   10.1.211.240   80      2m13s
```
- $ sudo vi /etc/hosts 添加一行, 对当前ip进行域名的配置
```
10.1.211.240  hello-world.info
```
- $ curl hello-world.info 访问域名，发现通了
```
Hello, world!
Version: 1.0.0
Hostname: web-6db77f5fdb-65wfv
```
- 这样，就完成了集群外的暴露，但是还需要再客户端机器或云服务器的域名解析，这里选择前者
  - 比如，在我的Mac电脑上连接当前 hello-world服务，这里前提是: Mac电脑和Centos可以连通
  - 在 Mac 上配置某个 Centos 的work node的host, $ sudo vi /etc/hosts
```
10.211.55.11  hello-world.info
```
  - 这里的 10.211.55.11 对应 work node 的 ip
- 在我的 Mac 上浏览器访问: http://hello-world.info:31686，如下
  - 像是这种访问不方便: http://hello-world.info:31686 这个端口比较麻烦
  - 可以修改成 80端口, 这样，就可以这样访问了：http://hello-world.info, 这里不演示了，参考如下
  - 参考: https://blog.csdn.net/qq_32060101/article/details/135691179
    - k8s修改NodePort支持80端口
  - 参考: https://blog.csdn.net/qq_32060101/article/details/135691441
    - ingress控制器修改NodePort成80端口