基于External-DNS的多集群Ingress DNS实践

概要

External-DNS提供了编程方式管理Kubernetes Ingress资源的DNS的功能，方便用户从Ingress管理DNS解析记录。而在kubernetes federation v2环境中，使用External-DNS可以快速的管理多个联邦集群的Ingress DNS解析，降低用户的操作成本。下面将简单介绍在阿里云容器服务环境中，如何使用External-DNS管理联邦集群的Ingress DNS解析。

联邦集群准备

参考阿里云Kubernetes容器服务上体验Federation v2 搭建两个集群组成的联邦集群（配置好kubeconfig，并完成两个集群的join）。

配置RAM信息

选择Kubernetes集群节点列表内任意一个Worker节点，打开对应的节点列表信息页面。

找到对应的 RAM 角色，打开RAM控制台，找到对应的角色名称，添加【AliyunDNSFullAccess】权限。

注意：每个集群都需要配置RAM信息。

部署External-DNS

配置RBAC

执行下面yaml：

apiVersion: v1
kind: ServiceAccount
metadata:name: external-dns
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:name: external-dns
rules:
- apiGroups: [""]resources: ["services"]verbs: ["get","watch","list"]
- apiGroups: [""]resources: ["pods"]verbs: ["get","watch","list"]
- apiGroups: ["extensions"]resources: ["ingresses"]verbs: ["get","watch","list"]
- apiGroups: [""]resources: ["nodes"]verbs: ["list"]
- apiGroups: ["multiclusterdns.federation.k8s.io"]resources: ["dnsendpoints"]verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:name: external-dns-viewer
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: external-dns
subjects:
- kind: ServiceAccountname: external-dnsnamespace: default

部署External-DNS服务

执行下面yaml：

apiVersion: extensions/v1beta1
kind: Deployment
metadata:name: external-dns
spec:strategy:type: Recreatetemplate:metadata:labels:app: external-dnsspec:serviceAccountName: external-dnscontainers:- name: external-dnsimage: registry.cn-beijing.aliyuncs.com/acs/external-dns:v0.5.8-27args:- --source=crd- --crd-source-apiversion=multiclusterdns.federation.k8s.io/v1alpha1- --crd-source-kind=DNSEndpoint- --provider=alibabacloud- --policy=sync # enable full synchronization- --registry=txt- --txt-prefix=cname- --txt-owner-id=my-identifier- --alibaba-cloud-config-file= # enable sts tokenvolumeMounts:- mountPath: /usr/share/zoneinfoname: hostpathvolumes:- name: hostpathhostPath:path: /usr/share/zoneinfotype: Directory

部署验证资源

创建FederatedDeployment和FederatedService：

apiVersion: v1
kind: Namespace
metadata:name: test-namespace---apiVersion: types.federation.k8s.io/v1alpha1
kind: FederatedNamespace
metadata:name: test-namespacenamespace: test-namespace
spec:placement:clusterNames:- cluster1- cluster2---apiVersion: types.federation.k8s.io/v1alpha1
kind: FederatedDeployment
metadata:name: test-deploymentnamespace: test-namespace
spec:template:metadata:labels:app: nginxspec:replicas: 2selector:matchLabels:app: nginxtemplate:metadata:labels:app: nginxspec:containers:- image: nginxname: nginxresources:limits:cpu: 500mrequests:cpu: 200mplacement:clusterNames:- cluster1- cluster2---apiVersion: types.federation.k8s.io/v1alpha1
kind: FederatedService
metadata:name: test-servicenamespace: test-namespace
spec:template:spec:selector:app: nginxtype: ClusterIPports:- name: httpport: 80placement:clusterNames:- cluster2- cluster1

各个集群ingress创建信息如下：

kubectl get ingress -n test-namespace --context cluster1
NAME           HOSTS   ADDRESS        PORTS   AGE
test-ingress   *       47.93.69.121   80      54mkubectl get ingress -n test-namespace --context cluster2
NAME           HOSTS   ADDRESS         PORTS   AGE
test-ingress   *       39.106.232.23   80      54m

创建FederatedIngress和IngressDNSRecord

apiVersion: types.federation.k8s.io/v1alpha1
kind: FederatedIngress
metadata:name: test-ingressnamespace: test-namespace
spec:template:spec:backend:serviceName: test-serviceservicePort: 80placement:clusterNames:- cluster2- cluster1        ---apiVersion: multiclusterdns.federation.k8s.io/v1alpha1
kind: IngressDNSRecord
metadata:name: test-ingressnamespace: test-namespace
spec:hosts:- ingress-example.example-domain.clubrecordTTL: 600

其中【ingress-example.example-domain.club】为测试阿里云托管的域名，请提前在阿里云上购买域名，并注意替换。

DNS解析验证

dig +short @dns7.hichina.com ingress-example.example-domain.club
47.93.69.121
39.106.232.23

可以看到我们绑定的域名已经解析到了cluster1和cluster2的ingress IP上了。
访问域名相应的服务：

curl ingress-example.sigma-host.club
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>body {width: 35em;margin: 0 auto;font-family: Tahoma, Verdana, Arial, sans-serif;}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p><p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p><p><em>Thank you for using nginx.</em></p>
</body>
</html>