在 Kubernetes 中,Pod 资源的控制器 Deployment、Replicaset、Daemonset 等常用于管理无状态应用,它们所管理的 Pod 对应的 IP、名字,启停顺序等都是随机的,Pod 之间也并不存在任何关联关系。而实际情况下,在应用集群部署时,实例彼此之间可能是需要存在关联关系的(启动顺序、角色),如 MySQL、MongoDB,所以 StatefulSet 就是为了运行有状态服务引入的一种资源类型,StatefulSet 为每个 Pod 维持一个唯一且固定的标识符,必要时还会为其创建专用的存储卷,当 Pod 被重建时,也依然能保持原来的标识符和存储卷。
完整的 StatefulSet 通常由三部分构成:StatefulSet
、VolumeClaimTemplate
、Headless Service
。
StatefulSet
用于 Pod 资源定义与管控,在 StatefulSet 模式下,Pod 有自己固定的命名规则(StatfulSet 名称 + Pod 创建时所在的索引),假设设置的 StatefulSet 名称为k8sdemo
,replicas 为3,则对应的 Pod 名称将分别是k8sdemo-0
、k8sdemo-1
、k8sdemo-0
,同时在进行 Pod 副本伸缩时也能做到按序号进行升降。
VolumeClaimTemplate
用于定义 Pod 所需存储的 PVC 声明 ,PVC 与 PV 进行绑定,提供专有固定的存储卷。
Headless Service
(clusterIP: None)用于为 Pod 生成可解析的 DNS 域名记录,基于 Pod 名称的有序规则,Pod 域名是不会变的(Pod 名称.serviceName),这也保证了 Pod 网络标识的稳定性。
下面继续以 .NET Core 项目构建的 beckjin/k8sdemo:1.2.0
镜像为例,增加了接口访问日志记录的功能。通过集成 log4net 将接口访问日志进行文件记录,日志将输出到 /Data/ 目录,每个 Pod 都会拥有自己的一份日志文件(这只是一个假设的场景,切勿较真,实际情况下日志记录一般都会使用统一的日志采集工具)。
定义资源
k8sdemo-statefulset.yaml
:
apiVersion: apps/v1
kind: StatefulSet
metadata:name: k8sdemo
spec:serviceName: "k8sdemo-service" # 需要与创建的 service name 一致replicas: 3selector:matchLabels:name: k8sdemotemplate:metadata:labels:name: k8sdemospec:containers:- name: k8sdemoimage: beckjin/k8sdemo:1.2.0imagePullPolicy: IfNotPresentvolumeMounts:- name: datamountPath: /app/Data # 将容器内的 Data 目录进行挂载volumeClaimTemplates: # 定义模板,自动创建 PVC- metadata:name: dataspec:accessModes:- ReadOnlyManyresources:requests:storage: 100MistorageClassName: "k8sdemo-sc" # 将自动与集群内 storageClassName 匹配的 PV 进行绑定
k8sdemo-service.yaml
:
apiVersion: v1
kind: Service
metadata:name: k8sdemo-service
spec:clusterIP: Noneports:- port: 80targetPort: 80selector:name: k8sdemo
StatefulSet 模式下需要设置 serviceName
字段,用来告诉 StatefulSet 控制器具体使用哪个 service 来解析它所管理的 Pod。同时通过 volumeClaimTemplates
字段进行 PVC 定义,StatefulSet 控制器会自动创建与 Pod 对应的 PVC,PVC 的名称为 (volumeClaimTemplateName)-(podName),然后 PVC 会自动与满足要求的 PV 进行绑定,PV 如果不支持自动创建可手动完成。另外当 Pod 被删除时 PVC 与 PV 依然会被保留,Pod 重建时会重新关联之前对应的 PVC 与 PV。
这里还是使用的 NFS 创建 PV 来实现存储,分别创建 3 个(data-k8sdemo-pv-[1~3])满足定义要求的 PV,如下:
apiVersion: v1
kind: PersistentVolume
metadata:name: data-k8sdemo-pv-1
spec:nfs:server: 192.168.124.21path: /statefulset/data1accessModes:- ReadOnlyManycapacity:storage: 100MistorageClassName: k8sdemo-sc
部署与测试
创建 PV 与 StatefulSet:
kubectl apply -f k8sdemo-statefulset-pv1.yaml
kubectl apply -f k8sdemo-statefulset-pv2.yaml
kubectl apply -f k8sdemo-statefulset-pv3.yaml
kubectl apply -f k8sdemo-statefulset.yaml
注意::PV 命名顺序并不代表被 PVC 的绑定顺序,这两者没有关系,所以不用对上图的数字编号对应关系有疑问。
创建 Service:
kubectl apply -f k8sdemo-service.yaml
因为 Service 定义的是 Headless 模式,所以需要进去 Pod 内进行接口访问测试,如:kubectl exec -it k8sdemo-0 bash
进入 k8sdemo-0 这个 Pod,通过域名 Pod 名称.serviceName 来访问,如下:
curl k8sdemo-0.k8sdemo-service/weatherforecast
curl k8sdemo-1.k8sdemo-service/weatherforecast
curl k8sdemo-2.k8sdemo-service/weatherforecast
在 NFS 挂载目录中查看接口访问日志,以下是 Pod k8sdemo-1 中的日志:
2020-09-20 06:01:17,451 [17] INFO [k8sdemo-1] - Request starting HTTP/1.1 GET http://k8sdemo-1.k8sdemo-service/weatherforecast
2020-09-20 06:01:17,455 [17] INFO [k8sdemo-1] - Executing endpoint 'T.K8SDemo.Controllers.WeatherForecastController.Get (T.K8SDemo)'
2020-09-20 06:01:17,458 [17] INFO [k8sdemo-1] - Route matched with {action = "Get", controller = "WeatherForecast"}. Executing controller action with signature System.Collections.Generic.IEnumerable`1[T.K8SDemo.WeatherForecast] Get() on controller T.K8SDemo.Controllers.WeatherForecastController (T.K8SDemo).
2020-09-20 06:01:17,459 [17] INFO [k8sdemo-1] - Executing ObjectResult, writing value of type 'T.K8SDemo.WeatherForecast[]'.
2020-09-20 06:01:17,460 [17] INFO [k8sdemo-1] - Executed action T.K8SDemo.Controllers.WeatherForecastController.Get (T.K8SDemo) in 2.3627ms
2020-09-20 06:01:17,460 [17] INFO [k8sdemo-1] - Executed endpoint 'T.K8SDemo.Controllers.WeatherForecastController.Get (T.K8SDemo)'
2020-09-20 06:01:17,461 [17] INFO [k8sdemo-1] - Request finished in 9.9194ms 200 application/json; charset=utf-8
执行 kubectl delete pod k8sdemo-1
删除 Pod k8sdemo-1,等待一会 k8sdemo-1 会自动恢复,然后重新访问 curl k8sdemo-1.k8sdemo-service/weatherforecast
,日志依然向原来的文件内追加,也说明保留了原来的状态。
2020-09-20 06:01:17,451 [17] INFO [k8sdemo-1] - Request starting HTTP/1.1 GET http://k8sdemo-1.k8sdemo-service/weatherforecast
2020-09-20 06:01:17,455 [17] INFO [k8sdemo-1] - Executing endpoint 'T.K8SDemo.Controllers.WeatherForecastController.Get (T.K8SDemo)'
2020-09-20 06:01:17,458 [17] INFO [k8sdemo-1] - Route matched with {action = "Get", controller = "WeatherForecast"}. Executing controller action with signature System.Collections.Generic.IEnumerable`1[T.K8SDemo.WeatherForecast] Get() on controller T.K8SDemo.Controllers.WeatherForecastController (T.K8SDemo).
2020-09-20 06:01:17,459 [17] INFO [k8sdemo-1] - Executing ObjectResult, writing value of type 'T.K8SDemo.WeatherForecast[]'.
2020-09-20 06:01:17,460 [17] INFO [k8sdemo-1] - Executed action T.K8SDemo.Controllers.WeatherForecastController.Get (T.K8SDemo) in 2.3627ms
2020-09-20 06:01:17,460 [17] INFO [k8sdemo-1] - Executed endpoint 'T.K8SDemo.Controllers.WeatherForecastController.Get (T.K8SDemo)'
2020-09-20 06:01:17,461 [17] INFO [k8sdemo-1] - Request finished in 9.9194ms 200 application/json; charset=utf-8
2020-09-20 06:17:06,467 [12] INFO [k8sdemo-1] - Request starting HTTP/1.1 GET http://k8sdemo-1.k8sdemo-service/weatherforecast
2020-09-20 06:17:06,494 [12] INFO [k8sdemo-1] - Executing endpoint 'T.K8SDemo.Controllers.WeatherForecastController.Get (T.K8SDemo)'
2020-09-20 06:17:06,527 [12] INFO [k8sdemo-1] - Route matched with {action = "Get", controller = "WeatherForecast"}. Executing controller action with signature System.Collections.Generic.IEnumerable`1[T.K8SDemo.WeatherForecast] Get() on controller T.K8SDemo.Controllers.WeatherForecastController (T.K8SDemo).
2020-09-20 06:17:06,533 [12] INFO [k8sdemo-1] - Executing ObjectResult, writing value of type 'T.K8SDemo.WeatherForecast[]'.
2020-09-20 06:17:06,548 [12] INFO [k8sdemo-1] - Executed action T.K8SDemo.Controllers.WeatherForecastController.Get (T.K8SDemo) in 17.1904ms
2020-09-20 06:17:06,549 [12] INFO [k8sdemo-1] - Executed endpoint 'T.K8SDemo.Controllers.WeatherForecastController.Get (T.K8SDemo)'
2020-09-20 06:17:06,550 [12] INFO [k8sdemo-1] - Request finished in 84.3414ms 200 application/json; charset=utf-8
另外对 Pod 副本进行伸缩时效果也是一样的,都会保持 Pod 具有的状态。当然文中的例子和一些组件的集群部署不太一样,比如像 MySQL 这类组件,各实例间还会做数据同步来实现数据的一致性,当然最终也是每个实例关联自己的数据存储卷。