1. 原理介绍

• 设置 HPA 每次最小扩容 Pod 数为可用区数量，以期可用区间 Pod 同步扩容
• 设置 TopologySpreadConstraints 可用区分散 maxSkew 为 1，以尽可能可用区间 Pod 均匀分布

2. 实验验证

2.1. 准备 Kind 集群

准备如下配置文件，命名为 kind-cluster.yaml

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-planeimage: kindest/node:v1.24.0@sha256:0866296e693efe1fed79d5e6c7af8df71fc73ae45e3679af05342239cdc5bc8e
- role: workerimage: kindest/node:v1.24.0@sha256:0866296e693efe1fed79d5e6c7af8df71fc73ae45e3679af05342239cdc5bc8elabels:topology.kubernetes.io/zone: "us-east-1a"
- role: workerimage: kindest/node:v1.24.0@sha256:0866296e693efe1fed79d5e6c7af8df71fc73ae45e3679af05342239cdc5bc8elabels:topology.kubernetes.io/zone: "us-east-1c"

上述配置为集群定义了 2 个工作节点，并分别打上了不同的可用区标签。
执行如下命令创建该 Kubernetes 集群：

$ kind create cluster --config cluster-1.24.yaml
Creating cluster "kind" ...✓ Ensuring node image (kindest/node:v1.24.0) 🖼 ✓ Preparing nodes 📦 📦 📦  ✓ Writing configuration 📜 ✓ Starting control-plane 🕹️ ✓ Installing CNI 🔌 ✓ Installing StorageClass 💾 ✓ Joining worker nodes 🚜 
Set kubectl context to "kind-kind"
You can now use your cluster with:kubectl cluster-info --context kind-kindHave a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂

检查集群运行正常：

$ kubectl get node --show-labels
NAME                 STATUS   ROLES           AGE    VERSION   LABELS
kind-control-plane   Ready    control-plane   161m   v1.24.0   beta.kubernetes.io/arch=arm64,beta.kubernetes.io/os=linux,kubernetes.io/arch=arm64,kubernetes.io/hostname=kind-control-plane,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node.kubernetes.io/exclude-from-external-load-balancers=
kind-worker          Ready    <none>          160m   v1.24.0   beta.kubernetes.io/arch=arm64,beta.kubernetes.io/os=linux,kubernetes.io/arch=arm64,kubernetes.io/hostname=kind-worker,kubernetes.io/os=linux,topology.kubernetes.io/zone=us-east-1a
kind-worker2         Ready    <none>          160m   v1.24.0   beta.kubernetes.io/arch=arm64,beta.kubernetes.io/os=linux,kubernetes.io/arch=arm64,kubernetes.io/hostname=kind-worker2,kubernetes.io/os=linux,topology.kubernetes.io/zone=us-east-1c

2.2. 安装 metrics-server 组件

HPA 依赖 metrics-server 提供监控指标，通过如下命令安装：

$ kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

提示: 国内网络不能直接下载到 registry.k8s.io/metrics-server/metrics-server:v0.6.4 镜像，可以替换为等同的 shidaqiu/metrics-server:v0.6.4。同时，关闭 tls 安全校验，如下图：

检查部署后的 metrics-server 运行正常：

$ kubectl top node
NAME                 CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%   
kind-control-plane   238m         5%     667Mi           8%        
kind-worker          76m          1%     207Mi           2%        
kind-worker2         41m          1%     110Mi           1% 

2.3. 部署测试服务

准备如下 YAML，命名为 hpa-php-demo.yaml

注意：Deployment 的 topologySpreadConstraints 配置为可用区分散！

apiVersion: apps/v1
kind: Deployment
metadata:name: php-web-demo
spec:selector:matchLabels:run: php-web-demoreplicas: 1template:metadata:labels:run: php-web-demospec:topologySpreadConstraints:- maxSkew: 1topologyKey: kubernetes.io/zonewhenUnsatisfiable: ScheduleAnywaylabelSelector:matchLabels:run: php-web-democontainers:- name: php-web-demoimage: shidaqiu/hpademo:latestports:- containerPort: 80resources:limits:cpu: 500mrequests:cpu: 200m
---
apiVersion: v1
kind: Service
metadata:name: php-web-demolabels:run: php-web-demo
spec:ports:- port: 80selector:run: php-web-demo

部署上述服务：

kubectl apply -f hpa-php-demo.yaml

2.4. 部署 HPA 配置

准备 HPA 配置文件，命名为 hpa-demo.yaml

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:name: php-web-demo
spec:scaleTargetRef:apiVersion: apps/v1kind: Deploymentname: php-web-demominReplicas: 2maxReplicas: 10metrics:- type: Resourceresource:name: cputarget:type: UtilizationaverageUtilization: 50behavior:scaleDown:stabilizationWindowSeconds: 300policies:- type: Percentvalue: 50periodSeconds: 15- type: Podsvalue: 2periodSeconds: 15scaleUp:stabilizationWindowSeconds: 0policies:- type: Percentvalue: 100periodSeconds: 15- type: Podsvalue: 2periodSeconds: 15 selectPolicy: Max

部署上述 HPA 配置：

$ kubectl apply -f hpa-demo.yaml

上述 HPA 通过 scaleUp 和 scaleDown 定义了扩容和缩容的行为，每次扩容一倍或 2 个 Pod（取较大者），每次缩容一半或 2 个 Pod（取较大者）。

2.5. 验证扩容

扩容前，观察 Pod 分别运行在两个区：

$ kubectl get pod -owide
NAME                           READY   STATUS    RESTARTS   AGE     IP           NODE           NOMINATED NODE   READINESS GATES
php-web-demo-d6d66c8d5-22tn6   1/1     Running   0          6m57s   10.244.2.3   kind-worker2   <none>           <none>
php-web-demo-d6d66c8d5-tz8m9   1/1     Running   0          76s     10.244.1.3   kind-worker    <none>           <none>

给服务施加压力：

$ kubectl run -it --rm load-generator --image=busybox /bin/sh
进入容器后，执行如下脚本：
while true; do wget -q -O- http://php-web-demo; done

可以观察到 Pod 扩容时，同时在两个可用区进行，实现了可用区同步扩容的效果

停止施加压力，可以观察到 Pod 缩容保持了可用区分散的状态

如何保证缩容后，Pod 仍在多可用区均匀分散？

可以考虑借助 descheduler 的 rebalance 能力，参考 https://github.com/kubernetes-sigs/descheduler?tab=readme-ov-file#removepodsviolatingtopologyspreadconstraint