【模块一】kubernetes容器编排进阶实战之kubernetes 资源限制

kubernetes 资源限制

kubernetes中资源限制概括

1.如果运行的容器没有定义资源(memory、CPU)等限制，但是在namespace定义了LimitRange限制，那么该容器会继承LimitRange中的默认限制。

2.如果namespace没有定义LimitRange限制，那么该容器可以只要宿主机的最大可用资源，直到无资源可用而触发宿主机(OOM Killer)。

为容器和 Pods 分配 CPU 资源 | Kubernetes本页面展示如何为容器设置 CPU request（请求）和 CPU limit（限制）。容器使用的 CPU 不能超过所配置的限制。如果系统有空闲的 CPU 时间，则可以保证给容器分配其所请求数量的 CPU 资源。准备开始你必须拥有一个 Kubernetes 的集群，且必须配置 kubectl 命令行工具让其与你的集群通信。建议运行本教程的集群至少有两个节点，且这两个节点不能作为控制平面主机。如果你还没有集群，你可以通过 Minikube 构建一个你自己的集群，或者你可以使用下面的 Kubernetes 练习环境之一：Killercoda 玩转 Kubernetes 要获知版本信息，请输入 kubectl version. 你的集群必须至少有 1 个 CPU 可用才能运行本任务中的示例。本页的一些步骤要求你在集群中运行 metrics-server 服务。如果你的集群中已经有正在运行的 metrics-server 服务，可以跳过这些步骤。如果你正在运行 Minikube，请运行以下命令启用 metrics-server：minikube addons enable metrics-server 查看 metrics-server（或者其他资源指标 API metrics.k8s.io 服务提供者）是否正在运行，请键入以下命令：kubectl get apiservices 如果资源指标 API 可用，则会输出将包含一个对 metrics.k8s.io 的引用。NAME v1beta1.metrics.k8s.io 创建一个名字空间创建一个名字空间，以便将本练习中创建的资源与集群的其余部分资源隔离。kubectl create namespace cpu-example 指定 CPU 请求和 CPU 限制要为容器指定 CPU 请求，请在容器资源清单中包含 resources: requests 字段。要指定 CPU 限制，请包含 resources:limits。https://kubernetes.io/zh/docs/tasks/configure-pod-container/assign-cpu-resource/

CPU 以核心为单位进行限制，单位可以是整核、浮点核心数或毫核(m/milli)：

2=2核心=200% 0.5=500m=50% 1.2=1200m=120%

为容器和 Pod 分配内存资源 | Kubernetes

memory 以字节为单位,单位可以是E、P、T、G、M、K、Ei、Pi、Ti、Gi、Mi、Ki

1536Mi=1.5Gi

requests(请求)为kubernetes scheduler执行pod调度时node节点至少需要拥有的资源。

limits(限制)为pod运行成功后最多可以使用的资源上限。

kubernetes对单个容器的CPU及memory实现资源限制

[root@k8s-master1 vip-limit-case]#cat case1-pod-memory-limit.yml
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
kind: Deployment
metadata:
name: limit-test-deployment
namespace: vip
spec:
replicas: 1
selector:
matchLabels: #rs or deployment
app: limit-test-pod
# matchExpressions:
# - {key: app, operator: In, values: [ng-deploy-80,ng-rs-81]}
template:
metadata:
labels:
app: limit-test-pod
spec:
containers:
- name: limit-test-container
image: lorel/docker-stress-ng
resources:
limits:
cpu: 1
memory: "256Mi"
requests:
cpu: 1
memory: "256Mi"
#command: ["stress"]
args: ["--vm", "2", "--vm-bytes", "256M"]
#nodeSelector:
# env: group1

[root@k8s-master1 vip-limit-case]#kubectl apply -f case1-pod-memory-limit.yml

[root@k8s-master1 vip-limit-case]#kubectl top pod -n vip
NAME CPU(cores) MEMORY(bytes)
limit-test-deployment-6d7c8cc78b-x868g 935m 246Mi

kubernetes对单个pod的CPU及memory实现资源限制

Limit Range是对具体某个Pod或容器的资源使用进行限制

限制范围（LimitRange） | Kubernetes

限制namespace中每个Pod或容器的最小与最大计算资源
限制namespace中每个Pod或容器计算资源request、limit之间的比例
限制namespace中每个存储卷声明（PersistentVolumeClaim）可使用的最小与最大存储空间
设置namespace中容器默认计算资源的request、limit，并在运行时自动注入到容器中

[root@k8s-master1 vip-limit-case]#cat case3-LimitRange.yaml
apiVersion: v1
kind: LimitRange
metadata:
name: limitrange-magedu
namespace: vip
spec:
limits:
- type: Container #限制的资源类型
max:
cpu: "2" #限制单个容器的最大CPU
memory: "2Gi" #限制单个容器的最大内存
min:
cpu: "500m" #限制单个容器的最小CPU
memory: "512Mi" #限制单个容器的最小内存
default:
cpu: "500m" #默认单个容器的CPU限制
memory: "512Mi" #默认单个容器的内存限制
defaultRequest:
cpu: "500m" #默认单个容器的CPU创建请求
memory: "512Mi" #默认单个容器的内存创建请求
maxLimitRequestRatio:
cpu: 2 #限制CPU limit/request比值最大为2
memory: 2 #限制内存limit/request比值最大为1.5
- type: Pod
max:
cpu: "4" #限制单个Pod的最大CPU
memory: "4Gi" #限制单个Pod最大内存
- type: PersistentVolumeClaim
max:
storage: 50Gi #限制PVC最大的requests.storage
min:
storage: 30Gi #限制PVC最小的requests.storage

限制案例：CPU与内存 RequestRatio比例限制与 CPU与内存或超分限制

[root@k8s-master1 magedu-limit-case]#cat ../metrics-server-0.6.1-case/tomcat-app1.yaml
kind: Deployment
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
metadata:labels:app: vip-tomcat-app1-deployment-labelname: vip-tomcat-app1-deploymentnamespace: vip
spec:replicas: 1selector:matchLabels:app: vip-tomcat-app1-selectortemplate:metadata:labels:app: vip-tomcat-app1-selectorspec:nodeName: 10.0.0.113containers:- name: vip-tomcat-app1-containerimage: tomcat:7.0.93-alpine #image: lorel/docker-stress-ng #args: ["--vm", "2", "--vm-bytes", "256M"]##command: ["/apps/tomcat/bin/run_tomcat.sh"]imagePullPolicy: IfNotPresent##imagePullPolicy: Alwaysports:- containerPort: 8080protocol: TCPname: httpenv:- name: "password"value: "123456"- name: "age"value: "18"resources:limits:cpu: 3memory: "512Mi"requests:cpu: 500mmemory: "512Mi"- name: vip-tomcat-app2-containerimage: tomcat:7.0.93-alpine#image: lorel/docker-stress-ng #args: ["--vm", "2", "--vm-bytes", "256M"]##command: ["/apps/tomcat/bin/run_tomcat.sh"]imagePullPolicy: IfNotPresent##imagePullPolicy: Alwaysports:- containerPort: 8080protocol: TCPname: httpenv:- name: "password"value: "123456"- name: "age"value: "18"resources:limits:cpu: 500mmemory: "500Mi"requests:cpu: 500mmemory: "500Mi"---
kind: Service
apiVersion: v1
metadata:labels:app: vip-tomcat-app1-service-labelname: vip-tomcat-app1-servicenamespace: vip
spec:type: NodePortports:- name: httpport: 80protocol: TCPtargetPort: 8080#nodePort: 40003selector:app: vip-tomcat-app1-selector

# kubectl apply  -f case3-LimitRange.yaml 
[root@k8s-master1 vip-limit-case]#kubectl describe limitranges -n vip
Name:                  limitrange-vip
Namespace:             magedu
Type                   Resource  Min    Max   Default Request  Default Limit  Max Limit/Request Ratio
----                   --------  ---    ---   ---------------  -------------  -----------------------
Container              cpu       500m   2     500m             500m           2
Container              memory    512Mi  2Gi   512Mi            512Mi          2
Pod                    cpu       -      4     -                -              -
Pod                    memory    -      4Gi   -                -              -
PersistentVolumeClaim  storage   30Gi   50Gi  -                -              -#kubectl apply  -f ../metrics-server-0.6.1-case/tomcat-app1.yaml#kubectl  get  deployment.apps/magedu-tomcat-app1-deployment -n vip  -o jsonmessage": "pods \"magedu-tomcat-app1-deployment-76dcc947d5-b25r6\" is forbidden: [minimum memory usage per Container is 512Mi, but request is 500Mi, maximum cpu usage per Container is 2, but limit is 3, cpu max limit to request ratio per Container is 2, but provided ratio is 6.000000
消息“:”pods \“magedu-tomcat-app1-deployment-76dcc947d5-b25r6\”被禁止：[每个容器的最小内存使用量是512Mi，但请求是500Mi，每个容器的最大cpu使用量是2，但限制是3，每个容器的cpu最大限制与请求比率是2，但提供的比率是6000000cpu比例等于   cpulimit/cpurequest  3/0.5=6

kubernetes对整个namespace的CPU及memory实现资源限制

资源配额 | Kubernetes
限定某个对象类型（如Pod、service）可创建对象的总数；
限定某个对象类型可消耗的计算资源（CPU、内存）与存储资源（存储卷声明）总数

[root@k8s-master1 vip-limit-case]#cat case6-ResourceQuota-vip.yaml 
apiVersion: v1
kind: ResourceQuota
metadata:name: quota-magedunamespace: vip
spec:hard:requests.cpu: "8"limits.cpu: "8"requests.memory: 4Gilimits.memory: 4Girequests.nvidia.com/gpu: 4pods: "2"services: "100"[root@k8s-master1 vip-limit-case]#kubectl get resourcequotas -n vip 
NAME           AGE     REQUEST                                                                                                 LIMIT
quota-vip   4m25s   pods: 0/100, requests.cpu: 0/8, requests.memory: 0/4Gi, requests.nvidia.com/gpu: 0/4, services: 0/100   limits.cpu: 0/8, limits.memory: 0/4Gi
[root@k8s-master1 vip-limit-case]#kubectl describe resourcequotas -n vip 
Name:                    quota-vip
Namespace:               vip
Resource                 Used  Hard
--------                 ----  ----
limits.cpu               0     8
limits.memory            0     4Gi
pods                     0     100
requests.cpu             0     8
requests.memory          0     4Gi
requests.nvidia.com/gpu  0     4
services                 0     100

限制案例1：验证namespace Pod副本数限制

[root@k8s-master1 ~]#kubectl get deployments.apps -n magedu 
NAME                      READY   UP-TO-DATE   AVAILABLE   AGE
magedu-nginx-deployment   2/3     2            2           38s
[root@k8s-master1 ~]#kubectl describe resourcequotas -n magedu 
Name:                    quota-magedu
Namespace:               magedu
Resource                 Used   Hard
--------                 ----   ----
limits.cpu               400m   8
limits.memory            424Mi  4Gi
pods                     2      2
requests.cpu             400m   8
requests.memory          424Mi  4Gi
requests.nvidia.com/gpu  0      4
services                 1      100kubectl get  -n magedu deployments.apps/magedu-nginx-deployment -o json"lastTransitionTime": "2024-12-27T08:52:29Z","lastUpdateTime": "2024-12-27T08:52:29Z","message": "pods \"magedu-nginx-deployment-7f548f9b4d-2kc42\" is forbidden: exceeded quota: quota-magedu, requested: pods=1, used: pods=2, limited: pods=2","reason": "FailedCreate","status": "True","type": "ReplicaFailure"},
消息：禁止创建 pod“magedu-nginx-deployment-7f548f9b4d-2kc42”：超出配额：quota-magedu，请求：pod=1，已用：pod=2，限制：pod=2

限制案例2：CPU总计核心数限制

[root@k8s-master1 vip-limit-case]#cat case7-namespace-pod-limit-test.yaml 
kind: Deployment
apiVersion: apps/v1
metadata:labels:app: vip-nginx-deployment-labelname: vip-nginx-deploymentnamespace: vip
spec:replicas: 1selector:matchLabels:app: vip-nginx-selectortemplate:metadata:labels:app: vip-nginx-selectorspec:nodeName: 10.0.0.113containers:- name: vip-nginx-containerimage: nginx:1.20.2-alpineimagePullPolicy: IfNotPresentports:- containerPort: 80protocol: TCPname: httpenv:- name: "password"value: "123456"- name: "age"value: "18"resources:limits:cpu: 5memory: 212Mirequests:cpu: 5memory: 212Mi

[root@k8s-master1 ~]#kubectl get  -n magedu deployments.apps/magedu-nginx-deployment -o json
"message": "pods \"magedu-nginx-deployment-5bccb4c76b-9857m\" is forbidden: exceeded quota: quota-magedu, requested: limits.cpu=5,pods=1,requests.cpu=5, used: limits.cpu=5005m,pods=2,requests.cpu=5005m, limited: limits.cpu=8,pods=2,requests.cpu=8",消息：禁止创建 pod "magedu-nginx-deployment-5bccb4c76b-9857m"：超出配额：quota-magedu，请求：limits.cpu=5，pods=1，requests.cpu=5，已使用：limits.cpu=5005m，pods=2，requests.cpu=5005m，限制：limits.cpu=8，pods=2，requests.cpu=8