3. kubernetes客户端crictl命令

kubernetes客户端crictl命令

crictl 是一个命令行工具，用于与容器运行时接口（CRI）兼容的容器运行时（如 containerd 和 CRI-O）进行交互。crictl 提供了许多有用的命令来管理容器、镜像和 sandboxes。

官方仓库地址：

kubernetes-sigs/cri-tools: CLI and validation tools for Kubelet Container Runtime Interface (CRI) . (github.com)

参考资料：

cri-tools/docs/crictl.md at master · kubernetes-sigs/cri-tools (github.com)
使用 crictl 对 Kubernetes 节点进行调试 | Kubernetes

1. 安装 crictl

cri-tools项目包括crictl和critest两个工具。下面是兼容性矩阵：

Kubernetes Version	cri-tools Version	cri-tools branch
≥ 1.27.x	≥ 1.27.x	master
≥ 1.16.x ≤ 1.26.x	≥ 1.16.x ≤ 1.26.x	master
1.15.X	v1.15.0	release-1.15
1.14.X	v1.14.0	release-1.14
1.13.X	v1.13.0	release-1.13
1.12.X	v1.12.0	release-1.12
1.11.X	v1.11.1	release-1.11
1.10.X	v1.0.0-beta.2	release-1.10
1.9.X	v1.0.0-alpha.1	release-1.9
1.8.X	v0.2	release-1.8
1.7.X	v0.1	release-1.7

一般情况下安装完成k8s后，会同步安装crictl。如果没有安装crictl工具，可以参考如下步骤手动安装：

下载 crictl 的二进制文件：

在Releases · kubernetes-sigs/cri-tools (github.com)查看版本列表，以下载安装v1.28.0版本为例：

VERSION="v1.28.0"# 使用wget
wget https://github.com/kubernetes-sigs/cri-tools/releases/download/$VERSION/crictl-$VERSION-linux-amd64.tar.gz# 使用curl
curl -L https://github.com/kubernetes-sigs/cri-tools/releases/download/$VERSION/crictl-${VERSION}-linux-amd64.tar.gz --output crictl-${VERSION}-linux-amd64.tar.gz

解压并将二进制文件移动到系统路径：

sudo tar zxvf crictl-$VERSION-linux-amd64.tar.gz -C /usr/local/bin

验证安装是否成功：

crictl --version

2. 配置 crictl

crictl在linux下默认会连接:

unix:///run/containerd/containerd.sock or
unix:///run/crio/crio.sock or
unix:///var/run/cri-dockerd.sock

在windows下默认会连接:

npipe:./pipe/containerd-containerd or
npipe:./pipe/cri-dockerd

其他运行时:

frakti: unix:///var/run/frakti.sock

可以用以下方法之一来为 crictl 设置端点：

设置参数 --runtime-endpoint 和 --image-endpoint。
设置环境变量 CONTAINER_RUNTIME_ENDPOINT 和 IMAGE_SERVICE_ENDPOINT。
在默认配置文件 --config=/etc/crictl.yaml 中设置端点。要设置不同的文件，可以在运行 crictl 时使用 --config=PATH_TO_FILE 标志。

这里采用配置文件方法，一劳永逸。默认配置文件路径是 /etc/crictl.yaml，使用containerd容器运行时，配置示例如下：

linux：

$ cat /etc/crictl.yaml
runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
timeout: 2
debug: true
pull-image-on-create: false

windows:

C:\> type %USERPROFILE%\.crictl\crictl.yaml
runtime-endpoint: npipe:./pipe/containerd-containerd
image-endpoint: npipe:./pipe/containerd-containerd
timeout: 2
debug: true
pull-image-on-create: false

如果endpoint没有设置， crictl 会尝试依次连接：

containerd
cri-o
cri-dockerd

3. crictl命令示例

注意：

如果使用 crictl 在正在运行的 Kubernetes 集群上创建 Pod 沙盒或容器， kubelet 最终将删除它们。 crictl 不是一个通用的工作流工具，而是一个对调试有用的工具。

打印 Pod 清单

打印所有 Pod 的清单：

crictl pods

输出类似于：

POD ID              CREATED              STATE               NAME                         NAMESPACE           ATTEMPT
926f1b5a1d33a       About a minute ago   Ready               sh-84d7dcf559-4r2gq          default             0
4dccb216c4adb       About a minute ago   Ready               nginx-65899c769f-wv2gp       default             0
a86316e96fa89       17 hours ago         Ready               kube-proxy-gblk4             kube-system         0
919630b8f81f1       17 hours ago         Ready               nvidia-device-plugin-zgbbv   kube-system         0

根据名称打印 Pod 清单：

crictl pods --name nginx-65899c769f-wv2gp

输出类似于这样：

POD ID              CREATED             STATE               NAME                     NAMESPACE           ATTEMPT
4dccb216c4adb       2 minutes ago       Ready               nginx-65899c769f-wv2gp   default             0

根据标签打印 Pod 清单：

crictl pods --label run=nginx

输出类似于这样：

POD ID              CREATED             STATE               NAME                     NAMESPACE           ATTEMPT
4dccb216c4adb       2 minutes ago       Ready               nginx-65899c769f-wv2gp   default             0

打印镜像清单

打印所有镜像清单：

crictl images

输出类似于这样：

IMAGE                                     TAG                 IMAGE ID            SIZE
busybox                                   latest              8c811b4aec35f       1.15MB
k8s-gcrio.azureedge.net/hyperkube-amd64   v1.10.3             e179bbfe5d238       665MB
k8s-gcrio.azureedge.net/pause-amd64       3.1                 da86e6ba6ca19       742kB
nginx                                     latest              cd5239a0906a6       109MB

根据仓库打印镜像清单：

crictl images nginx

输出类似于这样：

IMAGE               TAG                 IMAGE ID            SIZE
nginx               latest              cd5239a0906a6       109MB

只打印镜像 ID：

crictl images -q

输出类似于这样：

sha256:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a
sha256:e179bbfe5d238de6069f3b03fccbecc3fb4f2019af741bfff1233c4d7b2970c5
sha256:da86e6ba6ca197bf6bc5e9d900febd906b133eaa4750e6bed647b0fbe50ed43e
sha256:cd5239a0906a6ccf0562354852fae04bc5b52d72a2aff9a871ddb6bd57553569

打印容器清单

打印所有容器清单：

crictl ps -a

输出类似于这样：

CONTAINER ID        IMAGE                                                                                                             CREATED             STATE               NAME                       ATTEMPT
1f73f2d81bf98       busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47                                   7 minutes ago       Running             sh                         1
9c5951df22c78       busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47                                   8 minutes ago       Exited              sh                         0
87d3992f84f74       nginx@sha256:d0a8828cccb73397acb0073bf34f4d7d8aa315263f1e7806bf8c55d8ac139d5f                                     8 minutes ago       Running             nginx                      0
1941fb4da154f       k8s-gcrio.azureedge.net/hyperkube-amd64@sha256:00d814b1f7763f4ab5be80c58e98140dfc69df107f253d7fdd714b30a714260a   18 hours ago        Running             kube-proxy                 0

打印正在运行的容器清单：

crictl ps

输出类似于这样：

CONTAINER ID        IMAGE                                                                                                             CREATED             STATE               NAME                       ATTEMPT
1f73f2d81bf98       busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47                                   6 minutes ago       Running             sh                         1
87d3992f84f74       nginx@sha256:d0a8828cccb73397acb0073bf34f4d7d8aa315263f1e7806bf8c55d8ac139d5f                                     7 minutes ago       Running             nginx                      0
1941fb4da154f       k8s-gcrio.azureedge.net/hyperkube-amd64@sha256:00d814b1f7763f4ab5be80c58e98140dfc69df107f253d7fdd714b30a714260a   17 hours ago        Running             kube-proxy                 0

在正在运行的容器上执行命令

crictl exec -i -t 1f73f2d81bf98 ls

输出类似于这样：

bin   dev   etc   home  proc  root  sys   tmp   usr   var

获取容器日志

获取容器的所有日志：

crictl logs 87d3992f84f74

输出类似于这样：

10.240.0.96 - - [06/Jun/2018:02:45:49 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"
10.240.0.96 - - [06/Jun/2018:02:45:50 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"
10.240.0.96 - - [06/Jun/2018:02:45:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"

获取最近的 N 行日志：

crictl logs --tail=1 87d3992f84f74

输出类似于这样：

10.240.0.96 - - [06/Jun/2018:02:45:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"

运行 Pod 沙盒

用 crictl 运行 Pod 沙盒对容器运行时排错很有帮助。在运行的 Kubernetes 集群中，沙盒会随机地被 kubelet 停止和删除。

编写下面的 JSON 文件：

{"metadata": {"name": "nginx-sandbox","namespace": "default","attempt": 1,"uid": "hdishd83djaidwnduwk28bcsb"},"log_directory": "/tmp","linux": {}
}

使用 crictl runp 命令应用 JSON 文件并运行沙盒。
```
crictl runp pod-config.json
```
返回了沙盒的 ID。

创建容器

用 crictl 创建容器对容器运行时排错很有帮助。在运行的 Kubernetes 集群中，容器最终将被 kubelet 停止和删除。

拉取 busybox 镜像

crictl pull busybox

Image is up to date for busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d4

创建 Pod 和容器的配置：

Pod 配置：

{"metadata": {"name": "busybox-sandbox","namespace": "default","attempt": 1,"uid": "aewi4aeThua7ooShohbo1phoj"},"log_directory": "/tmp","linux": {}
}

容器配置：

container-config.json

{"metadata": {"name": "busybox"},"image":{"image": "busybox"},"command": ["top"],"log_path":"busybox.log","linux": {}
}

创建容器，传递先前创建的 Pod 的 ID、容器配置文件和 Pod 配置文件。返回容器的 ID。

crictl create f84dd361f8dc51518ed291fbadd6db537b0496536c1d2d6c05ff943ce8c9a54f container-config.json pod-config.json

查询所有容器并确认新创建的容器状态为 Created。

crictl ps -a

输出类似于这样：

CONTAINER ID        IMAGE               CREATED             STATE               NAME                ATTEMPT
3e025dd50a72d       busybox             32 seconds ago      Created             busybox             0

启动容器

要启动容器，要将容器 ID 传给 crictl start：

crictl start 3e025dd50a72d956c4f14881fbb5b1080c9275674e95fb67f965f6478a957d60

输出类似于这样：

3e025dd50a72d956c4f14881fbb5b1080c9275674e95fb67f965f6478a957d60

确认容器的状态为 Running。

crictl ps

输出类似于这样：

CONTAINER ID   IMAGE    CREATED              STATE    NAME     ATTEMPT
3e025dd50a72d  busybox  About a minute ago   Running  busybox  0

4. 排错

1 .CRI v1 image API is not implemented for endpoint

[root@k8s ~]# crictl images
DEBU[0000] get image connection
FATA[0000] validate service connection: CRI v1 image API is not implemented for endpoint "unix:///run/containerd/containerd.sock": rpc error: code = Unimplemented desc = unknown service runtime.v1.ImageService

原因为containerd禁用了cri插件：

[root@k8s ~]# cat /etc/containerd/config.toml | grep plugin
disabled_plugins = ["cri"]

将cri删除后，重启containerd服务后尝试。

创建Pod沙盒报错，OCI runtime create failed: runc create failed: expected cgroupsPath to be of format

root@master1:~# cat pod-config.json
{"metadata": {"name": "busybox-sandbox","namespace": "default","attempt": 1,"uid": "aewi4aeThua7ooShohbo1phoj"},"log_directory": "/tmp","linux": {}
}
root@master1:~# crictl runp pod-config.json
DEBU[0000] get runtime connection
DEBU[0000] RunPodSandboxRequest: &RunPodSandboxRequest{Config:&PodSandboxConfig{Metadata:&PodSandboxMetadata{Name:busybox-sandbox,Uid:aewi4aeThua7ooShohbo1phoj,Namespace:default,Attempt:1,},Hostname:,LogDirectory:/tmp,DnsConfig:nil,PortMappings:[]*PortMapping{},Labels:map[string]string{},Annotations:map[string]string{},Linux:&LinuxPodSandboxConfig{CgroupParent:,SecurityContext:nil,Sysctls:map[string]string{},Overhead:nil,Resources:nil,},Windows:nil,},RuntimeHandler:,}
E0622 17:09:26.796680   97165 remote_runtime.go:176] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: expected cgroupsPath to be of format \"slice:prefix:name\" for systemd cgroups, got \"/k8s.io/bcc06428ac84c7bf8bc431a1366b5c768b17e53cc04bc666195535a95f7325fc\" instead: unknown"
DEBU[0000] RunPodSandboxResponse:
FATA[0000] run pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: expected cgroupsPath to be of format "slice:prefix:name" for systemd cgroups, got "/k8s.io/bcc06428ac84c7bf8bc431a1366b5c768b17e53cc04bc666195535a95f7325fc" instead: unknown