文章目录
- 1. containerd安装
- 2. 源码编译
- 3. 验证编译的二进制文件是否含有调试需要的信息
- 3.1. objdump工具验证
- 3.2. file工具验证
- 3.3. dlv工具验证
- 4. debug
1. containerd安装
[Ubuntu 22.04] 安装containerd
2. 源码编译
主要步骤如下:
- 1、从github下载
containerd
源码git clone git@github.com:containerd/containerd.git
- 2、切换到想要
debug
的分支或者tag
,我这里选择切换到1.7.2
版本的tag
git branch v1.7.2 v1.7.2 && git checkout v1.7.2
- 3、下载
contaienrd
的依赖cd containerd && go mod tidy
- 4、编译
containerd
源码make build GODEBUG=true all
root@containerd:~/workspace/containerd#
root@containerd:~/workspace/containerd#
root@containerd:~/workspace/containerd#
root@containerd:~/workspace/containerd# make build GODEBUG=true all
+ build
+ bin/ctr
go build -gcflags=all="-N -l" -gcflags=-trimpath=/root/go/src -buildmode=pie -o bin/ctr -ldflags '-X github.com/containerd/containerd/version.Version=v1.7.2-2-gbe3ad13c1 -X github.com/containerd/containerd/version.Revision=be3ad13c14e0e1da2840fc6496f2bcefefb99764 -X github.com/containerd/containerd/version.Package=github.com/containerd/containerd ' -tags "urfave_cli_no_docs static_build" ./cmd/ctr
+ bin/containerd
go build -gcflags=all="-N -l" -gcflags=-trimpath=/root/go/src -buildmode=pie -o bin/containerd -ldflags '-X github.com/containerd/containerd/version.Version=v1.7.2-2-gbe3ad13c1 -X github.com/containerd/containerd/version.Revision=be3ad13c14e0e1da2840fc6496f2bcefefb99764 -X github.com/containerd/containerd/version.Package=github.com/containerd/containerd ' -tags "urfave_cli_no_docs static_build" ./cmd/containerd
+ bin/containerd-stress
go build -gcflags=all="-N -l" -gcflags=-trimpath=/root/go/src -buildmode=pie -o bin/containerd-stress -ldflags '-X github.com/containerd/containerd/version.Version=v1.7.2-2-gbe3ad13c1 -X github.com/containerd/containerd/version.Revision=be3ad13c14e0e1da2840fc6496f2bcefefb99764 -X github.com/containerd/containerd/version.Package=github.com/containerd/containerd ' -tags "urfave_cli_no_docs static_build" ./cmd/containerd-stress
+ bin/containerd-shim
+ bin/containerd-shim-runc-v1
+ bin/containerd-shim-runc-v2
+ binaries
root@containerd:~/workspace/containerd#
root@containerd:~/workspace/containerd#
root@containerd:~/workspace/containerd# ls -lhtr bin/
total 158M
-rwxr-xr-x 1 root root 31M Jul 29 14:34 ctr
-rwxr-xr-x 1 root root 61M Jul 29 14:34 containerd
-rwxr-xr-x 1 root root 29M Jul 29 14:34 containerd-stress
-rwxr-xr-x 1 root root 9.4M Jul 29 14:34 containerd-shim
-rwxr-xr-x 1 root root 12M Jul 29 14:34 containerd-shim-runc-v1
-rwxr-xr-x 1 root root 17M Jul 29 14:34 containerd-shim-runc-v2
root@containerd:~/workspace/containerd#
注意,编译的时候containerd
会依赖btrfs
文件系统,如果没有安装,编译的时候会报错(提示:linux/btrfs_tree.h: No such file or directory
)。按照文档第一步安装containerd
不会遇到这个问题。
3. 验证编译的二进制文件是否含有调试需要的信息
注意,以下三种验证方式,选择其中的一种验证成功即可。
3.1. objdump工具验证
正确的现象如下,执行objdump --syms bin/containerd
是可以看到调试信息的
root@containerd:~/workspace/containerd# objdump --syms bin/containerdbin/containerd: file format elf64-x86-64SYMBOL TABLE:
0000000000000000 l df *ABS* 0000000000000000 Scrt1.o
00000000000003b4 l O .note.ABI-tag 0000000000000020 __abi_tag
0000000000000000 l df *ABS* 0000000000000000 go.go
00000000003ad4c0 l F .text 0000000000000000 runtime.text
00000000003ad4c0 l F .text 0000000000000059 internal/cpu.Initialize
00000000003ad520 l F .text 0000000000000537 internal/cpu.processOptions
00000000003ada60 l F .text 0000000000000026 internal/cpu.indexByte
00000000003adaa0 l F .text 0000000000000925 internal/cpu.doinit
00000000003ae3e0 l F .text 0000000000000006 internal/cpu.isSet
00000000003ae400 l F .text 000000000000001b internal/cpu.cpuid.abi0
00000000003ae420 l F .text 0000000000000011 internal/cpu.xgetbv.abi0
00000000003ae440 l F .text 0000000000000009 internal/cpu.getGOAMD64level.abi0
00000000003ae460 l F .text 000000000000007a type:.eq.internal/cpu.option
00000000003ae4e0 l F .text 00000000000000e6 type:.eq.[6]internal/cpu.option
00000000003ae5e0 l F .text 0000000000000003 runtime/internal/atomic.(*Int32).Load
00000000003ae600 l F .text 0000000000000003 runtime/internal/atomic.(*Int32).Store
00000000003ae620 l F .text 000000000000000d runtime/internal/atomic.(*Int32).CompareAndSwap
00000000003ae640 l F .text 000000000000000a runtime/internal/atomic.(*Int32).Add
00000000003ae660 l F .text 0000000000000004 runtime/internal/atomic.(*Int64).Load
00000000003ae680 l F .text 0000000000000004 runtime/internal/atomic.(*Int64).Store
00000000003ae6a0 l F .text 000000000000000f runtime/internal/atomic.(*Int64).CompareAndSwap
00000000003ae6c0 l F .text 0000000000000007 runtime/internal/atomic.(*Int64).Swap
00000000003ae6e0 l F .text 000000000000000d runtime/internal/atomic.(*Int64).Add
00000000003ae700 l F .text 0000000000000003 runtime/internal/atomic.(*Uint8).Load
00000000003ae720 l F .text 0000000000000003 runtime/internal/atomic.(*Uint8).Store
00000000003ae740 l F .text 0000000000000004 runtime/internal/atomic.(*Uint8).And
00000000003ae760 l F .text 0000000000000004 runtime/internal/atomic.(*Uint8).Or
00000000003ae780 l F .text 0000000000000024 runtime/internal/atomic.(*Bool).Load
00000000003ae7c0 l F .text 000000000000001f runtime/internal/atomic.(*Bool).Store
00000000003ae7e0 l F .text 0000000000000003 runtime/internal/atomic.(*Uint32).Load
00000000003ae800 l F .text 0000000000000003 runtime/internal/atomic.(*Uint32).LoadAcquire
00000000003ae820 l F .text 0000000000000003 runtime/internal/atomic.(*Uint32).Store
00000000003ae840 l F .text 0000000000000003 runtime/internal/atomic.(*Uint32).StoreRelease
00000000003ae860 l F .text 000000000000000d runtime/internal/atomic.(*Uint32).CompareAndSwap
00000000003ae880 l F .text 000000000000000d runtime/internal/atomic.(*Uint32).CompareAndSwapRelease
00000000003ae8a0 l F .text 0000000000000005 runtime/internal/atomic.(*Uint32).Swap
00000000003ae8c0 l F .text 0000000000000004 runtime/internal/atomic.(*Uint32).And
00000000003ae8e0 l F .text 0000000000000004 runtime/internal/atomic.(*Uint32).Or
错误的信息如下,会提示no symbols
,这种二进制无法进行debug
root@containerd:~/workspace/containerd#
root@containerd:~/workspace/containerd# objdump --syms /usr/local/bin/containerd/usr/local/bin/containerd: file format elf64-x86-64SYMBOL TABLE:
no symbolsroot@containerd:~/workspace/containerd#
root@containerd:~/workspace/containerd#
root@containerd:~/workspace/containerd#
3.2. file工具验证
正确信息如下,会显示with debug_info, not stripped
root@containerd:~/workspace/containerd#
root@containerd:~/workspace/containerd#
root@containerd:~/workspace/containerd# file bin/containerd
bin/containerd: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=9bb291e61e1eceb23359dc29100845e5c1edf763, for GNU/Linux 3.2.0, with debug_info, not stripped
root@containerd:~/workspace/containerd#
root@containerd:~/workspace/containerd#
错误信息如下,会显示stripped
,说明不包含调试信息,无法进行debug
root@containerd:~/workspace/containerd#
root@containerd:~/workspace/containerd# file /usr/local/bin/containerd
/usr/local/bin/containerd: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=322f89b7e351fe2ccfaa0fe30de79c76d49d6e26, for GNU/Linux 3.2.0, stripped
root@containerd:~/workspace/containerd#
3.3. dlv工具验证
正确的调试信息如下:
root@containerd:~/workspace/containerd# dlv exec bin/containerd
Type 'help' for list of commands.
(dlv)
(dlv)
(dlv)
(dlv)
(dlv)
(dlv) exit
root@containerd:~/workspace/containerd#
root@containerd:~/workspace/containerd#
错误的调试信息如下,会提示:no debug info found
root@containerd:~/workspace/containerd#
root@containerd:~/workspace/containerd# dlv exec /usr/local/bin/containerd
Warning: no debug info found, some functionality will be missing such as stack traces and variable evaluation.
Type 'help' for list of commands.
(dlv)
(dlv)
(dlv)
(dlv)
(dlv) exit
4. debug
debug containerd
步骤如下:
- 1、利用
ps -ef|gerp containerd
查看containerd
启动所需要的参数,这一步特别重要,尤其是在debug k8s
源码的时候,k8s
的每一个组件都带了很多命令行参数,想要调试这些组件,必须把这些组件的启动参数原封不动的加入到dlv
调试命令当中- 注意,实际上执行上述命令之后会发现,
containerd
并没有启动参数,因此无需关心。
- 注意,实际上执行上述命令之后会发现,
- 2、通过
dlv
命令启动containerd
,启动命令我们可以从IDEA remote debug
功能拷贝过来dlv --listen=:12345 --headless=true --api-version=2 --accept-multiclient exec bin/containerd
- 注意:这里在启动
contaienrd
的时候没有指定任何参数,实际上也可以根据自己的情况加入containerd
参数,譬如指定containerd
的配置文件的位置(containerd
默认配置文件为:/etc/containerd/config.toml
),也可以指定调试的debug
级别,譬如:dlv --listen=:12345 --headless=true --api-version=2 --accept-multiclient exec bin/containerd -- --config=/root/mycontainerd/config.toml --log-level=debug
- 3、在
IDEA
启动debug
,连接到远程调试- 注意:在启动
IDEA
调试之前,你需要在想要debug
的位置增加断点,否则程序启动会直接运行起来,等你这个时候打断点,很可能就晚了。 - 注意:在启动
IDEA
调试之前,你需要修改IDEA
操作系统标识为Linux
- 注意:在启动
执行dlv --listen=:12345 --headless=true --api-version=2 --accept-multiclient exec bin/containerd -- --log-level=debug
命令之后,此时会阻塞在这里,千万不要使用ctrl + c
,只有当IDEA
连接上来的时候才会开始执行
root@containerd:~/workspace/containerd#
root@containerd:~/workspace/containerd# dlv --listen=:12345 --headless=true --api-version=2 --accept-multiclient exec bin/containerd -- --log-level=debug
API server listening at: [::]:12345
2023-07-29T15:06:45+08:00 warning layer=rpc Listening for remote connections (connections are not authenticated nor encrypted)