ubuntu 20.04 搭建crash dump问题分析环境

ubuntu 20.04 搭建crash dump问题分析环境

  • 1 安装依赖软件
    • 1.1 linux-dump
    • 1.2 kexec-tools
    • 1.3 安装crash工具
    • 1.4 安装gdb调试工具
    • 1.5 安装ubuntu内核调试符号
      • 1.5.1 GPG 秘钥导入
      • 1.5.2 添加仓库配置
      • 1.5.3 更新软件包
      • 1.5.4 下载和安装内核调试符号
      • 1.5.5 验证内核调试符号已经被安装
    • 1.6 配置转储内存大小
  • 2 触发系统异常验证
    • 2.1 开启kdump服务
    • 2.2 查看kdump服务当前的状态
    • 2.3 手动触发crash dump
      • 2.3.1 切换为 root 用户
      • 2.3.2 触发crash dump
  • 3 kernel dump文件分析
    • 3.1 crash dump文件的位置
    • 3.2 crash工具分析crash dump文件
  • 4 遇到的问题以及解决办法
    • 4.1 'makeinfo' is missing on your system
      • 4.1.1 问题现象
      • 4.1.2 解决方法
    • 4.2 crash工具和主机安装的gdb版本不一致导致的异常
      • 4.2.1 问题的现象
      • 4.2.2 解决办法
    • 4.3 /dev/mem: Operation not permitted
      • 4.3.1 问题的现象
      • 4.3.2 解决方法

主机环境和内核版本信息:

test@test:~/software/crash$ uname -r
5.15.0-74-generic
test@test:~/software/crash$test@test:~/software/crash$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.6 LTS"
test@test:~/software/crash$

1 安装依赖软件

参考了Ubuntu Kernel crash dump这篇文章

1.1 linux-dump

sudo apt install linux-crashdump

1.2 kexec-tools

sudo apt-get install kexec-tools

1.3 安装crash工具

sudo apt install crash

1.4 安装gdb调试工具

sudo apt-get install gdb

1.5 安装ubuntu内核调试符号

sudo apt-get install linux-image-$(uname -r)-dbgsym

如果通过上面的命令安装ubuntu内核调试符号,则需要通过下面的方法去安装,参考文章:安装ubuntu内核调试符号

1.5.1 GPG 秘钥导入

确保您拥有系统的 GPG 密钥。适用于16.04 及更高版本的 Ubuntu :

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C8CAB6595FDFF622

对于旧的发布版本:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys ECDCAD72428D7C0

1.5.2 添加仓库配置

codename=$(lsb_release -c | awk  '{print $2}')
sudo tee /etc/apt/sources.list.d/ddebs.list << EOF
deb http://ddebs.ubuntu.com/ ${codename}      main restricted universe multiverse
deb http://ddebs.ubuntu.com/ ${codename}-security main restricted universe multiverse
deb http://ddebs.ubuntu.com/ ${codename}-updates  main restricted universe multiverse
deb http://ddebs.ubuntu.com/ ${codename}-proposed main restricted universe multiverse
EOF

1.5.3 更新软件包

sudo apt-get update

1.5.4 下载和安装内核调试符号

sudo apt-get install linux-image-$(uname -r)-dbgsym

1.5.5 验证内核调试符号已经被安装

包含调试信息的文件名为 vmlinux-XXX-debug,其中 XXX 是内核版本。该文件存储在 /usr/lib/debug/boot 目录下。

1.6 配置转储内存大小

修改配置/etc/default/grub.d/kdump-tools.cfg,以支持kernel crash dump现场保存。

GRUB_CMDLINE_LINUX_DEFAULT="$GRUB_CMDLINE_LINUX_DEFAULT crashkernel=384M-:512M"

2 触发系统异常验证

2.1 开启kdump服务

systemctl start kdump-tools-dump.servicesystemctl enable kdump-tools-dump.service

2.2 查看kdump服务当前的状态

test@test:~/software/crash$ service kdump-tools-dump status
● kdump-tools-dump.service - Kernel crash dump capture serviceLoaded: loaded (/lib/systemd/system/kdump-tools-dump.service; static; vendor preset: enabled)Active: active (exited) since Tue 2023-11-14 10:19:58 CST; 56s agoMain PID: 126662 (code=exited, status=0/SUCCESS)Tasks: 0 (limit: 37610)Memory: 0BCGroup: /system.slice/kdump-tools-dump.service1114 10:19:58 test systemd[1]: Starting Kernel crash dump capture service...
1114 10:19:58 test kdump-tools[126662]: Starting kdump-tools:
1114 10:19:58 test kdump-tools[126669]:  * Cannot change symbolic links when kdump is loaded
1114 10:19:58 test systemd[1]: Finished Kernel crash dump capture service.
test@test:~/software/crash$

2.3 手动触发crash dump

2.3.1 切换为 root 用户

sudo -s

2.3.2 触发crash dump

echo 1 > /proc/sys/kernel/sysrqecho c > /proc/sysrq-trigger

3 kernel dump文件分析

3.1 crash dump文件的位置

test@test:~/software/crash$ ls /var/crash/
202311132045  kexec_cmd                                         linux-image-5.15.0-74-generic-202311132357.crash          _opt_kingsoft_wps-office_office6_wpscloudsvr.1000.uploaded  _usr_bin_crash.0.uploaded
202311132231  linux-image-5.15.0-74-generic-202311132045.crash  _opt_kingsoft_wps-office_office6_wpscloudsvr.1000.crash   _usr_bin_crash.0.crash
202311132357  linux-image-5.15.0-74-generic-202311132231.crash  _opt_kingsoft_wps-office_office6_wpscloudsvr.1000.upload  _usr_bin_crash.0.upload
test@test:~/software/crash$

3.2 crash工具分析crash dump文件

test@test:~/software/crash$ sudo crash -d /usr/bin/gdb /usr/lib/debug/boot/vmlinux-5.15.0-74-generic /var/crash/202311132357/dump.202311132357 crash 8.0.3++
Copyright (C) 2002-2022  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011, 2020-2022  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
Copyright (C) 2015, 2021  VMware, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.GNU gdb (GDB) 10.2
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
Find the GDB manual and other documentation resources online at:<http://www.gnu.org/software/gdb/documentation/>.For help, type "help".
Type "apropos word" to search for commands related to "word"...KERNEL: /usr/lib/debug/boot/vmlinux-5.15.0-74-genericDUMPFILE: /var/crash/202311132357/dump.202311132357  [PARTIAL DUMP]CPUS: 16DATE: Mon Nov 13 23:56:58 CST 2023UPTIME: 00:11:00
LOAD AVERAGE: 0.71, 0.84, 0.58TASKS: 1636NODENAME: testRELEASE: 5.15.0-74-genericVERSION: #81~20.04.2-Ubuntu SMP Fri May 26 19:56:20 UTC 2023MACHINE: x86_64  (2900 Mhz)MEMORY: 31.8 GBPANIC: "Kernel panic - not syncing: sysrq triggered crash"PID: 10269COMMAND: "bash"TASK: ffff89a4d1334d40  [THREAD_INFO: ffff89a4d1334d40]CPU: 0STATE: TASK_RUNNING (PANIC)crash> bt
PID: 10269    TASK: ffff89a4d1334d40  CPU: 0    COMMAND: "bash"#0 [ffff94d441397c48] machine_kexec at ffffffff9688afe0#1 [ffff94d441397ca8] __crash_kexec at ffffffff96998c02#2 [ffff94d441397d78] panic at ffffffff974cf4b6#3 [ffff94d441397df8] sysrq_handle_crash at ffffffff96feea5a#4 [ffff94d441397e08] __handle_sysrq.cold at ffffffff97524bf2#5 [ffff94d441397e50] write_sysrq_trigger at ffffffff96fef548#6 [ffff94d441397e68] proc_reg_write at ffffffff96c23b57#7 [ffff94d441397e88] vfs_write at ffffffff96b83a76#8 [ffff94d441397ec0] ksys_write at ffffffff96b85de7#9 [ffff94d441397f00] __x64_sys_write at ffffffff96b85e8a
#10 [ffff94d441397f10] do_syscall_64 at ffffffff97571d39
#11 [ffff94d441397f28] do_syscall_64 at ffffffff97571d49
#12 [ffff94d441397f50] entry_SYSCALL_64_after_hwframe at ffffffff97600099RIP: 00007fc202171077  RSP: 00007ffe9fcff018  RFLAGS: 00000246RAX: ffffffffffffffda  RBX: 0000000000000002  RCX: 00007fc202171077RDX: 0000000000000002  RSI: 000055c60be1a9e0  RDI: 0000000000000001RBP: 000055c60be1a9e0   R8: 000000000000000a   R9: 0000000000000001R10: 000055c60b144017  R11: 0000000000000246  R12: 0000000000000002R13: 00007fc2022506a0  R14: 00007fc20224c4a0  R15: 00007fc20224b8a0ORIG_RAX: 0000000000000001  CS: 0033  SS: 002b
crash>

4 遇到的问题以及解决办法

4.1 ‘makeinfo’ is missing on your system

4.1.1 问题现象

/home/test/software/crash/gdb-10.2/missing: line 81: makeinfo: command not found
WARNING: 'makeinfo' is missing on your system.You should only need it if you modified a '.texi' file, orany other file indirectly affecting the aspect of the manual.You might want to install the Texinfo package:<http://www.gnu.org/software/texinfo/>The spurious makeinfo call might also be the consequence ofusing a buggy 'make' (AIX, DU, IRIX), in which case you mightwant to install GNU make:<http://www.gnu.org/software/make/>
make[5]: *** [Makefile:542: bfd.info] Error 127
make[4]: *** [Makefile:1643: info-recursive] Error 1
make[3]: *** [Makefile:2771: all-bfd] Error 2
make[3]: *** Waiting for unfinished jobs....
config.status: creating Makefile
config.status: creating import/Makefile
config.status: creating config.h
config.status: executing depfiles commands
config.status: executing default commands
make[2]: *** [Makefile:860: all] Error 2
crash build failed
make[1]: *** [Makefile:263: gdb_merge] Error 1
make: *** [Makefile:254: all] Error 2

4.1.2 解决方法

sudo apt-get update
sudo apt-get install texinfo

4.2 crash工具和主机安装的gdb版本不一致导致的异常

4.2.1 问题的现象

GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...WARNING: kernel relocated [344MB]: patching 145829 gdb minimal_symbol valuesplease wait... (patching 145829 gdb minimal_symbol values) Segmentation faulttest@test:~/software/crash$ gdb --version
GNU gdb (Ubuntu 10.2-0ubuntu1~20.04~1) 10.2
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
test@test:~/software/crash$

4.2.2 解决办法

sudo apt purge crashgit clone https://github.com/crash-utility/crash.gitcd crashmake -j8
sudo make install
crash --version

4.3 /dev/mem: Operation not permitted

4.3.1 问题的现象

sudo crash /usr/lib/debug/boot/vmlinux-5.15.0-74-genericcrash 7.2.8
Copyright (C) 2002-2020  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.crash: /dev/mem: Operation not permitted

4.3.2 解决方法

参考1.6节,修改/etc/default/grub.d/kdump-tools.cfg文件,添加下面的配置以支持

GRUB_CMDLINE_LINUX_DEFAULT="$GRUB_CMDLINE_LINUX_DEFAULT` crashkernel=384M-:512M"

然后修改/dev/mem的权限

sudo chmod 777 /dev/mem

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.mzph.cn/news/155925.shtml

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

SELinux零知识学习二十三、SELinux策略语言之类型强制(8)

接前一篇文章&#xff1a;SELinux零知识学习二十二、SELinux策略语言之类型强制&#xff08;7&#xff09; 二、SELinux策略语言之类型强制 3. 访问向量规则 AV规则就是按照对客体类别的访问许可指定具体含义的规则&#xff0c;SELinux策略语言目前支持四类AV规则&#xff1a…

SpringBoot学习笔记-创建个人中心页面(下)

笔记内容转载自 AcWing 的 SpringBoot 框架课讲义&#xff0c;课程链接&#xff1a;AcWing SpringBoot 框架课。 CONTENTS 1. 实现个人中心页面2. POJO时区修改3. 集成代码编辑器 本节实现个人中心的前端页面&#xff0c;用户能够查看自己的 Bot 信息&#xff0c;并能创建、修改…

【Echart】Echart设置label太长隐藏:

文章目录 第一种&#xff1a;竖排显示第二种&#xff1a;显示部分第三种&#xff1a;强制显示所有标签并旋转 第一种&#xff1a;竖排显示 xAxis: {type: category,data: res.data.data.sz.xAxis,axisLabel:{fontSize:12,formatter: function(value) {return value.split().joi…

线程池有几种创建方式?

程序员的公众号&#xff1a;源1024&#xff0c;获取更多资料&#xff0c;无加密无套路&#xff01; 最近整理了一波电子书籍资料&#xff0c;包含《Effective Java中文版 第2版》《深入JAVA虚拟机》&#xff0c;《重构改善既有代码设计》&#xff0c;《MySQL高性能-第3版》&…

人员定位系统人员聚集风险监测预警功能为企业安全生产保驾护航!

危化企业生产区域由于装置设施开停工、设备检维修或其它原因&#xff0c;会导致在局部作业区域出现作业操作人员、指挥管理人员不定时、不定点的聚集情况&#xff0c;一旦发生泄漏中毒、火灾爆炸、高温高热物质喷溅等事故&#xff0c;人员聚集将会在一定程度上加重事故后果的严…

【测试开发】学习测试前的基础知识

文章目录 目录 前言 一、什么是软件测试 二、软件测试的发展史 三、软件测试和研发的区别 四、测试人员应该具备的能力 五、测试所需要知道的基本概念 1.需求 六、软件错误&#xff08;BUG&#xff09;的概念 前言 本篇文章主要讲解在学习测试前一些需要了解的知识。 一、什…

2023年【A特种设备相关管理(锅炉压力容器压力管道)】模拟考试题及A特种设备相关管理(锅炉压力容器压力管道)作业考试题库

题库来源&#xff1a;安全生产模拟考试一点通公众号小程序 A特种设备相关管理&#xff08;锅炉压力容器压力管道&#xff09;模拟考试题参考答案及A特种设备相关管理&#xff08;锅炉压力容器压力管道&#xff09;考试试题解析是安全生产模拟考试一点通题库老师及A特种设备相关…

【优秀毕设】基于vue+ssm+springboot的网上购物商城系统设计

摘 要 随着科学技术的飞速发展&#xff0c;社会的方方面面、各行各业都在努力与现代的先进技术接轨&#xff0c;通过科技手段来提高自身的优势&#xff0c;网上商城购物系统当然也不能排除在外。网上商城购物系统是以实际运用为开发背景&#xff0c;运用软件工程原理和开发方…

物联网AI MicroPython学习之语法 PWM脉宽调制模块

学物联网&#xff0c;来万物简单IoT物联网&#xff01;&#xff01; PWM 介绍 模块功能: PWM脉宽调制驱动模块 接口说明 PWM - 构建PWM对象 函数原型&#xff1a;PWM(ch, freq, duty)参数说明&#xff1a; 参数类型必选参数&#xff1f;说明chobjectYPin对象例如&#xf…

3、LeetCode之无重复字符的最长子串

给定一个字符串 s &#xff0c;请你找出其中不含有重复字符的 最长子串 的长度 输入: s "abcabcbb" 输出: 3 解释: 因为无重复字符的最长子串是 "abc"&#xff0c;所以其长度为 3。转载&#xff1a; C常用语法——unordered_set 题目主要思想&#xff…

jmeter 主从配置

配置 主从需要在同一子网。 可以修改从机的端口&#xff0c;主要是防火墙的原因&#xff0c;这样就可以在防火墙只配置一个端口&#xff0c;如果没有防火墙&#xff0c;可以不用配置 server_port1099 server.rmi.localport1099 centos防火墙开端口 sudo firewall-cmd --zonep…

直播间弹幕直播游戏开发教程

随着直播技术的不断发展&#xff0c;交互式弹幕直播游戏成为吸引用户参与的新兴方式。这种游戏融合了实时弹幕互动和直播视频&#xff0c;为观众和主播提供了更加丰富的互动体验。在这篇文章中&#xff0c;我们将探讨从概念到实现的步骤&#xff0c;帮助你打造一款引人入胜的交…

20天拿下华为OD笔试之【DP/贪心】2023B-观看文艺汇演【欧弟算法】全网注释最详细分类最全的华为OD真题题解

文章目录 【DP/贪心】2023B-观看文艺汇演题目描述与示例输入输出示例一输入输出说明 示例二输入输出说明 示例三输入输出 解题思路原始数据处理贪心思想求解问题动态规划求解问题 代码解法一&#xff1a;贪心时空复杂度 解法二&#xff1a;DP时空复杂度 华为OD算法冲刺训练 【D…

Frida hook android 应用程序

Frida hook android 应用程序 之前已经讲过了frida的安装和使用&#xff0c;本篇文章主要讲下怎么hook android 的应用程序。 1&#xff1a;编写hook脚本 如下test.js Java.perform(function () {var WelcomeActivity Java.use(com.tencent.mm.plugin.account.ui.WelcomeA…

五年程序员兼职接单的肺腑之言

不知不觉我已经参加工作&#xff0c;当一个程序员五年了&#xff0c;从一个职场菜鸟逐渐变成老油条&#xff0c;个中辛酸只有自己知道。这五年做过各种兼职接单&#xff0c;踩过不少坑&#xff0c;今天就把我在程序员接单上的一些心得体会分享给大家&#xff0c;希望能对兼职接…

【开题报告】基于uni-app的恋爱打卡app的设计与实现

1.选题背景 如今&#xff0c;随着移动互联网的普及和人们对生活品质的追求&#xff0c;恋爱已经成为了许多人关注的焦点。然而&#xff0c;在恋爱过程中&#xff0c;由于种种原因&#xff0c;往往会忽略掉一些重要的时刻和细节&#xff0c;导致感情降温甚至破裂。因此&#xf…

poetry执行报错 Reason: tried: ‘/opt/homebrew/Cellar/python@x.x

报错如下&#xff1a; ➜ poetry shell ➜ poetry run uvicorn main:app --reload --port 7000 dyld[42259]: Library not loaded: /opt/homebrew/Cellar/python3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/PythonReferenced from: <63F55A2A-2EB4-35B8-9170-9…

Redis--字符串

基础概念 字符串、数字&#xff0c;都会转化为字符串来进行存储 以二进制的方式存储在内存中 key的命名规范 key不宜过长&#xff0c;消耗内存&#xff0c;且在数据查找的这类键值的计算成本高&#xff08;通过hash计算来查找&#xff09;不宜过短&#xff0c;可读性查值一…

设计模式-命令模式-笔记

“行为变化”模式 在组件的构建过程中&#xff0c;组件行为的变化经常导致组件本身剧烈的变化。“行为变化”模式组件的行为和组件本身进行解耦&#xff0c;从而支持组件行为的变化&#xff0c;实现两者之间的松耦合。 经典模式&#xff1a;Command、Visitor 动机&#xff0…

C# - Opencv应用(2) 之矩阵Mat使用[矩阵创建、图像显示、像素读取与赋值]

C# - Opencv应用&#xff08;2&#xff09; 之矩阵Mat使用[矩阵创建、图像显示、像素读取与赋值] 矩阵创建图像显示与保存像素读取与赋值新建sample02项目&#xff0c;配置opencv4相关包&#xff0c;新建.cs进行测试 1.矩阵创建 //创建空白矩阵 var dst new Mat()//创建并赋…