jvm调优工具arthas(阿尔萨斯)安装与使用—实践
Arthas
是Alibaba开源的Java诊断工具,深受开发者喜爱。
当你遇到以下类似问题而束手无策时,Arthas
可以帮助你解决:
- 这个类从哪个 jar 包加载的?为什么会报各种类相关的 Exception?
- 我改的代码为什么没有执行到?难道是我没 commit?分支搞错了?
- 遇到问题无法在线上 debug,难道只能通过加日志再重新发布吗?
- 线上遇到某个用户的数据处理有问题,但线上同样无法 debug,线下无法重现!
- 是否有一个全局视角来查看系统的运行状况?
- 有什么办法可以监控到JVM的实时运行状态?
- 怎么快速定位应用的热点,生成火焰图?
- 怎样直接从JVM内查找某个类的实例?
Arthas
支持JDK 6+(4.x 版本不再支持 JDK 6 和 JDK 7),支持Linux/Mac/Windows,采用命令行交互模式,同时提供丰富的 Tab
自动补全功能,进一步方便进行问题的定位和诊断。
官方文档
1. 启动 math-game
curl -O https://arthas.aliyun.com/math-game.jar
java -jar math-game.jar
math-game
是一个简单的程序,每隔一秒生成一个随机数,再执行质因数分解,并打印出分解结果。
2. 启动 arthas
在命令行下面执行(使用和目标进程一致的用户启动,否则可能 attach 失败):
curl -O https://arthas.aliyun.com/arthas-boot.jar
java -jar arthas-boot.jar
- 执行该程序的用户需要和目标进程具有相同的权限。比如以
admin
用户来执行:sudo su admin && java -jar arthas-boot.jar
或sudo -u admin -EH java -jar arthas-boot.jar
。 - 如果 attach 不上目标进程,可以查看
~/logs/arthas/
目录下的日志。 - 如果下载速度比较慢,可以使用 aliyun 的镜像:
java -jar arthas-boot.jar --repo-mirror aliyun --use-http
java -jar arthas-boot.jar -h
打印更多参数信息。
选择应用 java 进程:
$ $ java -jar arthas-boot.jar
* [1]: 35542[2]: 71560 math-game.jar
math-game
进程是第 2 个,则输入 2,再输入回车/enter
。Arthas 会 attach 到目标进程上,并输出日志:
[INFO] Try to attach process 71560
[INFO] Attach process 71560 success.
[INFO] arthas-client connect 127.0.0.1 3658,---. ,------. ,--------.,--. ,--. ,---. ,---./ O \ | .--. ''--. .--'| '--' | / O \ ' .-'
| .-. || '--'.' | | | .--. || .-. |`. `-.
| | | || |\ \ | | | | | || | | |.-' |
`--' `--'`--' '--' `--' `--' `--'`--' `--'`-----'wiki: https://arthas.aliyun.com/doc
version: 3.0.5.20181127201536
pid: 71560
time: 2018-11-28 19:16:24$
3. 查看 dashboard
输入dashboard,按回车/enter
,会展示当前进程的信息,按ctrl+c
可以中断执行。
$ dashboard
ID NAME GROUP PRIORI STATE %CPU TIME INTERRU DAEMON
17 pool-2-thread-1 system 5 WAITIN 67 0:0 false false
27 Timer-for-arthas-dashb system 10 RUNNAB 32 0:0 false true
11 AsyncAppender-Worker-a system 9 WAITIN 0 0:0 false true
9 Attach Listener system 9 RUNNAB 0 0:0 false true
3 Finalizer system 8 WAITIN 0 0:0 false true
2 Reference Handler system 10 WAITIN 0 0:0 false true
4 Signal Dispatcher system 9 RUNNAB 0 0:0 false true
26 as-command-execute-dae system 10 TIMED_ 0 0:0 false true
13 job-timeout system 9 TIMED_ 0 0:0 false true
1 main main 5 TIMED_ 0 0:0 false false
14 nioEventLoopGroup-2-1 system 10 RUNNAB 0 0:0 false false
18 nioEventLoopGroup-2-2 system 10 RUNNAB 0 0:0 false false
23 nioEventLoopGroup-2-3 system 10 RUNNAB 0 0:0 false false
15 nioEventLoopGroup-3-1 system 10 RUNNAB 0 0:0 false false
Memory used total max usage GC
heap 32M 155M 1820M 1.77% gc.ps_scavenge.count 4
ps_eden_space 14M 65M 672M 2.21% gc.ps_scavenge.time(m 166
ps_survivor_space 4M 5M 5M s)
ps_old_gen 12M 85M 1365M 0.91% gc.ps_marksweep.count 0
nonheap 20M 23M -1 gc.ps_marksweep.time( 0
code_cache 3M 5M 240M 1.32% ms)
Runtime
os.name Mac OS X
os.version 10.13.4
java.version 1.8.0_162
java.home /Library/Java/JavaVirtualMachines/jdk1.8.0_162.jdk/Contents/Home/jre
4. 通过 thread 命令来获取到math-game
进程的 Main Class
thread 1
会打印线程 ID 1 的栈,通常是 main 函数的线程。
$ thread 1 | grep 'main('at demo.MathGame.main(MathGame.java:17)
5. 通过 jad 来反编译 Main Class
$ jad demo.MathGameClassLoader:
+-sun.misc.Launcher$AppClassLoader@3d4eac69+-sun.misc.Launcher$ExtClassLoader@66350f69Location:
/tmp/math-game.jar/** Decompiled with CFR 0_132.*/
package demo;import java.io.PrintStream;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.Random;
import java.util.concurrent.TimeUnit;public class MathGame {private static Random random = new Random();private int illegalArgumentCount = 0;public static void main(String[] args) throws InterruptedException {MathGame game = new MathGame();do {game.run();TimeUnit.SECONDS.sleep(1L);} while (true);}public void run() throws InterruptedException {try {int number = random.nextInt();List<Integer> primeFactors = this.primeFactors(number);MathGame.print(number, primeFactors);}catch (Exception e) {System.out.println(String.format("illegalArgumentCount:%3d, ", this.illegalArgumentCount) + e.getMessage());}}public static void print(int number, List<Integer> primeFactors) {StringBuffer sb = new StringBuffer("" + number + "=");Iterator<Integer> iterator = primeFactors.iterator();while (iterator.hasNext()) {int factor = iterator.next();sb.append(factor).append('*');}if (sb.charAt(sb.length() - 1) == '*') {sb.deleteCharAt(sb.length() - 1);}System.out.println(sb);}public List<Integer> primeFactors(int number) {if (number < 2) {++this.illegalArgumentCount;throw new IllegalArgumentException("number is: " + number + ", need >= 2");}ArrayList<Integer> result = new ArrayList<Integer>();int i = 2;while (i <= number) {if (number % i == 0) {result.add(i);number /= i;i = 2;continue;}++i;}return result;}
}Affect(row-cnt:1) cost in 970 ms.
6. watch
通过watch命令来查看demo.MathGame#primeFactors
函数的返回值:
$ watch demo.MathGame primeFactors returnObj
Press Ctrl+C to abort.
Affect(class-cnt:1 , method-cnt:1) cost in 107 ms.
ts=2018-11-28 19:22:30; [cost=1.715367ms] result=null
ts=2018-11-28 19:22:31; [cost=0.185203ms] result=null
ts=2018-11-28 19:22:32; [cost=19.012416ms] result=@ArrayList[@Integer[5],@Integer[47],@Integer[2675531],
]
ts=2018-11-28 19:22:33; [cost=0.311395ms] result=@ArrayList[@Integer[2],@Integer[5],@Integer[317],@Integer[503],@Integer[887],
]
ts=2018-11-28 19:22:34; [cost=10.136007ms] result=@ArrayList[@Integer[2],@Integer[2],@Integer[3],@Integer[3],@Integer[31],@Integer[717593],
]
ts=2018-11-28 19:22:35; [cost=29.969732ms] result=@ArrayList[@Integer[5],@Integer[29],@Integer[7651739],
]
7. 退出 arthas
如果只是退出当前的连接,可以用quit
或者exit
命令。Attach 到目标进程上的 arthas 还会继续运行,端口会保持开放,下次连接时可以直接连接上。
如果想完全退出 arthas,可以执行stop
命令。
参考文档
-
在线教程(推荐)
-
用户文档
-
安装
-
下载
-
快速入门
-
进阶使用
-
命令列表
-
WebConsole
-
Docker
-
Arthas Spring Boot Starter
-
用户案例
-
FAQ/常见问题
-
编译调试/参与贡献
-
Release Notes
案例展示
Dashboard
- https://arthas.aliyun.com/doc/dashboard
Thread
- https://arthas.aliyun.com/doc/thread
一目了然的了解系统的状态,哪些线程比较占cpu?他们到底在做什么?
$ thread -n 3
"as-command-execute-daemon" Id=29 cpuUsage=75% RUNNABLEat sun.management.ThreadImpl.dumpThreads0(Native Method)at sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:440)at com.taobao.arthas.core.command.monitor200.ThreadCommand$1.action(ThreadCommand.java:58)at com.taobao.arthas.core.command.handler.AbstractCommandHandler.execute(AbstractCommandHandler.java:238)at com.taobao.arthas.core.command.handler.DefaultCommandHandler.handleCommand(DefaultCommandHandler.java:67)at com.taobao.arthas.core.server.ArthasServer$4.run(ArthasServer.java:276)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)at java.lang.Thread.run(Thread.java:745)Number of locked synchronizers = 1- java.util.concurrent.ThreadPoolExecutor$Worker@6cd0b6f8"as-session-expire-daemon" Id=25 cpuUsage=24% TIMED_WAITINGat java.lang.Thread.sleep(Native Method)at com.taobao.arthas.core.server.DefaultSessionManager$2.run(DefaultSessionManager.java:85)"Reference Handler" Id=2 cpuUsage=0% WAITING on java.lang.ref.Reference$Lock@69ba0f27at java.lang.Object.wait(Native Method)- waiting on java.lang.ref.Reference$Lock@69ba0f27at java.lang.Object.wait(Object.java:503)at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
jad
- https://arthas.aliyun.com/doc/jad
对类进行反编译:
$ jad javax.servlet.ServletClassLoader:
+-java.net.URLClassLoader@6108b2d7+-sun.misc.Launcher$AppClassLoader@18b4aac2+-sun.misc.Launcher$ExtClassLoader@1ddf84b8Location:
/Users/xxx/work/test/lib/servlet-api.jar/** Decompiled with CFR 0_122.*/
package javax.servlet;import java.io.IOException;
import javax.servlet.ServletConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;public interface Servlet {public void init(ServletConfig var1) throws ServletException;public ServletConfig getServletConfig();public void service(ServletRequest var1, ServletResponse var2) throws ServletException, IOException;public String getServletInfo();public void destroy();
}
mc
- https://arthas.aliyun.com/doc/mc
Memory Compiler/内存编译器,编译.java
文件生成.class
。
mc /tmp/Test.java
retransform
- https://arthas.aliyun.com/doc/retransform
加载外部的.class
文件,retransform 热更新jvm已加载的类。
retransform /tmp/Test.class
retransform -c 327a647b /tmp/Test.class /tmp/Test\$Inner.class
sc
- https://arthas.aliyun.com/doc/sc
查找JVM中已经加载的类
$ sc -d org.springframework.web.context.support.XmlWebApplicationContextclass-info org.springframework.web.context.support.XmlWebApplicationContextcode-source /Users/xxx/work/test/WEB-INF/lib/spring-web-3.2.11.RELEASE.jarname org.springframework.web.context.support.XmlWebApplicationContextisInterface falseisAnnotation falseisEnum falseisAnonymousClass falseisArray falseisLocalClass falseisMemberClass falseisPrimitive falseisSynthetic falsesimple-name XmlWebApplicationContextmodifier publicannotationinterfacessuper-class +-org.springframework.web.context.support.AbstractRefreshableWebApplicationContext+-org.springframework.context.support.AbstractRefreshableConfigApplicationContext+-org.springframework.context.support.AbstractRefreshableApplicationContext+-org.springframework.context.support.AbstractApplicationContext+-org.springframework.core.io.DefaultResourceLoader+-java.lang.Objectclass-loader +-org.apache.catalina.loader.ParallelWebappClassLoader+-java.net.URLClassLoader@6108b2d7+-sun.misc.Launcher$AppClassLoader@18b4aac2+-sun.misc.Launcher$ExtClassLoader@1ddf84b8classLoaderHash 25131501
vmtool
- https://arthas.aliyun.com/doc/vmtool
从JVM heap中获取指定类的实例。
$ vmtool --action getInstances --className java.lang.String --limit 10
@String[][@String[com/taobao/arthas/core/shell/session/Session],@String[com.taobao.arthas.core.shell.session.Session],@String[com/taobao/arthas/core/shell/session/Session],@String[com/taobao/arthas/core/shell/session/Session],@String[com/taobao/arthas/core/shell/session/Session.class],@String[com/taobao/arthas/core/shell/session/Session.class],@String[com/taobao/arthas/core/shell/session/Session.class],@String[com/],@String[java/util/concurrent/ConcurrentHashMap$ValueIterator],@String[java/util/concurrent/locks/LockSupport],
]
stack
- https://arthas.aliyun.com/doc/stack
查看方法 test.arthas.TestStack#doGet
的调用堆栈:
$ stack test.arthas.TestStack doGet
Press Ctrl+C to abort.
Affect(class-cnt:1 , method-cnt:1) cost in 286 ms.
ts=2018-09-18 10:11:45;thread_name=http-bio-8080-exec-10;id=d9;is_daemon=true;priority=5;TCCL=org.apache.catalina.loader.ParallelWebappClassLoader@25131501@test.arthas.TestStack.doGet()at javax.servlet.http.HttpServlet.service(HttpServlet.java:624)at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)...at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:451)at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1121)at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)at java.lang.Thread.run(Thread.java:745)
Trace
- https://arthas.aliyun.com/doc/trace
观察方法执行的时候哪个子调用比较慢:
Watch
- https://arthas.aliyun.com/doc/watch
观察方法 test.arthas.TestWatch#doGet
执行的入参,仅当方法抛出异常时才输出。
$ watch test.arthas.TestWatch doGet {params[0], throwExp} -e
Press Ctrl+C to abort.
Affect(class-cnt:1 , method-cnt:1) cost in 65 ms.
ts=2018-09-18 10:26:28;result=@ArrayList[@RequestFacade[org.apache.catalina.connector.RequestFacade@79f922b2],@NullPointerException[java.lang.NullPointerException],
]
Monitor
- https://arthas.aliyun.com/doc/monitor
监控某个特殊方法的调用统计数据,包括总调用次数,平均rt,成功率等信息,每隔5秒输出一次。
$ monitor -c 5 org.apache.dubbo.demo.provider.DemoServiceImpl sayHello
Press Ctrl+C to abort.
Affect(class-cnt:1 , method-cnt:1) cost in 109 ms.timestamp class method total success fail avg-rt(ms) fail-rate
----------------------------------------------------------------------------------------------------------------------------2018-09-20 09:45:32 org.apache.dubbo.demo.provider.DemoServiceImpl sayHello 5 5 0 0.67 0.00%timestamp class method total success fail avg-rt(ms) fail-rate
----------------------------------------------------------------------------------------------------------------------------2018-09-20 09:45:37 org.apache.dubbo.demo.provider.DemoServiceImpl sayHello 5 5 0 1.00 0.00%timestamp class method total success fail avg-rt(ms) fail-rate
----------------------------------------------------------------------------------------------------------------------------2018-09-20 09:45:42 org.apache.dubbo.demo.provider.DemoServiceImpl sayHello 5 5 0 0.43 0.00%
Time Tunnel(tt)
- https://arthas.aliyun.com/doc/tt
记录方法调用信息,支持事后查看方法调用的参数,返回值,抛出的异常等信息,仿佛穿越时空隧道回到调用现场一般。
$ tt -t org.apache.dubbo.demo.provider.DemoServiceImpl sayHello
Press Ctrl+C to abort.
Affect(class-cnt:1 , method-cnt:1) cost in 75 ms.INDEX TIMESTAMP COST(ms) IS-RET IS-EXP OBJECT CLASS METHOD
-------------------------------------------------------------------------------------------------------------------------------------1000 2018-09-20 09:54:10 1.971195 true false 0x55965cca DemoServiceImpl sayHello1001 2018-09-20 09:54:11 0.215685 true false 0x55965cca DemoServiceImpl sayHello1002 2018-09-20 09:54:12 0.236303 true false 0x55965cca DemoServiceImpl sayHello1003 2018-09-20 09:54:13 0.159598 true false 0x55965cca DemoServiceImpl sayHello1004 2018-09-20 09:54:14 0.201982 true false 0x55965cca DemoServiceImpl sayHello1005 2018-09-20 09:54:15 0.214205 true false 0x55965cca DemoServiceImpl sayHello1006 2018-09-20 09:54:16 0.241863 true false 0x55965cca DemoServiceImpl sayHello1007 2018-09-20 09:54:17 0.305747 true false 0x55965cca DemoServiceImpl sayHello1008 2018-09-20 09:54:18 0.18468 true false 0x55965cca DemoServiceImpl sayHello
Classloader
- https://arthas.aliyun.com/doc/classloader
了解当前系统中有多少类加载器,以及每个加载器加载的类数量,帮助您判断是否有类加载器泄露。
$ classloadername numberOfInstances loadedCountTotalBootstrapClassLoader 1 3346com.taobao.arthas.agent.ArthasClassloader 1 1262java.net.URLClassLoader 2 1033org.apache.catalina.loader.ParallelWebappClassLoader 1 628sun.reflect.DelegatingClassLoader 166 166sun.misc.Launcher$AppClassLoader 1 31com.alibaba.fastjson.util.ASMClassLoader 6 15sun.misc.Launcher$ExtClassLoader 1 7org.jvnet.hk2.internal.DelegatingClassLoader 2 2sun.reflect.misc.MethodUtil 1 1
Web Console
- https://arthas.aliyun.com/doc/web-console
Profiler/FlameGraph/火焰图
- https://arthas.aliyun.com/doc/profiler
$ profiler start
Started [cpu] profiling
$ profiler stop
profiler output file: /tmp/demo/arthas-output/20211207-111550.html
OK
通过浏览器查看profiler结果:
Arthas Spring Boot Starter
- Arthas Spring Boot Starter